Commit Graph

96 Commits

Author SHA1 Message Date
Cassandra Heart
4696d5292c
restore verenc into hypergraph loop, thread hypergraph through token application for backcompat 2025-02-06 06:34:22 -06:00
Cassandra Heart
e32e71bb0c
Merge branch 'develop' into develop-2.1 2024-12-18 01:21:07 -06:00
petricadaipegsp
5ad9bdc468
Change default GOGC to 10 (#409)
* Set default GOGC to 10

* Do not skip frame prover trie mutex
2024-12-10 19:11:09 -06:00
petricadaipegsp
b728d8d76f
Centralize configuration defaults and upgrade message limits (#410)
* Apply config defaults early

* Apply engine config defaults early

* Apply P2P config defaults early

* Remove default duplicates

* Fix casing

* Add sync message size configuration
2024-12-10 19:10:49 -06:00
Cassandra Heart
b49ba62d9a
Merge branch 'develop' into develop-2.1 2024-12-09 05:44:53 -06:00
Cassandra Heart
4f742abd77
support state tree 2024-12-09 05:44:43 -06:00
petricadaipegsp
b9f1f0a6af
Frame pruning fixes (#405)
* Consistently close iterators

* Prune frames in batches

* Add compact on start flag
2024-12-07 21:19:04 -06:00
petricadaipegsp
89c85422be
Stop gRPC and HTTP servers on shutdown (#408)
* Stop gRPC and HTTP servers on shutdown

* Wait for executor to register
2024-12-07 21:12:26 -06:00
petricadaipegsp
8a7aae3557
Frame processing optimisations (#404)
* Make frame processing hashing parallel

* Do not build address for new style proofs

* Validate mint requests in parallel

* Do not recompute byte form of address in mint

* Remove unused output

* Use single append

* Cache verify challenge proof results
2024-12-04 22:57:59 -06:00
petricadaipegsp
1e441e280d
Memory management adjustments (#397)
* Do not vendor go-buffer-pool

* Do not change GOGC

* Use local buffer
2024-12-01 15:04:01 -06:00
petricadaipegsp
8ce9aeee4e
Disallow excessive GOMAXPROCS (#368) 2024-11-20 16:59:09 -06:00
petricadaipegsp
bc05a4d7b9
Adaptive reserved cores (#363)
* Add adaptive data worker count

* Use runtime worker count for validation workers

* Reserve cores for networking during transition application

* Automatically set GOGC and GOMEMLIMIT
2024-11-19 16:51:14 -06:00
Cassandra Heart
61179c13a0
node info should show worker counts 2024-11-18 02:03:37 -06:00
Cassandra Heart
29a129d543
allow choice of indexed or non-indexed batches 2024-11-17 07:58:15 -06:00
petricadaipegsp
80c7ec2889
Add initial Prometheus support (#353)
* Add Prometheus server

* Add Prometheus gRPC metrics

* Add BlossomSub metrics

---------

Co-authored-by: Cassandra Heart <7929478+CassOnMars@users.noreply.github.com>
2024-11-16 17:53:19 -06:00
Cassandra Heart
b0a87b2fe4
wrapping up 2024-11-13 06:03:14 -06:00
Cassandra Heart
d4a5e30ba2
v2.0.3-p3 2024-11-12 01:45:18 -06:00
Cassandra Heart
9d7db3c85e
add repair 2024-11-12 01:14:52 -06:00
Cassandra Heart
df0262769b
switch default behavior and config value 2024-11-09 17:03:12 -06:00
Cassandra Heart
67d454acb9
add light prover support 2024-11-09 14:46:53 -06:00
Cassandra Heart
80be91d067
v2.0.3-b3 2024-11-06 21:43:55 -06:00
petricadaipegsp
07ff232c7d
Use node configuration for genesis retrieval (#332) 2024-11-05 17:04:05 -06:00
Cassandra Heart
3dd9a0c5f3
get develop caught up (#322)
* Update qcommander.sh bootrap (#304)

* v2.0.1 (#308)

* roll up v2.0.1-b2 to develop

* b2-fixed

* adjust return data of fast sync so it doesn't return the earliest frame

* -b3

* fix: announce peer based on leading frame, not initial frame; fix: looping bug

* fix: last batch fails due to underflow; qol: make logging chattier

* -b4

* resolve frame cache issue

* fix: mint loop + re-migrate

* fix: register execution panic

* fix: mint loop, other side

* fix: handle unexpected return of nil status

* final -b4

* handle subtle change to migration

* qol: add heuristic to handle corruption scenario

* bump genesis

* qol: use separate channel for worker

* final parameterization, parallelize streams

* deprecate signers 10, 11, 14, 17

* adjust signatory check size to match rotated out signers

* V2.0.2.3 (#321)

* roll up v2.0.1-b2 to develop

* b2-fixed

* adjust return data of fast sync so it doesn't return the earliest frame

* -b3

* fix: announce peer based on leading frame, not initial frame; fix: looping bug

* fix: last batch fails due to underflow; qol: make logging chattier

* -b4

* resolve frame cache issue

* fix: mint loop + re-migrate

* fix: register execution panic

* fix: mint loop, other side

* fix: handle unexpected return of nil status

* final -b4

* handle subtle change to migration

* qol: add heuristic to handle corruption scenario

* bump genesis

* qol: use separate channel for worker

* final parameterization, parallelize streams

* Add direct peers to blossomsub (#309)

Co-authored-by: Tyler Sturos <tyler.john@qcommander.sh>

* chore(docker): add ca-certificates to fix x509 error. (#307)

* Update qcommander.sh bootrap (#304)

* chore(docker): add ca-certificates to fix x509 error.

---------

Co-authored-by: Tyler Sturos <55340199+tjsturos@users.noreply.github.com>

* deprecate signers 10, 11, 14, 17

* adjust signatory check size to match rotated out signers

* qol: sync by rebroadcast

* upgrade version

* more small adjustments

* wait a little longer

* fix: don't use iterator for frame directly until iterator is fixed

* change iterator, genesis for testnet

* adjust to previous sync handling

* adjust: don't grab the very latest while it's already being broadcasted

* ok, ready for testnet

* handle rebroadcast quirks

* more adjustments from testing

* faster

* temporarily bulk process on frame candidates

* resolve separate frames

* don't loop

* make worker reset resume to check where it should continue

* move window

* reduce signature count now that supermajority signed last

* resolve bottlenecks

* remove GOMAXPROCS limit for now

* revisions for v2.0.2.1

* bump version

* bulk import

* reintroduce sync

* small adustments to make life better

* check bitmask for peers and keep alive

* adjust reconnect

* ensure peer doesn't fall off address list

* adjust blossomsub to background discovery

* bump version

* remove dev check

* remove debug log line

* further adjustments

* a little more logic around connection management

* v2.0.2.3

* Fix peer discovery (#319)

* Fix peer discovery

* Make peer discovery connections parallel

* Monitor peers via pings (#317)

* Support QUILIBRIUM_SIGNATURE_CHECK in client (#314)

* Ensure direct peers are not pruned by resource limits (#315)

* Support pprof profiling via HTTP (#313)

* Fix CPU profiling

* Add pprof server support

* Additional peering connection improvements (#320)

* Lookup peers if not enough external peers are available

* Make bootstrap peer discovery sensitive to a lack of bootstrappers

---------

Co-authored-by: Tyler Sturos <55340199+tjsturos@users.noreply.github.com>
Co-authored-by: Tyler Sturos <tyler.john@qcommander.sh>
Co-authored-by: linquanisaac <33619994+linquanisaac@users.noreply.github.com>
Co-authored-by: petricadaipegsp <155911522+petricadaipegsp@users.noreply.github.com>

---------

Co-authored-by: Tyler Sturos <55340199+tjsturos@users.noreply.github.com>
Co-authored-by: Tyler Sturos <tyler.john@qcommander.sh>
Co-authored-by: linquanisaac <33619994+linquanisaac@users.noreply.github.com>
Co-authored-by: petricadaipegsp <155911522+petricadaipegsp@users.noreply.github.com>
2024-10-31 16:46:58 -05:00
Cassandra Heart
d74f26c7f3
remove GOMAXPROCS limit for now 2024-10-26 01:07:23 -05:00
Cassandra Heart
f5c7ade33d
reduce signature count now that supermajority signed last 2024-10-25 15:00:00 -05:00
Cassandra Heart
7296cb0ea2
more adjustments from testing 2024-10-25 00:46:38 -05:00
Cassandra Heart
b7d2e17c63
final parameterization, parallelize streams 2024-10-23 14:26:01 -05:00
Cassandra Heart
7944cb6ca5
roll up v2.0.1-b2 to develop 2024-10-20 20:00:36 -05:00
Cassandra Heart
1bd670e773
v2.0.0-p8 2024-10-18 01:55:51 -05:00
Cassandra Heart
5230ceb413
v2.0.0-p5 (#303) 2024-10-16 23:51:27 -05:00
Cassandra Heart
270b29b321
reduce to 8 2024-10-15 15:20:51 -07:00
Cassandra Heart
e87ec00c89
v2.0.0-p3 (#299) 2024-10-14 14:41:40 -07:00
Cassandra Heart
389ada9f28
v2.0.0-p1 + QClient (#296) 2024-10-13 18:37:19 -07:00
Cassandra Heart
b4051ccbc9
detangling merge of main node for v2 (#293) 2024-10-12 11:55:17 -07:00
Cassandra Heart
f640c09008
v1.4.21-p1 (#279) 2024-07-29 12:46:36 -05:00
Cassandra Heart
d526ec63d0
v1.4.21 (#255)
* v1.4.21

* bump name
2024-07-05 04:40:00 -05:00
Cassandra Heart
1a244f5154
v1.4.20-p1 (#251)
* v1.4.20-p1

* further refinements/tuning
2024-06-29 19:12:53 -05:00
Cassandra Heart
6c567a04c1
v1.4.20 (#244)
* v1.4.20 base

* add inmemory dev mock for hypergraph

* add simple rdf + tr

* Update config.go (#234)

2 of bootstrap nodes are going to be closed due to low performances. Will consider to replace with better specs.

* go mod tidy

* go mod tidy

* bump name in readme

---------

Co-authored-by: 0xOzgur <29779769+0xOzgur@users.noreply.github.com>
2024-06-21 12:46:36 -05:00
Cassandra Heart
35561a9e41
v1.4.19-p1 (#235)
* v1.4.19-p1

* remove integrity check from startup
2024-06-15 11:05:06 -05:00
Cassie Heart
ebb31fd792 v1.4.19 2024-06-08 11:32:45 +00:00
Cassandra Heart
58456c1057
v1.4.18-patch-2 (#230)
* feat: IPC for wesolowski

* update self peer info

* remove digests and signatures

* add new binaries and digests

* Signatory #13 added

* Signatory #4 added (#231)

* added sig.6 files (#232)

* Signatory #9 added (#233)

* Added signatories #1, #2, #3, #5, #8, #12, #14, #15, #16, #17

* remove binaries, release ready

---------

Co-authored-by: 0xOzgur <29779769+0xOzgur@users.noreply.github.com>
Co-authored-by: Demipoet <161999657+demipoet@users.noreply.github.com>
Co-authored-by: Freekers <1370857+Freekers@users.noreply.github.com>
2024-05-29 12:51:47 -05:00
Cassandra Heart
13bac91367
v1.4.18-patch-1 (#216)
* experiment: verify in channel (#215)

* Change volume mapping so .config folder is created inside node folder and not on root (#214)

* Update main.go to fix Q logo (#213)

Q logo is not appearing correctly on the terminal while running node. Added a new line character after "Signature check passed" to fix it

* switched get node info response to use masterClock frame for maxFrame field (#212)

* fix: keys file remains null (#217)

* Revert "Change volume mapping so .config folder is created inside node folder…" (#218)

This reverts commit 27f50a92c6.

* Docker split take 2 (#219)

* split runtime docker files into a docker subfolder

* split DOCKER-README.md

* updated docker instructions

* add restore command

* add image update related tasks

* add command to test if P2P port is visible

* Remove bootstrap peer (#189)

* Change bootstrap servers to DHT-only peers (#187)

* support voucher file-based claims (#183)

* Change bootstrap servers to DHT-only peers

Changing my bootstrap servers to DHT-only peers with somewhat lower
specs. One of the new ones is in the US and the other one is in
Switzerland. Both use reliable providers and have 10Gbps network
interfaces.

---------

Co-authored-by: Cassandra Heart <7929478+CassOnMars@users.noreply.github.com>

* Don't run self-test in DHT-only mode (#186)

* support voucher file-based claims (#183)

* Don't run self-test in DHT-only mode

The node tries to create a self-test when ran with the `-dht-only`
flag, but it doesn't load the KZG ceremony data in DHT-only mode
which leads to a crash.

Don't run self-test when the `-dht-only` flag is set.

I tested by starting a node locally with and without existing
self-test and with the `-dht-only` flag.

---------

Co-authored-by: Cassandra Heart <7929478+CassOnMars@users.noreply.github.com>

* Embed json files in binary (#182)

* Embed ceremony.json in binary

* Embed retroactive_peers.json in binary

* Signers build and verification tasks (#181)

* add signers specific Taskfile

* add verify tasks

* move signer task under signer folder

* create docker image specific for signers

* map current user into docker image and container

* ignore node-tmp-*

* add verify:build:internal

* prevent tasks with docker commands from being run inside a container

* rename *:internal to *:container

* add README.md

* add pem files to git

* Updating Q Guide link (#173)

* Update README.md

Updated link to Quilibrium guide to new website

* Update README.md

---------

Co-authored-by: littleblackcloud <163544315+littleblackcloud@users.noreply.github.com>
Co-authored-by: Agost Biro <5764438+agostbiro@users.noreply.github.com>
Co-authored-by: Cassandra Heart <7929478+CassOnMars@users.noreply.github.com>
Co-authored-by: Demipoet <161999657+demipoet@users.noreply.github.com>

* Signer related fixes (#220)

* add pems 16 and 17

* remove .bin extension from generated binaries

* no more json files to copy to docker image

* feat: recalibrate self-test on the fly (#221)

* fix: switch RPC for peer and node info (#222)

* replace binaries with patch build

* add digests

* Signatory #13 added

* Signatory #4 added (#223)

* Signatory #14 added

* Signatory #17 added

* Signatory #12 added

* Signatory #3 added

* Signatory #2 added

* Signatory #16 added

* Signatory #1 added

* Signatory #8 added

* remove binaries, release ready

---------

Co-authored-by: AvAcalho <158583728+AvAcalho@users.noreply.github.com>
Co-authored-by: Ravish Ahmad <ravishahmad16@gmail.com>
Co-authored-by: luk <luk@luktech.dev>
Co-authored-by: Marius Scurtescu <marius.scurtescu@gmail.com>
Co-authored-by: littleblackcloud <163544315+littleblackcloud@users.noreply.github.com>
Co-authored-by: Agost Biro <5764438+agostbiro@users.noreply.github.com>
Co-authored-by: Demipoet <161999657+demipoet@users.noreply.github.com>
Co-authored-by: 0xOzgur <29779769+0xOzgur@users.noreply.github.com>
2024-05-27 00:10:15 -05:00
Cassandra Heart
2bbd1e0690
v1.4.18 (#193)
* Remove bootstrap peer (#189)

* Change bootstrap servers to DHT-only peers (#187)

* support voucher file-based claims (#183)

* Change bootstrap servers to DHT-only peers

Changing my bootstrap servers to DHT-only peers with somewhat lower
specs. One of the new ones is in the US and the other one is in
Switzerland. Both use reliable providers and have 10Gbps network
interfaces.

---------

Co-authored-by: Cassandra Heart <7929478+CassOnMars@users.noreply.github.com>

* Don't run self-test in DHT-only mode (#186)

* support voucher file-based claims (#183)

* Don't run self-test in DHT-only mode

The node tries to create a self-test when ran with the `-dht-only`
flag, but it doesn't load the KZG ceremony data in DHT-only mode
which leads to a crash.

Don't run self-test when the `-dht-only` flag is set.

I tested by starting a node locally with and without existing
self-test and with the `-dht-only` flag.

---------

Co-authored-by: Cassandra Heart <7929478+CassOnMars@users.noreply.github.com>

* Embed json files in binary (#182)

* Embed ceremony.json in binary

* Embed retroactive_peers.json in binary

* Signers build and verification tasks (#181)

* add signers specific Taskfile

* add verify tasks

* move signer task under signer folder

* create docker image specific for signers

* map current user into docker image and container

* ignore node-tmp-*

* add verify:build:internal

* prevent tasks with docker commands from being run inside a container

* rename *:internal to *:container

* add README.md

* add pem files to git

* Updating Q Guide link (#173)

* Update README.md

Updated link to Quilibrium guide to new website

* Update README.md

* feat: network switching and namespaced announce strings/bitmasks (#190)

* feat: network switching and namespaced announce strings/bitmasks

* bump version name and logo

* feat: mini pomw proofs as part of peer manifest (#191)

* shift default config directory under current folder (#176)

* feat: signature check (#192)

* feat: signature check

* adjust docker command so it doesn't invoke sigcheck

* remove old version

* add binaries and digests

* fix bug, revert build

* shasum has weird byte at end

* proper binaries and digests

* Signatory #13 added

* Signatory #3 added

* Signer 4 (#194)

* Signatory #5 added

* Signatory #9 added (#195)

* Signatory #1 added

* added sig.6 files (#196)

* Signatories #8 and #16 added

* Signatory #12 added

* Add signature (#197)

* reset build for v1.4.18 after testnet bug

* updated build, resigned by #13

* Signatory #16 added

* added sig.6 files (#198)

* Signatory #8 added

* Signatory #17 added

* Signatory #1 added

* Signatory #7 added

* Signatory #4 added

* Signatory #14 added

* remove binaries, ready to ship

---------

Co-authored-by: littleblackcloud <163544315+littleblackcloud@users.noreply.github.com>
Co-authored-by: Agost Biro <5764438+agostbiro@users.noreply.github.com>
Co-authored-by: Marius Scurtescu <marius.scurtescu@gmail.com>
Co-authored-by: Demipoet <161999657+demipoet@users.noreply.github.com>
Co-authored-by: 0xOzgur <29779769+0xOzgur@users.noreply.github.com>
Co-authored-by: Freekers <1370857+Freekers@users.noreply.github.com>
2024-05-25 00:22:50 -05:00
Cassandra Heart
11ff272daf
v1.4.17 – Aurora (#169)
* v1.4.17 – Aurora

* binaries, digests, and first signature

* signer #3 verified

* signer #8 verified

* signer #14 verified

* signer #12 verified

* Add signatures (#170)

* Add files via upload (#171)

* signer #4 verified

* signer #11 verified

* signer #16 verified

* signer #1 verified

* remove binaries from repo, release ready

---------

Co-authored-by: Agost Biro <5764438+agostbiro@users.noreply.github.com>
Co-authored-by: Demipoet <161999657+demipoet@users.noreply.github.com>
2024-05-11 02:02:03 -05:00
Cassandra Heart
de115fbfad
Liberty (#167)
* And what country can preserve its liberties if their rulers are not warned from time to time that their people preserve the spirit of resistance?

* adjust check to handle peer id change
2024-04-22 20:38:20 -05:00
Sercan Tekin
f05bb62d63
Fixing print typo and adding peer score (#152) 2024-03-29 02:20:35 -05:00
Cassandra Heart
5a3af70dc8
v1.4.13 (#146)
* v1.4.13

* adjust blossomsub param
2024-03-27 03:50:52 -05:00
Cassandra Heart
0baa4ea341
qol: disable compaction completely for now, also weave in notfound when relevant 2024-03-24 21:00:45 -05:00
Marius Scurtescu
a0659fec83
add -node-info command (#141)
* add version label and trim repo url

* add README for protobufs

* add version to NodeInfoResponse

* add docker login task

* remove version from docker compose file

* return version with GetNodeInfo response

* add basic -node-info flag

* print max frame as well with -node-info

* expand protobuf README

* update node command examples to use node binary and make log commands follow

* return all of NodeInfo

* extract FormatVersion

* print version of running process
2024-03-24 03:11:58 -05:00
Cassandra Heart
5afabc401b
V1.4.12 (#142)
* v1.4.12

* small logging adjust
2024-03-23 15:26:57 -05:00