Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: implement peerDAS on fulu #6353

Draft
wants to merge 31 commits into
base: unstable
Choose a base branch
from
Draft

feat: implement peerDAS on fulu #6353

wants to merge 31 commits into from
+6,243 −4,912

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Jan 24, 2024

@ensi321 ensi321 mentioned this pull request Mar 14, 2024
Open
12 tasks
@g11tech g11tech force-pushed the electra-fork branch 2 times, most recently from 8a4f66e to 473a04f Compare April 11, 2024 11:12
@g11tech g11tech changed the base branch from electra-fork to unstable June 22, 2024 11:16
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jun 22, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 3c1cbef Previous: 6b20ef5 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.2497 ms/op 1.1193 ms/op 1.12
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 44.336 us/op 44.381 us/op 1.00
BLS verify - blst 1.0603 ms/op 1.1032 ms/op 0.96
BLS verifyMultipleSignatures 3 - blst 1.5656 ms/op 1.7066 ms/op 0.92
BLS verifyMultipleSignatures 8 - blst 2.3752 ms/op 2.1301 ms/op 1.12
BLS verifyMultipleSignatures 32 - blst 7.3035 ms/op 7.2117 ms/op 1.01
BLS verifyMultipleSignatures 64 - blst 12.813 ms/op 10.040 ms/op 1.28
BLS verifyMultipleSignatures 128 - blst 25.352 ms/op 20.704 ms/op 1.22
BLS deserializing 10000 signatures 776.12 ms/op 777.90 ms/op 1.00
BLS deserializing 100000 signatures 7.6382 s/op 7.4589 s/op 1.02
BLS verifyMultipleSignatures - same message - 3 - blst 1.0839 ms/op 947.32 us/op 1.14
BLS verifyMultipleSignatures - same message - 8 - blst 1.2605 ms/op 1.0696 ms/op 1.18
BLS verifyMultipleSignatures - same message - 32 - blst 1.8508 ms/op 1.7518 ms/op 1.06
BLS verifyMultipleSignatures - same message - 64 - blst 2.7896 ms/op 2.6847 ms/op 1.04
BLS verifyMultipleSignatures - same message - 128 - blst 4.5387 ms/op 4.5174 ms/op 1.00
BLS aggregatePubkeys 32 - blst 19.856 us/op 20.510 us/op 0.97
BLS aggregatePubkeys 128 - blst 72.338 us/op 72.828 us/op 0.99
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 61.949 ms/op 54.865 ms/op 1.13
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 58.045 ms/op 52.942 ms/op 1.10
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 47.639 ms/op 45.460 ms/op 1.05
getSlashingsAndExits - default max 90.034 us/op 90.749 us/op 0.99
getSlashingsAndExits - 2k 362.25 us/op 388.69 us/op 0.93
proposeBlockBody type=full, size=empty 6.5618 ms/op 6.5292 ms/op 1.00
isKnown best case - 1 super set check 216.00 ns/op 211.00 ns/op 1.02
isKnown normal case - 2 super set checks 209.00 ns/op 209.00 ns/op 1.00
isKnown worse case - 16 super set checks 209.00 ns/op 220.00 ns/op 0.95
InMemoryCheckpointStateCache - add get delete 2.7420 us/op 2.6770 us/op 1.02
validate api signedAggregateAndProof - struct 1.5402 ms/op 1.5274 ms/op 1.01
validate gossip signedAggregateAndProof - struct 1.5042 ms/op 1.5549 ms/op 0.97
batch validate gossip attestation - vc 640000 - chunk 32 133.69 us/op 146.36 us/op 0.91
batch validate gossip attestation - vc 640000 - chunk 64 113.83 us/op 124.28 us/op 0.92
batch validate gossip attestation - vc 640000 - chunk 128 104.00 us/op 115.77 us/op 0.90
batch validate gossip attestation - vc 640000 - chunk 256 109.69 us/op 117.39 us/op 0.93
pickEth1Vote - no votes 976.49 us/op 1.1143 ms/op 0.88
pickEth1Vote - max votes 6.2353 ms/op 10.679 ms/op 0.58
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.657 ms/op 16.373 ms/op 0.83
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 19.476 ms/op 24.153 ms/op 0.81
pickEth1Vote - Eth1Data fastSerialize value x2048 442.55 us/op 484.15 us/op 0.91
pickEth1Vote - Eth1Data fastSerialize tree x2048 3.8269 ms/op 2.7612 ms/op 1.39
bytes32 toHexString 392.00 ns/op 375.00 ns/op 1.05
bytes32 Buffer.toString(hex) 246.00 ns/op 245.00 ns/op 1.00
bytes32 Buffer.toString(hex) from Uint8Array 349.00 ns/op 372.00 ns/op 0.94
bytes32 Buffer.toString(hex) + 0x 242.00 ns/op 268.00 ns/op 0.90
Object access 1 prop 0.13100 ns/op 0.12000 ns/op 1.09
Map access 1 prop 0.15000 ns/op 0.12600 ns/op 1.19
Object get x1000 5.8250 ns/op 6.4860 ns/op 0.90
Map get x1000 6.1440 ns/op 6.7150 ns/op 0.91
Object set x1000 30.014 ns/op 33.598 ns/op 0.89
Map set x1000 20.113 ns/op 21.812 ns/op 0.92
Return object 10000 times 0.28520 ns/op 0.31740 ns/op 0.90
Throw Error 10000 times 4.3349 us/op 4.9739 us/op 0.87
toHex 132.81 ns/op 172.24 ns/op 0.77
Buffer.from 132.54 ns/op 152.78 ns/op 0.87
shared Buffer 81.144 ns/op 97.477 ns/op 0.83
fastMsgIdFn sha256 / 200 bytes 2.2300 us/op 2.4060 us/op 0.93
fastMsgIdFn h32 xxhash / 200 bytes 207.00 ns/op 237.00 ns/op 0.87
fastMsgIdFn h64 xxhash / 200 bytes 265.00 ns/op 302.00 ns/op 0.88
fastMsgIdFn sha256 / 1000 bytes 7.4410 us/op 8.0950 us/op 0.92
fastMsgIdFn h32 xxhash / 1000 bytes 336.00 ns/op 367.00 ns/op 0.92
fastMsgIdFn h64 xxhash / 1000 bytes 336.00 ns/op 357.00 ns/op 0.94
fastMsgIdFn sha256 / 10000 bytes 67.115 us/op 70.417 us/op 0.95
fastMsgIdFn h32 xxhash / 10000 bytes 1.8010 us/op 2.0000 us/op 0.90
fastMsgIdFn h64 xxhash / 10000 bytes 1.1740 us/op 1.3300 us/op 0.88
send data - 1000 256B messages 13.208 ms/op 14.686 ms/op 0.90
send data - 1000 512B messages 19.302 ms/op 19.746 ms/op 0.98
send data - 1000 1024B messages 28.598 ms/op 31.619 ms/op 0.90
send data - 1000 1200B messages 21.980 ms/op 26.669 ms/op 0.82
send data - 1000 2048B messages 22.466 ms/op 26.046 ms/op 0.86
send data - 1000 4096B messages 24.640 ms/op 35.477 ms/op 0.69
send data - 1000 16384B messages 57.657 ms/op 65.674 ms/op 0.88
send data - 1000 65536B messages 219.87 ms/op 221.78 ms/op 0.99
enrSubnets - fastDeserialize 64 bits 884.00 ns/op 952.00 ns/op 0.93
enrSubnets - ssz BitVector 64 bits 326.00 ns/op 337.00 ns/op 0.97
enrSubnets - fastDeserialize 4 bits 133.00 ns/op 136.00 ns/op 0.98
enrSubnets - ssz BitVector 4 bits 332.00 ns/op 349.00 ns/op 0.95
prioritizePeers score -10:0 att 32-0.1 sync 2-0 153.40 us/op 121.81 us/op 1.26
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 195.58 us/op 142.58 us/op 1.37
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 332.67 us/op 205.42 us/op 1.62
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 790.84 us/op 392.25 us/op 2.02
prioritizePeers score 0:0 att 64-1 sync 4-1 830.54 us/op 467.43 us/op 1.78
array of 16000 items push then shift 1.6569 us/op 1.6288 us/op 1.02
LinkedList of 16000 items push then shift 7.9400 ns/op 7.1380 ns/op 1.11
array of 16000 items push then pop 77.536 ns/op 77.223 ns/op 1.00
LinkedList of 16000 items push then pop 6.9840 ns/op 6.9840 ns/op 1.00
array of 24000 items push then shift 2.3936 us/op 2.4279 us/op 0.99
LinkedList of 24000 items push then shift 7.1560 ns/op 7.2680 ns/op 0.98
array of 24000 items push then pop 111.01 ns/op 109.17 ns/op 1.02
LinkedList of 24000 items push then pop 7.5530 ns/op 7.0920 ns/op 1.07
intersect bitArray bitLen 8 6.3420 ns/op 6.6570 ns/op 0.95
intersect array and set length 8 37.730 ns/op 39.456 ns/op 0.96
intersect bitArray bitLen 128 29.813 ns/op 31.401 ns/op 0.95
intersect array and set length 128 619.25 ns/op 639.11 ns/op 0.97
bitArray.getTrueBitIndexes() bitLen 128 1.2350 us/op 1.0560 us/op 1.17
bitArray.getTrueBitIndexes() bitLen 248 2.7440 us/op 1.8480 us/op 1.48
bitArray.getTrueBitIndexes() bitLen 512 5.3840 us/op 3.7250 us/op 1.45
Buffer.concat 32 items 674.00 ns/op 776.00 ns/op 0.87
Uint8Array.set 32 items 1.4180 us/op 1.0420 us/op 1.36
Buffer.copy 2.2410 us/op 2.1800 us/op 1.03
Uint8Array.set - with subarray 2.2820 us/op 1.6740 us/op 1.36
Uint8Array.set - without subarray 1.2530 us/op 985.00 ns/op 1.27
getUint32 - dataview 206.00 ns/op 201.00 ns/op 1.02
getUint32 - manual 140.00 ns/op 124.00 ns/op 1.13
Set add up to 64 items then delete first 2.2124 us/op 2.2737 us/op 0.97
OrderedSet add up to 64 items then delete first 3.2832 us/op 3.3744 us/op 0.97
Set add up to 64 items then delete last 2.4908 us/op 2.5015 us/op 1.00
OrderedSet add up to 64 items then delete last 3.7207 us/op 3.9413 us/op 0.94
Set add up to 64 items then delete middle 2.3520 us/op 2.5411 us/op 0.93
OrderedSet add up to 64 items then delete middle 5.3135 us/op 5.3414 us/op 0.99
Set add up to 128 items then delete first 5.1531 us/op 5.0173 us/op 1.03
OrderedSet add up to 128 items then delete first 8.1829 us/op 7.8060 us/op 1.05
Set add up to 128 items then delete last 5.0519 us/op 5.1580 us/op 0.98
OrderedSet add up to 128 items then delete last 7.8261 us/op 7.8659 us/op 0.99
Set add up to 128 items then delete middle 5.6771 us/op 5.1565 us/op 1.10
OrderedSet add up to 128 items then delete middle 15.488 us/op 16.680 us/op 0.93
Set add up to 256 items then delete first 10.726 us/op 10.161 us/op 1.06
OrderedSet add up to 256 items then delete first 16.401 us/op 16.428 us/op 1.00
Set add up to 256 items then delete last 10.437 us/op 11.439 us/op 0.91
OrderedSet add up to 256 items then delete last 17.421 us/op 16.104 us/op 1.08
Set add up to 256 items then delete middle 11.063 us/op 12.796 us/op 0.86
OrderedSet add up to 256 items then delete middle 51.417 us/op 41.959 us/op 1.23
transfer serialized Status (84 B) 2.6880 us/op 2.2800 us/op 1.18
copy serialized Status (84 B) 1.3500 us/op 1.2560 us/op 1.07
transfer serialized SignedVoluntaryExit (112 B) 2.4140 us/op 2.3210 us/op 1.04
copy serialized SignedVoluntaryExit (112 B) 1.7560 us/op 1.7040 us/op 1.03
transfer serialized ProposerSlashing (416 B) 2.9960 us/op 2.3510 us/op 1.27
copy serialized ProposerSlashing (416 B) 1.7070 us/op 1.3090 us/op 1.30
transfer serialized Attestation (485 B) 3.0500 us/op 2.3670 us/op 1.29
copy serialized Attestation (485 B) 1.8920 us/op 1.4380 us/op 1.32
transfer serialized AttesterSlashing (33232 B) 2.8810 us/op 2.6280 us/op 1.10
copy serialized AttesterSlashing (33232 B) 4.4240 us/op 3.5930 us/op 1.23
transfer serialized Small SignedBeaconBlock (128000 B) 3.3620 us/op 3.2030 us/op 1.05
copy serialized Small SignedBeaconBlock (128000 B) 10.656 us/op 9.3590 us/op 1.14
transfer serialized Avg SignedBeaconBlock (200000 B) 3.9260 us/op 3.6520 us/op 1.08
copy serialized Avg SignedBeaconBlock (200000 B) 15.118 us/op 15.116 us/op 1.00
transfer serialized BlobsSidecar (524380 B) 4.0760 us/op 3.9820 us/op 1.02
copy serialized BlobsSidecar (524380 B) 73.237 us/op 63.058 us/op 1.16
transfer serialized Big SignedBeaconBlock (1000000 B) 4.2510 us/op 4.0420 us/op 1.05
copy serialized Big SignedBeaconBlock (1000000 B) 118.08 us/op 115.59 us/op 1.02
pass gossip attestations to forkchoice per slot 2.6899 ms/op 3.1041 ms/op 0.87
forkChoice updateHead vc 100000 bc 64 eq 0 459.32 us/op 520.53 us/op 0.88
forkChoice updateHead vc 600000 bc 64 eq 0 2.8333 ms/op 3.3225 ms/op 0.85
forkChoice updateHead vc 1000000 bc 64 eq 0 4.7619 ms/op 5.5447 ms/op 0.86
forkChoice updateHead vc 600000 bc 320 eq 0 2.8305 ms/op 3.1281 ms/op 0.90
forkChoice updateHead vc 600000 bc 1200 eq 0 2.7920 ms/op 3.1741 ms/op 0.88
forkChoice updateHead vc 600000 bc 7200 eq 0 3.0239 ms/op 3.3316 ms/op 0.91
forkChoice updateHead vc 600000 bc 64 eq 1000 10.554 ms/op 11.619 ms/op 0.91
forkChoice updateHead vc 600000 bc 64 eq 10000 10.411 ms/op 11.824 ms/op 0.88
forkChoice updateHead vc 600000 bc 64 eq 300000 14.177 ms/op 13.822 ms/op 1.03
computeDeltas 500000 validators 300 proto nodes 3.9303 ms/op 4.0561 ms/op 0.97
computeDeltas 500000 validators 1200 proto nodes 3.9167 ms/op 3.9833 ms/op 0.98
computeDeltas 500000 validators 7200 proto nodes 4.1123 ms/op 4.0167 ms/op 1.02
computeDeltas 750000 validators 300 proto nodes 5.9182 ms/op 5.9429 ms/op 1.00
computeDeltas 750000 validators 1200 proto nodes 5.9001 ms/op 5.8939 ms/op 1.00
computeDeltas 750000 validators 7200 proto nodes 6.2350 ms/op 5.9715 ms/op 1.04
computeDeltas 1400000 validators 300 proto nodes 16.059 ms/op 11.543 ms/op 1.39
computeDeltas 1400000 validators 1200 proto nodes 12.276 ms/op 11.853 ms/op 1.04
computeDeltas 1400000 validators 7200 proto nodes 11.431 ms/op 11.980 ms/op 0.95
computeDeltas 2100000 validators 300 proto nodes 19.061 ms/op 18.129 ms/op 1.05
computeDeltas 2100000 validators 1200 proto nodes 18.318 ms/op 18.153 ms/op 1.01
computeDeltas 2100000 validators 7200 proto nodes 17.606 ms/op 19.086 ms/op 0.92
altair processAttestation - 250000 vs - 7PWei normalcase 2.3898 ms/op 3.8567 ms/op 0.62
altair processAttestation - 250000 vs - 7PWei worstcase 3.1105 ms/op 4.4464 ms/op 0.70
altair processAttestation - setStatus - 1/6 committees join 135.00 us/op 156.46 us/op 0.86
altair processAttestation - setStatus - 1/3 committees join 263.42 us/op 308.93 us/op 0.85
altair processAttestation - setStatus - 1/2 committees join 364.96 us/op 439.61 us/op 0.83
altair processAttestation - setStatus - 2/3 committees join 470.40 us/op 547.24 us/op 0.86
altair processAttestation - setStatus - 4/5 committees join 629.97 us/op 751.56 us/op 0.84
altair processAttestation - setStatus - 100% committees join 748.41 us/op 888.06 us/op 0.84
altair processBlock - 250000 vs - 7PWei normalcase 5.6651 ms/op 6.1619 ms/op 0.92
altair processBlock - 250000 vs - 7PWei normalcase hashState 40.187 ms/op 44.635 ms/op 0.90
altair processBlock - 250000 vs - 7PWei worstcase 56.672 ms/op 48.945 ms/op 1.16
altair processBlock - 250000 vs - 7PWei worstcase hashState 112.89 ms/op 103.93 ms/op 1.09
phase0 processBlock - 250000 vs - 7PWei normalcase 2.3733 ms/op 2.4448 ms/op 0.97
phase0 processBlock - 250000 vs - 7PWei worstcase 23.535 ms/op 29.840 ms/op 0.79
altair processEth1Data - 250000 vs - 7PWei normalcase 335.63 us/op 417.46 us/op 0.80
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 6.3660 us/op 5.6290 us/op 1.13
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 41.843 us/op 36.686 us/op 1.14
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 10.413 us/op 9.7100 us/op 1.07
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 7.3200 us/op 5.2490 us/op 1.39
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 145.88 us/op 166.08 us/op 0.88
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.4538 ms/op 1.4116 ms/op 1.03
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 2.2855 ms/op 1.9396 ms/op 1.18
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.4170 ms/op 1.9544 ms/op 0.72
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.4479 ms/op 4.4056 ms/op 0.78
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.7014 ms/op 2.0284 ms/op 0.84
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.6666 ms/op 4.2338 ms/op 0.87
Tree 40 250000 create 500.22 ms/op 728.10 ms/op 0.69
Tree 40 250000 get(125000) 142.84 ns/op 155.53 ns/op 0.92
Tree 40 250000 set(125000) 1.7384 us/op 2.8181 us/op 0.62
Tree 40 250000 toArray() 19.200 ms/op 24.525 ms/op 0.78
Tree 40 250000 iterate all - toArray() + loop 20.648 ms/op 24.353 ms/op 0.85
Tree 40 250000 iterate all - get(i) 61.011 ms/op 67.326 ms/op 0.91
Array 250000 create 2.6400 ms/op 3.0547 ms/op 0.86
Array 250000 clone - spread 846.64 us/op 5.9977 ms/op 0.14
Array 250000 get(125000) 0.41600 ns/op 0.49300 ns/op 0.84
Array 250000 set(125000) 0.44000 ns/op 0.53400 ns/op 0.82
Array 250000 iterate all - loop 104.28 us/op 124.74 us/op 0.84
phase0 afterProcessEpoch - 250000 vs - 7PWei 50.708 ms/op 63.532 ms/op 0.80
Array.fill - length 1000000 4.5562 ms/op 5.3551 ms/op 0.85
Array push - length 1000000 14.220 ms/op 20.234 ms/op 0.70
Array.get 0.29324 ns/op 0.33137 ns/op 0.88
Uint8Array.get 0.46429 ns/op 0.52692 ns/op 0.88
phase0 beforeProcessEpoch - 250000 vs - 7PWei 19.038 ms/op 24.587 ms/op 0.77
altair processEpoch - mainnet_e81889 351.94 ms/op 339.36 ms/op 1.04
mainnet_e81889 - altair beforeProcessEpoch 21.436 ms/op 23.150 ms/op 0.93
mainnet_e81889 - altair processJustificationAndFinalization 5.7620 us/op 6.6110 us/op 0.87
mainnet_e81889 - altair processInactivityUpdates 5.4110 ms/op 5.9430 ms/op 0.91
mainnet_e81889 - altair processRewardsAndPenalties 48.477 ms/op 50.877 ms/op 0.95
mainnet_e81889 - altair processRegistryUpdates 693.00 ns/op 858.00 ns/op 0.81
mainnet_e81889 - altair processSlashings 199.00 ns/op 231.00 ns/op 0.86
mainnet_e81889 - altair processEth1DataReset 181.00 ns/op 201.00 ns/op 0.90
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2820 ms/op 1.4094 ms/op 0.91
mainnet_e81889 - altair processSlashingsReset 1.0260 us/op 1.2360 us/op 0.83
mainnet_e81889 - altair processRandaoMixesReset 1.3270 us/op 1.4970 us/op 0.89
mainnet_e81889 - altair processHistoricalRootsUpdate 210.00 ns/op 217.00 ns/op 0.97
mainnet_e81889 - altair processParticipationFlagUpdates 565.00 ns/op 777.00 ns/op 0.73
mainnet_e81889 - altair processSyncCommitteeUpdates 141.00 ns/op 163.00 ns/op 0.87
mainnet_e81889 - altair afterProcessEpoch 55.869 ms/op 58.749 ms/op 0.95
capella processEpoch - mainnet_e217614 1.0145 s/op 1.1722 s/op 0.87
mainnet_e217614 - capella beforeProcessEpoch 73.718 ms/op 80.287 ms/op 0.92
mainnet_e217614 - capella processJustificationAndFinalization 6.1830 us/op 6.1000 us/op 1.01
mainnet_e217614 - capella processInactivityUpdates 17.206 ms/op 20.051 ms/op 0.86
mainnet_e217614 - capella processRewardsAndPenalties 202.49 ms/op 225.41 ms/op 0.90
mainnet_e217614 - capella processRegistryUpdates 7.0700 us/op 8.7980 us/op 0.80
mainnet_e217614 - capella processSlashings 190.00 ns/op 239.00 ns/op 0.79
mainnet_e217614 - capella processEth1DataReset 231.00 ns/op 215.00 ns/op 1.07
mainnet_e217614 - capella processEffectiveBalanceUpdates 15.389 ms/op 16.772 ms/op 0.92
mainnet_e217614 - capella processSlashingsReset 1.1750 us/op 1.1320 us/op 1.04
mainnet_e217614 - capella processRandaoMixesReset 1.4650 us/op 1.5280 us/op 0.96
mainnet_e217614 - capella processHistoricalRootsUpdate 180.00 ns/op 230.00 ns/op 0.78
mainnet_e217614 - capella processParticipationFlagUpdates 581.00 ns/op 651.00 ns/op 0.89
mainnet_e217614 - capella afterProcessEpoch 128.42 ms/op 148.36 ms/op 0.87
phase0 processEpoch - mainnet_e58758 311.34 ms/op 369.18 ms/op 0.84
mainnet_e58758 - phase0 beforeProcessEpoch 84.802 ms/op 89.301 ms/op 0.95
mainnet_e58758 - phase0 processJustificationAndFinalization 5.3910 us/op 6.8420 us/op 0.79
mainnet_e58758 - phase0 processRewardsAndPenalties 38.102 ms/op 40.430 ms/op 0.94
mainnet_e58758 - phase0 processRegistryUpdates 3.2560 us/op 3.9120 us/op 0.83
mainnet_e58758 - phase0 processSlashings 178.00 ns/op 258.00 ns/op 0.69
mainnet_e58758 - phase0 processEth1DataReset 172.00 ns/op 282.00 ns/op 0.61
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0244 ms/op 1.2097 ms/op 0.85
mainnet_e58758 - phase0 processSlashingsReset 1.0100 us/op 1.1940 us/op 0.85
mainnet_e58758 - phase0 processRandaoMixesReset 1.2660 us/op 1.6840 us/op 0.75
mainnet_e58758 - phase0 processHistoricalRootsUpdate 173.00 ns/op 223.00 ns/op 0.78
mainnet_e58758 - phase0 processParticipationRecordUpdates 899.00 ns/op 1.0190 us/op 0.88
mainnet_e58758 - phase0 afterProcessEpoch 41.723 ms/op 50.768 ms/op 0.82
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.4648 ms/op 1.9311 ms/op 1.28
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.9980 ms/op 2.0952 ms/op 0.95
altair processInactivityUpdates - 250000 normalcase 21.150 ms/op 22.482 ms/op 0.94
altair processInactivityUpdates - 250000 worstcase 19.175 ms/op 23.165 ms/op 0.83
phase0 processRegistryUpdates - 250000 normalcase 7.6050 us/op 12.150 us/op 0.63
phase0 processRegistryUpdates - 250000 badcase_full_deposits 269.33 us/op 326.09 us/op 0.83
phase0 processRegistryUpdates - 250000 worstcase 0.5 112.80 ms/op 129.61 ms/op 0.87
altair processRewardsAndPenalties - 250000 normalcase 46.458 ms/op 56.040 ms/op 0.83
altair processRewardsAndPenalties - 250000 worstcase 40.233 ms/op 56.459 ms/op 0.71
phase0 getAttestationDeltas - 250000 normalcase 7.2007 ms/op 13.365 ms/op 0.54
phase0 getAttestationDeltas - 250000 worstcase 6.1560 ms/op 7.4054 ms/op 0.83
phase0 processSlashings - 250000 worstcase 126.82 us/op 126.57 us/op 1.00
altair processSyncCommitteeUpdates - 250000 131.38 ms/op 154.49 ms/op 0.85
BeaconState.hashTreeRoot - No change 231.00 ns/op 246.00 ns/op 0.94
BeaconState.hashTreeRoot - 1 full validator 87.899 us/op 124.45 us/op 0.71
BeaconState.hashTreeRoot - 32 full validator 901.94 us/op 1.1806 ms/op 0.76
BeaconState.hashTreeRoot - 512 full validator 10.084 ms/op 14.176 ms/op 0.71
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 94.742 us/op 155.17 us/op 0.61
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.3987 ms/op 2.0018 ms/op 0.70
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 28.094 ms/op 33.212 ms/op 0.85
BeaconState.hashTreeRoot - 1 balances 90.560 us/op 115.10 us/op 0.79
BeaconState.hashTreeRoot - 32 balances 1.1922 ms/op 1.0671 ms/op 1.12
BeaconState.hashTreeRoot - 512 balances 9.2666 ms/op 10.262 ms/op 0.90
BeaconState.hashTreeRoot - 250000 balances 195.92 ms/op 202.87 ms/op 0.97
aggregationBits - 2048 els - zipIndexesInBitList 22.422 us/op 27.286 us/op 0.82
byteArrayEquals 32 53.890 ns/op 59.716 ns/op 0.90
Buffer.compare 32 17.323 ns/op 18.926 ns/op 0.92
byteArrayEquals 1024 1.5920 us/op 1.7809 us/op 0.89
Buffer.compare 1024 24.192 ns/op 27.409 ns/op 0.88
byteArrayEquals 16384 25.377 us/op 28.323 us/op 0.90
Buffer.compare 16384 204.71 ns/op 220.17 ns/op 0.93
byteArrayEquals 123687377 193.25 ms/op 242.93 ms/op 0.80
Buffer.compare 123687377 7.0360 ms/op 11.109 ms/op 0.63
byteArrayEquals 32 - diff last byte 51.895 ns/op 66.368 ns/op 0.78
Buffer.compare 32 - diff last byte 16.682 ns/op 21.906 ns/op 0.76
byteArrayEquals 1024 - diff last byte 1.5479 us/op 2.0911 us/op 0.74
Buffer.compare 1024 - diff last byte 25.114 ns/op 35.506 ns/op 0.71
byteArrayEquals 16384 - diff last byte 24.595 us/op 32.300 us/op 0.76
Buffer.compare 16384 - diff last byte 197.62 ns/op 260.39 ns/op 0.76
byteArrayEquals 123687377 - diff last byte 189.04 ms/op 231.17 ms/op 0.82
Buffer.compare 123687377 - diff last byte 6.3823 ms/op 9.7480 ms/op 0.65
byteArrayEquals 32 - random bytes 5.0060 ns/op 5.8020 ns/op 0.86
Buffer.compare 32 - random bytes 16.785 ns/op 18.768 ns/op 0.89
byteArrayEquals 1024 - random bytes 4.9640 ns/op 6.0990 ns/op 0.81
Buffer.compare 1024 - random bytes 16.953 ns/op 21.054 ns/op 0.81
byteArrayEquals 16384 - random bytes 5.0860 ns/op 6.3210 ns/op 0.80
Buffer.compare 16384 - random bytes 16.738 ns/op 19.309 ns/op 0.87
byteArrayEquals 123687377 - random bytes 6.2800 ns/op 7.2500 ns/op 0.87
Buffer.compare 123687377 - random bytes 18.000 ns/op 19.790 ns/op 0.91
regular array get 100000 times 40.071 us/op 46.305 us/op 0.87
wrappedArray get 100000 times 32.574 us/op 35.026 us/op 0.93
arrayWithProxy get 100000 times 14.124 ms/op 14.131 ms/op 1.00
ssz.Root.equals 47.322 ns/op 50.223 ns/op 0.94
byteArrayEquals 46.458 ns/op 50.174 ns/op 0.93
Buffer.compare 10.703 ns/op 11.151 ns/op 0.96
processSlot - 1 slots 10.086 us/op 12.644 us/op 0.80
processSlot - 32 slots 2.3163 ms/op 2.9197 ms/op 0.79
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 57.916 ms/op 61.769 ms/op 0.94
getCommitteeAssignments - req 1 vs - 250000 vc 2.0901 ms/op 2.7876 ms/op 0.75
getCommitteeAssignments - req 100 vs - 250000 vc 4.1035 ms/op 4.9822 ms/op 0.82
getCommitteeAssignments - req 1000 vs - 250000 vc 4.3013 ms/op 5.4714 ms/op 0.79
findModifiedValidators - 10000 modified validators 756.85 ms/op 806.94 ms/op 0.94
findModifiedValidators - 1000 modified validators 726.73 ms/op 797.15 ms/op 0.91
findModifiedValidators - 100 modified validators 303.68 ms/op 298.34 ms/op 1.02
findModifiedValidators - 10 modified validators 208.03 ms/op 194.73 ms/op 1.07
findModifiedValidators - 1 modified validators 138.86 ms/op 187.03 ms/op 0.74
findModifiedValidators - no difference 144.21 ms/op 249.90 ms/op 0.58
compare ViewDUs 6.5109 s/op 8.1246 s/op 0.80
compare each validator Uint8Array 2.0943 s/op 1.5982 s/op 1.31
compare ViewDU to Uint8Array 1.5623 s/op 1.4937 s/op 1.05
migrate state 1000000 validators, 24 modified, 0 new 852.64 ms/op 1.1881 s/op 0.72
migrate state 1000000 validators, 1700 modified, 1000 new 1.2979 s/op 1.5565 s/op 0.83
migrate state 1000000 validators, 3400 modified, 2000 new 1.5031 s/op 1.5227 s/op 0.99
migrate state 1500000 validators, 24 modified, 0 new 898.65 ms/op 1.1020 s/op 0.82
migrate state 1500000 validators, 1700 modified, 1000 new 1.1818 s/op 1.2817 s/op 0.92
migrate state 1500000 validators, 3400 modified, 2000 new 1.4051 s/op 1.4710 s/op 0.96
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.2900 ns/op 4.8800 ns/op 0.88
state getBlockRootAtSlot - 250000 vs - 7PWei 596.58 ns/op 618.96 ns/op 0.96
computeProposers - vc 250000 7.1009 ms/op 8.3383 ms/op 0.85
computeEpochShuffling - vc 250000 43.798 ms/op 48.988 ms/op 0.89
getNextSyncCommittee - vc 250000 139.33 ms/op 152.67 ms/op 0.91
computeSigningRoot for AttestationData 21.486 us/op 24.654 us/op 0.87
hash AttestationData serialized data then Buffer.toString(base64) 1.6281 us/op 1.8252 us/op 0.89
toHexString serialized data 992.21 ns/op 921.92 ns/op 1.08
Buffer.toString(base64) 154.69 ns/op 175.91 ns/op 0.88
nodejs block root to RootHex using toHex 141.19 ns/op 166.27 ns/op 0.85
nodejs block root to RootHex using toRootHex 94.497 ns/op 102.34 ns/op 0.92
browser block root to RootHex using the deprecated toHexString 211.11 ns/op 232.41 ns/op 0.91
browser block root to RootHex using toHex 171.73 ns/op 187.59 ns/op 0.92
browser block root to RootHex using toRootHex 161.68 ns/op 176.53 ns/op 0.92

by benchmarkbot/action

@g11tech g11tech force-pushed the peerDAS branch 4 times, most recently from 5fbbdb2 to 81aaeb5 Compare August 12, 2024 14:46
@matthewkeil
Squashed commit of the following:
5c24ff6 
commit e8bc729
Author: Matthew Keil <me@matthewkeil.com>
Date:   Tue Dec 3 10:08:30 2024 -0500

    refactor: peerdas types (#7243)

    * refactor: organize peerDAS types

    * refactor: DataColumnsData

    * refactor: rename BlockInputBlobs BlockInputColumnData

    * refactor: split up and rename BlockInputData

    * refactor: clean up BlobsData

    * refactor: clean up CachedData types

    * refactor: change from interface to type and update enum values for grafana

    * chore: lint

    * fix: remove extraneous lint fix

commit c8075d0
Author: Matthew Keil <me@matthewkeil.com>
Date:   Mon Nov 25 15:41:41 2024 +0800

    feat: log peer disconnect info (#7231)

    * feat: log disconnect reason

    * feat: log peerScore update

    * fix: pretty print peerId

    * fix: use prettyPrintPeerId

commit 8689c76
Author: Matthew Keil <me@matthewkeil.com>
Date:   Tue Oct 22 06:11:13 2024 -0400

    feat: check for no commitments on block or column in sidecar validation (#7184)

    * feat: check for no commitments on block or column in sidecar validation

    * test: add sanity check for empty blob commitments in column validation

    * fix: test bug

    * fix: but in test passing commitments

commit fccf9a2
Author: harkamal <gajinder@g11.in>
Date:   Tue Oct 8 12:44:11 2024 +0530

    add prevdownload tracker

commit 513bccc
Author: harkamal <gajinder@g11.in>
Date:   Wed Oct 2 16:46:40 2024 +0530

    fix

commit 1c08ab3
Author: harkamal <gajinder@g11.in>
Date:   Wed Oct 2 16:40:40 2024 +0530

    some fixing of beacon params

commit 7c9a01c
Author: harkamal <gajinder@g11.in>
Date:   Wed Oct 2 14:58:11 2024 +0530

    add debug and fix datacolumns migration and improve log

commit b04aaef
Author: harkamal <gajinder@g11.in>
Date:   Wed Oct 2 12:52:24 2024 +0530

    migrate datacolumns to finalized

commit a0e0087
Author: harkamal <gajinder@g11.in>
Date:   Wed Oct 2 00:36:03 2024 +0530

    more log

commit 2736b8c
Author: harkamal <gajinder@g11.in>
Date:   Tue Oct 1 19:43:59 2024 +0530

    add enhance datacolumn serving logs

commit cce193b
Author: harkamal <gajinder@g11.in>
Date:   Tue Oct 1 17:36:14 2024 +0530

    improve logging for debugging

commit a3de70f
Author: harkamal <gajinder@g11.in>
Date:   Thu Sep 26 14:43:49 2024 +0530

    turn persisting network identity to default true

commit cec27d6
Author: harkamal <gajinder@g11.in>
Date:   Sat Sep 21 22:07:58 2024 +0530

    handle edge case

commit 6a77828
Author: harkamal <gajinder@g11.in>
Date:   Sat Sep 21 20:33:53 2024 +0530

    add debug console log

commit 574837a
Author: harkamal <gajinder@g11.in>
Date:   Sat Sep 21 18:39:36 2024 +0530

    use sample subnets for data availability

commit fee7c08
Author: harkamal <gajinder@g11.in>
Date:   Tue Sep 17 16:42:45 2024 +0530

    validate inclusion proof

commit 20ef4c6
Author: Matthew Keil <me@matthewkeil.com>
Date:   Tue Sep 17 05:33:40 2024 -0400

    feat: validate data column sidecars (#7073)

    * feat: update c-kzg to final DAS version

    * refactor: use trusted-setup from c-kzg package

    * feat: implement validateDataColumnsSidecars

    * feat: check block and column commitments match

    * test: add unit test for validateDataColumnsSidecars

    * fix: invalid build and update validity condition of validateDataColumnsSidecars

    * fix: make error messages better

    * fix: electra vs peerdas type conflict

commit b1940ee
Author: Matthew Keil <me@matthewkeil.com>
Date:   Tue Sep 17 05:26:05 2024 -0400

    fix: remove ckzg build script (#7089)

    * fix: remove unused ckzg build script

    * fix: remove unused rsync dep from Dockerfile

commit bd4f7f9
Author: Matthew Keil <me@matthewkeil.com>
Date:   Mon Sep 16 06:31:52 2024 -0400

    feat: update ckzg to final DAS version (#7050)

    * feat: update c-kzg to final DAS version

    * refactor: use trusted-setup from c-kzg package

commit a33303f
Author: Matthew Keil <me@matthewkeil.com>
Date:   Mon Sep 16 04:08:29 2024 -0400

    feat: refactor and unit test getDataColumnSidecars (#7072)

    * refactor: getDataColumnSidecars

    * test: unit test getDataColumnSidecars with mocks from c-kzg library

    * refactor: use fromHex util

    * chore: update numbering on mocks

    * chore: update c-kzg to latest version

    * chore: fix type export syntax

    * test: add verification for cells from sidecars

    * test: add verification to DataColumnSidecars tests

    * refactor: getDataColumnSidecars for PR comments

    * feat: narrow type and remove unnecessary conditional

    * fix: getDataColumnSidecars param type

    * refactor: rename to computeDataColumnSidecars

commit 4ec7aff
Author: harkamal <gajinder@g11.in>
Date:   Fri Sep 13 18:19:45 2024 +0530

    edge case optimization

commit 3470076
Author: harkamal <gajinder@g11.in>
Date:   Fri Sep 13 17:31:52 2024 +0530

    more debug log

commit c4d04ee
Author: harkamal <gajinder@g11.in>
Date:   Fri Sep 13 15:35:32 2024 +0530

    fix the column id compute

commit cdd9bae
Author: harkamal <gajinder@g11.in>
Date:   Thu Sep 12 17:04:07 2024 +0530

    update compute spec tests

commit 2b10e4d
Author: harkamal <gajinder@g11.in>
Date:   Thu Sep 12 15:25:00 2024 +0530

    datacolumns retrival fix

commit 56c8c6e
Author: harkamal <gajinder@g11.in>
Date:   Thu Sep 12 13:57:51 2024 +0530

    custodied column fetch debugging log

commit af933fb
Author: harkamal <gajinder@g11.in>
Date:   Wed Sep 11 22:11:16 2024 +0530

    some fixes

commit 74d8122
Author: harkamal <gajinder@g11.in>
Date:   Wed Sep 11 21:57:41 2024 +0530

    add some log for debugging inbound data columns request

commit f7571f4
Author: harkamal <gajinder@g11.in>
Date:   Wed Sep 11 17:07:52 2024 +0530

    add some more loggig and availaibility tracking

commit d35873e
Author: harkamal <gajinder@g11.in>
Date:   Wed Sep 11 15:02:31 2024 +0530

    further wait till cutoff for all data to be available

commit 8c21168
Author: harkamal <gajinder@g11.in>
Date:   Wed Sep 11 00:02:09 2024 +0530

    make pull a little less agressive

commit de341b5
Author: harkamal <gajinder@g11.in>
Date:   Tue Sep 10 22:21:37 2024 +0530

    add send more log

commit bd84892
Author: harkamal <gajinder@g11.in>
Date:   Tue Sep 10 22:01:43 2024 +0530

    more log

commit d7721f8
Author: harkamal <gajinder@g11.in>
Date:   Tue Sep 10 21:19:11 2024 +0530

    fix bug

commit 5e1de6f
Author: harkamal <gajinder@g11.in>
Date:   Tue Sep 10 20:45:05 2024 +0530

    trying some fix

commit 2bc1a0d
Author: harkamal <gajinder@g11.in>
Date:   Tue Sep 10 20:11:05 2024 +0530

    add cache tracking

commit 387da88
Author: harkamal <gajinder@g11.in>
Date:   Tue Sep 10 19:27:05 2024 +0530

    add more log

commit aece0ab
Author: harkamal <gajinder@g11.in>
Date:   Tue Sep 10 17:46:52 2024 +0530

    fix add missing data availability resolutions

commit 006e781
Author: harkamal <gajinder@g11.in>
Date:   Sat Sep 7 20:35:00 2024 +0530

    add debug log

commit bf08852
Author: harkamal <gajinder@g11.in>
Date:   Sat Sep 7 20:08:10 2024 +0530

    resolve availability when datacolumns are downloaded and matched

commit 2833ac0
Author: harkamal <gajinder@g11.in>
Date:   Thu Sep 5 20:17:04 2024 +0530

    make the csc encoding updates as per latest spec

commit 585165e
Author: harkamal <gajinder@g11.in>
Date:   Wed Aug 28 17:35:31 2024 +0530

    remove banning unknown block, addmore log

commit a33a72f
Author: harkamal <gajinder@g11.in>
Date:   Tue Aug 27 21:59:03 2024 +0530

    subnet count 128

commit ae7678e
Author: harkamal <gajinder@g11.in>
Date:   Tue Aug 27 21:29:50 2024 +0530

    fx

commit 4b6f167
Author: harkamal <gajinder@g11.in>
Date:   Tue Aug 27 19:37:37 2024 +0530

    fix bug

commit 180f7d8
Author: harkamal <gajinder@g11.in>
Date:   Tue Aug 27 18:58:26 2024 +0530

    fix log

commit 54579b0
Author: harkamal <gajinder@g11.in>
Date:   Tue Aug 27 17:21:07 2024 +0530

    add more info for debugging

commit a3533f8
Author: harkamal <gajinder@g11.in>
Date:   Tue Aug 27 17:13:41 2024 +0530

    add supernode flag to configure node custody requirement and make it not required for validator

commit e6c613f
Author: harkamal <gajinder@g11.in>
Date:   Tue Aug 27 15:51:20 2024 +0530

    rename electra fork to peerdas for rebase and make csc in metadata uint8

commit 81aaeb5
Author: harkamal <gajinder@g11.in>
Date:   Mon Aug 12 15:43:39 2024 +0530

    feat: add and use metadatav3 for peer custody subnet

    fixes for metadata, working locally

    change the condition to update metadata csc change

commit c7f6341
Author: harkamal <gajinder@g11.in>
Date:   Fri Aug 9 17:06:52 2024 +0530

    fix the types/test

    rebase fixes

commit a0c5d27
Author: harkamal <gajinder@g11.in>
Date:   Tue Jul 16 18:54:18 2024 +0530

    fix: refactor to add and use nodeid computation and clear out nodeid tracking

    nodeid cleanup for network

commit d423004
Author: harkamal <gajinder@g11.in>
Date:   Mon Jul 15 03:12:52 2024 +0530

    feat: add the modifications to work with devnet2

    some network options to control peering behavior

    allow setting node custody capability via --params

    use eip754 names for the peerdas config

commit 47eedae
Author: harkamal <gajinder@g11.in>
Date:   Sun Jul 14 17:56:59 2024 +0530

    feat: get various sync mechanisms working with/without sharded data

commit 156ef53
Author: matthewkeil <me@matthewkeil.com>
Date:   Fri Jun 21 17:09:44 2024 +0200

    fix: docker build issue for c-kzg

    wip: REPLACE THIS COMMIT

    commit yarn lock

    rebase fixes

    fix: update c-zkg install workflow

    feat: add trustedSetupPrecompute cli flag

    fix: update trusted-setup for testing

    fix: update c-zkg install workflow to remove sudo

    fix: add rsync to apk deps

commit 499d93c
Author: harkamal <gajinder@g11.in>
Date:   Wed Jan 24 18:40:25 2024 +0530

    feat: implement peerDAS on electra

    add some presets

    add further params and types

    add data column to types repo and network

    move to max request data columns to preset

    add the datacolumns data in blockinput and fix breaking errors in seen gossip blockinput

    handle data columns in gossip and the seengossip

    further propagate forkaware blockdata and resolve build/type issues

    further handle datacolumns sync by range by root and forkaware data handling

    fix issues

    chore: update c-kzg to peerDas version

    feat: add peerDas ckzg functions to interface

    fix the lookups

    handle the publishing flow

    various sync try fixes

    fixes

    compute blob side car

    various misl debuggings and fixes

    debug and apply fixes and get range and by root sync to work will full custody

    enable syncing with lower custody requirement

    use node peerid rather than a dummy string

    get and use the nodeid from enr and correctly compute subnets and column indexes

    filterout and connect to peers only matching out custody requiremnt

    try adding custody requirement

    add protection for subnet calc

    get the sync working with devnet 0

    correctly set the enr with custody subnet info

    rebase fixes

    small refactor

commit 4805a2e
Author: harkamal <gajinder@g11.in>
Date:   Wed Jan 24 17:38:11 2024 +0530

    feat: placeholder PR for electra

    add types stub and epoch config

    fix types
@matthewkeil
fix: remove MAX_BLOBS_PER_BLOCK references
d591386
@matthewkeil
fix: types update from rebase
4e708df
@matthewkeil
fix: rebase inconsistencies in beacon-node
8b0674c
@matthewkeil
feat: change from peerdas to fulu for fork name
ad3c439
@matthewkeil
fix: electra block publish error
124c3bd
@matthewkeil
fix: update fork specific code for electra to process with deneb for …
24f006e 
…blobs
@matthewkeil
fix: update fulu types to extend electra instead of deneb
a45f8a9
@philknows philknows changed the title feat: implement peerDAS on electra feat: implement peerDAS on fulu Feb 5, 2025
@philknows philknows mentioned this pull request Feb 5, 2025
Open
3 tasks
@matthewkeil matthewkeil mentioned this pull request Feb 7, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matthewkeil matthewkeil matthewkeil left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@g11tech @matthewkeil