Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: update the test runner #7404

Open
wants to merge 41 commits into
base: unstable
Choose a base branch
from
Open

chore: update the test runner #7404

wants to merge 41 commits into from

Conversation

nazarhussain
Copy link
Contributor

@nazarhussain nazarhussain commented Jan 27, 2025 •
edited
Loading

Motivation

Update vitest and browser engine.

Description

  • Move root level shared configs to configs directory
  • Update to vitest 3.x
  • Migrate from webdriverio to playwright which is more stable and recommended by Vitest.

NOTE
Reverted the playwright upgrade. See the comments.

lodestar/configs/vitest.config.base.browser.ts

Lines 41 to 43 in d2a7af0

// Recommended provider is `playwright` but it's causing following error on CI
// Error: Failed to connect to the browser session "af5be85a-7f29-4299-b680-b07f0cfc2520" within the timeout.
// TODO: Debug the issue in later versions of playwright and vitest

Steps to test or reproduce

Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner January 27, 2025 11:28
@nazarhussain nazarhussain marked this pull request as draft January 27, 2025 11:28
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 27, 2025 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: d28a395 Previous: c43b070 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 933.35 us/op 749.50 us/op 1.25
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 34.204 us/op 29.973 us/op 1.14
BLS verify - blst 816.71 us/op 892.07 us/op 0.92
BLS verifyMultipleSignatures 3 - blst 1.1772 ms/op 1.3042 ms/op 0.90
BLS verifyMultipleSignatures 8 - blst 1.6181 ms/op 2.0867 ms/op 0.78
BLS verifyMultipleSignatures 32 - blst 4.8423 ms/op 4.5189 ms/op 1.07
BLS verifyMultipleSignatures 64 - blst 8.8932 ms/op 8.4002 ms/op 1.06
BLS verifyMultipleSignatures 128 - blst 17.120 ms/op 16.010 ms/op 1.07
BLS deserializing 10000 signatures 688.16 ms/op 594.65 ms/op 1.16
BLS deserializing 100000 signatures 6.8065 s/op 6.0440 s/op 1.13
BLS verifyMultipleSignatures - same message - 3 - blst 814.05 us/op 945.02 us/op 0.86
BLS verifyMultipleSignatures - same message - 8 - blst 1.0113 ms/op 1.0763 ms/op 0.94
BLS verifyMultipleSignatures - same message - 32 - blst 1.7022 ms/op 1.6872 ms/op 1.01
BLS verifyMultipleSignatures - same message - 64 - blst 2.6061 ms/op 2.5078 ms/op 1.04
BLS verifyMultipleSignatures - same message - 128 - blst 4.3538 ms/op 4.0633 ms/op 1.07
BLS aggregatePubkeys 32 - blst 19.499 us/op 17.367 us/op 1.12
BLS aggregatePubkeys 128 - blst 70.029 us/op 61.145 us/op 1.15
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 46.396 ms/op 39.819 ms/op 1.17
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 41.997 ms/op 46.088 ms/op 0.91
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 35.624 ms/op 34.808 ms/op 1.02
getSlashingsAndExits - default max 75.266 us/op 58.912 us/op 1.28
getSlashingsAndExits - 2k 281.99 us/op 203.28 us/op 1.39
proposeBlockBody type=full, size=empty 4.7918 ms/op 4.6684 ms/op 1.03
isKnown best case - 1 super set check 190.00 ns/op 387.00 ns/op 0.49
isKnown normal case - 2 super set checks 187.00 ns/op 386.00 ns/op 0.48
isKnown worse case - 16 super set checks 188.00 ns/op 387.00 ns/op 0.49
InMemoryCheckpointStateCache - add get delete 2.3270 us/op 2.4130 us/op 0.96
validate api signedAggregateAndProof - struct 1.3584 ms/op 1.9270 ms/op 0.70
validate gossip signedAggregateAndProof - struct 1.3710 ms/op 1.9389 ms/op 0.71
batch validate gossip attestation - vc 640000 - chunk 32 117.17 us/op 111.36 us/op 1.05
batch validate gossip attestation - vc 640000 - chunk 64 112.44 us/op 94.188 us/op 1.19
batch validate gossip attestation - vc 640000 - chunk 128 113.07 us/op 87.395 us/op 1.29
batch validate gossip attestation - vc 640000 - chunk 256 118.78 us/op 90.324 us/op 1.32
pickEth1Vote - no votes 996.41 us/op 723.65 us/op 1.38
pickEth1Vote - max votes 6.2509 ms/op 7.0104 ms/op 0.89
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.880 ms/op 14.933 ms/op 0.93
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.540 ms/op 20.980 ms/op 1.03
pickEth1Vote - Eth1Data fastSerialize value x2048 456.25 us/op 366.01 us/op 1.25
pickEth1Vote - Eth1Data fastSerialize tree x2048 3.4517 ms/op 4.2346 ms/op 0.82
bytes32 toHexString 371.00 ns/op 539.00 ns/op 0.69
bytes32 Buffer.toString(hex) 236.00 ns/op 421.00 ns/op 0.56
bytes32 Buffer.toString(hex) from Uint8Array 338.00 ns/op 494.00 ns/op 0.68
bytes32 Buffer.toString(hex) + 0x 242.00 ns/op 411.00 ns/op 0.59
Object access 1 prop 0.12000 ns/op 0.30500 ns/op 0.39
Map access 1 prop 0.12400 ns/op 0.31600 ns/op 0.39
Object get x1000 8.6690 ns/op 5.3690 ns/op 1.61
Map get x1000 6.8340 ns/op 5.9770 ns/op 1.14
Object set x1000 31.897 ns/op 18.979 ns/op 1.68
Map set x1000 21.458 ns/op 16.598 ns/op 1.29
Return object 10000 times 0.29850 ns/op 0.29490 ns/op 1.01
Throw Error 10000 times 4.5347 us/op 3.5006 us/op 1.30
toHex 141.94 ns/op 94.292 ns/op 1.51
Buffer.from 139.25 ns/op 88.918 ns/op 1.57
shared Buffer 91.234 ns/op 58.675 ns/op 1.55
fastMsgIdFn sha256 / 200 bytes 2.1930 us/op 1.8170 us/op 1.21
fastMsgIdFn h32 xxhash / 200 bytes 227.00 ns/op 374.00 ns/op 0.61
fastMsgIdFn h64 xxhash / 200 bytes 296.00 ns/op 440.00 ns/op 0.67
fastMsgIdFn sha256 / 1000 bytes 7.2830 us/op 5.8450 us/op 1.25
fastMsgIdFn h32 xxhash / 1000 bytes 358.00 ns/op 508.00 ns/op 0.70
fastMsgIdFn h64 xxhash / 1000 bytes 368.00 ns/op 514.00 ns/op 0.72
fastMsgIdFn sha256 / 10000 bytes 64.582 us/op 50.844 us/op 1.27
fastMsgIdFn h32 xxhash / 10000 bytes 1.8580 us/op 1.8840 us/op 0.99
fastMsgIdFn h64 xxhash / 10000 bytes 1.2570 us/op 1.3230 us/op 0.95
send data - 1000 256B messages 13.508 ms/op 9.7099 ms/op 1.39
send data - 1000 512B messages 17.894 ms/op 13.502 ms/op 1.33
send data - 1000 1024B messages 26.330 ms/op 19.898 ms/op 1.32
send data - 1000 1200B messages 23.872 ms/op 14.960 ms/op 1.60
send data - 1000 2048B messages 21.568 ms/op 14.644 ms/op 1.47
send data - 1000 4096B messages 23.881 ms/op 17.713 ms/op 1.35
send data - 1000 16384B messages 70.302 ms/op 44.959 ms/op 1.56
send data - 1000 65536B messages 230.83 ms/op 241.76 ms/op 0.95
enrSubnets - fastDeserialize 64 bits 924.00 ns/op 1.8000 us/op 0.51
enrSubnets - ssz BitVector 64 bits 345.00 ns/op 496.00 ns/op 0.70
enrSubnets - fastDeserialize 4 bits 150.00 ns/op 322.00 ns/op 0.47
enrSubnets - ssz BitVector 4 bits 347.00 ns/op 500.00 ns/op 0.69
prioritizePeers score -10:0 att 32-0.1 sync 2-0 116.56 us/op 100.09 us/op 1.16
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 139.63 us/op 115.71 us/op 1.21
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 201.03 us/op 169.78 us/op 1.18
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 376.85 us/op 311.84 us/op 1.21
prioritizePeers score 0:0 att 64-1 sync 4-1 470.87 us/op 366.01 us/op 1.29
array of 16000 items push then shift 1.6183 us/op 1.1878 us/op 1.36
LinkedList of 16000 items push then shift 7.0670 ns/op 7.0030 ns/op 1.01
array of 16000 items push then pop 74.238 ns/op 62.282 ns/op 1.19
LinkedList of 16000 items push then pop 6.8250 ns/op 5.9780 ns/op 1.14
array of 24000 items push then shift 2.3732 us/op 1.7413 us/op 1.36
LinkedList of 24000 items push then shift 7.1930 ns/op 6.4170 ns/op 1.12
array of 24000 items push then pop 103.91 ns/op 85.181 ns/op 1.22
LinkedList of 24000 items push then pop 6.8180 ns/op 6.0270 ns/op 1.13
intersect bitArray bitLen 8 6.2430 ns/op 5.3950 ns/op 1.16
intersect array and set length 8 37.214 ns/op 33.345 ns/op 1.12
intersect bitArray bitLen 128 29.152 ns/op 26.486 ns/op 1.10
intersect array and set length 128 608.34 ns/op 553.74 ns/op 1.10
bitArray.getTrueBitIndexes() bitLen 128 1.0480 us/op 1.1190 us/op 0.94
bitArray.getTrueBitIndexes() bitLen 248 1.8340 us/op 1.7860 us/op 1.03
bitArray.getTrueBitIndexes() bitLen 512 3.7070 us/op 3.2820 us/op 1.13
Buffer.concat 32 items 613.00 ns/op 791.00 ns/op 0.77
Uint8Array.set 32 items 1.0390 us/op 1.2430 us/op 0.84
Buffer.copy 2.3460 us/op 2.1220 us/op 1.11
Uint8Array.set - with subarray 2.4060 us/op 1.6990 us/op 1.42
Uint8Array.set - without subarray 1.3170 us/op 1.0780 us/op 1.22
getUint32 - dataview 204.00 ns/op 367.00 ns/op 0.56
getUint32 - manual 128.00 ns/op 300.00 ns/op 0.43
Set add up to 64 items then delete first 2.1283 us/op 1.6558 us/op 1.29
OrderedSet add up to 64 items then delete first 3.1524 us/op 2.5669 us/op 1.23
Set add up to 64 items then delete last 2.4165 us/op 1.8872 us/op 1.28
OrderedSet add up to 64 items then delete last 3.6562 us/op 2.8443 us/op 1.29
Set add up to 64 items then delete middle 2.5299 us/op 1.8866 us/op 1.34
OrderedSet add up to 64 items then delete middle 5.5878 us/op 4.3067 us/op 1.30
Set add up to 128 items then delete first 5.1296 us/op 3.7423 us/op 1.37
OrderedSet add up to 128 items then delete first 7.9479 us/op 5.9197 us/op 1.34
Set add up to 128 items then delete last 4.9205 us/op 3.5889 us/op 1.37
OrderedSet add up to 128 items then delete last 7.3486 us/op 5.5388 us/op 1.33
Set add up to 128 items then delete middle 4.8499 us/op 3.5991 us/op 1.35
OrderedSet add up to 128 items then delete middle 13.324 us/op 11.274 us/op 1.18
Set add up to 256 items then delete first 10.153 us/op 7.3338 us/op 1.38
OrderedSet add up to 256 items then delete first 16.056 us/op 11.796 us/op 1.36
Set add up to 256 items then delete last 9.3227 us/op 7.1036 us/op 1.31
OrderedSet add up to 256 items then delete last 14.546 us/op 11.197 us/op 1.30
Set add up to 256 items then delete middle 10.634 us/op 7.1072 us/op 1.50
OrderedSet add up to 256 items then delete middle 44.583 us/op 32.927 us/op 1.35
transfer serialized Status (84 B) 2.3150 us/op 2.0460 us/op 1.13
copy serialized Status (84 B) 1.2530 us/op 1.3050 us/op 0.96
transfer serialized SignedVoluntaryExit (112 B) 2.3180 us/op 1.9900 us/op 1.16
copy serialized SignedVoluntaryExit (112 B) 1.2750 us/op 1.2290 us/op 1.04
transfer serialized ProposerSlashing (416 B) 2.4260 us/op 2.1210 us/op 1.14
copy serialized ProposerSlashing (416 B) 1.7630 us/op 1.6750 us/op 1.05
transfer serialized Attestation (485 B) 2.4170 us/op 2.1340 us/op 1.13
copy serialized Attestation (485 B) 1.3320 us/op 1.3830 us/op 0.96
transfer serialized AttesterSlashing (33232 B) 2.5250 us/op 3.0010 us/op 0.84
copy serialized AttesterSlashing (33232 B) 3.7890 us/op 4.1940 us/op 0.90
transfer serialized Small SignedBeaconBlock (128000 B) 3.1130 us/op 2.9180 us/op 1.07
copy serialized Small SignedBeaconBlock (128000 B) 9.5610 us/op 8.9880 us/op 1.06
transfer serialized Avg SignedBeaconBlock (200000 B) 3.6400 us/op 2.9290 us/op 1.24
copy serialized Avg SignedBeaconBlock (200000 B) 16.863 us/op 12.681 us/op 1.33
transfer serialized BlobsSidecar (524380 B) 4.0500 us/op 3.3090 us/op 1.22
copy serialized BlobsSidecar (524380 B) 62.284 us/op 54.829 us/op 1.14
transfer serialized Big SignedBeaconBlock (1000000 B) 4.5530 us/op 3.6640 us/op 1.24
copy serialized Big SignedBeaconBlock (1000000 B) 115.13 us/op 167.50 us/op 0.69
pass gossip attestations to forkchoice per slot 2.9581 ms/op 2.3013 ms/op 1.29
forkChoice updateHead vc 100000 bc 64 eq 0 467.55 us/op 347.98 us/op 1.34
forkChoice updateHead vc 600000 bc 64 eq 0 2.9963 ms/op 2.1672 ms/op 1.38
forkChoice updateHead vc 1000000 bc 64 eq 0 4.7636 ms/op 3.7719 ms/op 1.26
forkChoice updateHead vc 600000 bc 320 eq 0 2.8048 ms/op 2.1259 ms/op 1.32
forkChoice updateHead vc 600000 bc 1200 eq 0 2.8792 ms/op 2.1320 ms/op 1.35
forkChoice updateHead vc 600000 bc 7200 eq 0 3.1502 ms/op 2.4135 ms/op 1.31
forkChoice updateHead vc 600000 bc 64 eq 1000 10.311 ms/op 9.1754 ms/op 1.12
forkChoice updateHead vc 600000 bc 64 eq 10000 10.209 ms/op 9.0711 ms/op 1.13
forkChoice updateHead vc 600000 bc 64 eq 300000 13.654 ms/op 10.936 ms/op 1.25
computeDeltas 500000 validators 300 proto nodes 3.9099 ms/op 3.1029 ms/op 1.26
computeDeltas 500000 validators 1200 proto nodes 3.9884 ms/op 3.1632 ms/op 1.26
computeDeltas 500000 validators 7200 proto nodes 3.9810 ms/op 3.2355 ms/op 1.23
computeDeltas 750000 validators 300 proto nodes 5.8683 ms/op 4.8869 ms/op 1.20
computeDeltas 750000 validators 1200 proto nodes 5.8484 ms/op 4.5519 ms/op 1.28
computeDeltas 750000 validators 7200 proto nodes 5.7885 ms/op 4.6906 ms/op 1.23
computeDeltas 1400000 validators 300 proto nodes 10.841 ms/op 8.7767 ms/op 1.24
computeDeltas 1400000 validators 1200 proto nodes 10.816 ms/op 8.7171 ms/op 1.24
computeDeltas 1400000 validators 7200 proto nodes 10.961 ms/op 8.6016 ms/op 1.27
computeDeltas 2100000 validators 300 proto nodes 16.555 ms/op 12.655 ms/op 1.31
computeDeltas 2100000 validators 1200 proto nodes 16.967 ms/op 12.797 ms/op 1.33
computeDeltas 2100000 validators 7200 proto nodes 19.024 ms/op 12.841 ms/op 1.48
altair processAttestation - 250000 vs - 7PWei normalcase 2.1195 ms/op 1.6911 ms/op 1.25
altair processAttestation - 250000 vs - 7PWei worstcase 3.1104 ms/op 2.5342 ms/op 1.23
altair processAttestation - setStatus - 1/6 committees join 231.90 us/op 93.804 us/op 2.47
altair processAttestation - setStatus - 1/3 committees join 259.43 us/op 187.46 us/op 1.38
altair processAttestation - setStatus - 1/2 committees join 347.90 us/op 278.95 us/op 1.25
altair processAttestation - setStatus - 2/3 committees join 524.51 us/op 341.71 us/op 1.53
altair processAttestation - setStatus - 4/5 committees join 627.41 us/op 510.93 us/op 1.23
altair processAttestation - setStatus - 100% committees join 775.17 us/op 599.04 us/op 1.29
altair processBlock - 250000 vs - 7PWei normalcase 5.4652 ms/op 3.8238 ms/op 1.43
altair processBlock - 250000 vs - 7PWei normalcase hashState 37.050 ms/op 33.493 ms/op 1.11
altair processBlock - 250000 vs - 7PWei worstcase 38.241 ms/op 40.801 ms/op 0.94
altair processBlock - 250000 vs - 7PWei worstcase hashState 93.632 ms/op 86.163 ms/op 1.09
phase0 processBlock - 250000 vs - 7PWei normalcase 1.8872 ms/op 1.9932 ms/op 0.95
phase0 processBlock - 250000 vs - 7PWei worstcase 30.950 ms/op 25.865 ms/op 1.20
altair processEth1Data - 250000 vs - 7PWei normalcase 338.63 us/op 257.31 us/op 1.32
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 6.0440 us/op 5.0580 us/op 1.19
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 43.914 us/op 24.138 us/op 1.82
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 9.4430 us/op 7.4970 us/op 1.26
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 7.6420 us/op 7.3410 us/op 1.04
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 179.22 us/op 106.06 us/op 1.69
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.0048 ms/op 1.1193 ms/op 0.90
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.4007 ms/op 986.18 us/op 1.42
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.4208 ms/op 1.0155 ms/op 1.40
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 4.2336 ms/op 2.4592 ms/op 1.72
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.6039 ms/op 999.14 us/op 1.61
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.8754 ms/op 2.5734 ms/op 1.51
Tree 40 250000 create 743.69 ms/op 371.24 ms/op 2.00
Tree 40 250000 get(125000) 148.60 ns/op 118.38 ns/op 1.26
Tree 40 250000 set(125000) 2.2150 us/op 1.2886 us/op 1.72
Tree 40 250000 toArray() 27.001 ms/op 9.9550 ms/op 2.71
Tree 40 250000 iterate all - toArray() + loop 24.403 ms/op 10.022 ms/op 2.43
Tree 40 250000 iterate all - get(i) 64.994 ms/op 38.922 ms/op 1.67
Array 250000 create 2.8746 ms/op 2.3702 ms/op 1.21
Array 250000 clone - spread 2.8004 ms/op 649.32 us/op 4.31
Array 250000 get(125000) 0.43600 ns/op 0.57700 ns/op 0.76
Array 250000 set(125000) 0.45400 ns/op 0.58400 ns/op 0.78
Array 250000 iterate all - loop 88.481 us/op 76.877 us/op 1.15
phase0 afterProcessEpoch - 250000 vs - 7PWei 51.939 ms/op 40.374 ms/op 1.29
Array.fill - length 1000000 6.9626 ms/op 2.3988 ms/op 2.90
Array push - length 1000000 19.137 ms/op 11.285 ms/op 1.70
Array.get 0.38476 ns/op 0.26129 ns/op 1.47
Uint8Array.get 0.47404 ns/op 0.34540 ns/op 1.37
phase0 beforeProcessEpoch - 250000 vs - 7PWei 28.987 ms/op 12.893 ms/op 2.25
altair processEpoch - mainnet_e81889 371.17 ms/op 273.18 ms/op 1.36
mainnet_e81889 - altair beforeProcessEpoch 28.728 ms/op 16.191 ms/op 1.77
mainnet_e81889 - altair processJustificationAndFinalization 6.6460 us/op 4.5950 us/op 1.45
mainnet_e81889 - altair processInactivityUpdates 6.4945 ms/op 3.3280 ms/op 1.95
mainnet_e81889 - altair processRewardsAndPenalties 62.492 ms/op 48.600 ms/op 1.29
mainnet_e81889 - altair processRegistryUpdates 1.2670 us/op 816.00 ns/op 1.55
mainnet_e81889 - altair processSlashings 234.00 ns/op 399.00 ns/op 0.59
mainnet_e81889 - altair processEth1DataReset 299.00 ns/op 396.00 ns/op 0.76
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.4415 ms/op 941.91 us/op 1.53
mainnet_e81889 - altair processSlashingsReset 1.3400 us/op 1.0960 us/op 1.22
mainnet_e81889 - altair processRandaoMixesReset 1.8680 us/op 1.5170 us/op 1.23
mainnet_e81889 - altair processHistoricalRootsUpdate 290.00 ns/op 412.00 ns/op 0.70
mainnet_e81889 - altair processParticipationFlagUpdates 870.00 ns/op 847.00 ns/op 1.03
mainnet_e81889 - altair processSyncCommitteeUpdates 164.00 ns/op 365.00 ns/op 0.45
mainnet_e81889 - altair afterProcessEpoch 59.790 ms/op 41.137 ms/op 1.45
capella processEpoch - mainnet_e217614 1.1388 s/op 901.94 ms/op 1.26
mainnet_e217614 - capella beforeProcessEpoch 98.069 ms/op 67.211 ms/op 1.46
mainnet_e217614 - capella processJustificationAndFinalization 8.5160 us/op 5.0100 us/op 1.70
mainnet_e217614 - capella processInactivityUpdates 26.007 ms/op 15.842 ms/op 1.64
mainnet_e217614 - capella processRewardsAndPenalties 245.34 ms/op 183.89 ms/op 1.33
mainnet_e217614 - capella processRegistryUpdates 7.0630 us/op 6.8570 us/op 1.03
mainnet_e217614 - capella processSlashings 245.00 ns/op 459.00 ns/op 0.53
mainnet_e217614 - capella processEth1DataReset 181.00 ns/op 395.00 ns/op 0.46
mainnet_e217614 - capella processEffectiveBalanceUpdates 14.709 ms/op 11.715 ms/op 1.26
mainnet_e217614 - capella processSlashingsReset 924.00 ns/op 1.2090 us/op 0.76
mainnet_e217614 - capella processRandaoMixesReset 1.3100 us/op 1.2620 us/op 1.04
mainnet_e217614 - capella processHistoricalRootsUpdate 184.00 ns/op 397.00 ns/op 0.46
mainnet_e217614 - capella processParticipationFlagUpdates 561.00 ns/op 735.00 ns/op 0.76
mainnet_e217614 - capella afterProcessEpoch 132.28 ms/op 105.42 ms/op 1.25
phase0 processEpoch - mainnet_e58758 326.76 ms/op 300.56 ms/op 1.09
mainnet_e58758 - phase0 beforeProcessEpoch 76.063 ms/op 86.306 ms/op 0.88
mainnet_e58758 - phase0 processJustificationAndFinalization 5.7000 us/op 5.9160 us/op 0.96
mainnet_e58758 - phase0 processRewardsAndPenalties 52.338 ms/op 41.483 ms/op 1.26
mainnet_e58758 - phase0 processRegistryUpdates 4.5830 us/op 2.7940 us/op 1.64
mainnet_e58758 - phase0 processSlashings 276.00 ns/op 405.00 ns/op 0.68
mainnet_e58758 - phase0 processEth1DataReset 189.00 ns/op 401.00 ns/op 0.47
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.1642 ms/op 840.16 us/op 1.39
mainnet_e58758 - phase0 processSlashingsReset 1.5610 us/op 1.1550 us/op 1.35
mainnet_e58758 - phase0 processRandaoMixesReset 1.7840 us/op 1.5880 us/op 1.12
mainnet_e58758 - phase0 processHistoricalRootsUpdate 326.00 ns/op 407.00 ns/op 0.80
mainnet_e58758 - phase0 processParticipationRecordUpdates 1.0500 us/op 1.2960 us/op 0.81
mainnet_e58758 - phase0 afterProcessEpoch 49.204 ms/op 35.591 ms/op 1.38
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.6119 ms/op 977.63 us/op 1.65
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 6.3706 ms/op 2.3424 ms/op 2.72
altair processInactivityUpdates - 250000 normalcase 25.664 ms/op 17.057 ms/op 1.50
altair processInactivityUpdates - 250000 worstcase 20.576 ms/op 18.277 ms/op 1.13
phase0 processRegistryUpdates - 250000 normalcase 9.6820 us/op 3.5640 us/op 2.72
phase0 processRegistryUpdates - 250000 badcase_full_deposits 344.75 us/op 309.42 us/op 1.11
phase0 processRegistryUpdates - 250000 worstcase 0.5 168.14 ms/op 106.82 ms/op 1.57
altair processRewardsAndPenalties - 250000 normalcase 61.448 ms/op 32.511 ms/op 1.89
altair processRewardsAndPenalties - 250000 worstcase 47.017 ms/op 51.066 ms/op 0.92
phase0 getAttestationDeltas - 250000 normalcase 13.810 ms/op 4.9848 ms/op 2.77
phase0 getAttestationDeltas - 250000 worstcase 8.7554 ms/op 6.5717 ms/op 1.33
phase0 processSlashings - 250000 worstcase 112.72 us/op 98.931 us/op 1.14
altair processSyncCommitteeUpdates - 250000 144.52 ms/op 112.26 ms/op 1.29
BeaconState.hashTreeRoot - No change 221.00 ns/op 447.00 ns/op 0.49
BeaconState.hashTreeRoot - 1 full validator 89.922 us/op 89.500 us/op 1.00
BeaconState.hashTreeRoot - 32 full validator 927.90 us/op 1.1925 ms/op 0.78
BeaconState.hashTreeRoot - 512 full validator 13.706 ms/op 12.112 ms/op 1.13
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 158.62 us/op 111.69 us/op 1.42
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.7419 ms/op 1.7367 ms/op 1.00
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 28.739 ms/op 28.048 ms/op 1.02
BeaconState.hashTreeRoot - 1 balances 95.321 us/op 72.065 us/op 1.32
BeaconState.hashTreeRoot - 32 balances 874.67 us/op 830.33 us/op 1.05
BeaconState.hashTreeRoot - 512 balances 12.978 ms/op 9.8756 ms/op 1.31
BeaconState.hashTreeRoot - 250000 balances 221.35 ms/op 171.93 ms/op 1.29
aggregationBits - 2048 els - zipIndexesInBitList 25.174 us/op 18.602 us/op 1.35
byteArrayEquals 32 55.876 ns/op 46.405 ns/op 1.20
Buffer.compare 32 17.764 ns/op 16.501 ns/op 1.08
byteArrayEquals 1024 1.6599 us/op 1.2831 us/op 1.29
Buffer.compare 1024 26.325 ns/op 32.906 ns/op 0.80
byteArrayEquals 16384 26.551 us/op 20.400 us/op 1.30
Buffer.compare 16384 209.25 ns/op 193.53 ns/op 1.08
byteArrayEquals 123687377 214.88 ms/op 150.00 ms/op 1.43
Buffer.compare 123687377 11.378 ms/op 5.8404 ms/op 1.95
byteArrayEquals 32 - diff last byte 56.626 ns/op 44.958 ns/op 1.26
Buffer.compare 32 - diff last byte 18.475 ns/op 15.206 ns/op 1.21
byteArrayEquals 1024 - diff last byte 1.7439 us/op 1.2269 us/op 1.42
Buffer.compare 1024 - diff last byte 33.951 ns/op 23.352 ns/op 1.45
byteArrayEquals 16384 - diff last byte 27.881 us/op 19.475 us/op 1.43
Buffer.compare 16384 - diff last byte 259.07 ns/op 170.00 ns/op 1.52
byteArrayEquals 123687377 - diff last byte 210.82 ms/op 152.85 ms/op 1.38
Buffer.compare 123687377 - diff last byte 8.6994 ms/op 3.9317 ms/op 2.21
byteArrayEquals 32 - random bytes 5.2770 ns/op 4.9300 ns/op 1.07
Buffer.compare 32 - random bytes 17.710 ns/op 16.923 ns/op 1.05
byteArrayEquals 1024 - random bytes 5.2800 ns/op 4.9250 ns/op 1.07
Buffer.compare 1024 - random bytes 17.647 ns/op 17.037 ns/op 1.04
byteArrayEquals 16384 - random bytes 5.3030 ns/op 4.8750 ns/op 1.09
Buffer.compare 16384 - random bytes 17.677 ns/op 17.061 ns/op 1.04
byteArrayEquals 123687377 - random bytes 6.7000 ns/op 7.8000 ns/op 0.86
Buffer.compare 123687377 - random bytes 19.200 ns/op 19.600 ns/op 0.98
regular array get 100000 times 42.665 us/op 30.739 us/op 1.39
wrappedArray get 100000 times 34.416 us/op 30.610 us/op 1.12
arrayWithProxy get 100000 times 14.562 ms/op 11.019 ms/op 1.32
ssz.Root.equals 47.309 ns/op 42.515 ns/op 1.11
byteArrayEquals 46.350 ns/op 43.225 ns/op 1.07
Buffer.compare 10.836 ns/op 10.551 ns/op 1.03
processSlot - 1 slots 11.624 us/op 8.9470 us/op 1.30
processSlot - 32 slots 2.9185 ms/op 2.1582 ms/op 1.35
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 56.721 ms/op 59.684 ms/op 0.95
getCommitteeAssignments - req 1 vs - 250000 vc 2.1708 ms/op 1.7046 ms/op 1.27
getCommitteeAssignments - req 100 vs - 250000 vc 4.3259 ms/op 3.3692 ms/op 1.28
getCommitteeAssignments - req 1000 vs - 250000 vc 4.5715 ms/op 3.5696 ms/op 1.28
compare ViewDUs 7.2859 s/op 6.0514 s/op 1.20
compare each validator Uint8Array 1.3754 s/op 863.10 ms/op 1.59
compare ViewDU to Uint8Array 1.2973 s/op 890.80 ms/op 1.46
migrate state 1000000 validators, 24 modified, 0 new 843.12 ms/op 793.03 ms/op 1.06
migrate state 1000000 validators, 1700 modified, 1000 new 1.3777 s/op 1.1929 s/op 1.15
migrate state 1000000 validators, 3400 modified, 2000 new 1.3900 s/op 1.1763 s/op 1.18
migrate state 1500000 validators, 24 modified, 0 new 1.0967 s/op 869.21 ms/op 1.26
migrate state 1500000 validators, 1700 modified, 1000 new 1.2286 s/op 1.0163 s/op 1.21
migrate state 1500000 validators, 3400 modified, 2000 new 1.5498 s/op 1.1690 s/op 1.33
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.5200 ns/op 5.6600 ns/op 0.80
state getBlockRootAtSlot - 250000 vs - 7PWei 607.02 ns/op 452.32 ns/op 1.34
computeProposers - vc 250000 8.4865 ms/op 6.9085 ms/op 1.23
computeEpochShuffling - vc 250000 43.671 ms/op 34.276 ms/op 1.27
getNextSyncCommittee - vc 250000 165.45 ms/op 98.239 ms/op 1.68
computeSigningRoot for AttestationData 37.914 us/op 21.912 us/op 1.73
hash AttestationData serialized data then Buffer.toString(base64) 1.6444 us/op 1.2096 us/op 1.36
toHexString serialized data 1.0011 us/op 725.45 ns/op 1.38
Buffer.toString(base64) 168.52 ns/op 114.65 ns/op 1.47
nodejs block root to RootHex using toHex 151.46 ns/op 113.96 ns/op 1.33
nodejs block root to RootHex using toRootHex 98.950 ns/op 74.577 ns/op 1.33
browser block root to RootHex using the deprecated toHexString 243.54 ns/op 195.20 ns/op 1.25
browser block root to RootHex using toHex 181.18 ns/op 160.01 ns/op 1.13
browser block root to RootHex using toRootHex 165.22 ns/op 147.10 ns/op 1.12

by benchmarkbot/action

@nazarhussain nazarhussain changed the title chore: update the test runner and tests tasks chore: update the test runner and browser engine Jan 28, 2025
@nazarhussain nazarhussain marked this pull request as ready for review January 28, 2025 15:14
@wemeetagain
Copy link
Member

does your vitest-in-process-pool work with vitest 3.x?

@nazarhussain
Copy link
Contributor Author

does your vitest-in-process-pool work with vitest 3.x?

No there is some issue that is causing it to not be compatible. Was looking into it ChainSafe/benchmark#29. Will upgrade it very soon.

@codecov Codecov
Copy link

codecov bot commented Jan 29, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 48.24%. Comparing base (ee9d4fa) to head (01c80ca).
Report is 4 commits behind head on unstable.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #7404      +/-   ##
============================================
- Coverage     48.45%   48.24%   -0.21%     
============================================
  Files           602      602              
  Lines         40387    40248     -139     
  Branches       2069     2069              
============================================
- Hits          19568    19417     -151     
- Misses        20781    20793      +12     
  Partials         38       38

@nazarhussain nazarhussain changed the title chore: update the test runner and browser engine chore: update the test runner Jan 30, 2025
@wemeetagain
Copy link
Member

does this PR need vitest 3.x? can we maintain 2.x for now?

@nazarhussain
Copy link
Contributor Author

does this PR need vitest 3.x? can we maintain 2.x for now?

The Vitest 3 comes with a lot of changes, but for our interests are:

Vite 6 is considered more performant (specially for frontend stack). I was more interested in workspaces and started this PR because of it, but could not get that fit in our workflow. One issue running tests from individual package folder.

So nothing critical for us, so can hold this PR merge or merge it. As you feel better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@nazarhussain @wemeetagain