Fuzz testing #1255
Fuzz testing #1255
Conversation
packages/token/src/tests/erc721/test_fuzz_erc721_enumerable.cairo
Outdated
Show resolved
Hide resolved
.github/workflows/test.yml
Outdated
| @@ -48,7 +48,7 @@ jobs: | |||
| run: scarb fmt --check --workspace | |||
|
|
|||
| - name: Run tests and generate coverage report | |||
| run: snforge test --workspace --coverage | |||
| run: snforge test --workspace --coverage --features fuzzing --fuzzer-runs 10000 | |||
There was a problem hiding this comment.
I have the feeling that this will soon make tests run timeout in the action.
There was a problem hiding this comment.
Tests are not running on the PR btw, when that's fixed I'm curious to see how much time will this take with this few examples.
There was a problem hiding this comment.
I reduced the number of fuzzer runs to 500 which seems a reasonable value to me, at least for now
With 10k runs the tests took 1h 55m to complete which is way too much, here are the run results: https://github.com/OpenZeppelin/cairo-contracts/actions/runs/12374506824/job/34537193923
With 1k runs it took 17m 27s to run all the tests. 10 minutes more than our usual test runs.
Results: https://github.com/OpenZeppelin/cairo-contracts/actions/runs/12379685661/job/34554383936
Although it's an acceptable duration, it can increase much when we add more fuzz test cases. So now with 500 runs the tests job takes 12 minutes (5 minutes more than it was without fuzz tests)
There was a problem hiding this comment.
Great job @immrsd, the approach looks good to me. With this said, note that snfoundry is working on improving its fuzz testing API, so we will probably need to adapt in the future, but this looks like a good foundation. cc @ggonzalez94 regarding snfoundry probably updating the API at some point this year.
Noted! |
PR Checklist