Adding a new fuzzer

[fEst1ck]

Integrated a new fuzzer and added a experiment request.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[fEst1ck]

Hi @DonggeLiu, can you help me on running an experiment on a new fuzzer? First time using fuzz bench.

[DonggeLiu]

Hi @fEst1ck, we are very happy to run experiments for you.

Some minor issues before I can launch the experiment:

1. Would you mind making a trivial modification to service/gcbrun_experiment.py?
   This will allow me to launch experiments in this PR before merging. Here is an example to add a dummy comment.

2. In addition, could you please write your experiment request in this format?
   You can swap the --experiment-name and --fuzzers parameters with your values.
   I would really appreciate that : )

3. BTW, you don't have to modify service/experiment-requests.yaml anymore. The two items above will suffice. Sorry that our documentation is a bit outdated.

Thanks!

[fEst1ck]

Hi @fEst1ck, we are very happy to run experiments for you.

Some minor issues before I can launch the experiment:

1. Would you mind making a trivial modification to service/gcbrun_experiment.py?
   This will allow me to launch experiments in this PR before merging. Here is an example to add a dummy comment.
2. In addition, could you please write your experiment request in this format?
   You can swap the --experiment-name and --fuzzers parameters with your values.
   I would really appreciate that : )
3. BTW, you don't have to modify service/experiment-requests.yaml anymore. The two items above will suffice. Sorry that our documentation is a bit outdated.

Thanks!

Hi @DonggeLiu, thanks for the quick reply! I've made the suggested changes. The command to run the experiment should be

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-01-22-path_afl --fuzzers aflplusplus path_afl

[DonggeLiu]

Hi @DonggeLiu, thanks for the quick reply! I've made the suggested changes. The command to run the experiment should be

gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-01-22-path_afl --fuzzers aflplusplus path_afl

Nice! Thanks for addressing this promptly. I will run the experiment below.
2 minor notes:

1. No need to include a core fuzzer name (e.g., afl++) as FuzzBench will automatically merge its past experiment result into your experiment. This avoids repeatedly running the baseline fuzzers.
2. Please avoid leaving comments when the experiment (fuzzbench) CI item is running : ) Otherwise our server may fail to grab the correct experiment request.

[DonggeLiu]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-01-23-path_afl --fuzzers path_afl

[fEst1ck]

I noticed the CI immediately failed because of linter warnings. I've fixed that now (checked with presubmit locally).
Do we need to restart the experiment?

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-01-23-path_afl-1 --fuzzers path_afl

@DonggeLiu

[DonggeLiu]

We don't mind minor linter warnings either because we no longer to have merge the fuzzer before running experiments : ) Please ignore the False-negative CI failures from Fuzzers+Benchmarks if you have tested building your fuzzer locally.

The experiment CI failed because the experiment name is invalid:

experiment.run_experiment.ValidationError: Experiment name "2025-01-23-path_afl" is invalid. Must match: "^[a-z0-9-]{0,30}$"

I will fix and re-run the command below.

[DonggeLiu]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-01-23-path-afl --fuzzers path_afl

[DonggeLiu]

Experiment 2025-01-23-path-afl data and results will be available later at:
The experiment data.
The experiment report.
The experiment report(experimental).

[fEst1ck]

The result is still not available after 24 hours. Wonder if this is normal? @DonggeLiu