Squash non-determinism

[gfrances]

There seems to be some source of non-determinism in the C++ backend of the planner, which provokes some slight variation of results. This can be reproduced with the following command:

python3 preprocessor/runner.py --tag t4 --instance /home/gfrances/projects/code/fs-benchmarks/benchmarks/blocksworld-fn/instance_20_2.pddl --driver lazybfws --options "successor_generation=naive,bfws.rs=sim,evaluator_t=adaptive" --run

The json files generated by the frontend are identical, but number of expansions, etc. seem to be different.
The logs generated by the (release-mode) planner are also equivalent.

[miquelramirez]

Is this confined to the functional domains? If so, surely is having an impact on the variance that I am seeing when using features.

[gfrances]

It seems so, although I'm not exactly sure what's going on... I've just found very small variations in the number of expanded nodes when running twice in a row the same command... I've only tried with the FN version of BW though...

[miquelramirez]

Ugh, I had forgotten about this. Today I sat for a long time with the proc. domains studying the effect of using features etc and observed wild variations in some domains like helper. Wild as in a 100% variation sometimes.

This variance happens even without features enabled, always when using BFWS (f7). For Blind BFWS, nothing out of place was observed after lots of runs done manually... Right now I am trying to do a controlled experiment on my laptop, to determine how big is the effect.