Make the use of substitutes on builders configurable

[dasJ]

No description provided.

[grahamc]

Looks good, can you document this option? One caveat here is users need to be very careful that the build machines do not substitute from a caching proxy. For example, consider a Hydra which pushes paths to S3, and serves them through CloudFront. Builders must not be configured to look at CloudFront for store paths, and must directly talk to S3.

[dasJ]

Is sounds pretty much like the issue we were solving with this option :D

[dasJ]

Any suggestion where to document this option?

[grahamc]

The main hydra should also not point to the cache :).

[grahamc]

Maybe start an (anemic) section here for "queue runner settings": https://github.com/NixOS/hydra/blob/master/doc/manual/src/configuration.md#statsd-configuration ?

[dasJ]

For reference:

hydra-queue-runner.cc:    , maxUnsupportedTime(config->getIntOption("max_unsupported_time", 0))
hydra-queue-runner.cc:    , dbPool(config->getIntOption("max_db_connections", 128))
hydra-queue-runner.cc:    , maxOutputSize(config->getIntOption("max_output_size", 2ULL << 30))
hydra-queue-runner.cc:    , maxLogSize(config->getIntOption("max_log_size", 64ULL << 20))
hydra-queue-runner.cc:    , uploadLogsToBinaryCache(config->getBoolOption("upload_logs_to_binary_cache", false))
hydra-queue-runner.cc:    , rootsDir(config->getStrOption("gc_roots_dir", fmt("%s/gcroots/per-user/%s/hydra-roots", settings.nixStateDir, getEnvOrDie("LOGNAME"))))
hydra-queue-runner.cc:    if (config->getStrOption("store_mode") != "")
hydra-queue-runner.cc:    if (config->getStrOption("binary_cache_dir") != "")
hydra-queue-runner.cc:    if (config->getStrOption("binary_cache_s3_bucket") != "")
hydra-queue-runner.cc:    if (config->getStrOption("binary_cache_secret_key_file") != "")
hydra-queue-runner.cc:    auto storeUri = config->getStrOption("store_uri");
hydra-queue-runner.cc:    useSubstitutes = config->getBoolOption("use-substitutes", false);
hydra-queue-runner.cc:    useSubstitutesOnRemoteBuilders = config->getBoolOption("use_substitutes_on_remote_builders", true);
hydra-queue-runner.cc:    for (auto & s : tokenizeString<Strings>(config->getStrOption("xxx-jobset-repeats"))) {

[grahamc]

Well, it would be amazing if you made a passing note of each of those. And, I'm not going to block this PR on it.

[dasJ]

Like this?