Pass parameters into a pipeline

[roberttriggs]

Hi,
I am really interested in this library, looking for some guidance on running mutliple pipelines with run specific parameters - my use case requires the same pipeline to run multiple times based on specific run parameters (e.q. from build queue to produce a pipeline run per customer)
so the pipeline steps are the same but passing in build parameters to make the run specific to the customer based on their build Params (these params are accessed in the modules).

At the moment I am created a new pipeline each time via a createPipeLine method:

  private static PipelineHostBuilder CreateuildPipeline(BuildParameters parameters)
  {
      var pipeline = PipelineHostBuilder.Create()
                      .ConfigureAppConfiguration((context, builder) =>
                      {
                          builder.AddJsonFile("appsettings.json")
                              .AddUserSecrets<Program>()
                              .AddEnvironmentVariables();
                      }).ConfigureServices((context, collection) =>
                      {
                          collection.Configure<SiteBuildRequest>(_ => GetSiteBuildRequest(parameters));
                      });
      pipeline.AddModule<FirstStepModule>();

      return pipeline;
  }

then execute each create pipeline

await pipeline.ExecutePipelineAsync();

Would this be the best way? (Not yet tried to see happens if I try to execute more than one concurrently yet and whether there is any issue having multiple instances of the pipeline host? (also can you confirm if multiple instances of the same pipeline would be ok?)

I wondered if there could be a way like having a version of ExecutePipeline that allows a parameters class to be passed in but the underlying pipeline is the same each time.
something like:
await pipeline.ExecutePipelineAsync(buildParameters)

[thomhurst]

I wondered if there could be a way like having a version of ExecutePipeline that allows a parameters class to be passed in but the underlying pipeline is the same each time.
something like:
await pipeline.ExecutePipelineAsync(buildParameters)

There's nothing like this currently but there's no reason you can't create a helper method for that yourself!

(Not yet tried to see happens if I try to execute more than one concurrently yet and whether there is any issue having multiple instances of the pipeline host? (also can you confirm if multiple instances of the same pipeline would be ok?)

The same object/instance can't be invoked multiple times - But if you create multiple instances, they can be each invoked concurrently. I don't know how this would perform because it depends what your pipeline is doing, and how well spec'd your machine is - So that'll be trial and error on your part I guess.

Would this be the best way?

Yes - I don't see anything wrong with your method.

[roberttriggs]

Thanks for your reply - I now have it working and it seems to run concurrently fairly nicely - the only issue I am seeing is I get the following warning:
System.InvalidOperationException: Trying to run one or more interactive functions concurrently. Operations with dynamic
displays (e.g. a prompt and a progress display) cannot be running at the same time
I believe this is down to how the module run and statuses are output to the console. I wondered if there was a way to configure a pipeline so turn that is turned off as I am handling the process with hooks and process the pipeline result object.

Not sure if this the right way but setting the pipeline LogLevel to "None" appears to make this error disappear and it's happy to run concurrent builds