Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce TrialRunner Abstraction #720

Merged
merged 279 commits into from
Jan 10, 2025
Merged

Introduce TrialRunner Abstraction #720

merged 279 commits into from
Jan 10, 2025

Conversation

bpkroth
Copy link
Contributor

@bpkroth bpkroth commented Mar 19, 2024 •
edited
Loading

This is another step in adding support for parallel trial execution #380.

Here we separate out the running of an individual trial to a single class - TrialRunner.

Multiple TrialRunners are instantiated at CLI invocation with the --num-trial-runners argument. 
Each TrialRunner associated with a single copy of the root Environment and its Services (important for eventual parallelization reasons), and made unique by means of a unique trial_runner_id value that's also included in that Environment's global_config.

In future PRs we will add:

  • New Scheduler implementations to run TrialRunners in parallel.
  • Async polling of status results in each TrialRunner independently.

motus and others added 30 commits February 27, 2024 17:42
@motus
better handling of warm-up phase (no redundant code)
b95100a
@motus
split the sccheduler implementation into the base class and the sync …
e15033d 
…scheduler implementation
@motus
use the new scheduler in _main()
6eab1b0
@motus
add scheduler config parameters that can be overridden from global co…
9c7f2cc 
…nfig
@motus
add todo comments
479a5ed
@motus
update the scores for launcher unit tests + fix teh regexps
50dad9f
@motus
add logging to the sync optimization loop
220ece1
@motus
add more logging to the scheduler class
29cec19
@motus
move (sync) implementation of the run_trial() to SyncScheduler; other…
6f8bb2c 
… minor refactorings
@bpkroth
wip
6adb2d0
@bpkroth
start tracking which trial runner a trial is assigned to
41a0c37
@motus
Merge branch 'main' of github.com:microsoft/MLOS into sergiym/run/sch…
2453427 
…eduler
@motus
Merge branch 'sergiym/run/scheduler' of github.com:motus/MLOS into se…
d8d8dfb 
…rgiym/run/scheduler
@bpkroth
wip: adding trial runner
8a32e5a
@bpkroth
Merge remote-tracking branch 'sergiy/sergiym/run/scheduler' into para…
57dc4c3 
…llel-async-trial-runners
@bpkroth
wip: integrating trial runner to merged branch
e55f33e
@motus
Roll back forceful assignment of PATH when invoking a local process
7df0770
@motus
instantiate Scheduler from JSON config in the launcher (no JSON schem…
da55c5e 
…a yet)
@motus
fix unit tests
f6eb5ef
@motus
add test for Launcher scheduler load in test_load_cli_config_examples…
97438e7 
…_via_launcher
@motus
Merge branch 'sergiym/local_exec/env' into sergiym/run/scheduler_load
715fab9
@motus
fix the way launcher handles trial_config_repeat_count
034aef9
@motus
minor type fixes
629236f
@motus
add required_keys for base Scheduler
049fdb6
@motus
remove unnecessary type annotation
094155c
@motus
typo in pylint exception
a6a7283
@motus
make all unit tests run
0a94a37
@motus
add a missing import
cf42730
@motus
add ConfigSchema.SCHEDULER (not defined yet)
6f31a2d
@motus
fix the teardown property propagation issue
e6ceb5c
bpkroth and others added 2 commits January 9, 2025 18:59
@bpkroth @motus
Update mlos_bench/mlos_bench/launcher.py
8b94f73 
Co-authored-by: Sergiy Matusevych <sergiy.matusevych@gmail.com>
@bpkroth @motus
Update mlos_bench/mlos_bench/schedulers/base_scheduler.py
bd85ea2 
Co-authored-by: Sergiy Matusevych <sergiy.matusevych@gmail.com>
bpkroth
bpkroth commented Jan 10, 2025
View reviewed changes
This was referenced Jan 10, 2025
Closed
Closed
motus
motus approved these changes Jan 10, 2025
View reviewed changes
Copy link
Member

@motus motus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Left a few minor comments

bpkroth and others added 5 commits January 10, 2025 16:09
@bpkroth @motus
Update mlos_bench/mlos_bench/tests/storage/trial_config_test.py
a7b1c61 
Co-authored-by: Sergiy Matusevych <sergiy.matusevych@gmail.com>
@bpkroth @motus
Update mlos_bench/mlos_bench/environments/base_environment.py
f503b85 
Co-authored-by: Sergiy Matusevych <sergiy.matusevych@gmail.com>
@bpkroth @motus
Update mlos_bench/mlos_bench/environments/base_environment.py
e4a3e9e 
Co-authored-by: Sergiy Matusevych <sergiy.matusevych@gmail.com>
@bpkroth
isort
6c743f4
@bpkroth
apply suggestions
6516eed
@bpkroth bpkroth enabled auto-merge (squash) January 10, 2025 22:15
@bpkroth bpkroth mentioned this pull request Jan 10, 2025
Merged
@bpkroth bpkroth merged commit 6ffe546 into microsoft:main Jan 10, 2025
15 checks passed
@bpkroth bpkroth deleted the trial-runner-abstraction branch January 10, 2025 22:30
This was referenced Jan 10, 2025
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@motus motus motus approved these changes

Assignees
No one assigned
Labels
ready for review Ready for review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bpkroth @motus