A Python library for autonomously learning simulation parameters from experimental data, from the micro to the macro, from laptops to clusters. This is done using either of two closely related frameworks:
- CoExSiST: Coupled Experimental-Simulational Study Tool.
- ACCES: Autonomous Characterisation and Calibration via Evolutionary Simulation.
Both libraries learn a given set of free parameters, such that an experiment is synchronised with an equivalent simulation; this synchronisation is done in one of two ways:
- CoExSiST calibrates microscopically: in a Discrete Element Method (DEM) context, all simulated particles follow their experimental counterparts exactly. Naturally, this technique is limited to dilute systems and experimental imaging techniques that can capture the 3D position of all moving particles (e.g. PIV) - however, it provides information about the fundamental aspects of particle collision.
- ACCES calibrates / optimises macroscopically: a given simulation reproduces a system-specific macroscopic quantity (e.g. residence time distribution, angle of repose). This technique is completely agnostic to the simulation method and the quantity to be reproduced. For example, it can train coarse-grained DEM simulations, using larger meso-particles to model multiple smaller ones.
ACCES is ready for production use; it was successfully used to calibrate coarse-grained DEM digital twins of GranuTools equipment (Andrei Leonard Nicusan and Dominik Werner, paper under review), CFDEM fluidised beds (Hanqiao Cha, paper under review) and even signal processing parameters in a PET scanner model (Matthew Herald, paper under review).
Example of an ACCES-calibrated DEM Digital Twin of a GranuTools GranuDrum; the calibration was done effectively against a single experimental data point - a photograph of the free surface shape yielded by MCC particles (left panel). The occupancy grid of a LIGGGHTS simulation was optimised against the free surface shape (right panel). The two superimposed grids amount to 4 mm² dissimilarity (dark blue pixels, middle panel).
ACCES was implemented in the coexist.Access class, providing an interface that is easy to use, but powerful enough to automatically parallelise arbitrary Python scripts through code inspection and metaprogramming. It was used successfully from laptop-scale shared-memory machines to multi-node supercomputing clusters.
This is a pure Python package that does not require any extra system configuration, supporting Python 3.6 and above (though it might work with even older versions) - to install it from PyPI, simply run:
pip install coexist
Or you can install the development version from the GitHub repository:
pip install git+https://github.com/uob-positron-imaging-centre/Coexist
The documentation website contains an ACCES tutorial with explained code and output figures produced by coexist; all public functionality is fully documented in the manual.
Want something more hands on? Check out the examples folder for example scripts using coexist.Coexist and coexist.Access; examples/access_simple is a very simple, hackable example script (remember that the simulation_script.py can execute anything). For a more involved, complete calibration of a GranuTools GranuDrum digital twin - using LIGGGHTS - see our collection of peer-reviewed digital twins repository.
The coexist.plots submodule can plot the convergence of ACCES onto optimally-calibrated parameters - or check intermediate results while your 50-hour simulations runs:
Alright, here's the gist of it: instead of having to rewrite your complex simulation into a function for calibration / optimisation (which is what virtually all optimisation frameworks require...), ACCES takes in an entire simulation script; here's a simple example, say in a file called simulation_script.py:
# File: simulation_script.py
# ACCESS PARAMETERS START
import coexist
parameters = coexist.create_parameters(
variables = ["CED", "CoR", "Epsilon", "Mu"],
minimums = [-5, -5, -5, -5],
maximums = [+5, +5, +5, +5],
values = [0, 0, 0, 0], # Optional, initial guess
)
access_id = 0 # Optional, unique ID for each simulation
# ACCESS PARAMETERS END
# Extract the free parameters' values - ACCES will modify / optimise them.
x, y, z, t = parameters["value"]
# Define the error value in *any* way - run a simulation, analyse data, etc.
#
# For simplicity, here's an analytical 4D Himmelblau function with 8 global
# and 2 local minima - the initial guess is very close to the local one!
error = (x**2 + y - 11)**2 + (x + y**2 - 7)**2 + z * tThen you can run in a separate script or Python console:
# File: access_learn.py
import coexist
# Use ACCESS to learn a simulation's parameters
access = coexist.Access("simulation_script.py")
access.learn(
num_solutions = 10, # Number of solutions per epoch
target_sigma = 0.1, # Target std-dev (accuracy) of solution
random_seed = 42, # Reproducible / deterministic optimisation
)
# Or simply
# coexist.Access("simulation_script.py").learn(random_seed = 42)That's it - ACCES will take the simulation_script.py, modify the free parameters, run the script in parallel (either on all processors of your local machine or on a distributed supercomputer) and optimise the error you defined; it'll look something like this:
================================================================================
Starting ACCES run at 08:32:23 on 01/02/2022 in directory `access_seed42`.
================================================================================
Epoch 0 | Population 10
--------------------------------------------------------------------------------
Scaled overall standard deviation: 1.0
CED CoR Epsilon Mu
estimate 0.0 0.000000 0.000000 0.000000
uncertainty 4.0 4.000050 4.000100 4.000150
scaled_std 1.0 1.000013 1.000025 1.000038
CED CoR Epsilon Mu error
0 1.218868 -4.159988 3.001880 3.762400 331.093228
1 -3.095859 -4.967675 0.511374 -1.265018 252.732762
2 -0.067205 -3.412218 3.517680 3.111284 239.466423
3 0.264123 4.509021 1.870084 -3.437299 219.638764
4 1.475003 -3.835578 3.513889 -0.199711 243.967225
5 -0.739449 -2.723752 4.825957 -0.618141 170.752129
6 -1.713311 -1.408552 2.129290 1.461831 138.135979
7 1.650930 1.723306 2.333195 -1.625721 44.785098
8 -2.048971 -3.255132 2.463979 4.516059 114.660635
9 -0.455790 -3.360668 -3.298007 2.602469 206.454823
Total function evaluations: 10
================================================================================
Epoch 1 | Population 10
--------------------------------------------------------------------------------
...
<output truncated>
...
================================================================================
Epoch 31 | Population 10
--------------------------------------------------------------------------------
Scaled overall standard deviation: 0.1032642976669169
CED CoR Epsilon Mu
estimate 3.621098 -1.748473 4.999445 -4.992881
uncertainty 0.052355 0.078080 0.284105 0.180196
scaled_std 0.013089 0.019520 0.071026 0.045049
CED CoR Epsilon Mu error
310 3.574473 -1.650134 4.999374 -4.901913 -23.996813
311 3.582026 -1.756384 4.863496 -4.973417 -24.071692
312 3.628789 -1.616891 4.859678 -4.992656 -23.386000
313 3.662351 -1.782704 4.982040 -4.998059 -24.478008
314 3.594971 -1.725715 4.999877 -4.898257 -24.269162
315 3.577725 -1.744971 4.998411 -4.956502 -24.629198
316 3.613914 -1.747412 4.983253 -4.996630 -24.690880
317 3.579212 -1.774811 4.852262 -4.972910 -24.055219
318 3.634952 -1.784927 4.999971 -4.999863 -24.783959
319 3.647169 -1.872419 4.999971 -4.978640 -24.685207
Total function evaluations: 320
Optimal solution found within `target_sigma`, i.e. 10.0%:
sigma = 0.08390460663313491 < 0.1
================================================================================
The best result was found in 32 epochs:
CED CoR Epsilon Mu error
value 3.569249 -1.813354 4.995112 -4.994052 -24.920092
std 0.042168 0.060116 0.203900 0.140874
These results were found for the job:
access_seed42/results/parameters.284.pickle
================================================================================
And you can access (pun intended) the results - even as ACCES is running - using:
>>> import coexist
>>> data = coexist.AccessData.read("access_seed42")
>>> data
AccessData
--------------------------------------------------------------------------------
paths ╎ AccessPaths(...)
parameters ╎ value min max sigma
╎ CED 3.569249 -5.0 5.0 0.052355
╎ CoR -1.813354 -5.0 5.0 0.078080
╎ Epsilon 4.995112 -5.0 5.0 0.284105
╎ Mu -4.994052 -5.0 5.0 0.180196
population ╎ 10
num_epochs ╎ 32
target ╎ 0.1
seed ╎ 42
epochs ╎ DataFrame(CED_mean, CoR_mean, Epsilon_mean, Mu_mean, CED_std,
╎ CoR_std, Epsilon_std, Mu_std, overall_std)
epochs_scaled ╎ DataFrame(CED_mean, CoR_mean, Epsilon_mean, Mu_mean, CED_std,
╎ CoR_std, Epsilon_std, Mu_std, overall_std)
results ╎ DataFrame(CED, CoR, Epsilon, Mu, error)
results_scaled ╎ DataFrame(CED, CoR, Epsilon, Mu, error)In this case a global optimum was found within 320 evaluations - this is of course problem-dependent, but you'll see that the optimum is often found much earlier if you check intermediate results (which you probably will when calibrating / optimising long-running simulations).
A tutorial with more detailed explanations is available here, including generating plots, checking intermediate results and running simulations on a SLURM-managed distributed cluster.
This library aims to be the state-of-the-art for simulation calibration, developed in the open using modern, collaborative coding approaches - no dragons shall be dwelling in the codebase. You are more than welcome to contribute to this library in the form of code improvements, documentation or helpful examples; please submit them either as:
- GitHub issues.
- Pull requests.
- Email me at a.l.nicusan@gmail.com.
We are more than happy to discuss the library architecture and calibration / optimisation approach with any potential contributors and users.
The authors gratefully acknowledge funding from the following UK funding bodies and industrial partners:
M²E³D: Multiphase Materials Exploration via Evolutionary Equation Discovery
Royce Materials 4.0 Feasibility and Pilot Scheme Grant, £57,477
CoExSiST: Coupled Experimental-Simulational Technique
EPSRC MAPP Grant, Feasibility Study, £60,246
ACCES: Autonomous Calibration and Characterisation via Evolutionary Simulation
EPSRC IAA, Follow-Up Grant to CoExSiST, £52,762
Improving ACCES: Towards the Multi-Tool Multi-Parameter Optimisation of Complex Particulate Systems
EPSRC MAPP, Grant, £48,871 + £48,871 matched funding from GranuTools Belgium
Thank you.
If you use this library in your research, you are kindly asked to cite:
[Paper after publication]
Until the ACCES paper is published, you may cite this repository:
Nicusan, A., Werner, D., Sykes, J. A., Seville, J., & Windows-Yule, K. (2022). ACCES: Autonomous Characterisation and Calibration via Evolutionary Simulation (Version 0.2.0) [Computer software]
ACCES is built on top of the excellent CMA-ES evolutionary algorithm - specifically the pycma implementation. If you use ACCES in your research, please also cite:
Nikolaus Hansen, Youhei Akimoto, and Petr Baudis. CMA-ES/pycma on Github. Zenodo, DOI:10.5281/zenodo.2559634, February 2019.
This library - in its general, domain-agnostic form - is free and open-source, published under the GNU General Public License v3.0.
If you are a company and would like to integrate ACCESS into your work - e.g. ACCESS-enabled equipment or general simulation calibration - please send an email to a.l.nicusan@bham.ac.uk to discuss commercial development of specific tools for your application. Relicensing for a closed-source / commercial project can be considered on an individual basis.
Copyright (C) 2020-2023 the Coexist developers. Until now, this library was built directly or indirectly through the brain-time of:
- Andrei Leonard Nicusan (University of Birmingham)
- Dominik Werner (University of Birmingham)
- Jack Sykes (University of Birmingham)
- Dr. Kit Windows-Yule (University of Birmingham)
- Prof. Jonathan Seville (University of Birmingham)
- Albert Bauer (TU Berlin)
Thank you.