Skip to content

An evaluation framework for machine learning models simulating high-throughput materials discovery.

License

Notifications You must be signed in to change notification settings

iProzd/matbench-discovery

 
 

Repository files navigation

Logo
Matbench Discovery

arXiv Tests GitHub Pages Requires Python 3.11+ PyPI

Matbench Discovery is an interactive leaderboard and associated PyPI package which rank ML models on a task designed to simulate high-throughput discovery of new stable inorganic crystals as well as predicting some of their properties.

We've tested models covering multiple methodologies including graph neural network (GNN) interatomic potentials, GNN one-shot predictors, iterative Bayesian optimizers and random forests with shallow-learning structure fingerprints.

Our results show that ML models have become robust enough to deploy them as triaging steps to more effectively allocate compute in high-throughput DFT relaxations. This work provides valuable insights for anyone looking to build large-scale materials databases.

If you'd like to refer to Matbench Discovery in a publication, please cite the preprint:

Janosh Riebesell, Rhys E. A. Goodall, Philipp Benner, Yuan Chiang, Bowen Deng, Alpha A. Lee, Anubhav Jain, and Kristin A. Persson. "Matbench Discovery -- A Framework to Evaluate Machine Learning Crystal Stability Predictions." arXiv, August 28, 2023. https://doi.org/10.48550/arXiv.2308.14920.

We welcome new models additions to the leaderboard through GitHub PRs. See the contributing guide for details and ask support questions via GitHub discussion.

For detailed results and analysis, check out the preprint.

Disclaimer: We evaluate how accurately ML models predict solid-state thermodynamic stability. Although this is an important aspect of high-throughput materials discovery, the ranking cannot give a complete picture of a model's general applicability to materials. A high ranking does not constitute endorsement by the Materials Project.

About

An evaluation framework for machine learning models simulating high-throughput materials discovery.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 79.6%
  • Svelte 12.4%
  • TypeScript 5.5%
  • CSS 1.0%
  • JavaScript 0.7%
  • Shell 0.5%
  • HTML 0.3%