Awesome Open-Ended AI

A curated list of AI open-ended learning resources. As the list grows longer, I will split it into more sections.

Table of Contents

• Papers
• Open-ended AI Safety
• Surveys and Perspectives

Contributing

When submitting a pull request, please put the new paper at the correct chronological position as the following format:

* **Paper Title** <br>
*Author(s)* <br>
Conference, Year. [[Paper]](link) [[Code]](link) [[Website]](link)

Papers

• Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions
  Rui Wang, Joel Lehman, Jeff Clune, Kenneth O. Stanley
  GECCO, 2019. [Paper] [Code] [Website]

• Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions
  Rui Wang, Joel Lehman, Aditya Rawal, Jiale Zhi, Yulun Li, Jeff Clune, Kenneth O. Stanley
  ICML, 2020. [Paper] [Code] [Website]

• Co-generation of game levels and game-playing agents
  Aaron Dharna, Julian Togelius, L.B.Soros
  AIIDE 2020. [Paper] [Code]

• Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design
  Michael Dennis, Natasha Jaques, Eugene Vinitsky, Alexandre Bayen, Stuart Russell, Andrew Critch, Sergey Levine
  NeurIPS, 2020. [Paper] [Code] [Website]

• Prioritized Level Replay
  Minqi Jiang, Edward Grefenstette, Tim Rocktäschel
  ICML, 2021. [Paper] [Code]

• Replay-Guided Adversarial Environment Design
  Minqi Jiang*, Michael Dennis*, Jack Parker-Holder, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel
  NeurIPS, 2021. [Paper] [Code]

• Environment Generation for Zero-Shot Compositional Reinforcement Learning
  Izzeddin Gur, Natasha Jaques, Yingjie Miao, Jongwook Choi, Manoj Tiwari, Honglak Lee, Aleksandra Faust
  NeurIPS, 2021. [Paper]

• MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research
  Mikayel Samvelyan, Robert Kirk, Vitaly Kurin, Jack Parker-Holder, Minqi Jiang, Eric Hambro, Fabio Petroni, Heinrich Küttler, Edward Grefenstette, Tim Rocktäschel
  NeurIPS, 2021. [Paper] [Code] [Website]

• Open-Ended Learning Leads to Generally Capable Agents
  Open Ended Learning Team, Adam Stooke, Anuj Mahajan, Catarina Barros, Charlie Deck, Jakob Bauer, Jakub Sygnowski, Maja Trebacz, Max Jaderberg, Michael Mathieu, Nat McAleese, Nathalie Bradley-Schmieg, Nathaniel Wong, Nicolas Porcel, Roberta Raileanu, Steph Hughes-Fitt, Valentin Dalibard, Wojciech Marian Czarnecki
  arXiv, 2021. [Paper] [Website]

• SPOTTER: Extending Symbolic Planning Operators through Targeted Reinforcement Learning
  Vasanth Sarathy, Daniel Kasenberg, Shivam Goel, Jivko Sinapov, Matthias Scheutz
  arXiv, 2021. [Paper] [Code]

• EvoCraft: A New Challenge for Open-Endedness
  Djordje Grbic, Rasmus Berg Palm, Elias Najarro, Claire Glanois, Sebastian Risi
  EvoStar, 2021. [Paper] [Website]

• Video Games as a Testbed for Open-Ended Phenomena
  Sam Earle; Julian Togelius; L. B. Soros
  IEEE Conference on Games, 2021. [Paper]

• Evolving Curricula with Regret-Based Environment Design
  Jack Parker-Holder*, Minqi Jiang*, Michael Dennis, Mikayel Samvelyan, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel
  ICML, 2022. [Paper] [Code] [Demo]

• Evolution through Large Models
  Joel Lehman, Jonathan Gordon, Shawn Jain, Kamal Ndousse, Cathy Yeh, Kenneth Stanley
  arXiv, 2022. [Paper] [Code]

• RAPid-Learn: A Framework for Learning to Recover for Handling Novelties in Open-World Environments
  Shivam Goel, Yash Shukla, Vasanth Sarathy, Matthias Scheutz, Jivko Sinapov
  arXiv, 2022. [Paper] [Code]

• Transfer Dynamics in Emergent Evolutionary Curricula
  Aaron Dharna, Amy K. Hoover, Julian Togelius, Lisa Soros
  IEEE Transactions on Games, 2022. [Paper] [Code]

• Watts: Infrastructure for Open-Ended Learning
  Aaron Dharna, Charlie Summers, Rohin Dasari, Julian Togelius, Amy K. Hoover
  ALOE Workshop 2022 [Paper] [Code]

• MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
  Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, Anima Anandkumar
  NeurIPS, 2022. [Paper] [Code] [Website]

• Grounding Aleatoric Uncertainty in Unsupervised Environment Design
  Minqi Jiang, Michael Dennis, Jack Parker-Holder, Andrei Lupu, Heinrich Küttler, Edward Grefenstette, Tim Rocktäschel, Jakob Foerster
  NeurIPS 2022. [Paper]

• Language and Culture Internalisation for Human-Like Autotelic AI
  Cédric Colas, Tristan Karch, Clément Moulin-Frier, Pierre-Yves Oudeyer
  Nature Machine Intelligence, 2022. [Paper] [Website]

• MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learning
  Mikayel Samvelyan, Akbir Khan, Michael Dennis, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, Roberta Raileanu, Tim Rocktäschel
  ICLR, 2023. [Paper] [Website]

• Powderworld: A Platform for Understanding Generalization via Rich Task Distributions
  Kevin Frans, Philip Isola
  ICLR, 2023. [Paper] [Website] [Code]

• Human-Timescale Adaptation in an Open-Ended Task Space
  Adaptive Agent Team, Jakob Bauer, Kate Baumli, Satinder Baveja, Feryal Behbahani, Avishkar Bhoopchand, Nathalie Bradley-Schmieg, Michael Chang, Natalie Clay, Adrian Collister, Vibhavari Dasagi, Lucy Gonzalez, Karol Gregor, Edward Hughes, Sheleem Kashem, Maria Loks-Thompson, Hannah Openshaw, Jack Parker-Holder, Shreya Pathak, Nicolas Perez-Nieves, Nemanja Rakicevic, Tim Rocktäschel, Yannick Schroecker, Jakub Sygnowski, Karl Tuyls, Sarah York, Alexander Zacherl, Lei Zhang
  ICML, 2023. [Paper] [Website]

• Voyager: An Open-Ended Embodied Agent with Large Language Models
  Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, Anima Anandkumar
  arXiv, 2023. [Paper] [Code] [Website]

• OMNI: Open-endedness via Models of human Notions of Interestingness
  Jenny Zhang, Joel Lehman, Kenneth Stanley, Jeff Clune
  arXiv, 2023. [Paper] [Code]

• Augmenting Autotelic Agents with Large Language Models
  Cédric Colas, Laetitia Teodorescu, Pierre-Yves Oudeyer, Xingdi Yuan, Marc-Alexandre Côté
  arXiv, 2023. [Paper]

Open-Ended AI Safety

• Open Questions in Creating Safe Open-ended AI: Tensions Between Control and Creativity
  Adrien Ecoffet, Jeff Clune, Joel Lehman
  arXiv, 2020. [Paper]

Surveys and Perspectives on Open-Endedness

• Open-endedness: The last grand challenge you’ve never heard of
  Kenneth O. Stanley, Joel Lehman, Lisa Soros
  O'Reilly Radar, 2017. [Paper]

• Why Greatness Cannot Be Planned: The Myth of the Objective
  Kenneth Stanley, Joel Lehman
  arXiv, 2019. [Book]

• AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
  Jeff Clune
  arXiv, 2019. [Paper]

• Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework
  Evana Gizzi, Lakshmi Nair, Sonia Chernova, Jivko Sinapov
  arXiv, 2022. [Paper]

• Executive Function: A Contrastive Value Policy for Resampling and Relabeling Perceptions via Hindsight Summarization?
  Chris Lengerich, Ben Lengerich
  arXiv, 2022. [Paper]

• General Intelligence Requires Rethinking Exploration
  Minqi Jiang, Tim Rocktäschel, Edward Grefenstette
  Royal Society Open Science, 2023. [Paper]