Getting Started | Docs | Baseline Results | Changelist
Dopamine is a research framework for fast prototyping of reinforcement learning algorithms. It aims to fill the need for a small, easily grokked codebase in which users can freely experiment with wild ideas (speculative research).
Our design principles are:
- Easy experimentation: Make it easy for new users to run benchmark experiments.
- Flexible development: Make it easy for new users to try out research ideas.
- Compact and reliable: Provide implementations for a few, battle-tested algorithms.
- Reproducible: Facilitate reproducibility in results. In particular, our setup follows the recommendations given by Machado et al. (2018).
Dopamine supports the following agents, implemented with jax:
- DQN (Mnih et al., 2015)
- C51 (Bellemare et al., 2017)
- Rainbow (Hessel et al., 2018)
- IQN (Dabney et al., 2018)
- SAC (Haarnoja et al., 2018)
- PPO (Schulman et al., 2017)
For more information on the available agents, see the docs.
Many of these agents also have a tensorflow (legacy) implementation, though newly added agents are likely to be jax-only.
This is not an official Google product.
We provide docker containers for using Dopamine. Instructions can be found here.
Alternatively, Dopamine can be installed from source (preferred) or installed with pip. For either of these methods, continue reading at prerequisites.
Dopamine supports Atari environments and Mujoco environments. Install the environments you intend to use before you install Dopamine:
Atari
- These should now come packaged with ale_py.
- You may need to manually run some steps to properly install
baselines, see instructions.
Mujoco
- Install Mujoco and get a license here.
- Run
pip install mujoco-py(we recommend using a virtual environment).
The most common way to use Dopamine is to install it from source and modify the source code directly:
git clone https://github.com/google/dopamine
After cloning, install dependencies:
pip install -r dopamine/requirements.txt
Dopamine supports tensorflow (legacy) and jax (actively maintained) agents. View the Tensorflow documentation for more information on installing tensorflow.
Note: We recommend using a virtual environment when working with Dopamine.
Note: We strongly recommend installing from source for most users.
Installing with pip is simple, but Dopamine is designed to be modified directly. We recommend installing from source for writing your own experiments.
pip install dopamine-rl
You can test whether the installation was successful by running the following from the dopamine root directory.
export PYTHONPATH=$PYTHONPATH:$PWD
python -m tests.dopamine.atari_init_test
View the docs for more information on training agents.
We supply baselines for each Dopamine agent.
We also provide a set of Colaboratory notebooks which demonstrate how to use Dopamine.
Mnih et al., Human-level Control through Deep Reinforcement Learning. Nature, 2015.
Schulman et al., Proximal Policy Optimization Algorithms.
If you use Dopamine in your work, we ask that you cite our white paper. Here is an example BibTeX entry:
@article{castro18dopamine,
author = {Pablo Samuel Castro and
Subhodeep Moitra and
Carles Gelada and
Saurabh Kumar and
Marc G. Bellemare},
title = {Dopamine: {A} {R}esearch {F}ramework for {D}eep {R}einforcement {L}earning},
year = {2018},
url = {http://arxiv.org/abs/1812.06110},
archivePrefix = {arXiv}
}