v0.4.0

v0.4.0 Release Notes

Important changes from this PR include the move away from observation and action flattening, the move to explicitly and fully support Dict, Tuple, Box, Discrete, and Text spaces, and the move to explicit dataset versioning. Additionally, we have added support for using a subset of an environment's action/observation spaces when creating a dataset.

Finally, we have released new versions of each dataset to make them compliant with our new dataset format. This includes all the changes listed in the "Dataset Updates" section of the release notes.

We have two new tutorials:

• Creating datasets with subset of observation space of environment
• Behavior cloning with rl_zoo3 and PyTorch DataLoader
  

Unflattened Dict and Tuple space support

The following exerpt from our documentation shows how unflattened gymnasium.spaces.Dict and gymnasium.spaces.Tuple are now supported.

In the case where, the observation space is a relatively complex Dict space with the following definition:

spaces.Dict(
    {
        "component_1": spaces.Box(low=-1, high=1, dtype=np.float32),
        "component_2": spaces.Dict(
            {
                "subcomponent_1": spaces.Box(low=2, high=3, dtype=np.float32),
                "subcomponent_2": spaces.Box(low=4, high=5, dtype=np.float32),
            }
        ),
    }
)

and the action space is Box space, the resulting HDF5 file will end up looking as follows:

📄 main_data.hdf5
├ 📁 episode_0
│  ├ 📁 observations
│  │  ├ 💾 component_1
│  │  └ 📁 component_2
│  │     ├ 💾 subcomponent_1
│  │     └ 💾 subcomponent_2
│  ├ 💾 actions
│  ├ 💾 terminations
│  ├ 💾 truncations
│  ├ 💾 rewards
│  ├ 📁 infos
│  │  ├ 💾 infos_datasets
│  │  └ 📁 infos_subgroup
│  │     └ 💾 more_datasets
│  └ 📁 additional_groups
│     └ 💾 additional_datasets
├ 📁 episode_1
├ 📁 episode_2
│
└ 📁 last_episode_id

Similarly, consider the case where we have a Box space as an observation space and a relatively complex Tuple space as an action space with the following definition:

spaces.Tuple(
    (
        spaces.Box(low=2, high=3, dtype=np.float32),
        spaces.Tuple(
            (
                spaces.Box(low=2, high=3, dtype=np.float32),
                spaces.Box(low=4, high=5, dtype=np.float32),
            )
        ),
    )
)

In this case, the resulting Minari dataset HDF5 file will end up looking as follows:

📄 main_data.hdf5
├ 📁 episode_0
│  ├ 💾 observations
│  ├ 📁 actions
│  │  ├ 💾 _index_0
│  │  └ 📁 _index_1
│  │     ├ 💾 _index_0
│  │     └ 💾 _index_0
│  ├ 💾 terminations
│  ├ 💾 truncations
│  ├ 💾 rewards
│  ├ 📁 infos
│  │  ├ 💾 infos_datasets
│  │  └ 📁 infos_subgroup
│  │     └ 💾 more_datasets
│  └ 📁 additional_groups
│     └ 💾 additional_datasets
├ 📁 episode_1
├ 📁 episode_2
│
└ 📁 last_episode_id

EpisodeData: Data format when sampling episodes

Episodes are now sampled as EpisodeData Instances that comply with the following format:

Field	Type	Description
id	np.int64	ID of the episode.
seed	np.int64	Seed used to reset the episode.
total_timesteps	np.int64	Number of timesteps in the episode.
observations	np.ndarray, list, tuple, dict	Observations for each timestep including initial observation.
actions	np.ndarray, list, tuple, dict	Actions for each timestep.
rewards	np.ndarray	Rewards for each timestep.
terminations	np.ndarray	Terminations for each timestep.
truncations	np.ndarray	Truncations for each timestep.

Breaking Changes

• Changed dataset format to support storing unflattened spaces and also subset spaces by @rodrigodelazcano, @balisujohn, and @younik in #77
• Refactored minari_dataset.py to not use h5py directly. This meant moving public function clear_episode_buffer to minari_storage.py by @balisujohn in #101
• Removed Python 3.7 compatibility due to EOL by @rodrigodelazcano in #107

New Features

• Added Python 3.11 support by @rodrigodelazcano in #73
• Reorganized tests and added more thorough testing of MinariStorage by @balisujohn in #75
• Added option to copy data(instead of reference based copy) when combining datasets by @Howuhh in #82
• Fail with a descriptive error if dataset env base library not installed by @shreyansjainn in #86
• Made EpisodeData into a dataclass by @younik in #88
• Added force_download flag for locally existing datasets by @shreyansjainn in #90
• Added support for text spaces by @younik in #99
• Added minari.get_normalized_scores that follows the evaluation process in the D4RL datasets. by @rodrigodelazcano in #110
• Added code to support Minari dataset version specifiers by @rodrigodelazcano in #107

Bug Fixes

• Fixed path printing in the CLI(previously incorrect) by @Howuhh in #83
• Copy infos from the previous episode if truncated or terminated without reset by @Howuhh in #96
• Ignore hidden files when listing local datasets by @enerrio #104
• h5py group creation bug fix by @rodrigodelazcano in #111

Documentation Changes

• Adding a table describing supported action and oibservation spaces by @tohsin in #84
• Adds test instructions to contributing.MD by @shreyansjainn in #86
• Adds installation instructions to basic usage section of doc and also doc build instructions to documentation by @enerrio in #105
• Added tutorial for space subsetting by @Bamboofungus in #108
• Added description of EpisodeData to documentation by @enerrio in #109
• Improved background about PID control in pointmaze dataset creation tutorial by @tohsin in #95
• Docs now show serialized dataset spaces by @rodrigodelazcano in #116
• Adds behavior cloning tutorial with Minari and PyTorchDataLoader by @younik in #102

Misc Changes

• Added a citation.cff file by @rodrigodelazcano
• Fixed some typos and type annotations, slightly ruggedized datacollector by @RedTachyon in #52
• Froze pyright in pre-commit to version 1.1.305 by @balisujohn
• Updated dataset used in readme example by @shreyansjainn in #80
• Corrected readme example by @younik in #87
• Added check to make sure observations and actions were in the right space in MinariStorage instances by @balisujohn in #92
• Ruggedized list_remote_datasets by @balisujohn in #93
• Added pre-commit and code-style black badges by @elliottower in #112

Dataset Updates

v1 versions of each provided dataset have been released and new dataset format has the following changes.

• Observation and action flattening have been removed for pointmaze datasets, as arbitrary nesting of Dict and Tuple spaces is now supported with the new dataset format.
• v1 and subsequent datasets now have action_space and observation_space fields which store a serialized representation of the observation and action spaces used for observations and actions in the dataset. It's important to note that this can be different from the spaces of the gymnasium environment mentioned in the dataset spec.
• v1 and subsequent datasets have the minari_version field which specify with which versions of Minari they are compatible.
• v1 pointmaze datasets copies the last info to the next episode as fixed in #96