Caution

Commercial use of RoScenes is strictly forbidden.

📰 Release Note

[2024-07-14] You can now download the dataset at ModelScope.

[2024-07-13] Devkit for RoScenes released.

[2024-07-01] Paper accepted to ECCV 2024! 🥳

[2024-05-28] Please stay tuned for the updates! We are doing final checks on data privacy.

🏙️ Features

🔖 Table of Contents

• 🔥 Quick Start
  • Download
  • Install via PyPI
  • Install Manually (for dev)
  • Start Using the Dataset
• 🔎 Explore the Dataset
• 👩‍💻 Examples
• 📈 Evaluation
• 🎯 To-do List

🔥 Quick Start

Download

Note

Please refer to ModelScope for downloading the dataset.

After download and extract, the dataset folder should be organized as follows:

. [DATA_ROOT] # Dataset root folder
├── 📂train # training set
│   ├── 📂s001_split_train_difficulty_mixed_ambience_day # scene 001's data
│   │   ├── 📂database # annotations, grouped by clip
│   │   │   ├── 📂0076fd69_clip_[0000000000000-0000000029529] # a clip's database, please use our devkit to read
│   │   │   └   ...
│   │   └── 📂images # images, grouped by clips
│   │       ├── 📂0076fd69
│   │       └   ...
│   ├── 📂s002_split_train_difficulty_mixed_ambience_day
│   ├── 📂s003_split_train_difficulty_mixed_ambience_day
│   ├── 📂s004_split_train_difficulty_mixed_ambience_day
│   └── 📂night_split_train_difficulty_mixed_ambience_night
│
│
├── 📂validation # validation set
│   ├── 📂s001_split_validation_difficulty_mixed_ambience_day # scene 001's data
│   ├── 📂s002_split_validation_difficulty_mixed_ambience_day
│   ├── 📂s003_split_validation_difficulty_mixed_ambience_day
│   ├── 📂s004_split_validation_difficulty_mixed_ambience_day
│   └── 📂night_split_validation_difficulty_mixed_ambience_night
│
│
└── 📂test # test set
    ├── 📂NO_GTs005_split_test_difficulty_mixed_ambience_day # scene 005's data
    ├── 📂NO_GTs006_split_test_difficulty_mixed_ambience_day
    ├── 📂NO_GTs007_split_test_difficulty_mixed_ambience_day
    ├── 📂NO_GTs008_split_test_difficulty_mixed_ambience_day
    ├── 📂NO_GTs009_split_test_difficulty_mixed_ambience_day
    ├── 📂NO_GTs010_split_test_difficulty_mixed_ambience_day
    ├── 📂NO_GTs011_split_test_difficulty_mixed_ambience_day
    ├── 📂NO_GTs012_split_test_difficulty_mixed_ambience_day
    ├── 📂NO_GTs013_split_test_difficulty_mixed_ambience_day
    └── 📂NO_GTs014_split_test_difficulty_mixed_ambience_day

Install via PyPI

Use PyPI to directly install RoScenes devkit:

pip install roscenes

Install Manually (for dev)

Also, you can clone this repository and install roscenes manually for developing.

git clone https://github.com/roscenes/RoScenes.git

cd RoScenes

pip install -e .

Start Using the Dataset

import roscenes as ro

# load the training set
dataset = ro.load('[DATA_ROOT]/train/*')
# number of total frames
print(len(dataset))

Then, we can iterate over the dataset, in two ways:

You can use indexing:

# use integer indexing
index = 10
# a Frame instance
print(type(dataset[index]))

for i in range(len(dataset)):
    # print num of objects for every frame
    print(len(dataset[index].boxes3D))

OR, you can directly iterate it:

# a frame instance
for frame in dataset:
    print(len(frame.boxes3D))

Important

Please refer to frame.py, camera.py for the detailed comments on box format, instrinsic and extrinsic definition, etc.

🔎 Explore the Dataset

python -m roscenes.visualizer [DATA_ROOT]/train/s001_split_train_difficulty_mixed_ambience_day 0 vis_result

👩‍💻 Examples

A example in companion with mmdet3d is given in examples.

Please go to that folder README.md for details.

📈 Evaluation

Note that we use nuScenes detection score (NDS) to evaluate performance. Additionally, we provide an optimized implementation of NDS which could significantly speed-up calculation. It should produce identical result with the official NDS under same inputs. Benchmark is shown below:

Input frames	Boxes/Frame	Original nuScenes devkit	Ours
360	109	1697s	118s

Unfortunately, since the code is built from scratch, our evaluation suite can not be an in-place replacement to the original one ---- You need to modify the code. To take evaluation, please refer to the example given in examples/mmdet3d/mmdet3d_plugin/datasets/roscenes_dataset.py and the README file in roscenes/evaluation/README.md.

🎯 To-do List

• Devkit release
• Dataset release
• Example dataset loader based on MMDetection3D
• 3D detection task and evaluation suite
• 3D tracking task and evaluation suite