develop/how_to/process_datasets/

[giscus[bot]]

develop/how_to/process_datasets/

A set of easy-to-use utilities that will come in handy in any computer vision project.

https://supervision.roboflow.com/develop/how_to/process_datasets/

[bmorlo]

Dear all,

thank you for your great work. I happen to have a question.

In my code I am trying to merge two sv.DetectionDataset objects. Unfortunately, I keep getting the following error:

ValueError                                Traceback (most recent call last)

<ipython-input-83-405bb67341d2> in <cell line: 124>()
    122 
    123 # Create the training and the testing detection datasets.
--> 124 detection_dataset_train, IMAGE_READ_INDEX = merged_detection_dataset_from_directory(TRAIN_JSON_DIR_PATH, IMAGE_READ_INDEX)
    125 detection_dataset_test, IMAGE_READ_INDEX = merged_detection_dataset_from_directory(TEST_JSON_DIR_PATH, IMAGE_READ_INDEX)

2 frames

<ipython-input-83-405bb67341d2> in merged_detection_dataset_from_directory(_directory_path, _image_read_index)
    113 
    114     # Merge the datasets
--> 115     merged_detection_dataset = sv.DetectionDataset.merge(datasets_to_merge)
    116 

/usr/local/lib/python3.10/dist-packages/supervision/dataset/core.py in merge(cls, dataset_list)
    311             annotations.update(dataset.annotations)
    312         for dataset in dataset_list:
--> 313             class_index_mapping = build_class_index_mapping(
    314                 source_classes=dataset.classes, target_classes=classes
    315             )

/usr/local/lib/python3.10/dist-packages/supervision/dataset/utils.py in build_class_index_mapping(source_classes, target_classes)
     69     for i, class_name in enumerate(source_classes):
     70         if class_name not in target_classes:
---> 71             raise ValueError(
     72                 f"Class {class_name} not found in target classes. "
     73                 "source_classes must be a subset of target_classes."

ValueError: Class PalletA not found in target classes. source_classes must be a subset of target_classes.

Obviously I've tried my best to debug it, but I guess the relation between source and target classes is not clear for me. My feeling is that, because both sv.DetectionDataset objects have EXACTLY the same class list, i.e. ['PalletA'], the merge function runs into an issue.

Could this be a bug inside the merge function? Or is it due to me not encoding class_id and classes correctly when creating the sv.DetectionDataset objects?

Am I correct, that the "classes" list of a sv.DetectionDataset object should only comprise a unique list of the found class names?

Thank you so much for your insights!

Best,
Benedikt

[LinasKo]

Good news.

In the last update we made merging treat classes as case-sensitive. This may have inadvertently fixed the bug you're witnessing 😄

https://github.com/roboflow/supervision/pull/1643/files

Which supervision do you have when the error is thrown? You can find out with pip freeze | grep supervision

[bmorlo]

Yeah, I really believe I was simply using an outdated version (0.22). Now when I install and work with the newest version, everything runs smoothly.

I hope I understand you correctly. I had only one class called "PalletA". No e.g. "palleta" class or something. Hence, I'm not so sure about this being an issue of case sensitivity.

[LinasKo]

Oh, the lower-casing was applied before the datasets were merged.

Step 1: lower-cased classes from all datasets were merged into a list. ['palleta', 'palletb', 'palletc'].
Step 2: it would immediately try to look for 'PalletA' in this list, and throw the error you're seeing.

This is exactly why the bug would happen. Glad to have it sorted at last.

[bmorlo]

Ah, now that makes sense! Thank you so much for your amazing support! All the best.

[LinasKo]

Glad to help!