BOP

[jeehart]

Hi,
I am trying to write the data of my scene in BOP format.
My issue is that the data is incomplete. I only have the following items :

• depth images
• rgb images
• scene_camera.json
• scene_gt.json

I am then missing scene_gt_info.json, mask and mask_visib.

My code is the following :

import blenderproc as bproc
import argparse
import os

parser = argparse.ArgumentParser()
parser.add_argument('camera', nargs='?', default="examples/resources/camera_positions", help="Path to the camera file")
parser.add_argument('scene', nargs='?', default="my_tests/tless/BOP_annotation/scene.blend", help="Path to the scene.blend file")
parser.add_argument('output_dir', nargs='?', default="my_tests/tless/BOP_annotation/output", help="Path to where the final files will be saved ")
args = parser.parse_args()

bproc.init()

# load the objects into the scene
objs = bproc.loader.load_blend(args.scene)

# Set some category ids for loaded objects
for j, obj in enumerate(objs):
    obj.set_cp("category_id", j+1)

# define a light and set its location and energy level
light = bproc.types.Light()
light.set_type("POINT")
light.set_location([5, -5, 5])
light.set_energy(1000)

# define the camera intrinsics
bproc.camera.set_resolution(512, 512)

# read the camera positions file and convert into homogeneous camera-world transformation
with open(args.camera, "r") as f:
    for line in f.readlines():
        line = [float(x) for x in line.split()]
        position, euler_rotation = line[:3], line[3:6]
        matrix_world = bproc.math.build_transformation_mat(position, euler_rotation)
        bproc.camera.add_camera_pose(matrix_world)

############### scene rendering and saving ###############
# activate normal and depth rendering
bproc.renderer.enable_normals_output()
bproc.renderer.enable_depth_output(activate_antialiasing=False)

# render the whole pipeline
data = bproc.renderer.render()

# Render segmentation masks (per class and per instance)
data.update(bproc.renderer.render_segmap(map_by=["class", "instance", "name"]))
# Render segmentation data and produce instance attribute maps
#seg_data = bproc.renderer.render_segmap(map_by=["instance", "class", "name"])

# write the data to a .hdf5 container
bproc.writer.write_hdf5(args.output_dir, data)

# write the data in bop format
bproc.writer.write_bop(os.path.join(args.output_dir, 'bop_data'),
                       target_objects = objs,
                       depths = data["depth"], 
                       colors = data["colors"], 
                       color_file_format = "JPEG",
                       save_world2cam = True,
                       )

Note that scene.blend scene from where I extract the objects I use is simply a scene with the objects with no annotation or parameters.

Also, the image from the camera is saved, but I also have an image of a view of the scene. Why is that?

What am I doing wrong ?

Thank you.

[MartinSmeyer]

Hey @jeehart,

the missing information can be generated using the bop_toolkit. Have a look here:
https://github.com/DLR-RM/BlenderProc/tree/main/examples/datasets/bop_challenge#complete-the-blenderproc4bop-datasets

Note that scene.blend scene from where I extract the objects I use is simply a scene with the objects with no annotation or parameters.

Also, the image from the camera is saved, but I also have an image of a view of the scene. Why is that?

My guess is that your scene.blend file already contains a camera that you would need to delete if it should not be rendered.

[jeehart]

I went through the code in the bop_toolkit, but my problem is that I'm not using any bop dataset like tless or another, I am using my own object models, but still would want the dataset in the complete BOP format. I am wondering if I can still use the bop_toolkit. or maybe there is a modified version that I could use?

If not, I was thinking of writing my own scripts using the segmentation data and images obtained after rendering with BlenderProc, but I am wondering if there was a way to have the segmentation masks for each object individually so I can calculate values like the visible fraction of each object, etc.

Thank you for your response and your time.

[MartinSmeyer]

Yes you can still use the bop_toolkit. The only thing you need to do is adding an entry for your new BOP dataset in dataset_params.py. Basically add the obj_ids here:
https://github.com/thodan/bop_toolkit/blob/af380d7a028b5c44903913e39d652c83a4bc2bdd/bop_toolkit_lib/dataset_params.py#L75-L89
and the dataset im_size and scene_ids potentially for different splits here:
https://github.com/thodan/bop_toolkit/blob/af380d7a028b5c44903913e39d652c83a4bc2bdd/bop_toolkit_lib/dataset_params.py#L363

That's it, then you should be able to use all the functionality in the bop_toolkit