This issue may be helpful for creating the mapping from object ID to label ID: #263

TLDR:

instance_id_to_label_id = {int(obj.id.split("_")[-1]): obj.category.index() for obj in sim.semantic_scene.objects}

I have gone through that issue and I can see that the information for the labels and the associated centeroid coordinates and dimensions can be extracted.
Can you shed some light on the dimensions that are being extracted?
This to be exact.

dims:{obj.aabb.sizes}

Are these cubical semantic masks?

I am not sure what a "cubical semantic mask" is, but here is what those are:

Each object has an associated 3D bounding box (an oriented bounding box in the case of objects). obj.aabb is then the axis aligned bounding box containing the OBB.
obj.aabb.sizes are the lengths of the sides of the bounding box.
obj.aabb.center is the center of the bounding box.

These are in Habitat's coordinate frame, which is different then the coordinate frame of the mp3d dataset. The mp3d dataset is +Z up while habitat is +Y up.

Thanks! I ended up working with the raw MP3d ply files and using the mapping that you suggested.

Couple of questions:

1. During the conversion of the instance id's to label id's it seems like there are a bunch of -1 label id's. Do these signify the un-classified but detected objects?
2. Is the world coordinate frame of matter port 3d (raw data from .ply files) and the habitat sim aligned? if not, do we have the information of the transform between them? I understand that mp3d is -z gravity and habitat sim is -y gravity. is there any other difference?

My understanding is that -1 is misc/unlabeled and these are generally not "objects". Note that we did not collect the Matterport3D dataset, so that may be wrong. You can read more about it here: https://niessner.github.io/Matterport/

Yes, that is the only difference between the two coordinate frames.

Thanks! I will try to make a pull request for extracting the scene semantic map(groundtruth) if I have the bandwidth. I have it working at my end.