Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[proto] Added functional perspective_bounding_box/segmentation_mask ops #5888

Merged
merged 6 commits into from
May 9, 2022

Conversation

vfdev-5
Copy link
Collaborator

@vfdev-5 vfdev-5 commented Apr 26, 2022

Related to #5514 and #5782

Description:

  • Added functional perspective_bounding_box/segmentation_mask ops
  • Added tests

Results on synthetic images/bboxes/segm mask:

Code
import numpy as np

import torch
import torchvision
from torchvision.prototype import features
from torchvision.prototype.transforms.functional import perspective_bounding_box, perspective_image_tensor, perspective_segmentation_mask

size = (64, 76)
# xyxy format
in_boxes = [
    [10, 15, 25, 35],
    [50, 5, 70, 22],
    [45, 46, 56, 62],
]
labels = [1, 2, 3]

im1 = 255 * np.ones(size + (3, ), dtype=np.uint8)
mask = np.zeros(size, dtype=np.int64)
for in_box, label in zip(in_boxes, labels):
    im1[in_box[1]:in_box[3], in_box[0]:in_box[2], :] = (127, 127, 127)
    mask[in_box[1]:in_box[3], in_box[0]:in_box[2]] = label
    
in_im = torch.tensor(im1).permute(2, 0, 1).view(1, 3, *size)

in_boxes = features.BoundingBox(
    in_boxes, format=features.BoundingBoxFormat.XYXY, image_size=size
)
in_mask = features.SegmentationMask(torch.tensor(mask)).view(1, *size)
    

coeffs = [3.7801878452301025,
 2.09562087059021,
 -59.95088195800781,
 0.9144350290298462,
 2.9016025066375732,
 -37.96672821044922,
 0.0310051292181015,
 0.01576310582458973]

out_boxes = perspective_bounding_box(
    in_boxes, 
    in_boxes.format,
    coeffs
)

out_mask = perspective_segmentation_mask(
    in_mask, 
    coeffs
)

out_im = perspective_image_tensor(in_im, coeffs, interpolation=torchvision.transforms.InterpolationMode.NEAREST)

image

Copy link
Contributor

@datumbox datumbox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vfdev-5 Thanks a lot for the PR, just a couple of minor comments. Let me know your thoughts.

@pmeier any thoughts on your end?

@vfdev-5
Copy link
Collaborator Author

vfdev-5 commented May 2, 2022

test/test_prototype_transforms_functional.py::test_correctness_perspective_bounding_box can be flaky on windows/macosx:

Maybe, we could define a bit higher rtol and atol ?

@vfdev-5 vfdev-5 requested a review from datumbox May 2, 2022 18:58
Copy link
Contributor

@datumbox datumbox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @vfdev-5.

@pmeier Any thoughts on the testing side?

@vfdev-5 vfdev-5 merged commit f079f5a into pytorch:main May 9, 2022
@vfdev-5 vfdev-5 deleted the proto-bbox-perspective branch May 9, 2022 11:09
facebook-github-bot pushed a commit that referenced this pull request May 11, 2022
…ion_mask` ops (#5888)

Summary:
* Added functional `perspective_bounding_box`/`perspective_segmentation_mask` ops

* Added more comments and added a code to assert denom != 0

* Put larger r/a tolerence when matching bboxes

Reviewed By: YosuaMichael

Differential Revision: D36281602

fbshipit-source-id: 5976f009e2dad93ff9a1a84129e8bbc066e91cf6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants