Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Segmentation Masks #269

Open
kne42 opened this issue May 17, 2019 · 6 comments
Open

Segmentation Masks #269

kne42 opened this issue May 17, 2019 · 6 comments
Labels
feature New feature or request
Milestone
0.4.10

Comments

@kne42
Copy link
Member

kne42 commented May 17, 2019

🚀 Feature

Add support for displaying segmentation instance masks.

Motivation

Many projects, including starfish and the 2018 kaggle data science bowl use segmentation masks to represent labeled regions in an image.

As of writing this, napari represents labeled regions in a "label image," a singular integer array where each positive integer marks a region. While it is trivial and non-lossy to convert a label image to a list of segmentation masks, it is lossy to convert a list of segmentation masks to a label image due to the potential of overlaps.

Pitch

Instead of representing the data contained in Labels layers with an MxN label image of P integers, represent it with a PxMxN boolean array of segmentation masks.

Alternatives

To save on memory, the segmentation masks could be represented with xarrays so as to only allocate memory for the bounding box of each mask, or use a sparse array.

Additional context

This feature request was spurred from a conversation with @sofroniewn and @ambrosejcarr
@kne42 kne42 added the feature New feature or request label May 17, 2019
@jni
Copy link
Member

jni commented May 18, 2019

We've used this model in skimage before, see skimage.future.manual_segmentation. imho this should go in a new layer type though, not Labels.

@sofroniewn
Copy link
Contributor

@jni curious how you think this might fit in with the idea to make Labels a subclass of Image. Would this then be a different subclass of Image or could there be another more intermediate subclass that both Labels and this class was then a subclass of? I feel like there would be a lot of code shared between this layer and a Labels layer.

The thing that is most challenging for me to imagine with this layer is how do we render overlapping regions? Do we blend the respective colors of the labels, do we just show the "largest" color, do we show something else? Blending probably makes the most sense, but I'm slightly nervous about it

@sofroniewn sofroniewn added this to the 0.2.0 milestone May 21, 2019
@jni
Copy link
Member

jni commented Jun 12, 2019

@sofroniewn I don't have an answer to your questions. Glad this is in 0.2.0 because I think there's a reason overlapping segmentations are generally poorly if at all supported in the libraries I've worked with: it's messy so no one wants to do it. I also think there's various workarounds, such as using multiple Labels layers.

@sofroniewn sofroniewn modified the milestones: 0.2.0, 0.3.0 Sep 14, 2019
@sofroniewn sofroniewn mentioned this issue Sep 17, 2019
Closed
@sofroniewn
Copy link
Contributor

Following discussion in zulip around blending with @bethac07 and @0x00b1

image

Beth Cimini: It shows outlines of all the objects, each object is its own color, and it shows a 3rd color (a blend would be ideal, but in CellProfiler it's pretty much just random IIRC) in overlapping regions

Allen Goodman: It depends. 😆 I'm a strong advocate for panoptic segmentation representation so I avoid making background (or non-foreground) classes transparent. But if you don't care about that, making foreground objects 0.5 works. It's what most software for this type of thing does.

@sofroniewn sofroniewn mentioned this issue Feb 18, 2021
Open
@0x00b1
Copy link

0x00b1 commented May 19, 2021

Hi, I started working on this today.

@sofroniewn
Copy link
Contributor

Hi, I started working on this today.

Awesome, not sure if you've been following this thread ome/ngff#19, but would be great to make sure that whatever support we have in napari naturally pairs well with whatever the ngff folks decide around overlapping labels (and vice versa) cc @joshmoore

Feel free to ping in zulip if you have any questions/ run into any stumbling blocks

@sofroniewn sofroniewn modified the milestones: 0.3, 0.4.9 May 19, 2021
@0x00b1 0x00b1 mentioned this issue May 19, 2021
Open
@sofroniewn sofroniewn modified the milestones: 0.4.9, 0.4.10 Jun 2, 2021
@sofroniewn sofroniewn mentioned this issue Jun 15, 2021
Closed
9 tasks
@0x00b1 0x00b1 removed their assignment Dec 13, 2021
@gselzer gselzer mentioned this issue Jan 20, 2022
Closed
Czaki pushed a commit that referenced this issue Nov 13, 2023
@psobolewskiPhD @Czaki
delete :orphan: (#269)
54c9f48 
# References and relevant issues
First noted here:
#6384 (comment)

# Description
There is an orphan `:orphan:` tag at the top of NAP-8. Pretty sure it's
not doing anything--the other naps don't have it. This deletes it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature New feature or request
Projects
None yet
Milestone
0.4.10
Development

No branches or pull requests
4 participants
@0x00b1 @jni @sofroniewn @kne42