-
-
Notifications
You must be signed in to change notification settings - Fork 7.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New data → color pipeline #28658
New data → color pipeline #28658
Conversation
Naming: Not sure Note 2: Yes, please move to a new file. Note 3: All agreed. Note that I've already simplified https://matplotlib.org/devdocs/gallery/images_contours_and_fields/multi_image.html through #28545. This still needs adaption to the Colorizer API, but only after we've untangled the Artist backreferences so that multiple Artists can share the same Colorizer/Colorbar |
My thoughts regarding the name have evolved as I have been working on the issue, which has led me to the following points:
That said, I don't think the name is of much significance, so just tell me again and I'll change it back to ¹ In my mind |
@timhoffm Also: |
lib/matplotlib/artist.py
Outdated
if np.ndim(data) == 0 and (isinstance(self, ScalarMappable) or | ||
isinstance(self, ColorizingArtist)): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if np.ndim(data) == 0 and (isinstance(self, ScalarMappable) or | |
isinstance(self, ColorizingArtist)): | |
if np.ndim(data) == 0 and isinstance(self, (ScalarMappable, ColorizingArtist)): |
lib/matplotlib/artist.py
Outdated
interval ``[0, 1]``. | ||
If a `str`, a `colors.Normalize` subclass is dynamically generated based | ||
on the scale with the corresponding name. | ||
If `cm.Colorizer`, the norm an colormap on the `cm.Colorizer` will be used |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would not overload the signature. A typical strategy would be to have the canoncial signature and create a factory classmethod for the other signature:
def __init__(colorizer):
self.colorizer = colorizer
@classmethod
def from_norm_and_cmap(cls, norm, cmap):
return cls(Colorizer(norm, cmap)
AFAICT, the situation here is a bit different, because users do not instantiate ColorizingArtist
directly, and hence, they won't use the factory method. We could do one of the two options:
- only implement
def __init__(colorizer)
. If subclasses need to initialize from norm + cmap, they'd simply doColorizingArtist.__init__(Colorizer(norm, cmap))
- create an alternative initializer
def _init_from_norm_and_cmap(norm, cmap): self.colorizer = Colorizer(norm, cmap)
Given that we'll only have a handfull of subclasses and the solution is quite simple, I'm leaning towards the first option.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The first alternative makes sense to me.
However, we still need the logic where the user can pass a Colorizer
object as part of the input to the plotting function.
I implemented this with the following function:
def _get_colorizer(cmap, norm):
"""
Passes or creates a Colorizer object.
Allows users to pass a Colorizer as the norm keyword
where a artist.ColorizingArtist is used as the artist.
If a Colorizer object is not passed, a Colorizer is created.
"""
if isinstance(norm, Colorizer):
if cmap:
raise ValueError("Providing a `cm.Colorizer` as the norm while "
"at the same time providing a `cmap` is not supported.")
return norm
return Colorizer(cmap, norm)
Such that the initiation of a ColorizingArtist
in e.g. Collection
is:
artist.ColorizingArtist.__init__(self, colorizer._get_colorizer(cmap, norm))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really want to pass a Colorizer
via the norm
parameter in public API? This feels very hacky. I'm inclined to add a colorizer
parameter instead, even if that means we have mutually exclusive parameters. (Note that even with the hack, we'd have to live with partially mutually exclusive parameters, because you cannot pass norm=mycolorizer, cmap='jet'
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have no strong opinions, and am inclined to agree with you @timhoffm.
Since this is a change to the call signature of the plotting functions, we should bring it up at a weekly meeting. @story645 could you add it on the agenda for the next meeting?
(My aversion to new keyword arguments is a result of a discussion at a weekly meeting a long time ago about multivariate color mapping, and how we do not want that to increase the number of keyword arguments, but this is a different case entirely.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry I didn't see this til now 😦 but also anyone can add stuff to the agenda (sorry me and Kyle didn't make that clear)! And also this got resolved as yes add new kwarg, I think.
ColorizerMappable? or ColorizerMappableInterface/colorizer.MappableInterface since this is aimed at providing an interface akin to the old Mappable Interface (I think?) |
lib/matplotlib/collections.py
Outdated
@@ -32,7 +32,7 @@ | |||
"linewidth": ["linewidths", "lw"], | |||
"offset_transform": ["transOffset"], | |||
}) | |||
class Collection(artist.Artist, cm.ScalarMappable): | |||
class Collection(artist.ColorizingArtist, colorizer.ColorizerShim): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As the colorizing artist gets built out/in later PRs, is the plan that the inherit from Shim goes away?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No. The shim provides the backward-compatible interface for the ScalarMappaple API. It‘s needed for the foreseeable future, but it’s reasonable to keep it separated from the ColorizingArtist API.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So where I'm confused is that I understand the compatibility layer as public API for downstream libraries, I'm confused about compatibility layer for private API where we can use the thing directly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean down the line, I understand for this and some foreseeable PRs the shim being here to show that the functionality is being maintained w/o having to reimplement everything.
try: | ||
scale_cls = scale._scale_mapping[norm] | ||
except KeyError: | ||
raise ValueError( | ||
"Invalid norm str name; the following values are " | ||
f"supported: {', '.join(scale._scale_mapping)}" | ||
) from None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a utility function for this, probably https://matplotlib.org/devdocs/api/_api_api.html#matplotlib._api.check_in_list
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code mirrors the existing code for ScalarMappable (cm.py lines 545→552), but I'll see if I can shift it to use the _api or if there is some reason why that has not been implemented for ScalarMappable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two points:
- This will change the error message from:
" Invalid norm str name; the following values are supported: 'linear', 'log', 'symlog', 'asinh', 'logit', 'function', 'functionlog'
to
' ' is not a valid value for norm; supported values are 'linear', 'log', 'symlog', 'asinh', 'logit', 'function', 'functionlog'
Which is slightly misleading, as norm supports other inputs than strings. We can mitigate this by intercepting the error message and replacing value with string, but this requires an additional try except block.
- I think this check naturally belongs in
_auto_norm_from_scale
, a function which is currently called by:
elif isinstance(norm, str):
try:
scale_cls = scale._scale_mapping[norm]
except KeyError:
raise ValueError(
"Invalid norm str name; the following values are "
f"supported: {', '.join(scale._scale_mapping)}"
) from None
norm = _auto_norm_from_scale(scale_cls)()
which I think should be replaced by:
elif isinstance(norm, str):
norm = _auto_norm_from_str(norm)()
(note the change in function name from _scale
to _str
)
This function then incorporates the check.
def _auto_norm_from_str(norm_str):
scale_cls = _api.check_getitem(scale._scale_mapping, norm=norm_str)
...
@timhoffm @story645 Let me know your opinions on point 1 and 2 above, and I will implement it in this PR.
Point 2. would be a useful change with regards to making a 'MultiNorm' in a later PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree the message would be slightly misleading. OTOH pushing this down does not make much better for the user. Thy still get the same exception message. Only the call stack is different. While the message in the deeper context is correct, it's further away from the public API and thus more difficult to trace back.
An alternative would be expanding _check_get_item
so that we can do _api.check_getitem(scale._scale_mapping, norm=norm_str, _type="str")
and get "' ' is not a valid str value ...", which would imply there may be other valid types.
I believe in practice, it does not matter. If the user has passed an invalid str, they most likely have a typo and still wanted to pass a str name. It's unlikely that the information being able to pass Norms directly would be helpful. To keep things simple, I slighly favor just using _check_get_item
as is in original context (this does not factor in any possible future needs). But overall no strong opinion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Point 2. would be a useful change with regards to making a 'MultiNorm' in a later PR.
Would it be easy to make this change inside the MultiNorm PR or better to have it here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's better to have it here, with the changes in #28690 that means we only have to move this function once.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would changing the get item check method in the way Tim described achieve the same ends? If so do that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If not, unfortunately I think moving it twice but making the change when folks can see why it helps is gonna be an easier sell.
c5393d5
to
2d4ae15
Compare
lib/matplotlib/artist.py
Outdated
"converted to float") | ||
|
||
self._A = A | ||
if not self.norm.scaled(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
are we sure that this will always have a .norm
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm thinking it will always have a norm, as I think that simplifies the implementation.
However, that norm might be NoNorm
.
The implementation here mirrors that of ScalarMappable
, which always has a Normalize
and Colormap
object, and I think this is sensible. For images that are already RGB(A) a norm (and cmap) is technically unnecessary, but I believe the overhead of creating them is negligible.
The norm and cmap properties only ever make sense with the |
2d4ae15
to
d130006
Compare
My motivation was to have a separation between "API we have to provide to ensure backward compatibility" and "API we want to provide". At least during the design phase this could make it more clear. We also need to decide at some point whether there is API that we want to discourage. This is all possible also with a single large class, but I have the impression that that is more mental load and we have to be extra careful to not miss anything. But no strong opinion. If you think you can keep the overview, that's ok. |
I'm going to make
Once/if |
lib/matplotlib/axes/_axes.py
Outdated
@@ -4696,7 +4697,7 @@ def invalid_shape_exception(csize, xsize): | |||
label_namer="y") | |||
@_docstring.interpd | |||
def scatter(self, x, y, s=None, c=None, marker=None, cmap=None, norm=None, | |||
vmin=None, vmax=None, alpha=None, linewidths=None, *, | |||
vmin=None, vmax=None, colorizer=None, alpha=None, linewidths=None, *, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This has to go after the *
or we break API (even though passing 10 positional arguments is a Bad Idea ™️ if someone was doing that we would now get the value of alpha
as colorizer
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Side-remark: We should consider tightening on the kw-only settings. It helps the understanding to collect related parameters next to each other. I'd be willing to force people with Bad Ideas™️ to adjust in this case.
But this doesn't help for now because we'd need to go through the deprecation cycle. So @tacaswell's comment is valid anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the feedback, I was unsure if it was OK to break the list of args for people with Bad Ideas™️, but I guess we should cater to everyone :)
The Colorizer class, ColorizerShim, and ColorableArtist which replaces the old ScalarMappable.
The colorizer keyword now always follows *args
This gives us a path to deprecate ScalarMappable with warnings if we want.
notes on possible _ScalarMappable removal
This is a hack, but not the _worst_ hack.
ce91be4
to
269ace9
Compare
The doc failure appears relevant, unfortunately. |
Thank you @trygvrad ! Congrats on driving a pretty big change to Matplotlib |
Just noting here that this needs a what's new entry |
The
Colorizer
class,ColorizerShim
, andColorableArtist
which replaces the oldScalarMappable
.PR summary
This PR implements the changes in #28428 where a container class is used to hold the$data\rightarrow color$ pipeline.
Schematically, this pipeline looks like this:
I have changed the names to be:
compared to the existing pipeline:
This is a minimal implementation so that the interested parties can take a look.
It does not include typing or new tests, but all existing tests pass
Note 1:
changed()
onColorizer
andColorableArtist
is executes:This is consistent with how the callbacks are handled on the norm
but different from how the callbacks are handled on
ScalarMappable
I believe this it is more consistent if the new classes do
self.callbacks.process('changed')
and I have not found any example where the additional argument matters.Note 2:
I would like to move
Colorizer
andColorizerShim
to a new file, instead ofcm.py
.If this is done,
cm.py
would only containColormapRegistry
(andScalarMappable
for compatibility).Note 3:
Here are some points that could be addressed in following PRs:
.colorizer
on both an artist and a colorbar.ColorableArtist
class has acbook.CallbackRegistry()
, but I believe this to be superfluous as all callbacks should in reality come from theColorizer
.Artist.format_cursor_data
should be moved toColorizableArtist
.cm.ScalarMappable
, where artists need additional ones. These should be changed to use the newColorizer
.Note 4:
The following code demonstrates how multiple axes can be connected to one
Colorizer
object:(I didn't re-record the video, it was orignally posted here and in order for changes made on the colorbar to propagate to the artist the
Colorbar
class needs some small changes [see point 1. in the list above], but changes on one of the images will propagate to all images and the colorbar as shown in the video.)https://github.com/user-attachments/assets/093d5fa1-0dfc-4a68-9197-24acd0ef4aed
PR checklist
¹ we should consider the existing tests for
cm.ScalarMappable
and see if any of them should be replicated to testColorizer
andColorableArtist
.