Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing background infrastructure for recursive types: Part 3 #7885

Merged
merged 18 commits into from
Nov 8, 2019

Conversation

ilevkivskyi
Copy link
Member

@ilevkivskyi ilevkivskyi commented Nov 5, 2019

This is the last part of plumbing for recursive types (previous #7366 and #7330). Here I implement visitors and related functions.

I convinced myself that we need to only be more careful when a recursive type is checked against another recursive one, so I only special-case these. Logic is similar to how protocols behave, because very roughly type alias can be imagined as a protocol with single property:

A = Union[T, Tuple[A[T], ...]]

class A(Protocol[T]):
    @property
    def __base__(self) -> Union[T, Tuple[A[T], ...]]: ...

but where TypeAliasType plays role of Instance and TypeAlias plays role of TypeInfo.

Next two pull requests will contain some non-trivial implementation logic.

Copy link
Collaborator

@msullivan msullivan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all seems pretty reasonable.

What is the testing situation here? Do you have an unpolished local implementation of the rest that gives some confidence that this works? I suppose I'm happy pushing ahead either way, though; it's not the end of the world if this needs further changes.

return self._visit(typs)

def _visit(self, typ_or_typs: Union[types.Type, Iterable[types.Type]]) -> Set[str]:
typs = [typ_or_typs] if isinstance(typ_or_typs, types.Type) else typ_or_typs
output = set() # type: Set[str]
for typ in typs:
if any(typ is s for s in self.seen):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems bad to be doing a search through every seen type so frequently. I assume it is done this way because it needs to be by identity and not by type equality, but then it would probably be better to have a set of ids.

(This is less important, I think, but would it be possible to only track seen TypeAliasTypes and to only do the check when visiting them?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I think it is fine to use equality here, I will add unit test to double-check.

mypy/join.py Outdated Show resolved Hide resolved
mypy/join.py Show resolved Hide resolved
mypy/meet.py Show resolved Hide resolved
mypy/nodes.py Outdated Show resolved Hide resolved
mypy/subtypes.py Show resolved Hide resolved
Comment on lines +237 to +241
def visit_type_alias_type(self, t: TypeAliasType) -> Type:
# This method doesn't have a default implementation for type translators,
# because type aliases are special: some information is contained in the
# TypeAlias node, and we normally don't generate new nodes. Every subclass
# must implement this depending on its semantics.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No reasonable way around this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately no. This is kind of a fundamental limitation. Technically we can put something moderately reasonable, but then there is a high chance we will forget to override this when needed in some visitors we will add in future.

Copy link
Collaborator

@JukkaL JukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a quick pass. Generally looks good!

Would it make sense to at least add some non-data-driven unit tests for recursive types at this point?

mypy/join.py Outdated Show resolved Hide resolved
@ilevkivskyi
Copy link
Member Author

Would it make sense to at least add some non-data-driven unit tests for recursive types at this point?

Yes, I think it would make sense to add some unit tests.

@ilevkivskyi
Copy link
Member Author

Yes, I think it would make sense to add some unit tests.

Actually I just discovered I am missing one implementation to be able to run unit tests, I will add it and add some tests tomorrow.

@ilevkivskyi
Copy link
Member Author

@msullivan @JukkaL Could one of you please take a look again?

Summary of changes:

  • Address review comments
  • Add some comments and docstrings
  • Added basic unit tests (basics, some visitors, subtyping)
  • Replaced ad-hoc type alias expansion with proper visitors
  • Shifted boundary of get_proper_type() around unions and expand_types() so that it happens later (previous location caused some conceptual difficulties)
  • Moved the shared state to TypeState where it belongs

Unfortunately the typeshed test is going to fail, because the type alias expansion clean-up discovered two errors in typeshed (previously Type[...] was not visited). Looking at it, it seems to me the check for class variables there is overly strict, we can move it from definition site to use site as we do for class methods. I will make a separate PR, and will merge this one on top.

Copy link
Collaborator

@msullivan msullivan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks!

What's next? Hooking it up in the semantic analyzer?

@ilevkivskyi
Copy link
Member Author

What's next? Hooking it up in the semantic analyzer?

Yes, exactly!

@ilevkivskyi ilevkivskyi merged commit 59617e8 into python:master Nov 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants