Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #2070 - bytes formatting incorrect in python 3 #2168

Merged
merged 14 commits into from
Sep 26, 2016

Conversation

buckbaskin
Copy link
Contributor

This pull request aims to fix the type checking of bytes strings. The code previously used a single path for both bytes and strings, which broke the example given in the bug (#2070) that expected a bytes output to a function for example. This pull request creates a separate method for handling bytes in the same pattern as strings, but with the correct type output.
At the time the pull request is created, there may be some errors in the types of inputs into the bytes % interpolation because it is using the type checking logic for string interpolation.

@buckbaskin buckbaskin closed this Sep 22, 2016
@buckbaskin buckbaskin reopened this Sep 22, 2016
@gvanrossum
Copy link
Member

Thanks! This looks fine, but I have a few requests:

  • Can you refactor check_str_interpolation() and check_bytes_interpolation() to reduce the code duplication? Maybe there should be just one function that doesn't return anything, and instead the return type should just be picked by the caller (that seems easy enough).
  • I don't like the cast. It would be better to pass in the string value and node context separately.
  • It seems the code here never supported UnicodeExpr, but in Python 2, expressions like u'I have %d shrubberies' % 12 should also be checked. Should be easy enough.

- check_str_interpolation accepts StrExpr, BytesExpr, UnicodeExpr
- remove a cast to a StrExpr
- Add "support" for UnicodeExpr

* Note that this does not implement special checks for differences
  between valid StrExpr and BytesExpr
Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would explain your test failure.

return self.strfrm_checker.check_byte_interpolation(e.left, e.right)
elif isinstance(e.left, StrExpr):
return self.strfrm_checker.check_str_interpolation(cast(StrExpr, e.left), e.right)
return self.strfrm_checker.check_str_interpolation(e.left, e.right)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That can't be right? What if e.left is not a string at all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the isinstance check back. It was my oversite that non StrExpr-like objects would get through to this point.

@@ -55,7 +55,7 @@ def __init__(self,
self.exprchk = exprchk
self.msg = msg

def check_str_interpolation(self, str: StrExpr, replacements: Node) -> Type:
def check_str_interpolation(self, str: Union[StrExpr, BytesExpr, UnicodeExpr], replacements: Node) -> Type:
"""Check the types of the 'replacements' in a string interpolation
expression: str % replacements
"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a TODO around here that in Python 3 the bytes format characters are actually more restricted than str/unicode format characters? (a) There are fewer supported characters; (b) %s only supports bytes. (In Python 2 these limitations do not apply.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also bytes formatting in Python 3 is only supported in 3.5 and up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the bytes formatting in 3.5 a TODO? Or is it something that I'll need to check for in code?

@gvanrossum
Copy link
Member

Up to you. If you have time to learn how to do it it would be great to add
this check (there's a pyversion variable used throughout the checker).

On Sun, Sep 25, 2016 at 9:16 PM, Buck Baskin notifications@github.com
wrote:

@buckbaskin commented on this pull request.

In mypy/checkstrformat.py #2168:

@@ -55,7 +55,7 @@ def init(self,
self.exprchk = exprchk
self.msg = msg

  • def check_str_interpolation(self, str: StrExpr, replacements: Node) -> Type:
  • def check_str_interpolation(self, str: Union[StrExpr, BytesExpr, UnicodeExpr], replacements: Node) -> Type:
    """Check the types of the 'replacements' in a string interpolation
    expression: str % replacements
    """

Is the bytes formatting in 3.5 a TODO? Or is it something that I'll need
to check for in code?


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#2168, or mute the thread
https://github.com/notifications/unsubscribe-auth/ACwrMlh-1BQHpJUtTe_S8SGDcrERQPZcks5qt0c4gaJpZM4KDicI
.

--Guido van Rossum (python.org/~guido)

@gvanrossum gvanrossum merged commit 2fbf387 into python:master Sep 26, 2016
@gvanrossum
Copy link
Member

Thank you for the patch! Are you interested in doing any of the follow-up TODOs?

@buckbaskin
Copy link
Contributor Author

I was planning on looking into the pyversion check next. What is the preferred process for completing a TODO? Should I file a bug and then make a pull request to fix it?

@gvanrossum
Copy link
Member

You can just send the PR, no issue required.

--Guido (mobile)

On Sep 26, 2016 10:29 AM, "Buck Baskin" notifications@github.com wrote:

I was planning on looking into the pyversion check next. What is the
preferred process for completing a TODO? Should I file a bug and then make
a pull request to fix it?


You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
#2168 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACwrMrd0lTGvx44n0WhhQQqxF2vOO8Knks5quAC9gaJpZM4KDicI
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants