logical_line
for call with f-string can result in invalid ast under Python 3.12 #1948
Description
how did you install flake8?
$ pip install flake8
unmodified output of flake8 --bug-report
{}
describe the problem
what I expected to happen
Some of the plugins in hacking
use a combo of ast.parse
and logical_line
to generate an AST for an individual line that is then fed into an ast.NodeVisitor
subclass. These work just fine on Python 3.11 and earlier, but I've noticed they fail under specific circumstances on Python 3.12. I've included a minimal reproducer below. There's a chance I am "holding it wrong" but I'd like to confirm this first.
sample code
test.py
import unittest
class TestFoo(unittest.TestCase):
def test_foo(self):
response_key = 'foo'
response_val = 'bar'
body_output = ''
self.assertEqual(
f'{{"{response_key}": "{response_val}"}}', body_output
)
checks.py
:
class NoneArgChecker(ast.NodeVisitor):
def __init__(self, func_name, num_args=2):
self.func_name = func_name
self.num_args = num_args
self.none_found = False
def visit_Call(self, node):
# snipped
...
def foo(logical_line, noqa):
if noqa:
return
for func_name in (
'assertEqual', 'assertIs', 'assertNotEqual', 'assertIsNot'
):
try:
start = logical_line.index('.%s(' % func_name) + 1
except ValueError:
continue
checker = NoneArgChecker(func_name)
# print(logical_line)
checker.visit(ast.parse(logical_line))
continue
if checker.none_found:
yield start, "H203: Use assertIs(Not)None to check for None"
tox.ini
:
[flake8:local-plugins]
extension =
X123 = checks:foo
paths = .
commands ran
$ flake8 test.py
wow.py:1:23: E999 SyntaxError: invalid syntax. Perhaps you forgot a comma?
If I comment out the print
statement, I see different results for Python 3.12 compared Pythons 3.10 and 3.11. Under 3.12:
self.assertEqual(f'x{x{response_key}xxxx{response_val}xx}', body_output)
Under 3.10 and 3.11:
self.assertEqual(f'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx', body_output)
additional information
The E999 error is associated with line 1 of the file, despite the error actually coming from a plugin. This led me on a wild goose chase as I tried to figure out why flake8 thought my file was invalid Python but Python itself did not. I suspect this might also be user error and I should be handling the potential exception from ast.parse
in my plugin but again, I just wanted to confirm that this was expected practice.