Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python experiment: adding entry definitions to the basic variable capture branch #15167

Draft
wants to merge 53 commits into
base: main
Choose a base branch
from

Conversation

yoff
Copy link
Contributor

@yoff yoff commented Dec 19, 2023

Some tests will likely fail, but hopefully only obvious stuff and I am also interested in the DCA run.

yoff and others added 30 commits December 14, 2023 10:20
This provides variable capture in standard situations:
- nested functions
- lambdas
There are some deficiencies:
- we do not yet handle objects capturing variables.
- we do not handle variables captured via the `nonlocal` keyword.
  This should be solved at the AST level, though, and then it
  should "just work".

There are still inconsistencies in the case where
a `SynthesizedCaptureNode` has a comprehensions
as its enclosing callable. In this case,
`TFunction(cn.getEnclosingCallable())` is not
defined and so getEnclosingCallable does not exist
for the `CaptureNode`.
TODO:
The member predicate `LibraryLambdaMethod::getACall` is
currently too permissive.
Ideally, we would have `libraryCallHasLambdaArg`
as in Ruby. But even a more precise
`libraryCall` predicate might be fine.
On the small test project, this reduces the number
of instances from 285 to 22.
These are the labmda self references. This is similar to
how `BlockParameterArgumentNode` is excluded for Ruby.

It is important that we restrict `call` in this logic.
Otherwise, we get a cartesian product and the consistency
check runs for a very long time...
This depends on the extractor fix
this must have been lost in my
clean-up rebase.
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
also improve comment
…atch.qll

Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
…ate.qll

Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
yoff and others added 21 commits December 15, 2023 14:31
introduced by bad merge
for `CapturingClosureArgumentNode`.

Unless we define it for every single `CallNode`, we need a more
sophisticated mutual recursion with the call graph construction.
There is built-in support for that, but we are currently not using it.
Otherwise we get loads of nodes with missing locations
from the brnach nodes that are not matched.
next to `SynthCaptureNode`
- on debug predicates
- on JS implementation
Avoid the term "closure" as it is somewhat academic.
This will be added in a follow-up PR instead.
I chose `category: majorAnalysis`, the description is
"An API has changed in a way that may affect the results produced
by a query that consumes the API."
The API in question here is `flowPath` which is used by all our
data flow queries.
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
yoff added 2 commits December 20, 2023 10:58
otherwise we confuse captured variables
in the single scope entry cfg node. Now
we have one for each defined variable.

TODO: Add test to demonstrate gain.
also, it is slightly incorrect...
@yoff yoff force-pushed the python/cv-basic-with-entry-definitions branch from d90b7e8 to c2dcd55 Compare December 20, 2023 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant