-
Notifications
You must be signed in to change notification settings - Fork 439
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds model verification testing #467
base: main
Are you sure you want to change the base?
Conversation
pyre_connection = PyreConnection(cur_dir / "verify_models") | ||
|
||
temp_file = tempfile.TemporaryFile(prefix=".pyre_configuration", suffix=".local", dir=cur_dir) | ||
temp_file.write(b'{"source_directories": ["verify_models"]}') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Put in taint models path as well pointing to the repositories stubs? This helps us make the action verify newer models as well.
temp_file = tempfile.TemporaryFile(prefix=".pyre_configuration", suffix=".local", dir=cur_dir) | ||
temp_file.write(b'{"source_directories": ["verify_models"]}') | ||
|
||
print(get_invalid_taint_models(pyre_connection)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What output do you get? Can we have a GitHub action that runs this script? We could mimic the pysa
action and run this script instead of running the deliberately vulnerable flask app's script.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes having a github action is the next step @abishekvashok but first need to find a way to make pyre_configuration.local play well with pyre_configuration at the root folder
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If doesn't play well, feel free to delete the .pyre_configuration file at the root of the dir and create a file at the your folder. For an action it should get you by :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah that's possible and is my last resort. Some clever use of tempfile seems cleaner though
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@PulkitMishra To avoid the issue with the .pyre_configuration located in the root folder the trick is to create a temporary directory in the global temporary folder (with something like tempfile.gettempdir()
) and create a folder there in which we can store the .pyre_configuration file. We can then pass to the PyreConnection the path of this folder so it will not conflict with the global .pyre_configuration.
We can create a function which does this in https://github.com/facebook/pyre-check/blob/main/api/connection.py and invoke it passing tempfile.gettempdir()
as root directory and the path to the virtualenvironment we created as a source dir. Example function:
def create_temporary_pyre_connection(
root_directory: pathlib.Path
targets: Optional[List[str]] = None,
source_directories: Optional[List[str]] = None,
) -> PyreConnection:
# We can compute the sha1 as folder name to avoid creating a new directory every time and use the same if the source
directories and targets do not change
root_directory.mkdir(exist_ok=True)
pyre_configuration_directory = (
root_directory
/ hashlib.sha1(
",".join((targets or []) + (source_directories or [])).encode()
).hexdigest()
)
pyre_configuration_directory.mkdir(exist_ok=True)
with open(
str(pyre_configuration_directory / ".pyre_configuration.local"), "w"
) as configuration:
configuration_contents: Dict[str, Any] = {}
if targets:
configuration_contents["targets"] = targets
if source_directories:
configuration_contents["source_directories"] = source_directories
json.dump(configuration_contents, configuration, indent=2)
return PyreConnection(pyre_configuration_directory)
WIP - creates a model validation testing system for the Opensource models