-
-
Notifications
You must be signed in to change notification settings - Fork 643
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add pickling support to ScmProjectTree #3189
Comments
stuhood
added a commit
that referenced
this issue
Apr 19, 2016
In order to properly invalidate for watchman events, we need to record that we walked symlinks in the product graph. This review adds symlink handling to `engine.exp.fs`, and improves the type safety of filesystem operations by requiring that a PathGlobs object matches exactly one of `Files`, `Dirs` or `Links`. - Replace `fs.Path` with `fs.Stat` subclasses: `fs.{File,Dir,Link}` - Add explicit support to PathGlobs for matching directories. A PathGlobs object matches either Files or Dirs, but not both. - Resolve symlinks by recursively requesting `Files`/`Dirs` for a `Path` projected from a `ReadLink` - Replace `RecursiveSubDirectories` with use of a `PathGlobs` object recursively matching `Dirs`. - Add ProjectTree.{lstat, readlink, listdir} - Split `test_fs.py` from `test_path_globs.py`, and prepare to test ScmProjectTree (see [#3189](#3189)) - Include tests in `test_fs.py` to validate the actual filesystem events that occurred. Testing Done: https://travis-ci.org/pantsbuild/pants/builds/124108442 Bugs closed: 3121, 3190 Reviewed at https://rbcommons.com/s/twitter/r/3691/
This codepath will be heavily exercised by https://github.com/pantsbuild/pants/blob/master/tests/python/pants_test/engine/exp/test_fs.py#L164 once it is pickleable. |
This was referenced Apr 27, 2016
Submitted as 265d140 Reviewed at https://rbcommons.com/s/twitter/r/3770/ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
ScmProjectTree is very not-pickleable, primarily because it eagerly constructs a
GitReader
object.Lazily constructing a GitReader and/or excluding the reader from pickling would go a long way.
The text was updated successfully, but these errors were encountered: