FlowKit should be able to automatically find FCS files based on uris in .wsp files
#166
Labels
enhancement
New feature or request
Milestone
Comments
|
Hi Hersh, Thanks for the PR, it's much appreciated. Our use cases generally do not have the referenced FCS files in the same locations as the original analyst so this functionality was never considered. One request is to make these changes off the I can do a proper review once you have these changes on the dev branch. Again, thanks for taking the time to do this! -Scott |
|
Thanks @whitews, I was guessing that some use cases must not have the paths written to the wsp files. My mistake, will fix the branch issue ASAP. Could you confirm how you run the tests? I get the following error when I run |
|
Looks like you are running the tests the correct way, some are just failing for some reason. At first I thought it might be due to you using Python 3.11 (not officially supported), but I set up a 3.11 venv and all the tests pass for me. Can you provide more details of your environment (OS, dep versions, etc.)? BTW, the tests passed for your new branch as well. |
|
MacOS 13.5, Python3.11.4. Apple Silicon.
|
|
Hi Hersh, My dev environment is Ubuntu 22.04, but I have an M2 Mac available. I updated that machine to Ventura, then installed Python 3.11.2 via pyenv. I created a venv for FlowKit testing from Python 3.11.2, and installed the dependencies using With this setup, the tests run without errors for me so I'm not sure what is happening on your side. I suspected numba since it always forces its NumPy version, but the tests passed after installing that too. Is the setup I described roughly what you did as well? Would you mind trying to replicate it again with a fresh venv? Sorry to sideline the PR, but I would like to know what is going on with those tests. From the output you provided, it wasn't a raised error that caused the failed test but a mismatch in a single element in a gate membership Boolean array (i.e. one event is supposed to be either included or excluded in the gate). Can you post the entire output of run_tests.py? Thanks, |
|
here is the output of the run in a new venv. pip list below. Also here is |
|
Thanks Hersh. After some experimenting I was able to replicate the same 9 failures on my M2 Mac. I updated XCode (and I guess the command line tools / compiler) and disabled caching for pip, which forced the venv to build the packages. I am quite sure this is due to differences in floating point precision across different architectures / compilers. Not sure what I'm going to do about that, but it has nothing to do with this PR. Back on topic, I'd like to add at least one test for the new functionality. However, the dynamic nature of the test environment makes hardcoding file paths in the .wsp file problematic. Using relative paths would solve this, do you know if relative paths work in FlowJo? |
|
Makes sense re the precision error. I've dealt with a similar issue before, and the solution was to enforce float precision rather than allow defaults. Not sure how feasible that's going to be in this case though. I edited a FlowJo WSP file and replaced the |
|
@whitews I made some changes in the PR that handle local paths. I was able to set up a test with static data for my own project here: https://github.com/hbhargava7/flowtx/tree/main/tests. You can try it with |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
tl;dr: It would be great if the FlowKit
Workspaceconstructor would find the FCS files automatically based on theuris written in the WSP files. Not only would this be hugely convenient, the current implementation is incompatible with custom Sample Display Naming in FlowJo, as well as FCS filenames that don't match the FlowJo sample names.My (not so esoteric, I'd guess) use case prevents FlowKit getting the FCS files no matter what: I have "Sample Display Naming" [1] in FlowJo set to a custom combination of properties, since I conduct timecourse experiments where each timepoint is its own experiment in BD FACSDiva, the software that controls our cytometer. This means FlowJo names the specimens with a name that is not the FCS filename.
My FCS files are stored like this
As a result, there are multiple FCS files with the same name (
specimen_A1.fcsfor multiple plates/timepoints/etc.), so that is not a suitable UID.Happily, there is a
urifield in the FlowJo WSP files that contains the absolute path to the FCS file for each specimen. It should be possible to read the FCS files from there,[1]: Here is the setting in FlowJo that's relevant
The text was updated successfully, but these errors were encountered: