Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Point to example data that is compatible with current specification #75

Merged
merged 3 commits into from
Sep 5, 2021

Conversation

rob-luke
Copy link
Member

@rob-luke rob-luke commented Sep 4, 2021

The current version of SNIRF sample data is outdated and does not meet the current specification. This is misleading to new users and causes people to waste their time. It wastes time when developing software, but also for maintaining software, I keep getting help requests from users saying my software is wrong because they took the time to test it with the official sample data.

This PR changes the example data to point to my repository of SNIRF files.

This is not ideal because its not maintained by the fNIRS GitHub account, also conflates the issue of BIDS, and the files are larger than I'd like for an example. But at least they are compliant with the specification.

This is likely a temporary fix until the sample data is updated. But every day the wrong link is provided it wastes peoples time and will leave a negative taste for the project.

@dboas
Copy link
Collaborator

dboas commented Sep 4, 2021

@sstucker and @jayd1860 , we need to update our data that is linked here. I guess we should do this once the first official release is made... hopefully very soon... and then we put up the snirf files that meet that official release.

@rob-luke
Copy link
Member Author

rob-luke commented Sep 4, 2021

@fangq has discussed this with me offline and raised the issue that example files should be 1) small, 2) address specific test cases, 3) be many examples that cover different scenarios. I agree with all this and in a perfect world someone would fix this today.

But my suggestion in this PR is the lesser of two evils. Currently the linked data is simply wrong. I know this because users have wasted days using it and complained to me. I agree with his points, and ideally the current examples would be fixed. But no one has done that, and every week the current files are linked it wastes peoples times and makes this project look unprofessional.

While my linked data isn't build for this purpose (they are example data for MNE and BIDS), they are correct according to the current specification. We can simply swap the link back to the ideal dataset when its updated.

@fangq
Copy link
Member

fangq commented Sep 4, 2021

with the above 2363e1c commit, can you let me know what else should be fixed in fNIRS-samples?

@rob-luke
Copy link
Member Author

rob-luke commented Sep 4, 2021

Are the sample files now compatible with the spec? If so, then just close this.

@rob-luke
Copy link
Member Author

rob-luke commented Sep 4, 2021

I just downloaded data from the readme link and tried snirf-samples-master/test/minimum_example.snirf and snirf-samples-master/basic/Simple_Probe.snirf and both threw errors in my reader. So I guess this isn't fixed (or my reader is broken).

I don't have time at the moment to specifically check what these issues were. I will leave this to you as owner of that data if you wish to fix the issue rather than merge this PR.

Edit: I also used the python snirf validator being developed by @dboas lab and the sample files dont even read in to it, just crashes. What software are you reading these files with to test them? The validator reads both my SNIRF files and the Homer SNIRF files, so yours must be something different. This just highlights the confusion/issue here and the time being wasted comparing files from different places.

@fangq
Copy link
Member

fangq commented Sep 5, 2021

go ahead and replace the sample submodule with yours.

@rob-luke
Copy link
Member Author

rob-luke commented Sep 5, 2021

Like this? Sorry I haven't used submodules before, might have made an error here.

@fangq fangq merged commit 01ec1d8 into fNIRS:master Sep 5, 2021
fangq added a commit that referenced this pull request Sep 5, 2021
@fangq
Copy link
Member

fangq commented Sep 5, 2021

done. see my above commit. submodules are pointers to a specific commit in a specific repo. after switching the URL in .gitmodules, you have to check out the updated module and run git add samples to update the commit hash.

nonetheless, my understand is that your sample repo serves as examples for BIDS conversion. We still have a need for a sample repo that provides small but wide variety of SNIRF data types (simple probe, complex probe, CW/TD/FD data, with/wo aux, ...) in order to perform efficient testing of parsers/validators/software.

if a validator crashes on the old sample data, something should be fixed, more likely it is the validator's bug. anyways, will look into when I have time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants