Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sketching paired-end read data #32

Closed
mw55309 opened this issue Jul 26, 2016 · 6 comments
Closed

Sketching paired-end read data #32

mw55309 opened this issue Jul 26, 2016 · 6 comments

Comments

@mw55309
Copy link

mw55309 commented Jul 26, 2016

Hi

I note no mention of paired-end data in the docs; sketching with multiple files creates a multi-file sketch, but if I want a single sketch per paired-end sample, how is this done?

I assume I should be able to use sub-shells and/or zcat and pipe to stdin; however when i did this on our cluster I got core dumps (admittedly I seem to be having some problems with zcat on our cluster....)

At the very least it may be a good idea to update your docs to reflect how to sketch paired-end read data

Cheers
Mick

@ondovb
Copy link
Member

ondovb commented Jul 27, 2016

You have the right idea catting to stdin. I added this step to the tutorial to clarify, but this could certainly be more elegant, maybe with a flag that pools all inputs into one sketch (possibly a behavior of the read flag -r).

If you want to try avoiding zcat, you can cat the gz files directly; mash will inflate each one in-stream.

@mw55309
Copy link
Author

mw55309 commented Aug 4, 2016

I do seem to have this persistent core dump:

zcat SRR1262647_1.fastq.gz | ../mash-Linux64-v1.1/mash sketch -k 21 -r -
Sketching from stdin...
Segmentation fault

Same with cat:

cat SRR1262647_1.fastq.gz | ../mash-Linux64-v1.1/mash sketch -k 21 -r -
Sketching from stdin...
Segmentation fault

Works fine without piping:

../mash-Linux64-v1.1/mash sketch -k 21 -r SRR1262647_1.fastq.gz
Sketching SRR1262647_1.fastq.gz...
Estimated genome size: 7.01971e+08
Estimated coverage: 2.942
Writing to SRR1262647_1.fastq.gz.msh...

Any ideas?

@ondovb
Copy link
Member

ondovb commented Aug 9, 2016

Looks look stdin input was broken in 1.1. A fix is now in the latest source and will be included in the next release.

@ondovb
Copy link
Member

ondovb commented Aug 29, 2016

Should be fixed in v1.1.1.

@ondovb ondovb closed this as completed Aug 29, 2016
@alienzj
Copy link

alienzj commented Apr 3, 2018

Hello,

cat sample_1.fq.gz sample_2.fq.gz | mash sketch -k 21 -s 10000 -r - -o sample
mash info sample.msh

output:

Header:
  Hash function (seed):          MurmurHash3_x64_128 (42)
  K-mer size:                    21 (64-bit hashes)
  Alphabet:                      ACGT (canonical)
  Target min-hashes per sketch:  10000
  Sketches:                      1

Sketches:
  [Hashes]  [Length]   [ID]  [Comment]

  10000     815194870  -      -

The ID is empty.
If we have many samples pair-ended fastq files, and get sketch for each sample, then paste all into a single file, then mash dist it, finally all ID is empty.

We can do bellow:

cat sample_1.fq.gz sample_2.fq.gz > sample.fq.gz
mash sketch -k 21 -s 10000 sample.fq.gz -o sample

so can avoid ID issue, but couldn't enjoy the pleasure of unix stream pipeline : )

Thanks for the author
Such a great and creative tool!

@ondovb
Copy link
Member

ondovb commented Sep 26, 2018

In latest source the -r flag will combine all input files and allows filling the empty fields with -I and -C.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants