Skip to content

Tags: BioInfoTools/samtools

Tags

1.9

Verified

This tag was signed with the committer’s verified signature.
samtools release 1.9:

 * Samtools mpileup VCF and BCF output is now deprecated.  It is
   still functional, but will warn.  Please use bcftools mpileup
   instead. (samtools#884)

 * Samtools mpileup now handles the '-d' max_depth option
   differently.  There is no longer an enforced minimum, and '-d
   0' is interpreted as limitless (no maximum - warning this may
   be slow).  The default per-file depth is now 8000, which
   matches the value mpileup used to use when processing a single
   sample.  To get the previous default behaviour use the higher
   of 8000 divided by the number of samples across all input
   files, or 250. (samtools#859)

 * Samtools stats new features:

   - The '--remove-overlaps' option discounts overlapping portions
     of templates when computing coverage and mapped base counting.
     (samtools#855)



   - When a target file is in use, the number of bases inside the
     target is printed and the percentage of target bases with
     coverage above a given threshold specified by the
     '--cov-threshold' option. (samtools#855)

   - Split base composition and length statistics by first and last
     reads. (samtools#814, samtools#816)

 * Samtools faidx new features:

   - Now takes long options. (samtools#509, thanks to Pierre Lindenbaum)

   - Now warns about zero-length and truncated sequences due to the
     requested range being beyond the end of the sequence. (samtools#834)



   - Gets a new option (--continue) that allows it to carry on when a
     requested sequence was not in the index. (samtools#834)

   - It is now possible to supply the list of regions to output in a
     text file using the new '--region-file' option. (samtools#840)

   - New '-i' option to make faidx return the reverse complement of
     the regions requested. (samtools#878)

   - faidx now works on FASTQ (returning FASTA) and added a new fqidx
     command to index and return FASTQ. (samtools#852)

 * Samtools collate now has a fast option '-f' that only operates on
   primary pairs, dropping secondary and supplementary.  It tries to
   write pairs to the final output file as soon as both reads have
   been found. (samtools#818)

 * Samtools bedcov gets a new '-j' option to make it ignore deletions
   (D) and reference skips (N) when computing coverage. (samtools#843)

 * Small speed up to samtools coordinate sort, by converting it to
   use radix sort. (samtools#835, thanks to Zhuravleva Aleksandra)

 * Samtools idxstats now works on SAM and CRAM files, however this
   isn't fast due to some information lacking from indices. (samtools#832)

 * Compression levels may now be specified with the level=N
   output-fmt-option.  E.g. with -O bam,level=3.

 * Various documentation improvements.

 * Bug-fixes:

   - Improved error reporting in several places. (samtools#827, samtools#834, samtools#877,
     cd7197)

   - Various test improvements.

   - Fixed failures in the multi-region iterator (view -M) when
     regions provided via BED files include overlaps (samtools#819,
     reported by Dave Larson).

   - Samtools stats now counts '=' and 'X' CIGAR operators when
     counting mapped bases. (samtools#855)

   - Samtools stats has fixes for insert size filtering (-m, -i).
     (samtools#845; samtools#697 reported by Soumitra Pal)

   - Samtools stats -F now longer negates an earlier -d option.
     (samtools#830)

   - Fix samtools stats crash when using a target region. (samtools#875,
     reported by John Marshall)

   - Samtools sort now keeps to a single thread when the -@ option is
     absent. Previously it would spawn a writer thread, which could
     cause the CPU usage to go slightly over 100%. (samtools#833, reported by
     Matthias Bernt)

   - Fixed samtools phase '-A' option which was incorrectly defined
     to take a parameter. (samtools#850; samtools#846 reported by Dianne Velasco)

   - Fixed compilation problems when using C_INCLUDE_PATH. (samtools#870;
     samtools#817 reported by Robert Boissy)

   - Fixed --version when built from a Git repository. (samtools#844, thanks
     to John Marshall)

   - Use noenhanced mode for title in plot-bamstats.  Prevents
     unwanted interpretation of characters like underscore in
     gnuplot version 5. (samtools#829, thanks to M. Zapukhlyak)

   - blast2sam.pl now reports perfect match hits (no indels or
     mismatches). (samtools#873, thanks to Nils Homer)

   - Fixed bug in fasta and fastq subcommands where stdout would not
     be flushed correctly if the -0 option was used.

   - Fixed invalid memory access in mpileup and depth on alignment
     records where the sequence is absent.

1.8

Verified

This tag was signed with the committer’s verified signature.
samtools release 1.8:

 * samtools calmd now has a quiet mode.  This can be enabled by
   passing `-Q` to calmd. (Thanks to Colin Davenport)

 * In samtools depth `-d 0` will effectively remove the depth limit.
   (samtools#764)

 * Improvements made to samtools collate's interface and
   documentation.  It is now possible to specify an output file
   name using `-o`, instead of deriving it from the prefix used
   for temporary files.  The prefix itself is now optional if
   `-o` or `-O` (to stdout) is used. (samtools#780)

 * Bug-fixes:

   - Make samtools addreplacerg choose output format by file
     extension. (samtools#767; reported by Argy Megalios)

   - Merge tests now work on ungzipped data, allowing tests to be run
     against different deflate libraries.

   - samtools markdup error messages about missing tags have been
     updated with the suggestion that samtools fixmate is run
     beforehand. (samtools#765; reported by Yudong Cai)

   - Enables the `--reference` option for samtools fastq.  Now works
     like other programs when a reference sequence is needed for CRAM
     files. (samtools#791, reported by Milana Kaljevic)

1.7

Verified

This tag was signed with the committer’s verified signature.
Samtools release 1.7: markdup supplementaries, multi-region iterator

* HTSlib, and so samtools, now support BAMs which include CIGARs with more
  than 65535 operations as per HTS-Specs 18th November (dab57f4 and 2f915a8).

* samtools quickcheck will now write a warning to stderr if it finds
  any problems.  These messages can be suppressed with a new `-q` option.

* samtools markdup can now mark supplementary alignments of reads where
  the primary alignment is found to be a duplicate.  Supplementary marking
  can be turned on by passing the `-S` option to markdup.  When this
  option is enabled, all the alignment data will be written to a temporary
  file so that supplementary alignments that occur before a duplicated
  primary can be correctly marked in the final output.  The location
  of this temporary file can be influenced using the new `-T` option.

* samtools view now supports HTSlib's new multi-region iterator.
  This can be enabled by passing the `-M` option to view.  When using
  this option:

  - The BED filter (`-L` option) will use the index to skip through the file
  - Reads from overlapping regions will only be output once

* samtools bedcov will now ignore BED comment and header lines (samtools#571; thanks
  to Daniel Baker).

* samtools collate now updates the @hd SO: and GO: tags, and sort will
  remove a GO: tag if present.  (samtools#757; reported by Imran Haque).

* Bug-fixes:

 - maq2sam now checks for input files that end early.  (samtools#751; patch supplied
   by Alexandre Rebert of the Mayhem team, via Andreas Tille from Debian.)

 - Fixed incorrect check when looking up header tags that could lead
   to a crash in samtools stats. (samtools#208; thanks to Dave Larson.)

 - Fixed bug in samtools fastq `-O` option where it would fail if
   the OQ tag in the input file had an unexpected type. (samtools#758;
   reported by Taejeong Bae)

 - The MD5 calculations in samtools dict and md5fa did not handle
   non-alphabetic characters in the same way as the CRAM MD5 function.
   They have now been updated to match. (samtools#704; reported by Chris Norman).

 - Fix possible infinite loop in samtools targetcut.

 - Building bam_tview_curses should no longer fail if a curses header file
   cannot be found.

1.6

Verified

This tag was signed with the committer’s verified signature.
Release 1.6: New markdup command; sort improvements

* Added new markdup sub-command and '-m' option for fixmate.  Used together,
  they allow duplicates to be marked and optionally removed.  This
  fixes a number of problems with the old 'rmdup' sub-command, for
  example samtools issue samtools#497.  'rmdup' is kept for backwards compatibility
  but 'markdup' should be used in preference.

* Sort is now much better at keeping within the requested memory limit.  It
  should also be slightly faster and need fewer temporary files when the file
  to be sorted does not fit in memory.  (samtools#593; thanks to Nathan Weeks.)

* Sort no longer rewrites the header when merging from files.  It can also
  now merge from memory, so fewer temporary files need to be written and
  it is better at sorting in parallel when everything fits in memory.

* Both sort and merge now resolve ties when merging based on the position
  in the input file(s).  This makes them fully stable for all ordering
  options.  (Previously position sort was stable, but name and by tag
  sorts were not).

* New --output-qname option for mpileup.

* Support for building on Windows using msys2/mingw64 or cygwin has
  been improved.

1.5

Release 1.5 [Solstice Release] (21st June 2017)

--------------------

* Samtools fastq now has a -i option to create a fastq file from an index
  tag, and a -T option (similar to -t) to add user specified aux tags to
  the fastq header line.

* Samtools fastq can now create compressed fastq files, by giving the
  output filenames an extention of .gq, .bgz, or .bgzf

* Samtools sort has a -t TAG option, that allows records to be sorted by
  the value of the specified aux tag, then by position or name.  Merge
  gets a similar option, allowing files sorted this way to be merged.
  (samtools#675; thanks to Patrick Marks of 10xgenomics).

1.4.1

Release 1.4.1 (8th May 2017)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Added options to fastq to create fastq files from BC (or other)
  tags.

* Samtools view has gained a -G <flags> option to exclude on all bits
  set.  For example to discard reads where neither end has been
  mapped use "-G 12".

* Samtools cat has a -b <fofn> option to ease concatenation of many
  files.

* Added misc/samtools_tab_completion for bash auto-completion of
  samtools sub-commands. (samtools#560)

* Samtools tview now has J and K keys for verticale movement by 20
  lines. (samtools#257)

* Various compilation / portability improvements.

* Fixed issue with more than 65536 CIGAR operations and SAM/CRAM files.
(samtools#667)

1.4

Release 1.4 (13 March 2017)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Noteworthy changes in samtools:

* Fixed Issue samtools#345 - out-by-one error in insert-size in samtools stats

* bam_split now add a @pg header to the bam file

* Added mate cigar tag support to fixmate

* Multi-threading is now supported for decoding BAM and CRAM (as well
  as the previously supported encoding).  Most commands that read BAM
  or CRAM have gained an -@ or --threads arguments, providing a
  significant speed bonus.  For commands that both read and write
  files the threads are shared between decoding and encoding tasks.

* Added -a option to samtools mpileup to show all locations, including
  sites with zero depth; repeating the option as -aa or -a -a additionally
  shows reference sequences without any reads mapped to them (samtools#496).

* The mpileup text output no longer contains empty columns at zero coverage
  positions.  Previously it would output "...0\t\t..." in some circumstances
  (zero coverage due to being below a minumum base quality); this has been
  fixed to output as "...0\t*\t*..." with placeholder '*' characters as in
  other zero coverage circumstances (see PR samtools#537).

* To stop it from creating too many temporary files, samtools sort
  will now not run unless its per-thread memory limit (-m) is set to
  at least 1 megabyte (samtools#547).

* The misc/plot-bamstats script now has a -l / --log-y option to change
  various graphs to display their Y axis log-scaled.  Currently this
  affects the Insert Size graph (PR samtools#589; thanks to Anton Kratz).

* Fixmate will now also add and update MC (mate CIGAR) tags.

1.3.1

Samtools release 1.3.1: bug fixes, notably sort temporary files

* The sort command creates any needed temporary files alongside the final
  output file (similarly to the pre-1.3 behaviour), and now aborts when
  it detects a collision with another sort invocation's temporary files.

  When the -T PREFIX option specified is a directory (or when sorting to
  standard output), a random component is now added to temporary filenames
  to try to avoid collisions (samtools#432, samtools#523, samtools#529, samtools#535, PR samtools#530).

* All samtools commands now check for I/O errors more carefully, especially
  when writing output files (samtools#111, samtools#253, samtools#470, PR samtools#467).

* Build fixes for 32-bit systems; be sure to run configure on such systems
  to enable large file support and access to 2GiB+ files.

* The fasta/fastq/bam2fq command no longer ignores reads when the -s option
  is used (samtools#532).

* The fastq -O option no longer crashes on reads that do not have an OQ tag
  field (samtools#517).

* The merge and sort commands now handle (unusual) BAM files that have no
  textual @sq headers (samtools#548, samtools#550).

* Sorting files containing @co headers no longer duplicates the comment
  headers, which previously happened on large sorts for which temporary
  files were needed (samtools#563).

* The rmdup and view -l commands no longer crash on @rg headers that do not
  have a LB field (samtools#538).

* Fixed miscellaneous issues samtools#128, samtools#130, samtools#131, samtools#489, and samtools#514.

1.3

Samtools release 1.3: many improvements, fixes, new commands

* The obsolete "samtools sort in.bam out.prefix" usage has been removed.
  If you are still using -f, -o, or out.prefix, convert to use -T PREFIX
  and/or -o FILE instead.  (samtools#295, samtools#349, samtools#356, samtools#418, PR samtools#441; see also
  discussions in samtools#171, samtools#213.)

* The "bamshuf" command has been renamed to "collate" (hence the term
  bamshuf no longer appears in the documentation, though it still works
  on the command line for compatibility with existing scripts).

* The mpileup command now outputs the unseen allele in VCF/BCF as <*>
  rather than X or <X> as previously, and now has AD, ADF, ADR, INFO/AD,
  INFO/ADF, INFO/ADR --output-tags annotations that largely supersede
  the existing DV, DP4, DPR annotations.

* The mpileup command now applies BAQ calculations at all base positions,
  regardless of which -l or -r options are used (previously with -l it was
  not applied to the first few tens of bases of each chromosome, leading
  to different mpileup results with -l vs. -r; samtools#79, samtools#125, samtools#286, samtools#407).

* Samtools now has a configure script which checks your build environment
  and facilitates choosing which HTSlib to build against.  See INSTALL
  for details.

* Samtools's Makefile now fully supports the standard convention of
  allowing CC/CPPFLAGS/CFLAGS/LDFLAGS/LIBS to be overridden as needed.
  Previously it listened to $(LDLIBS) instead; if you were overriding
  that, you should now override LIBS rather than LDLIBS.

* A new addreplacerg command that adds or alters @rg headers and RG:Z
  record tags has been added.

* The rmdup command no longer immediately aborts (previously it always
  aborted with "bam_get_library() not yet implemented"), but remains
  not recommended for most use (samtools#159, samtools#252, samtools#291, samtools#393).

* Merging files with millions of headers now completes in a reasonable
  amount of time (samtools#337, samtools#373, samtools#419, samtools#453; thanks to Nathan Weeks,
  Chris Smowton, Martin Pollard, Rob Davies).

* Samtools index's optional index output path argument works again (samtools#199).

* Fixed calmd, targetcut, and potential mpileup segfaults when given broken
  alignments with POS far beyond the end of their reference sequences.

* If you have source code using bam_md.c's bam_fillmd1_core(), bam_cap_mapQ(),
  or bam_prob_realn_core() functions, note that these now take an additional
  ref_len parameter.  (The versions named without "_core" are unchanged.)

* The tview command's colour scheme has been altered to be more suitable
  for users with colour blindness (samtools#457).

* Samtools depad command now handles CIGAR N operators and accepts
  CRAM files (samtools#201, samtools#404).

* Samtools stats now outputs separate "N" and "other" columns in the
  ACGT content per cycle section (samtools#376).

* Added -a option to samtools depth to show all locations, including
  zero depth sites (samtools#374).

* New samtools dict command, which creates a sequence dictionary
  (as used by Picard) from a FASTA reference file.

* Samtools stats --target-regions option works again.

* Added legacy API sam.h functions sam_index_load() and samfetch() providing
  bam_fetch()-style iteration over either BAM or CRAM files.  (In general
  we recommend recoding against the htslib API directly, but this addition
  may help existing libbam-using programs to be CRAM-enabled easily.)

* Fixed legacy API's samopen() to write headers only with "wh" when writing
  SAM files.  Plain "w" suppresses headers for SAM file output, but this
  was broken in 1.2.

* "samtools fixmate - -" works in pipelines again; with 1.0 to 1.2,
  this failed with "[bam_mating] cannot determine output format".

* Restored previous "samtools calmd -u" behaviour of writing compression
  level 0 BAM files.  Samtools 1.0 to 1.2 incorrectly wrote raw non-BGZF
  BAM files, which cannot be read by most other tools.  (Samtools commands
  other than calmd were unaffected by this bug.)

* Restored bam_nt16_nt4_table[] to legacy API header bam.h.

* Fixed bugs samtools#269, samtools#305, samtools#320, samtools#328, samtools#346, samtools#353, samtools#365, samtools#392, samtools#410, samtools#445,
  samtools#462, samtools#475, and samtools#495.

1.2

Bug fix release