Tag: Entrez

Updated and improved collection of RefSeq representative genome assemblies now available

We have updated the collection of representative genome assemblies for Bacteria and Archaea. As announced in April, this set is now recalculated three times a year. We selected a total of 11,727 prokaryotic assemblies to represent their respective species among the 192,000 assemblies in RefSeq. Six hundred and thirty-five species were included in the collection for the first time, while 395 organisms from undefined species (such as Bacillus bacterium) were removed. We were able to choose a higher-quality representative than in the previous set for 18% of Bacterial and Archaeal species due to improvements in the logic of the selection that is now based on the assembly length, number of pseudo CDSs called in the PGAP annotation, number of scaffolds, whether Gene IDs are available in the Gene database for the assembly that is currently representative, and type strain status. You can see the exact criteria in order of importance on the Prokaryotic RefSeq Genomes page. Now that the new selection process is in place, we expect future updates to have fewer changes. We will replace a representative only if the assembly has changed RefSeq status or if a substantially better assembly becomes available.

We have updated the database on the Microbial Nucleotide BLAST page as well as the basic nucleotide BLAST RefSeq Representative Genome Database, to reflect these changes.

You can download the reference and representative set from the Assembly resource. If you are interested in the annotation on these genomes, you can limit searches to proteins annotated on representative genomes by adding “refseq_select[filter]” to any query in the Protein database. For example, you can find all proteins annotated on representative genomes in the genus Klebsiella by using the query: “Klebsiella[organism] AND refseq_select[filter]“.  A BLAST database of proteins annotated on representative genomes will be coming soon. Stay tuned!

Improved Search Now Available Across NCBI Databases

Earlier this year, we announced the release of a new and improved search feature that interprets plain language to give better results for common searches. This feature, originally developed in NCBI Labs and later released on the NCBI All Databases search, is now available across several NCBI resources: Nucleotide, Protein, Gene, Genome, and Assembly. Whether you are searching for a specific gene or for a whole genome, you will now retrieve NCBI’s best results regardless of the database  you search.

The image below shows the results for a search for human INS in the Nucleotide database. Even though this is a Nucleotide search, the results include relevant information from Gene, Protein, Taxonomy,  plus links to the NCBI reference sequences (RefSeq) as well as access to BLAST and the insulin gene region in NCBI’s genome browser, the Genome Data Viewer.KIS_nuccore_smallFigure 1.  The new natural language search result in the Nucleotide database from a search for human INS.

Try out this new search capability and let us know what you think. And keep visiting the NCBI Labs search page to try our latest experiments, which we’ll also announce here on NCBI Insights.

 

Upcoming Changes to EST and GSS Databases

Upcoming Changes to EST and GSS Databases

Update: NCBI is now in the process of merging EST and GSS records into the Nucleotide database, and we expect to complete this process in early 2019. Accession.version and GI identifiers will not change during this process.

As of December 1, 2018, all records from the databases for Expressed Sequence Tags (EST) and Genome Survey Sequences (GSS) will reside in NCBI’s Nucleotide database. This change will provide a single point of access for all GenBank sequence data with a common look and feel.

Read more to learn about how this change affects these resources:

  • Websites (Entrez)
  • APIs (E-utilities)
  • FTP sites
  • Submission procedures
  • BLAST
  • TSA (have a look if you’re not familiar!)

Continue reading “Upcoming Changes to EST and GSS Databases”

Test drive a new sequence search experience at NCBI Labs

We know it’s not always easy to find the sequence data you’re after at NCBI. Maybe it’s because you’re no expert at constructing queries, and you end up with no results or too many results. Or maybe you’re an Entrez wizard, but creating a query full of Booleans and filters seems like overkill when you could just write a short natural language query, like you’re used to doing in Google.  The next time you search for a gene, transcript or genome assembly for a given organism, try the new search experience we’re piloting in NCBI Labs.

In NCBI Labs, you can now search for sequences using natural language and get the best results.

NCBI Labs transcript search interface
Figure 1. The new interface for specified transcript search.

The improved search experience now available in NCBI Labs addresses 3 types of queries that commonly fail in searches at NCBI: organism-gene (e.g. human BRCA1), organism-transcript (e.g. Mouse p53 transcripts) and organism-assembly (e.g. dog reference genome). For each of these query types in NCBI Labs, we now return NCBI’s highest quality sequence sets or reference and representative assemblies in an easy-to-view panel.

Example queries are shown below to get you started.

Continue reading “Test drive a new sequence search experience at NCBI Labs”

5 NCBI articles in 2018 Nucleic Acids Research database issue

5 NCBI articles in 2018 Nucleic Acids Research database issue

The 2018 Nucleic Acids Research database issue features several papers from NCBI staff that cover the status and future of databases including CCDS, ClinVar, GenBank and RefSeq. These papers are also available on PubMed. To read an article, click on the PMID number listed below.

Continue reading “5 NCBI articles in 2018 Nucleic Acids Research database issue”

GenBank release 223.0 is available via FTP, Entrez and BLAST

GenBank release 223.0 is available via FTP, Entrez and BLAST

GenBank release 223.0 (12/15/2017) has 206,293,625 traditional records (including non-bulk-oriented TSA) containing 249,722,163,594 base pairs of sequence data. In addition, there are 551,063,065 WGS records containing 2,466,098,053,327 base pairs of sequence data, 201,559,502 TSA records containing 181,394,660,188 base pairs of sequence data, and 12,695,198 TLS records containing 4,458,042,616 base pairs of sequence data.

Continue reading “GenBank release 223.0 is available via FTP, Entrez and BLAST”

GenBank release 221.0 is available via FTP, Entrez and BLAST

GenBank release 221.0 is available via FTP, Entrez and BLAST

GenBank release 221.0 (8/13/2017) has 203,180,606 traditional records containing 240,343,378,258 base pairs of sequence data. In addition, there are 499,965,722 WGS records containing 2,242,294,609,510 base pairs of sequence data, 186,777,106 TSA records containing 167,045,663,417 base pairs of sequence data, and 1,628,475 TLS records containing 824,191,338 base pairs of sequence data.

Continue reading “GenBank release 221.0 is available via FTP, Entrez and BLAST”

Exploring Entrez Direct: Parsing the XML Output of E-utilities

Entrez Direct is a UNIX/LINUX command-line interface to E-utilities, the API to the NCBI Entrez system. One of Entrez Direct’s most useful features is its ability to parse and reformat complex XML data returns from EFetch. In this post, we will explore how to use these features to parse, reformat and process specific data from PubMed records downloaded in XML using EFetch. Though this post focuses on PubMed, the technique is universal and applies to any XML returned by E-utilities from any database. The example explored here is also presented briefly in the Entrez Direct documentation; here we’ll dive in a bit depeer to see how it works. Let’s get started!

Continue reading “Exploring Entrez Direct: Parsing the XML Output of E-utilities”

The Second Offering of “A Librarian’s Guide to NCBI” at NIH

NCBI, in collaboration with NLM and the National Network of Libraries of Medicine NLM Training Center (NTC) at the University of Utah, recently presented the second offering of A Librarian’s Guide to NCBI. Health Sciences Librarians from 17 universities and two federal agencies attended the five-day intensive course on the NIH campus. This second offering of the training continues to prepare health science librarians for supporting NCBI molecular databases and tools, and training patrons in the use of NCBI resources at their own institutions.

Participants and instructors from the 2014 “A Librarian’s Guide to NCBI” outside the National Library of Medicine.
Participants and instructors from the 2014 “A Librarian’s Guide to NCBI” outside of the National Library of Medicine.

As before, all the course materials are available online. Feel free to learn from them, adapt them for your own teaching, and share them with others. You can use the links below to access the updated 2014 course materials. These include the slide sets with demonstrations and practice problems.

Continue reading “The Second Offering of “A Librarian’s Guide to NCBI” at NIH”