The Pfam protein families database

Using Wikipedia as a repository for protein family annotation

Historically, Pfam has provided only a basic level of textual annotation for each family. This has included a few sentences, with references, designed to give users an overview of the function(s) of the family or domain. However, rather than have the Pfam curators describe our families, we would strongly prefer to have annotations written by those who know the proteins and families best, namely the biologists and informaticians who work with them on a day-to-day basis. Harnessing the knowledge of these experts remains a significant challenge.

One recent approach has been to use Wikipedia as a source of scientific information (7,8). Wikipedia is the world's largest online encyclopedia, with over 3.7 million English language articles, and is widely acknowledged to be the most popular general reference work on the internet. A cornerstone of Wikipedia is that anyone can edit the content.

The Rfam database moved all of its family annotation into articles in Wikipedia in 2009, thereby allowing anyone to freely edit and improve their content. This experiment has proved successful, engaging the wider scientific community to provide expert annotations and improving the overall quality of the Rfam annotation (7). In light of this positive experience, we decided to adopt the same approach and use Wikipedia as the primary source of Pfam annotation (Figure 1A).

The Rfam database included around 600 families when the switch to Wikipedia was made. This made it feasible to assign existing articles where possible and to generate new ‘stub’ articles in Wikipedia for any family that still lacked any relevant article. These stubs have since been gradually expanded and improved by the Rfam and Wikipedia communities. At the time when Pfam began using Wikipedia for annotations, release 25.0, there were 12 273 Pfam-A families. We initially identified existing articles that described protein domains or families and which provided useful information about the Pfam family. These articles are now assigned as the primary annotation for the appropriate families. Given the number of families that remain without an article, however, it is simply not feasible to manually generate articles for all of them. For these families we continue to show the original annotation comments, which were written by the curator of the family, while encouraging our users to tell us about appropriate Wikipedia articles or to create them on our behalf. Furthermore, it is likely that there will be some families that are not sufficiently notable for inclusion in Wikipedia and we anticipate that many of these will remain without Wikipedia annotations for the long term, perhaps indefinitely.

As of release 26.0, there are 4909 Pfam families that link to 1016 Wikipedia articles. We invite readers and users of the Pfam website to edit and improve these articles in Wikipedia. Mapping of Pfam-A families to Wikipedia articles is available in JSON format, from: http://pfamsrv.sanger.ac.uk/cgi-bin/mapping.cgi?db=pfam.

Some Wikipedia articles cover multiple Pfam families, such as the Zinc finger article or the Interleukin article. Pfam contains a large series of 3526 families noted as DUFs. Virtually all of these DUF families link to a single Wikipedia article, which describes DUFs in general, and will do so until such time as their function is determined and they have an article of their own.

Although the process of manually creating a new Wikipedia article can be time-consuming and difficult, we are keen to increase the number of Wikipedia-annotated families in Pfam as much as possible. We have therefore developed a pipeline to generate stub articles automatically in a Wikipedia ‘sandbox’, often taking existing family annotations from the InterPro database (9) as the basis for the article. These stubs can then be reviewed and edited by our curators, before being moved out of the sandbox into Wikipedia proper and used to annotate families. We have implemented several automated procedures for augmenting the basic annotation text and expanding the content as far as possible before its final publication in Wikipedia.

A particularly useful feature of Wikipedia is the highlighting of terms within an article that are themselves described by another Wikipedia article. This network of linked terms allows readers to quickly understand the background to the article they are reading and, as such, they are crucial to the success of any article. To assist with the cross-linking of our new, automatically generated Wikipedia articles, we took the initial set of ∼700 Pfam Wikipedia articles and computationally collected a broad set of common terms. These terms were then automatically marked as links in the stub articles. Another essential feature of a Wikipedia page is the reference list. We used the TemplateFiller Perl module (http://search.cpan.org/dist/WWW-Wikipedia-TemplateFiller/) to retrieve and include the full details of the references cited in the InterPro annotation that we used as our starting point. Finally, we have automatically populated the ‘infobox’ (http://en.wikipedia.org/wiki/Help:Infobox) in our stub articles. This infobox (Figure 1A), located on the right-hand side of Wikipedia protein family pages, shows images of the relevant three-dimensional structures, where available, and additional database links. When an image of the protein structure was available from Wikimedia commons (http://commons.wikimedia.org/wiki/Main_Page), this was added to the top of the infobox, along with a caption. Further information was extracted from the Pfam database and added to the infobox, such as the Pfam clan accession and links to other database sites such as PROSITE (10), SCOP (11) and CAZy (12).

Altogether, the automatic article creation process generated 7823 articles in our Wikipedia sandbox. We continue to review, edit and move these stub articles into the main part of Wikipedia. In order to prioritize the best articles arising from the generation process, we calculated for each one a heuristic score, based on the size of the annotation, availability or otherwise of an image, the number of references and the number of links out to other databases. The score gave an overall measure of the level of information in the page and thus an indication of its potential for addition to Wikipedia. Already >200 of the highest scoring articles have been moved across to generate new Wikipedia pages.

One of the major concerns about Wikipedia generally is the risk of vandalism and deliberate errors being introduced into publicly edited articles. This is of particular concern to both the Pfam and Rfam projects, since we ‘scrape’ and re-display Wikipedia contents within our respective websites. In order to reduce the likelihood of blatant vandalism or egregious errors propagating through to our websites, we include an additional approval process before displaying newly edited Wikipedia articles. Our curators review and, if necessary, revert changes to articles on a daily basis and only after an article has been reviewed it is flagged for update and presentation within the Pfam website. In our experience, almost every case of vandalism is reverted by the Wikipedia community before we come to review the changes. Overall we have found that ∼1% of all edits are reverted by the Wikipedia community, suggesting an upper bound on the possible number of vandalism edits.

It is important to stress that the Wikipedia content displayed in Pfam family pages is an exact copy of the article that can be found on the main Wikipedia website, subject to a delay of a day or so for the approval process described above.

Family function annotation via the Pfam helpdesk

Val Wood of the Schizosaccharomyces pombe database, PomBase, routinely reports new findings from the literature to the Pfam helpdesk (pfam-help@sanger.ac.uk). Over the last 12 months, 74 such communications have been received, of which at least four provided evidence for the function of a DUF. A further 16 concerned hits of newly characterized S. pombe sequences to Pfam-B families, thus leading to the building of at least that number of new families.

One good example of a family that has been characterized in this way is DUF1709, in which a fission yeast anillin protein was characterized. Anillin proteins are actin-binding proteins involved in septin-organization, which are localized to the cleavage-furrow during cell division (13). The DUF has been re-named Anillin (PF08174).

Similarly, Pfam-B family PB008473 from Pfam release 24.0 was found to contain a fission yeast protein, Mtr4 (UniProtKB: Q9P795), which had been determined experimentally to be an essential RNA helicase that performs a critical role as an activator of the nuclear exosome in RNA processing and degradation. From this finding (14), family rRNA_proc-arch (PF13234) was built and described.

Other contributions from the community over the last year have included 51 direct annotation submissions (received via a web form available from the Pfam family pages) with suggestions for improvements and updates to the Pfam annotations; of these, 14 offered information about the function of DUFs, 4 of these coming from the InterPro team. A good example of how the functionality of InterPro benefits Pfam was a case where a team member flagged up one of our DUFs, DUF3462 (PF11945), as being the WASH subunit of the WASH complex (Wiskott–Aldrich Syndrome Protein and SCAR Homolog) that acts as an Arp2/3 activator necessary for Golgi-directed trafficking (15). The DUF was re-named as the WAHD domain of WASH complex.

Examples of cases where the determination of the three-dimensional structure of sequences from bacterial DUFs has led to the discovery of function are detailed in a dedicated issue of Acta Crystallographica, Section F (16).

Extending the community of Pfam curators

Although Wikipedia offers a mechanism for external scientists to contribute annotations using an established mechanism, it is restricted purely to functional annotation. As outlined above, the helpdesk provides a way of making more substantial contributions to the database, but the fraction of new families derived from helpdesk submissions is relatively small. Furthermore, contributions from the helpdesk are often in a different format and/or use a different sequence database to that used by Pfam. More often than not, a Pfam curator has to spend time understanding and modifying the submission to conform to the Pfam data model, which makes the helpdesk a far from ideal interface for bulk submissions.

Pfam is run by an international consortium of three groups, but until very recently our fundamental family data could be modified only at our Cambridge, UK, site. This has meant that even full consortium members have been unable to add their own families to Pfam. In order to remove this restriction, and with the goal of making it easier for members of the wider community to add families, we have developed a system that allows Pfam families to be added by registered users anywhere in the world. The distributed system involves the local installation of our family building pipeline (a set of Perl scripts and modules) and various quality control procedures. It allows the addition of new families and clans, as well as the modification of existing entries. Data are sent backwards and forwards between the user and a central, master server using HTTPS, and we are able to authenticate all traffic that results in changes to the database. Files are maintained using the widely used Subversion revision control system (http://subversion.apache.org/), thereby preventing the inadvertent conflicts that could occur when multiple users wish to make changes in a distributed environment.

Owing to the organizational changes detailed above, we have been able to embrace two external groups who work with data relevant to Pfam, giving them direct access to the Pfam submission pipeline. The Protein and Genome Evolution Research Group, run by L. Aravind at the National Center for Biotechnology Information (NCBI, USA), are experts in protein evolution, routinely publishing articles on large evolutionary related superfamilies. The second external contributing group is the one of Adam Godzik at the Burnham Institute (USA), part of the Joint Center for Structural Genomics (JCSG).

Members of both of these groups are now able to submit families and clans directly to Pfam, allowing them to improve Pfam data and to extend its reach to new communities and a broader audience. We see the introduction of this distributed curation model, in combination with the use of Wikipedia as a source of our annotations, as two important steps in making our database a community-based resource. Our goal is to provide an infrastructure that empowers scientists to contribute using whichever mechanism they feel most comfortable, while still allowing us to maintain oversight and control of the quality of our fundamental data.

Sunburst representation of the taxonomic tree

For each Pfam-A family we provide an interactive taxonomic tree, showing the species distribution of sequences in the family. However, due to the size of many families, this tree can be very large, making it difficult to gain a clear impression of the species distribution of the family. In order to address this problem, we have introduced a ‘sunburst’ representation of the species trees, as shown in Figure 1B. Sunbursts are a commonly used method of visualizing tree-like data sets, whereby the root of a tree is plotted as a circle, surrounded by concentric rings representing child nodes. In Pfam, each node of the taxonomic tree is drawn as an arc, whose distance from the centre corresponds to the taxonomic level of the node and whose length (or, equivalently, the angle subtended by the arc) is scaled to represent either the number of sequences or the number of species belonging to that node in the tree. The switch between scaling according to numbers of sequences or species may be changed interactively using a control in the page. Arcs are coloured according to kingdom. As the mouse pointer is moved across the sunburst, a tool-tip shows a summary of the current node, giving the species name for that node, along with the number of species and number of sequences beneath it. A summary panel also shows a simple graphical representation of the lineage of the relevant node. The overall size of the plot may be adjusted using a simple slider.

The sunburst tree is generated by mapping the UniProtKB assigned NCBI taxonomy identifiers onto the standard NCBI taxonomy. Unfortunately, there is not a perfect equivalence between taxonomy trees used by UniProtKB and NCBI, due simply to the fluid nature of the data and the different update cycles of the two resources. This mis-match inevitably generates cases where the mapping between the taxonomy identifiers in UniProtKB and NCBI breaks down. Species that cannot be assigned an exact node in the NCBI tree are shown as ‘Unclassified’ in the sunburst. Furthermore, because the NCBI taxonomy contains numerous levels that are not present across all species, we have attempted to normalize taxonomic levels to the eight major ones (domain, kingdom, phyla, class, order, family, genus, species). For example, the lineage of Bos taurus contains the sub-family level Bovinae, which we skip over and connect the genus directly to the family level, Bovidae. Some lineages also omit one or more of the major levels. Again, in the case of B. taurus, the level ‘order’ is omitted and the missing level is flagged with ‘No order’. We perform a node merger in the case of sub-species so that, for example, all sub-species of Escherichia coli are merged up to the species level and presented as E. coli sequences. These normalization steps allow us to draw every species with the same eight levels, making the outer ring of the sunburst complete and allowing the plot to represent more intuitively the distribution of sequences at each level.

Reinstatement of the PfamAlyzer tool for complex architecture queries

PfamAlyzer (17) is a Java applet that provides a user-friendly graphical interface to Pfam (Figure 1C). It was available in a previous version of the Pfam website (18) but was removed during development of the new website. It has now been reinstated and can be accessed through the search page. PfamAlyzer enables complex domain architecture queries to be specified using a simple drag-and-drop interface. The user can select a set of domains from drop-down lists of Pfam-A families or Pfam clans and drag and arrange them to build a query architecture. PfamAlyzer use has been described in detail elsewhere (17,18).

What are sequence and domain gathering thresholds?

The gathering thresholds, or GAs, are manually curated, family-specific, bit score thresholds that are chosen by Pfam curators at the time a family is built. Every family is given two GAs, a ‘sequence’ threshold, and a ‘domain’ threshold. In HMMER, the sequence bit score is the sum of all scoring matches between the sequence and the profile HMM. The domain bit score is the score assigned to each reported match between the sequence and the profile HMM. For a protein region to be considered as part of a family, both its sequence and domain bit scores must be equal to, or greater than, the corresponding GA. In families that contain sequences with multiple matches to the profile HMM, domain thresholds can be set to a value lower than sequence thresholds, in order to increase sensitivity. This is based on the assumption that finding multiple copies of a domain on the same sequence increases the chance of those instances being genuine matches, even when their E-values are not very significant when taken in isolation. This is particularly true for Pfam families assigned the type ‘repeat’, where instances of the repeating unit within a sequence diverge substantially from the consensus. In practice, only 2.3% of all families have different sequence and domain GAs.

Criteria for gathering threshold assignment

Family GAs are chosen with the goal of maximizing coverage while excluding any false positive matches. Although the number of false positives for a given threshold is generally unknown, one way to monitor the false positive-rate indirectly is to check for overlaps between one Pfam family and another. If the same region of a sequence matches two Pfam families, it should be considered a false positive in one of them. This holds true unless the two families are found to be in the same clan, i.e. the observed overlap is believed to reflect an evolutionary relationship between them.

When building a new family, therefore, the GA choice is often influenced by overlaps with other families. In general, overlap-resolution between old and new families leads to GAs being raised over time, since one way to resolve the overlap is to raise the GA in one or the other of the families. This means, for example, that when the UniProtKB dataset underlying Pfam is updated, i.e. at every new release, numerous GAs need to be modified. This is because new sequences will have introduced many new overlaps and, as stated above, Pfam does not allow overlaps between families that are not in the same clan. In a few cases, these sequences will indeed be judged ‘transitional’ between two families and the families will be added to a same clan. In Figure 3, we compare the values of sequence GAs for families in Pfam release 24.0 with GAs of the same families in release 26.0. Overall, 13% of GAs have changed, of which 91% have been raised.

Distribution of gathering thresholds of Pfam families and their relationship to E-values

The distributions of Pfam family GAs and corresponding E-values are shown in Figures 4A and B, respectively. The two GA peaks observed for intervals 25.0–26.0 and 27.0–28.0 (Figure 4A) are due to the fact that numerous Pfam GAs (∼27%) are set to fixed integer values of either 25.0 or 27.0. This is also the cause of the bimodal E-value distribution seen in Figure 4B. Historically, a large number of Pfam families were assigned a reference GA of 25.0. More recently, we have used a higher (guidance) reference threshold of 27.0. These values correspond roughly to ‘safe’ E-value thresholds of ∼10⁻² and their increase (from GA 25.0 to GA 27.0) reflects the increase in the size of the UniProtKB database (any particular bit score value will become less significant as the database size increases). In the absence of overlaps with other families, these thresholds are often left unchanged. In retrospect, however, these choices look too conservative, since most families that do not have thresholds of 25.0 or 27.0 have a distribution of GAs that is strongly shifted toward lower bit score values (median = 21.2, 25th percentile = 20.6, 75th percentile = 22.8) (Figure 4C, right side). This seems to indicate that most reference thresholds could be lowered, thereby increasing coverage.

Figure 4D reports the distribution of E-values that correspond to family GAs for all families (left side) or excluding those families with GA 25.0 or 27.0 (right side). In the latter case the distribution has a median of 0.18, a 25th percentile of 0.057 and a 75th percentile of 0.27. A handful of families have E-values that, at first sight, appear to be either too high or too low. In particular, there are 72 families with E-value >1, and 82 with E-value <10⁻⁶. A survey of these families indicates the following. High E-values arise for two possible reasons. Firstly, the E-value may have been set high because the model is very short and a more ‘realistic’ E-value would result in no matching sequences being reported. Alternatively, the high E-value may have been chosen because that was the relevant value for the size of the sequence database when the model was first built and this family has not been revised subsequently because no overlaps have been introduced. Low E-values, which often correspond to very long profile HMMs, are likely to have been set low in order to avoid inclusion of sequences from other families (overlaps). These overlaps frequently originate from the biased distribution of amino acids in these particular profile HMMs, such that the profile HMMs are too generic and capture equally biased but unrelated sequences (one example is coiled-coil families). We will revisit some of the families that may need re-thresholding in the near future, as part of a larger scale analysis of manually set GAs, and their discriminatory power versus using a uniform fixed threshold.