Validation of a New Web Application for Identification of Fungi by Use of Matrix-Assisted Laser Desorption Ionization–Time of Flight Mass Spectrometry

Experiment design.

We first tested a previously published panel of 422 strains (panel 1 [Table 1]) against the online algorithm and subsequently established identification criteria and threshold, since the online software, referred to as MSI (Mass Spectrometry Identification platform; https://biological-mass-spectrometry-identification.com/msi/), has never been assessed or published. The in-house database and the MSI software were then assessed by testing 2,004 spectra derived from 501 isolates prospectively collected in five hospitals in France and Belgium (panel 2). Panel 2 was also tested in parallel against the Bruker database and the IHEM/Marseille database using the MALDI Biotyper software from Bruker.

Threshold establishment.

Data extracted from the 1,688 spectra of the panel 1 strains are plotted in Fig. 1. The highest scores corresponding to false genus identifications peaked around 10, and very few false genus results exceeded 20, whereas most results corresponding to correct species identification surpassed this threshold. In contrast, it was markedly more difficult to define a satisfying cutoff that differentiated between accurate species identification and accurate genus but false species identification (Fig. 1).

Among the 1,688 spectra, only 47 corresponded to a false identification as an initial result. The scores corresponding to these 47 spectra ranged from 2.31 to 16.05 (mean = 10.26; standard deviation = 3.04), which was far below the selected threshold.

Description of the species identified in panel 2.

A total of 1,270 isolates were collected in five hospitals during the study period. However, only 501 isolates were identified via DNA sequencing, whereas the other isolates, which corresponded to common and easy-to-identify species, were identified using only morphological criteria and were thus not included in the study. Among the 501 isolates that were used to assess the in-house database and the online identification system, a total of 88 microbial taxa were identified, with a repartition from 9 to 43 taxa per hospital (Table 2). The details of the DNA identification results for these 501 isolates and the targets used are indicated in Table 2.

MALDI-TOF-based identification assessment of panel 2.

The 501 isolates included in panel 2 were identified using the MSI software with the defined threshold of 20. In parallel, the same 501 isolates were also identified using the Bruker-MALDI Biotyper and the IHEM/Marseille-MALDI Biotyper combinations. Two thresholds (1.7 as proposed by Normand et al. [18] and 2.0 as proposed by the supplier) were taken into account using the MALDI Biotyper software. For each isolate, four replicates were deposited, but only the deposit with the best identification score was taken into account, as previously proposed (18). The identifications obtained using the various algorithms were then classified into four categories (Table 3) according to the score and identification result. Regardless of the database/software combination, no false identifications were obtained. However, the percentage of isolate spectra that met the identification threshold varied greatly between the five identification systems (Table 3).

Panel 2 identification results per center.

The percentage of spectra that met the identification criteria per center and the percentage of correct identifications at the species and genus level are listed in Table 4.

Discrepancies between DNA identification and MSI results.

Twenty-six strains were incorrectly identified at the species level, although they reached the threshold for the MSI platform. The results obtained with these 26 isolates, the obtained scores, and the percentage of homology of corresponding DNA sequences are listed in Table 5. Most of these errors (20/26) corresponded to a misidentification among closely related species of the same species complex.

Time required to perform online application and comparison with the Bruker software.

Identification of a full plate of 96 spectra took 8 min (5 s per spectrum) using the MALDI Biotyper software with the Bruker database (604 references) and 43 min (27 s per spectrum) with the IHEM/Marseille database (5,676 references). The MSI software performed 96 identifications in 14.50 s (0.14 s per spectrum).

Online application compatibility.

The online application was tested on Windows 7 and Windows 10. Four Web browsers were successfully tested and yielded the same identification results. The tested versions included Mozilla Firefox v35.0 and v43.0, Internet Explorer v9.0.8112.16421, Google Chrome v42.0.2311.90m, and Safari v10.0 (10602.1.50.0.10).

Panel description.

Panel 1 comprised 422 strains selected from both the BCCM/IHEM collection and the Clinical Mycology Laboratory of Marseille Hospital. This panel has already been used to assess the identification algorithm using the MALDI Biotyper software; the description of this panel and the results of the assessment are published elsewhere (18). The details of panel 1 are found in Table 1. This panel constitutes a diverse group of mold species and genera (126 species, 38 genera), including 15 strains (13 species) for which the species was not represented in the reference spectrum database.

Panel 2 consisted of 501 isolates collected in five hospitals in France and Belgium. The isolates were collected at different time periods. Only a portion of the isolates collected in these five hospitals—those that had been sequenced using appropriate DNA targets—were included in panel 2 (501/1,269, 39.4%), while the remaining 768 isolates, which corresponded to easy-to-identify species, were identified exclusively based on phenotypic criteria. The species that were not represented in the database were intentionally retained in the panel as negative controls. Details concerning the isolates collected at each institution are provided in Table 1.

Fungal cultures.

Prior to identification, the five laboratories used different culture media: Sabouraud chloramphenicol gentamicin (Becton Dickinson [BD]) on Oxoid petri dishes or culture tubes, Sabouraud Actidione (Bio-Rad), CHROMagar Candida (BD), Sabouraud dextrose chloramphenicol (BD/Biokar), in-house made Lactrimel agar medium, Difco Sabouraud dextrose agar (BD) with 0.5 mg/liter chloramphenicol (Certa), and Sabouraud chloramphenicol 2 agar medium (bioMérieux). The laboratories also used different culture procedures, with culture temperatures ranging from 25 to 37°C and incubation times varying from 24 h to 28 days. Some strains were also received from other laboratories (mostly from bacteriology laboratories) for identification after growth on other culture media such as Can2, blood agar, chocolate agar, and Cepacia medium from various suppliers.

DNA sequencing.

All 923 strains (422 in panel 1 and 501 in panel 2) were identified via DNA sequencing in either Marseille or Brussels. The rRNA ITS2 region (primer sequences, ITS3 [GCA TCG ATG AAG AAC GCA GC] and ITS4c [TCC TCC GCT TAT TGA TAT GC]) was sequenced for all strains. When necessary, a second locus was sequenced, especially for certain relevant taxa: the partial beta-tubulin gene (primer sequences, Bt2A [GGT AAC CAA ATC GGT GCT GCT TTC] and Bt2B [ACC CTC AGT GTA GTG ACC CTT GGC]) was sequenced for Aspergillus, Penicillium, and Scedosporium species, while translation elongation factor 1-alpha (primer sequences, EF1 [ATG GGT AAG GAR GAC AAG AC] and EF2 [GGA RGT ACC AGT SAT CAT GTT]) was sequenced for Fusarium species. In all cases, identification was validated only when the obtained sequence was longer than 350 bp, with at least 99% homology using the NCBI Basic Local Alignment Search Tool (http://blast.ncbi.nlm.nih.gov/Blast.cgi) or the CBS nucleotide database (http://www.cbs.knaw.nl/Collections/BioloMICSSequences.aspx). For cases in which the DNA identification criteria mentioned above were not reached, the identification result was conserved only at the genus level. The details of the sequenced strains are listed in Table 2.

Mass spectrometry software.

Two software applications were used for mass spectrum identification: the MALDI Biotyper 3 by Bruker, with two thresholds (1.7 and 2.0) tested each time, and the MSI online application. Using the MSI software, identification of a given clinical sample was performed by successively aligning the spectrum with each reference spectrum and accounting for the common peaks between the two spectra. A maximum score of 100 was assigned when all peaks of the clinical spectrum were present at the same mass point (m/z) on the reference spectrum to which it was compared or when all peaks of the reference spectrum were present at the same exact mass point on the clinical spectrum. The score decreased when there was a shift, even a small one, between two peaks or when a peak found on one spectrum was lacking on the other. The MSI software not only provides the species corresponding to the reference spectrum with the highest score but also provides the best scores obtained with reference spectra corresponding to closely related species and to other species. The MSI software also provides a list of the 100 best recognized references, thus allowing the user to assess result consistency. Therefore, even with a score above 20, the user will determine whether high scores are also obtained with other species. The user also notice if the result is supported by only one or two references that are challenged by references corresponding to other species.

MS reference databases.

We used two different databases to perform the comparisons: the Bruker reference database and an in-house reference database. The Bruker database comprised 604 references of yeast and fungus (332 official references in the Bruker Taxonomy-Eukaryota and 272 unofficial references in the _DB_Content_FilFungi_111123 appendix). The in-house reference database was generated in coordination with the BCCM/IHEM culture collection and the Mycology Laboratory of Marseille Hospital using the protocol described by Normand et al. (5). The database used for this study comprised in-house references corresponding to 1,913 strains representing 938 species and 246 genera. The details concerning metaspectra construction are described elsewhere (5). Overall, a total of 5,676 references were used for identification purposes with the MALDI Biotyper. The same 1,913 strains were used to construct the MSI online reference database. No metaspectra were generated; instead, regardless of the fungus family, two spectra per culture/subculture were taken into account. Thus, a total of 11,851 spectra were included in the MSI reference database.

MS sample preparation.

As described in previous studies, all strains were subjected to protein extraction, and the extracts were deposited in quadruplicate for identification purposes (3, 9). The isolates were treated using a protocol described by Cassagne et al., which includes a formic acid and acetonitrile protein extraction step. Four aliquots of 1 μl of protein extract per isolate were deposited, and the samples were then covered with 1 μl of HCCA (α-cyano-4-hydroxycinnamic acid) matrix. MS acquisition was performed in the respective participating centers using the Bruker Microflex system.