Comparison methodology flowchart. 

Context in source publication

Context 1

... the last few years, sequencing technologies known as next-generation sequencing had a major impact on the availability of genomes in public data bases (Metzker, 2010). As whole genome sequencing became faster and inexpensive, new comparative analyses were possible, such as pan-genomic studies (Tettelin et al., 2008). The study of a single genome is not enough to determine the pool of genes present in bacterial species or to explain the vari- ability that determines, for instance, the pathogenicity of these bacterial species. Therefore, pan-genomic studies aim to characterize the complete genetic repertory of species through analysis of multiple strain genomes (Medini et al., 2005). The pan-genomic approach presents challenges associated with the management of the assembly and the automatic and manual annotation process of the many genome strains related to a project. To solve this issue we developed the PANNOTATOR workbench. This tool is com- posed of a relational database, interactive tools, several SQL reports, and a web-based in- terface. The workbench was initially developed as an in-house solution to manage the Co- rynebacterium pseudotuberculosis pan-genome project (Santos et al., 2012). Therefore, the relational schema was denominated the C. pseudotuberculosis Database (CpDB). Although it was initially conceived for C. pseudotuberculosis , it was used for other bacterial species as well (Carneiro et al., 2012). A parser to format entries to the PANNOTATOR workbench was also developed; it is capable of successfully interpreting genome annotations in EMBL and GenBank formats and converting these to our database format. Given a stored genome, the CpDB reports are capable of exporting files in EMBL format, an extension accepted by the Artemis program (Rutherford et al., 2000). PANNOTATOR’s main feature is its ability to produce an automatic annotation based on a manual curated genome. This workbench was conceived to reduce the workload required to generate reports and corrections of the various annotations during a pan-genome project. The idea was to transfer the annotation of gene names and functional products of a curated ge- nome, which is obtained using the alignments of protein sequence results as a linkage criteri- on. The cut-off parameters depend on how similar the protein products of the curated genome (source) are to those of the new genome (destiny), and the cut-off parameters of the quality of alignments allows the control of how much of the curated annotation will be incorporated into the new genome’s annotation. These parameters include the percentage of protein identity and the total extension of the alignment between amino acids. For instance, during the C. pseudo- tuberculosis pan-genome start (Ruiz et al., 2011), a threshold of 95% amino acid identity and sequence alignment was used, which is sufficient to correctly link most of the CDS among different strains. However, for the first automatic annotation of a C. pseudotuberculosis ge- nome, it was necessary to use the genome of C. diphtheriae (Cerdeño-Tárraga et al., 2003), the phylogenetically closest organism available, and a threshold of 65% protein identity and sequence alignment. When using a 95% threshold level, only the annotation of 4 ribosomal units was incorporated into the first C. pseudotuberculosis genome. Another useful report created by PANNOTATOR is a putative list of frame-shifts based on possible gene fragments from the destiny genome compared to source genome. The genomes compared were obtained from the NCBI website according to the follow- ing accession numbers: CP002251 ( C. pseudotuberculosis str. I19); NC_016932 ( C. pseudotu- berculosis str. 316); NC_002935 ( C. diphtheriae str. NCTC13129); NC_000913 ( Escherichia coli str. K-12 substr. MG1655); NC_010473 ( E. coli str. K-12 substr. DH10B); NC_012759 ( E. coli str. BW2952); and NC_011353 ( E. coli O157:H7 substr. EC4115). The PANNOTATOR pipeline (Figure 1) was implemented in the Ubuntu 12.04 oper- ating system. The Apache server was used to process web requests, and the web interface was developed using PHP (Hypertext Preprocessor). A number of inbuilt tools/components of the Linux operating system, such as “ sed ”, were used together with the software tools/components Bioperl for sequence file format conversions and feature extraction: BLAST and the Database Management System Postgres. PANNOTATOR mainly automates the process of annotation. The tool performs all required file conversions and modifications required by different software components. The entire process starts with 3 inputs by the user: a DNA strand, its gene prediction (destiny), and the curated genome (source). All predicted genes are compared to the ones of the genome curated. The gene name and product are assigned to a new genome based on BLAST similarity with the genes in the genome curated (Figure 1). Source and destiny genomes are evaluated using our in-house tool called parseEMBLtoCpDB. This parser comprises our annotation workbench (sourceforge.net/projects/cpdb) and is responsible for formatting data to feed the PANNOTATOR relational database schema. The destiny gene prediction is kept in a table denominated ‘gene’, which considers the locus tag and organism fields as discriminants. On the other hand, the source genome is kept in the ‘curated’ table, which is similar to the ‘gene’ table. After insertion of all annotation versions from each genome, several comparisons are performed between the source and the destiny genomes, with cut-off being those parameters selected by the user. The features considered in the analyses were named genes and products. The PANNOTATOR final result, an automated annotation of the destiny genome mediated by the source genome, only exists in the computer memory during an outer join SQL command, output written to an EMBL file. The outer join is essential in this situation because even without the existence of an acceptable similarity level between a gene from the destiny and all other genes from the source, it is still necessary to represent all the predicted genes from the destiny genome. PANNOTATOR uses the following color code for gene annotation: green for genes with a strong match (100%) to the source genome, yellow for matches between 100%, and the user specified cut-off value, or red otherwise. Genes colored red have no gene name or function linked. Furthermore, two kinds of RNA predictions (tRNA and rRNA) are automatically incorporated into the output file. Overlapping genes with RNA predictions are removed from the destiny genome, anticipating further GenBank demands in case a genome deposit process takes place. The genomes’ DNA strands of C. pseudotuberculosis str. I19 and E. coli str. K-12 substr. MG1655 were used to evaluate the tool and were submitted to the functional automated annotation tools BASys (Van Domselaar et al., 2005), RAST (Aziz et al., 2008), and PANNOTATOR (Figure 2). PANNOTATOR annotation transfer was performed using different strains and spe- cies, under different cut-off thresholds. For C. pseudotuberculosis comparison, strain 316 (Ra- mos et al., 2012), and the closely related species C. diphtheriae strain NCTC13129 (Cerdeño- Tárraga et al., 2003) were used as curated genomes; for E. coli comparison, strains BW2952, K-12 (substr. DH10B), and O157:H7 (substr. EC4115) were used as curated genomes. The main challenge to perform such comparisons resides in the fact that there is no common locus tag between a new gene prediction and the previous one present in the depos- ited version of a genome, known as the correct one or gold standard. In such situation, it is not possible to compare new features predicted (gene name and functional product) just using the locus tag. To work around such technical issue, we decided to take advantage of a relative conservation of the stop codon predictions as unique gene identifier between the genomes tested even when different gene predictors were used. Transferring annotation from close and more distant related curated genomes was used in order to evaluate the performance of the tool. The best results were achieved for C. pseudotuberculosis (Figures 3 and 4). Transferring the annotation from a different strain (316) resulted in 98% of gene names and 76% of products correctly assigned. When using the cut-off parameter of 60% similarity, considerable incorrect information was introduced in the new genome; since this value is less restrictive, it is more permissive to error introduction. Therefore, we advise the users to be careful while using a cut-off parameter below 70%. The worst results compared to RAST and BASys were obtained when using C. diph- theriae to transfer the annotation. Therefore, it is not just a matter of choosing a closely related organism for transfer, but it is also important for the annotation to be reliable (Richardson and Watson, 2013). It was suggested that the C. diphtheriae annotation was outdated, since it was deposited in 2003 (D’Afonseca et al., 2012). The only challenge in which PANNOTATOR was surpassed by another tool was regarding gene name assignment in E. coli (Figure 5). BASys had 92% of gene names correctly assigned while PANNOTATOR had, at best, 85%. Comparing the correct product assignment, PANNOTATOR outdid the other tools as observed in Figure 6. When transferring the annotation from a different species or distantly related strains, PANNOTATOR showed fairly comparable results compared to the other ...