Training AUGUSTUS

As a minimum for good performance you should have 200 gene structures for training. The more genes the training set comprises the better, however the incremental benefit of additional genes becomes very small at 1000 genes already. As a rule of thumb, if you have more than 1000 genes then rather prefer quality (fewer false annotations) over quantity. Make also sure that in particular the number of multi-exon genes is large. These are needed to train the introns. The gene structures of these genes should be as accurate as possible. However, it is not necessary that they are 100% correct, neither does the annotation have to be necessarily complete. It is more imporant that the start codon is correctly annotated than that the stop codon is correctly annotated.

Non-redundancy is important in particular if you plan to use part of it for training and another part for assessing prediction accuracy. If in two different sequences genes with an almost identical amino acid sequence is annotated then delete one of them. My criterion is: No two genes in the set are more than 80% identical on the amino acid level. The non-redundancy is very important for avoiding overfitting. This is of course also very important if you want to test the accuracy on a test set.

1.1 Options for compiling a set of gene structures

1.2 Split gene structure set into training and test set

Randomly split the set of annotated sequences in a training and a test set.

In rare cases it may be useful to specify a set of splice site sequences that should be used for training in addition to the complete genes. This would apply, e.g. if the number of intron examples in your training set is small but you have additional intron example, e.g. from spliced alignments of ESTs.
The format is as in the following example.

dss gccgagaactccgctcgttctgtgcgttctcctgtcccaggtagggaagaggggctgccgggcgcgctctgcgccccgtttc
dss cgtgattgtcggggggaaagacatccagggctccttgcaggtaacacatctgtttgagataacttgggttcaaggaggacat
dss agagaatcagagacagcctttcccaagagatgttggcaaggtaagtcagacaaacagcaaatgacaaaaacatgtttttatg
dss cattgtcactgttgtgtcacctgcgctgctggaccgagaggtgagctgaaaagaataccactttctttttcacgagaataga
dss tgacaaaaatgatcactcaccaaaattcaccaagaaagaggtaaacccctgtgccaaacaccaaccaccactgtggtcacag
ass gttagtatgcttctttaattttttttctccctgaaattataggaaccagatgttaaaaaattagaagaccaacttcaaggcg
ass --------------------------ggctttgtctttgcagaatttatagagcggcagcacgcaaagaacaggtattacta
ass gattccttgtgattagcctctcttgctccttttctccaccagcaaagtcgaccaagaaattatcaacattatgcaggatcgg
ass aaccgtagtaaacagcatgaatcgtgttttgtttttgaacagaccactggccttgtgggattggctgtgtgcaatactcctc

dss: donor (=5') splice site. 40 letters + gt + 40 letters
ass: donor (=3') splice site. 40 letters + ag + 40 letters
use '-' for unknown characters

2. CREATE A META PARAMETERS FILE FOR YOUR SPECIES

We call parameters like the size of the window of the splice site models and the order of the Markov model meta parameters, in contrast to parameters like the distribution of splice site patterns, the k-mer probabilities of coding and noncoding regions. There are a few dozen meta parameters but many thousands of parameters. The meta parameters determine how the parameters are calculated.

Create the files for training "bug" from a template.

contains output format options like

`protein`	turn on/off the inclusion of predicted peptide sequences in the output
`codingseq`	turn on/off the inclusion of predicted coding sequences in the output

and settings like

`alternatives-from-evidence`	turn on/of prediction of alternative transcripts based on hints
`UTR`	turn on/off the prediction of untranslated regions

All of the parameters set in this file can equally be given on the command line, e.g.

augustus --species=bug --protein=on input.fa

in which case they override the setting in this file. The parameters at the top of the file can be edited as desired or required for the species.

The optional file with the filename given by /IntronModel/splicefile may contain a list of sequence windows of known splice sites as described above.

For more info have a look at the comments in the file and at the README.TXT that is included with AUGUSTUS.

This section describes the meaning of the meta parameter /Constant/decomp_num_steps and the matrix in the file bug_weightmatrix.txt and you can savely skip this on first training.

For some species like humand and honey bee it makes sense to let the model parameters depend on the average frequencies of the 4 bases in the query sequence. For example, in human, the GC content stays consistently above or below average over long sequence stretches (isochores). AUGUSTUS can locally use different parameters that are adjusted to the base composition of this piece. I describe here only the dependency on the GC content.
/Constant/decomp_num_steps is the number of different levels of GC content that is taken into account, i.e. AUGUSTUS uses /Constant/decomp_num_steps different sets of parameters, each for a different GC content. Values between 1 and 10 are useful. The GC content ranges between /Constant/gc_range_min and /Constant/gc_range_max. These parameters can be set in the meta parameters file. Given a target GC content, each sequence in the training set is weighted depending on how similar its GC content is to the target GC content. It gets a weight that is the higher, the closer the GC content of the training sequence is to the target GC content.

The two non-zero numbers in the middle of the 4x4 Matrix in bug_weightmatrix.txt (default: 200) determine the influence of the deviation in GC content. A high value (like 300) means that mostly training sequences with a very similar GC content to the target GC content are taken into account. The advantage is that the training is more specific to the target GC content. The disadvantage of a high value is that this effectively reduces the size of the training set. A low value (like 150) means that all training sequences are taken into account, except that training sequences with a similar GC content are weighed somewhat stronger. Editing the weightmatrix file is not recommended for normal users.

3. MAKE AN INITIAL TRAINING

of the 100 genes 53 were predicted exactly
83.9% of the exons were predicted exactly
77.5% of the predicted exons were exactly as in the test set.

The fly parameters that come with augustus yield a somewhat better accuracy:

augustus --species=fly --stopCodonExcludedFromCDS=true genes.gb.test --UTR=off \
   | grep -A 22 Evaluation # takes ~1m

gives the following accuracy report:

*******      Evaluation of gene prediction     *******

---------------------------------------------\
                 | sensitivity | specificity |
---------------------------------------------|
nucleotide level |       0.987 |       0.896 |
---------------------------------------------/

----------------------------------------------------------------------------------------------------------\
           |  #pred |  #anno |      |    FP = false pos. |    FN = false neg. |             |             |
           | total/ | total/ |   TP |--------------------|--------------------| sensitivity | specificity |
           | unique | unique |      | part | ovlp | wrng | part | ovlp | wrng |             |             |
----------------------------------------------------------------------------------------------------------|
           |        |        |      |                 85 |                 45 |             |             |
exon level |    512 |    472 |  427 | ------------------ | ------------------ |       0.905 |       0.834 |
           |    512 |    472 |      |   29 |    2 |   54 |   30 |    1 |   14 |             |             |
----------------------------------------------------------------------------------------------------------/

----------------------------------------------------------------------------\
transcript | #pred | #anno |   TP |   FP |   FN | sensitivity | specificity |
----------------------------------------------------------------------------|
gene level |   105 |   100 |   67 |   38 |   33 |        0.67 |       0.638 |
----------------------------------------------------------------------------/

4. RUN THE SCRIPT optimize_augustus.pl

This script optimizes the prediction accuracy by adjusting the meta parameters in the *_parameters.cfg file. The script alternatingly used the programs augustus and etraining. This ususally increases prediction accuracy by a few percent points, but runs for hours or days. It may be skipped and only etraining be run once (step 3. above), which is very quick. augustus and etraining must be in the $PATH.

You need to tell optimize_augustus.pl, which metaparameters it should optimize. Do this by adjusting the file config/species/generic/generic_metapars.cfg. (You may also make a copy of it and then use the command line parameter --metapars=nameofmycopy to the script optimize_augustus.pl.)

Run

In an evaluation step this script does the following 10 fold cross validation. It splits the set genes.gb.train randomly into 10 sets (buckets) of ±1 equal size. Then takes 9 of the 10 sets for training and the other one for evaluating the prediction accuracy using the annotation of the genbank file. The 10 possible buckets for evaluating are rotated and each case the other sequences are used for training.

On each prediction it computes a single target value that is subject to optimization. The target is a a weighted average of the sensitivities and specificities on the base, exon and gene level. (Feel free to change the weighing to your preferences in the function 'sub gettarget'.)

For a meta parameter the script repeats above evaluation step for different values of the meta parameter. If it finds an improvement in the target value it adjusts the value of the meta parameter in your bug_parameters.cfg file. It tries new values for the parameter until it finds no more improvement. Then it optimizes the next meta parameter. When it has optimized all meta parameters once, it repeats with the first meta parameter.

It does at most 5 rounds of optimizations, but stops earlier if no improvements are found. It is possible to interrupt the script of you cannot wait for it to finish. This script probably has to run over night with nohup or screen. When it is done your bug_parameters.cfg should have better suited meta parameters for your species and your training set.

After optimize_augustus.pl has finished or (after you have interrupted it) you should (re)train AUGUSTUS with the meta parameters it has set.

If you have a test set, you can now check the prediction accuracy on this test set by running

The end of the output will then contain a summary of the accuracy of the prediction. If the gene level sensitivity is below 20% it is likely that the training set is not large enough, that it doesn't have a good quality or that the species is somehow 'special'.
If you succeeded in creating a good AUGUSTUS version for your species I would be very interested in your results. If possible please share your results by giving me the packed config/yourspecies folder.

AUGUSTUS can be told to use a different translation table, in particular one with a different set of stop codons. This is useful for a small number of species such as Tetrahymena thermophilia, in which some codons translate to a different amino acid than usual. If you train AUGUSTUS for such a species set the variable translation_table in the parameter file of your species. Further, adjust the stop codon probabilities in the same config file. E.g. say

translation_table 6
/Constant/amberprob 0 # Prob(stop codon = tag), if 0 tag is assumed to code for amino acid
/Constant/ochreprob 0 # Prob(stop codon = taa), if 0 taa is assumed to code for amino acid
/Constant/opalprob 1 # Prob(stop codon = tga), if 0 tga is assumed to code for amino acid

Choose the translation table number according to this table. translation_table=1 is the default value and the standard with stop codons taa, tga, tag. If you have a species with the standard genetic code you don't have to do anything. In case your species' code is not covered by this table send us a note with the string of 64 one-letter amino acid codes in the codon order below.

Training AUGUSTUS

1. COMPILE A SET OF TRAINING AND TEST GENES

1.1 Options for compiling a set of gene structures

1.2 Split gene structure set into training and test set

2. CREATE A META PARAMETERS FILE FOR YOUR SPECIES

3. MAKE AN INITIAL TRAINING

4. RUN THE SCRIPT `optimize_augustus.pl`

translation	a	a	a	a	a	a	a	a	a	a	a	a	a	a	a	a	c	c	c	c	c	c	c	c	c	c	c	c	c	c	c	c	g	g	g	g	g	g	g	g	g	g	g	g	g	g	g	g	t	t	t	t	t	t	t	t	t	t	t	t	t	t	t	t
table	a	a	a	a	c	c	c	c	g	g	g	g	t	t	t	t	a	a	a	a	c	c	c	c	g	g	g	g	t	t	t	t	a	a	a	a	c	c	c	c	g	g	g	g	t	t	t	t	a	a	a	a	c	c	c	c	g	g	g	g	t	t	t	t
number	a	c	g	t	a	c	g	t	a	c	g	t	a	c	g	t	a	c	g	t	a	c	g	t	a	c	g	t	a	c	g	t	a	c	g	t	a	c	g	t	a	c	g	t	a	c	g	t	a	c	g	t	a	c	g	t	a	c	g	t	a	c	g	t
1	K	N	K	N	T	T	T	T	R	S	R	S	I	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	*	Y	S	S	S	S	*	C	W	C	L	F	L	F
2	K	N	K	N	T	T	T	T	*	S	*	S	M	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	*	Y	S	S	S	S	W	C	W	C	L	F	L	F
3	K	N	K	N	T	T	T	T	R	S	R	S	M	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	T	T	T	T	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	*	Y	S	S	S	S	W	C	W	C	L	F	L	F
4	K	N	K	N	T	T	T	T	R	S	R	S	I	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	*	Y	S	S	S	S	W	C	W	C	L	F	L	F
5	K	N	K	N	T	T	T	T	S	S	S	S	M	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	*	Y	S	S	S	S	W	C	W	C	L	F	L	F
6	K	N	K	N	T	T	T	T	R	S	R	S	I	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	Q	Y	Q	Y	S	S	S	S	*	C	W	C	L	F	L	F
9	N	N	K	N	T	T	T	T	S	S	S	S	I	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	*	Y	S	S	S	S	W	C	W	C	L	F	L	F
10	K	N	K	N	T	T	T	T	R	S	R	S	I	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	*	Y	S	S	S	S	C	C	W	C	L	F	L	F
11	K	N	K	N	T	T	T	T	R	S	R	S	I	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	*	Y	S	S	S	S	*	C	W	C	L	F	L	F
12	K	N	K	N	T	T	T	T	R	S	R	S	I	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	S	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	*	Y	S	S	S	S	*	C	W	C	L	F	L	F
13	K	N	K	N	T	T	T	T	G	S	G	S	M	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	*	Y	S	S	S	S	W	C	W	C	L	F	L	F
14	N	N	K	N	T	T	T	T	S	S	S	S	I	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	Y	Y	*	Y	S	S	S	S	W	C	W	C	L	F	L	F
15	K	N	K	N	T	T	T	T	R	S	R	S	I	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	Q	Y	S	S	S	S	*	C	W	C	L	F	L	F
16	K	N	K	N	T	T	T	T	R	S	R	S	I	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	L	Y	S	S	S	S	*	C	W	C	L	F	L	F
21	N	N	K	N	T	T	T	T	S	S	S	S	M	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	*	Y	S	S	S	S	W	C	W	C	L	F	L	F
22	K	N	K	N	T	T	T	T	R	S	R	S	I	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	L	Y	*	S	S	S	*	C	W	C	L	F	L	F
23	K	N	K	N	T	T	T	T	R	S	R	S	I	I	M	I	Q	H	Q	H	P	P	P	P	R	R	R	R	L	L	L	L	E	D	E	D	A	A	A	A	G	G	G	G	V	V	V	V	*	Y	*	Y	S	S	S	S	*	C	W	C	*	F	L	F

Training AUGUSTUS

1. COMPILE A SET OF TRAINING AND TEST GENES

1.1 Options for compiling a set of gene structures

1.2 Split gene structure set into training and test set

2. CREATE A META PARAMETERS FILE FOR YOUR SPECIES

3. MAKE AN INITIAL TRAINING

4. RUN THE SCRIPT optimize_augustus.pl

4. RUN THE SCRIPT `optimize_augustus.pl`