bio-genome-annotation-functional-annotation

What this skill does

## Version Compatibility

Reference examples tested with: eggNOG-mapper 2.1.15 (pin for reproducibility), InterProScan 5.66+, KofamScan 1.3+, pandas 2.2+, AGAT 1.4+.

Before using code patterns, verify installed versions match. If versions differ:
- CLI: `<tool> --version` then `<tool> --help` to confirm flags
- Python: `pip show <package>` then `help(module.function)` to check signatures

Annotation content tracks **database release**: record the eggNOG DB version, InterPro/Pfam release, and KEGG/KofamScan profile date, and note whether InterProScan used the EBI precalculated lookup service. eggNOG-mapper v3 is under testing (not production) - pin v2.1.15. If code throws an error, introspect the installed tool and adapt rather than retrying.

# Functional Annotation

**"Functionally annotate my predicted proteins"** -> Transfer GO/KEGG/Pfam/EC/product labels from characterized proteins by orthology and domain signatures, attaching a confidence tier and provenance to each.
- CLI: `emapper.py -i proteins.faa --itype proteins -m diamond` (eggNOG-mapper), `interproscan.sh -i proteins.faa -f TSV,GFF3 -goterms -pa` (InterProScan)

## The Single Most Important Modern Insight -- Annotation Is a Propagated Hypothesis, Not a Measurement

Almost every label on a new genome is *transferred* by homology/orthology/ML from a small island of experimentally characterized proteins. The transfer chain is lossy and self-reinforcing - it behaves like a **percolation cascade** (Gilks 2002 *Bioinformatics* 18:1641): an over-specific name assigned in year 0, deposited with **no record that it was transferred**, becomes the nearest hit for the next genome, whose label becomes evidence for the next. By the time a query reaches NR, "number of hits agreeing" measures *how far an error spread*, not correctness. Schnoes 2009 (*PLoS Comput Biol* 5:e1000605) found misannotation reaching ~80% in bulk databases (TrEMBL/NR) and near-zero in curated Swiss-Prot - the gap *is* the curation. Three load-bearing consequences:

1. **The goal is not "maximally annotated" - it is "honestly tiered."** A genome that is 40% "hypothetical protein" with the rest correctly tiered by evidence is a better scientific object than one 95% named with half the names wrong. Prefer curated/orthology donors (Swiss-Prot, eggNOG OG consensus) over best-hits, and **demote specificity as identity/coverage fall** (full EC -> partial `1.1.1.-`; specific name -> superfamily; whole-protein -> per-domain). PI/reviewer pressure to "annotate everything" manufactures the next genome's percolating error.
2. **"Domain present" and "function known" are different claims.** A Pfam hit reports architecture, not activity - ~10% of the human kinome are catalytically dead pseudokinases that carry a confident "protein kinase" domain. Moonlighting (GAPDH), promiscuity, and mechanistically-diverse superfamilies (enolase, amidohydrolase, HAD, TIM-barrel: shared fold, divergent substrate) make this a first-order effect. **Fold conservation != function conservation** any more than sequence does - so structure-based transfer (Foldseek) inherits the same trap with *higher* false confidence.
3. **Record provenance on every label** (method, donor, donor evidence code, identity/coverage/bitscore, DB version). That is the only thing that stops the provenance-amnesia step that turns a transfer into a "fact."

## Tool Taxonomy

| Paradigm | Tool | Mechanism | Failure mode |
|----------|------|-----------|--------------|
| Orthology | eggNOG-mapper | seed-ortholog -> orthologous group -> consensus transfer | tax-scope sensitive; HGT/xenologs break the orthology assumption |
| Domain/signature | InterProScan | profile HMMs/matrices -> integrated InterPro entries | a domain implies a capability, not the substrate; broad families uninformative |
| KEGG ortholog | KofamScan | per-KO HMMs + adaptive thresholds | KO assignment, not pathway proof |
| Homology best-hit | DIAMOND vs Swiss-Prot | top-hit similarity, transfer label | best-hit != ortholog; transitive error propagation |
| ML / structure | DeepGO, DeepFRI, Foldseek | learned sequence/structure -> GO | low precision; ontology terms not products; reaches twilight zone only |

**Default workhorse pair:** eggNOG-mapper + InterProScan (orthogonal evidence: orthology vs signatures), reconciled afterward. Add KofamScan if KEGG pathway reconstruction is the goal (its adaptive per-KO thresholds are stricter than eggNOG's `KEGG_ko`). DIAMOND-vs-Swiss-Prot is the cheap product-name layer; never use it alone for GO.

## Decision Tree by Scenario

| Scenario | Recommended | Why |
|----------|-------------|-----|
| Bacterial isolate | Bakta/PGAP product names + eggNOG-mapper + InterProScan | structural pipeline first, then orthology + domains |
| Eukaryotic proteome | InterProScan (domains+GO+pathways) + eggNOG-mapper | orthogonal evidence, reconcile |
| Metagenome / MAG | eggNOG-mapper `--itype metagenome` (+ KofamScan, dbCAN) | built-in gene calling; KEGG modules |
| Twilight-zone / ORFan (no homolog) | ML (DeepGOPlus) or structure (ESMFold -> Foldseek -> DeepFRI) | only handle on the homology-free fraction; low-confidence leads |
| CAZymes / BGCs / AMR / signal peptides | -> dbCAN / antiSMASH / AMRFinderPlus / SignalP6 | a generic Pfam hit gives no substrate/phenotype/cluster |
| GO enrichment downstream | -> pathway-analysis/go-enrichment (mind IEA circularity) | enrichment on IEA partly tests the pipeline against itself |

## eggNOG-mapper

```bash
download_eggnog_data.py --data_dir db/ -y               # ~44 GB (DIAMOND DB installed by default; -D skips it)
emapper.py -i proteins.faa --itype proteins -m diamond \
    --tax_scope auto --data_dir db/ --cpu 16 -o annot --output_dir out/
```

Three stages: (1) **seed-ortholog search** (DIAMOND/MMseqs2/HMMER) anchors the query - this is a best-hit and is *not* the annotation; (2) **orthology assignment** retrieves the seed's fine-grained orthologs within the chosen taxonomic scope; (3) **functional transfer** pools terms across the *set of orthologs* (which damps single-entry misannotation - this is why eggNOG-mapper beats raw DIAMOND-vs-NR). `--tax_scope` is **the single most consequential parameter**: too broad gathers distant orthologs and over-generalizes function; `auto` lets each seed take its most-informative phylogenetic ceiling. `--itype {proteins,CDS,genome,metagenome}` (genome/metagenome runs Prodigal first). Output `.emapper.annotations` columns include `seed_ortholog`, `eggNOG_OGs`, `COG_category`, `Description`, `Preferred_name`, `GOs`, `EC`, `KEGG_ko`, `PFAMs` (read the actual header; `-` = empty).

## InterProScan

```bash
interproscan.sh -i proteins.faa -f TSV,GFF3 -goterms -pa -cpu 16
```

Runs member-database scanners (Pfam, PANTHER, NCBIfam, SUPERFAMILY, CDD, SMART, Gene3D, Hamap, PROSITE, ...) and **integrates overlapping signatures into InterPro entries** (stable IPRxxxxxx, with a type: Family/Domain/Repeat/Site/Homologous Superfamily). **Report at the InterPro-entry level** - it is the consensus that survives one member DB being wrong. `-goterms` adds the interpro2go mapping (these GO are IEA/electronic); `-pa` maps Reactome/MetaCyc. By default it queries the EBI precalculated lookup service (fast, MD5-keyed); `-dp` forces local compute (novel/confidential sequences, reproducibility). Java 11+ and a tens-of-GB data bundle required; for millions of proteins, chunk the FASTA into array jobs.

## Reconciling Multi-Tool Output with Python

**Goal:** Merge eggNOG and InterProScan per protein while preserving provenance, so a curated name is never silently overwritten by a generic domain.

**Approach:** Parse each tool's table, keep source namespaces separate, union GO with source tags, and prefer the orthology `Preferred_name`/`Description` for the human-readable product.

```python
import pandas as pd

def parse_eggnog(path):
    df = pd.read_csv(path, sep='\t', comment='#', header=None)
    cols = ['query', 'seed_ortholog', 'eval