bio-clinical-databases-myvariant-queries

What this skill does

## Version Compatibility

Reference examples tested with: myvariant 1.0.0+, requests 2.31+, pandas 2.2+. myvariant.info aggregates >=21 sources; the operative version of each source is queryable via the `_meta` field and the `/v1/metadata` endpoint.

Before using code patterns, verify installed versions match. If versions differ:
- Python: `pip show <package>` then `help(module.function)` to check signatures

If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying. dbNSFP version drift is the dominant staleness vector: AlphaMissense was added to dbNSFP v4.4 (~2024); querying `dbnsfp.alphamissense.score` returns whatever version of dbNSFP is currently loaded; check `_meta.src.dbnsfp.version`.

# MyVariant.info Queries; Aggregated Annotation

**'Annotate my variants with ClinVar + gnomAD + CADD + AlphaMissense in one batch'** -> Query the BioThings myvariant.info aggregator with field selection and version tracking, then parse nested responses.

- Python: `myvariant.MyVariantInfo().getvariant(hgvs_or_rsid, fields=['clinvar', 'gnomad_exome', 'dbnsfp'])`
- Python (batch): `mv.getvariants(ids_list, fields=...)`; up to 1000 IDs per request
- Python (search): `mv.query('clinvar.gene.symbol:BRCA1 AND clinvar.clinical_significance:Pathogenic')`
- REST: `GET https://myvariant.info/v1/variant/{hgvs_or_id}?fields=...`
- Bulk: `POST https://myvariant.info/v1/variant` with comma-separated IDs

## BioThings Architecture (Wu 2022 *Bioinformatics*)

myvariant.info is one of three flagship BioThings APIs (with MyGene.info and MyChem.info). All three share the BioThings SDK, which auto-deploys an Elasticsearch index from heterogeneous source files via per-source dataloaders. The 2022 paper formalized the SDK; the architecture itself is older (Xin 2016 *Genome Biol*).

- Elasticsearch-backed: queries use Lucene operators (AND, OR, NOT, range like `dbnsfp.cadd.phred:>20`)
- Dotted-field-name syntax for nested JSON
- The `_id` field is canonical HGVS-g per record (e.g., `chr7:g.117199644G>A`)

## Aggregated Sources: ~21 and Counting

| Source | What | Notes |
|--------|------|-------|
| ClinVar | Pathogenicity | Weekly refresh |
| gnomAD v4 exomes + genomes | Population AF | grpmax_faf95 surfaced |
| dbSNP Build 156 | rsID + alleles | RsMergeArch resolved |
| dbNSFP v4.x | Meta-aggregator of 40+ in silico predictors | Includes AlphaMissense, REVEL, BayesDel |
| CADD | Deleteriousness | Genome-wide |
| CIViC | Cancer interpretation | Per-disease |
| COSMIC | Somatic variants | Catalogue of Somatic Mutations |
| EVS | Exome Variant Server | Legacy (deprecated by gnomAD) |
| ExAC | ExAC frequencies | Legacy (superseded by gnomAD) |
| GRASP | GWAS associations | -- |
| GWAS Catalog | Curated GWAS | -- |
| Wellderly | Disease-resistant elderly cohort | -- |
| EMV | -- | -- |
| DOCM | Database of Curated Mutations | -- |
| ICGC | International cancer | -- |
| MutDB | -- | -- |
| GO | Gene Ontology | -- |
| Snpeff | snpEff annotations | -- |
| GeneReviews | Disease/gene reviews | -- |
| MutPred | Functional impact | -- |

**dbNSFP is itself an aggregator.** Querying `dbnsfp.alphamissense.score` returns the version that dbNSFP loaded, not AlphaMissense direct. The lag from publication (Cheng 2023 *Science*) to integration into myvariant.info is typically 6-18 months via dbNSFP.

## Scopes and Query Forms

| Endpoint | Method | Use |
|----------|--------|-----|
| `/v1/variant/{id}` | GET | Single canonical-ID lookup |
| `/v1/variant` | POST (batched IDs) | Batch lookup, up to 1000 IDs |
| `/v1/query?q={lucene}` | GET | Flexible Elasticsearch search |
| `/v1/metadata` | GET | Per-source versions |

**Scopes** (the `scopes` parameter on `/v1/query` POST) specifies which fields to match an input ID against: `hgvs`, `rsid`, `dbsnp.rsid`, `dbnsfp.genename`, `chrom`, `_id`. The `_id` is canonical HGVS-g.

## Reproducibility: The `_meta` Field

Every record carries `_meta.src` showing per-source version:

```python
mv = myvariant.MyVariantInfo()
record = mv.getvariant('chr7:g.140453136A>T', fields=['_meta', 'clinvar', 'dbnsfp.alphamissense'])
print(record['_meta']['src']['dbnsfp']['version'])  # e.g., '4.7a'
print(record['_meta']['src']['clinvar']['version'])  # e.g., '20250901'
```

For reproducibility, record per-source versions in analysis output alongside results.

## Comparison to Alternatives

| Tool | Approach | When to use |
|------|----------|-------------|
| **myvariant.info** | Cloud aggregator, ES-backed | Quick batch annotation, no local setup |
| **OpenCRAVAT** (Pagel 2020 *Cancer Res*) | Local install, modular annotators | Offline / PHI-sensitive |
| **VarSome** (Kopanos 2019 *Bioinformatics* 35:1978; commercial) | Hosted, 22 sources | 82% ACMG criteria auto-application (highest); clinical labs |
| **Franklin / Genoox** | Commercial hosted | 59 data sources; family/cohort analysis |
| **GeneBe.net** (Stawinski 2024 *Clin Genet*) | Open-source web + API | Free Tavtigian-point-system-based ACMG; comparable to VarSome |
| **ANNOVAR / VEP / snpEff** | Local annotation tools | Pipeline integration, batch annotation, no ACMG |

**myvariant.info does NOT produce ACMG calls**; it is purely an annotation aggregator. Pair with InterVar, GeneBe, or the `acmg-classification` skill for classification.

## Decision Tree by Query Scenario

| Scenario | Recommended path | Why |
|----------|------------------|-----|
| Single variant batch annotation | `getvariant(hgvs, fields=...)` | One call, all aggregated sources |
| 10-1000 variants | `getvariants(list, fields=...)` | Batch endpoint, up to 1000 |
| > 1000 variants | Chunk to 1000 + sleep | Rate limit + JSON size |
| Search by gene + pathogenicity | `mv.query('clinvar.gene.symbol:BRCA1 AND clinvar.clinical_significance:Pathogenic', size=200)` | Elasticsearch Lucene |
| ACMG-grade pipeline | myvariant for annotation -> InterVar / GeneBe for classification | myvariant does not produce ACMG calls |
| Offline / PHI-sensitive | OpenCRAVAT or VEP locally | myvariant requires HTTP |
| Reproducibility | Always record `_meta.src.<source>.version` | dbNSFP version is the dominant staleness vector |
| Source-specific deep dive | Use the source-specific skill (clinvar-lookup, gnomad-frequencies) | myvariant is aggregator-grade, not source-deep |

## Standard Annotation Workflow

**Goal:** Annotate a list of variants with the canonical clinical fields for downstream prioritization.

**Approach:** Batch `getvariants` with explicit field list; record `_meta` versions; convert to DataFrame.

```python
import myvariant
import pandas as pd

mv = myvariant.MyVariantInfo()

CLINICAL_FIELDS = [
    'clinvar.clinical_significance',
    'clinvar.review_status',
    'clinvar.variant_id',
    'gnomad_exome.faf95',
    'gnomad_exome.af.af',
    'gnomad_exome.an.an',
    'gnomad_genome.faf95',
    'gnomad_genome.af.af',
    'dbsnp.rsid',
    'dbnsfp.alphamissense.score',
    'dbnsfp.alphamissense.pred',
    'dbnsfp.revel.score',
    'dbnsfp.cadd.phred',
    'dbnsfp.spliceai.master_pred',
    'dbnsfp.spliceai.ds_max',
    'cosmic.cosmic_id',
    'civic.openCravatUrl',
    '_meta'
]

def annotate_variant_list(hgvs_list):
    '''Batch-annotate variants with ClinVar / gnomAD / dbNSFP / COSMIC / CIViC fields.'''
    chunked = [hgvs_list[i:i+1000] for i in range(0, len(hgvs_list), 1000)]
    rows = []
    versions = None
    for chunk in chunked:
        results = mv.getvariants(chunk, fields=CLINICAL_FIELDS)
        for r in results:
            if versions is None and r.get('_meta'):
                versions = {src: meta.get('version') for src, meta in r['_meta'].get('src', {}).items()}
            clinvar = r.get('clinvar', {}) or {}
            gnomad_e = r.get('gnomad_exome', {}) or {}
            gnomad_g = r.get('gnomad_genome', {}) or {}
            dbnsfp = r.get('dbnsfp', {}) or {}
            faf95 = (gnomad_e.get('faf95', {}) or gnomad_g.get('faf95', {})