bio-clinical-databases-gnomad-frequencies

What this skill does

## Version Compatibility

Reference examples tested with: requests 2.31+, hail 0.2.130+, pandas 2.2+, myvariant 1.0+. Current gnomAD release is **v4.1 (May 2024)**; v4.1 fixed the v4.0 AN under-counting issue that inflated rare-variant AF estimates by 5-10%.

Before using code patterns, verify installed versions match. If versions differ:
- Python: `pip show <package>` then `help(module.function)` to check signatures
- Hail: `hl.version()`; pin to >=0.2.130 for v4 schema

If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying. The gnomAD browser GraphQL API at `https://gnomad.broadinstitute.org/api` is the supported public endpoint; Hail Tables on Google Cloud Storage at `gs://gcp-public-data--gnomad/` are the supported bulk access.

# gnomAD Frequency Queries and Constraint

**'How rare is this variant in the general population?'** -> Pull allele frequency, grpmax FAF95 (the ACMG-grade frequency), LOEUF gene-level constraint, structural variant catalog, mtDNA frequencies, and the appropriate dataset version per use case.

- Python (single variant): GraphQL via `requests.post('https://gnomad.broadinstitute.org/api', json={'query': ..., 'variables': ...})`
- Python (aggregator): `myvariant.MyVariantInfo().getvariant(hgvs, fields=['gnomad_exome', 'gnomad_genome'])`
- Python (bulk): `hl.read_table('gs://gcp-public-data--gnomad/release/4.1/ht/exomes/gnomad.exomes.v4.1.sites.ht')`

## v2.1.1 / v3.1.2 / v4.x: When to Use Which

This is the most consequential decision in any gnomAD query. The releases are **not interchangeable**; choice determines what can and cannot be said about a variant.

| Release | Build | Samples | Use when | Fails when |
|---------|-------|---------|----------|-----------|
| **v2.1.1** | GRCh37 | 125,748 exomes + 15,708 genomes | Constraint metrics needed (LOEUF v2 most-validated); chrX/Y constraint required; GRCh37 native non-negotiable | GRCh38 native cohort; modern rare-variant FAF95 (use v4) |
| **v3.1.2** | GRCh38 | 76,156 genomes (NO exomes) | Non-coding region rare variants on GRCh38; mtDNA frequencies | Exome variants needed (no exomes); 76k cohort smaller than v4 |
| **v4.0/v4.1** | GRCh38 | 730,947 exomes + 76,215 genomes = **807,162 total** | Default for everything; rare-variant filtering, FAF95, gene queries | chrX/Y constraint (not released); cancer-cohort analysis (no TCGA in v4) |

**Critical caveats:**
- v4 genomes are the SAME 76,215 v3 samples reprocessed against GRCh38 with updated pipelines; not independent.
- ~81% of v2 genomes are also in v3; joint v2+v3 meta-analysis must dedupe at sample ID.
- v4 includes 416,555 UK Biobank exomes under a specific collaboration agreement; check use terms.
- **v4 does NOT include TCGA**, so the `non_cancer` subset is unnecessary; the v4 subset is `non_ukb` (excludes UKB exomes for ancestry rebalancing).
- Liftover v2 (GRCh37 -> GRCh38) is NOT equivalent to v4 native; variant representation differs at ~0.5-1% of sites due to assembly fixes.

## v4 Ancestry Groups: popmax -> grpmax Terminology

v4 ancestry groups: **AFR, AMR, ASJ, EAS, FIN, MID, NFE, SAS, AMI, REMAINING**. The **MID (Middle Eastern) group was new in v4**; previously absorbed into "OTH". The **REMAINING** group (31,256 v4 samples) is individuals who did not cluster with any reference; they contribute to overall AF but not to grpmax.

**Terminology shift:** gnomAD documentation and ACMG-facing narrative uses **grpmax** (genetic ancestry group max) -- replacing the older **popmax** ("population max") term -- to disambiguate genetic ancestry from self-reported race/ethnicity. The public GraphQL schema still exposes legacy field names containing `popmax` (e.g. `faf95.popmax`, `faf95.popmax_population`); these are the grpmax values under the modern terminology. Always check the schema version when writing queries; new browsers may rename these fields.

`grpmax_faf95` is the operational ACMG field. It computes the maximum 95% lower-CI allele frequency, **excluding bottleneck groups** (AMI, ASJ, FIN, REMAINING) because pathogenic founder variants in those groups would otherwise falsely trigger BS1/BA1. MID is included in grpmax but is the smallest non-bottleneck group with highest per-allele variance.

## Filtering Allele Frequency (FAF95): The ACMG-Grade AF

Whiffin 2017 *Genet Med* 19:1151 introduced FAF95 = Poisson lower bound of 95% CI for AF. By construction, AF > FAF95; FAF95 is the conservative frequency for ACMG application.

**Max-credible-AF formula:** `(prevalence x heterogeneity x allelic-contribution) / (penetrance x 2)`. Plug in disease parameters to get the gene-specific BA1 / BS1 threshold; compare against `grpmax_faf95`.

| Code | Threshold | Notes |
|------|-----------|-------|
| **BA1** | AF > 5% in any non-bottleneck group | ClinGen SVI default; VCEPs may override (Hearing Loss VCEP uses 0.5%) |
| **BS1** | AF > gene-specific max-credible-AF | Computed per gene via Whiffin formula |
| **PM2_Supporting** | Absent or ultra-rare in gnomAD | Downgraded from PM2_Moderate in SVI 2020 |

Use `grpmax_faf95`, not raw AF, for BS1/BA1 application; this is the ClinGen-recommended approach.

## Constraint Metrics: pLI, LOEUF, missense Z

Karczewski 2020 *Nature* 581:434 defined LOEUF as the upper bound of the 90% CI of observed/expected pLoF count per gene. LOEUF is **recommended over pLI** because it is continuous and accounts for gene size more rigorously.

| Metric | What | Interpretation |
|--------|------|----------------|
| **LOEUF** | Upper bound of 90% CI of LoF observed/expected ratio | Lower = more LoF-intolerant; **first decile (LOEUF < 0.35 v2; < 0.6 v4) = strongly intolerant** |
| **pLI** | Probability LoF intolerant | Still used; gnomAD team recommends LOEUF for ranking |
| **Missense Z** | Z-score of observed-vs-expected missense | Z > 3.09 = top 1% missense-constrained |
| **Missense O/E** | Observed/expected missense ratio | Continuous form of missense Z |

**Critical version mismatch:**
- v2.1.1 constraint metrics published 2020; v4 constraint published **March 2024** (4 months after v4 data release).
- **v4 constraint is autosomes only; chrX and chrY constraint metrics in v4 are NOT released**. For X/Y constraint, fall back to v2.1.1.
- **LOEUF first decile shifted v2 to v4**: v2 < 0.35; v4 < 0.6 (larger sample shifted the distribution). Gene rank in deciles is stable across versions but absolute thresholds are NOT interchangeable.

## Subsets: non_cancer, non_neuro, controls

| Release | Subset | Removes | Use when |
|---------|--------|---------|----------|
| v2.1.1 | `non_cancer` | TCGA | Cancer-related variant analysis (avoids circularity) |
| v2.1.1 | `non_neuro` | Psychiatric/neuro cohorts | Neuropsychiatric variant analysis |
| v2.1.1 | `controls` | Cases with known disease (~60k samples) | Disease-association calibration |
| v3.1.2 | `non_v2` | v2 overlapping samples | Independent of v2 |
| v3.1.2 | `controls_and_biobanks` | Disease cases retained, biobanks emphasized | Population-level reference |
| v4 | `non_ukb` | UK Biobank exomes | When EUR-skew of UKB problematic |
| v4 | `non_neuro` | Deprecated | -- |
| v4 | `non_cancer` | Unnecessary (no TCGA in v4) | -- |

## SV Catalog and CNV

| Resource | Release | Samples | Coverage |
|----------|---------|---------|----------|
| gnomAD-SV v2 | Collins 2020 *Nature* 581:444 | 14,891 unrelated WGS | 433k SVs, GRCh37 |
| gnomAD-SV v4 | Nov 2023 | 63,046 unrelated WGS | 1,199,117 high-confidence SVs, GRCh38 |
| gnomAD-CNV v4 | Nov 2023 | 464,297 individuals (exome-derived gCNV) | Rare (AF < 1%) autosomal coding CNVs |

gnomAD-CNV v4 is the resource that democratized exome-derived CNV background frequencies; previously only ExAC-CNV provided this at scale.

## mtDNA (Laricchia 2022 *Genome Res* 32:569)

10,850 unique mtDNA variants across 56,434 individuals (v3.1). Frequencies reported per nuclear-ancestry AND per mitochondrial-haplogroup.