bio-hi-c-analysis-compartment-analysis

What this skill does

## Version Compatibility

Reference examples tested with: cooler 0.10+, cooltools 0.7+, bioframe 0.7+

Before using code patterns, verify installed versions match. If versions differ:
- Python: `pip show <package>` then `help(module.function)` to check signatures

cooltools had a major API shift around 0.5 -> 0.7+ (functions standardized on `view_df`/viewframe arguments; `eigs_cis`, `expected_cis`, `saddle` signatures changed). The cooler MUST be balanced before any compartment analysis: `clr.matrix(balance=True)` requires a stored `weight` column. A `.mcool` is multi-resolution -- pass a single-resolution URI (`file.mcool::/resolutions/100000`), not the bare `.mcool`. The `phasing_track` MUST share the cooler's exact binning or phasing silently no-ops. If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.

# A/B Compartment Analysis

**"Which regions of my genome are in the active vs inactive compartment?"** -> Distance-normalize the cis matrix, take an eigenvector of its Pearson correlation matrix, then orient it by GC/gene density so positive = A (active) -- but verify the kept eigenvector is the compartment one, not an arm gradient.
- Python: `cooltools.eigs_cis(clr, gc_track, view_df=arms, n_eigs=3, sort_metric='pearsonr')`

## The Single Most Important Modern Insight -- E1 Is a Choice, Not an Output, and Its Sign Is Arbitrary Until Phased

The two most damaging beginner assumptions are "E1 = compartments" and "positive E1 = active." Both are false out of the box, and both fail *silently* -- the pipeline runs, returns a track, and is wrong.

1. **E1 is not guaranteed to be the compartment track.** On a *whole-chromosome* O/E correlation matrix the largest eigenvalue very often belongs to a smooth p-arm-vs-q-arm or centromere-to-telomere GRADIENT, not the plaid A/B checkerboard -- the real compartment signal then lands in E2 or E3. cooltools' own docs concede the first eigenvector "occasionally describes chromosomal arms or translocation blowouts." The compartment eigenvector is the one with the largest |correlation| to an activity track (GC, gene density, H3K27ac), not the one with the largest eigenvalue. The structural fix removes the gradient at the source: run `eigs_cis` per chromosome ARM (a `view_df` split at centromeres, from `bioframe.make_chromarms`), so the arm gradient is never in the within-arm matrix. Set `sort_metric='pearsonr'` so the returned eigenvectors are ordered by GC correlation, not eigenvalue -- otherwise the arm gradient is reported as "E1." A monotonic "compartment track" with no sign flips across a chromosome is the failure signature of a captured arm gradient.

2. **The sign is arbitrary until phased.** Eigenvectors are defined up to sign; the positive lobe is meaningless and can differ per chromosome AND per sample. The eigenvector MUST be oriented with an external active-chromatin track via the `phasing_track` argument so A = positive. GC content is the field default (it needs no extra assay and tracks compartment A; Lieberman-Aiden 2009 *Science* 326:289) -- compute it with `bioframe.frac_gc` at the compartment resolution, exactly matching the cooler's binning. Wrong/weak phasing flips A<->B silently, and every downstream saddle, switch call, and differential result inverts with no error. This is a classic source of irreproducible compartment papers.

3. **Compartments are an equilibrium phenomenon decoupled from TADs/loops.** Compartments = microphase separation of A/B chromatin states (cohesin-independent; survive cohesin loss, Schwarzer 2017 *Nature* 551:51, and CTCF loss, Nora 2017 *Cell* 169:930). TADs/loops = ATP-driven loop extrusion stalled at CTCF. Removing cohesin reinforces compartments while erasing TADs (Schwarzer 2017 reports reinforced compartmentalization on Nipbl loss; Rao 2017 *Cell* 171:305 eliminates all loop domains with compartments retained) -- loop extrusion actively mixes chromatin across compartment boundaries, so removing the extruder lets microphase separation run to completion (Nuebler 2018 *PNAS* 115:E6697). A preserved-or-stronger saddle after a cohesin/Nipbl/RAD21 perturbation is the EXPECTED result, not a bug; compartment-strength and TAD-strength are antagonistic. If a CTCF/cohesin perturbation makes compartments *vanish*, suspect a phasing artifact, not biology.

## Method / Output Taxonomy

| Output | Tool / call | What it is | When |
|--------|-------------|-----------|------|
| A/B eigenvector (E1) | `cooltools.eigs_cis` (cis, per-arm) | leading GC-phased eigenvector of the cis O/E correlation matrix; sign = A/B | standard A/B call, single map, per chromosome arm |
| Genome-wide A/B | `cooltools.eigs_trans` | eigenvector of inter-chromosomal blocks; immune to the cis arm-gradient | whole-genome A/B consensus with deep trans coverage |
| Compartment strength | `cooltools.saddle` + `saddle_strength` | (AA+BB)/(AB+BA) corner ratio of the saddle | comparing compartmentalization across conditions |
| 5-6 subcompartments | SNIPER (Xiong & Ma 2019 *Nat Commun* 10:5069) | autoencoder imputes inter-chr contacts -> MLP classifies A1/A2/B1/B2/B3 at 100kb | deep inter-chr data; Rao-style subcompartments |
| Continuous compartment rank | Calder (Liu 2021 *Nat Commun* 12:2439) | intra-chr divisive hierarchical clustering -> 0-1 multi-scale rank | cross-cell-line repositioning; modest coverage |
| Differential compartments | dcHiC (Chakraborty 2022 *Nat Commun* 13:6827) | quantile-normalized scores + multivariate Mahalanobis distance + significance; solves cross-sample sign flips | >=2 samples, "which bins switch A<->B" |
| Single-cell compartment | scA/B (Tan 2018 Dip-C *Science*) | CpG/activity proxy per locus; do NOT eigendecompose one sparse cell | scHi-C, ~20-50k contacts/cell |

## Decision Tree by Scenario

| Scenario | Recommended | Why |
|----------|-------------|-----|
| Matrix not yet balanced | `cooler balance` first (-> matrix-operations) | unbalanced -> O/E is all-NaN; meaningless eigenvector |
| Standard A/B call, one map | `eigs_cis` per arm at 100kb-1Mb, phase by GC | compartments are chromosome-scale; arms remove the centromere gradient |
| E1 looks monotonic / no sign flips | inspect E2/E3, pick by max |corr| with GC; or split by arm | E1 captured an arm/translocation gradient, not compartments |
| Sign of A/B seems inverted | confirm `phasing_track` is at the cooler's binning | weak/mismatched phasing flips A<->B silently |
| Want compartment STRENGTH | `saddle` + `saddle_strength`, fixed extent across samples | a single eigenvector does not quantify strength |
| Want 5-6 subcompartments | SNIPER or Calder, NOT more eigenvectors | subcompartments need inter-chr ML or hierarchical clustering, not `n_eigs` |
| Two+ conditions, compartment shift | -> hic-differential (dcHiC) | replicate-aware, sign-coherent across the cohort; hand-diffing eigenvectors flips signs |
| Single-cell Hi-C | scA/B (Dip-C), not a per-cell eigenvector | one sparse cell is too noisy to eigendecompose |
| Annotate switched bins with marks | -> chip-seq/chromatin-state-segmentation, chip-seq/peak-annotation | overlay ChromHMM/histone state on compartment calls |
| Render the eigenvector/saddle | -> hic-visualization; export bigWig -> genome-intervals/bigwig-tracks | track/heatmap conventions live there |

## Per-Arm Eigenvector with GC Phasing

**Goal:** Assign each genomic bin to the active (A) or inactive (B) compartment with a non-arbitrary sign, avoiding the centromere arm-gradient artifact.

**Approach:** Build a per-arm `view_df` (split at centromeres) so the arm gradient never enters the matrix; compute a GC-content phasing track at the cooler's exact binning; run `eigs_cis` with the GC track and `sort_metric='pearsonr'` so eigenvectors are ordered by GC correlation; then take the GC-correlated eigenvector as the compartment track.

```python
import cooler
import cooltools
import bioframe

c