nemo-mbridge-recipe-recommender

What this skill does

# Auto Recipe — Recipe Index & Recommendation

This skill indexes every shipped recipe and helps users pick the right starting
config, adjust parallelism, and avoid common pitfalls.

## How to Use This Skill

1. Ask the user for: **model name/size**, **GPU count & type**, **training goal**
   (pretrain / SFT / PEFT), and **sequence length** (if non-default).
2. Look up the best-match recipe in the index below.
3. Recommend the recipe function name + entry-point command.
4. Provide adjustment advice (parallelism resizing, batch tuning, pitfalls).

## First Answer Checklist

When recommending recipes, always include these distinctions before the long
index details:

1. **Library recipes** under `src/megatron/bridge/recipes/` are for functional
   training and use `scripts/training/run_recipe.py`.
2. **Performance recipes** under `scripts/performance/` are for upper-bound
   throughput benchmarks. They use mock data and should not be presented as
   production training recipes.
3. For a first-time Bridge smoke test, recommend `llama3_8b_sft_config` with
   mock data via `--dataset llm-pretrain-mock`. Do not use `llm-finetune` for
   the setup-only tryout unless the user specifically asks for an SFT data path.
4. For normal SFT recommendations, use `--dataset llm-finetune`; for pretrain
   and mock validation recommendations, use `--dataset llm-pretrain-mock`.
5. After the recipe and dataset, give the required resizing rules: TP must
   divide `num_key_value_heads`, keep TP within one node unless using
   NVL72-class interconnect, enable SP when TP > 1, configure CP for long
   context, DP is implicit, and reduce `micro_batch_size` first on OOM.

---

## Entry Points

### Library recipes (functional training)

```bash
# Pretrain with mock data
uv run python -m torch.distributed.run --nproc_per_node=8 scripts/training/run_recipe.py \
    --recipe <recipe_function_name> \
    --dataset llm-pretrain-mock

# SFT with SQuAD
uv run python -m torch.distributed.run --nproc_per_node=8 scripts/training/run_recipe.py \
    --recipe <recipe_function_name> \
    --dataset llm-finetune

# Override any field via CLI
uv run python -m torch.distributed.run --nproc_per_node=8 scripts/training/run_recipe.py \
    --recipe llama3_8b_pretrain_config \
    --dataset llm-pretrain-mock \
    'model.tensor_model_parallel_size=2' \
    'training.global_batch_size=64'
```

### Performance recipes (throughput benchmarks)

```bash
python scripts/performance/run_script.py \
    --recipe <model_family> \
    --gpu_type h100 \
    --num_gpus 64 \
    --data mock
```

See the Performance Recipe Index for important caveats before using these for anything beyond throughput benchmarking.

---

## Recipe Unification (Coming Soon — PR #2803)

PR [#2803](https://github.com/NVIDIA-NeMo/Megatron-Bridge/pull/2803) is
unifying performance recipes into the same **Python function** format used by
library recipes. Key changes:

- Perf recipes move from `scripts/performance/configs/` → `src/megatron/bridge/recipes/<family>/<model>_perf.py`
- Each perf recipe becomes a **self-contained Python function** (e.g. `llama3_8b_h100_bf16_pretrain_config()`)
- The old `WorkloadBaseConfig` → `set_workload_base_configs` → `get_perf_optimized_recipe` pipeline is removed
- Shared helpers: `_benchmark_common()` (50 iters, timing, TE RNG), `_perf_precision()` (bf16 / fp8_cs / fp8_mx / nvfp4)

**Why Python, not YAML?** Previous YAML-based approaches had problems:
recipe logic was split across multiple indirection layers, configs were not
self-contained, and the two-level pipeline made maintenance and debugging
difficult. Python functions are explicit, greppable, and composable.

After #2803 lands, both library and perf recipes will be invocable through the
same `run_recipe.py` entry point.

---

## Library Recipe Index

All recipes live under `src/megatron/bridge/recipes/`. Each function returns a
`ConfigContainer` with model, training, optimizer, and data settings.

### Llama

| Recipe | Mode | TP | PP | CP | SP | GPUs (min) | Seq Len |
|--------|------|----|----|----|----|------------|---------|
| `llama2_7b_pretrain_config` | Pretrain | 2 | 1 | — | — | 2 | 4K |
| `llama3_8b_pretrain_config` | Pretrain | 2 | 1 | — | ✓ | 2 | 8K |
| `llama3_8b_16k_pretrain_config` | Pretrain | 2 | 1 | 2 | ✓ | 4 | 16K |
| `llama3_8b_64k_pretrain_config` | Pretrain | 2 | 1 | 4 | ✓ | 8 | 64K |
| `llama3_8b_128k_pretrain_config` | Pretrain | 2 | 1 | 8 | ✓ | 16 | 128K |
| `llama3_70b_pretrain_config` | Pretrain | 8 | 4 | — | ✓ | 32 | 8K |
| `llama3_70b_16k_pretrain_config` | Pretrain | 8 | 4 | 2 | ✓ | 64 | 16K |
| `llama3_70b_64k_pretrain_config` | Pretrain | 8 | 4 | 4 | ✓ | 128 | 64K |
| `llama31_405b_pretrain_config` | Pretrain | 8 | 16 | — | ✓ | 128 | 8K |
| `llama3_8b_sft_config` | SFT | 2 | 1 | — | ✓ | 2 | 8K |
| `llama3_70b_sft_config` | SFT | 4 | 4 | — | ✓ | 16 | 8K |
| `llama31_405b_sft_config` | SFT | 8 | 8 | — | ✓ | 64 | 8K |
| `llama3_8b_peft_config` | PEFT | 1 | 1 | — | — | 1 | 8K |
| `llama3_70b_peft_config` | PEFT | 2 | 4 | — | ✓ | 8 | 8K |
| `llama31_405b_peft_config` | PEFT | 4 | 8 | — | ✓ | 32 | 8K |

### Qwen2 / Qwen2.5

| Recipe | Mode | TP | PP | Sizes |
|--------|------|----|----|-------|
| `qwen2_*_{pretrain,sft,peft}_config` | All | 1–8 | 1–4 | 500M, 1.5B, 7B, 14B, 32B, 72B |
| `qwen25_*_{pretrain,sft,peft}_config` | All | 1–8 | 1–4 | 500M, 1.5B, 3B, 7B, 14B, 32B, 72B |

### Qwen3 (Dense)

| Recipe | Mode | TP | PP | CP | Sizes |
|--------|------|----|----|-----|-------|
| `qwen3_*_pretrain_config` | Pretrain | 1–8 | 1–2 | — | 600M–32B |
| `qwen3_*_sft_config` | SFT | 1–8 | 1–2 | — | 600M–32B |
| `qwen3_600m_sft_128k_config` | SFT | 1 | 1 | 8 | 600M (128K seq) |
| `qwen3_*_peft_config` | PEFT | 1 | 1 | — | 600M–32B |

### Qwen3 MoE

| Recipe | Mode | TP | PP | EP | CP | GPUs |
|--------|------|----|----|----|----|------|
| `qwen3_30b_a3b_pretrain_config` | Pretrain | 1 | 1 | 8 | — | 8 |
| `qwen3_30b_a3b_sft_config` | SFT | 1 | 1 | 8 | — | 8 |
| `qwen3_30b_a3b_peft_config` | PEFT | 1 | 1 | 1 | — | 1 |
| `qwen3_235b_a22b_pretrain_config` | Pretrain | 4 | 16 | 8 | 2 | 512+ |
| `qwen3_235b_a22b_sft_config` | SFT | 4 | 8 | 8 | — | 256 |
| `qwen3_235b_a22b_peft_config` | PEFT | 1 | 4 | 4 | — | 16 |

### Qwen3-Next

| Recipe | Mode | TP | PP | EP |
|--------|------|----|----|-----|
| `qwen3_next_80b_a3b_pretrain_config` | Pretrain | 1 | 4 | 8 |
| `qwen3_next_80b_a3b_sft_config` | SFT | 1 | 2 | 8 |
| `qwen3_next_80b_a3b_peft_config` | PEFT | 1 | 1 | 4 |

### DeepSeek

| Recipe | Mode | TP | PP | EP | GPUs |
|--------|------|----|----|-----|------|
| `deepseek_v2_lite_pretrain_config` | Pretrain | 1 | 1 | 8 | 8 |
| `deepseek_v2_pretrain_config` | Pretrain | 1 | 4 | 32 | 128 |
| `deepseek_v3_pretrain_config` | Pretrain | 2 | 16 | 64 | 2048 |
| `deepseek_v3_pretrain_config_32nodes` | Pretrain | 2 | 8 | 32 | 256 |

### GLM-4.5

| Recipe | Mode | TP | PP | EP | GPUs |
|--------|------|----|----|-----|------|
| `glm45_355b_pretrain_config` | Pretrain | 2 | 8 | 16 | 256 |
| `glm45_air_106b_pretrain_config` | Pretrain | 1 | 4 | 8 | 32 |
| `glm45_355b_sft_config` | SFT | 2 | 8 | 16 | 256 |
| `glm45_air_106b_sft_config` | SFT | 1 | 4 | 8 | 32 |
| `glm45_355b_peft_config` | PEFT | 2 | 4 | 4 | 32 |
| `glm45_air_106b_peft_config` | PEFT | 1 | 2 | 4 | 8 |

### Gemma

| Recipe | Mode | TP | PP | Sizes |
|--------|------|----|----|-------|
| `gemma2_*_{pretrain,sft,peft}_config` | All | 2–8 | 1–2 | 2B, 9B, 27B |
| `gemma3_1b_{pretrain,sft,peft}_config` | All | 1 | 1 | 1B (32K seq) |

### NemotronH / Nemotron

| Recipe | Mode | TP | PP | EP | Notes |
|--------|------|----|----|-----|-------|
| `nemotronh_{4b,8b,47b,56b}_*_config` | P/S/PEFT | 1–8 | 1–4 | — | Dense SSM-hybrid |
| `nemotron_3_nano_*_config` | P/S/PEFT | varies | 1 | 8 | MoE + Mamba |
| `nemotron_3_super_*_config` | P/S/PEFT | 4 | 1 | 8 | MoE + Mamba, ~40% CUDA graph gain |
| `nemotron_nano_{9b,12b}_v2_*_config` | P/S/PEFT | varies | 1 | — | Dense |

### Other Models

| Rec