ai-engineering-from-scratch

What this skill does

```markdown
---
name: ai-engineering-from-scratch
description: Comprehensive AI engineering curriculum with 230+ hands-on lessons covering math, ML, DL, NLP, vision, transformers, LLMs, agents, and swarms across Python, TypeScript, Rust, and Julia.
triggers:
  - "help me follow the ai engineering from scratch course"
  - "set up the ai engineering curriculum project"
  - "work through a lesson in rohitg00 ai engineering"
  - "build a neural network from scratch following this course"
  - "implement an agent using the ai engineering course structure"
  - "run the jupyter notebooks for this ai course"
  - "add a new lesson to the ai engineering from scratch repo"
  - "explain the phase structure of this ai course"
---

# AI Engineering from Scratch

> Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection.

A structured, hands-on AI engineering curriculum with 230+ lessons across 20 phases. Covers everything from linear algebra to autonomous agent swarms. Every lesson produces reusable artifacts: prompts, skills, agents, MCP servers. Supports Python, TypeScript, Rust, and Julia.

---

## What This Project Does

- **20 phases** progressing from setup/tooling → math → classical ML → deep learning → NLP → vision → speech → transformers → LLMs → agents → swarms
- **230+ runnable lessons** — each in its own directory with code, notebooks, and docs
- **Multi-language** — Python (primary), TypeScript, Rust, Julia
- Each lesson outputs something reusable (a tool, prompt, agent, MCP server, or notebook)
- Designed for AI coding agents and humans to learn, build, and ship together

---

## Repository Structure

```
ai-engineering-from-scratch/
├── phases/
│   ├── 00-setup-and-tooling/
│   │   ├── 01-dev-environment/
│   │   ├── 02-git-and-collaboration/
│   │   └── ...
│   ├── 01-math-foundations/
│   ├── 02-ml-fundamentals/
│   ├── 03-deep-learning-core/
│   ├── 04-computer-vision/
│   ├── 05-nlp-foundations-to-advanced/
│   ├── 06-speech-and-audio/
│   └── ... (phases 07–19)
├── assets/
├── glossary/
│   └── terms.md
├── ROADMAP.md
├── CONTRIBUTING.md
└── README.md
```

Each lesson directory typically contains:
```
phases/NN-phase-name/NN-lesson-name/
├── README.md          # lesson explanation and goals
├── solution.py        # reference implementation
├── notebook.ipynb     # interactive walkthrough
├── requirements.txt   # lesson-specific dependencies
└── tests/             # validation tests
```

---

## Installation & Environment Setup

### Prerequisites

```bash
# Python 3.10+
python --version

# Node.js 18+ (for TypeScript lessons)
node --version

# Rust (for Rust lessons)
rustup --version

# Julia (for math lessons)
julia --version
```

### Clone and Bootstrap

```bash
git clone https://github.com/rohitg00/ai-engineering-from-scratch.git
cd ai-engineering-from-scratch
```

### Python Environment (recommended: per-phase venv)

```bash
# Create a base environment
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# Or use conda
conda create -n ai-eng python=3.11
conda activate ai-eng

# Install base dependencies
pip install -r requirements.txt  # if present at root

# Install lesson-specific deps
pip install -r phases/01-math-foundations/01-linear-algebra-intuition/requirements.txt
```

### Common Python Dependencies Across Lessons

```bash
pip install numpy scipy matplotlib pandas scikit-learn
pip install torch torchvision torchaudio          # deep learning
pip install transformers datasets tokenizers       # HuggingFace
pip install openai anthropic                       # LLM APIs
pip install jupyter notebook ipykernel             # notebooks
pip install pytest black ruff                      # dev tools
```

### API Keys (set as environment variables — never hardcode)

```bash
export OPENAI_API_KEY="your-key-here"
export ANTHROPIC_API_KEY="your-key-here"
export HUGGINGFACE_TOKEN="your-token-here"
export GOOGLE_API_KEY="your-key-here"
```

Or use a `.env` file with `python-dotenv`:

```python
from dotenv import load_dotenv
import os

load_dotenv()
api_key = os.getenv("OPENAI_API_KEY")
```

---

## Running Lessons

### Jupyter Notebooks

```bash
# Start Jupyter for a specific phase
cd phases/01-math-foundations/01-linear-algebra-intuition/
jupyter notebook notebook.ipynb

# Or JupyterLab
jupyter lab

# Run all notebooks in a phase non-interactively
jupyter nbconvert --to notebook --execute phases/01-math-foundations/*/notebook.ipynb
```

### Python Scripts

```bash
# Run a specific lesson's solution
python phases/02-ml-fundamentals/02-linear-regression/solution.py

# Run with arguments (common pattern)
python phases/03-deep-learning-core/01-the-perceptron/solution.py --epochs 100 --lr 0.01
```

### Tests

```bash
# Test a specific lesson
pytest phases/02-ml-fundamentals/02-linear-regression/tests/

# Test an entire phase
pytest phases/02-ml-fundamentals/

# All tests
pytest phases/
```

---

## Real Code Examples

### Phase 1 — Linear Algebra (Math Foundations)

```python
# phases/01-math-foundations/02-vectors-matrices-operations/solution.py
import numpy as np

# Vector operations
v1 = np.array([1, 2, 3])
v2 = np.array([4, 5, 6])

dot_product = np.dot(v1, v2)          # 32
cosine_sim = dot_product / (np.linalg.norm(v1) * np.linalg.norm(v2))

# Matrix multiplication
A = np.random.randn(3, 4)
B = np.random.randn(4, 5)
C = A @ B  # shape: (3, 5)

# Eigendecomposition
M = np.array([[4, 2], [1, 3]])
eigenvalues, eigenvectors = np.linalg.eig(M)
print(f"Eigenvalues: {eigenvalues}")
```

### Phase 2 — Linear Regression from Scratch

```python
# phases/02-ml-fundamentals/02-linear-regression/solution.py
import numpy as np

class LinearRegressionScratch:
    def __init__(self, lr=0.01, n_iters=1000):
        self.lr = lr
        self.n_iters = n_iters
        self.weights = None
        self.bias = None
        self.loss_history = []

    def fit(self, X, y):
        n_samples, n_features = X.shape
        self.weights = np.zeros(n_features)
        self.bias = 0.0

        for _ in range(self.n_iters):
            y_pred = X @ self.weights + self.bias
            loss = np.mean((y_pred - y) ** 2)
            self.loss_history.append(loss)

            # Gradients
            dw = (2 / n_samples) * X.T @ (y_pred - y)
            db = (2 / n_samples) * np.sum(y_pred - y)

            self.weights -= self.lr * dw
            self.bias    -= self.lr * db

        return self

    def predict(self, X):
        return X @ self.weights + self.bias

# Usage
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split

X, y = make_regression(n_samples=200, n_features=5, noise=10)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

model = LinearRegressionScratch(lr=0.001, n_iters=2000)
model.fit(X_train, y_train)
preds = model.predict(X_test)
```

### Phase 3 — Backpropagation from Scratch

```python
# phases/03-deep-learning-core/03-backpropagation-from-scratch/solution.py
import numpy as np

class Value:
    """Scalar autograd engine (micrograd-style)."""
    def __init__(self, data, _children=(), _op=''):
        self.data = data
        self.grad = 0.0
        self._backward = lambda: None
        self._prev = set(_children)
        self._op = _op

    def __add__(self, other):
        other = other if isinstance(other, Value) else Value(other)
        out = Value(self.data + other.data, (self, other), '+')
        def _backward():
            self.grad  += out.grad
            other.grad += out.grad
        out._backward = _backward
        return out

    def __mul__(self, other):
        other = other if isinstance(other, Value) else Value(other)
        out = Value(self.data * other.data, (self, other), '*')
        def _backward():
            self.grad  += other.data * out.grad
            other.grad += self.data  * out.grad
        out._backward = _backward
        return out

    def relu(self):
        out = Value(max(0, self.data), (self,), 'ReLU')