monitoring-dashboard

What this skill does

# Monitoring Dashboard

**Purpose:** Provide complete monitoring dashboard templates and setup scripts for ML training with TensorBoard and Weights & Biases (WandB).

**Activation Triggers:**
- Setting up training monitoring dashboards
- Tracking experiments and metrics in real-time
- Comparing multiple training runs
- Visualizing hyperparameters and results
- Integrating monitoring into existing training pipelines
- Logging custom metrics, images, and model artifacts

**Key Resources:**
- `scripts/setup-tensorboard.sh` - Install and configure TensorBoard
- `scripts/setup-wandb.sh` - Install and configure Weights & Biases
- `scripts/launch-monitoring.sh` - Launch monitoring dashboards
- `templates/tensorboard-config.yaml` - TensorBoard configuration template
- `templates/wandb-config.py` - WandB integration template
- `templates/logging-config.json` - Unified logging configuration
- `examples/tensorboard-integration.md` - Complete TensorBoard integration guide
- `examples/wandb-integration.md` - Complete WandB integration guide

## Quick Start

### 1. Choose Monitoring Solution

**TensorBoard (Local/Open Source):**
- Free, runs locally
- Best for: Single-user development, offline work
- Features: Metrics, histograms, graphs, images, embeddings
- Storage: Local filesystem

**Weights & Biases (Cloud/Collaboration):**
- Free tier available, cloud-hosted
- Best for: Team collaboration, experiment comparison, production
- Features: All TensorBoard features + collaboration, alerts, reports
- Storage: Cloud with unlimited history

**Both (Recommended for Production):**
- Use TensorBoard for local development
- Use WandB for team collaboration and production tracking

### 2. Setup TensorBoard

```bash
# Install and configure TensorBoard
./scripts/setup-tensorboard.sh

# Launch TensorBoard
./scripts/launch-monitoring.sh tensorboard --logdir ./runs
```

**Access:** Open browser to http://localhost:6006

### 3. Setup Weights & Biases

```bash
# Install and configure WandB
./scripts/setup-wandb.sh

# Login with API key
wandb login

# Launch monitoring
./scripts/launch-monitoring.sh wandb
```

**Access:** Dashboard at https://wandb.ai/your-username/your-project

## TensorBoard Integration

### Basic Setup

**Template:** `templates/tensorboard-config.yaml`

```python
from torch.utils.tensorboard import SummaryWriter
import datetime

# Create TensorBoard writer
log_dir = f"runs/experiment_{datetime.datetime.now().strftime('%Y%m%d-%H%M%S')}"
writer = SummaryWriter(log_dir=log_dir)

# Log scalar metrics
writer.add_scalar('Loss/train', train_loss, epoch)
writer.add_scalar('Loss/validation', val_loss, epoch)
writer.add_scalar('Accuracy/train', train_acc, epoch)
writer.add_scalar('Accuracy/validation', val_acc, epoch)

# Log learning rate
writer.add_scalar('Learning_Rate', optimizer.param_groups[0]['lr'], epoch)

# Close writer when done
writer.close()
```

### Advanced Logging

**Histograms (Weight Distributions):**
```python
# Log model weights
for name, param in model.named_parameters():
    writer.add_histogram(f'weights/{name}', param, epoch)
    writer.add_histogram(f'gradients/{name}', param.grad, epoch)
```

**Images:**
```python
# Log sample predictions
writer.add_image('predictions', image_grid, epoch)
writer.add_images('batch_samples', image_batch, epoch)
```

**Text:**
```python
# Log hyperparameters as text
config_text = '\n'.join([f'{k}: {v}' for k, v in config.items()])
writer.add_text('hyperparameters', config_text, 0)
```

**Model Graph:**
```python
# Log model architecture
writer.add_graph(model, input_tensor)
```

**Embeddings (t-SNE, PCA):**
```python
# Visualize embeddings
writer.add_embedding(embeddings, metadata=labels, label_img=images)
```

### Launch TensorBoard

```bash
# Basic launch
tensorboard --logdir runs

# Specify port
tensorboard --logdir runs --port 6007

# Load faster (sample data)
tensorboard --logdir runs --samples_per_plugin scalars=1000

# Enable reload
tensorboard --logdir runs --reload_interval 5
```

## Weights & Biases Integration

### Basic Setup

**Template:** `templates/wandb-config.py`

```python
import wandb

# Initialize WandB run
wandb.init(
    project="my-ml-project",
    name=f"experiment-{datetime.now().strftime('%Y%m%d-%H%M%S')}",
    config={
        "learning_rate": 0.001,
        "epochs": 100,
        "batch_size": 32,
        "model": "resnet50",
        "dataset": "imagenet"
    }
)

# Log metrics
wandb.log({
    "train_loss": train_loss,
    "val_loss": val_loss,
    "train_acc": train_acc,
    "val_acc": val_acc,
    "epoch": epoch
})

# Finish run
wandb.finish()
```

### Advanced Features

**Log Media:**
```python
# Log images
wandb.log({"predictions": [wandb.Image(img, caption=f"Pred: {pred}")]})

# Log tables
table = wandb.Table(columns=["epoch", "loss", "accuracy"], data=data)
wandb.log({"results_table": table})

# Log audio
wandb.log({"audio": wandb.Audio(audio_array, sample_rate=16000)})

# Log videos
wandb.log({"video": wandb.Video(video_path, fps=30)})
```

**Track Model Artifacts:**
```python
# Save model checkpoint
artifact = wandb.Artifact('model-checkpoint', type='model')
artifact.add_file('model.pth')
wandb.log_artifact(artifact)

# Load model from artifact
artifact = wandb.use_artifact('model-checkpoint:latest')
model_path = artifact.download()
```

**Hyperparameter Sweeps:**
```python
# Define sweep configuration
sweep_config = {
    'method': 'bayes',
    'metric': {'name': 'val_loss', 'goal': 'minimize'},
    'parameters': {
        'learning_rate': {'min': 0.0001, 'max': 0.1},
        'batch_size': {'values': [16, 32, 64]},
        'optimizer': {'values': ['adam', 'sgd', 'adamw']}
    }
}

# Initialize sweep
sweep_id = wandb.sweep(sweep_config, project="my-project")

# Run sweep agent
wandb.agent(sweep_id, function=train_model, count=10)
```

**Custom Charts:**
```python
# Create custom plot
data = [[x, y] for (x, y) in zip(x_values, y_values)]
table = wandb.Table(data=data, columns=["x", "y"])
wandb.log({
    "custom_plot": wandb.plot.line(table, "x", "y", title="Custom Plot")
})
```

**Alerts:**
```python
# Alert on metric threshold
if val_loss < 0.1:
    wandb.alert(
        title="Low Validation Loss",
        text=f"Validation loss dropped to {val_loss:.4f}",
        level=wandb.AlertLevel.INFO
    )
```

## Unified Logging Configuration

**Template:** `templates/logging-config.json`

Use this configuration to log to both TensorBoard and WandB simultaneously:

```python
import wandb
from torch.utils.tensorboard import SummaryWriter

class UnifiedLogger:
    def __init__(self, project_name, experiment_name, config):
        # TensorBoard
        self.tb_writer = SummaryWriter(
            log_dir=f"runs/{experiment_name}"
        )

        # WandB
        wandb.init(
            project=project_name,
            name=experiment_name,
            config=config
        )

    def log_metrics(self, metrics_dict, step):
        """Log to both TensorBoard and WandB"""
        # TensorBoard
        for key, value in metrics_dict.items():
            self.tb_writer.add_scalar(key, value, step)

        # WandB
        wandb.log(metrics_dict, step=step)

    def log_images(self, images_dict, step):
        """Log images to both platforms"""
        for key, image in images_dict.items():
            # TensorBoard
            self.tb_writer.add_image(key, image, step)

            # WandB
            wandb.log({key: wandb.Image(image)}, step=step)

    def log_model(self, model, input_sample):
        """Log model architecture"""
        # TensorBoard graph
        self.tb_writer.add_graph(model, input_sample)

        # WandB watches gradients
        wandb.watch(model, log="all", log_freq=100)

    def close(self):
        """Close both loggers"""
        self.tb_writer.close()
        wandb.finish()

# Usage
logger = UnifiedLogger(
    project_name="my-project",
    experiment_name="exp-001",
    config={"lr": 0.001, "batch_size": 32}
)

logger.log_metrics({
    "train_loss": 0.5,
    "v