monitoring-flower

What this skill does

# Flower Monitoring Skill

This skill provides comprehensive templates and configurations for setting up Flower, the real-time monitoring tool for Celery. Includes authentication, custom dashboards, Prometheus metrics integration, and production deployment patterns.

## Overview

Flower is a web-based monitoring and administration tool for Celery that provides:

1. **Real-time Monitoring** - Worker status, task progress, event tracking
2. **Task Management** - View, revoke, and retry tasks
3. **Authentication** - Basic auth, OAuth, and custom authentication
4. **Metrics Export** - Prometheus integration for external monitoring
5. **Custom Dashboards** - Tailored views for specific workflows

This skill covers production-ready Flower deployments with security and scalability.

## Available Scripts

### 1. Start Flower Server

**Script**: `scripts/start-flower.sh <broker-url> <port>`

**Purpose**: Starts Flower monitoring server with proper configuration

**Parameters**:
- `broker-url` - Redis/RabbitMQ broker URL (default: redis://localhost:6379/0)
- `port` - Port to run Flower on (default: 5555)

**Usage**:
```bash
# Start with default settings
./scripts/start-flower.sh

# Start with custom Redis broker
./scripts/start-flower.sh redis://redis:6379/0 5555

# Start with RabbitMQ
./scripts/start-flower.sh amqp://guest:guest@localhost:5672// 5555

# Start with authentication
FLOWER_BASIC_AUTH="user:password" ./scripts/start-flower.sh
```

**Environment Variables**:
- `FLOWER_BASIC_AUTH` - Basic auth credentials (user:password)
- `FLOWER_OAUTH2_REDIRECT_URI` - OAuth2 redirect URI
- `FLOWER_MAX_TASKS` - Maximum tasks to keep in memory (default: 10000)

**Output**: Flower web UI available at http://localhost:5555

### 2. Install Flower as Systemd Service

**Script**: `scripts/flower-systemd.service`

**Purpose**: Systemd service file for production Flower deployment

**Usage**:
```bash
# Copy service file
sudo cp scripts/flower-systemd.service /etc/systemd/system/flower.service

# Edit service file with your paths
sudo nano /etc/systemd/system/flower.service

# Reload systemd
sudo systemctl daemon-reload

# Enable and start service
sudo systemctl enable flower
sudo systemctl start flower

# Check status
sudo systemctl status flower
```

**Configuration Points**:
- `WorkingDirectory` - Your project directory
- `User` - User to run service as
- `Environment` - Broker URL and authentication
- `ExecStart` - Flower command with options

### 3. Test Flower Configuration

**Script**: `scripts/test-flower.sh <flower-url>`

**Purpose**: Validates Flower setup and connectivity

**Checks**:
- Flower web UI accessible
- Worker nodes visible
- Task history available
- Metrics endpoint working
- Authentication configured
- No security warnings

**Usage**:
```bash
# Test local Flower instance
./scripts/test-flower.sh http://localhost:5555

# Test with authentication
./scripts/test-flower.sh http://user:password@localhost:5555

# Test production instance
./scripts/test-flower.sh https://flower.example.com
```

**Exit Codes**:
- `0` - All checks passed
- `1` - Flower not accessible
- `2` - No workers detected
- `3` - Authentication issues

## Available Templates

### 1. Flower Configuration

**Template**: `templates/flower-config.py`

**Purpose**: Complete Flower configuration file with all options

**Features**:
- Broker and backend URLs
- Port and address binding
- Task retention settings
- URL prefix for reverse proxy
- Database persistence
- Max workers and tasks

**Usage**:
```python
# Save as flowerconfig.py in your project
# Flower will auto-detect this file

# Or specify explicitly:
celery -A myapp flower --conf=flowerconfig.py
```

**Key Configuration Options**:
- `broker_api` - Broker management API URL
- `persistent` - Enable database persistence
- `db` - SQLite database path
- `max_tasks` - Task history limit
- `url_prefix` - Prefix for reverse proxy

### 2. Flower Authentication

**Template**: `templates/flower-auth.py`

**Purpose**: Authentication configurations including basic auth and OAuth

**Authentication Methods**:

**Basic Authentication**:
```python
# Username/password protection
flower --basic_auth=user1:password1,user2:password2
```

**OAuth2 (Google)**:
```python
# Google OAuth integration
flower \
  --auth=".*@example\.com" \
  --oauth2_key=your_google_client_id_here \
  --oauth2_secret=your_google_client_secret_here \
  --oauth2_redirect_uri=http://localhost:5555/login
```

**Custom Authentication**:
```python
# Implement custom auth provider
from flower.views.auth import Auth

class CustomAuth(Auth):
    def authenticate(self, username, password):
        # Your authentication logic
        return username in allowed_users
```

**Security Notes**:
- Never hardcode credentials in config files
- Use environment variables for secrets
- Enable HTTPS in production
- Implement rate limiting
- Use OAuth for team access

### 3. Prometheus Metrics

**Template**: `templates/prometheus-metrics.py`

**Purpose**: Export Celery metrics to Prometheus

**Metrics Exposed**:
- `celery_tasks_total` - Total tasks by state
- `celery_workers_online` - Active worker count
- `celery_task_runtime_seconds` - Task execution time
- `celery_queue_length` - Queue depth by queue name

**Usage**:
```python
# Run metrics exporter alongside Flower
python templates/prometheus-metrics.py

# Metrics available at http://localhost:8000/metrics
```

**Prometheus Scrape Config**:
```yaml
scrape_configs:
  - job_name: 'celery'
    static_configs:
      - targets: ['localhost:8000']
```

**Grafana Integration**:
- Import Celery dashboard template
- Connect to Prometheus data source
- Visualize task rates, queue depths, worker health

### 4. Custom Dashboard

**Template**: `templates/custom-dashboard.py`

**Purpose**: Create custom Flower views for specific workflows

**Custom Views**:
- Task filtering by type
- Worker grouping by role
- Custom metrics display
- Workflow-specific dashboards

**Implementation**:
```python
from flower.views import BaseHandler

class CustomDashboard(BaseHandler):
    def get(self):
        # Your custom dashboard logic
        self.render("custom_dashboard.html", data=data)
```

**Template Variables**:
- `workers` - Active worker list
- `tasks` - Recent task history
- `queues` - Queue statistics
- `custom_metrics` - Your computed metrics

## Available Examples

### 1. Complete Flower Setup

**Example**: `examples/flower-setup.md`

**Covers**:
- Initial Flower installation
- Configuration file setup
- Authentication implementation
- Systemd service creation
- Reverse proxy configuration (Nginx)
- SSL/TLS setup
- Monitoring integration

**Step-by-Step Guide**:
1. Install Flower: `pip install flower`
2. Create configuration file
3. Configure authentication
4. Test locally
5. Deploy as systemd service
6. Configure reverse proxy
7. Enable SSL
8. Connect monitoring tools

**Production Checklist**:
- [ ] Authentication enabled
- [ ] HTTPS configured
- [ ] Database persistence enabled
- [ ] Task retention limits set
- [ ] Resource limits configured
- [ ] Monitoring integrated
- [ ] Backup strategy defined

### 2. Prometheus Integration

**Example**: `examples/prometheus-integration.md`

**Covers**:
- Metrics exporter setup
- Prometheus configuration
- Grafana dashboard creation
- Alerting rules
- Performance optimization

**Metrics Collection**:
```python
# Key metrics to monitor
- Task success/failure rates
- Average task duration
- Queue depths
- Worker availability
- Task retries
- Error rates by task type
```

**Alert Examples**:
- High task failure rate
- Queue depth exceeding threshold
- Worker offline detection
- Slow task execution
- Memory usage alerts

### 3. Custom Dashboards

**Example**: `examples/custom-dashboards.md`

**Covers**:
- Creating custom views
- Template customization
- Adding custom metrics
- Filtering and grouping
- Real-time updates

**Use Cases**:
- ML training job monitoring
- ETL pipeline tracking
- Report generation status
- Video pr