mqtt-development

What this skill does

# MQTT Development

You are an expert in MQTT (Message Queuing Telemetry Transport) protocol development for IoT and real-time messaging systems. Follow these best practices when building MQTT-based applications.

## Core Principles

- MQTT is designed as an extremely lightweight publish/subscribe messaging transport
- Ideal for connecting remote devices with small code footprint and minimal network bandwidth
- MQTT requires up to 80% less network bandwidth than HTTP for transmitting the same amount of data
- A minimal MQTT control message can be as little as two data bytes

## Architecture Overview

### Components

- **Message Broker**: Server that receives messages from publishing clients and routes them to destination clients
- **Clients**: Any device (microcontroller to server) running an MQTT library connected to a broker
- **Topics**: Hierarchical strings used to filter and route messages
- **Subscriptions**: Client registrations for specific topic patterns

## Topic Design Best Practices

### Topic Structure

- Use hierarchical topic structures with forward slashes as level separators
- Maximum of seven forward slashes (/) in topic names for AWS IoT Core compatibility
- Do NOT prefix topics with a forward slash - it counts towards topic levels and creates confusion
- Use meaningful, descriptive topic segments

### Topic Naming Conventions

```
{organization}/{location}/{device-type}/{device-id}/{data-type}
```

Example: `acme/building-1/sensor/temp-001/temperature`

### Wildcard Usage

- **Single-level wildcard (+)**: Matches one topic level - prefer for device subscriptions
- **Multi-level wildcard (#)**: Matches all remaining levels - use sparingly
- Never allow a device to subscribe to all topics using `#`
- Reserve multi-level wildcards for server-side rules engines
- Use single-level wildcards (+) for device subscriptions to prevent unintended consequences

## Quality of Service (QoS) Levels

### QoS 0 - At Most Once

- Fire and forget - no acknowledgment
- Fastest but least reliable
- Use for: Sensor data where occasional loss is acceptable, high-frequency telemetry

### QoS 1 - At Least Once

- Guaranteed delivery, may have duplicates
- Balance of reliability and performance
- Use for: Important notifications, commands that can be safely repeated

### QoS 2 - Exactly Once

- Guaranteed single delivery using four-way handshake
- Highest overhead but most reliable
- Use for: Financial transactions, critical commands, state changes

### Choosing QoS

- Match QoS to your reliability requirements
- Consider bandwidth constraints - higher QoS means more overhead
- Publisher and subscriber QoS are independent - broker delivers at lower of the two

## Session Management

### Clean Sessions

- `cleanSession=true`: No session state preserved, suitable for transient clients
- `cleanSession=false`: Broker stores subscriptions and queued messages for offline clients

### Persistent Sessions

- Enable for devices with intermittent connectivity
- Broker stores undelivered messages (based on QoS) for later delivery
- Set appropriate session expiry intervals
- Consider message queue limits on the broker

### Keep-Alive

- Configure keep-alive interval based on network conditions
- Broker uses keep-alive to detect dead connections
- Shorter intervals = faster detection, more overhead
- Typical values: 30-60 seconds for stable networks, 10-15 for mobile

## Last Will and Testament (LWT)

- Configure LWT message for each client
- Broker publishes LWT when client disconnects unexpectedly
- Use for: Device status updates, alerts, cleanup triggers
- LWT topic typically: `{base-topic}/status` with payload `offline`

## Security Best Practices

### Transport Security

- MQTT sends credentials in plain text by default
- Always use TLS to encrypt connections in production
- Default unencrypted port: 1883
- Encrypted port: 8883
- Verify broker certificates to prevent MITM attacks

### Authentication

- Use strong client credentials (username/password or certificates)
- Implement OAuth, TLS 1.3, or customer-managed certificates where supported
- Rotate credentials regularly
- Consider client certificate authentication for high-security scenarios

### Authorization

- Implement topic-level access control
- Clients should only access topics they need
- Use ACLs (Access Control Lists) on the broker
- Separate read and write permissions per topic

## Message Design

### Payload Format

- Use efficient serialization (JSON for readability, binary for efficiency)
- Keep payloads small - MQTT is designed for constrained environments
- Include timestamps in messages for time-series data
- Consider schema versioning for payload format changes

### Message Properties

- Use retained messages for current state (last known value)
- Set appropriate message expiry for time-sensitive data
- Use user properties for metadata without polluting payload

## Client Implementation

### Connection Handling

- Implement automatic reconnection with exponential backoff
- Handle connection loss gracefully
- Queue messages during disconnection for later delivery
- Use connection pooling for multi-threaded applications

### Subscription Management

- Subscribe to specific topics, avoid broad wildcards
- Unsubscribe when no longer needed
- Handle subscription acknowledgment failures
- Resubscribe after reconnection if using clean sessions

### Publishing Best Practices

- Validate messages before publishing
- Handle publish failures appropriately
- Use batching for high-frequency publishing where supported
- Consider message ordering requirements

## Broker Configuration

### Scalability

- Configure appropriate connection limits
- Set message queue sizes based on expected load
- Implement clustering for high availability
- Use load balancers for horizontal scaling

### Monitoring

- Track connection counts and rates
- Monitor message throughput and latency
- Alert on queue depth and memory usage
- Log authentication failures

## Testing

### Unit Testing

- Mock MQTT client for isolated testing
- Test message serialization/deserialization
- Verify QoS handling logic

### Integration Testing

- Test with real broker in test environment
- Verify reconnection scenarios
- Test LWT functionality
- Load test with realistic device counts

## Common Patterns

### Request/Response

- Use correlated topics: `request/{id}` and `response/{id}`
- Include correlation ID in message
- Implement timeouts for responses

### Device Shadow/Twin

- Maintain desired and reported state
- Use separate topics for state updates
- Handle state synchronization on reconnection

### Command and Control

- Use dedicated command topics per device
- Implement command acknowledgment
- Handle command queuing for offline devices