shadowbroker-osint-platform

What this skill does

```markdown
---
name: shadowbroker-osint-platform
description: Real-time multi-domain OSINT dashboard aggregating aircraft, ships, satellites, earthquakes, CCTV, GPS jamming, and geopolitical events on a unified map interface.
triggers:
  - set up shadowbroker osint dashboard
  - track private jets and aircraft with shadowbroker
  - add a new data layer to shadowbroker
  - configure shadowbroker backend api
  - deploy shadowbroker with docker
  - extend shadowbroker with custom feeds
  - shadowbroker satellite tracking setup
  - build osint map with shadowbroker
---

# ShadowBroker OSINT Platform

> Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection.

ShadowBroker is a self-hosted, real-time geospatial intelligence dashboard built with **Next.js** (frontend), **MapLibre GL** (map rendering), and **FastAPI + Python** (backend). It aggregates public OSINT feeds — ADS-B aircraft, AIS ships, satellite TLEs, USGS earthquakes, CCTV networks, GPS jamming, conflict events, and more — into a single dark-ops map interface.

---

## Installation

### Prerequisites
- Docker + Docker Compose **or** Podman + podman-compose
- Git

### Quick Start (Linux/macOS)

```bash
git clone https://github.com/BigBodyCobain/Shadowbroker.git
cd Shadowbroker
./compose.sh up -d
```

### Quick Start (Windows)

```bash
git clone https://github.com/BigBodyCobain/Shadowbroker.git
cd Shadowbroker
docker-compose up -d
```

Open **http://localhost:3000** in your browser.

### Force Podman engine

```bash
./compose.sh --engine podman up -d
```

### Update to latest version

```bash
git pull origin main

# Linux/macOS
./compose.sh down
./compose.sh up --build -d

# Windows
docker compose down
docker compose up --build -d
```

### Clear stale cache after update

```bash
docker compose build --no-cache
docker image prune -f
```

### View backend logs

```bash
./compose.sh logs -f backend
# or
docker compose logs -f backend
```

---

## Architecture Overview

```
Frontend (Next.js + MapLibre GL)  →  Backend (FastAPI + Python)  →  Public APIs
     :3000                                  :8000
```

- **Frontend**: TypeScript/Next.js app with MapLibre GL for WebGL map rendering, layer toggles, control panels.
- **Backend**: FastAPI server that proxies, caches, and aggregates public data feeds (OpenSky, CelesTrak, USGS, GDELT, aisstream.io, NASA FIRMS, etc.).
- **No external database required** — data is cached on disk and in memory.

### Kubernetes / Helm (Advanced)

```bash
helm repo add bjw-s-labs https://bjw-s-labs.github.io/helm-charts/
helm repo update
helm install shadowbroker ./helm/chart --create-namespace --namespace shadowbroker
```

---

## Environment Variables

Create a `.env` file in the project root (never commit secrets):

```env
# AIS vessel stream (required for maritime layer)
AISSTREAM_API_KEY=your_key_here

# NASA FIRMS fire hotspots (required for fire layer)
NASA_FIRMS_API_KEY=your_key_here

# Optional: override default ports
FRONTEND_PORT=3000
BACKEND_PORT=8000
```

Reference in code as `process.env.AISSTREAM_API_KEY` (frontend) or `os.environ["AISSTREAM_API_KEY"]` (backend).

---

## Key Data Layers & Sources

| Layer | Source | Notes |
|---|---|---|
| Commercial flights | OpenSky Network | ~5,000+ aircraft, no key needed |
| Private/military jets | adsb.lol | Includes owner identification |
| Maritime AIS | aisstream.io WebSocket | 25,000+ vessels, API key required |
| Satellites | CelesTrak TLE + SGP4 | 2,000+ active, no key needed |
| Earthquakes | USGS real-time feed | 24h window |
| Fire hotspots | NASA FIRMS VIIRS | API key required |
| Conflict events | GDELT | Last 8 hours |
| Ukraine frontline | DeepState Map GeoJSON | Live |
| CCTV cameras | TfL, TxDOT, NYC DOT, Singapore LTA | No key needed |
| GPS jamming | ADS-B NAC-P analysis | Computed from flight data |
| SDR receivers | KiwiSDR network | 500+ nodes |
| Satellite imagery | NASA GIBS MODIS / Esri / Sentinel-2 | No key needed |

---

## Backend API (FastAPI) — Key Endpoints

Base URL: `http://localhost:8000`

```
GET  /api/flights/commercial       # OpenSky commercial aircraft
GET  /api/flights/private          # Private/GA aircraft
GET  /api/flights/military         # Military aircraft
GET  /api/flights/private-jets     # HNW individual jets with owner info
GET  /api/ships                    # AIS vessel positions
GET  /api/satellites               # Real-time satellite positions (SGP4)
GET  /api/earthquakes              # USGS 24h earthquake feed
GET  /api/fires                    # NASA FIRMS hotspots
GET  /api/gdelt/conflict           # GDELT conflict events
GET  /api/carriers                 # US Navy carrier strike group positions
GET  /api/cctv                     # Aggregated CCTV camera feeds
GET  /api/gps-jamming              # Computed GPS jamming zones
GET  /api/space-weather            # NOAA geomagnetic Kp index
GET  /api/internet-outages         # Georgia Tech IODA outage data
GET  /api/news                     # SIGINT/OSINT RSS aggregation
GET  /api/region-dossier?lat=&lon= # Country profile + head of state
GET  /api/sentinel?lat=&lon=       # Sentinel-2 latest image for coordinates
```

---

## Frontend Usage — Adding a Custom Layer (TypeScript)

The frontend communicates with the backend via `fetch`. Example pattern for a new data layer:

```typescript
// src/hooks/useCustomLayer.ts
import { useEffect, useState } from "react";

interface CustomFeature {
  id: string;
  lat: number;
  lon: number;
  label: string;
}

export function useCustomLayer(enabled: boolean) {
  const [features, setFeatures] = useState<CustomFeature[]>([]);

  useEffect(() => {
    if (!enabled) return;

    const fetchData = async () => {
      const res = await fetch("http://localhost:8000/api/your-endpoint");
      const data = await res.json();
      setFeatures(data.features ?? []);
    };

    fetchData();
    const interval = setInterval(fetchData, 30_000); // refresh every 30s
    return () => clearInterval(interval);
  }, [enabled]);

  return features;
}
```

### Adding the layer to MapLibre GL

```typescript
// Inside your map component
import maplibregl from "maplibre-gl";

function addCustomLayer(map: maplibregl.Map, features: CustomFeature[]) {
  const sourceId = "custom-layer";
  const layerId = "custom-layer-points";

  const geojson: GeoJSON.FeatureCollection = {
    type: "FeatureCollection",
    features: features.map((f) => ({
      type: "Feature",
      geometry: { type: "Point", coordinates: [f.lon, f.lat] },
      properties: { id: f.id, label: f.label },
    })),
  };

  if (map.getSource(sourceId)) {
    (map.getSource(sourceId) as maplibregl.GeoJSONSource).setData(geojson);
  } else {
    map.addSource(sourceId, { type: "geojson", data: geojson });
    map.addLayer({
      id: layerId,
      type: "circle",
      source: sourceId,
      paint: {
        "circle-radius": 6,
        "circle-color": "#00ff88",
        "circle-opacity": 0.85,
      },
    });
  }
}
```

---

## Backend — Adding a Custom FastAPI Endpoint (Python)

```python
# backend/routers/custom.py
from fastapi import APIRouter
import httpx
import asyncio
from functools import lru_cache
import time

router = APIRouter()

_cache: dict = {"data": None, "ts": 0}
CACHE_TTL = 60  # seconds

async def fetch_source_data() -> list[dict]:
    async with httpx.AsyncClient(timeout=10) as client:
        resp = await client.get("https://example-public-api.org/data.json")
        resp.raise_for_status()
        return resp.json()

@router.get("/api/custom-feed")
async def custom_feed():
    now = time.time()
    if _cache["data"] is None or now - _cache["ts"] > CACHE_TTL:
        raw = await fetch_source_data()
        _cache["data"] = [
            {"id": item["id"], "lat": item["latitude"], "lon": item["longitude"], "label": item["name"]}
            for item in raw
        ]
        _cache["ts"] = now
    return {"features": _cache["data"], "count": len(_cache["data"])}
```

Register in `main.py`:

```python
from routers