netryx-street-level-geolocation

What this skill does

```markdown
---
name: netryx-street-level-geolocation
description: Expert skill for using Netryx, the open-source local-first street-level geolocation engine that identifies GPS coordinates from street photos using CosPlace, ALIKED/DISK, and LightGlue.
triggers:
  - geolocate a street photo
  - find GPS coordinates from an image
  - street level geolocation
  - index street view panoramas
  - use netryx to locate
  - run netryx geolocation pipeline
  - build a netryx index
  - identify location from street photo
---

# Netryx Street-Level Geolocation

> Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection.

Netryx is a locally-hosted, open-source geolocation engine that identifies precise GPS coordinates from any street-level photograph. It crawls Street View panoramas, indexes them as CosPlace embeddings, then uses a three-stage computer vision pipeline (global retrieval → local feature matching → refinement) to match a query image to a location. Sub-50m accuracy, no internet required at search time, runs entirely on local hardware.

---

## Installation

```bash
git clone https://github.com/sparkyniner/Netryx-OpenSource-Next-Gen-Street-Level-Geolocation.git
cd Netryx-OpenSource-Next-Gen-Street-Level-Geolocation

python3 -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate

pip install -r requirements.txt
pip install git+https://github.com/cvg/LightGlue.git   # required
pip install kornia                                        # optional: Ultra Mode (LoFTR)
```

### Optional: Gemini API key for AI Coarse mode
```bash
export GEMINI_API_KEY="your_key_here"   # from https://aistudio.google.com
```

### macOS tkinter fix (if GUI appears blank)
```bash
brew install [email protected]   # match your Python version
```

---

## Hardware Requirements

| Component | Minimum | Recommended |
|-----------|---------|-------------|
| GPU VRAM  | 4 GB    | 8 GB+       |
| RAM       | 8 GB    | 16 GB+      |
| Storage   | 10 GB   | 50 GB+      |
| Python    | 3.9+    | 3.10+       |

**GPU backends:** CUDA (NVIDIA) → uses ALIKED; MPS (Apple Silicon) → uses DISK; CPU → uses DISK (slow).

---

## Project Structure

```
netryx/
├── test_super.py          # Main GUI application (entry point)
├── cosplace_utils.py      # CosPlace model loading + descriptor extraction
├── build_index.py         # Standalone high-performance index builder
├── requirements.txt
├── cosplace_parts/        # Raw .npz embedding chunks (written during indexing)
└── index/
    ├── cosplace_descriptors.npy   # All 512-dim global descriptors
    └── metadata.npz               # lat/lon, heading, panoid per descriptor
```

---

## Launch the GUI

```bash
python test_super.py
```

The GUI has two modes: **Create** (index an area) and **Search** (geolocate a query image).

---

## Workflow

### Step 1 — Create an Index

Index a geographic area before searching. This crawls Street View panoramas and extracts CosPlace fingerprints.

**In GUI:**
1. Select **Create** mode
2. Enter center latitude/longitude
3. Set radius (km) — start with `0.5`–`1` for testing
4. Set grid resolution — default `300`, do not change
5. Click **Create Index**

**Indexing time estimates:**

| Radius | ~Panoramas | Time (M2 Max) | Index Size |
|--------|-----------|---------------|------------|
| 0.5 km | ~500      | 30 min        | ~60 MB     |
| 1 km   | ~2,000    | 1–2 hours     | ~250 MB    |
| 5 km   | ~30,000   | 8–12 hours    | ~3 GB      |
| 10 km  | ~100,000  | 24–48 hours   | ~7 GB      |

Indexing is **resumable** — if interrupted, re-run and it picks up where it left off.

**For large datasets, use the standalone builder:**
```bash
python build_index.py
```

### Step 2 — Search

1. Select **Search** mode
2. Upload a street-level photo
3. Choose search method:
   - **Manual**: provide approximate center lat/lon + radius
   - **AI Coarse**: Gemini analyzes visual clues to estimate region (requires `GEMINI_API_KEY`)
4. Click **Run Search** → **Start Full Search**
5. Result: GPS coordinates + confidence score displayed on map

**Enable Ultra Mode** for difficult images (night, blur, low texture). Adds LoFTR dense matching, descriptor hopping, and 100m neighborhood expansion. Slower but more robust.

---

## Pipeline Deep-Dive

### Stage 1 — Global Retrieval (CosPlace)
```
Query image → 512-dim CosPlace descriptor
             + flipped image → 512-dim descriptor
→ Cosine similarity against index (single matrix multiply, <1s)
→ Haversine radius filter
→ Top 500–1000 candidates
```

### Stage 2 — Geometric Verification (ALIKED/DISK + LightGlue)
```
For each candidate:
  Download Street View panorama (8 tiles, stitched)
  → Rectilinear crop at indexed heading
  → Multi-FOV crops: 70°, 90°, 110°
  → ALIKED (CUDA) or DISK (MPS/CPU) keypoint extraction
  → LightGlue deep feature matching vs query keypoints
  → RANSAC geometric verification → inlier count
Best match = highest inlier count
Processes 300–500 candidates in 2–5 minutes
```

### Stage 3 — Refinement
```
Top 15 candidates:
  → Heading refinement: ±45° at 15° steps, 3 FOVs
  → Spatial consensus: cluster into 50m cells
  → Confidence scoring: clustering strength + uniqueness ratio
→ Final GPS coordinates
```

### Ultra Mode extras
```
+ LoFTR detector-free dense matching (handles blur/low-contrast)
+ Descriptor hopping: re-search index using matched panorama's descriptor
+ Neighborhood expansion: search all panoramas within 100m of best match
```

---

## Using CosPlace Utilities Directly

```python
# cosplace_utils.py exposes model loading and descriptor extraction

from cosplace_utils import load_cosplace_model, get_descriptor
from PIL import Image
import torch

# Load model (cached after first call)
model = load_cosplace_model()  # auto-detects CUDA / MPS / CPU

# Extract a 512-dim descriptor from any PIL image
img = Image.open("query.jpg").convert("RGB")
descriptor = get_descriptor(model, img)   # returns np.ndarray shape (512,)
print(descriptor.shape)  # (512,)

# Compare two images via cosine similarity
import numpy as np
desc_a = get_descriptor(model, Image.open("a.jpg").convert("RGB"))
desc_b = get_descriptor(model, Image.open("b.jpg").convert("RGB"))
similarity = np.dot(desc_a, desc_b) / (np.linalg.norm(desc_a) * np.linalg.norm(desc_b))
print(f"Cosine similarity: {similarity:.4f}")
```

---

## Working with the Index Programmatically

```python
import numpy as np

# Load the compiled index
descriptors = np.load("index/cosplace_descriptors.npy")   # shape (N, 512)
meta = np.load("index/metadata.npz", allow_pickle=True)

lats      = meta["lats"]       # shape (N,)
lons      = meta["lons"]       # shape (N,)
headings  = meta["headings"]   # shape (N,)
panoids   = meta["panoids"]    # shape (N,)  — Street View panorama IDs

print(f"Index contains {len(lats):,} panorama views")
```

### Radius-filtered cosine search
```python
from math import radians, sin, cos, sqrt, atan2
import numpy as np

def haversine_km(lat1, lon1, lat2, lon2):
    R = 6371.0
    dlat = radians(lat2 - lat1)
    dlon = radians(lon2 - lon1)
    a = sin(dlat/2)**2 + cos(radians(lat1))*cos(radians(lat2))*sin(dlon/2)**2
    return R * 2 * atan2(sqrt(a), sqrt(1-a))

def search_index(query_descriptor, center_lat, center_lon, radius_km, top_k=500):
    """Return top_k candidate indices within radius_km of center."""
    descriptors = np.load("index/cosplace_descriptors.npy")
    meta = np.load("index/metadata.npz", allow_pickle=True)
    lats, lons = meta["lats"], meta["lons"]

    # Radius filter
    distances = np.array([
        haversine_km(center_lat, center_lon, lat, lon)
        for lat, lon in zip(lats, lons)
    ])
    in_radius = np.where(distances <= radius_km)[0]

    if len(in_radius) == 0:
        return []

    # Cosine similarity
    subset = descriptors[in_radius]
    q = query_descriptor / (np.linalg.norm(query_descriptor) + 1e-8)
    norms = np.linalg.norm(subset, axis=1, keepdims=True) + 1e-8
    sims = (subset / norms) @ q

    ranked = in_radi