creationix-rx-data-store

What this skill does

# RX Data Store

> Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection.

RX is an embedded data store for JSON-shaped data. Encode once, then query the encoded document in place — no parsing, no object graph, no GC pressure. Think of it as *no-SQL SQLite*: unstructured data with database-style random access.

**Key benefits:**
- O(1) array access, O(log n) object key lookup on encoded data
- Automatic deduplication of values and shared schemas
- Text-safe encoding (copy-paste friendly, no binary tooling needed)
- Minimal heap allocations (~10 vs millions for JSON parsing)
- ~18x compression on real deployment manifests (92 MB → 5.1 MB)

---

## Installation

```sh
npm install @creationix/rx        # library
npm install -g @creationix/rx     # CLI (global)
npx @creationix/rx data.rx        # CLI (one-off)
```

---

## Core API

### String API (most common)

```ts
import { stringify, parse } from "@creationix/rx";

// Encode
const rx = stringify({ users: ["alice", "bob"], version: 3 });
// Returns a string — store it anywhere you'd store JSON text

// Decode (returns a read-only Proxy)
const data = parse(rx) as any;
data.users[0]         // "alice"
data.version          // 3
Object.keys(data)     // ["users", "version"]
JSON.stringify(data)  // works — full JS interop
```

### Uint8Array API (performance-critical paths)

```ts
import { encode, open } from "@creationix/rx";

const buf = encode({ path: "/api/users", status: 200 });
const data = open(buf) as any;
data.path    // "/api/users"
data.status  // 200
```

### Inspect API (lazy AST)

```ts
import { encode, inspect } from "@creationix/rx";

const buf = encode({ name: "alice", scores: [10, 20, 30] });
const root = inspect(buf);

root.tag          // ":"
root[0].tag       // "," (a string key)
root[0].value     // "name"
root.length       // 4 (key, value, key, value)

// Iterate children
for (const child of root) {
  console.log(child.tag, child.b64);
}

// Object helpers
for (const [key, val] of root.entries()) { /* ... */ }
const node = root.index("name");   // key lookup → node
const elem = root.index(2);        // array index → node

// Filtered key search (O(log n + m) on indexed objects)
for (const [key, val] of root.filteredKeys("/api/")) { /* ... */ }
```

### Escape hatch to underlying buffer

```ts
import { handle } from "@creationix/rx";

const h = handle(data.nested);
// h.data: Uint8Array
// h.right: byte offset
```

---

## Encoding Options

```ts
stringify(data, {
  // Add sorted indexes to containers with >= N entries (enables O(log n) lookup)
  // Use 0 for all containers, false to disable entirely
  indexes: 10,

  // External refs — shared dictionary of known values for cross-document dedup
  refs: { R: ["/api/users", "/api/teams"] },

  // Streaming — receive chunks as they're produced
  onChunk: (chunk: string, offset: number) => process.stdout.write(chunk),
});

encode(data, {
  indexes: 10,
  refs: { R: ["/api/users", "/api/teams"] },
  onChunk: (chunk: Uint8Array, offset: number) => { /* ... */ },
});
```

**If the encoder used external refs, pass the same dictionary to the decoder:**

```ts
const data = parse(payload, { refs: { R: ["/api/users", "/api/teams"] } });
const data = open(buf, { refs: { R: ["/api/users", "/api/teams"] } });
```

---

## CLI

```sh
rx data.rx                         # pretty-print as tree (default on TTY)
rx data.rx -j                      # convert to JSON
rx data.json -r                    # convert to RX
cat data.rx | rx                   # read from stdin (auto-detect format)
rx data.rx -s key 0 sub            # select a sub-value: data["key"][0]["sub"]
rx data.rx -o out.json             # write to file
rx data.rx --ast                   # output encoding structure as JSON
rx data.rx -w                      # write converted file (.json↔.rx)
```

**Full CLI flags:**

| Flag | Description |
|------|-------------|
| `<file>` | Input file (format auto-detected) |
| `-` | Read from stdin explicitly |
| `-j`, `--json` | Output as JSON |
| `-r`, `--rexc` | Output as RX |
| `-t`, `--tree` | Output as tree |
| `-a`, `--ast` | Output encoding structure |
| `-s`, `--select <seg>...` | Select a sub-value by path segments |
| `-w`, `--write` | Write converted file |
| `-o`, `--out <path>` | Write to file instead of stdout |
| `-c`, `--color` / `--no-color` | Force or disable ANSI color |
| `--index-threshold <n>` | Index containers above n values (default: 16) |
| `--string-chain-threshold <n>` | Split strings longer than n (default: 64) |
| `--string-chain-delimiter <s>` | Delimiter for string chains (default: `/`) |
| `--key-complexity-threshold <n>` | Max object complexity for dedupe keys (default: 100) |

**Shell completions:**

```sh
rx --completions setup zsh   # or bash
```

**Tip — paged, colorized viewing:**

```sh
p() { rx "$1" -t -c | less -RFX; }
```

---

## Proxy Behavior

The value returned by `parse`/`open` is **read-only**:

```ts
obj.newKey = 1;      // throws TypeError
delete obj.key;      // throws TypeError
"key" in obj;        // works (zero-alloc key search)
obj.nested === obj.nested  // true (container Proxies are memoized)
```

**Supported operations on the Proxy:**
- Property access: `data.key`, `data[0]`
- `Object.keys()`, `Object.entries()`, `Object.values()`
- `for...of`, `for...in`
- `Array.isArray()`
- `.map()`, `.filter()`, `.find()`, `.reduce()`
- Spread and destructuring
- `JSON.stringify()`

---

## Common Patterns

### Build-step: convert JSON artifact to RX

```ts
import { readFileSync, writeFileSync } from "fs";
import { stringify } from "@creationix/rx";

const json = JSON.parse(readFileSync("manifest.json", "utf-8"));
const rx = stringify(json, { indexes: 10 });
writeFileSync("manifest.rx", rx, "utf-8");
```

### Runtime: sparse read from large RX file

```ts
import { readFileSync } from "fs";
import { parse } from "@creationix/rx";

const manifest = parse(readFileSync("manifest.rx", "utf-8")) as any;

// Only the accessed values are decoded — everything else is skipped
const route = manifest.routes["/dashboard/projects"];
console.log(route.title, route.component, route.auth);
```

### Streaming encode to stdout

```ts
import { stringify } from "@creationix/rx";

stringify(largeData, {
  onChunk: (chunk, offset) => process.stdout.write(chunk),
});
```

### Using external refs for cross-document deduplication

```ts
import { stringify, parse } from "@creationix/rx";

const sharedRefs = { R: ["/api/users", "/api/teams", "/api/projects"] };

// Encode multiple documents sharing the same ref dictionary
const doc1 = stringify(data1, { refs: sharedRefs });
const doc2 = stringify(data2, { refs: sharedRefs });

// Decode with the same dictionary
const val1 = parse(doc1, { refs: sharedRefs }) as any;
const val2 = parse(doc2, { refs: sharedRefs }) as any;
```

### Low-level inspect traversal (zero allocation)

```ts
import { encode, inspect } from "@creationix/rx";

const buf = encode(routes);
const root = inspect(buf);

// Walk object entries without creating JS objects
if (root.tag === ":") {
  for (const [key, val] of root.entries()) {
    if (key.value === "/dashboard") {
      console.log("Found:", val.index("title").value);
      break;
    }
  }
}
```

### Type-safe usage pattern

```ts
import { parse } from "@creationix/rx";

interface Route {
  title: string;
  component: string;
  auth: boolean;
}

interface Manifest {
  routes: Record<string, Route>;
  version: number;
}

// Cast after parse — RX Proxy supports all read operations
const manifest = parse(rxString) as unknown as Manifest;
const dashboard = manifest.routes["/dashboard"];
```

---

## Format Reference

RX is a **text encoding** — not human-readable like JSON, but safe to copy-paste.

Every value is read **right-to-left**. The parser scans left past base64 digits to find a **tag** character:

```
[body][tag][b64 varint]
            ◄── read this way ──
```

| JSON | RX | Description |
|------|----|-------------|
| `42` | `+1k` | tag `+`