jq

What this skill does

# jq - Command-Line JSON Processor

## Overview

jq is a small, fast command-line tool that **runs a *filter* program over JSON**. You write a
tiny program (the *filter*), jq reads JSON value(s) from stdin or files, runs the filter once
**per input value**, and prints each produced value back as JSON. Think of it as `sed`/`awk` for
structured JSON instead of lines of text.

**Key characteristics:**
- **A filter = a function** from one JSON value to *zero, one, or many* JSON values
- **Composable like a shell pipeline** — `|` pipes one filter's output into the next
- **Pure & functional** — no mutation; "assignment" returns a *modified copy*
- **Streaming by default** — reads whitespace-separated values, runs the filter on each
- **Deterministic, no TTY** — pure stdin→stdout, ideal for scripts, CI, and agents

### Mental model (read this first — it prevents the #1 source of confusion)

**A jq program is a filter: an input *stream* of JSON values → an output *stream* of JSON values.**

- The simplest filter is `.` (identity) — it copies input to output, pretty-printed.
- A filter can emit **0 values** (`empty`, a failed `select`), **1 value** (`.foo`), or
  **many** (`.[]`, `range(3)`). When a filter emits many values, everything downstream runs
  **once per value** (a generator / backtracking model).
- **`|` pipes**, exactly like a Unix shell: `f | g` runs `g` on each output of `f`.
- **`,` forks**: `f, g` emits all of `f`'s values, then all of `g`'s.
- jq never mutates the input. `|=` and `=` produce a *new* value with the change applied.

```bash
echo '{"a":1,"b":2}' | jq '.a'          # 1            (one value)
echo '[1,2,3]'       | jq '.[]'          # 1 2 3        (a stream of three)
echo '{"a":1,"b":2}' | jq '.a, .b'       # 1 2          (, forks the stream)
echo '[1,2,3]'       | jq '[.[] | .+10]' # [11,12,13]   (| pipes; [...] re-collects)
```

> **Disambiguation:** This skill documents the **`jq` CLI and its filter language** — invocation,
> I/O modes, the filter syntax, and the builtin library. It is **not** a generic JSON tutorial,
> and it does **not** cover jq-compatible reimplementations (`jaq`, `gojq`) or library bindings
> (Python `jq`, etc.) — for those, see their own docs.

## Prerequisites

**CRITICAL**: Before proceeding, verify jq is installed and check the version:

```bash
jq --version        # prints e.g. "jq-1.7.1"
```

**Version note:** This skill is documented against **jq 1.7 / 1.7.1** (the modern baseline, the big
release after a 5-year gap). Everything stable **at or before jq 1.6** is "bedrock" and shown
**unannotated**. Features added later are tagged inline as `(jq 1.7+)` / `(jq 1.8+)` **only where
the version is sourced** — see [references/version-features.md](references/version-features.md) for
the full feature → minimum-version map. Distros vary widely (many Linux LTS still ship 1.6 or
1.7); always confirm with `jq --version`. If a function errors with **"is not defined"**, it is
likely newer than your jq.

## Install

jq is a single self-contained binary with no runtime dependencies.

```bash
# macOS
brew install jq

# Debian/Ubuntu
sudo apt-get install jq

# Fedora/RHEL
sudo dnf install jq

# Arch
sudo pacman -S jq

# Alpine
apk add jq

# Windows
winget install jqlang.jq      # or: choco install jq / scoop install jq

# Docker (images moved to ghcr.io in 1.7)
docker run -i ghcr.io/jqlang/jq

# Static binaries / source: https://github.com/jqlang/jq/releases
```

Homepage **https://jqlang.org**; online playground **https://play.jqlang.org**.

## Invocation at a Glance

```bash
echo '<json>' | jq '<filter>'          # most common: pipe JSON in via stdin
jq '<filter>' file.json                # read from one or more files
jq -c '<filter>' file.json             # compact output (one value per line ⇒ NDJSON)
jq -r '<filter>'                       # raw output: strings without JSON quotes (shell capture)
jq -n '<filter>'                       # no input: filter starts from null (generate / use $ARGS,$ENV)
jq -f program.jq file.json             # load the filter from a file
jq --arg name "Ada" '<filter>'         # inject a shell value as $name (a string)
```

**Usage:** `jq [options] <filter> [file...]`. With no files, jq reads stdin. The filter is a
program in the jq language; the most useful invocation flags:

| Flag | Meaning |
|------|---------|
| `-n` / `--null-input` | Don't read input; run the filter once with `null`. For generators, `$ARGS`, `$ENV`. |
| `-r` / `--raw-output` | Output strings **without** JSON quotes/escapes (non-strings unchanged). |
| `-c` / `--compact-output` | Compact — one value per line ⇒ NDJSON output. |
| `-s` / `--slurp` | Read **all** inputs into a single array; run the filter once on that array. |
| `-R` / `--raw-input` | Read each line as a JSON **string** (not parsed JSON); with `-s`, the whole input as one string. |
| `-e` / `--exit-status` | Set exit code from the last output (see Troubleshooting). |
| `-j` / `--join-output` | Implies `-r`; no newline between outputs. |
| `-S` / `--sort-keys` | Sort object keys in output. |
| `-f file` / `--from-file` | Load the filter program from a file. |
| `--arg n v` / `--argjson n v` | Bind `$n` to a **string** / a parsed **JSON** value. |

> **Always single-quote the filter** in the shell — jq programs use `$`, `()`, `[]`, `|`, `"`,
> which the shell would otherwise mangle. Inject data with `--arg`/`--argjson`/`$ENV` rather than
> string-interpolating shell values into the program (safer; no quoting or injection bugs).

Full flag reference, I/O modes, arg-passing, exit codes, and module path:
[references/cli.md](references/cli.md).

## Core Workflows

All examples below are runnable and verified on jq 1.7.1.

### 1. Pretty-print / validate JSON

`jq .` reformats (pretty-prints) and exits non-zero on invalid JSON — a cheap validator.

```bash
jq . file.json                  # pretty-print
jq -c . file.json               # re-emit compact (one line)
jq -e . file.json >/dev/null    # validate: exit ≠ 0 ⇒ invalid JSON
```

### 2. Extract a field / nested path (use `-r` for shell capture)

```bash
jq '.user.name'    file.json    # nested object index → "Ada"  (JSON-quoted)
jq '.items[0].id'  file.json    # array index
jq -r '.user.name' file.json    # raw → Ada  (no quotes — what you want for the shell)
NAME=$(jq -r '.user.name' file.json)   # capture a string into a shell variable
```

Indexing a **missing** key yields `null` (not an error). Use `?` to suppress type errors
(`.a.b?`) and `// "default"` for a fallback (see Troubleshooting).

### 3. Filter a list of objects with `select` / pull a field from each

```bash
jq -c '.[] | select(.age > 30)' people.json   # one matching object per line
jq -r '.[] | .email'            people.json    # one email per line
jq    'map(select(.active))'    people.json    # keep the array shape, filtered
```

`.[]` iterates the array into a stream; `select(f)` keeps a value only when `f` is truthy
(only `false` and `null` are falsy — `0`, `""`, `[]` are **truthy**).

### 4. Reshape / build new objects

```bash
jq '[.[] | {id, name: .fullName, active: (.status == "on")}]' people.json
```

`{id}` is shorthand for `{id: .id}`. Object construction inside `[ ... ]` collects every produced
object back into one array. Computed keys: `{(.k): .v}`.

### 5. Emit CSV / TSV (and read line-oriented text)

```bash
# JSON array of objects → CSV rows (‑r so quotes/escapes are literal)
jq -r '.[] | [.id, .name, .amount] | @csv' file.json
jq -r '.[] | [.id, .name] | @tsv'          file.json   # tab-separated

# Lines of text → JSON array  (raw input + slurp = the whole stdin as one string)
jq -R -s 'split("\n")' file.txt
```

`@csv`/`@tsv` take an **array** and produce one delimited row, correctly quoting/escaping.
Format list (no `@base32` — it doesn't exist): `@text @json @base64 @base64d @uri @csv @tsv @html @sh`.

### 6. Slurp + aggregate (sum, average, group)

```bash
printf '{"x":1}\n{"x":2}\n{"x":3}\n' | jq -s 'map(.x) |