image-generator-sd-webui

What this skill does

# Image Generator (sd-webui API)

## Overview

Drive a Stable Diffusion WebUI / Forge server through its REST API to enumerate available resources, run `txt2img`, poll progress, and interrupt jobs. All scripts under `scripts/` are thin `curl` wrappers; they print JSON or extracted fields to stdout so the agent can pipe / parse them.

## Server connection

Before doing anything, confirm the server URL (and optional HTTP Basic Auth) with the user. Pass them as environment variables to every script:

```bash
export SD_WEBUI_URL="http://localhost:7860"   # required, no trailing slash
export SD_WEBUI_USER=""                       # optional, HTTP Basic Auth
export SD_WEBUI_PASS=""                       # optional
```

If unset, scripts default to `http://localhost:7860` with no auth.

Quick connectivity test (returns `OK <url>` on success, exits non-zero on failure):

```bash
scripts/probe.sh
```

## Workflow

1. **Probe** — Verify the server is reachable (`scripts/probe.sh`). On failure, ask the user for the correct URL / credentials.
2. **Enumerate & choose** — List the resources to pick (models, modules, samplers, schedulers, styles) and ask the user to choose. Capture their choice **verbatim** in the API's English `name` / `title` / `model_name` — sd-webui matches exactly, do not translate or rename.
3. **Prompt** — Obtain the positive prompt, negative prompt, and any extra params (steps, CFG, size). See "Prompt engineering" for sourcing these.
4. **Generate** — Call `scripts/generate.sh` with a request JSON. It returns a JSON object containing the base64 PNG image and the generation `info`.
5. **(Optional) Track progress** — While generation is running (in another shell / background), call `scripts/progress.sh` to print `progress` (0–1), `eta_relative`, and `state`.
6. **(Optional) Cancel** — Call `scripts/cancel.sh` to interrupt the current job.

## Tasks

### Listing available resources

| User wants | Command | API endpoint |
|---|---|---|
| Checkpoints (models) | `scripts/list.sh models` | `GET /sdapi/v1/sd-models` → array of `{title, model_name, hash, ...}` |
| Extra modules (TE / VAE, Forge-only) | `scripts/list.sh modules` | `GET /sdapi/v1/sd-modules` → array of `{model_name, ...}` |
| Samplers | `scripts/list.sh samplers` | `GET /sdapi/v1/samplers` → array of `{name, aliases}` |
| Schedulers | `scripts/list.sh schedulers` | `GET /sdapi/v1/schedulers` → array of `{name, label}` |
| Style presets | `scripts/list.sh styles` | `GET /sdapi/v1/prompt-styles` → array of `{name, prompt, negative_prompt}` |
| Upscalers | `scripts/list.sh upscalers` | `GET /sdapi/v1/upscalers` |
| LoRAs | `scripts/list.sh loras` | `GET /sdapi/v1/loras` |
| Embeddings | `scripts/list.sh embeddings` | `GET /sdapi/v1/embeddings` |

`scripts/list.sh <kind>` prints the canonical English identifier for each entry, one per line — pipe to `column`, `fzf`, etc. Add `--json` for the raw JSON.

After listing, present the options to the user (use `ask_user` with an enum if the list is short). For models, prefer the full `title` (which embeds the hash suffix, e.g. `anima/animaika_v36.safetensors [d50fb5b9a0]`) over `model_name` because the title is unambiguous — if the user supplies a bare filename without the hash, verify it via `list.sh models` and substitute the exact title before sending it to the API. For schedulers, `list.sh schedulers` prints the human-readable `label` (e.g. `Beta`); both `label` and the lowercase `name` (`beta`) are accepted by the txt2img `scheduler` field.

### Generating an image (txt2img)

1. Build a JSON request. Required field: `prompt`. Recommended: `negative_prompt`, `steps`, `cfg_scale`, `width`, `height`, `sampler_name`, `scheduler`, `styles` (array of style names), and `override_settings.sd_model_checkpoint` (model title) / `override_settings.forge_additional_modules` (array of module names, Forge only). See `references/txt2img-parameters.md` for every field.
2. Run:
   ```bash
   scripts/generate.sh request.json > result.json
   # or pipe:
   cat request.json | scripts/generate.sh - > result.json
   ```
3. Extract the image (base64 PNG):
   ```bash
   jq -r '.images[0]' result.json | base64 -d > out.png
   ```
4. The `info` field is a JSON string with `seed`, `all_prompts`, `sampler_name`, etc. — parse with `jq -r '.info | fromjson'`.

**Important behaviour notes:**

- **`samples_format` pre-pin**: sd-webui/Forge validates `samples_format` *before* applying `override_settings`, so if the server's persistent value is unsupported (e.g. `avif`), txt2img fails. `generate.sh` preemptively `POST`s `samples_format=png` to `/sdapi/v1/options` **and** redundantly injects `override_settings.samples_format=png`. ⚠️ The pre-pin mutates the server's persistent default to `"png"` — `override_settings_restore_afterwards` cannot undo it. If the user shares the server with clients expecting a different default, restore manually after: `scripts/options.sh set samples_format '"webp"'`. Convert locally if you need non-PNG output (see "Converting to another format" below).
- `override_settings_restore_afterwards: true` is forced on by `generate.sh` so the *other* `override_settings` keys (model checkpoint, modules, VAE) do not stick.
- Generation is **synchronous** — the POST blocks until the image is ready. The script uses a 600s curl timeout; override with `SD_WEBUI_TIMEOUT=900 scripts/generate.sh ...`.

#### Converting to another format

If the user wants the output in a non-PNG format (WebP, AVIF, JPEG, etc.), do **not** try to re-enable a different `samples_format` on the server. Instead, convert locally while preserving the embedded sd-webui generation metadata:

1. Check whether **both** `format-converter.sh` and `copy-info.sh` are available on `PATH` (e.g. `command -v format-converter.sh && command -v copy-info.sh`).
2. If both are present, run `format-converter.sh` on the PNG — it calls `copy-info.sh` internally to carry the parameters over. Run `format-converter.sh -h` to see the current usage.
3. If either is missing, guide the user to install the helper project once: <https://github.com/jim60105/sd-image-format-converter>. It has system dependencies that must be set up manually, so it can't be auto-installed. After install, both scripts should be on `PATH` and `-h` will show usage.

### Tracking progress

Call from another terminal (or background the `generate.sh` call with `&` first):

```bash
scripts/progress.sh                  # one-shot, prints JSON
scripts/progress.sh --watch          # poll every 1s until progress reaches 1.0 or state.job is empty
scripts/progress.sh --watch --interval 2
scripts/progress.sh --field progress # just the numeric 0..1 value
scripts/progress.sh --field state.job
```

Endpoint: `GET /sdapi/v1/progress?skip_current_image=true`. Key response fields:

- `progress` — float 0..1, fraction of current job complete.
- `eta_relative` — estimated seconds remaining.
- `state.job` — current job name (empty string when idle).
- `state.sampling_step` / `state.sampling_steps` — current step index / total.
- `current_image` — base64 PNG preview of the in-progress image (omitted by the script via `skip_current_image=true` to keep responses small; fetch raw with `curl` if needed).

### Cancelling

```bash
scripts/cancel.sh         # POST /sdapi/v1/interrupt — stop current job, return current partial result
scripts/cancel.sh --skip  # POST /sdapi/v1/skip — skip current job in a batch
```

Note: `interrupt` is *cooperative* — it tells the sampler to stop at the next step. The pending `generate.sh` call will return with whatever the model produced so far (often a usable but partial image). It does **not** raise an HTTP error on the txt2img call.

### Global options (advanced)

`scripts/options.sh` wraps `GET /sdapi/v1/options` and `POST /sdapi/v1/options`:

```bash
scripts/options.sh get                                # print all options as JSON
scripts/options.sh get sd_model_checkpoint            # print one key
scripts/options.sh set