openless-voice-input

What this skill does

# OpenLess Voice Input

> Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection.

OpenLess is a cross-platform (macOS 12+, Windows 10+) voice-input app built with Tauri 2 + Rust + React/TypeScript. Press a global hotkey, speak, release — the app records audio, transcribes via Volcengine streaming ASR or Whisper, polishes the transcript with an LLM, and inserts the result at the active cursor in any app. It is a fully open-source alternative to Typeless, Wispr Flow, and Superwhisper.

---

## Installation (End Users)

### macOS
1. Download `OpenLess_<version>_aarch64.dmg` from [Releases](https://github.com/appergb/openless/releases/latest).
2. Open the DMG, drag `OpenLess.app` to `/Applications`.
3. Launch, grant **Microphone** and **Accessibility** permissions when prompted.
4. **Quit and reopen** — Accessibility only takes effect after a restart.
5. Open Settings → fill in ASR + LLM credentials.

### Windows
1. Download `OpenLess_<version>_x64-setup.exe` from Releases.
2. Run the installer.
3. Grant Microphone access when prompted.
4. Open Settings → Permissions → verify the global hotkey listener is active.
5. Fill in ASR + LLM credentials in Settings.

---

## Build from Source (Developers)

### Prerequisites
- Node.js 18+, npm
- Rust 1.77+ (`rustup`)
- Tauri CLI v2 (`cargo install tauri-cli --version "^2"`)
- macOS: Xcode Command Line Tools
- Windows: MSVC build tools or MinGW (see `openless-all/README.md`)

### Steps

```bash
git clone https://github.com/appergb/openless.git
cd openless/openless-all/app

npm ci

# Development (Vite at :1420 + Tauri shell with hot reload)
npm run tauri dev

# Production build — macOS (signs, installs, resets TCC)
./scripts/build-mac.sh

# Build only, skip install step
INSTALL=0 ./scripts/build-mac.sh

# Rust type-check without full compile
cargo check --manifest-path src-tauri/Cargo.toml

# Frontend TypeScript type-check
npm run build
```

### Log locations
- **macOS**: `~/Library/Logs/OpenLess/openless.log`
- **Windows**: `%LOCALAPPDATA%\OpenLess\Logs\openless.log`

---

## Configuration & Credentials

Credentials are stored in the platform Keychain (service = `com.openless.app`). A plaintext fallback is written to `~/.openless/credentials.json` (mode `0600`) when Keychain is unavailable in dev mode.

**Never commit API keys.** Reference them via environment variables or enter them in the Settings UI.

### Required credentials

| Key | Where to get it |
|-----|----------------|
| Volcengine ASR APP ID | Volcengine console → Speech Recognition |
| Volcengine ASR Access Token | Same console |
| Volcengine ASR Resource ID | Same console |
| Ark/LLM API Key | Volcengine Ark console or any OpenAI-compatible provider |
| Ark Model ID | e.g. `ep-XXXXXXXX` or a DeepSeek model ID |
| Ark Endpoint | Default: `https://ark.cn-beijing.volces.com/api/v3/chat/completions` |

### Setting credentials programmatically (Rust, for tests/CI)

```rust
// src-tauri/src/persistence.rs pattern
use crate::types::Credentials;

let creds = Credentials {
    asr_app_id: std::env::var("OPENLESS_ASR_APP_ID").unwrap(),
    asr_token: std::env::var("OPENLESS_ASR_TOKEN").unwrap(),
    asr_resource_id: std::env::var("OPENLESS_ASR_RESOURCE_ID").unwrap(),
    ark_api_key: std::env::var("OPENLESS_ARK_API_KEY").unwrap(),
    ark_model_id: std::env::var("OPENLESS_ARK_MODEL_ID").unwrap(),
    ark_endpoint: std::env::var("OPENLESS_ARK_ENDPOINT")
        .unwrap_or_else(|_| "https://ark.cn-beijing.volces.com/api/v3/chat/completions".to_string()),
};
// Pass to coordinator via Tauri state
```

---

## Architecture Overview

```
openless-all/app/
├── src/                    # React/TypeScript frontend
│   ├── pages/
│   │   ├── _atoms.tsx      # Recoil global state atoms
│   │   ├── Home.tsx
│   │   ├── History.tsx
│   │   ├── Dictionary.tsx
│   │   └── Settings.tsx
│   └── lib/
│       └── ipc.ts          # All Tauri invoke() calls (IPC surface)
└── src-tauri/src/          # Rust backend
    ├── types.rs            # Value types: DictationSession, PolishMode, errors
    ├── hotkey.rs           # CGEventTap (macOS) / WH_KEYBOARD_LL (Windows)
    ├── recorder.rs         # Mic → 16 kHz mono Int16 PCM + RMS callback
    ├── asr/                # Volcengine WebSocket ASR + Whisper HTTP
    ├── polish.rs           # OpenAI-compatible chat-completions
    ├── insertion.rs        # AX focused-element → clipboard+paste → copy fallback
    ├── persistence.rs      # History / prefs / vocab JSON + Keychain
    ├── permissions.rs      # TCC checks
    ├── coordinator.rs      # State machine: Idle→Starting→Listening→Processing
    └── commands.rs         # Tauri #[tauri::command] IPC surface
```

### Dictation pipeline

```
hotkey DOWN
  → coordinator: Idle → Starting → Listening
  → recorder.start() + asr.open_session()
  → [audio frames streamed to ASR via WebSocket]

hotkey UP
  → recorder.stop() + asr.send_last_frame()
  → coordinator: Listening → Processing
  → polish(transcript, mode) → LLM API call
  → insertion.insert_at_cursor(polished_text)
      ├─ AX focused element write (macOS Accessibility API)
      ├─ clipboard + Cmd+V / Ctrl+V paste
      └─ copy-only fallback (text in clipboard, user pastes manually)
  → history.save(session)
  → coordinator: Processing → Idle
```

`Esc` cancels at **any** phase including polish/insert.

---

## Polish Modes

| Mode | Tauri enum | Behaviour |
|------|-----------|-----------|
| Raw | `PolishMode::Raw` | Transcript verbatim, no LLM call |
| Light | `PolishMode::Light` | Remove filler words, fix punctuation |
| Structured | `PolishMode::Structured` | **AI-prompt mode** — reshapes speech into a structured, context-rich prompt |
| Formal | `PolishMode::Formal` | Formal prose, fixes grammar, organises paragraphs |

### Structured mode — what it does

Input (spoken): *"uh so I want ChatGPT to write me a SQL query from the orders table get last month's orders group by customer sort by amount desc top ten"*

Output inserted at cursor:
```text
Please write a SQL query that:

- Pulls orders from last month from the `orders` table.
- Groups by customer.
- Sorts by total amount, descending.
- Returns the top 10 rows only.
```

**Key invariant**: the LLM only *reshapes* your text. It does not answer questions or execute commands. If you say "what features are missing?", the output is `What features are missing?` — not a feature list.

---

## IPC Surface (Frontend → Backend)

All calls go through `src/lib/ipc.ts` using Tauri's `invoke()`.

```typescript
// src/lib/ipc.ts — representative subset

import { invoke } from "@tauri-apps/api/core";
import type { DictationSession, PolishMode, HotkeyBinding } from "./types";

// Save credentials (written to Keychain)
export async function saveCredentials(creds: {
  asrAppId: string;
  asrToken: string;
  asrResourceId: string;
  arkApiKey: string;
  arkModelId: string;
  arkEndpoint: string;
}): Promise<void> {
  return invoke("save_credentials", { creds });
}

// Load saved credentials (for Settings UI pre-fill)
export async function loadCredentials(): Promise<typeof creds | null> {
  return invoke("load_credentials");
}

// Get dictation history
export async function getHistory(): Promise<DictationSession[]> {
  return invoke("get_history");
}

// Set active polish mode
export async function setPolishMode(mode: PolishMode): Promise<void> {
  return invoke("set_polish_mode", { mode });
}

// Update hotkey binding
export async function setHotkey(binding: HotkeyBinding): Promise<void> {
  return invoke("set_hotkey", { binding });
}

// Check platform permissions
export async function checkPermissions(): Promise<{
  microphone: boolean;
  accessibility: boolean;
}> {
  return invoke("check_permissions");
}

// Add a vocabulary/dictionary entry
export async function addVocabEntry(entry: {
  word: string;
  category: string;
  notes: string;
}): Promise<void> {
  return invoke("add_vocab_entry", { entry });
}
```

---

## Recoil State Atoms

```typescript
// src/pages/_atoms.tsx