computer-control

Included with Lifetime

$97 forever

Automates desktop GUI workflows via computer use API with screenshot capture. Use when scripting GUI interactions or recording browser sessions for tutorials.

Backend & APIs

What this skill does

# Computer Control Skill

Use Claude's Computer Use API to see and control desktop
environments through screenshots and mouse/keyboard actions.

## When To Use

- Automating GUI-based workflows that lack CLI alternatives
- Testing web applications through visual interaction
- Filling forms, navigating menus, or interacting with desktop apps
- Building automation pipelines that need visual verification

## When NOT To Use

- Tasks achievable through CLI or API (no GUI needed)
- Browser automation better served by Playwright or CDP

> **Why this stays opt-in.** Per
> [docs/inclusive-defaults.md][inc] (TRUE-exception
> category 4), Computer Use takes screenshots and
> synthesizes keyboard/mouse input: cross-process side
> effects that must always be explicitly invoked, never
> default-on.

[inc]: ../../../../docs/inclusive-defaults.md

## Architecture

The computer use system has three layers:

1. **Display Toolkit** (`phantom.display`) - executes OS-level
   actions via xdotool/scrot on the real or virtual display
2. **Agent Loop** (`phantom.loop`) - manages the conversation
   cycle between Claude API and the display toolkit
3. **CLI** (`phantom.cli`) - command-line interface for running
   tasks or checking environment readiness

```
User Task
    |
    v
Agent Loop  <---->  Claude API (beta)
    |                   |
    v                   v
Display Toolkit    tool_use responses
    |              (click, type, screenshot)
    v
OS Commands (xdotool, scrot)
    |
    v
Display (X11 / Xvfb / WSLg)
```

## Quick Start

### Check environment

```bash
cd plugins/phantom
uv run python -m phantom.cli --check
```

### Run a task

```bash
export ANTHROPIC_API_KEY="sk-ant-..."
uv run python -m phantom.cli "Open Firefox and search for Claude AI"
```

### Use in Python

```python
from phantom.display import DisplayConfig, DisplayToolkit
from phantom.loop import LoopConfig, run_loop

result = run_loop(
    task="Take a screenshot of the desktop",
    api_key="sk-ant-...",
    loop_config=LoopConfig(
        model="claude-sonnet-4-6",
        max_iterations=10,
    ),
    display_config=DisplayConfig(width=1920, height=1080),
)

print(f"Done in {result.iterations} iterations")
print(result.final_text)
```

## API Versions

| Model | Tool Version | Beta Flag |
|-------|-------------|-----------|
| Opus 4.6, Sonnet 4.6, Opus 4.5 | `computer_20251124` | `computer-use-2025-11-24` |
| Sonnet 4.5, Haiku 4.5, older | `computer_20250124` | `computer-use-2025-01-24` |

The `resolve_tool_version()` function handles this mapping
automatically based on the model name.

## Available Actions

**All versions:**
- `screenshot` - capture display
- `left_click` - click at `[x, y]`
- `type` - type text string
- `key` - press key combo (e.g., `ctrl+s`)
- `mouse_move` - move cursor

**Enhanced (20250124+):**
- `scroll` - scroll with direction and amount
- `left_click_drag` - drag between coordinates
- `right_click`, `middle_click`, `double_click`, `triple_click`
- `hold_key` - hold key for duration
- `wait` - pause between actions

**Latest (20251124):**
- `zoom` - inspect screen region at full resolution

## Safety

Computer use carries risks. Follow these guidelines:

1. **Use a sandbox**: Run in Docker or a VM, not your main OS
2. **Limit access**: Do not provide login credentials unless
   necessary, and never for banking or sensitive services
3. **Set iteration caps**: Always use `max_iterations` to
   prevent runaway API costs
4. **Human approval**: For actions with real-world consequences,
   add confirmation callbacks via `on_action`
5. **Close sensitive apps**: Claude sees the full screen via
   screenshots; close anything private before starting

## Environment Requirements

**Linux (native or WSL2 with WSLg):**

```bash
sudo apt install xdotool scrot xclip
```

**Headless (Docker/CI):**

```bash
# Install Xvfb for virtual display
sudo apt install xvfb xdotool scrot xclip
Xvfb :1 -screen 0 1920x1080x24 &
export DISPLAY=:1
```

## Prompting Tips

1. Be specific about each step of the task
2. Add "After each step, take a screenshot and verify" to
   catch mistakes early
3. Use keyboard shortcuts when UI elements are hard to click
4. Provide example screenshots for repeatable workflows
5. Set a system prompt with domain-specific instructions

Files: 1

Size: 4.4 KB

Complexity: 13/100

Category: Backend & APIs

Source: https://github.com/athola/claude-night-market/tree/main/plugins/phantom/skills/computer-control

Related in Backend & APIs

jfrog

Included

Interact with the JFrog Platform via the JFrog CLI and REST/GraphQL APIs. Use this skill when the user wants to manage Artifactory repositories, upload or download artifacts, manage builds, configure permissions, manage users and groups, work with access tokens, configure JFrog CLI servers, search artifacts, manage properties, set up replication, manage JFrog Projects, run security audits or scans, look up CVE details, query exposures scan results from JFrog Advanced Security, manage release bundles and lifecycle operations, aggregate or export platform data, or perform any JFrog Platform administration task. Also use when the user mentions jf, jfrog, artifactory, xray, distribution, evidence, apptrust, onemodel, graphql, workers, mission control, curation, advanced security, exposures, or any JFrog product name.

Backend & APIsscripts

cupynumeric-migration-readiness

Included

Pre-migration readiness assessor for porting NumPy to cuPyNumeric. Use BEFORE substantial porting work begins when the user asks whether code will scale on GPU, whether they should migrate to cuPyNumeric, which NumPy patterns transfer cleanly, what must be refactored before porting, or mentions pre-port assessment, scaling analysis, or refactor planning. Inspect the user's source code, look up NumPy usage, cross-reference the cuPyNumeric API support manifest, and distinguish distributed-scaling-friendly patterns from blockers such as unsupported APIs, scalar synchronization, host round-trips, Python/object-heavy control flow, shape/data-dependent branching, and in-place mutation hazards. Produce a verdict of READY, LIGHT REFACTOR, SIGNIFICANT REFACTOR, or NOT RECOMMENDED, with concrete refactor pointers.

Backend & APIsscripts

alibabacloud-data-agent-skill

Included

Invoke Alibaba Cloud Apsara Data Agent for Analytics via CLI to perform natural language-driven data analysis on enterprise databases. Data Agent for Analytics is an intelligent data analysis agent developed by Alibaba Cloud Database team for enterprise users. It automatically completes requirement analysis, data understanding, analysis insights, and report generation based on natural language descriptions. This tool supports: discovering data resources (instances/databases/tables) managed in DMS, initiating query or deep analysis sessions, real-time progress tracking, and retrieving analysis conclusions and generated reports. Use this Skill when users need to query databases, analyze data trends, generate data reports, ask questions in natural language, or mention "Data Agent", "data analysis", "database query", "SQL analysis", "data insights".

Backend & APIsscripts

token-optimizer

Included

Reduce OpenClaw token usage and API costs through smart model routing, heartbeat optimization, budget tracking, and native 2026.2.15 features (session pruning, bootstrap size limits, cache TTL alignment). Use when token costs are high, API rate limits are being hit, or hosting multiple agents at scale. The 4 executable scripts (context_optimizer, model_router, heartbeat_optimizer, token_tracker) are local-only — no network requests, no subprocess calls, no system modifications. Reference files (PROVIDERS.md, config-patches.json) document optional multi-provider strategies that require external API keys and network access if you choose to use them. See SECURITY.md for full breakdown.

Backend & APIsscripts

resend-cli

Included

Use this skill when the task is specifically about operating Resend from an AI agent, terminal session, or CI job via the official resend CLI: installing/authenticating the CLI, sending/listing/updating/cancelling emails, batch sends, domains and DNS, webhooks and local listeners, inbound receiving, contacts, topics, segments, broadcasts, templates, API keys, profiles, or debugging Resend CLI/API failures. Trigger on mentions of Resend CLI, `resend`, `resend doctor`, `resend emails send`, `resend domains`, `resend webhooks listen`, `resend emails receiving`, or agent-friendly terminal automation.

Backend & APIsscripts

alibabacloud-odps-maxframe-coding

Included

Use this skill for MaxFrame SDK development and documentation navigation on Alibaba Cloud MaxCompute (ODPS). Helps answer MaxFrame API, concept, official example, and supported pandas API questions; create data processing programs; read/write MaxCompute tables; debug jobs (remote or local); and build custom DPE runtime images. Trigger when users mention MaxFrame, MaxCompute with MaxFrame, ODPS table processing, DPE runtime, MaxFrame docs/examples, DataFrame/Tensor operations, or GPU runtime setup. Works for both English and Chinese queries about Alibaba Cloud data processing with MaxFrame.

Backend & APIsscripts