grepai-search-advanced

What this skill does

# GrepAI Advanced Search Options

This skill covers advanced search options including JSON output, compact mode, and integration with AI agents.

## When to Use This Skill

- Integrating GrepAI with scripts or tools
- Using GrepAI with AI agents (Claude, GPT)
- Processing search results programmatically
- Reducing token usage in AI contexts

## Command-Line Options

| Option | Description |
|--------|-------------|
| `--limit N` | Number of results (default: 10) |
| `--json` / `-j` | JSON output format |
| `--toon` / `-t` | TOON output format (~50% fewer tokens than JSON) |
| `--compact` / `-c` | Compact output (no content, works with --json or --toon) |

> **Note:** `--json` and `--toon` are mutually exclusive.

## JSON Output

### Standard JSON

```bash
grepai search "authentication" --json
```

Output:
```json
{
  "query": "authentication",
  "results": [
    {
      "score": 0.89,
      "file": "src/auth/middleware.go",
      "start_line": 15,
      "end_line": 45,
      "content": "func AuthMiddleware() gin.HandlerFunc {\n    return func(c *gin.Context) {\n        token := c.GetHeader(\"Authorization\")\n        if token == \"\" {\n            c.AbortWithStatus(401)\n            return\n        }\n        claims, err := ValidateToken(token)\n        ...\n    }\n}"
    },
    {
      "score": 0.82,
      "file": "src/auth/jwt.go",
      "start_line": 23,
      "end_line": 55,
      "content": "func ValidateToken(tokenString string) (*Claims, error) {\n    ..."
    }
  ],
  "total": 2
}
```

### Compact JSON (AI Optimized)

```bash
grepai search "authentication" --json --compact
```

Output:
```json
{
  "q": "authentication",
  "r": [
    {
      "s": 0.89,
      "f": "src/auth/middleware.go",
      "l": "15-45"
    },
    {
      "s": 0.82,
      "f": "src/auth/jwt.go",
      "l": "23-55"
    }
  ],
  "t": 2
}
```

**Key differences:**
- Abbreviated keys (`s` vs `score`, `f` vs `file`)
- No content (just file locations)
- ~80% fewer tokens for AI agents

## TOON Output (v0.26.0+)

TOON (Token-Oriented Object Notation) is an even more compact format, optimized for AI agents.

### Standard TOON

```bash
grepai search "authentication" --toon
```

Output:
```
[2]{content,end_line,file_path,score,start_line}:
  "func AuthMiddleware()...",45,src/auth/middleware.go,0.89,15
  "func ValidateToken()...",55,src/auth/jwt.go,0.82,23
```

### Compact TOON (Best for AI)

```bash
grepai search "authentication" --toon --compact
```

Output:
```
[2]{end_line,file_path,score,start_line}:
  45,src/auth/middleware.go,0.89,15
  55,src/auth/jwt.go,0.82,23
```

### TOON vs JSON Comparison

| Format | Tokens (5 results) | Best For |
|--------|-------------------|----------|
| JSON | ~1,500 | Scripts, parsing |
| JSON compact | ~300 | AI agents |
| TOON | ~250 | AI agents |
| **TOON compact** | **~150** | **Token-constrained AI** |

### When to Use TOON

- **Use TOON** when integrating with AI agents that support it
- **Use TOON compact** for maximum token efficiency (~50% smaller than JSON compact)
- **Stick with JSON** for traditional scripting (jq, programming languages)

## Compact Format Reference

| Full Key | Compact Key | Description |
|----------|-------------|-------------|
| `query` | `q` | Search query |
| `results` | `r` | Results array |
| `score` | `s` | Similarity score |
| `file` | `f` | File path |
| `start_line`/`end_line` | `l` | Line range ("15-45") |
| `total` | `t` | Total results |

## Combining Options

```bash
# 5 results in compact JSON
grepai search "error handling" --limit 5 --json --compact

# 20 results in full JSON
grepai search "database" --limit 20 --json
```

## AI Agent Integration

### For Claude/GPT Prompts

Use compact mode to minimize tokens:

```bash
# Agent asks for context
grepai search "payment processing" --json --compact --limit 5
```

Then provide results to the AI with file read tool for details.

### Workflow Example

1. **Search for relevant code:**
```bash
grepai search "authentication middleware" --json --compact --limit 3
```

2. **Get response:**
```json
{
  "q": "authentication middleware",
  "r": [
    {"s": 0.92, "f": "src/auth/middleware.go", "l": "15-45"},
    {"s": 0.85, "f": "src/auth/jwt.go", "l": "23-55"},
    {"s": 0.78, "f": "src/handlers/auth.go", "l": "10-40"}
  ],
  "t": 3
}
```

3. **Read specific files:**
AI reads `src/auth/middleware.go:15-45` for full context.

## Scripting with JSON

### Bash + jq

```bash
# Get just file paths
grepai search "config" --json | jq -r '.results[].file'

# Filter by score
grepai search "config" --json | jq '.results[] | select(.score > 0.8)'

# Count results
grepai search "config" --json | jq '.total'
```

### Python

```python
import subprocess
import json

result = subprocess.run(
    ['grepai', 'search', 'authentication', '--json'],
    capture_output=True,
    text=True
)

data = json.loads(result.stdout)
for r in data['results']:
    print(f"{r['score']:.2f} | {r['file']}:{r['start_line']}")
```

### Node.js

```javascript
const { execSync } = require('child_process');

const output = execSync('grepai search "authentication" --json');
const data = JSON.parse(output);

data.results.forEach(r => {
    console.log(`${r.score.toFixed(2)} | ${r.file}:${r.start_line}`);
});
```

## MCP Integration

GrepAI provides MCP tools with format selection (v0.26.0+):

```bash
# Start MCP server
grepai mcp-serve
```

MCP tools support JSON (default) or TOON format:

| MCP Tool | Parameters |
|----------|------------|
| `grepai_search` | `query`, `limit`, `compact`, `format` |
| `grepai_trace_callers` | `symbol`, `compact`, `format` |
| `grepai_trace_callees` | `symbol`, `compact`, `format` |
| `grepai_trace_graph` | `symbol`, `depth`, `format` |
| `grepai_index_status` | `format` |

### Format Parameter

```json
{
  "name": "grepai_search",
  "arguments": {
    "query": "authentication",
    "format": "toon",
    "compact": true
  }
}
```

Valid values: `"json"` (default) or `"toon"`

## Token Optimization

### Token Comparison

For a typical search with 5 results:

| Format | Approximate Tokens |
|--------|-------------------|
| Human-readable | ~2,000 |
| JSON full | ~1,500 |
| JSON compact | ~300 |

### When to Use Each Format

| Format | Use Case |
|--------|----------|
| Human-readable | Manual inspection |
| JSON full | Scripts needing content |
| JSON compact | AI agents, token-limited contexts |

## Piping Results

### To File

```bash
grepai search "authentication" --json > results.json
```

### To Another Tool

```bash
# Open results in VS Code
grepai search "config" --json | jq -r '.results[0].file' | xargs code

# Copy first result path to clipboard (macOS)
grepai search "config" --json | jq -r '.results[0].file' | pbcopy
```

## Batch Searches

Run multiple searches:

```bash
#!/bin/bash
queries=("authentication" "database" "logging" "error handling")

for q in "${queries[@]}"; do
    echo "=== $q ==="
    grepai search "$q" --json --compact --limit 3
    echo
done
```

## Error Handling

### JSON Error Response

When search fails:

```json
{
  "error": "Index not found. Run 'grepai watch' first.",
  "code": "INDEX_NOT_FOUND"
}
```

### Checking for Errors in Scripts

```bash
result=$(grepai search "query" --json)
if echo "$result" | jq -e '.error' > /dev/null 2>&1; then
    echo "Error: $(echo "$result" | jq -r '.error')"
    exit 1
fi
```

## Best Practices

1. **Use compact for AI agents:** 80% token savings
2. **Use full JSON for scripts:** When you need content
3. **Use human-readable for debugging:** Easier to read
4. **Limit results appropriately:** Don't fetch more than needed
5. **Check for errors:** Parse JSON response properly

## Output Format

Advanced search output (JSON compact):

```json
{
  "q": "authentication middleware",
  "r": [
    {"s": 0.92, "f": "src/auth/middleware.go", "l": "15-45"},
    {"s": 0.85, "f": "src/auth/jwt.go", "l": "23-55"},
    {"s": 0.78, "f": "src/handlers/auth.go", "l": "10-40"}
  ],
  "t": 3
}
```

**Token estimate:** ~80