Claude
Skills
Sign in
Back

web-search-techniques

Included with Lifetime
$97 forever

Knowledge base for web search query techniques, source authority ranking, WebFetch/WebSearch best practices, and bot-block fallback via webfetch.py. Used by quick-searcher and deep-researcher in plugins/research/. TRIGGER WHEN: performing web research with WebSearch or WebFetch. DO NOT TRIGGER WHEN: searching local codebase (use Grep or Glob directly).

General

What this skill does


# Web Search Techniques

Shared knowledge base for `research:quick-searcher` and `research:deep-researcher`. Scope: web-only. Covers query formulation, source authority, tool usage, bot-block fallback, and anti-loop rules.

## Query Formulation

Extract core concepts from the question before querying:
- Identify synonyms and domain terminology (e.g. "authentication": auth, login, signin, session, token, jwt, oauth, credentials)
- Account for abbreviations and full forms
- Add the year ("2026") when the query has temporal dependency
- Add "official" or "documentation" to push toward authoritative sources
- Quote exact phrases for precise matches
- Use `site:` to restrict to known-good domains (e.g. `site:developer.mozilla.org`)

Start broad, narrow progressively. Overly specific first queries miss adjacent information. Each refinement round incorporates terms surfaced in prior results.

## Source Authority Ranking

Rank every source before citing:

1. **Official documentation sites and API references** -- highest authority
2. **RFC and specification documents** -- canonical for standards
3. **GitHub issues, discussions, and source code** -- authoritative for specific libraries
4. **Peer-reviewed or community-validated content** -- Stack Overflow with high votes, maintainers' blogs
5. **General blog posts and tutorials** -- use only when nothing better exists
6. **Deprioritize** -- SEO content farms, AI-generated summaries, scraped aggregators

Currency checks:
- Last modified date on the page
- Version numbers cited vs latest release
- Deprecation warnings

## WebSearch Techniques

Query operators (standard search-engine conventions, usually respected by WebSearch):
- `site:` -- restrict to a domain
- `"exact phrase"` -- match the phrase verbatim
- Year token -- add "2026" for recency
- `"official"` or `"documentation"` -- bias toward authoritative
- Version numbers when relevant (e.g. `react 19`)

## WebFetch Guidance

- Prefer documentation pages and API references over blog posts
- Evaluate fetched content -- low-authority source means discard and re-search
- Large pages may be truncated -- target specific sections (anchor URLs) when possible
- Track the accessed URL with date for citation

## webfetch.py Fallback

When WebFetch returns a bot-block (403, 429, Cloudflare challenge) or thin content (under ~200 chars of useful text), fall back to the plugin's stealth fetcher:

```bash
python3 ${CLAUDE_PLUGIN_ROOT}/scripts/webfetch.py <url>
```

Behavior:
- Impersonates Chrome TLS fingerprint via curl_cffi
- Returns clean extracted text on stdout
- Exits 0 on success, 1 on timeout or error
- On failure, proceed without the result (do not retry in a loop)

Invocation options:
- `--timeout SECONDS` (default: 30)
- `--max-chars CHARS` (truncate output)
- `--raw` (return raw HTML instead of extracted text)

Requires `Bash` tool in the agent's `tools:` frontmatter.

## Anti-Loop Rules

- Never repeat the exact same query or search parameters
- If a search returns nothing, change terminology, broaden the regex, or switch tool/target
- Maximum 2 failed attempts per sub-topic before pivoting or escalating
- After 2 failed attempts on the same angle, document the gap and proceed with what you have

Related in General