brightdata-sdk

What this skill does

# Bright Data SDK

Access web data through a unified Python SDK. One client, eight service categories: platform scraping, platform search, web search (SERP), AI-powered discovery, datasets, web unlocking, browser automation, and scraper studio.

Always use the client as a context manager. In synchronous environments (scripts, notebooks, Claude Code), use `SyncBrightDataClient`. In async environments, use `BrightDataClient`. Both use the same method names — the sync client wraps calls automatically. Note: the sync client currently has limited platform coverage — see the sync compatibility note in `references/scrapers.md` for details. For unsupported platforms or the datasets API, use the async client (`BrightDataClient`).

## Service Selection (decide first, then look up the specific method)

Use this decision tree to pick the right service BEFORE reaching for any specific method. Most routing failures come from skipping this step and pattern-matching on user keywords instead.

```
Have a URL?
  ├── On a supported platform (Amazon, LinkedIn, Facebook, Instagram, YouTube,
  │   TikTok, Reddit, ChatGPT, Perplexity, Pinterest, DigiKey)?
  │     → Platform scraping: client.scrape.<platform>.<method>(url=...)
  │
  ├── Generic page (not on any supported platform)?
  │     → Web unlocker: client.scrape_url(url=...)
  │
  └── Need login / JavaScript / click-scroll-fill / CAPTCHA / multi-step navigation?
        → Browser API: client.browser.get_connect_url() (then connect with Playwright)

No URL?
  ├── Want entities matching natural-language criteria
  │   ("find AI startups in Berlin", "competitors of Acme Corp", "people who worked at X")?
  │     → Discover: client.discover(query=..., intent=...)
  │
  ├── Want web pages / articles / search-result links
  │   ("search Google for X", "find pages about Y")?
  │     → SERP: client.search.google(query=...) [or .bing / .yandex]
  │
  ├── Want to search WITHIN a specific platform
  │   ("find products on Amazon", "TikTok videos by hashtag", "Pinterest pins about recipes")?
  │     → Platform search: client.search.<platform>.<method>(...)
  │
  └── Want bulk historical data at scale
      ("all LinkedIn companies in tech", "historical Amazon prices", "every Zillow listing in Texas")?
        → Datasets: client.datasets.<name>(filter=...) (then .download(snapshot_id))
```

Edge cases:
- URL on supported platform BUT user explicitly mentions login/click/scroll/JS → Browser API (the interaction trumps the platform).
- URL on supported platform BUT scraper returns 403/blocked → fall back to `client.scrape_url()` (web unlocker).
- "Find/research/who are X" with a URL alongside (e.g., "find competitors of acme.com") → still Discover; the URL is context, not the scrape target.

## Method Names: Verify Before Asserting

Before claiming any platform method exists, doesn't exist, or asserting a platform isn't supported, you MUST consult `references/scrapers.md` first. Past evals show the model has hallucinated method names (e.g., `client.scrape.linkedin.people` — does NOT exist; use `.profiles`) and falsely claimed platforms unsupported (e.g., Pinterest is supported via both `client.scrape.pinterest` and `client.search.pinterest`).

**Rule:** load `references/scrapers.md` before naming any specific platform method. The reference file lists every platform, every method signature, and the sync/async availability matrix. Verify, don't assume. If you've already loaded `references/scrapers.md` in this session, consult what's in context — no need to reload.

**Known hallucinations** (these names do NOT exist in the SDK — the model has invented them in past evals):

| Hallucinated | Correct replacement |
|---|---|
| `client.scrape.linkedin.people(...)` | `client.scrape.linkedin.profiles(url=...)` |
| `client.scrape.instagram.users(...)` | `client.scrape.instagram.profiles(url=...)` |
| `client.list_datasets()` | `client.datasets.list()` |
| `asyncio.gather(*[client.scrape.X.<quick>(...) for ...])` | Trigger pattern — see Batch gotcha |

This list grows as new hallucinations are observed in evals. If you're tempted to write a method name that "feels right" but you haven't seen in `references/scrapers.md`, treat it as a likely hallucination — load the reference and verify.

## Useful Standalone Methods

Methods that don't belong to any specific workflow — easy to overlook because they're not tied to a platform or a routing decision. The model has hallucinated some of these (`client.list_datasets()` instead of `client.datasets.list()`); use the canonical names below.

| Method | What it does |
|---|---|
| `client.datasets.list()` | List all 310+ datasets at runtime. Do NOT use `dir()` or introspection — use this method. |
| `client.discover(query=, intent=)` | AI-ranked entity search (companies, people, products). See Service Selection above. |
| `client.scrape_url(url=...)` | Web unlocker for any URL. Use for sites without a dedicated platform scraper. |
| `client.browser.get_connect_url()` | CDP WebSocket URL for Playwright / Puppeteer / Selenium. |
| `client.list_zones()` | List active Bright Data zones. |
| `client.delete_zone(name)` | Remove a zone. |
| `client.test_connection()` | Verify the API token works. |
| `client.get_account_info()` | Usage, quotas, active zones. |

## How to Handle Requests

### Exploring capabilities

If the user wants to know what's available or asks "what can this do?", describe these 8 categories. Each follows the template: **Name** — what it does — example invocation — when to use.

1. **Platform scraping** — extract structured data from 11 supported platforms by URL. `client.scrape.<platform>.<method>(url=...)` (platforms: Amazon, LinkedIn, Facebook, Instagram, YouTube, TikTok, Reddit, ChatGPT, Perplexity, Pinterest, DigiKey). Use when the user has a URL on a supported platform and wants structured fields (price, profile data, post engagement, etc.).

2. **Platform search** — search within a specific platform by keyword/profile/filter. `client.search.<platform>.<method>(...)`. Use for "find products on Amazon by keyword", "discover TikTok videos by hashtag", "find Pinterest pins about recipes" — i.e., the user wants to search WITHIN a platform but doesn't have a specific URL.

3. **Web search (SERP)** — get structured search engine results (titles, links, snippets, rankings). `client.search.google(query=...)` (or `.bing` / `.yandex`). Use for "search Google for X", "find pages/articles about Y", "look up news on Z" — i.e., the user wants web pages, not entities.

4. **Discover (AI-powered)** — `client.discover(query=..., intent=...)` to find entities (companies, people, products, places) matching natural-language criteria. Use for "find AI startups in Berlin", "competitors of Acme Corp", "people who worked at Stripe", "research the SaaS pricing landscape" — i.e., the user wants a list of entities matching a description, not web pages.

5. **Datasets** — access 310+ pre-built datasets with historical/bulk data at scale. `client.datasets.<name>(filter=...)` (then `.download(snapshot_id)`). Use for "bulk LinkedIn company data", "historical Amazon prices for electronics", "all Zillow listings in Texas" — i.e., the user wants many records at once, not live data on one page.

6. **Web unlocker** — scrape any URL with anti-bot bypass for sites without a dedicated platform scraper. `client.scrape_url(url=...)`. Use when the URL is on a generic website (no dedicated scraper) or as fallback when a platform scraper returns 403/blocked.

7. **Browser API** — connect to a remote browser via CDP (Chrome DevTools Protocol) for real-browser interaction. `client.browser.get_connect_url()` (then connect with Playwright/Puppeteer). Use for login flows, JavaScript-heavy single-page apps, click/scroll/fill interactions, CAPTCHA — i.e., anything requiring a real browser session. Most expensive option; use only when simpler methods can't accomplish the task.

8. **Scraper Studio** — run pre-built or custom scraping