browserless

What this skill does

## Troubleshooting

If requests fail, run `zero doctor check-connector --env-name BROWSERLESS_TOKEN` or `zero doctor check-connector --url https://production-sfo.browserless.io/scrape --method POST`

## How to Use

### 1. Scrape Data (CSS Selectors)

Extract structured JSON using CSS selectors:

Write to `/tmp/browserless_request.json`:

```json
{
  "url": "https://example.com",
  "elements": [
    {"selector": "h1"},
    {"selector": "p"}
  ]
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/scrape?token=$BROWSERLESS_TOKEN" --header "Content-Type: application/json" -d @/tmp/browserless_request.json
```

**With wait options:**

Write to `/tmp/browserless_request.json`:

```json
{
  "url": "https://news.ycombinator.com",
  "elements": [{"selector": ".titleline > a"}],
  "gotoOptions": {
    "waitUntil": "networkidle2",
    "timeout": 30000
  }
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/scrape?token=$BROWSERLESS_TOKEN" --header "Content-Type: application/json" -d @/tmp/browserless_request.json | jq '.data[0].results[:3]'
```

### 2. Take Screenshots

**Full page screenshot:**

Write to `/tmp/browserless_request.json`:

```json
{
  "url": "https://example.com",
  "options": {
    "fullPage": true,
    "type": "png"
  }
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/screenshot?token=$BROWSERLESS_TOKEN" --header "Content-Type: application/json" -d @/tmp/browserless_request.json --output screenshot.png
```

**Element screenshot:**

Write to `/tmp/browserless_request.json`:

```json
{
  "url": "https://example.com",
  "options": {
    "type": "png"
  },
  "selector": "h1"
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/screenshot?token=$BROWSERLESS_TOKEN" --header "Content-Type: application/json" -d @/tmp/browserless_request.json --output element.png
```

**With viewport size:**

Write to `/tmp/browserless_request.json`:

```json
{
  "url": "https://example.com",
  "viewport": {
    "width": 1920,
    "height": 1080
  },
  "options": {
    "type": "jpeg",
    "quality": 80
  }
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/screenshot?token=$BROWSERLESS_TOKEN" --header "Content-Type: application/json" -d @/tmp/browserless_request.json --output screenshot.jpg
```

### 3. Generate PDF

Write to `/tmp/browserless_request.json`:

```json
{
  "url": "https://example.com",
  "options": {
    "format": "A4",
    "printBackground": true,
    "margin": {
      "top": "1cm",
      "bottom": "1cm"
    }
  }
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/pdf?token=$BROWSERLESS_TOKEN" --header "Content-Type: application/json" -d @/tmp/browserless_request.json --output page.pdf
```

### 4. Get Rendered HTML

Get fully rendered HTML after JavaScript execution:

Write to `/tmp/browserless_request.json`:

```json
{
  "url": "https://example.com",
  "gotoOptions": {
    "waitUntil": "networkidle0"
  }
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/content?token=$BROWSERLESS_TOKEN" --header "Content-Type: application/json" -d @/tmp/browserless_request.json
```

### 5. Execute Custom JavaScript (Click, Type, etc.)

Run Puppeteer code with full interaction support:

**Click element:**

Write to `/tmp/browserless_function.js`:

```javascript
export default async ({ page }) => {
  await page.goto("https://example.com");
  await page.click("a");
  return { data: { url: page.url() }, type: "application/json" };
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/function?token=$BROWSERLESS_TOKEN" -H "Content-Type: application/javascript" -d @/tmp/browserless_function.js
```

**Type into input:**

Write to `/tmp/browserless_function.js`:

```javascript
export default async ({ page }) => {
  await page.goto("https://duckduckgo.com");
  await page.waitForSelector("input[name=q]");
  await page.type("input[name=q]", "hello world");
  const val = await page.$eval("input[name=q]", e => e.value);
  return { data: { typed: val }, type: "application/json" };
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/function?token=$BROWSERLESS_TOKEN" -H "Content-Type: application/javascript" -d @/tmp/browserless_function.js
```

**Form submission:**

Write to `/tmp/browserless_function.js`:

```javascript
export default async ({ page }) => {
  await page.goto("https://duckduckgo.com");
  await page.type("input[name=q]", "test query");
  await page.keyboard.press("Enter");
  await page.waitForNavigation();
  return { data: { title: await page.title() }, type: "application/json" };
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/function?token=$BROWSERLESS_TOKEN" -H "Content-Type: application/javascript" -d @/tmp/browserless_function.js
```

**Extract data with custom script:**

Write to `/tmp/browserless_function.js`:

```javascript
export default async ({ page }) => {
  await page.goto("https://news.ycombinator.com");
  const links = await page.$$eval(".titleline > a", els => els.slice(0,5).map(a => ({title: a.innerText, url: a.href})));
  return { data: links, type: "application/json" };
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/function?token=$BROWSERLESS_TOKEN" -H "Content-Type: application/javascript" -d @/tmp/browserless_function.js
```

### 6. Unblock Protected Sites

Bypass bot detection:

Write to `/tmp/browserless_request.json`:

```json
{
  "url": "https://example.com",
  "browserWSEndpoint": false,
  "cookies": false,
  "content": true,
  "screenshot": false
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/unblock?token=$BROWSERLESS_TOKEN" --header "Content-Type: application/json" -d @/tmp/browserless_request.json
```

### 7. Stealth Mode

Enable stealth mode to avoid detection:

Write to `/tmp/browserless_request.json`:

```json
{
  "url": "https://example.com",
  "elements": [{"selector": "body"}]
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/scrape?token=$BROWSERLESS_TOKEN&stealth=true" --header "Content-Type: application/json" -d @/tmp/browserless_request.json
```

### 8. Export Page with Resources

Fetch a URL and get content in native format. Can bundle all resources (CSS, JS, images) as zip:

**Basic export:**

Write to `/tmp/browserless_request.json`:

```json
{
  "url": "https://example.com"
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/export?token=$BROWSERLESS_TOKEN" --header "Content-Type: application/json" -d @/tmp/browserless_request.json --output page.html
```

**Export with all resources as ZIP:**

Write to `/tmp/browserless_request.json`:

```json
{
  "url": "https://example.com",
  "includeResources": true
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/export?token=$BROWSERLESS_TOKEN" --header "Content-Type: application/json" -d @/tmp/browserless_request.json --output webpage.zip
```

### 9. Performance Audit (Lighthouse)

Run Lighthouse audits for accessibility, performance, SEO, best practices:

**Full audit:**

Write to `/tmp/browserless_request.json`:

```json
{
  "url": "https://example.com"
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/performance?token=$BROWSERLESS_TOKEN" --header "Content-Type: application/json" -d @/tmp/browserless_request.json | jq '.data.categories | to_entries[] | {category: .key, score: .value.score}'
```

**Specific category (accessibility, performance, seo, best-practices, pwa):**

Write to `/tmp/browserless_request.json`:

```json
{
  "url": "https://example.com",
  "config": {
    "extends": "lighthouse:default",
    "settings": {
      "onlyCategories": ["performance"]
    }
  }
}
```

Then run:

```bash
curl -s -X POST "https://production-sfo.browserless.io/performance?token=$BROWSERLESS_TOKEN" --header