brightdata-local-dev-loop
Configure Bright Data local development with hot reload and testing. Use when setting up a development environment, configuring test workflows, or establishing a fast iteration cycle with Bright Data. Trigger with phrases like "brightdata dev setup", "brightdata local development", "brightdata dev environment", "develop with brightdata".
What this skill does
# Bright Data Local Dev Loop
## Overview
Set up a fast, reproducible local development workflow for Bright Data scraping projects with mocked proxy responses, cached results, and vitest integration.
## Prerequisites
- Completed `brightdata-install-auth` setup
- Node.js 18+ with npm/pnpm
- brd-ca.crt SSL certificate downloaded
## Instructions
### Step 1: Create Project Structure
```
my-scraper/
├── src/
│ ├── brightdata/
│ │ ├── proxy.ts # Proxy configuration helper
│ │ ├── scraper.ts # Scraping functions
│ │ └── cache.ts # Response caching for dev
│ └── index.ts
├── tests/
│ ├── fixtures/ # Cached HTML responses
│ │ └── example.html
│ └── scraper.test.ts
├── .env.local # Local credentials (git-ignored)
├── .env.example # Template for team
├── brd-ca.crt # Bright Data SSL cert (git-ignored)
└── package.json
```
### Step 2: Build Proxy Configuration Module
```typescript
// src/brightdata/proxy.ts
import 'dotenv/config';
export interface BrightDataProxy {
host: string;
port: number;
auth: { username: string; password: string };
}
export function getProxy(options?: {
country?: string;
city?: string;
session?: string;
}): BrightDataProxy {
const { BRIGHTDATA_CUSTOMER_ID, BRIGHTDATA_ZONE, BRIGHTDATA_ZONE_PASSWORD } = process.env;
if (!BRIGHTDATA_CUSTOMER_ID || !BRIGHTDATA_ZONE || !BRIGHTDATA_ZONE_PASSWORD) {
throw new Error('Missing BRIGHTDATA_* environment variables');
}
let username = `brd-customer-${BRIGHTDATA_CUSTOMER_ID}-zone-${BRIGHTDATA_ZONE}`;
if (options?.country) username += `-country-${options.country}`;
if (options?.city) username += `-city-${options.city}`;
if (options?.session) username += `-session-${options.session}`;
return {
host: 'brd.superproxy.io',
port: 33335,
auth: { username, password: BRIGHTDATA_ZONE_PASSWORD },
};
}
```
### Step 3: Add Response Cache for Development
```typescript
// src/brightdata/cache.ts — cache scraped pages to avoid burning proxy credits
import { createHash } from 'crypto';
import { existsSync, readFileSync, writeFileSync, mkdirSync } from 'fs';
import { join } from 'path';
const CACHE_DIR = join(process.cwd(), '.scrape-cache');
export function getCachedResponse(url: string): string | null {
const key = createHash('md5').update(url).digest('hex');
const path = join(CACHE_DIR, `${key}.html`);
return existsSync(path) ? readFileSync(path, 'utf-8') : null;
}
export function setCachedResponse(url: string, html: string): void {
mkdirSync(CACHE_DIR, { recursive: true });
const key = createHash('md5').update(url).digest('hex');
writeFileSync(join(CACHE_DIR, `${key}.html`), html);
}
```
### Step 4: Configure Testing with Mocked Responses
```typescript
// tests/scraper.test.ts
import { describe, it, expect, vi, beforeEach } from 'vitest';
import axios from 'axios';
vi.mock('axios');
const mockedAxios = vi.mocked(axios);
describe('Bright Data Scraper', () => {
beforeEach(() => vi.clearAllMocks());
it('should scrape through proxy and return HTML', async () => {
mockedAxios.get.mockResolvedValueOnce({
status: 200,
data: '<html><head><title>Test</title></head></html>',
});
const { scrape } = await import('../src/brightdata/scraper');
const html = await scrape('https://example.com');
expect(html).toContain('<title>Test</title>');
// Verify proxy was configured
expect(mockedAxios.get).toHaveBeenCalledWith(
'https://example.com',
expect.objectContaining({
proxy: expect.objectContaining({ host: 'brd.superproxy.io' }),
}),
);
});
it('should retry on 502 proxy errors', async () => {
mockedAxios.get
.mockRejectedValueOnce({ response: { status: 502 } })
.mockResolvedValueOnce({ status: 200, data: '<html>OK</html>' });
const { scrapeWithRetry } = await import('../src/brightdata/scraper');
const html = await scrapeWithRetry('https://example.com');
expect(html).toContain('OK');
expect(mockedAxios.get).toHaveBeenCalledTimes(2);
});
});
```
### Step 5: Package Scripts
```json
{
"scripts": {
"dev": "tsx watch src/index.ts",
"scrape": "tsx src/index.ts",
"test": "vitest",
"test:watch": "vitest --watch",
"test:live": "BRIGHTDATA_LIVE=1 vitest --testPathPattern=integration"
}
}
```
## Output
- Proxy config module with geo-targeting support
- Response cache to avoid burning credits during development
- Mocked test suite that doesn't require live proxy access
- Live integration test flag (`BRIGHTDATA_LIVE=1`)
## Error Handling
| Error | Cause | Solution |
|-------|-------|----------|
| `Missing BRIGHTDATA_* vars` | No `.env.local` | Copy from `.env.example` |
| Cache stale | Old cached HTML | Delete `.scrape-cache/` directory |
| Mock not working | Import order | Use `vi.mock()` before dynamic imports |
| SSL errors in tests | CA cert path | Tests use mocks, not live proxy |
## Resources
- Bright Data Proxy Docs
- [Vitest Documentation](https://vitest.dev/)
- [tsx Documentation](https://github.com/esbuild-kit/tsx)
## Next Steps
See `brightdata-sdk-patterns` for production-ready code patterns.
Related in Code Review
gstack
IncludedFast headless browser for QA testing and site dogfooding. Navigate pages, interact with elements, verify state, diff before/after, take annotated screenshots, test responsive layouts, forms, uploads, dialogs, and capture bug evidence. Use when asked to open or test a site, verify a deployment, dogfood a user flow, or file a bug with screenshots. (gstack)
startup-due-diligence
IncludedLegal due diligence review for seed-stage and Series A startups (US, Delaware C-Corp focus). Supports both investor and founder perspectives. Capabilities include: (1) Interactive document review and issue spotting; (2) Document request list generation; (3) Cap table and SAFE/convertible note analysis; (4) Red flag identification with severity ratings; (5) Diligence report generation. TRIGGERS: due diligence, DD, startup investment, cap table review, Series A, seed round, investor diligence, legal review startup, SAFE analysis, convertible note, 409A, founder vesting.
interview-master
IncludedThis skill should be used when the user asks to "generate interview questions", "prepare for interview", "optimize resume", "conduct mock interview", "analyze git commits for resume", "generate resume from code", "review my resume", or mentions interview preparation, career assistance, or extracting project experience from git history. Provides comprehensive interview and career development guidance for both job seekers and interviewers.
fix-issue
IncludedFixes GitHub issues using parallel analysis agents for root cause investigation, code exploration, and regression detection. Reads issue context from gh CLI, searches codebase and memory for related patterns, generates a fix with tests, and links the resolution back to the issue via PR. Includes prevention analysis to avoid recurrence. Use when debugging errors, resolving regressions, fixing bugs, or triaging issues.
sf-apex
IncludedGenerates and reviews Salesforce Apex code with 150-point scoring. TRIGGER when: user writes, reviews, or fixes Apex classes, triggers, test classes, batch/queueable/schedulable jobs, or touches .cls/.trigger files. DO NOT TRIGGER when: LWC JavaScript (use sf-lwc), Flow XML (use sf-flow), SOQL-only queries (use sf-soql), or non-Salesforce code.
swift-development
IncludedComprehensive Swift development for building, testing, and deploying iOS/macOS applications. Use when Claude needs to: (1) Build Swift packages or Xcode projects from command line, (2) Run tests with XCTest or Swift Testing framework, (3) Manage iOS simulators with simctl, (4) Handle code signing, provisioning profiles, and app distribution, (5) Format or lint Swift code with SwiftFormat/SwiftLint, (6) Work with Swift Package Manager (SPM), (7) Implement Swift 6 concurrency patterns (async/await, actors, Sendable), (8) Create SwiftUI views with MVVM architecture, (9) Set up Core Data or SwiftData persistence, or any other Swift/iOS/macOS development tasks.