apify-js-sdk

What this skill does

# Apify-Js-Sdk Skill

Comprehensive assistance with Apify JavaScript SDK development for web scraping, crawling, and Actor creation. This skill provides access to official Apify documentation covering the API, SDK, and platform features.

## When to Use This Skill

This skill should be triggered when:
- Building web scrapers or crawlers with Apify
- Working with Apify Actors (creation, management, deployment)
- Using the Apify JavaScript Client to interact with the Apify API
- Managing Apify datasets, key-value stores, or request queues
- Implementing data extraction with Cheerio or other parsing libraries
- Setting up crawling workflows with link extraction and filtering
- Debugging Apify code or Actor runs
- Configuring logging and monitoring for Apify Actors
- Learning Apify platform best practices

## Key Concepts

### Actors
Serverless cloud programs running on the Apify platform. Actors can perform various tasks like web scraping, data processing, or automation.

### Datasets
Storage for structured data (results from scraping). Each Actor run can have an associated dataset where scraped data is stored.

### Key-Value Stores
Storage for arbitrary data like files, screenshots, or configuration. Each Actor run has a default key-value store.

### Request Queue
Queue for managing URLs to be crawled. Handles URL deduplication and retry logic automatically.

### Apify Client
JavaScript/Python library for interacting with the Apify API programmatically from your code.

## Quick Reference

### Basic Link Extraction with Cheerio

Extract all links from a webpage using Cheerio:

```javascript
import * as cheerio from 'cheerio';
import { gotScraping } from 'got-scraping';

const storeUrl = 'https://warehouse-theme-metal.myshopify.com/collections/sales';

const response = await gotScraping(storeUrl);
const html = response.body;

const $ = cheerio.load(html);

// Select all anchor elements
const links = $('a');

// Extract href attributes
for (const link of links) {
    const url = $(link).attr('href');
    console.log(url);
}
```

### Running an Actor with Apify Client

Call an Actor and wait for results:

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({
    token: 'YOUR_API_TOKEN',
});

// Run an Actor and wait for it to finish
const run = await client.actor('some_actor_id').call();

// Get dataset items from the run
const { items } = await client.dataset(run.defaultDatasetId).listItems();

console.log(items);
```

### Creating and Managing Datasets

Store scraped data in a dataset:

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({
    token: 'YOUR_API_TOKEN',
});

// Create a new dataset
const dataset = await client.datasets().getOrCreate('my-dataset');

// Add items to the dataset
await client.dataset(dataset.id).pushItems([
    { title: 'Product 1', price: 29.99 },
    { title: 'Product 2', price: 39.99 },
]);

// Retrieve items
const { items } = await client.dataset(dataset.id).listItems();
```

### Key-Value Store Operations

Store and retrieve arbitrary data:

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({
    token: 'YOUR_API_TOKEN',
});

const store = await client.keyValueStores().getOrCreate('my-store');

// Store a value
await client.keyValueStore(store.id).setRecord({
    key: 'config',
    value: { apiUrl: 'https://api.example.com' },
});

// Retrieve a value
const record = await client.keyValueStore(store.id).getRecord('config');
console.log(record.value);
```

### Logging Configuration

Set up proper logging for Apify Actors:

```python
import logging
from apify.log import ActorLogFormatter

async def main() -> None:
    handler = logging.StreamHandler()
    handler.setFormatter(ActorLogFormatter())

    apify_logger = logging.getLogger('apify')
    apify_logger.setLevel(logging.DEBUG)
    apify_logger.addHandler(handler)
```

### Using the Actor Context

Access Actor run context and storage:

```python
from apify import Actor

async def main() -> None:
    async with Actor:
        # Log messages
        Actor.log.info('Starting Actor run')

        # Access input
        actor_input = await Actor.get_input()

        # Save data to dataset
        await Actor.push_data({
            'url': 'https://example.com',
            'title': 'Example Page'
        })

        # Save to key-value store
        await Actor.set_value('OUTPUT', {'status': 'done'})
```

### Running an Actor Task

Execute a pre-configured Actor task:

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({
    token: 'YOUR_API_TOKEN',
});

// Run a task with custom input
const run = await client.task('task-id').call({
    startUrls: ['https://example.com'],
    maxPages: 10,
});

console.log(`Task run: ${run.id}`);
```

### Redirecting Logs from Called Actors

Redirect logs from a called Actor to the parent run:

```python
from apify import Actor

async def main() -> None:
    async with Actor:
        # Default redirect logger
        await Actor.call(actor_id='some_actor_id')

        # No redirect logger
        await Actor.call(actor_id='some_actor_id', logger=None)

        # Custom redirect logger
        await Actor.call(
            actor_id='some_actor_id',
            logger=logging.getLogger('custom_logger')
        )
```

### Getting Actor Run Details

Retrieve information about an Actor run:

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({
    token: 'YOUR_API_TOKEN',
});

// Get run details
const run = await client.run('run-id').get();

console.log(`Status: ${run.status}`);
console.log(`Started: ${run.startedAt}`);
console.log(`Finished: ${run.finishedAt}`);
```

### Listing Actor Builds

Get all builds for a specific Actor:

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({
    token: 'YOUR_API_TOKEN',
});

const { items } = await client.actor('actor-id').builds().list({
    limit: 10,
    desc: true,
});

for (const build of items) {
    console.log(`Build ${build.buildNumber}: ${build.status}`);
}
```

## Reference Files

This skill includes comprehensive documentation in the `references/` directory:

### llms-txt.md
Complete API reference documentation with detailed information on:
- **Actor Management**: Creating, updating, and running Actors
- **Builds**: Managing Actor builds and versions
- **Runs**: Controlling Actor execution and monitoring
- **Tasks**: Pre-configured Actor executions
- **Datasets**: Structured data storage and retrieval
- **Key-Value Stores**: Arbitrary data storage
- **Request Queues**: URL queue management
- **Client SDK**: JavaScript/Python client libraries
- **Logging**: Configuring and managing logs

### llms-full.md
Extensive documentation covering:
- Complete Apify API v2 reference
- All API endpoints with request/response examples
- Authentication and rate limiting
- Error handling
- Webhooks and integrations

### llms.md
High-level overview and getting started guide with:
- Platform concepts and architecture
- Quick start examples
- Common patterns and workflows
- Best practices for web scraping

## Working with This Skill

### For Beginners

Start with these concepts:
1. **Understanding Actors**: Review the Actors introduction to learn about the core building block
2. **First Scraper**: Use the link extraction examples to build your first web scraper
3. **Data Storage**: Learn about Datasets and Key-Value Stores for storing results
4. **API Basics**: Get familiar with the Apify Client for programmatic access

Key reference: `llms.md` for platform overview and getting started guides

### For Intermediate Users

Focus on these areas:
1. **Advanced Crawling**: Implement request queues and link filtering
2. **Actor Tasks**: Set up pre-configured runs with custom inputs
3. **Logging**: Configure proper logging with ActorLogFormatter
4. **Error Handling**: Implement retry logi