cheerio
Parse and extract data from HTML with Cheerio. Use when a user asks to scrape static web pages, parse HTML files, extract data from HTML, build a web scraper for server-rendered pages, extract text or links from HTML documents, parse RSS/XML feeds, transform HTML content, or process HTML emails. Covers jQuery-style selectors, DOM traversal, text extraction, attribute parsing, and integration with HTTP clients for web scraping pipelines.
What this skill does
# Cheerio
## Overview
Cheerio is a fast, lightweight HTML/XML parser for Node.js that implements a jQuery-like API. Unlike Puppeteer, it does not run a browser — it parses raw HTML strings, making it 100x faster and ideal for scraping server-rendered pages, parsing HTML files, and transforming HTML content. Pair it with `fetch` or `axios` for web scraping, or use it standalone for HTML processing.
## Instructions
### Step 1: Installation
```bash
npm install cheerio
```
### Step 2: Parse HTML and Extract Data
```javascript
// parse_html.js — Load HTML and extract structured data with CSS selectors
import * as cheerio from 'cheerio'
const html = `
<html>
<body>
<h1>Products</h1>
<div class="product" data-id="1">
<h2>Widget Pro</h2>
<span class="price">$29.99</span>
<a href="/products/widget-pro">Details</a>
</div>
<div class="product" data-id="2">
<h2>Gadget Max</h2>
<span class="price">$49.99</span>
<a href="/products/gadget-max">Details</a>
</div>
</body>
</html>`
const $ = cheerio.load(html)
// Extract all products
const products = []
$('.product').each((i, el) => {
products.push({
id: $(el).attr('data-id'),
title: $(el).find('h2').text().trim(),
price: $(el).find('.price').text().trim(),
link: $(el).find('a').attr('href'),
})
})
console.log(products)
// [{ id: '1', title: 'Widget Pro', price: '$29.99', link: '/products/widget-pro' }, ...]
```
### Step 3: Web Scraping with Fetch
```javascript
// scrape_site.js — Fetch a page and extract data
import * as cheerio from 'cheerio'
async function scrape(url) {
const response = await fetch(url)
const html = await response.text()
const $ = cheerio.load(html)
// Extract all links
const links = []
$('a[href]').each((i, el) => {
links.push({
text: $(el).text().trim(),
href: $(el).attr('href'),
})
})
// Extract meta tags
const meta = {
title: $('title').text(),
description: $('meta[name="description"]').attr('content'),
ogImage: $('meta[property="og:image"]').attr('content'),
}
return { links, meta }
}
```
### Step 4: Advanced Selectors and Traversal
```javascript
// selectors.js — Complex CSS selectors and DOM traversal
const $ = cheerio.load(html)
// Attribute selectors
$('a[href^="https"]') // links starting with https
$('img[src$=".png"]') // PNG images
$('div[class*="product"]') // divs with "product" in class
// Traversal
$('.product').first() // first product
$('.product').last() // last product
$('.product').eq(2) // third product (0-indexed)
$('.price').parent() // parent of each .price element
$('.product').children('h2') // direct h2 children
$('.product').find('.price') // descendants matching .price
$('.product').next() // next sibling
$('.product').prev() // previous sibling
// Filtering
$('.product').filter((i, el) => {
const price = parseFloat($(el).find('.price').text().replace('$', ''))
return price < 50
})
// Text and HTML
$('.product').first().text() // all text content, flattened
$('.product').first().html() // inner HTML
```
### Step 5: Table Extraction
```javascript
// extract_table.js — Parse HTML tables into structured data
function extractTable($, tableSelector) {
/**
* Convert an HTML table to an array of objects using headers as keys.
* Args:
* $: Cheerio instance
* tableSelector: CSS selector for the table element
*/
const headers = []
$(`${tableSelector} thead th`).each((i, el) => {
headers.push($(el).text().trim())
})
const rows = []
$(`${tableSelector} tbody tr`).each((i, tr) => {
const row = {}
$(tr).find('td').each((j, td) => {
row[headers[j]] = $(td).text().trim()
})
rows.push(row)
})
return rows
}
// Usage
const tableData = extractTable($, '#pricing-table')
// [{ Plan: 'Free', Price: '$0', Users: '1' }, { Plan: 'Pro', Price: '$29', Users: '10' }]
```
### Step 6: HTML Transformation
```javascript
// transform.js — Modify HTML content
const $ = cheerio.load(html)
// Add class
$('.product').addClass('featured')
// Remove elements
$('.ad-banner').remove()
// Replace content
$('h1').text('Updated Title')
// Wrap elements
$('.product').wrap('<section class="product-section"></section>')
// Add attributes
$('a').attr('target', '_blank')
$('img').attr('loading', 'lazy')
// Get modified HTML
const modifiedHtml = $.html()
```
## Examples
### Example 1: Build a price monitoring scraper
**User prompt:** "Scrape product prices from 5 competitor websites daily and save to a CSV. The sites are server-rendered (no JavaScript needed)."
The agent will:
1. Use `fetch` + cheerio for each site (no browser overhead).
2. Write site-specific selectors for product name, price, and availability.
3. Parse prices into numbers, normalize currency.
4. Append results to a CSV with timestamps.
5. Set up as a cron job for daily execution.
### Example 2: Extract and clean article content from HTML
**User prompt:** "I have 1,000 saved HTML pages from a blog. Extract just the article title, author, date, and body text from each, ignoring navigation, ads, and footers."
The agent will:
1. Read each HTML file, load with cheerio.
2. Extract content using article-specific selectors (`article`, `.post-content`, etc.).
3. Strip HTML tags from body, normalize whitespace.
4. Output structured JSON with title, author, date, and clean text.
## Guidelines
- Use cheerio for server-rendered pages (where the HTML contains the data you need). For SPAs or JavaScript-rendered content, use Puppeteer instead.
- Cheerio does not execute JavaScript, fetch external resources, or render CSS — it only parses the HTML string you give it.
- Always call `.trim()` on extracted text — HTML often contains whitespace, newlines, and indentation that clutters results.
- Use `.attr('href')` and `.attr('src')` to get link/image URLs. Remember these may be relative — resolve them against the base URL.
- For large-scale scraping, cheerio is 100x faster than Puppeteer and uses negligible memory. It can process thousands of pages per second.
- Combine cheerio with `fetch` or `axios` for scraping, and add delays between requests to avoid overwhelming target servers.
Related in Web Dev
generating-lwc-components
IncludedLightning Web Components with PICKLES methodology and 165-point scoring. Use this skill when the user creates or edits LWC components, builds wire service patterns, or writes Jest tests for LWC. TRIGGER when: user creates/edits LWC components, touches lwc/**/*.js, .html, .css, .js-meta.xml files, or asks about wire service, SLDS, or Jest LWC tests. DO NOT TRIGGER when: Apex classes (use generating-apex), Aura components, or Visualforce.
tanstack-query
IncludedManage server state in React with TanStack Query v5. Set up queries with useQuery, mutations with useMutation, configure QueryClient caching strategies, implement optimistic updates, and handle infinite scroll with useInfiniteQuery. Use when: setting up data fetching in React projects, migrating from v4 to v5, or fixing object syntax required errors, query callbacks removed issues, cacheTime renamed to gcTime, isPending vs isLoading confusion, keepPreviousData removed problems.
document-processor-api
IncludedProcess documents with Nutrient DWS. Use when the user wants to generate PDFs from HTML or URLs, convert Office/images/PDFs, assemble or split packets, OCR scans, extract text/tables/key-value pairs, redact PII, watermark, sign, fill forms, optimize PDFs, or produce compliance outputs like PDF/A or PDF/UA. Triggers include convert to PDF, merge these PDFs, OCR this scan, extract tables, redact PII, sign this PDF, make this PDF/A, or linearize for web delivery.
nutrient-document-processing
IncludedProcess documents with Nutrient DWS. Use when the user wants to generate PDFs from HTML or URLs, convert Office/images/PDFs, assemble or split packets, OCR scans, extract text/tables/key-value pairs, redact PII, watermark, sign, fill forms, optimize PDFs, or produce compliance outputs like PDF/A or PDF/UA. Triggers include convert to PDF, merge these PDFs, OCR this scan, extract tables, redact PII, sign this PDF, make this PDF/A, or linearize for web delivery.
tanstack-query
IncludedManage server state in React with TanStack Query v5. Covers useMutationState, simplified optimistic updates, throwOnError, network mode (offline/PWA), and infiniteQueryOptions. Use when setting up data fetching, fixing v4→v5 migration errors (object syntax, gcTime, isPending, keepPreviousData), or debugging SSR/hydration issues with streaming server components.
accelint-nextjs-best-practices
IncludedNext.js performance optimization and best practices. Use when writing Next.js code (App Router or Pages Router); implementing Server Components, Server Actions, or API routes; optimizing RSC serialization, data fetching, or server-side rendering; reviewing Next.js code for performance issues; fixing authentication in Server Actions; or implementing Suspense boundaries, parallel data fetching, or request deduplication.