document-extraction

What this skill does

# Document Extraction (ADE)

## Overview

LandingAI's Agentic Document Extraction (ADE) is a document processing SaaS that parses, extracts, and classifies documents without requiring templates or training. It provides three main capabilities:

1. **Parse**: Convert documents into structured Markdown with hierarchical JSON representation
2. **Extract**: Pull specific structured data using JSON schemas or Pydantic models
3. **Split**: Classify and separate multi-document batches by type

**Key Benefits:**
- No ML training or templates required
- Layout-agnostic parsing (works with any document structure)
- Supports 20+ file formats (PDF, images, spreadsheets, presentations)
- Precise visual grounding (bounding boxes, page numbers)
- Multiple models optimized for different document types

## Quick Start

### 1. Installation

Never install packages globally without user approval. Always check for a local Python environment first.

```
1. .venv/bin/python       — uv-managed (this project)
2. venv/bin/python        — standard Python venv
3. uv run python          — if pyproject.toml exists
4. poetry run python      — if poetry.lock exists
5. python3                — system fallback; warn the user
```
Use the local environment to install: `landingai-ade`, `python-dotenv`

### 2. API Key Setup

The user may have already setup a `.env` file in the same directory as the `document-extraction` skill with the API key. You MUST check this path first (ls -la .*/skills/document-extraction/.env). Also try checking on the same directory as this SKILL.md file.

If not, provide instructions to create one. The script below will search for `.env` in common locations and load it.

```bash
.venv/bin/python - << 'EOF'
import os
from pathlib import Path
from dotenv import load_dotenv

# Load API key: prefer existing env var, then .env file lookup
if os.environ.get("VISION_AGENT_API_KEY"):
    print("API key found in existing environment variable")
else:
    def _find_env():
        for d in [Path.cwd().resolve(), *Path.cwd().resolve().parents]:
            for candidate in [
                # ADD the directory where the document-extraction skill is located
                d / '.env',
                d / 'document-extraction/.env',
                d / 'skills/document-extraction/.env',
            ]:
                if candidate.is_file():
                    return candidate
        return None
    env = _find_env()
    if env:
        load_dotenv(env)
        print(f"API key loaded from: {env}")
    else:
        print("Warning: VISION_AGENT_API_KEY not set and no .env found")
EOF
```

If not key is found instruct the user to get an API key from [https://va.landing.ai/settings/api-key](https://va.landing.ai/settings/api-key)

Copy `.env-sample` to `.env` and add your API key:

```bash
cp .env-sample .env
```

Edit `.env` and add your key:
```
VISION_AGENT_API_KEY=your_actual_api_key_here
```

**Note:** The `.env` file is gitignored for security. Advanced users can also set the environment variable directly: `export VISION_AGENT_API_KEY=<your-key>`

**EU Endpoint:** If using the EU endpoint, set `environment="eu"` when initializing the client.

### 3. Basic Parse Example

```python
from dotenv import load_dotenv
load_dotenv()  # Load API key from .env

from landingai_ade import LandingAIADE
from pathlib import Path

client = LandingAIADE()

# Parse a document
response = client.parse(
    document=Path("document.pdf"),
    model="dpt-2-latest"
)

# Access results
print(f"Pages: {response.metadata.page_count}")
print(f"Chunks: {len(response.chunks)}")
print("\nMarkdown output:")
print(response.markdown[:500])  # First 500 chars

# Save Markdown for extraction
with open("output.md", "w", encoding="utf-8") as f:
    f.write(response.markdown)
```

### 4. Basic Extract Example

```python
from dotenv import load_dotenv
load_dotenv()

from landingai_ade import LandingAIADE
from landingai_ade.lib import pydantic_to_json_schema
from pydantic import BaseModel, Field
from pathlib import Path

# Define extraction schema using Pydantic
class Invoice(BaseModel):
    invoice_number: str = Field(description="Invoice number")
    invoice_date: str = Field(description="Invoice date")
    total_amount: float = Field(description="Total amount in USD")
    vendor_name: str = Field(description="Vendor name")

# Convert to JSON schema
schema = pydantic_to_json_schema(Invoice)

client = LandingAIADE()

# Extract from parsed markdown
response = client.extract(
    schema=schema,
    markdown=Path("output.md"),  # From parse step
    model="extract-latest"
)

# Access extracted data
print(response.extraction)
# Output: {'invoice_number': 'INV-12345', 'invoice_date': '2024-01-15', ...}

# Check extraction metadata (traceability)
print(response.extraction_metadata)
```

## Document Parsing

### Parse Local Files

```python
from dotenv import load_dotenv
load_dotenv()

from landingai_ade import LandingAIADE
from pathlib import Path

client = LandingAIADE()

response = client.parse(
    document=Path("/path/to/document.pdf"),
    model="dpt-2-latest"
)

# Work with chunks
for chunk in response.chunks:
    print(f"Type: {chunk.type}, Page: {chunk.grounding.page}")
    print(f"Content: {chunk.markdown[:100]}...")
```

### Parse Remote URLs

```python
response = client.parse(
    document_url="https://example.com/document.pdf",
    model="dpt-2-latest"
)
```

### Parse Spreadsheets

Spreadsheets (CSV, XLSX) return a **different response type** than documents. Key differences:

| Field | Documents (`ParseResponse`) | Spreadsheets (`SpreadsheetParseResponse`) |
|---|---|---|
| `metadata.page_count` | ✓ | ✗ (uses `sheet_count`, `total_rows`, `total_cells`, `total_chunks`, `total_images`) |
| `splits[].pages` | ✓ | ✗ (uses `sheets` — array of sheet indices) |
| `grounding` (top-level) | ✓ | ✗ (not present for spreadsheets) |
| Chunk grounding | Always present | Optional (null for table chunks, present for embedded image chunks) |

```python
response = client.parse(
    document=Path("data.xlsx"),
    model="dpt-2-latest"
)

# Spreadsheet metadata
print(f"Sheets: {response.metadata.sheet_count}")
print(f"Total rows: {response.metadata.total_rows}")
print(f"Total cells: {response.metadata.total_cells}")

# Splits use 'sheets' instead of 'pages'
for split in response.splits:
    print(f"Sheet indices: {split.sheets}")
    print(f"Markdown: {split.markdown[:200]}...")
```

### Model Selection

Choose the right model for your documents:

| Model | Best For | Chunk Types |
|-------|----------|-------------|
| **dpt-2-latest** | Complex documents with logos, signatures, ID cards | text, table, figure, logo, card, attestation, scan_code, marginalia |
| **dpt-2-mini** | Simple, digitally-native documents (faster, cheaper) | text, table, figure, marginalia |
| **dpt-1** | ⚠️ **Deprecated March 31, 2026** — migrate to dpt-2 | text, table, figure, marginalia |

**Recommendation:** Use `dpt-2-latest` unless you have simple documents where cost/speed is critical.

**Version Pinning:** For production, use dated versions (e.g., `dpt-2-20260302`) for reproducibility.

### Parse Large Files (Async)

For files up to 1 GB or 6,000 pages, use Parse Jobs:

```python
import time
from dotenv import load_dotenv
load_dotenv()

from landingai_ade import LandingAIADE
from pathlib import Path

client = LandingAIADE()

# Step 1: Create parse job
job = client.parse_jobs.create(
    document=Path("large_document.pdf"),
    model="dpt-2-latest"
)

job_id = job.job_id
print(f"Job {job_id} created")

# Step 2: Poll for completion
while True:
    response = client.parse_jobs.get(job_id)
    if response.status == "completed":
        print(f"Job {job_id} completed")
        break
    print(f"Progress: {response.progress * 100:.0f}%")
    time.sleep(5)

# Step 3: Access results
# Results are in response.data (or response.output_url for large results)
if response.data:
    print(f"Chunks: {len(response.data.chunks)}")
    with open("output.md", "