vercel-ai-sdk-knowledge-patch

Included with Lifetime

$97 forever

Vercel AI SDK changes since training cutoff (latest: 4.1.0) — PDF files, computer use tools, continuation, image generation, stream smoothing, createDataStreamResponse, tool call repair. Load before working with Vercel AI SDK.

Image & Video

What this skill does


# Vercel AI SDK Knowledge Patch

Claude Opus 4.6 knows AI SDK through 3.x. It is **unaware** of the features below, which cover AI SDK 4.0 (2024-11-18) through 4.1 (2025-01-20).

## Index

| Topic | Reference | Key features |
|---|---|---|
| 4.0 & 4.1 features | [references/ai-sdk-4-features.md](references/ai-sdk-4-features.md) | PDF files, computer use tools, continuation, image generation, stream smoothing, createDataStreamResponse, tool context/repair, structured outputs with tools, new providers |

---

## PDF Support (4.0)

Send PDFs as `file` type in message content:

```ts
const result = await generateText({
  model: anthropic('claude-3-5-sonnet-20241022'),
  messages: [{
    role: 'user',
    content: [
      { type: 'text', text: 'Summarize this.' },
      { type: 'file', data: fs.readFileSync('./doc.pdf'), mimeType: 'application/pdf' },
    ],
  }],
});
```

Works with Anthropic, Google Generative AI, and Vertex AI.

---

## Computer Use (4.0)

Anthropic tools accessible via `anthropic.tools`:

```ts
const computerTool = anthropic.tools.computer_20241022({
  displayWidthPx: 1920, displayHeightPx: 1080,
  execute: async ({ action, coordinate, text }) => { /* implement actions */ },
  experimental_toToolResultContent: (result) => /* format result */,
});

await generateText({
  model: anthropic('claude-3-5-sonnet-20241022'),
  tools: { computer: computerTool },
  maxSteps: 10,
});
```

Also: `anthropic.tools.textEditor_20241022()`, `anthropic.tools.bash_20241022()`.

---

## Continuation (4.0)

Auto-continue when generation hits length limit:

```ts
await generateText({
  model: openai('gpt-4o'),
  maxSteps: 5,
  experimental_continueSteps: true,
  prompt: 'Write a long essay...',
});
```

---

## Image Generation (4.1)

```ts
import { experimental_generateImage as generateImage } from 'ai';
import { replicate } from '@ai-sdk/replicate';

const { image } = await generateImage({
  model: replicate.image('black-forest-labs/flux-1.1-pro-ultra'),
  prompt: 'A cityscape at sunset',
  size: '16:9',
  n: 3,
});
// image.base64, image.uint8Array
```

Providers: `replicate.image()`, `openai.image()`, `vertex.image()`, `fireworks.image()`.

---

## Stream Smoothing (4.1)

```ts
import { smoothStream, streamText } from 'ai';

const result = streamText({
  model, prompt,
  experimental_transform: smoothStream(),
});
```

---

## Non-blocking Data Streaming (4.1)

Stream custom data before/alongside LLM output:

```ts
import { createDataStreamResponse, streamText } from 'ai';

return createDataStreamResponse({
  execute: async (dataStream) => {
    dataStream.writeData({ type: 'source', url: '...' });
    const result = streamText({ model, messages });
    result.mergeIntoDataStream(dataStream);
  },
});
```

---

## Tool Improvements (4.1)

Execute context: `execute(args, { toolCallId, messages, abortSignal })`.

Tool call repair:

```ts
await generateText({
  model, tools, prompt,
  experimental_repairToolCall: async ({ toolCall, tools, parameterSchema, error }) => {
    if (NoSuchToolError.isInstance(error)) return null;
    const { object } = await generateObject({ model, schema: tools[toolCall.toolName].parameters, prompt: '...' });
    return { ...toolCall, args: JSON.stringify(object) };
  },
});
```

Structured outputs with tools via `experimental_output: Output.object({ schema })` (OpenAI only).

---

## New Providers

| Package | Provider |
|---|---|
| `@ai-sdk/xai` | xAI Grok |
| `@ai-sdk/groq` | Groq |
| `@ai-sdk/replicate` | Replicate (image) |
| `@ai-sdk/fireworks` | Fireworks (language + image) |
| `@ai-sdk/deepinfra` | DeepInfra |
| `@ai-sdk/deepseek` | DeepSeek |
| `@ai-sdk/cerebras` | Cerebras |

---

## Reference Files

| File | Contents |
|---|---|
| [ai-sdk-4-features.md](references/ai-sdk-4-features.md) | Complete API reference for all 4.0 and 4.1 features with full code examples |

Files: 7

Size: 19.2 KB

Complexity: 43/100

Category: Image & Video

Source: https://github.com/nevaberry/nevaberry-plugins/tree/main/plugins/vercel-ai-sdk-knowledge-patch/skills/vercel-ai-sdk-knowledge-patch

Related in Image & Video

watch

Included

Watch a video (URL or local path). Downloads with yt-dlp, extracts auto-scaled frames with ffmpeg, pulls the transcript from captions (or Whisper API fallback), and hands the result to Claude so it can answer questions about what's in the video.

Image & Videoscriptsfeatured

physical-ai-defect-image-generation

Included

Use when the user wants to orchestrate defect image generation, run associated setup, or handle outputs on OSMO. The Day 0 path handles cold-start with USD-to-ROI, image-edit augmentation, and AnomalyGen to create initial PCBA datasets. The Day 1 path performs inference and labeling on real images. This skill helps with first-time asset setup, creation of finetuning checkpoints, and configuring deployment. Trigger keywords: defect image generation, dig workflow, dig pipeline, defect image detection workflow, aoi pipeline, aoi anomalygen, usd2roi anomalygen, day 0 pcba, day 1 pcba, day 1 real-photo alignment, day 1 manual roi, metal surface anomaly, glass defect, anomalygen finetune, setup_pcb, setup_metal, setup_glass, setup_pretrained, dig setup, dig datasets, dig pretrained checkpoint, dig image-edit endpoint.

Image & Videoscripts

accelint-react-best-practices

Included

React performance optimization and best practices. ALWAYS use this skill when working with any React code - writing components, hooks, JSX; refactoring; optimizing re-renders, memoization, state management; reviewing for performance; fixing hydration mismatches; debugging infinite re-renders, stale closures, input focus loss, animations restarting; preventing remounting; implementing transitions, lazy initialization, effect dependencies. Even simple React tasks benefit from these patterns. Covers React 19+ (useEffectEvent, Activity, ref props). Triggers - useEffect, useState, useMemo, useCallback, memo, inline components, nested components, components inside components, re-render, performance, hydration, SSR, Next.js, useDeferredValue, combined hooks.

Image & Videoscripts

elevenlabs-agents

Included

Build conversational AI voice agents with ElevenLabs Platform using React, JavaScript, React Native, or Swift SDKs. Configure agents, tools (client/server/MCP), RAG knowledge bases, multi-voice, and Scribe real-time STT. Use when: building voice chat interfaces, implementing AI phone agents with Twilio, configuring agent workflows or tools, adding RAG knowledge bases, testing with CLI "agents as code", or troubleshooting deprecated @11labs packages, Android audio cutoff, CSP violations, dynamic variables, or WebRTC config. Keywords: ElevenLabs Agents, ElevenLabs voice agents, AI voice agents, conversational AI, @elevenlabs/react, @elevenlabs/client, @elevenlabs/react-native, @elevenlabs/elevenlabs-js, @elevenlabs/agents-cli, elevenlabs SDK, voice AI, TTS, text-to-speech, ASR, speech recognition, turn-taking model, WebRTC voice, WebSocket voice, ElevenLabs conversation, agent system prompt, agent tools, agent knowledge base, RAG voice agents, multi-voice agents, pronunciation dictionary, voice speed control, elevenlabs scribe, @11labs deprecated, Android audio cutoff, CSP violation elevenlabs, dynamic variables elevenlabs, case-sensitive tool names, webhook authentication

Image & Videoscripts

humanizer

Included

Humanize AI-generated text by detecting and removing patterns typical of LLM output. Rewrites text to sound natural, specific, and human. Uses 28 pattern detectors, 560+ AI vocabulary terms across 3 tiers, and statistical analysis (burstiness, type-token ratio, readability) for comprehensive detection. Use when asked to humanize text, de-AI writing, make content sound more natural/human, review writing for AI patterns, score text for AI detection, or improve AI-generated drafts. Covers content, language, style, communication, and filler categories.

Image & Videoscripts

generating-mermaid-diagrams

Included

Salesforce architecture diagrams using Mermaid with ASCII fallback. Use this skill when generating text-based diagrams for Salesforce architecture, OAuth flows, ERDs, integration sequences, or Agentforce structure. TRIGGER when: user says "diagram", "visualize", "ERD", or asks for sequence diagrams, flowcharts, class diagrams, or architecture visualizations in Mermaid. DO NOT TRIGGER when: user wants PNG/SVG image output (use generating-visual-diagrams), or asks about non-Salesforce systems.

Image & Videoscripts