livekit-nextjs-frontend
Build and review production-grade web and mobile frontends using LiveKit with Next.js. Covers real-time video/audio/data communication, WebRTC connections, track management, and best practices for LiveKit React components.
What this skill does
# LiveKit Next.js Frontend Development
This skill guides the development and review of production-grade web and mobile frontends using LiveKit with Next.js. Use this when building real-time communication features including video conferencing, live streaming, audio rooms, or data synchronization.
## Overview
LiveKit is a WebRTC-based platform for building real-time video, audio, and data applications. The official React components library (`@livekit/components-react`) provides battle-tested hooks and components for Next.js applications.
**Latest Versions (as of 2025):**
- `@livekit/components-react`: v2.9.16+
- `livekit-client`: Latest
- `livekit-server-sdk`: v2+ (supports Node.js, Deno, and Bun)
### Key Dependencies
```json
{
"dependencies": {
"livekit-client": "latest",
"@livekit/components-react": "latest",
"livekit-server-sdk": "latest"
},
"devDependencies": {
"tailwindcss": "latest",
"autoprefixer": "latest",
"postcss": "latest"
}
}
```
**Optional (for custom UI with icons):**
```bash
npm install lucide-react
```
The examples use Tailwind CSS for styling and lucide-react for icons. These are optional - you can use your own styling solution and icons/text alternatives.
## Architecture Patterns
### 1. Token-Based Authentication
LiveKit uses JWT-based access tokens signed with your API secret. Tokens must be generated server-side to prevent secret exposure.
**Environment Setup (.env.local):**
```env
# Client-accessible (for LiveKitRoom component)
NEXT_PUBLIC_LIVEKIT_URL=wss://your-project.livekit.cloud
# Server-only (never exposed to client)
LIVEKIT_API_KEY=your-api-key
LIVEKIT_API_SECRET=your-api-secret
```
**Note:** For server-side features like recording, you may also need:
```env
LIVEKIT_URL=wss://your-project.livekit.cloud
```
**Token Generation API Route (app/api/token/route.ts):**
```typescript
import { AccessToken } from 'livekit-server-sdk';
import { NextRequest, NextResponse } from 'next/server';
export async function GET(request: NextRequest) {
const roomName = request.nextUrl.searchParams.get('room');
const participantName = request.nextUrl.searchParams.get('username');
if (!roomName || !participantName) {
return NextResponse.json(
{ error: 'Missing room or username' },
{ status: 400 }
);
}
const at = new AccessToken(
process.env.LIVEKIT_API_KEY!,
process.env.LIVEKIT_API_SECRET!,
{
identity: participantName,
ttl: '6h', // Token expires after 6 hours
}
);
// Set permissions
at.addGrant({
roomJoin: true,
room: roomName,
canPublish: true,
canSubscribe: true,
canPublishData: true,
});
const token = await at.toJwt();
return NextResponse.json({ token });
}
```
**Security Best Practices:**
- Never expose API secrets in client-side code
- Validate user identity before issuing tokens
- Set appropriate token TTL based on use case
- Implement rate limiting on token endpoint
- Use HTTPS in production
### 2. Room Connection Pattern
**Basic Room Component:**
```typescript
'use client';
import { LiveKitRoom, VideoConference } from '@livekit/components-react';
import '@livekit/components-styles';
import { useEffect, useState } from 'react';
interface RoomPageProps {
roomName: string;
username: string;
}
export default function RoomPage({ roomName, username }: RoomPageProps) {
const [token, setToken] = useState('');
useEffect(() => {
// Fetch token from API route
fetch(`/api/token?room=${roomName}&username=${username}`)
.then(res => res.json())
.then(data => setToken(data.token));
}, [roomName, username]);
if (!token) {
return <div>Loading...</div>;
}
return (
<LiveKitRoom
token={token}
serverUrl={process.env.NEXT_PUBLIC_LIVEKIT_URL!}
connect={true}
video={true}
audio={true}
onDisconnected={() => {
// Handle disconnection
}}
onError={(error) => {
console.error('Room error:', error);
}}
>
<VideoConference />
</LiveKitRoom>
);
}
```
### 3. Custom Components with Hooks
**CRITICAL BEST PRACTICE:** Always use LiveKit's provided hooks instead of creating custom implementations. These hooks manage React state and are rigorously tested.
**Essential Hooks:**
- `useRoom()` - Access room state and events
- `useTracks()` - Subscribe to track updates
- `useParticipants()` - Get participant list
- `useLocalParticipant()` - Access local participant
- `useTrackToggle()` - Toggle audio/video
- `useLiveKitRoom()` - Lower-level room management
**Custom Controls Example:**
```typescript
'use client';
import { useRoom, useLocalParticipant, useTrackToggle } from '@livekit/components-react';
import { Track } from 'livekit-client';
export function CustomControls() {
const room = useRoom();
const { localParticipant } = useLocalParticipant();
// Use built-in hook for track toggling
const { buttonProps: audioProps, enabled: audioEnabled } = useTrackToggle({
source: Track.Source.Microphone,
});
const { buttonProps: videoProps, enabled: videoEnabled } = useTrackToggle({
source: Track.Source.Camera,
});
return (
<div className="controls">
<button {...audioProps}>
{audioEnabled ? 'Mute' : 'Unmute'}
</button>
<button {...videoProps}>
{videoEnabled ? 'Stop Video' : 'Start Video'}
</button>
<button onClick={() => room.disconnect()}>
Leave Room
</button>
</div>
);
}
```
### 4. Track Management
**Publishing Tracks:**
```typescript
import { useLocalParticipant } from '@livekit/components-react';
import { Track } from 'livekit-client';
function ScreenShareButton() {
const { localParticipant } = useLocalParticipant();
const startScreenShare = async () => {
await localParticipant.setScreenShareEnabled(true);
};
const stopScreenShare = async () => {
await localParticipant.setScreenShareEnabled(false);
};
return (
<button onClick={startScreenShare}>Share Screen</button>
);
}
```
**Subscribing to Remote Tracks:**
```typescript
import { useTracks, VideoTrack } from '@livekit/components-react';
import { Track } from 'livekit-client';
function RemoteParticipants() {
// Subscribe to all camera tracks
const tracks = useTracks([
{ source: Track.Source.Camera, withPlaceholder: true }
]);
return (
<div className="participants-grid">
{tracks.map((track) => (
<VideoTrack key={track.participant.sid} trackRef={track} />
))}
</div>
);
}
```
### 5. Data Messages
**IMPORTANT:** LiveKit recommends using higher-level APIs like text streams, byte streams, or RPC for most use cases. Use the low-level `publishData` API only when you need advanced control over individual packet behavior.
**Message Size Limits:**
- **Reliable packets**: 16KiB (16,384 bytes) recommended maximum for compatibility
- **Lossy packets**: 1,300 bytes maximum to stay within network MTU (1,400 bytes)
- Larger messages in lossy mode get fragmented; if any fragment is lost, the entire message is lost
**Sending Data:**
```typescript
import { useLocalParticipant } from '@livekit/components-react';
function ChatComponent() {
const { localParticipant } = useLocalParticipant();
const sendMessage = (message: string) => {
const encoder = new TextEncoder();
const data = encoder.encode(JSON.stringify({ message }));
// Validate size (16KiB limit for reliable messages)
if (data.byteLength > 16 * 1024) {
console.error('Message too large');
return;
}
// Use topic to differentiate message types
localParticipant.publishData(data, {
reliable: true, // Reliable delivery with retransmission
topic: 'chat', // Topic helps filter different message types
});
};
return (
<button onClick={() => sendMessage('Hello!')}>
Send Message
</button>
);
}
```
**Receiving Data:**
```typescript
import { useRoom } from '@livekRelated in Image & Video
watch
IncludedWatch a video (URL or local path). Downloads with yt-dlp, extracts auto-scaled frames with ffmpeg, pulls the transcript from captions (or Whisper API fallback), and hands the result to Claude so it can answer questions about what's in the video.
physical-ai-defect-image-generation
IncludedUse when the user wants to orchestrate defect image generation, run associated setup, or handle outputs on OSMO. The Day 0 path handles cold-start with USD-to-ROI, image-edit augmentation, and AnomalyGen to create initial PCBA datasets. The Day 1 path performs inference and labeling on real images. This skill helps with first-time asset setup, creation of finetuning checkpoints, and configuring deployment. Trigger keywords: defect image generation, dig workflow, dig pipeline, defect image detection workflow, aoi pipeline, aoi anomalygen, usd2roi anomalygen, day 0 pcba, day 1 pcba, day 1 real-photo alignment, day 1 manual roi, metal surface anomaly, glass defect, anomalygen finetune, setup_pcb, setup_metal, setup_glass, setup_pretrained, dig setup, dig datasets, dig pretrained checkpoint, dig image-edit endpoint.
accelint-react-best-practices
IncludedReact performance optimization and best practices. ALWAYS use this skill when working with any React code - writing components, hooks, JSX; refactoring; optimizing re-renders, memoization, state management; reviewing for performance; fixing hydration mismatches; debugging infinite re-renders, stale closures, input focus loss, animations restarting; preventing remounting; implementing transitions, lazy initialization, effect dependencies. Even simple React tasks benefit from these patterns. Covers React 19+ (useEffectEvent, Activity, ref props). Triggers - useEffect, useState, useMemo, useCallback, memo, inline components, nested components, components inside components, re-render, performance, hydration, SSR, Next.js, useDeferredValue, combined hooks.
elevenlabs-agents
IncludedBuild conversational AI voice agents with ElevenLabs Platform using React, JavaScript, React Native, or Swift SDKs. Configure agents, tools (client/server/MCP), RAG knowledge bases, multi-voice, and Scribe real-time STT. Use when: building voice chat interfaces, implementing AI phone agents with Twilio, configuring agent workflows or tools, adding RAG knowledge bases, testing with CLI "agents as code", or troubleshooting deprecated @11labs packages, Android audio cutoff, CSP violations, dynamic variables, or WebRTC config. Keywords: ElevenLabs Agents, ElevenLabs voice agents, AI voice agents, conversational AI, @elevenlabs/react, @elevenlabs/client, @elevenlabs/react-native, @elevenlabs/elevenlabs-js, @elevenlabs/agents-cli, elevenlabs SDK, voice AI, TTS, text-to-speech, ASR, speech recognition, turn-taking model, WebRTC voice, WebSocket voice, ElevenLabs conversation, agent system prompt, agent tools, agent knowledge base, RAG voice agents, multi-voice agents, pronunciation dictionary, voice speed control, elevenlabs scribe, @11labs deprecated, Android audio cutoff, CSP violation elevenlabs, dynamic variables elevenlabs, case-sensitive tool names, webhook authentication
humanizer
IncludedHumanize AI-generated text by detecting and removing patterns typical of LLM output. Rewrites text to sound natural, specific, and human. Uses 28 pattern detectors, 560+ AI vocabulary terms across 3 tiers, and statistical analysis (burstiness, type-token ratio, readability) for comprehensive detection. Use when asked to humanize text, de-AI writing, make content sound more natural/human, review writing for AI patterns, score text for AI detection, or improve AI-generated drafts. Covers content, language, style, communication, and filler categories.
generating-mermaid-diagrams
IncludedSalesforce architecture diagrams using Mermaid with ASCII fallback. Use this skill when generating text-based diagrams for Salesforce architecture, OAuth flows, ERDs, integration sequences, or Agentforce structure. TRIGGER when: user says "diagram", "visualize", "ERD", or asks for sequence diagrams, flowcharts, class diagrams, or architecture visualizations in Mermaid. DO NOT TRIGGER when: user wants PNG/SVG image output (use generating-visual-diagrams), or asks about non-Salesforce systems.