speak-sdk-patterns

Included with Lifetime

$97 forever

Production patterns for Speak language learning API: conversation sessions, pronunciation assessment, audio preprocessing, and batch operations. Use when implementing sdk patterns features, or troubleshooting Speak language learning integration issues. Trigger with phrases like "speak sdk patterns", "speak sdk patterns".

Image & Videosaasspeakapi

What this skill does

# Speak SDK Patterns

## Overview

Production patterns for Speak language learning API: conversation sessions, pronunciation assessment, audio preprocessing, and batch operations.

## Prerequisites

- Completed `speak-install-auth` setup
- Valid API credentials configured
- ffmpeg installed for audio processing

## Instructions

### Pattern 1: Conversation Session Manager

```typescript
class ConversationManager {
  private client: SpeakClient;
  private sessions: Map<string, SessionState> = new Map();

  async startLesson(language: string, scenario: string, level: string) {
    const session = await this.client.startConversation({
      scenario, language, level, nativeLanguage: 'en',
    });
    this.sessions.set(session.id, {
      turns: [], startTime: Date.now(), language,
    });
    return session;
  }

  async submitResponse(sessionId: string, audioPath: string) {
    const turn = await this.client.sendTurn(sessionId, { audioPath });
    this.sessions.get(sessionId)?.turns.push(turn);
    return turn;
  }

  async endAndReport(sessionId: string) {
    const summary = await this.client.endSession(sessionId);
    const state = this.sessions.get(sessionId)!;
    return {
      ...summary,
      duration: (Date.now() - state.startTime) / 1000,
      totalTurns: state.turns.length,
      avgPronunciation: state.turns.reduce((s, t) =>
        s + (t.pronunciationScore || 0), 0) / state.turns.length,
    };
  }
}
```

### Pattern 2: Audio Preprocessor

```typescript
import { execSync } from 'child_process';

function preprocessAudio(inputPath: string): string {
  const outputPath = inputPath.replace(/\.[^.]+$/, '.processed.wav');
  // Convert to WAV 16kHz mono PCM — required by Speak API
  execSync(
    `ffmpeg -y -i "${inputPath}" -ar 16000 -ac 1 -c:a pcm_s16le "${outputPath}"`,
    { stdio: 'pipe' }
  );
  return outputPath;
}
```

### Pattern 3: Retry with Backoff

```typescript
async function withRetry<T>(fn: () => Promise<T>, maxRetries = 3): Promise<T> {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await fn();
    } catch (err: any) {
      if (err.response?.status === 429 && i < maxRetries - 1) {
        const wait = parseInt(err.response.headers['retry-after'] || '5');
        await new Promise(r => setTimeout(r, wait * 1000));
        continue;
      }
      throw err;
    }
  }
  throw new Error('Max retries exceeded');
}
```

### Pattern 4: Progress Tracker

```typescript
class LearningProgress {
  private history: SessionSummary[] = [];

  addSession(summary: SessionSummary) {
    this.history.push(summary);
  }

  getReport() {
    const recent = this.history.slice(-10);
    return {
      totalSessions: this.history.length,
      avgPronunciation: recent.reduce((s, h) => s + h.avgPronunciationScore, 0) / recent.length,
      totalMinutes: this.history.reduce((s, h) => s + h.durationMinutes, 0),
      vocabularyLearned: [...new Set(this.history.flatMap(h => h.newWords))].length,
    };
  }
}
```

## Output

- Patterns implementation complete
- Speak API integration verified
- Production-ready patterns applied

## Error Handling

| Error | Cause | Solution |
|-------|-------|----------|
| 401 Unauthorized | Invalid API key | Verify SPEAK_API_KEY environment variable |
| 429 Rate Limited | Too many requests | Wait Retry-After seconds, use backoff |
| Audio format error | Wrong codec/sample rate | Convert to WAV 16kHz mono with ffmpeg |
| Session expired | Timeout after 30 min | Start a new conversation session |

## Resources

- [Speak Website](https://speak.com)
- [OpenAI Realtime API](https://platform.openai.com/docs/guides/realtime)
- [Speak GPT-4 Blog](https://speak.com/blog/speak-gpt-4)

## Next Steps

See `speak-prod-checklist` for production readiness.

## Examples

**Basic**: Apply sdk patterns with default configuration for a standard Speak integration.

**Advanced**: Customize for production with error recovery, monitoring, and team-specific requirements.

Files: 2

Size: 7.6 KB

Complexity: 27/100

Category: Image & Video

Source: https://github.com/jeremylongshore/claude-code-plugins-plus-skills/tree/main/plugins/saas-packs/speak-pack/skills/speak-sdk-patterns

Related in Image & Video

watch

Included

Watch a video (URL or local path). Downloads with yt-dlp, extracts auto-scaled frames with ffmpeg, pulls the transcript from captions (or Whisper API fallback), and hands the result to Claude so it can answer questions about what's in the video.

Image & Videoscriptsfeatured

physical-ai-defect-image-generation

Included

Use when the user wants to orchestrate defect image generation, run associated setup, or handle outputs on OSMO. The Day 0 path handles cold-start with USD-to-ROI, image-edit augmentation, and AnomalyGen to create initial PCBA datasets. The Day 1 path performs inference and labeling on real images. This skill helps with first-time asset setup, creation of finetuning checkpoints, and configuring deployment. Trigger keywords: defect image generation, dig workflow, dig pipeline, defect image detection workflow, aoi pipeline, aoi anomalygen, usd2roi anomalygen, day 0 pcba, day 1 pcba, day 1 real-photo alignment, day 1 manual roi, metal surface anomaly, glass defect, anomalygen finetune, setup_pcb, setup_metal, setup_glass, setup_pretrained, dig setup, dig datasets, dig pretrained checkpoint, dig image-edit endpoint.

Image & Videoscripts

accelint-react-best-practices

Included

React performance optimization and best practices. ALWAYS use this skill when working with any React code - writing components, hooks, JSX; refactoring; optimizing re-renders, memoization, state management; reviewing for performance; fixing hydration mismatches; debugging infinite re-renders, stale closures, input focus loss, animations restarting; preventing remounting; implementing transitions, lazy initialization, effect dependencies. Even simple React tasks benefit from these patterns. Covers React 19+ (useEffectEvent, Activity, ref props). Triggers - useEffect, useState, useMemo, useCallback, memo, inline components, nested components, components inside components, re-render, performance, hydration, SSR, Next.js, useDeferredValue, combined hooks.

Image & Videoscripts

elevenlabs-agents

Included

Build conversational AI voice agents with ElevenLabs Platform using React, JavaScript, React Native, or Swift SDKs. Configure agents, tools (client/server/MCP), RAG knowledge bases, multi-voice, and Scribe real-time STT. Use when: building voice chat interfaces, implementing AI phone agents with Twilio, configuring agent workflows or tools, adding RAG knowledge bases, testing with CLI "agents as code", or troubleshooting deprecated @11labs packages, Android audio cutoff, CSP violations, dynamic variables, or WebRTC config. Keywords: ElevenLabs Agents, ElevenLabs voice agents, AI voice agents, conversational AI, @elevenlabs/react, @elevenlabs/client, @elevenlabs/react-native, @elevenlabs/elevenlabs-js, @elevenlabs/agents-cli, elevenlabs SDK, voice AI, TTS, text-to-speech, ASR, speech recognition, turn-taking model, WebRTC voice, WebSocket voice, ElevenLabs conversation, agent system prompt, agent tools, agent knowledge base, RAG voice agents, multi-voice agents, pronunciation dictionary, voice speed control, elevenlabs scribe, @11labs deprecated, Android audio cutoff, CSP violation elevenlabs, dynamic variables elevenlabs, case-sensitive tool names, webhook authentication

Image & Videoscripts

humanizer

Included

Humanize AI-generated text by detecting and removing patterns typical of LLM output. Rewrites text to sound natural, specific, and human. Uses 28 pattern detectors, 560+ AI vocabulary terms across 3 tiers, and statistical analysis (burstiness, type-token ratio, readability) for comprehensive detection. Use when asked to humanize text, de-AI writing, make content sound more natural/human, review writing for AI patterns, score text for AI detection, or improve AI-generated drafts. Covers content, language, style, communication, and filler categories.

Image & Videoscripts

generating-mermaid-diagrams

Included

Salesforce architecture diagrams using Mermaid with ASCII fallback. Use this skill when generating text-based diagrams for Salesforce architecture, OAuth flows, ERDs, integration sequences, or Agentforce structure. TRIGGER when: user says "diagram", "visualize", "ERD", or asks for sequence diagrams, flowcharts, class diagrams, or architecture visualizations in Mermaid. DO NOT TRIGGER when: user wants PNG/SVG image output (use generating-visual-diagrams), or asks about non-Salesforce systems.

Image & Videoscripts