whatsapp-voice-talk

What this skill does

# WhatsApp Voice Talk

Turn WhatsApp voice messages into real-time conversations. This skill provides a complete pipeline: **voice → transcription → intent detection → response generation → text-to-speech**.

Perfect for:
- Voice assistants on WhatsApp
- Hands-free command interfaces  
- Multi-lingual chatbots
- IoT voice control (drones, smart home, etc.)

## Quick Start

### 1. Install Dependencies

```bash
pip install openai-whisper soundfile numpy
```

### 2. Process a Voice Message

```javascript
const { processVoiceNote } = require('./scripts/voice-processor');
const fs = require('fs');

// Read a voice message (OGG, WAV, MP3, etc.)
const buffer = fs.readFileSync('voice-message.ogg');

// Process it
const result = await processVoiceNote(buffer);

console.log(result);
// {
//   status: 'success',
//   response: "Current weather in Delhi is 19°C, haze. Humidity is 56%.",
//   transcript: "What's the weather today?",
//   intent: 'weather',
//   language: 'en',
//   timestamp: 1769860205186
// }
```

### 3. Run Auto-Listener

For automatic processing of incoming WhatsApp voice messages:

```bash
node scripts/voice-listener-daemon.js
```

This watches `~/.clawdbot/media/inbound/` every 5 seconds and processes new voice files.

## How It Works

```
Incoming Voice Message
        ↓
    Transcribe (Whisper API)
        ↓
  "What's the weather?"
        ↓
  Detect Language & Intent
        ↓
   Match against INTENTS
        ↓
   Execute Handler
        ↓
   Generate Response
        ↓
   Convert to TTS
        ↓
  Send back via WhatsApp
```

## Key Features

✅ **Zero Setup Complexity** - No FFmpeg, no complex dependencies. Uses soundfile + Whisper.

✅ **Multi-Language** - Automatic English/Hindi detection. Extend easily.

✅ **Intent-Driven** - Define custom intents with keywords and handlers.

✅ **Real-Time Processing** - 5-10 seconds per message (after first model load).

✅ **Customizable** - Add weather, status, commands, or anything else.

✅ **Production Ready** - Built from real usage in Clawdbot.

## Common Use Cases

### Weather Bot
```javascript
// User says: "What's the weather in Bangalore?"
// Response: "Current weather in Delhi is 19°C..."

// (Built-in intent, just enable it)
```

### Smart Home Control
```javascript
// User says: "Turn on the lights"
// Handler: Sends signal to smart home API
// Response: "Lights turned on"
```

### Task Manager
```javascript
// User says: "Add milk to shopping list"
// Handler: Adds to database
// Response: "Added milk to your list"
```

### Status Checker
```javascript
// User says: "Is the system running?"
// Handler: Checks system status
// Response: "All systems online"
```

## Customization

### Add a Custom Intent

Edit `voice-processor.js`:

1. **Add to INTENTS map:**
```javascript
const INTENTS = {
  'shopping': {
    keywords: ['shopping', 'list', 'buy', 'खरीद'],
    handler: 'handleShopping'
  }
};
```

2. **Add handler:**
```javascript
const handlers = {
  async handleShopping(language = 'en') {
    return {
      status: 'success',
      response: language === 'en' 
        ? "What would you like to add to your shopping list?"
        : "आप अपनी शॉपिंग लिस्ट में क्या जोड़ना चाहते हैं?"
    };
  }
};
```

### Support More Languages

1. Update `detectLanguage()` for your language's Unicode:
```javascript
const urduChars = /[\u0600-\u06FF]/g; // Add this
```

2. Add language code to returns:
```javascript
return language === 'ur' ? 'Urdu response' : 'English response';
```

3. Set language in `transcribe.py`:
```python
result = model.transcribe(data, language="ur")
```

### Change Transcription Model

In `transcribe.py`:
```python
model = whisper.load_model("tiny")    # Fastest, 39MB
model = whisper.load_model("base")    # Default, 140MB  
model = whisper.load_model("small")   # Better, 466MB
model = whisper.load_model("medium")  # Good, 1.5GB
```

## Architecture

**Scripts:**
- `transcribe.py` - Whisper transcription (Python)
- `voice-processor.js` - Core logic (intent parsing, handlers)
- `voice-listener-daemon.js` - Auto-listener watching for new messages

**References:**
- `SETUP.md` - Installation and configuration
- `API.md` - Detailed function documentation

## Integration with Clawdbot

If running as a Clawdbot skill, hook into message events:

```javascript
// In your Clawdbot handler
const { processVoiceNote } = require('skills/whatsapp-voice-talk/scripts/voice-processor');

message.on('voice', async (audioBuffer) => {
  const result = await processVoiceNote(audioBuffer, message.from);
  
  // Send response back
  await message.reply(result.response);
  
  // Or send as voice (requires TTS)
  await sendVoiceMessage(result.response);
});
```

## Performance

- **First run:** ~30 seconds (downloads Whisper model, ~140MB)
- **Typical:** 5-10 seconds per message
- **Memory:** ~1.5GB (base model)
- **Languages:** English, Hindi (easily extended)

## Supported Audio Formats

OGG (Opus), WAV, FLAC, MP3, CAF, AIFF, and more via libsndfile.

WhatsApp uses Opus-coded OGG by default — works out of the box.

## Troubleshooting

**"No module named 'whisper'"**
```bash
pip install openai-whisper
```

**"No module named 'soundfile'"**
```bash
pip install soundfile
```

**Voice messages not processing?**
1. Check: `clawdbot status` (is it running?)
2. Check: `~/.clawdbot/media/inbound/` (files arriving?)
3. Run daemon manually: `node scripts/voice-listener-daemon.js` (see logs)

**Slow transcription?**
Use smaller model: `whisper.load_model("base")` or `"tiny"`

## Further Reading

- **Setup Guide:** See `references/SETUP.md` for detailed installation and configuration
- **API Reference:** See `references/API.md` for function signatures and examples
- **Examples:** Check `scripts/` for working code

## License

MIT - Use freely, customize, contribute back!

---

Built for real-world use in Clawdbot. Battle-tested with multiple languages and use cases.