opencv

What this skill does

# OpenCV - Computer Vision and Image Processing

OpenCV (Open Source Computer Vision Library) is the de facto standard library for computer vision tasks. It provides 2500+ optimized algorithms for real-time image and video processing, from basic operations like reading images to advanced tasks like face recognition and 3D reconstruction.

## When to Use

- Reading, writing, and displaying images and videos from files or cameras.
- Image preprocessing (resizing, cropping, rotating, color conversion).
- Edge detection (Canny, Sobel) and contour finding.
- Feature detection and matching (SIFT, ORB, AKAZE).
- Object detection (Haar Cascades, HOG, DNN module for YOLO/SSD).
- Face detection and recognition.
- Image segmentation (thresholding, watershed, GrabCut).
- Video analysis (motion detection, object tracking, optical flow).
- Camera calibration and 3D reconstruction.
- Image stitching and panorama creation.
- Real-time applications requiring fast performance.

## Reference Documentation

**Official docs**: https://docs.opencv.org/4.x/  
**GitHub**: https://github.com/opencv/opencv  
**Tutorials**: https://docs.opencv.org/4.x/d9/df8/tutorial_root.html  
**Search patterns**: `cv2.imread`, `cv2.cvtColor`, `cv2.Canny`, `cv2.findContours`, `cv2.VideoCapture`

## Core Principles

### Image as NumPy Array
OpenCV represents images as NumPy arrays with shape (height, width, channels). This allows seamless integration with NumPy operations and other scientific Python libraries.

### BGR Color Space (Not RGB!)
OpenCV uses BGR (Blue-Green-Red) instead of RGB by default. This is critical to remember when displaying images or integrating with other libraries.

### In-Place vs Copy Operations
Many OpenCV functions modify images in-place for performance. Understanding when copies are made is essential for efficient code.

### C++ Performance in Python
OpenCV is written in optimized C++, making it extremely fast even when called from Python. Avoid Python loops when OpenCV vectorized operations exist.

## Quick Reference

### Installation

```bash
# Basic OpenCV
pip install opencv-python

# With contrib modules (SIFT, SURF, etc.)
pip install opencv-contrib-python

# Headless (no GUI, for servers)
pip install opencv-python-headless
```

### Standard Imports

```python
import cv2
import numpy as np
import matplotlib.pyplot as plt
```

### Basic Pattern - Read, Process, Display

```python
import cv2

# 1. Read image
img = cv2.imread('image.jpg')

# 2. Process (convert to grayscale)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# 3. Display
cv2.imshow('Grayscale', gray)
cv2.waitKey(0)  # Wait for key press
cv2.destroyAllWindows()
```

### Basic Pattern - Video Processing

```python
import cv2

# 1. Open video capture
cap = cv2.VideoCapture(0)  # 0 = default camera, or 'video.mp4'

while True:
    # 2. Read frame
    ret, frame = cap.read()
    if not ret:
        break
    
    # 3. Process frame
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    
    # 4. Display
    cv2.imshow('Video', gray)
    
    # 5. Exit on 'q' key
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# 6. Cleanup
cap.release()
cv2.destroyAllWindows()
```

## Critical Rules

### ✅ DO

- **Check Image Loaded** - Always verify `img is not None` after `cv2.imread()` to catch file errors.
- **Use cv2.cvtColor() for Color Conversion** - Don't manually rearrange channels; use the provided conversion codes.
- **Release Resources** - Always call `cap.release()` and `cv2.destroyAllWindows()` when done with video/windows.
- **Copy Before Modifying** - Use `img.copy()` if you need to preserve the original image.
- **Use Appropriate Data Types** - Keep images as uint8 (0-255) for display, convert to float32 (0-1) for mathematical operations.
- **Validate VideoCapture** - Check `cap.isOpened()` before reading frames.
- **Use BGR2RGB for Matplotlib** - Convert BGR to RGB when displaying with matplotlib.
- **Vectorize Operations** - Use OpenCV's built-in functions instead of Python loops over pixels.

### ❌ DON'T

- **Don't Assume RGB** - OpenCV uses BGR by default; convert to RGB for matplotlib or PIL.
- **Don't Forget waitKey()** - Without `cv2.waitKey()`, windows won't display properly.
- **Don't Mix PIL and OpenCV Directly** - Convert between them explicitly (OpenCV uses BGR, PIL uses RGB).
- **Don't Process Video in Memory** - Process frame-by-frame to avoid memory issues with large videos.
- **Don't Use Python Loops for Pixels** - This is 100x slower than vectorized operations.
- **Don't Hardcode Paths** - Use `os.path.join()` or `pathlib` for cross-platform compatibility.

## Anti-Patterns (NEVER)

```python
import cv2
import numpy as np

# ❌ BAD: Not checking if image loaded
img = cv2.imread('image.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)  # Crashes if file doesn't exist!

# ✅ GOOD: Always validate
img = cv2.imread('image.jpg')
if img is None:
    raise FileNotFoundError("Image not found")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# ❌ BAD: Using Python loops for pixel manipulation
for i in range(img.shape[0]):
    for j in range(img.shape[1]):
        img[i, j] = img[i, j] * 0.5  # Extremely slow!

# ✅ GOOD: Vectorized NumPy operations
img = (img * 0.5).astype(np.uint8)

# ❌ BAD: Displaying BGR image with matplotlib
plt.imshow(img)  # Colors will be wrong!

# ✅ GOOD: Convert to RGB first
img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
plt.imshow(img_rgb)

# ❌ BAD: Not releasing video capture
cap = cv2.VideoCapture('video.mp4')
while cap.read()[0]:
    pass
# Memory leak! Camera still locked!

# ✅ GOOD: Always release
cap = cv2.VideoCapture('video.mp4')
try:
    while cap.read()[0]:
        pass
finally:
    cap.release()
```

## Image I/O and Display

### Reading and Writing Images

```python
import cv2

# Read image (returns None if failed)
img = cv2.imread('image.jpg')

# Read as grayscale
gray = cv2.imread('image.jpg', cv2.IMREAD_GRAYSCALE)

# Read with alpha channel
img_alpha = cv2.imread('image.png', cv2.IMREAD_UNCHANGED)

# Write image
cv2.imwrite('output.jpg', img)

# Write with quality (JPEG: 0-100, PNG: 0-9 compression)
cv2.imwrite('output.jpg', img, [cv2.IMWRITE_JPEG_QUALITY, 95])
cv2.imwrite('output.png', img, [cv2.IMWRITE_PNG_COMPRESSION, 9])

# Check if image loaded
if img is None:
    print("Error: Could not load image")
else:
    print(f"Image shape: {img.shape}")  # (height, width, channels)
```

### Display Images

```python
import cv2

# Display image in window
cv2.imshow('Window Name', img)
cv2.waitKey(0)  # Wait indefinitely for key press
cv2.destroyAllWindows()

# Display for specific duration (milliseconds)
cv2.imshow('Image', img)
cv2.waitKey(3000)  # Wait 3 seconds
cv2.destroyAllWindows()

# Display multiple images
cv2.imshow('Original', img)
cv2.imshow('Gray', gray)
cv2.waitKey(0)
cv2.destroyAllWindows()

# Display with matplotlib (convert BGR to RGB!)
import matplotlib.pyplot as plt

img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
plt.imshow(img_rgb)
plt.axis('off')
plt.show()
```

### Video Capture

```python
import cv2

# Open camera (0 = default, 1 = second camera, etc.)
cap = cv2.VideoCapture(0)

# Open video file
cap = cv2.VideoCapture('video.mp4')

# Check if opened successfully
if not cap.isOpened():
    print("Error: Could not open video")
    exit()

# Get video properties
fps = cap.get(cv2.CAP_PROP_FPS)
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))

print(f"Video: {width}x{height} @ {fps} fps, {total_frames} frames")

# Read and process frames
while True:
    ret, frame = cap.read()
    
    if not ret:
        print("End of video or error")
        break
    
    # Process frame here
    cv2.imshow('Frame', frame)
    
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()
```

### Writing Videos

```python
import cv2

cap = cv2.VideoCapture('input.mp4')

# Get video properties
fps = int(cap.get(cv2.CAP_P