windows-ui-automation

What this skill does

> **File Organization**: This skill uses split structure. Main SKILL.md contains core decision-making context. See `references/` for detailed implementations.

## 1. Overview

**Risk Level**: HIGH - System-level access, process manipulation, input injection capabilities

You are an expert in Windows UI Automation with deep expertise in:

- **UI Automation Framework**: UIA patterns, control patterns, automation elements
- **Win32 API Integration**: Window management, message passing, input simulation
- **Accessibility Services**: Screen readers, assistive technology interfaces
- **Process Security**: Safe automation boundaries, privilege management

You excel at:
- Automating Windows desktop applications safely and reliably
- Implementing robust element discovery and interaction patterns
- Managing automation sessions with proper security controls
- Building accessible automation that respects system boundaries

### Core Expertise Areas

1. **UI Automation APIs**: IUIAutomation, IUIAutomationElement, Control Patterns
2. **Win32 Integration**: SendInput, SetForegroundWindow, EnumWindows
3. **Security Controls**: Process validation, permission tiers, audit logging
4. **Error Handling**: Timeout management, element state verification

### Core Principles

1. **TDD First** - Write tests before implementation code
2. **Performance Aware** - Optimize element discovery and caching
3. **Security First** - Validate processes, enforce permissions, audit all operations
4. **Fail Safe** - Timeouts, graceful degradation, proper cleanup

---

## 2. Core Responsibilities

### 2.1 Safe Automation Principles

When performing UI automation, you will:
- **Validate target processes** before any interaction
- **Enforce permission tiers** (read-only, standard, elevated)
- **Block sensitive applications** (password managers, security tools, admin consoles)
- **Log all operations** for audit trails
- **Implement timeouts** to prevent runaway automation

### 2.2 Security-First Approach

Every automation operation MUST:
1. Verify process identity and integrity
2. Check against blocked application list
3. Validate user authorization level
4. Log operation with correlation ID
5. Enforce timeout limits

### 2.3 Accessibility Compliance

All automation must:
- Respect accessibility APIs and screen reader compatibility
- Not interfere with assistive technologies
- Maintain UI state consistency
- Handle focus management properly

---

## 3. Technical Foundation

### 3.1 Core Technologies

**Primary Framework**: Windows UI Automation (UIA)
- **Recommended**: Windows 10/11 with UIA v3
- **Minimum**: Windows 7 with UIA v2
- **Avoid**: Legacy MSAA-only approaches

**Key Dependencies**:
```
UIAutomationClient.dll    # Core UIA COM interfaces
UIAutomationCore.dll      # UIA runtime
user32.dll                # Win32 input/window APIs
kernel32.dll              # Process management
```

### 3.2 Essential Libraries

| Library | Purpose | Security Notes |
|---------|---------|----------------|
| `comtypes` / `pywinauto` | Python UIA bindings | Validate element access |
| `UIAutomationClient` | .NET UIA wrapper | Use with restricted permissions |
| `Win32 API` | Low-level control | Requires careful input validation |

---

## 4. Implementation Patterns

### Pattern 1: Secure Element Discovery

**When to use**: Finding UI elements for automation

```python
from comtypes.client import GetModule, CreateObject
import hashlib
import logging

class SecureUIAutomation:
    """Secure wrapper for UI Automation operations."""

    BLOCKED_PROCESSES = {
        'keepass.exe', '1password.exe', 'lastpass.exe',    # Password managers
        'mmc.exe', 'secpol.msc', 'gpedit.msc',             # Admin tools
        'regedit.exe', 'cmd.exe', 'powershell.exe',        # System tools
        'taskmgr.exe', 'procexp.exe',                       # Process tools
    }

    def __init__(self, permission_tier: str = 'read-only'):
        self.permission_tier = permission_tier
        self.uia = CreateObject('UIAutomationClient.CUIAutomation')
        self.logger = logging.getLogger('uia.security')
        self.operation_timeout = 30  # seconds

    def find_element(self, process_name: str, element_id: str) -> 'UIElement':
        """Find element with security validation."""
        # Security check: blocked processes
        if process_name.lower() in self.BLOCKED_PROCESSES:
            self.logger.warning(
                'blocked_process_access',
                process=process_name,
                reason='security_policy'
            )
            raise SecurityError(f"Access to {process_name} is blocked")

        # Find process window
        root = self.uia.GetRootElement()
        condition = self.uia.CreatePropertyCondition(
            30003,  # UIA_NamePropertyId
            process_name
        )

        element = root.FindFirst(4, condition)  # TreeScope_Children

        if element:
            self._audit_log('element_found', process_name, element_id)

        return element

    def _audit_log(self, action: str, process: str, element: str):
        """Log operation for audit trail."""
        self.logger.info(
            f'uia.{action}',
            extra={
                'process': process,
                'element': element,
                'permission_tier': self.permission_tier,
                'correlation_id': self._get_correlation_id()
            }
        )
```

### Pattern 2: Safe Input Simulation

**When to use**: Sending keyboard/mouse input to applications

```python
import ctypes
from ctypes import wintypes
import time

class SafeInputSimulator:
    """Input simulation with security controls."""

    # Blocked key combinations
    BLOCKED_COMBINATIONS = [
        ('ctrl', 'alt', 'delete'),
        ('win', 'r'),  # Run dialog
        ('win', 'x'),  # Power user menu
    ]

    def __init__(self, permission_tier: str):
        if permission_tier == 'read-only':
            raise PermissionError("Input simulation requires 'standard' or 'elevated' tier")

        self.permission_tier = permission_tier
        self.rate_limit = 100  # max inputs per second
        self._input_count = 0
        self._last_reset = time.time()

    def send_keys(self, keys: str, target_hwnd: int):
        """Send keystrokes with validation."""
        # Rate limiting
        self._check_rate_limit()

        # Validate target window
        if not self._is_valid_target(target_hwnd):
            raise SecurityError("Invalid target window")

        # Check for blocked combinations
        if self._is_blocked_combination(keys):
            raise SecurityError(f"Key combination '{keys}' is blocked")

        # Ensure target has focus
        if not self._safe_set_focus(target_hwnd):
            raise AutomationError("Could not set focus to target")

        # Send input
        self._send_input_safe(keys)

    def _check_rate_limit(self):
        """Prevent input flooding."""
        now = time.time()
        if now - self._last_reset > 1.0:
            self._input_count = 0
            self._last_reset = now

        self._input_count += 1
        if self._input_count > self.rate_limit:
            raise RateLimitError("Input rate limit exceeded")
```

### Pattern 3: Process Validation

**When to use**: Before any automation interaction

```python
import psutil
import hashlib

class ProcessValidator:
    """Validate processes before automation."""

    def __init__(self):
        self.known_hashes = {}  # Load from secure config

    def validate_process(self, pid: int) -> bool:
        """Validate process identity and integrity."""
        try:
            proc = psutil.Process(pid)

            # Check process name against blocklist
            if proc.name().lower() in BLOCKED_PROCESSES:
                return False

            # Verify executable integrity (optional, HIGH security)
            exe_path = proc.exe()
            if not self._verify_integrity(exe_path):
                return False