ert

What this skill does

# ERT: Emacs Lisp Regression Testing

ERT is Emacs's built-in testing framework for automated testing of Emacs Lisp code. It provides facilities for defining tests, running them interactively or in batch mode, and debugging failures with integrated tooling.

## Overview

ERT (Emacs Lisp Regression Testing) is included with Emacs and requires no additional installation. It leverages Emacs's dynamic and interactive nature to provide powerful testing capabilities for unit tests, integration tests, and regression prevention.

**Key Characteristics:**
- Built into Emacs (available in Emacs 24+)
- Interactive debugging with backtrace inspection
- Flexible test selection and organization
- Batch mode for CI/CD integration
- Dynamic binding for easy mocking
- No external dependencies

## Core Concepts

### Test Definition

Tests are defined using `ert-deftest`, which creates a named test function:

```elisp
(ert-deftest test-name ()
  "Docstring describing what the test verifies."
  (should (= 2 (+ 1 1))))
```

### Assertions (Should Forms)

ERT provides three assertion macros:

- `should` - Assert that a form evaluates to non-nil
- `should-not` - Assert that a form evaluates to nil
- `should-error` - Assert that a form signals an error

Unlike `cl-assert`, these macros provide detailed error reporting including the form, evaluated subexpressions, and resulting values.

### Test Selectors

Selectors specify which tests to run:

- `t` - All tests
- `"regex"` - Tests matching regular expression
- `:tag symbol` - Tests tagged with symbol
- `:failed` - Tests that failed in last run
- `:passed` - Tests that passed in last run
- Combinations using `(and ...)`, `(or ...)`, `(not ...)`

## API Reference

### Defining Tests

#### `ert-deftest`
```elisp
(ert-deftest NAME () [DOCSTRING] [:tags (TAG...)] BODY...)
```

Define a test named NAME.

**Parameters:**
- `NAME` - Symbol naming the test
- `DOCSTRING` - Optional description of what the test verifies
- `:tags` - Optional list of tags for test organization
- `BODY` - Test code containing assertions

**Example:**
```elisp
(ert-deftest test-addition ()
  "Test basic arithmetic addition."
  :tags '(arithmetic quick)
  (should (= 4 (+ 2 2)))
  (should (= 0 (+ -1 1))))
```

### Assertion Macros

#### `should`
```elisp
(should FORM)
```

Assert that FORM evaluates to non-nil. On failure, displays the form and all evaluated subexpressions.

**Example:**
```elisp
(should (string-match "foo" "foobar"))
(should (< 1 2))
(should (member 'x '(x y z)))
```

#### `should-not`
```elisp
(should-not FORM)
```

Assert that FORM evaluates to nil.

**Example:**
```elisp
(should-not (string-match "baz" "foobar"))
(should-not (> 1 2))
```

#### `should-error`
```elisp
(should-error FORM [:type TYPE])
```

Assert that FORM signals an error. Optional `:type` specifies the expected error type.

**Example:**
```elisp
;; Any error accepted
(should-error (/ 1 0))

;; Specific error type required
(should-error (/ 1 0) :type 'arith-error)

;; Wrong error type would fail
(should-error (error "message") :type 'arith-error)  ; fails
```

### Running Tests Interactively

#### `ert` / `ert-run-tests-interactively`
```elisp
M-x ert RET SELECTOR RET
```

Run tests matching SELECTOR and display results in interactive buffer.

**Common selectors:**
- `t` - Run all tests
- `"^my-package-"` - Tests matching regex
- `:failed` - Re-run failed tests

**Interactive debugging commands:**
- `.` - Jump to test definition
- `d` - Re-run test with debugger enabled
- `b` - Show backtrace of failed test
- `r` - Re-run test at point
- `R` - Re-run all tests
- `l` - Show executed `should` forms
- `m` - Show messages from test
- `TAB` - Expand/collapse test details

### Running Tests in Batch Mode

#### `ert-run-tests-batch-and-exit`
```elisp
(ert-run-tests-batch-and-exit [SELECTOR])
```

Run tests in batch mode and exit with status code (0 for success, non-zero for failure).

**Command line usage:**
```bash
# Run all tests
emacs -batch -l ert -l my-tests.el -f ert-run-tests-batch-and-exit

# Run specific tests
emacs -batch -l ert -l my-tests.el \
  --eval '(ert-run-tests-batch-and-exit "^test-feature-")'

# Quiet mode (only unexpected results)
emacs -batch -l ert -l my-tests.el \
  --eval '(let ((ert-quiet t)) (ert-run-tests-batch-and-exit))'
```

#### `ert-run-tests-batch`
```elisp
(ert-run-tests-batch [SELECTOR])
```

Run tests in batch mode but do not exit. Useful when running tests is part of a larger batch script.

### Test Organization

#### Tags

Use `:tags` in `ert-deftest` to organize tests:

```elisp
(ert-deftest test-fast-operation ()
  :tags '(quick unit)
  (should (fast-function)))

(ert-deftest test-slow-integration ()
  :tags '(slow integration)
  (should (slow-integration-test)))
```

Run tagged tests:
```elisp
;; Run only quick tests
M-x ert RET :tag quick RET

;; Run integration tests
M-x ert RET :tag integration RET
```

#### Test Naming Conventions

Prefix test names with the package name:

```elisp
(ert-deftest my-package-test-feature ()
  "Test feature implementation."
  ...)

(ert-deftest my-package-test-edge-case ()
  "Test handling of edge case."
  ...)
```

This enables:
- Running all package tests: `M-x ert RET "^my-package-" RET`
- Clear test ownership and organization
- Avoiding name collisions

### Skipping Tests

#### `skip-unless`
```elisp
(skip-unless CONDITION)
```

Skip test if CONDITION is nil.

**Example:**
```elisp
(ert-deftest test-graphical-feature ()
  "Test feature requiring graphical display."
  (skip-unless (display-graphic-p))
  (should (graphical-operation)))
```

#### `skip-when`
```elisp
(skip-when CONDITION)
```

Skip test if CONDITION is non-nil.

**Example:**
```elisp
(ert-deftest test-unix-specific ()
  "Test Unix-specific functionality."
  (skip-when (eq system-type 'windows-nt))
  (should (unix-specific-function)))
```

### Programmatic Test Execution

#### `ert-run-tests`
```elisp
(ert-run-tests SELECTOR LISTENER)
```

Run tests matching SELECTOR, reporting results to LISTENER.

**Example:**
```elisp
;; Run all tests silently
(ert-run-tests t #'ert-quiet-listener)

;; Custom listener
(defun my-listener (event-type &rest data)
  (pcase event-type
    ('test-started ...)
    ('test-passed ...)
    ('test-failed ...)
    ('run-ended ...)))

(ert-run-tests "^my-" #'my-listener)
```

## Best Practices

### Test Structure

**1. Use descriptive test names:**

```elisp
;; Good
(ert-deftest my-package-parse-valid-json ()
  ...)

;; Poor
(ert-deftest test1 ()
  ...)
```

**2. Include clear docstrings:**

```elisp
(ert-deftest my-package-handle-empty-input ()
  "Verify that empty input returns nil without error."
  (should-not (my-package-process "")))
```

**3. One logical assertion per test:**

```elisp
;; Good - focused test
(ert-deftest my-package-parse-returns-alist ()
  "Parser returns result as alist."
  (should (listp (my-package-parse "data")))
  (should (eq 'cons (type-of (car (my-package-parse "data"))))))

;; Better - even more focused
(ert-deftest my-package-parse-returns-list ()
  "Parser returns a list."
  (should (listp (my-package-parse "data"))))

(ert-deftest my-package-parse-list-contains-alist-entries ()
  "Parser list contains alist entries."
  (should (eq 'cons (type-of (car (my-package-parse "data"))))))
```

### Test Environment

**1. Isolate tests from environment:**

```elisp
(ert-deftest my-package-test-configuration ()
  "Test respects custom configuration."
  ;; Save and restore configuration
  (let ((my-package-option 'custom-value))
    (should (eq 'custom-value (my-package-get-option)))))
```

**2. Use temporary buffers:**

```elisp
(ert-deftest my-package-buffer-manipulation ()
  "Test buffer manipulation functions."
  (with-temp-buffer
    (insert "test content")
    (my-package-process-buffer)
    (should (string= (buffer-string) "processed content"))))
```

**3. Clean up with `unwind-protect`:**

```elisp
(ert-deftest my-package-file-operation ()
  "Test file operations with cleanup."