umbraco-e2e-testing

What this skill does

# Umbraco E2E Testing

End-to-end testing for Umbraco backoffice extensions using Playwright and `@umbraco/playwright-testhelpers`. This approach tests against a real running Umbraco instance, validating complete user workflows.

## Critical: Use Testhelpers for Core Umbraco

Use `@umbraco/playwright-testhelpers` for **core Umbraco operations**:

| Package | Purpose | Why Required |
|---------|---------|--------------|
| `@umbraco/playwright-testhelpers` | UI and API helpers | Handles auth, navigation, core entity CRUD |
| `@umbraco/json-models-builders` | Test data builders | Creates valid Umbraco entities with correct structure |

**Why use testhelpers for core Umbraco?**
- Umbraco uses `data-mark` instead of `data-testid` - testhelpers handle this
- Auth token management is complex - testhelpers manage `STORAGE_STAGE_PATH`
- API setup/teardown requires specific payload formats - builders ensure correctness
- Selectors change between versions - testhelpers abstract these away

```typescript
// WRONG - Raw Playwright for core Umbraco (brittle)
await page.goto('/umbraco');
await page.fill('[name="email"]', '[email protected]');

// CORRECT - Testhelpers for core Umbraco
import { test } from '@umbraco/playwright-testhelpers';

test('my test', async ({ umbracoApi, umbracoUi }) => {
  await umbracoUi.goToBackOffice();
  await umbracoUi.login.enterEmail('[email protected]');
});
```

### When to Use Raw Playwright

For **custom extensions**, use `umbracoUi.page` (raw Playwright) because testhelpers don't know about your custom elements:

```typescript
test('my custom extension', async ({ umbracoUi }) => {
  // Testhelpers for core navigation
  await umbracoUi.goToBackOffice();
  await umbracoUi.content.goToSection(ConstantHelper.sections.settings);

  // Raw Playwright for YOUR custom elements
  await umbracoUi.page.getByRole('link', { name: 'My Custom Item' }).click();
  await expect(umbracoUi.page.locator('my-custom-workspace')).toBeVisible();
});
```

| Use Testhelpers For | Use `umbracoUi.page` For |
|---------------------|--------------------------|
| Login/logout | Custom tree items |
| Navigate to ANY section (including custom) | Custom workspace elements |
| Create/edit documents via API | Custom entity actions |
| Built-in UI interactions | Custom UI components |

## When to Use

- Testing complete user workflows
- Testing data persistence
- Testing authentication/authorization
- Acceptance testing before release
- Integration testing with real API responses

## Related Skills

- **umbraco-testing** - Master skill for testing overview
- **umbraco-playwright-testhelpers** - Full reference for the testhelpers package
- **umbraco-test-builders** - JsonModels.Builders for test data
- **umbraco-mocked-backoffice** - Test without real backend (faster)

## Documentation

- **Playwright**: https://playwright.dev/docs/intro
- **Reference tests**: `Umbraco-CMS/tests/Umbraco.Tests.AcceptanceTest`

---

## Setup

### Dependencies

Add to `package.json`:

```json
{
  "devDependencies": {
    "@playwright/test": "^1.56",
    "@umbraco/playwright-testhelpers": "^17.0.15",
    "@umbraco/json-models-builders": "^2.0.42",
    "dotenv": "^16.3.1"
  },
  "scripts": {
    "test:e2e": "playwright test",
    "test:e2e:ui": "playwright test --ui",
    "test:e2e:debug": "playwright test --debug"
  }
}
```

Then run:
```bash
npm install
npx playwright install chromium
```

**Version Compatibility**: Match testhelpers to your Umbraco version:
| Umbraco | Testhelpers |
|---------|-------------|
| 17.1.x (pre-release) | `17.1.0-beta.x` |
| 17.0.x | `^17.0.15` |
| 14.x | `^14.x` |

### Configuration

Create `playwright.config.ts`:

```typescript
import { defineConfig, devices } from '@playwright/test';
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);

export const STORAGE_STATE = join(__dirname, 'tests/e2e/.auth/user.json');

// CRITICAL: Testhelpers read auth tokens from this file
process.env.STORAGE_STAGE_PATH = STORAGE_STATE;

export default defineConfig({
  testDir: './tests/e2e',
  timeout: 30 * 1000,
  expect: { timeout: 5000 },
  fullyParallel: false,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: 1,
  reporter: process.env.CI ? 'line' : 'html',
  use: {
    baseURL: process.env.UMBRACO_URL || 'https://localhost:44325',
    trace: 'retain-on-failure',
    ignoreHTTPSErrors: true,
    // CRITICAL: Umbraco uses 'data-mark' not 'data-testid'
    testIdAttribute: 'data-mark',
  },
  projects: [
    {
      name: 'setup',
      testMatch: '**/*.setup.ts',
    },
    {
      name: 'e2e',
      testMatch: '**/*.spec.ts',
      dependencies: ['setup'],
      use: {
        ...devices['Desktop Chrome'],
        ignoreHTTPSErrors: true,
        storageState: STORAGE_STATE,
      },
    },
  ],
});
```

### Critical Settings

| Setting | Value | Why Required |
|---------|-------|--------------|
| `testIdAttribute` | `'data-mark'` | Umbraco uses `data-mark`, not `data-testid` |
| `STORAGE_STAGE_PATH` | Path to user.json | Testhelpers read auth tokens from this file |
| `ignoreHTTPSErrors` | `true` | For local dev with self-signed certs |

**Without `testIdAttribute: 'data-mark'`, all `getByTestId()` calls will fail.**

### Authentication Setup

Create `tests/e2e/auth.setup.ts`:

```typescript
import { test as setup } from '@playwright/test';
import { STORAGE_STATE } from '../../playwright.config';
import { ConstantHelper, UiHelpers } from '@umbraco/playwright-testhelpers';

setup('authenticate', async ({ page }) => {
  const umbracoUi = new UiHelpers(page);

  await umbracoUi.goToBackOffice();
  await umbracoUi.login.enterEmail(process.env.UMBRACO_USER_LOGIN!);
  await umbracoUi.login.enterPassword(process.env.UMBRACO_USER_PASSWORD!);
  await umbracoUi.login.clickLoginButton();
  await umbracoUi.login.goToSection(ConstantHelper.sections.settings);
  await page.context().storageState({ path: STORAGE_STATE });
});
```

### Environment Variables

Create `.env` (add to `.gitignore`):

```bash
UMBRACO_URL=https://localhost:44325
[email protected]
UMBRACO_USER_PASSWORD=yourpassword
UMBRACO_DATA_PATH=/path/to/Umbraco.Web.UI/App_Data  # Optional: for data reset
```

| Variable | Required | Purpose |
|----------|----------|---------|
| `UMBRACO_URL` | Yes | Backoffice URL |
| `UMBRACO_USER_LOGIN` | Yes | Admin email |
| `UMBRACO_USER_PASSWORD` | Yes | Admin password |
| `UMBRACO_DATA_PATH` | No | App_Data path for test data reset (see "Testing with Persistent Data") |

### Directory Structure

```
my-extension/
├── src/
│   └── ...
├── tests/
│   └── e2e/
│       ├── .auth/
│       │   └── user.json       # Auth state (gitignored)
│       ├── auth.setup.ts       # Authentication
│       └── my-extension.spec.ts
├── playwright.config.ts
├── .env                        # Gitignored
├── .env.example
└── package.json
```

---

## Patterns

### Test Fixtures

```typescript
import { test } from '@umbraco/playwright-testhelpers';

test('my test', async ({ umbracoApi, umbracoUi }) => {
  // umbracoApi - API helpers for setup/teardown
  // umbracoUi - UI helpers for backoffice interaction
});
```

### AAA Pattern (Arrange-Act-Assert)

```typescript
test('can create content', async ({ umbracoApi, umbracoUi }) => {
  // Arrange - Setup via API
  await umbracoApi.documentType.createDefaultDocumentType('TestDocType');

  // Act - Perform user actions via UI
  await umbracoUi.goToBackOffice();
  await umbracoUi.content.goToSection(ConstantHelper.sections.content);
  await umbracoUi.content.clickActionsMenuAtRoot();

  // Assert - Verify results
  expect(await umbracoApi.document.