masterdata-storage-strategy

What this skill does

# Master Data storage strategy

## When this skill applies

Use this skill **before creating any new Master Data entity** or when auditing existing usage. It helps you answer:

- Is Master Data the **right** storage for this data, or would Catalog, OMS, VBase, or an external database serve better?
- How should I design the **JSON Schema** for performance and security?
- Which fields should I **index** (`v-indexed`), and which should I **not**?
- Should I enable or disable **caching** (`v-cache`)?
- Do I need **triggers** (`v-triggers`), or is an event-driven IO approach better?
- How do I plan for **capacity** and **lifecycle** of schemas and documents?

Do not use this skill for:

- VTEX IO app integration patterns (MasterDataClient, `masterdata` builder, CRUD in code) — use `vtex-io-masterdata`
- Performance patterns for IO services (LRU, VBase caching layers) — use `vtex-io-application-performance`

## Decision rules

### When to use Master Data

Master Data is a good fit when **all** of the following are true:

1. **Document-oriented access** — Your data is naturally key-value or document-shaped (JSON documents with variable schemas). You query by indexed fields and retrieve full or partial documents.
2. **Platform-integrated** — You benefit from VTEX-native features: `v-triggers` for automated workflows, `v-security` for per-field public access, `v-indexed` for search/filter, and the `masterdata` builder for schema-as-code.
3. **Moderate volume** — Your entity will hold thousands to low millions of documents. MD handles this well with proper indexing.
4. **Not on the purchase critical path** — MD is not optimized for sub-10ms latency. Synchronous MD reads in checkout/cart/payment flows risk conversion if MD is slow.
5. **No better native fit** — The data doesn't belong in Catalog (product/SKU attributes), OMS (order data), CL/AD (customer profiles/addresses), or VBase (app-specific cache/state).

### When NOT to use Master Data

| Data type                          | Better storage                                     | Why                                                        |
| ---------------------------------- | -------------------------------------------------- | ---------------------------------------------------------- |
| Product attributes, specifications | **Catalog** (specifications, unstructured specs)   | Native indexing, search integration, catalog APIs          |
| Order data, order history          | **OMS** (via OMS APIs + BFF cache)                 | Single source of truth; duplicating to MD creates drift    |
| Customer profiles, addresses       | **CL/AD native entities**                          | Platform-managed, already indexed and cached               |
| App-specific cache or temp state   | **VBase**                                          | Designed for per-app ephemeral storage, no schema overhead |
| Application logs, debug traces     | **`ctx.vtex.logger`**                              | Structured logging infrastructure, not a database          |
| High-throughput time-series data   | **External database** (SQL, NoSQL, time-series DB) | MD is not designed for millions of writes/day              |
| Relational data with joins         | **External SQL database**                          | MD has no join support; denormalize or use a relational DB |
| Data requiring strong consistency  | **External database**                              | MD is eventually consistent for indexed fields             |

### Schema design principles

- **One entity per concept** — Don't mix unrelated data in a single entity. Each entity should represent a clear business concept (e.g. `reviews`, `wishlists`, `legacyOrders`).
- **Index what you query** — Only fields in `v-indexed` can be used in `where` clauses. But **don't over-index**: each indexed field increases write latency and storage because the index is updated on every document change.
- **Minimal `v-default-fields`** — Return only the fields most consumers need by default. Large default payloads waste bandwidth.
- **`v-cache` matches the workload** — Leave `true` (default) for read-heavy entities. Set to `false` for entities with high write frequency where consumers need immediate consistency after writes.
- **`v-security` is explicit** — Set `allowGetAll: false` unless unauthenticated list access is intentional. Use `publicRead`, `publicWrite`, `publicFilter` only for fields that must be accessible without authentication.

### VTEX schema extensions (`v-*` fields) — reference

Master Data v2 extends standard JSON Schema with `v-*` properties that control indexing, caching, security, defaults, triggers, and schema inheritance. These are **VTEX-specific**; standard JSON Schema validators ignore them.

#### `v-indexed`

Array of field names that Master Data will create secondary indexes for.

- **Only** indexed fields can appear in `where` clauses for `searchDocuments` and `scrollDocuments`. Queries on non-indexed fields trigger **full document scans** that time out on large datasets.
- Each index is updated on **every** document write. Over-indexing increases write latency and storage cost proportionally.
- **When to index**: fields used in `where` filters, sort expressions, or `publicFilter`. **When not to index**: large text fields (`description`, `notes`), fields never queried, or fields only read by document ID (indexing adds no benefit for `getDocument`).

```json
{ "v-indexed": ["email", "status", "createdAt"] }
```

#### `v-cache`

Boolean (default `true`). Controls whether Master Data caches GET responses for individual documents.

- **`true` (default)** — Read-heavy entities benefit from caching. Most entities should leave this as default.
- **`false`** — Use for entities with **high write frequency** where consumers need **fresh reads** immediately after writes (e.g. real-time counters, configuration flags, session-like state).

```json
{ "v-cache": false }
```

#### `v-default-fields`

Array of field names returned when the caller does **not** specify a `fields` parameter in the API request.

- Keep this **minimal** — only the fields most consumers need by default.
- Reduces payload size for common queries.

```json
{ "v-default-fields": ["email", "status", "score", "createdAt"] }
```

#### `v-security`

Object controlling **unauthenticated** (public) access to fields. By default, all fields require authentication.

| Property       | Type       | Description                                                                                                   |
| -------------- | ---------- | ------------------------------------------------------------------------------------------------------------- |
| `allowGetAll`  | `boolean`  | If `true`, unauthenticated users can list all documents. **Default `false`; keep it off unless intentional.** |
| `publicRead`   | `string[]` | Fields readable without authentication                                                                        |
| `publicWrite`  | `string[]` | Fields writable without authentication                                                                        |
| `publicFilter` | `string[]` | Fields usable in `where` clauses without authentication (must also be in `v-indexed`)                         |

```json
{
  "v-security": {
    "allowGetAll": false,
    "publicRead": ["status", "displayName", "rating"],
    "publicWrite": [],
    "publicFilter": ["status"]
  }
}
```

**Never** include PII (email, phone, addresses), internal IDs, or business-sensitive data in `publicRead` or `publicFilter`.

#### `v-triggers`

Array of trigger objects that define automated actions executed when documents are created or updated and meet specified conditions.

| Property          | Type      | Description                                                                       |
| ----------------- | --------- | --------------------------------------------------------------------------------- |
| `name`            | `st