market-data

What this skill does

# Market Data

## Purpose

Guide the design and management of market data infrastructure for financial services
firms. Covers real-time and delayed market data, depth of book levels, consolidated
tape and direct feeds, data vendor selection and management, market data licensing and
entitlements, data distribution architecture, and market data quality management.
Enables building or evaluating market data infrastructure that delivers accurate,
timely data to trading, portfolio management, and client-facing systems.

## Layer

13 — Data Integration (Reference Data & Integration)

## Direction

both

## When to Use

- A firm is designing or upgrading its market data infrastructure
- Questions arise about real-time vs delayed data or Level 1/2/3 requirements
- A firm is evaluating market data vendors (Bloomberg, Refinitiv, ICE, FactSet)
- Questions involve consolidated tape (SIP) vs direct exchange feeds
- A firm needs to manage exchange data licensing, entitlements, or usage reporting
- A technology team is designing ticker plants, fan-out, or distribution architecture
- Data quality issues arise: stale quotes, missing ticks, erroneous prints
- Trigger phrases: "market data," "Level 1/2/3," "depth of book," "consolidated tape,"
  "SIP," "direct feed," "NBBO," "ticker plant," "B-PIPE," "data license," "non-display
  use," "market data entitlements," "conflation," "tick data," "real-time feed"

## Core Concepts

### 1. Market Data Types

- **Trade data:** Last sale price, quantity, timestamp, condition codes (regular, odd lot,
  opening/closing), cumulative volume, VWAP.
- **Quote data:** NBBO (best bid/offer across all exchanges), bid/ask sizes. Quote updates
  vastly outnumber trade updates in liquid instruments.
- **Depth of book:** Multiple price levels beyond the NBBO with resting order quantities.
  Aggregated depth (size per level) or order-by-order (individual orders visible).
- **Index data:** Real-time index values, composition, weightings, total return values.
  Sources include exchange-calculated (S&P 500 via Cboe) and third-party (MSCI, FTSE Russell).
- **Fixed income pricing:** Dealer quotes, evaluated pricing (ICE, Bloomberg BVAL), TRACE
  trade reports for corporates, EMMA for municipals. Inherently less standardized than equities.
- **Options data:** Chains (strikes/expirations), Greeks, implied volatility, volume, open
  interest. OPRA provides the consolidated options feed.
- **Fundamental data:** Earnings, financial statements, corporate actions, analyst estimates.
  Sourced from vendors (Bloomberg, FactSet, S&P Capital IQ) rather than exchange feeds.
- **News and events:** Headlines, economic calendar (FOMC, employment), corporate events
  (earnings dates, ex-dates), sentiment scores.

### 2. Data Levels

**Level 1 — Top of Book:** NBBO, last sale, volume, daily OHLC. Sufficient for portfolio
management, client reporting, and order entry. Lowest cost and bandwidth.

**Level 2 — Market Depth:** Multiple price levels with aggregate size (top 5-20 levels per
side). Reveals liquidity beyond the NBBO. Essential for active trading, market impact
assessment, and algorithmic execution (TWAP, VWAP). Higher cost and bandwidth.

**Level 3 — Full Order Book:** Individual order detail (price, size, order ID) enabling
complete book reconstruction and order lifecycle tracking. Provided by direct feeds (Nasdaq
ITCH, NYSE Arca). Required for market making, HFT, and queue position modeling. Highest
cost — hundreds of thousands of messages per second per exchange.

| Use Case | Level | Rationale |
|---|---|---|
| Portfolio management / reporting | Level 1 | NBBO and last sale sufficient for valuation |
| Active equity trading desk | Level 2 | Traders assess depth before large orders |
| Algorithmic execution | Level 2 | Algorithms adapt pace based on available liquidity |
| Market making / HFT | Level 3 | Requires queue position and order flow modeling |
| Client-facing app (delayed) | Level 1 (delayed) | Display only, 15-minute delay acceptable |

### 3. Consolidated Tape vs Direct Feeds

**Securities Information Processors (SIPs):** CTA/CQS for NYSE-listed (Tape A/B), UTP for
Nasdaq-listed (Tape C), OPRA for options. SIPs collect data from all exchanges, compute the
NBBO, and disseminate a consolidated stream. Under Reg NMS, the SIP NBBO is the regulatory
benchmark for best execution.

**Direct exchange feeds:** Proprietary feeds from individual exchanges (NYSE Arca, Nasdaq
TotalView/ITCH, Cboe PITCH, IEX DEEP) delivering order-by-order data with lower latency
than the SIP. A firm must subscribe to multiple feeds and compute NBBO internally. Each
exchange uses different protocols requiring per-exchange parsers.

| Dimension | SIP (Consolidated) | Direct Feeds |
|---|---|---|
| Latency | Higher (~10-50 microseconds SIP processing) | Lower (bypasses SIP) |
| NBBO | Provided directly | Must compute from multiple feeds |
| Data depth | Level 1 (NBBO + last sale) | Level 2/3 (full depth, order-by-order) |
| Cost | Lower, predictable | Higher, scales with exchange count |
| Normalization | Pre-normalized | Requires per-exchange parsers |
| Typical consumer | Buy-side, advisory, retail | Prop trading, market making, HFT |

### 4. Market Data Vendors

**Bloomberg:** Terminal ($20K-$25K/user/year), B-PIPE (enterprise real-time feed), Data
License (bulk EOD/reference data), BEAP (cloud API).

**Refinitiv (LSEG):** Eikon (desktop, lower cost than Bloomberg, strong FX/FI), Elektron/
LSEG Real-Time (enterprise feed), DataScope (bulk EOD), Tick History (historical ticks).

**ICE Data Services:** Consolidated feeds, evaluated fixed income pricing (widely used for
NAV and regulatory reporting), ICE Benchmark Administration.

**FactSet:** Research-oriented, flexible API delivery, competitive pricing for smaller
buy-side, strong Excel/portfolio management integration.

**S&P Capital IQ / Market Intelligence:** Comprehensive fundamentals, credit ratings,
company filings. **Morningstar:** Fund/ETF data, ratings, Morningstar Direct for research.

**Free/open sources:** Exchange websites and financial portals provide delayed (15-min)
quotes. Useful for non-time-sensitive display but limited reliability and coverage.

**Vendor selection criteria:** Asset class coverage, latency, reliability/uptime SLA, API
quality, total licensing cost (including exchange fees), historical data depth, support,
data quality handling.

### 5. Market Data Licensing and Entitlements

**License categories:** Non-professional (retail, personal use, lower fees) vs professional
(business use, significantly higher). Display (human views on screen) vs non-display
(automated systems: algorithms, risk engines, pricing — fees based on application type, not
per-user). Derived data (substantially transformed; redistribution may be permitted if
original data cannot be reverse-engineered; policies vary by exchange).

**Licensing models:** Per-user/per-device (exact monthly count required), enterprise (flat
fee covering a defined entity), usage-based non-display (fees by application category:
trading, risk, valuation).

**Reporting obligations:** Monthly/quarterly subscriber counts submitted to each exchange or
via data vendor. Under-reporting triggers back-billing, penalties, and contract termination.

**Redistribution:** Raw exchange data requires explicit redistribution agreements and
additional fees for client-facing display. Vendors typically handle redistribution for data
consumed through their platforms.

**Cost management:** Audit usage periodically to eliminate unused subscriptions. Use delayed
data where real-time is unnecessary. Track non-display use — many firms discover unreported
non-display obligations only during exchange audits.

### 6. Market Data Distribution Architecture

**Ticker plant:** Central ingestion and normalization layer. Parses exchange protocols (ITCH,
PITCH, FIX), normalizes to unified schema, maps symbology, caches latest values, applies
conflation, and monitors feed heal