integration-patterns

What this skill does

# Integration Patterns

## Purpose

Guide the design and implementation of integration architectures connecting financial systems. Covers API design conventions for financial data services, the FIX protocol for order and market data messaging, ISO 20022 XML-based financial messaging and SWIFT migration, event-driven architectures for trade and settlement event propagation, batch file processing for custodian feeds and end-of-day reconciliation, idempotency and exactly-once semantics for financial transactions, error handling and resilience patterns (retry, circuit breaker, dead letter queue, compensating transaction), data transformation and mapping between systems with different schemas and identifier conventions, and security and compliance requirements for integration infrastructure. Enables building or evaluating integration architectures that reliably connect portfolio management, trading, settlement, custodian, and reporting systems while maintaining data integrity and audit trails.

## Layer

13 -- Data Integration (Reference Data & Integration)

## Direction

both

## When to Use

- Designing a custodian integration architecture for an advisory firm or asset manager
- Evaluating FIX protocol connectivity for order routing or market data
- Implementing ISO 20022 messaging for payments, securities settlement, or SWIFT migration
- Designing event-driven trade notification or settlement status systems
- Building batch file processing pipelines for custodian feeds, reconciliation files, or EOD settlement
- Implementing idempotency for financial transaction APIs to prevent duplicate processing
- Designing retry and circuit breaker patterns for unreliable upstream systems
- Mapping data between systems with different schemas, identifiers, or conventions
- Evaluating API design patterns (REST, WebSocket, gRPC) for financial data services
- Implementing mTLS, encryption, and audit logging for integration infrastructure
- Troubleshooting integration failures causing reconciliation breaks or settlement delays
- Trigger phrases: "FIX protocol," "ISO 20022," "event-driven," "batch processing," "custodian feed," "API design," "idempotency," "circuit breaker," "dead letter queue," "data mapping," "integration architecture," "message broker," "file feed," "SWIFT migration," "mTLS"

## Core Concepts

### 1. API Design for Financial Systems

Financial APIs serve position data, transaction history, account information, reference data, and order submission. Design conventions differ from general-purpose APIs due to the sensitivity, auditability, and volume of financial data.

**REST conventions:** Resource-oriented design with nouns for financial entities -- `/accounts/{id}/positions`, `/transactions`, `/orders/{id}/executions`. Use HTTP methods semantically: GET for reads (positions, balances), POST for actions (order submission, transfer initiation), PUT/PATCH for updates (account preferences, model assignments). Return standard HTTP status codes with domain-specific error bodies including error codes, human-readable messages, and correlation IDs for traceability. Financial APIs must distinguish between synchronous operations (position lookup returns immediately) and asynchronous operations (transfer initiation returns a 202 Accepted with a status polling URL or webhook callback). Long-running operations such as bulk rebalancing or batch trade submission should use the async pattern to avoid client-side timeouts.

**WebSocket and streaming:** For real-time use cases (position updates, order status, market data), WebSocket connections provide server-push capability without polling overhead. Financial WebSocket APIs require heartbeat/ping-pong to detect stale connections, automatic reconnection with state recovery (the client must receive missed updates on reconnect), and back-pressure handling when the consumer cannot keep pace with the producer.

**Versioning:** URL-based versioning (`/v2/positions`) is the dominant pattern in financial APIs due to its visibility and cacheability. Breaking changes (field removal, type change, semantic change) require a new version; additive changes (new optional fields) do not. Maintain at least two concurrent versions with a published deprecation timeline (typically 12-18 months in financial services).

**Pagination for large datasets:** Position and transaction endpoints routinely return thousands of records. Cursor-based pagination (opaque next-page token) is preferred over offset-based for consistency during concurrent writes. Include total count estimates, page size limits, and sorting parameters. For bulk data extraction, offer a separate export/download endpoint returning files rather than paginated API calls.

**Rate limiting and throttling:** Protect systems from burst traffic during market events (open, close, volatility spikes). Use token bucket or sliding window algorithms. Return `429 Too Many Requests` with `Retry-After` headers. Distinguish rate limits per client, per endpoint, and per operation type (reads vs writes).

**Authentication and authorization:** OAuth 2.0 client credentials flow for service-to-service. API keys with HMAC request signing for simpler integrations. Mutual TLS (mTLS) for high-security custodian and clearing connections. Role-based access control scoped to accounts, operations (read-only vs read-write), and data sensitivity levels. Token expiration and rotation policies must be automated -- expired credentials are a leading cause of integration outages.

**Integration testing:** Financial API integrations require dedicated testing environments (UAT/sandbox) that mirror production behavior including realistic data volumes, error scenarios, and latency characteristics. Contract testing (verifying both producer and consumer conform to the API specification) prevents integration regressions during independent deployments. Custodians and data vendors typically provide certification environments; completing certification is a prerequisite for production connectivity. Maintain a suite of integration tests that exercise the happy path, each documented error code, timeout behavior, pagination boundaries, and rate limit responses.

### 2. FIX Protocol

The Financial Information eXchange (FIX) protocol is the dominant messaging standard for electronic trading, connecting buy-side firms, sell-side firms, exchanges, ECNs, and alternative trading systems.

**Protocol structure:** FIX messages are sequences of tag=value pairs delimited by SOH (ASCII 0x01). Each message has a header (BeginString, BodyLength, MsgType, SenderCompID, TargetCompID, MsgSeqNum, SendingTime), a body (message-type-specific fields), and a trailer (CheckSum). Tags are numeric (e.g., Tag 35 = MsgType, Tag 55 = Symbol, Tag 44 = Price, Tag 38 = OrderQty).

**Session layer:** Manages connectivity, sequencing, and recovery. Logon (MsgType=A) establishes the session with sequence number synchronization. Heartbeat (MsgType=0) and TestRequest (MsgType=1) monitor connection health. ResendRequest (MsgType=2) and SequenceReset (MsgType=4) handle gap recovery after disconnection. Logout (MsgType=5) terminates the session. Sequence numbers are strictly monotonic per session per direction; gaps trigger automatic recovery.

**Key application messages:** NewOrderSingle (MsgType=D) submits an order. ExecutionReport (MsgType=8) reports fills, partial fills, cancellations, and rejects. OrderCancelRequest (MsgType=F) and OrderCancelReplaceRequest (MsgType=G) modify or cancel orders. MarketDataRequest (MsgType=V) subscribes to market data. MarketDataSnapshotFullRefresh (MsgType=W) and MarketDataIncrementalRefresh (MsgType=X) deliver market data.

**FIX versions:** FIX 4.2 remains widely deployed for equity order routing. FIX 4.4 added improved support for multi-leg instruments, allocations, and position management. FIX 5.0 (with FIXT 1.1 transport) decoupled the session and application layers, enabling transport independence and versioned applicati