Changelog

Every change to the schema, the REST and WebSocket APIs and the Python SDK, plus every data revision, newest first. API v1 and schema 0.9 are drafts until Phase 1, when the schema becomes 1.0.

Until 1.0, a draft may change in ways that break compatibility, and each such change is listed here. From 1.0, breaking schema changes are announced at least 90 days ahead and the old and new versions are served in parallel for at least 12 months. Pin a version with the FDC-Schema-Version header (see REST) or the SDK's minor version (see Python SDK). Corrections to published data appear as DATA REVISION entries; how revisions work is on Data quality. The reasoning behind the current draft is in Schema v0.9 draft, and comments are welcome at daniel@financedatacloud.com.

v0.9 draft, October update

Added

• WebSocket: prev_seq on every market data message, so a client can detect a missed message without venue-specific sequence rules.
• WebSocket: ts_send and region on every message.
• REST: X-RateLimit-Concurrent and X-RateLimit-Burst headers.
• chain_flows: link_venue, link_symbol and link_trade_id, joining a flow to the trade it was matched to.
• SDK: on_live callback, and on_gap accepting "raise", "resync" or a callable.

Changed

• WebSocket replay buffer for resume_from raised from 6 to 24 hours on Desk and from 24 to 72 hours on Enterprise.
• WebSocket: authentication failures now send an error frame before closing with 4401 or 4403 (previously the connection closed with no frame). The auth frame must arrive within 5 seconds.
• REST: maximum JSON page size raised from 5,000 to 10,000 rows.
• cxl_fill window shortened from 60 s to 10 s, with the level-1 and top-5 variants l1 and l5.

Deprecated

• The ts_recv alias for ts_capture, kept since v0.3, will be removed in schema 1.0.

Data revision: book_l2, Hyperliquid

Revision 1 stays readable for 24 months as an S3 object version and through revision=1.

v0.9 draft

The draft we expect to become schema 1.0 with only small changes. Background and open questions are in the announcement post.

Added

• region common column (apne1, euw1, later use1), and source = "fdc_derived" for rows we compute: candles, checkpoint snapshots and extended fields.
• revision parameter on REST and in the SDK, and Parquet footer keys fdc.schema_version, fdc.dataset, fdc.revision, fdc.row_count, fdc.gap_count, fdc.seq_range and fdc.sha256.
• FDC-Schema-Version request and response header.
• extended: emit, ts_available and ref_id, per-trade markout events, and custom profiles for Enterprise.
• funding: kind, interval_s, rate_1h and rate_ann, plus an hourly grid (ts_grid, predicted_rate_1h, stale, age_s); the SDK and REST gained grid="1h".
• instruments: valid_from, valid_to, successor_id, and the as_of and history parameters.
• gaps: detected_by and revision.
• Multiplier contracts such as 1000PEPE-PERP (Hyperliquid kPEPE), with multiplier in instruments.
• GET /v1/usage.

Changed

• extended moved to a long layout: field, variant, value, on_grid. The wide layout is still available with layout=wide in REST and wide=True in the SDK.
• OFI-ML variants are now l1, l5, l10 and norm (previously five per-level columns).
• Daily files are partitioned on the UTC day of ts_capture and sorted by ts_capture, then seq. REST time filters default to ts_capture.
• format=parquet responses are no longer paginated and cover up to 31 days per request.
• Book requests with start begin with a synthetic snapshot of the book at start.
• Error bodies standardized on status, code, message and request_id.

Deprecated

• The depth="full" alias in the SDK. Use depth="L2" with levels=None.

v0.8

Added

• mark_index.oracle_price and basis_bps.
• S3 daily manifests with a SHA-256 per file.
• fdc.download(), writing files in the S3 layout and returning their paths.
• Hourly full-depth snapshots in every book file, with the 00:00 UTC snapshot opening the file.

Changed

• Inverse contracts carry INV in the symbol: BTC-USD-INV-PERP (was BTC-USD-PERP) and BTC-USD-INV-26MAR27, so the margin type is never implied. See Symbology.
• Dated futures use BTC-26MAR27 (was BTC-20270326), matching option symbols such as BTC-26MAR27-90000-C.

Deprecated

• side values b and s. Use buy and sell. Removed in v0.9.

v0.7

Added

• chain_entities and chain_flows (L4+), with cluster_method, link_type, calibrated confidence and model_version. See Chain intelligence.
• book_l4.entity_id, filled for accounts with chain intelligence and null otherwise.
• part participant fields in extended, on venues with L4.
• options_chain, with IV and greeks where the venue publishes them.
• WebSocket draft: subscribe, unsubscribe, server-side snapshots and resume_from.

Changed

• chain_entities is versioned hourly with valid_from and valid_to, so clusterings can be queried as of any past time.

v0.6

Added

• extended: queue, cxl_fill, resil and markout.
• Candles at 1s and 5m.

Changed

• Markouts are now published on the grid row at which their horizon has elapsed. Earlier drafts stamped them at trade time, which placed future prices on past rows. Values are unchanged; their timestamps moved.
• S3 candle prefixes split by interval: candles_1m, candles_1h and so on.

v0.5

Added

• The extended table, with ofi_ml, vpin and lambda on a 100 ms grid.
• Retry-After and X-RateLimit-* headers.

Changed

• REST pagination moved from offset to opaque cursors, so page boundaries no longer shift when data is appended.

Deprecated

• The offset parameter. Removed in v0.7.

v0.4

Added

• book_l4: book_l3 plus participant_id and, on fills, counterparty_id.
• open_interest, liquidations and mark_index.
• The gaps table and GET /v1/gaps.

Changed

• book_l3.size is the remaining resting size after the event. The change is derived from the new prev_size column, and fills carry fill_size.

v0.3

Added

• source column on every table: fdc_capture or venue_archive.
• venue_symbol on every table.

Changed

• Prices and sizes changed from float64 to decimal128(38,18) in Parquet and to strings in JSON, so no precision is lost.
• ts_recv renamed to ts_capture. Both are int64 nanoseconds since the Unix epoch, UTC.

Deprecated

• ts_recv, kept as an alias.

v0.2

Added

• book_l3 with add, modify, cancel and fill events.
• candles at 1m, 1h and 1d, and instruments.

Changed

• Normalized symbology: BTC-USDT for spot, and BTC-PERP for each venue's main linear perpetual (USDT-margined on Binance, Bybit, OKX and Bitget; USDC-margined on Hyperliquid and dYdX).

v0.1

Added

• First shared draft of the schema: trades, quotes, book_l2 and funding.
• Common columns venue, symbol, ts_exchange, ts_recv and seq.
• Daily Parquet layout {dataset}/venue={venue}/date={YYYY-MM-DD}/{symbol}.parquet, unchanged since.