A One-Byte, Options-Free Market-State Monitor: Detection-Preserving Compression of Financial Data Streams with a Class-Discriminant Token

Abstract

A One-Byte, Options-Free Market-State Monitor: Detection-Preserving Compression of Financial Data Streams with a Class-Discriminant Token Randolph James Ferlic, M.D., and Kimberly Kate Ferlic — Fieldstone Analytics, LLC Correspondence: randolphf@fieldstoneanalyticsllc.com Preprint · Zenodo DOI: 10.5281/zenodo.22101085 · CC-BY 4.0 Abstract A previously described class-discriminant encoder reduces a window of a multivariate stream to a single 8-bit token (statistical features projected onto a shrinkage-regularized linear-discriminant and principal-component subspace, quantized to a k-means centroid, read by a lightweight head), trading thousands-fold compression for a preserved decision. We ask, under strict pre-registration, how much of that property survives on financial data — the hardest domain for naive machine learning. Across five asset classes (equities, foreign exchange, rates, cryptocurrency, commodities) and thousands of trading days of real daily price/volume data, we find a sharp, consistent boundary. As a detector, the token is near-lossless: an unsupervised codebook fit on calm windows only flags market-stress windows by nearest-centroid distance at a mean AUC of about 0.90, within 0.01–0.04 of a full 50-feature detector, at roughly 500× compression and with no labels. Pooled across a thirteen-name basket, a one-byte-per-instrument, options-free Token Market-State Index tracks the VIX volatility index (Spearman 0.62) and detects market stress that neither a full-feature detector nor VIX statistically outperforms at this sample size — a result that survives purged, embargoed walk-forward validation and block-bootstrap confidence intervals across 2010–2026, and generalizes across all five asset classes, to intraday (hourly) frequency, and to a distinct cross-asset macro risk-off state. A second, complementary detector built from the same tokens — the cross-sectional co-movement of the per-instrument token distances — flags correlation and contagion regimes, adding information beyond volatility (a +0.11 AUC point-estimate increment) and rivaling the absorption-ratio systemic-risk measure computed from the full return covariance. As a classifier or forecaster, the same token is honestly limited: it pays a real 0.07–0.16 AUC tax on supervised volatility-regime and market-state classification (only partly recovered by multi-token product quantization), it is coincident rather than leading, and it shows no directional-return skill at any horizon. The unifying regularity is that the encoder retains what a detector needs and discards what a classifier or forecaster needs — the same compression boundary observed for physiological and industrial signals, now mapped in finance. We report all negatives, including two pre-registered red flags that caught bugs in our own code before they became false results. Highlights · The single 8-bit token is a strong coincident detector of market stress: an unsupervised calm-fit codebook flags stress windows at a mean AUC of ~0.90, within 0.01–0.04 of a full 50-feature detector, at ~500× — indeed as few as two bits — compression, with no labels. · A pooled, options-free, one-byte-per-instrument Token Market-State Index tracks VIX (Spearman 0.62) and detects market stress that neither a full-feature detector nor VIX statistically outperforms at this sample size — surviving purged/embargoed walk-forward and block-bootstrap intervals across 2010–2026, and generalizing across equities, FX, rates, cryptocurrency and commodities, to intraday frequency, and to a distinct macro risk-off state. · A second detector from the same tokens — cross-sectional co-movement — flags contagion / correlation regimes, adding information beyond volatility (a +0.11 AUC point-estimate increment) and rivaling the absorption-ratio systemic-risk measure, from compressed tokens rather than the full covariance. · Honest limits, fully reported: the token pays a real 0.07–0.16 AUC classification tax, is coincident, not leading, and shows no directional-return skill at any horizon; a coincident de-risking rule reduces drawdown but is presented explicitly not as a trading strategy. Two self-caught bugs (a lookahead label and a zeroed factor) are disclosed. · No new method is claimed: the contribution is a pre-registered map of where an extreme-compression class-discriminant token is a detector and where it is not, on real public market data, with all outcomes — including those that missed their bands — reported verbatim. Statistical candor is explicit: only the market-state result carries bootstrap confidence intervals; the other comparative figures are single-split point estimates, pre-registered but multiplicity-uncorrected. What this record contains · The manuscript (PDF), with a fourteen-study summary table and eight figures. · A reproducibility archive (`PAPER_39_ZENODO_ARCHIVE.zip`): the eight pre-registration scopes with frozen outcome bands, the deterministic per-study runners (FIN-1 through FIN-14), the per-study JSON result summaries behind every figure and table value, and the figures. All data are public daily and hourly OHLCV series and the CBOE Volatility Index (VIX); no raw data is redistributed — the loaders fetch the public source at run time. Cite as R. J. Ferlic and K. K. Ferlic, "A one-byte, options-free market-state monitor: detection-preserving compression of financial data streams with a class-discriminant token," Zenodo, 2026, doi: 10.5281/zenodo.22101085. License and patent notice Released under CC-BY 4.0. Consistent with Section 2(b) of that license, no patent, patent-application, or other intellectual-property right of the authors is licensed, waived, granted, or otherwise conveyed by this publication or by any reuse of it; the methods described herein — including the single-token class-discriminant encoder, its multi-token product-quantization variant, and its unsupervised nearest-centroid-distance monitoring mode — are the subject of filed and pending U.S. patent applications, and attribution under CC-BY does not extend to those rights. Inquiries regarding licensing of the encoding methods may be directed to randolphf@fieldstoneanalyticsllc.com. Companion deposits Part of the single-token class-discriminant codebook family on Zenodo (community: spiral-domain-encoder-campaign), which includes the single-token industrial sensor substrate (doi: 10.5281/zenodo.20854722), the hardening-and-generality characterization (doi: 10.5281/zenodo.20802759), and the deterministic multi-token token-ladder (doi: 10.5281/zenodo.22003179). This deposit applies the same previously described encoding method to a new input domain — financial data streams. Keywords decision-preserving compression, class-discriminant codebook, market-stress detection, contagion and correlation regime, systemic risk, volatility regime, VIX, anomaly detection, edge computing, pre-registration, honest negatives.