Feature engineering is the process of transforming raw market data — prices, volumes, fundamentals, order-book snapshots, text — into the numerical inputs ("features") that a machine-learning model actually consumes. In quantitative trading it is widely regarded as the highest-leverage and most domain-specific step in the modeling pipeline: model choice (XGBoost vs. a neural net) often matters less than which features you feed in. Its core tension is unique to finance: market data has an extremely low signal-to-noise ratio and is non-stationary, so the same transformations that make a series statistically usable (e.g. differencing to achieve stationarity) tend to destroy the predictive "memory" the model needs — and the same flexibility that lets you build thousands of candidate features makes it trivially easy to engineer a feature that secretly contains information from the future.

What gets engineered

Features in market ML fall into a few canonical families:

• Price/return transforms — log returns over multiple horizons, lagged returns, cumulative returns, normalized price (e.g. distance from a moving average).
• Technical indicators as features — moving averages (SMA/EMA), RSI, MACD, Bollinger %B, ATR-based volatility, rate-of-change. Here the indicator is not a standalone signal but one column among many.
• Volatility & risk features — realized volatility (rolling std of returns), Garman-Klass/Parkinson range estimators, downside deviation, beta.
• Volume/liquidity features — dollar volume, Amihud illiquidity, turnover, bid-ask spread, order-flow imbalance.
• Cross-sectional features — a stock's characteristic ranked or z-scored against the universe on the same date (the standard form in cross-sectional return prediction).
• Fundamental features — valuation multiples, margins, leverage, growth rates — used point-in-time.
• Alternative-data features — news/sentiment scores, web-traffic, satellite counts.

A large empirical study by Gu, Kelly & Xiu (Review of Financial Studies, 2020) used 94 firm characteristics, 8 macro time-series and 74 industry dummies — well over 900 derived signals — and found that across all ML methods the dominant predictive features clustered around momentum, liquidity, and volatility. That is the closest thing the field has to a consensus on which engineered features carry the most weight.

Core techniques

• Lagging & windowing — every feature must be computed from data available strictly before the prediction timestamp. Rolling windows (mean, std, min/max, rank) are the workhorse.
• Stationarity transforms — models assume reasonably stationary inputs, but raw prices are non-stationary. Standard fixes (integer differencing → returns) make the series stationary but wipe out long-memory. Fractional differentiation (López de Prado, Advances in Financial Machine Learning, Ch. 5) differences by a real exponent d between 0 and 1, choosing the minimum d that passes a stationarity test (e.g. ADF) so the series is stationary while retaining as much memory as possible. This is one of the few finance-specific feature techniques with a clear theoretical motivation.
• Scaling/normalization — standardization (z-score), min-max, or cross-sectional ranking; tree models are scale-invariant, but distance- and gradient-based models are not.
• Cross-sectional normalization — ranking or z-scoring each feature within each date removes time-varying market-wide level effects and is standard in equity cross-section work.
• Dimensionality reduction — PCA or autoencoders to compress correlated features; surveys report PCA as a frequently top-performing extraction step, though it sacrifices interpretability.
• Feature selection — tree-based importance (e.g. Random Forest / gain importance), but López de Prado warns that naïve importance is unreliable under correlated features and recommends methods like Mean-Decrease-Accuracy with purged cross-validation.

How it's used in practice

The pipeline is: assemble point-in-time raw data → engineer a wide candidate feature set → align every feature and the prediction label on a strict time axis → validate with walk-forward / purged cross-validation (not random k-fold, which leaks across time) → prune to a robust subset. Practitioners deliberately engineer features at multiple horizons (e.g. 5-, 21-, 63-day windows) so the model can find the relevant timescale. Labels are themselves "engineered" — fixed-horizon forward returns, or López de Prado's triple-barrier method (first touch of a profit-take, stop, or time barrier), which produces more economically meaningful targets than a fixed-bar return.

Standing & evidence

Feature engineering is universally adopted across quant finance, from retail Kaggle-style models to institutional equity factor desks. The contested question is not whether to do it but how much it survives out-of-sample. The dominant honest finding in the literature is that the bottleneck is rarely cleverness of features and almost always overfitting and leakage: with hundreds of candidate features and noisy data, it is easy to "discover" a spurious predictor. López de Prado's central thesis is that most published financial ML results are false discoveries from improper validation. Gu/Kelly/Xiu's positive result — that ML on engineered characteristics produced real, large economic gains, mostly via nonlinear interactions among momentum/liquidity/volatility features — is credible precisely because it was rigorously out-of-sample (the study spans 1957–2016, with the last 30 years, 1987–2016, reserved for out-of-sample testing). The reconciliation: well-known, economically grounded features (momentum, value, volatility) carry durable signal; exotic, mined features usually do not.

Strengths & limitations

Strengths. Encoding domain knowledge (volatility regimes, mean-reversion, cross-sectional rank) lets simpler, more robust models work; good features improve sample efficiency, which matters because market data is scarce relative to its noise.

Limitations / failure modes.

• Lookahead bias / leakage is the #1 failure — any feature (or label, or scaler fit on the full sample) that uses future information produces a spectacular backtest that collapses live. The classic trap is a rolling/centered statistic or a normalization fit across the whole dataset.
• Non-stationarity / regime change — a feature's relationship to returns can decay or invert; features learned in one regime fail in another.
• Multiple-testing / overfitting — more candidate features mean more chances to fit noise; without purged, embargoed walk-forward validation, feature importance is meaningless.
• The single most common misuse is treating feature engineering as a search for the most predictive feature in-sample, rather than the most robust one out-of-sample with no leakage.

System relevance

Within Delvantic this node sits under Machine Learning in Markets and is upstream of any model that Cairn (track-record measurement) or the regime engine might consume. It is distinct from the indicator-definition nodes in the Technical Analysis branch: those define RSI/MACD/ATR as signals; here they are raw material for a feature matrix. Hard caveat for any system that ingests engineered features: the feature pipeline must be point-in-time and leakage-free, and feature importance must come from purged walk-forward validation — an impressive in-sample feature set is the most common way a market ML result is wrong.

Sources

• Gu, Kelly & Xiu, "Empirical Asset Pricing via Machine Learning," Review of Financial Studies 33(5), 2020 — academic.oup.com/rfs/article/33/5/2223; NBER WP 25398 (dominant features = momentum/liquidity/volatility; out-of-sample gains via nonlinear interactions).
• López de Prado, Advances in Financial Machine Learning (Wiley, 2018), Ch. 5 (fractional differentiation), Ch. 3 (triple-barrier labels), Ch. 7–8 (purged CV, feature importance) — oreilly.com/library/view/advances-in-financial/9781119482086.
• Hudson & Thames, "Fractional Differentiation" — hudsonthames.org/fractional-differentiation/ (stationarity-vs-memory tradeoff, minimum-d selection).
• "Survey of feature selection and extraction techniques for stock market prediction," PMC9834034 — pmc.ncbi.nlm.nih.gov/articles/PMC9834034/ (PCA as top extraction method; selection to avoid overfitting).
• Kyle Jones, "Data Leakage, Lookahead Bias, and Causality in Time Series" — medium.com/@kyle-t-jones (rolling-window misalignment as primary leakage conduit).
• ResearchGate, "Feature Engineering for Financial Market Prediction" (2024) — feature families overview; cross-checked against the survey above.

Flagged dispute: the field genuinely disagrees on how much mined-feature edge survives live trading; this doc takes López de Prado's skeptical/validation-first position as the responsible default, supported by the rigor of the Gu/Kelly/Xiu out-of-sample result.