Financial Forecasting — Baseline Comparison

▶ Live results site — the report below, rendered and rebuilt from reports/summary.md on every run.

A leakage-safe, walk-forward benchmark that pits a next-day price forecaster against honest baselines: naive random-walk, moving-average, linear (Ridge), gradient-boosted trees (XGBoost), and an LSTM. Real data — 20 equities over 2009–2025 from yfinance, plus the 10-year Treasury yield from FRED (the same multi-source setup as the underlying research: Stooq / yfinance / FRED).

The honest finding

The random-walk (naive) baseline is not beaten on RMSE. Across 16 years and 20 stocks, no model lowers next-day price RMSE below "tomorrow = today." This is the expected, correct result for near-efficient daily prices — and the reason a rigorous study reports it rather than a tuned number that won't reproduce.

Predictive value shows up elsewhere: the linear model matches naive on RMSE while reaching ~52% directional accuracy (above chance), and models' relative behavior differs by volatility regime.

Walk-forward results (6 folds, averaged)

Model	RMSE	MAE	SMAPE%	Dir. acc	RMSE vs naive
naive	2.410	1.355	1.20	—	+0.00%
linear	2.414	1.355	1.20	51.9%	−0.17%
lstm	2.493	1.404	1.24	48.7%	−3.46%
xgboost	2.693	1.582	1.38	51.3%	−11.75%
moving_avg	3.509	2.047	1.80	50.6%	−45.63%

Full report (regime breakdown + feature ablation): reports/summary.md.

Why this is the right way to do it

• No leakage: expanding-window walk-forward; training data is always strictly before the scored days (enforced by a test).
• Honest baselines first: naive and moving-average before any learned model — you cannot claim a win without beating them.
• Stationary target: models forecast next-day return and reconstruct the price level, so RMSE is comparable to the naive random walk.
• Diagnostics, not just a leaderboard: directional accuracy, volatility-regime breakdown, and a feature ablation.

Run it

pip install -r requirements.txt
python -m src.forecast.run                 # real data (yfinance + FRED, cached)
python -m src.forecast.run --synthetic     # offline, documented simulation
pytest -q                                  # 15 tests incl. no-leakage guarantees

First real run downloads and caches the panel to data/; later runs are offline. Tests use the synthetic panel and need no network.

Layout

src/forecast/
  data.py         yfinance + FRED panel (real) / documented synthetic fallback
  features.py     leakage-safe technical + macro + regime features
  models.py       naive, moving-avg, linear, xgboost, lstm (common interface)
  walkforward.py  expanding-window folds + metric evaluation
  metrics.py      RMSE, MAE, SMAPE, directional accuracy
  run.py          orchestration + regime breakdown + ablation + report
tests/            metrics, models, and the no-leakage / naive-baseline guarantees