Next Steps

Marginalising over missing values

dataframes/step3_interpolating_over_nan.py already interpolates every NaN in the raw yield data before PCA runs, so no missing values currently reach either Bayesian track — the "Bayesian PCA marginalises over missing data" idea in concerns.md is a well-known advantage of probabilistic PCA in general, not something either track actually does yet.

A future extension: skip interpolation and let the model marginalise over missing values directly instead, so the uncertainty from not knowing a value propagates through inference rather than pretending an interpolated guess was the truth.

The natural worry — that each missing value adds processing time — mostly dissolves for a Gaussian-likelihood model like the full-latent (bayesian_latent) track. Missing entries can be marginalised out analytically: for a row with some missing columns, evaluate the likelihood using the marginal Normal over just that row's observed columns (dropping the missing ones from $\boldsymbol{\mu}$ and the covariance), rather than treating each missing value as its own sampled latent parameter. This is exact, not an approximation, and adds essentially no MCMC dimensionality — the real cost is bookkeeping (each distinct missingness pattern needs its own covariance submatrix), not sampling overhead.

Comparing classical vs. Bayesian PCA around a QE regime shift

Fit classical PCA on data truncated at some point before it became clear the central bank would continue quantitative easing, then fit a comparable Bayesian PCA on that same truncated window but with a prior reflecting the market's dawning understanding that QE would continue. Comparing the two would show what incorporating that piece of expert judgement via the prior actually buys you over the classical model, which only sees the truncated data itself and has no way to encode that forward-looking belief.