What Would Success Look Like? (Latent Track)
The aim
With uninformative priors, the Bayesian PCA (Latent) model should largely reproduce what classical PCA already found — that's the point of the "comparing output" step in the Aims plan. Concretely: does the classical loading matrix sit somewhere near the centre of the posterior this model produces, rather than out in the tails?
If it does, that's a basic consistency check passing: the Bayesian machinery isn't distorting the answer relative to the classical one when it isn't given any extra information to justify a different answer. If it doesn't, that's a signal something is wrong — not evidence of a genuinely different, better answer — since no informative prior has been introduced yet to justify moving away from classical.
Why "near the centre" isn't a straightforward column comparison
$\mathbf{W}$ in the latent model is only identified up to rotation and reflection: the likelihood depends on $\mathbf{W}$ only through $\mathbf{W}\mathbf{W}^\top$ (see Where the covariance matrix fits in), so infinitely many rotated/reflected versions of the same $\mathbf{W}$ describe the data identically well. That rules out lining up posterior draws' columns against classical PC1/PC2/PC3 directly — a posterior draw could describe the identical fitted model as classical yet look totally different column by column.
Two ways round this, not yet decided between:
- Compare at the covariance level. $\mathbf{W}\mathbf{W}^\top + \sigma^2\mathbf{I}$ is rotation-invariant, so the classical covariance matrix $\mathbf{A}$ (Step 6) can be compared directly against the distribution of $\mathbf{W}\mathbf{W}^\top + \sigma^2\mathbf{I}$ implied by posterior draws, without touching the rotation problem at all.
- Align, then compare columns. Rotate each posterior draw's $\mathbf{W}$ onto the classical loadings first (e.g. via a Procrustes rotation), then compare the aligned columns to PC1/PC2/PC3 individually — useful if the aim is to say something about individual components rather than just the overall fitted shape.
Which of these (or both) gets used is deferred to whichever step first needs to make the comparison.