GitHub profile

2 Layers of Assumptions

If we are modelling principal components using a normal distribution, that is the distribution used to calculate the likelihood. In a way the form of distribution for the likelihood is also an assumption. So bayesian inference has two layers of assumptions: structural (likelihood) and prior (uncertainty about parameters) In summary, the likelihood reflects modelling assumptions, while the prior reflects parameter uncertainty within that model.

open decision: we have not yet decided what the bayesian approach will actually model. Two candidates on the table: (a) a light extension - keep the classical PCA loadings/eigenvectors fixed, and only put priors on the parameters (mean, variance) of the normal distribution used for the component scores, with the posterior describing that normal distribution. (b) something closer to Bishop's Bayesian PCA - a fully probabilistic generative model where the loadings themselves are also inferred (e.g. via an ARD prior), alongside the scores. These are materially different in scope: (a) stays close to classical PCA and industry practice, only adding uncertainty quantification on scores; (b) is a bigger undertaking but could address the independence/orthogonality issues raised later in this doc. Which one we mean changes what the "2 layers" framing above is actually describing.

For the future consideration perhaps:

Questions

what is PCA

Principal Component Analysis (PCA) is a dimensionality reduction technique that transforms a dataset into a set of orthogonal components, ordered by the amount of variance they capture, to simplify data analysis while preserving as much information as possible.

eigenvalue

% of variability explained by principal component

Principal Component

A vector that describes deviation from a mean level. Principal components are orthogonal to each other. The complete set of principal components explains all of variability in data set.

layperson explanations

principal components / eigenvectors are in effect patterns of movement.

and data shows how much each pattern is active at any one time

under classical patterns are fixed .. we say these are the patterns and these are the scores

under bayesian we say we are not entirely what the scores are so lets treat them as uncertain and estimate them

the priors come in : before looking at the data , what do i expect them to look like.

i guess we could go with pure parallel, slope and twist

priors stop overfitting and inject commonsense

Industry Practice - PCA based stress testing

The life industry typically uses classical PCA. note to self: evidence this point

The output of PCA is reinterpreted as a set of orthogonal economic stress scenarios scaled to regulatory confidence intervals

The method then treats the principal components as independent risk factors and stresses them either in isolation or combined.

This approach is valid if the PCA decomposition is stable and the orthogonality assumption holds out-of-sample. note to self: expand on this point

probablistic PCA

Under probablistic PCA the principal components are assumed to have a normal distribution. From this it follows that that the principal components are independent. Why? A normal distribution is fully characterised by its mean and variance. Given that principal components are by definition uncorrelated, there is no additional structure present within which dependence could be encoded.

In practice component scores are ??MORE OFTEN THAN NOT?? modelled using normal distribution.

Industry PCA

theoretical inconsistencies in industry practice

treating principal components as statistically independent

non gaussian modelling of components

under these conditions ICA becomes more theoretically correct tool

test image

test image

Summary