Classical Principal Component Analysis

Finds a low dimensional linear subspace that maximises variance (or equivalently minimises reconstruction error). A purely algebraic operation. No notion of noise. No uncertainty in parameters

Probablistic Principal Component Analysis

Reframes PCA a maximum likelihood under a probabilistic latent factor model. Still no priors. "Latent" here implies an unobserved \ hidden variables $\mathbf{z} \in \mathbb{R}^k, k≪d$ that cannot be measured directly. Instead the observed data is generated by said hidden variables. Each observation is a vector: $\mathbf{x} \in \mathbb{R}^d$ and $n$ observations produce a matrix: $\mathbf{X} \in \mathbb{R}^{n \times d}$
Recovery is the process of finding the most likely values of principal components given an observation using $\mathbf{z} = \mathbf{W}^T(\mathbf{x} - \boldsymbol{\mu})$. The maxiumum likelihood solution for $W$ recovers the classical PCA directions exactly. Generative model: $\mathbf{x} = \mathbf{W}\mathbf{z} + \boldsymbol{\mu} + \boldsymbol{\epsilon}$ Mean: $\boldsymbol{\mu} \in \mathbb{R}^d$ — mean of the observed data

Bayesian Principal Component Analaysis

Bayesian PCA is an extension of Probablistic PCA and therefore assumes that latent variables are normally distributed.