PCA Concepts
Contents
Loadings
The entries (weights) of an eigenvector of the covariance matrix.
Each loading is the weight given to an original variable when forming a principal component — i.e., one of the coefficients in $\widetilde{\mathbf{Y}}_c\mathbf{w}_k$ — so the loadings describe how the component is constructed from the original variables.
The loadings for one principal component collectively form a loading vector $\mathbf{w}_k$, and stacking the loading vectors for all components as columns gives the loading matrix $\mathbf{W}$.
$\mathbf{w}_k$ is the same object as the generic eigenvector $\vec{v}$ defined on the Linear Algebra page — just relabeled once we're specifically talking about PCA/loadings rather than eigenvectors in general (so $\mathbf{W}$ corresponds to $\mathbf{V}$).
When we move from classical to probabilistic PCA, "loadings"/$\mathbf{W}$ becomes the preferred term over "eigenvectors."
Principal component loadings
Synonymous with Loadings above.
Principal component scores
The coordinates of the original observations once projected onto the principal component axes: $\mathbf{z} = \widetilde{\mathbf{Y}}_c\mathbf{W}$. Since $\mathbf{W}$'s columns are orthonormal, using all $N$ of them makes this a rotation of the data into a new coordinate system aligned with the directions of maximum variance — no information lost, just re-expressed in new axes. (Keeping only the first few columns of $\mathbf{W}$, as in $\mathbf{z}_3 = \widetilde{\mathbf{Y}}_c\mathbf{W}_3$, turns it into a rotation plus a dimensionality-reducing projection.) Scores are values, one per observation per component, describing where each observation sits in the new, lower-dimensional space — as opposed to loadings, which describe the axes themselves.