Whitening transformation

A whitening transformation is a linear transformation that transforms a vector of random variables with a known covariance matrix into a set of new variables whose covariance is the identity matrix meaning that they are uncorrelated and all have variance 1.^[1] The transformation is called "whitening" because it changes the input vector into a white noise vector.

Several other transformations are closely related to whitening: 1) the decorrelation transform removes only the correlations but leaves variances intact, 2) the standardization transform sets variances to 1 but leaves correlations intact, and 3) a coloring transformation transforms a vector of white random variables into a random vector with a specified covariance matrix.^[2]

Definition

Suppose $X$ is a random (column) vector with non-singular covariance matrix $M$ and mean $0$. Then the transformation $Y = W X$ with a whitening matrix $W$ satisfying the condition $W^T W = M^{-1}$ yields the whitened random vector $Y$ with unit diagonal covariance.

There are infinitely many possible whitening matrices $W$ that all satisfy the above condition. Commonly used choices are $W = M^{-1/2}$ (Mahalanobis or ZCA whitening), the Cholesky decomposition of $M^{-1}$ (Cholesky whitening), or the eigen-system of $M$ (PCA whitening).

Kessy et al. (2015) demonstrate that optimal whitening transforms can be singled out by investigating the cross-covariance and cross-correlation of $X$ and $Y$. For example, the unique optimal whitening transformation achieving maximal component-wise correlation between original $X$ and whitened $Y$ is produced by the whitening matrix $W = P^{-1/2} V^{-1/2}$ where $P$ is the correlation matrix and $V$ the variance matrix.

Whitening a data matrix

Whitening a data matrix follows the same transformation as for random variables. An empirical whitening transform is obtained by estimating the covariance (e.g. by maximum likelihood) and subsequently constructing a corresponding estimated whitening matrix (e.g. by Cholesky decomposition).

See also

• Decorrelation
• Principal component analysis

References

1. Kessy, A.; Lewin, A.; Strimmer, K. (2015). "Optimal whitening and decorrelation". arXiv:1512.00809. Retrieved 28 March 2016.
2. Hossain, Miliha. "Whitening and Coloring Transforms for Multivariate Gaussian Random Variables". Project Rhea. Retrieved 21 March 2016.

External links

• http://courses.media.mit.edu/2010fall/mas622j/whiten.pdf
• The ZCA whitening transformation. Appendix A of Learning Multiple Layers of Features from Tiny Images by A. Krizhevsky.