Data processing inequality

The Data processing inequality is an information theoretic concept which states that the information content of a signal cannot be increased via a local physical operation. This can be expressed concisely as 'post-processing cannot increase information'.^[1] As explained by Kinney and Atwal, the DPI means that information is generally lost (never gained) when transmitted through a noisy channel.^[2]

Example

Let be an Markov chain $X \rightarrow Y \rightarrow Z$
Then,
$I(x;y) \geqslant I(x;z)$ with
$I(x;y) = I(x;z)$ if and only if $X \rightarrow Z \rightarrow Y$
where $I(x;y)$ is the Mutual information

See also

• Garbage in, garbage out

References

1. Beaudry, Normand (2012), "An intuitive proof of the data processing inequality", Quantum Information & Computation 12 (5-6): 432–441, arXiv:1107.0740
2. "Equitability, mutual information, and the maximal information coefficient.". Proc Natl Acad Sci U S A 111: 3354–9. Mar 2014. doi:10.1073/pnas.1309933111. PMID 24550517.

External links

• http://www.scholarpedia.org/article/Mutual_information