HGR

HGR stands for Hirschfeld–Gebelein–Rényi maximum correlation. It is a correlation metric in statistics. Compared with commonly-used Pearson's correlation coefficient, it has the advantage to handle non-linear statistical dependency. The benefit for such generalization comes with computational cost. While correlation coefficient can be computed by definition, HGR maximum correlation should be approximated by ACE ( Alternating Conditional Expectations) algorithm. The application of HGR and its extension has been found in the field of information theory, machine learning and so on.

History
HGR maximum correlation is independently proposed by three mathematicians in 20th century. Hans proposed that the correlation can be computed by series expansion, which is not efficient and replaced by ACE latterly.

Mathematical Description
Let $X$ and $Y$ be random variables, $f$ and $g$ be smooth transformation of $X$, $Y$ respectively, HGR is defined as
 * $$ \rho = \mathop{\max_{\mathbb{E}[f(X)]=0,\mathbb{E}[g(Y)]=0}}_{ \textrm{Var}[f(X)]=1, \textrm{Var}[g(Y)]=1} \mathbb{E}[f(X) g(Y)]

$$

Information Theory
The connection between HGR and other information-theoretic metric is discussed thoroughly with local assumption.