Residual Entropy
The residual entropy, or erasure entropy, is a dual to the binding information.
\[\begin{split}\R[X_{0:n}] &= \sum \H[X_i | X_{\{0..n\}/i}] \\
&= - \sum_{x_{0:n} \in X_{0:n}} p(x_{0:n}) \log_2 \prod p(x_i|x_{\{0:n\}/i})\end{split}\]
-
residual_entropy(dist, rvs=None, crvs=None, rv_names=None)[source]
Parameters : |
- dist (Distribution) –
The distribution from which the residual entropy is calculated.
- rvs (list, None) –
The indexes of the random variable used to calculate the residual
entropy. If None, then the total correlation is calculated
over all random variables.
- crvs (list, None) –
The indexes of the random variables to condition on. If None, then no
variables are condition on.
- rv_names (bool) –
If True, then the elements of rvs are treated as random variable
names. If False, then the elements of rvs are treated as random
variable indexes. If None, then the value True is used if the
distribution has specified names for its random variables.
|
Returns: | R (float) –
The residual entropy
|
Raises : | ditException –
Raised if dist is not a joint distribution.
|