Perplexity
The perplexity is a trivial measure to make the entropy more intuitive.
\[\P[X] = 2^{\H[X]}\]
The perplexity of a random variable is the size of a uniform distribution that
would have the same entropy. For example, a distribution with 2 bits of entropy
has a perplexity of 4, and so could be said to be “as random” as a four-sided
die.
The conditional perplexity is defined in the natural way:
\[\P[X|Y] = 2^{\H[X|Y]}\]
-
perplexity(dist, rvs=None, crvs=None, rv_names=None)[source]
Parameters : |
- dist (Distribution) –
The distribution from which the perplexity is calculated.
- rvs (list, None) –
The indexes of the random variable used to calculate the perplexity.
If None, then the perpelxity is calculated over all random variables.
- crvs (list, None) –
The indexes of the random variables to condition on. If None, then no
variables are condition on.
- rv_names (bool) –
If True, then the elements of rvs are treated as random variable
names. If False, then the elements of rvs are treated as random
variable indexes. If None, then the value True is used if the
distribution has specified names for its random variables.
|
Returns: | P (float) –
The perplexity.
|