Perplexity

The perplexity is a trivial measure to make the entropy more intuitive.

\[\P[X] = 2^{\H[X]}\]

The perplexity of a random variable is the size of a uniform distribution that would have the same entropy. For example, a distribution with 2 bits of entropy has a perplexity of 4, and so could be said to be “as random” as a four-sided die.

The conditional perplexity is defined in the natural way:

\[\P[X|Y] = 2^{\H[X|Y]}\]
perplexity(dist, rvs=None, crvs=None, rv_names=None)[source]
Parameters :
  • dist (Distribution) – The distribution from which the perplexity is calculated.
  • rvs (list, None) – The indexes of the random variable used to calculate the perplexity. If None, then the perpelxity is calculated over all random variables.
  • crvs (list, None) – The indexes of the random variables to condition on. If None, then no variables are condition on.
  • rv_names (bool) – If True, then the elements of rvs are treated as random variable names. If False, then the elements of rvs are treated as random variable indexes. If None, then the value True is used if the distribution has specified names for its random variables.
Returns:

P (float) – The perplexity.

Read the Docs v: dev
Versions
latest
dev
Downloads
PDF
HTML
Epub
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.