Let X be a discrete random variable taking values {x1, x2, . . . , xn} with probability {p1, p2, . . . , pn}. The entropy of the random variable is defined as H(X) = −sigma(pilog(pi)) ( where sigma takes value i=1 to n ) Find the probability mass function for the above discrete random variable that maximizes the entropy
Continuous Probability Distributions
Probability distributions are of two types, which are continuous probability distributions and discrete probability distributions. A continuous probability distribution contains an infinite number of values. For example, if time is infinite: you could count from 0 to a trillion seconds, billion seconds, so on indefinitely. A discrete probability distribution consists of only a countable set of possible values.
Normal Distribution
Suppose we had to design a bathroom weighing scale, how would we decide what should be the range of the weighing machine? Would we take the highest recorded human weight in history and use that as the upper limit for our weighing scale? This may not be a great idea as the sensitivity of the scale would get reduced if the range is too large. At the same time, if we keep the upper limit too low, it may not be usable for a large percentage of the population!
Let X be a discrete random variable taking values {x1, x2, . . . , xn} with
of the random variable is defined as
H(X) = −sigma(pilog(pi)) ( where sigma takes value i=1 to n )
Find the probability mass
Trending now
This is a popular solution!
Step by step
Solved in 2 steps
