A discrete-valued random variable is defined by its probability mass function P = {p1;P2, . .. Pn}, where 0 < p; < 1 is the probability of outcome X;. One interpretation of the Shannon entropy of a random variable X is the degree of uncertainty in the outcome of X. For discrete-valued random variables the Shannon entropy is calculated as S(X) = -Ep: log(pi).
Continuous Probability Distributions
Probability distributions are of two types, which are continuous probability distributions and discrete probability distributions. A continuous probability distribution contains an infinite number of values. For example, if time is infinite: you could count from 0 to a trillion seconds, billion seconds, so on indefinitely. A discrete probability distribution consists of only a countable set of possible values.
Normal Distribution
Suppose we had to design a bathroom weighing scale, how would we decide what should be the range of the weighing machine? Would we take the highest recorded human weight in history and use that as the upper limit for our weighing scale? This may not be a great idea as the sensitivity of the scale would get reduced if the range is too large. At the same time, if we keep the upper limit too low, it may not be usable for a large percentage of the population!
Step by step
Solved in 2 steps
