Chapter 9, Problem 104SE

a.

To determine

Provide an estimator ${\widehat{\theta }}_1$ for $\theta$ by the method of moments.

Answer to Problem 104SE

The method of moments estimator of $\theta$ is ${\widehat{\theta }}_1=\overline{Y}-1$.

Explanation of Solution

Calculation:

The expectation of the given random variable Y is obtained as follows:

$\begin{array}{c}E\left(Y\right)={\displaystyle \underset{\theta }{\overset{\infty }{\int }}y{e}^{-\left(y-\theta \right)}dy}\\ =e^{\theta }{\displaystyle \underset{\theta }{\overset{\infty }{\int }}y{e}^{-y}dy}\\ =e^{\theta }\left({\left[-y{e}^{-y}\right]}_{\theta }^{\infty }+{\displaystyle \underset{\theta }{\overset{\infty }{\int }}{e}^{-y}\left(\frac{dy}{dy}\right)dy}\right)\\ =e^{\theta }\left(\theta e^{-\theta }+{\left[-e^{-y}\right]}_{\theta }^{\infty }\right)\\ =e^{\theta }\left(\theta e^{-\theta }+e^{-\theta }\right)\\ =\theta +1\end{array}$

The method of moments estimator of $\theta$ is ${\widehat{\theta }}_1$.

Now, to obtained the method of moments estimator of $\theta$, it is needed to equate the expectation with the sample mean. That is,

$\begin{array}{c}{\widehat{\theta }}_1+1=\overline{Y}\\ {\widehat{\theta }}_1=\overline{Y}-1\end{array}$

Hence, the method of moments estimator of $\theta$ is ${\widehat{\theta }}_1=\overline{Y}-1$.

b.

To determine

Provide an estimator ${\widehat{\theta }}_2$ for $\theta$ by the method of maximum likelihood.

Answer to Problem 104SE

An estimator ${\widehat{\theta }}_2$ for $\theta$ by the method of maximum likelihood is ${\widehat{\theta }}_2=Y_{\left(1\right)}$.

Explanation of Solution

Calculation:

Consider an indicator function $I(.)$, such that, the value of the indicator function will be 1 if the condition is true otherwise the indicator function will take the value as 0.

Now, the likelihood function of the given random variable Y is defined as follows:

$\begin{array}{c}L\left(Y_1,Y_2,\mathrm{....},Y_n|\theta \right)={\displaystyle \prod _{i=1}^n{e}^{-n\left(Y_i-\theta \right)}I\left(Y_i\ge \theta \right)}\\ =e^{-n\left({\displaystyle \sum _{i=1}^n{Y}_i}-\theta \right)}{\displaystyle \prod _{i=1}^{n}I\left(Y_i\ge \theta \right)}\\ =e^{-\left({\displaystyle \sum _{i=1}^n{Y}_i}-n\theta \right)}I\left(Y_{\left(1\right)}\ge \theta \right)\end{array}$

To maximize the likelihood it is needed to choose $\theta$ such that the $\theta$ is smaller than all $Y_i\text{'}\text{s}$. Hence, if the lowest value of $Y_i\text{'}\text{s}$ is greater than $\theta$, then all of the other values are also greater than $\theta$.

So, it is needed to choose $\theta$ as large as that $I\left(Y_{\left(1\right)}\ge \theta \right)=1$.

Hence, ${\widehat{\theta }}_2=Y_{\left(1\right)}$.

Hence, an estimator ${\widehat{\theta }}_2$ for $\theta$ by the method of maximum likelihood is ${\widehat{\theta }}_2=Y_{\left(1\right)}$.

c.

To determine

Make some adjustment to make ${\widehat{\theta }}_1\text{and }{\widehat{\theta }}_2$ as unbiased.

Also find the efficiency of the adjusted ${\widehat{\theta }}_1$ relative to the adjusted ${\widehat{\theta }}_2$.

Explanation of Solution

Calculation:

From Part (a), it is obtained that method of moments estimator of $\theta$ is ${\widehat{\theta }}_1=\overline{Y}-1$.

It is also known that $E\left(\overline{Y}\right)=\theta +1$.

That is,

$\begin{array}{c}E\left({\widehat{\theta }}_1\right)=E\left(\overline{Y}\right)-1\\ =\theta +1-1\\ =\theta \end{array}$

That is, ${\widehat{\theta }}_1$ is itself an unbiased estimator of $\theta$.

The distribution function of $Y_{\left(1\right)}$ is obtained as,

$\begin{array}{c}{F}_{Y_{\left(1\right)}}\left(y\right)=1-P\left(Y_{\left(1\right)}\ge y\right)\\ =1-P\left(Y_1\ge y\right)\mathrm{...}P\left(Y_n\ge y\right)\\ =1-{\left[e^{-\left(y-\theta \right)}\right]}^n\\ =1-e^{-n\left(y-\theta \right)}\end{array}$

The probability density function of $Y_{\left(1\right)}$ is obtained as,

$\begin{array}{c}{f}_{Y_{\left(1\right)}}\left(y\right)=\frac{d\left(F_{Y_{\left(1\right)}}\left(y\right)\right)}{dy}\\ =\frac{d\left(1-e^{-n\left(y-\theta \right)}\right)}{dy}\\ =n{e}^{-n\left(y-\theta \right)}\end{array}$

Hence,

$\begin{array}{c}E\left(Y_{\left(1\right)}\right)={\displaystyle \underset{\theta }{\overset{\infty }{\int }}yn{e}^{-n\left(y-\theta \right)}dy}\\ =e^{n\theta }{\displaystyle \underset{\theta }{\overset{\infty }{\int }}yn{e}^{-ny}dy}\\ =\frac{e^{n\theta }}{n}{\displaystyle \underset{n\theta }{\overset{\infty }{\int }}t{e}^{-t}dt}\\ =\frac{e^{n\theta }}{n}\left[n\theta e^{-n\theta }+e^{-n\theta }\right]\\ =\theta +\frac{1}{n}\end{array}$

Thus,

$\begin{array}{c}E\left({\widehat{\theta }}_2\right)=E\left(Y_{\left(1\right)}\right)\\ E\left({\widehat{\theta }}_2\right)=\theta +\frac{1}{n}\\ E\left({\widehat{\theta }}_2\right)-\frac{1}{n}=\theta \end{array}$

Thus, the unbiased estimator of $\theta$ is $E\left({\widehat{\theta }}_2\right)-\frac{1}{n}\left(={\widehat{\theta }}_2{}^{*}\right)$.

Now,

$\begin{array}{c}E\left(Y_{\left(1\right)}^2\right)={\displaystyle \underset{\theta }{\overset{\infty }{\int }}{y}^{2}n{e}^{-n\left(y-\theta \right)}dy}\\ =e^{n\theta }{\displaystyle \underset{\theta }{\overset{\infty }{\int }}{y}^{2}n{e}^{-ny}dy}\\ =\frac{e^{n\theta }}{n^2}{\displaystyle \underset{n\theta }{\overset{\infty }{\int }}{t}^2{e}^{-t}dt}\left[\text{consider }ny=t\right]\\ =\frac{e^{n\theta }}{n^2}\left[n^2{\theta }^2{e}^{-n\theta }+2n1e^{-n\theta }+2e^{-n\theta }\right]\\ ={\theta }^2+\frac{2\theta }{n}+\frac{2}{n^2}\end{array}$

This, the variance of $Y_{\left(1\right)}$ is,

$\begin{array}{c}V\left(Y_{\left(1\right)}\right)=E\left(Y_{\left(1\right)}^2\right)-E^2\left(Y_{\left(1\right)}\right)\\ ={\theta }^2+\frac{2\theta }{n}+\frac{2}{n^2}-{\left(\theta +\frac{1}{n}\right)}^2\\ ={\theta }^2+\frac{2\theta }{n}+\frac{2}{n^2}-{\theta }^2-\frac{1}{n^2}-\frac{2\theta }{n}\\ =\frac{1}{n^2}\end{array}$

Thus,

$\begin{array}{c}V\left({\widehat{\theta }}_2\right)=V\left(Y_{\left(1\right)}\right)\\ V\left({\widehat{\theta }}_2\right)=\frac{1}{n^2}\end{array}$

Now, the expected value of $Y^2$ is obtained as follows:

$\begin{array}{c}E\left(Y^2\right)={\displaystyle \underset{\theta }{\overset{\infty }{\int }}{y}^2{e}^{-\left(y-\theta \right)}dy}\\ =e^{\theta }{\displaystyle \underset{\theta }{\overset{\infty }{\int }}{y}^2{e}^{-y}dy}\\ =e^{\theta }\left({\theta }^2{e}^{-\theta }+2\theta e^{-\theta }++2e^{-\theta }\right)\\ ={\theta }^2+2\theta +1\end{array}$

Thus, the variance of Y is,

$\begin{array}{c}V\left(Y\right)=E\left(Y^2\right)-E^2\left(Y\right)\\ ={\theta }^2+2\theta +2-{\theta }^2-2\theta -1\\ =1\end{array}$

Thus,

$\begin{array}{c}V\left(\overline{Y}\right)=V\left(\frac{{\displaystyle \sum _{i-1}^n{Y}_i}}{n}\right)\\ =\frac{{\displaystyle \sum _{i-1}^{n}V\left(Y_i\right)}}{n^2}\\ =\frac{n}{n^2}\\ =\frac{1}{n}\end{array}$

Thus, the efficiency of the adjusted ${\widehat{\theta }}_1$ relative to the adjusted ${\widehat{\theta }}_2$ is,

$\begin{array}{c}Eff\left({\widehat{\theta }}_1,{\widehat{\theta }}^{*}{}_2\right)=\frac{Var\left({\widehat{\theta }}^{*}{}_2\right)}{Var\left({\widehat{\theta }}_1\right)}\\ =\frac{\frac{1}{n^2}}{\frac{1}{n}}\\ =\frac{1}{n}\end{array}$

Thus, the efficiency of the adjusted ${\widehat{\theta }}_1$ relative to the adjusted ${\widehat{\theta }}_2$ is $\frac{1}{n}$.