Chapter 13.1, Problem 11E

a.

To determine

Express the $i\text{th}$ residual in the form of ${\displaystyle \sum c_j{Y}_j}$.

Verify that $V\left(Y_i-{\widehat{Y}}_i\right)$ is given by the Expression 13.2.

Explanation of Solution

Calculation:

Consider,

$\begin{array}{l}\widehat{y_i}=\widehat{{\beta }_0}-\widehat{{\beta }_1}{x}_i\\ \widehat{{\beta }_0}=\overline{y}-\widehat{{\beta }_1}\overline{x}\end{array}$

Also, the sum of difference of each x value from mean is 0. That is, ${\displaystyle \sum _i^n\left(x_i-\overline{x}\right)}=0,{\displaystyle \sum _i^n\left(y_i-\overline{y}\right)}$

$\begin{array}{c}{e}_i=y_i-\widehat{y_i}\\ =y_i-\widehat{{\beta }_0}-\widehat{{\beta }_1}{x}_i\\ =y_i-\overline{y}-\widehat{{\beta }_1}\left(x_i-\overline{x}\right)\end{array}$

When $i=j$,

$\begin{array}{c}{\displaystyle \sum _j^{}{e}_i}={\displaystyle \sum _j^{}{y}_i-\overline{y}-\widehat{{\beta }_1}{\displaystyle \sum _i^n\left(x_i-\overline{x}\right)}}\\ =Y_i-\frac{1}{n}{\displaystyle \sum _j{Y}_j}-\frac{\left(x_i-\overline{x}\right){\displaystyle \sum _j\left(x_j-\overline{x}\right)}\left(Y_j-\overline{Y}\right)}{{{\displaystyle \sum _j\left(x_j-\overline{x}\right)}}^2}\\ =Y_i-\frac{1}{n}{\displaystyle \sum _j{Y}_j}-\frac{\left(x_i-\overline{x}\right){\displaystyle \sum _j\left(x_j-\overline{x}\right)}{Y}_j}{{{\displaystyle \sum _j\left(x_j-\overline{x}\right)}}^2}\\ =\left[1-\frac{1}{n}-\frac{{\left(x_i-\overline{x}\right)}^2}{n{\displaystyle \sum {\left(x_i-\overline{x}\right)}^2}}\right]Y_j\\ ={\displaystyle \sum _j{c}_j{Y}_j}\end{array}$

When $i\ne j$.

$\begin{array}{c}{e}_i=y_i-\widehat{y_i}\\ =y_i-\widehat{{\beta }_0}-\widehat{{\beta }_1}{x}_i\\ =y_i-\overline{y}-\widehat{{\beta }_1}\left(x_i-\overline{x}\right)\\ {\displaystyle \sum _j^{}{e}_i}={\displaystyle \sum _j^{}{y}_i-\overline{y}-\widehat{{\beta }_1}{\displaystyle \sum _i^n\left(x_i-\overline{x}\right)}}\end{array}$

$\begin{array}{l}=Y_i-\frac{1}{n}{\displaystyle \sum _j{Y}_j}-\frac{\left(x_i-\overline{x}\right){\displaystyle \sum _j\left(x_j-\overline{x}\right)}\left(Y_j-\overline{Y}\right)}{{{\displaystyle \sum _j\left(x_j-\overline{x}\right)}}^2}\\ =Y_i-\frac{1}{n}{\displaystyle \sum _j{Y}_j}-\frac{\left(x_i-\overline{x}\right){\displaystyle \sum _j\left(x_j-\overline{x}\right)}{Y}_j}{{{\displaystyle \sum _j\left(x_j-\overline{x}\right)}}^2}\\ =\left[1-\frac{1}{n}-\frac{\left(x_i-\overline{x}\right)\left(x_j-\overline{x}\right)}{{\displaystyle \sum {\left(x_j-\overline{x}\right)}^2}}\right]Y_j\\ ={\displaystyle \sum _j{c}_j{Y}_j}\end{array}$

Verification:

The expression given in 13.2 is given below:

$\begin{array}{l}V\left(y_i-{\widehat{y}}_i\right)={\sigma }^2\left[1-\frac{1}{n}-\frac{{\left(x_i-\overline{x}\right)}^2}{S_{xx}}\right]\\ V\left(e_i\right)={\sigma }^2\left[1-\frac{1}{n}-\frac{{\left(x_i-\overline{x}\right)}^2}{n{\displaystyle \sum {\left(x_j-\overline{x}\right)}^2}}\right]\end{array}$

Here,

$\begin{array}{c}V\left(Y-{\widehat{Y}}_i\right)={\displaystyle \sum V\left(c_j{Y}_j\right)}\\ ={\displaystyle \sum c_j^2}V\left(Y_j\right)\\ ={\displaystyle \sum c_j^2}{\sigma }^2\end{array}$

Where,

The variance is calculated by using the formula $V\left(aX\right)=a^{2}V\left(X\right)$ and $V\left(Y_j\right)={\sigma }^2$

Thus,

$\begin{array}{c}V\left(Y_i-{\widehat{Y}}_i\right)=c_j^2{\sigma }^2\\ =\left[1-\frac{1}{n}-\frac{{\left(x_i-\overline{x}\right)}^2}{n{\displaystyle \sum {\left(x_j-\overline{x}\right)}^2}}\right]{\sigma }^2\end{array}$

Hence, the $V\left(Y_i-{\widehat{Y}}_i\right)$ is equal to the Expression 13.2.

b.

To determine

Use the fact that predicted value and residuals are independentof each other, the expression for $V\left(Y\right)$ given in the Section 12.4 and the relation $Y_i={\widehat{Y}}_i+\left(Y_i-{\widehat{Y}}_i\right)$ to verify the Expression 13.2.

Explanation of Solution

Calculation:

The expression for $V\left(\widehat{Y}\right)$ given in the Section 12.4 is,

$V\left(\widehat{Y}\right)={\sigma }^2\left[\frac{1}{n}+\frac{{\left(x_i-\overline{x}\right)}^2}{S_{xx}}\right]$

The variance ${\sigma }^2$ can be expressed in the form of

$\begin{array}{c}{\sigma }^2=V\left(Y_i\right)\\ =V\left[{\widehat{Y}}_i+\left(Y_i-{\widehat{Y}}_i\right)\right]\\ =V\left({\widehat{Y}}_i\right)+V\left(Y_i-{\widehat{Y}}_i\right)\end{array}$

Substituting the $V\left(\widehat{Y}\right)={\sigma }^2\left[\frac{1}{n}+\frac{{\left(x_i-\overline{x}\right)}^2}{S_{xx}}\right]$ in the above expression yields,

$\begin{array}{c}V\left(Y_i-{\widehat{Y}}_i\right)={\sigma }^2-V\left({\widehat{Y}}_i\right)\\ ={\sigma }^2-{\sigma }^2\left[\frac{1}{n}+\frac{{\left(x_i-\overline{x}\right)}^2}{S_{xx}}\right]\\ =\left[1-\frac{1}{n}-\frac{{\left(x_i-\overline{x}\right)}^2}{S_{xx}}\right]{\sigma }^2\end{array}$

Thus, the resultant equation is same as the Expression 13.2.

c.

To determine

Identify the changes in $V\left({\widehat{Y}}_i\right)\text{ and }V\left(Y{-}_i{\widehat{Y}}_i\right)$ when $x_i$ moves away from $\overline{x}$.

Explanation of Solution

From the expression,

$\begin{array}{c}V\left(Y_i-{\widehat{Y}}_i\right)={\sigma }^2-V\left({\widehat{Y}}_i\right)\\ ={\sigma }^2-{\sigma }^2\left[\frac{1}{n}+\frac{{\left(x_i-\overline{x}\right)}^2}{S_{xx}}\right]\end{array}$

It can be observed that as $x_i$ moves away from $\overline{x}$ then $V\left({\widehat{Y}}_i\right)$ increases due to the fact that ${\left(x_i-\overline{x}\right)}^2$ has a positive sign and $V\left({\widehat{Y}}_i-Y_i\right)$ decreases because ${\left(x_i-\overline{x}\right)}^2$ has a negative sign in that expression.