Chapter 14, Problem 20CE

a.

To determine

Make the correlation matrix.

Find the independent variable that has the strongest correlation with the dependent variable.

Explain whether the strong correlation between some independent variables indicate any problem.

Answer to Problem 20CE

The correlation matrix is obtained as,

The independent variable that has the strongest correlation with the dependent variable is “Years of experience”.

Explanation of Solution

Multiple linear regression model:

A multiple linear regression model is given as $\widehat{y}=a+b_1{x}_1+b_2{x}_2+b_3{x}_3+\mathrm{...}+b_k{x}_k$ where y is the response or dependent variable, and $x_1,x_2,\mathrm{...},x_k$ are the k quantitative independent variables where k is a positive integer.

Here, a is the intercept term of the regression model, that is, the value of predicted value of y when X’s are 0 and $b_i$’s are the slopes, that is, the amount of change of the predicted value of y for one unit increase in $x_i$ when all other independent variables are constant.

Dummy variable:

A dichotomous variable is defined as a dummy variable, where one outcome is defined as 1 another as 0.

In the given problem the predicted dependent variable y is the salary. The years of experience, the Principal’s rating, and the Master’s Degree, are defined as $x_1,x_2,\text{and}{x}_3$, respectively.

The independent random variable $x_3$ is defined as dummy variable.

Hence,

$x_2=\{\begin{array}{l}1,\text{Master degree}\\ \text{0,}\text{No master degree}\end{array}$.

Step by step procedure to obtain the correlation matrix using MINITAB software is given below:

• Choose Stat > Basic Statistics > Correlation.
• Select Years, Rating and Masters under Variables tab.
• Click OK.

The MINITAB output is obtained.

According to the obtained output there is a strongest correlation between the independent variable “Years” and the dependent variable “Salary”. The correlation coefficient between “Years” and “Salary” is 0.868.

Thus, it implies that as the years of experience increases the salary also increases.

Multicollinearity:

In a multiple regression model, when there is high correlation between two or more independent variables, then multicollinearity occurs.

The correlation between the independent variables “Rating” and “Masters” is 0.458, which is maximum among the correlation between the independent variables.

Thus, there is no chance of occurrence of multicollinearity in the regression model.

b.

To determine

Find the regression equation.

Find the estimated salary for a teacher with 5 years’ experience, a rating by the principal of 60 and no master’s degree.

Answer to Problem 20CE

The regression equation is $\underset{\_}{\widehat{y}=19.92+0.8994x_1+0.1539x_2-0.67x_3}$.

The estimated salary for a teacher with 5 years’ experience, a rating by the principal of 60 and no master’s degree is $33,651.

Explanation of Solution

Calculation:

Step by step procedure to obtain the regression equation using MINITAB software:

• Choose Stat > Regression > Regression > Fit Regression Model.
• Under Responses, enter the column of Salary.
• Under Continuous predictors, enter the columns of Years, Rating and Masters.
• Click OK.

Output using MINITAB software is given below:

Thus, the regression equation is $\underset{\_}{\widehat{y}=19.92+0.8994x_1+0.1539x_2-0.67x_3}$.

Now, substitute $x_1=5$,  $x_2=60$ and $x_3=0$ in the obtained regression equation.

Hence,

$\begin{array}{c}\widehat{y}=19.92+0.8994\left(5\right)+0.1539\left(60\right)-0.67\left(0\right)\\ =19.92+4.497+9.234-0\\ =33.651\end{array}$

Thus, the estimated salary for a teacher with 5 years’ experience, a rating by the principal of 60 and no master’s degree is $33,651.

c.

To determine

Perform a global hypothesis test to check whether any of the regression coefficients differ from zero at 0.05 level of significance.

Answer to Problem 20CE

There is strong evidence that any of the regression coefficient differ from 0 at 0.05 significance level.

Explanation of Solution

Calculation:

Consider that y is dependent variable and $x_i\text{'}\text{s}$ are the independent variables where ${\beta }_i\text{'}\text{s}$ are the corresponding population regression coefficient for all $i=1,2,3$.

State the hypotheses:

Null hypothesis:

$H_0:{\beta }_1={\beta }_2={\beta }_3=0$.

That is, the model is not significant.

Alternative hypothesis:

$H_1:\text{At least one }{\beta }_i\text{ is not equal to }0$.

That is, the model is significant.

In case of global test the F test statistic is defined as,

$F=\frac{\frac{\text{SSR}}{k}}{\frac{\text{SSE}}{n-k-1}}$, where SSR, SSE, n and k are the regression sum of square, error sum of square, sample size and the number of independent variables.

According to the output in Part (b) the value of F statistic is 52.72 with numerator degrees of freedom 3 and denominator degrees of freedom 16.

The level of significance is $\alpha =0.05$.

Decision rule:

• If $p\text{-value}\le \alpha$, then reject the null hypothesis.
• Otherwise failed to reject the null hypothesis.

Conclusion:

Here, p-value corresponding to the global test is 0.

Hence, $p\text{-value}\left(=0\right)<\alpha \left(=0.05\right)$.

That is, the p-value is less than the level of significance.

Therefore, reject the null hypothesis.

Hence, it can be concluded that any of the regression coefficient differ from 0 at 0.05 significance level.

d.

To determine

Perform individual tests of each independent variable at 0.05 significance level.

Explain whether any of the independent variable will be eliminated.

Answer to Problem 20CE

There is no significant relation between y and $x_3$, whereas there is significant relation between y and $x_1\text{and}{x}_2$.

The independent random variable “Master’s degree” can be eliminated.

Explanation of Solution

Calculation:

For independent variable $x_1$:

Consider that ${\beta }_1$ is the population regression coefficient of independent variable $x_1$.

State the hypotheses:

Null hypothesis:

$H_0:{\beta }_1=0$.

That is, there is no significant relationship between y and $x_1$.

Alternative hypothesis:

$H_1:{\beta }_1\ne 0$.

That is, there is significant relationship between y and $x_1$.

In case of individual regression coefficient test the t test statistic is defined as,

$t=\frac{b_i}{s_{b_i}}$, where $b_i$ and $s_{b_i}$ are the i^th regression coefficient and the standard deviation of the i^th regression coefficient.

According to the output in Part (a) the t statistic value corresponding to $x_1$ is 10.26 with 16 degrees of freedom.

Conclusion:

Here, p-value corresponding to the “Years of experience”$\left(x_1\right)$ is 0.

Hence, $p\text{-value}\left(=0\right)<\alpha \left(=0.05\right)$.

That is, the p-value is less than the level of significance.

Therefore, reject the null hypothesis.

Hence, it can be concluded that there is significant relationship between y and $x_1$.

For independent variable $x_2$:

Consider that ${\beta }_2$ is the population regression coefficient of independent variable $x_2$.

State the hypotheses:

Null hypothesis:

$H_0:{\beta }_2=0$.

That is, there is no significant relationship between y and $x_2$.

Alternative hypothesis:

$H_1:{\beta }_2\ne 0$.

That is, there is significant relationship between y and $x_2$.

According to the output in Part (a) the value of t test statistic corresponding to $x_2$ is 4.9 with 16 degrees of freedom.

Conclusion:

Here, p-value corresponding to the “Principal’s rating”$\left(x_2\right)$ is 0.

Hence, $p\text{-value}\left(=0\right)<\alpha \left(=0.05\right)$.

That is, the p-value is less than the level of significance.

Therefore, reject the null hypothesis.

Hence, it can be concluded that there is significant relationship between y and $x_2$.

For independent variable $x_3$:

Consider that ${\beta }_3$ is the population regression coefficient of independent variable $x_3$.

State the hypotheses:

Null hypothesis:

$H_0:{\beta }_3=0$.

That is, there is no significant relationship between y and $x_3$.

Alternative hypothesis:

$H_1:{\beta }_3\ne 0$.

That is, there is significant relationship between y and $x_3$.

According to the output in Part (a) the value of t test statistic corresponding to $x_3$ is –0.59 with 5 degrees of freedom.

Conclusion:

Here, p-value corresponding to the income $\left(x_3\right)$ is 0.59.

Hence, $p\text{-value}\left(=0.59\right)>\alpha \left(=0.05\right)$.

That is, the p-value is greater than the level of significance.

Therefore, fail to reject the null hypothesis.

Hence, it can be concluded that there is no significant relationship between y and $x_3$.

As there are no significant relationship between the dependent variable and the independent variable $x_3$, it is better to eliminate this variable.

Hence, it can be said that there is no significant relationship between the salary and the Master’s degree. Thus, it is better to omit the independent random variable “Master’s degree”.

e.

To determine

Perform the regression analysis by omitting the insignificant random variable.

Explanation of Solution

Calculation:

The regression analysis is performed after omitting the independent variable Master’s degree.

Step by step procedure to obtain the regression equation using MINITAB software:

• Choose Stat > Regression > Regression > Fit Regression Model.
• Under Responses, enter the column of Salary.
• Under Continuous predictors, enter the columns of Years, and Rating
• Choose Graphs.
• Under Residual plot select Histogram of residuals and Residual Versus fit.
• Click OK.
• Choose Storage.
• Under Regression storage select Residuals.
• Click OK
• Click OK.

Output using MINITAB software is given below:

Residuals
–1.2802
2.726697
–0.0664
0.711748
–2.0206
0.676788
2.725551
2.531216
0.795113
–4.26789
0.300234
–0.41968
–1.14257
0.284365
–2.38135
–2.67229
1.368873
–2.06188
–0.81143
5.00372

Thus, the regression equation is $\underset{\_}{\widehat{y}=20.12+0.8926x_1+{0.1464}_2}$.

Global test:

State the hypotheses:

Null hypothesis:

$H_0:{\beta }_1={\beta }_2=0$.

That is, the model is not significant.

Alternative hypothesis:

$H_1:\text{At least one }{\beta }_i\text{ is not equal to }0$.

That is, the model is significant.

In case of global test the F test statistic is defined as,

$F=\frac{\frac{\text{SSR}}{k}}{\frac{\text{SSE}}{n-k-1}}$, where SSR, SSE, n and k are the regression sum of square, error sum of square, sample size and the number of independent variables.

According to the output in the value of F statistic is 82.31 with numerator degrees of freedom 2 and denominator degrees of freedom 17.

Conclusion:

Here, p-value corresponding to the global test is 0.

Hence, $p\text{-value}\left(=0\right)<\alpha \left(=0.05\right)$.

That is, the p-value is less than the level of significance.

Therefore, reject the null hypothesis.

Hence, it can be concluded that any of the regression coefficient differ from 0 at 0.05 significance level.

Individual test:

For independent variable $x_1$:

Null hypothesis:

$H_0:{\beta }_1=0$.

That is, there is no significant relationship between y and $x_1$.

Alternative hypothesis:

$H_1:{\beta }_1\ne 0$.

That is, there is significant relationship between y and $x_1$.

According to the output the t statistic value corresponding to $x_1$ is 10.5 with 17 degrees of freedom.

Conclusion:

Here, p-value corresponding to the “Years of experience”$\left(x_1\right)$ is 0.

Hence, $p\text{-value}\left(=0\right)<\alpha \left(=0.05\right)$.

That is, the p-value is less than the level of significance.

Therefore, reject the null hypothesis.

Hence, it can be concluded that there is significant relationship between y and $x_1$.

For independent variable $x_2$:

Null hypothesis:

$H_0:{\beta }_2=0$.

That is, there is no significant relationship between y and $x_2$.

Alternative hypothesis:

$H_1:{\beta }_2\ne 0$.

That is, there is significant relationship between y and $x_2$.

According to the output the value of t test statistic corresponding to $x_2$ is 5.28 with 17 degrees of freedom.

Conclusion:

Here, p-value corresponding to the “Principal’s rating”$\left(x_2\right)$ is 0.

Hence, $p\text{-value}\left(=0\right)<\alpha \left(=0.05\right)$.

That is, the p-value is less than the level of significance.

Therefore, reject the null hypothesis.

Hence, it can be concluded that there is significant relationship between y and $x_2$.

f.

To determine

Find the residuals for the equation of Part (e).

Find a stem-and-leaf plot or a histogram to verify whether the distribution of the residuals is approximately normal.

Explanation of Solution

Calculation:

From Part (e), the residuals are obtained as,

Residuals
–1.2802
2.726697
–0.0664
0.711748
–2.0206
0.676788
2.725551
2.531216
0.795113
–4.26789
0.300234
–0.41968
–1.14257
0.284365
–2.38135
–2.67229
1.368873
–2.06188
–0.81143
5.00372

Histogram:

From Part (e), the histogram is obtained as,

Assumption of normality from histogram:

• The majority of the observation in the middle and centered on the mean of 0.
• There are lower frequencies on the tails of the distributions.

According to the given histogram, the most of the observations are centered on the mean of 0 and there are less frequencies on the tails of the distributions. However, it is roughly symmetric.

Hence, the normality assumptions appear somehow.

g.

To determine

Plot the residual plot.

Explain whether the plot reveals any violations of assumptions of regression.

Explanation of Solution

From Part (e), the residual plot is obtained as,

Assumption for residual analysis for the regression model:

• The plot of the residuals vs. the observed values of the predictor variable should fall roughly in a horizontal band and symmetric about x-axis.
• For a normal probability plot, residuals should be roughly linear.
• There should not be any observable pattern.

According to the given residual plot, the points are roughly in a horizontal band and more or less symmetric about x-axis. Moreover, there is no particular pattern in the residual plot. A complete haphazard and random nature has observed.

Hence, the assumptions of the residual plot are not violated.