Stat_TF

STAT 151A Final Review 2023 Part 1 4. (TRUE / FALSE) If an observation with high leverage is dropped from the model fitting, then the estimate for the coefficients will become more precise. FALSE . If the observation is far from the mean ¯ x and falls exactly on the regression line (that is, it is not an outlier), then the estimate will become less precise when the observation is removed. 6. (TRUE / FALSE) The maximum likelihood estimator is a biased estimator. FALSE . The MLE for β in the regular linear model is unbiased since it is just the OLS estimator, which we know is unbiased. 7. (TRUE / FALSE) If AIC and Mallow’s C p choose a best model and these both have the same number of parameters, then the models are identical (i.e., they include the same explanatory variables). TRUE . Models with the same number of parameters will be placed in the same order by C p and AIC because the ordering depends only on the sum of squared deviations. 9. (TRUE / FALSE) Suppose you have a logistic regression model with p parameters. If you add another variable to the existing model, the log-likelihood l ( ˆ β ) will never decrease. TRUE . The first model is a specialization of the second (since the coefficient of the added variable is 0 for the first model), so the log-likelihood of the first model will be at most the log-likelihood of the second and larger model. 10. (TRUE / FALSE) In linear regression it is better to use ANOVA (F-test) to de cide whether a single (continuous) explanatory variable is needed in the model, rather than the standard t -test that is reported in summary (Im). FALSE . Both will give the same p-values when testing a single coefficient. 11. (TRUE / FALSE) In a model with p continuous explanatory variables, Mallow’s C p must be p + 1. FALSE . We have C p = ( p + 1) + ( k − p )( F − 1). Since F is non-negative and can be less than 1 , C p can be less than p. + 1 12. (TRUE / FALSE) R 2 can be used as a model selection criterion in the linear model. FALSE . R 2 does not decrease as we add variables to the model. 1

Part 2 6. Indicate whether the following statements are TRUE or FALSE by circling your choice. Explain your answer with a counter example, short proof, or a clear argument. (3 points each) Assume the linear model y = X β + ϵ where X is a fixed matrix and ϵ ∼ N ( 0 , σ 2 I ) (a) (True/False): You are using OLS to fit a regression equation. If you exclude a variable from an equation, but the excluded variable is orthogonal to the other variables in the equation, you will not bias the estimated coefficients of the remaining variables TRUE . (b) (True/False): R 2 is a measure of the validity of your model. FALSE . (c) (True/False): Collinearity leads to bias in the estimated standard errors for the slope coefficients. FALSE . (leads to higher estimated SEs) (d) (True/False): Mallows’s C p for the full model always equals k +1 where k is the number of explanatory variables in the model. FALSE . (e) (True/False): For the residual bootstrap to work for inference in the linear model, it is necessary that some of the OLS assumptions be violated. FALSE . (f) (True/False): The leverages are always between 0 and 1 inclusive 0 ≤ h ii ≤ 1. TRUE . (h) (True/False): Over the period 1950-1999, the correlation between the population density in the US and the death rate from lung cancer was 0.92 . This indicates that population density may be a cause of lung cancer. FALSE . (i) (True/False): If I regress the residuals from the fitted linear model on one of the ex- planatory variables from the model, then the estimated slope coefficient will be identically 0. TRUE . 2

(d) In a model with p continuous explanatory variables, Mallow’s C p must be ≥ p + 1. (2 points). FALSE . The formula for C p is p +1+( k − p )( F − 1), where k is the number of non-intercept parameters in the full model, p is the number in the model under current consideration, and F is the F -statistic comparing the reduced and full. models. C p will be lower than p + 1 whenever p < k and F < 1 (an event which happens with nonzero probability under the null hypothesis that the reduced model is correct). (e) R 2 is an effective model selection criterion for deciding the best size for a linear model. (2 points). FALSE . R 2 always increases as more variables are added to the linear model, so it will always choose a larger model over a submodel, even if the additional covariates used are pure noise. (f) In simple regression, rejecting the null hypothesis that β = 0 is equivalent to concluding that x has a causal effect on y . (2 points). FALSE . Even if changing x has no effect on y , if both x and y are highly correlated with a third variable z not present in the regression, then x and y may have a strong correlation. In this case, the null hypothesis β = 0 will likely be rejected but this is not enough to demonstrate a causal effect. Midterm 2018 (a) (True/False) Let e be the residuals from the least squares fit for the linear model and ˆ y be the fitted values, then we have that e ⊥ ˆ y . TRUE . e = ( I − H ) y ˆ y = HY H = X ( X T X ) − 1 X T Thus, e T ˆ y = y ( I − H ) Hy = y ( H − H 2 ) y = y ( H − H ) y = 0 (b) (True/False) The residuals are homoscedastic. FALSE . Variance of residuals equal var( e ) = var(( I − H ) y ) = σ 2 ( I − H ) var( y ) = σ 2 I The variances are the diagonal elements of σ 2 ( I − H ). Thus, var ( e i ) = σ 2 (1 − h ii ) 4

which need not be the same. (c) (True/False) Regressing the residuals e on one of the explanatory variables x j will give us the partial coefficient of x j from the full model. FALSE . It should give us 0 since residuals are orthogonal to the span(X). (d) (True/False) Anthropologists often use a regression model to estimate the living height of a deceased person (”living stature”), using the maximal femur (thighbone) length and tibia (shinbone) length as explanatory variables. A graduate student in physical anthropology fits such a model and rejects the hypothesis that the coefficient of the variable tibia is zero, since the P -value is less than 0.0001 . This must mean that the estimated coefficient for tibia is large. FALSE . That only means ˆ β SE ( ˆ β ) is large. (e) (True/False) The residuals e and the estimated coefficients ˆ β are independent. TRUE . e = ( I − H ) Y and ˆ β = X ( X T X ) − 1 X T Y . Since both e and ˆ β are linear transfor- mations of a multivariate normal (MVN) random variable Y, [ e, ˆ β ] T is also MVN. Thus, to show independence, we need only show the Cov( e, ˆ β ) = 0 Cov( e, ˆ β ) = Cov ( I − H ) Y, X ( X T X ) − 1 X T Y = ( I − H ) Cov( Y ) X ( X T X ) − 1 X T = σ 2 ( I − H ) X ( X T X ) − 1 X T = 0 since ( I − H ) X = X − HX = X − X = 0 (f) (True/False) In the summary of regression that we see in R (from lm ), we see the value of the F -statistic, along with a P -value. If this statistic is significant, the linear model is validated. FALSE . The F-test assumes a model under the null and cannot be used to tell if the model is appropriate. If the test statistic is significant, it means that at least one of the coefficients is statistically significantly different from 0 , but we do not know which of them are 0 . (g) (True/False) The assumption of normality is essential in order show that the estimator of the coefficients is the best linear unbiased estimator. FALSE . Normality is necessary for exact inference of coefficient estimates under small sample sizes. Not needed for Gauss Markov Theorem. 5

6. (TRUE / FALSE) For a multiple regression with p explanatory variables and an intercept, it is always the case that the predicted value for (¯ x 1 , ¯ x 2 , . . . ¯ x p ) is ¯ y . TRUE . X y i = X ˆ y i + X e i so ¯ y = ˆ y and X ˆ y i = nα + ˆ β 1 X x 1 ,i + · · · + ˆ β p X x p,i which implies ¯ y = ˆ y = α + ˆ β 1 x 1 + · · · + ˆ β p x p 7. (TRUE / FALSE) If you exclude a variable from the equation, but the excluded variable is orthogonal to the other variables in the equation, you won’t bias the estimated coefficients of the remaining variables. TRUE . Since x is orthogonal to the other variables, it’s coefficient in the multiple regression is determined by the projection of y onto x (i.e. is the same as for the SLR) and so dropping it from the MLR will not change (or bias) the estimates of the coefficients of the remaining variables. 8. (TRUE / FALSE) Var( β ) = σ 2 ( X ′ X ) − 1 , where X is the design matrix. FALSE . This is the variance-covariance of the estimator ˆ β . The parameter β is constant. 7

9. (TRUE / FALSE) Collinearity in the explanatory variables in the model leads to bias in the estimated coefficients. FALSE . It leads to an inflated SE of the coefficients. 10. (TRUE / FALSE) E ( ϵ ′ ϵ ) = nσ 2 , where ϵ is the n × 1 vector of ϵ i . TRUE . E ( ϵ ′ ϵ ) = E (∑ i ϵ 2 i ) = n E ( ϵ 2 1 ) = nσ 2 11. (TRUE / FALSE) Consider the simple linear model where we regress y on the categorical variable z where z has 3 levels. Let the design matrix consist of the 1 vector and two dummy vectors D 1 and D 2 for the first two levels of z . The columns of the design matrix are orthogonal. FALSE . The two dummy vectors D 1 and D 2 are orthogonal, but the 1 vector is not orthogonal to D 1 or to D 2 . 8