Midterm 2 Practice Test Solutions

1. Variable Transformations First, we’d like to build a simple linear regression that predicts poverty with median_hh_income. The plot below shows the relationship between these two variables. 1.1. Linearity Assumption We then fit the following three linear regression models that involve these two variables. A. 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 � = 35.25 − 0.0004 𝑚𝑚𝑝𝑝𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 _ ℎℎ _ 𝑚𝑚𝑚𝑚𝑖𝑖𝑝𝑝𝑚𝑚𝑝𝑝 B. log ( 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 ) � = 3.95 − 0.00002498 𝑚𝑚𝑝𝑝𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 _ ℎℎ _ 𝑚𝑚𝑚𝑚𝑖𝑖𝑝𝑝𝑚𝑚𝑝𝑝 C. 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 2 � = 997.13 − 0.0136 𝑚𝑚𝑝𝑝𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 _ ℎℎ _ 𝑚𝑚𝑚𝑚𝑖𝑖𝑝𝑝𝑚𝑚𝑝𝑝 Match the model to the most likely corresponding fitted values vs. residuals plot. Explanation not needed, but may help with partial credit if you are wrong.

2. Slope Interpretations Next, suppose we fit the following linear regression model that predicts unemployment_rate given: • Per_capita_income • Median_hh_income • Homeownership rate Here is some information about these four numerical variables in the training dataset. Correlations Summary Statistics

3. Regularization Models 3.1. Model Slope Matching LASSO and Ridge Regression Suppose that we fit the following models with our scaled features matrix as well as the target array for the training dataset. Each of these models predicts unemployment rate with: median_hh_income, per_capita_income, homeownership, and multi_unit. 1. Non-Regularized Linear Regression – Model D a. This model should have no zero slopes and it’s squared slope sum should be larger than that of the two ridge regression models. 2. LASSO linear regression with 𝜆𝜆 = 0.1 – Model C a. A LASSO model is allowed to have zero slopes. There are two models with zero slopes. One has two zero slopes, the other has one zero slope. The LASSO model with the higher 𝜆𝜆 is likely to have more zero slopes. 3. LASSO linear regression with 𝜆𝜆 = 0.05 – Model E a. A LASSO model is allowed to have zero slopes. There are two models with zero slopes. One has two zero slopes, the other has one zero slope. The LASSO model with the higher 𝜆𝜆 is likely to have more zero slopes. 4. Linear ridge regression with 𝜆𝜆 = 100 – Model A a. A ridge regression model should have no zero slopes and a ridge regression model with the highest 𝜆𝜆 should have a squared slope sum that is the smallest compared to that of the other ridge regression model and the non-regularized linear regression model. 5. Linear ridge regression with 𝜆𝜆 = 10 – Model B a. A ridge regression model should have no zero slopes and a ridge regression model with the highest 𝜆𝜆 should have a squared slope sum that is the smallest compared to that of the other ridge regression model and the non-regularized linear regression model. Match the types of models (1-5 above) to each of the corresponding model slopes below (A-E).

4. Fitting a Logistic Regression Model We fit a logistic regression model that predicts the probability that a county has a metropolitan area given the following explanatory variables: • Median_education level: (some_college, hs_diploma, bachelors , below_hs) • Unemployment rate 4.1. Use the summary output table above to write out the logistic regression equation that predicts the probability that a county has a metropolitan county. Make sure to use the appropriate notation seen in class. 𝑝𝑝̂ = 1 1 + 𝑝𝑝 − ( 1 . 6703−19 . 1483𝑚𝑚𝑝𝑝𝑚𝑚𝑟𝑟𝑟𝑟𝑚𝑚 _ 𝑝𝑝𝑚𝑚𝑒𝑒 [ 𝑏𝑏𝑝𝑝𝑏𝑏𝑝𝑝𝑏𝑏 _ ℎ𝑠𝑠 ] −2 . 655𝑚𝑚𝑝𝑝𝑚𝑚𝑟𝑟𝑟𝑟𝑚𝑚 _ 𝑝𝑝𝑚𝑚𝑒𝑒 [ ℎ𝑠𝑠 _ 𝑚𝑚𝑟𝑟𝑝𝑝𝑏𝑏𝑝𝑝𝑚𝑚𝑟𝑟 ] −1 . 4544𝑚𝑚𝑝𝑝𝑚𝑚𝑟𝑟𝑟𝑟𝑚𝑚 _ 𝑝𝑝𝑚𝑚𝑒𝑒 [ 𝑠𝑠𝑝𝑝𝑚𝑚𝑝𝑝 _ 𝑐𝑐𝑝𝑝𝑏𝑏𝑏𝑏𝑝𝑝𝑐𝑐𝑝𝑝 ] −0 . 0169𝑒𝑒𝑚𝑚𝑝𝑝𝑚𝑚𝑝𝑝𝑏𝑏𝑝𝑝𝑝𝑝𝑚𝑚𝑝𝑝𝑚𝑚𝑝𝑝 _ 𝑝𝑝𝑟𝑟𝑝𝑝𝑝𝑝 )) 4.2. Use the summary output table above to write out the logistic regression equation that predicts the odds that a county has a metropolitan county. Make sure to use the appropriate notation seen in class. 𝑝𝑝𝑚𝑚𝑚𝑚𝑜𝑜 � = 𝑝𝑝̂ 1 − 𝑝𝑝̂ = 𝑝𝑝 ( 1 . 6703−19 . 1483𝑚𝑚𝑝𝑝𝑚𝑚𝑟𝑟𝑟𝑟𝑚𝑚 _ 𝑝𝑝𝑚𝑚𝑒𝑒 [ 𝑏𝑏𝑝𝑝𝑏𝑏𝑝𝑝𝑏𝑏 _ ℎ𝑠𝑠 ] −2 . 655𝑚𝑚𝑝𝑝𝑚𝑚𝑟𝑟𝑟𝑟𝑚𝑚 _ 𝑝𝑝𝑚𝑚𝑒𝑒 [ ℎ𝑠𝑠 _ 𝑚𝑚𝑟𝑟𝑝𝑝𝑏𝑏𝑝𝑝𝑚𝑚𝑟𝑟 ] −1 . 4544𝑚𝑚𝑝𝑝𝑚𝑚𝑟𝑟𝑟𝑟𝑚𝑚 _ 𝑝𝑝𝑚𝑚𝑒𝑒 [ 𝑠𝑠𝑝𝑝𝑚𝑚𝑝𝑝 _ 𝑐𝑐𝑝𝑝𝑏𝑏𝑏𝑏𝑝𝑝𝑐𝑐𝑝𝑝 ] −0 . 0169𝑒𝑒𝑚𝑚𝑝𝑝𝑚𝑚𝑝𝑝𝑏𝑏𝑝𝑝𝑝𝑝𝑚𝑚𝑝𝑝𝑚𝑚𝑝𝑝 _ 𝑝𝑝𝑟𝑟𝑝𝑝𝑝𝑝 ))

6.3. Predict the log - odds that a county with an unemployment rate of 6 (percent) that has a median education level of “hs_diploma” has a metropolitan area. Put these numerical odds into prose format. ln �𝑝𝑝𝑚𝑚𝑚𝑚𝑜𝑜 � � = ln � 𝑝𝑝̂ 1 − 𝑝𝑝̂ � = ln � 0.25 1 − 0.25 � = ln(0.33) = − 1.1087 7. Classification Classify the county with an unemployment rate of 6 (percent) that has a median education level of “hs_diploma” is a metropolitan county as either: having or not having a metropolitan area. Use a predictive probability threshold of 𝑝𝑝̂ 0 = 0.5 to classify this observation. Because 𝑝𝑝̂ = 0.25 < 0.5 , we classify it as a 0 (ie. a county that does not have a metropolitan area). Part 3 – Conceptual Questions The questions in this section are conceptual and do not necessarily relate to the dataset discussed in part 1 and 2. 8. R^2 and Adjusted R^2 Conceptual Questions Select whether each statement below is True or False. 8.1. The adjusted R^2 of a model represents the percent of response variable variability that is explained by the model. False . This is the definition of R^2. The adjusted R^2 is simply just a metric that measures parsimoniousness of a linear regression model. 8.2. The adjusted R^2 is a measure of how parsimonious a linear regression model is. True . 8.3. By adding another explanatory variable to a linear regression model, the adjusted R^2 will never decrease. False . It will decrease if the explanatory variables does not bring “enough” predictive power to the model (according to the adjusted R^2) and thus may be contributing to overfitting of the model.

11. More Cross-Validation Conceptual Questions Suppose that we are considering two linear regression models for a given dataset. • Model A o This model achieved a test R^2 = 0.75 using the train-test-split method. o This model achieved an average test R^2 = 0.65 using k=5 fold cross-validation. • Model B o This model achieved a test R^2 = 0.65 using the train-test-split method. o This model achieved an average test R^2 = 0.75 using k=5 fold cross-validation. If you were to select one of these models to predict the response variable values for new datasets , which model would you choose and why? o Model B. o In k=5 cross-validation we randomly created 5 sets of training and test datasets. We fit 5 models with our 5 training datasets and test the 5 models with each of our 5 test datasets. We then calculate the average test R^2 performance of the model. o Thus model B did better on average (considering multiple test R^2 values) than model A did. o By evaluating a model based on the average of multiple test R^2 values, we are able to decrease the amount of model variability that we might see in our test R^2 results (as opposed to the train-test-split method which evaluates just a single test R^2). Using the average test R^2 may lead to a higher degree of confidence with respect to how a given model might perform for new datasets, which most likely will not look EXACTLY like the test dataset. 12. Feature Selection Conceptual Questions 12.1. True or False: A backwards elimination algorithm is guaranteed to find the combination of explanatory variables that yield the linear regression model with the highest adjusted R^2. False 12.2. True or False: If a numerical explanatory variable has a strong linear relationship with the response variable, then it will not be dropped in a backwards elimination algorithm. False. For instance, it’s possible that this numerical explanatory variable 𝑥𝑥 1 has a strong linear relationship with another explanatory variable 𝑥𝑥 2 (ie. our model has an issue with multicollinearity). In a situation like this, 𝑥𝑥 1 may get dropped by a backwards elimination algorithm because 𝑥𝑥 2 may already be making the same type of contribution to the predictive power that 𝑥𝑥 1 may have. Thus, by including 𝑥𝑥 1 in addition to 𝑥𝑥 2 , 𝑥𝑥 1 may not bring enough additional predictive power to the model and may get dropped.