EC2C3 MT Lecture 7

Effect of Adding Controls 2 Regression is automated matching. We estimate the effect of interest while holding fixed controls. (1) Is our estimate with all controls causal? (2) Can we quantify how estimates change when adding another control variable?

Setting Consider the data from the example in lecture 6 . 𝑌 ? = ? + ? ? ? + ?? ? + ? ? (long regression) Does the inclusion of the dummy for ? ? matter for the estimate of ?? 𝑌 ? = ? ? + ? ? ? ? + ? ? ? (short regression) We have two regression estimates, መ ? = 10,000 ෢ ? ? = 20,000 Where does the difference of 10,000 between these estimates come from? (“Long” and “short” are relative to each other, with “long” meaning at least one additional variable is added). 5

Omitted Variable Bias (OVB) The difference between the short and long regression coefficients, ? ? and ? , is called omitted variable bias (OVB): Omitted Variable Bias = Coef in Short − Coef in Long From the previous derivation, OVB = ? ? − ? = ? 𝜋 1 = { Coefficient on the omitted in the Long Regression } × Coefficient on the variable of interest in the Auxiliary 8 𝑌 ? = ? + ? ? ? + ? ? ? + ? ? (Long Regression) 𝑌 ? = ? ? + ? ? ? ? + ? ? ? Short Regression ? ? = 𝜋 0 + 𝜋 1 ? ? + ? ? (Auxiliary Regression)

Auxiliary Regression with Other ? 11 • Long : ln(????𝑖??? ? ) = ? 0 + ? 1 ??𝑖???? ? + ? 2 ?𝐴? 𝑖 100 + ? 3 ?????? ??𝐼? ? + ? 4 2???? ? + ? 5 3???? ? + ? 6 4???? ? + ? ? • Short : ln(????𝑖??? ? ) = ? 0 ? + ? 1 ? ??𝑖???? ? + ? 3 ? ?????? ??𝐼? ? + ? 4 ? 2???? ? + ? 5 ? 3???? ? + ? 6 ? 4???? ? + ? ? ? • The auxiliary has the omitted variable from the short as the outcome and the ? in the short as regressors, ??? ? 100 = 𝜋 0 + 𝜋 1 ??𝑖???? ? + 𝜋 2 ?????? ??𝐼? ? + 𝜋 3 2???? ? + 𝜋 4 3???? ? + 𝜋 5 4???? ? + ? ? • Plugging the auxiliary into the long regression and simplifying results in the short regression. ln(????𝑖???) = ? 0 + ? 1 ??𝑖???? + ? 2 (𝜋 0 + 𝜋 1 ??𝑖???? ? + ⋯ ) + ? 3 ?????? ??𝐼? ? + ? 4 2???? ? + ? 5 3???? ? + ? 6 4???? ? + ? ? ln ????𝑖??? = ? 0 + ? 2 𝜋 0 + ? 1 + ? 2 𝜋 1 ??𝑖???? ? + ⋯ + ? ? ? ? 1 ? = ? 1 + ? 2 𝜋 1 . The principle of the OVB formula is the same: ?ℎ??? = 𝐿??? + ?????𝑖?𝑖??? ?? ??𝑖???? 𝑖? 𝐿??? × ?????𝑖?𝑖??? ?? ???𝑖???? ?? 𝑖??????? 𝑖? ???𝑖?𝑖???

Without looking at results from the auxiliary regression, do students who attended private colleges, on average: A. Score higher on the SAT than their public college counterparts B. Score lower on the SAT than their public college counterparts C. Score similarly on the SAT as their public college counterparts D. We don’t know 14 Practice OVB = ?ℎ??? − 𝐿??? > 0 OVB = Coefficient on variable of interest in Auxiliary × Coefficient on omitted in Long > 0 Because Coefficient on omitted in Long > 0 , we know Coefficient on variable of interest in Auxiliary > 0 , thus, the omitted variable and variable of interest are positively correlated. That is, SAT score is positively correlated with attending a private college. Go to ttpoll.eu and use session id “EC2C3”

Relationship Between Selection Bias and OVB Selection bias : The difference between our estimate of the causal effect and the true causal effect. The bias that occurs when we compare the treated and control group without controlling for all confounders that affect treatment and also the outcome (selection into treatment is not fully controlled for). • E.g., private school applicants may be more motivated or born into richer families on average. These factors affect income. • Issue: We may not observe all factors that influence selection into treatment, so we can’t include them all in the regression as controls to remove their OVB. OVB : the mathematical relationship between coefficients for the same variable in any two regressions which differ only in that one regression contains at least one additional regressor. • Adding variables to remove OVB per se does not imply a long regression has a causal interpretation . • OVB is only a mathematical relationship. • The OVB formula holds for any paired long and short regressions. • Removing OVB will reduce selection bias if we control for good confounders. 17

OVB = Relationship between ?? ? and ? ? × {Coefficient on ?? ? in Long Regression} Which coefficient captures the relationship between ?? ? and ? ? ? A. ? B. ? 3 C. 𝜋 1 D. 𝜋 3 20 Practice Go to ttpoll.eu and use session id “EC2C3”

OVB Due to Omitting Family Size OVB = Relationship between ?? ? and ? ? × { Coefficient on ?? ? in Long Regression } OVB = ? ? − ? = 𝜋 1 × ? 3 21 ln 𝑌 ? = ? ? + ? ? ? ? + σ ? ? ? ? ????? ?? + ? 1 ? ??? ? + ? 2 ? ln ?𝐼 ? + ? ? ? If we observe family size and include it in the regression, ln 𝑌 ? = ? + ? ? ? + σ ? ? ? ????? ?? + ? 1 ??? ? + ? 2 ln ?𝐼 ? + ? 3 ?? ? + ? ? The auxiliary regression is, ?? ? = 𝜋 0 + 𝜋 1 ? ? + σ ? 𝜃 ? ????? ?? + 𝜋 2 ??? ? + 𝜋 3 ln ?𝐼 ? + ? ?

How is this Useful? • If we don’t observe ?? , we can’t run either the long or auxiliary regression. How does the OVB formula help? • The coefficient for ?? ? in the long regression, namely ? 3 , is likely negative. • Kids from larger families often earn less. (They receive less parental attention as children). • The relationship between ?? ? and ? ? , namely 𝜋 1 , is likely negative. • Kids from larger families are less likely to attend private universities. • Using the OVB formula: OVB = ? ? − ? = 𝜋 1 × ? 3 = negative × negative = positive • Even without knowing the estimates, we can predict the sign of the OVB. • This allows us to interpret our results as likely greater than the true effect, ? ? > ? . 22

“Signing” the Bias OVB = Coefficient on variable of interest in Auxiliary × {Coefficient on omitted in Long Regression} • OVB = ? ? − ? • ? ? = ? + ??? • If ??? > 0 , then ? ? > ? . ? ? is “biased up” or has “upward bias” and is an overestimate. • If ??? < 0 , then ? ? < ? . ? ? is “biased down” or has “downward bias” and is an underestimate. 23 ??? Coefficient on variable of interest in Auxiliary + − Coefficient on omitted in Long Regression + ??? > 0 ??? < 0 − ??? < 0 ??? > 0

Context of Harvard and U-Mass • Many omitted variables would likely be of the style of family size, affecting earnings and the probability of attending private schools in the same direction. • This means OVB would be positive, and results are biased up. • Including variables such as family size would lower the estimate of the private school treatment effect, which is basically zero, even further. • The story can go in the opposite way. Individuals going to U-Mass who also were admitted to Harvard may be super smart and got scholarships to attend U-Mass. • The OVB would be negative, and results would be biased down. 24

Application Essay by Hugh Gallagher, age 19 Hugh went to NYU Topic: Describe significant experiences or accomplishments that have helped define you as a person. 25 Essay: I am a dynamic figure. I ride the Tour de France and cook Thirty-Minute Brownies in twenty minutes. I am an expert in stucco, a veteran in love, and an outlaw in Peru. On Wednesdays after school, I repair appliances for the homeless. I am an abstract artist, a concrete analyst, and a ruthless bookie. I weave, I dodge, I frolic, and my bills are all paid. I have won spelling bees at the Kremlin, bullfights in San Juan, and cliff-diving competitions in Sri Lanka. I’ve played Hamlet, performed open-heart surgery, and spoken with Elvis. But I have not yet gone to college. You can see from his surreal achievements, it’s almost impossible to “keep factors fixed (equal)” if we have people like Hugh in our dataset.

Terminology Recap • Outcome: the variable on the left side of a regression. Also called the “dependent variable” . • Regressor: any variable on the right side of a regression, also called a “covariate”, “right -side variable”, or “independent variable” . • Regressor of interest: the variable whose causal effect on the outcome is what we want to study. Also called “treatment” . • Confounder: a variable that is causally related with both the treatment and the outcome. It is not necessarily a variable in our regression. • Control: a regressor (or matching variable) we use to hold confounders constant. For a causal interpretation of the coefficient for the regressor of interest, controls need to capture all confounders . 26

Omitted Variable Bias: Summary • The interpretation of regression estimates as causal effects hinges on our ability to control for all confounders. • We can only control for observables. • The OVB formula helps us reason about the effect of potential unobserved confounders. OVB = Coefficient on variable of interest in Auxiliary × Coefficient on omitted in Long • The formula is a mathematical relationship between coefficients that holds irrespective of the causal interpretation of the estimates. 27