solutions02

Econ 410 Discussion Section Handout 2: Simple Regression Introduction to Regression and Ordinary Least Squares Suppose there is a true line that describes the relationship between y and x : y i = β 0 + β 1 x i + u i So we collect a simple random sample: ( x 1 , y 1 ) , ..., ( x n , y n ). We use this data to estimate the true regression line. The estimated line is written as: ˆ y i = ˆ β 0 + ˆ β 1 x i Exercise 1: What is β 0 and how does it differ from ˆ β 0 ? What is u i and how does it differ from ˆ u i ? What is y i and how does it differ from ˆ y i ? β 0 is the true intercept while ˆ β 0 is our estimate of the intercept. Similarly, β 1 is the true slope while ˆ β 1 is our estimate of the slope. u i is the vertical distance between data point i and the true line (known as the error), while ˆ u i is the vertical distance between data point i and the estimated line (known as the residual). Finally, y i is actual value of y for observation i , while ˆ y i is the predicted (or fitted) value of y for observation i based on the estimated regression line. Typically we estimate the regression line using the ordinary least squares (OLS) method, which is obtained by minimizing the sum of squared residuals: min n X i =1 ( y i − ˆ β 0 − ˆ β 1 x i ) 2 Which yields the following OLS estimators: ˆ β 1 = d Cov( x, y ) / d Var( x ) ˆ β 0 = ¯ y − ˆ β 1 ¯ x Exercise 2: When a regression line is estimated using OLS, what will the sum of the residuals be equal to? Explain your reasoning. The first order conditions (FOC) of the OLS minimization problem are: n X i =1 ( y i − ˆ β 0 − ˆ β 1 x i ) = 0 n X i =1 ( y i − ˆ β 0 − ˆ β 1 x i ) x i = 0 1

Recall that the residual is defined as the gap between y i and the regression line: ˆ u i = y i − ˆ y i = y i − ˆ β 0 − ˆ β 1 x i Plugging this equation into the first FOC, we obtain: n X i =1 ˆ u i = 0 Which implies that the sum of the OLS residuals is equal to zero. We’ll write the OLS slope estimator in a variety of ways. Here’s a figure to help you famil- iarize yourself with the many valid ways this equation can be written: P n i =1 x i ( y i - ¯ y ) P n i =1 x i ( x i - ¯ x ) P n i =1 ( x i - ¯ x )( y i - ¯ y ) P n i =1 ( x i - ¯ x ) 2 P n i =1 ( x i - ¯ x ) y i P n i =1 x i ( x i - ¯ x ) P n i =1 d i y i P n i =1 d 2 i P n i =1 d i y i SST x \ Cov( x, y ) \ Var( x ) β 1 + P n i =1 ( x i - ¯ x ) u i P n i =1 x i ( x i - ¯ x ) The multitude of ways we’ll be writing the simple OLS slope estimator Exercise 3: Consider the following regression output from Stata (the units of the wage variable is dollars per hour and the units of the education variable is years of education): . regress wage educ 2

Exercise 4: When we perform OLS on a simple regression, which of the following expressions are equal to zero and why? • ¯ ˆ u and ˆ ux These both must equal zero. This is because they are the first order conditions (FOC) from the minimization problem used to derive the OLS estimators. From calculus we know that FOC are necessary conditions for a minimum, so if the FOC aren’t satisfied, then we haven’t estimated our regression line using OLS. • E ( u ) and E ( xu ) These are equal to zero by assumption. This is because we’ve assumed E ( u | x ) = 0 (next week in section we’ll begin calling this assumption SLR.4), which implies that E ( u ) = 0 and E ( ux ) = 0: E [ u ] = E [ E ( u | x )] = E [0] = 0 E [ xu ] = E [ E ( xu | x )] = E [ xE ( u | x )] = E [ x 0] = 0 • E (¯ u ) This is also zero. Since the expected value of each error is zero by assumption, one would also expect the average error to be zero: E (¯ u ) = E ∑ n i =1 u i n = ∑ n i =1 E ( u ) n = ∑ n i =1 0 n = 0 • ¯ u Since we don’t observe the errors in our data, we aren’t able to calculate their sample mean. But, in principle, if we could somehow calculate the sample average of the errors, it’s unlikely to be exactly zero. While we expect each error (and thus their sample average) to be zero, this doesn’t imply that the average will be exactly zero in any given sample. (As an analogy, the expected roll for a six-sided die is 3.5, but in any given sample of 100 die rolls the sample average is unlikely to be exactly 3.5.) 4