Interpreting Regression Output ( Without all the Statistics Theory)

Interpreting Regression Output (Without all the Statistics Theory) Regression analysis is one of multiple data analysis techniques used in business and social sciences. The regression analysis technique is built on many statistical concepts, including sampling, probability, correlation, distributions, central limit theorem, confidence intervals, z-scores, t-scores, hypothesis testing, and more. However, you may not have studied these concepts. And if you did study these concepts, you may not remember all the statistical concepts underlying regression analysis. The ‘ Interpreting Regression Output Without all the Statistics Theory ’ book is for you to read and interpret regression analysis data without knowing all the underlying statistical concepts. Who is this book for? This book is primarily written for graduate or undergraduate business or humanities students interested in understanding and interpreting regression analysis output tables. This book is also helpful for executives and professionals interested in interpreting and using regression analysis. It is a wonderful resource for students or professionals looking for a quick refresher before exams or interviewing for jobs in the data analysis industry. This book is not intended to replace a statistics textbook or be a complete regression analysis guide. Instead, it is intended to be a quick and easy-to-follow summary of the regression analysis output. ‘Interpreting Regression Output Without all the Statistics Theory’ focuses only on basic insights the regression output gives you. This book does not assume that the reader is familiar with statistical concepts underlying regression analysis. For example, the reader is not expected to know the central limit theorem or hypothesis testing process. In addition, the reader is NOT expected to be an expert in Microsoft Excel, R, Python, or any other software that may perform a regression analysis. This book is not intended to replace a statistics text book or to be a complete guide to regression analysis. Interpreting Regression Output Without all the Statistics Theory is based on Senith Mathews’ experience tutoring students and executives in statistics and data analysis over 10 years. $0.99 Interpreting Regression... Shop now S IGN U P N OW abc@gmail.com Joe Brown 201 352 5654 Enter a note to your tuto Get Live Tutoring Get Live Tutoring R ECENT P OSTS Estimating the Sample Size Required for a Target Margin of Error (MoE) Accounts Receivables, Allowances fo Uncollectable Debt, Bad Debt Expenses When should I use the t-test vs. z-tes (t distribution vs. z distribution)? When to use Norm.Dist vs. Norm.In What cash flow value do use to calculate terminal value if you are using the perpetual growth method? GraduateTutor.com Make Learning Easier Call us: +1 – 732 510 0607 Email: care@graduatetutor.com Linkedin / Twitter HOME HOW IT WORKS PRICING CONTACT US ABOUT US CAREERS ETHICS

5 Chapters on Regression Basics The first chapter of this book shows you what the regression output looks like in different software tools. The second chapter of Interpreting Regression Output Without all the Statistics Theory helps you get a high-level overview of the regression model. You will understand how ‘good’ or reliable the model is. The second chapter helps you address the following questions: What does the the Multiple R tell me about the relationships between the X and Y variables? R-Squared or Multiple R-Squared tell me about the regression model? How is the Adjusted R-Squared different from the R-Squared? How is the standard Error useful? What does the Significance F tell me about the regression model? The third chapter of Interpreting Regression Output Without all the Statistics Theory discusses the regression equation and helps you find the ingredients of the regression equation. This chapter helps you address the following questions: What is the regression equation? Where can I get the ingredients of the regression equation? What does the regression equation tell me? What does the intercept indicate? What do the coefficients indicate? What do the signs of coefficients indicate? The fourth chapter of this book digs deeper into the regression equation. It helps you interpret the equation and understand its components. The fourth chapter of Interpreting Regression Output Without all the Statistics Theory helps you address the following questions: How do I interpret the Standard Error of the coefficients for each variable in a regression output? How is the t-statistic or the t-value computed and what does it indicate? How can I interpret the P-values in a regression model? What does the 95% Confidence Interval for each variable indicate? The fifth chapter addresses important points you must keep in mind when using regression analysis. It includes a brief discussion on some of the following aspects given regression analysis: Causation vs Correlation Man with a hammer syndrome Do outliers impact regression output? Underlying Assumptions of regression analysis Enjoy reading this book. Should you need more assistance with interpreting regression analysis output, please do not hesitate to call us or send us an email. One of our statistics tutors will be more than happy to assist you with interpreting your regression analysis output. Regression Data Used

Note that the regression analysis output provides essentially the same information in all these cases, although it is presented in different formats or designs. Every number in the regression output indicates something. We will address only the most frequently used numbers in this book. Chapter 2: The Big Picture / Understanding the Model The first set of numbers my eyes wander to are at the top of the regression output in Microsoft Excel under the heading Regression Statistics. This data is presented in the last few rows of the regression output in R. This set of data gives you the big picture about your regression output. It allows you to answer questions such as: How good is your model? What percentage of the variation is explained by the variables included? 2.1 The Multiple R The multiple R is the absolute value of the correlation coefficient of the two variables (X and Y) being evaluated. The correlation coefficient indicates how closely two variables move in tandem with each other. It assumes that the relationship is linear and so measures the linear relationship between the two variables X and Y. The correlation coefficient has a value between +1 and − 1. A correlation coefficient of +1 indicates that the variables move in perfect tandem and in the same direction. A correlation coefficient of 0 indicates that there is no relationship between the variables. A correlation coefficient of − 1 indicates that the variables move in perfect tandem but in the OPPOSITE direction. However, since the multiple R is the absolute value of the correlation coefficient, we do not get to know if the correlation is positive or negative! This means that we do not see the direction of the relationship and only know the strength of the relationship. The correlation coefficient is also referred to as the Pearson correlation coefficient or Pearson’s r. The Multiple R in our example indicates a strong correlation between the amount spent on TV ads and sales. As indicated above, the Multiple R will not tell us if the correlation is positive or negative. 2.2 R-Squared or Multiple R-Squared

The R-Squared (in Microsoft Excel) or Multiple R-Squared (in R) indicates how well the model or regression line “fits” the data. It indicates the proportion of variance in the dependent variable (Y) explained by the independent variable (X). We know a variable could be impacted by one or more factors. Therefore, the R-Squared indicates the percentage of variation in the dependent variable explained by the independent variables. In our example, we know that the unit sales of a product will be influenced by various factors such as price, competitors’ actions, economy, etc., and not just by the advertisement expenditure. When we run a regression with sales as the dependent Y variable and only advertisement expenditure as the independent X variable, the R-square indicates the percentage of variation in unit sales explained by the advertisement expenditure. It tells you the percentage of change in sales that is caused by varying the advertisement expenditure. This also means that we can compute the percentage of variation explained by factors other than advertisement expenditure such as the economy, competition, price, etc. The percentage of variation that is explained by factors other than advertisement expenditure will be 100%- R-square . Our regression output indicates that 81.48% of the variation in unit sales is explained by the advertisement budget. And 18.52% (100%-81.48%) of the variation is caused by factors other than advertisement expenditure. (Also, note that as the name suggests, the R-square is equal to the square of the multiple R!) 2.3 Adjusted R-Squared Adjusted R-Squared is used only when analyzing multiple regression output and ignored when analyzing simple linear regression output. When we have more than one independent variable in our analysis, the computation process inflates the R-squared. As the name indicates, the Adjusted R-Squared is the R-Square adjusted for this inflation when performing multiple regression. The interpretation of the Adjusted R-Squared is similar to the R-square and used only when analyzing multiple regression output. 2.4 The Standard Error The standard error in the regression output is a very important number to understand when interpreting regression data. The standard error is a measure of the precision of the model. It reflects the average error of the regression model. In other words, if we were using the regression model to predict or estimate the dependent variable or variable of interest, the standard error shows you how wrong you could be if you used the regression model to make predictions. As the standard error reflects how wrong you could be, we want the standard error to be as small as possible. The standard error is used to help you get a confidence interval for your predicted values. 2.5 Significance F The simplest way to understand the significance F is to think of it as the probability that our regression model is wrong and needs to be discarded!! The significance F gives you the probability that the model is wrong. We want the significance F or the probability of being wrong to be as small as possible. Significance F: Smaller is better….

regression output. X and Y are variables and will take on different values at different points in time. The values of a and b are substituted in the regression equation to get the relationship between X and Y as follows: Y = 437.88 + 16.95*X This can also be expressed in the context of the example or question making the relationship more meaningful. Sales = 437.88 + 16.95*Advertising budget 3.3 What does the regression equation tell me? What this regression model indicates is that sales are dependent on the advertising budget. If I spend $1 on advertising, I can expect to have sales of $454.83 (Sales = 437.88 + 16.95*$1) If I spend $2 on advertising, I can expect to have sales of $471.78 (Sales = 437.88 + 16.95*$2) If I spend $3 on advertising, I can expect to have sales of $488.73 (Sales = 437.88 + 16.95*$3) 3.4 What does the intercept indicate? The intercept of 437.88 indicates that sales will be 437.88 if we do not spend any money on advertising. This is because when advertising spend is zero, it (zero) is multiplied by the slope or b (here 16.95), resulting in a zero. This is added to your intercept, leaving you only the intercept value 437.88. If I spend $0 on advertising, I can expect to have sales of $437.88 (Sales = 437.88 + 16.95*$0) 3.5 What do the coefficients indicate? The coefficient b (here 16.95) indicates that for every unit increase in the X variable (here TV spend), the Y variable (here sales) will change by the amount of the coefficient 16.95. It is also referred to as the slope of the line in a simple linear equation. 3.6 What do the signs of coefficients indicate? If the coefficient of the independent variable X is positive, it indicates for every unit increase in the independent variable; the dependent variable will increase by the value of the coefficient. This also means that for every unit decrease in the independent variable, the dependent variable will decrease by the value of the coefficient. On the other hand, if the coefficient of the independent variable X is negative, for every unit increase in the independent RegressionStatistics Multiple R 0.9027 R Square 0.8148 Adjusted R Sq 0.8016 Standard Error 148.1904 O b s e r a t i o n s . 16 SUMMARY OUTPUT A N O V A Regression Residual Total Intercept STVads SS MS 1 1,352.635.29 1,352.635.29 14 307.445.65 21,960.40 15 1.680.080.94 C o e ff i c i e n t s S t a n d e r d E rr o r 437.8799 82.3477 16.9448 2.1591 † Stat 5.3175 7.8482 6159 Significance f 0.00000171 Povalue 0.0001 0.0000 Lower 95% 261.2617 12.3141 Upper 95% Lower 95.0% Upper 95.0% 614.4981 261.2617 614.4981 21.5756 12.3141 21.5756

variable, the dependent variable will decrease by the value of the coefficient. Correspondingly, for every unit decrease in the independent variable, the dependent variable will increase by the value of the coefficient. In our example, the sign of coefficient b is positive (here, it is +16.95). Therefore for every $1 increase in TV spend, sales can be expected to increase by $16.95 (the value of the coefficient). We have only one independent variable in this example. Since you have only one independent variable, it is called simple linear regression. When you have more than one independent variable, it will be called multiple regression. Therefore, you will see a coefficient for every independent variable in the multiple regression output. The interpretation of these coefficients will be the same. Chapter 4: More on the Regression Equation In the previous chapter, we understood the regression equation and how good or reliable the regression is. We also learned how to find the intercept and coefficients of the regression equation. In this chapter, we will look more deeply into the components of the regression equation. 4.1 Standard Error of the coefficients The standard error of the coefficients reflects the variability of the coefficient. It reflects the average error of the regression model. In other words, when we use the regression model to estimate the coefficient of an independent variable, the standard error shows you how wrong the estimated coefficient could be if you use it to make predictions. Again, because the standard error reflects how wrong you could be, we want the standard error to be small in relation to its coefficient. The standard error is used to help you get a confidence interval for your coefficient values. This is discussed further in section __. We notice that the standard error of our variable 2.16 is small relative to its coefficient of 16.95. 4.2 t-statistic or the t-value The t value or t statistic is not a number we recommend you focus on. It is computed by dividing the coefficient by its standard error and is hard to interpret on its own. If you think about a coefficient and its standard error, you will see that the larger the coefficient is compared to its standard error, the more reliable it will be. This will indicate that the larger the t value, the more reliable the coefficient. While the t value is not very helpful by itself, it is needed to compute a handy number – the P-value. The t value is used to look up the Student’s t distribution to determine the P-value. The Student’s t distribution shows you how the mean will behave given the size of your sample. The P-value is a really important and useful number and will be discussed next. 4.3 P-values The P-value indicates the probability that the estimated coefficient is wrong or unreliable. The best way to understand the P-value is as the “probability of an error.” We want the P-value to be as small as possible. How small should the P-value be? That depends on a cut-off level that we decide on separately. This cut-off level is called the significance level. The cutoff selected depends on the nature of the data studied and the different error types. The cutoff or significance level is usually 1%, 5%, or 10%. Generally, a cutoff point of 5% is used.

5.3 Sample Remember that regression analysis relies on sample data and reflects the relationship of the data in the sample. We assume that the sample reflects the true population, but this need not be so. 5.4 Outliers Regression analysis is sensitive to outliers. Therefore, the data being analyzed must be scrubbed for outliers. 5.5 Underlying Assumptions Regression analysis is built on four underlying assumptions. If these assumptions do not hold, the regression analysis will not be reliable. Linear relationship Multivariate normality No or little multicollinearity No auto-correlation Homoscedasticity Select Resources on Regression Here are a few resources that will help you learn more about interpreting regression analysis data. Regression Primer from PennState More in the F test from the Minitab blog Another example on interpreting regression output Regression hypothesis and the F value interpretation Note: When you look at the regression output in R, you will see a summary of the residuals. The interpretation of residuals becomes easy. Bu default, the average of the residuals is zero. The median should not be far from zero, and the minimum and maximum should be roughly equal in absolute value in an ideal scenario. C ONTACT US GraduateTutor.com Forest View Drive Avenel, NJ 07001 Call us:+1 – (732) 510-0607, E-mail: care@graduatetutor.com Privacy , FAQ Core Areas of Tutoring Accounting Tutoring Economics Tutoring Finance Tutoring Operations Research Tutoring Statistics for Managers Microsoft Excel Modeling Private Equity Interview Prep Copyright 2007-2023 Sign up Now abc@gmail.com Joe Brown 201 352 5654 Enter a note to your tutor Get Live Tutoring Get Live Tutoring