Chapter 14.7, Problem 61E

Method of Least Squares Suppose that a scientist has reason to believe that two quantities $x$ and $y$ are related linearly, that is, $y=mx+b$ , at least approximately, for some values of $m$ and $b$ . The scientist performs an experiment and collects data in the form of points $\left(x_1,y_1\right),\left(x_2,y_2\right),\dots ,\left(x_n,y_n\right)$ , and then plots these points. The points don't lie exactly on a straight line, so the scientist wants to find constants $m$ and $b$ so that the line $y=mx+b$ "fits" the points as well as possible (see the figure).

Let $d_i=y_i-\left(m{x}_i+b\right)$ be the vertical deviation of the point $\left(x_i,y_i\right)$ from the line. The method of least squares determines $m$ and $b$ so as to minimize ${\displaystyle \underset{i=1}{\overset{n}{\sum }}{d}_i^2}$ , the sum of the squares of these deviations. Show that, according to this method, the line of best fit is obtained when $m{\displaystyle \underset{i=1}{\overset{n}{\sum }}{x}_i}+bn={\displaystyle \underset{i=1}{\overset{n}{\sum }}{y}_i}$ and $m{\displaystyle \underset{i=1}{\overset{n}{\sum }}{x}_i^2}+b{\displaystyle \underset{i=1}{\overset{n}{\sum }}{x}_i}={\displaystyle \underset{i=1}{\overset{n}{\sum }}{x}_i}{y}_i$

Thus the line is found by solving these two equations in the two unknowns $m$ and $b$ . (See Section 1.2 for a further discussion and applications of the method of least squares.)