Chapter 14.7, Problem 59E

Suppose that a scientist has reason to believe that two quantities x and y are related linearly, that is, $y=mx+b$, at least approximately, for some values of m and b. The scientist performs an experiment and collects data in the form of points $\left(x_1,y_1\right),\left(x,y_2\right),\mathrm{...},\left(x_n,y_n\right),$ and then plots these points. The points don’t lie exactly on a straight line, so the scientist wants to find constants m and b so that the line $y=mx+b$ “fits” the points as well as possible (see the figure).

Let $d_i=y_i-\left(m{x}_i+b\right)$ be the vertical deviation of the point $\left(x_i,y_i\right)$ from the line. The method of least squares determines m and b so as to minimize ${\displaystyle \sum {}_{i=1}^n{d}_i^2}$, the sum of the squares of these deviations. Show that, according to this method, the line of best fit is obtained when

$m{\displaystyle \underset{i=1}{\overset{n}{\sum }}{x}_i+bn={\displaystyle \underset{i=1}{\overset{n}{\sum }}{y}_i}}$

and $m{\displaystyle \underset{i=1}{\overset{n}{\sum }}{x}_i^2+b{\displaystyle \underset{i=1}{\overset{n}{\sum }}{x}_i={\displaystyle \underset{i=1}{\overset{n}{\sum }}{x}_i{y}_i}}}$

Thus the line is found by solving these two equations in the two unknowns m and b. (See Section 1.2 for a further discussion and applications of the method of least squares.)