Chapter 13.8, Problem 50ES

A common problem in experimental work is to obtain a mathematical relationship $y=f\left(x\right)$ between two variables x and y by "fitting" a curve to points in the plane that correspond to experimentally determined values of x and y, say $\left(x_1,y_1\right),\left(x_2,y_2\right),\mathrm{...},\left(x_n,y_n\right)$ The curve $y=f\left(x\right)$ is called a mathematical model of the data. The general form of the function f is commonly determined by some underlying physical principle, but sometimes it is just determined by the pattern of the data. We are concerned with fitting a straight line $y=mx+b$ to data. Usually, the data will not lie on a line (possibly due to experimental error or variations in experimental conditions), so the problem is to find a line that fits the data "best" according to some criterion. One criterion for selecting the line of best fit is to choose m and b to minimize the function $g\left(m,b\right)={\displaystyle \underset{i=1}{\overset{n}{\sum }}{\left(m{x}_i+b-y_i\right)}^2}$ This is called the method of least squares, and the resulting line is called the regression line or the least squares line of best fit. Geometrically, $\left|m{x}_i+b-y_i\right|$ is the vertical distance between the data point $\left(x_i,y_i\right)$ and the line $y=mx+b.$

These vertical distances arc called the residuals of the data points, so the effect of minimizing $g\left(m,b\right)$ is to minimize the sum of the squares of the residuals. In these exercises, we will derive a formula for the regression line.

Assume that not all the $x_i\text{'}\text{s}$ are the same, so that $g\left(m,b\right)$ has a unique critical point at the values of m and b obtained in Exercise 49(c). The purpose of this exercise is to show that g has an absolute minimum value at this point.

(a) Find the partial derivatives $g_{mm}\left(m,b\right),g_{bb}\left(m,b\right),\text{and}{g}_{mb}\left(m,b\right),$ and then apply the second partials test to show that g has a relative minimum at the critical point obtained in Exercise 49.

(b) Show that the graph of the equation $z=g\left(m,b\right)$ is a quadric surface.

(c) It can be proved that the graph of $z=g\left(m,b\right)$ is an elliptic paraboloid. Accepting this to be so, show that this paraboloid opens in the positive z-direction, and explain how this shows that g has an absolute minimum at the critical point obtained in Exercise 49.