PlanetPhysics/Example of Linear Least Squares

The assumption of linear least squares is that there is a linear relationship between our measurements $$z$$ and the variables to be estimated $$x$$

$$ z = Mx + b $$

For this example let us assume that our measurements are given in Table 1 and you can see them plotted in Figure 1.

{\mathbf Table 1:} Example Data

The linear least squares solution to fit the given data is given by the equation

$$ x_{fit} = (A^TA)^{-1}A^Tz $$

The only not so obvious step before using a tool like Matlab, is to form the $$A$$ matrix, which is a combination of an identity vector and $$x$$ as column vectors, such that

$$ A = [ \,\, 1 \,\, | \,\,x \,\,] $$

This is clarified by looking at the example code in Matlab, LinearLeastSquares.m. A plot of fitting the measurement data with a line such that it minimizes the the mean square of the error is shown in Figure 1.

The equation of the line to fit this data is then

$$z = 0.543 x + 0.947$$

\includegraphics[scale=.6]{LinearLeastSquares2.eps}

{\mathbf Figure 1:} Linear Fit of Example Data (Matlab)

\includegraphics[scale=.8]{least_squares.eps}

{\mathbf Figure 1:} Linear Fit of Example Data (rlplot)