Least Squares Fitting:
Physics 218

In Chapter 8 of An Introduction to Error Analysis, by John R. Taylor, the author discusses the general theme of least-squares fitting.  This is based on the normal distribution discussion in Chapter 5.  Here, we will take the theoretical background as given, and show how the minimization of the least square difference leads to the standard results for some simple cases.

Computing the Mean

As a simple application, we will use the principle of least squares to calculate the mean of a set of numbers.

First, we will make up a data set of random numbers.  The following command creates a list of 5000 numbers selected from a normal distribution with a mean of 2.0 and a standard deviation of 0.1

rdata = RandomReal[NormalDistribution[2, 0.1], 5000];
npts = Length[rdata];
{Mean[rdata], StandardDeviation[rdata]}
Histogram[rdata, LabelStyle-> Larger]

Computing (χ^(2))

Next, we calculate (χ^2).   For N data points with a mean value of X, (and assuming all have the same uncertainty) the formula is

chisq1[data_, X_] := (1/(npts - 1))(∑_(i=1)^npts)(((data[[i]] - X))^2)

Of course, we don’t know X yet.  The purpose of this exercise is to find the value of X which minimizes (χ^2).  We claim that value of X is our best estimate of the mean.  Here is a plot showing (χ^2) for various guesses of X.

Plot[chisq1[rdata, X], {X, 1, 3}, AxesLabel->{"mean", "(χ^2)"}]

FindMinimum[chisq1[rdata, X], X]

This tells us the value of X  that minimizes (χ^2), and also what that minimum value is.

Sqrt[%[[1]]] (* This will be the standard deviation *)

Mean and Standard Deviation

The usual definition for the mean results from minimizing (χ^2), and that minimum value is the same as the usual definition for the standard deviation.

{Mean[rdata], StandardDeviation[rdata]}