Previous Next Table of Contents

6. Goodness of Fit and Uncertainty in Variables

This topic is discussed in detail in Chapter 5 of Matt Newville's Feffit document. For good literature references see Data Reduction and Error Analysis for the Physical Sciences by Philip R. Bevington and Numerical Recipes by Press, et al. Rather than rehashing what is said in those places, I will only point out differences between the error analysis in Feffit and Phit.

The most important difference is that the summations for chi-square and the R-factor are over both the real and imaginary parts in Feffit. Since Phit deals strictly with real valued data, these summations are only over real values. Thus, the R-factor is defined as the sum of the squares of the residuals divided by the sum of squares of the data. (See Eq. 5.7 in the Feffit document.)

In the Feffit document, the measurement uncertainty is referred to as epsilon. Here I have been calling it sigma. Ok. In Feffit the uncertainty is estimated from a part of the Fourier transformed signal that is spectrally distinct from the analyzed part of the data. Phit cannot use any such tricks. If the measurement uncertainty at each point is somehow already known, then it can be input into the program as part of the data file. This requires using the keyword sigma. Alternatively, a single value for sigma can be entered in the input file with the keyword sigdat. If no information about sigma is provided by the user, then Phit will use the standard deviation of the residuals as the value of sigma. The residuals are the differences at each point between the data and the sum of functions evaluated at the best fit values of the parameters.

Using the standard deviation of the residuals as the value of sigma assures that the value of chi-square for any fit will be nearly 1. This is true even in the case of a fit that is obviously poor by visual inspection. If this option for evaluating sigma is used, chi-square is not a valid criteria for ascertaining the quality of the fit. The R factor, however, is independent of the choice of sigma and should be a good criteria for judging a fit.

The merit of using a value of sigma that assures unit chi-square is that the error bars on the variables will be correctly normalized. If the fit is of acceptable quality, than the error bars reported in the log file will be those for a variable that is normally distributed about its best-fit value.

Since there is no signal processing involved in Phit, there is no way to evaluate the number of independent measurements in the data. The standard assumption in this situation is that each input data point is an independent measurement. Chi-square is normalized under this assumption. Phit assumes the number of independent points is the number of data points falling within the fitting ranges minus the number of guess values.

Whatever evaluation of sigma is used, chi-square is calculated from the residual at each point weighted by the appropriate value of sigma, as in Eq. 5.2 of the Feffit document. This value of chi-square is then used in calculating error bars and correlations between variables. The normalized chi-square reported in phit.log is this chi-square divided by the number of independent points.

The error bars on the guess values are calculated assuming the fitting function is normally distributed about the best-fit values. With this assumption, the error bars reported are one-sigma values for the variation in the guessed variables. Error bars for the evaluation of set expressions are also reported in phit.log. These are propagated numerically from the error bars in the guessed variables and include the correlation terms. See page ?? in Bevington's book for the details of this calculation. Error bars are reported neither for set expressions that are independent of guessed variables nor for set expressions that depend on the x-axis.

Please read Chapter 5 of the Feffit document carefully. It is a good discussion of the issues of non-linear minimization and provides information on interpreting the results of the error analysis performed by Phit.


Previous Next Table of Contents