I am trying a composite function for fitting muon data obtained at ISIS. I do not understand what “NumDeriv” is and what does it mean for it to be true or false. There is no documentation regarding that.
composite=CompositeFunction,NumDeriv=true;name=ExpDecayMuon,A=0.259619,Lambda=0.0851644;(composite=ProductFunction,NumDeriv=true;name=MuH,A0=1.20046,NuD=0.00110003,Lambda=-4.75099,Sigma=0.619706,Phi=1.76129;name=ExpDecayMuon,A=3.1181,Lambda=4.20696)
moreover, after the fitting curve, I see that the errorbars are ‘NaN’ when you export results to table; why? For example, if you have a fitting function in the form AB + C, and you want to initially fit the data with only C, and later on turn on AB part. How to do that?
Thanks
using Mantid version 6.11.0 (23 Oct 2024) on Windows 11
I’m looking at the code for ExpDecayMuon and I cannot see a NumDeriv value. To help us recreate what you are trying to do please can you provide us with the steps you followed to create the composite function?
NumDeriv specifies whether or not to use a numerical derivative of the function with respect to the parameters, e.g. finite difference.
As @jpcstfc says NumDeriv option tells Mantid to take the numerical derivative using the finite-difference method when calculating the first-derivative when estimating the jacobian (which can then be used to estimate the Hessian in the default LM minimiser).
Some fit functions in Mantid have an analytical form for the jacobian and so don’t require NumDeriv=True.
If you are using the python scripting you can do it on the CompositeFunction object like so
cf = CompositeFunction(Gaussian(), LinearBackground(), NumDeriv=True)
Or in a function string as you have above.
It is mentioned in some places in the docs, such as here
https://mantidproject.github.io/docs-nightly/fitting/fitfunctions/DiffRotDiscreteCircle.html#func-diffrotdiscretecircle
but I think it’s a fair point that it should be clearer/better sign-posted in the Fit documentation.
If you see that NumDeriv=True behaves/fits a lot better than NumDeriv=False this could indicate there is a bug in the analytical jacobian defined, so do let us know!
In terms of NaN errors, this is done when the errors are invalid and there could be a number of reasons: it can be because the hessian becomes singular (i.e. can’t be inverted to produce the covariance), this can indicate you need a different step-size in the finite-difference (try calling Fit with StepSizeMethod='Sqrt epsilon') . The diagonal elements in the covariance can also come out negative, this can happen if you’re not quite at the minimum in the cost function etc. Also using the Hessian to get the errors is not perfect, makes several assumptions that aren’t always true. You can always try the ProfileChiSquared1D algorithm
https://docs.mantidproject.org/nightly/algorithms/ProfileChiSquared1D-v1.html
Or look at bootstrap methods or Monte-carlo simulations (i.e. fit many simulated datasets generated using the observed data and errorbars - note it is often important to use Posison noise not Gaussian/Normal if the stats are low.)
In terms of initially setting parameters to 0 etc. this can be done using ties - here I tie the value of the flat background A0
composite=CompositeFunction,NumDeriv=true;name=BackToBackExponential,I=25149.2,A=0.0780062,B=0.0149591,X0=29995.3,S=31.1924;name=FlatBackground,A0=48.9564,ties=(A0=48.9564)
@AnthonyLim has also had a look at your function and noticed that the MuH already includes an exponential decay so there is no need for the ProductFunction and indeed this may be the cause of some of your problems (the fit is over-parameterised - there will be no unique solution I don’t think).
I agree. I am now using just MuH.
Thanks everyone for the clarity on NumDeriv.