Default hyperparameter values

[jcjaskula-aws]

Hello,

This commit changed the default value of epsfcn. If I want to recover the old behavior, I understand that I would need to pass epsfcn for all the Models individually.
As the default value for scipy.optimize is None, I wonder if lmfit should not follow this default choice instead of setting it to an arbitrary value. On the other hand, if we stick to this solution, could we find a way to define these default value globally and enable users to change them with something like lmfit.set_default_values({"epsfcn": None})?

[newville]

@jcjaskula-aws Does the default value of 1.e-10 for epsfcn cause problems for you? I would be surprised to hear about significant differences resulting from changing that value. If you do see a problem, post a minimal example showing that.

The default value for epsfcn will typically be 1.e-15, though somewhere in minpack I remember it was set to 2.2e-16 = 2**(-52). The default step size for a variable when calculating the Jacobian will be sqrt(epsfcn). That means a variable's value will be multiplied by 1+sqrt(epsfcn) to construct the derivative unless the value is close to zero (I think that is abs(value) < sqrt(epsfcn)) in which case sqrt(epsfcn) will be added to the value.

The challenge is that sqrt(1.e-15) is still smaller than machine precision for float32. So if someone gives independent data of float32 we want to not promote those to float64, but also want derivatives from calculations with these to not be all zero.

I don't think there is any obvious best value for epsfcn. Like, the fact the step size is the square-root of that "near machine precision" is just sort of odd. I think the original authors wanted a default value around 1.e-8 for double precision and 1.e-4 for single precision and to avoid rounding errors.

It is certainly possible to explicitly set epsfcn value for any particular fit. I would be surprised if this was needed very often. Lmfit does not currently have any globally configured values. I think adding those would be a significant change.

[jcjaskula-aws]

Yes, changing default value 1.e-10 has an impact on data processing and I can retrieve good fits by simply using calling .fit(..., fit_kws={"epsfcn": None}). I would defer on providing a MVE as it is mainly depending on data and guesses. As for my other question, this is mainly causing a problem of consistency with previous versions and would require me to adapt heavily my code and warn my users.
I am mainly looking for a minimal change either in lmfit or in my code to avoid updating my data-processing code and all my model fitting.

[newville]

@jcjaskula-aws If you do not produce a script that demonstrates the behavior you report as a problem, how are we supposed to understand or address that report?

Changing the default value of epsfcn changed the results on our numerous test runs so that the previously found values we correct to 4 decimal places or better. Those indicate that there are no significant changes in the least-squares fit. Again, I would be surprised to hear of cases where the results were significantly different. If there was such a case, I would not necessarily assume that the older results are more accurate than the newer results.

But, you can certainly change the value of epsfcn if you want. If consistency is your goal, would not recommend a value of None, as that just pushes the selection of the value to somewhere else.