Curve fitting (linear and polynomial regression and more)

[RobinSchmidt]

i have just implemented polynomial regression, which is a generalization of linear regression. in linear regression, you find the straight line that best fits the data (in the least squares sense). in polynomial regression, you do the same thing with a polynomial with a degree > 1. see https://en.wikipedia.org/wiki/Polynomial_regression

here i have created some data from a 4th degree polynomial (black) plus noise (blue) and then fitted a 4th degree polynomial to that noisy data (green)

in the case without noise, a perfect agreement between data an polynomial model is expected when the degree of the model is >= the degree of the data generator polynomial (in the plot, we have the == situation). this served me as a test - and it indeed does work. the disagreement between model (green) and generator (black) in this plot is attributable to the noise. in fact the green curve fits the noisy blue data visually even better than the original black generator curve

[RobinSchmidt]

it's not yet ready for production - i just worked out the required math - but in the end, it shall be available in the library in a form something like this:

template<class T>
RAPT::rsPolynomial<T> fitPolynomial(int numDataPoints, T* x, T* y, int degree)

i'm still cleaning everything up and contemplating the API for this functionality. i just realized a few days ago that polynomial regression can actually be framed in terms of multiple linear regression:

https://en.wikipedia.org/wiki/Linear_regression#Simple_and_multiple_linear_regression

it's just a special case thereof - so i think, we should have an API for multiple regression and on top of that one for polynomial regression. starting from there, we could also start experimenting with other sets of basis functions instead of polynomials.....like....maybe sinusoids....because sinusoids - we are audio people!

[elanhickler]

woah nice i never saw this. did you do this when I said curve fitting might be better than peak finding for smoothing out a messy harmonic (amplitude-wise)?

[RobinSchmidt]

hmmm - if you refer to this:

#295 (comment)

then apparently, i did this before you mentioned curve fitting. i was actually prompted to do this by a youtube video from a german math channel which explained multiple linear regression really well. it made me realize that i almost had all the required ingredients already available in my library and just needed to plug them together in the right way. ...and polynomial regression is then really just a little bit of simple code on top of the multiple linear regression code. i wanted to have polynomial regression in the library since a long time but kinda thought, it would be more complicated to implement. i was pleased to realize, that it was not that hard after all

i'm not sure, if a polynomial fit is a good idea to approximate an amplitude envelope, though. do you think, it is? we can certainly try it

...btw. i hope, you are all well! i kinda was thrown off track quite a bit by this corona stuff. i didn't catch it (yet? :-O) - but i was totally blown away - watching news all day and couldn't really focus on doing any work. but i think, i'm getting back on track....

[RobinSchmidt]

OK - it's ready to use. i have put it into rsCurveFitter::fitPolynomial. example code how to use it is in polynomialRegression() in MathExperiments.cpp. this creates the plot above. the general usage is like:

rsPolynomial<double> p = rsCurveFitter::fitPolynomial(x, y, numDataPoints, polyDegree); 

which gives you a polynomial object. x and y are the data-arrays.

[elanhickler]

Could you hop over to #295 and let me know how to test the latest iteration of partial beating remover?