Interpolation functions

[pgree]

Summary

I added a function that does interpolation of a set of function values, (x_i, y_i), specified by the user. The algorithm works by first doing a linear interpolation between points and then smoothing that linear interpolation by convolution with a Gaussian.

In order to enforce that the interpolated function values coincide with the user-inputted function values, the algorithm at each step convolves a Gaussian with a new piecewise linear function in which the function values are shifted by the difference between the most recent interpolation and the desired, original value.

The interpolated function can be evaluated analytically aside from an evaluation of the error function. The derivative of the interpolated function can be evaluated analytically.

This is a work-in-progress. I have discussed some of the design choices with @bbbales2 but that's it.

Tests

There are tests for the "accuracy" of the interpolation and tests for the derivative of the interpolation that compare autodiff to finite differences.

Side Effects

The variance of the Gaussian in the convolution is currently 1/10 the minimum distance between the x-values specified by the user. This has the potentially to lead to undesired behavior.

There is a tolerance hardcoded for when to stop the iterative algorithm in terms of the maximum distance between the interpolated values at the original points and the user-specified values at those points.

Checklist

• Math issue Interpolation function(s) #58

• Copyright holder: Columbia University

• the basic tests are passing

• the code is written in idiomatic C++ and changes are documented in the doxygen

• the new changes are tested

[SteveBronder]

Thanks for submitting!

Couple quick review comments. In general I think you need to take the stuff in prim and flesh it out to work for fvar and var types for any of the parameters since you only have a rev specialization for the 5th argument to conv_gaus_line

I also think the names for these should be fleshed out. conv_gaus_line could be something like gauss_line_convolution

[SteveBronder]

Generally we want prim functions to be templated generically to work with any type, then the specialization in rev will be for analytic calculations.

[pgree]

moved this function to the rev version of this file

[SteveBronder]

Need full docs for external facing functions

[SteveBronder]

Put using aliases together at the top of the function

[SteveBronder]

Suggested change

	double sig = sqrt(sig2);
	const double sig = sqrt(sig2);

[SteveBronder]

Suggested change

	double alpha = sqrt(2 * pi * sig2);
	const double alpha = sqrt(2 * pi * sig2);

[SteveBronder]

Suggested change

	if (x <= xs[0])
	return ys[0];
	if (x >= xs[n - 1])
	return ys[n - 1];
	if (x <= xs[0]) {
	return ys[0];
	}
	if (x >= xs[n - 1]) {
	return ys[n - 1];
	}

[SteveBronder]

These should all be const& parameters so copies are not made

[SteveBronder]

This is true for everywhere you are using std::vector, arithmetic types can be passed by value

[pgree]

I made these changes generally and also I created a new file for lin_interp. It seemed to make sense to separate the linear interpolation from this smooth interpolation.

[SteveBronder]

Reserve the vector size ahead of the loop to avoid reallocation

Suggested change

	vector<double> ys_new;
	vector<double> ys_new;
	ys_new.reserve(n_new);

[SteveBronder]

If this is only used in this file as an internal function then it should be in the internal namespace

[SteveBronder]

idt this will work for fvar<var>

[pgree]

I think this is no longer relevant, because now the xs (and ys) will only be inputted as data

[pgree]

@SteveBronder thanks for reviewing and for these comments. There is some discussion -- stan-dev/design-docs#18 -- about how and if this should be implemented so I will wait until there is some agreement before iterating further on this.

[syclik]

Just changed the title to reflect this is WIP.

[SteveBronder]

@pgree are there any blockers for this?

[pgree]

@SteveBronder sorry for the delay on this, I got distracted and had some Catalina-related problems. I will get to this asap

[SteveBronder]

Cool no worries just wanted to check in!

[stan-buildbot]

---

Name	Old Result	New Result	Ratio	Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan	3.02	3.02	1.0	-0.11% slower
low_dim_corr_gauss/low_dim_corr_gauss.stan	0.02	0.02	1.0	0.39% faster
eight_schools/eight_schools.stan	0.12	0.11	1.02	2.18% faster
gp_regr/gp_regr.stan	0.17	0.17	1.01	0.67% faster
irt_2pl/irt_2pl.stan	5.69	5.67	1.0	0.4% faster
performance.compilation	88.16	85.49	1.03	3.03% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan	8.47	8.49	1.0	-0.24% slower
pkpd/one_comp_mm_elim_abs.stan	30.86	29.95	1.03	2.94% faster
sir/sir.stan	136.36	135.2	1.01	0.85% faster
gp_regr/gen_gp_data.stan	0.04	0.04	0.99	-1.35% slower
low_dim_gauss_mix/low_dim_gauss_mix.stan	2.96	2.96	1.0	0.17% faster
pkpd/sim_one_comp_mm_elim_abs.stan	0.38	0.38	1.01	1.44% faster
arK/arK.stan	1.76	1.78	0.99	-1.09% slower
arma/arma.stan	0.59	0.6	1.0	-0.35% slower
garch/garch.stan	0.75	0.74	1.01	0.92% faster
Mean result: 1.00676635704

---

Jenkins Console Log
Blue Ocean
Commit hash: 57c2f8b

---

Machine information

ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

[rok-cesnovar]

Bump. What is the status of this PR? Waiting on a re-review or something else?

[pgree]

@rok-cesnovar I think all the requested changes were made so just waiting for a re-review.

[wds15]

Didn't we also have a design doc for this. Not sure what the status is there?

[pgree]

Didn't we also have a design doc for this. Not sure what the status is there?

Yes, exactly. This PR reflects what was discussed there, including your suggestions @wds15

[pgree]

@bgoodri - per our discussion today, here's the PR

[bgoodri]

Sorry, didn't realize this existed until today. It sounds like it is already been reviewed, in which case we should merge it once the conflict with fun.hpp is fixed. But what were the pros and cons of this interpolation scheme as compared to any of the ones in Boost that we could wrap?

[pgree]

But what were the pros and cons of this interpolation scheme as compared to any of the ones in Boost that we could wrap?

This interpolation schemes has a few of advantages

• the interpolated function is smooth
• the interpolated function can be evaluated with a formula
• the derivative of the interpolated function can be evaluated with a formula
• the interpolated function coincides with the inputted points

At the time we decided on this interpolation scheme, Boost didn't have anything with these features

[bob-carpenter]

It'd be great to add an interpolation function to Stan. Andrew and others have been asking for something like the interpolation function in BUGS for ages.

Is the approach unlike any of the Boost implementations? https://www.boost.org/doc/libs/1_77_0/libs/math/doc/html/interpolation.html

[bgoodri]

At the time we decided on this interpolation scheme, Boost didn't have anything with these features

Fair enough, although since we will probably include pchip at some point, we should think about having some sort of a common interface for them, at least at the Stan Language level. It would be great it we could soon have callable f = interpolation(x, y); in the transformed data block.

[pgree]

Is the approach unlike any of the Boost implementations? https://www.boost.org/doc/libs/1_77_0/libs/math/doc/html/interpolation.html

From above:

This interpolation scheme has a few of advantages

• the interpolated function is smooth
• the interpolated function can be evaluated with a formula
• the derivative of the interpolated function can be evaluated with a formula
• the interpolated function coincides with the inputted points

Also, you end up with non-oscillatory behavior between reference points. At the time we decided on this interpolation scheme, Boost didn't have anything with these features

The plots for the Akima spline look nice, but I'm not sure how they would do in the environments where people will use them. I don't know if one of the Boost interpolation schemes would work well enough for users' needs. Maybe they would.

we should think about having some sort of a common interface for them, at least at the Stan Language level. It would be great it we could soon have callable f = interpolation(x, y); in the transformed data block.

That's pretty much what we did. I discussed this with (I think) @SteveBronder a while back and we ended up settling on

lin_interp(xs, ys, x)
gaus_interp(xs, ys, x)

which are both in this PR

[bob-carpenter]

Thanks.

lin_interp(xs, ys, x)
gaus_interp(xs, ys, x)

1. We've used "normal" rather than "gauss" everywhere in our math lib. If you want to call out Gauss, it should at least be spelled out all the way to "gauss".

2. The BUGS function is interp.lin. It's documented on p. 13 of http://www.mrc-bsu.cam.ac.uk/wp-content/uploads/manual14.pdf. The advantage of interp_lin over lin_interp is that it's better for autocomplete and indexing.

I probably won't be reviewing this and we don't double review, but I strongly urge you not to use gaus_ whatever other decisions are made.

[pgree]

great, will change to interp_lin. Since interp_normal doesn't sound like a useful name, how about interp_smooth?

[bob-carpenter]

I like the more descriptive name. I'm OK with interp_gauss if you think interp_normal is too vague. I just really don't like just gaus.

[bgoodri]

Since all of the interpolation schemes we might implement in Stan in the future will presumably be smooth to some extent, I don't think interp_smooth would disambiguate. Maybe interp_convolve?

[bob-carpenter]

If there aren't likely to be other normal or Gaussian interpolations, I'd prefer

• interp_normal, or
• interp_gauss.

If there might be other Gaussian interpolations that don't involve convolution, then I'd prefer

• interp_normal_conv,
• interp_gauss_conv,
• interp_normal_convolve, or
• interp_gauss_convolve.

[syclik]

@pgree, sorry about the delay. I'm closing the PR for the moment. Please reopen if it's still active.

When that happens, I'll clear out the old review and give it a fresh one and turn that around quickly.

Right now, the tests fail, so we'll need to trigger the build again anyway.