Use of "back calculation" introducing bias to Rt estimates

[seabbs]

It appears (though very happy to be corrected) that you have used back calculation by applying the discrete delay + incubation pmf to case counts iteratively to get infections.

LocalCovidTracker/prep/daily_tracker_scraper.R

Line 113 in fba6017

utlas.incidence.backcalculation <- lapply(1:length(utlas.alphabetical), function(x) {

Just to flag this approach has been shown to not properly reconstruct infections with a forwards process being needed. Assuming the same delay in both direction leads to over smoothing which in turns creates bias in the Rt estimates (towards 1). You can test this by applying your delay pmf to the reconstructed infections and plotting against the observed data.

See here and here for details.

It's fairly easy to implement a forwards looking deconvolution/back-calculation approach with some versions implemented here (with the generative version using the renewal equation giving better real time estimates and the data led gaussian process reconstruction being several orders of magnitude faster). There is also a fixed delay approach built into the surveillance package.

[MichelleKendall]

Thanks for bringing this to our attention, we're going to look into it asap.

[seabbs]

No problem.

This appears to also impact the analyses (both growth rate and Rt) in this paper: https://www.thelancet.com/journals/landig/article/PIIS2589-7500(20)30241-7

This plot (from some simulation work quite some time ago so I apologise for its lack of clarity) highlights the issue we had when initially using the same back calculation approach vs deconvolution on simulated data:

(with sampled I think being comparable to a pmf convolution approach)

[seabbs]

Just to note the second paper I shared has a v2 which has more clarity on a potential solution.