Open
Description
It would be nice to have an epi archive backed by disk / many files + we talked about the current exploration framework based on the targets package + annoyances going through epix_slide
when also looping across things other than forecast_date (+ annoyances with epix_slide when the slide function outputs errors/etc.) I think this points to:
- epi_archive should be something more general that can have various backends. I sketched some backends we might imagine here.
- Maybe we want just a smarter/faster
epix_as_of
rather thanepix_slide
. (epix_slide
isn't smart at all yet, but it'd be easier than making epix_as_of smarter. Makingepix_as_of
smarter probably involves caching the last query & result or some sort of partition of the data, whileepix_slide
would get to order things as it likes, partition once, etc.) - We often want a way to let slide computations output errors/warnings instead/alongside "real" results, rather than have
epix_slide
completely give up or spit out all the warnings without associatedforecast_as_of
s at the end. I thinkpurrr
might have some useful helper functions here (purrr::safely
?). - Would we want a
derived_epi_archive
that does some of this targets/targets-like stuff, or based on some (hypothetical?) library that does parallelism with only one pool of workers & adjusts for BLAS/gurobi/etc. using parallelism to prevent CPU/swap thrashing / OOM killing (investigatecoro
?)? It might still have some of the awkwardness ofepix_slide
, but might make some things look nicer; e.g., metaforecaster's forecasts would be aderived_epi_archive
maybe atop amerge_epi_archive
atop (a) thederived_epi_archive
holding the base forecaster's forecasts and (b) the response data'sram_epi_archive
.
Alternatively, if we don't have derived or cached archived concepts, can we integrate our slide computations with a framework that does? E.g., targets
.
Metadata
Metadata
Assignees
Labels
No labels