see if we get speedup by not using full tau matrix

[willwerscheid]

For example, we might get faster updates by using just a scalar (rather than an n x p matrix) when var_type = "constant". Try with "constant" first to see what difference it makes.

[willwerscheid]

Results of quick implementation and test on GTEx "strong" data, var_type = "constant":
Greedy, 5 factors: before changes 44.8s, after changes 27.6s (38% speedup).
Backfit of same 5 factors: before 140s, after 85s (39%).

Results will probably not be as dramatic for var_type = "constant". Nonetheless, this speedup seems significant enough that implementing this change would be worth our while.

Calls used:
flash_add_greedy(strong, 5, var_type="constant", nullcheck = FALSE, verbose=FALSE)
flash_backfit(strong, fl, var_type="constant", nullcheck = FALSE, verbose=FALSE)

[willwerscheid]

From profiling the above backfit, it seems like we could get a further speedup of 25% by just skipping the subsetting of Rk in calc_ebnm_l_args when no factors/loadings are fixed (this takes a full 17s, apparently). Another small but easy perfomance gain can be obtained by skipping the subsetting of R2 in compute_precision when there is no missing data (3.5s). Finally, we compute the likelihood twice in every iteration of ebnm_pn (once in mle_normal_logscale_grad and then again in grad_negloglik_normal). We could shave off up to 5 more seconds by eliminating this redundancy. So, if the profiling results are correct, we could get the backfit down to 60s (from an original 140!).

[pcarbo]

@willwerscheid That's great! But keep in mind there is often a tradeoff between optimizing code and keeping code simple. If you have both, that is great. Also, optimizing memory is way more important, because memory is a fixed constraint, but time isn't (unless you have a conference deadline).

[willwerscheid]

@pcarbo Thanks, great point. The redundancies I've identified above should also help a bit with memory (a lot of unnecessary copies) when that becomes an issue. But there are probably other things we can do as well.

[willwerscheid]

When making these changes, it will be helpful to exploit the fact that ebnm_pn and ebnm_ash can accept a scalar argument for s.

[willwerscheid]

Tests of suggested changes here.