Performance regression in invquad

[mohamed82008]

Hi,

It seems that the invquad function is significantly slower on the master branch compared to the latest release v0.9.7.

using PDMats, LinearAlgebra, ForwardDiff, BenchmarkTools

A = PDMat(Matrix{Float64}(I, 250, 250));
x = rand(250);

@btime invquad($A, $x);

# v0.9.7
## 42.301 μs (1 allocation: 2.13 KiB)

# master
## 80.592 μs (3 allocations: 490.52 KiB)

### Duals ###

xd = ForwardDiff.Dual{Nothing}.(x)
@btime invquad($A, $xd);

# v0.9.7
## 40.114 μs (4 allocations: 2.17 KiB)

# master
## 330.757 μs (7 allocations: 978.94 KiB)

This slowdown is because the implementation of invquad for vectors went from:

invquad(M::PDMat, x) = dot(x, M.chol \ x)

to

invquad(M::PDMat, x) = sum(abs2, M.chol.L \ x)

[mohamed82008]

M.chol.L seems to copy the array. M.chol.U' is faster and allocates less.

[mohamed82008]

Not for the duals though. It seems that conversion happens for the generic fallback.

[mohamed82008]

In https://github.com/JuliaLang/julia/blob/df9210ebfb6d04832a2cbf1f44357c959a0b3457/stdlib/LinearAlgebra/src/triangular.jl#L1881, copying happens. I suppose this is necessary for BLAS calls, but when the eltype is the same as TAB and for pure Julia implementations, this doesn't seem necessary IIUC.

[mohamed82008]

It seems that copying here can be completely avoided since dispatch happens on ldiv! anyways for BLAS types. But I may be wrong.