Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define functions for Cholesky #168

Merged
merged 12 commits into from
Oct 13, 2023
Merged

Define functions for Cholesky #168

merged 12 commits into from
Oct 13, 2023

Conversation

devmotion
Copy link
Member

@devmotion devmotion commented Jun 22, 2022

This PR defines

  • dim
  • whiten, whiten!
  • unwhiten, unwhiten!
  • quad, quad!
  • invquad, invquad!
  • X_A_Xt, Xt_A_X, X_invA_Xt, Xt_invA_X

for Cholesky.

Ultimately, I think we should forward (some of) the generic fallbacks for AbstractMatrix to these functions (I noticed they could also be cleaned a bit since e.g.

invquad(a::AbstractMatrix, x::AbstractVecOrMat) = x' / a * x
is never hit for x::AbstractMatrix due to the definition in
function invquad(a::AbstractMatrix{T}, x::AbstractMatrix{S}) where {T<:Real, S<:Real}
; additionally, it is problematic e.g. for StaticArrays that out-of-place methods fall back to in-place methods). However, I assumed it would be cleaner to not touch the dispatch pipeline in this PR but only add and test definitions for Cholesky.

This PR also provides a solution to #140 @st--: If one does not want to construct the full matrix, it is possible to work with Cholesky directly (at least when using the PDMats-specific API).

@codecov-commenter
Copy link

codecov-commenter commented Jun 22, 2022

Codecov Report

Attention: 8 lines in your changes are missing coverage. Please review.

Files Coverage Δ
src/chol.jl 80.48% <75.75%> (-19.52%) ⬇️

📢 Thoughts on this report? Let us know!.

@btmit
Copy link

btmit commented Jun 24, 2022

It might be worth extending addition as well. Below uses a full rank update to add two matrices directly using their Cholesky decompositions. Obviously slower than just adding the matrices if you have them, but if you're starting and ending with the Cholesky this seems the way to go. IIRC this is more numerically stable as well. Unfortunately, this doesn't work for StaticArrays #JuliaArrays/StaticArrays.jl#930

using LinearAlgebra, PDMats, BenchmarkTools

function fullrankupdate(c1::Cholesky, c2::Cholesky)
    cout = copy(c1)
    return fullrankupdate!(cout, c2)
end

function fullrankupdate!(c1::Cholesky, c2::Cholesky)
    for c in eachcol(PDMats.chol_lower(c2))
        lowrankupdate!(c1, c)
    end
    return c1
end

c1 = cholesky([1.0 0.5; 0.5 2.0])
c2 = cholesky([2.0 0.2; 0.2 1.0])

@btime PDMat($c1) + PDMat($c2);

@btime fullrankupdate($c1, $c2);

@assert PDMats.chol_lower(cholesky(PDMat(c1) + PDMat(c2)))  PDMats.chol_lower(fullrankupdate(c1, c2))
458.122 ns (12 allocations: 688 bytes)
70.585 ns (5 allocations: 240 bytes)
true

@devmotion
Copy link
Member Author

I don't think this should be done in this PR. In general, I'm not sure if it's a good idea at all to define addition for Cholesky at all but if it is desired it should probably be done in base.

@@ -52,6 +55,33 @@ using StaticArrays
@test Xt_invA_X(A, Y) isa SMatrix{10, 10, Float64}
@test Xt_invA_X(A, Y) ≈ Matrix(Y)' * (Matrix(A) \ Matrix(Y))
end

# TODO: Fix for `PDMat` and `PDiagMat`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there an issue tracking this TODO?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, no issue. But it will be fixed by #183 🙂

size_of_one_copy = sizeof(C)
@assert size_of_one_copy > 100 # ensure the matrix is large enough that few-byte allocations don't matter

@test chol_lower(C) ≈ chol_upper(C)'
@test (@allocated chol_lower(C)) < 1.05 * size_of_one_copy # allow 5% overhead
@test (@allocated chol_upper(C)) < 1.05 * size_of_one_copy

X = randn(d, 10)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this one?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

d should be 100 and used above as well. It's a remaining (final?) issue from the merge commit afaf699 that broke tests a bit.

@devmotion devmotion merged commit f28cdfb into master Oct 13, 2023
11 checks passed
@devmotion devmotion deleted the dw/cholesky branch October 13, 2023 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants