WIP: positive definite kernels, randomly pivoted Cholesky, and more (rebase of PR #72)

[rileyjmurray]

Notable changes so far:

Infrastructure:

• Updates RandBLAS
• Update RandLAPACK.hh to start with config files and RandBLAS.hh.

Abstract linear operators (see /misc/rl_linops.hh):

• Added operator()(int64_t i, int64_t j) to SymmetricLinearOperator.
• Created RegExplicitSymLinOp, a subclass of SymmetricLinearOperator that carries around regularization parameter(s).
• Added SpectralPrecond. This doesn't subclass anything. It implements the old evaluate function as well as operator()(... BLAS-like args ...) functions. The reason for the class is to seamlessly handle working with separable problems vs with block problems, and to transform the eigenvalues/singular values so that a matvec with the preconditioner takes two matvecs with the factor matrices, instead of three.

Positive definite kernels (see /misc/rl_pdkernels.hh):

• Various helper functions for evaluating the Gaussian kernel (or any kernel that primarily relies on the Euclidean distance matrix).
• Add a block_arrowhead_multiply function to avoid duplicate kernel evaluations when accessing a kernel matrix as a black-box linear operator.
• Add a subclass of SymmetricLinearOperator called "RBFKernelMatrix." This carries around regularization data much like RegExplicitSymLinOp, but everything else about it reflects working with an implicit matrix defined as the RBF kernel matrix of a low-dimensional dataset.

Randomly pivoted Cholesky (see /comps/rl_rpchol.hh):

• Utility functions
  • compute_columns evaluates kernel matrices in parallel where the kernel matrix is specified by a function handle callable as K_stateless(i, j).
  • pack_selected_rows
  • downdate_d_and_cdf
• rp_cholesky: implements Algorithm 4 from https://arxiv.org/abs/2304.12465.

Deterministic iterative solvers

• Rename pcg to pcg_saddle.
• Add lockorblock_pcg, which can take a lockstep approach to solving a separable problem or it can use an O'Leary subspace method. The implementation required utility functions (psd_sqrt_pinv and posm_square) and a StatefulSeminorm abstraction. A note: it might be tempting to abbreviate "lockorblock_pcg" by "lobpcg", but lobpcg already means something very specific.

Preconditioner generation

• make_right_orthogonalizer accepts an argument specifying the number of columns in V. This is needed when V comes from an eigendecomposition of a properly low-rank PSD matrix (which arises in low-memory preconditioning, a-la Nystrom or rpcholesky).
• Added rpchol_pc_data, which behaves similarly to nystrom_pc_data but obtains the low-rank approximation by RPCholesky.

[rileyjmurray]

@TeachRaccooon, one of the QB tests is failing. Can you take a look?

Note: Google Test filter = TestQB.Polynomial_Decay_general1
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from TestQB
[ RUN      ] TestQB.Polynomial_Decay_general1
Inner dimension of QB: 50                       
FRO NORM OF A - QB:    1.129707e-15
FRO NORM OF Q'Q - I:   2.560494e-15
FRO NORM OF A_k - QB:  5.062664e+00
/Users/rjmurr/Documents/randnla/RandLAPACK/test/comps/test_qb.cc:170: Failure
The difference between norm_test_4 and 0 is 5.0626636198116017, which exceeds test_tol, where
norm_test_4 evaluates to 5.0626636198116017,
0 evaluates to 0, and
test_tol evaluates to 1.6463612699567982e-10.

[  FAILED  ] TestQB.Polynomial_Decay_general1 (7 ms)
[----------] 1 test from TestQB (7 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (7 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] TestQB.Polynomial_Decay_general1

 1 FAILED TEST


0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.05 sec

The following tests FAILED:
        346 - TestQB.Polynomial_Decay_general1 (Failed)
Errors while running CTest

Seems this is compiler-dependent. For posterity, my compiler and cmake are

(rb311) (base) s1104997ca:RandLAPACK-build rjmurr$ clang -v
Homebrew clang version 17.0.6
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm/bin

(rb311) (base) s1104997ca:RandLAPACK-build rjmurr$ cmake --version
cmake version 3.27.7

CMake suite maintained and supported by Kitware (kitware.com/cmake).

I ended up being able to fix this by disabling compiler optimizations in Random123's boxmuller.hpp file. Don't ask me how I zero'd in on that.

[rileyjmurray]

This is superseded by PR #100.