diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json
index 1a37f076f..e915cbd81 100644
--- a/dev/.documenter-siteinfo.json
+++ b/dev/.documenter-siteinfo.json
@@ -1 +1 @@
-{"documenter":{"julia_version":"1.10.5","generation_timestamp":"2024-09-30T07:20:57","documenter_version":"1.7.0"}}
\ No newline at end of file
+{"documenter":{"julia_version":"1.10.5","generation_timestamp":"2024-10-02T20:04:09","documenter_version":"1.7.0"}}
\ No newline at end of file
diff --git a/dev/api/index.html b/dev/api/index.html
index 7a4864504..678657bba 100644
--- a/dev/api/index.html
+++ b/dev/api/index.html
@@ -1,40 +1,40 @@
-API · Krylov.jl
where q₂, q₁ and q₀ are real. Care is taken to avoid numerical cancellation. Optionally, nitref steps of iterative refinement may be performed to improve accuracy. By default, nitref=1.
where q₂, q₁ and q₀ are real. Care is taken to avoid numerical cancellation. Optionally, nitref steps of iterative refinement may be performed to improve accuracy. By default, nitref=1.
Block-GMRES can be warm-started from an initial guess X0 where kwargs are the same keyword arguments as above.
Solve the linear system AX = B of size n with p right-hand sides using block-GMRES.
Input arguments
A: a linear operator that models a matrix of dimension n;
B: a matrix of size n × p.
Optional argument
X0: a matrix of size n × p that represents an initial guess of the solution X.
Keyword arguments
memory: if restart = true, the restarted version block-GMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
restart: restart the method after memory iterations;
reorthogonalization: reorthogonalize the new matrices of the block-Krylov basis against all previous matrix;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2 * div(n,p);
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms;
callback: function or functor called as callback(solver) that returns true if the block-Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
X: a dense matrix of size n × p;
stats: statistics collected on the run in a SimpleStats structure.
Block-GMRES can be warm-started from an initial guess X0 where kwargs are the same keyword arguments as above.
Solve the linear system AX = B of size n with p right-hand sides using block-GMRES.
Input arguments
A: a linear operator that models a matrix of dimension n;
B: a matrix of size n × p.
Optional argument
X0: a matrix of size n × p that represents an initial guess of the solution X.
Keyword arguments
memory: if restart = true, the restarted version block-GMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
restart: restart the method after memory iterations;
reorthogonalization: reorthogonalize the new matrices of the block-Krylov basis against all previous matrix;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2 * div(n,p);
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms;
callback: function or functor called as callback(solver) that returns true if the block-Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
X: a dense matrix of size n × p;
stats: statistics collected on the run in a SimpleStats structure.
If the vectors $b$ and $c$ in the non-Hermitian Lanczos process are replaced by matrices $B$ and $C$ with both $p$ columns, we can derive the block non-Hermitian Lanczos process.
After $k$ iterations of the block non-Hermitian Lanczos process, the situation may be summarized as
\[\begin{align*}
+\end{bmatrix}.\]
The function hermitian_lanczos returns $V_{k+1}$, $\Psi_1$ and $T_{k+1,k}$.
If the vectors $b$ and $c$ in the non-Hermitian Lanczos process are replaced by matrices $B$ and $C$ with both $p$ columns, we can derive the block non-Hermitian Lanczos process.
After $k$ iterations of the block non-Hermitian Lanczos process, the situation may be summarized as
If the vectors $b$ and $c$ in the Saunders-Simon-Yip process are replaced by matrices $B$ and $C$ with both $p$ columns, we can derive the block Saunders-Simon-Yip process.
After $k$ iterations of the block Saunders-Simon-Yip process, the situation may be summarized as
\[\begin{align*}
+\end{bmatrix}.\]
The function golub_kahan returns $V_{k+1}$, $U_{k+1}$, $\Psi_1$ and $L_{k+1}$.
If the vectors $b$ and $c$ in the Saunders-Simon-Yip process are replaced by matrices $B$ and $C$ with both $p$ columns, we can derive the block Saunders-Simon-Yip process.
After $k$ iterations of the block Saunders-Simon-Yip process, the situation may be summarized as
If the vectors $b$ and $c$ in the Montoison-Orban process are replaced by matrices $D$ and $C$ with both $p$ columns, we can derive the block Montoison-Orban process.
After $k$ iterations of the block Montoison-Orban process, the situation may be summarized as
\[\begin{align*}
+\end{bmatrix}.\]
The function saunders_simon_yip returns $V_{k+1}$, $\Psi_1$, $T_{k+1,k}$, $U_{k+1}$, $\Phi_1^H$ and $T_{k,k+1}^H$.
If the vectors $b$ and $c$ in the Montoison-Orban process are replaced by matrices $D$ and $C$ with both $p$ columns, we can derive the block Montoison-Orban process.
After $k$ iterations of the block Montoison-Orban process, the situation may be summarized as
Each Krylov method is able to call a callback function as callback(solver) at each iteration. The callback should return true if the main loop should terminate, and false otherwise. If the method terminated because of the callback, the output status will be "user-requested exit". For example, if the user defines minres_callback(solver::MinresSolver), it can be passed to the solver using
Each Krylov method is able to call a callback function as callback(solver) at each iteration. The callback should return true if the main loop should terminate, and false otherwise. If the method terminated because of the callback, the output status will be "user-requested exit". For example, if the user defines minres_callback(solver::MinresSolver), it can be passed to the solver using
If you need to write a callback that uses variables that are not in a KrylovSolver, use a closure:
function custom_stopping_condition(solver::KrylovSolver, A, b, r, tol)
mul!(r, A, solver.x)
r .-= b # r := b - Ax
bool = norm(r) ≤ tol # tolerance based on the 2-norm of the residual
@@ -50,4 +50,4 @@
return false # We don't want to add new stopping conditions
end
-(x, stats) = gmres(A, b, callback = gmres_callback)
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 30 September 2024. Using Julia version 1.10.5.
using Krylov
using LinearAlgebra, Printf
m = 5
@@ -21,12 +21,12 @@
@printf("Error in y: %7.1e\n", norm(λ * y - xy_exact[n+1:n+m]) / norm(xy_exact[n+1:n+m]))
end
All methods are factorization-free, which means that you only need to provide operator-vector products.
The A or B input arguments of Krylov.jl solvers can be any object that represents a linear operator. That object must implement mul!, for multiplication with a vector, size() and eltype(). For certain methods it must also implement adjoint().
Some methods only require A * v products, whereas other ones also require A' * u products. In the latter case, adjoint(A) must also be implemented.
A * v
A * v and A' * u
CG, CR, CAR
CGLS, CRLS, CGNE, CRMR
SYMMLQ, CG-LANCZOS, MINRES, MINRES-QLP, MINARES
LSLQ, LSQR, LSMR, LNLQ, CRAIG, CRAIGMR
DIOM, FOM, DQGMRES, GMRES, FGMRES, BLOCK-GMRES
BiLQ, QMR, BiLQR, USYMLQ, USYMQR, TriLQR
CGS, BICGSTAB
TriCG, TriMR
Info
GPMR is the only method that requires A * v and B * w products.
Preconditioners M, N, C, D, E or F can be also linear operators and must implement mul! or ldiv!.
In the field of nonlinear optimization, finding critical points of a continuous function frequently involves linear systems with a Hessian or Jacobian as coefficient. Materializing such operators as matrices is expensive in terms of operations and memory consumption and is unreasonable for high-dimensional problems. However, it is often possible to implement efficient Hessian-vector and Jacobian-vector products, for example with the help of automatic differentiation tools, and used within Krylov solvers. We now illustrate variants with explicit matrices and with matrix-free operators for two well-known optimization methods.
At each iteration of Newton's method applied to a $\mathcal{C}^2$ strictly convex function $f : \mathbb{R}^n \rightarrow \mathbb{R}$, a descent direction direction is determined by minimizing the quadratic Taylor model of $f$:
At each iteration of the Gauss-Newton method applied to a nonlinear least-squares objective $f(x) = \tfrac{1}{2}\| F(x)\|^2$ where $F : \mathbb{R}^n \rightarrow \mathbb{R}^m$ is $\mathcal{C}^1$, we solve the subproblem:
\[\min_{d \in \mathbb{R}^n}~~\tfrac{1}{2}~\|J(x_k) d + F(x_k)\|^2,\]
where $J(x)$ is the Jacobian of $F$ at $x$.
An appropriate iterative method to solve the above linear least-squares problems is LSMR. We could pass the explicit Jacobian to LSMR as follows:
Note that preconditioners can be also implemented as abstract operators. For instance, we could compute the Cholesky factorization of $M$ and $N$ and create linear operators that perform the forward and backsolves.
Krylov methods combined with factorization free operators allow to reduce computation time and memory requirements considerably by avoiding building and storing the system matrix. In the field of partial differential equations, the implementation of high-performance factorization free operators and assembly free preconditioning is a subject of active research.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 30 September 2024. Using Julia version 1.10.5.
diff --git a/dev/gpu/index.html b/dev/gpu/index.html
index 3bf483b4c..0ac11ac38 100644
--- a/dev/gpu/index.html
+++ b/dev/gpu/index.html
@@ -1,5 +1,5 @@
-GPU support · Krylov.jl
Krylov methods are well suited for GPU computations because they only require matrix-vector products ($u \leftarrow Av$, $u \leftarrow A^{H}w$) and vector operations ($\|v\|$, $u^H v$, $v \leftarrow \alpha u + \beta v$), which are highly parallelizable.
The implementations in Krylov.jl are generic so as to take advantage of the multiple dispatch and broadcast features of Julia. Those allow the implementations to be specialized automatically by the compiler for both CPU and GPU. Thus, Krylov.jl works with GPU backends that build on GPUArrays.jl, such as CUDA.jl, AMDGPU.jl, oneAPI.jl or Metal.jl.
All solvers in Krylov.jl can be used with CUDA.jl and allow computations on Nvidia GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (CuMatrix and CuVector).
Krylov methods are well suited for GPU computations because they only require matrix-vector products ($u \leftarrow Av$, $u \leftarrow A^{H}w$) and vector operations ($\|v\|$, $u^H v$, $v \leftarrow \alpha u + \beta v$), which are highly parallelizable.
The implementations in Krylov.jl are generic so as to take advantage of the multiple dispatch and broadcast features of Julia. Those allow the implementations to be specialized automatically by the compiler for both CPU and GPU. Thus, Krylov.jl works with GPU backends that build on GPUArrays.jl, such as CUDA.jl, AMDGPU.jl, oneAPI.jl or Metal.jl.
All solvers in Krylov.jl can be used with CUDA.jl and allow computations on Nvidia GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (CuMatrix and CuVector).
using CUDA, Krylov
if CUDA.functional()
# CPU Arrays
@@ -30,11 +30,11 @@
x_csc, stats_csc = lslq(A_csc_gpu, b_gpu)
x_csr, stats_csr = lsqr(A_csr_gpu, b_gpu)
x_coo, stats_coo = lsmr(A_coo_gpu, b_gpu)
-end
If you use a Krylov method that only requires A * v products (see here), the most efficient format is CuSparseMatrixCSR. Optimized operator-vector and operator-matrix products that exploit GPU features can be also used by means of linear operators.
For instance, when executing sparse matrix products on NVIDIA GPUs, the mul! function utilized by Krylov.jl internally calls three routines from CUSPARSE. The first one conducts an analysis of the sparse matrix's structure, the second one determines the size of the buffer that needs to be allocated, and the last one handles the product using the allocated buffer. To circumvent the need for repeated analysis computations and buffer allocation/deallocation with each product, we introduce a KrylovOperator in KrylovPreconditioners.jl. This operator performs the analysis and allocates the buffer only once.
using Krylov, KrylovPreconditioners
+end
If you use a Krylov method that only requires A * v products (see here), the most efficient format is CuSparseMatrixCSR. Optimized operator-vector and operator-matrix products that exploit GPU features can be also used by means of linear operators.
For instance, when executing sparse matrix products on NVIDIA GPUs, the mul! function utilized by Krylov.jl internally calls three routines from CUSPARSE. The first one conducts an analysis of the sparse matrix's structure, the second one determines the size of the buffer that needs to be allocated, and the last one handles the product using the allocated buffer. To circumvent the need for repeated analysis computations and buffer allocation/deallocation with each product, we introduce a KrylovOperator in KrylovPreconditioners.jl. This operator performs the analysis and allocates the buffer only once.
using Krylov, KrylovPreconditioners
# A_gpu can be a sparse COO, CSC or CSR matrix
opA_gpu = KrylovOperator(A_gpu)
-x_gpu, stats = gmres(opA_gpu, b_gpu)
Preconditioners, especially incomplete Cholesky or Incomplete LU factorizations that involve triangular solves, can be applied directly on GPU thanks to efficient operators that take advantage of CUSPARSE routines.
using SparseArrays, Krylov, LinearOperators
+x_gpu, stats = gmres(opA_gpu, b_gpu)
Preconditioners, especially incomplete Cholesky or incomplete LU factorizations that involve sparse triangular solves, can be applied directly on GPU thanks to efficient operators (like TriangularOperator) that take advantage of CUSPARSE routines.
using SparseArrays, Krylov, LinearOperators
using CUDA, CUDA.CUSPARSE
if CUDA.functional()
@@ -148,7 +148,7 @@
# A_gpu can be a sparse COO, CSC or CSR matrix
opA_gpu = KrylovOperator(A_gpu)
-x_gpu, stats = bicgstab(opA_gpu, b_gpu)
All solvers in Krylov.jl can be used with oneAPI.jl and allow computations on Intel GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (oneMatrix and oneVector).
using Krylov, oneAPI
+x_gpu, stats = bicgstab(opA_gpu, b_gpu)
Preconditioners, especially incomplete Cholesky or incomplete LU factorizations that involve sparse triangular solves, can be applied directly on GPU thanks to efficient operators (like TriangularOperator) that take advantage of rocSPARSE routines.
All solvers in Krylov.jl can be used with oneAPI.jl and allow computations on Intel GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (oneMatrix and oneVector).
using Krylov, oneAPI
if oneAPI.functional()
T = Float32 # oneAPI.jl also works with ComplexF32
@@ -180,4 +180,4 @@
b_gpu = MtlVector(b_cpu)
# Solve a dense least-norm problem on an Apple M1 GPU
-x, stats = craig(A_gpu, b_gpu)
Warning
Metal.jl is under heavy development and is considered experimental for now.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 30 September 2024. Using Julia version 1.10.5.
+x, stats = craig(A_gpu, b_gpu)
Warning
Metal.jl is under heavy development and is considered experimental for now.
Settings
This document was generated with Documenter.jl version 1.7.0 on Wednesday 2 October 2024. Using Julia version 1.10.5.
should be solved when A is column rank-deficient but b is in the range of A (consistent systems), regardless of the shape of A. This situation mainly occurs when
A is square and singular,
A is short and wide.
Overdetermined systems are less common but also occur.
4 - Adjoint systems
\[ Ax = b \quad \text{and} \quad A^H y = c\]
where A can have any shape.
5 - Saddle-point and Hermitian quasi-definite systems
\[ \begin{bmatrix} M & \phantom{-}A \\ A^H & -N \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \left(\begin{bmatrix} b \\ 0 \end{bmatrix},\begin{bmatrix} 0 \\ c \end{bmatrix},\begin{bmatrix} b \\ c \end{bmatrix}\right)\]
where A can have any shape.
6 - Generalized saddle-point and non-Hermitian partitioned systems
\[ \begin{bmatrix} M & A \\ B & N \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} b \\ c \end{bmatrix}\]
where A can have any shape and B has the shape of Aᴴ. A, B, b and c must be all nonzero.
Krylov solvers are particularly appropriate in situations where such problems must be solved but a factorization is not possible, either because:
A is not available explicitly,
A would be dense or would consume an excessive amount of memory if it were materialized,
factors would consume an excessive amount of memory.
Iterative methods are recommended in either of the following situations:
the problem is sufficiently large that a factorization is not feasible or would be slow,
an effective preconditioner is known in cases where the problem has unfavorable spectral structure,
the operator can be represented efficiently as a sparse matrix,
the operator is fast, i.e., can be applied with better complexity than if it were materialized as a matrix. Certain fast operators would materialize as dense matrices.
should be solved when A is column rank-deficient but b is in the range of A (consistent systems), regardless of the shape of A. This situation mainly occurs when
A is square and singular,
A is short and wide.
Overdetermined systems are less common but also occur.
4 - Adjoint systems
\[ Ax = b \quad \text{and} \quad A^H y = c\]
where A can have any shape.
5 - Saddle-point and Hermitian quasi-definite systems
\[ \begin{bmatrix} M & \phantom{-}A \\ A^H & -N \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \left(\begin{bmatrix} b \\ 0 \end{bmatrix},\begin{bmatrix} 0 \\ c \end{bmatrix},\begin{bmatrix} b \\ c \end{bmatrix}\right)\]
where A can have any shape.
6 - Generalized saddle-point and non-Hermitian partitioned systems
\[ \begin{bmatrix} M & A \\ B & N \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} b \\ c \end{bmatrix}\]
where A can have any shape and B has the shape of Aᴴ. A, B, b and c must be all nonzero.
Krylov solvers are particularly appropriate in situations where such problems must be solved but a factorization is not possible, either because:
A is not available explicitly,
A would be dense or would consume an excessive amount of memory if it were materialized,
factors would consume an excessive amount of memory.
Iterative methods are recommended in either of the following situations:
the problem is sufficiently large that a factorization is not feasible or would be slow,
an effective preconditioner is known in cases where the problem has unfavorable spectral structure,
the operator can be represented efficiently as a sparse matrix,
the operator is fast, i.e., can be applied with better complexity than if it were materialized as a matrix. Certain fast operators would materialize as dense matrices.
If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.
If you want to ask a question not suited for a bug report, feel free to start a discussion here. This forum is for general discussion about this repository and the JuliaSmoothOptimizers organization, so questions about any of our packages are welcome.
Settings
This document was generated with Documenter.jl version 1.7.0 on Monday 30 September 2024. Using Julia version 1.10.5.
If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.
If you want to ask a question not suited for a bug report, feel free to start a discussion here. This forum is for general discussion about this repository and the JuliaSmoothOptimizers organization, so questions about any of our packages are welcome.
Settings
This document was generated with Documenter.jl version 1.7.0 on Wednesday 2 October 2024. Using Julia version 1.10.5.
All solvers in Krylov.jl have an in-place variant implemented in a method whose name ends with !. A workspace (KrylovSolver) that contains the storage needed by a Krylov method can be used to solve multiple linear systems that have the same dimensions in the same floating-point precision. Each KrylovSolver has two constructors:
All solvers in Krylov.jl have an in-place variant implemented in a method whose name ends with !. A workspace (KrylovSolver), which contains the storage needed by a Krylov method, can be used to solve multiple linear systems with the same dimensions and the same floating-point precision. The section storage requirements specifies the memory needed for each Krylov method. Each KrylovSolver has two constructors:
XyzSolver(A, b)
XyzSolver(m, n, S)
Xyz is the name of the Krylov method with lowercase letters except its first one (Cg, Minres, Lsmr, Bicgstab, ...). Given an operator A and a right-hand side b, you can create a KrylovSolver based on the size of A and the type of b or explicitly give the dimensions (m, n) and the storage type S.
For example, use S = Vector{Float64} if you want to solve linear systems in double precision on the CPU and S = CuVector{Float32} if you want to solve linear systems in single precision on an Nvidia GPU.
Note
DiomSolver, FomSolver, DqgmresSolver, GmresSolver, BlockGmresSolver, FgmresSolver, GpmrSolver and CgLanczosShiftSolver require an additional argument (memory or nshifts).
The workspace is always the first argument of the in-place methods:
In-place methods return an updated solver workspace. Solutions and statistics can be recovered via solver.x, solver.y and solver.stats. Functions solution and statistics can be also used.
Return the solution(s) stored in the solver. Optionally you can specify which solution you want to recover, solution(solver, 1) returns x and solution(solver, 2) returns y.
We illustrate the use of in-place Krylov solvers with two well-known optimization methods. The details of the optimization methods are described in the section about Factorization-free operators.
In-place methods return an updated solver workspace. Solutions and statistics can be recovered via solver.x, solver.y and solver.stats. Functions solution and statistics can be also used.
Return the solution(s) stored in the solver. Optionally you can specify which solution you want to recover, solution(solver, 1) returns x and solution(solver, 2) returns y.
We illustrate the use of in-place Krylov solvers with two well-known optimization methods. The details of the optimization methods are described in the section about Factorization-free operators.
All methods are matrix-free, which means that you only need to provide operator-vector products.
The A or B input arguments of Krylov.jl solvers can be any object that represents a linear operator. That object must implement mul!, for multiplication with a vector, size() and eltype(). For certain methods it must also implement adjoint().
Some methods only require A * v products, whereas other ones also require A' * u products. In the latter case, adjoint(A) must also be implemented.
A * v
A * v and A' * u
CG, CR, CAR
CGLS, CRLS, CGNE, CRMR
SYMMLQ, CG-LANCZOS, MINRES, MINRES-QLP, MINARES
LSLQ, LSQR, LSMR, LNLQ, CRAIG, CRAIGMR
DIOM, FOM, DQGMRES, GMRES, FGMRES, BLOCK-GMRES
BiLQ, QMR, BiLQR, USYMLQ, USYMQR, TriLQR
CGS, BICGSTAB
TriCG, TriMR
Info
GPMR is the only method that requires A * v and B * w products.
Preconditioners M, N, C, D, E or F can be also linear operators and must implement mul! or ldiv!.
In the field of nonlinear optimization, finding critical points of a continuous function frequently involves linear systems with a Hessian or Jacobian as coefficient. Materializing such operators as matrices is expensive in terms of operations and memory consumption and is unreasonable for high-dimensional problems. However, it is often possible to implement efficient Hessian-vector and Jacobian-vector products, for example with the help of automatic differentiation tools, and used within Krylov solvers. We now illustrate variants with explicit matrices and with matrix-free operators for two well-known optimization methods.
At each iteration of Newton's method applied to a $\mathcal{C}^2$ strictly convex function $f : \mathbb{R}^n \rightarrow \mathbb{R}$, a descent direction direction is determined by minimizing the quadratic Taylor model of $f$:
At each iteration of the Gauss-Newton method applied to a nonlinear least-squares objective $f(x) = \tfrac{1}{2}\| F(x)\|^2$ where $F : \mathbb{R}^n \rightarrow \mathbb{R}^m$ is $\mathcal{C}^1$, we solve the subproblem:
\[\min_{d \in \mathbb{R}^n}~~\tfrac{1}{2}~\|J(x_k) d + F(x_k)\|^2,\]
where $J(x)$ is the Jacobian of $F$ at $x$.
An appropriate iterative method to solve the above linear least-squares problems is LSMR. We could pass the explicit Jacobian to LSMR as follows:
Note that preconditioners can be also implemented as abstract operators. For instance, we could compute the Cholesky factorization of $M$ and $N$ and create linear operators that perform the forward and backsolves.
Krylov methods combined with factorization free operators allow to reduce computation time and memory requirements considerably by avoiding building and storing the system matrix. In the field of partial differential equations, the implementation of high-performance factorization free operators and assembly free preconditioning is a subject of active research.
Settings
This document was generated with Documenter.jl version 1.7.0 on Wednesday 2 October 2024. Using Julia version 1.10.5.
The solvers in Krylov.jl support preconditioners, i.e., transformations that modify a linear system $Ax = b$ into an equivalent form that may yield faster convergence in finite-precision arithmetic. Preconditioning can be used to reduce the condition number of the problem or cluster its eigenvalues or singular values for instance.
The design of preconditioners is highly dependent on the origin of the problem and most preconditioners need to take application-dependent information and structure into account. Specialized preconditioners generally outperform generic preconditioners such as incomplete factorizations.
The construction of a preconditioner necessitates trade-offs because we need to apply it at least once per iteration within a Krylov method. Hence, a preconditioner must be constructed such that it is cheap to apply, while also capturing the characteristics of the original system in some sense.
where $P_{\ell}$ and $P_r$ are square and nonsingular.
The left preconditioning preserves the error $x_k - x^{\star}$ whereas the right preconditioning keeps invariant the residual $b - A x_k$. Two-sided preconditioning is the only variant that allows to preserve the hermicity of a linear system.
Note
Because det$(P^{-1}A - \lambda I)$ = det$(A - \lambda P)$ det$(P^{-1})$ = det$(AP^{-1} - \lambda I)$, the eigenvalues of $P^{-1}A$ and $AP^{-1}$ are identical. If $P = LL^{H}$, $L^{-1}AL^{-H}$ also has the same spectrum.
In Krylov.jl, we call $P_{\ell}^{-1}$ and $P_r^{-1}$ the preconditioners and we assume that we can apply them with the operation $y \leftarrow P^{-1} * x$. It is also common to call $P_{\ell}$ and $P_r$ the preconditioners if the equivalent operation $y \leftarrow P~\backslash~x$ is available. Krylov.jl supports both approaches thanks to the argument ldiv of the Krylov solvers.
When $A$ is Hermitian, we can only use centered preconditioning $L^{-1}AL^{-H}y = L^{-1}b$ with $x = L^{-H}y$. Centered preconditioning is a special case of two-sided preconditioning with $P_{\ell} = L = P_r^H$ that maintains hermicity. However, there is no need to specify $L$ and one may specify $P_c = LL^H$ or its inverse directly.
Preconditioners
$P_c^{-1}$
$P_c$
Arguments
M with ldiv=false
M with ldiv=true
Warning
The preconditioner M must be hermitian and positive definite.
RandomizedPreconditioners.jl uses randomized numerical linear algebra to construct approximate inverses of matrices.
BasicLU.jl uses a sparse LU factorization to compute a maximum volume basis that can be used as a preconditioner for least-norm and least-squares problems.
The solvers in Krylov.jl support preconditioners, i.e., transformations that modify a linear system $Ax = b$ into an equivalent form that may yield faster convergence in finite-precision arithmetic. Preconditioning can be used to reduce the condition number of the problem or cluster its eigenvalues or singular values for instance.
The design of preconditioners is highly dependent on the origin of the problem and most preconditioners need to take application-dependent information and structure into account. Specialized preconditioners generally outperform generic preconditioners such as incomplete factorizations.
The construction of a preconditioner necessitates trade-offs because we need to apply it at least once per iteration within a Krylov method. Hence, a preconditioner must be constructed such that it is cheap to apply, while also capturing the characteristics of the original system in some sense.
where $P_{\ell}$ and $P_r$ are square and nonsingular.
The left preconditioning preserves the error $x_k - x^{\star}$ whereas the right preconditioning keeps invariant the residual $b - A x_k$. Two-sided preconditioning is the only variant that allows to preserve the hermicity of a linear system.
Note
Because det$(P^{-1}A - \lambda I)$ = det$(A - \lambda P)$ det$(P^{-1})$ = det$(AP^{-1} - \lambda I)$, the eigenvalues of $P^{-1}A$ and $AP^{-1}$ are identical. If $P = LL^{H}$, $L^{-1}AL^{-H}$ also has the same spectrum.
In Krylov.jl, we call $P_{\ell}^{-1}$ and $P_r^{-1}$ the preconditioners and we assume that we can apply them with the operation $y \leftarrow P^{-1} * x$. It is also common to call $P_{\ell}$ and $P_r$ the preconditioners if the equivalent operation $y \leftarrow P~\backslash~x$ is available. Krylov.jl supports both approaches thanks to the argument ldiv of the Krylov solvers.
When $A$ is Hermitian, we can only use centered preconditioning $L^{-1}AL^{-H}y = L^{-1}b$ with $x = L^{-H}y$. Centered preconditioning is a special case of two-sided preconditioning with $P_{\ell} = L = P_r^H$ that maintains hermicity. However, there is no need to specify $L$ and one may specify $P_c = LL^H$ or its inverse directly.
Preconditioners
$P_c^{-1}$
$P_c$
Arguments
M with ldiv=false
M with ldiv=true
Warning
The preconditioner M must be hermitian and positive definite.
RandomizedPreconditioners.jl uses randomized numerical linear algebra to construct approximate inverses of matrices.
BasicLU.jl uses a sparse LU factorization to compute a maximum volume basis that can be used as a preconditioner for least-norm and least-squares problems.
","category":"page"},{"location":"factorization-free/#factorization-free","page":"Factorization-free operators","title":"Factorization-free operators","text":"","category":"section"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"All methods are factorization-free, which means that you only need to provide operator-vector products.","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"The A or B input arguments of Krylov.jl solvers can be any object that represents a linear operator. That object must implement mul!, for multiplication with a vector, size() and eltype(). For certain methods it must also implement adjoint().","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"Some methods only require A * v products, whereas other ones also require A' * u products. In the latter case, adjoint(A) must also be implemented.","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"A * v A * v and A' * u\nCG, CR, CAR CGLS, CRLS, CGNE, CRMR\nSYMMLQ, CG-LANCZOS, MINRES, MINRES-QLP, MINARES LSLQ, LSQR, LSMR, LNLQ, CRAIG, CRAIGMR\nDIOM, FOM, DQGMRES, GMRES, FGMRES, BLOCK-GMRES BiLQ, QMR, BiLQR, USYMLQ, USYMQR, TriLQR\nCGS, BICGSTAB TriCG, TriMR","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"info: Info\nGPMR is the only method that requires A * v and B * w products.","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"Preconditioners M, N, C, D, E or F can be also linear operators and must implement mul! or ldiv!.","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"We strongly recommend LinearOperators.jl to model matrix-free operators, but other packages such as LinearMaps.jl, DiffEqOperators.jl or your own operator can be used as well.","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"With LinearOperators.jl, operators are defined as","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"A = LinearOperator(type, nrows, ncols, symmetric, hermitian, prod, tprod, ctprod)","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"where","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"type is the operator element type;\nnrow and ncol are its dimensions;\nsymmetric and hermitian should be set to true or false;\nprod(y, v), tprod(y, w) and ctprod(u, w) are called when writing mul!(y, A, v), mul!(y, transpose(A), w), and mul!(y, A', u), respectively.","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"See the tutorial and the detailed documentation for more information on LinearOperators.jl.","category":"page"},{"location":"factorization-free/#Examples","page":"Factorization-free operators","title":"Examples","text":"","category":"section"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"In the field of nonlinear optimization, finding critical points of a continuous function frequently involves linear systems with a Hessian or Jacobian as coefficient. Materializing such operators as matrices is expensive in terms of operations and memory consumption and is unreasonable for high-dimensional problems. However, it is often possible to implement efficient Hessian-vector and Jacobian-vector products, for example with the help of automatic differentiation tools, and used within Krylov solvers. We now illustrate variants with explicit matrices and with matrix-free operators for two well-known optimization methods.","category":"page"},{"location":"factorization-free/#Example-1:-Newton's-Method-for-convex-optimization","page":"Factorization-free operators","title":"Example 1: Newton's Method for convex optimization","text":"","category":"section"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"At each iteration of Newton's method applied to a mathcalC^2 strictly convex function f mathbbR^n rightarrow mathbbR, a descent direction direction is determined by minimizing the quadratic Taylor model of f:","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"min_d in mathbbR^nf(x_k) + nabla f(x_k)^T d + tfrac12d^T nabla^2 f(x_k) d","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"which is equivalent to solving the symmetric and positive-definite system","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"nabla^2 f(x_k) d = -nabla f(x_k)","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"The system above can be solved with the conjugate gradient method as follows, using the explicit Hessian:","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"using ForwardDiff, Krylov\n\nxk = -ones(4)\n\nf(x) = (x[1] - 1)^2 + (x[2] - 2)^2 + (x[3] - 3)^2 + (x[4] - 4)^2\n\ng(x) = ForwardDiff.gradient(f, x)\n\nH(x) = ForwardDiff.hessian(f, x)\n\nd, stats = cg(H(xk), -g(xk))","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"The explicit Hessian can be replaced by a linear operator that only computes Hessian-vector products:","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"using ForwardDiff, LinearOperators, Krylov\n\nxk = -ones(4)\n\nf(x) = (x[1] - 1)^2 + (x[2] - 2)^2 + (x[3] - 3)^2 + (x[4] - 4)^2\n\ng(x) = ForwardDiff.gradient(f, x)\n\nH(y, v) = ForwardDiff.derivative!(y, t -> g(xk + t * v), 0)\nopH = LinearOperator(Float64, 4, 4, true, true, (y, v) -> H(y, v))\n\ncg(opH, -g(xk))","category":"page"},{"location":"factorization-free/#Example-2:-The-Gauss-Newton-Method-for-Nonlinear-Least-Squares","page":"Factorization-free operators","title":"Example 2: The Gauss-Newton Method for Nonlinear Least Squares","text":"","category":"section"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"At each iteration of the Gauss-Newton method applied to a nonlinear least-squares objective f(x) = tfrac12 F(x)^2 where F mathbbR^n rightarrow mathbbR^m is mathcalC^1, we solve the subproblem:","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"min_d in mathbbR^ntfrac12J(x_k) d + F(x_k)^2","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"where J(x) is the Jacobian of F at x.","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"An appropriate iterative method to solve the above linear least-squares problems is LSMR. We could pass the explicit Jacobian to LSMR as follows:","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"using ForwardDiff, Krylov\n\nxk = ones(2)\n\nF(x) = [x[1]^4 - 3; exp(x[2]) - 2; log(x[1]) - x[2]^2]\n\nJ(x) = ForwardDiff.jacobian(F, x)\n\nd, stats = lsmr(J(xk), -F(xk))","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"However, the explicit Jacobian can be replaced by a linear operator that only computes Jacobian-vector and transposed Jacobian-vector products:","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"using LinearAlgebra, ForwardDiff, LinearOperators, Krylov\n\nxk = ones(2)\n\nF(x) = [x[1]^4 - 3; exp(x[2]) - 2; log(x[1]) - x[2]^2]\n\nJ(y, v) = ForwardDiff.derivative!(y, t -> F(xk + t * v), 0)\nJᵀ(y, u) = ForwardDiff.gradient!(y, x -> dot(F(x), u), xk)\nopJ = LinearOperator(Float64, 3, 2, false, false, (y, v) -> J(y, v),\n (y, w) -> Jᵀ(y, w),\n (y, u) -> Jᵀ(y, u))\n\nlsmr(opJ, -F(xk))","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"Note that preconditioners can be also implemented as abstract operators. For instance, we could compute the Cholesky factorization of M and N and create linear operators that perform the forward and backsolves.","category":"page"},{"location":"factorization-free/","page":"Factorization-free operators","title":"Factorization-free operators","text":"Krylov methods combined with factorization free operators allow to reduce computation time and memory requirements considerably by avoiding building and storing the system matrix. In the field of partial differential equations, the implementation of high-performance factorization free operators and assembly free preconditioning is a subject of active research.","category":"page"},{"location":"inplace/#In-place-methods","page":"In-place methods","title":"In-place methods","text":"","category":"section"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"All solvers in Krylov.jl have an in-place variant implemented in a method whose name ends with !. A workspace (KrylovSolver) that contains the storage needed by a Krylov method can be used to solve multiple linear systems that have the same dimensions in the same floating-point precision. Each KrylovSolver has two constructors:","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"XyzSolver(A, b)\nXyzSolver(m, n, S)","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"Xyz is the name of the Krylov method with lowercase letters except its first one (Cg, Minres, Lsmr, Bicgstab, ...). Given an operator A and a right-hand side b, you can create a KrylovSolver based on the size of A and the type of b or explicitly give the dimensions (m, n) and the storage type S.","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"For example, use S = Vector{Float64} if you want to solve linear systems in double precision on the CPU and S = CuVector{Float32} if you want to solve linear systems in single precision on an Nvidia GPU.","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"note: Note\nDiomSolver, FomSolver, DqgmresSolver, GmresSolver, BlockGmresSolver, FgmresSolver, GpmrSolver and CgLanczosShiftSolver require an additional argument (memory or nshifts).","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"The workspace is always the first argument of the in-place methods:","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"minres_solver = MinresSolver(n, n, Vector{Float64})\nminres!(minres_solver, A1, b1)\n\ndqgmres_solver = DqgmresSolver(n, n, memory, Vector{BigFloat})\ndqgmres!(dqgmres_solver, A2, b2)\n\nlsqr_solver = LsqrSolver(m, n, CuVector{Float32})\nlsqr!(lsqr_solver, A3, b3)","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"A generic function solve! is also available and dispatches to the appropriate Krylov method.","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"Krylov.solve!","category":"page"},{"location":"inplace/#Krylov.solve!","page":"In-place methods","title":"Krylov.solve!","text":"solve!(solver, args...; kwargs...)\n\nUse the in-place Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"In-place methods return an updated solver workspace. Solutions and statistics can be recovered via solver.x, solver.y and solver.stats. Functions solution and statistics can be also used.","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"Krylov.nsolution\nKrylov.solution\nKrylov.statistics\nKrylov.issolved","category":"page"},{"location":"inplace/#Krylov.nsolution","page":"In-place methods","title":"Krylov.nsolution","text":"nsolution(solver)\n\nReturn the number of outputs of solution(solver).\n\n\n\n\n\n","category":"function"},{"location":"inplace/#Krylov.solution","page":"In-place methods","title":"Krylov.solution","text":"solution(solver)\n\nReturn the solution(s) stored in the solver. Optionally you can specify which solution you want to recover, solution(solver, 1) returns x and solution(solver, 2) returns y.\n\n\n\n\n\n","category":"function"},{"location":"inplace/#Krylov.statistics","page":"In-place methods","title":"Krylov.statistics","text":"statistics(solver)\n\nReturn the statistics stored in the solver.\n\n\n\n\n\n","category":"function"},{"location":"inplace/#Krylov.issolved","page":"In-place methods","title":"Krylov.issolved","text":"issolved(solver)\n\nReturn a boolean that determines whether the Krylov method associated to solver succeeded.\n\n\n\n\n\n","category":"function"},{"location":"inplace/#Examples","page":"In-place methods","title":"Examples","text":"","category":"section"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"We illustrate the use of in-place Krylov solvers with two well-known optimization methods. The details of the optimization methods are described in the section about Factorization-free operators.","category":"page"},{"location":"inplace/#Example-1:-Newton's-method-for-convex-optimization-without-linesearch","page":"In-place methods","title":"Example 1: Newton's method for convex optimization without linesearch","text":"","category":"section"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"using Krylov\n\nfunction newton(∇f, ∇²f, x₀; itmax = 200, tol = 1e-8)\n\n n = length(x₀)\n x = copy(x₀)\n gx = ∇f(x)\n \n iter = 0\n S = typeof(x)\n solver = CgSolver(n, n, S)\n Δx = solver.x\n\n solved = false\n tired = false\n\n while !(solved || tired)\n \n Hx = ∇²f(x) # Compute ∇²f(xₖ)\n cg!(solver, Hx, -gx) # Solve ∇²f(xₖ)Δx = -∇f(xₖ)\n x = x + Δx # Update xₖ₊₁ = xₖ + Δx\n gx = ∇f(x) # ∇f(xₖ₊₁)\n \n iter += 1\n solved = norm(gx) ≤ tol\n tired = iter ≥ itmax\n end\n return x\nend","category":"page"},{"location":"inplace/#Example-2:-The-Gauss-Newton-method-for-nonlinear-least-squares-without-linesearch","page":"In-place methods","title":"Example 2: The Gauss-Newton method for nonlinear least squares without linesearch","text":"","category":"section"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"using Krylov\n\nfunction gauss_newton(F, JF, x₀; itmax = 200, tol = 1e-8)\n\n n = length(x₀)\n x = copy(x₀)\n Fx = F(x)\n m = length(Fx)\n \n iter = 0\n S = typeof(x)\n solver = LsmrSolver(m, n, S)\n Δx = solver.x\n\n solved = false\n tired = false\n\n while !(solved || tired)\n \n Jx = JF(x) # Compute J(xₖ)\n lsmr!(solver, Jx, -Fx) # Minimize ‖J(xₖ)Δx + F(xₖ)‖\n x = x + Δx # Update xₖ₊₁ = xₖ + Δx\n Fx_old = Fx # F(xₖ)\n Fx = F(x) # F(xₖ₊₁)\n \n iter += 1\n solved = norm(Fx - Fx_old) / norm(Fx) ≤ tol\n tired = iter ≥ itmax\n end\n return x\nend","category":"page"},{"location":"warm-start/#warm-start","page":"Warm-start","title":"Warm-start","text":"","category":"section"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"Most Krylov methods in this module accept a starting point as argument. The starting point is used as initial approximation to a solution.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"solver = CgSolver(A, b)\ncg!(solver, A, b, itmax=100)\nif !issolved(solver)\n cg!(solver, A, b, solver.x, itmax=100) # cg! uses the approximate solution `solver.x` as starting point\nend","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"If the user has an initial guess x0, it can be provided directly.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"cg(A, b, x0)","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"It is also possible to use the warm_start! function to feed the starting point into the solver.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"warm_start!(solver, x0)\ncg!(solver, A, b)\n# the previous two lines are equivalent to cg!(solver, A, b, x0)","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"If a Krylov method doesn't have the option to warm start, it can still be done explicitly. We provide an example with cg_lanczos!.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"solver = CgLanczosSolver(A, b)\ncg_lanczos!(solver, A, b)\nx₀ = solver.x # Ax₀ ≈ b\nr = b - A * x₀ # r = b - Ax₀\ncg_lanczos!(solver, A, r)\nΔx = solver.x # AΔx = r\nx = x₀ + Δx # Ax = b","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"Explicit restarts cannot be avoided in certain block methods, such as TriMR, due to the preconditioners.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"# [E A] [x] = [b]\n# [Aᴴ F] [y] [c]\nM = inv(E)\nN = inv(F)\nx₀, y₀, stats = trimr(A, b, c, M=M, N=N)\n\n# E and F are not available inside TriMR\nb₀ = b - Ex₀ - Ay\nc₀ = c - Aᴴx₀ - Fy\n\nΔx, Δy, stats = trimr(A, b₀, c₀, M=M, N=N)\nx = x₀ + Δx\ny = y₀ + Δy","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"# ## Restarted methods\n#\n# The storage requirements of Krylov methods based on the Arnoldi process, such as FOM and GMRES, increase as the iteration progresses.\n# For very large problems, the storage costs become prohibitive after only few iterations and restarted variants FOM(k) and GMRES(k) are preferred.\n# In this section, we show how to use warm starts to implement GMRES(k) and FOM(k).\n#\n# ```julia\n# k = 50\n# solver = GmresSolver(A, b, k) # FomSolver(A, b, k)\n# solver.x .= 0 # solver.x .= x₀ \n# nrestart = 0\n# while !issolved(solver) || nrestart ≤ 10\n# solve!(solver, A, b, solver.x, itmax=k)\n# nrestart += 1\n# end\n# ```","category":"page"},{"location":"block_krylov/#Block-GMRES","page":"Block-Krylov methods","title":"Block-GMRES","text":"","category":"section"},{"location":"block_krylov/","page":"Block-Krylov methods","title":"Block-Krylov methods","text":"note: Note\nblock_gmres works on GPUs with Julia 1.11.","category":"page"},{"location":"block_krylov/","page":"Block-Krylov methods","title":"Block-Krylov methods","text":"If you want to use block_gmres on previous Julia versions, you can overload the function Krylov.copy_triangle with the following code:","category":"page"},{"location":"block_krylov/","page":"Block-Krylov methods","title":"Block-Krylov methods","text":"using KernelAbstractions, Krylov\n\n@kernel function copy_triangle_kernel!(dest, src)\n i, j = @index(Global, NTuple)\n if j >= i\n @inbounds dest[i, j] = src[i, j]\n end\nend\n\nfunction Krylov.copy_triangle(Q::AbstractMatrix{FC}, R::AbstractMatrix{FC}, k::Int) where FC <: Krylov.FloatOrComplex\n backend = get_backend(Q)\n ndrange = (k, k)\n copy_triangle_kernel!(backend)(R, Q; ndrange=ndrange)\n KernelAbstractions.synchronize(backend)\nend","category":"page"},{"location":"block_krylov/","page":"Block-Krylov methods","title":"Block-Krylov methods","text":"block_gmres\nblock_gmres!","category":"page"},{"location":"block_krylov/#Krylov.block_gmres","page":"Block-Krylov methods","title":"Krylov.block_gmres","text":"(X, stats) = block_gmres(A, b::AbstractMatrix{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(X, stats) = block_gmres(A, B, X0::AbstractMatrix; kwargs...)\n\nBlock-GMRES can be warm-started from an initial guess X0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system AX = B of size n with p right-hand sides using block-GMRES.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nB: a matrix of size n × p.\n\nOptional argument\n\nX0: a matrix of size n × p that represents an initial guess of the solution X.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version block-GMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new matrices of the block-Krylov basis against all previous matrix;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2 * div(n,p);\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms;\ncallback: function or functor called as callback(solver) that returns true if the block-Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nX: a dense matrix of size n × p;\nstats: statistics collected on the run in a SimpleStats structure.\n\n\n\n\n\n","category":"function"},{"location":"block_krylov/#Krylov.block_gmres!","page":"Block-Krylov methods","title":"Krylov.block_gmres!","text":"solver = block_gmres!(solver::BlockGmresSolver, B; kwargs...)\nsolver = block_gmres!(solver::BlockGmresSolver, B, X0; kwargs...)\n\nwhere kwargs are keyword arguments of block_gmres.\n\nSee BlockGmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"block_processes/#block-krylov-processes","page":"Block Krylov processes","title":"Block Krylov processes","text":"","category":"section"},{"location":"block_processes/#block-hermitian-lanczos","page":"Block Krylov processes","title":"Block Hermitian Lanczos","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vector b in the Hermitian Lanczos process is replaced by a matrix B with p columns, we can derive the block Hermitian Lanczos process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_hermitian_lanczos)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Hermitian Lanczos process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A V_k = V_k+1 T_k+1k \n V_k^H V_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where V_k is an orthonormal basis of the block Krylov subspace mathcalK_k^square(AB),","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"T_k+1k =\nbeginbmatrix\n Omega_1 Psi_2^H \n Psi_2 Omega_2 ddots \n ddots ddots Psi_k^H \n Psi_k Omega_k \n Psi_k+1\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function hermitian_lanczos returns V_k+1, Psi_1 and T_k+1k.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"hermitian_lanczos(::Any, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.hermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.hermitian_lanczos","text":"V, Ψ, T = hermitian_lanczos(A, B, k; algo=\"householder\")\n\nInput arguments\n\nA: a linear operator that models an Hermitian matrix of dimension n;\nB: a matrix of size n × p;\nk: the number of iterations of the block Hermitian Lanczos process.\n\nKeyword arguments\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\n\nOutput arguments\n\nV: a dense n × p(k+1) matrix;\nΨ: a dense p × p upper triangular matrix such that V₁Ψ = B;\nT: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-nonhermitian-lanczos","page":"Block Krylov processes","title":"Block Non-Hermitian Lanczos","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vectors b and c in the non-Hermitian Lanczos process are replaced by matrices B and C with both p columns, we can derive the block non-Hermitian Lanczos process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_nonhermitian_lanczos)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block non-Hermitian Lanczos process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A V_k = V_k+1 T_k+1k \n A^H U_k = U_k+1 T_kk+1^H \n V_k^H U_k = U_k^H V_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where V_k and U_k are bases of the block Krylov subspaces mathcalK^square_k(AB) and mathcalK^square_k (A^HC), respectively,","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"T_k+1k =\nbeginbmatrix\n Omega_1 Phi_2 \n Psi_2 Omega_2 ddots \n ddots ddots Phi_k \n Psi_k Omega_k \n Psi_k+1\nendbmatrix\n qquad\nT_kk+1^H =\nbeginbmatrix\n Omega_1^H Psi_2^H \n Phi_2^H Omega_2^H ddots \n ddots ddots Psi_k^H \n Phi_k^H Omega_k^H \n Phi_k+1^H\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function nonhermitian_lanczos returns V_k+1, Psi_1, T_k+1k, U_k+1 Phi_1^H and T_kk+1^H.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"nonhermitian_lanczos(::Any, ::AbstractMatrix{FC}, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.nonhermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.nonhermitian_lanczos","text":"V, Ψ, T, U, Φᴴ, Tᴴ = nonhermitian_lanczos(A, B, C, k)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nB: a matrix of size n × p;\nC: a matrix of size n × p;\nk: the number of iterations of the block non-Hermitian Lanczos process.\n\nOutput arguments\n\nV: a dense n × p(k+1) matrix;\nΨ: a dense p × p upper triangular matrix such that V₁Ψ = B;\nT: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p;\nU: a dense n × p(k+1) matrix;\nΦᴴ: a dense p × p upper triangular matrix such that U₁Φᴴ = C;\nTᴴ: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-arnoldi","page":"Block Krylov processes","title":"Block Arnoldi","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vector b in the Arnoldi process is replaced by a matrix B with p columns, we can derive the block Arnoldi process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_arnoldi)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Arnoldi process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A V_k = V_k+1 H_k+1k \n V_k^H V_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where V_k is an orthonormal basis of the block Krylov subspace mathcalK_k^square(AB),","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"H_k+1k =\nbeginbmatrix\n Psi_11 Psi_12 ldots Psi_1k \n Psi_21 ddots ddots vdots \n ddots ddots Psi_k-1k \n Psi_kk-1 Psi_kk \n Psi_k+1k\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function arnoldi returns V_k+1, Gamma, and H_k+1k.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"Related method: BLOCK-GMRES.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"arnoldi(::Any, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.arnoldi-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.arnoldi","text":"V, Γ, H = arnoldi(A, B, k; algo=\"householder\", reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nB: a matrix of size n × p;\nk: the number of iterations of the block Arnoldi process.\n\nKeyword arguments\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\nreorthogonalization: reorthogonalize the new matrices of the block Krylov basis against all previous matrices.\n\nOutput arguments\n\nV: a dense n × p(k+1) matrix;\nΓ: a dense p × p upper triangular matrix such that V₁Γ = B;\nH: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-golub-kahan","page":"Block Krylov processes","title":"Block Golub-Kahan","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vector b in the Golub-Kahan process is replaced by a matrix B with p columns, we can derive the block Golub-Kahan process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_golub_kahan)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Golub-Kahan process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A V_k = U_k+1 B_k \n A^H U_k+1 = V_k+1 L_k+1^H \n V_k^H V_k = U_k^H U_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where V_k and U_k are bases of the block Krylov subspaces mathcalK_k^square(A^HAA^HB) and mathcalK_k^square(AA^HB), respectively,","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"B_k =\nbeginbmatrix\n Omega_1 \n Psi_2 Omega_2 \n ddots ddots \n Psi_k Omega_k \n Psi_k+1 \nendbmatrix\n qquad\nL_k+1^H =\nbeginbmatrix\n Omega_1^H Psi_2^H \n Omega_2^H ddots \n ddots Psi_k^H \n Omega_k^H Psi_k+1^H \n Omega_k+1^H \nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function golub_kahan returns V_k+1, U_k+1, Psi_1 and L_k+1.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"golub_kahan(::Any, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.golub_kahan-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.golub_kahan","text":"V, U, Ψ, L = golub_kahan(A, B, k; algo=\"householder\")\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a matrix of size m × p;\nk: the number of iterations of the block Golub-Kahan process.\n\nKeyword argument\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\n\nOutput arguments\n\nV: a dense n × p(k+1) matrix;\nU: a dense m × p(k+1) matrix;\nΨ: a dense p × p upper triangular matrix such that U₁Ψ = B;\nL: a sparse p(k+1) × p(k+1) block lower bidiagonal matrix with a lower bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-saunders-simon-yip","page":"Block Krylov processes","title":"Block Saunders-Simon-Yip","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vectors b and c in the Saunders-Simon-Yip process are replaced by matrices B and C with both p columns, we can derive the block Saunders-Simon-Yip process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_saunders_simon_yip)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Saunders-Simon-Yip process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A U_k = V_k+1 T_k+1k \n A^H V_k = U_k+1 T_kk+1^H \n V_k^H V_k = U_k^H U_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A A^H 0 endbmatrix beginbmatrix B 0 0 C endbmatrixright),","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"T_k+1k =\nbeginbmatrix\n Omega_1 Phi_2 \n Psi_2 Omega_2 ddots \n ddots ddots Phi_k \n Psi_k Omega_k \n Psi_k+1\nendbmatrix\n qquad\nT_kk+1^H =\nbeginbmatrix\n Omega_1^H Psi_2^H \n Phi_2^H Omega_2^H ddots \n ddots ddots Psi_k^H \n Phi_k^H Omega_k^H \n Phi_k+1^H\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function saunders_simon_yip returns V_k+1, Psi_1, T_k+1k, U_k+1, Phi_1^H and T_kk+1^H.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"saunders_simon_yip(::Any, ::AbstractMatrix{FC}, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.saunders_simon_yip-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.saunders_simon_yip","text":"V, Ψ, T, U, Φᴴ, Tᴴ = saunders_simon_yip(A, B, C, k; algo=\"householder\")\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a matrix of size m × p;\nC: a matrix of size n × p;\nk: the number of iterations of the block Saunders-Simon-Yip process.\n\nKeyword argument\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\n\nOutput arguments\n\nV: a dense m × p(k+1) matrix;\nΨ: a dense p × p upper triangular matrix such that V₁Ψ = B;\nT: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p;\nU: a dense n × p(k+1) matrix;\nΦᴴ: a dense p × p upper triangular matrix such that U₁Φᴴ = C;\nTᴴ: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-montoison-orban","page":"Block Krylov processes","title":"Block Montoison-Orban","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vectors b and c in the Montoison-Orban process are replaced by matrices D and C with both p columns, we can derive the block Montoison-Orban process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_montoison_orban)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Montoison-Orban process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A U_k = V_k+1 H_k+1k \n B V_k = U_k+1 F_k+1k \n V_k^H V_k = U_k^H U_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A B 0 endbmatrix beginbmatrix D 0 0 C endbmatrixright),","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"H_k+1k =\nbeginbmatrix\n Psi_11 Psi_12 ldots Psi_1k \n Psi_21 ddots ddots vdots \n ddots ddots Psi_k-1k \n Psi_kk-1 Psi_kk \n Psi_k+1k\nendbmatrix\n qquad\nF_k+1k =\nbeginbmatrix\n Phi_11 Phi_12 ldots Phi_1k \n Phi_21 ddots ddots vdots \n ddots ddots Phi_k-1k \n Phi_kk-1 Phi_kk \n Phi_k+1k\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function montoison_orban returns V_k+1, Gamma, H_k+1k, U_k+1, Lambda, and F_k+1k.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"montoison_orban(::Any, ::Any, ::AbstractMatrix{FC}, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.montoison_orban-Union{Tuple{FC}, Tuple{Any, Any, AbstractMatrix{FC}, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.montoison_orban","text":"V, Γ, H, U, Λ, F = montoison_orban(A, B, D, C, k; algo=\"householder\", reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a linear operator that models a matrix of dimension n × m;\nD: a matrix of size m × p;\nC: a matrix of size n × p;\nk: the number of iterations of the block Montoison-Orban process.\n\nKeyword arguments\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\nreorthogonalization: reorthogonalize the new matrices of the block Krylov basis against all previous matrices.\n\nOutput arguments\n\nV: a dense m × p(k+1) matrix;\nΓ: a dense p × p upper triangular matrix such that V₁Γ = D;\nH: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p;\nU: a dense n × p(k+1) matrix;\nΛ: a dense p × p upper triangular matrix such that U₁Λ = C;\nF: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"examples/tricg/","page":"TriCG","title":"TriCG","text":"using Krylov, LinearOperators\nusing LinearAlgebra, Printf, SparseArrays\n\n# Identity matrix.\neye(n::Int) = sparse(1.0 * I, n, n)\n\n# Symmetric quasi-definite systems and variants\nn = m = 5\nA = [2^(i/j)*j + (-1)^(i-j) * n*(i-1) for i = 1:n, j = 1:n]\nb = ones(n)\nM = diagm(0 => [3.0 * i for i = 1:n])\nN = diagm(0 => [5.0 * i for i = 1:n])\nc = -b\n\n# [I A] [x] = [b]\n# [Aᴴ -I] [y] [c]\n(x, y, stats) = tricg(A, b, c)\nK = [eye(m) A; A' -eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [-I A] [x] = [b]\n# [ Aᴴ I] [y] [c]\n(x, y, stats) = tricg(A, b, c, flip=true)\nK = [-eye(m) A; A' eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [I A] [x] = [b]\n# [Aᴴ I] [y] [c]\n(x, y, stats) = tricg(A, b, c, spd=true)\nK = [eye(m) A; A' eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [-I A] [x] = [b]\n# [ Aᴴ -I] [y] [c]\n(x, y, stats) = tricg(A, b, c, snd=true)\nK = [-eye(m) A; A' -eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [τI A] [x] = [b]\n# [ Aᴴ νI] [y] [c]\n(τ, ν) = (1e-4, 1e2)\n(x, y, stats) = tricg(A, b, c, τ=τ, ν=ν)\nK = [τ*eye(m) A; A' ν*eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [M⁻¹ A ] [x] = [b]\n# [Aᴴ -N⁻¹] [y] [c]\n(x, y, stats) = tricg(A, b, c, M=M, N=N, verbose=1)\nK = [inv(M) A; A' -inv(N)]\nH = BlockDiagonalOperator(M, N)\nB = [b; c]\nr = B - K * [x; y]\nresid = sqrt(dot(r, H * r))\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)","category":"page"},{"location":"solvers/as/","page":"Adjoint systems","title":"Adjoint systems","text":"# Adjoint systems","category":"page"},{"location":"solvers/as/#BiLQR","page":"Adjoint systems","title":"BiLQR","text":"","category":"section"},{"location":"solvers/as/","page":"Adjoint systems","title":"Adjoint systems","text":"bilqr\nbilqr!","category":"page"},{"location":"solvers/as/#Krylov.bilqr","page":"Adjoint systems","title":"Krylov.bilqr","text":"(x, y, stats) = bilqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n transfer_to_bicg::Bool=true, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, y, stats) = bilqr(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)\n\nBiLQR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.\n\nCombine BiLQ and QMR to solve adjoint systems.\n\n[0 A] [y] = [b]\n[Aᴴ 0] [x] [c]\n\nThe relation bᴴc ≠ 0 must be satisfied. BiLQ is used for solving primal system Ax = b of size n. QMR is used for solving dual system Aᴴy = c of size n.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n;\nc: a vector of length n.\n\nOptional arguments\n\nx0: a vector of length n that represents an initial guess of the solution x;\ny0: a vector of length n that represents an initial guess of the solution y.\n\nKeyword arguments\n\ntransfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length n;\nstats: statistics collected on the run in an AdjointStats structure.\n\nReference\n\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/as/#Krylov.bilqr!","page":"Adjoint systems","title":"Krylov.bilqr!","text":"solver = bilqr!(solver::BilqrSolver, A, b, c; kwargs...)\nsolver = bilqr!(solver::BilqrSolver, A, b, c, x0, y0; kwargs...)\n\nwhere kwargs are keyword arguments of bilqr.\n\nSee BilqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/as/#TriLQR","page":"Adjoint systems","title":"TriLQR","text":"","category":"section"},{"location":"solvers/as/","page":"Adjoint systems","title":"Adjoint systems","text":"trilqr\ntrilqr!","category":"page"},{"location":"solvers/as/#Krylov.trilqr","page":"Adjoint systems","title":"Krylov.trilqr","text":"(x, y, stats) = trilqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n transfer_to_usymcg::Bool=true, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, y, stats) = trilqr(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)\n\nTriLQR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.\n\nCombine USYMLQ and USYMQR to solve adjoint systems.\n\n[0 A] [y] = [b]\n[Aᴴ 0] [x] [c]\n\nUSYMLQ is used for solving primal system Ax = b of size m × n. USYMQR is used for solving dual system Aᴴy = c of size n × m.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional arguments\n\nx0: a vector of length n that represents an initial guess of the solution x;\ny0: a vector of length m that represents an initial guess of the solution y.\n\nKeyword arguments\n\ntransfer_to_usymcg: transfer from the USYMLQ point to the USYMCG point, when it exists. The transfer is based on the residual norm;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in an AdjointStats structure.\n\nReference\n\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/as/#Krylov.trilqr!","page":"Adjoint systems","title":"Krylov.trilqr!","text":"solver = trilqr!(solver::TrilqrSolver, A, b, c; kwargs...)\nsolver = trilqr!(solver::TrilqrSolver, A, b, c, x0, y0; kwargs...)\n\nwhere kwargs are keyword arguments of trilqr.\n\nSee TrilqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"api/#Stats-Types","page":"API","title":"Stats Types","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Krylov.KrylovStats\nKrylov.SimpleStats\nKrylov.LanczosStats\nKrylov.LanczosShiftStats\nKrylov.SymmlqStats\nKrylov.AdjointStats\nKrylov.LNLQStats\nKrylov.LSLQStats\nKrylov.LsmrStats","category":"page"},{"location":"api/#Krylov.KrylovStats","page":"API","title":"Krylov.KrylovStats","text":"Abstract type for statistics returned by a solver\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.SimpleStats","page":"API","title":"Krylov.SimpleStats","text":"Type for statistics returned by the majority of Krylov solvers, the attributes are:\n\nniter\nsolved\ninconsistent\nresiduals\nAresiduals\nAcond\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LanczosStats","page":"API","title":"Krylov.LanczosStats","text":"Type for statistics returned by CG-LANCZOS, the attributes are:\n\nniter\nsolved\nresiduals\nindefinite\nAnorm\nAcond\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LanczosShiftStats","page":"API","title":"Krylov.LanczosShiftStats","text":"Type for statistics returned by CG-LANCZOS with shifts, the attributes are:\n\nniter\nsolved\nresiduals\nindefinite\nAnorm\nAcond\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.SymmlqStats","page":"API","title":"Krylov.SymmlqStats","text":"Type for statistics returned by SYMMLQ, the attributes are:\n\nniter\nsolved\nresiduals\nresidualscg\nerrors\nerrorscg\nAnorm\nAcond\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.AdjointStats","page":"API","title":"Krylov.AdjointStats","text":"Type for statistics returned by adjoint systems solvers BiLQR and TriLQR, the attributes are:\n\nniter\nsolved_primal\nsolved_dual\nresiduals_primal\nresiduals_dual\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LNLQStats","page":"API","title":"Krylov.LNLQStats","text":"Type for statistics returned by the LNLQ method, the attributes are:\n\nniter\nsolved\nresiduals\nerrorwithbnd\nerrorbndx\nerrorbndy\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LSLQStats","page":"API","title":"Krylov.LSLQStats","text":"Type for statistics returned by the LSLQ method, the attributes are:\n\nniter\nsolved\ninconsistent\nresiduals\nAresiduals\nerr_lbnds\nerrorwithbnd\nerrubndslq\nerrubndscg\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LsmrStats","page":"API","title":"Krylov.LsmrStats","text":"Type for statistics returned by LSMR. The attributes are:\n\nniter\nsolved\ninconsistent\nresiduals\nAresiduals\nAcond\nAnorm\nxNorm\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Solver-Types","page":"API","title":"Solver Types","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"KrylovSolver\nBlockKrylovSolver\nMinresSolver\nMinaresSolver\nCgSolver\nCrSolver\nCarSolver\nSymmlqSolver\nCgLanczosSolver\nCgLanczosShiftSolver\nMinresQlpSolver\nDiomSolver\nFomSolver\nDqgmresSolver\nGmresSolver\nUsymlqSolver\nUsymqrSolver\nTricgSolver\nTrimrSolver\nTrilqrSolver\nCgsSolver\nBicgstabSolver\nBilqSolver\nQmrSolver\nBilqrSolver\nCglsSolver\nCrlsSolver\nCgneSolver\nCrmrSolver\nLslqSolver\nLsqrSolver\nLsmrSolver\nLnlqSolver\nCraigSolver\nCraigmrSolver\nGpmrSolver\nFgmresSolver\nBlockGmresSolver","category":"page"},{"location":"api/#Krylov.KrylovSolver","page":"API","title":"Krylov.KrylovSolver","text":"Abstract type for using Krylov solvers in-place\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BlockKrylovSolver","page":"API","title":"Krylov.BlockKrylovSolver","text":"Abstract type for using block Krylov solvers in-place\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.MinresSolver","page":"API","title":"Krylov.MinresSolver","text":"Type for storing the vectors required by the in-place version of MINRES.\n\nThe outer constructors\n\nsolver = MinresSolver(m, n, S; window :: Int=5)\nsolver = MinresSolver(A, b; window :: Int=5)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.MinaresSolver","page":"API","title":"Krylov.MinaresSolver","text":"Type for storing the vectors required by the in-place version of MINARES.\n\nThe outer constructors\n\nsolver = MinaresSolver(m, n, S)\nsolver = MinaresSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgSolver","page":"API","title":"Krylov.CgSolver","text":"Type for storing the vectors required by the in-place version of CG.\n\nThe outer constructors\n\nsolver = CgSolver(m, n, S)\nsolver = CgSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CrSolver","page":"API","title":"Krylov.CrSolver","text":"Type for storing the vectors required by the in-place version of CR.\n\nThe outer constructors\n\nsolver = CrSolver(m, n, S)\nsolver = CrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CarSolver","page":"API","title":"Krylov.CarSolver","text":"Type for storing the vectors required by the in-place version of CAR.\n\nThe outer constructors\n\nsolver = CarSolver(m, n, S)\nsolver = CarSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.SymmlqSolver","page":"API","title":"Krylov.SymmlqSolver","text":"Type for storing the vectors required by the in-place version of SYMMLQ.\n\nThe outer constructors\n\nsolver = SymmlqSolver(m, n, S)\nsolver = SymmlqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgLanczosSolver","page":"API","title":"Krylov.CgLanczosSolver","text":"Type for storing the vectors required by the in-place version of CG-LANCZOS.\n\nThe outer constructors\n\nsolver = CgLanczosSolver(m, n, S)\nsolver = CgLanczosSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgLanczosShiftSolver","page":"API","title":"Krylov.CgLanczosShiftSolver","text":"Type for storing the vectors required by the in-place version of CG-LANCZOS-SHIFT.\n\nThe outer constructors\n\nsolver = CgLanczosShiftSolver(m, n, nshifts, S)\nsolver = CgLanczosShiftSolver(A, b, nshifts)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.MinresQlpSolver","page":"API","title":"Krylov.MinresQlpSolver","text":"Type for storing the vectors required by the in-place version of MINRES-QLP.\n\nThe outer constructors\n\nsolver = MinresQlpSolver(m, n, S)\nsolver = MinresQlpSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.DiomSolver","page":"API","title":"Krylov.DiomSolver","text":"Type for storing the vectors required by the in-place version of DIOM.\n\nThe outer constructors\n\nsolver = DiomSolver(m, n, memory, S)\nsolver = DiomSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.FomSolver","page":"API","title":"Krylov.FomSolver","text":"Type for storing the vectors required by the in-place version of FOM.\n\nThe outer constructors\n\nsolver = FomSolver(m, n, memory, S)\nsolver = FomSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.DqgmresSolver","page":"API","title":"Krylov.DqgmresSolver","text":"Type for storing the vectors required by the in-place version of DQGMRES.\n\nThe outer constructors\n\nsolver = DqgmresSolver(m, n, memory, S)\nsolver = DqgmresSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.GmresSolver","page":"API","title":"Krylov.GmresSolver","text":"Type for storing the vectors required by the in-place version of GMRES.\n\nThe outer constructors\n\nsolver = GmresSolver(m, n, memory, S)\nsolver = GmresSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.UsymlqSolver","page":"API","title":"Krylov.UsymlqSolver","text":"Type for storing the vectors required by the in-place version of USYMLQ.\n\nThe outer constructors\n\nsolver = UsymlqSolver(m, n, S)\nsolver = UsymlqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.UsymqrSolver","page":"API","title":"Krylov.UsymqrSolver","text":"Type for storing the vectors required by the in-place version of USYMQR.\n\nThe outer constructors\n\nsolver = UsymqrSolver(m, n, S)\nsolver = UsymqrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.TricgSolver","page":"API","title":"Krylov.TricgSolver","text":"Type for storing the vectors required by the in-place version of TRICG.\n\nThe outer constructors\n\nsolver = TricgSolver(m, n, S)\nsolver = TricgSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.TrimrSolver","page":"API","title":"Krylov.TrimrSolver","text":"Type for storing the vectors required by the in-place version of TRIMR.\n\nThe outer constructors\n\nsolver = TrimrSolver(m, n, S)\nsolver = TrimrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.TrilqrSolver","page":"API","title":"Krylov.TrilqrSolver","text":"Type for storing the vectors required by the in-place version of TRILQR.\n\nThe outer constructors\n\nsolver = TrilqrSolver(m, n, S)\nsolver = TrilqrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgsSolver","page":"API","title":"Krylov.CgsSolver","text":"Type for storing the vectors required by the in-place version of CGS.\n\nThe outer constructorss\n\nsolver = CgsSolver(m, n, S)\nsolver = CgsSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BicgstabSolver","page":"API","title":"Krylov.BicgstabSolver","text":"Type for storing the vectors required by the in-place version of BICGSTAB.\n\nThe outer constructors\n\nsolver = BicgstabSolver(m, n, S)\nsolver = BicgstabSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BilqSolver","page":"API","title":"Krylov.BilqSolver","text":"Type for storing the vectors required by the in-place version of BILQ.\n\nThe outer constructors\n\nsolver = BilqSolver(m, n, S)\nsolver = BilqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.QmrSolver","page":"API","title":"Krylov.QmrSolver","text":"Type for storing the vectors required by the in-place version of QMR.\n\nThe outer constructors\n\nsolver = QmrSolver(m, n, S)\nsolver = QmrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BilqrSolver","page":"API","title":"Krylov.BilqrSolver","text":"Type for storing the vectors required by the in-place version of BILQR.\n\nThe outer constructors\n\nsolver = BilqrSolver(m, n, S)\nsolver = BilqrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CglsSolver","page":"API","title":"Krylov.CglsSolver","text":"Type for storing the vectors required by the in-place version of CGLS.\n\nThe outer constructors\n\nsolver = CglsSolver(m, n, S)\nsolver = CglsSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CrlsSolver","page":"API","title":"Krylov.CrlsSolver","text":"Type for storing the vectors required by the in-place version of CRLS.\n\nThe outer constructors\n\nsolver = CrlsSolver(m, n, S)\nsolver = CrlsSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgneSolver","page":"API","title":"Krylov.CgneSolver","text":"Type for storing the vectors required by the in-place version of CGNE.\n\nThe outer constructors\n\nsolver = CgneSolver(m, n, S)\nsolver = CgneSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CrmrSolver","page":"API","title":"Krylov.CrmrSolver","text":"Type for storing the vectors required by the in-place version of CRMR.\n\nThe outer constructors\n\nsolver = CrmrSolver(m, n, S)\nsolver = CrmrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LslqSolver","page":"API","title":"Krylov.LslqSolver","text":"Type for storing the vectors required by the in-place version of LSLQ.\n\nThe outer constructors\n\nsolver = LslqSolver(m, n, S)\nsolver = LslqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LsqrSolver","page":"API","title":"Krylov.LsqrSolver","text":"Type for storing the vectors required by the in-place version of LSQR.\n\nThe outer constructors\n\nsolver = LsqrSolver(m, n, S)\nsolver = LsqrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LsmrSolver","page":"API","title":"Krylov.LsmrSolver","text":"Type for storing the vectors required by the in-place version of LSMR.\n\nThe outer constructors\n\nsolver = LsmrSolver(m, n, S)\nsolver = LsmrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LnlqSolver","page":"API","title":"Krylov.LnlqSolver","text":"Type for storing the vectors required by the in-place version of LNLQ.\n\nThe outer constructors\n\nsolver = LnlqSolver(m, n, S)\nsolver = LnlqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CraigSolver","page":"API","title":"Krylov.CraigSolver","text":"Type for storing the vectors required by the in-place version of CRAIG.\n\nThe outer constructors\n\nsolver = CraigSolver(m, n, S)\nsolver = CraigSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CraigmrSolver","page":"API","title":"Krylov.CraigmrSolver","text":"Type for storing the vectors required by the in-place version of CRAIGMR.\n\nThe outer constructors\n\nsolver = CraigmrSolver(m, n, S)\nsolver = CraigmrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.GpmrSolver","page":"API","title":"Krylov.GpmrSolver","text":"Type for storing the vectors required by the in-place version of GPMR.\n\nThe outer constructors\n\nsolver = GpmrSolver(m, n, memory, S)\nsolver = GpmrSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n + m if the value given is larger than n + m.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.FgmresSolver","page":"API","title":"Krylov.FgmresSolver","text":"Type for storing the vectors required by the in-place version of FGMRES.\n\nThe outer constructors\n\nsolver = FgmresSolver(m, n, memory, S)\nsolver = FgmresSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BlockGmresSolver","page":"API","title":"Krylov.BlockGmresSolver","text":"Type for storing the vectors required by the in-place version of BLOCK-GMRES.\n\nThe outer constructors\n\nsolver = BlockGmresSolver(m, n, p, memory, SV, SM)\nsolver = BlockGmresSolver(A, B; memory=5)\n\nmay be used in order to create these vectors. memory is set to div(n,p) if the value given is larger than div(n,p).\n\n\n\n\n\n","category":"type"},{"location":"api/#Utilities","page":"API","title":"Utilities","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Krylov.roots_quadratic\nKrylov.sym_givens\nKrylov.to_boundary\nKrylov.vec2str\nKrylov.ktypeof\nKrylov.kzeros\nKrylov.kones\nKrylov.vector_to_matrix\nKrylov.matrix_to_vector","category":"page"},{"location":"api/#Krylov.roots_quadratic","page":"API","title":"Krylov.roots_quadratic","text":"roots = roots_quadratic(q₂, q₁, q₀; nitref)\n\nFind the real roots of the quadratic\n\nq(x) = q₂ x² + q₁ x + q₀,\n\nwhere q₂, q₁ and q₀ are real. Care is taken to avoid numerical cancellation. Optionally, nitref steps of iterative refinement may be performed to improve accuracy. By default, nitref=1.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.sym_givens","page":"API","title":"Krylov.sym_givens","text":"(c, s, ρ) = sym_givens(a, b)\n\nNumerically stable symmetric Givens reflection. Given a and b reals, return (c, s, ρ) such that\n\n[ c s ] [ a ] = [ ρ ]\n[ s -c ] [ b ] = [ 0 ].\n\n\n\n\n\nNumerically stable symmetric Givens reflection. Given a and b complexes, return (c, s, ρ) with c real and (s, ρ) complexes such that\n\n[ c s ] [ a ] = [ ρ ]\n[ s̅ -c ] [ b ] = [ 0 ].\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.to_boundary","page":"API","title":"Krylov.to_boundary","text":"roots = to_boundary(n, x, d, radius; flip, xNorm2, dNorm2)\n\nGiven a trust-region radius radius, a vector x lying inside the trust-region and a direction d, return σ1 and σ2 such that\n\n‖x + σi d‖ = radius, i = 1, 2\n\nin the Euclidean norm. n is the length of vectors x and d. If known, ‖x‖² and ‖d‖² may be supplied with xNorm2 and dNorm2.\n\nIf flip is set to true, σ1 and σ2 are computed such that\n\n‖x - σi d‖ = radius, i = 1, 2.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.vec2str","page":"API","title":"Krylov.vec2str","text":"s = vec2str(x; ndisp)\n\nDisplay an array in the form\n\n[ -3.0e-01 -5.1e-01 1.9e-01 ... -2.3e-01 -4.4e-01 2.4e-01 ]\n\nwith (ndisp - 1)/2 elements on each side.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.ktypeof","page":"API","title":"Krylov.ktypeof","text":"S = ktypeof(v)\n\nReturn the most relevant storage type S based on the type of v.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.kzeros","page":"API","title":"Krylov.kzeros","text":"v = kzeros(S, n)\n\nCreate a vector of storage type S of length n only composed of zero.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.kones","page":"API","title":"Krylov.kones","text":"v = kones(S, n)\n\nCreate a vector of storage type S of length n only composed of one.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.vector_to_matrix","page":"API","title":"Krylov.vector_to_matrix","text":"M = vector_to_matrix(S)\n\nReturn the dense matrix storage type M related to the dense vector storage type S.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.matrix_to_vector","page":"API","title":"Krylov.matrix_to_vector","text":"S = matrix_to_vector(M)\n\nReturn the dense vector storage type S related to the dense matrix storage type M.\n\n\n\n\n\n","category":"function"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"# Thanks Morten Piibeleht for the hack with the tables!","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"","category":"page"},{"location":"storage/#storage-requirements","page":"Storage requirements","title":"Storage requirements","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"This section provides the storage requirements of all Krylov methods available in Krylov.jl.","category":"page"},{"location":"storage/#Notation","page":"Storage requirements","title":"Notation","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"We denote by m and n the number of rows and columns of the linear problem. The memory parameter of DIOM, FOM, DQGMRES, GMRES, FGMRES and GPMR is k. The numbers of shifts of CG-LANCZOS-SHIFT is p.","category":"page"},{"location":"storage/#Theoretical-storage-requirements","page":"Storage requirements","title":"Theoretical storage requirements","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"The following tables provide the number of coefficients that must be allocated for each Krylov method. The coefficients have the same type as those that compose the linear problem we seek to solve. Each table summarizes the storage requirements of Krylov methods recommended to a specific linear problem.","category":"page"},{"location":"storage/#Hermitian-positive-definite-linear-systems","page":"Storage requirements","title":"Hermitian positive definite linear systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods CG CR CAR CG-LANCZOS CG-LANCZOS-SHIFT\nStorage 4n 5n 7n 5n 3n + 2np + 5p","category":"page"},{"location":"storage/#Hermitian-indefinite-linear-systems","page":"Storage requirements","title":"Hermitian indefinite linear systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods SYMMLQ MINRES MINRES-QLP MINARES\nStorage 5n 6n 6n 8n","category":"page"},{"location":"storage/#Non-Hermitian-square-linear-systems","page":"Storage requirements","title":"Non-Hermitian square linear systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods CGS BICGSTAB BiLQ QMR\nStorage 6n 6n 8n 9n","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods DIOM DQGMRES\nStorage n(2k+1) + 2k - 1 n(2k+2) + 3k + 1","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods FOM GMRES FGMRES\nStoragedfrac n(2+k) +2k + dfrack(k + 1)2 n(2+k) + 3k + dfrack(k + 1)2 n(2+2k) + 3k + dfrack(k + 1)2","category":"page"},{"location":"storage/#Least-norm-problems","page":"Storage requirements","title":"Least-norm problems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods USYMLQ CGNE CRMR LNLQ CRAIG CRAIGMR\nStorage 5n + 3m 3n + 2m 3n + 2m 3n + 4m 3n + 4m 4n + 5m","category":"page"},{"location":"storage/#Least-squares-problems","page":"Storage requirements","title":"Least-squares problems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods USYMQR CGLS CRLS LSLQ LSQR LSMR\nStorage 6n + 3m 3n + 2m 4n + 3m 4n + 2m 4n + 2m 5n + 2m","category":"page"},{"location":"storage/#Adjoint-systems","page":"Storage requirements","title":"Adjoint systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods BiLQR TriLQR\nStorage 11n 6m + 5n","category":"page"},{"location":"storage/#Saddle-point-and-Hermitian-quasi-definite-systems","page":"Storage requirements","title":"Saddle-point and Hermitian quasi-definite systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods TriCG TriMR\nStorage 6n + 6m 8n + 8m","category":"page"},{"location":"storage/#Generalized-saddle-point-and-non-Hermitian-partitioned-systems","page":"Storage requirements","title":"Generalized saddle-point and non-Hermitian partitioned systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Method GPMR\nStorage (2+k)(n+m) + 2k^2 + 11k","category":"page"},{"location":"storage/#Practical-storage-requirements","page":"Storage requirements","title":"Practical storage requirements","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Each method has its own KrylovSolver that contains all the storage needed by the method. In the REPL, the size in bytes of each attribute and the total amount of memory allocated by the solver are displayed when we show a KrylovSolver.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"using Krylov\n\nm = 5000\nn = 12000\nA = rand(Float64, m, n)\nb = rand(Float64, m)\nsolver = LsmrSolver(A, b)\nshow(stdout, solver, show_stats=false)","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"If we want the total number of bytes used by the solver, we can call nbytes = sizeof(solver).","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"nbytes = sizeof(solver)","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Thereafter, we can use Base.format_bytes(nbytes) to recover what is displayed in the REPL.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Base.format_bytes(nbytes)","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"To verify that we match the theoretical results, we just need to multiply the storage requirement of a method by the number of bytes associated to the precision of the linear problem. For instance, we need 4 bytes for the precision Float32, 8 bytes for precisions Float64 and ComplexF32, and 16 bytes for the precision ComplexF64.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"FC = Float64 # precision of the least-squares problem\nncoefs_lsmr = 5*n + 2*m # number of coefficients\nnbytes_lsmr = sizeof(FC) * ncoefs_lsmr # number of bytes","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Therefore, you can check that you have enough memory in RAM to allocate a KrylovSolver.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"free_nbytes = Sys.free_memory()\nBase.format_bytes(free_nbytes) # Total free memory in RAM in bytes.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"note: Note\nBeyond having faster operations, using low precisions, such as simple precision, allows to store more coefficients in RAM and solve larger linear problems.\nIn the file test_allocations.jl, we use the macro @allocated to test that we match the expected storage requirement of each method with a tolerance of 2%.","category":"page"},{"location":"examples/craig/","page":"CRAIG","title":"CRAIG","text":"using Krylov\nusing LinearAlgebra, Printf\n\nm = 5\nn = 8\nλ = 1.0e-3\nA = rand(m, n)\nb = A * ones(n)\nxy_exact = [A λ*I] \\ b # In Julia, this is the min-norm solution!\n\n(x, y, stats) = craig(A, b, λ=λ, atol=0.0, rtol=1.0e-20, verbose=1)\nshow(stats)\n\n# Check that we have a minimum-norm solution.\n# When λ > 0 we solve min ‖(x,s)‖ s.t. Ax + λs = b, and we get s = λy.\n@printf(\"Primal feasibility: %7.1e\\n\", norm(b - A * x - λ^2 * y) / norm(b))\n@printf(\"Dual feasibility: %7.1e\\n\", norm(x - A' * y) / norm(x))\n@printf(\"Error in x: %7.1e\\n\", norm(x - xy_exact[1:n]) / norm(xy_exact[1:n]))\nif λ > 0.0\n @printf(\"Error in y: %7.1e\\n\", norm(λ * y - xy_exact[n+1:n+m]) / norm(xy_exact[n+1:n+m]))\nend","category":"page"},{"location":"examples/cgne/","page":"CGNE","title":"CGNE","text":"using Krylov, HarwellRutherfordBoeing, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"wm2\")\npath = fetch_ssmc(matrix, format=\"RB\")\n\nA = RutherfordBoeingData(joinpath(path[1], \"$(matrix.name[1]).rb\")).data\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\nx_exact = A' * ones(m)\nx_exact_norm = norm(x_exact)\nx_exact /= x_exact_norm\nb = A * x_exact\n(x, stats) = cgne(A, b)\nshow(stats)\nresid = norm(A * x - b) / norm(b)\n@printf(\"CGNE: Relative residual: %7.1e\\n\", resid)\n@printf(\"CGNE: ‖x - x*‖₂: %7.1e\\n\", norm(x - x_exact))","category":"page"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"# Least-squares problems","category":"page"},{"location":"solvers/ls/#CGLS","page":"Least-squares problems","title":"CGLS","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"cgls\ncgls!","category":"page"},{"location":"solvers/ls/#Krylov.cgls","page":"Least-squares problems","title":"Krylov.cgls","text":"(x, stats) = cgls(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, radius::T=zero(T),\n λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ‖x‖₂²\n\nof size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations\n\n(AᴴA + λI) x = Aᴴb\n\nbut is more stable.\n\nCGLS produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSQR, though can be slightly less accurate, but simpler to implement.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, 49(6), pp. 409–436, 1952.\nA. Björck, T. Elfving and Z. Strakos, Stability of Conjugate Gradient and Lanczos Methods for Linear Least Squares Problems, SIAM Journal on Matrix Analysis and Applications, 19(3), pp. 720–736, 1998.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.cgls!","page":"Least-squares problems","title":"Krylov.cgls!","text":"solver = cgls!(solver::CglsSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of cgls.\n\nSee CglsSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#CRLS","page":"Least-squares problems","title":"CRLS","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"crls\ncrls!","category":"page"},{"location":"solvers/ls/#Krylov.crls","page":"Least-squares problems","title":"Krylov.crls","text":"(x, stats) = crls(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, radius::T=zero(T),\n λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ‖x‖₂²\n\nof size m × n using the Conjugate Residuals (CR) method. This method is equivalent to applying MINRES to the normal equations\n\n(AᴴA + λI) x = Aᴴb.\n\nThis implementation recurs the residual r := b - Ax.\n\nCRLS produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSMR, though can be substantially less accurate, but simpler to implement.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nD. C.-L. Fong, Minimum-Residual Methods for Sparse, Least-Squares using Golubg-Kahan Bidiagonalization, Ph.D. Thesis, Stanford University, 2011.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.crls!","page":"Least-squares problems","title":"Krylov.crls!","text":"solver = crls!(solver::CrlsSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of crls.\n\nSee CrlsSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#LSLQ","page":"Least-squares problems","title":"LSLQ","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"lslq\nlslq!","category":"page"},{"location":"solvers/ls/#Krylov.lslq","page":"Least-squares problems","title":"Krylov.lslq","text":"(x, stats) = lslq(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n window::Int=5, transfer_to_lsqr::Bool=false,\n sqd::Bool=false, λ::T=zero(T),\n σ::T=zero(T), etol::T=√eps(T),\n utol::T=√eps(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ²‖x‖₂²\n\nof size m × n using the LSLQ method, where λ ≥ 0 is a regularization parameter. LSLQ is formally equivalent to applying SYMMLQ to the normal equations\n\n(AᴴA + λ²I) x = Aᴴb\n\nbut is more stable.\n\nIf λ > 0, we solve the symmetric and quasi-definite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ -λ²F ] [ x ] = [ 0 ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSLQ is then equivalent to applying SYMMLQ to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).\n\nIf λ = 0, we solve the symmetric and indefinite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ 0 ] [ x ] = [ 0 ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹.\n\nIn this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).\n\nMain features\n\nthe solution estimate is updated along orthogonal directions\nthe norm of the solution estimate ‖xᴸₖ‖₂ is increasing\nthe error ‖eₖ‖₂ := ‖xᴸₖ - x*‖₂ is decreasing\nit is possible to transition cheaply from the LSLQ iterate to the LSQR iterate if there is an advantage (there always is in terms of error)\nif A is rank deficient, identify the minimum least-squares solution\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\ntransfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nσ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;\netol: stopping tolerance based on the lower bound on the error;\nutol: stopping tolerance based on the upper bound on the error;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a LSLQStats structure.\nstats.err_lbnds is a vector of lower bounds on the LQ error–-the vector is empty if window is set to zero\nstats.err_ubnds_lq is a vector of upper bounds on the LQ error–-the vector is empty if σ == 0 is left at zero\nstats.err_ubnds_cg is a vector of upper bounds on the CG error–-the vector is empty if σ == 0 is left at zero\nstats.error_with_bnd is a boolean indicating whether there was an error in the upper bounds computation (cancellation errors, too large σ ...)\n\nStopping conditions\n\nThe iterations stop as soon as one of the following conditions holds true:\n\nthe optimality residual is sufficiently small (stats.status = \"found approximate minimum least-squares solution\") in the sense that either\n‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ atol, or\n1 + ‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ 1\nan approximate zero-residual solution has been found (stats.status = \"found approximate zero-residual solution\") in the sense that either\n‖r‖ / ‖b‖ ≤ btol + atol ‖A‖ * ‖xᴸ‖ / ‖b‖, or\n1 + ‖r‖ / ‖b‖ ≤ 1\nthe estimated condition number of A is too large in the sense that either\n1/cond(A) ≤ 1/conlim (stats.status = \"condition number exceeds tolerance\"), or\n1 + 1/cond(A) ≤ 1 (stats.status = \"condition number seems too large for this machine\")\nthe lower bound on the LQ forward error is less than etol * ‖xᴸ‖\nthe upper bound on the CG forward error is less than utol * ‖xᶜ‖\n\nReferences\n\nR. Estrin, D. Orban and M. A. Saunders, Euclidean-norm error bounds for SYMMLQ and CG, SIAM Journal on Matrix Analysis and Applications, 40(1), pp. 235–253, 2019.\nR. Estrin, D. Orban and M. A. Saunders, LSLQ: An Iterative Method for Linear Least-Squares with an Error Minimization Property, SIAM Journal on Matrix Analysis and Applications, 40(1), pp. 254–275, 2019.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.lslq!","page":"Least-squares problems","title":"Krylov.lslq!","text":"solver = lslq!(solver::LslqSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lslq.\n\nSee LslqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#LSQR","page":"Least-squares problems","title":"LSQR","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"lsqr\nlsqr!","category":"page"},{"location":"solvers/ls/#Krylov.lsqr","page":"Least-squares problems","title":"Krylov.lsqr","text":"(x, stats) = lsqr(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n window::Int=5, sqd::Bool=false, λ::T=zero(T),\n radius::T=zero(T), etol::T=√eps(T),\n axtol::T=√eps(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=zero(T),\n rtol::T=zero(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ²‖x‖₂²\n\nof size m × n using the LSQR method, where λ ≥ 0 is a regularization parameter. LSQR is formally equivalent to applying CG to the normal equations\n\n(AᴴA + λ²I) x = Aᴴb\n\n(and therefore to CGLS) but is more stable.\n\nLSQR produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to CGLS, though can be slightly more accurate.\n\nIf λ > 0, LSQR solves the symmetric and quasi-definite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ -λ²F ] [ x ] = [ 0 ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSQR is then equivalent to applying CG to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).\n\nIf λ = 0, we solve the symmetric and indefinite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ 0 ] [ x ] = [ 0 ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹.\n\nIn this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\netol: stopping tolerance based on the lower bound on the error;\naxtol: tolerance on the backward error;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nC. C. Paige and M. A. Saunders, LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares, ACM Transactions on Mathematical Software, 8(1), pp. 43–71, 1982.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.lsqr!","page":"Least-squares problems","title":"Krylov.lsqr!","text":"solver = lsqr!(solver::LsqrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lsqr.\n\nSee LsqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#LSMR","page":"Least-squares problems","title":"LSMR","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"lsmr\nlsmr!","category":"page"},{"location":"solvers/ls/#Krylov.lsmr","page":"Least-squares problems","title":"Krylov.lsmr","text":"(x, stats) = lsmr(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n window::Int=5, sqd::Bool=false, λ::T=zero(T),\n radius::T=zero(T), etol::T=√eps(T),\n axtol::T=√eps(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=zero(T),\n rtol::T=zero(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ²‖x‖₂²\n\nof size m × n using the LSMR method, where λ ≥ 0 is a regularization parameter. LSMR is formally equivalent to applying MINRES to the normal equations\n\n(AᴴA + λ²I) x = Aᴴb\n\n(and therefore to CRLS) but is more stable.\n\nLSMR produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to CRLS, though can be substantially more accurate.\n\nLSMR can be also used to find a null vector of a singular matrix A by solving the problem min ‖Aᴴx - b‖ with any nonzero vector b. At a minimizer, the residual vector r = b - Aᴴx will satisfy Ar = 0.\n\nIf λ > 0, we solve the symmetric and quasi-definite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ -λ²F ] [ x ] = [ 0 ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSMR is then equivalent to applying MINRES to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).\n\nIf λ = 0, we solve the symmetric and indefinite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ 0 ] [ x ] = [ 0 ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹.\n\nIn this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\netol: stopping tolerance based on the lower bound on the error;\naxtol: tolerance on the backward error;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a LsmrStats structure.\n\nReference\n\nD. C.-L. Fong and M. A. Saunders, LSMR: An Iterative Algorithm for Sparse Least Squares Problems, SIAM Journal on Scientific Computing, 33(5), pp. 2950–2971, 2011.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.lsmr!","page":"Least-squares problems","title":"Krylov.lsmr!","text":"solver = lsmr!(solver::LsmrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lsmr.\n\nSee LsmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#USYMQR","page":"Least-squares problems","title":"USYMQR","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"usymqr\nusymqr!","category":"page"},{"location":"solvers/ls/#Krylov.usymqr","page":"Least-squares problems","title":"Krylov.usymqr","text":"(x, stats) = usymqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = usymqr(A, b, c, x0::AbstractVector; kwargs...)\n\nUSYMQR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nUSYMQR solves the linear least-squares problem min ‖b - Ax‖² of size m × n. USYMQR solves Ax = b if it is consistent.\n\nUSYMQR is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The residual norm ‖b - Ax‖ monotonously decreases in USYMQR. When A is Hermitian and b = c, QMR is equivalent to MINRES. USYMQR is considered as a generalization of MINRES.\n\nIt can also be applied to under-determined and over-determined problems. USYMQR finds the minimum-norm solution if problems are inconsistent.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. A. Saunders, H. D. Simon, and E. L. Yip, Two Conjugate-Gradient-Type Methods for Unsymmetric Linear Equations, SIAM Journal on Numerical Analysis, 25(4), pp. 927–940, 1988.\nA. Buttari, D. Orban, D. Ruiz and D. Titley-Peloquin, A tridiagonalization method for symmetric saddle-point and quasi-definite systems, SIAM Journal on Scientific Computing, 41(5), pp. 409–432, 2019.\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.usymqr!","page":"Least-squares problems","title":"Krylov.usymqr!","text":"solver = usymqr!(solver::UsymqrSolver, A, b, c; kwargs...)\nsolver = usymqr!(solver::UsymqrSolver, A, b, c, x0; kwargs...)\n\nwhere kwargs are keyword arguments of usymqr.\n\nSee UsymqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"preconditioners/#preconditioners","page":"Preconditioners","title":"Preconditioners","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The solvers in Krylov.jl support preconditioners, i.e., transformations that modify a linear system Ax = b into an equivalent form that may yield faster convergence in finite-precision arithmetic. Preconditioning can be used to reduce the condition number of the problem or cluster its eigenvalues or singular values for instance.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The design of preconditioners is highly dependent on the origin of the problem and most preconditioners need to take application-dependent information and structure into account. Specialized preconditioners generally outperform generic preconditioners such as incomplete factorizations.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The construction of a preconditioner necessitates trade-offs because we need to apply it at least once per iteration within a Krylov method. Hence, a preconditioner must be constructed such that it is cheap to apply, while also capturing the characteristics of the original system in some sense.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"There exist three variants of preconditioning:","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Left preconditioning Two-sided preconditioning Right preconditioning\nP_ell^-1Ax = P_ell^-1b P_ell^-1AP_r^-1y = P_ell^-1btextwithx = P_r^-1y AP_r^-1y = btextwithx = P_r^-1y","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"where P_ell and P_r are square and nonsingular.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The left preconditioning preserves the error x_k - x^star whereas the right preconditioning keeps invariant the residual b - A x_k. Two-sided preconditioning is the only variant that allows to preserve the hermicity of a linear system.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"note: Note\nBecause det(P^-1A - lambda I) = det(A - lambda P) det(P^-1) = det(AP^-1 - lambda I), the eigenvalues of P^-1A and AP^-1 are identical. If P = LL^H, L^-1AL^-H also has the same spectrum.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"In Krylov.jl, we call P_ell^-1 and P_r^-1 the preconditioners and we assume that we can apply them with the operation y leftarrow P^-1 * x. It is also common to call P_ell and P_r the preconditioners if the equivalent operation y leftarrow Pbackslashx is available. Krylov.jl supports both approaches thanks to the argument ldiv of the Krylov solvers.","category":"page"},{"location":"preconditioners/#How-to-use-preconditioners-in-Krylov.jl?","page":"Preconditioners","title":"How to use preconditioners in Krylov.jl?","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"info: Info\nA preconditioner only needs to support the operation mul!(y, P⁻¹, x) when ldiv=false or ldiv!(y, P, x) when ldiv=true to be used in Krylov.jl.\nAdditional support for adjoint with preconditioners is required in the methods BILQ and QMR.\nThe default value of a preconditioner in Krylov.jl is the identity operator I.","category":"page"},{"location":"preconditioners/#Square-non-Hermitian-linear-systems","page":"Preconditioners","title":"Square non-Hermitian linear systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: CGS, BILQ, QMR, BiCGSTAB, DQGMRES, GMRES, BLOCK-GMRES, FGMRES, DIOM and FOM.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"A Krylov method dedicated to non-Hermitian linear systems allows the three variants of preconditioning.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners P_ell^-1 P_ell P_r^-1 P_r\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/#Hermitian-linear-systems","page":"Preconditioners","title":"Hermitian linear systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: SYMMLQ, CG, CG-LANCZOS, CG-LANCZOS-SHIFT, CR, CAR, MINRES, MINRES-QLP and MINARES.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"When A is Hermitian, we can only use centered preconditioning L^-1AL^-Hy = L^-1b with x = L^-Hy. Centered preconditioning is a special case of two-sided preconditioning with P_ell = L = P_r^H that maintains hermicity. However, there is no need to specify L and one may specify P_c = LL^H or its inverse directly.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners P_c^-1 P_c\nArguments M with ldiv=false M with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioner M must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Linear-least-squares-problems","page":"Preconditioners","title":"Linear least-squares problems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: CGLS, CRLS, LSLQ, LSQR and LSMR.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nleast-squares problem min tfrac12 b - Ax^2_2 min tfrac12 b - Ax^2_E^-1\nNormal equation A^HAx = A^Hb A^HE^-1Ax = A^HE^-1b\nAugmented system beginbmatrix I A A^H 0 endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix beginbmatrix E A A^H 0 endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"LSLQ, LSQR and LSMR also handle regularized least-squares problems.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nleast-squares problem min tfrac12 b - Ax^2_2 + tfrac12 lambda^2 x^2_2 min tfrac12 b - Ax^2_E^-1 + tfrac12 lambda^2 x^2_F\nNormal equation (A^HA + lambda^2 I)x = A^Hb (A^HE^-1A + lambda^2 F)x = A^HE^-1b\nAugmented system beginbmatrix I A A^H -lambda^2 I endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix beginbmatrix E A A^H -lambda^2 F endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners E^-1 E F^-1 F\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioners M and N must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Linear-least-norm-problems","page":"Preconditioners","title":"Linear least-norm problems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: CGNE, CRMR, LNLQ, CRAIG and CRAIGMR.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nminimum-norm problem min tfrac12 x^2_2textstAx = b min tfrac12 x^2_FtextstAx = b\nNormal equation AA^Hy = btextwithx = A^Hy AF^-1A^Hy = btextwithx = F^-1A^Hy\nAugmented system beginbmatrix -I A^H phantom-A 0 endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix beginbmatrix -F A^H phantom-A 0 endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"LNLQ, CRAIG and CRAIGMR also handle penalized minimum-norm problems.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nminimum-norm problem min tfrac12 x^2_2 + tfrac12 y^2_2textstAx + lambda^2 y = b min tfrac12 x^2_F + tfrac12 y^2_EtextstAx + lambda^2 Ey = b\nNormal equation (AA^H + lambda^2 I)y = btextwithx = A^Hy (AF^-1A^H + lambda^2 E)y = btextwithx = F^-1A^Hy\nAugmented system beginbmatrix -I A^H phantom-A lambda^2 I endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix beginbmatrix -F A^H phantom-A lambda^2 E endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners E^-1 E F^-1 F\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioners M and N must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Saddle-point-and-symmetric-quasi-definite-systems","page":"Preconditioners","title":"Saddle-point and symmetric quasi-definite systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"TriCG and TriMR can take advantage of the structure of Hermitian systems Kz = d with the 2x2 block structure","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":" beginbmatrix tau E phantom-A A^H nu F endbmatrix beginbmatrix x y endbmatrix = beginbmatrix b c endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners E^-1 E F^-1 F\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioners M and N must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Generalized-saddle-point-and-unsymmetric-partitioned-systems","page":"Preconditioners","title":"Generalized saddle-point and unsymmetric partitioned systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"GPMR can take advantage of the structure of general square systems Kz = d with the 2x2 block structure","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":" beginbmatrix lambda M A B mu N endbmatrix beginbmatrix x y endbmatrix = beginbmatrix b c endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Relations CE = M^-1 EC = M DF = N^-1 FD = N\nArguments C and E with ldiv=false C and E with ldiv=true D and F with ldiv=false D and F with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"note: Note\nOur implementations of BiLQ, QMR, BiLQR, USYMLQ, USYMQR and TriLQR don't support preconditioning.","category":"page"},{"location":"preconditioners/#Packages-that-provide-preconditioners","page":"Preconditioners","title":"Packages that provide preconditioners","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"IncompleteLU.jl implements the left-looking and Crout versions of ILU decompositions.\nILUZero.jl is a Julia implementation of incomplete LU factorization with zero level of fill-in. \nLimitedLDLFactorizations.jl for limited-memory LDLᵀ factorization of symmetric matrices.\nAlgebraicMultigrid.jl provides two algebraic multigrid (AMG) preconditioners.\nRandomizedPreconditioners.jl uses randomized numerical linear algebra to construct approximate inverses of matrices.\nBasicLU.jl uses a sparse LU factorization to compute a maximum volume basis that can be used as a preconditioner for least-norm and least-squares problems.","category":"page"},{"location":"preconditioners/#Examples","page":"Preconditioners","title":"Examples","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using Krylov\nn, m = size(A)\nd = [A[i,i] ≠ 0 ? 1 / abs(A[i,i]) : 1 for i=1:n] # Jacobi preconditioner\nP⁻¹ = diagm(d)\nx, stats = symmlq(A, b, M=P⁻¹)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using Krylov\nn, m = size(A)\nd = [1 / norm(A[:,i]) for i=1:m] # diagonal preconditioner\nP⁻¹ = diagm(d)\nx, stats = minres(A, b, M=P⁻¹)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using IncompleteLU, Krylov\nPℓ = ilu(A)\nx, stats = gmres(A, b, M=Pℓ, ldiv=true) # left preconditioning","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using LimitedLDLFactorizations, Krylov\nP = lldl(A)\nP.D .= abs.(P.D)\nx, stats = cg(A, b, M=P, ldiv=true) # centered preconditioning","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using ILUZero, Krylov\nPᵣ = ilu0(A)\nx, stats = bicgstab(A, b, N=Pᵣ, ldiv=true) # right preconditioning","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using LDLFactorizations, Krylov\n\nM = ldl(E)\nN = ldl(F)\n\n# [E A] [x] = [b]\n# [Aᴴ -F] [y] [c]\nx, y, stats = tricg(A, b, c, M=M, N=N, ldiv=true)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using SuiteSparse, Krylov\nimport LinearAlgebra.ldiv!\n\nM = cholesky(E)\n\n# ldiv! is not implemented for the sparse Cholesky factorization (SuiteSparse.CHOLMOD)\nldiv!(y::Vector{T}, F::SuiteSparse.CHOLMOD.Factor{T}, x::Vector{T}) where T = (y .= F \\ x)\n\n# [E A] [x] = [b]\n# [Aᴴ 0] [y] [c]\nx, y, stats = trimr(A, b, c, M=M, sp=true, ldiv=true)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using Krylov\n\nC = lu(M)\n\n# [M A] [x] = [b]\n# [B 0] [y] [c]\nx, y, stats = gpmr(A, B, b, c, C=C, gsp=true, ldiv=true)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"import BasicLU\nusing LinearOperators, Krylov\n\n# Least-squares problem\nm, n = size(A)\nAᴴ = sparse(A')\nbasis, B = BasicLU.maxvolbasis(Aᴴ)\nopA = LinearOperator(A)\nB⁻ᴴ = LinearOperator(Float64, n, n, false, false, (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'T')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'N')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'N')))\n\nd, stats = lsmr(opA * B⁻ᴴ, b) # min ‖AB⁻ᴴd - b‖₂\nx = B⁻ᴴ * d # recover the solution of min ‖Ax - b‖₂\n\n# Least-norm problem\nm, n = size(A)\nbasis, B = maxvolbasis(A)\nopA = LinearOperator(A)\nB⁻¹ = LinearOperator(Float64, m, m, false, false, (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'N')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'T')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'T')))\n\nx, y, stats = craigmr(B⁻¹ * opA, B⁻¹ * b) # min ‖x‖₂ s.t. B⁻¹Ax = B⁻¹b","category":"page"},{"location":"reference/#Reference","page":"Reference","title":"Reference","text":"","category":"section"},{"location":"reference/#Index","page":"Reference","title":"Index","text":"","category":"section"},{"location":"reference/","page":"Reference","title":"Reference","text":"","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"Krylov.FloatOrComplex\nKrylov.niterations\nKrylov.Aprod\nKrylov.Atprod\nKrylov.kstdout\nKrylov.extract_parameters\nBase.show","category":"page"},{"location":"reference/#Krylov.FloatOrComplex","page":"Reference","title":"Krylov.FloatOrComplex","text":"FloatOrComplex{T}\n\nUnion type of T and Complex{T} where T is an AbstractFloat.\n\n\n\n\n\n","category":"type"},{"location":"reference/#Krylov.niterations","page":"Reference","title":"Krylov.niterations","text":"niterations(solver)\n\nReturn the number of iterations performed by the Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"reference/#Krylov.Aprod","page":"Reference","title":"Krylov.Aprod","text":"Aprod(solver)\n\nReturn the number of operator-vector products with A performed by the Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"reference/#Krylov.Atprod","page":"Reference","title":"Krylov.Atprod","text":"Atprod(solver)\n\nReturn the number of operator-vector products with A' performed by the Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"reference/#Krylov.kstdout","page":"Reference","title":"Krylov.kstdout","text":"Default I/O stream for all Krylov methods.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#Krylov.extract_parameters","page":"Reference","title":"Krylov.extract_parameters","text":"arguments = extract_parameters(ex::Expr)\n\nExtract the arguments of an expression that is keyword parameter tuple. Implementation suggested by Mitchell J. O'Sullivan (@mosullivan93).\n\n\n\n\n\n","category":"function"},{"location":"reference/#Base.show","page":"Reference","title":"Base.show","text":"show(io, solver; show_stats=true)\n\nStatistics of solver are displayed if show_stats is set to true.\n\n\n\n\n\n","category":"function"},{"location":"examples/block_gmres/","page":"BLOCK-GMRES","title":"BLOCK-GMRES","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"sherman2\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nB = Matrix{Float64}(I, n, 5)\nB_norm = norm(B)\n\n# Solve Ax = B\nX, stats = block_gmres(A, B)\nshow(stats)\nR = B - A * X\n@printf(\"Relative residual: %8.1e\\n\", norm(R) / B_norm)","category":"page"},{"location":"examples/cg/","page":"CG","title":"CG","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"bcsstk09\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = ones(n)\nb_norm = norm(b)\n\n# Solve Ax = b.\n(x, stats) = cg(A, b)\nshow(stats)\nr = b - A * x\n@printf(\"Relative residual: %8.1e\\n\", norm(r) / b_norm)","category":"page"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"# Non-Hermitian square linear systems","category":"page"},{"location":"solvers/unsymmetric/#BiLQ","page":"Non-Hermitian square linear systems","title":"BiLQ","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"bilq\nbilq!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.bilq","page":"Non-Hermitian square linear systems","title":"Krylov.bilq","text":"(x, stats) = bilq(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, transfer_to_bicg::Bool=true,\n M=I, N=I, ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = bilq(A, b, x0::AbstractVector; kwargs...)\n\nBiLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the square linear system Ax = b of size n using BiLQ. BiLQ is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, BiLQ is equivalent to SYMMLQ. BiLQ requires support for adjoint(M) and adjoint(N) if preconditioners are provided.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\ntransfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\nR. Fletcher, Conjugate gradient methods for indefinite systems, Numerical Analysis, Springer, pp. 73–89, 1976.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.bilq!","page":"Non-Hermitian square linear systems","title":"Krylov.bilq!","text":"solver = bilq!(solver::BilqSolver, A, b; kwargs...)\nsolver = bilq!(solver::BilqSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of bilq.\n\nSee BilqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#QMR","page":"Non-Hermitian square linear systems","title":"QMR","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"qmr\nqmr!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.qmr","page":"Non-Hermitian square linear systems","title":"Krylov.qmr","text":"(x, stats) = qmr(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, M=I, N=I, ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf, verbose::Int=0,\n history::Bool=false, callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = qmr(A, b, x0::AbstractVector; kwargs...)\n\nQMR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the square linear system Ax = b of size n using QMR.\n\nQMR is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, QMR is equivalent to MINRES. QMR requires support for adjoint(M) and adjoint(N) if preconditioners are provided.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nR. W. Freund and N. M. Nachtigal, QMR : a quasi-minimal residual method for non-Hermitian linear systems, Numerische mathematik, Vol. 60(1), pp. 315–339, 1991.\nR. W. Freund and N. M. Nachtigal, An implementation of the QMR method based on coupled two-term recurrences, SIAM Journal on Scientific Computing, Vol. 15(2), pp. 313–337, 1994.\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.qmr!","page":"Non-Hermitian square linear systems","title":"Krylov.qmr!","text":"solver = qmr!(solver::QmrSolver, A, b; kwargs...)\nsolver = qmr!(solver::QmrSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of qmr.\n\nSee QmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#CGS","page":"Non-Hermitian square linear systems","title":"CGS","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"cgs\ncgs!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.cgs","page":"Non-Hermitian square linear systems","title":"Krylov.cgs","text":"(x, stats) = cgs(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, M=I, N=I,\n ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = cgs(A, b, x0::AbstractVector; kwargs...)\n\nCGS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the consistent linear system Ax = b of size n using CGS. CGS requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.\n\nFrom \"Iterative Methods for Sparse Linear Systems (Y. Saad)\" :\n\n«The method is based on a polynomial variant of the conjugate gradients algorithm. Although related to the so-called bi-conjugate gradients (BCG) algorithm, it does not involve adjoint matrix-vector multiplications, and the expected convergence rate is about twice that of the BCG algorithm.\n\nThe Conjugate Gradient Squared algorithm works quite well in many cases. However, one difficulty is that, since the polynomials are squared, rounding errors tend to be more damaging than in the standard BCG algorithm. In particular, very high variations of the residual vectors often cause the residual norms computed to become inaccurate.\n\nTFQMR and BICGSTAB were developed to remedy this difficulty.»\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nP. Sonneveld, CGS, A Fast Lanczos-Type Solver for Nonsymmetric Linear systems, SIAM Journal on Scientific and Statistical Computing, 10(1), pp. 36–52, 1989.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.cgs!","page":"Non-Hermitian square linear systems","title":"Krylov.cgs!","text":"solver = cgs!(solver::CgsSolver, A, b; kwargs...)\nsolver = cgs!(solver::CgsSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of cgs.\n\nSee CgsSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#BiCGSTAB","page":"Non-Hermitian square linear systems","title":"BiCGSTAB","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"bicgstab\nbicgstab!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.bicgstab","page":"Non-Hermitian square linear systems","title":"Krylov.bicgstab","text":"(x, stats) = bicgstab(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, M=I, N=I,\n ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = bicgstab(A, b, x0::AbstractVector; kwargs...)\n\nBICGSTAB can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the square linear system Ax = b of size n using BICGSTAB. BICGSTAB requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.\n\nThe Biconjugate Gradient Stabilized method is a variant of BiCG, like CGS, but using different updates for the Aᴴ-sequence in order to obtain smoother convergence than CGS.\n\nIf BICGSTAB stagnates, we recommend DQGMRES and BiLQ as alternative methods for unsymmetric square systems.\n\nBICGSTAB stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖b‖ * rtol.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nH. A. van der Vorst, Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems, SIAM Journal on Scientific and Statistical Computing, 13(2), pp. 631–644, 1992.\nG. L.G. Sleijpen and D. R. Fokkema, BiCGstab(ℓ) for linear equations involving unsymmetric matrices with complex spectrum, Electronic Transactions on Numerical Analysis, 1, pp. 11–32, 1993.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.bicgstab!","page":"Non-Hermitian square linear systems","title":"Krylov.bicgstab!","text":"solver = bicgstab!(solver::BicgstabSolver, A, b; kwargs...)\nsolver = bicgstab!(solver::BicgstabSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of bicgstab.\n\nSee BicgstabSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#DIOM","page":"Non-Hermitian square linear systems","title":"DIOM","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"diom\ndiom!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.diom","page":"Non-Hermitian square linear systems","title":"Krylov.diom","text":"(x, stats) = diom(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n reorthogonalization::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = diom(A, b, x0::AbstractVector; kwargs...)\n\nDIOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the consistent linear system Ax = b of size n using DIOM.\n\nDIOM only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If CG is well defined on Ax = b and memory = 2, DIOM is theoretically equivalent to CG. If k ≤ memory where k is the number of iterations, DIOM is theoretically equivalent to FOM. Otherwise, DIOM interpolates between CG and FOM and is similar to CG with partial reorthogonalization.\n\nAn advantage of DIOM is that non-Hermitian or Hermitian indefinite or both non-Hermitian and indefinite systems of linear equations can be handled by this single algorithm.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad, Practical use of some krylov subspace methods for solving indefinite and nonsymmetric linear systems, SIAM journal on scientific and statistical computing, 5(1), pp. 203–228, 1984.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.diom!","page":"Non-Hermitian square linear systems","title":"Krylov.diom!","text":"solver = diom!(solver::DiomSolver, A, b; kwargs...)\nsolver = diom!(solver::DiomSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of diom.\n\nNote that the memory keyword argument is the only exception. It's required to create a DiomSolver and can't be changed later.\n\nSee DiomSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#FOM","page":"Non-Hermitian square linear systems","title":"FOM","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"fom\nfom!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.fom","page":"Non-Hermitian square linear systems","title":"Krylov.fom","text":"(x, stats) = fom(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = fom(A, b, x0::AbstractVector; kwargs...)\n\nFOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system Ax = b of size n using FOM.\n\nFOM algorithm is based on the Arnoldi process and a Galerkin condition.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version FOM(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad, Krylov subspace methods for solving unsymmetric linear systems, Mathematics of computation, Vol. 37(155), pp. 105–126, 1981.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.fom!","page":"Non-Hermitian square linear systems","title":"Krylov.fom!","text":"solver = fom!(solver::FomSolver, A, b; kwargs...)\nsolver = fom!(solver::FomSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of fom.\n\nNote that the memory keyword argument is the only exception. It's required to create a FomSolver and can't be changed later.\n\nSee FomSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#DQGMRES","page":"Non-Hermitian square linear systems","title":"DQGMRES","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"dqgmres\ndqgmres!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.dqgmres","page":"Non-Hermitian square linear systems","title":"Krylov.dqgmres","text":"(x, stats) = dqgmres(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n reorthogonalization::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = dqgmres(A, b, x0::AbstractVector; kwargs...)\n\nDQGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the consistent linear system Ax = b of size n using DQGMRES.\n\nDQGMRES algorithm is based on the incomplete Arnoldi orthogonalization process and computes a sequence of approximate solutions with the quasi-minimal residual property.\n\nDQGMRES only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If MINRES is well defined on Ax = b and memory = 2, DQGMRES is theoretically equivalent to MINRES. If k ≤ memory where k is the number of iterations, DQGMRES is theoretically equivalent to GMRES. Otherwise, DQGMRES interpolates between MINRES and GMRES and is similar to MINRES with partial reorthogonalization.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad and K. Wu, DQGMRES: a quasi minimal residual algorithm based on incomplete orthogonalization, Numerical Linear Algebra with Applications, Vol. 3(4), pp. 329–343, 1996.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.dqgmres!","page":"Non-Hermitian square linear systems","title":"Krylov.dqgmres!","text":"solver = dqgmres!(solver::DqgmresSolver, A, b; kwargs...)\nsolver = dqgmres!(solver::DqgmresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of dqgmres.\n\nNote that the memory keyword argument is the only exception. It's required to create a DqgmresSolver and can't be changed later.\n\nSee DqgmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#GMRES","page":"Non-Hermitian square linear systems","title":"GMRES","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"gmres\ngmres!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.gmres","page":"Non-Hermitian square linear systems","title":"Krylov.gmres","text":"(x, stats) = gmres(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = gmres(A, b, x0::AbstractVector; kwargs...)\n\nGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system Ax = b of size n using GMRES.\n\nGMRES algorithm is based on the Arnoldi process and computes a sequence of approximate solutions with the minimum residual.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version GMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad and M. H. Schultz, GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems, SIAM Journal on Scientific and Statistical Computing, Vol. 7(3), pp. 856–869, 1986.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.gmres!","page":"Non-Hermitian square linear systems","title":"Krylov.gmres!","text":"solver = gmres!(solver::GmresSolver, A, b; kwargs...)\nsolver = gmres!(solver::GmresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of gmres.\n\nNote that the memory keyword argument is the only exception. It's required to create a GmresSolver and can't be changed later.\n\nSee GmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#FGMRES","page":"Non-Hermitian square linear systems","title":"FGMRES","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"fgmres\nfgmres!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.fgmres","page":"Non-Hermitian square linear systems","title":"Krylov.fgmres","text":"(x, stats) = fgmres(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = fgmres(A, b, x0::AbstractVector; kwargs...)\n\nFGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system Ax = b of size n using FGMRES.\n\nFGMRES computes a sequence of approximate solutions with minimum residual. FGMRES is a variant of GMRES that allows changes in the right preconditioner at each iteration.\n\nThis implementation allows a left preconditioner M and a flexible right preconditioner N. A situation in which the preconditioner is \"not constant\" is when a relaxation-type method, a Chebyshev iteration or another Krylov subspace method is used as a preconditioner. Compared to GMRES, there is no additional cost incurred in the arithmetic but the memory requirement almost doubles. Thus, GMRES is recommended if the right preconditioner N is constant.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version FGMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad, A Flexible Inner-Outer Preconditioned GMRES Algorithm, SIAM Journal on Scientific Computing, Vol. 14(2), pp. 461–469, 1993.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.fgmres!","page":"Non-Hermitian square linear systems","title":"Krylov.fgmres!","text":"solver = fgmres!(solver::FgmresSolver, A, b; kwargs...)\nsolver = fgmres!(solver::FgmresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of fgmres.\n\nNote that the memory keyword argument is the only exception. It's required to create a FgmresSolver and can't be changed later.\n\nSee FgmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"examples/crmr/","page":"CRMR","title":"CRMR","text":"using Krylov, HarwellRutherfordBoeing, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"gemat1\")\npath = fetch_ssmc(matrix, format=\"RB\")\n\nA = RutherfordBoeingData(joinpath(path[1], \"$(matrix.name[1]).rb\")).data\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\nx_exact = A' * ones(m)\nx_exact_norm = norm(x_exact)\nx_exact /= x_exact_norm\nb = A * x_exact\n(x, stats) = crmr(A, b)\nshow(stats)\nresid = norm(A * x - b) / norm(b)\n@printf(\"CRMR: Relative residual: %7.1e\\n\", resid)\n@printf(\"CRMR: ‖x - x*‖₂: %7.1e\\n\", norm(x - x_exact))","category":"page"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"# Hermitian indefinite linear systems","category":"page"},{"location":"solvers/sid/#SYMMLQ","page":"Hermitian indefinite linear systems","title":"SYMMLQ","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"symmlq\nsymmlq!","category":"page"},{"location":"solvers/sid/#Krylov.symmlq","page":"Hermitian indefinite linear systems","title":"Krylov.symmlq","text":"(x, stats) = symmlq(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, window::Int=5,\n transfer_to_cg::Bool=true, λ::T=zero(T),\n λest::T=zero(T), etol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = symmlq(A, b, x0::AbstractVector; kwargs...)\n\nSYMMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above\n\nSolve the shifted linear system\n\n(A + λI) x = b\n\nof size n using the SYMMLQ method, where λ is a shift parameter, and A is Hermitian.\n\nSYMMLQ produces monotonic errors ‖x* - x‖₂.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\ntransfer_to_cg: transfer from the SYMMLQ point to the CG point, when it exists. The transfer is based on the residual norm;\nλ: regularization parameter;\nλest: positive strict lower bound on the smallest eigenvalue λₘᵢₙ when solving a positive-definite system, such as λest = (1-10⁻⁷)λₘᵢₙ;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\netol: stopping tolerance based on the lower bound on the error;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SymmlqStats structure.\n\nReference\n\nC. C. Paige and M. A. Saunders, Solution of Sparse Indefinite Systems of Linear Equations, SIAM Journal on Numerical Analysis, 12(4), pp. 617–629, 1975.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.symmlq!","page":"Hermitian indefinite linear systems","title":"Krylov.symmlq!","text":"solver = symmlq!(solver::SymmlqSolver, A, b; kwargs...)\nsolver = symmlq!(solver::SymmlqSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of symmlq.\n\nSee SymmlqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#MINRES","page":"Hermitian indefinite linear systems","title":"MINRES","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"minres\nminres!","category":"page"},{"location":"solvers/sid/#Krylov.minres","page":"Hermitian indefinite linear systems","title":"Krylov.minres","text":"(x, stats) = minres(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, window::Int=5,\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), etol::T=√eps(T),\n conlim::T=1/√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = minres(A, b, x0::AbstractVector; kwargs...)\n\nMINRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the shifted linear least-squares problem\n\nminimize ‖b - (A + λI)x‖₂²\n\nor the shifted linear system\n\n(A + λI) x = b\n\nof size n using the MINRES method, where λ ≥ 0 is a shift parameter, where A is Hermitian.\n\nMINRES is formally equivalent to applying CR to Ax=b when A is positive definite, but is typically more stable and also applies to the case where A is indefinite.\n\nMINRES produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\netol: stopping tolerance based on the lower bound on the error;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nC. C. Paige and M. A. Saunders, Solution of Sparse Indefinite Systems of Linear Equations, SIAM Journal on Numerical Analysis, 12(4), pp. 617–629, 1975.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.minres!","page":"Hermitian indefinite linear systems","title":"Krylov.minres!","text":"solver = minres!(solver::MinresSolver, A, b; kwargs...)\nsolver = minres!(solver::MinresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of minres.\n\nSee MinresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#MINRES-QLP","page":"Hermitian indefinite linear systems","title":"MINRES-QLP","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"minres_qlp\nminres_qlp!","category":"page"},{"location":"solvers/sid/#Krylov.minres_qlp","page":"Hermitian indefinite linear systems","title":"Krylov.minres_qlp","text":"(x, stats) = minres_qlp(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, Artol::T=√eps(T),\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = minres_qlp(A, b, x0::AbstractVector; kwargs...)\n\nMINRES-QLP can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nMINRES-QLP is the only method based on the Lanczos process that returns the minimum-norm solution on singular inconsistent systems (A + λI)x = b of size n, where λ is a shift parameter. It is significantly more complex but can be more reliable than MINRES when A is ill-conditioned.\n\nM also indicates the weighted norm in which residuals are measured.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nArtol: relative stopping tolerance based on the Aᴴ-residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nS.-C. T. Choi, Iterative methods for singular linear equations and least-squares problems, Ph.D. thesis, ICME, Stanford University, 2006.\nS.-C. T. Choi, C. C. Paige and M. A. Saunders, MINRES-QLP: A Krylov subspace method for indefinite or singular symmetric systems, SIAM Journal on Scientific Computing, Vol. 33(4), pp. 1810–1836, 2011.\nS.-C. T. Choi and M. A. Saunders, Algorithm 937: MINRES-QLP for symmetric and Hermitian linear equations and least-squares problems, ACM Transactions on Mathematical Software, 40(2), pp. 1–12, 2014.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.minres_qlp!","page":"Hermitian indefinite linear systems","title":"Krylov.minres_qlp!","text":"solver = minres_qlp!(solver::MinresQlpSolver, A, b; kwargs...)\nsolver = minres_qlp!(solver::MinresQlpSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of minres_qlp.\n\nSee MinresQlpSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#MINARES","page":"Hermitian indefinite linear systems","title":"MINARES","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"minares\nminares!","category":"page"},{"location":"solvers/sid/#Krylov.minares","page":"Hermitian indefinite linear systems","title":"Krylov.minares","text":"(x, stats) = minares(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false,\n λ::T = zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), Artol::T = √eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = minares(A, b, x0::AbstractVector; kwargs...)\n\nMINARES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nMINARES solves the Hermitian linear system Ax = b of size n. MINARES minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.\n\nInput arguments\n\nA: a linear operator that models a Hermitian positive definite matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nArtol: relative stopping tolerance based on the Aᴴ-residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nA. Montoison, D. Orban and M. A. Saunders, MinAres: An Iterative Solver for Symmetric Linear Systems, Cahier du GERAD G-2023-40, GERAD, Montréal, 2023.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.minares!","page":"Hermitian indefinite linear systems","title":"Krylov.minares!","text":"solver = minares!(solver::MinaresSolver, A, b; kwargs...)\nsolver = minares!(solver::MinaresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of minares.\n\nSee MinaresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"# Least-norm problems","category":"page"},{"location":"solvers/ln/#CGNE","page":"Least-norm problems","title":"CGNE","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"cgne\ncgne!","category":"page"},{"location":"solvers/ln/#Krylov.cgne","page":"Least-norm problems","title":"Krylov.cgne","text":"(x, stats) = cgne(A, b::AbstractVector{FC};\n N=I, ldiv::Bool=false,\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the consistent linear system\n\nAx + √λs = b\n\nof size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations of the second kind\n\n(AAᴴ + λI) y = b\n\nbut is more stable. When λ = 0, this method solves the minimum-norm problem\n\nmin ‖x‖₂ s.t. Ax = b.\n\nWhen λ > 0, it solves the problem\n\nmin ‖(x,s)‖₂ s.t. Ax + √λs = b.\n\nCGNE produces monotonic errors ‖x-x*‖₂ but not residuals ‖r‖₂. It is formally equivalent to CRAIG, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nN: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nJ. E. Craig, The N-step iteration procedures, Journal of Mathematics and Physics, 34(1), pp. 64–73, 1955.\nJ. E. Craig, Iterations Procedures for Simultaneous Equations, Ph.D. Thesis, Department of Electrical Engineering, MIT, 1954.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.cgne!","page":"Least-norm problems","title":"Krylov.cgne!","text":"solver = cgne!(solver::CgneSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of cgne.\n\nSee CgneSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#CRMR","page":"Least-norm problems","title":"CRMR","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"crmr\ncrmr!","category":"page"},{"location":"solvers/ln/#Krylov.crmr","page":"Least-norm problems","title":"Krylov.crmr","text":"(x, stats) = crmr(A, b::AbstractVector{FC};\n N=I, ldiv::Bool=false,\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the consistent linear system\n\nAx + √λs = b\n\nof size m × n using the Conjugate Residual (CR) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CR to the normal equations of the second kind\n\n(AAᴴ + λI) y = b\n\nbut is more stable. When λ = 0, this method solves the minimum-norm problem\n\nmin ‖x‖₂ s.t. x ∈ argmin ‖Ax - b‖₂.\n\nWhen λ > 0, this method solves the problem\n\nmin ‖(x,s)‖₂ s.t. Ax + √λs = b.\n\nCRMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRAIG-MR, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nN: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nD. Orban and M. Arioli, Iterative Solution of Symmetric Quasi-Definite Linear Systems, Volume 3 of Spotlights. SIAM, Philadelphia, PA, 2017.\nD. Orban, The Projected Golub-Kahan Process for Constrained Linear Least-Squares Problems. Cahier du GERAD G-2014-15, 2014.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.crmr!","page":"Least-norm problems","title":"Krylov.crmr!","text":"solver = crmr!(solver::CrmrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of crmr.\n\nSee CrmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#LNLQ","page":"Least-norm problems","title":"LNLQ","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"lnlq\nlnlq!","category":"page"},{"location":"solvers/ln/#Krylov.lnlq","page":"Least-norm problems","title":"Krylov.lnlq","text":"(x, y, stats) = lnlq(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n transfer_to_craig::Bool=true,\n sqd::Bool=false, λ::T=zero(T),\n σ::T=zero(T), utolx::T=√eps(T),\n utoly::T=√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nFind the least-norm solution of the consistent linear system\n\nAx + λ²y = b\n\nof size m × n using the LNLQ method, where λ ≥ 0 is a regularization parameter.\n\nFor a system in the form Ax = b, LNLQ method is equivalent to applying SYMMLQ to AAᴴy = b and recovering x = Aᴴy but is more stable. Note that y are the Lagrange multipliers of the least-norm problem\n\nminimize ‖x‖ s.t. Ax = b.\n\nIf λ > 0, LNLQ solves the symmetric and quasi-definite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A λ²E ] [ y ] = [ b ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LNLQ is then equivalent to applying SYMMLQ to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.\n\nIf λ = 0, LNLQ solves the symmetric and indefinite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A 0 ] [ y ] = [ b ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖x‖²_F s.t. Ax = b.\n\nIn this case, M can still be specified and indicates the weighted norm in which residuals are measured.\n\nIn this implementation, both the x and y-parts of the solution are returned.\n\nutolx and utoly are tolerances on the upper bound of the distance to the solution ‖x-x‖ and ‖y-y‖, respectively. The bound is valid if λ>0 or σ>0 where σ should be strictly smaller than the smallest positive singular value. For instance σ:=(1-1e-7)σₘᵢₙ .\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\ntransfer_to_craig: transfer from the LNLQ point to the CRAIG point, when it exists. The transfer is based on the residual norm;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nσ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;\nutolx: tolerance on the upper bound on the distance to the solution ‖x-x*‖;\nutoly: tolerance on the upper bound on the distance to the solution ‖y-y*‖;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in a LNLQStats structure.\n\nReference\n\nR. Estrin, D. Orban, M.A. Saunders, LNLQ: An Iterative Method for Least-Norm Problems with an Error Minimization Property, SIAM Journal on Matrix Analysis and Applications, 40(3), pp. 1102–1124, 2019.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.lnlq!","page":"Least-norm problems","title":"Krylov.lnlq!","text":"solver = lnlq!(solver::LnlqSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lnlq.\n\nSee LnlqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#CRAIG","page":"Least-norm problems","title":"CRAIG","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"craig\ncraig!","category":"page"},{"location":"solvers/ln/#Krylov.craig","page":"Least-norm problems","title":"Krylov.craig","text":"(x, y, stats) = craig(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n transfer_to_lsqr::Bool=false, sqd::Bool=false,\n λ::T=zero(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nFind the least-norm solution of the consistent linear system\n\nAx + λ²y = b\n\nof size m × n using the Golub-Kahan implementation of Craig's method, where λ ≥ 0 is a regularization parameter. This method is equivalent to CGNE but is more stable.\n\nFor a system in the form Ax = b, Craig's method is equivalent to applying CG to AAᴴy = b and recovering x = Aᴴy. Note that y are the Lagrange multipliers of the least-norm problem\n\nminimize ‖x‖ s.t. Ax = b.\n\nIf λ > 0, CRAIG solves the symmetric and quasi-definite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A λ²E ] [ y ] = [ b ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. CRAIG is then equivalent to applying CG to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.\n\nIf λ = 0, CRAIG solves the symmetric and indefinite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A 0 ] [ y ] = [ b ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖x‖²_F s.t. Ax = b.\n\nIn this case, M can still be specified and indicates the weighted norm in which residuals are measured.\n\nIn this implementation, both the x and y-parts of the solution are returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\ntransfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nJ. E. Craig, The N-step iteration procedures, Journal of Mathematics and Physics, 34(1-4), pp. 64–73, 1955.\nC. C. Paige and M. A. Saunders, LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares, ACM Transactions on Mathematical Software, 8(1), pp. 43–71, 1982.\nM. A. Saunders, Solutions of Sparse Rectangular Systems Using LSQR and CRAIG, BIT Numerical Mathematics, 35(4), pp. 588–604, 1995.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.craig!","page":"Least-norm problems","title":"Krylov.craig!","text":"solver = craig!(solver::CraigSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of craig.\n\nSee CraigSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#CRAIGMR","page":"Least-norm problems","title":"CRAIGMR","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"craigmr\ncraigmr!","category":"page"},{"location":"solvers/ln/#Krylov.craigmr","page":"Least-norm problems","title":"Krylov.craigmr","text":"(x, y, stats) = craigmr(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n sqd::Bool=false, λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the consistent linear system\n\nAx + λ²y = b\n\nof size m × n using the CRAIGMR method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying the Conjugate Residuals method to the normal equations of the second kind\n\n(AAᴴ + λ²I) y = b\n\nbut is more stable. When λ = 0, this method solves the minimum-norm problem\n\nmin ‖x‖ s.t. x ∈ argmin ‖Ax - b‖.\n\nIf λ > 0, CRAIGMR solves the symmetric and quasi-definite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A λ²E ] [ y ] = [ b ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. CRAIGMR is then equivalent to applying MINRES to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.\n\nIf λ = 0, CRAIGMR solves the symmetric and indefinite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A 0 ] [ y ] = [ b ].\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F s.t. Ax = b.\n\nIn this case, M can still be specified and indicates the weighted norm in which residuals are measured.\n\nCRAIGMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRMR, though can be slightly more accurate, and intricate to implement. Both the x- and y-parts of the solution are returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nD. Orban and M. Arioli. Iterative Solution of Symmetric Quasi-Definite Linear Systems, Volume 3 of Spotlights. SIAM, Philadelphia, PA, 2017.\nD. Orban, The Projected Golub-Kahan Process for Constrained, Linear Least-Squares Problems. Cahier du GERAD G-2014-15, 2014.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.craigmr!","page":"Least-norm problems","title":"Krylov.craigmr!","text":"solver = craigmr!(solver::CraigmrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of craigmr.\n\nSee CraigmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#USYMLQ","page":"Least-norm problems","title":"USYMLQ","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"usymlq\nusymlq!","category":"page"},{"location":"solvers/ln/#Krylov.usymlq","page":"Least-norm problems","title":"Krylov.usymlq","text":"(x, stats) = usymlq(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n transfer_to_usymcg::Bool=true, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = usymlq(A, b, c, x0::AbstractVector; kwargs...)\n\nUSYMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nUSYMLQ determines the least-norm solution of the consistent linear system Ax = b of size m × n.\n\nUSYMLQ is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The error norm ‖x - x*‖ monotonously decreases in USYMLQ. When A is Hermitian and b = c, USYMLQ is equivalent to SYMMLQ. USYMLQ is considered as a generalization of SYMMLQ.\n\nIt can also be applied to under-determined and over-determined problems. In all cases, problems must be consistent.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\ntransfer_to_usymcg: transfer from the USYMLQ point to the USYMCG point, when it exists. The transfer is based on the residual norm;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. A. Saunders, H. D. Simon, and E. L. Yip, Two Conjugate-Gradient-Type Methods for Unsymmetric Linear Equations, SIAM Journal on Numerical Analysis, 25(4), pp. 927–940, 1988.\nA. Buttari, D. Orban, D. Ruiz and D. Titley-Peloquin, A tridiagonalization method for symmetric saddle-point and quasi-definite systems, SIAM Journal on Scientific Computing, 41(5), pp. 409–432, 2019.\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.usymlq!","page":"Least-norm problems","title":"Krylov.usymlq!","text":"solver = usymlq!(solver::UsymlqSolver, A, b, c; kwargs...)\nsolver = usymlq!(solver::UsymlqSolver, A, b, c, x0; kwargs...)\n\nwhere kwargs are keyword arguments of usymlq.\n\nSee UsymlqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"","category":"page"},{"location":"processes/#krylov-processes","page":"Krylov processes","title":"Krylov processes","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Krylov processes are the foundation of Krylov methods, they generate bases of Krylov subspaces. Depending on the Krylov subspaces generated, Krylov processes are more or less specialized for a subset of linear problems. The following table summarizes the most relevant processes for each linear problem.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Linear problems Processes\nHermitian linear systems Hermitian Lanczos\nSquare Non-Hermitian linear systems Non-Hermitian Lanczos – Arnoldi\nLeast-squares problems Golub-Kahan – Saunders-Simon-Yip\nLeast-norm problems Golub-Kahan – Saunders-Simon-Yip\nSaddle-point and Hermitian quasi-definite systems Golub-Kahan – Saunders-Simon-Yip\nGeneralized saddle-point and non-Hermitian partitioned systems Montoison-Orban","category":"page"},{"location":"processes/#Notation","page":"Krylov processes","title":"Notation","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"For a matrix A, A^H denotes the conjugate transpose of A. It coincides with A^T, the transpose of A, for real matrices. Define V_k = beginbmatrix v_1 ldots v_k endbmatrix enspace and enspace U_k = beginbmatrix u_1 ldots u_k endbmatrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"For a matrix C in mathbbC^n times n and a vector t in mathbbC^n, the k-th Krylov subspace generated by C and t is","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"mathcalK_k(C t) =\nleftsum_i=0^k-1 omega_i C^i t middle vert omega_i in mathbbC0 le i le k-1 right","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"For matrices C in mathbbC^n times n enspace and enspace T in mathbbC^n times p, the k-th block Krylov subspace generated by C and T is","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"mathcalK_k^square(C T) =\nleftsum_i=0^k-1 C^i T Omega_i middle vert Omega_i in mathbbC^p times p0 le i le k-1 right","category":"page"},{"location":"processes/#hermitian-lanczos","page":"Krylov processes","title":"Hermitian Lanczos","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: hermitian_lanczos)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Hermitian Lanczos process, the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = V_k T_k + beta_k+1k v_k+1 e_k^T = V_k+1 T_k+1k \n V_k^H V_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k is an orthonormal basis of the Krylov subspace mathcalK_k(Ab),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"T_k =\nbeginbmatrix\n alpha_1 barbeta_2 \n beta_2 alpha_2 ddots \n ddots ddots barbeta_k \n beta_k alpha_k\nendbmatrix\n qquad\nT_k+1k =\nbeginbmatrix\n T_k \n beta_k+1 e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Note that depending on how we normalize the vectors that compose V_k, T_k+1k can be a real tridiagonal matrix even if A is a complex matrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function hermitian_lanczos returns V_k+1, beta_1 and T_k+1k.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: SYMMLQ, CG, CR, CAR, MINRES, MINRES-QLP, MINARES, CGLS, CRLS, CGNE, CRMR, CG-LANCZOS and CG-LANCZOS-SHIFT.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"hermitian_lanczos(::Any, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.hermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.hermitian_lanczos","text":"V, β, T = hermitian_lanczos(A, b, k)\n\nInput arguments\n\nA: a linear operator that models an Hermitian matrix of dimension n;\nb: a vector of length n;\nk: the number of iterations of the Hermitian Lanczos process.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nT: a sparse (k+1) × k tridiagonal matrix.\n\nReference\n\nC. Lanczos, An Iteration Method for the Solution of the Eigenvalue Problem of Linear Differential and Integral Operators, Journal of Research of the National Bureau of Standards, 45(4), pp. 225–280, 1950.\n\n\n\n\n\n","category":"method"},{"location":"processes/#nonhermitian-lanczos","page":"Krylov processes","title":"Non-Hermitian Lanczos","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: nonhermitian_lanczos)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the non-Hermitian Lanczos process (also named the Lanczos biorthogonalization process), the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = V_k T_k + beta_k+1 v_k+1 e_k^T = V_k+1 T_k+1k \n A^H U_k = U_k T_k^H + bargamma_k+1 u_k+1 e_k^T = U_k+1 T_kk+1^H \n V_k^H U_k = U_k^H V_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k and U_k are bases of the Krylov subspaces mathcalK_k (Ab) and mathcalK_k (A^Hc), respectively,","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"T_k = \nbeginbmatrix\n alpha_1 gamma_2 \n beta_2 alpha_2 ddots \n ddots ddots gamma_k \n beta_k alpha_k\nendbmatrix\n qquad\nT_k+1k =\nbeginbmatrix\n T_k \n beta_k+1 e_k^T\nendbmatrix\n qquad\nT_kk+1 =\nbeginbmatrix\n T_k gamma_k+1 e_k\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function nonhermitian_lanczos returns V_k+1, beta_1, T_k+1k, U_k+1, bargamma_1 and T_kk+1^H.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: BiLQ, QMR, BiLQR, CGS and BICGSTAB.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe scaling factors used in our implementation are beta_k = u_k^H v_k^tfrac12 and gamma_k = (u_k^H v_k) beta_k. With these scaling factors, the non-Hermitian Lanczos process coincides with the Hermitian Lanczos process when A = A^H and b = c.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"nonhermitian_lanczos(::Any, ::AbstractVector{FC}, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.nonhermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.nonhermitian_lanczos","text":"V, β, T, U, γᴴ, Tᴴ = nonhermitian_lanczos(A, b, c, k)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nb: a vector of length n;\nc: a vector of length n;\nk: the number of iterations of the non-Hermitian Lanczos process.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nT: a sparse (k+1) × k tridiagonal matrix;\nU: a dense n × (k+1) matrix;\nγᴴ: a coefficient such that γᴴu₁ = c;\nTᴴ: a sparse (k+1) × k tridiagonal matrix.\n\nReference\n\nC. Lanczos, An Iteration Method for the Solution of the Eigenvalue Problem of Linear Differential and Integral Operators, Journal of Research of the National Bureau of Standards, 45(4), pp. 225–280, 1950.\n\n\n\n\n\n","category":"method"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The non-Hermitian Lanczos process can be also implemented without A^H (transpose-free variant). To derive it, we can observe that beta_k+1 v_k+1 = P_k(A) b and bargamma_k+1 u_k+1 = Q_k(A^H) c where P_k and Q_k are polynomials of degree k. The polynomials are defined from the recursions:","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n P_0(A) = I_n \n P_1(A) = left(dfracA - alpha_1 I_nbeta_1right) P_0(A) \n P_k(A) = left(dfracA - alpha_k I_nbeta_kright) P_k-1(A) - dfracgamma_kbeta_k-1 P_k-2(A) quad k ge 2 \n \n Q_0(A^H) = I_n \n Q_1(A^H) = left(dfracA^H - baralpha_1 I_nbargamma_1right) Q_0(A^H) \n Q_k(A^H) = left(dfracA^H - baralpha_k I_nbargamma_kright) Q_k-1(A^H) - dfracbarbeta_kbargamma_k-1 Q_k-2(A^H) quad k ge 2\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Because alpha_k = u_k^H A v_k and (bargamma_k+1 u_k+1)^H (beta_k+1 v_k+1) = gamma_k+1 beta_k+1, we can determine the coefficients of T_k+1k and T_kk+1^H as follows:","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n alpha_k = dfrac1gamma_k beta_k langleQ_k-1(A^H) c A P_k-1(A) brangle \n = dfrac1gamma_k beta_k langlec barQ_k-1(A) A P_k-1(A) brangle \n \n beta_k+1 gamma_k+1 = langleQ_k(A^H) c P_k(A) brangle \n = langlec barQ_k(A) P_k(A) brangle\nendalign*","category":"page"},{"location":"processes/#arnoldi","page":"Krylov processes","title":"Arnoldi","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: arnoldi)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Arnoldi process, the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = V_k H_k + h_k+1k v_k+1 e_k^T = V_k+1 H_k+1k \n V_k^H V_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k is an orthonormal basis of the Krylov subspace mathcalK_k (Ab),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"H_k =\nbeginbmatrix\n h_11 h_12 ldots h_1k \n h_21 ddots ddots vdots \n ddots ddots h_k-1k \n h_kk-1 h_kk\nendbmatrix\n qquad\nH_k+1k =\nbeginbmatrix\n H_k \n h_k+1k e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function arnoldi returns V_k+1, beta and H_k+1k.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: DIOM, FOM, DQGMRES, GMRES and FGMRES.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Arnoldi process coincides with the Hermitian Lanczos process when A is Hermitian.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"arnoldi(::Any, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.arnoldi-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.arnoldi","text":"V, β, H = arnoldi(A, b, k; reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nb: a vector of length n;\nk: the number of iterations of the Arnoldi process.\n\nKeyword argument\n\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nH: a dense (k+1) × k upper Hessenberg matrix.\n\nReference\n\nW. E. Arnoldi, The principle of minimized iterations in the solution of the matrix eigenvalue problem, Quarterly of Applied Mathematics, 9, pp. 17–29, 1951.\n\n\n\n\n\n","category":"method"},{"location":"processes/#golub-kahan","page":"Krylov processes","title":"Golub-Kahan","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: golub_kahan)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Golub-Kahan bidiagonalization process, the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = U_k+1 B_k \n A^H U_k+1 = V_k B_k^H + baralpha_k+1 v_k+1 e_k+1^T = V_k+1 L_k+1^H \n V_k^H V_k = U_k^H U_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k and U_k are bases of the Krylov subspaces mathcalK_k (A^HAA^Hb) and mathcalK_k (AA^Hb), respectively,","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"L_k =\nbeginbmatrix\n alpha_1 \n beta_2 alpha_2 \n ddots ddots \n beta_k alpha_k\nendbmatrix\n qquad\nB_k =\nbeginbmatrix\n alpha_1 \n beta_2 alpha_2 \n ddots ddots \n beta_k alpha_k \n beta_k+1 \nendbmatrix\n=\nbeginbmatrix\n L_k \n beta_k+1 e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Note that depending on how we normalize the vectors that compose V_k and U_k, L_k can be a real bidiagonal matrix even if A is a complex matrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function golub_kahan returns V_k+1, U_k+1, beta_1 and L_k+1.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: LNLQ, CRAIG, CRAIGMR, LSLQ, LSQR and LSMR.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Golub-Kahan process coincides with the Hermitian Lanczos process applied to the normal equations A^HA x = A^Hb and AA^H x = b. It is also related to the Hermitian Lanczos process applied to beginbmatrix 0 A A^H 0 endbmatrix with initial vector beginbmatrix b 0 endbmatrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"golub_kahan(::Any, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.golub_kahan-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.golub_kahan","text":"V, U, β, L = golub_kahan(A, b, k)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nk: the number of iterations of the Golub-Kahan process.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nU: a dense m × (k+1) matrix;\nβ: a coefficient such that βu₁ = b;\nL: a sparse (k+1) × (k+1) lower bidiagonal matrix.\n\nReferences\n\nG. H. Golub and W. Kahan, Calculating the Singular Values and Pseudo-Inverse of a Matrix, SIAM Journal on Numerical Analysis, 2(2), pp. 225–224, 1965.\nC. C. Paige, Bidiagonalization of Matrices and Solution of Linear Equations, SIAM Journal on Numerical Analysis, 11(1), pp. 197–209, 1974.\n\n\n\n\n\n","category":"method"},{"location":"processes/#saunders-simon-yip","page":"Krylov processes","title":"Saunders-Simon-Yip","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: saunders_simon_yip)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Saunders-Simon-Yip process (also named the orthogonal tridiagonalization process), the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A U_k = V_k T_k + beta_k+1 v_k+1 e_k^T = V_k+1 T_k+1k \n A^H V_k = U_k T_k^H + bargamma_k+1 u_k+1 e_k^T = U_k+1 T_kk+1^H \n V_k^H V_k = U_k^H U_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A A^H 0 endbmatrix beginbmatrix b 0 0 c endbmatrixright),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"T_k = \nbeginbmatrix\n alpha_1 gamma_2 \n beta_2 alpha_2 ddots \n ddots ddots gamma_k \n beta_k alpha_k\nendbmatrix\n qquad\nT_k+1k =\nbeginbmatrix\n T_k \n beta_k+1 e_k^T\nendbmatrix\n qquad\nT_kk+1 =\nbeginbmatrix\n T_k gamma_k+1 e_k\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function saunders_simon_yip returns V_k+1, beta_1, T_k+1k, U_k+1, bargamma_1 and T_kk+1^H.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: USYMLQ, USYMQR, TriLQR, TriCG and TriMR.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Saunders-Simon-Yip is equivalent to the block-Lanczos process applied to beginbmatrix 0 A A^H 0 endbmatrix with initial matrix beginbmatrix b 0 0 c endbmatrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"saunders_simon_yip(::Any, ::AbstractVector{FC}, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.saunders_simon_yip-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.saunders_simon_yip","text":"V, β, T, U, γᴴ, Tᴴ = saunders_simon_yip(A, b, c, k)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n;\nk: the number of iterations of the Saunders-Simon-Yip process.\n\nOutput arguments\n\nV: a dense m × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nT: a sparse (k+1) × k tridiagonal matrix;\nU: a dense n × (k+1) matrix;\nγᴴ: a coefficient such that γᴴu₁ = c;\nTᴴ: a sparse (k+1) × k tridiagonal matrix.\n\nReference\n\nM. A. Saunders, H. D. Simon, and E. L. Yip, Two Conjugate-Gradient-Type Methods for Unsymmetric Linear Equations, SIAM Journal on Numerical Analysis, 25(4), pp. 927–940, 1988.\n\n\n\n\n\n","category":"method"},{"location":"processes/#montoison-orban","page":"Krylov processes","title":"Montoison-Orban","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: montoison_orban)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Montoison-Orban process (also named the orthogonal Hessenberg reduction process), the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A U_k = V_k H_k + h_k+1k v_k+1 e_k^T = V_k+1 H_k+1k \n B V_k = U_k F_k + f_k+1k u_k+1 e_k^T = U_k+1 F_k+1k \n V_k^H V_k = U_k^H U_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A B 0 endbmatrix beginbmatrix b 0 0 c endbmatrixright),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"H_k =\nbeginbmatrix\n h_11 h_12 ldots h_1k \n h_21 ddots ddots vdots \n ddots ddots h_k-1k \n h_kk-1 h_kk\nendbmatrix\n qquad\nF_k =\nbeginbmatrix\n f_11 f_12 ldots f_1k \n f_21 ddots ddots vdots \n ddots ddots f_k-1k \n f_kk-1 f_kk\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"H_k+1k =\nbeginbmatrix\n H_k \n h_k+1k e_k^T\nendbmatrix\n qquad\nF_k+1k =\nbeginbmatrix\n F_k \n f_k+1k e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function montoison_orban returns V_k+1, beta, H_k+1k, U_k+1, gamma and F_k+1k.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: GPMR.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Montoison-Orban is equivalent to the block-Arnoldi process applied to beginbmatrix 0 A B 0 endbmatrix with initial matrix beginbmatrix b 0 0 c endbmatrix. It also coincides with the Saunders-Simon-Yip process when B = A^H.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"montoison_orban(::Any, ::Any, ::AbstractVector{FC}, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.montoison_orban-Union{Tuple{FC}, Tuple{Any, Any, AbstractVector{FC}, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.montoison_orban","text":"V, β, H, U, γ, F = montoison_orban(A, B, b, c, k; reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a linear operator that models a matrix of dimension n × m;\nb: a vector of length m;\nc: a vector of length n;\nk: the number of iterations of the Montoison-Orban process.\n\nKeyword argument\n\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors.\n\nOutput arguments\n\nV: a dense m × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nH: a dense (k+1) × k upper Hessenberg matrix;\nU: a dense n × (k+1) matrix;\nγ: a coefficient such that γu₁ = c;\nF: a dense (k+1) × k upper Hessenberg matrix.\n\nReference\n\nA. Montoison and D. Orban, GPMR: An Iterative Method for Unsymmetric Partitioned Linear Systems, SIAM Journal on Matrix Analysis and Applications, 44(1), pp. 293–311, 2023.\n\n\n\n\n\n","category":"method"},{"location":"examples/crls/","page":"CRLS","title":"CRLS","text":"using MatrixMarket, SuiteSparseMatrixCollection\nusing Krylov, LinearOperators\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"well1850\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1])_b.mtx\"))[:]\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\n# Define a regularization parameter.\nλ = 1.0e-3\n\n(x, stats) = crls(A, b, λ=λ)\nshow(stats)\nresid = norm(A' * (A * x - b) + λ * x) / norm(b)\n@printf(\"CRLS: Relative residual: %8.1e\\n\", resid)\n@printf(\"CRLS: ‖x‖: %8.1e\\n\", norm(x))","category":"page"},{"location":"callbacks/#callbacks","page":"Callbacks","title":"Callbacks","text":"","category":"section"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"Each Krylov method is able to call a callback function as callback(solver) at each iteration. The callback should return true if the main loop should terminate, and false otherwise. If the method terminated because of the callback, the output status will be \"user-requested exit\". For example, if the user defines minres_callback(solver::MinresSolver), it can be passed to the solver using","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"(x, stats) = minres(A, b, callback = minres_callback)","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"If you need to write a callback that uses variables that are not in a KrylovSolver, use a closure:","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"function custom_stopping_condition(solver::KrylovSolver, A, b, r, tol)\n mul!(r, A, solver.x)\n r .-= b # r := b - Ax\n bool = norm(r) ≤ tol # tolerance based on the 2-norm of the residual\n return bool\nend\n\ncg_callback(solver) = custom_stopping_condition(solver, A, b, r, tol)\n(x, stats) = cg(A, b, callback = cg_callback)","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"Alternatively, use a structure and make it callable:","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"mutable struct CallbackWorkspace{T}\n A::Matrix{T}\n b::Vector{T}\n r::Vector{T}\n tol::T\nend\n\nfunction (workspace::CallbackWorkspace)(solver::KrylovSolver)\n mul!(workspace.r, workspace.A, solver.x)\n workspace.r .-= workspace.b\n bool = norm(workspace.r) ≤ workspace.tol\n return bool\nend\n\nbicgstab_callback = CallbackWorkspace(A, b, r, tol)\n(x, stats) = bicgstab(A, b, callback = bicgstab_callback)","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"Although the main goal of a callback is to add new stopping conditions, it can also retrieve information from the workspace of a Krylov method along the iterations. We now illustrate how to store all iterates x_k of the GMRES method.","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"S = Krylov.ktypeof(b)\nglobal X = S[] # Storage for GMRES iterates\n\nfunction gmres_callback(solver)\n z = solver.z\n k = solver.inner_iter\n nr = sum(1:k)\n V = solver.V\n R = solver.R\n y = copy(z)\n\n # Solve Rk * yk = zk\n for i = k : -1 : 1\n pos = nr + i - k\n for j = k : -1 : i+1\n y[i] = y[i] - R[pos] * y[j]\n pos = pos - j + 1\n end\n y[i] = y[i] / R[pos]\n end\n\n # xk = Vk * yk\n xk = sum(V[i] * y[i] for i = 1:k)\n push!(X, xk)\n\n return false # We don't want to add new stopping conditions\nend\n\n(x, stats) = gmres(A, b, callback = gmres_callback)","category":"page"},{"location":"gpu/#gpu","page":"GPU support","title":"GPU support","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Krylov methods are well suited for GPU computations because they only require matrix-vector products (u leftarrow Av, u leftarrow A^Hw) and vector operations (v, u^H v, v leftarrow alpha u + beta v), which are highly parallelizable.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"The implementations in Krylov.jl are generic so as to take advantage of the multiple dispatch and broadcast features of Julia. Those allow the implementations to be specialized automatically by the compiler for both CPU and GPU. Thus, Krylov.jl works with GPU backends that build on GPUArrays.jl, such as CUDA.jl, AMDGPU.jl, oneAPI.jl or Metal.jl.","category":"page"},{"location":"gpu/#Nvidia-GPUs","page":"GPU support","title":"Nvidia GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with CUDA.jl and allow computations on Nvidia GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (CuMatrix and CuVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using CUDA, Krylov\n\nif CUDA.functional()\n # CPU Arrays\n A_cpu = rand(20, 20)\n b_cpu = rand(20)\n\n # GPU Arrays\n A_gpu = CuMatrix(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # Solve a square and dense system on an Nivida GPU\n x, stats = bilq(A_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Sparse matrices have a specific storage on Nvidia GPUs (CuSparseMatrixCSC, CuSparseMatrixCSR or CuSparseMatrixCOO):","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using CUDA, Krylov\nusing CUDA.CUSPARSE, SparseArrays\n\nif CUDA.functional()\n # CPU Arrays\n A_cpu = sprand(200, 100, 0.3)\n b_cpu = rand(200)\n\n # GPU Arrays\n A_csc_gpu = CuSparseMatrixCSC(A_cpu)\n A_csr_gpu = CuSparseMatrixCSR(A_cpu)\n A_coo_gpu = CuSparseMatrixCOO(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # Solve a rectangular and sparse system on an Nvidia GPU\n x_csc, stats_csc = lslq(A_csc_gpu, b_gpu)\n x_csr, stats_csr = lsqr(A_csr_gpu, b_gpu)\n x_coo, stats_coo = lsmr(A_coo_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"If you use a Krylov method that only requires A * v products (see here), the most efficient format is CuSparseMatrixCSR. Optimized operator-vector and operator-matrix products that exploit GPU features can be also used by means of linear operators.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"For instance, when executing sparse matrix products on NVIDIA GPUs, the mul! function utilized by Krylov.jl internally calls three routines from CUSPARSE. The first one conducts an analysis of the sparse matrix's structure, the second one determines the size of the buffer that needs to be allocated, and the last one handles the product using the allocated buffer. To circumvent the need for repeated analysis computations and buffer allocation/deallocation with each product, we introduce a KrylovOperator in KrylovPreconditioners.jl. This operator performs the analysis and allocates the buffer only once.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, KrylovPreconditioners\n\n# A_gpu can be a sparse COO, CSC or CSR matrix\nopA_gpu = KrylovOperator(A_gpu)\nx_gpu, stats = gmres(opA_gpu, b_gpu)","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Preconditioners, especially incomplete Cholesky or Incomplete LU factorizations that involve triangular solves, can be applied directly on GPU thanks to efficient operators that take advantage of CUSPARSE routines.","category":"page"},{"location":"gpu/#Example-with-a-symmetric-positive-definite-system","page":"GPU support","title":"Example with a symmetric positive-definite system","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using SparseArrays, Krylov, LinearOperators\nusing CUDA, CUDA.CUSPARSE\n\nif CUDA.functional()\n # Transfer the linear system from the CPU to the GPU\n A_gpu = CuSparseMatrixCSR(A_cpu) # A_gpu = CuSparseMatrixCSC(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # IC(0) decomposition LLᴴ ≈ A for CuSparseMatrixCSC or CuSparseMatrixCSR matrices\n P = ic02(A_gpu)\n\n # Additional vector required for solving triangular systems\n n = length(b_gpu)\n T = eltype(b_gpu)\n z = CUDA.zeros(T, n)\n\n # Solve Py = x\n function ldiv_ic0!(P::CuSparseMatrixCSR, x, y, z)\n ldiv!(z, LowerTriangular(P), x) # Forward substitution with L\n ldiv!(y, LowerTriangular(P)', z) # Backward substitution with Lᴴ\n return y\n end\n\n function ldiv_ic0!(P::CuSparseMatrixCSC, x, y, z)\n ldiv!(z, UpperTriangular(P)', x) # Forward substitution with L\n ldiv!(y, UpperTriangular(P), z) # Backward substitution with Lᴴ\n return y\n end\n\n # Operator that model P⁻¹\n symmetric = hermitian = true\n opM = LinearOperator(T, n, n, symmetric, hermitian, (y, x) -> ldiv_ic0!(P, x, y, z))\n\n # Solve an Hermitian positive definite system with an IC(0) preconditioner on GPU\n x, stats = cg(A_gpu, b_gpu, M=opM)\nend","category":"page"},{"location":"gpu/#Example-with-a-general-square-system","page":"GPU support","title":"Example with a general square system","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using SparseArrays, Krylov, LinearOperators\nusing CUDA, CUDA.CUSPARSE, CUDA.CUSOLVER\n\nif CUDA.functional()\n # Optional -- Compute a permutation vector p such that A[:,p] has no zero diagonal\n p = zfd(A_cpu)\n p .+= 1\n A_cpu = A_cpu[:,p]\n\n # Transfer the linear system from the CPU to the GPU\n A_gpu = CuSparseMatrixCSR(A_cpu) # A_gpu = CuSparseMatrixCSC(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # ILU(0) decomposition LU ≈ A for CuSparseMatrixCSC or CuSparseMatrixCSR matrices\n P = ilu02(A_gpu)\n\n # Additional vector required for solving triangular systems\n n = length(b_gpu)\n T = eltype(b_gpu)\n z = CUDA.zeros(T, n)\n\n # Solve Py = x\n function ldiv_ilu0!(P::CuSparseMatrixCSR, x, y, z)\n ldiv!(z, UnitLowerTriangular(P), x) # Forward substitution with L\n ldiv!(y, UpperTriangular(P), z) # Backward substitution with U\n return y\n end\n\n function ldiv_ilu0!(P::CuSparseMatrixCSC, x, y, z)\n ldiv!(z, LowerTriangular(P), x) # Forward substitution with L\n ldiv!(y, UnitUpperTriangular(P), z) # Backward substitution with U\n return y\n end\n\n # Operator that model P⁻¹\n symmetric = hermitian = false\n opM = LinearOperator(T, n, n, symmetric, hermitian, (y, x) -> ldiv_ilu0!(P, x, y, z))\n\n # Solve a non-Hermitian system with an ILU(0) preconditioner on GPU\n x̄, stats = bicgstab(A_gpu, b_gpu, M=opM)\n\n # Recover the solution of Ax = b with the solution of A[:,p]x̄ = b\n invp = invperm(p)\n x = x̄[invp]\nend","category":"page"},{"location":"gpu/#AMD-GPUs","page":"GPU support","title":"AMD GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with AMDGPU.jl and allow computations on AMD GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (ROCMatrix and ROCVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, AMDGPU\n\nif AMDGPU.functional()\n # CPU Arrays\n A_cpu = rand(ComplexF64, 20, 20)\n A_cpu = A_cpu + A_cpu'\n b_cpu = rand(ComplexF64, 20)\n\n A_gpu = ROCMatrix(A_cpu)\n b_gpu = ROCVector(b_cpu)\n\n # Solve a dense Hermitian system on an AMD GPU\n x, stats = minres(A_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Sparse matrices have a specific storage on AMD GPUs (ROCSparseMatrixCSC, ROCSparseMatrixCSR or ROCSparseMatrixCOO):","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using AMDGPU, Krylov\nusing AMDGPU.rocSPARSE, SparseArrays\n\nif AMDGPU.functional()\n # CPU Arrays\n A_cpu = sprand(100, 200, 0.3)\n b_cpu = rand(100)\n\n # GPU Arrays\n A_csc_gpu = ROCSparseMatrixCSC(A_cpu)\n A_csr_gpu = ROCSparseMatrixCSR(A_cpu)\n A_coo_gpu = ROCSparseMatrixCOO(A_cpu)\n b_gpu = ROCVector(b_cpu)\n\n # Solve a rectangular and sparse system on an AMD GPU\n x_csc, y_csc, stats_csc = lnlq(A_csc_gpu, b_gpu)\n x_csr, y_csr, stats_csr = craig(A_csr_gpu, b_gpu)\n x_coo, y_coo, stats_coo = craigmr(A_coo_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"When executing sparse products on AMD GPUs, the mul! function utilized by Krylov.jl internally calls three routines from rocSPARSE. The first one conducts an analysis of the sparse matrix's structure, the second one determines the size of the buffer that needs to be allocated, and the last one handles the product using the allocated buffer. To circumvent the need for repeated analysis computations and buffer allocation/deallocation with each product, we introduce a KrylovOperator in KrylovPreconditioners.jl. This operator performs the analysis and allocates the buffer only once.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, KrylovPreconditioners\n\n# A_gpu can be a sparse COO, CSC or CSR matrix\nopA_gpu = KrylovOperator(A_gpu)\nx_gpu, stats = bicgstab(opA_gpu, b_gpu)","category":"page"},{"location":"gpu/#Intel-GPUs","page":"GPU support","title":"Intel GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with oneAPI.jl and allow computations on Intel GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (oneMatrix and oneVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, oneAPI\n\nif oneAPI.functional()\n T = Float32 # oneAPI.jl also works with ComplexF32\n m = 20\n n = 10\n\n # CPU Arrays\n A_cpu = rand(T, m, n)\n b_cpu = rand(T, m)\n\n # GPU Arrays\n A_gpu = oneMatrix(A_cpu)\n b_gpu = oneVector(b_cpu)\n\n # Solve a dense least-squares problem on an Intel GPU\n x, stats = lsqr(A_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"note: Note\nThe library oneMKL is interfaced in oneAPI.jl and accelerates linear algebra operations on Intel GPUs. Only dense linear systems are supported for the time being because sparse linear algebra routines are not interfaced yet.","category":"page"},{"location":"gpu/#Apple-M1-GPUs","page":"GPU support","title":"Apple M1 GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with Metal.jl and allow computations on Apple M1 GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (MtlMatrix and MtlVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, Metal\n\nT = Float32 # Metal.jl also works with ComplexF32\nn = 10\nm = 20\n\n# CPU Arrays\nA_cpu = rand(T, n, m)\nb_cpu = rand(T, n)\n\n# GPU Arrays\nA_gpu = MtlMatrix(A_cpu)\nb_gpu = MtlVector(b_cpu)\n\n# Solve a dense least-norm problem on an Apple M1 GPU\nx, stats = craig(A_gpu, b_gpu)","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"warning: Warning\nMetal.jl is under heavy development and is considered experimental for now.","category":"page"},{"location":"#Home","page":"Home","title":"Krylov.jl documentation","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"This package provides implementations of certain of the most useful Krylov method for a variety of problems:","category":"page"},{"location":"","page":"Home","title":"Home","text":"1 - Square or rectangular full-rank systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" Ax = b","category":"page"},{"location":"","page":"Home","title":"Home","text":"should be solved when b lies in the range space of A. This situation occurs when","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is square and nonsingular,\nA is tall and has full column rank and b lies in the range of A.","category":"page"},{"location":"","page":"Home","title":"Home","text":"2 - Linear least-squares problems","category":"page"},{"location":"","page":"Home","title":"Home","text":" min b - Ax","category":"page"},{"location":"","page":"Home","title":"Home","text":"should be solved when b is not in the range of A (inconsistent systems), regardless of the shape and rank of A. This situation mainly occurs when","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is square and singular,\nA is tall and thin.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Underdetermined systems are less common but also occur.","category":"page"},{"location":"","page":"Home","title":"Home","text":"If there are infinitely many such x (because A is column rank-deficient), one with minimum norm is identified","category":"page"},{"location":"","page":"Home","title":"Home","text":" min x quad textsubject to quad x in argmin b - Ax","category":"page"},{"location":"","page":"Home","title":"Home","text":"3 - Linear least-norm problems","category":"page"},{"location":"","page":"Home","title":"Home","text":" min x quad textsubject to quad Ax = b","category":"page"},{"location":"","page":"Home","title":"Home","text":"should be solved when A is column rank-deficient but b is in the range of A (consistent systems), regardless of the shape of A. This situation mainly occurs when","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is square and singular,\nA is short and wide.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Overdetermined systems are less common but also occur.","category":"page"},{"location":"","page":"Home","title":"Home","text":"4 - Adjoint systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" Ax = b quad textand quad A^H y = c","category":"page"},{"location":"","page":"Home","title":"Home","text":"where A can have any shape.","category":"page"},{"location":"","page":"Home","title":"Home","text":"5 - Saddle-point and Hermitian quasi-definite systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" beginbmatrix M phantom-A A^H -N endbmatrix beginbmatrix x y endbmatrix = left(beginbmatrix b 0 endbmatrixbeginbmatrix 0 c endbmatrixbeginbmatrix b c endbmatrixright)","category":"page"},{"location":"","page":"Home","title":"Home","text":"where A can have any shape.","category":"page"},{"location":"","page":"Home","title":"Home","text":"6 - Generalized saddle-point and non-Hermitian partitioned systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" beginbmatrix M A B N endbmatrix beginbmatrix x y endbmatrix = beginbmatrix b c endbmatrix","category":"page"},{"location":"","page":"Home","title":"Home","text":"where A can have any shape and B has the shape of Aᴴ. A, B, b and c must be all nonzero.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Krylov solvers are particularly appropriate in situations where such problems must be solved but a factorization is not possible, either because:","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is not available explicitly,\nA would be dense or would consume an excessive amount of memory if it were materialized,\nfactors would consume an excessive amount of memory.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Iterative methods are recommended in either of the following situations:","category":"page"},{"location":"","page":"Home","title":"Home","text":"the problem is sufficiently large that a factorization is not feasible or would be slow,\nan effective preconditioner is known in cases where the problem has unfavorable spectral structure,\nthe operator can be represented efficiently as a sparse matrix,\nthe operator is fast, i.e., can be applied with better complexity than if it were materialized as a matrix. Certain fast operators would materialize as dense matrices.","category":"page"},{"location":"#Features","page":"Home","title":"Features","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"All solvers in Krylov.jl have in-place version, are compatible with GPU and work in any floating-point data type.","category":"page"},{"location":"#How-to-Install","page":"Home","title":"How to Install","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Krylov can be installed and tested through the Julia package manager:","category":"page"},{"location":"","page":"Home","title":"Home","text":"julia> ]\npkg> add Krylov\npkg> test Krylov","category":"page"},{"location":"#Bug-reports-and-discussions","page":"Home","title":"Bug reports and discussions","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.","category":"page"},{"location":"","page":"Home","title":"Home","text":"If you want to ask a question not suited for a bug report, feel free to start a discussion here. This forum is for general discussion about this repository and the JuliaSmoothOptimizers organization, so questions about any of our packages are welcome.","category":"page"},{"location":"examples/cgls/","page":"CGLS","title":"CGLS","text":"using MatrixMarket, SuiteSparseMatrixCollection\nusing Krylov, LinearOperators\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"well1033\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1])_b.mtx\"))[:]\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\n# Define a regularization parameter.\nλ = 1.0e-3\n\n(x, stats) = cgls(A, b, λ=λ)\nshow(stats)\nresid = norm(A' * (A * x - b) + λ * x) / norm(b)\n@printf(\"CGLS: Relative residual: %8.1e\\n\", resid)\n@printf(\"CGLS: ‖x‖: %8.1e\\n\", norm(x))","category":"page"}]
+[{"location":"examples/minares/","page":"MINARES","title":"MINARES","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"GHS_indef\", \"laser\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = ones(n)\n\n# Solve Ax = b.\nx, stats = minares(A, b)\nshow(stats)\nr = b - A * x\nAr = A * r\n@printf(\"Relative A-residual: %8.1e\\n\", norm(A * r) / norm(A * b))","category":"page"},{"location":"examples/trimr/","page":"TriMR","title":"TriMR","text":"using Krylov, LinearOperators, LDLFactorizations\nusing LinearAlgebra, Printf, SparseArrays\n\n# Identity matrix.\neye(n::Int) = sparse(1.0 * I, n, n)\n\n# Saddle-point systems\nn = m = 5\nA = [2^(i/j)*j + (-1)^(i-j) * n*(i-1) for i = 1:n, j = 1:n]\nb = ones(n)\nD = diagm(0 => [2.0 * i for i = 1:n])\nm, n = size(A)\nc = -b\n\n# [D A] [x] = [b]\n# [Aᴴ 0] [y] [c]\nllt_D = cholesky(D)\nopD⁻¹ = LinearOperator(Float64, 5, 5, true, true, (y, v) -> ldiv!(y, llt_D, v))\nopH⁻¹ = BlockDiagonalOperator(opD⁻¹, eye(n))\n(x, y, stats) = trimr(A, b, c, M=opD⁻¹, sp=true)\nK = [D A; A' zeros(n,n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = sqrt(dot(r, opH⁻¹ * r))\n@printf(\"TriMR: Relative residual: %8.1e\\n\", resid)\n\n# Symmetric quasi-definite systems\nn = m = 5\nA = [2^(i/j)*j + (-1)^(i-j) * n*(i-1) for i = 1:n, j = 1:n]\nb = ones(n)\nM = diagm(0 => [3.0 * i for i = 1:n])\nN = diagm(0 => [5.0 * i for i = 1:n])\nc = -b\n\n# [I A] [x] = [b]\n# [Aᴴ -I] [y] [c]\n(x, y, stats) = trimr(A, b, c)\nK = [eye(m) A; A' -eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriMR: Relative residual: %8.1e\\n\", resid)\n\n# [M A] [x] = [b]\n# [Aᴴ -N] [y] [c]\nldlt_M = ldl(M)\nldlt_N = ldl(N)\nopM⁻¹ = LinearOperator(Float64, size(M,1), size(M,2), true, true, (y, v) -> ldiv!(y, ldlt_M, v))\nopN⁻¹ = LinearOperator(Float64, size(N,1), size(N,2), true, true, (y, v) -> ldiv!(y, ldlt_N, v))\nopH⁻¹ = BlockDiagonalOperator(opM⁻¹, opN⁻¹)\n(x, y, stats) = trimr(A, b, c, M=opM⁻¹, N=opN⁻¹, verbose=1)\nK = [M A; A' -N]\nB = [b; c]\nr = B - K * [x; y]\nresid = sqrt(dot(r, opH⁻¹ * r))\n@printf(\"TriMR: Relative residual: %8.1e\\n\", resid)","category":"page"},{"location":"solvers/spd/","page":"Hermitian positive definite linear systems","title":"Hermitian positive definite linear systems","text":"# Hermitian positive definite linear systems","category":"page"},{"location":"solvers/spd/#CG","page":"Hermitian positive definite linear systems","title":"CG","text":"","category":"section"},{"location":"solvers/spd/","page":"Hermitian positive definite linear systems","title":"Hermitian positive definite linear systems","text":"cg\ncg!","category":"page"},{"location":"solvers/spd/#Krylov.cg","page":"Hermitian positive definite linear systems","title":"Krylov.cg","text":"(x, stats) = cg(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, radius::T=zero(T),\n linesearch::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = cg(A, b, x0::AbstractVector; kwargs...)\n\nCG can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nThe conjugate gradient method to solve the Hermitian linear system Ax = b of size n.\n\nThe method does not abort if A is not definite. M also indicates the weighted norm in which residuals are measured.\n\nInput arguments\n\nA: a linear operator that models a Hermitian positive definite matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\nlinesearch: if true, indicate that the solution is to be used in an inexact Newton method with linesearch. If negative curvature is detected at iteration k > 0, the solution of iteration k-1 is returned. If negative curvature is detected at iteration 0, the right-hand side is returned (i.e., the negative gradient);\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nM. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, 49(6), pp. 409–436, 1952.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#Krylov.cg!","page":"Hermitian positive definite linear systems","title":"Krylov.cg!","text":"solver = cg!(solver::CgSolver, A, b; kwargs...)\nsolver = cg!(solver::CgSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of cg.\n\nSee CgSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#CR","page":"Hermitian positive definite linear systems","title":"CR","text":"","category":"section"},{"location":"solvers/spd/","page":"Hermitian positive definite linear systems","title":"Hermitian positive definite linear systems","text":"cr\ncr!","category":"page"},{"location":"solvers/spd/#Krylov.cr","page":"Hermitian positive definite linear systems","title":"Krylov.cr","text":"(x, stats) = cr(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, radius::T=zero(T),\n linesearch::Bool=false, γ::T=√eps(T),\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = cr(A, b, x0::AbstractVector; kwargs...)\n\nCR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nA truncated version of Stiefel’s Conjugate Residual method to solve the Hermitian linear system Ax = b of size n or the least-squares problem min ‖b - Ax‖ if A is singular. The matrix A must be Hermitian semi-definite. M also indicates the weighted norm in which residuals are measured.\n\nInput arguments\n\nA: a linear operator that models a Hermitian positive definite matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\nlinesearch: if true, indicate that the solution is to be used in an inexact Newton method with linesearch. If negative curvature is detected at iteration k > 0, the solution of iteration k-1 is returned. If negative curvature is detected at iteration 0, the right-hand side is returned (i.e., the negative gradient);\nγ: tolerance to determine that the curvature of the quadratic model is nonpositive;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, 49(6), pp. 409–436, 1952.\nE. Stiefel, Relaxationsmethoden bester Strategie zur Losung linearer Gleichungssysteme, Commentarii Mathematici Helvetici, 29(1), pp. 157–179, 1955.\nD. G. Luenberger, The conjugate residual method for constrained minimization problems, SIAM Journal on Numerical Analysis, 7(3), pp. 390–398, 1970.\nM-A. Dahito and D. Orban, The Conjugate Residual Method in Linesearch and Trust-Region Methods, SIAM Journal on Optimization, 29(3), pp. 1988–2025, 2019.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#Krylov.cr!","page":"Hermitian positive definite linear systems","title":"Krylov.cr!","text":"solver = cr!(solver::CrSolver, A, b; kwargs...)\nsolver = cr!(solver::CrSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of cr.\n\nSee CrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#CAR","page":"Hermitian positive definite linear systems","title":"CAR","text":"","category":"section"},{"location":"solvers/spd/","page":"Hermitian positive definite linear systems","title":"Hermitian positive definite linear systems","text":"car\ncar!","category":"page"},{"location":"solvers/spd/#Krylov.car","page":"Hermitian positive definite linear systems","title":"Krylov.car","text":"(x, stats) = car(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = car(A, b, x0::AbstractVector; kwargs...)\n\nCAR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nCAR solves the Hermitian and positive definite linear system Ax = b of size n. CAR minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.\n\nInput arguments\n\nA: a linear operator that models a Hermitian positive definite matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nA. Montoison, D. Orban and M. A. Saunders, MinAres: An Iterative Solver for Symmetric Linear Systems, Cahier du GERAD G-2023-40, GERAD, Montréal, 2023.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#Krylov.car!","page":"Hermitian positive definite linear systems","title":"Krylov.car!","text":"solver = car!(solver::CarSolver, A, b; kwargs...)\nsolver = car!(solver::CarSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of car.\n\nSee CarSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#CG-LANCZOS","page":"Hermitian positive definite linear systems","title":"CG-LANCZOS","text":"","category":"section"},{"location":"solvers/spd/","page":"Hermitian positive definite linear systems","title":"Hermitian positive definite linear systems","text":"cg_lanczos\ncg_lanczos!","category":"page"},{"location":"solvers/spd/#Krylov.cg_lanczos","page":"Hermitian positive definite linear systems","title":"Krylov.cg_lanczos","text":"(x, stats) = cg_lanczos(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false,\n check_curvature::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = cg_lanczos(A, b, x0::AbstractVector; kwargs...)\n\nCG-LANCZOS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nThe Lanczos version of the conjugate gradient method to solve the Hermitian linear system Ax = b of size n.\n\nThe method does not abort if A is not definite.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\ncheck_curvature: if true, check that the curvature of the quadratic along the search direction is positive, and abort if not, unless linesearch is also true;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a LanczosStats structure.\n\nReferences\n\nA. Frommer and P. Maass, Fast CG-Based Methods for Tikhonov-Phillips Regularization, SIAM Journal on Scientific Computing, 20(5), pp. 1831–1850, 1999.\nC. C. Paige and M. A. Saunders, Solution of Sparse Indefinite Systems of Linear Equations, SIAM Journal on Numerical Analysis, 12(4), pp. 617–629, 1975.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#Krylov.cg_lanczos!","page":"Hermitian positive definite linear systems","title":"Krylov.cg_lanczos!","text":"solver = cg_lanczos!(solver::CgLanczosSolver, A, b; kwargs...)\nsolver = cg_lanczos!(solver::CgLanczosSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of cg_lanczos.\n\nSee CgLanczosSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#CG-LANCZOS-SHIFT","page":"Hermitian positive definite linear systems","title":"CG-LANCZOS-SHIFT","text":"","category":"section"},{"location":"solvers/spd/","page":"Hermitian positive definite linear systems","title":"Hermitian positive definite linear systems","text":"cg_lanczos_shift\ncg_lanczos_shift!","category":"page"},{"location":"solvers/spd/#Krylov.cg_lanczos_shift","page":"Hermitian positive definite linear systems","title":"Krylov.cg_lanczos_shift","text":"(x, stats) = cg_lanczos_shift(A, b::AbstractVector{FC}, shifts::AbstractVector{T};\n M=I, ldiv::Bool=false,\n check_curvature::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nThe Lanczos version of the conjugate gradient method to solve a family of shifted systems\n\n(A + αI) x = b (α = α₁, ..., αₚ)\n\nof size n. The method does not abort if A + αI is not definite.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n;\nshifts: a vector of length p.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\ncheck_curvature: if true, check that the curvature of the quadratic along the search direction is positive, and abort if not, unless linesearch is also true;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a vector of p dense vectors, each one of length n;\nstats: statistics collected on the run in a LanczosShiftStats structure.\n\nReferences\n\nA. Frommer and P. Maass, Fast CG-Based Methods for Tikhonov-Phillips Regularization, SIAM Journal on Scientific Computing, 20(5), pp. 1831–1850, 1999.\nC. C. Paige and M. A. Saunders, Solution of Sparse Indefinite Systems of Linear Equations, SIAM Journal on Numerical Analysis, 12(4), pp. 617–629, 1975.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#Krylov.cg_lanczos_shift!","page":"Hermitian positive definite linear systems","title":"Krylov.cg_lanczos_shift!","text":"solver = cg_lanczos!(solver::CgLanczosShiftSolver, A, b, shifts; kwargs...)\n\nwhere kwargs are keyword arguments of cg_lanczos_shift.\n\nSee CgLanczosShiftSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"examples/craigmr/","page":"CRAIGMR","title":"CRAIGMR","text":"using Krylov, HarwellRutherfordBoeing, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"wm1\")\npath = fetch_ssmc(matrix, format=\"RB\")\n\nA = RutherfordBoeingData(joinpath(path[1], \"$(matrix.name[1]).rb\")).data\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\nx_exact = A' * ones(m)\nx_exact_norm = norm(x_exact)\nx_exact /= x_exact_norm\nb = A * x_exact\n(x, y, stats) = craigmr(A, b)\nshow(stats)\nresid = norm(A * x - b) / norm(b)\n@printf(\"CRAIGMR: Relative residual: %7.1e\\n\", resid)\n@printf(\"CRAIGMR: ‖x - x*‖₂: %7.1e\\n\", norm(x - x_exact))\n@printf(\"CRAIGMR: %d iterations\\n\", length(stats.residuals))","category":"page"},{"location":"examples/symmlq/","page":"SYMMLQ","title":"SYMMLQ","text":"using Krylov\nusing LinearAlgebra, Printf\n\nA = diagm([1.0; 2.0; 3.0; 0.0])\nn = size(A, 1)\nb = [1.0; 2.0; 3.0; 0.0]\nb_norm = norm(b)\n\n# SYMMLQ returns the minimum-norm solution of symmetric, singular and consistent systems\n(x, stats) = symmlq(A, b, transfer_to_cg=false);\nr = b - A * x;\n\n@printf(\"Residual r: %s\\n\", Krylov.vec2str(r))\n@printf(\"Relative residual norm ‖r‖: %8.1e\\n\", norm(r) / b_norm)\n@printf(\"Solution x: %s\\n\", Krylov.vec2str(x))\n@printf(\"Minimum-norm solution? %s\\n\", x ≈ [1.0; 1.0; 1.0; 0.0])","category":"page"},{"location":"examples/dqgmres/","page":"DQGMRES","title":"DQGMRES","text":"using Krylov, LinearOperators, ILUZero, MatrixMarket\nusing LinearAlgebra, Printf, SuiteSparseMatrixCollection\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"Simon\", \"raefsky1\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = A * ones(n)\n\nF = ilu0(A)\n\n@printf(\"nnz(ILU) / nnz(A): %7.1e\\n\", nnz(F) / nnz(A))\n\n# Solve Ax = b with DQGMRES and an ILU(0) preconditioner\n# Remark: DIOM, FOM and GMRES can be used in the same way\nopM = LinearOperator(Float64, n, n, false, false, (y, v) -> forward_substitution!(y, F, v))\nopN = LinearOperator(Float64, n, n, false, false, (y, v) -> backward_substitution!(y, F, v))\nopP = LinearOperator(Float64, n, n, false, false, (y, v) -> ldiv!(y, F, v))\n\n# Without preconditioning\nx, stats = dqgmres(A, b, memory=50, history=true)\nr = b - A * x\n@printf(\"[Without preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Without preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)\n\n# Split preconditioning\nx, stats = dqgmres(A, b, memory=50, history=true, M=opM, N=opN)\nr = b - A * x\n@printf(\"[Split preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Split preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)\n\n# Left preconditioning\nx, stats = dqgmres(A, b, memory=50, history=true, M=opP)\nr = b - A * x\n@printf(\"[Left preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Left preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)\n\n# Right preconditioning\nx, stats = dqgmres(A, b, memory=50, history=true, N=opP)\nr = b - A * x\n@printf(\"[Right preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Right preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)","category":"page"},{"location":"solvers/sp_sqd/","page":"Saddle-point and Hermitian quasi-definite systems","title":"Saddle-point and Hermitian quasi-definite systems","text":"# Saddle-point and Hermitian quasi-definite systems","category":"page"},{"location":"solvers/sp_sqd/#TriCG","page":"Saddle-point and Hermitian quasi-definite systems","title":"TriCG","text":"","category":"section"},{"location":"solvers/sp_sqd/","page":"Saddle-point and Hermitian quasi-definite systems","title":"Saddle-point and Hermitian quasi-definite systems","text":"tricg\ntricg!","category":"page"},{"location":"solvers/sp_sqd/#Krylov.tricg","page":"Saddle-point and Hermitian quasi-definite systems","title":"Krylov.tricg","text":"(x, y, stats) = tricg(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n spd::Bool=false, snd::Bool=false,\n flip::Bool=false, τ::T=one(T),\n ν::T=-one(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, y, stats) = tricg(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)\n\nTriCG can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.\n\nGiven a matrix A of dimension m × n, TriCG solves the Hermitian linear system\n\n[ τE A ] [ x ] = [ b ]\n[ Aᴴ νF ] [ y ] [ c ],\n\nof size (n+m) × (n+m) where τ and ν are real numbers, E = M⁻¹ ≻ 0 and F = N⁻¹ ≻ 0. b and c must both be nonzero. TriCG could breakdown if τ = 0 or ν = 0. It's recommended to use TriMR in these cases.\n\nBy default, TriCG solves Hermitian and quasi-definite linear systems with τ = 1 and ν = -1.\n\nTriCG is based on the preconditioned orthogonal tridiagonalization process and its relation with the preconditioned block-Lanczos process.\n\n[ M 0 ]\n[ 0 N ]\n\nindicates the weighted norm in which residuals are measured. It's the Euclidean norm when M and N are identity operators.\n\nTriCG stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional arguments\n\nx0: a vector of length m that represents an initial guess of the solution x;\ny0: a vector of length n that represents an initial guess of the solution y.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the partitioned system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the partitioned system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nspd: if true, set τ = 1 and ν = 1 for Hermitian and positive-definite linear system;\nsnd: if true, set τ = -1 and ν = -1 for Hermitian and negative-definite linear systems;\nflip: if true, set τ = -1 and ν = 1 for another known variant of Hermitian quasi-definite systems;\nτ and ν: diagonal scaling factors of the partitioned Hermitian linear system;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length m;\ny: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nA. Montoison and D. Orban, TriCG and TriMR: Two Iterative Methods for Symmetric Quasi-Definite Systems, SIAM Journal on Scientific Computing, 43(4), pp. 2502–2525, 2021.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sp_sqd/#Krylov.tricg!","page":"Saddle-point and Hermitian quasi-definite systems","title":"Krylov.tricg!","text":"solver = tricg!(solver::TricgSolver, A, b, c; kwargs...)\nsolver = tricg!(solver::TricgSolver, A, b, c, x0, y0; kwargs...)\n\nwhere kwargs are keyword arguments of tricg.\n\nSee TricgSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sp_sqd/#TriMR","page":"Saddle-point and Hermitian quasi-definite systems","title":"TriMR","text":"","category":"section"},{"location":"solvers/sp_sqd/","page":"Saddle-point and Hermitian quasi-definite systems","title":"Saddle-point and Hermitian quasi-definite systems","text":"trimr\ntrimr!","category":"page"},{"location":"solvers/sp_sqd/#Krylov.trimr","page":"Saddle-point and Hermitian quasi-definite systems","title":"Krylov.trimr","text":"(x, y, stats) = trimr(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n spd::Bool=false, snd::Bool=false,\n flip::Bool=false, sp::Bool=false,\n τ::T=one(T), ν::T=-one(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, y, stats) = trimr(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)\n\nTriMR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.\n\nGiven a matrix A of dimension m × n, TriMR solves the symmetric linear system\n\n[ τE A ] [ x ] = [ b ]\n[ Aᴴ νF ] [ y ] [ c ],\n\nof size (n+m) × (n+m) where τ and ν are real numbers, E = M⁻¹ ≻ 0, F = N⁻¹ ≻ 0. b and c must both be nonzero. TriMR handles saddle-point systems (τ = 0 or ν = 0) and adjoint systems (τ = 0 and ν = 0) without any risk of breakdown.\n\nBy default, TriMR solves symmetric and quasi-definite linear systems with τ = 1 and ν = -1.\n\nTriMR is based on the preconditioned orthogonal tridiagonalization process and its relation with the preconditioned block-Lanczos process.\n\n[ M 0 ]\n[ 0 N ]\n\nindicates the weighted norm in which residuals are measured. It's the Euclidean norm when M and N are identity operators.\n\nTriMR stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional arguments\n\nx0: a vector of length m that represents an initial guess of the solution x;\ny0: a vector of length n that represents an initial guess of the solution y.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the partitioned system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the partitioned system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nspd: if true, set τ = 1 and ν = 1 for Hermitian and positive-definite linear system;\nsnd: if true, set τ = -1 and ν = -1 for Hermitian and negative-definite linear systems;\nflip: if true, set τ = -1 and ν = 1 for another known variant of Hermitian quasi-definite systems;\nsp: if true, set τ = 1 and ν = 0 for saddle-point systems;\nτ and ν: diagonal scaling factors of the partitioned Hermitian linear system;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length m;\ny: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nA. Montoison and D. Orban, TriCG and TriMR: Two Iterative Methods for Symmetric Quasi-Definite Systems, SIAM Journal on Scientific Computing, 43(4), pp. 2502–2525, 2021.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sp_sqd/#Krylov.trimr!","page":"Saddle-point and Hermitian quasi-definite systems","title":"Krylov.trimr!","text":"solver = trimr!(solver::TrimrSolver, A, b, c; kwargs...)\nsolver = trimr!(solver::TrimrSolver, A, b, c, x0, y0; kwargs...)\n\nwhere kwargs are keyword arguments of trimr.\n\nSee TrimrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"examples/car/","page":"CAR","title":"CAR","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"Nasa\", \"nasa2146\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = ones(n)\nb_norm = norm(b)\n\n# Solve Ax = b.\n(x, stats) = car(A, b)\nshow(stats)\nr = b - A * x\n@printf(\"Relative residual: %8.1e\\n\", norm(r) / b_norm)","category":"page"},{"location":"examples/bicgstab/","page":"BICGSTAB","title":"BICGSTAB","text":"using Krylov, LinearOperators, IncompleteLU, HarwellRutherfordBoeing\nusing LinearAlgebra, Printf, SuiteSparseMatrixCollection, SparseArrays\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"sherman5\")\npath = fetch_ssmc(matrix, format=\"RB\")\n\nn = matrix.nrows[1]\nA = RutherfordBoeingData(joinpath(path[1], \"$(matrix.name[1]).rb\")).data\nb = A * ones(n)\n\nF = ilu(A, τ = 0.05)\n\n@printf(\"nnz(ILU) / nnz(A): %7.1e\\n\", nnz(F) / nnz(A))\n\n# Solve Ax = b with BICGSTAB and an incomplete LU factorization\n# Remark: CGS can be used in the same way\nopM = LinearOperator(Float64, n, n, false, false, (y, v) -> forward_substitution!(y, F, v))\nopN = LinearOperator(Float64, n, n, false, false, (y, v) -> backward_substitution!(y, F, v))\nopP = LinearOperator(Float64, n, n, false, false, (y, v) -> ldiv!(y, F, v))\n\n# Without preconditioning\nx, stats = bicgstab(A, b, history=true)\nr = b - A * x\n@printf(\"[Without preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Without preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)\n\n# Split preconditioning\nx, stats = bicgstab(A, b, history=true, M=opM, N=opN)\nr = b - A * x\n@printf(\"[Split preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Split preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)\n\n# Left preconditioning\nx, stats = bicgstab(A, b, history=true, M=opP)\nr = b - A * x\n@printf(\"[Left preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Left preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)\n\n# Right preconditioning\nx, stats = bicgstab(A, b, history=true, N=opP)\nr = b - A * x\n@printf(\"[Right preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Right preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"The most expensive procedures in Krylov methods are matrix-vector products (y ← Ax) and vector operations (dot products, vector norms, y ← αx + βy). Therefore they directly affect the efficiency of the methods provided by Krylov.jl. In this section, we present various optimizations based on multithreading to speed up these procedures. Multithreading is a form of shared memory parallelism that makes use of multiple cores to perform tasks.","category":"page"},{"location":"tips/#Multi-threaded-BLAS-and-LAPACK-operations","page":"Performance tips","title":"Multi-threaded BLAS and LAPACK operations","text":"","category":"section"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"OPENBLAS_NUM_THREADS=N julia # Set the number of OpenBLAS threads to N\nMKL_NUM_THREADS=N julia # Set the number of MKL threads to N","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"If you don't know the maximum number of threads available on your computer, you can obtain it with","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"NMAX = Sys.CPU_THREADS","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"and define the number of OpenBLAS/MKL threads at runtime with","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"BLAS.set_num_threads(N) # 1 ≤ N ≤ NMAX\nBLAS.get_num_threads()","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"The recommended number of BLAS threads is the number of physical and not logical cores, which is in general N = NMAX / 2 if your CPU supports simultaneous multithreading (SMT).","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"By default Julia ships with OpenBLAS but it's also possible to use Intel MKL BLAS and LAPACK with MKL.jl. If your operating system is MacOS 13.4 or later, it's recommended to use Accelerate BLAS and LAPACK with AppleAccelerate.jl.","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"using LinearAlgebra\nBLAS.get_config() # BLAS.vendor() for Julia 1.6","category":"page"},{"location":"tips/#Multi-threaded-sparse-matrix-vector-products","page":"Performance tips","title":"Multi-threaded sparse matrix-vector products","text":"","category":"section"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"For sparse matrices, the Julia implementation of mul! of SparseArrays library is not parallelized. A significant speed-up can be observed with the multhreaded mul! of MKLSparse.jl or ThreadedSparseCSR.jl.","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"It's also possible to implement a generic multithreaded julia version. For instance, the following function can be used for symmetric matrices","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"using Base.Threads\n\nfunction threaded_mul!(y::Vector{T}, A::SparseMatrixCSC{T}, x::Vector{T}) where T <: Number\n A.m == A.n || error(\"A is not a square matrix!\")\n @threads for i = 1 : A.n\n tmp = zero(T)\n @inbounds for j = A.colptr[i] : (A.colptr[i+1] - 1)\n tmp += A.nzval[j] * x[A.rowval[j]]\n end\n @inbounds y[i] = tmp\n end\n return y\nend","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"and wrapped inside a linear operator to solve symmetric linear systems","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"using LinearOperators\n\nn, m = size(A)\nsym = herm = true\nT = eltype(A)\nopA = LinearOperator(T, n, m, sym, herm, (y, v) -> threaded_mul!(y, A, v))","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"To enable multi-threading with Julia, you can start julia with the environment variable JULIA_NUM_THREADS or the options -t and --threads","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"julia -t auto # alternative: --threads auto\njulia -t N # alternative: --threads N\n\nJULIA_NUM_THREADS=N julia","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"Thereafter, you can verify the number of threads usable by Julia","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"using Base.Threads\nnthreads()","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"The following benchmarks illustrate the time required in seconds to compute 1000 sparse matrix-vector products with symmetric matrices of the SuiteSparse Matrix Collection. The computer used for the benchmarks has 2 physical cores and Julia was launched with JULIA_NUM_THREADS=2.","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"(Image: benchmarks)","category":"page"},{"location":"solvers/gsp/","page":"Generalized saddle-point and non-Hermitian partitioned systems","title":"Generalized saddle-point and non-Hermitian partitioned systems","text":"# Generalized saddle-point and non-Hermitian partitioned systems","category":"page"},{"location":"solvers/gsp/#GPMR","page":"Generalized saddle-point and non-Hermitian partitioned systems","title":"GPMR","text":"","category":"section"},{"location":"solvers/gsp/","page":"Generalized saddle-point and non-Hermitian partitioned systems","title":"Generalized saddle-point and non-Hermitian partitioned systems","text":"gpmr\ngpmr!","category":"page"},{"location":"solvers/gsp/#Krylov.gpmr","page":"Generalized saddle-point and non-Hermitian partitioned systems","title":"Krylov.gpmr","text":"(x, y, stats) = gpmr(A, B, b::AbstractVector{FC}, c::AbstractVector{FC};\n memory::Int=20, C=I, D=I, E=I, F=I,\n ldiv::Bool=false, gsp::Bool=false,\n λ::FC=one(FC), μ::FC=one(FC),\n reorthogonalization::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, y, stats) = gpmr(A, B, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)\n\nGPMR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.\n\nGiven matrices A of dimension m × n and B of dimension n × m, GPMR solves the non-Hermitian partitioned linear system\n\n[ λIₘ A ] [ x ] = [ b ]\n[ B μIₙ ] [ y ] [ c ],\n\nof size (n+m) × (n+m) where λ and μ are real or complex numbers. A can have any shape and B has the shape of Aᴴ. A, B, b and c must be all nonzero.\n\nThis implementation allows left and right block diagonal preconditioners\n\n[ C ] [ λM A ] [ E ] [ E⁻¹x ] = [ Cb ]\n[ D ] [ B μN ] [ F ] [ F⁻¹y ] [ Dc ],\n\nand can solve\n\n[ λM A ] [ x ] = [ b ]\n[ B μN ] [ y ] [ c ]\n\nwhen CE = M⁻¹ and DF = N⁻¹.\n\nBy default, GPMR solves unsymmetric linear systems with λ = 1 and μ = 1.\n\nGPMR is based on the orthogonal Hessenberg reduction process and its relations with the block-Arnoldi process. The residual norm ‖rₖ‖ is monotonically decreasing in GPMR.\n\nGPMR stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a linear operator that models a matrix of dimension n × m;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional arguments\n\nx0: a vector of length m that represents an initial guess of the solution x;\ny0: a vector of length n that represents an initial guess of the solution y.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version GPMR(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nC: linear operator that models a nonsingular matrix of size m, and represents the first term of the block-diagonal left preconditioner;\nD: linear operator that models a nonsingular matrix of size n, and represents the second term of the block-diagonal left preconditioner;\nE: linear operator that models a nonsingular matrix of size m, and represents the first term of the block-diagonal right preconditioner;\nF: linear operator that models a nonsingular matrix of size n, and represents the second term of the block-diagonal right preconditioner;\nldiv: define whether the preconditioners use ldiv! or mul!;\ngsp: if true, set λ = 1 and μ = 0 for generalized saddle-point systems;\nλ and μ: diagonal scaling factors of the partitioned linear system;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length m;\ny: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nA. Montoison and D. Orban, GPMR: An Iterative Method for Unsymmetric Partitioned Linear Systems, SIAM Journal on Matrix Analysis and Applications, 44(1), pp. 293–311, 2023.\n\n\n\n\n\n","category":"function"},{"location":"solvers/gsp/#Krylov.gpmr!","page":"Generalized saddle-point and non-Hermitian partitioned systems","title":"Krylov.gpmr!","text":"solver = gpmr!(solver::GpmrSolver, A, B, b, c; kwargs...)\nsolver = gpmr!(solver::GpmrSolver, A, B, b, c, x0, y0; kwargs...)\n\nwhere kwargs are keyword arguments of gpmr.\n\nNote that the memory keyword argument is the only exception. It's required to create a GpmrSolver and can't be changed later.\n\nSee GpmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"examples/lsmr/","page":"LSMR","title":"LSMR","text":"using MatrixMarket, SuiteSparseMatrixCollection\nusing Krylov, LinearOperators\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"illc1850\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1])_b.mtx\"))[:]\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\n# Define a regularization parameter.\nλ = 1.0e-3\n\n(x, stats) = lsmr(A, b, λ=λ, atol=0.0, btol=0.0)\nshow(stats)\nresid = norm(A' * (A * x - b) + λ * x) / norm(b)\n@printf(\"LSMR: Relative residual: %8.1e\\n\", resid)\n@printf(\"LSMR: ‖x‖: %8.1e\\n\", norm(x))","category":"page"},{"location":"examples/cg_lanczos_shift/","page":"CG-LANCZOS-SHIFT","title":"CG-LANCZOS-SHIFT","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nfunction residuals(A, b, shifts, x)\n nshifts = length(shifts)\n r = [ (b - A * x[i] - shifts[i] * x[i]) for i = 1 : nshifts ]\n return r\nend\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"1138_bus\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nn, m = size(A)\nb = ones(n)\n\n# Solve (A + αI)x = b.\nshifts = [1.0, 2.0, 3.0, 4.0]\n(x, stats) = cg_lanczos_shift(A, b, shifts)\nshow(stats)\nr = residuals(A, b, shifts, x)\nresids = map(norm, r) / norm(b)\n@printf(\"Relative residuals with shifts:\\n\")\nfor resid in resids\n @printf(\" %8.1e\", resid)\nend\n@printf(\"\\n\")","category":"page"},{"location":"examples/minres_qlp/","page":"MINRES-QLP","title":"MINRES-QLP","text":"using Krylov\nusing LinearAlgebra, Printf\n\nA = diagm([1.0; 2.0; 3.0; 0.0])\nn = size(A, 1)\nb = [1.0; 2.0; 3.0; 4.0]\nb_norm = norm(b)\n\n# MINRES-QLP returns the minimum-norm solution of symmetric, singular and inconsistent systems\n(x, stats) = minres_qlp(A, b);\nr = b - A * x;\n\n@printf(\"Residual r: %s\\n\", Krylov.vec2str(r))\n@printf(\"Relative residual norm ‖r‖: %8.1e\\n\", norm(r) / b_norm)\n@printf(\"Solution x: %s\\n\", Krylov.vec2str(x))\n@printf(\"Minimum-norm solution? %s\\n\", x ≈ [1.0; 1.0; 1.0; 0.0])","category":"page"},{"location":"examples/lsqr/","page":"LSQR","title":"LSQR","text":"using MatrixMarket, SuiteSparseMatrixCollection\nusing Krylov, LinearOperators\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"illc1033\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1])_b.mtx\"))[:]\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\n# Define a regularization parameter.\nλ = 1.0e-3\n\n(x, stats) = lsqr(A, b, λ=λ, atol=0.0, btol=0.0)\nshow(stats)\nresid = norm(A' * (A * x - b) + λ * x) / norm(b)\n@printf(\"LSQR: Relative residual: %8.1e\\n\", resid)\n@printf(\"LSQR: ‖x‖: %8.1e\\n\", norm(x))","category":"page"},{"location":"inplace/#in-place","page":"In-place methods","title":"In-place methods","text":"","category":"section"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"All solvers in Krylov.jl have an in-place variant implemented in a method whose name ends with !. A workspace (KrylovSolver), which contains the storage needed by a Krylov method, can be used to solve multiple linear systems with the same dimensions and the same floating-point precision. The section storage requirements specifies the memory needed for each Krylov method. Each KrylovSolver has two constructors:","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"XyzSolver(A, b)\nXyzSolver(m, n, S)","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"Xyz is the name of the Krylov method with lowercase letters except its first one (Cg, Minres, Lsmr, Bicgstab, ...). Given an operator A and a right-hand side b, you can create a KrylovSolver based on the size of A and the type of b or explicitly give the dimensions (m, n) and the storage type S.","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"For example, use S = Vector{Float64} if you want to solve linear systems in double precision on the CPU and S = CuVector{Float32} if you want to solve linear systems in single precision on an Nvidia GPU.","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"note: Note\nDiomSolver, FomSolver, DqgmresSolver, GmresSolver, BlockGmresSolver, FgmresSolver, GpmrSolver and CgLanczosShiftSolver require an additional argument (memory or nshifts).","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"The workspace is always the first argument of the in-place methods:","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"minres_solver = MinresSolver(n, n, Vector{Float64})\nminres!(minres_solver, A1, b1)\n\ndqgmres_solver = DqgmresSolver(n, n, memory, Vector{BigFloat})\ndqgmres!(dqgmres_solver, A2, b2)\n\nlsqr_solver = LsqrSolver(m, n, CuVector{Float32})\nlsqr!(lsqr_solver, A3, b3)","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"A generic function solve! is also available and dispatches to the appropriate Krylov method.","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"Krylov.solve!","category":"page"},{"location":"inplace/#Krylov.solve!","page":"In-place methods","title":"Krylov.solve!","text":"solve!(solver, args...; kwargs...)\n\nUse the in-place Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"In-place methods return an updated solver workspace. Solutions and statistics can be recovered via solver.x, solver.y and solver.stats. Functions solution and statistics can be also used.","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"Krylov.nsolution\nKrylov.solution\nKrylov.statistics\nKrylov.issolved","category":"page"},{"location":"inplace/#Krylov.nsolution","page":"In-place methods","title":"Krylov.nsolution","text":"nsolution(solver)\n\nReturn the number of outputs of solution(solver).\n\n\n\n\n\n","category":"function"},{"location":"inplace/#Krylov.solution","page":"In-place methods","title":"Krylov.solution","text":"solution(solver)\n\nReturn the solution(s) stored in the solver. Optionally you can specify which solution you want to recover, solution(solver, 1) returns x and solution(solver, 2) returns y.\n\n\n\n\n\n","category":"function"},{"location":"inplace/#Krylov.statistics","page":"In-place methods","title":"Krylov.statistics","text":"statistics(solver)\n\nReturn the statistics stored in the solver.\n\n\n\n\n\n","category":"function"},{"location":"inplace/#Krylov.issolved","page":"In-place methods","title":"Krylov.issolved","text":"issolved(solver)\n\nReturn a boolean that determines whether the Krylov method associated to solver succeeded.\n\n\n\n\n\n","category":"function"},{"location":"inplace/#Examples","page":"In-place methods","title":"Examples","text":"","category":"section"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"We illustrate the use of in-place Krylov solvers with two well-known optimization methods. The details of the optimization methods are described in the section about Factorization-free operators.","category":"page"},{"location":"inplace/#Example-1:-Newton's-method-for-convex-optimization-without-linesearch","page":"In-place methods","title":"Example 1: Newton's method for convex optimization without linesearch","text":"","category":"section"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"using Krylov\n\nfunction newton(∇f, ∇²f, x₀; itmax = 200, tol = 1e-8)\n\n n = length(x₀)\n x = copy(x₀)\n gx = ∇f(x)\n \n iter = 0\n S = typeof(x)\n solver = CgSolver(n, n, S)\n Δx = solver.x\n\n solved = false\n tired = false\n\n while !(solved || tired)\n \n Hx = ∇²f(x) # Compute ∇²f(xₖ)\n cg!(solver, Hx, -gx) # Solve ∇²f(xₖ)Δx = -∇f(xₖ)\n x = x + Δx # Update xₖ₊₁ = xₖ + Δx\n gx = ∇f(x) # ∇f(xₖ₊₁)\n \n iter += 1\n solved = norm(gx) ≤ tol\n tired = iter ≥ itmax\n end\n return x\nend","category":"page"},{"location":"inplace/#Example-2:-The-Gauss-Newton-method-for-nonlinear-least-squares-without-linesearch","page":"In-place methods","title":"Example 2: The Gauss-Newton method for nonlinear least squares without linesearch","text":"","category":"section"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"using Krylov\n\nfunction gauss_newton(F, JF, x₀; itmax = 200, tol = 1e-8)\n\n n = length(x₀)\n x = copy(x₀)\n Fx = F(x)\n m = length(Fx)\n \n iter = 0\n S = typeof(x)\n solver = LsmrSolver(m, n, S)\n Δx = solver.x\n\n solved = false\n tired = false\n\n while !(solved || tired)\n \n Jx = JF(x) # Compute J(xₖ)\n lsmr!(solver, Jx, -Fx) # Minimize ‖J(xₖ)Δx + F(xₖ)‖\n x = x + Δx # Update xₖ₊₁ = xₖ + Δx\n Fx_old = Fx # F(xₖ)\n Fx = F(x) # F(xₖ₊₁)\n \n iter += 1\n solved = norm(Fx - Fx_old) / norm(Fx) ≤ tol\n tired = iter ≥ itmax\n end\n return x\nend","category":"page"},{"location":"warm-start/#warm-start","page":"Warm-start","title":"Warm-start","text":"","category":"section"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"Most Krylov methods in this module accept a starting point as argument. The starting point is used as initial approximation to a solution.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"solver = CgSolver(A, b)\ncg!(solver, A, b, itmax=100)\nif !issolved(solver)\n cg!(solver, A, b, solver.x, itmax=100) # cg! uses the approximate solution `solver.x` as starting point\nend","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"If the user has an initial guess x0, it can be provided directly.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"cg(A, b, x0)","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"It is also possible to use the warm_start! function to feed the starting point into the solver.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"warm_start!(solver, x0)\ncg!(solver, A, b)\n# the previous two lines are equivalent to cg!(solver, A, b, x0)","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"If a Krylov method doesn't have the option to warm start, it can still be done explicitly. We provide an example with cg_lanczos!.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"solver = CgLanczosSolver(A, b)\ncg_lanczos!(solver, A, b)\nx₀ = solver.x # Ax₀ ≈ b\nr = b - A * x₀ # r = b - Ax₀\ncg_lanczos!(solver, A, r)\nΔx = solver.x # AΔx = r\nx = x₀ + Δx # Ax = b","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"Explicit restarts cannot be avoided in certain block methods, such as TriMR, due to the preconditioners.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"# [E A] [x] = [b]\n# [Aᴴ F] [y] [c]\nM = inv(E)\nN = inv(F)\nx₀, y₀, stats = trimr(A, b, c, M=M, N=N)\n\n# E and F are not available inside TriMR\nb₀ = b - Ex₀ - Ay\nc₀ = c - Aᴴx₀ - Fy\n\nΔx, Δy, stats = trimr(A, b₀, c₀, M=M, N=N)\nx = x₀ + Δx\ny = y₀ + Δy","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"# ## Restarted methods\n#\n# The storage requirements of Krylov methods based on the Arnoldi process, such as FOM and GMRES, increase as the iteration progresses.\n# For very large problems, the storage costs become prohibitive after only few iterations and restarted variants FOM(k) and GMRES(k) are preferred.\n# In this section, we show how to use warm starts to implement GMRES(k) and FOM(k).\n#\n# ```julia\n# k = 50\n# solver = GmresSolver(A, b, k) # FomSolver(A, b, k)\n# solver.x .= 0 # solver.x .= x₀ \n# nrestart = 0\n# while !issolved(solver) || nrestart ≤ 10\n# solve!(solver, A, b, solver.x, itmax=k)\n# nrestart += 1\n# end\n# ```","category":"page"},{"location":"block_krylov/#Block-GMRES","page":"Block-Krylov methods","title":"Block-GMRES","text":"","category":"section"},{"location":"block_krylov/","page":"Block-Krylov methods","title":"Block-Krylov methods","text":"note: Note\nblock_gmres works on GPUs with Julia 1.11.","category":"page"},{"location":"block_krylov/","page":"Block-Krylov methods","title":"Block-Krylov methods","text":"If you want to use block_gmres on previous Julia versions, you can overload the function Krylov.copy_triangle with the following code:","category":"page"},{"location":"block_krylov/","page":"Block-Krylov methods","title":"Block-Krylov methods","text":"using KernelAbstractions, Krylov\n\n@kernel function copy_triangle_kernel!(dest, src)\n i, j = @index(Global, NTuple)\n if j >= i\n @inbounds dest[i, j] = src[i, j]\n end\nend\n\nfunction Krylov.copy_triangle(Q::AbstractMatrix{FC}, R::AbstractMatrix{FC}, k::Int) where FC <: Krylov.FloatOrComplex\n backend = get_backend(Q)\n ndrange = (k, k)\n copy_triangle_kernel!(backend)(R, Q; ndrange=ndrange)\n KernelAbstractions.synchronize(backend)\nend","category":"page"},{"location":"block_krylov/","page":"Block-Krylov methods","title":"Block-Krylov methods","text":"block_gmres\nblock_gmres!","category":"page"},{"location":"block_krylov/#Krylov.block_gmres","page":"Block-Krylov methods","title":"Krylov.block_gmres","text":"(X, stats) = block_gmres(A, b::AbstractMatrix{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(X, stats) = block_gmres(A, B, X0::AbstractMatrix; kwargs...)\n\nBlock-GMRES can be warm-started from an initial guess X0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system AX = B of size n with p right-hand sides using block-GMRES.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nB: a matrix of size n × p.\n\nOptional argument\n\nX0: a matrix of size n × p that represents an initial guess of the solution X.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version block-GMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new matrices of the block-Krylov basis against all previous matrix;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2 * div(n,p);\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms;\ncallback: function or functor called as callback(solver) that returns true if the block-Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nX: a dense matrix of size n × p;\nstats: statistics collected on the run in a SimpleStats structure.\n\n\n\n\n\n","category":"function"},{"location":"block_krylov/#Krylov.block_gmres!","page":"Block-Krylov methods","title":"Krylov.block_gmres!","text":"solver = block_gmres!(solver::BlockGmresSolver, B; kwargs...)\nsolver = block_gmres!(solver::BlockGmresSolver, B, X0; kwargs...)\n\nwhere kwargs are keyword arguments of block_gmres.\n\nSee BlockGmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"block_processes/#block-krylov-processes","page":"Block Krylov processes","title":"Block Krylov processes","text":"","category":"section"},{"location":"block_processes/#block-hermitian-lanczos","page":"Block Krylov processes","title":"Block Hermitian Lanczos","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vector b in the Hermitian Lanczos process is replaced by a matrix B with p columns, we can derive the block Hermitian Lanczos process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_hermitian_lanczos)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Hermitian Lanczos process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A V_k = V_k+1 T_k+1k \n V_k^H V_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where V_k is an orthonormal basis of the block Krylov subspace mathcalK_k^square(AB),","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"T_k+1k =\nbeginbmatrix\n Omega_1 Psi_2^H \n Psi_2 Omega_2 ddots \n ddots ddots Psi_k^H \n Psi_k Omega_k \n Psi_k+1\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function hermitian_lanczos returns V_k+1, Psi_1 and T_k+1k.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"hermitian_lanczos(::Any, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.hermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.hermitian_lanczos","text":"V, Ψ, T = hermitian_lanczos(A, B, k; algo=\"householder\")\n\nInput arguments\n\nA: a linear operator that models an Hermitian matrix of dimension n;\nB: a matrix of size n × p;\nk: the number of iterations of the block Hermitian Lanczos process.\n\nKeyword arguments\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\n\nOutput arguments\n\nV: a dense n × p(k+1) matrix;\nΨ: a dense p × p upper triangular matrix such that V₁Ψ = B;\nT: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-nonhermitian-lanczos","page":"Block Krylov processes","title":"Block Non-Hermitian Lanczos","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vectors b and c in the non-Hermitian Lanczos process are replaced by matrices B and C with both p columns, we can derive the block non-Hermitian Lanczos process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_nonhermitian_lanczos)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block non-Hermitian Lanczos process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A V_k = V_k+1 T_k+1k \n A^H U_k = U_k+1 T_kk+1^H \n V_k^H U_k = U_k^H V_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where V_k and U_k are bases of the block Krylov subspaces mathcalK^square_k(AB) and mathcalK^square_k (A^HC), respectively,","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"T_k+1k =\nbeginbmatrix\n Omega_1 Phi_2 \n Psi_2 Omega_2 ddots \n ddots ddots Phi_k \n Psi_k Omega_k \n Psi_k+1\nendbmatrix\n qquad\nT_kk+1^H =\nbeginbmatrix\n Omega_1^H Psi_2^H \n Phi_2^H Omega_2^H ddots \n ddots ddots Psi_k^H \n Phi_k^H Omega_k^H \n Phi_k+1^H\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function nonhermitian_lanczos returns V_k+1, Psi_1, T_k+1k, U_k+1 Phi_1^H and T_kk+1^H.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"nonhermitian_lanczos(::Any, ::AbstractMatrix{FC}, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.nonhermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.nonhermitian_lanczos","text":"V, Ψ, T, U, Φᴴ, Tᴴ = nonhermitian_lanczos(A, B, C, k)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nB: a matrix of size n × p;\nC: a matrix of size n × p;\nk: the number of iterations of the block non-Hermitian Lanczos process.\n\nOutput arguments\n\nV: a dense n × p(k+1) matrix;\nΨ: a dense p × p upper triangular matrix such that V₁Ψ = B;\nT: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p;\nU: a dense n × p(k+1) matrix;\nΦᴴ: a dense p × p upper triangular matrix such that U₁Φᴴ = C;\nTᴴ: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-arnoldi","page":"Block Krylov processes","title":"Block Arnoldi","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vector b in the Arnoldi process is replaced by a matrix B with p columns, we can derive the block Arnoldi process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_arnoldi)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Arnoldi process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A V_k = V_k+1 H_k+1k \n V_k^H V_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where V_k is an orthonormal basis of the block Krylov subspace mathcalK_k^square(AB),","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"H_k+1k =\nbeginbmatrix\n Psi_11 Psi_12 ldots Psi_1k \n Psi_21 ddots ddots vdots \n ddots ddots Psi_k-1k \n Psi_kk-1 Psi_kk \n Psi_k+1k\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function arnoldi returns V_k+1, Gamma, and H_k+1k.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"Related method: BLOCK-GMRES.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"arnoldi(::Any, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.arnoldi-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.arnoldi","text":"V, Γ, H = arnoldi(A, B, k; algo=\"householder\", reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nB: a matrix of size n × p;\nk: the number of iterations of the block Arnoldi process.\n\nKeyword arguments\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\nreorthogonalization: reorthogonalize the new matrices of the block Krylov basis against all previous matrices.\n\nOutput arguments\n\nV: a dense n × p(k+1) matrix;\nΓ: a dense p × p upper triangular matrix such that V₁Γ = B;\nH: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-golub-kahan","page":"Block Krylov processes","title":"Block Golub-Kahan","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vector b in the Golub-Kahan process is replaced by a matrix B with p columns, we can derive the block Golub-Kahan process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_golub_kahan)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Golub-Kahan process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A V_k = U_k+1 B_k \n A^H U_k+1 = V_k+1 L_k+1^H \n V_k^H V_k = U_k^H U_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where V_k and U_k are bases of the block Krylov subspaces mathcalK_k^square(A^HAA^HB) and mathcalK_k^square(AA^HB), respectively,","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"B_k =\nbeginbmatrix\n Omega_1 \n Psi_2 Omega_2 \n ddots ddots \n Psi_k Omega_k \n Psi_k+1 \nendbmatrix\n qquad\nL_k+1^H =\nbeginbmatrix\n Omega_1^H Psi_2^H \n Omega_2^H ddots \n ddots Psi_k^H \n Omega_k^H Psi_k+1^H \n Omega_k+1^H \nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function golub_kahan returns V_k+1, U_k+1, Psi_1 and L_k+1.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"golub_kahan(::Any, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.golub_kahan-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.golub_kahan","text":"V, U, Ψ, L = golub_kahan(A, B, k; algo=\"householder\")\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a matrix of size m × p;\nk: the number of iterations of the block Golub-Kahan process.\n\nKeyword argument\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\n\nOutput arguments\n\nV: a dense n × p(k+1) matrix;\nU: a dense m × p(k+1) matrix;\nΨ: a dense p × p upper triangular matrix such that U₁Ψ = B;\nL: a sparse p(k+1) × p(k+1) block lower bidiagonal matrix with a lower bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-saunders-simon-yip","page":"Block Krylov processes","title":"Block Saunders-Simon-Yip","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vectors b and c in the Saunders-Simon-Yip process are replaced by matrices B and C with both p columns, we can derive the block Saunders-Simon-Yip process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_saunders_simon_yip)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Saunders-Simon-Yip process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A U_k = V_k+1 T_k+1k \n A^H V_k = U_k+1 T_kk+1^H \n V_k^H V_k = U_k^H U_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A A^H 0 endbmatrix beginbmatrix B 0 0 C endbmatrixright),","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"T_k+1k =\nbeginbmatrix\n Omega_1 Phi_2 \n Psi_2 Omega_2 ddots \n ddots ddots Phi_k \n Psi_k Omega_k \n Psi_k+1\nendbmatrix\n qquad\nT_kk+1^H =\nbeginbmatrix\n Omega_1^H Psi_2^H \n Phi_2^H Omega_2^H ddots \n ddots ddots Psi_k^H \n Phi_k^H Omega_k^H \n Phi_k+1^H\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function saunders_simon_yip returns V_k+1, Psi_1, T_k+1k, U_k+1, Phi_1^H and T_kk+1^H.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"saunders_simon_yip(::Any, ::AbstractMatrix{FC}, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.saunders_simon_yip-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.saunders_simon_yip","text":"V, Ψ, T, U, Φᴴ, Tᴴ = saunders_simon_yip(A, B, C, k; algo=\"householder\")\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a matrix of size m × p;\nC: a matrix of size n × p;\nk: the number of iterations of the block Saunders-Simon-Yip process.\n\nKeyword argument\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\n\nOutput arguments\n\nV: a dense m × p(k+1) matrix;\nΨ: a dense p × p upper triangular matrix such that V₁Ψ = B;\nT: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p;\nU: a dense n × p(k+1) matrix;\nΦᴴ: a dense p × p upper triangular matrix such that U₁Φᴴ = C;\nTᴴ: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-montoison-orban","page":"Block Krylov processes","title":"Block Montoison-Orban","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vectors b and c in the Montoison-Orban process are replaced by matrices D and C with both p columns, we can derive the block Montoison-Orban process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_montoison_orban)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Montoison-Orban process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A U_k = V_k+1 H_k+1k \n B V_k = U_k+1 F_k+1k \n V_k^H V_k = U_k^H U_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A B 0 endbmatrix beginbmatrix D 0 0 C endbmatrixright),","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"H_k+1k =\nbeginbmatrix\n Psi_11 Psi_12 ldots Psi_1k \n Psi_21 ddots ddots vdots \n ddots ddots Psi_k-1k \n Psi_kk-1 Psi_kk \n Psi_k+1k\nendbmatrix\n qquad\nF_k+1k =\nbeginbmatrix\n Phi_11 Phi_12 ldots Phi_1k \n Phi_21 ddots ddots vdots \n ddots ddots Phi_k-1k \n Phi_kk-1 Phi_kk \n Phi_k+1k\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function montoison_orban returns V_k+1, Gamma, H_k+1k, U_k+1, Lambda, and F_k+1k.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"montoison_orban(::Any, ::Any, ::AbstractMatrix{FC}, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.montoison_orban-Union{Tuple{FC}, Tuple{Any, Any, AbstractMatrix{FC}, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.montoison_orban","text":"V, Γ, H, U, Λ, F = montoison_orban(A, B, D, C, k; algo=\"householder\", reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a linear operator that models a matrix of dimension n × m;\nD: a matrix of size m × p;\nC: a matrix of size n × p;\nk: the number of iterations of the block Montoison-Orban process.\n\nKeyword arguments\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\nreorthogonalization: reorthogonalize the new matrices of the block Krylov basis against all previous matrices.\n\nOutput arguments\n\nV: a dense m × p(k+1) matrix;\nΓ: a dense p × p upper triangular matrix such that V₁Γ = D;\nH: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p;\nU: a dense n × p(k+1) matrix;\nΛ: a dense p × p upper triangular matrix such that U₁Λ = C;\nF: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"examples/tricg/","page":"TriCG","title":"TriCG","text":"using Krylov, LinearOperators\nusing LinearAlgebra, Printf, SparseArrays\n\n# Identity matrix.\neye(n::Int) = sparse(1.0 * I, n, n)\n\n# Symmetric quasi-definite systems and variants\nn = m = 5\nA = [2^(i/j)*j + (-1)^(i-j) * n*(i-1) for i = 1:n, j = 1:n]\nb = ones(n)\nM = diagm(0 => [3.0 * i for i = 1:n])\nN = diagm(0 => [5.0 * i for i = 1:n])\nc = -b\n\n# [I A] [x] = [b]\n# [Aᴴ -I] [y] [c]\n(x, y, stats) = tricg(A, b, c)\nK = [eye(m) A; A' -eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [-I A] [x] = [b]\n# [ Aᴴ I] [y] [c]\n(x, y, stats) = tricg(A, b, c, flip=true)\nK = [-eye(m) A; A' eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [I A] [x] = [b]\n# [Aᴴ I] [y] [c]\n(x, y, stats) = tricg(A, b, c, spd=true)\nK = [eye(m) A; A' eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [-I A] [x] = [b]\n# [ Aᴴ -I] [y] [c]\n(x, y, stats) = tricg(A, b, c, snd=true)\nK = [-eye(m) A; A' -eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [τI A] [x] = [b]\n# [ Aᴴ νI] [y] [c]\n(τ, ν) = (1e-4, 1e2)\n(x, y, stats) = tricg(A, b, c, τ=τ, ν=ν)\nK = [τ*eye(m) A; A' ν*eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [M⁻¹ A ] [x] = [b]\n# [Aᴴ -N⁻¹] [y] [c]\n(x, y, stats) = tricg(A, b, c, M=M, N=N, verbose=1)\nK = [inv(M) A; A' -inv(N)]\nH = BlockDiagonalOperator(M, N)\nB = [b; c]\nr = B - K * [x; y]\nresid = sqrt(dot(r, H * r))\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)","category":"page"},{"location":"solvers/as/","page":"Adjoint systems","title":"Adjoint systems","text":"# Adjoint systems","category":"page"},{"location":"solvers/as/#BiLQR","page":"Adjoint systems","title":"BiLQR","text":"","category":"section"},{"location":"solvers/as/","page":"Adjoint systems","title":"Adjoint systems","text":"bilqr\nbilqr!","category":"page"},{"location":"solvers/as/#Krylov.bilqr","page":"Adjoint systems","title":"Krylov.bilqr","text":"(x, y, stats) = bilqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n transfer_to_bicg::Bool=true, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, y, stats) = bilqr(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)\n\nBiLQR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.\n\nCombine BiLQ and QMR to solve adjoint systems.\n\n[0 A] [y] = [b]\n[Aᴴ 0] [x] [c]\n\nThe relation bᴴc ≠ 0 must be satisfied. BiLQ is used for solving primal system Ax = b of size n. QMR is used for solving dual system Aᴴy = c of size n.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n;\nc: a vector of length n.\n\nOptional arguments\n\nx0: a vector of length n that represents an initial guess of the solution x;\ny0: a vector of length n that represents an initial guess of the solution y.\n\nKeyword arguments\n\ntransfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length n;\nstats: statistics collected on the run in an AdjointStats structure.\n\nReference\n\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/as/#Krylov.bilqr!","page":"Adjoint systems","title":"Krylov.bilqr!","text":"solver = bilqr!(solver::BilqrSolver, A, b, c; kwargs...)\nsolver = bilqr!(solver::BilqrSolver, A, b, c, x0, y0; kwargs...)\n\nwhere kwargs are keyword arguments of bilqr.\n\nSee BilqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/as/#TriLQR","page":"Adjoint systems","title":"TriLQR","text":"","category":"section"},{"location":"solvers/as/","page":"Adjoint systems","title":"Adjoint systems","text":"trilqr\ntrilqr!","category":"page"},{"location":"solvers/as/#Krylov.trilqr","page":"Adjoint systems","title":"Krylov.trilqr","text":"(x, y, stats) = trilqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n transfer_to_usymcg::Bool=true, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, y, stats) = trilqr(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)\n\nTriLQR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.\n\nCombine USYMLQ and USYMQR to solve adjoint systems.\n\n[0 A] [y] = [b]\n[Aᴴ 0] [x] [c]\n\nUSYMLQ is used for solving primal system Ax = b of size m × n. USYMQR is used for solving dual system Aᴴy = c of size n × m.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional arguments\n\nx0: a vector of length n that represents an initial guess of the solution x;\ny0: a vector of length m that represents an initial guess of the solution y.\n\nKeyword arguments\n\ntransfer_to_usymcg: transfer from the USYMLQ point to the USYMCG point, when it exists. The transfer is based on the residual norm;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in an AdjointStats structure.\n\nReference\n\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/as/#Krylov.trilqr!","page":"Adjoint systems","title":"Krylov.trilqr!","text":"solver = trilqr!(solver::TrilqrSolver, A, b, c; kwargs...)\nsolver = trilqr!(solver::TrilqrSolver, A, b, c, x0, y0; kwargs...)\n\nwhere kwargs are keyword arguments of trilqr.\n\nSee TrilqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"api/#Stats-Types","page":"API","title":"Stats Types","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Krylov.KrylovStats\nKrylov.SimpleStats\nKrylov.LanczosStats\nKrylov.LanczosShiftStats\nKrylov.SymmlqStats\nKrylov.AdjointStats\nKrylov.LNLQStats\nKrylov.LSLQStats\nKrylov.LsmrStats","category":"page"},{"location":"api/#Krylov.KrylovStats","page":"API","title":"Krylov.KrylovStats","text":"Abstract type for statistics returned by a solver\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.SimpleStats","page":"API","title":"Krylov.SimpleStats","text":"Type for statistics returned by the majority of Krylov solvers, the attributes are:\n\nniter\nsolved\ninconsistent\nresiduals\nAresiduals\nAcond\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LanczosStats","page":"API","title":"Krylov.LanczosStats","text":"Type for statistics returned by CG-LANCZOS, the attributes are:\n\nniter\nsolved\nresiduals\nindefinite\nAnorm\nAcond\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LanczosShiftStats","page":"API","title":"Krylov.LanczosShiftStats","text":"Type for statistics returned by CG-LANCZOS with shifts, the attributes are:\n\nniter\nsolved\nresiduals\nindefinite\nAnorm\nAcond\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.SymmlqStats","page":"API","title":"Krylov.SymmlqStats","text":"Type for statistics returned by SYMMLQ, the attributes are:\n\nniter\nsolved\nresiduals\nresidualscg\nerrors\nerrorscg\nAnorm\nAcond\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.AdjointStats","page":"API","title":"Krylov.AdjointStats","text":"Type for statistics returned by adjoint systems solvers BiLQR and TriLQR, the attributes are:\n\nniter\nsolved_primal\nsolved_dual\nresiduals_primal\nresiduals_dual\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LNLQStats","page":"API","title":"Krylov.LNLQStats","text":"Type for statistics returned by the LNLQ method, the attributes are:\n\nniter\nsolved\nresiduals\nerrorwithbnd\nerrorbndx\nerrorbndy\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LSLQStats","page":"API","title":"Krylov.LSLQStats","text":"Type for statistics returned by the LSLQ method, the attributes are:\n\nniter\nsolved\ninconsistent\nresiduals\nAresiduals\nerr_lbnds\nerrorwithbnd\nerrubndslq\nerrubndscg\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LsmrStats","page":"API","title":"Krylov.LsmrStats","text":"Type for statistics returned by LSMR. The attributes are:\n\nniter\nsolved\ninconsistent\nresiduals\nAresiduals\nAcond\nAnorm\nxNorm\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Solver-Types","page":"API","title":"Solver Types","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"KrylovSolver\nBlockKrylovSolver\nMinresSolver\nMinaresSolver\nCgSolver\nCrSolver\nCarSolver\nSymmlqSolver\nCgLanczosSolver\nCgLanczosShiftSolver\nMinresQlpSolver\nDiomSolver\nFomSolver\nDqgmresSolver\nGmresSolver\nUsymlqSolver\nUsymqrSolver\nTricgSolver\nTrimrSolver\nTrilqrSolver\nCgsSolver\nBicgstabSolver\nBilqSolver\nQmrSolver\nBilqrSolver\nCglsSolver\nCrlsSolver\nCgneSolver\nCrmrSolver\nLslqSolver\nLsqrSolver\nLsmrSolver\nLnlqSolver\nCraigSolver\nCraigmrSolver\nGpmrSolver\nFgmresSolver\nBlockGmresSolver","category":"page"},{"location":"api/#Krylov.KrylovSolver","page":"API","title":"Krylov.KrylovSolver","text":"Abstract type for using Krylov solvers in-place\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BlockKrylovSolver","page":"API","title":"Krylov.BlockKrylovSolver","text":"Abstract type for using block Krylov solvers in-place\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.MinresSolver","page":"API","title":"Krylov.MinresSolver","text":"Type for storing the vectors required by the in-place version of MINRES.\n\nThe outer constructors\n\nsolver = MinresSolver(m, n, S; window :: Int=5)\nsolver = MinresSolver(A, b; window :: Int=5)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.MinaresSolver","page":"API","title":"Krylov.MinaresSolver","text":"Type for storing the vectors required by the in-place version of MINARES.\n\nThe outer constructors\n\nsolver = MinaresSolver(m, n, S)\nsolver = MinaresSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgSolver","page":"API","title":"Krylov.CgSolver","text":"Type for storing the vectors required by the in-place version of CG.\n\nThe outer constructors\n\nsolver = CgSolver(m, n, S)\nsolver = CgSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CrSolver","page":"API","title":"Krylov.CrSolver","text":"Type for storing the vectors required by the in-place version of CR.\n\nThe outer constructors\n\nsolver = CrSolver(m, n, S)\nsolver = CrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CarSolver","page":"API","title":"Krylov.CarSolver","text":"Type for storing the vectors required by the in-place version of CAR.\n\nThe outer constructors\n\nsolver = CarSolver(m, n, S)\nsolver = CarSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.SymmlqSolver","page":"API","title":"Krylov.SymmlqSolver","text":"Type for storing the vectors required by the in-place version of SYMMLQ.\n\nThe outer constructors\n\nsolver = SymmlqSolver(m, n, S)\nsolver = SymmlqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgLanczosSolver","page":"API","title":"Krylov.CgLanczosSolver","text":"Type for storing the vectors required by the in-place version of CG-LANCZOS.\n\nThe outer constructors\n\nsolver = CgLanczosSolver(m, n, S)\nsolver = CgLanczosSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgLanczosShiftSolver","page":"API","title":"Krylov.CgLanczosShiftSolver","text":"Type for storing the vectors required by the in-place version of CG-LANCZOS-SHIFT.\n\nThe outer constructors\n\nsolver = CgLanczosShiftSolver(m, n, nshifts, S)\nsolver = CgLanczosShiftSolver(A, b, nshifts)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.MinresQlpSolver","page":"API","title":"Krylov.MinresQlpSolver","text":"Type for storing the vectors required by the in-place version of MINRES-QLP.\n\nThe outer constructors\n\nsolver = MinresQlpSolver(m, n, S)\nsolver = MinresQlpSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.DiomSolver","page":"API","title":"Krylov.DiomSolver","text":"Type for storing the vectors required by the in-place version of DIOM.\n\nThe outer constructors\n\nsolver = DiomSolver(m, n, memory, S)\nsolver = DiomSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.FomSolver","page":"API","title":"Krylov.FomSolver","text":"Type for storing the vectors required by the in-place version of FOM.\n\nThe outer constructors\n\nsolver = FomSolver(m, n, memory, S)\nsolver = FomSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.DqgmresSolver","page":"API","title":"Krylov.DqgmresSolver","text":"Type for storing the vectors required by the in-place version of DQGMRES.\n\nThe outer constructors\n\nsolver = DqgmresSolver(m, n, memory, S)\nsolver = DqgmresSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.GmresSolver","page":"API","title":"Krylov.GmresSolver","text":"Type for storing the vectors required by the in-place version of GMRES.\n\nThe outer constructors\n\nsolver = GmresSolver(m, n, memory, S)\nsolver = GmresSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.UsymlqSolver","page":"API","title":"Krylov.UsymlqSolver","text":"Type for storing the vectors required by the in-place version of USYMLQ.\n\nThe outer constructors\n\nsolver = UsymlqSolver(m, n, S)\nsolver = UsymlqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.UsymqrSolver","page":"API","title":"Krylov.UsymqrSolver","text":"Type for storing the vectors required by the in-place version of USYMQR.\n\nThe outer constructors\n\nsolver = UsymqrSolver(m, n, S)\nsolver = UsymqrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.TricgSolver","page":"API","title":"Krylov.TricgSolver","text":"Type for storing the vectors required by the in-place version of TRICG.\n\nThe outer constructors\n\nsolver = TricgSolver(m, n, S)\nsolver = TricgSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.TrimrSolver","page":"API","title":"Krylov.TrimrSolver","text":"Type for storing the vectors required by the in-place version of TRIMR.\n\nThe outer constructors\n\nsolver = TrimrSolver(m, n, S)\nsolver = TrimrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.TrilqrSolver","page":"API","title":"Krylov.TrilqrSolver","text":"Type for storing the vectors required by the in-place version of TRILQR.\n\nThe outer constructors\n\nsolver = TrilqrSolver(m, n, S)\nsolver = TrilqrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgsSolver","page":"API","title":"Krylov.CgsSolver","text":"Type for storing the vectors required by the in-place version of CGS.\n\nThe outer constructorss\n\nsolver = CgsSolver(m, n, S)\nsolver = CgsSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BicgstabSolver","page":"API","title":"Krylov.BicgstabSolver","text":"Type for storing the vectors required by the in-place version of BICGSTAB.\n\nThe outer constructors\n\nsolver = BicgstabSolver(m, n, S)\nsolver = BicgstabSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BilqSolver","page":"API","title":"Krylov.BilqSolver","text":"Type for storing the vectors required by the in-place version of BILQ.\n\nThe outer constructors\n\nsolver = BilqSolver(m, n, S)\nsolver = BilqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.QmrSolver","page":"API","title":"Krylov.QmrSolver","text":"Type for storing the vectors required by the in-place version of QMR.\n\nThe outer constructors\n\nsolver = QmrSolver(m, n, S)\nsolver = QmrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BilqrSolver","page":"API","title":"Krylov.BilqrSolver","text":"Type for storing the vectors required by the in-place version of BILQR.\n\nThe outer constructors\n\nsolver = BilqrSolver(m, n, S)\nsolver = BilqrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CglsSolver","page":"API","title":"Krylov.CglsSolver","text":"Type for storing the vectors required by the in-place version of CGLS.\n\nThe outer constructors\n\nsolver = CglsSolver(m, n, S)\nsolver = CglsSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CrlsSolver","page":"API","title":"Krylov.CrlsSolver","text":"Type for storing the vectors required by the in-place version of CRLS.\n\nThe outer constructors\n\nsolver = CrlsSolver(m, n, S)\nsolver = CrlsSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgneSolver","page":"API","title":"Krylov.CgneSolver","text":"Type for storing the vectors required by the in-place version of CGNE.\n\nThe outer constructors\n\nsolver = CgneSolver(m, n, S)\nsolver = CgneSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CrmrSolver","page":"API","title":"Krylov.CrmrSolver","text":"Type for storing the vectors required by the in-place version of CRMR.\n\nThe outer constructors\n\nsolver = CrmrSolver(m, n, S)\nsolver = CrmrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LslqSolver","page":"API","title":"Krylov.LslqSolver","text":"Type for storing the vectors required by the in-place version of LSLQ.\n\nThe outer constructors\n\nsolver = LslqSolver(m, n, S)\nsolver = LslqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LsqrSolver","page":"API","title":"Krylov.LsqrSolver","text":"Type for storing the vectors required by the in-place version of LSQR.\n\nThe outer constructors\n\nsolver = LsqrSolver(m, n, S)\nsolver = LsqrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LsmrSolver","page":"API","title":"Krylov.LsmrSolver","text":"Type for storing the vectors required by the in-place version of LSMR.\n\nThe outer constructors\n\nsolver = LsmrSolver(m, n, S)\nsolver = LsmrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LnlqSolver","page":"API","title":"Krylov.LnlqSolver","text":"Type for storing the vectors required by the in-place version of LNLQ.\n\nThe outer constructors\n\nsolver = LnlqSolver(m, n, S)\nsolver = LnlqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CraigSolver","page":"API","title":"Krylov.CraigSolver","text":"Type for storing the vectors required by the in-place version of CRAIG.\n\nThe outer constructors\n\nsolver = CraigSolver(m, n, S)\nsolver = CraigSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CraigmrSolver","page":"API","title":"Krylov.CraigmrSolver","text":"Type for storing the vectors required by the in-place version of CRAIGMR.\n\nThe outer constructors\n\nsolver = CraigmrSolver(m, n, S)\nsolver = CraigmrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.GpmrSolver","page":"API","title":"Krylov.GpmrSolver","text":"Type for storing the vectors required by the in-place version of GPMR.\n\nThe outer constructors\n\nsolver = GpmrSolver(m, n, memory, S)\nsolver = GpmrSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n + m if the value given is larger than n + m.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.FgmresSolver","page":"API","title":"Krylov.FgmresSolver","text":"Type for storing the vectors required by the in-place version of FGMRES.\n\nThe outer constructors\n\nsolver = FgmresSolver(m, n, memory, S)\nsolver = FgmresSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BlockGmresSolver","page":"API","title":"Krylov.BlockGmresSolver","text":"Type for storing the vectors required by the in-place version of BLOCK-GMRES.\n\nThe outer constructors\n\nsolver = BlockGmresSolver(m, n, p, memory, SV, SM)\nsolver = BlockGmresSolver(A, B; memory=5)\n\nmay be used in order to create these vectors. memory is set to div(n,p) if the value given is larger than div(n,p).\n\n\n\n\n\n","category":"type"},{"location":"api/#Utilities","page":"API","title":"Utilities","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Krylov.roots_quadratic\nKrylov.sym_givens\nKrylov.to_boundary\nKrylov.vec2str\nKrylov.ktypeof\nKrylov.kzeros\nKrylov.kones\nKrylov.vector_to_matrix\nKrylov.matrix_to_vector","category":"page"},{"location":"api/#Krylov.roots_quadratic","page":"API","title":"Krylov.roots_quadratic","text":"roots = roots_quadratic(q₂, q₁, q₀; nitref)\n\nFind the real roots of the quadratic\n\nq(x) = q₂ x² + q₁ x + q₀,\n\nwhere q₂, q₁ and q₀ are real. Care is taken to avoid numerical cancellation. Optionally, nitref steps of iterative refinement may be performed to improve accuracy. By default, nitref=1.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.sym_givens","page":"API","title":"Krylov.sym_givens","text":"(c, s, ρ) = sym_givens(a, b)\n\nNumerically stable symmetric Givens reflection. Given a and b reals, return (c, s, ρ) such that\n\n[ c s ] [ a ] = [ ρ ]\n[ s -c ] [ b ] = [ 0 ].\n\n\n\n\n\nNumerically stable symmetric Givens reflection. Given a and b complexes, return (c, s, ρ) with c real and (s, ρ) complexes such that\n\n[ c s ] [ a ] = [ ρ ]\n[ s̅ -c ] [ b ] = [ 0 ].\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.to_boundary","page":"API","title":"Krylov.to_boundary","text":"roots = to_boundary(n, x, d, radius; flip, xNorm2, dNorm2)\n\nGiven a trust-region radius radius, a vector x lying inside the trust-region and a direction d, return σ1 and σ2 such that\n\n‖x + σi d‖ = radius, i = 1, 2\n\nin the Euclidean norm. n is the length of vectors x and d. If known, ‖x‖² and ‖d‖² may be supplied with xNorm2 and dNorm2.\n\nIf flip is set to true, σ1 and σ2 are computed such that\n\n‖x - σi d‖ = radius, i = 1, 2.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.vec2str","page":"API","title":"Krylov.vec2str","text":"s = vec2str(x; ndisp)\n\nDisplay an array in the form\n\n[ -3.0e-01 -5.1e-01 1.9e-01 ... -2.3e-01 -4.4e-01 2.4e-01 ]\n\nwith (ndisp - 1)/2 elements on each side.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.ktypeof","page":"API","title":"Krylov.ktypeof","text":"S = ktypeof(v)\n\nReturn the most relevant storage type S based on the type of v.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.kzeros","page":"API","title":"Krylov.kzeros","text":"v = kzeros(S, n)\n\nCreate a vector of storage type S of length n only composed of zero.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.kones","page":"API","title":"Krylov.kones","text":"v = kones(S, n)\n\nCreate a vector of storage type S of length n only composed of one.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.vector_to_matrix","page":"API","title":"Krylov.vector_to_matrix","text":"M = vector_to_matrix(S)\n\nReturn the dense matrix storage type M related to the dense vector storage type S.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.matrix_to_vector","page":"API","title":"Krylov.matrix_to_vector","text":"S = matrix_to_vector(M)\n\nReturn the dense vector storage type S related to the dense matrix storage type M.\n\n\n\n\n\n","category":"function"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"# Thanks Morten Piibeleht for the hack with the tables!","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"","category":"page"},{"location":"storage/#storage-requirements","page":"Storage requirements","title":"Storage requirements","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"This section provides the storage requirements of all Krylov methods available in Krylov.jl.","category":"page"},{"location":"storage/#Notation","page":"Storage requirements","title":"Notation","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"We denote by m and n the number of rows and columns of the linear problem. The memory parameter of DIOM, FOM, DQGMRES, GMRES, FGMRES and GPMR is k. The numbers of shifts of CG-LANCZOS-SHIFT is p.","category":"page"},{"location":"storage/#Theoretical-storage-requirements","page":"Storage requirements","title":"Theoretical storage requirements","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"The following tables provide the number of coefficients that must be allocated for each Krylov method. The coefficients have the same type as those that compose the linear problem we seek to solve. Each table summarizes the storage requirements of Krylov methods recommended to a specific linear problem.","category":"page"},{"location":"storage/#Hermitian-positive-definite-linear-systems","page":"Storage requirements","title":"Hermitian positive definite linear systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods CG CR CAR CG-LANCZOS CG-LANCZOS-SHIFT\nStorage 4n 5n 7n 5n 3n + 2np + 5p","category":"page"},{"location":"storage/#Hermitian-indefinite-linear-systems","page":"Storage requirements","title":"Hermitian indefinite linear systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods SYMMLQ MINRES MINRES-QLP MINARES\nStorage 5n 6n 6n 8n","category":"page"},{"location":"storage/#Non-Hermitian-square-linear-systems","page":"Storage requirements","title":"Non-Hermitian square linear systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods CGS BICGSTAB BiLQ QMR\nStorage 6n 6n 8n 9n","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods DIOM DQGMRES\nStorage n(2k+1) + 2k - 1 n(2k+2) + 3k + 1","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods FOM GMRES FGMRES\nStoragedfrac n(2+k) +2k + dfrack(k + 1)2 n(2+k) + 3k + dfrack(k + 1)2 n(2+2k) + 3k + dfrack(k + 1)2","category":"page"},{"location":"storage/#Least-norm-problems","page":"Storage requirements","title":"Least-norm problems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods USYMLQ CGNE CRMR LNLQ CRAIG CRAIGMR\nStorage 5n + 3m 3n + 2m 3n + 2m 3n + 4m 3n + 4m 4n + 5m","category":"page"},{"location":"storage/#Least-squares-problems","page":"Storage requirements","title":"Least-squares problems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods USYMQR CGLS CRLS LSLQ LSQR LSMR\nStorage 6n + 3m 3n + 2m 4n + 3m 4n + 2m 4n + 2m 5n + 2m","category":"page"},{"location":"storage/#Adjoint-systems","page":"Storage requirements","title":"Adjoint systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods BiLQR TriLQR\nStorage 11n 6m + 5n","category":"page"},{"location":"storage/#Saddle-point-and-Hermitian-quasi-definite-systems","page":"Storage requirements","title":"Saddle-point and Hermitian quasi-definite systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods TriCG TriMR\nStorage 6n + 6m 8n + 8m","category":"page"},{"location":"storage/#Generalized-saddle-point-and-non-Hermitian-partitioned-systems","page":"Storage requirements","title":"Generalized saddle-point and non-Hermitian partitioned systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Method GPMR\nStorage (2+k)(n+m) + 2k^2 + 11k","category":"page"},{"location":"storage/#Practical-storage-requirements","page":"Storage requirements","title":"Practical storage requirements","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Each method has its own KrylovSolver that contains all the storage needed by the method. In the REPL, the size in bytes of each attribute and the total amount of memory allocated by the solver are displayed when we show a KrylovSolver.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"using Krylov\n\nm = 5000\nn = 12000\nA = rand(Float64, m, n)\nb = rand(Float64, m)\nsolver = LsmrSolver(A, b)\nshow(stdout, solver, show_stats=false)","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"If we want the total number of bytes used by the solver, we can call nbytes = sizeof(solver).","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"nbytes = sizeof(solver)","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Thereafter, we can use Base.format_bytes(nbytes) to recover what is displayed in the REPL.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Base.format_bytes(nbytes)","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"To verify that we match the theoretical results, we just need to multiply the storage requirement of a method by the number of bytes associated to the precision of the linear problem. For instance, we need 4 bytes for the precision Float32, 8 bytes for precisions Float64 and ComplexF32, and 16 bytes for the precision ComplexF64.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"FC = Float64 # precision of the least-squares problem\nncoefs_lsmr = 5*n + 2*m # number of coefficients\nnbytes_lsmr = sizeof(FC) * ncoefs_lsmr # number of bytes","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Therefore, you can check that you have enough memory in RAM to allocate a KrylovSolver.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"free_nbytes = Sys.free_memory()\nBase.format_bytes(free_nbytes) # Total free memory in RAM in bytes.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"note: Note\nBeyond having faster operations, using low precisions, such as simple precision, allows to store more coefficients in RAM and solve larger linear problems.\nIn the file test_allocations.jl, we use the macro @allocated to test that we match the expected storage requirement of each method with a tolerance of 2%.","category":"page"},{"location":"examples/craig/","page":"CRAIG","title":"CRAIG","text":"using Krylov\nusing LinearAlgebra, Printf\n\nm = 5\nn = 8\nλ = 1.0e-3\nA = rand(m, n)\nb = A * ones(n)\nxy_exact = [A λ*I] \\ b # In Julia, this is the min-norm solution!\n\n(x, y, stats) = craig(A, b, λ=λ, atol=0.0, rtol=1.0e-20, verbose=1)\nshow(stats)\n\n# Check that we have a minimum-norm solution.\n# When λ > 0 we solve min ‖(x,s)‖ s.t. Ax + λs = b, and we get s = λy.\n@printf(\"Primal feasibility: %7.1e\\n\", norm(b - A * x - λ^2 * y) / norm(b))\n@printf(\"Dual feasibility: %7.1e\\n\", norm(x - A' * y) / norm(x))\n@printf(\"Error in x: %7.1e\\n\", norm(x - xy_exact[1:n]) / norm(xy_exact[1:n]))\nif λ > 0.0\n @printf(\"Error in y: %7.1e\\n\", norm(λ * y - xy_exact[n+1:n+m]) / norm(xy_exact[n+1:n+m]))\nend","category":"page"},{"location":"examples/cgne/","page":"CGNE","title":"CGNE","text":"using Krylov, HarwellRutherfordBoeing, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"wm2\")\npath = fetch_ssmc(matrix, format=\"RB\")\n\nA = RutherfordBoeingData(joinpath(path[1], \"$(matrix.name[1]).rb\")).data\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\nx_exact = A' * ones(m)\nx_exact_norm = norm(x_exact)\nx_exact /= x_exact_norm\nb = A * x_exact\n(x, stats) = cgne(A, b)\nshow(stats)\nresid = norm(A * x - b) / norm(b)\n@printf(\"CGNE: Relative residual: %7.1e\\n\", resid)\n@printf(\"CGNE: ‖x - x*‖₂: %7.1e\\n\", norm(x - x_exact))","category":"page"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"# Least-squares problems","category":"page"},{"location":"solvers/ls/#CGLS","page":"Least-squares problems","title":"CGLS","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"cgls\ncgls!","category":"page"},{"location":"solvers/ls/#Krylov.cgls","page":"Least-squares problems","title":"Krylov.cgls","text":"(x, stats) = cgls(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, radius::T=zero(T),\n λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ‖x‖₂²\n\nof size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations\n\n(AᴴA + λI) x = Aᴴb\n\nbut is more stable.\n\nCGLS produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSQR, though can be slightly less accurate, but simpler to implement.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, 49(6), pp. 409–436, 1952.\nA. Björck, T. Elfving and Z. Strakos, Stability of Conjugate Gradient and Lanczos Methods for Linear Least Squares Problems, SIAM Journal on Matrix Analysis and Applications, 19(3), pp. 720–736, 1998.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.cgls!","page":"Least-squares problems","title":"Krylov.cgls!","text":"solver = cgls!(solver::CglsSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of cgls.\n\nSee CglsSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#CRLS","page":"Least-squares problems","title":"CRLS","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"crls\ncrls!","category":"page"},{"location":"solvers/ls/#Krylov.crls","page":"Least-squares problems","title":"Krylov.crls","text":"(x, stats) = crls(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, radius::T=zero(T),\n λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ‖x‖₂²\n\nof size m × n using the Conjugate Residuals (CR) method. This method is equivalent to applying MINRES to the normal equations\n\n(AᴴA + λI) x = Aᴴb.\n\nThis implementation recurs the residual r := b - Ax.\n\nCRLS produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSMR, though can be substantially less accurate, but simpler to implement.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nD. C.-L. Fong, Minimum-Residual Methods for Sparse, Least-Squares using Golubg-Kahan Bidiagonalization, Ph.D. Thesis, Stanford University, 2011.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.crls!","page":"Least-squares problems","title":"Krylov.crls!","text":"solver = crls!(solver::CrlsSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of crls.\n\nSee CrlsSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#LSLQ","page":"Least-squares problems","title":"LSLQ","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"lslq\nlslq!","category":"page"},{"location":"solvers/ls/#Krylov.lslq","page":"Least-squares problems","title":"Krylov.lslq","text":"(x, stats) = lslq(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n window::Int=5, transfer_to_lsqr::Bool=false,\n sqd::Bool=false, λ::T=zero(T),\n σ::T=zero(T), etol::T=√eps(T),\n utol::T=√eps(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ²‖x‖₂²\n\nof size m × n using the LSLQ method, where λ ≥ 0 is a regularization parameter. LSLQ is formally equivalent to applying SYMMLQ to the normal equations\n\n(AᴴA + λ²I) x = Aᴴb\n\nbut is more stable.\n\nIf λ > 0, we solve the symmetric and quasi-definite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ -λ²F ] [ x ] = [ 0 ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSLQ is then equivalent to applying SYMMLQ to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).\n\nIf λ = 0, we solve the symmetric and indefinite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ 0 ] [ x ] = [ 0 ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹.\n\nIn this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).\n\nMain features\n\nthe solution estimate is updated along orthogonal directions\nthe norm of the solution estimate ‖xᴸₖ‖₂ is increasing\nthe error ‖eₖ‖₂ := ‖xᴸₖ - x*‖₂ is decreasing\nit is possible to transition cheaply from the LSLQ iterate to the LSQR iterate if there is an advantage (there always is in terms of error)\nif A is rank deficient, identify the minimum least-squares solution\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\ntransfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nσ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;\netol: stopping tolerance based on the lower bound on the error;\nutol: stopping tolerance based on the upper bound on the error;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a LSLQStats structure.\nstats.err_lbnds is a vector of lower bounds on the LQ error–-the vector is empty if window is set to zero\nstats.err_ubnds_lq is a vector of upper bounds on the LQ error–-the vector is empty if σ == 0 is left at zero\nstats.err_ubnds_cg is a vector of upper bounds on the CG error–-the vector is empty if σ == 0 is left at zero\nstats.error_with_bnd is a boolean indicating whether there was an error in the upper bounds computation (cancellation errors, too large σ ...)\n\nStopping conditions\n\nThe iterations stop as soon as one of the following conditions holds true:\n\nthe optimality residual is sufficiently small (stats.status = \"found approximate minimum least-squares solution\") in the sense that either\n‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ atol, or\n1 + ‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ 1\nan approximate zero-residual solution has been found (stats.status = \"found approximate zero-residual solution\") in the sense that either\n‖r‖ / ‖b‖ ≤ btol + atol ‖A‖ * ‖xᴸ‖ / ‖b‖, or\n1 + ‖r‖ / ‖b‖ ≤ 1\nthe estimated condition number of A is too large in the sense that either\n1/cond(A) ≤ 1/conlim (stats.status = \"condition number exceeds tolerance\"), or\n1 + 1/cond(A) ≤ 1 (stats.status = \"condition number seems too large for this machine\")\nthe lower bound on the LQ forward error is less than etol * ‖xᴸ‖\nthe upper bound on the CG forward error is less than utol * ‖xᶜ‖\n\nReferences\n\nR. Estrin, D. Orban and M. A. Saunders, Euclidean-norm error bounds for SYMMLQ and CG, SIAM Journal on Matrix Analysis and Applications, 40(1), pp. 235–253, 2019.\nR. Estrin, D. Orban and M. A. Saunders, LSLQ: An Iterative Method for Linear Least-Squares with an Error Minimization Property, SIAM Journal on Matrix Analysis and Applications, 40(1), pp. 254–275, 2019.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.lslq!","page":"Least-squares problems","title":"Krylov.lslq!","text":"solver = lslq!(solver::LslqSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lslq.\n\nSee LslqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#LSQR","page":"Least-squares problems","title":"LSQR","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"lsqr\nlsqr!","category":"page"},{"location":"solvers/ls/#Krylov.lsqr","page":"Least-squares problems","title":"Krylov.lsqr","text":"(x, stats) = lsqr(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n window::Int=5, sqd::Bool=false, λ::T=zero(T),\n radius::T=zero(T), etol::T=√eps(T),\n axtol::T=√eps(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=zero(T),\n rtol::T=zero(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ²‖x‖₂²\n\nof size m × n using the LSQR method, where λ ≥ 0 is a regularization parameter. LSQR is formally equivalent to applying CG to the normal equations\n\n(AᴴA + λ²I) x = Aᴴb\n\n(and therefore to CGLS) but is more stable.\n\nLSQR produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to CGLS, though can be slightly more accurate.\n\nIf λ > 0, LSQR solves the symmetric and quasi-definite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ -λ²F ] [ x ] = [ 0 ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSQR is then equivalent to applying CG to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).\n\nIf λ = 0, we solve the symmetric and indefinite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ 0 ] [ x ] = [ 0 ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹.\n\nIn this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\netol: stopping tolerance based on the lower bound on the error;\naxtol: tolerance on the backward error;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nC. C. Paige and M. A. Saunders, LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares, ACM Transactions on Mathematical Software, 8(1), pp. 43–71, 1982.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.lsqr!","page":"Least-squares problems","title":"Krylov.lsqr!","text":"solver = lsqr!(solver::LsqrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lsqr.\n\nSee LsqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#LSMR","page":"Least-squares problems","title":"LSMR","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"lsmr\nlsmr!","category":"page"},{"location":"solvers/ls/#Krylov.lsmr","page":"Least-squares problems","title":"Krylov.lsmr","text":"(x, stats) = lsmr(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n window::Int=5, sqd::Bool=false, λ::T=zero(T),\n radius::T=zero(T), etol::T=√eps(T),\n axtol::T=√eps(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=zero(T),\n rtol::T=zero(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ²‖x‖₂²\n\nof size m × n using the LSMR method, where λ ≥ 0 is a regularization parameter. LSMR is formally equivalent to applying MINRES to the normal equations\n\n(AᴴA + λ²I) x = Aᴴb\n\n(and therefore to CRLS) but is more stable.\n\nLSMR produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to CRLS, though can be substantially more accurate.\n\nLSMR can be also used to find a null vector of a singular matrix A by solving the problem min ‖Aᴴx - b‖ with any nonzero vector b. At a minimizer, the residual vector r = b - Aᴴx will satisfy Ar = 0.\n\nIf λ > 0, we solve the symmetric and quasi-definite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ -λ²F ] [ x ] = [ 0 ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSMR is then equivalent to applying MINRES to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).\n\nIf λ = 0, we solve the symmetric and indefinite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ 0 ] [ x ] = [ 0 ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹.\n\nIn this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\netol: stopping tolerance based on the lower bound on the error;\naxtol: tolerance on the backward error;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a LsmrStats structure.\n\nReference\n\nD. C.-L. Fong and M. A. Saunders, LSMR: An Iterative Algorithm for Sparse Least Squares Problems, SIAM Journal on Scientific Computing, 33(5), pp. 2950–2971, 2011.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.lsmr!","page":"Least-squares problems","title":"Krylov.lsmr!","text":"solver = lsmr!(solver::LsmrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lsmr.\n\nSee LsmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#USYMQR","page":"Least-squares problems","title":"USYMQR","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"usymqr\nusymqr!","category":"page"},{"location":"solvers/ls/#Krylov.usymqr","page":"Least-squares problems","title":"Krylov.usymqr","text":"(x, stats) = usymqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = usymqr(A, b, c, x0::AbstractVector; kwargs...)\n\nUSYMQR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nUSYMQR solves the linear least-squares problem min ‖b - Ax‖² of size m × n. USYMQR solves Ax = b if it is consistent.\n\nUSYMQR is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The residual norm ‖b - Ax‖ monotonously decreases in USYMQR. When A is Hermitian and b = c, QMR is equivalent to MINRES. USYMQR is considered as a generalization of MINRES.\n\nIt can also be applied to under-determined and over-determined problems. USYMQR finds the minimum-norm solution if problems are inconsistent.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. A. Saunders, H. D. Simon, and E. L. Yip, Two Conjugate-Gradient-Type Methods for Unsymmetric Linear Equations, SIAM Journal on Numerical Analysis, 25(4), pp. 927–940, 1988.\nA. Buttari, D. Orban, D. Ruiz and D. Titley-Peloquin, A tridiagonalization method for symmetric saddle-point and quasi-definite systems, SIAM Journal on Scientific Computing, 41(5), pp. 409–432, 2019.\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.usymqr!","page":"Least-squares problems","title":"Krylov.usymqr!","text":"solver = usymqr!(solver::UsymqrSolver, A, b, c; kwargs...)\nsolver = usymqr!(solver::UsymqrSolver, A, b, c, x0; kwargs...)\n\nwhere kwargs are keyword arguments of usymqr.\n\nSee UsymqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"preconditioners/#preconditioners","page":"Preconditioners","title":"Preconditioners","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The solvers in Krylov.jl support preconditioners, i.e., transformations that modify a linear system Ax = b into an equivalent form that may yield faster convergence in finite-precision arithmetic. Preconditioning can be used to reduce the condition number of the problem or cluster its eigenvalues or singular values for instance.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The design of preconditioners is highly dependent on the origin of the problem and most preconditioners need to take application-dependent information and structure into account. Specialized preconditioners generally outperform generic preconditioners such as incomplete factorizations.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The construction of a preconditioner necessitates trade-offs because we need to apply it at least once per iteration within a Krylov method. Hence, a preconditioner must be constructed such that it is cheap to apply, while also capturing the characteristics of the original system in some sense.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"There exist three variants of preconditioning:","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Left preconditioning Two-sided preconditioning Right preconditioning\nP_ell^-1Ax = P_ell^-1b P_ell^-1AP_r^-1y = P_ell^-1btextwithx = P_r^-1y AP_r^-1y = btextwithx = P_r^-1y","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"where P_ell and P_r are square and nonsingular.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The left preconditioning preserves the error x_k - x^star whereas the right preconditioning keeps invariant the residual b - A x_k. Two-sided preconditioning is the only variant that allows to preserve the hermicity of a linear system.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"note: Note\nBecause det(P^-1A - lambda I) = det(A - lambda P) det(P^-1) = det(AP^-1 - lambda I), the eigenvalues of P^-1A and AP^-1 are identical. If P = LL^H, L^-1AL^-H also has the same spectrum.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"In Krylov.jl, we call P_ell^-1 and P_r^-1 the preconditioners and we assume that we can apply them with the operation y leftarrow P^-1 * x. It is also common to call P_ell and P_r the preconditioners if the equivalent operation y leftarrow Pbackslashx is available. Krylov.jl supports both approaches thanks to the argument ldiv of the Krylov solvers.","category":"page"},{"location":"preconditioners/#How-to-use-preconditioners-in-Krylov.jl?","page":"Preconditioners","title":"How to use preconditioners in Krylov.jl?","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"info: Info\nA preconditioner only needs to support the operation mul!(y, P⁻¹, x) when ldiv=false or ldiv!(y, P, x) when ldiv=true to be used in Krylov.jl.\nAdditional support for adjoint with preconditioners is required in the methods BILQ and QMR.\nThe default value of a preconditioner in Krylov.jl is the identity operator I.","category":"page"},{"location":"preconditioners/#Square-non-Hermitian-linear-systems","page":"Preconditioners","title":"Square non-Hermitian linear systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: CGS, BILQ, QMR, BiCGSTAB, DQGMRES, GMRES, BLOCK-GMRES, FGMRES, DIOM and FOM.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"A Krylov method dedicated to non-Hermitian linear systems allows the three variants of preconditioning.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners P_ell^-1 P_ell P_r^-1 P_r\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/#Hermitian-linear-systems","page":"Preconditioners","title":"Hermitian linear systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: SYMMLQ, CG, CG-LANCZOS, CG-LANCZOS-SHIFT, CR, CAR, MINRES, MINRES-QLP and MINARES.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"When A is Hermitian, we can only use centered preconditioning L^-1AL^-Hy = L^-1b with x = L^-Hy. Centered preconditioning is a special case of two-sided preconditioning with P_ell = L = P_r^H that maintains hermicity. However, there is no need to specify L and one may specify P_c = LL^H or its inverse directly.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners P_c^-1 P_c\nArguments M with ldiv=false M with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioner M must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Linear-least-squares-problems","page":"Preconditioners","title":"Linear least-squares problems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: CGLS, CRLS, LSLQ, LSQR and LSMR.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nleast-squares problem min tfrac12 b - Ax^2_2 min tfrac12 b - Ax^2_E^-1\nNormal equation A^HAx = A^Hb A^HE^-1Ax = A^HE^-1b\nAugmented system beginbmatrix I A A^H 0 endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix beginbmatrix E A A^H 0 endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"LSLQ, LSQR and LSMR also handle regularized least-squares problems.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nleast-squares problem min tfrac12 b - Ax^2_2 + tfrac12 lambda^2 x^2_2 min tfrac12 b - Ax^2_E^-1 + tfrac12 lambda^2 x^2_F\nNormal equation (A^HA + lambda^2 I)x = A^Hb (A^HE^-1A + lambda^2 F)x = A^HE^-1b\nAugmented system beginbmatrix I A A^H -lambda^2 I endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix beginbmatrix E A A^H -lambda^2 F endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners E^-1 E F^-1 F\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioners M and N must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Linear-least-norm-problems","page":"Preconditioners","title":"Linear least-norm problems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: CGNE, CRMR, LNLQ, CRAIG and CRAIGMR.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nminimum-norm problem min tfrac12 x^2_2textstAx = b min tfrac12 x^2_FtextstAx = b\nNormal equation AA^Hy = btextwithx = A^Hy AF^-1A^Hy = btextwithx = F^-1A^Hy\nAugmented system beginbmatrix -I A^H phantom-A 0 endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix beginbmatrix -F A^H phantom-A 0 endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"LNLQ, CRAIG and CRAIGMR also handle penalized minimum-norm problems.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nminimum-norm problem min tfrac12 x^2_2 + tfrac12 y^2_2textstAx + lambda^2 y = b min tfrac12 x^2_F + tfrac12 y^2_EtextstAx + lambda^2 Ey = b\nNormal equation (AA^H + lambda^2 I)y = btextwithx = A^Hy (AF^-1A^H + lambda^2 E)y = btextwithx = F^-1A^Hy\nAugmented system beginbmatrix -I A^H phantom-A lambda^2 I endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix beginbmatrix -F A^H phantom-A lambda^2 E endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners E^-1 E F^-1 F\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioners M and N must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Saddle-point-and-symmetric-quasi-definite-systems","page":"Preconditioners","title":"Saddle-point and symmetric quasi-definite systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"TriCG and TriMR can take advantage of the structure of Hermitian systems Kz = d with the 2x2 block structure","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":" beginbmatrix tau E phantom-A A^H nu F endbmatrix beginbmatrix x y endbmatrix = beginbmatrix b c endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners E^-1 E F^-1 F\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioners M and N must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Generalized-saddle-point-and-unsymmetric-partitioned-systems","page":"Preconditioners","title":"Generalized saddle-point and unsymmetric partitioned systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"GPMR can take advantage of the structure of general square systems Kz = d with the 2x2 block structure","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":" beginbmatrix lambda M A B mu N endbmatrix beginbmatrix x y endbmatrix = beginbmatrix b c endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Relations CE = M^-1 EC = M DF = N^-1 FD = N\nArguments C and E with ldiv=false C and E with ldiv=true D and F with ldiv=false D and F with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"note: Note\nOur implementations of BiLQ, QMR, BiLQR, USYMLQ, USYMQR and TriLQR don't support preconditioning.","category":"page"},{"location":"preconditioners/#Packages-that-provide-preconditioners","page":"Preconditioners","title":"Packages that provide preconditioners","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"KrylovPreconditioners.jl implements block-Jacobi, IC(0) and ILU(0) preconditioners.\nILUZero.jl is a Julia implementation of incomplete LU factorization with zero level of fill-in. \nLimitedLDLFactorizations.jl for limited-memory LDLᵀ factorization of symmetric matrices.\nAlgebraicMultigrid.jl provides two algebraic multigrid (AMG) preconditioners.\nRandomizedPreconditioners.jl uses randomized numerical linear algebra to construct approximate inverses of matrices.\nBasicLU.jl uses a sparse LU factorization to compute a maximum volume basis that can be used as a preconditioner for least-norm and least-squares problems.","category":"page"},{"location":"preconditioners/#Examples","page":"Preconditioners","title":"Examples","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using KrylovPreconditioners, Krylov\n\nP⁻¹ = BlockJacobiPreconditioner(A) # Block-Jacobi preconditioner\nx, stats = gmres(A, b, M=P⁻¹)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using Krylov\nn, m = size(A)\nd = [A[i,i] ≠ 0 ? 1 / abs(A[i,i]) : 1 for i=1:n] # Jacobi preconditioner\nP⁻¹ = diagm(d)\nx, stats = symmlq(A, b, M=P⁻¹)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using Krylov\nn, m = size(A)\nd = [1 / norm(A[:,i]) for i=1:m] # diagonal preconditioner\nP⁻¹ = diagm(d)\nx, stats = minres(A, b, M=P⁻¹)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using KrylovPreconditioners, Krylov\nPℓ = ilu(A)\nx, stats = gmres(A, b, M=Pℓ, ldiv=true) # left preconditioning","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using LimitedLDLFactorizations, Krylov\nP = lldl(A)\nP.D .= abs.(P.D)\nx, stats = cg(A, b, M=P, ldiv=true) # centered preconditioning","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using ILUZero, Krylov\nPᵣ = ilu0(A)\nx, stats = bicgstab(A, b, N=Pᵣ, ldiv=true) # right preconditioning","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using LDLFactorizations, Krylov\n\nM = ldl(E)\nN = ldl(F)\n\n# [E A] [x] = [b]\n# [Aᴴ -F] [y] [c]\nx, y, stats = tricg(A, b, c, M=M, N=N, ldiv=true)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using SuiteSparse, Krylov\nimport LinearAlgebra.ldiv!\n\nM = cholesky(E)\n\n# ldiv! is not implemented for the sparse Cholesky factorization (SuiteSparse.CHOLMOD)\nldiv!(y::Vector{T}, F::SuiteSparse.CHOLMOD.Factor{T}, x::Vector{T}) where T = (y .= F \\ x)\n\n# [E A] [x] = [b]\n# [Aᴴ 0] [y] [c]\nx, y, stats = trimr(A, b, c, M=M, sp=true, ldiv=true)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using Krylov\n\nC = lu(M)\n\n# [M A] [x] = [b]\n# [B 0] [y] [c]\nx, y, stats = gpmr(A, B, b, c, C=C, gsp=true, ldiv=true)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"import BasicLU\nusing LinearOperators, Krylov\n\n# Least-squares problem\nm, n = size(A)\nAᴴ = sparse(A')\nbasis, B = BasicLU.maxvolbasis(Aᴴ)\nopA = LinearOperator(A)\nB⁻ᴴ = LinearOperator(Float64, n, n, false, false, (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'T')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'N')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'N')))\n\nd, stats = lsmr(opA * B⁻ᴴ, b) # min ‖AB⁻ᴴd - b‖₂\nx = B⁻ᴴ * d # recover the solution of min ‖Ax - b‖₂\n\n# Least-norm problem\nm, n = size(A)\nbasis, B = maxvolbasis(A)\nopA = LinearOperator(A)\nB⁻¹ = LinearOperator(Float64, m, m, false, false, (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'N')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'T')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'T')))\n\nx, y, stats = craigmr(B⁻¹ * opA, B⁻¹ * b) # min ‖x‖₂ s.t. B⁻¹Ax = B⁻¹b","category":"page"},{"location":"reference/#Reference","page":"Reference","title":"Reference","text":"","category":"section"},{"location":"reference/#Index","page":"Reference","title":"Index","text":"","category":"section"},{"location":"reference/","page":"Reference","title":"Reference","text":"","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"Krylov.FloatOrComplex\nKrylov.niterations\nKrylov.Aprod\nKrylov.Atprod\nKrylov.kstdout\nKrylov.extract_parameters\nBase.show","category":"page"},{"location":"reference/#Krylov.FloatOrComplex","page":"Reference","title":"Krylov.FloatOrComplex","text":"FloatOrComplex{T}\n\nUnion type of T and Complex{T} where T is an AbstractFloat.\n\n\n\n\n\n","category":"type"},{"location":"reference/#Krylov.niterations","page":"Reference","title":"Krylov.niterations","text":"niterations(solver)\n\nReturn the number of iterations performed by the Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"reference/#Krylov.Aprod","page":"Reference","title":"Krylov.Aprod","text":"Aprod(solver)\n\nReturn the number of operator-vector products with A performed by the Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"reference/#Krylov.Atprod","page":"Reference","title":"Krylov.Atprod","text":"Atprod(solver)\n\nReturn the number of operator-vector products with A' performed by the Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"reference/#Krylov.kstdout","page":"Reference","title":"Krylov.kstdout","text":"Default I/O stream for all Krylov methods.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#Krylov.extract_parameters","page":"Reference","title":"Krylov.extract_parameters","text":"arguments = extract_parameters(ex::Expr)\n\nExtract the arguments of an expression that is keyword parameter tuple. Implementation suggested by Mitchell J. O'Sullivan (@mosullivan93).\n\n\n\n\n\n","category":"function"},{"location":"reference/#Base.show","page":"Reference","title":"Base.show","text":"show(io, solver; show_stats=true)\n\nStatistics of solver are displayed if show_stats is set to true.\n\n\n\n\n\n","category":"function"},{"location":"examples/block_gmres/","page":"BLOCK-GMRES","title":"BLOCK-GMRES","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"sherman2\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nB = Matrix{Float64}(I, n, 5)\nB_norm = norm(B)\n\n# Solve Ax = B\nX, stats = block_gmres(A, B)\nshow(stats)\nR = B - A * X\n@printf(\"Relative residual: %8.1e\\n\", norm(R) / B_norm)","category":"page"},{"location":"examples/cg/","page":"CG","title":"CG","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"bcsstk09\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = ones(n)\nb_norm = norm(b)\n\n# Solve Ax = b.\n(x, stats) = cg(A, b)\nshow(stats)\nr = b - A * x\n@printf(\"Relative residual: %8.1e\\n\", norm(r) / b_norm)","category":"page"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"# Non-Hermitian square linear systems","category":"page"},{"location":"solvers/unsymmetric/#BiLQ","page":"Non-Hermitian square linear systems","title":"BiLQ","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"bilq\nbilq!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.bilq","page":"Non-Hermitian square linear systems","title":"Krylov.bilq","text":"(x, stats) = bilq(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, transfer_to_bicg::Bool=true,\n M=I, N=I, ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = bilq(A, b, x0::AbstractVector; kwargs...)\n\nBiLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the square linear system Ax = b of size n using BiLQ. BiLQ is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, BiLQ is equivalent to SYMMLQ. BiLQ requires support for adjoint(M) and adjoint(N) if preconditioners are provided.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\ntransfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\nR. Fletcher, Conjugate gradient methods for indefinite systems, Numerical Analysis, Springer, pp. 73–89, 1976.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.bilq!","page":"Non-Hermitian square linear systems","title":"Krylov.bilq!","text":"solver = bilq!(solver::BilqSolver, A, b; kwargs...)\nsolver = bilq!(solver::BilqSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of bilq.\n\nSee BilqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#QMR","page":"Non-Hermitian square linear systems","title":"QMR","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"qmr\nqmr!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.qmr","page":"Non-Hermitian square linear systems","title":"Krylov.qmr","text":"(x, stats) = qmr(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, M=I, N=I, ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf, verbose::Int=0,\n history::Bool=false, callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = qmr(A, b, x0::AbstractVector; kwargs...)\n\nQMR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the square linear system Ax = b of size n using QMR.\n\nQMR is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, QMR is equivalent to MINRES. QMR requires support for adjoint(M) and adjoint(N) if preconditioners are provided.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nR. W. Freund and N. M. Nachtigal, QMR : a quasi-minimal residual method for non-Hermitian linear systems, Numerische mathematik, Vol. 60(1), pp. 315–339, 1991.\nR. W. Freund and N. M. Nachtigal, An implementation of the QMR method based on coupled two-term recurrences, SIAM Journal on Scientific Computing, Vol. 15(2), pp. 313–337, 1994.\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.qmr!","page":"Non-Hermitian square linear systems","title":"Krylov.qmr!","text":"solver = qmr!(solver::QmrSolver, A, b; kwargs...)\nsolver = qmr!(solver::QmrSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of qmr.\n\nSee QmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#CGS","page":"Non-Hermitian square linear systems","title":"CGS","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"cgs\ncgs!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.cgs","page":"Non-Hermitian square linear systems","title":"Krylov.cgs","text":"(x, stats) = cgs(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, M=I, N=I,\n ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = cgs(A, b, x0::AbstractVector; kwargs...)\n\nCGS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the consistent linear system Ax = b of size n using CGS. CGS requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.\n\nFrom \"Iterative Methods for Sparse Linear Systems (Y. Saad)\" :\n\n«The method is based on a polynomial variant of the conjugate gradients algorithm. Although related to the so-called bi-conjugate gradients (BCG) algorithm, it does not involve adjoint matrix-vector multiplications, and the expected convergence rate is about twice that of the BCG algorithm.\n\nThe Conjugate Gradient Squared algorithm works quite well in many cases. However, one difficulty is that, since the polynomials are squared, rounding errors tend to be more damaging than in the standard BCG algorithm. In particular, very high variations of the residual vectors often cause the residual norms computed to become inaccurate.\n\nTFQMR and BICGSTAB were developed to remedy this difficulty.»\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nP. Sonneveld, CGS, A Fast Lanczos-Type Solver for Nonsymmetric Linear systems, SIAM Journal on Scientific and Statistical Computing, 10(1), pp. 36–52, 1989.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.cgs!","page":"Non-Hermitian square linear systems","title":"Krylov.cgs!","text":"solver = cgs!(solver::CgsSolver, A, b; kwargs...)\nsolver = cgs!(solver::CgsSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of cgs.\n\nSee CgsSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#BiCGSTAB","page":"Non-Hermitian square linear systems","title":"BiCGSTAB","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"bicgstab\nbicgstab!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.bicgstab","page":"Non-Hermitian square linear systems","title":"Krylov.bicgstab","text":"(x, stats) = bicgstab(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, M=I, N=I,\n ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = bicgstab(A, b, x0::AbstractVector; kwargs...)\n\nBICGSTAB can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the square linear system Ax = b of size n using BICGSTAB. BICGSTAB requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.\n\nThe Biconjugate Gradient Stabilized method is a variant of BiCG, like CGS, but using different updates for the Aᴴ-sequence in order to obtain smoother convergence than CGS.\n\nIf BICGSTAB stagnates, we recommend DQGMRES and BiLQ as alternative methods for unsymmetric square systems.\n\nBICGSTAB stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖b‖ * rtol.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nH. A. van der Vorst, Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems, SIAM Journal on Scientific and Statistical Computing, 13(2), pp. 631–644, 1992.\nG. L.G. Sleijpen and D. R. Fokkema, BiCGstab(ℓ) for linear equations involving unsymmetric matrices with complex spectrum, Electronic Transactions on Numerical Analysis, 1, pp. 11–32, 1993.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.bicgstab!","page":"Non-Hermitian square linear systems","title":"Krylov.bicgstab!","text":"solver = bicgstab!(solver::BicgstabSolver, A, b; kwargs...)\nsolver = bicgstab!(solver::BicgstabSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of bicgstab.\n\nSee BicgstabSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#DIOM","page":"Non-Hermitian square linear systems","title":"DIOM","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"diom\ndiom!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.diom","page":"Non-Hermitian square linear systems","title":"Krylov.diom","text":"(x, stats) = diom(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n reorthogonalization::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = diom(A, b, x0::AbstractVector; kwargs...)\n\nDIOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the consistent linear system Ax = b of size n using DIOM.\n\nDIOM only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If CG is well defined on Ax = b and memory = 2, DIOM is theoretically equivalent to CG. If k ≤ memory where k is the number of iterations, DIOM is theoretically equivalent to FOM. Otherwise, DIOM interpolates between CG and FOM and is similar to CG with partial reorthogonalization.\n\nAn advantage of DIOM is that non-Hermitian or Hermitian indefinite or both non-Hermitian and indefinite systems of linear equations can be handled by this single algorithm.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad, Practical use of some krylov subspace methods for solving indefinite and nonsymmetric linear systems, SIAM journal on scientific and statistical computing, 5(1), pp. 203–228, 1984.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.diom!","page":"Non-Hermitian square linear systems","title":"Krylov.diom!","text":"solver = diom!(solver::DiomSolver, A, b; kwargs...)\nsolver = diom!(solver::DiomSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of diom.\n\nNote that the memory keyword argument is the only exception. It's required to create a DiomSolver and can't be changed later.\n\nSee DiomSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#FOM","page":"Non-Hermitian square linear systems","title":"FOM","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"fom\nfom!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.fom","page":"Non-Hermitian square linear systems","title":"Krylov.fom","text":"(x, stats) = fom(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = fom(A, b, x0::AbstractVector; kwargs...)\n\nFOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system Ax = b of size n using FOM.\n\nFOM algorithm is based on the Arnoldi process and a Galerkin condition.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version FOM(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad, Krylov subspace methods for solving unsymmetric linear systems, Mathematics of computation, Vol. 37(155), pp. 105–126, 1981.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.fom!","page":"Non-Hermitian square linear systems","title":"Krylov.fom!","text":"solver = fom!(solver::FomSolver, A, b; kwargs...)\nsolver = fom!(solver::FomSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of fom.\n\nNote that the memory keyword argument is the only exception. It's required to create a FomSolver and can't be changed later.\n\nSee FomSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#DQGMRES","page":"Non-Hermitian square linear systems","title":"DQGMRES","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"dqgmres\ndqgmres!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.dqgmres","page":"Non-Hermitian square linear systems","title":"Krylov.dqgmres","text":"(x, stats) = dqgmres(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n reorthogonalization::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = dqgmres(A, b, x0::AbstractVector; kwargs...)\n\nDQGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the consistent linear system Ax = b of size n using DQGMRES.\n\nDQGMRES algorithm is based on the incomplete Arnoldi orthogonalization process and computes a sequence of approximate solutions with the quasi-minimal residual property.\n\nDQGMRES only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If MINRES is well defined on Ax = b and memory = 2, DQGMRES is theoretically equivalent to MINRES. If k ≤ memory where k is the number of iterations, DQGMRES is theoretically equivalent to GMRES. Otherwise, DQGMRES interpolates between MINRES and GMRES and is similar to MINRES with partial reorthogonalization.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad and K. Wu, DQGMRES: a quasi minimal residual algorithm based on incomplete orthogonalization, Numerical Linear Algebra with Applications, Vol. 3(4), pp. 329–343, 1996.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.dqgmres!","page":"Non-Hermitian square linear systems","title":"Krylov.dqgmres!","text":"solver = dqgmres!(solver::DqgmresSolver, A, b; kwargs...)\nsolver = dqgmres!(solver::DqgmresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of dqgmres.\n\nNote that the memory keyword argument is the only exception. It's required to create a DqgmresSolver and can't be changed later.\n\nSee DqgmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#GMRES","page":"Non-Hermitian square linear systems","title":"GMRES","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"gmres\ngmres!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.gmres","page":"Non-Hermitian square linear systems","title":"Krylov.gmres","text":"(x, stats) = gmres(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = gmres(A, b, x0::AbstractVector; kwargs...)\n\nGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system Ax = b of size n using GMRES.\n\nGMRES algorithm is based on the Arnoldi process and computes a sequence of approximate solutions with the minimum residual.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version GMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad and M. H. Schultz, GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems, SIAM Journal on Scientific and Statistical Computing, Vol. 7(3), pp. 856–869, 1986.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.gmres!","page":"Non-Hermitian square linear systems","title":"Krylov.gmres!","text":"solver = gmres!(solver::GmresSolver, A, b; kwargs...)\nsolver = gmres!(solver::GmresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of gmres.\n\nNote that the memory keyword argument is the only exception. It's required to create a GmresSolver and can't be changed later.\n\nSee GmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#FGMRES","page":"Non-Hermitian square linear systems","title":"FGMRES","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"fgmres\nfgmres!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.fgmres","page":"Non-Hermitian square linear systems","title":"Krylov.fgmres","text":"(x, stats) = fgmres(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = fgmres(A, b, x0::AbstractVector; kwargs...)\n\nFGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system Ax = b of size n using FGMRES.\n\nFGMRES computes a sequence of approximate solutions with minimum residual. FGMRES is a variant of GMRES that allows changes in the right preconditioner at each iteration.\n\nThis implementation allows a left preconditioner M and a flexible right preconditioner N. A situation in which the preconditioner is \"not constant\" is when a relaxation-type method, a Chebyshev iteration or another Krylov subspace method is used as a preconditioner. Compared to GMRES, there is no additional cost incurred in the arithmetic but the memory requirement almost doubles. Thus, GMRES is recommended if the right preconditioner N is constant.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version FGMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad, A Flexible Inner-Outer Preconditioned GMRES Algorithm, SIAM Journal on Scientific Computing, Vol. 14(2), pp. 461–469, 1993.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.fgmres!","page":"Non-Hermitian square linear systems","title":"Krylov.fgmres!","text":"solver = fgmres!(solver::FgmresSolver, A, b; kwargs...)\nsolver = fgmres!(solver::FgmresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of fgmres.\n\nNote that the memory keyword argument is the only exception. It's required to create a FgmresSolver and can't be changed later.\n\nSee FgmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"examples/crmr/","page":"CRMR","title":"CRMR","text":"using Krylov, HarwellRutherfordBoeing, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"gemat1\")\npath = fetch_ssmc(matrix, format=\"RB\")\n\nA = RutherfordBoeingData(joinpath(path[1], \"$(matrix.name[1]).rb\")).data\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\nx_exact = A' * ones(m)\nx_exact_norm = norm(x_exact)\nx_exact /= x_exact_norm\nb = A * x_exact\n(x, stats) = crmr(A, b)\nshow(stats)\nresid = norm(A * x - b) / norm(b)\n@printf(\"CRMR: Relative residual: %7.1e\\n\", resid)\n@printf(\"CRMR: ‖x - x*‖₂: %7.1e\\n\", norm(x - x_exact))","category":"page"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"# Hermitian indefinite linear systems","category":"page"},{"location":"solvers/sid/#SYMMLQ","page":"Hermitian indefinite linear systems","title":"SYMMLQ","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"symmlq\nsymmlq!","category":"page"},{"location":"solvers/sid/#Krylov.symmlq","page":"Hermitian indefinite linear systems","title":"Krylov.symmlq","text":"(x, stats) = symmlq(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, window::Int=5,\n transfer_to_cg::Bool=true, λ::T=zero(T),\n λest::T=zero(T), etol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = symmlq(A, b, x0::AbstractVector; kwargs...)\n\nSYMMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above\n\nSolve the shifted linear system\n\n(A + λI) x = b\n\nof size n using the SYMMLQ method, where λ is a shift parameter, and A is Hermitian.\n\nSYMMLQ produces monotonic errors ‖x* - x‖₂.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\ntransfer_to_cg: transfer from the SYMMLQ point to the CG point, when it exists. The transfer is based on the residual norm;\nλ: regularization parameter;\nλest: positive strict lower bound on the smallest eigenvalue λₘᵢₙ when solving a positive-definite system, such as λest = (1-10⁻⁷)λₘᵢₙ;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\netol: stopping tolerance based on the lower bound on the error;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SymmlqStats structure.\n\nReference\n\nC. C. Paige and M. A. Saunders, Solution of Sparse Indefinite Systems of Linear Equations, SIAM Journal on Numerical Analysis, 12(4), pp. 617–629, 1975.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.symmlq!","page":"Hermitian indefinite linear systems","title":"Krylov.symmlq!","text":"solver = symmlq!(solver::SymmlqSolver, A, b; kwargs...)\nsolver = symmlq!(solver::SymmlqSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of symmlq.\n\nSee SymmlqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#MINRES","page":"Hermitian indefinite linear systems","title":"MINRES","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"minres\nminres!","category":"page"},{"location":"solvers/sid/#Krylov.minres","page":"Hermitian indefinite linear systems","title":"Krylov.minres","text":"(x, stats) = minres(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, window::Int=5,\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), etol::T=√eps(T),\n conlim::T=1/√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = minres(A, b, x0::AbstractVector; kwargs...)\n\nMINRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the shifted linear least-squares problem\n\nminimize ‖b - (A + λI)x‖₂²\n\nor the shifted linear system\n\n(A + λI) x = b\n\nof size n using the MINRES method, where λ ≥ 0 is a shift parameter, where A is Hermitian.\n\nMINRES is formally equivalent to applying CR to Ax=b when A is positive definite, but is typically more stable and also applies to the case where A is indefinite.\n\nMINRES produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\netol: stopping tolerance based on the lower bound on the error;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nC. C. Paige and M. A. Saunders, Solution of Sparse Indefinite Systems of Linear Equations, SIAM Journal on Numerical Analysis, 12(4), pp. 617–629, 1975.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.minres!","page":"Hermitian indefinite linear systems","title":"Krylov.minres!","text":"solver = minres!(solver::MinresSolver, A, b; kwargs...)\nsolver = minres!(solver::MinresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of minres.\n\nSee MinresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#MINRES-QLP","page":"Hermitian indefinite linear systems","title":"MINRES-QLP","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"minres_qlp\nminres_qlp!","category":"page"},{"location":"solvers/sid/#Krylov.minres_qlp","page":"Hermitian indefinite linear systems","title":"Krylov.minres_qlp","text":"(x, stats) = minres_qlp(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, Artol::T=√eps(T),\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = minres_qlp(A, b, x0::AbstractVector; kwargs...)\n\nMINRES-QLP can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nMINRES-QLP is the only method based on the Lanczos process that returns the minimum-norm solution on singular inconsistent systems (A + λI)x = b of size n, where λ is a shift parameter. It is significantly more complex but can be more reliable than MINRES when A is ill-conditioned.\n\nM also indicates the weighted norm in which residuals are measured.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nArtol: relative stopping tolerance based on the Aᴴ-residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nS.-C. T. Choi, Iterative methods for singular linear equations and least-squares problems, Ph.D. thesis, ICME, Stanford University, 2006.\nS.-C. T. Choi, C. C. Paige and M. A. Saunders, MINRES-QLP: A Krylov subspace method for indefinite or singular symmetric systems, SIAM Journal on Scientific Computing, Vol. 33(4), pp. 1810–1836, 2011.\nS.-C. T. Choi and M. A. Saunders, Algorithm 937: MINRES-QLP for symmetric and Hermitian linear equations and least-squares problems, ACM Transactions on Mathematical Software, 40(2), pp. 1–12, 2014.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.minres_qlp!","page":"Hermitian indefinite linear systems","title":"Krylov.minres_qlp!","text":"solver = minres_qlp!(solver::MinresQlpSolver, A, b; kwargs...)\nsolver = minres_qlp!(solver::MinresQlpSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of minres_qlp.\n\nSee MinresQlpSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#MINARES","page":"Hermitian indefinite linear systems","title":"MINARES","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"minares\nminares!","category":"page"},{"location":"solvers/sid/#Krylov.minares","page":"Hermitian indefinite linear systems","title":"Krylov.minares","text":"(x, stats) = minares(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false,\n λ::T = zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), Artol::T = √eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = minares(A, b, x0::AbstractVector; kwargs...)\n\nMINARES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nMINARES solves the Hermitian linear system Ax = b of size n. MINARES minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.\n\nInput arguments\n\nA: a linear operator that models a Hermitian positive definite matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nArtol: relative stopping tolerance based on the Aᴴ-residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nA. Montoison, D. Orban and M. A. Saunders, MinAres: An Iterative Solver for Symmetric Linear Systems, Cahier du GERAD G-2023-40, GERAD, Montréal, 2023.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.minares!","page":"Hermitian indefinite linear systems","title":"Krylov.minares!","text":"solver = minares!(solver::MinaresSolver, A, b; kwargs...)\nsolver = minares!(solver::MinaresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of minares.\n\nSee MinaresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"# Least-norm problems","category":"page"},{"location":"solvers/ln/#CGNE","page":"Least-norm problems","title":"CGNE","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"cgne\ncgne!","category":"page"},{"location":"solvers/ln/#Krylov.cgne","page":"Least-norm problems","title":"Krylov.cgne","text":"(x, stats) = cgne(A, b::AbstractVector{FC};\n N=I, ldiv::Bool=false,\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the consistent linear system\n\nAx + √λs = b\n\nof size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations of the second kind\n\n(AAᴴ + λI) y = b\n\nbut is more stable. When λ = 0, this method solves the minimum-norm problem\n\nmin ‖x‖₂ s.t. Ax = b.\n\nWhen λ > 0, it solves the problem\n\nmin ‖(x,s)‖₂ s.t. Ax + √λs = b.\n\nCGNE produces monotonic errors ‖x-x*‖₂ but not residuals ‖r‖₂. It is formally equivalent to CRAIG, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nN: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nJ. E. Craig, The N-step iteration procedures, Journal of Mathematics and Physics, 34(1), pp. 64–73, 1955.\nJ. E. Craig, Iterations Procedures for Simultaneous Equations, Ph.D. Thesis, Department of Electrical Engineering, MIT, 1954.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.cgne!","page":"Least-norm problems","title":"Krylov.cgne!","text":"solver = cgne!(solver::CgneSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of cgne.\n\nSee CgneSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#CRMR","page":"Least-norm problems","title":"CRMR","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"crmr\ncrmr!","category":"page"},{"location":"solvers/ln/#Krylov.crmr","page":"Least-norm problems","title":"Krylov.crmr","text":"(x, stats) = crmr(A, b::AbstractVector{FC};\n N=I, ldiv::Bool=false,\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the consistent linear system\n\nAx + √λs = b\n\nof size m × n using the Conjugate Residual (CR) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CR to the normal equations of the second kind\n\n(AAᴴ + λI) y = b\n\nbut is more stable. When λ = 0, this method solves the minimum-norm problem\n\nmin ‖x‖₂ s.t. x ∈ argmin ‖Ax - b‖₂.\n\nWhen λ > 0, this method solves the problem\n\nmin ‖(x,s)‖₂ s.t. Ax + √λs = b.\n\nCRMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRAIG-MR, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nN: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nD. Orban and M. Arioli, Iterative Solution of Symmetric Quasi-Definite Linear Systems, Volume 3 of Spotlights. SIAM, Philadelphia, PA, 2017.\nD. Orban, The Projected Golub-Kahan Process for Constrained Linear Least-Squares Problems. Cahier du GERAD G-2014-15, 2014.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.crmr!","page":"Least-norm problems","title":"Krylov.crmr!","text":"solver = crmr!(solver::CrmrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of crmr.\n\nSee CrmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#LNLQ","page":"Least-norm problems","title":"LNLQ","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"lnlq\nlnlq!","category":"page"},{"location":"solvers/ln/#Krylov.lnlq","page":"Least-norm problems","title":"Krylov.lnlq","text":"(x, y, stats) = lnlq(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n transfer_to_craig::Bool=true,\n sqd::Bool=false, λ::T=zero(T),\n σ::T=zero(T), utolx::T=√eps(T),\n utoly::T=√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nFind the least-norm solution of the consistent linear system\n\nAx + λ²y = b\n\nof size m × n using the LNLQ method, where λ ≥ 0 is a regularization parameter.\n\nFor a system in the form Ax = b, LNLQ method is equivalent to applying SYMMLQ to AAᴴy = b and recovering x = Aᴴy but is more stable. Note that y are the Lagrange multipliers of the least-norm problem\n\nminimize ‖x‖ s.t. Ax = b.\n\nIf λ > 0, LNLQ solves the symmetric and quasi-definite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A λ²E ] [ y ] = [ b ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LNLQ is then equivalent to applying SYMMLQ to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.\n\nIf λ = 0, LNLQ solves the symmetric and indefinite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A 0 ] [ y ] = [ b ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖x‖²_F s.t. Ax = b.\n\nIn this case, M can still be specified and indicates the weighted norm in which residuals are measured.\n\nIn this implementation, both the x and y-parts of the solution are returned.\n\nutolx and utoly are tolerances on the upper bound of the distance to the solution ‖x-x‖ and ‖y-y‖, respectively. The bound is valid if λ>0 or σ>0 where σ should be strictly smaller than the smallest positive singular value. For instance σ:=(1-1e-7)σₘᵢₙ .\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\ntransfer_to_craig: transfer from the LNLQ point to the CRAIG point, when it exists. The transfer is based on the residual norm;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nσ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;\nutolx: tolerance on the upper bound on the distance to the solution ‖x-x*‖;\nutoly: tolerance on the upper bound on the distance to the solution ‖y-y*‖;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in a LNLQStats structure.\n\nReference\n\nR. Estrin, D. Orban, M.A. Saunders, LNLQ: An Iterative Method for Least-Norm Problems with an Error Minimization Property, SIAM Journal on Matrix Analysis and Applications, 40(3), pp. 1102–1124, 2019.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.lnlq!","page":"Least-norm problems","title":"Krylov.lnlq!","text":"solver = lnlq!(solver::LnlqSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lnlq.\n\nSee LnlqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#CRAIG","page":"Least-norm problems","title":"CRAIG","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"craig\ncraig!","category":"page"},{"location":"solvers/ln/#Krylov.craig","page":"Least-norm problems","title":"Krylov.craig","text":"(x, y, stats) = craig(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n transfer_to_lsqr::Bool=false, sqd::Bool=false,\n λ::T=zero(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nFind the least-norm solution of the consistent linear system\n\nAx + λ²y = b\n\nof size m × n using the Golub-Kahan implementation of Craig's method, where λ ≥ 0 is a regularization parameter. This method is equivalent to CGNE but is more stable.\n\nFor a system in the form Ax = b, Craig's method is equivalent to applying CG to AAᴴy = b and recovering x = Aᴴy. Note that y are the Lagrange multipliers of the least-norm problem\n\nminimize ‖x‖ s.t. Ax = b.\n\nIf λ > 0, CRAIG solves the symmetric and quasi-definite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A λ²E ] [ y ] = [ b ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. CRAIG is then equivalent to applying CG to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.\n\nIf λ = 0, CRAIG solves the symmetric and indefinite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A 0 ] [ y ] = [ b ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖x‖²_F s.t. Ax = b.\n\nIn this case, M can still be specified and indicates the weighted norm in which residuals are measured.\n\nIn this implementation, both the x and y-parts of the solution are returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\ntransfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nJ. E. Craig, The N-step iteration procedures, Journal of Mathematics and Physics, 34(1-4), pp. 64–73, 1955.\nC. C. Paige and M. A. Saunders, LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares, ACM Transactions on Mathematical Software, 8(1), pp. 43–71, 1982.\nM. A. Saunders, Solutions of Sparse Rectangular Systems Using LSQR and CRAIG, BIT Numerical Mathematics, 35(4), pp. 588–604, 1995.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.craig!","page":"Least-norm problems","title":"Krylov.craig!","text":"solver = craig!(solver::CraigSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of craig.\n\nSee CraigSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#CRAIGMR","page":"Least-norm problems","title":"CRAIGMR","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"craigmr\ncraigmr!","category":"page"},{"location":"solvers/ln/#Krylov.craigmr","page":"Least-norm problems","title":"Krylov.craigmr","text":"(x, y, stats) = craigmr(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n sqd::Bool=false, λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the consistent linear system\n\nAx + λ²y = b\n\nof size m × n using the CRAIGMR method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying the Conjugate Residuals method to the normal equations of the second kind\n\n(AAᴴ + λ²I) y = b\n\nbut is more stable. When λ = 0, this method solves the minimum-norm problem\n\nmin ‖x‖ s.t. x ∈ argmin ‖Ax - b‖.\n\nIf λ > 0, CRAIGMR solves the symmetric and quasi-definite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A λ²E ] [ y ] = [ b ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. CRAIGMR is then equivalent to applying MINRES to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.\n\nIf λ = 0, CRAIGMR solves the symmetric and indefinite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A 0 ] [ y ] = [ b ].\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F s.t. Ax = b.\n\nIn this case, M can still be specified and indicates the weighted norm in which residuals are measured.\n\nCRAIGMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRMR, though can be slightly more accurate, and intricate to implement. Both the x- and y-parts of the solution are returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nD. Orban and M. Arioli. Iterative Solution of Symmetric Quasi-Definite Linear Systems, Volume 3 of Spotlights. SIAM, Philadelphia, PA, 2017.\nD. Orban, The Projected Golub-Kahan Process for Constrained, Linear Least-Squares Problems. Cahier du GERAD G-2014-15, 2014.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.craigmr!","page":"Least-norm problems","title":"Krylov.craigmr!","text":"solver = craigmr!(solver::CraigmrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of craigmr.\n\nSee CraigmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#USYMLQ","page":"Least-norm problems","title":"USYMLQ","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"usymlq\nusymlq!","category":"page"},{"location":"solvers/ln/#Krylov.usymlq","page":"Least-norm problems","title":"Krylov.usymlq","text":"(x, stats) = usymlq(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n transfer_to_usymcg::Bool=true, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = usymlq(A, b, c, x0::AbstractVector; kwargs...)\n\nUSYMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nUSYMLQ determines the least-norm solution of the consistent linear system Ax = b of size m × n.\n\nUSYMLQ is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The error norm ‖x - x*‖ monotonously decreases in USYMLQ. When A is Hermitian and b = c, USYMLQ is equivalent to SYMMLQ. USYMLQ is considered as a generalization of SYMMLQ.\n\nIt can also be applied to under-determined and over-determined problems. In all cases, problems must be consistent.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\ntransfer_to_usymcg: transfer from the USYMLQ point to the USYMCG point, when it exists. The transfer is based on the residual norm;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. A. Saunders, H. D. Simon, and E. L. Yip, Two Conjugate-Gradient-Type Methods for Unsymmetric Linear Equations, SIAM Journal on Numerical Analysis, 25(4), pp. 927–940, 1988.\nA. Buttari, D. Orban, D. Ruiz and D. Titley-Peloquin, A tridiagonalization method for symmetric saddle-point and quasi-definite systems, SIAM Journal on Scientific Computing, 41(5), pp. 409–432, 2019.\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.usymlq!","page":"Least-norm problems","title":"Krylov.usymlq!","text":"solver = usymlq!(solver::UsymlqSolver, A, b, c; kwargs...)\nsolver = usymlq!(solver::UsymlqSolver, A, b, c, x0; kwargs...)\n\nwhere kwargs are keyword arguments of usymlq.\n\nSee UsymlqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"","category":"page"},{"location":"processes/#krylov-processes","page":"Krylov processes","title":"Krylov processes","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Krylov processes are the foundation of Krylov methods, they generate bases of Krylov subspaces. Depending on the Krylov subspaces generated, Krylov processes are more or less specialized for a subset of linear problems. The following table summarizes the most relevant processes for each linear problem.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Linear problems Processes\nHermitian linear systems Hermitian Lanczos\nSquare Non-Hermitian linear systems Non-Hermitian Lanczos – Arnoldi\nLeast-squares problems Golub-Kahan – Saunders-Simon-Yip\nLeast-norm problems Golub-Kahan – Saunders-Simon-Yip\nSaddle-point and Hermitian quasi-definite systems Golub-Kahan – Saunders-Simon-Yip\nGeneralized saddle-point and non-Hermitian partitioned systems Montoison-Orban","category":"page"},{"location":"processes/#Notation","page":"Krylov processes","title":"Notation","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"For a matrix A, A^H denotes the conjugate transpose of A. It coincides with A^T, the transpose of A, for real matrices. Define V_k = beginbmatrix v_1 ldots v_k endbmatrix enspace and enspace U_k = beginbmatrix u_1 ldots u_k endbmatrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"For a matrix C in mathbbC^n times n and a vector t in mathbbC^n, the k-th Krylov subspace generated by C and t is","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"mathcalK_k(C t) =\nleftsum_i=0^k-1 omega_i C^i t middle vert omega_i in mathbbC0 le i le k-1 right","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"For matrices C in mathbbC^n times n enspace and enspace T in mathbbC^n times p, the k-th block Krylov subspace generated by C and T is","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"mathcalK_k^square(C T) =\nleftsum_i=0^k-1 C^i T Omega_i middle vert Omega_i in mathbbC^p times p0 le i le k-1 right","category":"page"},{"location":"processes/#hermitian-lanczos","page":"Krylov processes","title":"Hermitian Lanczos","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: hermitian_lanczos)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Hermitian Lanczos process, the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = V_k T_k + beta_k+1k v_k+1 e_k^T = V_k+1 T_k+1k \n V_k^H V_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k is an orthonormal basis of the Krylov subspace mathcalK_k(Ab),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"T_k =\nbeginbmatrix\n alpha_1 barbeta_2 \n beta_2 alpha_2 ddots \n ddots ddots barbeta_k \n beta_k alpha_k\nendbmatrix\n qquad\nT_k+1k =\nbeginbmatrix\n T_k \n beta_k+1 e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Note that depending on how we normalize the vectors that compose V_k, T_k+1k can be a real tridiagonal matrix even if A is a complex matrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function hermitian_lanczos returns V_k+1, beta_1 and T_k+1k.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: SYMMLQ, CG, CR, CAR, MINRES, MINRES-QLP, MINARES, CGLS, CRLS, CGNE, CRMR, CG-LANCZOS and CG-LANCZOS-SHIFT.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"hermitian_lanczos(::Any, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.hermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.hermitian_lanczos","text":"V, β, T = hermitian_lanczos(A, b, k)\n\nInput arguments\n\nA: a linear operator that models an Hermitian matrix of dimension n;\nb: a vector of length n;\nk: the number of iterations of the Hermitian Lanczos process.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nT: a sparse (k+1) × k tridiagonal matrix.\n\nReference\n\nC. Lanczos, An Iteration Method for the Solution of the Eigenvalue Problem of Linear Differential and Integral Operators, Journal of Research of the National Bureau of Standards, 45(4), pp. 225–280, 1950.\n\n\n\n\n\n","category":"method"},{"location":"processes/#nonhermitian-lanczos","page":"Krylov processes","title":"Non-Hermitian Lanczos","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: nonhermitian_lanczos)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the non-Hermitian Lanczos process (also named the Lanczos biorthogonalization process), the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = V_k T_k + beta_k+1 v_k+1 e_k^T = V_k+1 T_k+1k \n A^H U_k = U_k T_k^H + bargamma_k+1 u_k+1 e_k^T = U_k+1 T_kk+1^H \n V_k^H U_k = U_k^H V_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k and U_k are bases of the Krylov subspaces mathcalK_k (Ab) and mathcalK_k (A^Hc), respectively,","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"T_k = \nbeginbmatrix\n alpha_1 gamma_2 \n beta_2 alpha_2 ddots \n ddots ddots gamma_k \n beta_k alpha_k\nendbmatrix\n qquad\nT_k+1k =\nbeginbmatrix\n T_k \n beta_k+1 e_k^T\nendbmatrix\n qquad\nT_kk+1 =\nbeginbmatrix\n T_k gamma_k+1 e_k\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function nonhermitian_lanczos returns V_k+1, beta_1, T_k+1k, U_k+1, bargamma_1 and T_kk+1^H.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: BiLQ, QMR, BiLQR, CGS and BICGSTAB.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe scaling factors used in our implementation are beta_k = u_k^H v_k^tfrac12 and gamma_k = (u_k^H v_k) beta_k. With these scaling factors, the non-Hermitian Lanczos process coincides with the Hermitian Lanczos process when A = A^H and b = c.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"nonhermitian_lanczos(::Any, ::AbstractVector{FC}, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.nonhermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.nonhermitian_lanczos","text":"V, β, T, U, γᴴ, Tᴴ = nonhermitian_lanczos(A, b, c, k)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nb: a vector of length n;\nc: a vector of length n;\nk: the number of iterations of the non-Hermitian Lanczos process.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nT: a sparse (k+1) × k tridiagonal matrix;\nU: a dense n × (k+1) matrix;\nγᴴ: a coefficient such that γᴴu₁ = c;\nTᴴ: a sparse (k+1) × k tridiagonal matrix.\n\nReference\n\nC. Lanczos, An Iteration Method for the Solution of the Eigenvalue Problem of Linear Differential and Integral Operators, Journal of Research of the National Bureau of Standards, 45(4), pp. 225–280, 1950.\n\n\n\n\n\n","category":"method"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The non-Hermitian Lanczos process can be also implemented without A^H (transpose-free variant). To derive it, we can observe that beta_k+1 v_k+1 = P_k(A) b and bargamma_k+1 u_k+1 = Q_k(A^H) c where P_k and Q_k are polynomials of degree k. The polynomials are defined from the recursions:","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n P_0(A) = I_n \n P_1(A) = left(dfracA - alpha_1 I_nbeta_1right) P_0(A) \n P_k(A) = left(dfracA - alpha_k I_nbeta_kright) P_k-1(A) - dfracgamma_kbeta_k-1 P_k-2(A) quad k ge 2 \n \n Q_0(A^H) = I_n \n Q_1(A^H) = left(dfracA^H - baralpha_1 I_nbargamma_1right) Q_0(A^H) \n Q_k(A^H) = left(dfracA^H - baralpha_k I_nbargamma_kright) Q_k-1(A^H) - dfracbarbeta_kbargamma_k-1 Q_k-2(A^H) quad k ge 2\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Because alpha_k = u_k^H A v_k and (bargamma_k+1 u_k+1)^H (beta_k+1 v_k+1) = gamma_k+1 beta_k+1, we can determine the coefficients of T_k+1k and T_kk+1^H as follows:","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n alpha_k = dfrac1gamma_k beta_k langleQ_k-1(A^H) c A P_k-1(A) brangle \n = dfrac1gamma_k beta_k langlec barQ_k-1(A) A P_k-1(A) brangle \n \n beta_k+1 gamma_k+1 = langleQ_k(A^H) c P_k(A) brangle \n = langlec barQ_k(A) P_k(A) brangle\nendalign*","category":"page"},{"location":"processes/#arnoldi","page":"Krylov processes","title":"Arnoldi","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: arnoldi)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Arnoldi process, the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = V_k H_k + h_k+1k v_k+1 e_k^T = V_k+1 H_k+1k \n V_k^H V_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k is an orthonormal basis of the Krylov subspace mathcalK_k (Ab),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"H_k =\nbeginbmatrix\n h_11 h_12 ldots h_1k \n h_21 ddots ddots vdots \n ddots ddots h_k-1k \n h_kk-1 h_kk\nendbmatrix\n qquad\nH_k+1k =\nbeginbmatrix\n H_k \n h_k+1k e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function arnoldi returns V_k+1, beta and H_k+1k.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: DIOM, FOM, DQGMRES, GMRES and FGMRES.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Arnoldi process coincides with the Hermitian Lanczos process when A is Hermitian.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"arnoldi(::Any, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.arnoldi-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.arnoldi","text":"V, β, H = arnoldi(A, b, k; reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nb: a vector of length n;\nk: the number of iterations of the Arnoldi process.\n\nKeyword argument\n\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nH: a dense (k+1) × k upper Hessenberg matrix.\n\nReference\n\nW. E. Arnoldi, The principle of minimized iterations in the solution of the matrix eigenvalue problem, Quarterly of Applied Mathematics, 9, pp. 17–29, 1951.\n\n\n\n\n\n","category":"method"},{"location":"processes/#golub-kahan","page":"Krylov processes","title":"Golub-Kahan","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: golub_kahan)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Golub-Kahan bidiagonalization process, the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = U_k+1 B_k \n A^H U_k+1 = V_k B_k^H + baralpha_k+1 v_k+1 e_k+1^T = V_k+1 L_k+1^H \n V_k^H V_k = U_k^H U_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k and U_k are bases of the Krylov subspaces mathcalK_k (A^HAA^Hb) and mathcalK_k (AA^Hb), respectively,","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"L_k =\nbeginbmatrix\n alpha_1 \n beta_2 alpha_2 \n ddots ddots \n beta_k alpha_k\nendbmatrix\n qquad\nB_k =\nbeginbmatrix\n alpha_1 \n beta_2 alpha_2 \n ddots ddots \n beta_k alpha_k \n beta_k+1 \nendbmatrix\n=\nbeginbmatrix\n L_k \n beta_k+1 e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Note that depending on how we normalize the vectors that compose V_k and U_k, L_k can be a real bidiagonal matrix even if A is a complex matrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function golub_kahan returns V_k+1, U_k+1, beta_1 and L_k+1.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: LNLQ, CRAIG, CRAIGMR, LSLQ, LSQR and LSMR.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Golub-Kahan process coincides with the Hermitian Lanczos process applied to the normal equations A^HA x = A^Hb and AA^H x = b. It is also related to the Hermitian Lanczos process applied to beginbmatrix 0 A A^H 0 endbmatrix with initial vector beginbmatrix b 0 endbmatrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"golub_kahan(::Any, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.golub_kahan-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.golub_kahan","text":"V, U, β, L = golub_kahan(A, b, k)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nk: the number of iterations of the Golub-Kahan process.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nU: a dense m × (k+1) matrix;\nβ: a coefficient such that βu₁ = b;\nL: a sparse (k+1) × (k+1) lower bidiagonal matrix.\n\nReferences\n\nG. H. Golub and W. Kahan, Calculating the Singular Values and Pseudo-Inverse of a Matrix, SIAM Journal on Numerical Analysis, 2(2), pp. 225–224, 1965.\nC. C. Paige, Bidiagonalization of Matrices and Solution of Linear Equations, SIAM Journal on Numerical Analysis, 11(1), pp. 197–209, 1974.\n\n\n\n\n\n","category":"method"},{"location":"processes/#saunders-simon-yip","page":"Krylov processes","title":"Saunders-Simon-Yip","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: saunders_simon_yip)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Saunders-Simon-Yip process (also named the orthogonal tridiagonalization process), the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A U_k = V_k T_k + beta_k+1 v_k+1 e_k^T = V_k+1 T_k+1k \n A^H V_k = U_k T_k^H + bargamma_k+1 u_k+1 e_k^T = U_k+1 T_kk+1^H \n V_k^H V_k = U_k^H U_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A A^H 0 endbmatrix beginbmatrix b 0 0 c endbmatrixright),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"T_k = \nbeginbmatrix\n alpha_1 gamma_2 \n beta_2 alpha_2 ddots \n ddots ddots gamma_k \n beta_k alpha_k\nendbmatrix\n qquad\nT_k+1k =\nbeginbmatrix\n T_k \n beta_k+1 e_k^T\nendbmatrix\n qquad\nT_kk+1 =\nbeginbmatrix\n T_k gamma_k+1 e_k\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function saunders_simon_yip returns V_k+1, beta_1, T_k+1k, U_k+1, bargamma_1 and T_kk+1^H.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: USYMLQ, USYMQR, TriLQR, TriCG and TriMR.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Saunders-Simon-Yip is equivalent to the block-Lanczos process applied to beginbmatrix 0 A A^H 0 endbmatrix with initial matrix beginbmatrix b 0 0 c endbmatrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"saunders_simon_yip(::Any, ::AbstractVector{FC}, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.saunders_simon_yip-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.saunders_simon_yip","text":"V, β, T, U, γᴴ, Tᴴ = saunders_simon_yip(A, b, c, k)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n;\nk: the number of iterations of the Saunders-Simon-Yip process.\n\nOutput arguments\n\nV: a dense m × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nT: a sparse (k+1) × k tridiagonal matrix;\nU: a dense n × (k+1) matrix;\nγᴴ: a coefficient such that γᴴu₁ = c;\nTᴴ: a sparse (k+1) × k tridiagonal matrix.\n\nReference\n\nM. A. Saunders, H. D. Simon, and E. L. Yip, Two Conjugate-Gradient-Type Methods for Unsymmetric Linear Equations, SIAM Journal on Numerical Analysis, 25(4), pp. 927–940, 1988.\n\n\n\n\n\n","category":"method"},{"location":"processes/#montoison-orban","page":"Krylov processes","title":"Montoison-Orban","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: montoison_orban)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Montoison-Orban process (also named the orthogonal Hessenberg reduction process), the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A U_k = V_k H_k + h_k+1k v_k+1 e_k^T = V_k+1 H_k+1k \n B V_k = U_k F_k + f_k+1k u_k+1 e_k^T = U_k+1 F_k+1k \n V_k^H V_k = U_k^H U_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A B 0 endbmatrix beginbmatrix b 0 0 c endbmatrixright),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"H_k =\nbeginbmatrix\n h_11 h_12 ldots h_1k \n h_21 ddots ddots vdots \n ddots ddots h_k-1k \n h_kk-1 h_kk\nendbmatrix\n qquad\nF_k =\nbeginbmatrix\n f_11 f_12 ldots f_1k \n f_21 ddots ddots vdots \n ddots ddots f_k-1k \n f_kk-1 f_kk\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"H_k+1k =\nbeginbmatrix\n H_k \n h_k+1k e_k^T\nendbmatrix\n qquad\nF_k+1k =\nbeginbmatrix\n F_k \n f_k+1k e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function montoison_orban returns V_k+1, beta, H_k+1k, U_k+1, gamma and F_k+1k.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: GPMR.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Montoison-Orban is equivalent to the block-Arnoldi process applied to beginbmatrix 0 A B 0 endbmatrix with initial matrix beginbmatrix b 0 0 c endbmatrix. It also coincides with the Saunders-Simon-Yip process when B = A^H.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"montoison_orban(::Any, ::Any, ::AbstractVector{FC}, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.montoison_orban-Union{Tuple{FC}, Tuple{Any, Any, AbstractVector{FC}, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.montoison_orban","text":"V, β, H, U, γ, F = montoison_orban(A, B, b, c, k; reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a linear operator that models a matrix of dimension n × m;\nb: a vector of length m;\nc: a vector of length n;\nk: the number of iterations of the Montoison-Orban process.\n\nKeyword argument\n\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors.\n\nOutput arguments\n\nV: a dense m × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nH: a dense (k+1) × k upper Hessenberg matrix;\nU: a dense n × (k+1) matrix;\nγ: a coefficient such that γu₁ = c;\nF: a dense (k+1) × k upper Hessenberg matrix.\n\nReference\n\nA. Montoison and D. Orban, GPMR: An Iterative Method for Unsymmetric Partitioned Linear Systems, SIAM Journal on Matrix Analysis and Applications, 44(1), pp. 293–311, 2023.\n\n\n\n\n\n","category":"method"},{"location":"examples/crls/","page":"CRLS","title":"CRLS","text":"using MatrixMarket, SuiteSparseMatrixCollection\nusing Krylov, LinearOperators\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"well1850\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1])_b.mtx\"))[:]\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\n# Define a regularization parameter.\nλ = 1.0e-3\n\n(x, stats) = crls(A, b, λ=λ)\nshow(stats)\nresid = norm(A' * (A * x - b) + λ * x) / norm(b)\n@printf(\"CRLS: Relative residual: %8.1e\\n\", resid)\n@printf(\"CRLS: ‖x‖: %8.1e\\n\", norm(x))","category":"page"},{"location":"callbacks/#callbacks","page":"Callbacks","title":"Callbacks","text":"","category":"section"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"Each Krylov method is able to call a callback function as callback(solver) at each iteration. The callback should return true if the main loop should terminate, and false otherwise. If the method terminated because of the callback, the output status will be \"user-requested exit\". For example, if the user defines minres_callback(solver::MinresSolver), it can be passed to the solver using","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"(x, stats) = minres(A, b, callback = minres_callback)","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"If you need to write a callback that uses variables that are not in a KrylovSolver, use a closure:","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"function custom_stopping_condition(solver::KrylovSolver, A, b, r, tol)\n mul!(r, A, solver.x)\n r .-= b # r := b - Ax\n bool = norm(r) ≤ tol # tolerance based on the 2-norm of the residual\n return bool\nend\n\ncg_callback(solver) = custom_stopping_condition(solver, A, b, r, tol)\n(x, stats) = cg(A, b, callback = cg_callback)","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"Alternatively, use a structure and make it callable:","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"mutable struct CallbackWorkspace{T}\n A::Matrix{T}\n b::Vector{T}\n r::Vector{T}\n tol::T\nend\n\nfunction (workspace::CallbackWorkspace)(solver::KrylovSolver)\n mul!(workspace.r, workspace.A, solver.x)\n workspace.r .-= workspace.b\n bool = norm(workspace.r) ≤ workspace.tol\n return bool\nend\n\nbicgstab_callback = CallbackWorkspace(A, b, r, tol)\n(x, stats) = bicgstab(A, b, callback = bicgstab_callback)","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"Although the main goal of a callback is to add new stopping conditions, it can also retrieve information from the workspace of a Krylov method along the iterations. We now illustrate how to store all iterates x_k of the GMRES method.","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"S = Krylov.ktypeof(b)\nglobal X = S[] # Storage for GMRES iterates\n\nfunction gmres_callback(solver)\n z = solver.z\n k = solver.inner_iter\n nr = sum(1:k)\n V = solver.V\n R = solver.R\n y = copy(z)\n\n # Solve Rk * yk = zk\n for i = k : -1 : 1\n pos = nr + i - k\n for j = k : -1 : i+1\n y[i] = y[i] - R[pos] * y[j]\n pos = pos - j + 1\n end\n y[i] = y[i] / R[pos]\n end\n\n # xk = Vk * yk\n xk = sum(V[i] * y[i] for i = 1:k)\n push!(X, xk)\n\n return false # We don't want to add new stopping conditions\nend\n\n(x, stats) = gmres(A, b, callback = gmres_callback)","category":"page"},{"location":"gpu/#gpu","page":"GPU support","title":"GPU support","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Krylov methods are well suited for GPU computations because they only require matrix-vector products (u leftarrow Av, u leftarrow A^Hw) and vector operations (v, u^H v, v leftarrow alpha u + beta v), which are highly parallelizable.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"The implementations in Krylov.jl are generic so as to take advantage of the multiple dispatch and broadcast features of Julia. Those allow the implementations to be specialized automatically by the compiler for both CPU and GPU. Thus, Krylov.jl works with GPU backends that build on GPUArrays.jl, such as CUDA.jl, AMDGPU.jl, oneAPI.jl or Metal.jl.","category":"page"},{"location":"gpu/#Nvidia-GPUs","page":"GPU support","title":"Nvidia GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with CUDA.jl and allow computations on Nvidia GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (CuMatrix and CuVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using CUDA, Krylov\n\nif CUDA.functional()\n # CPU Arrays\n A_cpu = rand(20, 20)\n b_cpu = rand(20)\n\n # GPU Arrays\n A_gpu = CuMatrix(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # Solve a square and dense system on an Nivida GPU\n x, stats = bilq(A_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Sparse matrices have a specific storage on Nvidia GPUs (CuSparseMatrixCSC, CuSparseMatrixCSR or CuSparseMatrixCOO):","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using CUDA, Krylov\nusing CUDA.CUSPARSE, SparseArrays\n\nif CUDA.functional()\n # CPU Arrays\n A_cpu = sprand(200, 100, 0.3)\n b_cpu = rand(200)\n\n # GPU Arrays\n A_csc_gpu = CuSparseMatrixCSC(A_cpu)\n A_csr_gpu = CuSparseMatrixCSR(A_cpu)\n A_coo_gpu = CuSparseMatrixCOO(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # Solve a rectangular and sparse system on an Nvidia GPU\n x_csc, stats_csc = lslq(A_csc_gpu, b_gpu)\n x_csr, stats_csr = lsqr(A_csr_gpu, b_gpu)\n x_coo, stats_coo = lsmr(A_coo_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"If you use a Krylov method that only requires A * v products (see here), the most efficient format is CuSparseMatrixCSR. Optimized operator-vector and operator-matrix products that exploit GPU features can be also used by means of linear operators.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"For instance, when executing sparse matrix products on NVIDIA GPUs, the mul! function utilized by Krylov.jl internally calls three routines from CUSPARSE. The first one conducts an analysis of the sparse matrix's structure, the second one determines the size of the buffer that needs to be allocated, and the last one handles the product using the allocated buffer. To circumvent the need for repeated analysis computations and buffer allocation/deallocation with each product, we introduce a KrylovOperator in KrylovPreconditioners.jl. This operator performs the analysis and allocates the buffer only once.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, KrylovPreconditioners\n\n# A_gpu can be a sparse COO, CSC or CSR matrix\nopA_gpu = KrylovOperator(A_gpu)\nx_gpu, stats = gmres(opA_gpu, b_gpu)","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Preconditioners, especially incomplete Cholesky or incomplete LU factorizations that involve sparse triangular solves, can be applied directly on GPU thanks to efficient operators (like TriangularOperator) that take advantage of CUSPARSE routines.","category":"page"},{"location":"gpu/#Example-with-a-symmetric-positive-definite-system","page":"GPU support","title":"Example with a symmetric positive-definite system","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using SparseArrays, Krylov, LinearOperators\nusing CUDA, CUDA.CUSPARSE\n\nif CUDA.functional()\n # Transfer the linear system from the CPU to the GPU\n A_gpu = CuSparseMatrixCSR(A_cpu) # A_gpu = CuSparseMatrixCSC(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # IC(0) decomposition LLᴴ ≈ A for CuSparseMatrixCSC or CuSparseMatrixCSR matrices\n P = ic02(A_gpu)\n\n # Additional vector required for solving triangular systems\n n = length(b_gpu)\n T = eltype(b_gpu)\n z = CUDA.zeros(T, n)\n\n # Solve Py = x\n function ldiv_ic0!(P::CuSparseMatrixCSR, x, y, z)\n ldiv!(z, LowerTriangular(P), x) # Forward substitution with L\n ldiv!(y, LowerTriangular(P)', z) # Backward substitution with Lᴴ\n return y\n end\n\n function ldiv_ic0!(P::CuSparseMatrixCSC, x, y, z)\n ldiv!(z, UpperTriangular(P)', x) # Forward substitution with L\n ldiv!(y, UpperTriangular(P), z) # Backward substitution with Lᴴ\n return y\n end\n\n # Operator that model P⁻¹\n symmetric = hermitian = true\n opM = LinearOperator(T, n, n, symmetric, hermitian, (y, x) -> ldiv_ic0!(P, x, y, z))\n\n # Solve an Hermitian positive definite system with an IC(0) preconditioner on GPU\n x, stats = cg(A_gpu, b_gpu, M=opM)\nend","category":"page"},{"location":"gpu/#Example-with-a-general-square-system","page":"GPU support","title":"Example with a general square system","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using SparseArrays, Krylov, LinearOperators\nusing CUDA, CUDA.CUSPARSE, CUDA.CUSOLVER\n\nif CUDA.functional()\n # Optional -- Compute a permutation vector p such that A[:,p] has no zero diagonal\n p = zfd(A_cpu)\n p .+= 1\n A_cpu = A_cpu[:,p]\n\n # Transfer the linear system from the CPU to the GPU\n A_gpu = CuSparseMatrixCSR(A_cpu) # A_gpu = CuSparseMatrixCSC(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # ILU(0) decomposition LU ≈ A for CuSparseMatrixCSC or CuSparseMatrixCSR matrices\n P = ilu02(A_gpu)\n\n # Additional vector required for solving triangular systems\n n = length(b_gpu)\n T = eltype(b_gpu)\n z = CUDA.zeros(T, n)\n\n # Solve Py = x\n function ldiv_ilu0!(P::CuSparseMatrixCSR, x, y, z)\n ldiv!(z, UnitLowerTriangular(P), x) # Forward substitution with L\n ldiv!(y, UpperTriangular(P), z) # Backward substitution with U\n return y\n end\n\n function ldiv_ilu0!(P::CuSparseMatrixCSC, x, y, z)\n ldiv!(z, LowerTriangular(P), x) # Forward substitution with L\n ldiv!(y, UnitUpperTriangular(P), z) # Backward substitution with U\n return y\n end\n\n # Operator that model P⁻¹\n symmetric = hermitian = false\n opM = LinearOperator(T, n, n, symmetric, hermitian, (y, x) -> ldiv_ilu0!(P, x, y, z))\n\n # Solve a non-Hermitian system with an ILU(0) preconditioner on GPU\n x̄, stats = bicgstab(A_gpu, b_gpu, M=opM)\n\n # Recover the solution of Ax = b with the solution of A[:,p]x̄ = b\n invp = invperm(p)\n x = x̄[invp]\nend","category":"page"},{"location":"gpu/#AMD-GPUs","page":"GPU support","title":"AMD GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with AMDGPU.jl and allow computations on AMD GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (ROCMatrix and ROCVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, AMDGPU\n\nif AMDGPU.functional()\n # CPU Arrays\n A_cpu = rand(ComplexF64, 20, 20)\n A_cpu = A_cpu + A_cpu'\n b_cpu = rand(ComplexF64, 20)\n\n A_gpu = ROCMatrix(A_cpu)\n b_gpu = ROCVector(b_cpu)\n\n # Solve a dense Hermitian system on an AMD GPU\n x, stats = minres(A_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Sparse matrices have a specific storage on AMD GPUs (ROCSparseMatrixCSC, ROCSparseMatrixCSR or ROCSparseMatrixCOO):","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using AMDGPU, Krylov\nusing AMDGPU.rocSPARSE, SparseArrays\n\nif AMDGPU.functional()\n # CPU Arrays\n A_cpu = sprand(100, 200, 0.3)\n b_cpu = rand(100)\n\n # GPU Arrays\n A_csc_gpu = ROCSparseMatrixCSC(A_cpu)\n A_csr_gpu = ROCSparseMatrixCSR(A_cpu)\n A_coo_gpu = ROCSparseMatrixCOO(A_cpu)\n b_gpu = ROCVector(b_cpu)\n\n # Solve a rectangular and sparse system on an AMD GPU\n x_csc, y_csc, stats_csc = lnlq(A_csc_gpu, b_gpu)\n x_csr, y_csr, stats_csr = craig(A_csr_gpu, b_gpu)\n x_coo, y_coo, stats_coo = craigmr(A_coo_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"When executing sparse products on AMD GPUs, the mul! function utilized by Krylov.jl internally calls three routines from rocSPARSE. The first one conducts an analysis of the sparse matrix's structure, the second one determines the size of the buffer that needs to be allocated, and the last one handles the product using the allocated buffer. To circumvent the need for repeated analysis computations and buffer allocation/deallocation with each product, we introduce a KrylovOperator in KrylovPreconditioners.jl. This operator performs the analysis and allocates the buffer only once.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, KrylovPreconditioners\n\n# A_gpu can be a sparse COO, CSC or CSR matrix\nopA_gpu = KrylovOperator(A_gpu)\nx_gpu, stats = bicgstab(opA_gpu, b_gpu)","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Preconditioners, especially incomplete Cholesky or incomplete LU factorizations that involve sparse triangular solves, can be applied directly on GPU thanks to efficient operators (like TriangularOperator) that take advantage of rocSPARSE routines.","category":"page"},{"location":"gpu/#Intel-GPUs","page":"GPU support","title":"Intel GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with oneAPI.jl and allow computations on Intel GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (oneMatrix and oneVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, oneAPI\n\nif oneAPI.functional()\n T = Float32 # oneAPI.jl also works with ComplexF32\n m = 20\n n = 10\n\n # CPU Arrays\n A_cpu = rand(T, m, n)\n b_cpu = rand(T, m)\n\n # GPU Arrays\n A_gpu = oneMatrix(A_cpu)\n b_gpu = oneVector(b_cpu)\n\n # Solve a dense least-squares problem on an Intel GPU\n x, stats = lsqr(A_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"note: Note\nThe library oneMKL is interfaced in oneAPI.jl and accelerates linear algebra operations on Intel GPUs. Only dense linear systems are supported for the time being because sparse linear algebra routines are not interfaced yet.","category":"page"},{"location":"gpu/#Apple-M1-GPUs","page":"GPU support","title":"Apple M1 GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with Metal.jl and allow computations on Apple M1 GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (MtlMatrix and MtlVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, Metal\n\nT = Float32 # Metal.jl also works with ComplexF32\nn = 10\nm = 20\n\n# CPU Arrays\nA_cpu = rand(T, n, m)\nb_cpu = rand(T, n)\n\n# GPU Arrays\nA_gpu = MtlMatrix(A_cpu)\nb_gpu = MtlVector(b_cpu)\n\n# Solve a dense least-norm problem on an Apple M1 GPU\nx, stats = craig(A_gpu, b_gpu)","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"warning: Warning\nMetal.jl is under heavy development and is considered experimental for now.","category":"page"},{"location":"#Home","page":"Home","title":"Krylov.jl documentation","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"This package provides implementations of certain of the most useful Krylov method for a variety of problems:","category":"page"},{"location":"","page":"Home","title":"Home","text":"1 - Square or rectangular full-rank systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" Ax = b","category":"page"},{"location":"","page":"Home","title":"Home","text":"should be solved when b lies in the range space of A. This situation occurs when","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is square and nonsingular,\nA is tall and has full column rank and b lies in the range of A.","category":"page"},{"location":"","page":"Home","title":"Home","text":"2 - Linear least-squares problems","category":"page"},{"location":"","page":"Home","title":"Home","text":" min b - Ax","category":"page"},{"location":"","page":"Home","title":"Home","text":"should be solved when b is not in the range of A (inconsistent systems), regardless of the shape and rank of A. This situation mainly occurs when","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is square and singular,\nA is tall and thin.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Underdetermined systems are less common but also occur.","category":"page"},{"location":"","page":"Home","title":"Home","text":"If there are infinitely many such x (because A is column rank-deficient), one with minimum norm is identified","category":"page"},{"location":"","page":"Home","title":"Home","text":" min x quad textsubject to quad x in argmin b - Ax","category":"page"},{"location":"","page":"Home","title":"Home","text":"3 - Linear least-norm problems","category":"page"},{"location":"","page":"Home","title":"Home","text":" min x quad textsubject to quad Ax = b","category":"page"},{"location":"","page":"Home","title":"Home","text":"should be solved when A is column rank-deficient but b is in the range of A (consistent systems), regardless of the shape of A. This situation mainly occurs when","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is square and singular,\nA is short and wide.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Overdetermined systems are less common but also occur.","category":"page"},{"location":"","page":"Home","title":"Home","text":"4 - Adjoint systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" Ax = b quad textand quad A^H y = c","category":"page"},{"location":"","page":"Home","title":"Home","text":"where A can have any shape.","category":"page"},{"location":"","page":"Home","title":"Home","text":"5 - Saddle-point and Hermitian quasi-definite systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" beginbmatrix M phantom-A A^H -N endbmatrix beginbmatrix x y endbmatrix = left(beginbmatrix b 0 endbmatrixbeginbmatrix 0 c endbmatrixbeginbmatrix b c endbmatrixright)","category":"page"},{"location":"","page":"Home","title":"Home","text":"where A can have any shape.","category":"page"},{"location":"","page":"Home","title":"Home","text":"6 - Generalized saddle-point and non-Hermitian partitioned systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" beginbmatrix M A B N endbmatrix beginbmatrix x y endbmatrix = beginbmatrix b c endbmatrix","category":"page"},{"location":"","page":"Home","title":"Home","text":"where A can have any shape and B has the shape of Aᴴ. A, B, b and c must be all nonzero.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Krylov solvers are particularly appropriate in situations where such problems must be solved but a factorization is not possible, either because:","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is not available explicitly,\nA would be dense or would consume an excessive amount of memory if it were materialized,\nfactors would consume an excessive amount of memory.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Iterative methods are recommended in either of the following situations:","category":"page"},{"location":"","page":"Home","title":"Home","text":"the problem is sufficiently large that a factorization is not feasible or would be slow,\nan effective preconditioner is known in cases where the problem has unfavorable spectral structure,\nthe operator can be represented efficiently as a sparse matrix,\nthe operator is fast, i.e., can be applied with better complexity than if it were materialized as a matrix. Certain fast operators would materialize as dense matrices.","category":"page"},{"location":"#Features","page":"Home","title":"Features","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"All solvers in Krylov.jl have in-place version, are compatible with GPU and work in any floating-point data type.","category":"page"},{"location":"#How-to-Install","page":"Home","title":"How to Install","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Krylov can be installed and tested through the Julia package manager:","category":"page"},{"location":"","page":"Home","title":"Home","text":"julia> ]\npkg> add Krylov\npkg> test Krylov","category":"page"},{"location":"#Bug-reports-and-discussions","page":"Home","title":"Bug reports and discussions","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.","category":"page"},{"location":"","page":"Home","title":"Home","text":"If you want to ask a question not suited for a bug report, feel free to start a discussion here. This forum is for general discussion about this repository and the JuliaSmoothOptimizers organization, so questions about any of our packages are welcome.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"","category":"page"},{"location":"matrix_free/#matrix-free","page":"Matrix-free operators","title":"Matrix-free operators","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"All methods are matrix-free, which means that you only need to provide operator-vector products.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The A or B input arguments of Krylov.jl solvers can be any object that represents a linear operator. That object must implement mul!, for multiplication with a vector, size() and eltype(). For certain methods it must also implement adjoint().","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Some methods only require A * v products, whereas other ones also require A' * u products. In the latter case, adjoint(A) must also be implemented.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"A * v A * v and A' * u\nCG, CR, CAR CGLS, CRLS, CGNE, CRMR\nSYMMLQ, CG-LANCZOS, MINRES, MINRES-QLP, MINARES LSLQ, LSQR, LSMR, LNLQ, CRAIG, CRAIGMR\nDIOM, FOM, DQGMRES, GMRES, FGMRES, BLOCK-GMRES BiLQ, QMR, BiLQR, USYMLQ, USYMQR, TriLQR\nCGS, BICGSTAB TriCG, TriMR","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"info: Info\nGPMR is the only method that requires A * v and B * w products.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Preconditioners M, N, C, D, E or F can be also linear operators and must implement mul! or ldiv!.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"We strongly recommend LinearOperators.jl to model matrix-free operators, but other packages such as LinearMaps.jl, DiffEqOperators.jl or your own operator can be used as well.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"With LinearOperators.jl, operators are defined as","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"A = LinearOperator(type, nrows, ncols, symmetric, hermitian, prod, tprod, ctprod)","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"where","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"type is the operator element type;\nnrow and ncol are its dimensions;\nsymmetric and hermitian should be set to true or false;\nprod(y, v), tprod(y, w) and ctprod(u, w) are called when writing mul!(y, A, v), mul!(y, transpose(A), w), and mul!(y, A', u), respectively.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"See the tutorial and the detailed documentation for more information on LinearOperators.jl.","category":"page"},{"location":"matrix_free/#Examples","page":"Matrix-free operators","title":"Examples","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In the field of nonlinear optimization, finding critical points of a continuous function frequently involves linear systems with a Hessian or Jacobian as coefficient. Materializing such operators as matrices is expensive in terms of operations and memory consumption and is unreasonable for high-dimensional problems. However, it is often possible to implement efficient Hessian-vector and Jacobian-vector products, for example with the help of automatic differentiation tools, and used within Krylov solvers. We now illustrate variants with explicit matrices and with matrix-free operators for two well-known optimization methods.","category":"page"},{"location":"matrix_free/#Example-1:-Newton's-Method-for-convex-optimization","page":"Matrix-free operators","title":"Example 1: Newton's Method for convex optimization","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"At each iteration of Newton's method applied to a mathcalC^2 strictly convex function f mathbbR^n rightarrow mathbbR, a descent direction direction is determined by minimizing the quadratic Taylor model of f:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"min_d in mathbbR^nf(x_k) + nabla f(x_k)^T d + tfrac12d^T nabla^2 f(x_k) d","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"which is equivalent to solving the symmetric and positive-definite system","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"nabla^2 f(x_k) d = -nabla f(x_k)","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The system above can be solved with the conjugate gradient method as follows, using the explicit Hessian:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using ForwardDiff, Krylov\n\nxk = -ones(4)\n\nf(x) = (x[1] - 1)^2 + (x[2] - 2)^2 + (x[3] - 3)^2 + (x[4] - 4)^2\n\ng(x) = ForwardDiff.gradient(f, x)\n\nH(x) = ForwardDiff.hessian(f, x)\n\nd, stats = cg(H(xk), -g(xk))","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The explicit Hessian can be replaced by a linear operator that only computes Hessian-vector products:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using ForwardDiff, LinearOperators, Krylov\n\nxk = -ones(4)\n\nf(x) = (x[1] - 1)^2 + (x[2] - 2)^2 + (x[3] - 3)^2 + (x[4] - 4)^2\n\ng(x) = ForwardDiff.gradient(f, x)\n\nH(y, v) = ForwardDiff.derivative!(y, t -> g(xk + t * v), 0)\nopH = LinearOperator(Float64, 4, 4, true, true, (y, v) -> H(y, v))\n\ncg(opH, -g(xk))","category":"page"},{"location":"matrix_free/#Example-2:-The-Gauss-Newton-Method-for-Nonlinear-Least-Squares","page":"Matrix-free operators","title":"Example 2: The Gauss-Newton Method for Nonlinear Least Squares","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"At each iteration of the Gauss-Newton method applied to a nonlinear least-squares objective f(x) = tfrac12 F(x)^2 where F mathbbR^n rightarrow mathbbR^m is mathcalC^1, we solve the subproblem:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"min_d in mathbbR^ntfrac12J(x_k) d + F(x_k)^2","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"where J(x) is the Jacobian of F at x.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"An appropriate iterative method to solve the above linear least-squares problems is LSMR. We could pass the explicit Jacobian to LSMR as follows:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using ForwardDiff, Krylov\n\nxk = ones(2)\n\nF(x) = [x[1]^4 - 3; exp(x[2]) - 2; log(x[1]) - x[2]^2]\n\nJ(x) = ForwardDiff.jacobian(F, x)\n\nd, stats = lsmr(J(xk), -F(xk))","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"However, the explicit Jacobian can be replaced by a linear operator that only computes Jacobian-vector and transposed Jacobian-vector products:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using LinearAlgebra, ForwardDiff, LinearOperators, Krylov\n\nxk = ones(2)\n\nF(x) = [x[1]^4 - 3; exp(x[2]) - 2; log(x[1]) - x[2]^2]\n\nJ(y, v) = ForwardDiff.derivative!(y, t -> F(xk + t * v), 0)\nJᵀ(y, u) = ForwardDiff.gradient!(y, x -> dot(F(x), u), xk)\nopJ = LinearOperator(Float64, 3, 2, false, false, (y, v) -> J(y, v),\n (y, w) -> Jᵀ(y, w),\n (y, u) -> Jᵀ(y, u))\n\nlsmr(opJ, -F(xk))","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Note that preconditioners can be also implemented as abstract operators. For instance, we could compute the Cholesky factorization of M and N and create linear operators that perform the forward and backsolves.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Krylov methods combined with factorization free operators allow to reduce computation time and memory requirements considerably by avoiding building and storing the system matrix. In the field of partial differential equations, the implementation of high-performance factorization free operators and assembly free preconditioning is a subject of active research.","category":"page"},{"location":"examples/cgls/","page":"CGLS","title":"CGLS","text":"using MatrixMarket, SuiteSparseMatrixCollection\nusing Krylov, LinearOperators\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"well1033\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1])_b.mtx\"))[:]\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\n# Define a regularization parameter.\nλ = 1.0e-3\n\n(x, stats) = cgls(A, b, λ=λ)\nshow(stats)\nresid = norm(A' * (A * x - b) + λ * x) / norm(b)\n@printf(\"CGLS: Relative residual: %8.1e\\n\", resid)\n@printf(\"CGLS: ‖x‖: %8.1e\\n\", norm(x))","category":"page"}]
}
diff --git a/dev/solvers/as/index.html b/dev/solvers/as/index.html
index 30f1ab0f9..52b2d8270 100644
--- a/dev/solvers/as/index.html
+++ b/dev/solvers/as/index.html
@@ -1,14 +1,14 @@
-Adjoint systems · Krylov.jl
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
(x, y, stats) = bilqr(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)
BiLQR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.
Combine BiLQ and QMR to solve adjoint systems.
[0 A] [y] = [b]
-[Aᴴ 0] [x] [c]
The relation bᴴc ≠ 0 must be satisfied. BiLQ is used for solving primal system Ax = b of size n. QMR is used for solving dual system Aᴴy = c of size n.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n;
c: a vector of length n.
Optional arguments
x0: a vector of length n that represents an initial guess of the solution x;
y0: a vector of length n that represents an initial guess of the solution y.
Keyword arguments
transfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
y: a dense vector of length n;
stats: statistics collected on the run in an AdjointStats structure.
The relation bᴴc ≠ 0 must be satisfied. BiLQ is used for solving primal system Ax = b of size n. QMR is used for solving dual system Aᴴy = c of size n.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n;
c: a vector of length n.
Optional arguments
x0: a vector of length n that represents an initial guess of the solution x;
y0: a vector of length n that represents an initial guess of the solution y.
Keyword arguments
transfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
y: a dense vector of length n;
stats: statistics collected on the run in an AdjointStats structure.
GPMR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.
Given matrices A of dimension m × n and B of dimension n × m, GPMR solves the non-Hermitian partitioned linear system
[ λIₘ A ] [ x ] = [ b ]
[ B μIₙ ] [ y ] [ c ],
of size (n+m) × (n+m) where λ and μ are real or complex numbers. A can have any shape and B has the shape of Aᴴ. A, B, b and c must be all nonzero.
This implementation allows left and right block diagonal preconditioners
[ C ] [ λM A ] [ E ] [ E⁻¹x ] = [ Cb ]
[ D ] [ B μN ] [ F ] [ F⁻¹y ] [ Dc ],
and can solve
[ λM A ] [ x ] = [ b ]
-[ B μN ] [ y ] [ c ]
when CE = M⁻¹ and DF = N⁻¹.
By default, GPMR solves unsymmetric linear systems with λ = 1 and μ = 1.
GPMR is based on the orthogonal Hessenberg reduction process and its relations with the block-Arnoldi process. The residual norm ‖rₖ‖ is monotonically decreasing in GPMR.
GPMR stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
B: a linear operator that models a matrix of dimension n × m;
b: a vector of length m;
c: a vector of length n.
Optional arguments
x0: a vector of length m that represents an initial guess of the solution x;
y0: a vector of length n that represents an initial guess of the solution y.
Keyword arguments
memory: if restart = true, the restarted version GPMR(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
C: linear operator that models a nonsingular matrix of size m, and represents the first term of the block-diagonal left preconditioner;
D: linear operator that models a nonsingular matrix of size n, and represents the second term of the block-diagonal left preconditioner;
E: linear operator that models a nonsingular matrix of size m, and represents the first term of the block-diagonal right preconditioner;
F: linear operator that models a nonsingular matrix of size n, and represents the second term of the block-diagonal right preconditioner;
ldiv: define whether the preconditioners use ldiv! or mul!;
gsp: if true, set λ = 1 and μ = 0 for generalized saddle-point systems;
λ and μ: diagonal scaling factors of the partitioned linear system;
reorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length m;
y: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
This document was generated with Documenter.jl version 1.7.0 on Monday 30 September 2024. Using Julia version 1.10.5.
+[ B μN ] [ y ] [ c ]
when CE = M⁻¹ and DF = N⁻¹.
By default, GPMR solves unsymmetric linear systems with λ = 1 and μ = 1.
GPMR is based on the orthogonal Hessenberg reduction process and its relations with the block-Arnoldi process. The residual norm ‖rₖ‖ is monotonically decreasing in GPMR.
GPMR stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
B: a linear operator that models a matrix of dimension n × m;
b: a vector of length m;
c: a vector of length n.
Optional arguments
x0: a vector of length m that represents an initial guess of the solution x;
y0: a vector of length n that represents an initial guess of the solution y.
Keyword arguments
memory: if restart = true, the restarted version GPMR(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
C: linear operator that models a nonsingular matrix of size m, and represents the first term of the block-diagonal left preconditioner;
D: linear operator that models a nonsingular matrix of size n, and represents the second term of the block-diagonal left preconditioner;
E: linear operator that models a nonsingular matrix of size m, and represents the first term of the block-diagonal right preconditioner;
F: linear operator that models a nonsingular matrix of size n, and represents the second term of the block-diagonal right preconditioner;
ldiv: define whether the preconditioners use ldiv! or mul!;
gsp: if true, set λ = 1 and μ = 0 for generalized saddle-point systems;
λ and μ: diagonal scaling factors of the partitioned linear system;
reorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length m;
y: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Solve the consistent linear system
Ax + √λs = b
of size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations of the second kind
(AAᴴ + λI) y = b
but is more stable. When λ = 0, this method solves the minimum-norm problem
min ‖x‖₂ s.t. Ax = b.
When λ > 0, it solves the problem
min ‖(x,s)‖₂ s.t. Ax + √λs = b.
CGNE produces monotonic errors ‖x-x*‖₂ but not residuals ‖r‖₂. It is formally equivalent to CRAIG, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
N: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Solve the consistent linear system
Ax + √λs = b
of size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations of the second kind
(AAᴴ + λI) y = b
but is more stable. When λ = 0, this method solves the minimum-norm problem
min ‖x‖₂ s.t. Ax = b.
When λ > 0, it solves the problem
min ‖(x,s)‖₂ s.t. Ax + √λs = b.
CGNE produces monotonic errors ‖x-x*‖₂ but not residuals ‖r‖₂. It is formally equivalent to CRAIG, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
N: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Solve the consistent linear system
Ax + √λs = b
of size m × n using the Conjugate Residual (CR) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CR to the normal equations of the second kind
(AAᴴ + λI) y = b
but is more stable. When λ = 0, this method solves the minimum-norm problem
min ‖x‖₂ s.t. x ∈ argmin ‖Ax - b‖₂.
When λ > 0, this method solves the problem
min ‖(x,s)‖₂ s.t. Ax + √λs = b.
CRMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRAIG-MR, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
N: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
(x, y, stats) = lnlq(A, b::AbstractVector{FC};
+ callback=solver->false, iostream::IO=kstdout)
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Solve the consistent linear system
Ax + √λs = b
of size m × n using the Conjugate Residual (CR) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CR to the normal equations of the second kind
(AAᴴ + λI) y = b
but is more stable. When λ = 0, this method solves the minimum-norm problem
min ‖x‖₂ s.t. x ∈ argmin ‖Ax - b‖₂.
When λ > 0, this method solves the problem
min ‖(x,s)‖₂ s.t. Ax + √λs = b.
CRMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRAIG-MR, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
N: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Find the least-norm solution of the consistent linear system
Ax + λ²y = b
of size m × n using the LNLQ method, where λ ≥ 0 is a regularization parameter.
For a system in the form Ax = b, LNLQ method is equivalent to applying SYMMLQ to AAᴴy = b and recovering x = Aᴴy but is more stable. Note that y are the Lagrange multipliers of the least-norm problem
minimize ‖x‖ s.t. Ax = b.
If λ > 0, LNLQ solves the symmetric and quasi-definite system
[ -F Aᴴ ] [ x ] [ 0 ]
[ A λ²E ] [ y ] = [ b ],
where E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.
The system above represents the optimality conditions of
min ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.
For a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LNLQ is then equivalent to applying SYMMLQ to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.
If λ = 0, LNLQ solves the symmetric and indefinite system
[ -F Aᴴ ] [ x ] [ 0 ]
-[ A 0 ] [ y ] = [ b ].
The system above represents the optimality conditions of
minimize ‖x‖²_F s.t. Ax = b.
In this case, M can still be specified and indicates the weighted norm in which residuals are measured.
In this implementation, both the x and y-parts of the solution are returned.
utolx and utoly are tolerances on the upper bound of the distance to the solution ‖x-x‖ and ‖y-y‖, respectively. The bound is valid if λ>0 or σ>0 where σ should be strictly smaller than the smallest positive singular value. For instance σ:=(1-1e-7)σₘᵢₙ .
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
ldiv: define whether the preconditioners use ldiv! or mul!;
transfer_to_craig: transfer from the LNLQ point to the CRAIG point, when it exists. The transfer is based on the residual norm;
sqd: if true, set λ=1 for Hermitian quasi-definite systems;
λ: regularization parameter;
σ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;
utolx: tolerance on the upper bound on the distance to the solution ‖x-x*‖;
utoly: tolerance on the upper bound on the distance to the solution ‖y-y*‖;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
y: a dense vector of length m;
stats: statistics collected on the run in a LNLQStats structure.
(x, y, stats) = craig(A, b::AbstractVector{FC};
+[ A 0 ] [ y ] = [ b ].
The system above represents the optimality conditions of
minimize ‖x‖²_F s.t. Ax = b.
In this case, M can still be specified and indicates the weighted norm in which residuals are measured.
In this implementation, both the x and y-parts of the solution are returned.
utolx and utoly are tolerances on the upper bound of the distance to the solution ‖x-x‖ and ‖y-y‖, respectively. The bound is valid if λ>0 or σ>0 where σ should be strictly smaller than the smallest positive singular value. For instance σ:=(1-1e-7)σₘᵢₙ .
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
ldiv: define whether the preconditioners use ldiv! or mul!;
transfer_to_craig: transfer from the LNLQ point to the CRAIG point, when it exists. The transfer is based on the residual norm;
sqd: if true, set λ=1 for Hermitian quasi-definite systems;
λ: regularization parameter;
σ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;
utolx: tolerance on the upper bound on the distance to the solution ‖x-x*‖;
utoly: tolerance on the upper bound on the distance to the solution ‖y-y*‖;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
y: a dense vector of length m;
stats: statistics collected on the run in a LNLQStats structure.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Find the least-norm solution of the consistent linear system
Ax + λ²y = b
of size m × n using the Golub-Kahan implementation of Craig's method, where λ ≥ 0 is a regularization parameter. This method is equivalent to CGNE but is more stable.
For a system in the form Ax = b, Craig's method is equivalent to applying CG to AAᴴy = b and recovering x = Aᴴy. Note that y are the Lagrange multipliers of the least-norm problem
minimize ‖x‖ s.t. Ax = b.
If λ > 0, CRAIG solves the symmetric and quasi-definite system
[ -F Aᴴ ] [ x ] [ 0 ]
[ A λ²E ] [ y ] = [ b ],
where E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.
The system above represents the optimality conditions of
min ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.
For a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. CRAIG is then equivalent to applying CG to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.
If λ = 0, CRAIG solves the symmetric and indefinite system
[ -F Aᴴ ] [ x ] [ 0 ]
-[ A 0 ] [ y ] = [ b ].
The system above represents the optimality conditions of
minimize ‖x‖²_F s.t. Ax = b.
In this case, M can still be specified and indicates the weighted norm in which residuals are measured.
In this implementation, both the x and y-parts of the solution are returned.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
ldiv: define whether the preconditioners use ldiv! or mul!;
transfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;
sqd: if true, set λ=1 for Hermitian quasi-definite systems;
λ: regularization parameter;
btol: stopping tolerance used to detect zero-residual problems;
conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
y: a dense vector of length m;
stats: statistics collected on the run in a SimpleStats structure.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Solve the consistent linear system
Ax + λ²y = b
of size m × n using the CRAIGMR method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying the Conjugate Residuals method to the normal equations of the second kind
(AAᴴ + λ²I) y = b
but is more stable. When λ = 0, this method solves the minimum-norm problem
min ‖x‖ s.t. x ∈ argmin ‖Ax - b‖.
If λ > 0, CRAIGMR solves the symmetric and quasi-definite system
[ -F Aᴴ ] [ x ] [ 0 ]
[ A λ²E ] [ y ] = [ b ],
where E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.
The system above represents the optimality conditions of
min ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.
For a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. CRAIGMR is then equivalent to applying MINRES to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.
If λ = 0, CRAIGMR solves the symmetric and indefinite system
[ -F Aᴴ ] [ x ] [ 0 ]
-[ A 0 ] [ y ] = [ b ].
The system above represents the optimality conditions of
min ‖x‖²_F s.t. Ax = b.
In this case, M can still be specified and indicates the weighted norm in which residuals are measured.
CRAIGMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRMR, though can be slightly more accurate, and intricate to implement. Both the x- and y-parts of the solution are returned.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
ldiv: define whether the preconditioners use ldiv! or mul!;
sqd: if true, set λ=1 for Hermitian quasi-definite systems;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
y: a dense vector of length m;
stats: statistics collected on the run in a SimpleStats structure.
(x, stats) = usymlq(A, b::AbstractVector{FC}, c::AbstractVector{FC};
+[ A 0 ] [ y ] = [ b ].
The system above represents the optimality conditions of
min ‖x‖²_F s.t. Ax = b.
In this case, M can still be specified and indicates the weighted norm in which residuals are measured.
CRAIGMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRMR, though can be slightly more accurate, and intricate to implement. Both the x- and y-parts of the solution are returned.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
ldiv: define whether the preconditioners use ldiv! or mul!;
sqd: if true, set λ=1 for Hermitian quasi-definite systems;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
y: a dense vector of length m;
stats: statistics collected on the run in a SimpleStats structure.
USYMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
USYMLQ determines the least-norm solution of the consistent linear system Ax = b of size m × n.
USYMLQ is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The error norm ‖x - x*‖ monotonously decreases in USYMLQ. When A is Hermitian and b = c, USYMLQ is equivalent to SYMMLQ. USYMLQ is considered as a generalization of SYMMLQ.
It can also be applied to under-determined and over-determined problems. In all cases, problems must be consistent.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m;
c: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
transfer_to_usymcg: transfer from the USYMLQ point to the USYMCG point, when it exists. The transfer is based on the residual norm;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
USYMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
USYMLQ determines the least-norm solution of the consistent linear system Ax = b of size m × n.
USYMLQ is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The error norm ‖x - x*‖ monotonously decreases in USYMLQ. When A is Hermitian and b = c, USYMLQ is equivalent to SYMMLQ. USYMLQ is considered as a generalization of SYMMLQ.
It can also be applied to under-determined and over-determined problems. In all cases, problems must be consistent.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m;
c: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
transfer_to_usymcg: transfer from the USYMLQ point to the USYMCG point, when it exists. The transfer is based on the residual norm;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Solve the regularized linear least-squares problem
minimize ‖b - Ax‖₂² + λ‖x‖₂²
of size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations
(AᴴA + λI) x = Aᴴb
but is more stable.
CGLS produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSQR, though can be slightly less accurate, but simpler to implement.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Solve the regularized linear least-squares problem
minimize ‖b - Ax‖₂² + λ‖x‖₂²
of size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations
(AᴴA + λI) x = Aᴴb
but is more stable.
CGLS produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSQR, though can be slightly less accurate, but simpler to implement.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Solve the linear least-squares problem
minimize ‖b - Ax‖₂² + λ‖x‖₂²
of size m × n using the Conjugate Residuals (CR) method. This method is equivalent to applying MINRES to the normal equations
(AᴴA + λI) x = Aᴴb.
This implementation recurs the residual r := b - Ax.
CRLS produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSMR, though can be substantially less accurate, but simpler to implement.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
Reference
D. C.-L. Fong, Minimum-Residual Methods for Sparse, Least-Squares using Golubg-Kahan Bidiagonalization, Ph.D. Thesis, Stanford University, 2011.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Solve the linear least-squares problem
minimize ‖b - Ax‖₂² + λ‖x‖₂²
of size m × n using the Conjugate Residuals (CR) method. This method is equivalent to applying MINRES to the normal equations
(AᴴA + λI) x = Aᴴb.
This implementation recurs the residual r := b - Ax.
CRLS produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSMR, though can be substantially less accurate, but simpler to implement.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
Reference
D. C.-L. Fong, Minimum-Residual Methods for Sparse, Least-Squares using Golubg-Kahan Bidiagonalization, Ph.D. Thesis, Stanford University, 2011.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Solve the regularized linear least-squares problem
minimize ‖b - Ax‖₂² + λ²‖x‖₂²
of size m × n using the LSLQ method, where λ ≥ 0 is a regularization parameter. LSLQ is formally equivalent to applying SYMMLQ to the normal equations
(AᴴA + λ²I) x = Aᴴb
but is more stable.
If λ > 0, we solve the symmetric and quasi-definite system
[ E A ] [ r ] [ b ]
[ Aᴴ -λ²F ] [ x ] = [ 0 ],
where E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.
The system above represents the optimality conditions of
minimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.
For a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSLQ is then equivalent to applying SYMMLQ to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).
If λ = 0, we solve the symmetric and indefinite system
[ E A ] [ r ] [ b ]
-[ Aᴴ 0 ] [ x ] = [ 0 ].
The system above represents the optimality conditions of
minimize ‖b - Ax‖²_E⁻¹.
In this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).
Main features
the solution estimate is updated along orthogonal directions
the norm of the solution estimate ‖xᴸₖ‖₂ is increasing
the error ‖eₖ‖₂ := ‖xᴸₖ - x*‖₂ is decreasing
it is possible to transition cheaply from the LSLQ iterate to the LSQR iterate if there is an advantage (there always is in terms of error)
if A is rank deficient, identify the minimum least-squares solution
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
ldiv: define whether the preconditioners use ldiv! or mul!;
window: number of iterations used to accumulate a lower bound on the error;
transfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;
sqd: if true, set λ=1 for Hermitian quasi-definite systems;
λ: regularization parameter;
σ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;
etol: stopping tolerance based on the lower bound on the error;
utol: stopping tolerance based on the upper bound on the error;
btol: stopping tolerance used to detect zero-residual problems;
conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a LSLQStats structure.
stats.err_lbnds is a vector of lower bounds on the LQ error–-the vector is empty if window is set to zero
stats.err_ubnds_lq is a vector of upper bounds on the LQ error–-the vector is empty if σ == 0 is left at zero
stats.err_ubnds_cg is a vector of upper bounds on the CG error–-the vector is empty if σ == 0 is left at zero
stats.error_with_bnd is a boolean indicating whether there was an error in the upper bounds computation (cancellation errors, too large σ ...)
Stopping conditions
The iterations stop as soon as one of the following conditions holds true:
the optimality residual is sufficiently small (stats.status = "found approximate minimum least-squares solution") in the sense that either
‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ atol, or
1 + ‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ 1
an approximate zero-residual solution has been found (stats.status = "found approximate zero-residual solution") in the sense that either
‖r‖ / ‖b‖ ≤ btol + atol ‖A‖ * ‖xᴸ‖ / ‖b‖, or
1 + ‖r‖ / ‖b‖ ≤ 1
the estimated condition number of A is too large in the sense that either
1/cond(A) ≤ 1/conlim (stats.status = "condition number exceeds tolerance"), or
1 + 1/cond(A) ≤ 1 (stats.status = "condition number seems too large for this machine")
the lower bound on the LQ forward error is less than etol * ‖xᴸ‖
the upper bound on the CG forward error is less than utol * ‖xᶜ‖
The system above represents the optimality conditions of
minimize ‖b - Ax‖²_E⁻¹.
In this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).
Main features
the solution estimate is updated along orthogonal directions
the norm of the solution estimate ‖xᴸₖ‖₂ is increasing
the error ‖eₖ‖₂ := ‖xᴸₖ - x*‖₂ is decreasing
it is possible to transition cheaply from the LSLQ iterate to the LSQR iterate if there is an advantage (there always is in terms of error)
if A is rank deficient, identify the minimum least-squares solution
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
ldiv: define whether the preconditioners use ldiv! or mul!;
window: number of iterations used to accumulate a lower bound on the error;
transfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;
sqd: if true, set λ=1 for Hermitian quasi-definite systems;
λ: regularization parameter;
σ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;
etol: stopping tolerance based on the lower bound on the error;
utol: stopping tolerance based on the upper bound on the error;
btol: stopping tolerance used to detect zero-residual problems;
conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a LSLQStats structure.
stats.err_lbnds is a vector of lower bounds on the LQ error–-the vector is empty if window is set to zero
stats.err_ubnds_lq is a vector of upper bounds on the LQ error–-the vector is empty if σ == 0 is left at zero
stats.err_ubnds_cg is a vector of upper bounds on the CG error–-the vector is empty if σ == 0 is left at zero
stats.error_with_bnd is a boolean indicating whether there was an error in the upper bounds computation (cancellation errors, too large σ ...)
Stopping conditions
The iterations stop as soon as one of the following conditions holds true:
the optimality residual is sufficiently small (stats.status = "found approximate minimum least-squares solution") in the sense that either
‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ atol, or
1 + ‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ 1
an approximate zero-residual solution has been found (stats.status = "found approximate zero-residual solution") in the sense that either
‖r‖ / ‖b‖ ≤ btol + atol ‖A‖ * ‖xᴸ‖ / ‖b‖, or
1 + ‖r‖ / ‖b‖ ≤ 1
the estimated condition number of A is too large in the sense that either
1/cond(A) ≤ 1/conlim (stats.status = "condition number exceeds tolerance"), or
1 + 1/cond(A) ≤ 1 (stats.status = "condition number seems too large for this machine")
the lower bound on the LQ forward error is less than etol * ‖xᴸ‖
the upper bound on the CG forward error is less than utol * ‖xᶜ‖
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Solve the regularized linear least-squares problem
minimize ‖b - Ax‖₂² + λ²‖x‖₂²
of size m × n using the LSQR method, where λ ≥ 0 is a regularization parameter. LSQR is formally equivalent to applying CG to the normal equations
(AᴴA + λ²I) x = Aᴴb
(and therefore to CGLS) but is more stable.
LSQR produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to CGLS, though can be slightly more accurate.
If λ > 0, LSQR solves the symmetric and quasi-definite system
[ E A ] [ r ] [ b ]
[ Aᴴ -λ²F ] [ x ] = [ 0 ],
where E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.
The system above represents the optimality conditions of
minimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.
For a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSQR is then equivalent to applying CG to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).
If λ = 0, we solve the symmetric and indefinite system
[ E A ] [ r ] [ b ]
-[ Aᴴ 0 ] [ x ] = [ 0 ].
The system above represents the optimality conditions of
minimize ‖b - Ax‖²_E⁻¹.
In this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
ldiv: define whether the preconditioners use ldiv! or mul!;
window: number of iterations used to accumulate a lower bound on the error;
sqd: if true, set λ=1 for Hermitian quasi-definite systems;
λ: regularization parameter;
radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
etol: stopping tolerance based on the lower bound on the error;
axtol: tolerance on the backward error;
btol: stopping tolerance used to detect zero-residual problems;
conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
The system above represents the optimality conditions of
minimize ‖b - Ax‖²_E⁻¹.
In this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
ldiv: define whether the preconditioners use ldiv! or mul!;
window: number of iterations used to accumulate a lower bound on the error;
sqd: if true, set λ=1 for Hermitian quasi-definite systems;
λ: regularization parameter;
radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
etol: stopping tolerance based on the lower bound on the error;
axtol: tolerance on the backward error;
btol: stopping tolerance used to detect zero-residual problems;
conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
Solve the regularized linear least-squares problem
minimize ‖b - Ax‖₂² + λ²‖x‖₂²
of size m × n using the LSMR method, where λ ≥ 0 is a regularization parameter. LSMR is formally equivalent to applying MINRES to the normal equations
(AᴴA + λ²I) x = Aᴴb
(and therefore to CRLS) but is more stable.
LSMR produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to CRLS, though can be substantially more accurate.
LSMR can be also used to find a null vector of a singular matrix A by solving the problem min ‖Aᴴx - b‖ with any nonzero vector b. At a minimizer, the residual vector r = b - Aᴴx will satisfy Ar = 0.
If λ > 0, we solve the symmetric and quasi-definite system
[ E A ] [ r ] [ b ]
[ Aᴴ -λ²F ] [ x ] = [ 0 ],
where E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.
The system above represents the optimality conditions of
minimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.
For a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSMR is then equivalent to applying MINRES to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).
If λ = 0, we solve the symmetric and indefinite system
[ E A ] [ r ] [ b ]
-[ Aᴴ 0 ] [ x ] = [ 0 ].
The system above represents the optimality conditions of
minimize ‖b - Ax‖²_E⁻¹.
In this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
ldiv: define whether the preconditioners use ldiv! or mul!;
window: number of iterations used to accumulate a lower bound on the error;
sqd: if true, set λ=1 for Hermitian quasi-definite systems;
λ: regularization parameter;
radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
etol: stopping tolerance based on the lower bound on the error;
axtol: tolerance on the backward error;
btol: stopping tolerance used to detect zero-residual problems;
conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a LsmrStats structure.
The system above represents the optimality conditions of
minimize ‖b - Ax‖²_E⁻¹.
In this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
ldiv: define whether the preconditioners use ldiv! or mul!;
window: number of iterations used to accumulate a lower bound on the error;
sqd: if true, set λ=1 for Hermitian quasi-definite systems;
λ: regularization parameter;
radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
etol: stopping tolerance based on the lower bound on the error;
axtol: tolerance on the backward error;
btol: stopping tolerance used to detect zero-residual problems;
conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a LsmrStats structure.
USYMQR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
USYMQR solves the linear least-squares problem min ‖b - Ax‖² of size m × n. USYMQR solves Ax = b if it is consistent.
USYMQR is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The residual norm ‖b - Ax‖ monotonously decreases in USYMQR. When A is Hermitian and b = c, QMR is equivalent to MINRES. USYMQR is considered as a generalization of MINRES.
It can also be applied to under-determined and over-determined problems. USYMQR finds the minimum-norm solution if problems are inconsistent.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m;
c: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
USYMQR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
USYMQR solves the linear least-squares problem min ‖b - Ax‖² of size m × n. USYMQR solves Ax = b if it is consistent.
USYMQR is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The residual norm ‖b - Ax‖ monotonously decreases in USYMQR. When A is Hermitian and b = c, QMR is equivalent to MINRES. USYMQR is considered as a generalization of MINRES.
It can also be applied to under-determined and over-determined problems. USYMQR finds the minimum-norm solution if problems are inconsistent.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m;
c: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
MINRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the shifted linear least-squares problem
minimize ‖b - (A + λI)x‖₂²
or the shifted linear system
(A + λI) x = b
of size n using the MINRES method, where λ ≥ 0 is a shift parameter, where A is Hermitian.
MINRES is formally equivalent to applying CR to Ax=b when A is positive definite, but is typically more stable and also applies to the case where A is indefinite.
MINRES produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂.
Input arguments
A: a linear operator that models a Hermitian matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
window: number of iterations used to accumulate a lower bound on the error;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
etol: stopping tolerance based on the lower bound on the error;
conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
MINRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the shifted linear least-squares problem
minimize ‖b - (A + λI)x‖₂²
or the shifted linear system
(A + λI) x = b
of size n using the MINRES method, where λ ≥ 0 is a shift parameter, where A is Hermitian.
MINRES is formally equivalent to applying CR to Ax=b when A is positive definite, but is typically more stable and also applies to the case where A is indefinite.
MINRES produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂.
Input arguments
A: a linear operator that models a Hermitian matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
window: number of iterations used to accumulate a lower bound on the error;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
etol: stopping tolerance based on the lower bound on the error;
conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
MINRES-QLP can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
MINRES-QLP is the only method based on the Lanczos process that returns the minimum-norm solution on singular inconsistent systems (A + λI)x = b of size n, where λ is a shift parameter. It is significantly more complex but can be more reliable than MINRES when A is ill-conditioned.
M also indicates the weighted norm in which residuals are measured.
Input arguments
A: a linear operator that models a Hermitian matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
Artol: relative stopping tolerance based on the Aᴴ-residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
References
S.-C. T. Choi, Iterative methods for singular linear equations and least-squares problems, Ph.D. thesis, ICME, Stanford University, 2006.
MINRES-QLP can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
MINRES-QLP is the only method based on the Lanczos process that returns the minimum-norm solution on singular inconsistent systems (A + λI)x = b of size n, where λ is a shift parameter. It is significantly more complex but can be more reliable than MINRES when A is ill-conditioned.
M also indicates the weighted norm in which residuals are measured.
Input arguments
A: a linear operator that models a Hermitian matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
Artol: relative stopping tolerance based on the Aᴴ-residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
References
S.-C. T. Choi, Iterative methods for singular linear equations and least-squares problems, Ph.D. thesis, ICME, Stanford University, 2006.
MINARES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
MINARES solves the Hermitian linear system Ax = b of size n. MINARES minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.
Input arguments
A: a linear operator that models a Hermitian positive definite matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
Artol: relative stopping tolerance based on the Aᴴ-residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
MINARES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
MINARES solves the Hermitian linear system Ax = b of size n. MINARES minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.
Input arguments
A: a linear operator that models a Hermitian positive definite matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
λ: regularization parameter;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
Artol: relative stopping tolerance based on the Aᴴ-residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
(x, y, stats) = tricg(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)
TriCG can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.
Given a matrix A of dimension m × n, TriCG solves the Hermitian linear system
[ τE A ] [ x ] = [ b ]
[ Aᴴ νF ] [ y ] [ c ],
of size (n+m) × (n+m) where τ and ν are real numbers, E = M⁻¹ ≻ 0 and F = N⁻¹ ≻ 0. b and c must both be nonzero. TriCG could breakdown if τ = 0 or ν = 0. It's recommended to use TriMR in these cases.
By default, TriCG solves Hermitian and quasi-definite linear systems with τ = 1 and ν = -1.
TriCG is based on the preconditioned orthogonal tridiagonalization process and its relation with the preconditioned block-Lanczos process.
[ M 0 ]
-[ 0 N ]
indicates the weighted norm in which residuals are measured. It's the Euclidean norm when M and N are identity operators.
TriCG stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m;
c: a vector of length n.
Optional arguments
x0: a vector of length m that represents an initial guess of the solution x;
y0: a vector of length n that represents an initial guess of the solution y.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the partitioned system;
N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the partitioned system;
ldiv: define whether the preconditioners use ldiv! or mul!;
spd: if true, set τ = 1 and ν = 1 for Hermitian and positive-definite linear system;
snd: if true, set τ = -1 and ν = -1 for Hermitian and negative-definite linear systems;
flip: if true, set τ = -1 and ν = 1 for another known variant of Hermitian quasi-definite systems;
τ and ν: diagonal scaling factors of the partitioned Hermitian linear system;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length m;
y: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
(x, y, stats) = trimr(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)
TriMR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.
Given a matrix A of dimension m × n, TriMR solves the symmetric linear system
[ τE A ] [ x ] = [ b ]
[ Aᴴ νF ] [ y ] [ c ],
of size (n+m) × (n+m) where τ and ν are real numbers, E = M⁻¹ ≻ 0, F = N⁻¹ ≻ 0. b and c must both be nonzero. TriMR handles saddle-point systems (τ = 0 or ν = 0) and adjoint systems (τ = 0 and ν = 0) without any risk of breakdown.
By default, TriMR solves symmetric and quasi-definite linear systems with τ = 1 and ν = -1.
TriMR is based on the preconditioned orthogonal tridiagonalization process and its relation with the preconditioned block-Lanczos process.
[ M 0 ]
-[ 0 N ]
indicates the weighted norm in which residuals are measured. It's the Euclidean norm when M and N are identity operators.
TriMR stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.
Input arguments
A: a linear operator that models a matrix of dimension m × n;
b: a vector of length m;
c: a vector of length n.
Optional arguments
x0: a vector of length m that represents an initial guess of the solution x;
y0: a vector of length n that represents an initial guess of the solution y.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the partitioned system;
N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the partitioned system;
ldiv: define whether the preconditioners use ldiv! or mul!;
spd: if true, set τ = 1 and ν = 1 for Hermitian and positive-definite linear system;
snd: if true, set τ = -1 and ν = -1 for Hermitian and negative-definite linear systems;
flip: if true, set τ = -1 and ν = 1 for another known variant of Hermitian quasi-definite systems;
sp: if true, set τ = 1 and ν = 0 for saddle-point systems;
τ and ν: diagonal scaling factors of the partitioned Hermitian linear system;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length m;
y: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
CG can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
The conjugate gradient method to solve the Hermitian linear system Ax = b of size n.
The method does not abort if A is not definite. M also indicates the weighted norm in which residuals are measured.
Input arguments
A: a linear operator that models a Hermitian positive definite matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
linesearch: if true, indicate that the solution is to be used in an inexact Newton method with linesearch. If negative curvature is detected at iteration k > 0, the solution of iteration k-1 is returned. If negative curvature is detected at iteration 0, the right-hand side is returned (i.e., the negative gradient);
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
CG can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
The conjugate gradient method to solve the Hermitian linear system Ax = b of size n.
The method does not abort if A is not definite. M also indicates the weighted norm in which residuals are measured.
Input arguments
A: a linear operator that models a Hermitian positive definite matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
linesearch: if true, indicate that the solution is to be used in an inexact Newton method with linesearch. If negative curvature is detected at iteration k > 0, the solution of iteration k-1 is returned. If negative curvature is detected at iteration 0, the right-hand side is returned (i.e., the negative gradient);
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
CR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
A truncated version of Stiefel’s Conjugate Residual method to solve the Hermitian linear system Ax = b of size n or the least-squares problem min ‖b - Ax‖ if A is singular. The matrix A must be Hermitian semi-definite. M also indicates the weighted norm in which residuals are measured.
Input arguments
A: a linear operator that models a Hermitian positive definite matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
linesearch: if true, indicate that the solution is to be used in an inexact Newton method with linesearch. If negative curvature is detected at iteration k > 0, the solution of iteration k-1 is returned. If negative curvature is detected at iteration 0, the right-hand side is returned (i.e., the negative gradient);
γ: tolerance to determine that the curvature of the quadratic model is nonpositive;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
CR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
A truncated version of Stiefel’s Conjugate Residual method to solve the Hermitian linear system Ax = b of size n or the least-squares problem min ‖b - Ax‖ if A is singular. The matrix A must be Hermitian semi-definite. M also indicates the weighted norm in which residuals are measured.
Input arguments
A: a linear operator that models a Hermitian positive definite matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
linesearch: if true, indicate that the solution is to be used in an inexact Newton method with linesearch. If negative curvature is detected at iteration k > 0, the solution of iteration k-1 is returned. If negative curvature is detected at iteration 0, the right-hand side is returned (i.e., the negative gradient);
γ: tolerance to determine that the curvature of the quadratic model is nonpositive;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
CAR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
CAR solves the Hermitian and positive definite linear system Ax = b of size n. CAR minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.
Input arguments
A: a linear operator that models a Hermitian positive definite matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
CAR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
CAR solves the Hermitian and positive definite linear system Ax = b of size n. CAR minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.
Input arguments
A: a linear operator that models a Hermitian positive definite matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
CG-LANCZOS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
The Lanczos version of the conjugate gradient method to solve the Hermitian linear system Ax = b of size n.
The method does not abort if A is not definite.
Input arguments
A: a linear operator that models a Hermitian matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
check_curvature: if true, check that the curvature of the quadratic along the search direction is positive, and abort if not, unless linesearch is also true;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a LanczosStats structure.
CG-LANCZOS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
The Lanczos version of the conjugate gradient method to solve the Hermitian linear system Ax = b of size n.
The method does not abort if A is not definite.
Input arguments
A: a linear operator that models a Hermitian matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
check_curvature: if true, check that the curvature of the quadratic along the search direction is positive, and abort if not, unless linesearch is also true;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a LanczosStats structure.
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
The Lanczos version of the conjugate gradient method to solve a family of shifted systems
(A + αI) x = b (α = α₁, ..., αₚ)
of size n. The method does not abort if A + αI is not definite.
Input arguments
A: a linear operator that models a Hermitian matrix of dimension n;
b: a vector of length n;
shifts: a vector of length p.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
check_curvature: if true, check that the curvature of the quadratic along the search direction is positive, and abort if not, unless linesearch is also true;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a vector of p dense vectors, each one of length n;
stats: statistics collected on the run in a LanczosShiftStats structure.
This document was generated with Documenter.jl version 1.7.0 on Monday 30 September 2024. Using Julia version 1.10.5.
+ callback=solver->false, iostream::IO=kstdout)
T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.
The Lanczos version of the conjugate gradient method to solve a family of shifted systems
(A + αI) x = b (α = α₁, ..., αₚ)
of size n. The method does not abort if A + αI is not definite.
Input arguments
A: a linear operator that models a Hermitian matrix of dimension n;
b: a vector of length n;
shifts: a vector of length p.
Keyword arguments
M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
ldiv: define whether the preconditioner uses ldiv! or mul!;
check_curvature: if true, check that the curvature of the quadratic along the search direction is positive, and abort if not, unless linesearch is also true;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a vector of p dense vectors, each one of length n;
stats: statistics collected on the run in a LanczosShiftStats structure.
BiLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the square linear system Ax = b of size n using BiLQ. BiLQ is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, BiLQ is equivalent to SYMMLQ. BiLQ requires support for adjoint(M) and adjoint(N) if preconditioners are provided.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
c: the second initial vector of length n required by the Lanczos biorthogonalization process;
transfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
BiLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the square linear system Ax = b of size n using BiLQ. BiLQ is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, BiLQ is equivalent to SYMMLQ. BiLQ requires support for adjoint(M) and adjoint(N) if preconditioners are provided.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
c: the second initial vector of length n required by the Lanczos biorthogonalization process;
transfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
QMR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the square linear system Ax = b of size n using QMR.
QMR is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, QMR is equivalent to MINRES. QMR requires support for adjoint(M) and adjoint(N) if preconditioners are provided.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
c: the second initial vector of length n required by the Lanczos biorthogonalization process;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
QMR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the square linear system Ax = b of size n using QMR.
QMR is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, QMR is equivalent to MINRES. QMR requires support for adjoint(M) and adjoint(N) if preconditioners are provided.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
c: the second initial vector of length n required by the Lanczos biorthogonalization process;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
CGS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the consistent linear system Ax = b of size n using CGS. CGS requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.
From "Iterative Methods for Sparse Linear Systems (Y. Saad)" :
«The method is based on a polynomial variant of the conjugate gradients algorithm. Although related to the so-called bi-conjugate gradients (BCG) algorithm, it does not involve adjoint matrix-vector multiplications, and the expected convergence rate is about twice that of the BCG algorithm.
The Conjugate Gradient Squared algorithm works quite well in many cases. However, one difficulty is that, since the polynomials are squared, rounding errors tend to be more damaging than in the standard BCG algorithm. In particular, very high variations of the residual vectors often cause the residual norms computed to become inaccurate.
TFQMR and BICGSTAB were developed to remedy this difficulty.»
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
c: the second initial vector of length n required by the Lanczos biorthogonalization process;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
CGS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the consistent linear system Ax = b of size n using CGS. CGS requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.
From "Iterative Methods for Sparse Linear Systems (Y. Saad)" :
«The method is based on a polynomial variant of the conjugate gradients algorithm. Although related to the so-called bi-conjugate gradients (BCG) algorithm, it does not involve adjoint matrix-vector multiplications, and the expected convergence rate is about twice that of the BCG algorithm.
The Conjugate Gradient Squared algorithm works quite well in many cases. However, one difficulty is that, since the polynomials are squared, rounding errors tend to be more damaging than in the standard BCG algorithm. In particular, very high variations of the residual vectors often cause the residual norms computed to become inaccurate.
TFQMR and BICGSTAB were developed to remedy this difficulty.»
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
c: the second initial vector of length n required by the Lanczos biorthogonalization process;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
BICGSTAB can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the square linear system Ax = b of size n using BICGSTAB. BICGSTAB requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.
The Biconjugate Gradient Stabilized method is a variant of BiCG, like CGS, but using different updates for the Aᴴ-sequence in order to obtain smoother convergence than CGS.
If BICGSTAB stagnates, we recommend DQGMRES and BiLQ as alternative methods for unsymmetric square systems.
BICGSTAB stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖b‖ * rtol.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
c: the second initial vector of length n required by the Lanczos biorthogonalization process;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
G. L.G. Sleijpen and D. R. Fokkema, BiCGstab(ℓ) for linear equations involving unsymmetric matrices with complex spectrum, Electronic Transactions on Numerical Analysis, 1, pp. 11–32, 1993.
BICGSTAB can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the square linear system Ax = b of size n using BICGSTAB. BICGSTAB requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.
The Biconjugate Gradient Stabilized method is a variant of BiCG, like CGS, but using different updates for the Aᴴ-sequence in order to obtain smoother convergence than CGS.
If BICGSTAB stagnates, we recommend DQGMRES and BiLQ as alternative methods for unsymmetric square systems.
BICGSTAB stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖b‖ * rtol.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
c: the second initial vector of length n required by the Lanczos biorthogonalization process;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
G. L.G. Sleijpen and D. R. Fokkema, BiCGstab(ℓ) for linear equations involving unsymmetric matrices with complex spectrum, Electronic Transactions on Numerical Analysis, 1, pp. 11–32, 1993.
DIOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the consistent linear system Ax = b of size n using DIOM.
DIOM only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If CG is well defined on Ax = b and memory = 2, DIOM is theoretically equivalent to CG. If k ≤ memory where k is the number of iterations, DIOM is theoretically equivalent to FOM. Otherwise, DIOM interpolates between CG and FOM and is similar to CG with partial reorthogonalization.
An advantage of DIOM is that non-Hermitian or Hermitian indefinite or both non-Hermitian and indefinite systems of linear equations can be handled by this single algorithm.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
memory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
reorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
DIOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the consistent linear system Ax = b of size n using DIOM.
DIOM only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If CG is well defined on Ax = b and memory = 2, DIOM is theoretically equivalent to CG. If k ≤ memory where k is the number of iterations, DIOM is theoretically equivalent to FOM. Otherwise, DIOM interpolates between CG and FOM and is similar to CG with partial reorthogonalization.
An advantage of DIOM is that non-Hermitian or Hermitian indefinite or both non-Hermitian and indefinite systems of linear equations can be handled by this single algorithm.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
memory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
reorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
FOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the linear system Ax = b of size n using FOM.
FOM algorithm is based on the Arnoldi process and a Galerkin condition.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
memory: if restart = true, the restarted version FOM(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
restart: restart the method after memory iterations;
reorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
FOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the linear system Ax = b of size n using FOM.
FOM algorithm is based on the Arnoldi process and a Galerkin condition.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
memory: if restart = true, the restarted version FOM(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
restart: restart the method after memory iterations;
reorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
DQGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the consistent linear system Ax = b of size n using DQGMRES.
DQGMRES algorithm is based on the incomplete Arnoldi orthogonalization process and computes a sequence of approximate solutions with the quasi-minimal residual property.
DQGMRES only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If MINRES is well defined on Ax = b and memory = 2, DQGMRES is theoretically equivalent to MINRES. If k ≤ memory where k is the number of iterations, DQGMRES is theoretically equivalent to GMRES. Otherwise, DQGMRES interpolates between MINRES and GMRES and is similar to MINRES with partial reorthogonalization.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
memory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
reorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;
ldiv: define whether the preconditioners use ldiv! or mul!;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
DQGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the consistent linear system Ax = b of size n using DQGMRES.
DQGMRES algorithm is based on the incomplete Arnoldi orthogonalization process and computes a sequence of approximate solutions with the quasi-minimal residual property.
DQGMRES only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If MINRES is well defined on Ax = b and memory = 2, DQGMRES is theoretically equivalent to MINRES. If k ≤ memory where k is the number of iterations, DQGMRES is theoretically equivalent to GMRES. Otherwise, DQGMRES interpolates between MINRES and GMRES and is similar to MINRES with partial reorthogonalization.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
memory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
reorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;
ldiv: define whether the preconditioners use ldiv! or mul!;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
GMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the linear system Ax = b of size n using GMRES.
GMRES algorithm is based on the Arnoldi process and computes a sequence of approximate solutions with the minimum residual.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
memory: if restart = true, the restarted version GMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
restart: restart the method after memory iterations;
reorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
GMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the linear system Ax = b of size n using GMRES.
GMRES algorithm is based on the Arnoldi process and computes a sequence of approximate solutions with the minimum residual.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
memory: if restart = true, the restarted version GMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
restart: restart the method after memory iterations;
reorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
FGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the linear system Ax = b of size n using FGMRES.
FGMRES computes a sequence of approximate solutions with minimum residual. FGMRES is a variant of GMRES that allows changes in the right preconditioner at each iteration.
This implementation allows a left preconditioner M and a flexible right preconditioner N. A situation in which the preconditioner is "not constant" is when a relaxation-type method, a Chebyshev iteration or another Krylov subspace method is used as a preconditioner. Compared to GMRES, there is no additional cost incurred in the arithmetic but the memory requirement almost doubles. Thus, GMRES is recommended if the right preconditioner N is constant.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
memory: if restart = true, the restarted version FGMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
restart: restart the method after memory iterations;
reorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
FGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.
Solve the linear system Ax = b of size n using FGMRES.
FGMRES computes a sequence of approximate solutions with minimum residual. FGMRES is a variant of GMRES that allows changes in the right preconditioner at each iteration.
This implementation allows a left preconditioner M and a flexible right preconditioner N. A situation in which the preconditioner is "not constant" is when a relaxation-type method, a Chebyshev iteration or another Krylov subspace method is used as a preconditioner. Compared to GMRES, there is no additional cost incurred in the arithmetic but the memory requirement almost doubles. Thus, GMRES is recommended if the right preconditioner N is constant.
Input arguments
A: a linear operator that models a matrix of dimension n;
b: a vector of length n.
Optional argument
x0: a vector of length n that represents an initial guess of the solution x.
Keyword arguments
memory: if restart = true, the restarted version FGMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
ldiv: define whether the preconditioners use ldiv! or mul!;
restart: restart the method after memory iterations;
reorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;
atol: absolute stopping tolerance based on the residual norm;
rtol: relative stopping tolerance based on the residual norm;
itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
timemax: the time limit in seconds;
verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
iostream: stream to which output is logged.
Output arguments
x: a dense vector of length n;
stats: statistics collected on the run in a SimpleStats structure.
