diff --git a/stable b/stable
index d2781879..bbe495df 120000
--- a/stable
+++ b/stable
@@ -1 +1 @@
-v0.8.7
\ No newline at end of file
+v0.8.8
\ No newline at end of file
diff --git a/v0.8 b/v0.8
index d2781879..bbe495df 120000
--- a/v0.8
+++ b/v0.8
@@ -1 +1 @@
-v0.8.7
\ No newline at end of file
+v0.8.8
\ No newline at end of file
diff --git a/v0.8.8/acceleration/index.html b/v0.8.8/acceleration/index.html
new file mode 100644
index 00000000..ecde9b95
--- /dev/null
+++ b/v0.8.8/acceleration/index.html
@@ -0,0 +1,7 @@
+
+
By default COSMO's ADMM algorithm is wrapped in a safeguarded acceleration method to achieve faster convergence to higher precision. COSMO uses accelerators from the COSMOAccelerators.jl package.
By default, the solver uses the accelerator type AndersonAccelerator{T, Type2{QRDecomp}, RestartedMemory, NoRegularizer}. This is the classic type-II Anderson acceleration method where the least squares subproblem is solved using an updated QR method. Moreover, the method is restarted, i.e. the history of iterates is deleted, after mem steps and no regularisation for the least-squares method is used.
In addition, the method is safeguarded (safeguard = true), i.e. the residual-norm of the accelerated point can not deviate too much from the current point. Otherwise, the point is discarded and the ADMM algorithm performs a normal step instead.
The acceleration method can be altered as usual via the solver settings and the accelerator keyword. To deactivate acceleration pass an EmptyAccelerator:
The optional keyword arguments x0 and y0 can be used to provide the solver with warm starting values for the primal variable x and the dual variable y.
The optional arguments dim and indices can be used to specify A and b for subparts of variable x. If x has dimension dim = 4, then x[2] and x[3] can be constrained to the zero cone in the following way:
Examples
julia> c = COSMO.Constraint([1 0;0 1], zeros(2), COSMO.ZeroSet, 4, 2:3)
+Constraint
+Size of A: (2, 4)
+ConvexSet: COSMO.ZeroSet{Float64}
Notice that extra columns of A have been added automatically.
Creates the cone of symmetric positive semidefinite matrices $\mathcal{S}_+^{dim}$. The entries of the matrix X are stored column-by-column in the vector x of dimension dim. Accordingly $X \in \mathbb{S}_+ \Rightarrow x \in \mathcal{S}_+^{dim}$, where $X = \text{mat}(x)$.
Creates the cone of symmetric positive semidefinite matrices. The entries of the upper-triangular part of matrix X are stored in the vector x of dimension dim. A $r \times r$ matrix has $r(r+1)/2$ upper triangular elements and results in a vector of $\mathrm{dim} = r(r+1)/2$.
If you find COSMO useful in your project, we kindly request that you cite the following paper:
@Article{Garstka_2021,
+ author = {Michael Garstka and Mark Cannon and Paul Goulart},
+ journal = {Journal of Optimization Theory and Applications},
+ title = {{COSMO}: A Conic Operator Splitting Method for Convex Conic Problems},
+ volume = {190},
+ number = {3},
+ pages = {779--810},
+ year = {2021},
+ publisher = {Springer},
+ doi = {10.1007/s10957-021-01896-x},
+ url = {https://doi.org/10.1007/s10957-021-01896-x}
+}
The acceleration strategy is described in this paper:
@InProceedings{Garstka_2022,
+ title={Safeguarded Anderson acceleration for parametric nonexpansive operators},
+ author={Garstka, Michael and Cannon, Mark and Goulart, Paul},
+ booktitle={2022 European Control Conference (ECC)},
+ pages={435--440},
+ year={2022},
+ organization={IEEE}
+}
If you want to contribute features, bug fixes, etc, please take a look at our Code Style Guide below
Please report any issues and bugs that you encounter in Issues
As an open source project we are also interested in any projects and applications that use COSMO. Please let us know via email to: michael.garstka[at]eng.ox.ac.uk
The code in this repository follows the naming and style conventions of Julia Base with a few modifications. This style guide is heavily "inspired" by the guides of John Myles White and JuMP.
Branch names should be prepended with the initials of the creator and a forward slash, e.g. mg/newIdea instead of newIdea
Commit messages should have the following format:
<#IssueId> Short (72 chars or less) summary
+
+More detailed explanatory text. Wrap it to 72 characters. The blank
+line separating the summary from the body is critical.
+
+Imperative style for the commit message: "Fix bug" and not "Fixed
+bug" or "Fixes bug."
+
+The issue id can be ommitted if the commit does not related to a specific open issue
Settings
This document was generated with Documenter.jl version 0.27.25 on Sunday 23 July 2023. Using Julia version 1.9.2.
For very large sparse semidefinite programs (SDPs) it is often helpful to analyse the sparsity structure of the PSD constraint(s). If the equality constraints impose a sparsity structure on the matrix variable, one PSD constraint on a large matrix variable can be decomposed into several smaller constraints. This results usually in a significant speedup and reduction in solve time.
The following example gives a short overview on chordal decomposition and clique merging. For more details, take a look at our paper on clique merging or watch the corresponding presentation that I gave on the topic.
Since all the data matrices have zeros in the same places, the equality constraint of the problem tells us that our matrix variable $S$ will also have zeros in these places. The aggregated pattern matrix of $S$ is shown in the following figure, where a dot represents a nonzero entry:
Since the matrices are symmetric we only show the lower triangle. We can represent the aggregated sparsity pattern of the problem using a graph $G(V, E)$, where the vertex set $V$ is given by the column indices of the matrix and we introduce an edge $(i,j) \in E$ for every nonzero matrix element $S_{ij}$. The graph for the example problem is shown in the right figure. In order to decompose the problem, we have to find the cliques, i.e. completely connected subgraphs, of the graph $G$. These cliques represent the dense subblocks of non-zero entries in the matrix (see the colored entries in the figure). In order for the theory to work, we also have to require $G$ to be a chordal graph. For the purpose of this example, we just assume that $G$ is chordal and keep in mind that we can always make a graph chordal by adding more edges.
COSMO finds the cliques of the graph automatically, if a constraint of type COSMO.PsdCone or COSMO.PsdConeTriangle is present in the problem and additional equality constraints impose a structure on them. For the example problem COSMO finds the following cliques: $\mathcal{C_1}=\{1,3,6\}, \; \mathcal{C_2}=\{2,3\}, \; \mathcal{C_3}=\{3,6,7,8\}, \; \mathcal{C_4}=\{4,5,8\}$ and $\mathcal{C_5}=\{6,7,8,9\}$.
Let's denote the set of cliques $\mathcal{B}=\{\mathcal{C}_1,\ldots,\mathcal{C_p}\}$. To represent the relationship between different cliques, e.g. in terms of overlapping entries, it is helpful to represent them either as a clique tree $\mathcal{T}(\mathcal{B}, \mathcal{E})$ (left) or a clique graph $\mathcal{G}(\mathcal{B}, \xi)$ (right), shown in the following figures:
Once the cliques have been found, we can use the following theorem to decompose the problem (and speed up the solver significantly).
Theorem (Agler's Theorem)
Let $G(V,E)$ be a chordal graph with a set of maximal cliques $\{\mathcal{C}_1,\ldots,\mathcal{C}_p \}$. Then $S \in \mathbb{S}^n_+(E,0)$ if and only if there exist matrices $S_\ell \in S^{|\mathcal{C}_\ell|}$ for $\ell = 1,\ldots,p$ such that
where $T_\ell$ is an entry-selector matrix that maps the subblocks $S_\ell$ into the correct entries of the original matrix $S$.
In other words, instead of solving the original problem and choosing $S$ to be positive semidefinite, we only have to choose the subblocks $S_\ell$ of the matrix $S$ to be positive semidefinite and make sure that the individual blocks "add up" to $S$. Thus, we can solve the equivalent problem:
\[\begin{array}{ll}
+ \text{minimize} & c^\top x \\
+ \text{subject to} &\displaystyle \sum_{i=1}^m A_i x + \sum_{\ell = 1}^p T_l^\top S_\ell T_\ell = B\\
+ & S_\ell \in \mathbb{S}^{|\mathcal{C}_\ell|}_+, \; l = 1, \ldots, p.
+\end{array}\]
Compare this again with the problem at the top of the page. We have replaced the positive semidefinite constraint on the matrix variable $S$ by $p$ constraints on the smaller subblocks $S_\ell$. Why is this useful?
When solving an SDP, the major linear algebra operation is a projection onto the constraint set, see Method. This projection is done at each iteration of the algorithm and for positive semidefinite constraints involves an eigenvalue decomposition of the matrix iterate. Unfortunately, an eigenvalue decomposition of a matrix of dimension $n$ has a complexity of roughly $\mathcal{O}(n^3)$.
For our example this means that we now project one $2\times 2$ block, two $3 \times 3$ blocks and two $4 \times 4$ blocks instead of one $9 \times 9$ block. For small problems, this doesn't seem to make much of a difference, but consider of a problem with 1000 $11 \times 11$ blocks along the diagonal and overlapping by one entry. Taking into account that the computational cost scales cubicly, we can improve the performance of the solver by projecting 1000 $11\times 11$ blocks instead of projecting one giant $10001\times 10001$ block at each iteration.
Let's go back to our example and solve it with COSMO and JuMP:
model = JuMP.Model(with_optimizer(COSMO.Optimizer, decompose = true, merge_strategy = COSMO.NoMerge));
+@variable(model, x[1:2]);
+@objective(model, Min, c' * x )
+@constraint(model, Symmetric(B - A1 .* x[1] - A2 .* x[2] ) in JuMP.PSDCone());
+
+JuMP.optimize!(model)
Notice that we set the solver settings decompose = true to allow COSMO to decompose the problem. Furthermore, we set the clique merging strategy to COSMO.NoMerge. This just means that after we have found the cliques we don't attempt to merge some of them (more about clique merging below). We get the following output from the solver:
------------------------------------------------------------------
+ COSMO v0.5.0 - A Quadratic Objective Conic Solver
+ Michael Garstka
+ University of Oxford, 2017 - 2019
+------------------------------------------------------------------
+
+Problem: x ∈ R^{13},
+ constraints: A ∈ R^{35x13} (70 nnz),
+ matrix size to factor: 48x48 (188 nnz)
+Sets: PsdConeTriangle of dim: 10
+ PsdConeTriangle of dim: 10
+ PsdConeTriangle of dim: 6
+ PsdConeTriangle of dim: 6
+ PsdConeTriangle of dim: 3
+Decomp: Num of original PSD cones: 1
+ Num decomposable PSD cones: 1
+ Num PSD cones after decomposition: 5
+ Merge Strategy: NoMerge
+Settings: ϵ_abs = 1.0e-04, ϵ_rel = 1.0e-04,
+ ϵ_prim_inf = 1.0e-06, ϵ_dual_inf = 1.0e-04,
+ ρ = 0.1, σ = 1.0e-6, α = 1.6,
+ max_iter = 2500,
+ scaling iter = 10 (on),
+ check termination every 40 iter,
+ check infeasibility every 40 iter,
+ KKT system solver: QDLDL
+Setup Time: 0.14ms
+
+Iter: Objective: Primal Res: Dual Res: Rho:
+40 -1.4134e+00 1.2320e-05 3.0409e-05 1.0000e-01
+
+------------------------------------------------------------------
+>>> Results
+Status: Solved
+Iterations: 40
+Optimal objective: -1.4134
+Runtime: 0.002s (1.93ms)
Under sets we can indeed see that COSMO solved a problem with five PSD constraints corresponding to the cliques $\mathcal{C}_1,\ldots, \mathcal{C}_5$ discovered in the sparsity pattern. (Note that the dimension printed is the number of entries in the upper triangle of the matrix block.)
After we have found the cliques of the sparsity pattern, we are allowed to merge some of them back together. For the graph of the sparsity pattern this just means adding more edges or treating some structural zeros as numerical zeros. The main reason to merge two cliques is that they might overlap a lot and therefore it is not advantageous to treat them as two different blocks. Consider the two extreme cases below:
In the left figure we have the ideal case that all the blocks overlap in just one entry. A full decomposition would leave us with a large number of small blocks. The sparsity pattern in the right figure has two large blocks overlapping almost entirely. In this case it would be disadvantageous to decompose the blocks. Instead, we would do the initial decomposition, realize the large overlap, and then merge the two blocks back together. For sparsity patterns that arise from real applications the case is not always as clear and we have to use more sophisticated strategies to decide which blocks to merge.
COSMO currently provides three different strategies that can be selected by the user:
The merge strategy suggested in Sun and Andersen - Decomposition in conic optimization with partially separable structure (2014). The initial clique tree is traversed in topological order and a clique $\mathcal{C}_\ell$ is greedily merged to its parent clique $\mathcal{C}_{par(\ell)}$ if at least one of the two conditions are met
The (default) merge strategy based on the reduced clique graph $\mathcal{G}(\mathcal{B}, \xi)$, for a set of cliques $\mathcal{B} = \{ \mathcal{C}_1, \dots, \mathcal{C}_p\}$ where the edge set $\xi$ is obtained by taking the edges of the union of clique trees.
Moreover, given an edge weighting function $e(\mathcal{C}_i,\mathcal{C}_j) = w_{ij}$, we compute a weight for each edge that quantifies the computational savings of merging the two cliques. After the initial weights are computed, we merge cliques in a loop:
while clique graph contains positive weights:
select two permissible cliques with the highest weight $w_{ij}$
merge cliques $\rightarrow$ update clique graph
recompute weights for updated clique graph
Custom edge weighting functions can be used by defining your own CustomEdgeWeight <: AbstractEdgeWeight and a corresponding edge_metric method. By default, the ComplexityWeight <: AbstractEdgeWeight is used which computes the weight based on the cardinalities of the cliques: $e(\mathcal{C}_i,\mathcal{C}_j) = |\mathcal{C}_i|^3 + |\mathcal{C}_j|^3 - |\mathcal{C}_i \cup \mathcal{C}_j|^3$.
See also: Garstka, Cannon, Goulart - A clique graph based merging strategy for decomposable SDPs (2019)
In our example problem we have two cliques $\mathcal{C}_3 = \{ 3,6,7,8\}$ and $\mathcal{C}_5 = \{6,7,8,9 \}$ that overlap in three entries. Let's solve the problem again and choose the default clique merging strategy merge_strategy = COSMO.CliqueGraphMerge:
model = JuMP.Model(with_optimizer(COSMO.Optimizer, decompose = true, merge_strategy = COSMO.CliqueGraphMerge));
+@variable(model, x[1:2]);
+@objective(model, Min, c' * x )
+@constraint(model, Symmetric(B - A1 .* x[1] - A2 .* x[2] ) in JuMP.PSDCone());
+
+JuMP.optimize!(model)
------------------------------------------------------------------
+ COSMO v0.5.0 - A Quadratic Objective Conic Solver
+ Michael Garstka
+ University of Oxford, 2017 - 2019
+------------------------------------------------------------------
+
+Problem: x ∈ R^{7},
+ constraints: A ∈ R^{30x7} (58 nnz),
+ matrix size to factor: 37x37 (153 nnz)
+Sets: PsdConeTriangle of dim: 15
+ PsdConeTriangle of dim: 6
+ PsdConeTriangle of dim: 6
+ PsdConeTriangle of dim: 3
+Decomp: Num of original PSD cones: 1
+ Num decomposable PSD cones: 1
+ Num PSD cones after decomposition: 4
+ Merge Strategy: CliqueGraphMerge
+Settings: ϵ_abs = 1.0e-04, ϵ_rel = 1.0e-04,
+ ϵ_prim_inf = 1.0e-06, ϵ_dual_inf = 1.0e-04,
+ ρ = 0.1, σ = 1.0e-6, α = 1.6,
+ max_iter = 2500,
+ scaling iter = 10 (on),
+ check termination every 40 iter,
+ check infeasibility every 40 iter,
+ KKT system solver: QDLDL
+Setup Time: 0.26ms
+
+Iter: Objective: Primal Res: Dual Res: Rho:
+40 -1.4134e+00 1.7924e-05 1.0416e-04 1.0000e-01
+
+------------------------------------------------------------------
+>>> Results
+Status: Solved
+Iterations: 40
+Optimal objective: -1.4134
+Runtime: 0.003s (2.68ms)
Unsurprisingly, we can see in the output that COSMO solved a problem with four PSD constraints. One of them is of dimension 15, i.e. a $5\times 5$ block, which correspond to the merged clique $\mathcal{C_3} \cup \mathcal{C}_5 = \{3,6,7,8,9 \}$.
After a decomposed problem is solved, we can recover the solution to the original problem by assembling the matrix variable $S$ from its subblocks $S_\ell$:
Following Agler's Theorem, $S$ will be a positive semidefinite matrix. However, this is not true for the corresponding dual variable matrix $Y$. The dual variable returned after solving the decomposed problem will be in the space of PSD completable matrices $Y \in \mathbb{S}_+^n(E,?)$. This means that the entries in $Y$ corresponding to the blocks $S_\ell$ (black dots) have been chosen correctly. The numerical values for all the other entries (corresponding to the zeros in $S$ and denoted with a red dot) have to be chosen in the right way to make $Y$ positive semidefinite.
For more information about PSD matrix completion and the completion algorithm used in COSMO take a look at Vandenberghe and Andersen - Chordal Graphs and Semidefinite Optimization (Ch.10). To configure COSMO to complete the dual variable after solving the problem you have to set the complete_dual option:
model = JuMP.Model(with_optimizer(COSMO.Optimizer, complete_dual = true));
We consider the problem of finding the closest correlation matrix $X$ to a given random matrix $C$. With closest correlation matrix we mean a positive semidefinite matrix with ones on the diagonal. The problem is given by:
Notice that we use JuMP to model the problem. COSMO is chosen as the backend solver. And COSMO-specific settings are passed using the optimizer_with_attributes() function.
using COSMO, JuMP, LinearAlgebra, SparseArrays, Test, Random
rng = Random.MersenneTwister(12345);
+# create a random test matrix C
+n = 8;
+C = -1 .+ rand(rng, n, n) .* 2;
+c = vec(C);
Some example problems that are solved with COSMO can be found in the /examples folder.
In the first example the native solver interface is used to define and solve a Linear Program. In the second example JuMP is used to describe the problem and COSMO is set as the solver backend.
We consider the problem of finding the closest correlation matrix X to a given random matrix C. With closest correlation matrix we mean a positive semidefinite matrix with ones on the diagonal. The problem is given by:
Logistic regression problems can be solved using exponential cone constraints. An example on how to use COSMO to solve a logistic regression problem is presented in /examples/logistic_regression_regularization.ipynb.
Settings
This document was generated with Documenter.jl version 0.27.25 on Sunday 23 July 2023. Using Julia version 1.9.2.
In this example we use logistic regression to estimate the parameters $\theta_i$ of a logistic model in a classification problem. We will first transform the logistic regression problem into an exponential cone optimisation problem. We will then solve the optimisation problem with COSMO and determine the model parameters and the decision boundary.
The plot shows two test scores of $n$ microchip samples from a fabrication plant and whether the chip passed the quality check. Based on this data we would like to build a logistic model that takes into account the test scores and helps us predict the likelihood of a chip being accepted.
The vector $x$ represents the independent variables and $\theta$ represents the model parameters. For our samples we set the dependent variable $y =1$ if the chip was accepted and $y = 0$ otherwise.
The function $h_\theta(x)$ can be interpreted as the probability of the outcome being true rather than false. We want to find the parameters $\theta$ that maximize the log-likelihood over all (independently Bernoulli distributed) observations
As our dataset only has two independent variables (the test scores) our model $y = \theta_0 + \theta_1 x_1 + \theta_2 x_2$ will have the form of a straight line. Looking at the plot one can see that a line will not perform well in separating the samples. Therefore, we will create more features based on each data point by mapping the original features ($x_1$, $x_2$) into all polynomial terms of $x_1$ and $x_2$ up to the 6th power:
We can rewrite above likelihood maximisation problem as a conic optimisation problem with exponential-cone-, second-order-cone-, equality-, and inequality constraints:
Implementing the constraint $\log(1 + \exp(z)) \leq \epsilon$ for each of the $n$ samples requires two exponential cone constraints, one inequality constraint and two equality constraints. To see this, take the exponential on both sides and then divide by $\exp(\epsilon)$ to get:
\[K_{\text{exp}} = \{(r, s, t) \mid s >0, s \exp(r/s) \leq t \} \cup \{ r \leq 0, s = 0, t \geq 0 \}.\]
Based on this transformation our optimisation problem will have $5n + n_\theta + 1$ variables, 1 SOCP constraint, $2n$ exponential cone constraints, $n$ inequality constraints and $2n$ equality constraints. Let's model the problem with JuMP and COSMO:
n_theta = size(X, 2)
+n = n_data
+μ = 1.
+
+m = JuMP.Model(COSMO.Optimizer)
+@variable(m, v)
+@variable(m, θ[1:n_theta])
+@variable(m, e[1:n])
+@variable(m, t1[1:n])
+@variable(m, t2[1:n])
+@variable(m, s1[1:n])
+@variable(m, s2[1:n])
+
+@objective(m, Min, μ * v + sum(e))
+@constraint(m, [v; θ] in MOI.SecondOrderCone(n_theta + 1))
+
+# create the constraints for each sample points
+for i = 1:n
+ yi = y[i]
+ x = X[i, :]
+ yi == 1. ? (a = -1) : (a = 1)
+ @constraint(m, [a * dot(θ, x) - e[i]; s1[i]; t1[i] ] in MOI.ExponentialCone())
+ @constraint(m, [-e[i]; s2[i]; t2[i]] in MOI.ExponentialCone())
+ @constraint(m, t1[i] + t2[i] <= 1)
+ @constraint(m, s1[i] == 1)
+ @constraint(m, s2[i] == 1)
+end
+JuMP.optimize!(m)
------------------------------------------------------------------
+ COSMO v0.8.8 - A Quadratic Objective Conic Solver
+ Michael Garstka
+ University of Oxford, 2017 - 2022
+------------------------------------------------------------------
+
+Problem: x ∈ R^{619},
+ constraints: A ∈ R^{1091x619} (4513 nnz),
+ matrix size to factor: 1710x1710,
+ Floating-point precision: Float64
+Sets: ZeroSet of dim: 236
+ Nonnegatives of dim: 118
+ SecondOrderCone of dim: 29
+ ExponentialCone of dim: 3
+ ExponentialCone of dim: 3
+ ... and 234 more
+Settings: ϵ_abs = 1.0e-05, ϵ_rel = 1.0e-05,
+ ϵ_prim_inf = 1.0e-04, ϵ_dual_inf = 1.0e-04,
+ ρ = 0.1, σ = 1e-06, α = 1.6,
+ max_iter = 5000,
+ scaling iter = 10 (on),
+ check termination every 25 iter,
+ check infeasibility every 40 iter,
+ KKT system solver: QDLDL
+Acc: Anderson Type2{QRDecomp},
+ Memory size = 15, RestartedMemory,
+ Safeguarded: true, tol: 2.0
+Setup Time: 9.91ms
+
+Iter: Objective: Primal Res: Dual Res: Rho:
+1 -1.2883e+03 2.5314e+01 1.7951e+00 1.0000e-01
+25 4.5884e+01 3.4757e-02 1.0876e-02 1.0000e-01
+50 5.2041e+01 3.4769e-04 1.3217e-04 1.0000e-01
+75 5.1927e+01 1.6111e-05 1.1864e-05 1.0000e-01
+
+------------------------------------------------------------------
+>>> Results
+Status: Solved
+Iterations: 75
+Optimal objective: 51.93
+Runtime: 0.654s (654.34ms)
Once we have solved the optimisation problem and obtained the parameter vector $\theta$, we can plot the decision boundary. This can be done by evaluating our model over a grid of points $(u,v)$ and then plotting the contour line where the function $g$ returns a probability of $p=0.5$.
# First we evaluate our model over a grid of points z = θ' x
+u = collect(range(-1., stop = 1.5, length = 50))
+v = collect(range(-1., stop = 1.5, length = 50))
+z = zeros(length(u), length(v));
+for i = 1:length(u), j = 1:length(v)
+ z[i, j] = dot(map_feature(u[i], v[j]), theta);
+end
To add the decision boundary we have to plot the line indicating $50\%$ probability of acceptance, i.e. $g(\theta^\top x) = g(z) = 0.5$ which we get at $z=0$.
plot(x1[1:n_half-1], x2[1:n_half-1], color = :blue, st = :scatter, markershape = :cross, aspect_ratio=:equal, label = "Accepted", xlabel = "x1 - Microchip Test Score 1", ylabel = "x2 - Microchip Test Score 2")
+plot!(x1[n_half:end], x2[n_half:end], color = :red, st = :scatter, markershape = :circle, label = "Rejected")
+contour!(u, v, z', levels = [0.], c = :black, linewidth = 2)
We can solve the problem directly in COSMO by using its modeling interface. The problem will have $nn = 5 n + n_\theta + 1$ variables. Let us define the cost function $\frac{1}{2}x^\top P x + q^\top x$:
nn = 5 * n + n_theta + 1
+P = spzeros(nn, nn)
+q = zeros(nn)
+q[1] = μ
+for i = 1:n
+ q[1 + n_theta + (i - 1) * 5 + 1] = 1.
+end
Next we define a function that creates the COSMO.Constraints for a given sample:
# the order of the variables
+# v, thetas, [e1 t11 t12 s11 s12] [e2 t21 t22 s21 s22] ...
+# for each sample create two exponential cone constraints,
+# 1 nonnegatives constraint, 2 zeroset constraints
+function add_log_regression_constraints!(constraint_list, x, y, n, sample_num)
+ num_thetas = length(x)
+ # 1st exponential cone constraint (zi - ei, s1, t1) in Kexp
+ c_start = 1 + num_thetas + (sample_num - 1) * 5 + 1
+ A = spzeros(3, n)
+ A[1, c_start] = -1.
+ y == 1. ? (a = -1) : (a = 1)
+ for k = 1:num_thetas
+ A[1, 2 + k - 1] = a * x[k]
+ end
+ A[2, c_start + 3] = 1.
+ A[3, c_start + 1] = 1.
+ b = zeros(3)
+ push!(constraint_list, COSMO.Constraint(A, b, COSMO.ExponentialCone))
+
+ # 2nd exponential cone constraint (-e, s2, t2)
+ A = spzeros(3, n)
+ A[1, c_start] = -1.
+ A[2, c_start + 4] = 1.
+ A[3, c_start + 2] = 1.
+ b = zeros(3)
+ push!(constraint_list, COSMO.Constraint(A, b, COSMO.ExponentialCone))
+
+ # Nonnegatives constraint t1 + t2 <= 1
+ A = spzeros(1, n)
+ A[1, c_start + 1] = -1.
+ A[1, c_start + 2] = -1.
+ b = [1.]
+ push!(constraint_list, COSMO.Constraint(A, b, COSMO.Nonnegatives))
+
+ # ZeroSet constraint s1 == 1, s2 == 1
+ A = spzeros(2, n)
+ A[1, c_start + 3] = 1.
+ A[2, c_start + 4] = 1.
+ b = -1 * ones(2)
+ push!(constraint_list, COSMO.Constraint(A, b, COSMO.ZeroSet))
+end
+nothing
Now we can use this function to loop over the sample points and add the constraints to our constraint list:
constraint_list = Array{COSMO.Constraint{Float64}}(undef, 0)
+for i = 1:n
+ add_log_regression_constraints!(constraint_list, X[i, :], y[i], nn, i )
+end
It remains to add a second order cone constraint for the regularisation term: $\|\theta \|_2 \leq v$
The Lovász theta function is an important concept in graph theory. Consider an undirected graph $G = (V,E)$ with vertex set $V = \{1,\dots,n\}$ and edge set $E$ with $E_{ij} = 1$ if there exists an edge between the vertices $i$ and $j$. A stable set (or independent set) is a subset of $V$ such that the induced subgraph does not contain edges. The stability number$\alpha(G)$ of the graph is equal to the cardinality of the largest stable set. It is closely related to the Shannon capacity$\Theta(G)$ that models the amount of information that a noisy communication channel can carry if certain signal values can be confused with each other. Unfortunately, the determination of $\alpha(G)$ is an NP-hard problem and the computational complexity to determine $\Theta(G)$ remains unknown. The Lovász theta function $\vartheta(G)$ was introduced in [1], can be computed in polynomial time, and represents an upper bound on both the stability number and the Shannon capacity:
\[\alpha(G) \leq \Theta(G) \leq \vartheta(G).\]
The value of $\vartheta(G)$ can be determined by computing the optimal value $p^*$ of the following SDP:
with matrix variable $X$, matrix $J \in \mathbf{R}^{n \times n}$ of all ones and $\text{Tr}()$ denoting the matrix trace. In this simple example we will compute the value of the Lovász theta function for the Petersen Graph (which is known to be $4$).
We are interested in solving an approximation to the maximum cut problem using semidefinite programming. Consider the graph $G(V,E)$ with weights $w_{ij}$ shown below:
The maximum cut problem tries to find a cut or a partition of the graph's vertices into two complementary sets $S$ and $\bar{S}$ such that the total weight of the edges between the two sets is maximized. For this small graph the problem is trivial. The optimal solution is $S = \{1,2,3 \}$ and $\bar{S}=\{4\}$. Formally, this problem can be written as a mixed-integer optimisation problem:
where $y_i = 1$ indicates that $v_i \in S$ and $y_i = -1$ indicates that $v_i \in \bar{S}$. This problem is of interest in the field of integrated circuit design, where one tries to minimize the number of cross-layer connections in a circuit under layout constraints.
For more complicated graphs this problem quickly becomes hard to solve to optimality and in fact is known to be NP-hard. For this example we are interested in the randomized approximation algorithm that relaxes the problem to an SDP and was devised in Goemans and Williamson (1995) [1] (see for more details).
The approach can be divided into three steps:
Solve a relaxed SDP to obtain $Y^*$
Recover an approximate solution $V$ via a Cholesky factorisation $Y^* = V^\top V$
Round the approximate solution using a random vector $r$ from the unit sphere.
The authors showed that this approach guarantees a solution of at least $0.87856$ times the optimal solution.
Notice that the decision matrix $Y$ is generally dense (as correctly classified in the solver output above). Therefore, we won't be able to utilize COSMO's chordal decomposition features. However, assuming strong duality, it turns out that we can also solve the dual problem, which is given by:
\[\begin{array}{ll} \text{minimize} & \sum_i \gamma_i \\
+\text{subject to} & \text{diag}(\gamma) - \frac{1}{4} L = S \\
+ & S \in \mathbf{S}_{+}^n.
+\end{array}\]
As you can see, the matrix $S$ is constrained to have zeros in places specified by the graph (Laplacian). Therefore, COSMO can try to decompose this SDP (see solver output) and speed up its algorithm:
To get the correct positive semidefinite dual variable, we have to enable PSD completion of the dual variable in COSMO. Now, that we have a solution we can perform the remaining steps in the approximation algorithm.
We consider a single-period Markowitz portfolio optimisation example. Assume that we have a portfolio with $n$ assets at the beginning of time period $t$. Given some forecasts on risks and expected returns we try to find the optimal trade vector that rebalances the portfolio to achieve a good balance between expected risk (variance) $x^\top \Sigma x$ and returns $\mu^\top x$. In it's most simple form we want to solve:
\[\begin{array}{ll} \text{maximize} & \mu^\top x - \gamma (x^\top \Sigma x)\\
+\text{subject to} & 1^\top x = d + 1^\top x^0 \\
+ & x \geq 0,
+\end{array}\]
with variable $x \in \mathbf{R}^n$, $\mu$ forecasted (expected) returns, $\gamma > 0 $ risk aversion parameter. $x^0_i$ represents the initial investment in asset $i$ and $d$ represents the cash reserve. Consequently, the equality constraint tells us that the sum of the new allocation vector $x$ has to equal the initial allocation plus the cash reserve. Furthermore, the covariance matrix of our risk model is given by $\Sigma \in \mathbf{S}_+^n$. Here we assume a factor risk model, i.e. we can write $\Sigma$ as $\Sigma = D + F F^\top$ where $D$ is diagonal and the factor matrix $F$ has a lower rank $k < n$. This approach allows us to reduce the number of nonzeros in the problem. Furthermore, note that we don't consider shortselling in this example. Let's generate some problem data:
using LinearAlgebra, SparseArrays, Random, COSMO, JuMP, Test
+
+# generate the data
+rng = Random.MersenneTwister(1)
+k = 5; # number of factors
+n = k * 10; # number of assets
+D = spdiagm(0 => rand(rng, n) .* sqrt(k))
+F = sprandn(rng, n, k, 0.5); # factor loading matrix
+μ = (3 .+ 9. * rand(rng, n)) / 100. # expected returns between 3% - 12%
+γ = 1.0; # risk aversion parameter
+d = 1 # we are starting from all cash
+x0 = zeros(n);
We can now write the problem as a QP:
\[\begin{array}{ll} \text{minimize} & x^\top D x + y^\top y - \gamma^{-1} \mu^\top x \\
+\text{subject to} & y = F^\top x \\
+ & 1^\top x = d + 1^\top x^0 \\
+ & x \geq 0.
+\end{array}\]
Before considering other effects, let's create the model in JuMP and solve it using COSMO:
model = JuMP.Model(COSMO.Optimizer);
+@variable(model, x[1:n]);
+@variable(model, y[1:k]);
+@objective(model, Min, x' * D * x + y' * y - 1/γ * μ' * x);
+@constraint(model, y .== F' * x);
+@constraint(model, sum(x) == d + sum(x0));
+@constraint(model, x .>= 0);
+JuMP.optimize!(model)
It is pointed out in [1] that above problem formulation can lead to numerical problems, e.g. if $\Sigma$ is not strictly positive semidefinite. Another option is to formulate the risk constraint in terms of the standard deviation $\|M^\top x \|$ where $M M^\top = D + F F^\top$ and bound it using a second-order cone constraint:
\[\begin{array}{ll} \text{minimize} & - \mu^\top x \\
+\text{subject to} & \|M^\top x\| \leq \gamma \\
+ & 1^\top x = d + 1^\top x^0 \\
+ & x \geq 0.
+\end{array}\]
Note that the result is different from the example above because $\gamma$ scales the problem in a different way. Here it can be seen as an upper bound on the standard deviation of the portfolio.
The above portfolio optimisation approach yields the optimal expected return for a given level of risk. The result is obviously impacted by the risk aversion $\gamma$ parameter. To visualise the trade-off and present the investor with an efficient Pareto optimal portfolio for their risk appetite we can compute the optimal portfolio for many choices of $\gamma$ and plot the corresponding risk-return trade-off curve.
gammas = [ 0.001, 0.01, 0.1, 0.5, 1., 3., 10, 100, 1000]
+risks = zeros(length(gammas))
+returns = zeros(length(gammas))
+model = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, "verbose" => false));
+@variable(model, x[1:n]);
+@variable(model, y[1:k]);
+@objective(model, Min, x' * D * x + y' * y - 1/γ * μ' * x);
+@constraint(model, y .== F' * x);
+@constraint(model, sum(x) == d + sum(x0));
+@constraint(model, x .>= 0);
+
+# solve the same problem for different values of γ
+for (k, gamma) in enumerate(gammas)
+ coeff = - 1/gamma * μ
+ JuMP.set_objective_coefficient.(model, x, coeff)
+ JuMP.optimize!(model)
+ local x_opt = JuMP.value.(x);
+ local y_opt = JuMP.value.(y);
+ returns[k] = dot(μ, x_opt)
+ risks[k] = sqrt(dot(y_opt, y_opt))
+end
We can now plot the risk-return trade-off curve:
using Plots
+Plots.plot(risks, returns, xlabel = "Standard deviation (risk)", ylabel = "Expected return", title = "Risk-return trade-off for efficient portfolios", legend = false)
+
+
Note
When the model is updated in JuMP as above the JuMP.model is copied in full to COSMO. We are trying to improve the interface with respect to model updates in the future. Until then you can use Model Updates in COSMOs native interface.
In the model above we assume that trading the assets is free and does not impact the market. However, this is clearly not the case in reality. To make the example more realistic consider the following cost $c_j$ associated with the trade $δ_j = x_j - x_j^0$:
where the first term models the bid-ask spread and broker fees for asset $j$. The second term models the impact on the market that our trade has. This is obviously only a factor if the volume of our trade is significant. The constant $b_j$ is a function of the total volume traded in the considered time periode and the price volatility of the asset and has to be estimated by the trader. To make this example simple we consider the same coefficients $a$ and $b$ for every asset. The $|\delta_j|^{3/2}$ term can be easily modeled using a power cone constraint $\mathcal{K}_{pow} = \{(x, y, z) \mid x^\alpha y^{(1-\alpha)} \geq |z|, x \geq 0, y \geq 0, 0 \leq \alpha \leq 1 \}$. In fact this can be used to model any market impact function with exponent greater than 1. We can write the total transaction cost $a^\top s + b^\top t$ where $s_j$ bounds the absolute value of $\delta_j$ and $t_{j}$ is used to bound the term $|x_j - x_j^0|^{3/2} \leq t_{j}$ using a power cone formulation: $(t_{j}, 1, x_j - x_j^0) \in \mathcal{K}_{pow}(2/3)$.
a = 1e-3
+b = 1e-1
+γ = 1.0;
+model = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, "eps_abs" => 1e-5, "eps_rel" => 1e-5));
+@variable(model, x[1:n]);
+@variable(model, y[1:k]);
+@variable(model, t[1:n]);
+@variable(model, s[1:n]);
+@objective(model, Min, x' * D * x + y' * y - 1/γ * μ' * x);
+@constraint(model, y .== F' * x);
+@constraint(model, x .>= 0);
+
+# transaction costs
+@constraint(model, sum(x) + a * sum(s) + b * sum(t) == d + sum(x0) );
+@constraint(model, [i = 1:n], x[i] - x0[i] <= s[i]); # model the absolute value with slack variable s
+@constraint(model, [i = 1:n], x0[i] - x[i] <= s[i]);
+@constraint(model, [i = 1:n], [t[i], 1, x[i] - x0[i]] in MOI.PowerCone(2/3));
+JuMP.optimize!(model)
------------------------------------------------------------------
+ COSMO v0.8.8 - A Quadratic Objective Conic Solver
+ Michael Garstka
+ University of Oxford, 2017 - 2022
+------------------------------------------------------------------
+
+Problem: x ∈ R^{155},
+ constraints: A ∈ R^{306x155} (628 nnz),
+ matrix size to factor: 461x461,
+ Floating-point precision: Float64
+Sets: Nonnegatives of dim: 150
+ ZeroSet of dim: 6
+ PowerCone of dim: 3
+ PowerCone of dim: 3
+ PowerCone of dim: 3
+ ... and 47 more
+Settings: ϵ_abs = 1.0e-05, ϵ_rel = 1.0e-05,
+ ϵ_prim_inf = 1.0e-04, ϵ_dual_inf = 1.0e-04,
+ ρ = 0.1, σ = 1e-06, α = 1.6,
+ max_iter = 5000,
+ scaling iter = 10 (on),
+ check termination every 25 iter,
+ check infeasibility every 40 iter,
+ KKT system solver: QDLDL
+Acc: Anderson Type2{QRDecomp},
+ Memory size = 15, RestartedMemory,
+ Safeguarded: true, tol: 2.0
+Setup Time: 4.81ms
+
+Iter: Objective: Primal Res: Dual Res: Rho:
+1 -1.5637e-01 1.1398e+00 6.3250e-02 1.0000e-01
+25 -7.7854e-02 2.0863e-03 5.8259e-05 1.0000e-01
+50 -7.7862e-02 4.1404e-05 3.8814e-07 1.0000e-01
+75 -7.7862e-02 1.7501e-06 3.3924e-08 1.0000e-01
+
+------------------------------------------------------------------
+>>> Results
+Status: Solved
+Iterations: 80 (incl. 5 safeguarding iter)
+Optimal objective: -0.07786
+Runtime: 0.1s (100.2ms)
Let's look at the expected return and the total transaction cost:
This document was generated with Documenter.jl version 0.27.25 on Sunday 23 July 2023. Using Julia version 1.9.2.
diff --git a/v0.8.8/examples/sum_abs_k_eigenvalues/index.html b/v0.8.8/examples/sum_abs_k_eigenvalues/index.html
new file mode 100644
index 00000000..799c6be6
--- /dev/null
+++ b/v0.8.8/examples/sum_abs_k_eigenvalues/index.html
@@ -0,0 +1,87 @@
+
+Minimizing the sum of the k-largest λ · COSMO.jl
We show how to find the sum of absolute value of the k largest eigenvalues of a symmetric matrix $A \in \mathbb{S}^n$. This problem can be solved as a semidefinite program. The primal and dual forms are stated in Alizadeh [1]:
\[\begin{array}{llll} \text{maximize} & \text{Tr}(AY) - \text{Tr}(AW) & \text{minimize} & kz + Tr(U) + Tr(V) \\
+\text{subject to} & \text{Tr}(Y + W) = k & \text{subject to} & zI + V - A \succeq 0 \\
+ & 0 \preceq Y \preceq I & & zI + U + A \succeq 0 \\
+ & 0 \preceq W \preceq I & & U, V \succeq 0,
+\end{array}\]
where $Y, W$ are the variables of the primal and $U, V$ are the variables of the dual problem.
We are interested in minimizing the sum of absolute values of the k=3 largest eigenvalues. Let's formulate the problem in JuMP with COSMO as the backend solver:
Above problems are mostly helpful for illustrative purpose. It is obviously easier to find the sum of the k-largest eigenvalues by simply computing the eigenvalues of $A$. However, above results become useful if finding $A$ itself is part of the problem. For example, assume we want to find a valid matrix $A$ under the constraints: $C\, \text{vec}(A) = b$ with the minimum sum of absolute values of the k-largest eigenvalues. We can then solve the equivalent problem:
\[\begin{array}{ll} \text{minimize} & kz + Tr(U) + Tr(V) \\
+ \text{subject to} & C \text{vec}(A) = b \\
+ & zI + V - A \succeq 0 \\
+ & zI + U + A \succeq 0 \\
+ & U, V \succeq 0.
+\end{array}\]
where $\lambda$ is a hyperparameter. This problem can be solved as a quadratic program. We can rewrite above problem into an optimisation problem in primal form by introducing the auxiliary slack variables $t_i$:
The problem can also be solved by transforming it directly into COSMO's problem format. Define COSMO`s $x$-variable to be $x=[w, t]^\top$ and choose $P$, $q$, accordingly:
Next we transform the first constraint $y_i (w^\top x_i - b) \geq 1 - t_i, \quad \text{for } i = 1,\ldots, m$ into COSMO's constraint format: $Ax + b \in \mathcal{K}$.
We consider the (nonconvex) two-way partitioning problem from Boyd and Vandenberghe (2004), p.219 [1]:
\[\begin{array}{ll} \text{minimize} & x^\top W x \\
+\text{subject to} & x_i^2 = 1, \quad i=1,\dots, n,
+\end{array}\]
with $x \in \mathbf{R}^n$ and $W \in \mathbf{S}^n$. The problem can be interpreted as finding a partition of the points $i = 1,\dots, n$ into two sets, where the cost of two points in the same set is $W_{ij}$ and $-W_{ij}$ otherwise. Brute forcing the solution $x^*$ takes $2^n$ attempts and becomes quickly intractable, e.g. for $n \geq 30$. However, a lower-bound for this problem can be computed by solving the convex dual of the problem:
Solving this problem with optimal objective $d^*$ yields a lower bound as least as good as a lower bound based on the minimum eigenvalue $d^* \geq \lambda_{\text{min}}(W)$. Let's set up the problem for $n = 20$ and with a specially structured (banded) $W$:
As you can see, the matrix $W$ imposes a structure on the LMI-constraint. Accordingly, COSMO will be able to use chordal decomposition to decompose the LMI constraint into multiple smaller constraints. This can make a significant difference in the performance of the algorithm for large $n$.
model = JuMP.Model(COSMO.Optimizer);
+@variable(model, ν[1:n]);
+@objective(model, Max, -ones(n)' * ν )
+@constraint(model, Symmetric(W + diagm(ν) ) in JuMP.PSDCone());
+JuMP.optimize!(model)
------------------------------------------------------------------
+ COSMO v0.8.8 - A Quadratic Objective Conic Solver
+ Michael Garstka
+ University of Oxford, 2017 - 2022
+------------------------------------------------------------------
+
+Problem: x ∈ R^{38},
+ constraints: A ∈ R^{57x38} (56 nnz),
+ matrix size to factor: 95x95,
+ Floating-point precision: Float64
+Sets: PsdConeTriangle of dim: 3 (2x2)
+ PsdConeTriangle of dim: 3 (2x2)
+ PsdConeTriangle of dim: 3 (2x2)
+ PsdConeTriangle of dim: 3 (2x2)
+ PsdConeTriangle of dim: 3 (2x2)
+ ... and 14 more
+Decomp: Num of original PSD cones: 1
+ Num decomposable PSD cones: 1
+ Num PSD cones after decomposition: 19
+ Merge Strategy: CliqueGraphMerge
+Settings: ϵ_abs = 1.0e-05, ϵ_rel = 1.0e-05,
+ ϵ_prim_inf = 1.0e-04, ϵ_dual_inf = 1.0e-04,
+ ρ = 0.1, σ = 1e-06, α = 1.6,
+ max_iter = 5000,
+ scaling iter = 10 (on),
+ check termination every 25 iter,
+ check infeasibility every 40 iter,
+ KKT system solver: QDLDL
+Acc: Anderson Type2{QRDecomp},
+ Memory size = 15, RestartedMemory,
+ Safeguarded: true, tol: 2.0
+Setup Time: 0.14ms
+
+Iter: Objective: Primal Res: Dual Res: Rho:
+1 -6.1146e+02 1.7072e+01 5.9998e-01 1.0000e-01
+25 2.3958e+01 2.9812e-01 1.1258e-09 1.0000e-01
+50 2.5146e+01 2.0059e-01 1.6359e-08 1.0000e-01
+75 2.5430e+01 1.3731e-01 4.0566e-11 1.0000e-01
+100 2.5624e+01 1.2992e-10 3.3307e-16 1.0000e-01
+
+------------------------------------------------------------------
+>>> Results
+Status: Solved
+Iterations: 121 (incl. 21 safeguarding iter)
+Optimal objective: 25.62
+Runtime: 0.098s (98.42ms)
Looking at the solver output you can see how the PSD constraint was decomposed into 19 PSD constraints. Let's look at the lower bound:
JuMP.objective_value(model)
-25.623709242674522
As $n$ is small, we can verify our result by finding the optimal solution by trying out all possible combinations:
function brute_force_optimisation(W, n)
+ opt_obj = Inf
+ opt_x = Inf * ones(n)
+
+ for xt in Iterators.product([[1.; -1.] for k = 1:n]...)
+ x = [i for i in xt]
+ obj_val = x' * W * x
+ if obj_val < opt_obj
+ opt_obj = obj_val
+ opt_x = x
+ end
+ end
+ return opt_obj, opt_x
+end
+opt_obj, opt_x = brute_force_optimisation(W, n)
This user guide describes the basic structures and functions to define an optimisation problem, to solve the problem and to analyse the result. If you want to use JuMP to describe the problem, see the JuMP Interface section.
COSMO solves optimisation problems in the following format:
\[\begin{array}{ll} \text{minimize} & \textstyle{\frac{1}{2}}x^\top Px + q^\top x\\ \text{subject to} & Ax + s = b \\ & s \in \mathcal{K}, \end{array}\]
with decision variables $x \in \mathbb{R}^n$, $s \in \mathbb{R}^m$ and data matrices $P=P^\top \succeq 0$, $q \in \mathbb{R}^n$, $A \in \mathbb{R}^{m \times n}$, and $b \in \mathbb{R}^m$. The convex set $\mathcal{K}$ is a composition of convex sets and cones.
To set the objective function of your optimisation problem simply define the square positive semidefinite matrix $P \in \mathrm{R}^{n\times n}$ and the vector $q \in \mathrm{R}^{n}$. You might have to transform your optimisation problem into a solver compatible format for this step.
The COSMO interface expects constraints to have the form $A_i x + b_i \in \mathcal{K}_i$, where $\mathcal{K}_i$ is one of the convex sets defined below:
Convex Set
Description
ZeroSet
The set $\{ 0 \}^{dim}$ that contains the origin
Nonnegatives
The nonnegative orthant $\{ x \in \mathbb{R}^{dim} : x_i \ge 0, \forall i=1,\dots,\mathrm{dim} \}$
Box(l, u)
The hyperbox $\{ x \in \mathbb{R}^{dim} : l \leq x \leq u\}$ with vectors $l \in \mathbb{R}^{dim} \cup \{-\infty\}$ and $u \in \mathbb{R}^{dim} \cup \{+\infty\}$
SecondOrderCone
The second-order (Lorenz) cone $\{ (t,x) \in \mathbb{R}^{dim} : |x|_2 \leq t \}$
PsdCone
The vectorized positive semidefinite cone $\mathcal{S}_+^{dim}$. $x$ is the vector obtained by stacking the columns of the positive semidefinite matrix $X$, i.e. $X \in \mathbb{S}^{\sqrt{dim}}_+ \rarr \text{vec}(X) = x \in \mathcal{S}_+^{dim}$
PsdConeTriangle
The vectorized positive semidefinite cone $\mathcal{S}_+^{dim}$. $x$ is the vector obtained by stacking the columns of the upper triangular part of the positive semidefinite matrix $X$, i.e. $X \in \mathbb{S}^{d}_+ \rarr \text{svec}(X) = x \in \mathcal{S}_+^{dim}$ where $d=\sqrt{1/4 + 2 \text{dim}} - 1/2$
ExponentialCone
The exponential cone $\mathcal{K}_{exp} = \{(x, y, z) \mid y \geq 0, ye^{x/y} ≤ z\} \cup \{ (x,y,z) \mid x \leq 0, y = 0, z \geq 0 \}$
DualExponentialCone
The dual exponential cone $\mathcal{K}^*_{exp} = \{(x, y, z) \mid x < 0, -xe^{y/x} \leq e^1 z \} \cup \{ (0,y,z) \mid y \geq 0, z \geq 0 \}$
PowerCone(alpha)
The 3d power cone $\mathcal{K}_{pow} = \{(x, y, z) \mid x^\alpha y^{(1-\alpha)} \geq |z|, x \geq 0, y \geq 0 \}$ with $0 < \alpha < 1$
DualPowerCone(alpha)
The 3d dual power cone $\mathcal{K}^*_{pow} = \{(u, v, w) \mid \left( \frac{u}{\alpha}\right)^\alpha \left( \frac{v}{1-\alpha}\right)^{(1-\alpha)} \geq |w|, u \geq 0, v \geq 0 \}$ with $0 < \alpha < 1$
The constructor for a constraint expects a matrix A, a vector b and a convex_set.
Lets consider a problem with a decision variable $x \in \mathbb{R}^5$. Suppose we want to create the two constraint $x_2 + 5 \geq 0$ and $x_3 - 3 \geq 0$. We can do this either by creating two constraints and adding them to an array:
Another way to construct the constraint is to used the optional arguments dim, the dimension of x, and indices, the elements of x that appear in the constraint. When specifying these arguments, A and b only refer to the elements of x in indices:
Consider as a second example the positive semidefinite constraint on a matrix $X \in \mathbb{S}_+^{3}$. Our decision variable is the vector $x$ obtained by stacking the columns of $X$. We can specify the constraint on $x$ in the following way:
It is usually enough to pass the convex_set as a type. However, some convex sets like Box, PowerCone and DualPowerCone require more information to be created. In that case you have to pass an object to the constructor, e.g.
To adjust those values, either pass your preferred option and parameter as a key-value pair to the constructor or edit the corresponding field afterwards. For example if you want to enable verbose printing and increase the solver accuracy, you can type
settings = COSMO.Settings(verbose = true, eps_abs = 1e-5, eps_rel = 1e-5)
+# the following is equivalent
+settings = COSMO.Settings()
+settings.verbose = true
+settings.eps_abs = 1e-5
+settings.eps_rel = 1e-5
Once the objective function and an array of constraints have been defined, we can assemble the model with
COSMO.assemble!(model, P, q, constraints)
This simply sets the corresponding variables in the model and transforms the array of constraints into the problem format defined at the top of the page.
If you want to change the default settings, you can pass your settings object custom_settings to the assemble! function:
COSMO.assemble!(model, P, q, constraints, settings = custom_settings)
One of the advantages of ADMM-based solvers is that they can be easily warm started. By providing starting values for the primal variable x and/or the dual variable y in the vicinity of their optimal values, the number of iterations to convergence can often be dramatically decreased.
Consider the case where you have a decision variable $x \in \mathbb{R}^3$ and a dual variable $y \in \mathbb{R}^2$. Assume you expect their optimal values to be close to $x_0 = (1, 5, 3)$ and $y_0 = (1, 2)$. You can pass these values when assembling the model.
After the model has been assembled, we can solve the problem by typing
results = COSMO.optimize!(model)
Once the solver algorithm terminates, it will return a Results object that gives information about the status of the solver. If successful, it contains the optimal objective value and optimal primal and dual variables. For more information see the following section.
In some cases we want to solve a large number of similar models. COSMO allows you to update the model problem data vectors q and b after the first call of optimize!(). After changing the problem data, COSMO can reuse the factorisation step of the KKT matrix from the previous problem which can save a lot of time in the case of LPs and QPs.
COSMO.jl is a Julia implementation of the Conic Operator Splitting Method. The underlying ADMM-algorithm is well-suited for large convex conic problems. COSMO solves the following problem:
\[\begin{array}{ll} \text{minimize} & \textstyle{\frac{1}{2}}x^\top Px + q^\top x\\ \text{subject to} & Ax + s = b \\ & s \in \mathcal{K},
+\end{array}\]
with decision variables $x \in \mathbb{R}^n$, $s \in \mathbb{R}^m$ and data matrices $P=P^\top \succeq 0$, $q \in \mathbb{R}^n$, $A \in \mathbb{R}^{m \times n}$, and $b \in \mathbb{R}^m$. The convex set $\mathcal{K}$ is a composition of convex sets and cones.
Versatile: COSMO solves linear programs, quadratic programs, second-order cone programs, semidefinite programs and problems involving exponential and power cones
Quad SDPs: Positive semidefinite programs with quadratic objective functions are natively supported
Safeguarded acceleration: robust and faster convergence to higher precision using COSMOAccelerators
Infeasibility detection: Infeasible problems are detected without a homogeneous self-dual embedding of the problem
JuMP / Convex.jl support: We provide an interface to MathOptInterface (MOI), which allows you to describe your problem in JuMP and Convex.jl.
Chordal decomposition: COSMO tries to decompose large structured PSD constraints into multiple smaller PSD constraints using chordal decomposition techniques. This often results in a significant speedup compared to solving the original problem.
Smart clique merging: After an initial decomposition of a structured SDP, COSMO recombines overlapping cliques/blocks to speed up the algorithm.
Warm starting: COSMO supports warm starting of the decision variables
Arbitrary precision types: You can solve problems with any floating point precision.
Open Source: Our code is available on GitHub and distributed under the Apache 2.0 Licence
Our JuMP interface allows you to describe and modify your optimisation problem with JuMP and use COSMO as the backend solver. The interface is defined in /src/MOIWrapper.jl.
Note
We assume here that the latest JuMP release (~0.21.0) is used.
To specify COSMO as the solver for your JuMP model, load the solver module with using COSMO and then pass a COSMO.Optimizer when initialising the JuMP model:
Solver-specific settings can be passed using the optimizer_with_attributes() function. For example, if you want to adjust the maximum number of iterations and turn on verbose printing use
m = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, "max_iter" => 5000, "verbose" => true));
Note that the attributes are passed as key-value pairs and the keys are strings. This is slightly different to using the native COSMO interface. Equivalently, one can also use:
m = JuMP.Model(COSMO.Optimizer);
+set_optimizer_attribute(m, "max_iter", 5000)
+set_optimizer_attribute(m, "verbose", true)
The full list of available settings can be found in the Settings section. All of them should be compatible with JuMP.
In case JuMP.termination_status(m) is MOI.ITERATION_LIMIT, JuMP.primal_status(m) (resp. JuMP.dual_status(m)) should be checked to determine the feasibility status of the primal (resp. dual) solutions. A primal (resp. dual) solution is considered feasible (resported as MOI.FEASIBLE_POINT) if
$r$ is the primal (resp. dual) residual r_prim$= \lVert Ax + s - b \rVert_\infty$
(resp. r_dual$= \lVert P x + q - A^\top * μ \rVert_\infty$),
$s$ is the primal (resp. dual) scaling of this residual
max_norm_prim$= \max(\lVert Ax \rVert_\infty, \lVert s \rVert_\infty, \lVert b \rVert_\infty)$ (resp. max_norm_dual$= \max(\lVert P x \rVert_\infty, \lVert q \rVert_\infty, \lVert A^\top * μ \rVert_\infty)$).
$\epsilon_{\text{abs}}$ is the optimizer attribute eps_abs and
$\epsilon_{\text{rel}}$ is the optimizer attribute eps_rel.
A primal (resp. dual solution) is considered nearly feasible (resported as MOI.NEARLY_FEASIBLE_POINT) if it is not feasible but the same inequality is satisfied where the right-hand side is multiplied by the optimizer attribute nearly_ratio.
Otherwise, the solution is considered infeasible (resported as MOI.INFEASIBLE_POINT). Note that this does not mean that the problem is infeasible, it only means that the current primal (resp. dual) solution is infeasible.
Note that, in case JuMP.termination_status(m) is MOI.ITERATION_LIMIT, the feasibility of the solution is checked with the current optimizer attributes, eps_abs, eps_rel and nearly_ratio. This means that if they are changed after optimize!, the status may change! This can be used to check whether the solution is feasible up to a relaxed tolerance. For instance, the following could happen:
optimize!(m)
+primal_status(m) # MOI.INFEASIBLE_POINT
+# This means that `r_prim >= nearly_ratio * (eps_abs * max_norm_prim * eps_rel)`
+set_optimizer_attribute(m, "eps_abs", 1e-4)
+primal_status(m) # MOI.NEARLY_FEASIBLE_POINT
+# This means that `1 <= r_prim / (eps_abs * max_norm_prim * eps_rel) < nearly_ratio`
The values of r_prim, max_norm_prim, r_dual and max_norm_dual can also be accessed as the fields of the res_info struct that can be obtained as follows
res_info = MOI.get(m, COSMO.RawResult()).info
Then, the feasibility can either be checked manually as in
One major step of COSMO's ADMM algorithm is solving a linear system of equations at each iteration. Fortunately, the left-hand matrix is only dependent on the problem data and therefore only needs to be factored once. Depending on the problem class this factorisation can be the computationally most expensive step of the algorithm (LPs, QPs). See the Method section for a more detailed description of the linear system.
COSMO allows you to specify the linear system solver that performs the factorisation and back-substitution. We also support indirect system solver which are useful for very large problems where a factorisation becomes inefficient. The table below shows the currently supported linear system solver:
Type
Solver
Description
CholmodKKTSolver
Cholmod
Julia's default linear system solver (from SuiteSparse)
Conjugate Gradients on the reduced KKT linear system.
MINRESIndirectKKTSolver
IterativeSolvers.jl
MINRES on the (full) KKT linear system.
Note
To use the Pardiso and Intel MKL Pardiso solver, you have to install the respective libraries and the corresponding Julia wrapper. For more information about installing these, visit the Pardiso.jl repository page. Likewise in order to use Indirect(Reduced)KKTSolver you have to install IterativeSolvers.jl (v0.9+) and LinearMaps.jl. We are using the Requires package for lazy loading of code related to Pardiso and IterativeSolvers. This means in order to use Pardiso / IterativeSolvers, you'll have to load these packages alongside COSMO, i.e. using Pardiso and using IterativeSolvers, LinearMaps.
COSMO uses the Cholmod linear system solver by default. You can specify a different solver in the settings by using the kkt_solver keyword and the respective type:
COSMO also allows you to pass in solver-specific options with the with_options(solver_type, args...; kwargs...) syntax. For example, if you want to use Pardiso with verbose printing use the following syntax:
Likewise, CGIndirectKKTSolver and MINRESIndirectKKTSolver are also parameterizable with with_options(solver_type, args...; kwargs...) and accept the following arguments:
Argument
Description
Values (default)
tol_constant::T and tol_exponent::T
Parameter that defines the solution tolerance for the iterative solvers across iterations. In particular, the solution tolerance at every iteration is defined as $\text{tol\_constant} / \text{iteration}^{\text{tol\_exponent}}$
1.0, 1.5
This also works if you want to use this configuration with JuMP:
model = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, "kkt_solver" => with_options(PardisoDirectKKTSolver, msg_level_on = true));
+
Or alternatively:
model = JuMP.Model(COSMO.Optimizer);
+set_optimizer_attribute(model, "kkt_solver", with_options(PardisoDirectKKTSolver, msg_level_on = true));
Settings
This document was generated with Documenter.jl version 0.27.25 on Sunday 23 July 2023. Using Julia version 1.9.2.
diff --git a/v0.8.8/literate/arbitrary_precision.jl b/v0.8.8/literate/arbitrary_precision.jl
new file mode 100644
index 00000000..5cae23c3
--- /dev/null
+++ b/v0.8.8/literate/arbitrary_precision.jl
@@ -0,0 +1,41 @@
+# # Arbitrary Precision
+#
+# COSMO allows you to solve problems with arbitrary floating-point precision, e.g. by using `BigFloat` problem data. To do this, the desired floating point type has to be consistent across the model `COSMO.Model{<: AbstractFloat}`, the input data and the (optional) settings object `COSMO.Settings{<: AbstractFloat}`.
+# As an example, assume we want to solve the following quadratic program:
+#
+# $$
+# \begin{array}{ll} \text{minimize} & 1/2 x^\top P x + q^\top x \\
+# \text{subject to} & l \leq A x \leq u
+# \end{array}
+# $$
+# where $P = \begin{bmatrix} 4 & 1 \\ 1 & 2\end{bmatrix}$, $q = [1, 1]^\top$, $A = \begin{bmatrix} 1 & 1 \\ 1 & 0 \\ 0 & 1\end{bmatrix}$ and $l= [1,0, 0]^\top$, $u=[1, 0.7, 0.7]^\top$. We start by creating the model with the desired precision:
+using COSMO, LinearAlgebra, SparseArrays
+model = COSMO.Model{BigFloat}()
+
+#-
+# Next, we define the problem data as `BigFloat` arrays and create the constraint:
+q = BigFloat[1; 1.]
+P = sparse(BigFloat[4. 1; 1 2])
+A = BigFloat[1. 1; 1 0; 0 1]
+l = BigFloat[1.; 0; 0]
+u = BigFloat[1; 0.7; 0.7]
+
+constraint = COSMO.Constraint(A, zeros(BigFloat, 3), COSMO.Box(l, u))
+
+# Notice that the constraint type parameter is dependent on the input data. The same is true for the constraint set `Box`. Next, we define the settings
+settings = COSMO.Settings{BigFloat}(verbose = true, kkt_solver = QdldlKKTSolver)
+
+# and assemble and solve the problem:
+assemble!(model, P, q, constraint, settings = settings)
+result = COSMO.optimize!(model);
+
+
+# Moreover, notice that when no type parameter is specified, all objects default to `Float64`:
+model = COSMO.Model()
+
+# Two limitations to arbitrary precision:
+# - Since we call the LAPACK function `syevr` for eigenvalue decompositions, we currently only support solving problems with PSD constraints in `Float32` and `Float64`.
+# - We suggest to use the pure Julia QDLDL linear system solver (`kkt_solver = QdldlKKTSolver`) when working with arbitrary precision types as some of the other available solvers don't support all available precisions.
+
+#md # !!! note
+#md # `JuMP` does not currently support arbitrary precision. However, if you want to use `COSMO` directly with `MathOptInterface`, you can use: `COSMO.Optimizer{<: AbstractFloat}` as your optimizer. Again, the problem data precision of your MathOptInterface-model has to agree with the optimizer's precision.
diff --git a/v0.8.8/literate/build/arbitrary_precision/index.html b/v0.8.8/literate/build/arbitrary_precision/index.html
new file mode 100644
index 00000000..28025490
--- /dev/null
+++ b/v0.8.8/literate/build/arbitrary_precision/index.html
@@ -0,0 +1,54 @@
+
+Arbitrary Precision · COSMO.jl
COSMO allows you to solve problems with arbitrary floating-point precision, e.g. by using BigFloat problem data. To do this, the desired floating point type has to be consistent across the model COSMO.Model{<: AbstractFloat}, the input data and the (optional) settings object COSMO.Settings{<: AbstractFloat}. As an example, assume we want to solve the following quadratic program:
\[\begin{array}{ll} \text{minimize} & 1/2 x^\top P x + q^\top x \\
+\text{subject to} & l \leq A x \leq u
+\end{array}\]
where $P = \begin{bmatrix} 4 & 1 \\ 1 & 2\end{bmatrix}$, $q = [1, 1]^\top$, $A = \begin{bmatrix} 1 & 1 \\ 1 & 0 \\ 0 & 1\end{bmatrix}$ and $l= [1,0, 0]^\top$, $u=[1, 0.7, 0.7]^\top$. We start by creating the model with the desired precision:
using COSMO, LinearAlgebra, SparseArrays
+model = COSMO.Model{BigFloat}()
A COSMO Model with Float precision: BigFloat
+
Next, we define the problem data as BigFloat arrays and create the constraint:
Moreover, notice that when no type parameter is specified, all objects default to Float64:
model = COSMO.Model()
A COSMO Model with Float precision: Float64
+
Two limitations to arbitrary precision:
Since we call the LAPACK function syevr for eigenvalue decompositions, we currently only support solving problems with PSD constraints in Float32 and Float64.
We suggest to use the pure Julia QDLDL linear system solver (kkt_solver = QdldlKKTSolver) when working with arbitrary precision types as some of the other available solvers don't support all available precisions.
Note
JuMP does not currently support arbitrary precision. However, if you want to use COSMO directly with MathOptInterface, you can use: COSMO.Optimizer{<: AbstractFloat} as your optimizer. Again, the problem data precision of your MathOptInterface-model has to agree with the optimizer's precision.
This example demonstrate how the user can extend COSMO with his own custom convex cones and use them in constraints. To make things easy, we will implement the simple cone of Nonpositives, i.e. $\mathbb{R}_{-}^n = \{x \in \mathbb{R}^n \mid x_i \leq 0\}$.
We start by creating a concrete subtype of our cone interface-type: COSMO.AbstractConvexCone{T}, where T is a parameter for the floating-point precision. The only field that is strictly required for our cone definition is dim, which is used by the solver to store information about the cone dimension. However, you can also use the object to store other information, or allocate the workspace necessary for the projection step.
using COSMO, LinearAlgebra, SparseArrays, Test
+
+# define new cone type
+struct Nonpositives{T} <: COSMO.AbstractConvexCone{T}
+ dim::Int64
+end
Next, we define a projection method for Nonpositives{T} that takes a vector x and projects it onto our custom cone. For the cone of nonpositive vectors the projection simply sets all positive elements of x to zero:
function COSMO.project!(x::AbstractVector{T}, C::Nonpositives{T}) where {T <: AbstractFloat}
+ x .= min.(x, zero(T))
+end
This is all that is required to get a basic constraint working. Let's test our new cone/constraint by solving the following LP:
The problem was solved using constraints involving our new cone and yields the expected result. Next, we want to add the ability to detect infeasible problems.
If no further information about the new cone is provided, the infeasibility detection is disabled. However, by defining the two additional methods in_dual and in_pol_recc we can support infeasibility detection. More information on how infeasible problems are detected in COSMO can be found here [1]. We will have to add the ability for the solver to check whether a vector is in the dual cone$\mathcal{K}^*$ of our new cone $\mathcal{K}$ and whether a vector is in the recession cone of the polar cone${\mathcal{K}^\circ}^\infty$ of our cone. The first function is used to check for primal infeasibility, while the second function is used to check for dual infeasibility. Luckily, for the Nonpositives-cone this is straightforward:
Notice that for numerical reasons, we give the check-functions a bit of slack using the tolerance parameter tol. To test the infeasibility detection, we will attempt to use our new cone to solve the clearly dual infeasible problem:
The nice composibility of Julia-code allows us to use our new cone definition even when the problem is modelled with JuMP. For this to work, we define a concrete subtype of MOI.AbstractSet, that JuMP can use to build a constraint.
using MathOptInterface, JuMP
+import Base.copy
+const MOI = MathOptInterface
+
+struct NonPos <: MOI.AbstractVectorSet
+ dimension::Int
+end
+Base.copy(set::NonPos) = set #this is needed for MOI
WARNING: using JuMP.Nonpositives in module Main conflicts with an existing identifier.
Next, we tell COSMO that whenever JuMP wants to solve a problem with a NonPos-cone it should translate it into a Nonpositives constraint and we also tell MOI that we now support constraints of this new mysterious cone NonPos.
function COSMO.processSet!(b::Vector{T}, rows::UnitRange{Int}, cs, s::NonPos) where {T <: AbstractFloat}
+ push!(cs, Nonpositives{T}(length(rows)))
+ nothing
+end
+#tell MOI, that COSMO supports constraints with this new cone
+MOI.supports_constraint(optimizer::COSMO.Optimizer, ::Type{<:Union{MOI.VectorOfVariables, MOI.VectorAffineFunction{Float64}}}, ::Type{NonPos}) = true
We should now be able to model the LP from above in JuMP and solve it internally using our Nonpositives cone and projection function.
You can see in the solver output that indeed a set Nonpositives of dim: 2 was used.
The discussed cone of Nonpositives is of course trivial, but the ability to define new cones and constraints for COSMO can be very powerful to model complex problems.
When a pareto-optimal front in portfolio optimisation is computed for different risk-aversion parameters $\gamma$, we are repeatedly solving the same model with only a slight change in the cost function. This allows us to reuse the KKT factorisation from previous solves and recompute the solution very efficiently, see Algorithm. For more information about solving portfolio optimsiation problems with COSMO see Portfolio Optimisation. We are planning to solve the following QP for different values of $\gamma$:
\[\begin{array}{ll} \text{minimize} & x^\top D x + y^\top y - \gamma^{-1} \mu^\top x \\
+\text{subject to} & y = F^\top x \\
+ & 1^\top x = d + 1^\top x^0 \\
+ & x \geq 0,
+\end{array}\]
As you can see $\gamma$ enters only in the linear part of the cost function, i.e. our vector q. Let's generate some example data:
using LinearAlgebra, SparseArrays, Random, COSMO
+
+rng = Random.MersenneTwister(1)
+k = 5; # number of factors
+n = k * 10; # number of assets
+D = spdiagm(0 => rand(rng, n) .* sqrt(k))
+F = sprandn(rng, n, k, 0.5); # factor loading matrix
+μ = (3 .+ 9. * rand(rng, n)) / 100. # expected returns between 3% - 12%
+d = 1 # we are starting from all cash
+x0 = zeros(n);
We want to solve this problem for the following risk-parameters:
After a successful solve, we can use the update!(model, q = nothing, b = nothing) function to update our model and re-solve the problem efficiently:
# solve the problem efficiently for the remaining γ using model updates
+function solve_repeatedly!(returns, risks, gammas, model, k, n, μ)
+ for (i, γ) in enumerate(gammas[2:end])
+ q_new = [-μ ./ (2 * γ); zeros(k)];
+
+ # here we use the update! function to change the model
+ update!(model, q = q_new)
+ result = COSMO.optimize!(model);
+
+ x_opt = view(result.x, 1:n)
+ y_opt = view(result.x, n+1:n+k)
+ returns[i + 1] = dot(μ, x_opt)
+ risks[i + 1] = sqrt(dot(y_opt, y_opt))
+ end
+end
+solve_repeatedly!(returns, risks, gammas, model, k, n, μ);
We can now plot the risk-return trade-off curve:
using Plots
+Plots.plot(risks, returns, xlabel = "Standard deviation (risk)", ylabel = "Expected return", title = "Risk-return trade-off for efficient portfolios", legend = false)
This document was generated with Documenter.jl version 0.27.25 on Sunday 23 July 2023. Using Julia version 1.9.2.
diff --git a/v0.8.8/literate/custom_cone.jl b/v0.8.8/literate/custom_cone.jl
new file mode 100644
index 00000000..327cc256
--- /dev/null
+++ b/v0.8.8/literate/custom_cone.jl
@@ -0,0 +1,126 @@
+# # Custom Cone Constraint
+#
+# This example demonstrate how the user can extend `COSMO` with his own custom convex cones and use them in constraints. To make things easy, we will implement the simple cone of **Nonpositives**, i.e. $\mathbb{R}_{-}^n = \{x \in \mathbb{R}^n \mid x_i \leq 0\}$.
+# ## Cone definition and projection function
+# We start by creating a concrete subtype of our cone interface-type: `COSMO.AbstractConvexCone{T}`, where `T` is a parameter for the floating-point precision. The only field that is strictly required for our cone definition is `dim`, which is used by the solver to store information about the cone dimension. However, you can also use the object to store other information, or allocate the workspace necessary for the projection step.
+using COSMO, LinearAlgebra, SparseArrays, Test
+
+## define new cone type
+struct Nonpositives{T} <: COSMO.AbstractConvexCone{T}
+ dim::Int64
+end
+#-
+# Next, we define a projection method for `Nonpositives{T}` that takes a vector `x` and projects it onto our custom cone. For the cone of nonpositive vectors the projection simply sets all positive elements of `x` to zero:
+
+function COSMO.project!(x::AbstractVector{T}, C::Nonpositives{T}) where {T <: AbstractFloat}
+ x .= min.(x, zero(T))
+end
+
+#-
+# This is all that is required to get a basic constraint working. Let's test our new cone/constraint by solving the following LP:
+# $$
+# \begin{array}{ll} \text{maximize} & x_1 + x_2 + x_3\\
+# \text{subject to} & x_1 \leq 3 \\
+# & x_2 \leq 2 \\
+# & x_1 + x_3 = 5
+# \end{array}
+# $$
+# We define our decision vector $x = (x_1, x_2, x_3)$ and setup the problem:
+n = 3
+P = spzeros(n, n)
+q = -ones(n)
+
+## A1 * x + b1 <= 0
+A1 = diagm(0 => ones(2))
+b1 = [-3.; -2.]
+c1 = COSMO.Constraint(A1, b1, Nonpositives, n, 1:2); #here we use our new cone
+
+## x_1 + x_3 == 5
+A2 = [1. 0 1.]
+b2 = [-5.]
+c2 = COSMO.Constraint(A2, b2, COSMO.ZeroSet);
+
+## assemple and solve
+model = COSMO.Model();
+assemble!(model, P, q, [c1; c2]);
+res = COSMO.optimize!(model);
+
+#-
+res.x
+
+#-
+obj_val = -dot(q, res.x)
+# The problem was solved using constraints involving our new cone and yields the expected result. Next, we want to add the ability to detect infeasible problems.
+#
+# ## Support infeasibility detection
+# If no further information about the new cone is provided, the infeasibility detection is disabled. However, by defining the two additional methods `in_dual` and `in_pol_recc` we can support infeasibility detection. More information on how infeasible problems are detected in `COSMO` can be found here \[1\]. We will have to add the ability for the solver to check whether a vector is in the [dual cone](https://en.wikipedia.org/wiki/Dual_cone_and_polar_cone) $\mathcal{K}^*$ of our new cone $\mathcal{K}$ and whether a vector is in the recession cone of the [polar cone](https://en.wikipedia.org/wiki/Dual_cone_and_polar_cone) ${\mathcal{K}^\circ}^\infty$ of our cone. The first function is used to check for primal infeasibility, while the second function is used to check for dual infeasibility. Luckily, for the Nonpositives-cone this is straightforward:
+#
+# $$
+# {\mathbb{R}_{-}^n}^* = \mathbb{R}_{-}^n, \quad {{\mathbb{R}_{-}^n}^\circ}^\infty = \mathbb{R}_{+}^n.
+# $$
+
+function COSMO.in_dual(x::AbstractVector{T}, C::Nonpositives{T}, tol::T) where {T <: AbstractFloat}
+ return !any( x-> (x > -tol), x)
+end
+
+function COSMO.in_pol_recc(x::AbstractVector{T}, C::Nonpositives{T}, tol::T) where {T <: AbstractFloat}
+ return !any( x-> (x < tol), x)
+end
+# Notice that for numerical reasons, we give the check-functions a bit of slack using the tolerance parameter `tol`.
+# To test the infeasibility detection, we will attempt to use our new cone to solve the clearly dual infeasible problem:
+# $$
+# \begin{array}{ll} \text{minimize} & x\\
+# \text{subject to} & x\leq 3
+# \end{array}
+# $$
+n = 1
+P = spzeros(n, n)
+q = [1.]
+
+## x <= 3 <=> x - 3 in Nonpositives
+Ai = [1.]
+bi = [-3.]
+ci = COSMO.Constraint(Ai, bi, Nonpositives);
+
+model = COSMO.Model();
+assemble!(model, P, q, [ci]);
+res = COSMO.optimize!(model);
+#-
+@test res.status == :Dual_infeasible
+
+# which is what we would expect.
+# ## Using new cone with JuMP
+# The nice composibility of Julia-code allows us to use our new cone definition even when the problem is modelled with `JuMP`. For this to work, we define a concrete subtype of `MOI.AbstractSet`, that JuMP can use to build a constraint.
+using MathOptInterface, JuMP
+import Base.copy
+const MOI = MathOptInterface
+
+struct NonPos <: MOI.AbstractVectorSet
+ dimension::Int
+end
+Base.copy(set::NonPos) = set #this is needed for MOI
+# Next, we tell `COSMO` that whenever `JuMP` wants to solve a problem with a `NonPos`-cone it should translate it into a `Nonpositives` constraint and we also tell `MOI` that we now support constraints of this new mysterious cone `NonPos`.
+function COSMO.processSet!(b::Vector{T}, rows::UnitRange{Int}, cs, s::NonPos) where {T <: AbstractFloat}
+ push!(cs, Nonpositives{T}(length(rows)))
+ nothing
+end
+#tell MOI, that COSMO supports constraints with this new cone
+MOI.supports_constraint(optimizer::COSMO.Optimizer, ::Type{<:Union{MOI.VectorOfVariables, MOI.VectorAffineFunction{Float64}}}, ::Type{NonPos}) = true
+
+
+# We should now be able to model the LP from above in `JuMP` and solve it internally using our `Nonpositives` cone and projection function.
+model = JuMP.Model(COSMO.Optimizer);
+@variable(model, x[1:3]);
+@objective(model, Max, x[1] + x[2] + x[3]);
+@constraint(model, A1 * x[1:2] .+ b1 in NonPos(2));
+@constraint(model, x[1] + x[3] == 5);
+JuMP.optimize!(model)
+
+#-
+x_opt = JuMP.value.(x)
+# You can see in the solver output that indeed a set `Nonpositives` of `dim: 2` was used.
+#
+# The discussed cone of Nonpositives is of course trivial, but the ability to define new cones and constraints for `COSMO` can be very powerful to model complex problems.
+
+# ## References
+# [1] Garstka et al. - COSMO: A conic operator splitting method for convex conic problems (2020)
diff --git a/v0.8.8/literate/portfolio_model_updates.jl b/v0.8.8/literate/portfolio_model_updates.jl
new file mode 100644
index 00000000..c77adc24
--- /dev/null
+++ b/v0.8.8/literate/portfolio_model_updates.jl
@@ -0,0 +1,80 @@
+# # Model Updates
+#
+# When a pareto-optimal front in portfolio optimisation is computed for different risk-aversion parameters $\gamma$, we are repeatedly solving the same model with only a slight change in the cost function. This allows us to reuse the KKT factorisation from previous solves and recompute the solution very efficiently, see [Algorithm](@ref).
+# For more information about solving portfolio optimsiation problems with COSMO see [Portfolio Optimisation](@ref).
+# We are planning to solve the following QP for different values of $\gamma$:
+# $$
+# \begin{array}{ll} \text{minimize} & x^\top D x + y^\top y - \gamma^{-1} \mu^\top x \\
+# \text{subject to} & y = F^\top x \\
+# & 1^\top x = d + 1^\top x^0 \\
+# & x \geq 0,
+# \end{array}
+# $$
+# As you can see $\gamma$ enters only in the linear part of the cost function, i.e. our vector `q`. Let's generate some example data:
+using LinearAlgebra, SparseArrays, Random, COSMO
+
+rng = Random.MersenneTwister(1)
+k = 5; # number of factors
+n = k * 10; # number of assets
+D = spdiagm(0 => rand(rng, n) .* sqrt(k))
+F = sprandn(rng, n, k, 0.5); # factor loading matrix
+μ = (3 .+ 9. * rand(rng, n)) / 100. # expected returns between 3% - 12%
+d = 1 # we are starting from all cash
+x0 = zeros(n);
+
+# We want to solve this problem for the following risk-parameters:
+gammas = [ 0.001, 0.01, 0.1, 0.5, 1., 3., 10, 100, 1000]
+
+# Let's solve the problem for the first parameter:
+## x = (x, y)
+γ = gammas[1]
+P = blockdiag(D, spdiagm(0 => ones(k)))
+q = [-μ ./ (2 * γ); zeros(k)];
+
+Aeq = [ transpose(F) -diagm(0 => ones(k));
+ ones(1, n) zeros(1, k)]
+beq = [zeros(k); -1.0]
+Aineq = [spdiagm(0 => ones(n)) spzeros(n, k)]
+
+cs1 = COSMO.Constraint(Aeq, beq, COSMO.ZeroSet);
+cs2 = COSMO.Constraint(Aineq, zeros(n), COSMO.Nonnegatives);
+cosmo_settings = COSMO.Settings(verbose = false)
+
+## solve the problem once
+model = COSMO.Model();
+assemble!(model, P, q, [cs1; cs2], settings = cosmo_settings);
+result = COSMO.optimize!(model);
+x_opt = view(result.x, 1:n);
+y_opt = view(result.x, n+1:n+k);
+
+## allocate some space for results
+risks = zeros(length(gammas));
+returns = zeros(length(gammas));
+returns[1] = dot(μ, x_opt);
+risks[1] = sqrt(dot(y_opt, y_opt));
+
+
+
+# After a successful solve, we can use the `update!(model, q = nothing, b = nothing)` function to update our model and re-solve the problem efficiently:
+
+
+## solve the problem efficiently for the remaining γ using model updates
+function solve_repeatedly!(returns, risks, gammas, model, k, n, μ)
+ for (i, γ) in enumerate(gammas[2:end])
+ q_new = [-μ ./ (2 * γ); zeros(k)];
+
+ ## here we use the update! function to change the model
+ update!(model, q = q_new)
+ result = COSMO.optimize!(model);
+
+ x_opt = view(result.x, 1:n)
+ y_opt = view(result.x, n+1:n+k)
+ returns[i + 1] = dot(μ, x_opt)
+ risks[i + 1] = sqrt(dot(y_opt, y_opt))
+ end
+end
+solve_repeatedly!(returns, risks, gammas, model, k, n, μ);
+
+# We can now plot the risk-return trade-off curve:
+using Plots
+Plots.plot(risks, returns, xlabel = "Standard deviation (risk)", ylabel = "Expected return", title = "Risk-return trade-off for efficient portfolios", legend = false)
diff --git a/v0.8.8/logistic_regression/index.html b/v0.8.8/logistic_regression/index.html
new file mode 100644
index 00000000..608b7cf3
--- /dev/null
+++ b/v0.8.8/logistic_regression/index.html
@@ -0,0 +1,6 @@
+
+Logistic Regression · COSMO.jl
Logistic regression problems can be solved using exponential cone constraints. An example on how to use COSMO to solve a logistic regression problem is presented in /examples/logistic_regression_regularization.ipynb.
Settings
This document was generated with Documenter.jl version 0.27.25 on Sunday 23 July 2023. Using Julia version 1.9.2.
This section describes COSMO's underlying ADMM algorithm and how the user can use the settings to adjust this algorithm. For a more detailed explanation take a look at the associated publication in Citing COSMO.
COSMO solves problems with quadratic objective function and a number of conic constraints in the following form:
\[\begin{array}{ll} \text{minimize} & \textstyle{\frac{1}{2}}x^\top Px + q^\top x\\ \text{subject to} & Ax + s = b \\ & s \in \mathcal{K}, \end{array}\]
with primal decision variable $x \in \mathbb{R}^n$, primal slack variable $s \in \mathbb{R}^m$. The objective function is defined by positive semidefinite matrix $P=P^\top \succeq 0$ and vector $q \in \mathbb{R}^n$. The constraints are defined by matrix $A \in \mathbb{R}^{m \times n}$, vector $b \in \mathbb{R}^m$ and a non-empty, closed convex set $\mathcal{K}$. The convex set itself can be a Cartesian product of convex sets in the form:
The algorithm considers a slightly transformed problem. By introducing two dummy variables $\tilde{x} = x$ and $\tilde{s} = s$ we can rewrite the original problem:
where indicator functions $\mathcal{I}$ were used to move the constraints into the objective function. The resulting problem is now in the right format to apply the alternating direction method of multipliers (ADMM). To apply ADMM we first find the augmented Lagrangian $L$:
Minimizing the Lagrangian in an alternating fashion with respect to the two variable pairs $(\tilde{x},\tilde{s})$ and $(x,s)$ yields the following algorithm steps:
By the construction of the ADMM method those iterates are converging to the global solution. These steps are executed in a loop until convergence. Two important parameters are the ADMM steps sizes $\rho$ (Settings.rho) and $\sigma$ (Settings.sigma) which can be adjusted via the solver settings.
The two most important steps of the algorithm happen in the first and third line. The evaluation of the first line turns out to be an equality constrained quadratic program. We get a solution for $( \tilde{x}^{k+1},\tilde{s}^{k+1})$ at every iteration by solving the following linear system:
Fortunately, the left hand matrix doesn't change, which is why COSMO only has to factor the matrix once at the start of the algorithm.
The second important step in the algorithm is the update equation for $s^{k+1}$ which can be interpreted as a projection onto the constraint set $\mathcal{K}$:
The computational cost of this projection is highly dependent on the constraints of the problem. While projections onto the zero set or the nonnegative orthant are inexpensive, projections onto the positive semidefinite cone of order $N$ involve an eigen-decomposition. Since methods for eigen-decompositions have a complexity of $\mathcal{O}(N^3)$ the projection can become the computationally most expensive operation of the algorithm.
The convergence of ADMM-based algorithms depends on the relative scaling of the problem data. Especially to improve the convergence of badly scaled problems, COSMO tries to rescale the data in a preprocessing step.
We rescale the equality constraints with diagonal positive semidefinite matrices $D$ and $E$. The scaled problem is given by:
and the scaled convex cone $E\mathcal{K} = \{Ev \in \mathbb{R}^m \mid v \in \mathcal{K} \}$. After solving the scaled problem the original solution is obtained by reversing the scaling:
\[ x = D\hat{x}, \quad s = E^{-1}\hat{s}, \quad y = E\hat{y}.\]
To obtain the scaling matrices $D$ and $E$ we use a modified Ruiz equilibration algorithm which involves a certain number of scaling iterations to equilibrate the column norms of the data matrices $P$ and $A$. The number of these iterations can be adjusted by the user with scaling. To disable the scaling step set scaling = 0.
The COSMO algorithm can terminate for four reasons:
The maximum number of allowed iterations has been reached. The user can specify this value in the solver settings with max_iter.
The solver runtime reaches the time limit specified by the user (time_limit).
COSMO detects an infeasible problem.
The iterates fulfil the termination criteria for convergence.
COSMO uses the primal and dual residuals of the problem to determine if the algorithm has converged. The primal and dual residuals are given by:
\[\begin{aligned}
+r_p &= Ax + s -b,\\
+r_d &= Px + q + A^T y.
+\end{aligned}\]
The solver terminates when the $\infty$-norms of the residuals lie below a specified tolerance. COSMO uses the sum of an absolute and relative tolerance term:
The absolute and relative tolerances $\epsilon_{\mathrm{abs}}$and $\epsilon_{\mathrm{rel}}$ can be set by the user by specifying eps_abs and eps_rel. Furthermore, the user can adjust the number of iterations after which the convergence criteria are checked (check_termination).
COSMO uses conditions based on separating hyperplanes to detect infeasible problems. The conditions for COSMO's problem format have been developed in [1]. Define the convex set $\mathcal{C} = \mathcal{-K} + \{b\}$ then we can use the following infeasibility conditions:
The existence of some $y \in \mathcal{D}$ is a certificate that the problem is primal infeasible, while the existence of some $x \in \mathcal{P}$ is a certificate for dual infeasibility. COSMO regularly checks above conditions to detect infeasible problems. If the detection is successful, the solver terminates and returns the status codes :Primal_infeasible or :Dual_infeasible. COSMO checks the conditions every check_infeasibility iterations, which can be adjusted by the user.
There are a number of ways to improve the performance of the solver given a particular problem. If you are not satisfied with the performance there are a number of things you have to determine first. Is the solver slow because
it's the first time you ran it in the current Julia session or
is it because the solver needs a lot of iterations (convergence) or
is each iteration or the initial factorisation slow (computational performance)?
Whenever a new Julia session is started, the first run will trigger a compilation of all functions based on their arguments used in your script. This will slow the first execution of COSMO down. After that Julia will call the fast compiled functions. To get around this, you can either keep your current Julia session open and discard the first run. Alternatively, if your problem is very large, you could solve a small version of your problem first to trigger the compilation. Another option is to use PackageCompiler to save compiled functions into a sysimage that can be loaded at startup.
It is often instructive to look at the detailed solver timing results for your problem. This can reveal where most of the time is spent. To achieve this, run COSMO with the setting verbose_timing = true. After solving the problem with result = COSMO.optimize!(model) you can look at result.times for a breakdown of the times spent in different parts of the algorithm, see Timings for more details. Especially take a look at the ratio of factorisation time and iteration time. If you use JuMP to solve the problem, you can take a look at the timings with backend(model).optimizer.model.optimizer.results.times.
You could try changing any of the following parameters:
rho: The initial algorithm step parameter has a large influence on the convergence. Try different values between 1e-5 and 10.
adaptive_rho = false: You can try to disable the automatic rho adaption and use different rho values.
adaptive_rho_interval: This specifies after how many iterations COSMO tries to adapt the rho parameter. You can also set adaptive_rho_interval = 0 which adapts the rho parameter after the time spent iterating passes 40% of the factorisation time. This is currently the default in OSQP and works well with QPs.
alpha = 1.0: This disables the over-relaxation that is used in the algorithm. We recommend values between 1.0 - 1.6.
scaling = 0: This disables the problem scaling.
eps_abs and eps_rel: Check the impact of modifying the stopping accuracies.
The number of iterations can be dramatically decreased by providing a good initial guess for x, s and y. Examples where warm starting is commonly used are model predictive control and portfolio backtests, see Warm starting.
We experienced significant performance improvements on Intel CPUs if Julia is compiled with MKL BLAS. This is because Julia's linear algebra function will use Intel MKL BLAS and LAPACK functions that are optimised for Intel hardware. The effect is especially significant for SDPs because most of the time is spent in the LAPACK function syevr. If you are running Julia on Intel hardware, an easy way to compile Julia with MKL is to add and build the MKL package, see MKL.jl. To verify your current BLAS vendor you can use julia> LinearAlgebra.BLAS.vendor().
COSMO uses QDLDL.jl as the default linear system solver. In our experience this seems to be a competitive choice until about 1e5 - 1e6 nonzeros in the constraint matrix. After that it is worth trying one of the indirect system solvers, such as CG or MINRES. Furthermore, we also recommend trying Pardiso (or MKLPardiso) for problems of that dimension. More details can be found here: Linear System Solver.
In some cases the computations can be speed-up if certain constraints in the problem allow the implementation of a fast projection function. We allow the user to define their own custom convex cone with a corresponding projection function. The custom cone has to be defined as struct CustomCone{T} <: COSMO.AbstractConvexSet{T}. Furthermore, the user has to define a function that projects an input vector x onto the custom cone, i.e. function COSMO.project!(x::AbstractVector{T}, C::CustomCone{T}) where {T} ... end.
COSMO allows the execution of the projection step for multiple constraints in parallel using Julia's multithreading features. This is currently not enabled in the tagged release because of stability issues in earlier Julia versions. To use multithreading checkout the branch with_multi_threading, which we keep in sync with the latest tagged release. This can be installed via Julia's package manager with pkg> add COSMO#with_multi_threading. Afterwards, before starting Julia, set export JULIA_NUM_THREADS=[NUMBER_PHYSICAL_CORES_HERE]. In Julia you can verify the number of threads with julia> Threads.nthreads().
Notice that the extra overhead for multithreading can slow the solver down if the problem is small. However, we noticed significant performance improvements if the problem contained multiple positive semidefinite constraints or when one large constraint was decomposed. In that case it also helps to restrict the number of BLAS threads per Julia thread with julia> BLAS.set_num_threads(1) to prevent oversubscription of the available cores.
Multithreading can also be used in the factorisation step if the Pardiso or MKLPardiso solver are selected. This is only advisable for constraint matrices with more than 1e5 nonzeros.
When solving large structured and sparse SDPs significant performance improvements are achievable if the problem is passed to COSMO in the right way. This means the solver has to be able to infer the structure of the positive semidefinite variable from the constraint. See the section on Chordal Decomposition for more details. In some cases the primal SDP doesn't allow decomposition but the dual SDP does, consider the Maximum Cut Problem and the Relaxed Two-Way Partitioning Problem for examples.
If the problem is decomposable it is also worth experimenting with different clique merging strategies to see how they impact the performance. More details can be found here: Clique merging.
Settings
This document was generated with Documenter.jl version 0.27.25 on Sunday 23 July 2023. Using Julia version 1.9.2.
This document was generated with Documenter.jl version 0.27.25 on Sunday 23 July 2023. Using Julia version 1.9.2.
diff --git a/v0.8.8/search_index.js b/v0.8.8/search_index.js
new file mode 100644
index 00000000..3bf92e05
--- /dev/null
+++ b/v0.8.8/search_index.js
@@ -0,0 +1,3 @@
+var documenterSearchIndex = {"docs":
+[{"location":"logistic_regression/#Logistic-Regression","page":"Logistic Regression","title":"Logistic Regression","text":"","category":"section"},{"location":"logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"Logistic regression problems can be solved using exponential cone constraints. An example on how to use COSMO to solve a logistic regression problem is presented in /examples/logistic_regression_regularization.ipynb.","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"The source files for all examples can be found in /examples.","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"EditURL = \"https://github.com/oxfordcontrol/COSMO.jl/blob/master/examples/logistic_regression.jl\"","category":"page"},{"location":"examples/logistic_regression/#Logistic-Regression","page":"Logistic Regression","title":"Logistic Regression","text":"","category":"section"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"The presented example is adapted from the Machine Learning - course by Andrew Ng.","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"In this example we use logistic regression to estimate the parameters theta_i of a logistic model in a classification problem. We will first transform the logistic regression problem into an exponential cone optimisation problem. We will then solve the optimisation problem with COSMO and determine the model parameters and the decision boundary.","category":"page"},{"location":"examples/logistic_regression/#Visualizing-the-data","page":"Logistic Regression","title":"Visualizing the data","text":"","category":"section"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"Before we start, let's load and take a look at the example data from examples/chip_data.txt:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"using LinearAlgebra, SparseArrays, COSMO, JuMP\nusing Plots\n\n# load example data\nf = open(joinpath(@__DIR__, \"chip_data.txt\"))\nlines = readlines(f)\nclose(f)\nn_data = length(lines)\nn_half = div(n_data, 2)\nx1 = zeros(n_data)\nx2 = zeros(n_data)\ny = zeros(Float64, n_data)\nfor (i, l) in enumerate(lines)\n s = split(l, \",\")\n x1[i] = parse(Float64, s[1])\n x2[i] = parse(Float64, s[2])\n y[i] = parse(Float64, s[3])\nend\n\n\n# visualize data\nplot(x1[1:n_half-1], x2[1:n_half-1], color = :blue, st=:scatter, markershape = :cross, aspect_ratio=:equal, label = \"Accepted\", xlabel = \"x1 - Microchip Test Score 1\", ylabel = \"x2 - Microchip Test Score 2\")\nplot!(x1[n_half:end], x2[n_half:end], color = :red, st=:scatter, markershape = :circle, label = \"Rejected\")","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"The plot shows two test scores of n microchip samples from a fabrication plant and whether the chip passed the quality check. Based on this data we would like to build a logistic model that takes into account the test scores and helps us predict the likelihood of a chip being accepted.","category":"page"},{"location":"examples/logistic_regression/#Defining-the-logistic-model","page":"Logistic Regression","title":"Defining the logistic model","text":"","category":"section"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"The logistic regression hypothesis is given by","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"h_theta(x) = g(theta^top x)","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"where g is the sigmoid function:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"g(theta^top x) = frac11+exp(-theta^top x)","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"The vector x represents the independent variables and theta represents the model parameters. For our samples we set the dependent variable y =1 if the chip was accepted and y = 0 otherwise.","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"The function h_theta(x) can be interpreted as the probability of the outcome being true rather than false. We want to find the parameters theta that maximize the log-likelihood over all (independently Bernoulli distributed) observations","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"J(theta) = sum_i y_i = 1 log h_theta(x_i) + sum_i y_i = 0 log (1-h_theta(x_i))","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"Consequently, we want to solve the following optimization problem:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"textminimize quad -J(theta) + mu theta _2","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"where we added a regularization term with parameter mu to prevent overfitting.","category":"page"},{"location":"examples/logistic_regression/#Feature-mapping","page":"Logistic Regression","title":"Feature mapping","text":"","category":"section"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"As our dataset only has two independent variables (the test scores) our model y = theta_0 + theta_1 x_1 + theta_2 x_2 will have the form of a straight line. Looking at the plot one can see that a line will not perform well in separating the samples. Therefore, we will create more features based on each data point by mapping the original features (x_1, x_2) into all polynomial terms of x_1 and x_2 up to the 6th power:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"textmap_feature(x_1x_2) = 1 x_1 x_2 x_1^2 x_1x_2 x_2^2 x_1^3 dots x_1x_2^5 x_2^6 ","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"This will create 28 features for each sample.","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"function map_feature(x1, x2)\n deg = 6\n x_new = ones(length(x1))\n for i = 1:deg, j = 0:i\n x_new = hcat(x_new, x1.^(i-j) .* x2.^j)\n end\n return x_new\nend\n\nX = map_feature(x1, x2);\nsize(X)","category":"page"},{"location":"examples/logistic_regression/#Transformation-into-a-conic-optimisation-problem","page":"Logistic Regression","title":"Transformation into a conic optimisation problem","text":"","category":"section"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"We can rewrite above likelihood maximisation problem as a conic optimisation problem with exponential-cone-, second-order-cone-, equality-, and inequality constraints:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"beginarrayll\ntextminimize sum_i^n epsilon_i + mu v\ntextsubject to theta _2 leq v\n log(1 + exp(-theta^top x_i)) leq epsilon_i quad textif y_i = 1 \n log(1 + exp(theta^top x_i)) leq epsilon_i quadtext otherwise\nendarray","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"Implementing the constraint log(1 + exp(z)) leq epsilon for each of the n samples requires two exponential cone constraints, one inequality constraint and two equality constraints. To see this, take the exponential on both sides and then divide by exp(epsilon) to get:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"exp(-epsilon) + exp(z - epsilon) leq 1","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"This constraint is equivalent to:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"beginarrayll\n(z - epsilon s_1 t_1) in K_textexp \n(-epsilon s_2 t_2) in K_textexp \nt_1 + t_2 leq 1\ns_1 = s_2 = 1\nendarray","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"where we defined the exponential cone as:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"K_textexp = (r s t) mid s 0 s exp(rs) leq t cup r leq 0 s = 0 t geq 0 ","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"Based on this transformation our optimisation problem will have 5n + n_theta + 1 variables, 1 SOCP constraint, 2n exponential cone constraints, n inequality constraints and 2n equality constraints. Let's model the problem with JuMP and COSMO:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"n_theta = size(X, 2)\nn = n_data\nμ = 1.\n\nm = JuMP.Model(COSMO.Optimizer)\n@variable(m, v)\n@variable(m, θ[1:n_theta])\n@variable(m, e[1:n])\n@variable(m, t1[1:n])\n@variable(m, t2[1:n])\n@variable(m, s1[1:n])\n@variable(m, s2[1:n])\n\n@objective(m, Min, μ * v + sum(e))\n@constraint(m, [v; θ] in MOI.SecondOrderCone(n_theta + 1))\n\n# create the constraints for each sample points\nfor i = 1:n\n yi = y[i]\n x = X[i, :]\n yi == 1. ? (a = -1) : (a = 1)\n @constraint(m, [a * dot(θ, x) - e[i]; s1[i]; t1[i] ] in MOI.ExponentialCone())\n @constraint(m, [-e[i]; s2[i]; t2[i]] in MOI.ExponentialCone())\n @constraint(m, t1[i] + t2[i] <= 1)\n @constraint(m, s1[i] == 1)\n @constraint(m, s2[i] == 1)\nend\nJuMP.optimize!(m)","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"theta = value.(θ)","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"Once we have solved the optimisation problem and obtained the parameter vector theta, we can plot the decision boundary. This can be done by evaluating our model over a grid of points (uv) and then plotting the contour line where the function g returns a probability of p=05.","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"# First we evaluate our model over a grid of points z = θ' x\nu = collect(range(-1., stop = 1.5, length = 50))\nv = collect(range(-1., stop = 1.5, length = 50))\nz = zeros(length(u), length(v));\nfor i = 1:length(u), j = 1:length(v)\n z[i, j] = dot(map_feature(u[i], v[j]), theta);\nend","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"To add the decision boundary we have to plot the line indicating 50 probability of acceptance, i.e. g(theta^top x) = g(z) = 05 which we get at z=0.","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"plot(x1[1:n_half-1], x2[1:n_half-1], color = :blue, st = :scatter, markershape = :cross, aspect_ratio=:equal, label = \"Accepted\", xlabel = \"x1 - Microchip Test Score 1\", ylabel = \"x2 - Microchip Test Score 2\")\nplot!(x1[n_half:end], x2[n_half:end], color = :red, st = :scatter, markershape = :circle, label = \"Rejected\")\ncontour!(u, v, z', levels = [0.], c = :black, linewidth = 2)","category":"page"},{"location":"examples/logistic_regression/#Solving-the-optimisation-problem-directly-with-COSMO","page":"Logistic Regression","title":"Solving the optimisation problem directly with COSMO","text":"","category":"section"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"We can solve the problem directly in COSMO by using its modeling interface. The problem will have nn = 5 n + n_theta + 1 variables. Let us define the cost function frac12x^top P x + q^top x:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"nn = 5 * n + n_theta + 1\nP = spzeros(nn, nn)\nq = zeros(nn)\nq[1] = μ\nfor i = 1:n\n q[1 + n_theta + (i - 1) * 5 + 1] = 1.\nend","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"Next we define a function that creates the COSMO.Constraints for a given sample:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"# the order of the variables\n# v, thetas, [e1 t11 t12 s11 s12] [e2 t21 t22 s21 s22] ...\n# for each sample create two exponential cone constraints,\n# 1 nonnegatives constraint, 2 zeroset constraints\nfunction add_log_regression_constraints!(constraint_list, x, y, n, sample_num)\n num_thetas = length(x)\n # 1st exponential cone constraint (zi - ei, s1, t1) in Kexp\n c_start = 1 + num_thetas + (sample_num - 1) * 5 + 1\n A = spzeros(3, n)\n A[1, c_start] = -1.\n y == 1. ? (a = -1) : (a = 1)\n for k = 1:num_thetas\n A[1, 2 + k - 1] = a * x[k]\n end\n A[2, c_start + 3] = 1.\n A[3, c_start + 1] = 1.\n b = zeros(3)\n push!(constraint_list, COSMO.Constraint(A, b, COSMO.ExponentialCone))\n\n # 2nd exponential cone constraint (-e, s2, t2)\n A = spzeros(3, n)\n A[1, c_start] = -1.\n A[2, c_start + 4] = 1.\n A[3, c_start + 2] = 1.\n b = zeros(3)\n push!(constraint_list, COSMO.Constraint(A, b, COSMO.ExponentialCone))\n\n # Nonnegatives constraint t1 + t2 <= 1\n A = spzeros(1, n)\n A[1, c_start + 1] = -1.\n A[1, c_start + 2] = -1.\n b = [1.]\n push!(constraint_list, COSMO.Constraint(A, b, COSMO.Nonnegatives))\n\n # ZeroSet constraint s1 == 1, s2 == 1\n A = spzeros(2, n)\n A[1, c_start + 3] = 1.\n A[2, c_start + 4] = 1.\n b = -1 * ones(2)\n push!(constraint_list, COSMO.Constraint(A, b, COSMO.ZeroSet))\nend\nnothing","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"Now we can use this function to loop over the sample points and add the constraints to our constraint list:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"constraint_list = Array{COSMO.Constraint{Float64}}(undef, 0)\nfor i = 1:n\n add_log_regression_constraints!(constraint_list, X[i, :], y[i], nn, i )\nend","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"It remains to add a second order cone constraint for the regularisation term: theta _2 leq v","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"push!(constraint_list, COSMO.Constraint(Matrix(1.0I, n_theta + 1, n_theta + 1), zeros(n_theta + 1), COSMO.SecondOrderCone, nn, 1:n_theta+1));\nnothing #hide","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"We can now create, assemble, and solve our COSMO.Model:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"model = COSMO.Model()\nassemble!(model, P, q, constraint_list, settings = COSMO.Settings(verbose=true))\nres = COSMO.optimize!(model);\nnothing #hide","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"Let us double check that we get the same theta as in the previous section:","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"using Test\ntheta_cosmo = res.x[2:2+n_theta-1]\n@test norm(theta_cosmo - theta) < 1e-10","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"","category":"page"},{"location":"examples/logistic_regression/","page":"Logistic Regression","title":"Logistic Regression","text":"This page was generated using Literate.jl.","category":"page"},{"location":"contributing/#Contributing","page":"Contributing","title":"Contributing","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Contributions are always welcome:","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"If you want to contribute features, bug fixes, etc, please take a look at our Code Style Guide below\nPlease report any issues and bugs that you encounter in Issues\nAs an open source project we are also interested in any projects and applications that use COSMO. Please let us know via email to: michael.garstka[at]eng.ox.ac.uk","category":"page"},{"location":"contributing/#Code-Style-Guide","page":"Contributing","title":"Code Style Guide","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"The code in this repository follows the naming and style conventions of Julia Base with a few modifications. This style guide is heavily \"inspired\" by the guides of John Myles White and JuMP.","category":"page"},{"location":"contributing/#Formatting","page":"Contributing","title":"Formatting","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Use one tab when indenting a new block (except module)\nUse spaces between operators, except for ^, ', and :\nUse single space after commas and semicolons\nDon't use spaces around parentheses, or braces","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Bad: f(x,y) = [5*sin(x+y);y'] Good: f(x, y) = [5 * sin(x + y); y']","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Use spacing with keyword arguments","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Bad: foo(x::Integer=1) Good: foo(x::Integer = 1)","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Don't parenthesize conditions","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Bad: if (a == b) Good: if a == b","category":"page"},{"location":"contributing/#Naming","page":"Contributing","title":"Naming","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Modules and Type names use capitilization and camel case, e.g. module LinearAlgebra, struct ConvexSets.\nFunctions are lowercase and use underscores to seperate words, e.g. has_key(x), is_valid(y).\nNormal variables are lowercase and use underscores like functions, e.g. convex_set\nConstants are uppercase, e.g. const MY_CONSTANT\nAlways append ! to names of functions that modify their arguments.\nFunction arguments that are mutated come first. Otherwise follow the rules layed out in Julia Base Argument ordering\nFiles are named like functions, e.g. my_new_file.jl","category":"page"},{"location":"contributing/#Syntax","page":"Contributing","title":"Syntax","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Use 1.0 instead of 1.","category":"page"},{"location":"contributing/#Git(hub)-specific-conventions","page":"Contributing","title":"Git(hub)-specific conventions","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Branch names should be prepended with the initials of the creator and a forward slash, e.g. mg/newIdea instead of newIdea\nCommit messages should have the following format:","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"<#IssueId> Short (72 chars or less) summary\n\nMore detailed explanatory text. Wrap it to 72 characters. The blank\nline separating the summary from the body is critical.\n\nImperative style for the commit message: \"Fix bug\" and not \"Fixed\nbug\" or \"Fixes bug.\"\n\nThe issue id can be ommitted if the commit does not related to a specific open issue","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"EditURL = \"https://github.com/oxfordcontrol/COSMO.jl/blob/master/docs/src/literate/custom_cone.jl\"","category":"page"},{"location":"literate/build/custom_cone/#Custom-Cone-Constraint","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"","category":"section"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"This example demonstrate how the user can extend COSMO with his own custom convex cones and use them in constraints. To make things easy, we will implement the simple cone of Nonpositives, i.e. mathbbR_-^n = x in mathbbR^n mid x_i leq 0.","category":"page"},{"location":"literate/build/custom_cone/#Cone-definition-and-projection-function","page":"Custom Cone Constraint","title":"Cone definition and projection function","text":"","category":"section"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"We start by creating a concrete subtype of our cone interface-type: COSMO.AbstractConvexCone{T}, where T is a parameter for the floating-point precision. The only field that is strictly required for our cone definition is dim, which is used by the solver to store information about the cone dimension. However, you can also use the object to store other information, or allocate the workspace necessary for the projection step.","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"using COSMO, LinearAlgebra, SparseArrays, Test\n\n# define new cone type\nstruct Nonpositives{T} <: COSMO.AbstractConvexCone{T}\n dim::Int64\nend","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"Next, we define a projection method for Nonpositives{T} that takes a vector x and projects it onto our custom cone. For the cone of nonpositive vectors the projection simply sets all positive elements of x to zero:","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"function COSMO.project!(x::AbstractVector{T}, C::Nonpositives{T}) where {T <: AbstractFloat}\n x .= min.(x, zero(T))\nend","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"This is all that is required to get a basic constraint working. Let's test our new cone/constraint by solving the following LP:","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"beginarrayll textmaximize x_1 + x_2 + x_3\ntextsubject to x_1 leq 3 \n x_2 leq 2 \n x_1 + x_3 = 5\nendarray","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"We define our decision vector x = (x_1 x_2 x_3) and setup the problem:","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"n = 3\nP = spzeros(n, n)\nq = -ones(n)\n\n# A1 * x + b1 <= 0\nA1 = diagm(0 => ones(2))\nb1 = [-3.; -2.]\nc1 = COSMO.Constraint(A1, b1, Nonpositives, n, 1:2); #here we use our new cone\n\n# x_1 + x_3 == 5\nA2 = [1. 0 1.]\nb2 = [-5.]\nc2 = COSMO.Constraint(A2, b2, COSMO.ZeroSet);\n\n# assemple and solve\nmodel = COSMO.Model();\nassemble!(model, P, q, [c1; c2]);\nres = COSMO.optimize!(model);\nnothing #hide","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"res.x","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"obj_val = -dot(q, res.x)","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"The problem was solved using constraints involving our new cone and yields the expected result. Next, we want to add the ability to detect infeasible problems.","category":"page"},{"location":"literate/build/custom_cone/#Support-infeasibility-detection","page":"Custom Cone Constraint","title":"Support infeasibility detection","text":"","category":"section"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"If no further information about the new cone is provided, the infeasibility detection is disabled. However, by defining the two additional methods in_dual and in_pol_recc we can support infeasibility detection. More information on how infeasible problems are detected in COSMO can be found here [1]. We will have to add the ability for the solver to check whether a vector is in the dual cone mathcalK^* of our new cone mathcalK and whether a vector is in the recession cone of the polar cone mathcalK^circ^infty of our cone. The first function is used to check for primal infeasibility, while the second function is used to check for dual infeasibility. Luckily, for the Nonpositives-cone this is straightforward:","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"mathbbR_-^n^* = mathbbR_-^n quad mathbbR_-^n^circ^infty = mathbbR_+^n","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"function COSMO.in_dual(x::AbstractVector{T}, C::Nonpositives{T}, tol::T) where {T <: AbstractFloat}\n\treturn !any( x-> (x > -tol), x)\nend\n\nfunction COSMO.in_pol_recc(x::AbstractVector{T}, C::Nonpositives{T}, tol::T) where {T <: AbstractFloat}\n\treturn !any( x-> (x < tol), x)\nend","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"Notice that for numerical reasons, we give the check-functions a bit of slack using the tolerance parameter tol. To test the infeasibility detection, we will attempt to use our new cone to solve the clearly dual infeasible problem:","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"beginarrayll textminimize x\ntextsubject to xleq 3\nendarray","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"n = 1\nP = spzeros(n, n)\nq = [1.]\n\n# x <= 3 <=> x - 3 in Nonpositives\nAi = [1.]\nbi = [-3.]\nci = COSMO.Constraint(Ai, bi, Nonpositives);\n\nmodel = COSMO.Model();\nassemble!(model, P, q, [ci]);\nres = COSMO.optimize!(model);\nnothing #hide","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"@test res.status == :Dual_infeasible","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"which is what we would expect.","category":"page"},{"location":"literate/build/custom_cone/#Using-new-cone-with-JuMP","page":"Custom Cone Constraint","title":"Using new cone with JuMP","text":"","category":"section"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"The nice composibility of Julia-code allows us to use our new cone definition even when the problem is modelled with JuMP. For this to work, we define a concrete subtype of MOI.AbstractSet, that JuMP can use to build a constraint.","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"using MathOptInterface, JuMP\nimport Base.copy\nconst MOI = MathOptInterface\n\nstruct NonPos <: MOI.AbstractVectorSet\n\tdimension::Int\nend\nBase.copy(set::NonPos) = set #this is needed for MOI","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"Next, we tell COSMO that whenever JuMP wants to solve a problem with a NonPos-cone it should translate it into a Nonpositives constraint and we also tell MOI that we now support constraints of this new mysterious cone NonPos.","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"function COSMO.processSet!(b::Vector{T}, rows::UnitRange{Int}, cs, s::NonPos) where {T <: AbstractFloat}\n push!(cs, Nonpositives{T}(length(rows)))\n nothing\nend\n#tell MOI, that COSMO supports constraints with this new cone\nMOI.supports_constraint(optimizer::COSMO.Optimizer, ::Type{<:Union{MOI.VectorOfVariables, MOI.VectorAffineFunction{Float64}}}, ::Type{NonPos}) = true","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"We should now be able to model the LP from above in JuMP and solve it internally using our Nonpositives cone and projection function.","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"model = JuMP.Model(COSMO.Optimizer);\n@variable(model, x[1:3]);\n@objective(model, Max, x[1] + x[2] + x[3]);\n@constraint(model, A1 * x[1:2] .+ b1 in NonPos(2));\n@constraint(model, x[1] + x[3] == 5);\nJuMP.optimize!(model)","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"x_opt = JuMP.value.(x)","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"You can see in the solver output that indeed a set Nonpositives of dim: 2 was used.","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"The discussed cone of Nonpositives is of course trivial, but the ability to define new cones and constraints for COSMO can be very powerful to model complex problems.","category":"page"},{"location":"literate/build/custom_cone/#References","page":"Custom Cone Constraint","title":"References","text":"","category":"section"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"[1] Garstka et al. - COSMO: A conic operator splitting method for convex conic problems (2020)","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"","category":"page"},{"location":"literate/build/custom_cone/","page":"Custom Cone Constraint","title":"Custom Cone Constraint","text":"This page was generated using Literate.jl.","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"The source files for all examples can be found in /examples.","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"EditURL = \"https://github.com/oxfordcontrol/COSMO.jl/blob/master/examples/lp.jl\"","category":"page"},{"location":"examples/lp/#Linear-Program","page":"Linear Program","title":"Linear Program","text":"","category":"section"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"We want to solve the following linear program with decision variable x:","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"beginarrayll textminimize c^top x\ntextsubject to A x leq b \n x geq 1 \n x_2 geq 5 \n x_1 + x_3 geq 4\nendarray","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"The problem can be solved with COSMO in the following way:","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"using COSMO, LinearAlgebra, SparseArrays, Test","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"##Define problem data:\nc = [1; 2; 3; 4.];\nA = Matrix(1.0I, 4, 4);\nb = [10.; 10; 10; 10];\nn = 4;\nnothing #hide","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"Create the constraints Ax + b in mathcalK:","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"# Ax <= b\nc1 = COSMO.Constraint(-A, b, COSMO.Nonnegatives);\n# x >= 1\nc2 = COSMO.Constraint(Matrix(1.0I, n, n), -ones(n), COSMO.Nonnegatives);\n# x2 >= 5\nc3 = COSMO.Constraint(1, -5, COSMO.Nonnegatives, n, 2:2);\n# x1 + x3 >= 4\nc4 = COSMO.Constraint([1 0 1 0], -4, COSMO.Nonnegatives);\nnothing #hide","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"Define matrix P and vector q for the objective function:","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"P = spzeros(4, 4);\nq = c;\nnothing #hide","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"Create, assemble and solve the model:","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"settings = COSMO.Settings(verbose=true, eps_abs = 1e-4, eps_rel = 1e-5);\nmodel = COSMO.Model();\nassemble!(model, P, q, [c1; c2; c3; c4], settings = settings);\nres = COSMO.optimize!(model)","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"Compare the result to the known solution:","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"@test isapprox(res.x[1:4], [3; 5; 1; 1], atol=1e-2, norm = (x -> norm(x, Inf)))","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"@test isapprox(res.obj_val, 20.0, atol=1e-2)","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"","category":"page"},{"location":"examples/lp/","page":"Linear Program","title":"Linear Program","text":"This page was generated using Literate.jl.","category":"page"},{"location":"performance/#Performance-Tips","page":"Performance Tips","title":"Performance Tips","text":"","category":"section"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"There are a number of ways to improve the performance of the solver given a particular problem. If you are not satisfied with the performance there are a number of things you have to determine first. Is the solver slow because","category":"page"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"it's the first time you ran it in the current Julia session or\nis it because the solver needs a lot of iterations (convergence) or\nis each iteration or the initial factorisation slow (computational performance)?","category":"page"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"Let's see how each point can be addressed.","category":"page"},{"location":"performance/#First-run","page":"Performance Tips","title":"First run","text":"","category":"section"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"Whenever a new Julia session is started, the first run will trigger a compilation of all functions based on their arguments used in your script. This will slow the first execution of COSMO down. After that Julia will call the fast compiled functions. To get around this, you can either keep your current Julia session open and discard the first run. Alternatively, if your problem is very large, you could solve a small version of your problem first to trigger the compilation. Another option is to use PackageCompiler to save compiled functions into a sysimage that can be loaded at startup.","category":"page"},{"location":"performance/#Solver-Timings","page":"Performance Tips","title":"Solver Timings","text":"","category":"section"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"It is often instructive to look at the detailed solver timing results for your problem. This can reveal where most of the time is spent. To achieve this, run COSMO with the setting verbose_timing = true. After solving the problem with result = COSMO.optimize!(model) you can look at result.times for a breakdown of the times spent in different parts of the algorithm, see Timings for more details. Especially take a look at the ratio of factorisation time and iteration time. If you use JuMP to solve the problem, you can take a look at the timings with backend(model).optimizer.model.optimizer.results.times.","category":"page"},{"location":"performance/#Convergence","page":"Performance Tips","title":"Convergence","text":"","category":"section"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"It is possible that COSMO converges slowly, i.e. needs a large number of iterations, for your problem given its default parameters.","category":"page"},{"location":"performance/#Parameters","page":"Performance Tips","title":"Parameters","text":"","category":"section"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"You could try changing any of the following parameters:","category":"page"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"rho: The initial algorithm step parameter has a large influence on the convergence. Try different values between 1e-5 and 10.\nadaptive_rho = false: You can try to disable the automatic rho adaption and use different rho values.\nadaptive_rho_interval: This specifies after how many iterations COSMO tries to adapt the rho parameter. You can also set adaptive_rho_interval = 0 which adapts the rho parameter after the time spent iterating passes 40% of the factorisation time. This is currently the default in OSQP and works well with QPs.\nalpha = 1.0: This disables the over-relaxation that is used in the algorithm. We recommend values between 1.0 - 1.6.\nscaling = 0: This disables the problem scaling.\neps_abs and eps_rel: Check the impact of modifying the stopping accuracies.","category":"page"},{"location":"performance/#Use-warm-starting","page":"Performance Tips","title":"Use warm starting","text":"","category":"section"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"The number of iterations can be dramatically decreased by providing a good initial guess for x, s and y. Examples where warm starting is commonly used are model predictive control and portfolio backtests, see Warm starting.","category":"page"},{"location":"performance/#Computational-performance","page":"Performance Tips","title":"Computational performance","text":"","category":"section"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"If the convergence of the algorithm is not an issue, there are still a number of steps you can take to make COSMO faster.","category":"page"},{"location":"performance/#Intel-MKL-BLAS/LAPACK","page":"Performance Tips","title":"Intel MKL BLAS/LAPACK","text":"","category":"section"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"We experienced significant performance improvements on Intel CPUs if Julia is compiled with MKL BLAS. This is because Julia's linear algebra function will use Intel MKL BLAS and LAPACK functions that are optimised for Intel hardware. The effect is especially significant for SDPs because most of the time is spent in the LAPACK function syevr. If you are running Julia on Intel hardware, an easy way to compile Julia with MKL is to add and build the MKL package, see MKL.jl. To verify your current BLAS vendor you can use julia> LinearAlgebra.BLAS.vendor().","category":"page"},{"location":"performance/#Linear-system-solver","page":"Performance Tips","title":"Linear system solver","text":"","category":"section"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"COSMO uses QDLDL.jl as the default linear system solver. In our experience this seems to be a competitive choice until about 1e5 - 1e6 nonzeros in the constraint matrix. After that it is worth trying one of the indirect system solvers, such as CG or MINRES. Furthermore, we also recommend trying Pardiso (or MKLPardiso) for problems of that dimension. More details can be found here: Linear System Solver.","category":"page"},{"location":"performance/#Custom-cones","page":"Performance Tips","title":"Custom cones","text":"","category":"section"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"In some cases the computations can be speed-up if certain constraints in the problem allow the implementation of a fast projection function. We allow the user to define their own custom convex cone with a corresponding projection function. The custom cone has to be defined as struct CustomCone{T} <: COSMO.AbstractConvexSet{T}. Furthermore, the user has to define a function that projects an input vector x onto the custom cone, i.e. function COSMO.project!(x::AbstractVector{T}, C::CustomCone{T}) where {T} ... end.","category":"page"},{"location":"performance/#Multithreading","page":"Performance Tips","title":"Multithreading","text":"","category":"section"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"COSMO allows the execution of the projection step for multiple constraints in parallel using Julia's multithreading features. This is currently not enabled in the tagged release because of stability issues in earlier Julia versions. To use multithreading checkout the branch with_multi_threading, which we keep in sync with the latest tagged release. This can be installed via Julia's package manager with pkg> add COSMO#with_multi_threading. Afterwards, before starting Julia, set export JULIA_NUM_THREADS=[NUMBER_PHYSICAL_CORES_HERE]. In Julia you can verify the number of threads with julia> Threads.nthreads().","category":"page"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"Notice that the extra overhead for multithreading can slow the solver down if the problem is small. However, we noticed significant performance improvements if the problem contained multiple positive semidefinite constraints or when one large constraint was decomposed. In that case it also helps to restrict the number of BLAS threads per Julia thread with julia> BLAS.set_num_threads(1) to prevent oversubscription of the available cores.","category":"page"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"Multithreading can also be used in the factorisation step if the Pardiso or MKLPardiso solver are selected. This is only advisable for constraint matrices with more than 1e5 nonzeros.","category":"page"},{"location":"performance/#Chordal-decomposition-and-Clique-merging","page":"Performance Tips","title":"Chordal decomposition and Clique merging","text":"","category":"section"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"When solving large structured and sparse SDPs significant performance improvements are achievable if the problem is passed to COSMO in the right way. This means the solver has to be able to infer the structure of the positive semidefinite variable from the constraint. See the section on Chordal Decomposition for more details. In some cases the primal SDP doesn't allow decomposition but the dual SDP does, consider the Maximum Cut Problem and the Relaxed Two-Way Partitioning Problem for examples.","category":"page"},{"location":"performance/","page":"Performance Tips","title":"Performance Tips","text":"If the problem is decomposable it is also worth experimenting with different clique merging strategies to see how they impact the performance. More details can be found here: Clique merging.","category":"page"},{"location":"decomposition/#Chordal-Decomposition","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"","category":"section"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"For very large sparse semidefinite programs (SDPs) it is often helpful to analyse the sparsity structure of the PSD constraint(s). If the equality constraints impose a sparsity structure on the matrix variable, one PSD constraint on a large matrix variable can be decomposed into several smaller constraints. This results usually in a significant speedup and reduction in solve time.","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"The following example gives a short overview on chordal decomposition and clique merging. For more details, take a look at our paper on clique merging or watch the corresponding presentation that I gave on the topic.","category":"page"},{"location":"decomposition/#Example-problem","page":"Chordal Decomposition","title":"Example problem","text":"","category":"section"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Let's consider the following SDP in standard dual form:","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"beginarrayll\n textminimize c^top x \n textsubject to displaystyle sum_i=1^m A_i x_i + S = B\n S in mathbbS^n_+\nendarray","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"with problem data matrices A_1 ldots A_m B in mathbbS^n, vector variable x in mathbbR^n, and matrix variable S in mathbbS^n_+.","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Let's look at the following example problem with m=2 and n=9:","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"A1 = [-4.0 0.0 -2.0 0.0 0.0 -1.0 0.0 0.0 0.0; 0.0 -3.0 -1.0 0.0 0.0 0.0 0.0 0.0 0.0; -2.0 -1.0 -2.0 0.0 0.0 5.0 4.0 -4.0 0.0; 0.0 0.0 0.0 -4.0 -5.0 0.0 0.0 3.0 0.0; 0.0 0.0 0.0 -5.0 4.0 0.0 0.0 2.0 0.0; -1.0 0.0 5.0 0.0 0.0 5.0 -4.0 -4.0 -5.0; 0.0 0.0 4.0 0.0 0.0 -4.0 -1.0 -1.0 -3.0; 0.0 0.0 -4.0 3.0 2.0 -4.0 -1.0 2.0 -2.0; 0.0 0.0 0.0 0.0 0.0 -5.0 -3.0 -2.0 -3.0];\n\nA2 = [-5.0 0.0 3.0 0.0 0.0 -2.0 0.0 0.0 0.0; 0.0 -3.0 -5.0 0.0 0.0 0.0 0.0 0.0 0.0; 3.0 -5.0 3.0 0.0 0.0 5.0 -4.0 -5.0 0.0; 0.0 0.0 0.0 3.0 2.0 0.0 0.0 -2.0 0.0; 0.0 0.0 0.0 2.0 4.0 0.0 0.0 -3.0 0.0; -2.0 0.0 5.0 0.0 0.0 1.0 -5.0 -2.0 -4.0; 0.0 0.0 -4.0 0.0 0.0 -5.0 -2.0 -3.0 3.0; 0.0 0.0 -5.0 -2.0 -3.0 -2.0 -3.0 5.0 3.0; 0.0 0.0 0.0 0.0 0.0 -4.0 3.0 3.0 -4.0];\n\nB = [-0.11477375644968069 0.0 6.739182490600791 0.0 0.0 -1.2185593245043502 0.0 0.0 0.0; 0.0 1.2827680528587497 -5.136452036888789 0.0 0.0 0.0 0.0 0.0 0.0; 6.739182490600791 -5.136452036888789 7.344770673489607 0.0 0.0 -0.2224400187044442 -10.505300166831221 -1.2627361794562273 0.0; 0.0 0.0 0.0 10.327710040060499 8.91534585379813 0.0 0.0 -6.525873789637007 0.0; 0.0 0.0 0.0 8.91534585379813 0.8370459338528677 0.0 0.0 -6.210900615408826 0.0; -1.2185593245043502 0.0 -0.2224400187044442 0.0 0.0 -3.8185953011245024 -0.994033914192722 2.8156077981712997 1.4524716674219218; 0.0 0.0 -10.505300166831221 0.0 0.0 -0.994033914192722 0.029162208619863517 -2.8123790276830745 7.663416446183705; 0.0 0.0 -1.2627361794562273 -6.525873789637007 -6.210900615408826 2.8156077981712997 -2.8123790276830745 4.71893305728242 6.322431630550857; 0.0 0.0 0.0 0.0 0.0 1.4524716674219218 7.663416446183705 6.322431630550857 0.5026094532322212];\n\nc = [-0.21052661285686525, -1.263324575834677];","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Note that the data matrices A_1dots A_m B all have the same sparsity pattern (common zeros in certain entries). Take a look at A_1:","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"A1 = [-4.0 0.0 -2.0 0.0 0.0 -1.0 0.0 0.0 0.0; 0.0 -3.0 -1.0 0.0 0.0 0.0 0.0 0.0 0.0; -2.0 -1.0 -2.0 0.0 0.0 5.0 4.0 -4.0 0.0; 0.0 0.0 0.0 -4.0 -5.0 0.0 0.0 3.0 0.0; 0.0 0.0 0.0 -5.0 4.0 0.0 0.0 2.0 0.0; -1.0 0.0 5.0 0.0 0.0 5.0 -4.0 -4.0 -5.0; 0.0 0.0 4.0 0.0 0.0 -4.0 -1.0 -1.0 -3.0; 0.0 0.0 -4.0 3.0 2.0 -4.0 -1.0 2.0 -2.0; 0.0 0.0 0.0 0.0 0.0 -5.0 -3.0 -2.0 -3.0] # hide\nA1","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Since all the data matrices have zeros in the same places, the equality constraint of the problem tells us that our matrix variable S will also have zeros in these places. The aggregated pattern matrix of S is shown in the following figure, where a dot represents a nonzero entry:","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"(Image: )","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Since the matrices are symmetric we only show the lower triangle. We can represent the aggregated sparsity pattern of the problem using a graph G(V E), where the vertex set V is given by the column indices of the matrix and we introduce an edge (ij) in E for every nonzero matrix element S_ij. The graph for the example problem is shown in the right figure. In order to decompose the problem, we have to find the cliques, i.e. completely connected subgraphs, of the graph G. These cliques represent the dense subblocks of non-zero entries in the matrix (see the colored entries in the figure). In order for the theory to work, we also have to require G to be a chordal graph. For the purpose of this example, we just assume that G is chordal and keep in mind that we can always make a graph chordal by adding more edges.","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"COSMO finds the cliques of the graph automatically, if a constraint of type COSMO.PsdCone or COSMO.PsdConeTriangle is present in the problem and additional equality constraints impose a structure on them. For the example problem COSMO finds the following cliques: mathcalC_1=136 mathcalC_2=23 mathcalC_3=3678 mathcalC_4=458 and mathcalC_5=6789.","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Let's denote the set of cliques mathcalB=mathcalC_1ldotsmathcalC_p. To represent the relationship between different cliques, e.g. in terms of overlapping entries, it is helpful to represent them either as a clique tree mathcalT(mathcalB mathcalE) (left) or a clique graph mathcalG(mathcalB xi) (right), shown in the following figures:","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"(Image: ) (Image: )","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Once the cliques have been found, we can use the following theorem to decompose the problem (and speed up the solver significantly).","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Theorem (Agler's Theorem)","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Let G(VE) be a chordal graph with a set of maximal cliques mathcalC_1ldotsmathcalC_p . Then S in mathbbS^n_+(E0) if and only if there exist matrices S_ell in S^mathcalC_ell for ell = 1ldotsp such that","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"S = displaystyle sum_ell = 1^p T_ell^top S_ell T_ell","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"where T_ell is an entry-selector matrix that maps the subblocks S_ell into the correct entries of the original matrix S.","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"In other words, instead of solving the original problem and choosing S to be positive semidefinite, we only have to choose the subblocks S_ell of the matrix S to be positive semidefinite and make sure that the individual blocks \"add up\" to S. Thus, we can solve the equivalent problem:","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"beginarrayll\n textminimize c^top x \n textsubject to displaystyle sum_i=1^m A_i x + sum_ell = 1^p T_l^top S_ell T_ell = B\n S_ell in mathbbS^mathcalC_ell_+ l = 1 ldots p\nendarray","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Compare this again with the problem at the top of the page. We have replaced the positive semidefinite constraint on the matrix variable S by p constraints on the smaller subblocks S_ell. Why is this useful?","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"When solving an SDP, the major linear algebra operation is a projection onto the constraint set, see Method. This projection is done at each iteration of the algorithm and for positive semidefinite constraints involves an eigenvalue decomposition of the matrix iterate. Unfortunately, an eigenvalue decomposition of a matrix of dimension n has a complexity of roughly mathcalO(n^3).","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"For our example this means that we now project one 2times 2 block, two 3 times 3 blocks and two 4 times 4 blocks instead of one 9 times 9 block. For small problems, this doesn't seem to make much of a difference, but consider of a problem with 1000 11 times 11 blocks along the diagonal and overlapping by one entry. Taking into account that the computational cost scales cubicly, we can improve the performance of the solver by projecting 1000 11times 11 blocks instead of projecting one giant 10001times 10001 block at each iteration.","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Let's go back to our example and solve it with COSMO and JuMP:","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"model = JuMP.Model(with_optimizer(COSMO.Optimizer, decompose = true, merge_strategy = COSMO.NoMerge));\n@variable(model, x[1:2]);\n@objective(model, Min, c' * x )\n@constraint(model, Symmetric(B - A1 .* x[1] - A2 .* x[2] ) in JuMP.PSDCone());\n\nJuMP.optimize!(model)","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Notice that we set the solver settings decompose = true to allow COSMO to decompose the problem. Furthermore, we set the clique merging strategy to COSMO.NoMerge. This just means that after we have found the cliques we don't attempt to merge some of them (more about clique merging below). We get the following output from the solver:","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"------------------------------------------------------------------\n COSMO v0.5.0 - A Quadratic Objective Conic Solver\n Michael Garstka\n University of Oxford, 2017 - 2019\n------------------------------------------------------------------\n\nProblem: x ∈ R^{13},\n constraints: A ∈ R^{35x13} (70 nnz),\n matrix size to factor: 48x48 (188 nnz)\nSets: PsdConeTriangle of dim: 10\n PsdConeTriangle of dim: 10\n PsdConeTriangle of dim: 6\n PsdConeTriangle of dim: 6\n PsdConeTriangle of dim: 3\nDecomp: Num of original PSD cones: 1\n Num decomposable PSD cones: 1\n Num PSD cones after decomposition: 5\n Merge Strategy: NoMerge\nSettings: ϵ_abs = 1.0e-04, ϵ_rel = 1.0e-04,\n ϵ_prim_inf = 1.0e-06, ϵ_dual_inf = 1.0e-04,\n ρ = 0.1, σ = 1.0e-6, α = 1.6,\n max_iter = 2500,\n scaling iter = 10 (on),\n check termination every 40 iter,\n check infeasibility every 40 iter,\n KKT system solver: QDLDL\nSetup Time: 0.14ms\n\nIter: Objective: Primal Res: Dual Res: Rho:\n40 -1.4134e+00 1.2320e-05 3.0409e-05 1.0000e-01\n\n------------------------------------------------------------------\n>>> Results\nStatus: Solved\nIterations: 40\nOptimal objective: -1.4134\nRuntime: 0.002s (1.93ms)","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Under sets we can indeed see that COSMO solved a problem with five PSD constraints corresponding to the cliques mathcalC_1ldots mathcalC_5 discovered in the sparsity pattern. (Note that the dimension printed is the number of entries in the upper triangle of the matrix block.)","category":"page"},{"location":"decomposition/#Clique-merging","page":"Chordal Decomposition","title":"Clique merging","text":"","category":"section"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"After we have found the cliques of the sparsity pattern, we are allowed to merge some of them back together. For the graph of the sparsity pattern this just means adding more edges or treating some structural zeros as numerical zeros. The main reason to merge two cliques is that they might overlap a lot and therefore it is not advantageous to treat them as two different blocks. Consider the two extreme cases below:","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"(Image: ) (Image: )","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"In the left figure we have the ideal case that all the blocks overlap in just one entry. A full decomposition would leave us with a large number of small blocks. The sparsity pattern in the right figure has two large blocks overlapping almost entirely. In this case it would be disadvantageous to decompose the blocks. Instead, we would do the initial decomposition, realize the large overlap, and then merge the two blocks back together. For sparsity patterns that arise from real applications the case is not always as clear and we have to use more sophisticated strategies to decide which blocks to merge.","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"COSMO currently provides three different strategies that can be selected by the user:","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"COSMO.NoMerge\nCOSMO.ParentChildMerge\nCOSMO.CliqueGraphMerge","category":"page"},{"location":"decomposition/#COSMO.NoMerge","page":"Chordal Decomposition","title":"COSMO.NoMerge","text":"NoMerge <: AbstractMergeStrategy\n\nA strategy that does not merge cliques.\n\n\n\n\n\n","category":"type"},{"location":"decomposition/#COSMO.ParentChildMerge","page":"Chordal Decomposition","title":"COSMO.ParentChildMerge","text":"ParentChildMerge(t_fill = 8, t_size = 8) <: AbstractTreeBasedMerge\n\nThe merge strategy suggested in Sun and Andersen - Decomposition in conic optimization with partially separable structure (2014). The initial clique tree is traversed in topological order and a clique mathcalC_ell is greedily merged to its parent clique mathcalC_par(ell) if at least one of the two conditions are met\n\n( mathcalC_par(ell) - eta_ell) (mathcalC_ell - eta_ell) leq t_textfill (fill-in condition)\nmax left nu_ell nu_par(ell) right leq t_textsize (supernode size condition)\n\n\n\n\n\n","category":"type"},{"location":"decomposition/#COSMO.CliqueGraphMerge","page":"Chordal Decomposition","title":"COSMO.CliqueGraphMerge","text":"CliqueGraphMerge(edge_weight::AbstractEdgeWeight = ComplexityWeight()) <: AbstractGraphBasedMerge\n\nThe (default) merge strategy based on the reduced clique graph mathcalG(mathcalB xi), for a set of cliques mathcalB = mathcalC_1 dots mathcalC_p where the edge set xi is obtained by taking the edges of the union of clique trees.\n\nMoreover, given an edge weighting function e(mathcalC_imathcalC_j) = w_ij, we compute a weight for each edge that quantifies the computational savings of merging the two cliques. After the initial weights are computed, we merge cliques in a loop:\n\nwhile clique graph contains positive weights:\n\nselect two permissible cliques with the highest weight w_ij\nmerge cliques rightarrow update clique graph\nrecompute weights for updated clique graph\n\nCustom edge weighting functions can be used by defining your own CustomEdgeWeight <: AbstractEdgeWeight and a corresponding edge_metric method. By default, the ComplexityWeight <: AbstractEdgeWeight is used which computes the weight based on the cardinalities of the cliques: e(mathcalC_imathcalC_j) = mathcalC_i^3 + mathcalC_j^3 - mathcalC_i cup mathcalC_j^3.\n\nSee also: Garstka, Cannon, Goulart - A clique graph based merging strategy for decomposable SDPs (2019)\n\n\n\n\n\n","category":"type"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"In our example problem we have two cliques mathcalC_3 = 3678 and mathcalC_5 = 6789 that overlap in three entries. Let's solve the problem again and choose the default clique merging strategy merge_strategy = COSMO.CliqueGraphMerge:","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"model = JuMP.Model(with_optimizer(COSMO.Optimizer, decompose = true, merge_strategy = COSMO.CliqueGraphMerge));\n@variable(model, x[1:2]);\n@objective(model, Min, c' * x )\n@constraint(model, Symmetric(B - A1 .* x[1] - A2 .* x[2] ) in JuMP.PSDCone());\n\nJuMP.optimize!(model)","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"------------------------------------------------------------------\n COSMO v0.5.0 - A Quadratic Objective Conic Solver\n Michael Garstka\n University of Oxford, 2017 - 2019\n------------------------------------------------------------------\n\nProblem: x ∈ R^{7},\n constraints: A ∈ R^{30x7} (58 nnz),\n matrix size to factor: 37x37 (153 nnz)\nSets: PsdConeTriangle of dim: 15\n PsdConeTriangle of dim: 6\n PsdConeTriangle of dim: 6\n PsdConeTriangle of dim: 3\nDecomp: Num of original PSD cones: 1\n Num decomposable PSD cones: 1\n Num PSD cones after decomposition: 4\n Merge Strategy: CliqueGraphMerge\nSettings: ϵ_abs = 1.0e-04, ϵ_rel = 1.0e-04,\n ϵ_prim_inf = 1.0e-06, ϵ_dual_inf = 1.0e-04,\n ρ = 0.1, σ = 1.0e-6, α = 1.6,\n max_iter = 2500,\n scaling iter = 10 (on),\n check termination every 40 iter,\n check infeasibility every 40 iter,\n KKT system solver: QDLDL\nSetup Time: 0.26ms\n\nIter: Objective: Primal Res: Dual Res: Rho:\n40 -1.4134e+00 1.7924e-05 1.0416e-04 1.0000e-01\n\n------------------------------------------------------------------\n>>> Results\nStatus: Solved\nIterations: 40\nOptimal objective: -1.4134\nRuntime: 0.003s (2.68ms)","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Unsurprisingly, we can see in the output that COSMO solved a problem with four PSD constraints. One of them is of dimension 15, i.e. a 5times 5 block, which correspond to the merged clique mathcalC_3 cup mathcalC_5 = 36789 .","category":"page"},{"location":"decomposition/#Completing-the-dual-variable","page":"Chordal Decomposition","title":"Completing the dual variable","text":"","category":"section"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"After a decomposed problem is solved, we can recover the solution to the original problem by assembling the matrix variable S from its subblocks S_ell:","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"S = displaystyle sum_ell = 1^p T_ell^top S_ell T_ell","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"Following Agler's Theorem, S will be a positive semidefinite matrix. However, this is not true for the corresponding dual variable matrix Y. The dual variable returned after solving the decomposed problem will be in the space of PSD completable matrices Y in mathbbS_+^n(E). This means that the entries in Y corresponding to the blocks S_ell (black dots) have been chosen correctly. The numerical values for all the other entries (corresponding to the zeros in S and denoted with a red dot) have to be chosen in the right way to make Y positive semidefinite.","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"(Image: )","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"For more information about PSD matrix completion and the completion algorithm used in COSMO take a look at Vandenberghe and Andersen - Chordal Graphs and Semidefinite Optimization (Ch.10). To configure COSMO to complete the dual variable after solving the problem you have to set the complete_dual option:","category":"page"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"model = JuMP.Model(with_optimizer(COSMO.Optimizer, complete_dual = true));","category":"page"},{"location":"decomposition/#Example-Code","page":"Chordal Decomposition","title":"Example Code","text":"","category":"section"},{"location":"decomposition/","page":"Chordal Decomposition","title":"Chordal Decomposition","text":"The code used for this example can be found in /examples/chordal_decomposition.jl.","category":"page"},{"location":"acceleration/#Acceleration","page":"Acceleration","title":"Acceleration","text":"","category":"section"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"By default COSMO's ADMM algorithm is wrapped in a safeguarded acceleration method to achieve faster convergence to higher precision. COSMO uses accelerators from the COSMOAccelerators.jl package.","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"By default, the solver uses the accelerator type AndersonAccelerator{T, Type2{QRDecomp}, RestartedMemory, NoRegularizer}. This is the classic type-II Anderson acceleration method where the least squares subproblem is solved using an updated QR method. Moreover, the method is restarted, i.e. the history of iterates is deleted, after mem steps and no regularisation for the least-squares method is used.","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"In addition, the method is safeguarded (safeguard = true), i.e. the residual-norm of the accelerated point can not deviate too much from the current point. Otherwise, the point is discarded and the ADMM algorithm performs a normal step instead.","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"The acceleration method can be altered as usual via the solver settings and the accelerator keyword. To deactivate acceleration pass an EmptyAccelerator:","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"settings = COSMO.Settings(accelerator = EmptyAccelerator)","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"To use the default accelerator but with a different memory size (number of stored iterates) use:","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"settings = COSMO.Settings(accelerator = with_options(AndersonAccelerator, mem = 15))","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"To turn the safeguarding off use:","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"settings = COSMO.Settings(safeguard = false)","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"To use an Anderson Accelerator of Type-I with a rolling-memory (oldest iterate replaced by newest) approach, use:","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"settings = COSMO.Settings(accelerator = AndersonAccelerator{Float64, Type1, RollingMemory, NoRegularizer})","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"For more fine-grained control look at the implementation of the accelerator here.","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"When JuMP is used, the accelerator settings can be passed in the usual way:","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"model = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, \"accelerator\" => with_options(AndersonAccelerator, mem = 15)))","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"Or using the set_optimizer_attribute() method:","category":"page"},{"location":"acceleration/","page":"Acceleration","title":"Acceleration","text":"model = JuMP.Model(COSMO.Optimizer);\nset_optimizer_attribute(model, \"accelerator\", with_options(AndersonAccelerator, mem = 15))","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"The source files for all examples can be found in /examples.","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"EditURL = \"https://github.com/oxfordcontrol/COSMO.jl/blob/master/examples/sum_abs_k_eigenvalues.jl\"","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/#Minimizing-the-sum-of-the-k-largest-λ","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"","category":"section"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"We show how to find the sum of absolute value of the k largest eigenvalues of a symmetric matrix A in mathbbS^n. This problem can be solved as a semidefinite program. The primal and dual forms are stated in Alizadeh [1]:","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"beginarrayllll textmaximize textTr(AY) - textTr(AW) textminimize kz + Tr(U) + Tr(V) \ntextsubject to textTr(Y + W) = k textsubject to zI + V - A succeq 0 \n 0 preceq Y preceq I zI + U + A succeq 0 \n 0 preceq W preceq I U V succeq 0\nendarray","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"where Y W are the variables of the primal and U V are the variables of the dual problem.","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"using LinearAlgebra, JuMP, COSMO, Random\nrng = Random.MersenneTwister(212)\n\nn = 10\nA = 5 .* randn(rng, 10, 10)\nA = Symmetric(A, :U)","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"We are interested in minimizing the sum of absolute values of the k=3 largest eigenvalues. Let's formulate the problem in JuMP with COSMO as the backend solver:","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"k = 3\nmodel = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, \"verbose\" => true));\n@variable(model, Y[1:n, 1:n], PSD);\n@variable(model, W[1:n, 1:n], PSD);\n\n@objective(model, Max, tr(A * Y) - tr(A * W));\n@constraint(model, tr(Y + W) == k);\n@constraint(model, Symmetric(I - Y) in PSDCone());\n@constraint(model, Symmetric(I - W) in PSDCone());\nstatus = JuMP.optimize!(model)","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"opt_objective = JuMP.objective_value(model)","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"Now, we can check the solution by computing the sum of the absolute value of the 3-largest eigenvalues:","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"k_λ_abs = sum(sort(abs.(eigen(A).values), rev = true)[1:k])","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/#Solve-the-dual","page":"Minimizing the sum of the k-largest λ","title":"Solve the dual","text":"","category":"section"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"Alternatively, we can solve the dual problem:","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"model = JuMP.Model(with_optimizer(COSMO.Optimizer, verbose=true));\n@variable(model, V[1:n, 1:n], PSD);\n@variable(model, U[1:n, 1:n], PSD);\n@variable(model, z);\n\n@objective(model, Min, k * z + tr(V) + tr(U));\n@constraint(model, Symmetric(z .* diagm(0 => ones(n)) + V - A) in PSDCone());\n@constraint(model, Symmetric(z .* diagm(0 => ones(n)) + U + A) in PSDCone());\nstatus = JuMP.optimize!(model)","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"opt_objective = JuMP.objective_value(model)","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"This gives the same result.","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/#Problem-with-A-as-variable","page":"Minimizing the sum of the k-largest λ","title":"Problem with A as variable","text":"","category":"section"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"Above problems are mostly helpful for illustrative purpose. It is obviously easier to find the sum of the k-largest eigenvalues by simply computing the eigenvalues of A. However, above results become useful if finding A itself is part of the problem. For example, assume we want to find a valid matrix A under the constraints: C textvec(A) = b with the minimum sum of absolute values of the k-largest eigenvalues. We can then solve the equivalent problem:","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"beginarrayll textminimize kz + Tr(U) + Tr(V) \n textsubject to C textvec(A) = b \n zI + V - A succeq 0 \n zI + U + A succeq 0 \n U V succeq 0\nendarray","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/#References","page":"Minimizing the sum of the k-largest λ","title":"References","text":"","category":"section"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"[1] Alizadeh - Interior point methods in semidefinite programming with applications to combinatorial optimization (1995)","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"","category":"page"},{"location":"examples/sum_abs_k_eigenvalues/","page":"Minimizing the sum of the k-largest λ","title":"Minimizing the sum of the k-largest λ","text":"This page was generated using Literate.jl.","category":"page"},{"location":"api/#API-Reference","page":"API Reference","title":"API Reference","text":"","category":"section"},{"location":"api/#Model","page":"API Reference","title":"Model","text":"","category":"section"},{"location":"api/","page":"API Reference","title":"API Reference","text":"COSMO.Model\nCOSMO.assemble!\nCOSMO.set!\nCOSMO.empty_model!\nCOSMO.warm_start_primal!\nCOSMO.warm_start_slack!\nCOSMO.warm_start_dual!","category":"page"},{"location":"api/#COSMO.Model","page":"API Reference","title":"COSMO.Model","text":"Model{T <: AbstractFloat}()\n\nInitializes an empty COSMO model that can be filled with problem data using assemble!(model, P, q,constraints; [settings, x0, s0, y0]).\n\n\n\n\n\n","category":"type"},{"location":"api/#COSMO.assemble!","page":"API Reference","title":"COSMO.assemble!","text":"assemble!(model, P, q, constraint(s); [settings, x0, y0, s0])\n\nAssembles a COSMO.Model with a cost function defind by P and q, and a number of constraints.\n\nThe positive semidefinite matrix P and vector q are used to specify the cost function of the optimization problem:\n\nmin 1/2 x'Px + q'x\ns.t. Ax + b ∈ C\n\nconstraints is a COSMO.Constraint or an array of COSMO.Constraint objects that are used to describe the constraints on x.\n\n\n\nThe optional keyword argument settings can be used to pass custom solver settings:\n\ncustom_settings = COSMO.Settings(verbose = true);\nassemble!(model, P, q, constraints, settings = custom_settings)\n\n\n\nThe optional keyword arguments x0 and y0 can be used to provide the solver with warm starting values for the primal variable x and the dual variable y.\n\nx_0 = [1.0; 5.0; 3.0]\nCOSMO.assemble!(model, P, q, constraints, x0 = x_0)\n\n\n\n\n\n","category":"function"},{"location":"api/#COSMO.set!","page":"API Reference","title":"COSMO.set!","text":"set!(model, P, q, A, b, convex_sets, [settings])\n\nSets model data directly based on provided fields.\n\n\n\n\n\n","category":"function"},{"location":"api/#COSMO.empty_model!","page":"API Reference","title":"COSMO.empty_model!","text":"empty_model!(model)\n\nResets all the fields of model to that of a model created with COSMO.Model() (apart from the settings).\n\n\n\n\n\n","category":"function"},{"location":"api/#COSMO.warm_start_primal!","page":"API Reference","title":"COSMO.warm_start_primal!","text":"warm_start_primal!(model, x0, [ind])\n\nProvides the COSMO.Model with warm starting values for the primal variable x. ind can be used to warm start certain components of x.\n\n\n\n\n\n","category":"function"},{"location":"api/#COSMO.warm_start_slack!","page":"API Reference","title":"COSMO.warm_start_slack!","text":"warm_start_slack!(model, s0, [ind])\n\nProvides the COSMO.Model with warm starting values for the primal slack variable s. ind can be used to warm start certain components of s.\n\n\n\n\n\n","category":"function"},{"location":"api/#COSMO.warm_start_dual!","page":"API Reference","title":"COSMO.warm_start_dual!","text":"warm_start_dual!(model, y0, [ind])\n\nProvides the COSMO.Model with warm starting values for the dual variable y. ind can be used to warm start certain components of y.\n\n\n\n\n\n","category":"function"},{"location":"api/#Constraints","page":"API Reference","title":"Constraints","text":"","category":"section"},{"location":"api/","page":"API Reference","title":"API Reference","text":"COSMO.Constraint\nCOSMO.ZeroSet\nCOSMO.Nonnegatives\nCOSMO.Box\nCOSMO.SecondOrderCone\nCOSMO.PsdCone\nCOSMO.PsdConeTriangle\nCOSMO.ExponentialCone\nCOSMO.DualExponentialCone\nCOSMO.PowerCone\nCOSMO.DualPowerCone","category":"page"},{"location":"api/#COSMO.Constraint","page":"API Reference","title":"COSMO.Constraint","text":"Constraint{T <: AbstractFloat}(A, b, convex_set_type, dim = 0, indices = 0:0)\n\nCreates a COSMO constraint: Ax + b ∈ convex_set.\n\nBy default the following convex set types are supported: ZeroSet, Nonnegatives, SecondOrderCone, PsdCone, PsdConeTriangle.\n\nExamples\n\njulia> COSMO.Constraint([1 0;0 1], zeros(2), COSMO.Nonnegatives)\nConstraint\nSize of A: (2, 2)\nConvexSet: COSMO.Nonnegatives{Float64}\n\nFor convex sets that require their own data, it is possible to pass the pass the instantiated object directly rather than the type name.\n\nExamples\n\njulia> COSMO.Constraint([1 0;0 1], zeros(2), COSMO.Box([-1.;-1.],[1.;1.]))\nConstraint\nSize of A: (2, 2)\nConvexSet: COSMO.Box{Float64}\n\n\n\nThe optional arguments dim and indices can be used to specify A and b for subparts of variable x. If x has dimension dim = 4, then x[2] and x[3] can be constrained to the zero cone in the following way:\n\nExamples\n\njulia> c = COSMO.Constraint([1 0;0 1], zeros(2), COSMO.ZeroSet, 4, 2:3)\nConstraint\nSize of A: (2, 4)\nConvexSet: COSMO.ZeroSet{Float64}\n\nNotice that extra columns of A have been added automatically.\n\njulia>Matrix(c.A)\n2×4 Array{Float64,2}:\n0.0 1.0 0.0 0.0\n0.0 0.0 1.0 0.0\n\n\n\n\n\n","category":"type"},{"location":"api/#COSMO.ZeroSet","page":"API Reference","title":"COSMO.ZeroSet","text":"ZeroSet(dim)\n\nCreates the zero set 0 ^dim of dimension dim. If x ∈ ZeroSet then all entries of x are zero.\n\n\n\n\n\n","category":"type"},{"location":"api/#COSMO.Nonnegatives","page":"API Reference","title":"COSMO.Nonnegatives","text":"Nonnegatives(dim)\n\nCreates the nonnegative orthant x in mathbbR^dim x ge 0 of dimension dim.\n\n\n\n\n\n","category":"type"},{"location":"api/#COSMO.Box","page":"API Reference","title":"COSMO.Box","text":"Box(l, u)\n\nCreates a box or intervall with lower boundary vector l in mathbbR^m cup -infty^m and upper boundary vectoru in mathbbR^mcup +infty^m.\n\n\n\n\n\n","category":"type"},{"location":"api/#COSMO.SecondOrderCone","page":"API Reference","title":"COSMO.SecondOrderCone","text":"SecondOrderCone(dim)\n\nCreates the second-order cone (or Lorenz cone) (tx) in mathrmR^dim x _2 leq t .\n\n\n\n\n\n","category":"type"},{"location":"api/#COSMO.PsdCone","page":"API Reference","title":"COSMO.PsdCone","text":"PsdCone(dim)\n\nCreates the cone of symmetric positive semidefinite matrices mathcalS_+^dim. The entries of the matrix X are stored column-by-column in the vector x of dimension dim. Accordingly X in mathbbS_+ Rightarrow x in mathcalS_+^dim, where X = textmat(x).\n\n\n\n\n\n","category":"type"},{"location":"api/#COSMO.PsdConeTriangle","page":"API Reference","title":"COSMO.PsdConeTriangle","text":"PsdConeTriangle(dim)\n\nCreates the cone of symmetric positive semidefinite matrices. The entries of the upper-triangular part of matrix X are stored in the vector x of dimension dim. A r times r matrix has r(r+1)2 upper triangular elements and results in a vector of mathrmdim = r(r+1)2.\n\nExamples\n\nThe matrix\n\nbeginbmatrix x_1 x_2 x_4 x_2 x_3 x_5 x_4 x_5 x_6 endbmatrix\n\nis transformed to the vector x_1 x_2 x_3 x_4 x_5 x_6^top with corresponding constraint PsdConeTriangle(6).\n\n\n\n\n\n","category":"type"},{"location":"api/#COSMO.ExponentialCone","page":"API Reference","title":"COSMO.ExponentialCone","text":"ExponentialCone(MAX_ITERS = 100, EXP_TOL = 1e-8)\n\nCreates the exponential cone mathcalK_exp = (x y z) mid y geq 0 ye^xy z cup (xyz) mid x leq 0 y = 0 z geq 0 \n\n\n\n\n\n","category":"type"},{"location":"api/#COSMO.DualExponentialCone","page":"API Reference","title":"COSMO.DualExponentialCone","text":"DualExponentialCone(MAX_ITERS::Int = 100, EXP_TOL = 1e-8)\n\nCreates the dual exponential cone mathcalK^*_exp = (x y z) mid x 0 -xe^yx leq e^1 z cup (0yz) mid y geq 0 z geq 0 \n\n\n\n\n\n","category":"type"},{"location":"api/#COSMO.PowerCone","page":"API Reference","title":"COSMO.PowerCone","text":"PowerCone(alpha::Float64, MAX_ITERS::Int = 20, POW_TOL = 1e-8)\n\nCreates the 3-d power cone mathcalK_pow = (x y z) mid x^alpha y^(1-alpha) geq z x geq 0 y geq 0 with 0 alpha 1\n\n\n\n\n\n","category":"type"},{"location":"api/#COSMO.DualPowerCone","page":"API Reference","title":"COSMO.DualPowerCone","text":"DualPowerCone(alpha::Float64, MAX_ITERS::Int = 20, POW_TOL = 1e-8)\n\nCreates the 3-d dual power cone mathcalK^*_pow = (u v w) mid left( fracualpharight)^alpha left( fracv1-alpharight)^(1-alpha) geq w u geq 0 v geq 0 with 0 alpha 1\n\n\n\n\n\n","category":"type"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"EditURL = \"https://github.com/oxfordcontrol/COSMO.jl/blob/master/docs/src/literate/arbitrary_precision.jl\"","category":"page"},{"location":"literate/build/arbitrary_precision/#Arbitrary-Precision","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"","category":"section"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"COSMO allows you to solve problems with arbitrary floating-point precision, e.g. by using BigFloat problem data. To do this, the desired floating point type has to be consistent across the model COSMO.Model{<: AbstractFloat}, the input data and the (optional) settings object COSMO.Settings{<: AbstractFloat}. As an example, assume we want to solve the following quadratic program:","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"beginarrayll textminimize 12 x^top P x + q^top x \ntextsubject to l leq A x leq u\nendarray","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"where P = beginbmatrix 4 1 1 2endbmatrix, q = 1 1^top, A = beginbmatrix 1 1 1 0 0 1endbmatrix and l= 10 0^top, u=1 07 07^top. We start by creating the model with the desired precision:","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"using COSMO, LinearAlgebra, SparseArrays\nmodel = COSMO.Model{BigFloat}()","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"Next, we define the problem data as BigFloat arrays and create the constraint:","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"q = BigFloat[1; 1.]\nP = sparse(BigFloat[4. 1; 1 2])\nA = BigFloat[1. 1; 1 0; 0 1]\nl = BigFloat[1.; 0; 0]\nu = BigFloat[1; 0.7; 0.7]\n\nconstraint = COSMO.Constraint(A, zeros(BigFloat, 3), COSMO.Box(l, u))","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"Notice that the constraint type parameter is dependent on the input data. The same is true for the constraint set Box. Next, we define the settings","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"settings = COSMO.Settings{BigFloat}(verbose = true, kkt_solver = QdldlKKTSolver)","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"and assemble and solve the problem:","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"assemble!(model, P, q, constraint, settings = settings)\nresult = COSMO.optimize!(model);\nnothing #hide","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"Moreover, notice that when no type parameter is specified, all objects default to Float64:","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"model = COSMO.Model()","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"Two limitations to arbitrary precision:","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"Since we call the LAPACK function syevr for eigenvalue decompositions, we currently only support solving problems with PSD constraints in Float32 and Float64.\nWe suggest to use the pure Julia QDLDL linear system solver (kkt_solver = QdldlKKTSolver) when working with arbitrary precision types as some of the other available solvers don't support all available precisions.","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"note: Note\nJuMP does not currently support arbitrary precision. However, if you want to use COSMO directly with MathOptInterface, you can use: COSMO.Optimizer{<: AbstractFloat} as your optimizer. Again, the problem data precision of your MathOptInterface-model has to agree with the optimizer's precision.","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"","category":"page"},{"location":"literate/build/arbitrary_precision/","page":"Arbitrary Precision","title":"Arbitrary Precision","text":"This page was generated using Literate.jl.","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"The source files for all examples can be found in /examples.","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"EditURL = \"https://github.com/oxfordcontrol/COSMO.jl/blob/master/examples/maxcut.jl\"","category":"page"},{"location":"examples/maxcut/#Maximum-Cut-Problem","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"","category":"section"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"We are interested in solving an approximation to the maximum cut problem using semidefinite programming. Consider the graph G(VE) with weights w_ij shown below:","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"# create the weighted adjacency matrix of the graph and plot the graph\nusing LinearAlgebra, Plots, GraphRecipes\nn = 4;\nW = zeros(n, n);\nW[1, 2] = 1; W[1, 4] = 8;\nW[2, 3] = 2; W[2, 4] = 10;\nW[3, 4] = 6;\nW = Symmetric(W)\ngraphplot(W, names = 1:n, edgelabel = W, x = [0; 1; 1; 0], y = [1; 1; 0; 0], fontsize = 12, nodeshape =:circle)","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"The maximum cut problem tries to find a cut or a partition of the graph's vertices into two complementary sets S and barS such that the total weight of the edges between the two sets is maximized. For this small graph the problem is trivial. The optimal solution is S = 123 and barS=4. Formally, this problem can be written as a mixed-integer optimisation problem:","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"beginarrayll textmaximize frac12 sum_i j w_ij(1 - y_i y_j)\ntextsubject to y_i in -1 1 quad forall i in V\nendarray","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"where y_i = 1 indicates that v_i in S and y_i = -1 indicates that v_i in barS. This problem is of interest in the field of integrated circuit design, where one tries to minimize the number of cross-layer connections in a circuit under layout constraints.","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"For more complicated graphs this problem quickly becomes hard to solve to optimality and in fact is known to be NP-hard. For this example we are interested in the randomized approximation algorithm that relaxes the problem to an SDP and was devised in Goemans and Williamson (1995) [1] (see for more details).","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"The approach can be divided into three steps:","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"Solve a relaxed SDP to obtain Y^*\nRecover an approximate solution V via a Cholesky factorisation Y^* = V^top V\nRound the approximate solution using a random vector r from the unit sphere.","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"The authors showed that this approach guarantees a solution of at least 087856 times the optimal solution.","category":"page"},{"location":"examples/maxcut/#Solving-the-primal-SDP","page":"Maximum Cut Problem","title":"Solving the primal SDP","text":"","category":"section"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"Before we formulate the SDP, let's compute the Laplacian matrix L:","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"using LinearAlgebra, Random, SparseArrays, COSMO, JuMP\n\nrng = Random.MersenneTwister(1)\n\n# Compute the Laplacian matrix of the graph\nL = diagm(0 => W * ones(n)) - W;\nnothing #hide","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"Given the Laplacian matrix L, we are looking for the optimal Y^* that solves the following SDP:","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"beginarrayll textmaximize frac14 langle L Y rangle \ntextsubject to Y_ii = 1 quad textfor i = 1dotsn\n Y in mathbfS_+^n\nendarray","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"Notice that the solution Y^* can be viewed as a correlation matrix. Let's solve the problem using COSMO and JuMP.","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"model = JuMP.Model(COSMO.Optimizer);\n@variable(model, Y[1:n, 1:n], PSD);\n@objective(model, Max, 1 / 4 * dot(L, Y));\n@constraint(model, [i = 1:n], Y[i, i] == 1.);\nJuMP.optimize!(model)","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"Yopt = JuMP.value.(Y);\nobj_val = JuMP.objective_value(model)","category":"page"},{"location":"examples/maxcut/#Solving-the-dual-SDP","page":"Maximum Cut Problem","title":"Solving the dual SDP","text":"","category":"section"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"Notice that the decision matrix Y is generally dense (as correctly classified in the solver output above). Therefore, we won't be able to utilize COSMO's chordal decomposition features. However, assuming strong duality, it turns out that we can also solve the dual problem, which is given by:","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"beginarrayll textminimize sum_i gamma_i \ntextsubject to textdiag(gamma) - frac14 L = S \n S in mathbfS_+^n\nendarray","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"As you can see, the matrix S is constrained to have zeros in places specified by the graph (Laplacian). Therefore, COSMO can try to decompose this SDP (see solver output) and speed up its algorithm:","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"model_dual = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, \"complete_dual\" => true));\n@variable(model_dual, γ[1:n]);\n@objective(model_dual, Min, sum(γ));\n@constraint(model_dual, lmi, Symmetric(-1/4 * L + diagm(γ)) in JuMP.PSDCone());\nJuMP.optimize!(model_dual)\nobj_val = JuMP.objective_value(model_dual)","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"The primal variable Y^* can be recovered from the dual variable associated with the LMI-constraint:","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"Yopt = dual(lmi)","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"To get the correct positive semidefinite dual variable, we have to enable PSD completion of the dual variable in COSMO. Now, that we have a solution we can perform the remaining steps in the approximation algorithm.","category":"page"},{"location":"examples/maxcut/#Cholesky-factorisation-of-Y","page":"Maximum Cut Problem","title":"Cholesky factorisation of Y","text":"","category":"section"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"Compute the Cholesky factorisation of Y = V^top V to find the unit vectors v_1 dots v_n.","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"factor = cholesky(Yopt, Val(true); check = false);\nV = Matrix((factor.P * factor.L)');\n# normalize columns\nfor i in 1:n\n V[:, i] ./= norm(V[:, i]);\nend","category":"page"},{"location":"examples/maxcut/#Rounding-the-approximate-solution","page":"Maximum Cut Problem","title":"Rounding the approximate solution","text":"","category":"section"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"It remains to round the unit vectors v_i using a random vector r with each component drawn from mathcalU(0 1), to obtain the y_i's:","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"r = rand(rng, n)\nr /= norm(r, 2)\ny = ones(n)\nfor i in 1:n\n dot(r, V[:, i]) <= 0 && (y[i] = -1)\nend\ny","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"For larger graphs this rounding step could be repeated several times to improve the rounding. As expected S = 1 2 3 and barS= 4.","category":"page"},{"location":"examples/maxcut/#References","page":"Maximum Cut Problem","title":"References","text":"","category":"section"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"[1] Goemans and Williamson - Improved Approximation Algorithms for Maximum Cut and Satisfiability Problems Using Semidefinite Programs (1995)","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"[2] Post-processing code steps 2 and 3 from this JuMP example","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"","category":"page"},{"location":"examples/maxcut/","page":"Maximum Cut Problem","title":"Maximum Cut Problem","text":"This page was generated using Literate.jl.","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"The source files for all examples can be found in /examples.","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"EditURL = \"https://github.com/oxfordcontrol/COSMO.jl/blob/master/examples/svm_primal.jl\"","category":"page"},{"location":"examples/svm_primal/#Support-Vector-Machine","page":"Support Vector Machine","title":"Support Vector Machine","text":"","category":"section"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"We are showing how to solve a support vector machine problem with COSMO (and JuMP).","category":"page"},{"location":"examples/svm_primal/#Generating-the-Dataset","page":"Support Vector Machine","title":"Generating the Dataset","text":"","category":"section"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"We want to classify the points in this example dataset with m = 100 samples and n = 2 features:","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"using Distributions: MvNormal\nusing Plots, LinearAlgebra, SparseArrays, Random, Test","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"# Generate dataset\nrng = Random.MersenneTwister(123);\nnum_samples = 100;\nXpos = rand(rng, MvNormal([1.5, 1.5], 1.25), div(num_samples, 2))';\nXneg = rand(rng, MvNormal([-1.5, -1.5], 1.25), div(num_samples, 2))';\nypos = ones(div(num_samples, 2));\nyneg = -ones(div(num_samples, 2));\nnothing #hide","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"# Plot dataset\nplot(Xpos[:, 1], Xpos[:, 2], color = :red, st=:scatter, markershape = :rect, label = \"positive\", xlabel = \"x1\", ylabel = \"x2\")\nplot!(Xneg[:, 1], Xneg[:, 2], color = :blue, st=:scatter, markershape = :circle, label = \"negative\")","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"with samples (x_1 x_2 ldots x_m) in mathbbR^2 and labels y_i in -11.","category":"page"},{"location":"examples/svm_primal/#Solving-SVM-as-a-QP","page":"Support Vector Machine","title":"Solving SVM as a QP","text":"","category":"section"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"We want to compute the weights w and bias term b of the (soft-margin) SVM classifier:","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"beginarrayll\n textminimize w^2 + lambda sum_i=1^m textmax(0 1 - y_i(w^top x_i - b))\nendarray","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"where lambda is a hyperparameter. This problem can be solved as a quadratic program. We can rewrite above problem into an optimisation problem in primal form by introducing the auxiliary slack variables t_i:","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"t_i = textmax(0 1 - y_i(w^T x_i - b)) quad t_i geq 0","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"This allows us to write the problems in standard QP format:","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"beginarrayll\n textminimize w^2 + lambda sum_i=1^m t_i\n textsubject to y_i (w^top x_i - b) geq 1 - t_i quad textfor i = 1ldots m\n t_i geq 0 quad textfor i = 1ldots m\nendarray","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"Next, we will remove the bias term b by adding an initial feature x_0 = -1 to each sample (now: n = 3):","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"X = [-ones(num_samples) [Xpos; Xneg]];\ny = [ypos; yneg];\nm, n = size(X)","category":"page"},{"location":"examples/svm_primal/#Modelling-in-JuMP","page":"Support Vector Machine","title":"Modelling in JuMP","text":"","category":"section"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"We can model this problem using JuMP and then hand it to COSMO:","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"using JuMP, COSMO","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"λ = 1.0; # hyperparameter\nmodel = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, \"verbose\" => true));\n\n\n@variable(model, w[1:n]);\n@variable(model, t[1:m] >= 0.);\n@objective(model, Min, w' * w + λ * ones(m)' * t);\n@constraint(model, diagm(0 => y) * X * w .+ t .- 1 .>= 0);\nstatus = JuMP.optimize!(model)","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"The optimal weights w = w_0 w_1 w_2^top (where w_0 = b) are:","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"w_opt = JuMP.value.(w)","category":"page"},{"location":"examples/svm_primal/#Plotting-the-hyperplane","page":"Support Vector Machine","title":"Plotting the hyperplane","text":"","category":"section"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"The separating hyperplane is defined by w^top x - b = 0. To plot the hyperplane, we calculate x_2 over a range of x_1 values:","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"x_2 = (-w_1 x_1 - w_0) w_2 text where w_0 = b","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"x1 = -4:0.1:4;\nx2 = (-w_opt[2] * x1 .- w_opt[1]) / w_opt[3]\nplot!(x1, x2, label = \"SVM separator\", legend = :topleft)","category":"page"},{"location":"examples/svm_primal/#Modelling-with-COSMO","page":"Support Vector Machine","title":"Modelling with COSMO","text":"","category":"section"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"The problem can also be solved by transforming it directly into COSMO's problem format. Define COSMO`s x-variable to be x=w t^top and choose P, q, accordingly:","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"P = blockdiag(spdiagm(0 => ones(n)), spzeros(m, m));\nq = [zeros(n); 0.5 * λ * ones(m)];\nnothing #hide","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"Next we transform the first constraint y_i (w^top x_i - b) geq 1 - t_i quad textfor i = 1ldots m into COSMO's constraint format: Ax + b in mathcalK.","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"A1 = [(spdiagm(0 => y) * X) spdiagm(0 => ones(m))];\nb1 = -ones(m);\ncs1 = COSMO.Constraint(A1, b1, COSMO.Nonnegatives);\nnothing #hide","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"It remains to specify the constraint t_i geq 0 quad textfor i = 1ldots m:","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"A2 = spdiagm(0 => ones(m));\nb2 = zeros(m);\ncs2 = COSMO.Constraint(A2, b2, COSMO.Nonnegatives, m+n, n+1:m+n);\nnothing #hide","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"Create, assemble and solve the COSMO.Model:","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"model2 = COSMO.Model();\nassemble!(model2, P, q, [cs1; cs2]);\nresult2 = COSMO.optimize!(model2);\nw_opt2 = result2.x[1:3];\n@test norm(w_opt2 - w_opt, Inf) < 1e-3","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"","category":"page"},{"location":"examples/svm_primal/","page":"Support Vector Machine","title":"Support Vector Machine","text":"This page was generated using Literate.jl.","category":"page"},{"location":"jump/#JuMP-Interface","page":"JuMP Interface","title":"JuMP Interface","text":"","category":"section"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"Our JuMP interface allows you to describe and modify your optimisation problem with JuMP and use COSMO as the backend solver. The interface is defined in /src/MOIWrapper.jl.","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"note: Note\nWe assume here that the latest JuMP release (~0.21.0) is used.","category":"page"},{"location":"jump/#Use-COSMO","page":"JuMP Interface","title":"Use COSMO","text":"","category":"section"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"To specify COSMO as the solver for your JuMP model, load the solver module with using COSMO and then pass a COSMO.Optimizer when initialising the JuMP model:","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"m = JuMP.Model(COSMO.Optimizer);","category":"page"},{"location":"jump/#Specify-Solver-Settings","page":"JuMP Interface","title":"Specify Solver Settings","text":"","category":"section"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"Solver-specific settings can be passed using the optimizer_with_attributes() function. For example, if you want to adjust the maximum number of iterations and turn on verbose printing use","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"m = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, \"max_iter\" => 5000, \"verbose\" => true));","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"Note that the attributes are passed as key-value pairs and the keys are strings. This is slightly different to using the native COSMO interface. Equivalently, one can also use:","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"m = JuMP.Model(COSMO.Optimizer);\nset_optimizer_attribute(m, \"max_iter\", 5000)\nset_optimizer_attribute(m, \"verbose\", true)","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"The full list of available settings can be found in the Settings section. All of them should be compatible with JuMP.","category":"page"},{"location":"jump/#Results","page":"JuMP Interface","title":"Results","text":"","category":"section"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"After solving the problem the result can be obtained using the standard JuMP commands. To see if the optimisation was successful use","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"JuMP.termination_status(m)\nJuMP.primal_status(m)","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"If a solution is available, the optimal objective value can be queried using","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"JuMP.objective_value(m)","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"and the value of a decision variable x can be obtained with","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"JuMP.value.(x)","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"To query the number of iterations of the algorithm use:","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"iter = MOI.get(m, COSMO.ADMMIterations())","category":"page"},{"location":"jump/#Feasibility-of-solution","page":"JuMP Interface","title":"Feasibility of solution","text":"","category":"section"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"In case JuMP.termination_status(m) is MOI.ITERATION_LIMIT, JuMP.primal_status(m) (resp. JuMP.dual_status(m)) should be checked to determine the feasibility status of the primal (resp. dual) solutions. A primal (resp. dual) solution is considered feasible (resported as MOI.FEASIBLE_POINT) if","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"r epsilon_textabs + sepsilon_textrel","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"where","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"r is the primal (resp. dual) residual r_prim = lVert Ax + s - b rVert_infty","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"(resp. r_dual = lVert P x + q - A^top * μ rVert_infty),","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"s is the primal (resp. dual) scaling of this residual","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"max_norm_prim = max(lVert Ax rVert_infty lVert s rVert_infty lVert b rVert_infty) (resp. max_norm_dual = max(lVert P x rVert_infty lVert q rVert_infty lVert A^top * μ rVert_infty)).","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"epsilon_textabs is the optimizer attribute eps_abs and\nepsilon_textrel is the optimizer attribute eps_rel.","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"A primal (resp. dual solution) is considered nearly feasible (resported as MOI.NEARLY_FEASIBLE_POINT) if it is not feasible but the same inequality is satisfied where the right-hand side is multiplied by the optimizer attribute nearly_ratio.","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"Otherwise, the solution is considered infeasible (resported as MOI.INFEASIBLE_POINT). Note that this does not mean that the problem is infeasible, it only means that the current primal (resp. dual) solution is infeasible.","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"Note that, in case JuMP.termination_status(m) is MOI.ITERATION_LIMIT, the feasibility of the solution is checked with the current optimizer attributes, eps_abs, eps_rel and nearly_ratio. This means that if they are changed after optimize!, the status may change! This can be used to check whether the solution is feasible up to a relaxed tolerance. For instance, the following could happen:","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"optimize!(m)\nprimal_status(m) # MOI.INFEASIBLE_POINT\n# This means that `r_prim >= nearly_ratio * (eps_abs * max_norm_prim * eps_rel)`\nset_optimizer_attribute(m, \"eps_abs\", 1e-4)\nprimal_status(m) # MOI.NEARLY_FEASIBLE_POINT\n# This means that `1 <= r_prim / (eps_abs * max_norm_prim * eps_rel) < nearly_ratio`","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"The values of r_prim, max_norm_prim, r_dual and max_norm_dual can also be accessed as the fields of the res_info struct that can be obtained as follows","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"res_info = MOI.get(m, COSMO.RawResult()).info","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"Then, the feasibility can either be checked manually as in","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"res_info.r_prim < eps_abs + res_info.max_norm_prim * eps_rel","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"or using one of the following:","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"COSMO.is_primal_feasible(res_info, eps_abs, eps_rel)\nCOSMO.is_primal_nearly_feasible(res_info, eps_abs, eps_rel)\nCOSMO.is_dual_feasible(res_info, eps_abs, eps_rel)\nCOSMO.is_dual_nearly_feasible(res_info, eps_abs, eps_rel)","category":"page"},{"location":"jump/","page":"JuMP Interface","title":"JuMP Interface","text":"For more information on how to use JuMP check the JuMP documentation.","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"EditURL = \"https://github.com/oxfordcontrol/COSMO.jl/blob/master/docs/src/literate/portfolio_model_updates.jl\"","category":"page"},{"location":"literate/build/portfolio_model_updates/#Model-Updates","page":"Model Updates","title":"Model Updates","text":"","category":"section"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"When a pareto-optimal front in portfolio optimisation is computed for different risk-aversion parameters gamma, we are repeatedly solving the same model with only a slight change in the cost function. This allows us to reuse the KKT factorisation from previous solves and recompute the solution very efficiently, see Algorithm. For more information about solving portfolio optimsiation problems with COSMO see Portfolio Optimisation. We are planning to solve the following QP for different values of gamma:","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"beginarrayll textminimize x^top D x + y^top y - gamma^-1 mu^top x \ntextsubject to y = F^top x \n 1^top x = d + 1^top x^0 \n x geq 0\nendarray","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"As you can see gamma enters only in the linear part of the cost function, i.e. our vector q. Let's generate some example data:","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"using LinearAlgebra, SparseArrays, Random, COSMO\n\nrng = Random.MersenneTwister(1)\nk = 5; # number of factors\nn = k * 10; # number of assets\nD = spdiagm(0 => rand(rng, n) .* sqrt(k))\nF = sprandn(rng, n, k, 0.5); # factor loading matrix\nμ = (3 .+ 9. * rand(rng, n)) / 100. # expected returns between 3% - 12%\nd = 1 # we are starting from all cash\nx0 = zeros(n);\nnothing #hide","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"We want to solve this problem for the following risk-parameters:","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"gammas = [ 0.001, 0.01, 0.1, 0.5, 1., 3., 10, 100, 1000]","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"Let's solve the problem for the first parameter:","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"# x = (x, y)\nγ = gammas[1]\nP = blockdiag(D, spdiagm(0 => ones(k)))\nq = [-μ ./ (2 * γ); zeros(k)];\n\nAeq = [ transpose(F) -diagm(0 => ones(k));\n ones(1, n) zeros(1, k)]\nbeq = [zeros(k); -1.0]\nAineq = [spdiagm(0 => ones(n)) spzeros(n, k)]\n\ncs1 = COSMO.Constraint(Aeq, beq, COSMO.ZeroSet);\ncs2 = COSMO.Constraint(Aineq, zeros(n), COSMO.Nonnegatives);\ncosmo_settings = COSMO.Settings(verbose = false)\n\n# solve the problem once\nmodel = COSMO.Model();\nassemble!(model, P, q, [cs1; cs2], settings = cosmo_settings);\nresult = COSMO.optimize!(model);\nx_opt = view(result.x, 1:n);\ny_opt = view(result.x, n+1:n+k);\n\n# allocate some space for results\nrisks = zeros(length(gammas));\nreturns = zeros(length(gammas));\nreturns[1] = dot(μ, x_opt);\nrisks[1] = sqrt(dot(y_opt, y_opt));\nnothing #hide","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"After a successful solve, we can use the update!(model, q = nothing, b = nothing) function to update our model and re-solve the problem efficiently:","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"# solve the problem efficiently for the remaining γ using model updates\nfunction solve_repeatedly!(returns, risks, gammas, model, k, n, μ)\n for (i, γ) in enumerate(gammas[2:end])\n q_new = [-μ ./ (2 * γ); zeros(k)];\n\n # here we use the update! function to change the model\n update!(model, q = q_new)\n result = COSMO.optimize!(model);\n\n x_opt = view(result.x, 1:n)\n y_opt = view(result.x, n+1:n+k)\n returns[i + 1] = dot(μ, x_opt)\n risks[i + 1] = sqrt(dot(y_opt, y_opt))\n end\nend\nsolve_repeatedly!(returns, risks, gammas, model, k, n, μ);\nnothing #hide","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"We can now plot the risk-return trade-off curve:","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"using Plots\nPlots.plot(risks, returns, xlabel = \"Standard deviation (risk)\", ylabel = \"Expected return\", title = \"Risk-return trade-off for efficient portfolios\", legend = false)","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"","category":"page"},{"location":"literate/build/portfolio_model_updates/","page":"Model Updates","title":"Model Updates","text":"This page was generated using Literate.jl.","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"The source files for all examples can be found in /examples.","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"EditURL = \"https://github.com/oxfordcontrol/COSMO.jl/blob/master/examples/portfolio_optimisation.jl\"","category":"page"},{"location":"examples/portfolio_optimisation/#Portfolio-Optimisation","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"","category":"section"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"We consider a single-period Markowitz portfolio optimisation example. Assume that we have a portfolio with n assets at the beginning of time period t. Given some forecasts on risks and expected returns we try to find the optimal trade vector that rebalances the portfolio to achieve a good balance between expected risk (variance) x^top Sigma x and returns mu^top x. In it's most simple form we want to solve:","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"beginarrayll textmaximize mu^top x - gamma (x^top Sigma x)\ntextsubject to 1^top x = d + 1^top x^0 \n x geq 0\nendarray","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"with variable x in mathbfR^n, mu forecasted (expected) returns, gamma 0 risk aversion parameter x^0_i represents the initial investment in asset i and d represents the cash reserve. Consequently, the equality constraint tells us that the sum of the new allocation vector x has to equal the initial allocation plus the cash reserve. Furthermore, the covariance matrix of our risk model is given by Sigma in mathbfS_+^n. Here we assume a factor risk model, i.e. we can write Sigma as Sigma = D + F F^top where D is diagonal and the factor matrix F has a lower rank k n. This approach allows us to reduce the number of nonzeros in the problem. Furthermore, note that we don't consider shortselling in this example. Let's generate some problem data:","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"using LinearAlgebra, SparseArrays, Random, COSMO, JuMP, Test\n\n# generate the data\nrng = Random.MersenneTwister(1)\nk = 5; # number of factors\nn = k * 10; # number of assets\nD = spdiagm(0 => rand(rng, n) .* sqrt(k))\nF = sprandn(rng, n, k, 0.5); # factor loading matrix\nμ = (3 .+ 9. * rand(rng, n)) / 100. # expected returns between 3% - 12%\nγ = 1.0; # risk aversion parameter\nd = 1 # we are starting from all cash\nx0 = zeros(n);\nnothing #hide","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"We can now write the problem as a QP:","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"beginarrayll textminimize x^top D x + y^top y - gamma^-1 mu^top x \ntextsubject to y = F^top x \n 1^top x = d + 1^top x^0 \n x geq 0\nendarray","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"Before considering other effects, let's create the model in JuMP and solve it using COSMO:","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"model = JuMP.Model(COSMO.Optimizer);\n@variable(model, x[1:n]);\n@variable(model, y[1:k]);\n@objective(model, Min, x' * D * x + y' * y - 1/γ * μ' * x);\n@constraint(model, y .== F' * x);\n@constraint(model, sum(x) == d + sum(x0));\n@constraint(model, x .>= 0);\nJuMP.optimize!(model)","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"After solving the problem, we can calculate the expected return and risk sigma= sqrtx^* top Sigma x^*:","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"x_opt = JuMP.value.(x);\ny_opt = JuMP.value.(y);\nexpected_return_basic = dot(μ, x_opt)","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"expected_risk_basic = sqrt(dot(y_opt, y_opt))","category":"page"},{"location":"examples/portfolio_optimisation/#Using-standard-deviation-in-the-model","page":"Portfolio Optimisation","title":"Using standard deviation in the model","text":"","category":"section"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"It is pointed out in [1] that above problem formulation can lead to numerical problems, e.g. if Sigma is not strictly positive semidefinite. Another option is to formulate the risk constraint in terms of the standard deviation M^top x where M M^top = D + F F^top and bound it using a second-order cone constraint:","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"beginarrayll textminimize - mu^top x \ntextsubject to M^top x leq gamma \n 1^top x = d + 1^top x^0 \n x geq 0\nendarray","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"Mt = [D.^0.5; F']\nmodel = JuMP.Model(COSMO.Optimizer);\n@variable(model, x[1:n]);\n@objective(model, Min, - μ' * x);\n@constraint(model, [γ; Mt * x] in SecondOrderCone()); # ||M'x|| <= γ\n@constraint(model, sum(x) == d + sum(x0));\n@constraint(model, x .>= 0);\nJuMP.optimize!(model)","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"Note that the result is different from the example above because gamma scales the problem in a different way. Here it can be seen as an upper bound on the standard deviation of the portfolio.","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"x_opt = JuMP.value.(x);\nexpected_return = dot(μ, x_opt)","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"Let us verify that the bound holds:","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"@test norm(Mt * x_opt) <= γ","category":"page"},{"location":"examples/portfolio_optimisation/#Pareto-optimal-front","page":"Portfolio Optimisation","title":"Pareto-optimal front","text":"","category":"section"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"The above portfolio optimisation approach yields the optimal expected return for a given level of risk. The result is obviously impacted by the risk aversion gamma parameter. To visualise the trade-off and present the investor with an efficient Pareto optimal portfolio for their risk appetite we can compute the optimal portfolio for many choices of gamma and plot the corresponding risk-return trade-off curve.","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"gammas = [ 0.001, 0.01, 0.1, 0.5, 1., 3., 10, 100, 1000]\nrisks = zeros(length(gammas))\nreturns = zeros(length(gammas))\nmodel = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, \"verbose\" => false));\n@variable(model, x[1:n]);\n@variable(model, y[1:k]);\n@objective(model, Min, x' * D * x + y' * y - 1/γ * μ' * x);\n@constraint(model, y .== F' * x);\n@constraint(model, sum(x) == d + sum(x0));\n@constraint(model, x .>= 0);\n\n# solve the same problem for different values of γ\nfor (k, gamma) in enumerate(gammas)\n coeff = - 1/gamma * μ\n JuMP.set_objective_coefficient.(model, x, coeff)\n JuMP.optimize!(model)\n local x_opt = JuMP.value.(x);\n local y_opt = JuMP.value.(y);\n returns[k] = dot(μ, x_opt)\n risks[k] = sqrt(dot(y_opt, y_opt))\nend","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"We can now plot the risk-return trade-off curve:","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"using Plots\nPlots.plot(risks, returns, xlabel = \"Standard deviation (risk)\", ylabel = \"Expected return\", title = \"Risk-return trade-off for efficient portfolios\", legend = false)","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"note: Note\nWhen the model is updated in JuMP as above the JuMP.model is copied in full to COSMO. We are trying to improve the interface with respect to model updates in the future. Until then you can use Model Updates in COSMOs native interface.","category":"page"},{"location":"examples/portfolio_optimisation/#Transaction-costs","page":"Portfolio Optimisation","title":"Transaction costs","text":"","category":"section"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"In the model above we assume that trading the assets is free and does not impact the market. However, this is clearly not the case in reality. To make the example more realistic consider the following cost c_j associated with the trade δ_j = x_j - x_j^0:","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"c_j(delta_j) = a_j delta_j + b_j delta_j^32","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"where the first term models the bid-ask spread and broker fees for asset j. The second term models the impact on the market that our trade has. This is obviously only a factor if the volume of our trade is significant. The constant b_j is a function of the total volume traded in the considered time periode and the price volatility of the asset and has to be estimated by the trader. To make this example simple we consider the same coefficients a and b for every asset. The delta_j^32 term can be easily modeled using a power cone constraint mathcalK_pow = (x y z) mid x^alpha y^(1-alpha) geq z x geq 0 y geq 0 0 leq alpha leq 1 . In fact this can be used to model any market impact function with exponent greater than 1. We can write the total transaction cost a^top s + b^top t where s_j bounds the absolute value of delta_j and t_j is used to bound the term x_j - x_j^0^32 leq t_j using a power cone formulation: (t_j 1 x_j - x_j^0) in mathcalK_pow(23).","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"a = 1e-3\nb = 1e-1\nγ = 1.0;\nmodel = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, \"eps_abs\" => 1e-5, \"eps_rel\" => 1e-5));\n@variable(model, x[1:n]);\n@variable(model, y[1:k]);\n@variable(model, t[1:n]);\n@variable(model, s[1:n]);\n@objective(model, Min, x' * D * x + y' * y - 1/γ * μ' * x);\n@constraint(model, y .== F' * x);\n@constraint(model, x .>= 0);\n\n# transaction costs\n@constraint(model, sum(x) + a * sum(s) + b * sum(t) == d + sum(x0) );\n@constraint(model, [i = 1:n], x[i] - x0[i] <= s[i]); # model the absolute value with slack variable s\n@constraint(model, [i = 1:n], x0[i] - x[i] <= s[i]);\n@constraint(model, [i = 1:n], [t[i], 1, x[i] - x0[i]] in MOI.PowerCone(2/3));\nJuMP.optimize!(model)","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"Let's look at the expected return and the total transaction cost:","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"x_opt = JuMP.value.(x);\ny_opt = JuMP.value.(y);\ns_opt = JuMP.value.(s);\nt_opt = JuMP.value.(t);\nexpected_return = dot(μ, x_opt)","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"expected_risk = dot(y_opt, y_opt)","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"transaction_cost = a * sum(s_opt) + b * sum( t_opt)","category":"page"},{"location":"examples/portfolio_optimisation/#References","page":"Portfolio Optimisation","title":"References","text":"","category":"section"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"[1] Mosek Case Studies","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"","category":"page"},{"location":"examples/portfolio_optimisation/","page":"Portfolio Optimisation","title":"Portfolio Optimisation","text":"This page was generated using Literate.jl.","category":"page"},{"location":"getting_started/#Getting-Started","page":"Getting Started","title":"Getting Started","text":"","category":"section"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"This user guide describes the basic structures and functions to define an optimisation problem, to solve the problem and to analyse the result. If you want to use JuMP to describe the problem, see the JuMP Interface section.","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"COSMO solves optimisation problems in the following format:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"beginarrayll textminimize textstylefrac12x^top Px + q^top x textsubject to Ax + s = b s in mathcalK endarray","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"with decision variables x in mathbbR^n, s in mathbbR^m and data matrices P=P^top succeq 0, q in mathbbR^n, A in mathbbR^m times n, and b in mathbbR^m. The convex set mathcalK is a composition of convex sets and cones.","category":"page"},{"location":"getting_started/#Model","page":"Getting Started","title":"Model","text":"","category":"section"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"The problem data, user settings and workspace variables are all stored in a Model. To get started define an empty model:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"model = COSMO.Model()","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"To initialize the model with an optimisation problem we need to define three more things:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"the objective function, i.e. the matrix P and the vector q in frac12x^top P x + q^top x\nan array of constraints\na Settings object that specifies how COSMO solves the problem (optional)","category":"page"},{"location":"getting_started/#Objective-Function","page":"Getting Started","title":"Objective Function","text":"","category":"section"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"To set the objective function of your optimisation problem simply define the square positive semidefinite matrix P in mathrmR^ntimes n and the vector q in mathrmR^n. You might have to transform your optimisation problem into a solver compatible format for this step.","category":"page"},{"location":"getting_started/#Constraints","page":"Getting Started","title":"Constraints","text":"","category":"section"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"The COSMO interface expects constraints to have the form A_i x + b_i in mathcalK_i, where mathcalK_i is one of the convex sets defined below:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"Convex Set Description\nZeroSet The set 0 ^dim that contains the origin\nNonnegatives The nonnegative orthant x in mathbbR^dim x_i ge 0 forall i=1dotsmathrmdim \nBox(l, u) The hyperbox x in mathbbR^dim l leq x leq u with vectors l in mathbbR^dim cup -infty and u in mathbbR^dim cup +infty\nSecondOrderCone The second-order (Lorenz) cone (tx) in mathbbR^dim x_2 leq t \nPsdCone The vectorized positive semidefinite cone mathcalS_+^dim. x is the vector obtained by stacking the columns of the positive semidefinite matrix X, i.e. X in mathbbS^sqrtdim_+ rarr textvec(X) = x in mathcalS_+^dim\nPsdConeTriangle The vectorized positive semidefinite cone mathcalS_+^dim. x is the vector obtained by stacking the columns of the upper triangular part of the positive semidefinite matrix X, i.e. X in mathbbS^d_+ rarr textsvec(X) = x in mathcalS_+^dim where d=sqrt14 + 2 textdim - 12\nExponentialCone The exponential cone mathcalK_exp = (x y z) mid y geq 0 ye^xy z cup (xyz) mid x leq 0 y = 0 z geq 0 \nDualExponentialCone The dual exponential cone mathcalK^*_exp = (x y z) mid x 0 -xe^yx leq e^1 z cup (0yz) mid y geq 0 z geq 0 \nPowerCone(alpha) The 3d power cone mathcalK_pow = (x y z) mid x^alpha y^(1-alpha) geq z x geq 0 y geq 0 with 0 alpha 1\nDualPowerCone(alpha) The 3d dual power cone mathcalK^*_pow = (u v w) mid left( fracualpharight)^alpha left( fracv1-alpharight)^(1-alpha) geq w u geq 0 v geq 0 with 0 alpha 1","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"The constructor for a constraint expects a matrix A, a vector b and a convex_set.","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"Lets consider a problem with a decision variable x in mathbbR^5. Suppose we want to create the two constraint x_2 + 5 geq 0 and x_3 - 3 geq 0. We can do this either by creating two constraints and adding them to an array:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":" constraint1 = COSMO.Constraint([0.0 1.0 0.0 0.0 0.0], 5.0, COSMO.Nonnegatives)\n constraint2 = COSMO.Constraint([0.0 0.0 1.0 0.0 0.0], -3.0, COSMO.Nonnegatives)\n constraints = [constraint1; constraint2]","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"The second option is to include both in one constraint:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"constraint1 = COSMO.Constraint([0.0 1.0 0.0 0.0 0.0; 0.0 0.0 1.0 0.0 0.0], [5.0; -3.0], COSMO.Nonnegatives)","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"Another way to construct the constraint is to used the optional arguments dim, the dimension of x, and indices, the elements of x that appear in the constraint. When specifying these arguments, A and b only refer to the elements of x in indices:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"constraint1 = COSMO.Constraint([1.0 0.0; 0.0 1.0], [5.0; -3.0], COSMO.Nonnegatives, 5, 2:3)","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"Consider as a second example the positive semidefinite constraint on a matrix X in mathbbS_+^3. Our decision variable is the vector x obtained by stacking the columns of X. We can specify the constraint on x in the following way:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"I_9 x + 0_9 in mathcalS_+^9","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"or in Julia:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"constraint1 = COSMO.Constraint(Matrix(1.0I, 9, 9), zeros(9), COSMO.PsdCone)","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"Several constraints can be combined in an array:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"constraints = [constraint_1, constraint_2, ..., constraint_N]","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"It is usually enough to pass the convex_set as a type. However, some convex sets like Box, PowerCone and DualPowerCone require more information to be created. In that case you have to pass an object to the constructor, e.g.","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"l = -ones(2)\nu = ones(2)\nconstraint = COSMO.Constraint(Matrix(1.0I, 2, 2), zeros(2), COSMO.Box(l, u))","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"or in the case of a power Cone you specify the alpha:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"constraint = COSMO.Constraint(Matrix(1.0I, 3, 3), zeros(3), COSMO.PowerCone(0.6))","category":"page"},{"location":"getting_started/#Settings","page":"Getting Started","title":"Settings","text":"","category":"section"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"The solver settings are stored in a Settings object and can be adjusted by the user. To create a Settings object just call the constructor:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"COSMO.Settings","category":"page"},{"location":"getting_started/#COSMO.Settings","page":"Getting Started","title":"COSMO.Settings","text":"COSMO.Settings{T}(; kwargs) where {T <: AbstractFloat}\n\nCreates a COSMO settings object that is used to pass user settings to the solver.\n\nArgument Description Values (default)\nrho ADMM rho step 0.1\nsigma ADMM sigma step 1e-6\nalpha Relaxation parameter 1.6\neps_abs Absolute residual tolerance 1e-5\neps_rel Relative residual tolerance 1e-5\nnearly_ratio Residual tolerance ratio between MOI.NEARLY_FEASIBLE_POINT and MOIFEASIBLE_POINT 100\neps_prim_inf Primal infeasibility tolerance 1e-5\neps_dual_inf Dual infeasibility tolerance 1e-5\nmax_iter Maximum number of iterations 5000\nverbose Verbose printing false\nverbose_timing Verbose timing false\nkkt_solver Linear System solver QdldlKKTSolver\ncheck_termination Check termination interval 25\ncheck_infeasibility Check infeasibility interval 40\nscaling Number of scaling iterations 10\nadaptive_rho Automatic adaptation of step size parameter true\nadaptiverhomax_adaptions Max number of rho adaptions typemax(Int64) (deactivated)\ndecompose Activate to decompose chordal psd constraints true\ncomplete_dual Activate to complete the dual variable after decomposition false\nmerge_strategy Choose a strategy for clique merging CliqueGraphMerge\ncompact_transformation Choose how a decomposed problem is transformed true\ntime_limit Set solver time limit in s 0 (deactivated)\naccelerator Acceleration scheme AndersonAccelerator{T, Type2{QRDecomp}, RestartedMemory, NoRegularizer}\naccelerator_activation Accelerator activation ImmediateActivation\nsafeguard Accelerator safeguarding true\nsafeguard_tol Safeguarding tolerance 2.0\n\n\n\n\n\n","category":"type"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"To adjust those values, either pass your preferred option and parameter as a key-value pair to the constructor or edit the corresponding field afterwards. For example if you want to enable verbose printing and increase the solver accuracy, you can type","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"settings = COSMO.Settings(verbose = true, eps_abs = 1e-5, eps_rel = 1e-5)\n# the following is equivalent\nsettings = COSMO.Settings()\nsettings.verbose = true\nsettings.eps_abs = 1e-5\nsettings.eps_rel = 1e-5","category":"page"},{"location":"getting_started/#Assembling-the-model","page":"Getting Started","title":"Assembling the model","text":"","category":"section"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"Once the objective function and an array of constraints have been defined, we can assemble the model with","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"COSMO.assemble!(model, P, q, constraints)","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"This simply sets the corresponding variables in the model and transforms the array of constraints into the problem format defined at the top of the page.","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"If you want to change the default settings, you can pass your settings object custom_settings to the assemble! function:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"COSMO.assemble!(model, P, q, constraints, settings = custom_settings)","category":"page"},{"location":"getting_started/#Warm-starting","page":"Getting Started","title":"Warm starting","text":"","category":"section"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"One of the advantages of ADMM-based solvers is that they can be easily warm started. By providing starting values for the primal variable x and/or the dual variable y in the vicinity of their optimal values, the number of iterations to convergence can often be dramatically decreased.","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"Consider the case where you have a decision variable x in mathbbR^3 and a dual variable y in mathbbR^2. Assume you expect their optimal values to be close to x_0 = (1 5 3) and y_0 = (1 2). You can pass these values when assembling the model.","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"x_0 = [1.0; 5.0; 3.0]\ny_0 = [1.0; 2.0]\nCOSMO.assemble!(model, P, q, constraints, x0 = x_0, y0 = y_0)","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"Another option is to use","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"COSMO.assemble!(model, P, q, constraints)\nwarm_start_primal!(model, x_0)\nwarm_start_dual!(model, y_0)","category":"page"},{"location":"getting_started/#Solving","page":"Getting Started","title":"Solving","text":"","category":"section"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"After the model has been assembled, we can solve the problem by typing","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"results = COSMO.optimize!(model)","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"Once the solver algorithm terminates, it will return a Results object that gives information about the status of the solver. If successful, it contains the optimal objective value and optimal primal and dual variables. For more information see the following section.","category":"page"},{"location":"getting_started/#Results","page":"Getting Started","title":"Results","text":"","category":"section"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"After attempting to solve the problem, COSMO will return a result object with the following fields:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"COSMO.Result\nCOSMO.ResultInfo","category":"page"},{"location":"getting_started/#COSMO.Result","page":"Getting Started","title":"COSMO.Result","text":"Result{T <: AbstractFloat}\n\nObject returned by the COSMO solver after calling optimize!(model). It has the following fields:\n\nFieldname Type Description\nx Vector{T} Primal variable\ny Vector{T} Dual variable\ns Vector{T} (Primal) set variable\nobj_val T Objective value\niter Int Total number of ADMM iterations (incl. safeguarding_iter)\nsafeguarding_iter Int Number of iterations due to safeguarding of accelerator\nstatus Symbol Solution status\ninfo COSMO.ResultInfo Struct with more information\ntimes COSMO.ResultTimes Struct with several measured times\n\n\n\n\n\n","category":"type"},{"location":"getting_started/#COSMO.ResultInfo","page":"Getting Started","title":"COSMO.ResultInfo","text":"ResultInfo{T <: AbstractFloat}\n\nObject that contains further information about the primal residual, the dual residuals and the rho updates.\n\n\n\n\n\n","category":"type"},{"location":"getting_started/#Status-Codes","page":"Getting Started","title":"Status Codes","text":"","category":"section"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"COSMO will return one of the following statuses:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"Status Code Description\n:Solved An optimal solution was found\n:Unsolved Default value\n:Max_iter_reached Solver reached iteration limit (set with Settings.max_iter)\n:Time_limit_reached Solver reached time limit (set with Settings.time_limit)\n:Primal_infeasible Problem is primal infeasible\n:Dual_infeasible Problem is dual infeasible","category":"page"},{"location":"getting_started/#Timings","page":"Getting Started","title":"Timings","text":"","category":"section"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"If settings.verbose_timing is set to true, COSMO will report the following times in result.times:","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"COSMO.ResultTimes","category":"page"},{"location":"getting_started/#COSMO.ResultTimes","page":"Getting Started","title":"COSMO.ResultTimes","text":"ResultTimes\n\nPart of the Result object returned by the solver. ResultTimes contains timing results for certain parts of the algorithm:\n\nTime Name Description\nsolver_time Total time used to solve the problem\nsetup_time Setup time = graph_time + init_factor_time + scaling_time\nscaling_time Time to scale the problem data\ngraph_time Time used to perform chordal decomposition\ninit_factor_time Time used for initial factorisation of the system of linear equations\nfactor_update_time Sum of times used to refactor the system of linear equations due to rho\niter_time Time spent in iteration loop\nproj_time Time spent in projection functions\npost_time Time used for post processing\nupdate_time Time spent in the update! function of the accelerator\naccelerate_time Time spent in the accelerate! function of the accelerator\n\nBy default COSMO only measures solver_time, setup_time and proj_time. To measure the other times set verbose_timing = true.\n\n\n\n\n\n","category":"type"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"It holds: solver_time = setup_time+ iter_time + factor_update_time + post_time,","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"setup_time = graph_time+ init_factor_time + scaling_time,","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"proj_time is a subset of iter_time.","category":"page"},{"location":"getting_started/#Updating-the-model","page":"Getting Started","title":"Updating the model","text":"","category":"section"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"In some cases we want to solve a large number of similar models. COSMO allows you to update the model problem data vectors q and b after the first call of optimize!(). After changing the problem data, COSMO can reuse the factorisation step of the KKT matrix from the previous problem which can save a lot of time in the case of LPs and QPs.","category":"page"},{"location":"getting_started/","page":"Getting Started","title":"Getting Started","text":"COSMO.update!","category":"page"},{"location":"getting_started/#COSMO.update!","page":"Getting Started","title":"COSMO.update!","text":"update!(model, q = nothing, b = nothing)\n\nUpdates the model data for b or q. This can be done without refactoring the KKT matrix. The vectors will be appropriatly scaled.\n\n\n\n\n\n","category":"function"},{"location":"method/#Method","page":"Method","title":"Method","text":"","category":"section"},{"location":"method/","page":"Method","title":"Method","text":"This section describes COSMO's underlying ADMM algorithm and how the user can use the settings to adjust this algorithm. For a more detailed explanation take a look at the associated publication in Citing COSMO.","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"COSMO solves problems with quadratic objective function and a number of conic constraints in the following form:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"beginarrayll textminimize textstylefrac12x^top Px + q^top x textsubject to Ax + s = b s in mathcalK endarray","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"with primal decision variable x in mathbbR^n, primal slack variable s in mathbbR^m. The objective function is defined by positive semidefinite matrix P=P^top succeq 0 and vector q in mathbbR^n. The constraints are defined by matrix A in mathbbR^m times n, vector b in mathbbR^m and a non-empty, closed convex set mathcalK. The convex set itself can be a Cartesian product of convex sets in the form:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":" mathcalK = mathcalK_1^m_1 times mathcalK_2^m_2 times cdots times mathcalK_N^m_N","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"Accordingly, by an appropriate choice of convex sets one can represent any LP, QP, SOCP or SDP.","category":"page"},{"location":"method/#Dual-problem","page":"Method","title":"Dual problem","text":"","category":"section"},{"location":"method/","page":"Method","title":"Method","text":"The dual problem of the optimisation problem above is given by:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"beginarrayll\ntextmaximize -textstylefrac12x^top Px - b^top y - textsup_s in mathcalK(-y^top s ) \ntextsubject to Px + A^top y = -q\n y in (mathcalK^infty)^* \nendarray","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"with dual variable y in mathbbR^m.","category":"page"},{"location":"method/#Algorithm","page":"Method","title":"Algorithm","text":"","category":"section"},{"location":"method/","page":"Method","title":"Method","text":"The algorithm considers a slightly transformed problem. By introducing two dummy variables tildex = x and tildes = s we can rewrite the original problem:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"beginarrayll\ntextminimize textstylefrac12tildex^top P tildex + q^top tildex + mathcalI_Ax+s=b(tildextildes) + mathcalI_mathcalK(s)\ntextsubject to (tildextildes) = (xs)\nendarray","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"where indicator functions mathcalI were used to move the constraints into the objective function. The resulting problem is now in the right format to apply the alternating direction method of multipliers (ADMM). To apply ADMM we first find the augmented Lagrangian L:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":" L(xstildextildeslambday) = textstylefrac12tildex^top Ptildex + q^top tildex + mathcalI_Ax+s=b(tildextildes) + mathcalI_mathcalK(s) + fracsigma2 tildex - x + textstylefrac1sigma λ _2^2 + fracrho2 tildes - s + textstylefrac1rho y _2^2","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"Minimizing the Lagrangian in an alternating fashion with respect to the two variable pairs (tildextildes) and (xs) yields the following algorithm steps:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"beginaligned\n ( tildex^k+1tildes^k+1) rarr undersettildextildestextargmin Lleft( tildextildesx^ks^ky^k right)\n x^k+1 larr tildex^k+1\n s^k+1 larr undersetstextargmin fracrho2 tildes^k+1 - s+textstylefrac1rhoy^k _2^2 + I_mathcalK(s) \n y^k+1 larr y^k + rho left( tildes^k+1 -s^k+1 right)\nendaligned","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"By the construction of the ADMM method those iterates are converging to the global solution. These steps are executed in a loop until convergence. Two important parameters are the ADMM steps sizes rho (Settings.rho) and sigma (Settings.sigma) which can be adjusted via the solver settings.","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"The two most important steps of the algorithm happen in the first and third line. The evaluation of the first line turns out to be an equality constrained quadratic program. We get a solution for ( tildex^k+1tildes^k+1) at every iteration by solving the following linear system:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"beginaligned\nbeginbmatrix\nP + sigma I A^top A - frac1rhoI\n endbmatrixbeginbmatrixtildex^k+1 nu^k+1endbmatrix= beginbmatrix-q+sigma x^k b-s^k+frac1rhoy^kendbmatrix\ntildes^k+1 = s^k - frac1rholeft(nu^k+1 + y^kright)\nendaligned","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"Fortunately, the left hand matrix doesn't change, which is why COSMO only has to factor the matrix once at the start of the algorithm.","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"The second important step in the algorithm is the update equation for s^k+1 which can be interpreted as a projection onto the constraint set mathcalK:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"s^k+1 = Pi_mathcalKleft( tildes^k+1 + frac1rhoy^kright)","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"The computational cost of this projection is highly dependent on the constraints of the problem. While projections onto the zero set or the nonnegative orthant are inexpensive, projections onto the positive semidefinite cone of order N involve an eigen-decomposition. Since methods for eigen-decompositions have a complexity of mathcalO(N^3) the projection can become the computationally most expensive operation of the algorithm.","category":"page"},{"location":"method/#Scaling","page":"Method","title":"Scaling","text":"","category":"section"},{"location":"method/","page":"Method","title":"Method","text":"The convergence of ADMM-based algorithms depends on the relative scaling of the problem data. Especially to improve the convergence of badly scaled problems, COSMO tries to rescale the data in a preprocessing step.","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"We rescale the equality constraints with diagonal positive semidefinite matrices D and E. The scaled problem is given by:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"beginarrayll\ntextminimize textstylefrac12 hatx^top hatP hatx + hatq^top hatx\ntextsubject to hatA hatx + hats = hatb \n hats in EmathcalK\nendarray","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"with scaled problem data","category":"page"},{"location":"method/","page":"Method","title":"Method","text":" hatP=DPD quad hatq=Dq quadhatA=EAD quad hatb=Eb","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"and the scaled convex cone EmathcalK = Ev in mathbbR^m mid v in mathcalK . After solving the scaled problem the original solution is obtained by reversing the scaling:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":" x = Dhatx quad s = E^-1hats quad y = Ehaty","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"To obtain the scaling matrices D and E we use a modified Ruiz equilibration algorithm which involves a certain number of scaling iterations to equilibrate the column norms of the data matrices P and A. The number of these iterations can be adjusted by the user with scaling. To disable the scaling step set scaling = 0.","category":"page"},{"location":"method/#Termination-criteria","page":"Method","title":"Termination criteria","text":"","category":"section"},{"location":"method/","page":"Method","title":"Method","text":"The COSMO algorithm can terminate for four reasons:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"The maximum number of allowed iterations has been reached. The user can specify this value in the solver settings with max_iter.\nThe solver runtime reaches the time limit specified by the user (time_limit).\nCOSMO detects an infeasible problem.\nThe iterates fulfil the termination criteria for convergence.","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"COSMO uses the primal and dual residuals of the problem to determine if the algorithm has converged. The primal and dual residuals are given by:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"beginaligned\nr_p = Ax + s -b\nr_d = Px + q + A^T y\nendaligned","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"The solver terminates when the infty-norms of the residuals lie below a specified tolerance. COSMO uses the sum of an absolute and relative tolerance term:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"beginaligned\n r_p^k _infty leq epsilon_mathrmabs + epsilon_mathrmrel textmax left Ax^k _inftys^k_infty b_infty right\n r_d^k_infty leq epsilon_mathrmabs + epsilon_mathrmrel textmax leftPx^k_inftyq_infty A^top y^k_infty right\nendaligned","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"The absolute and relative tolerances epsilon_mathrmabsand epsilon_mathrmrel can be set by the user by specifying eps_abs and eps_rel. Furthermore, the user can adjust the number of iterations after which the convergence criteria are checked (check_termination).","category":"page"},{"location":"method/#Infeasibility-detection","page":"Method","title":"Infeasibility detection","text":"","category":"section"},{"location":"method/","page":"Method","title":"Method","text":"COSMO uses conditions based on separating hyperplanes to detect infeasible problems. The conditions for COSMO's problem format have been developed in [1]. Define the convex set mathcalC = mathcal-K + b then we can use the following infeasibility conditions:","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"beginaligned\nmathcalP = leftx in mathbbR^n mid Px = 0 Ax in mathcalC^infty langle qx rangle 0 right \nmathcalD = lefty in mathbbR^m mid A^top y = 0 sigma_mathcalC(y) 0 right\nendaligned","category":"page"},{"location":"method/","page":"Method","title":"Method","text":"The existence of some y in mathcalD is a certificate that the problem is primal infeasible, while the existence of some x in mathcalP is a certificate for dual infeasibility. COSMO regularly checks above conditions to detect infeasible problems. If the detection is successful, the solver terminates and returns the status codes :Primal_infeasible or :Dual_infeasible. COSMO checks the conditions every check_infeasibility iterations, which can be adjusted by the user.","category":"page"},{"location":"method/#References","page":"Method","title":"References","text":"","category":"section"},{"location":"method/","page":"Method","title":"Method","text":"[1] Banjac, G. et al. Infeasibility detection in the alternating direction method of multipliers for convex optimization. Preprint, 2017.","category":"page"},{"location":"examples/#Examples","page":"Examples","title":"Examples","text":"","category":"section"},{"location":"examples/","page":"Examples","title":"Examples","text":"Some example problems that are solved with COSMO can be found in the /examples folder.","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"In the first example the native solver interface is used to define and solve a Linear Program. In the second example JuMP is used to describe the problem and COSMO is set as the solver backend.","category":"page"},{"location":"examples/#Linear-Program","page":"Examples","title":"Linear Program","text":"","category":"section"},{"location":"examples/","page":"Examples","title":"Examples","text":"We want to solve the following linear program with decision variable x:","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"beginarrayll textminimize c^top x\ntextsubject to A x leq b \n x geq 1 \n x_2 geq 5 \n x_1 + x_3 geq 4\nendarray","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"The problem can be solved with COSMO in the following way:","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"using COSMO, LinearAlgebra, SparseArrays, Test\n\nc = [1; 2; 3; 4.]\nA = Matrix(1.0I, 4, 4)\nb = [10; 10; 10; 10]\nn = 4\n# -------------------\n# create constraints A * x + b in set\n# -------------------\n# Ax <= b\nc1 = COSMO.Constraint(-A, b, COSMO.Nonnegatives)\n# x >= 1\nc2 = COSMO.Constraint(Matrix(1.0I, n, n), -ones(n), COSMO.Nonnegatives)\n# x2 >= 5\nc3 = COSMO.Constraint(1, -5, COSMO.Nonnegatives, n, 2:2)\n# x1 + x3 >= 4\nc4 = COSMO.Constraint([1 0 1 0], -4, COSMO.Nonnegatives)\n\n# -------------------\n# define cost function\n# -------------------\nP = spzeros(4, 4)\nq = c\n\n# -------------------\n# assemble solver model\n# -------------------\nsettings = COSMO.Settings(max_iter=2500, verbose=true, eps_abs = 1e-4, eps_rel = 1e-5)\nmodel = COSMO.Model()\nassemble!(model, P, q, [c1; c2; c3; c4], settings = settings)\nres = COSMO.optimize!(model);\n\n@testset \"Linear Problem\" begin\n @test isapprox(res.x[1:4], [3; 5; 1; 1], atol=1e-2, norm = (x -> norm(x, Inf)))\n @test isapprox(res.obj_val, 20.0, atol=1e-2)\nend","category":"page"},{"location":"examples/#Closest-Correlation-Matrix","page":"Examples","title":"Closest Correlation Matrix","text":"","category":"section"},{"location":"examples/","page":"Examples","title":"Examples","text":"We consider the problem of finding the closest correlation matrix X to a given random matrix C. With closest correlation matrix we mean a positive semidefinite matrix with ones on the diagonal. The problem is given by:","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"beginarrayll textminimize frac12X - C_F^2\ntextsubject to X_ii = 1 quad i=1dotsn \n X succeq 0\nendarray","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"Notice how JuMP is used to describe the problem. COSMO is chosen as the backend solver using JuMP's optimizer_with_attributes() function.","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"using COSMO, JuMP, LinearAlgebra, SparseArrays, Test, Random\nrng = Random.MersenneTwister(12345);\n\n# create a random test matrix C\nn = 8\nC = -1 .+ rand(rng, n, n) .* 2;\nc = vec(C);\n\n# define problem in JuMP\nq = -vec(C);\nr = 0.5 * vec(C)' * vec(C);\nm = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, \"verbose\" => true, \"eps_abs\" => 1e-4));\n@variable(m, X[1:n, 1:n], PSD);\nx = vec(X);\n@objective(m, Min, 0.5 * x' * x + q' * x + r)\nfor i = 1:n\n @constraint(m, X[i, i] == 1.)\nend\n\n# solve and get results\nstatus = JuMP.optimize!(m)\nobj_val = JuMP.objective_value(m)\nX_sol = JuMP.value.(X)","category":"page"},{"location":"examples/#Logistic-Regression","page":"Examples","title":"Logistic Regression","text":"","category":"section"},{"location":"examples/","page":"Examples","title":"Examples","text":"Logistic regression problems can be solved using exponential cone constraints. An example on how to use COSMO to solve a logistic regression problem is presented in /examples/logistic_regression_regularization.ipynb.","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"The source files for all examples can be found in /examples.","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"EditURL = \"https://github.com/oxfordcontrol/COSMO.jl/blob/master/examples/qp.jl\"","category":"page"},{"location":"examples/qp/#Quadratic-Program","page":"Quadratic Program","title":"Quadratic Program","text":"","category":"section"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"We want to solve the following quadratic program with decision variable x:","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"beginarrayll textminimize 12 x^top P x + q^top x \ntextsubject to l leq A x leq u\nendarray","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"The problem can be solved with COSMO in the following way. Start by defining the problem data","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"using COSMO, SparseArrays, LinearAlgebra, Test\n\nq = [1; 1.];\nP = sparse([4. 1; 1 2]);\nA = [1. 1; 1 0; 0 1];\nl = [1.; 0; 0];\nu = [1; 0.7; 0.7];\nnothing #hide","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"First, we decide to solve the problem with two one-sided constraints using COSMO.Nonnegatives as the convex set:","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"Aa = [-A; A]\nba = [u; -l]\nconstraint1 = COSMO.Constraint(Aa, ba, COSMO.Nonnegatives);\nnothing #hide","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"Next, we define the settings object, the model and then assemble everything:","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"settings = COSMO.Settings(verbose=true);\nmodel = COSMO.Model();\nassemble!(model, P, q, constraint1, settings = settings);\nres = COSMO.optimize!(model);\nnothing #hide","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"Alternatively, we can also use two-sided constraints with COSMO.Box:","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"constraint1 = COSMO.Constraint(A, zeros(3), COSMO.Box(l, u));\n\nmodel = COSMO.Model();\nassemble!(model, P, q, constraint1, settings = settings);\nres_box = COSMO.optimize!(model);\nnothing #hide","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"Let's check that the solution is correct:","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"@testset \"QP Problem\" begin\n @test norm(res.x[1:2] - [0.3; 0.7], Inf) < 1e-3\n @test norm(res_box.x[1:2] - [0.3; 0.7], Inf) < 1e-3\n @test abs(res.obj_val - 1.88) < 1e-3\n @test abs(res_box.obj_val - 1.88) < 1e-3\nend\nnothing","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"","category":"page"},{"location":"examples/qp/","page":"Quadratic Program","title":"Quadratic Program","text":"This page was generated using Literate.jl.","category":"page"},{"location":"lin_solver/#Linear-System-Solver","page":"Linear System Solver","title":"Linear System Solver","text":"","category":"section"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"One major step of COSMO's ADMM algorithm is solving a linear system of equations at each iteration. Fortunately, the left-hand matrix is only dependent on the problem data and therefore only needs to be factored once. Depending on the problem class this factorisation can be the computationally most expensive step of the algorithm (LPs, QPs). See the Method section for a more detailed description of the linear system.","category":"page"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"COSMO allows you to specify the linear system solver that performs the factorisation and back-substitution. We also support indirect system solver which are useful for very large problems where a factorisation becomes inefficient. The table below shows the currently supported linear system solver:","category":"page"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"Type Solver Description\nCholmodKKTSolver Cholmod Julia's default linear system solver (from SuiteSparse)\nQdldlKKTSolver QDLDL For more information QDLDL.jl\nPardisoDirectKKTSolver Pardiso (direct) Pardiso 6.0 direct solver\nPardisoIndirectKKTSolver Pardiso (indirect) Pardiso 6.0 indirect solver\nMKLPardisoKKTSolver Intel MKL Pardiso Pardiso optimised for Intel platforms\nCGIndirectKKTSolver IterativeSolvers.jl Conjugate Gradients on the reduced KKT linear system.\nMINRESIndirectKKTSolver IterativeSolvers.jl MINRES on the (full) KKT linear system.","category":"page"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"note: Note\nTo use the Pardiso and Intel MKL Pardiso solver, you have to install the respective libraries and the corresponding Julia wrapper. For more information about installing these, visit the Pardiso.jl repository page. Likewise in order to use Indirect(Reduced)KKTSolver you have to install IterativeSolvers.jl (v0.9+) and LinearMaps.jl. We are using the Requires package for lazy loading of code related to Pardiso and IterativeSolvers. This means in order to use Pardiso / IterativeSolvers, you'll have to load these packages alongside COSMO, i.e. using Pardiso and using IterativeSolvers, LinearMaps.","category":"page"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"COSMO uses the Cholmod linear system solver by default. You can specify a different solver in the settings by using the kkt_solver keyword and the respective type:","category":"page"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"settings = COSMO.Settings(kkt_solver = CholmodKKTSolver)\n","category":"page"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"COSMO also allows you to pass in solver-specific options with the with_options(solver_type, args...; kwargs...) syntax. For example, if you want to use Pardiso with verbose printing use the following syntax:","category":"page"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"settings = COSMO.Settings(kkt_solver = with_options(PardisoDirectKKTSolver, msg_level_on = true))","category":"page"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"Likewise, CGIndirectKKTSolver and MINRESIndirectKKTSolver are also parameterizable with with_options(solver_type, args...; kwargs...) and accept the following arguments:","category":"page"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"Argument Description Values (default)\ntol_constant::T and tol_exponent::T Parameter that defines the solution tolerance for the iterative solvers across iterations. In particular, the solution tolerance at every iteration is defined as texttol_constant textiteration^texttol_exponent 1.0, 1.5","category":"page"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"This also works if you want to use this configuration with JuMP:","category":"page"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"model = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, \"kkt_solver\" => with_options(PardisoDirectKKTSolver, msg_level_on = true));\n","category":"page"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"Or alternatively:","category":"page"},{"location":"lin_solver/","page":"Linear System Solver","title":"Linear System Solver","text":"model = JuMP.Model(COSMO.Optimizer);\nset_optimizer_attribute(model, \"kkt_solver\", with_options(PardisoDirectKKTSolver, msg_level_on = true));","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"The source files for all examples can be found in /examples.","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"EditURL = \"https://github.com/oxfordcontrol/COSMO.jl/blob/master/examples/closest_correlation_matrix.jl\"","category":"page"},{"location":"examples/closest_correlation_matrix/#Closest-Correlation-Matrix","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"","category":"section"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"We consider the problem of finding the closest correlation matrix X to a given random matrix C. With closest correlation matrix we mean a positive semidefinite matrix with ones on the diagonal. The problem is given by:","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"beginarrayll textminimize frac12X - C_F^2\ntextsubject to X_ii = 1 quad i=1dotsn \n X succeq 0\nendarray","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"Notice that we use JuMP to model the problem. COSMO is chosen as the backend solver. And COSMO-specific settings are passed using the optimizer_with_attributes() function.","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"using COSMO, JuMP, LinearAlgebra, SparseArrays, Test, Random","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"rng = Random.MersenneTwister(12345);\n# create a random test matrix C\nn = 8;\nC = -1 .+ rand(rng, n, n) .* 2;\nc = vec(C);\nnothing #hide","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"Define problem in JuMP:","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"q = -vec(C);\nr = 0.5 * vec(C)' * vec(C);\nm = JuMP.Model(optimizer_with_attributes(COSMO.Optimizer, \"verbose\" => true));\n@variable(m, X[1:n, 1:n], PSD);\nx = vec(X);\n@objective(m, Min, 0.5 * x' * x + q' * x + r);\nfor i = 1:n\n @constraint(m, X[i, i] == 1.);\nend","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"Solve the JuMP model with COSMO and query the solution X_sol:","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"status = JuMP.optimize!(m);\nobj_val = JuMP.objective_value(m);\nX_sol = JuMP.value.(X);\nnothing #hide","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"Double check result against known solution:","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"known_opt_val = 12.5406\nknown_solution = [\n 1.0 0.732562 -0.319491 -0.359985 -0.287543 -0.15578 0.0264044 -0.271438;\n 0.732562 1.0 0.0913246 -0.0386357 0.299199 -0.122733 0.126612 -0.187489;\n -0.319491 0.0913246 1.0 -0.0863377 0.432948 0.461783 -0.248641 -0.395299;\n -0.359985 -0.0386357 -0.0863377 1.0 0.503379 0.250601 0.141151 0.286088;\n -0.287543 0.299199 0.432948 0.503379 1.0 -0.0875199 0.137518 0.0262425;\n -0.15578 -0.122733 0.461783 0.250601 -0.0875199 1.0 -0.731556 0.0841783;\n 0.0264044 0.126612 -0.248641 0.141151 0.137518 -0.731556 1.0 -0.436274;\n -0.271438 -0.187489 -0.395299 0.286088 0.0262425 0.0841783 -0.436274 1.0 ];\n@test isapprox(obj_val, known_opt_val , atol=1e-3)","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"@test norm(X_sol - known_solution, Inf) < 1e-3","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"","category":"page"},{"location":"examples/closest_correlation_matrix/","page":"Closest Correlation Matrix","title":"Closest Correlation Matrix","text":"This page was generated using Literate.jl.","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"The source files for all examples can be found in /examples.","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"EditURL = \"https://github.com/oxfordcontrol/COSMO.jl/blob/master/examples/lovasz_petersen.jl\"","category":"page"},{"location":"examples/lovasz_petersen/#Lovasz-Theta-Function","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"","category":"section"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"The Lovász theta function is an important concept in graph theory. Consider an undirected graph G = (VE) with vertex set V = 1dotsn and edge set E with E_ij = 1 if there exists an edge between the vertices i and j. A stable set (or independent set) is a subset of V such that the induced subgraph does not contain edges. The stability number alpha(G) of the graph is equal to the cardinality of the largest stable set. It is closely related to the Shannon capacity Theta(G) that models the amount of information that a noisy communication channel can carry if certain signal values can be confused with each other. Unfortunately, the determination of alpha(G) is an NP-hard problem and the computational complexity to determine Theta(G) remains unknown. The Lovász theta function vartheta(G) was introduced in [1], can be computed in polynomial time, and represents an upper bound on both the stability number and the Shannon capacity:","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"alpha(G) leq Theta(G) leq vartheta(G)","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"The value of vartheta(G) can be determined by computing the optimal value p^* of the following SDP:","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"beginarrayll textmaximize textTr(JX)\ntextsubject to textTr(X) = 1 \n X_ij = 0 quad (i j) in E \n X succeq 0\nendarray","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"with matrix variable X, matrix J in mathbfR^n times n of all ones and textTr() denoting the matrix trace. In this simple example we will compute the value of the Lovász theta function for the Petersen Graph (which is known to be 4).","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"using LinearAlgebra, COSMO, JuMP, Plots, GraphRecipes\n\n# let's define the Petersen graph\nn = 10\nE = zeros(n, n)\nE[1, 2] = E[1, 5] = E[1, 6] = 1.\nE[2, 3] = E[2, 7] = 1.\nE[3, 4] = E[3, 8] = 1.\nE[4, 5] = E[4, 9] = 1.\nE[5, 10] = 1.\nE[6, 8] = E[6, 9] = 1.\nE[7, 9] = E[7, 10] = 1.\nE[8, 10] = 1.\n\n# plot the graph\nri = 1.\nro = 2.\ncoordinates = []\nfor θ = 90:-72:-198\n push!(coordinates, [ro * cosd(θ), ro * sind(θ)])\nend\nfor θ = 90:-72:-198\n push!(coordinates, [ri * cosd(θ), ri * sind(θ)])\nend\ngraphplot(E, names = 1:n, x = getindex.(coordinates, 1), y = getindex.(coordinates, 2), fontsize = 10, nodesize = 1, nodeshape =:circle, curvature = 0.)","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"Let's solve the SDP with COSMO and JuMP:","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"model = JuMP.Model(COSMO.Optimizer);\n\n@variable(model, X[1:n, 1:n], PSD)\nx = vec(X)\n@objective(model, Max, sum(x))\n@constraint(model, tr(X)== 1.)\nfor j = 1:n\n for i = 1:j-1\n if E[i, j] == 1.\n @constraint(model, X[i, j] == 0)\n end\n end\nend\nstatus = JuMP.optimize!(model)","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"The optimal objective is given by:","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"JuMP.objective_value(model)","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"Which is the correct known value for the Petersen Graph.","category":"page"},{"location":"examples/lovasz_petersen/#References","page":"Lovász Theta Function","title":"References","text":"","category":"section"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"[1] Lovász - On the Shannon Capacity of a Graph, IEEE Transactions on Information Theory (1979)","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"","category":"page"},{"location":"examples/lovasz_petersen/","page":"Lovász Theta Function","title":"Lovász Theta Function","text":"This page was generated using Literate.jl.","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"The source files for all examples can be found in /examples.","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"EditURL = \"https://github.com/oxfordcontrol/COSMO.jl/blob/master/examples/two_way_partitioning.jl\"","category":"page"},{"location":"examples/two_way_partitioning/#Relaxed-Two-Way-Partitioning-Problem","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"","category":"section"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"We consider the (nonconvex) two-way partitioning problem from Boyd and Vandenberghe (2004), p.219 [1]:","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"beginarrayll textminimize x^top W x \ntextsubject to x_i^2 = 1 quad i=1dots n\nendarray","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"with x in mathbfR^n and W in mathbfS^n. The problem can be interpreted as finding a partition of the points i = 1dots n into two sets, where the cost of two points in the same set is W_ij and -W_ij otherwise. Brute forcing the solution x^* takes 2^n attempts and becomes quickly intractable, e.g. for n geq 30. However, a lower-bound for this problem can be computed by solving the convex dual of the problem:","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"beginarrayll textmaximize -sum_i nu \ntextsubject to W + textdiag(nu) in mathbfS_+^n\nendarray","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"Solving this problem with optimal objective d^* yields a lower bound as least as good as a lower bound based on the minimum eigenvalue d^* geq lambda_textmin(W). Let's set up the problem for n = 20 and with a specially structured (banded) W:","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"using LinearAlgebra, Random, COSMO, JuMP, IterTools\n\nrng = Random.MersenneTwister(212313);\nn = 20;\n\nW = diagm(0 => randn(rng, n));\nW += diagm(1 => randn(rng, n-1));\nW = Symmetric(W)","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"As you can see, the matrix W imposes a structure on the LMI-constraint. Accordingly, COSMO will be able to use chordal decomposition to decompose the LMI constraint into multiple smaller constraints. This can make a significant difference in the performance of the algorithm for large n.","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"model = JuMP.Model(COSMO.Optimizer);\n@variable(model, ν[1:n]);\n@objective(model, Max, -ones(n)' * ν )\n@constraint(model, Symmetric(W + diagm(ν) ) in JuMP.PSDCone());\nJuMP.optimize!(model)","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"Looking at the solver output you can see how the PSD constraint was decomposed into 19 PSD constraints. Let's look at the lower bound:","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"JuMP.objective_value(model)","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"As n is small, we can verify our result by finding the optimal solution by trying out all possible combinations:","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"function brute_force_optimisation(W, n)\n opt_obj = Inf\n opt_x = Inf * ones(n)\n\n for xt in Iterators.product([[1.; -1.] for k = 1:n]...)\n x = [i for i in xt]\n obj_val = x' * W * x\n if obj_val < opt_obj\n opt_obj = obj_val\n opt_x = x\n end\n end\n return opt_obj, opt_x\nend\nopt_obj, opt_x = brute_force_optimisation(W, n)","category":"page"},{"location":"examples/two_way_partitioning/#References","page":"Relaxed Two-Way Partitioning Problem","title":"References","text":"","category":"section"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"[1] Boyd and Vandenberghe - Convex Optimization, Cambridge University Press (2004)","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"","category":"page"},{"location":"examples/two_way_partitioning/","page":"Relaxed Two-Way Partitioning Problem","title":"Relaxed Two-Way Partitioning Problem","text":"This page was generated using Literate.jl.","category":"page"},{"location":"","page":"Home","title":"Home","text":"COSMO.jl is a Julia implementation of the Conic Operator Splitting Method. The underlying ADMM-algorithm is well-suited for large convex conic problems. COSMO solves the following problem:","category":"page"},{"location":"","page":"Home","title":"Home","text":"beginarrayll textminimize textstylefrac12x^top Px + q^top x textsubject to Ax + s = b s in mathcalK\nendarray","category":"page"},{"location":"","page":"Home","title":"Home","text":"with decision variables x in mathbbR^n, s in mathbbR^m and data matrices P=P^top succeq 0, q in mathbbR^n, A in mathbbR^m times n, and b in mathbbR^m. The convex set mathcalK is a composition of convex sets and cones.","category":"page"},{"location":"#Features","page":"Home","title":"Features","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Versatile: COSMO solves linear programs, quadratic programs, second-order cone programs, semidefinite programs and problems involving exponential and power cones\nQuad SDPs: Positive semidefinite programs with quadratic objective functions are natively supported\nSafeguarded acceleration: robust and faster convergence to higher precision using COSMOAccelerators\nInfeasibility detection: Infeasible problems are detected without a homogeneous self-dual embedding of the problem\nJuMP / Convex.jl support: We provide an interface to MathOptInterface (MOI), which allows you to describe your problem in JuMP and Convex.jl.\nChordal decomposition: COSMO tries to decompose large structured PSD constraints into multiple smaller PSD constraints using chordal decomposition techniques. This often results in a significant speedup compared to solving the original problem.\nSmart clique merging: After an initial decomposition of a structured SDP, COSMO recombines overlapping cliques/blocks to speed up the algorithm.\nWarm starting: COSMO supports warm starting of the decision variables\nArbitrary precision types: You can solve problems with any floating point precision.\nOpen Source: Our code is available on GitHub and distributed under the Apache 2.0 Licence","category":"page"},{"location":"#Installation","page":"Home","title":"Installation","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"COSMO can be installed using the Julia package manager for Julia v1.0 and higher. Inside the Julia REPL, type ] to enter the Pkg REPL mode then run","category":"page"},{"location":"","page":"Home","title":"Home","text":"pkg> add COSMO","category":"page"},{"location":"","page":"Home","title":"Home","text":"If you want to install the latest version from master run","category":"page"},{"location":"","page":"Home","title":"Home","text":"pkg> add COSMO#master","category":"page"},{"location":"#Quick-Example","page":"Home","title":"Quick Example","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Consider the following 2x2 semidefinite program with decision variable X:","category":"page"},{"location":"","page":"Home","title":"Home","text":"beginarrayll textminimize texttr(CX)\ntextsubject to texttr(A X) = b \n X succeq 0\nendarray","category":"page"},{"location":"","page":"Home","title":"Home","text":"with problem data A, b and C:","category":"page"},{"location":"","page":"Home","title":"Home","text":"A = beginbmatrix 1 5 5 2endbmatrix\nC = beginbmatrix 1 2 2 2endbmatrix\nb = 4","category":"page"},{"location":"","page":"Home","title":"Home","text":"where tr denotes the trace of a matrix. We can solve this problem either using COSMO's interface:","category":"page"},{"location":"","page":"Home","title":"Home","text":"using COSMO, LinearAlgebra\n\nC = [1. 2; 2 2]\nA = [1. 5; 5 2]\nb = 4.0;\n\nmodel = COSMO.Model();\n\n# define the cost function\nP = zeros(4, 4)\nq = vec(C)\n\n# define the constraints\n# A x = b\ncs1 = COSMO.Constraint(vec(A)', -b, COSMO.ZeroSet)\n# X in PSD cone\ncs2 = COSMO.Constraint(Matrix(1.0I, 4, 4), zeros(4), COSMO.PsdCone)\nconstraints = [cs1; cs2]\n\n# assemble and solve the model\nassemble!(model, P, q, constraints)\nresult = COSMO.optimize!(model);\n\nX_sol = reshape(result.x, 2, 2)\nobj_value = result.obj_val","category":"page"},{"location":"","page":"Home","title":"Home","text":"or we can describe the problem using JuMP and use COSMO as the backend solver:","category":"page"},{"location":"","page":"Home","title":"Home","text":"using COSMO, JuMP, LinearAlgebra\n\nC = [1. 2; 2 2]\nA = [1. 5; 5 2]\nb = 4.0;\nm = JuMP.Model(COSMO.Optimizer);\n@variable(m, X[1:2, 1:2], PSD)\n@objective(m, Min, tr(C * X));\n@constraint(m, tr(A * X) == b);\nJuMP.optimize!(m);\n\nstatus = JuMP.termination_status(m)\nX_sol = JuMP.value.(X)\nobj_value = JuMP.objective_value(m)","category":"page"},{"location":"#Related-talks","page":"Home","title":"Related talks","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"A video of my first presentation of COSMO at JuMP-dev (Santiago) is available here: COSMO.jl at JuMP-dev\nA more detailed presentation on chordal decomposition and clique merging is available here: Chordal decomposition and clique merging\nI gave an overview on recent features of COSMO such as acceleration methods at JuliaCon 2021: What's new in COSMO?\nMore details on the acceleration methods were presented in my ECC 2022 talk Safeguarded Anderson acceleration for parametric nonexpansive operators","category":"page"},{"location":"#Credits","page":"Home","title":"Credits","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"The following people are involved in the development of COSMO:","category":"page"},{"location":"","page":"Home","title":"Home","text":"Michael Garstka (main development)\nNikitas Rontsis (algorithm performance)\nPaul Goulart (code architecture, maths and algorithms)\nMark Cannon (maths and algorithms)","category":"page"},{"location":"","page":"Home","title":"Home","text":"*all contributors are affiliated with the University of Oxford.","category":"page"},{"location":"","page":"Home","title":"Home","text":"If this project is useful for your work please consider","category":"page"},{"location":"","page":"Home","title":"Home","text":"Citing the relevant papers\nLeaving a star on the GitHub repository","category":"page"},{"location":"#Licence","page":"Home","title":"Licence","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"COSMO.jl is licensed under the Apache License 2.0. For more details click here.","category":"page"},{"location":"citing/#Citing-COSMO","page":"Citing COSMO","title":"Citing COSMO","text":"","category":"section"},{"location":"citing/","page":"Citing COSMO","title":"Citing COSMO","text":"If you find COSMO useful in your project, we kindly request that you cite the following paper:","category":"page"},{"location":"citing/","page":"Citing COSMO","title":"Citing COSMO","text":"@Article{Garstka_2021,\n author = {Michael Garstka and Mark Cannon and Paul Goulart},\n journal = {Journal of Optimization Theory and Applications},\n title = {{COSMO}: A Conic Operator Splitting Method for Convex Conic Problems},\n volume = {190},\n number = {3},\n pages = {779--810},\n year = {2021},\n publisher = {Springer},\n doi = {10.1007/s10957-021-01896-x},\n url = {https://doi.org/10.1007/s10957-021-01896-x}\n}","category":"page"},{"location":"citing/","page":"Citing COSMO","title":"Citing COSMO","text":"The article is available under Open Access here.","category":"page"},{"location":"citing/","page":"Citing COSMO","title":"Citing COSMO","text":"The following paper describes the algorithm used for our clique graph based merging strategy:","category":"page"},{"location":"citing/","page":"Citing COSMO","title":"Citing COSMO","text":"@InProceedings{Garstka_2020,\n author = {Michael Garstka and Mark Cannon and Paul Goulart},\n title = {A clique graph based merging strategy for decomposable {SDPs}},\n year = {2020},\n note = {21th IFAC World Congress},\n number = {2},\n pages = {7355-7361},\n volume = {53},\n doi = {10.1016/j.ifacol.2020.12.1255},\n issn = {2405-8963},\n journal = {IFAC-PapersOnLine},\n}","category":"page"},{"location":"citing/","page":"Citing COSMO","title":"Citing COSMO","text":"A preprint can be downloaded here.","category":"page"},{"location":"citing/","page":"Citing COSMO","title":"Citing COSMO","text":"The acceleration strategy is described in this paper:","category":"page"},{"location":"citing/","page":"Citing COSMO","title":"Citing COSMO","text":"@InProceedings{Garstka_2022,\n title={Safeguarded Anderson acceleration for parametric nonexpansive operators},\n author={Garstka, Michael and Cannon, Mark and Goulart, Paul},\n booktitle={2022 European Control Conference (ECC)},\n pages={435--440},\n year={2022},\n organization={IEEE}\n}","category":"page"},{"location":"citing/","page":"Citing COSMO","title":"Citing COSMO","text":"A preprint can be downloaded here.","category":"page"}]
+}
diff --git a/v0.8.8/siteinfo.js b/v0.8.8/siteinfo.js
new file mode 100644
index 00000000..5b6546bf
--- /dev/null
+++ b/v0.8.8/siteinfo.js
@@ -0,0 +1 @@
+var DOCUMENTER_CURRENT_VERSION = "v0.8.8";
diff --git a/versions.js b/versions.js
index a44b2b2a..137b1126 100644
--- a/versions.js
+++ b/versions.js
@@ -7,5 +7,5 @@ var DOC_VERSIONS = [
"v0.4",
"dev",
];
-var DOCUMENTER_NEWEST = "v0.8.7";
+var DOCUMENTER_NEWEST = "v0.8.8";
var DOCUMENTER_STABLE = "stable";