Remove qpOASES from the Dynamics component and the superbuild?

[traversaro]

The qpOASES library served us well for years, but it is a bit complicated to maintain for these reasons:

• The upstream build system is not working properly, especially for what it concerns buildings bindings, and so we need to mantain a custom qpOASES fork at https://github.com/robotology-dependencies/qpOASES
• There are some critical bugs related to qpOASES marking as unfeasible problems that are feasible (see Possible bug in qpOASES when using equality constraints robotology-dependencies/qpOASES#4). We do not have the expertise to fix the problem ourself, and we have reported the problem upstream years ago, but no one is currently working on it.

The OSQP QP solver recently emerged and is a well maintained and with an active upstream library, that does not have the problem of the build systems and of bugs of qpOASES. OSQP is already part of the superbuild. Ideally, we should be able to just use OSQP for all our QP needs, removing the need of the qpOASES and reducing the compilation time of the superbuild and our maintenance burden.

Currently this is blocked by the following issues:

• Add OSQP block to deprecate the QpOases block  wb-toolbox#180

[traversaro]

If anyone thinks that for any reason qpOASES should be maintained inside the superbuild, feel free to comment, thanks!

[traversaro]

cc @DanielePucci

[traversaro]

It turns out that sometimes OSQP "generates a solution that is outside the boundary. I've noticed this behaviour when the problem is ill conditioned", see robotology-dependencies/qpOASES#4 (comment) . To be completely honest, if we are not able to reproduce the error I would not consider this as an issue blocking us from completly migrate to OSQP, at least in the default builds of the superbuild.

[arocchi]

I had the same experience with OSQP, but to be honest I did not try again after the report here osqp/OSQP.jl#47
My problem could have even been different altogether, as I remember the issue was that because of the relatively large eps_rel, the algorithm would converge with huge residuals.

[traversaro]

Thanks for chiming in and for the pointers @arocchi !
@GiulioRomualdi @MiladShafiee If the problem you experienced is indeed related to osqp/osqp#151 , possibly disabling adaptive_rho (if I understood correctly) should avoid the problems, but I am not sure about the other effects of it (increase in the computational time?)

[traversaro]

Other reports of problems on which OSQP fails: tesseract-robotics/trajopt#146 (comment) .

[traversaro]

@robotology/iit-dic As it seems that we are not able to gain consensus of the usefulness of moving away from qpOASES, to reduce the maintenance burden it would be at least useful to remove the compilation of the qpOASES Matlab bindings from the ROBOLOTY_USES_MATLAB option.
The only public use of matlab bindings of qpOASES is in a deprecated package, but I noticed a few use in private repos of @Giulero @isorrentino @gabrielenava @lrapetti .

If we want to keep it around, I am afraid I need someone to take responsibility of the repo https://github.com/robotology-dependencies/qpOASES as a maintainer. cc @DanielePucci

[lrapetti]

The only public use of matlab bindings of qpOASES is in a deprecated package, but I noticed a few use in private repos of @Giulero @isorrentino @gabrielenava @lrapetti .

If we want to keep it around, I am afraid I need someone to take responsibility of the repo https://github.com/robotology-dependencies/qpOASES as a maintainer.

Just as a side note, I think that robotology/wb-toolbox#180 would deprecate the usage for all the four of us (and might be somehow less expensive at the end that maintaining robotology-dependencies/qpOASES)

[traversaro]

Just as a side note, I think that robotology/wb-toolbox#180 would deprecate the usage for all the four of us (and might be somehow less expensive at the end that maintaining robotology-dependencies/qpOASES)

Great!

[traversaro]

Actually the usage of qpOASES matlab bindings that I found was actually due to @Yeshasvitvs, but I think they are just old controllers.

[gabrielenava]

From my side:

• whole-body-controllers uses the QPOASES block block from WBToolbox, so no issue for this repo;

• controllers inside robotology-legacy are deprecated and already broken for other reasons so no issue also from here;

• mex-wholebodymodel does depend on QPOASES matlab bindings, but the repo should go in legacy: see Move this repo to Robotology Legacy? robotology-legacy/mex-wholebodymodel#91;

• Inside the DIC organization: the jerk-control does not use any QP. So there is only one controller that is still using QPOASES matlab bindings: iRonCub controller. But I see no limitations for moving to the usage of QPOASES block inside this controller.

We are going to handle this modification (we = @gabrielenava @Giulero @bemilio) so that all the (still used) above mentioned controllers are exposed to QPOASES only through WBTollbox, and they will comply to any future decision concerning QPOASES.

[traversaro]

Thanks @gabrielenava !

[fiorisi]

cc @RAR1989

[isorrentino]

I use qpOASES Matlab bindings for the insoles calibration, but I'll change the solver as soon as possible. From my side, there aren't problems with removing it.

[traversaro]

Periodic remainder that qpOASES is a source of problems, see for example #418 .

[traversaro]

Given the recurring problems related to qpOASES and the need to maintain its CMake build system, unless someone volunteers to maintain qpOASES my plan is to remove qpOASES from the superbuild by the end of 2020.

cc @robotology/iit-dynamic-interaction-control

[diegoferigo]

From the WB-Toolbox side I think there're won't be many problems. If qpOASES is not found, the code of the block is not compiled (but the block shows up in the library, though it will not work).

[traversaro]

From the WB-Toolbox side I think there're won't be many problems. If qpOASES is not found, the code of the block is not compiled (but the block shows up in the library, though it will not work).

I hope someone will migrate the WB-Toolbox block to osqp/OsqpEigen by then. : )

[diegoferigo]

I hope so too :)

[nunoguedelha]

Since I will be working on robotology/blockfactory#59, I might have some time to fit this migration in my tasks...

[traversaro]

As qpOASES is still used in WB-Toolbox, I think a good compromise to reduce the maintenance burden is to disable compilation of MATLAB bindings, that are the part that use a custom fork and the one that gave most of the problems. Furthemore, MATLAB bindings for a state of the art QP solver will be added in #585, so anyone interested can use those. Unless someone objects on this, I would proceed with this change in 7 days.

[traversaro]

To clarify, this will not change the fact that qpOASES has well known shortcomings (i.e. feasible QP problems that are classified as unfeasible) described in coin-or/qpOASES#85 that could kick in at any moment (even when controlling robot in real world experiments) and for that working on robotology/wb-toolbox#180 could make sense.

[diegoferigo]

After 1.5 years from the publishing of this issue few things have changed in the lab, I noticed that the usage of CasADI significantly increased. Thanks to @GiulioRomualdi, after #526 the superbuild can optionally compile CasADI from sources, and by default it enables the IPOPT and (updated) OSQP backends.

Having as many available backend as possible is imho a good idea, regardless of the limitations of qpOASES. What's important is being aware of them. This being said, if the maintenance burden comes from the need of MATLAB bindings and if without them we can switch back to using upstream, I would like to discuss about keeping qpOASES inside the superbuild.

I recently found useful this version of CasADI to get out-of-the-box a complete Python package, that is similar to the one published in PyPI but built from sources against the system's libraries. Unfortunately also CasADI (as many other C++ hybrid packages) suffers from incompatibilities when loading multiple C++ modules casadi/casadi#1012, and a source compilation often helps.

[traversaro]

Yes, I have no problem in having qpOASES in the superbuild if we can use the upstream repo (and dropping MATLAB bindings is the first step in that direction) and as long users are aware that may not a good idea to use it by default.

[traversaro]

On a related note, if someone wants to enable qpOASES in the CasADi build of the superbuild, that's a good idea as well.

[diegoferigo]

On a related note, if someone wants to enable qpOASES in the CasADi build of the superbuild, that's a good idea as well.

Yep I agree, we should pass -DWITH_LAPACK=ON WITH_MUMPS=ON WITH_QPOASES=ON.

Then, since we're there, by passing WITH_PYTHON=ON WITH_PYTHON3=ON we can enable python support. There are two caveats:

1. The module could not be imported. I had to apply the following patch: sed -i "s/class _copyableObject(_object):/class _copyableObject():/g" /usr/local/src/robotology-superbuild/build/install/python/casadi/casadi.py. I guess it is a Python2 related problem.
2. The Python bindings are installed into <build>/install/python.

Maybe this is not the right place where to write this info, but for the moment let's log it here.

[traversaro]

In coin-or/qpOASES#108 and coin-or/qpOASES#109 I contributed upstream the modification of qpOASES that we need, and after that we can switch to use qpOASES upstream (just waiting on a proper release, see coin-or/qpOASES#113). In any case, given that at this point we do not have anymore mantainance overhead coming from qpOASES, I think we can close the issue.