Python API does not allow using keywords to pass arguments, making model building code harder to read

[severinh]

Context

Out team is about to rewrite our large MILP problem using HiGHS, in particular its Matrix API.
One of our goals is to make the mode building code as readable as possible to increase our team's velocity.

Pain point

One significant issue with developer ergonomics we've identified with the HiGHS API is the following: Its Python API only allows arguments to be passed as positional arguments, not keyword arguments.

Consider the following simple model building code:

import highspy
import numpy as np

h = highspy.Highs()

vars = h.addBinaries(3)

h.addRow(-highspy.kHighsInf, 4, 3, np.array([0, 1, 2]), np.array([1, 2, 3]))
h.addRow(1, highspy.kHighsInf, 2, np.array([0, 1]), np.array([1, 1]))

h.changeColsCost(3, vars, np.array([1, 1, 2]))
result = h.maximize()

For someone who doesn't write HiGHS code regularly, it's

• not clear meaning of the many parameters is. What is 4? What is np.array([1, 2, 3])?
• easy to make a mistake and pass arguments in the wrong order.

Desired change

I think the code would be a lot easier to understand for developers if they were allowed to pass arguments by keyword too (if they prefer to), not just positionally.

import highspy
import numpy as np

h = highspy.Highs()

vars = h.addBinaries(3)

h.addRow(
    lower_bound=-highspy.kHighsInf,
    upper_bound=4,
    num_elements=3,
    indices=np.array([0, 1, 2]),
    values=np.array([1, 2, 3]),
)
h.addRow(
    lower_bound=1,
    upper_bound=highspy.kHighsInf,
    num_elements=2,
    indices=np.array([0, 1]),
    values=np.array([1, 1]),
)

h.changeColsCost(num_cols=3, cols=vars, costs=np.array([1, 1, 2]))
result = h.maximize()

Surprisingly, this is currently not allowed by HiGHS. When running the above code, I get the following traceback:

Traceback (most recent call last):
  File "/Users/severinh/Documents/archlet-app/python/bid_data/highs_matrix.py", line 8, in <module>
    h.addRow(
    ~~~~~~~~^
        lower_bound=-highspy.kHighsInf,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<3 lines>...
        values=np.array([1, 2, 3]),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
TypeError: addRow(): incompatible function arguments. The following argument types are supported:
    1. (self: highspy._core._Highs, arg0: float, arg1: float, arg2: int, arg3: numpy.ndarray[numpy.int32], arg4: numpy.ndarray[numpy.float64]) -> highspy._core.HighsStatus

Invoked with: <highspy.highs.Highs object at 0x100b3b6b0>; kwargs: lower_bound=-inf, upper_bound=4, num_elements=3, indices=array([0, 1, 2]), values=array([1, 2, 3])

That is surprising. When using HiGHS for the first time I lost an hour or more to figure out that this was not because there was something wrong with the values I passed. I've never seen such a behavior in any Python API, which disallowed passing arguments by keyword. There seems to be some extra checking that HiGHS does here.

As a work-around, we can of course introduce our own wrapper methods - but I feel strongly that this should not be necessary.

What do you think?

And above everything else, THANK YOU for building such an amazing Open Source solver.

[jajhall]

I see what you mean if you look at highspy from a Python programmer's point of view. However highspy is only a wrapper around the C++ API, and in many cases there's no intermediate code.

We wrote highspy as a means by which Python users could get access to the C++ API, but clearly it's become a heavily used interface to HiGHS.

If what you're wanting were done for the whole of highspy, it would require a lot of work, for which we don't have the resources.

I guess we could introduce methods for addCol and addRow, though.

[lesshaste]

Is the scipy interface a solution for this?

[jajhall]

Is the scipy interface a solution for this?

I don't think that has any model building calls like addCol and addRow. For HiGHS, it only allows cold-started runs for a given LP or MIP, rather than

[severinh]

Thanks for taking the time to give context, @jajhall. I see that the unexpected behavior was not a conscious decision on the part of the HiGHS team - just a consequence of the direct API mapping with pybind.

I agree that investing into a nicer Python abstraction layer and APIs is not cheap. And I would argue that the time of the core team is better spent focusing on improving the core solver itself, since it's something few people are able to.

As our team starts working more and more with the HiGHS Python API in the coming months, we might be able to propose some PRs for the Python interface for you to review.

[nickmasonsmith]

Is the cvxpy interface a solution for this?