Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend valid types for PyROS solver argument uncertain_params #3439

Open
wants to merge 34 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
31ae3ff
Extend `InputDataStandardizer` to allow multiple component types
shermanjasonaf Nov 30, 2024
7df8dec
Allow PyROS to accept `Var`s as uncertain params
shermanjasonaf Nov 30, 2024
24f0c99
Test uncertain param substitution in working model startup
shermanjasonaf Nov 30, 2024
40a134b
Apply black
shermanjasonaf Nov 30, 2024
2b9bd29
Remove unused import
shermanjasonaf Nov 30, 2024
90e4e15
Account for `Var`-type uncertain params in preprocessor tests
shermanjasonaf Nov 30, 2024
3d7b8f0
Maintain named expressions in `Param` for `Var` substitutions
shermanjasonaf Nov 30, 2024
972ae6c
Modify uncertain parameter replacement logging messages
shermanjasonaf Nov 30, 2024
c7cf91a
Note Var bounds/domains/fixing ignored in PyROS docstring
shermanjasonaf Nov 30, 2024
05dfbb6
Modify test for `InputDataStandardizer.domain_name()`
shermanjasonaf Nov 30, 2024
13e6ac5
Apply black
shermanjasonaf Nov 30, 2024
24d49c1
Merge branch 'main' into pyros-vars-as-params
shermanjasonaf Nov 30, 2024
3b4c59d
Tweak documentation of PyROS `uncertain_params` argument
shermanjasonaf Dec 1, 2024
ec086dd
Make some coeff matching tests more rigorous
shermanjasonaf Dec 1, 2024
d9588f1
Apply black
shermanjasonaf Dec 1, 2024
2243d2e
Tweak new var as uncertain param test
shermanjasonaf Dec 1, 2024
35e6487
Modify uncertain parameter refs in online docs
shermanjasonaf Dec 2, 2024
88d48c6
Merge branch 'main' into pyros-vars-as-params
shermanjasonaf Dec 2, 2024
3db32c9
Merge branch 'pyros-vars-as-params' of github.com:shermanjasonaf/pyom…
shermanjasonaf Dec 2, 2024
c372d98
Tweak note on vars as params
shermanjasonaf Dec 2, 2024
d265469
Merge branch 'main' into pyros-vars-as-params
shermanjasonaf Dec 20, 2024
21e28a1
Update version number, changelog
shermanjasonaf Dec 20, 2024
1bde454
Add bullet on DEBUG-level output logs
shermanjasonaf Dec 20, 2024
57ed57c
Merge branch 'main' into pyros-vars-as-params
blnicho Jan 14, 2025
1521928
Merge branch 'main' into pyros-vars-as-params
shermanjasonaf Jan 18, 2025
c3fc428
Require that `Vars` passed as uncertain params be fixed
shermanjasonaf Jan 20, 2025
04eb8aa
Apply black
shermanjasonaf Jan 20, 2025
df3ee82
Complete black reformatting
shermanjasonaf Jan 20, 2025
d2b98c1
More rigorously test uncertain param data validator
shermanjasonaf Jan 20, 2025
e55d0c5
Add note on uncertain params fixed to other uncertain params
shermanjasonaf Jan 20, 2025
7663111
Make PyROS uncertain params as vars tests more rigorous
shermanjasonaf Jan 20, 2025
af16736
Apply black
shermanjasonaf Jan 20, 2025
f85f103
Adjust PyROS online doc to new uncertain parameter restriction
shermanjasonaf Jan 20, 2025
17233f7
Tweak doc note on support for `Var`s as uncertain parameters
shermanjasonaf Jan 20, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 54 additions & 31 deletions doc/OnlineDocs/explanation/solvers/pyros.rst
Original file line number Diff line number Diff line change
Expand Up @@ -329,23 +329,30 @@ The deterministic Pyomo model for *hydro* is shown below.

.. note::
Primitive data (Python literals) that have been hard-coded within a
deterministic model cannot be later considered uncertain,
unless they are first converted to ``Param`` objects within
the ``ConcreteModel`` object.
Furthermore, any ``Param`` object that is to be later considered
uncertain must have the property ``mutable=True``.
deterministic model (:class:`~pyomo.core.base.PyomoModel.ConcreteModel`)
cannot be later considered uncertain,
unless they are first converted to Pyomo
:class:`~pyomo.core.base.param.Param` instances declared on the
:class:`~pyomo.core.base.PyomoModel.ConcreteModel` object.
Furthermore, any :class:`~pyomo.core.base.param.Param`
object that is to be later considered uncertain must be instantiated
with the argument ``mutable=True``.

.. note::
In case modifying the ``mutable`` property inside the deterministic
model object itself is not straightforward in your context,
you may consider adding the following statement **after**
If specifying/modifying the ``mutable`` argument in the
:class:`~pyomo.core.base.param.Param` declarations
of your deterministic model source code
is not straightforward in your context, then
you may consider adding **after** the line
``import pyomo.environ as pyo`` but **before** defining the model
object: ``pyo.Param.DefaultMutable = True``.
For all ``Param`` objects declared after this statement,
the attribute ``mutable`` is set to ``True`` by default.
Hence, non-mutable ``Param`` objects are now declared by
explicitly passing the argument ``mutable=False`` to the
``Param`` constructor.
object the statement: ``pyo.Param.DefaultMutable = True``.
For all :class:`~pyomo.core.base.param.Param`
objects declared after this statement,
the attribute ``mutable`` is set to True by default.
Hence, non-mutable :class:`~pyomo.core.base.param.Param`
objects are now declared by explicitly passing the argument
``mutable=False`` to the :class:`~pyomo.core.base.param.Param`
constructor.

.. doctest::

Expand Down Expand Up @@ -428,22 +435,37 @@ The deterministic Pyomo model for *hydro* is shown below.
Step 2: Define the Uncertainty
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

First, we need to collect into a list those ``Param`` objects of our model
that represent potentially uncertain parameters.
For the purposes of our example, we shall assume uncertainty in the model
parameters ``[m.p[0], m.p[1], m.p[2], m.p[3]]``, for which we can
conveniently utilize the object ``m.p`` (itself an indexed ``Param`` object).
We first collect the components of our model that represent the
uncertain parameters.
In this example, we assume uncertainty in
the parameter objects ``m.p[0]``, ``m.p[1]``, ``m.p[2]``, and ``m.p[3]``.
Since these objects comprise the mutable :class:`~pyomo.core.base.param.Param`
object ``m.p``, we can conveniently specify:

.. doctest::

>>> # === Specify which parameters are uncertain ===
>>> # We can pass IndexedParams this way to PyROS,
>>> # or as an expanded list per index
>>> uncertain_parameters = [m.p]
>>> uncertain_params = m.p

Equivalently, we may instead set ``uncertain_params`` to
either ``[m.p]``, ``[m.p[0], m.p[1], m.p[2], m.p[3]]``,
or ``list(m.p.values())``.

.. note::
Any :class:`~pyomo.core.base.param.Param` object that is
to be considered uncertain by PyROS must have the property
``mutable=True``.

.. note::
Any ``Param`` object that is to be considered uncertain by PyROS
must have the property ``mutable=True``.
PyROS also allows uncertain parameters to be implemented as
:class:`~pyomo.core.base.var.Var` objects declared on the
deterministic model.
This may be convenient for users transitioning to PyROS from
parameter estimation and/or uncertainty quantification workflows,
in which the uncertain parameters are
often represented by :class:`~pyomo.core.base.var.Var` objects.
Prior to invoking PyROS,
all such :class:`~pyomo.core.base.var.Var` objects should be fixed.


PyROS will seek to identify solutions that remain feasible for any
realization of these parameters included in an uncertainty set.
Expand Down Expand Up @@ -555,7 +577,7 @@ correspond to first-stage degrees of freedom.
... model=m,
... first_stage_variables=first_stage_variables,
... second_stage_variables=second_stage_variables,
... uncertain_params=uncertain_parameters,
... uncertain_params=uncertain_params,
... uncertainty_set=box_uncertainty_set,
... local_solver=local_solver,
... global_solver=global_solver,
Expand Down Expand Up @@ -648,7 +670,7 @@ In this example, we select affine decision rules by setting
... model=m,
... first_stage_variables=first_stage_variables,
... second_stage_variables=second_stage_variables,
... uncertain_params=uncertain_parameters,
... uncertain_params=uncertain_params,
... uncertainty_set=box_uncertainty_set,
... local_solver=local_solver,
... global_solver=global_solver,
Expand Down Expand Up @@ -702,7 +724,7 @@ could have been equivalently written as:
... model=m,
... first_stage_variables=first_stage_variables,
... second_stage_variables=second_stage_variables,
... uncertain_params=uncertain_parameters,
... uncertain_params=uncertain_params,
... uncertainty_set=box_uncertainty_set,
... local_solver=local_solver,
... global_solver=global_solver,
Expand Down Expand Up @@ -768,7 +790,7 @@ instance and invoking the PyROS solver:
... model=m,
... first_stage_variables=first_stage_variables,
... second_stage_variables=second_stage_variables,
... uncertain_params=uncertain_parameters,
... uncertain_params=uncertain_params,
... uncertainty_set= box_uncertainty_set,
... local_solver=local_solver,
... global_solver=global_solver,
Expand Down Expand Up @@ -860,7 +882,8 @@ for a basic tutorial, see the :doc:`logging HOWTO <python:howto/logging>`.
* Iteration log table
* Termination details: message, timing breakdown, summary of statistics
* - :py:obj:`logging.DEBUG`
- * Termination outcomes and summary of statistics for
- * Progress through the various preprocessing subroutines
* Termination outcomes and summary of statistics for
every master feasility, master, and DR polishing problem
* Progress updates for the separation procedure
* Separation subproblem initial point infeasibilities
Expand Down Expand Up @@ -935,7 +958,7 @@ Observe that the log contains the following information:
:linenos:

==============================================================================
PyROS: The Pyomo Robust Optimization Solver, v1.3.1.
PyROS: The Pyomo Robust Optimization Solver, v1.3.2.
Pyomo version: 6.9.0
Commit hash: unknown
Invoked at UTC 2024-11-01T00:00:00.000000
Expand Down
7 changes: 7 additions & 0 deletions pyomo/contrib/pyros/CHANGELOG.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@
PyROS CHANGELOG
===============

-------------------------------------------------------------------------------
PyROS 1.3.2 29 Nov 2024
-------------------------------------------------------------------------------
- Allow Var/VarData objects to be specified as uncertain parameters
through the `uncertain_params` argument to `PyROS.solve()`


-------------------------------------------------------------------------------
PyROS 1.3.1 25 Nov 2024
-------------------------------------------------------------------------------
Expand Down
130 changes: 82 additions & 48 deletions pyomo/contrib/pyros/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,8 @@
Interfaces for managing PyROS solver options.
"""

from collections.abc import Iterable
import logging

from pyomo.common.collections import ComponentSet
from pyomo.common.config import (
ConfigDict,
ConfigValue,
Expand All @@ -14,7 +12,6 @@
NonNegativeFloat,
InEnum,
Path,
_domain_name,
)
from pyomo.common.errors import ApplicationError, PyomoException
from pyomo.core.base import Var, VarData
Expand Down Expand Up @@ -61,64 +58,98 @@ def positive_int_or_minus_one(obj):
positive_int_or_minus_one.domain_name = "positive int or -1"


def mutable_param_validator(param_obj):
def uncertain_param_validator(uncertain_obj):
"""
Check that Param-like object has attribute `mutable=True`.
Check that a component object modeling an
uncertain parameter in PyROS is appropriately constructed,
initialized, and/or mutable, where applicable.

Parameters
----------
param_obj : Param or ParamData
Param-like object of interest.
uncertain_obj : Param or Var
Object on which to perform checks.

Raises
------
ValueError
If lengths of the param object and the accompanying
index set do not match. This may occur if some entry
of the Param is not initialized.
ValueError
If attribute `mutable` is of value False.
If the length of the component (data) object does not
match that of its index set, or the object is a Param
with attribute `mutable=False`.
"""
if len(param_obj) != len(param_obj.index_set()):
if len(uncertain_obj) != len(uncertain_obj.index_set()):
raise ValueError(
f"Length of Param component object with "
f"name {param_obj.name!r} is {len(param_obj)}, "
f"Length of {type(uncertain_obj).__name__} object with "
f"name {uncertain_obj.name!r} is {len(uncertain_obj)}, "
"and does not match that of its index set, "
f"which is of length {len(param_obj.index_set())}. "
"Check that all entries of the component object "
"have been initialized."
f"which is of length {len(uncertain_obj.index_set())}. "
"Check that the component has been properly constructed, "
"and all entries have been initialized. "
)
if isinstance(uncertain_obj, Param) and not uncertain_obj.mutable:
raise ValueError(
f"{type(uncertain_obj).__name__} object with name {uncertain_obj.name!r} "
"is immutable."
)
if not param_obj.mutable:
raise ValueError(f"Param object with name {param_obj.name!r} is immutable.")


def uncertain_param_data_validator(uncertain_obj):
"""
Validator for component data object specified as an
uncertain parameter.

Parameters
----------
uncertain_obj : Param or Var
Object on which to perform checks.

Raises
------
ValueError
If `uncertain_obj` is a VarData object
that is not fixed explicitly via VarData.fixed
or implicitly via bounds.
"""
if isinstance(uncertain_obj, VarData):
is_fixed_var = uncertain_obj.fixed or (
uncertain_obj.lower is uncertain_obj.upper
and uncertain_obj.lower is not None
)
if not is_fixed_var:
raise ValueError(
f"{type(uncertain_obj).__name__} object with name "
f"{uncertain_obj.name!r} is not fixed."
)


class InputDataStandardizer(object):
"""
Standardizer for objects castable to a list of Pyomo
component types.
Domain validator for an object that is castable to
a list of Pyomo component data objects.

Parameters
----------
ctype : type
Pyomo component type, such as Component, Var or Param.
cdatatype : type
Corresponding Pyomo component data type, such as
ctype : type or tuple of type
Valid Pyomo component type(s),
such as Component, Var or Param.
cdatatype : type or tuple of type
Valid Pyomo component data type(s), such as
ComponentData, VarData, or ParamData.
ctype_validator : callable, optional
Validator function for objects of type `ctype`.
cdatatype_validator : callable, optional
Validator function for objects of type `cdatatype`.
allow_repeats : bool, optional
True to allow duplicate component data entries in final
list to which argument is cast, False otherwise.
True to allow duplicate component data object
entries in final list to which argument is cast,
False otherwise.

Attributes
----------
ctype
cdatatype
ctype_validator
cdatatype_validator
allow_repeats
ctype : type or tuple of type
cdatatype : type or tuple of type
ctype_validator : callable or None
cdatatype_validator : callable or None
allow_repeats : bool
"""

def __init__(
Expand Down Expand Up @@ -151,13 +182,10 @@ def __call__(self, obj, from_iterable=None, allow_repeats=None):
True if list can contain repeated entries,
False otherwise.

Raises
------
TypeError
If all entries in the resulting list
are not of type ``self.cdatatype``.
ValueError
If the resulting list contains duplicate entries.
Returns
-------
list of ComponentData
Each entry is an instance of ``self.cdatatype``.
"""
return standardize_component_data(
obj=obj,
Expand All @@ -171,9 +199,12 @@ def __call__(self, obj, from_iterable=None, allow_repeats=None):

def domain_name(self):
"""Return str briefly describing domain encompassed by self."""
cdt = _domain_name(self.cdatatype)
ct = _domain_name(self.ctype)
return f"{cdt}, {ct}, or Iterable[{cdt}/{ct}]"
ctypes_tup = (self.ctype,) if isinstance(self.ctype, type) else self.ctype
cdtypes_tup = (
(self.cdatatype,) if isinstance(self.cdatatype, type) else self.cdatatype
)
alltypes_desc = ", ".join(vtype.__name__ for vtype in ctypes_tup + cdtypes_tup)
return f"(iterable of) {alltypes_desc}"


class SolverNotResolvable(PyomoException):
Expand Down Expand Up @@ -506,16 +537,19 @@ def pyros_config():
ConfigValue(
default=[],
domain=InputDataStandardizer(
ctype=Param,
cdatatype=ParamData,
ctype_validator=mutable_param_validator,
ctype=(Param, Var),
cdatatype=(ParamData, VarData),
ctype_validator=uncertain_param_validator,
cdatatype_validator=uncertain_param_data_validator,
allow_repeats=False,
),
description=(
"""
Uncertain model parameters.
The `mutable` attribute for all uncertain parameter
objects should be set to True.
Of every constituent `Param` object,
the `mutable` attribute must be set to True.
Of every constituent `Var` and `VarData` object,
the domain, declared bounds, and fixing are ignored.
"""
),
visibility=1,
Expand Down
10 changes: 6 additions & 4 deletions pyomo/contrib/pyros/pyros.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
)


__version__ = "1.3.1"
__version__ = "1.3.2"


default_pyros_solver_logger = setup_pyros_logger()
Expand Down Expand Up @@ -299,10 +299,12 @@ def solve(
First-stage model variables (or design variables).
second_stage_variables: VarData, Var, or iterable of VarData/Var
Second-stage model variables (or control variables).
uncertain_params: ParamData, Param, or iterable of ParamData/Param
uncertain_params: (iterable of) Param, Var, ParamData, or VarData
Uncertain model parameters.
The `mutable` attribute for all uncertain parameter objects
must be set to True.
Of every constituent `Param` object,
the `mutable` attribute must be set to True.
All constituent `Var`/`VarData` objects should be
fixed.
uncertainty_set: UncertaintySet
Uncertainty set against which the solution(s) returned
will be confirmed to be robust.
Expand Down
Loading
Loading