Skip to content

Commit

Permalink
Replace current_case and target_case with ensemble
Browse files Browse the repository at this point in the history
Deprecate current-case and target-case in CLI
  • Loading branch information
dafeda committed Mar 6, 2024
1 parent 9b704c2 commit c029007
Show file tree
Hide file tree
Showing 64 changed files with 572 additions and 412 deletions.
8 changes: 4 additions & 4 deletions docs/about/release_notes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2041,13 +2041,13 @@ that this requires a bit of care due to the lack of metadata in current storage.
We are aiming at resolving this in the future in the new storage that soon will
be the standard.

After selecting the ES-MDA algorithm, you first need to set `Current case` to
After selecting the ES-MDA algorithm, you first need to set `Current ensemble` to
the case you intend to be your first case to reevaluate. After which the
iteration number will be wrongly injected into the `Target case format`, which
you have to remove manually (reset it to the original `Target case format`).
iteration number will be wrongly injected into the `Target ensemble format`, which
you have to remove manually (reset it to the original `Target ensemble format`).
After which you have to set `Start iteration` to the iteration for which you
are to start from; this number must correspond to the iteration number of the
case you have selected as your `Current case`. We recognize that this induces
case you have selected as your `Current ensemble`. We recognize that this induces
some manual checking, but this is due to the lack of metadata mentioned above.
We still hope that this can aid our users and prevent the need of a complete
restart due to cluster issues etc.
Expand Down
8 changes: 4 additions & 4 deletions docs/reference/configuration/data_types.rst
Original file line number Diff line number Diff line change
Expand Up @@ -398,9 +398,9 @@ To define a triangular distribution with a minimum of 1, mode (peak) of 3, and m
Loading GEN_KW values from an external file
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The default use of the GEN_KW keyword is to let the ERT application sample
The default use of the GEN_KW keyword is to let ert sample
random values for the elements in the GEN_KW instance, but it is also possible
to tell ERT to load a precreated set of data files, this can for instance be
to tell ert to load a precreated set of data files. This can for instance be
used as a component in an experimental design based workflow. When using external
files to initialize the GEN_KW instances you supply an extra keyword
``INIT_FILE:/path/to/priors/files%d`` which tells where the prior files are:
Expand All @@ -411,7 +411,7 @@ files to initialize the GEN_KW instances you supply an extra keyword

In the example above you must prepare files priors/multflt/faults0,
priors/multflt/faults1, ... priors/multflt/faultsn which ert will load when you
initialize the case. The format of the GEN_KW input files can be of two
initialize the ensemble. The format of the GEN_KW input files can be of two
varieties:

1. The files can be plain ASCII text files with a list of numbers:
Expand Down Expand Up @@ -521,7 +521,7 @@ attribute when it used as *the way* to initialize fields, and when it is used in
combination with ``FORWARD_INIT:True``. When ``INIT_FILES:`` is used alone the
filename given should contain a ``%d`` which will be replaced with realization
number, when used with ``FORWARD_INIT:True`` that is not necessary. Furthermore
in the ``FORWARD_INIT:True`` case the *the path is interpreted relative to the
in the ``FORWARD_INIT:True`` case *the path is interpreted relative to the
runpath folder*, whereas in the other case the path is interpreted relative to
the location of the main ERT configuration file.

Expand Down
80 changes: 66 additions & 14 deletions src/ert/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -283,7 +283,13 @@ def get_ert_parser(parser: Optional[ArgumentParser] = None) -> ArgumentParser:
"--current-case",
type=valid_name,
default="default",
help="Name of the case where the results for the experiment "
help="Deprecated: This argument is deprecated and will be "
"removed in future versions. Use --current-ensemble instead.",
)
test_run_parser.add_argument(
"--current-ensemble",
type=valid_name,
help="Name of the ensemble where the results for the experiment "
"using the prior parameters will be stored.",
)

Expand All @@ -308,7 +314,14 @@ def get_ert_parser(parser: Optional[ArgumentParser] = None) -> ArgumentParser:
"--current-case",
type=valid_case,
default="default",
help="Name of the case where the results for the experiment "
help="Deprecated: This argument is deprecated and will be "
"removed in future versions. Use --current-ensemble instead.",
)
ensemble_experiment_parser.add_argument(
"--current-ensemble",
type=valid_case,
default="default",
help="Name of the ensemble where the results for the experiment "
"using the prior parameters will be stored.",
)
ensemble_experiment_parser.add_argument(
Expand Down Expand Up @@ -341,7 +354,13 @@ def get_ert_parser(parser: Optional[ArgumentParser] = None) -> ArgumentParser:
"--target-case",
type=valid_name,
required=True,
help="Name of the case where the results for the "
help="Deprecated: This argument is deprecated and will be "
"removed in future versions. Use --target-ensemble instead.",
)
ensemble_smoother_parser.add_argument(
"--target-ensemble",
type=valid_name,
help="Name of the ensemble where the results for the "
"updated parameters will be stored.",
)
ensemble_smoother_parser.add_argument(
Expand All @@ -356,7 +375,14 @@ def get_ert_parser(parser: Optional[ArgumentParser] = None) -> ArgumentParser:
"--current-case",
type=valid_name,
default="default",
help="Name of the case where the results for the experiment "
help="Deprecated: This argument is deprecated and will be "
"removed in future versions. Use --current-ensemble instead.",
)
ensemble_smoother_parser.add_argument(
"--current-ensemble",
type=valid_name,
default="default",
help="Name of the ensemble where the results for the experiment "
"using the prior parameters will be stored.",
)

Expand All @@ -374,10 +400,16 @@ def get_ert_parser(parser: Optional[ArgumentParser] = None) -> ArgumentParser:
"--target-case",
type=valid_name_format,
required=True,
help="The iterative ensemble smoother creates multiple cases for the different "
"iterations. The case names will follow the specified format. "
"For example, 'Target case format: iter_%%d' will generate "
"cases with the names iter_0, iter_1, iter_2, iter_3, ....",
help="Deprecated: This argument is deprecated and will be "
"removed in future versions. Use --target-ensemble instead.",
)
iterative_ensemble_smoother_parser.add_argument(
"--target-ensemble",
type=valid_name_format,
help="The iterative ensemble smoother creates multiple ensembles for the different "
"iterations. The ensemble names will follow the specified format. "
"For example, 'Target ensemble format: iter_%%d' will generate "
"ensembles with the names iter_0, iter_1, iter_2, iter_3, ....",
)
iterative_ensemble_smoother_parser.add_argument(
"--realizations",
Expand All @@ -391,7 +423,14 @@ def get_ert_parser(parser: Optional[ArgumentParser] = None) -> ArgumentParser:
"--current-case",
type=valid_name,
default="default",
help="Name of the case where the results for the experiment "
help="Deprecated: This argument is deprecated and will be "
"removed in future versions. Use --current-ensemble instead.",
)
iterative_ensemble_smoother_parser.add_argument(
"--current-ensemble",
type=valid_name,
default="default",
help="Name of the ensemble where the results for the experiment "
"using the prior parameters will be stored.",
)
iterative_ensemble_smoother_parser.add_argument(
Expand All @@ -409,10 +448,16 @@ def get_ert_parser(parser: Optional[ArgumentParser] = None) -> ArgumentParser:
es_mda_parser.add_argument(
"--target-case",
type=valid_name_format,
help="The es_mda creates multiple cases for the different "
"iterations. The case names will follow the specified format. "
"For example, 'Target case format: iter-%%d' will generate "
"cases with the names iter-0, iter-1, iter-2, iter-3, ....",
help="Deprecated: This argument is deprecated and will be "
"removed in future versions. Use --target-ensemble instead.",
)
es_mda_parser.add_argument(
"--target-ensemble",
type=valid_name_format,
help="The es_mda creates multiple ensembles for the different "
"iterations. The ensemble names will follow the specified format. "
"For example, 'Target ensemble format: iter-%%d' will generate "
"ensembles with the names iter-0, iter-1, iter-2, iter-3, ....",
)
es_mda_parser.add_argument(
"--realizations",
Expand All @@ -434,7 +479,14 @@ def get_ert_parser(parser: Optional[ArgumentParser] = None) -> ArgumentParser:
"--restart-case",
type=valid_name,
default=None,
help="Name of the case where the results for the experiment "
help="Deprecated: This argument is deprecated and will be "
"removed in future versions. Use --restart-ensemble instead.",
)
es_mda_parser.add_argument(
"--restart-ensemble",
type=valid_name,
default=None,
help="Name of the ensemble where the results for the experiment "
"using the prior parameters will be stored. Iteration number is read "
"from this case. If provided this will be a restart a run",
)
Expand Down
8 changes: 4 additions & 4 deletions src/ert/analysis/_es_update.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,8 +68,8 @@ def __post_init__(self) -> None:

@dataclass
class SmootherSnapshot:
source_case: str
target_case: str
source_ensemble: str
target_ensemble: str
alpha: float
std_cutoff: float
global_scaling: float
Expand Down Expand Up @@ -672,8 +672,8 @@ def _write_update_report(path: Path, snapshot: SmootherSnapshot, run_id: str) ->
fout.write("=" * 150 + "\n")
timestamp = datetime.now().strftime("%Y.%m.%d %H:%M:%S")
fout.write(f"Time: {timestamp}\n")
fout.write(f"Parent ensemble: {snapshot.source_case}\n")
fout.write(f"Target ensemble: {snapshot.target_case}\n")
fout.write(f"Parent ensemble: {snapshot.source_ensemble}\n")
fout.write(f"Target ensemble: {snapshot.target_ensemble}\n")
fout.write(f"Alpha: {snapshot.alpha}\n")
fout.write(f"Global scaling: {snapshot.global_scaling}\n")
fout.write(f"Standard cutoff: {snapshot.std_cutoff}\n")
Expand Down
24 changes: 24 additions & 0 deletions src/ert/cli/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import logging
import os
import sys
import warnings
from typing import Any, TextIO

from _ert.threading import ErtThread
Expand Down Expand Up @@ -34,6 +35,29 @@ class ErtTimeoutError(Exception):


def run_cli(args: Namespace, _: Any = None) -> None:
with warnings.catch_warnings():
warnings.simplefilter("default", DeprecationWarning)
if getattr(args, "current_case", None):
warnings.warn(
"The '--current-case' argument is deprecated and will be removed in future versions. Use '--current-ensemble' instead.",
DeprecationWarning,
stacklevel=1,
)

if getattr(args, "target_case", None):
warnings.warn(
"The '--target-case' argument is deprecated and will be removed in future versions. Use '--target-ensemble' instead.",
DeprecationWarning,
stacklevel=1,
)

if getattr(args, "restart_case", None):
warnings.warn(
"The '--restart-case' argument is deprecated and will be removed in future versions. Use '--restart-ensemble' instead.",
DeprecationWarning,
stacklevel=1,
)

ert_dir = os.path.abspath(os.path.dirname(args.config))
os.chdir(ert_dir)
# Changing current working directory means we need to update
Expand Down
66 changes: 48 additions & 18 deletions src/ert/cli/model_factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -97,10 +97,16 @@ def create_model(
def _setup_single_test_run(
config: ErtConfig, storage: Storage, args: Namespace
) -> SingleTestRun:

current_ensemble = getattr(args, "current_case", "default")

if current_ensemble == "default":
current_ensemble = getattr(args, "current_ensemble", "default")

return SingleTestRun(
SingleTestRunArguments(
random_seed=config.random_seed,
current_case=args.current_case,
current_ensemble=current_ensemble,
minimum_required_realizations=1,
ensemble_size=config.model_config.num_realizations,
stop_long_running=config.analysis_config.stop_long_running,
Expand All @@ -126,12 +132,19 @@ def _setup_ensemble_experiment(
)
experiment_name = args.experiment_name
assert experiment_name is not None

current_ensemble = getattr(args, "current_case", "default")
if current_ensemble == "default":
current_ensemble = getattr(args, "current_ensemble", "default")

return EnsembleExperiment(
EnsembleExperimentRunArguments(
random_seed=config.random_seed,
active_realizations=active_realizations.tolist(),
current_case=args.current_case,
iter_num=int(args.iter_num),
current_ensemble=current_ensemble,
iter_num=int(
getattr(args, "iter_num", 0)
), # Safely access iter_num with 0 as a default
minimum_required_realizations=config.analysis_config.minimum_required_realizations,
ensemble_size=config.model_config.num_realizations,
stop_long_running=config.analysis_config.stop_long_running,
Expand Down Expand Up @@ -161,7 +174,7 @@ def _setup_evaluate_ensemble(
EvaluateEnsembleRunArguments(
random_seed=config.random_seed,
active_realizations=active_realizations.tolist(),
current_case=args.ensemble_name,
current_ensemble=args.ensemble_name,
minimum_required_realizations=config.analysis_config.minimum_required_realizations,
ensemble_size=config.model_config.num_realizations,
stop_long_running=config.analysis_config.stop_long_running,
Expand All @@ -179,18 +192,27 @@ def _setup_ensemble_smoother(
args: Namespace,
update_settings: UpdateSettings,
) -> EnsembleSmoother:

current_ensemble = getattr(args, "current_case", "default")
if current_ensemble == "default":
current_ensemble = getattr(args, "current_ensemble", "default")

target_ensemble = getattr(args, "target_case", None)
if target_ensemble is None:
target_ensemble = args.target_ensemble

return EnsembleSmoother(
ESRunArguments(
random_seed=config.random_seed,
active_realizations=_realizations(
args, config.model_config.num_realizations
).tolist(),
current_case=args.current_case,
target_case=args.target_case,
current_ensemble=current_ensemble,
target_ensemble=target_ensemble,
minimum_required_realizations=config.analysis_config.minimum_required_realizations,
ensemble_size=config.model_config.num_realizations,
stop_long_running=config.analysis_config.stop_long_running,
experiment_name=args.experiment_name,
experiment_name=getattr(args, "experiment_name", ""),
),
config,
storage,
Expand All @@ -209,9 +231,11 @@ def _determine_restart_info(args: Namespace) -> Tuple[bool, str]:
to run from.
"""
if hasattr(args, "restart_case"):
# When running from CLI
restart_run = args.restart_case is not None
prior_ensemble = args.restart_case
else:
# When running from GUI
restart_run = args.restart_run
prior_ensemble = args.prior_ensemble
return restart_run, prior_ensemble
Expand All @@ -231,7 +255,7 @@ def _setup_multiple_data_assimilation(
active_realizations=_realizations(
args, config.model_config.num_realizations
).tolist(),
target_case=_iterative_case_format(config, args),
target_ensemble=_iterative_ensemble_format(config, args),
weights=args.weights,
restart_run=restart_run,
prior_ensemble=prior_ensemble,
Expand Down Expand Up @@ -260,8 +284,8 @@ def _setup_iterative_ensemble_smoother(
active_realizations=_realizations(
args, config.model_config.num_realizations
).tolist(),
current_case=args.current_case,
target_case=_iterative_case_format(config, args),
current_ensemble=args.current_case,
target_ensemble=_iterative_ensemble_format(config, args),
num_iterations=_num_iterations(config, args),
minimum_required_realizations=config.analysis_config.minimum_required_realizations,
ensemble_size=config.model_config.num_realizations,
Expand All @@ -285,20 +309,26 @@ def _realizations(args: Namespace, ensemble_size: int) -> npt.NDArray[np.bool_]:
)


def _iterative_case_format(config: ErtConfig, args: Namespace) -> str:
def _iterative_ensemble_format(config: ErtConfig, args: Namespace) -> str:
"""
When a RunModel runs multiple iterations, a case format will be used.
E.g. when starting from the case 'case', subsequent runs can be named
'case_0', 'case_1', 'case_2', etc.
When a RunModel runs multiple iterations, an ensemble format will be used.
E.g. when starting from the ensemble 'ensemble', subsequent runs can be named
'ensemble_0', 'ensemble_1', 'ensemble_2', etc.
This format can be set from the commandline via the `target_case` option,
This format can be set from the commandline via the `target_ensemble` option,
and via the config file via the `ITER_CASE` keyword. If none of these are
set we use the name of the current case and add `_%d` to it.
set we use the name of the current ensemble and add `_%d` to it.
"""
if getattr(args, "target_case", None) is not None:
target_ensemble = args.target_case
elif hasattr(args, "target_ensemble"):
target_ensemble = args.target_ensemble
else:
target_ensemble = None
return (
args.target_case
target_ensemble
or config.analysis_config.case_format
or f"{getattr(args, 'current_case', None) or 'default'}_%d"
or f"{getattr(args, 'current_ensemble', None) or 'default'}_%d"
)


Expand Down
2 changes: 1 addition & 1 deletion src/ert/dark_storage/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ def data_for_key(
key: str,
) -> pd.DataFrame:
"""Returns a pandas DataFrame with the datapoints for a given key for a
given case. The row index is the realization number, and the columns are an
given ensemble. The row index is the realization number, and the columns are an
index over the indexes/dates"""
if key.startswith("LOG10_"):
key = key[6:]
Expand Down
Loading

0 comments on commit c029007

Please sign in to comment.