Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pre-commit for testing style #6720

Merged
merged 3 commits into from
Dec 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2731,4 +2731,4 @@
],
"datetime": "2023-08-08T06:53:56.313769",
"version": "4.0.0"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2731,4 +2731,4 @@
],
"datetime": "2023-10-05T01:17:55.846789",
"version": "4.0.0"
}
}
2 changes: 1 addition & 1 deletion .coveragerc
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[report]
exclude_lines =
exclude_lines =
pragma: no cover
if TYPE_CHECKING:
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ _Short description of the approach_
- [ ] Prepare changes in small commits for more convenient review (optional)
- [ ] PR title captures the intent of the changes, and is fitting for release notes.
- [ ] Updated documentation
- [ ] Ensured that unit tests are added for all new behavior (See
- [ ] Ensured that unit tests are added for all new behavior (See
[Ground Rules](https://github.com/equinor/ert/blob/main/CONTRIBUTING.md#ground-rules)),
and changes to existing code have good test coverage.

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/doctest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ on:

jobs:
python-doctest:
name:
name:
timeout-minutes: 40
runs-on: ubuntu-latest
strategy:
Expand Down
15 changes: 2 additions & 13 deletions .github/workflows/style.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,21 +33,10 @@ jobs:
run: |
pip install ".[style]"

- name: Clang Format
run: ./script/clang-format --check

- name: CMake Format
if: ${{ always() }}
run: ./script/cmake-format --check

- name: Add matching rule
run: echo ::add-matcher::.github/flake8-matcher.json

- name: Run ruff
- name: Run style check
if: ${{ always() }}
run: |
ruff check .

- name: Run black
if: ${{ always() }}
uses: psf/black@stable
pre-commit run --all-files --show-diff-on-failure
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
*.user.*
.nfs*
*.swp
.pre-commit-config.yaml
.idea
.vscode
.pytest_cache
Expand Down
2 changes: 1 addition & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[submodule "test-data/block_storage"]
path = test-data/block_storage
url = https://github.com/equinor/ert-testdata
ignore = dirty
ignore = dirty
34 changes: 34 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.5.0
hooks:
- id: check-json
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.1.6
hooks:
- id: ruff
args: [ --fix ]
oyvindeide marked this conversation as resolved.
Show resolved Hide resolved

- repo: https://github.com/ambv/black
rev: 23.11.0
hooks:
- id: black

- repo: https://github.com/pre-commit/mirrors-clang-format
rev: v17.0.5
hooks:
- id: clang-format
args: [ --style=file, --Werror]
exclude: .json

- repo: https://github.com/cheshirekow/cmake-format-precommit
rev: v0.6.10
hooks:
- id: cmake-format
- id: cmake-lint
exclude: FindFilesystem
args: [ "--disable:C0301,C0111,C0113" ]
2 changes: 1 addition & 1 deletion CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ In addition to any definition or interpretation of the open source code of condu

If at any point concerns are raised that a group or member is not acting according to our values, the behaviour in question should cease, be discussed, and tried to be resolved.

We expect everyone to have a low threshold for raising issues, or in general discuss how we live up to our values.
We expect everyone to have a low threshold for raising issues, or in general discuss how we live up to our values.
Equally, we encourage all community members to appreciate when concerns are raised and do their best to solve them.

Project maintainers are responsible both for *what* is delivered, as well as *how* it is delivered.
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ noise in the review process.

You can build the documentation after installation by running
```bash
pip install "ert[dev]"
pip install "ert[dev]"
sphinx-build -n -v -E -W ./docs ./tmp/ert_docs
```
and then open the generated `./tmp/ert_docs/index.html` in a browser.
Expand Down
13 changes: 12 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ pip install --editable .
Additional development packages must be installed to run the test suite:

```sh
pip install "ert[dev]"
pip install "ert[dev]"
pytest tests/
```

Expand All @@ -150,6 +150,17 @@ If you checked out submodules without having git lfs installed, you can force gi
git submodule foreach "git lfs pull"
```


### Style requirements

There are a set of style requirements, which are gathered in the `pre-commit`
configuration, to have it automatically run on each commit do:

``` sh
$ pip install pre-commit
$ pre-commit install
```

### Trouble with setup

If you encounter problems during install, try deleting the `_skbuild` folder before reinstalling.
Expand Down
2 changes: 1 addition & 1 deletion docs/developer_documentation/concepts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ that in real life (as real as it gets on a computer at least) evaluations fail.
One can think of it as evaluating a realisation will with high probability give
you :code:`y = fm(x)`, but you might also get a failure (:code:`None`). Since
this is an inherent part of our domain we need to be able to communicate and
reason about it as a first class citizen of our problem domain.
reason about it as a first class citizen of our problem domain.

Realisation response
~~~~~~~~~~~~~~~~~~~~
Expand Down
2 changes: 1 addition & 1 deletion docs/developer_documentation/forward_model.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,4 +57,4 @@ of the runpath. After this, response loading is initiated by the core. In
particular, summary data is loaded from the configured :code:`ECLBASE` and in
addition the various :code:`GEN_DATA` etc. responses are loaded from their
configured files. If the loading of responses is also successful, the forward
model in its entirety is deemed successful by the core.
model in its entirety is deemed successful by the core.
2 changes: 1 addition & 1 deletion docs/developer_documentation/qt.rst
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ We should write:
dialog = MyDialog()
dialog.open()
qtbot.waitExposed(dialog)

# do something with dialog

dialog.close()
Expand Down
58 changes: 29 additions & 29 deletions docs/getting_started/configuration/poly_new/guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,16 @@ Configuration guide
This guide will introduce you to the basic concepts of ERT by demonstrating
a project setup that uses a simple polynomial as the :ref:`forward model <forward_model_chapter>`.

Minimal Configuration
Minimal Configuration
---------------------
We first create a minimal configuration and run an experiment that
We first create a minimal configuration and run an experiment that
doesn't execute any computations, but only generates the necessary folders and files.

1. **Create a Folder**:
1. **Create a Folder**:
Start by creating a folder named ``poly_example``.
This folder will contain all required files for our example.

Create a Configuration File
Create a Configuration File
***************************
Running ERT requires a dedicated configuration file, typically with the extension `.ert`.

Expand Down Expand Up @@ -64,7 +64,7 @@ Once all the realizations are complete, close all ERT windows.
Inspect the results
*******************

After running the experiment, two new folders, ``simulations`` and ``storage``, should appear in the project folder.
After running the experiment, two new folders, ``simulations`` and ``storage``, should appear in the project folder.

* ``storage`` contains ERT's internal data, which should remain untouched.
* ``simulations`` is generated based on the :ref:`RUNPATH <runpath>` configuration and contains the realizations of your experiment.
Expand Down Expand Up @@ -104,7 +104,7 @@ Do not modify these files, either manually or through your experiments' jobs.
Adding a Forward Model
----------------------
As mentioned, this example experiment will use a simple polynomial evaluation as the forward
model. In real-world scenarios, this would typically involve executing a physics simulator,
model. In real-world scenarios, this would typically involve executing a physics simulator,
such as Eclipse, Flow, pyWake, etc.

To get started, the forward model will be implemented as a simple Python script named
Expand All @@ -114,8 +114,8 @@ As we progress, both the script and the configuration file will be expanded to s

Changing the Default Runpath
****************************
The minimal configuration utilized the default :ref:`RUNPATH <runpath>`,
causing the realization to execute within the directory structure ``simulations/realization-<IENS>/iter-<ITER>``.
The minimal configuration utilized the default :ref:`RUNPATH <runpath>`,
causing the realization to execute within the directory structure ``simulations/realization-<IENS>/iter-<ITER>``.

You can alter this default location by modifying the runpath in the ``poly.ert`` file.
Simply add the following line to specify a new location:
Expand All @@ -131,7 +131,7 @@ giving you more control over the organization of your experiment's outputs.

Creating a Script to Define the Forward Model
*********************************************
The forward model for our example will be defined in a Python script named ``poly_eval.py``.
The forward model for our example will be defined in a Python script named ``poly_eval.py``.
The script will evaluate a polynomial with fixed coefficients at specific points, writing the results to a file.

1. **Create the Script**: Within your project folder, create a file named ``poly_eval.py`` with the following content:
Expand Down Expand Up @@ -242,10 +242,10 @@ In the following section, you'll learn how to use ERT to automatically vary para

Creating Parameters
-------------------
To sample different parameters across realizations in ERT,
you'll need to define the prior distribution for each parameter.
Furthermore, you'll detail how ERT can identify and inject these parameters into
each simulation run via a templating mechanism.
To sample different parameters across realizations in ERT,
you'll need to define the prior distribution for each parameter.
Furthermore, you'll detail how ERT can identify and inject these parameters into
each simulation run via a templating mechanism.

Adding Prior Distributions
**************************
Expand All @@ -257,10 +257,10 @@ Create a file named ``coeff_priors`` and add the following content:
Each line of this file defines a parameter:

- The first part is the name of the parameter (e.g., ``a``).
- The second part defines the type of distribution from which to sample the parameter.
- The second part defines the type of distribution from which to sample the parameter.
In this case, we're using a uniform distribution (``UNIFORM``).
- The remaining parts describe the distribution's specific characteristics.
For a uniform distribution, these are the lower and upper bounds.
- The remaining parts describe the distribution's specific characteristics.
For a uniform distribution, these are the lower and upper bounds.
Different distributions will require different arguments.

Configuring the parameter set
Expand Down Expand Up @@ -305,9 +305,9 @@ Increasing the number of realizations
*************************************
Let's increase the number of realizations to obtain a larger sample size.

1. **Increase the Number of Realizations**: Set the ``NUM_REALIZATIONS`` value to ``100`` to instruct ERT to run 100 simulations.
1. **Increase the Number of Realizations**: Set the ``NUM_REALIZATIONS`` value to ``100`` to instruct ERT to run 100 simulations.

2. **Configure Parallel Execution**: To make the experiment run faster, you can specify the number of simultaneous simulations that the system can execute.
2. **Configure Parallel Execution**: To make the experiment run faster, you can specify the number of simultaneous simulations that the system can execute.
Use the :ref:`queue option <queue_option>` ``MAX_RUNNING`` for the ``LOCAL`` queue and set it to ``50``:

.. code-block:: shell
Expand All @@ -331,7 +331,7 @@ This ensures that you'll only see the new data in your results.

2. **Start the Experiment**: Choose "Ensemble Experiment" in the "Simulation mode" and initiate the simulation.

3. **Create Plot**: Once the simulations are complete, you have the option to press the "Create Plot" button either in the progress window or in the main window.
3. **Create Plot**: Once the simulations are complete, you have the option to press the "Create Plot" button either in the progress window or in the main window.
This action will open the "Plotting" window.

4. **View Distributions**: In the "Plotting" window, you can now observe the distributions of the three different parameters you created: ``COEFFS:a``, ``COEFFS:b``, and ``COEFFS:c``.
Expand All @@ -351,7 +351,7 @@ Inspecting the Parameters and Results
*************************************
The sampled parameters and resulting simulation data can be inspected within each runpath.

1. **Inspecting the Parameters**: You should find the instantiated parameter template files named ``coeffs.json`` in each runpath.
1. **Inspecting the Parameters**: You should find the instantiated parameter template files named ``coeffs.json`` in each runpath.
To look at a specific realization, you might run a command such as:

.. code-block:: shell
Expand Down Expand Up @@ -393,16 +393,16 @@ This should return something similar to:

Adding observations
-------------------
The simple polynomial in our example serves as a model of a real-world process,
representing our best current understanding of how this process behaves.
The accuracy of this model hinges on how well a polynomial mirrors reality and how precise the parameters `a`, `b`, and `c` are.
The simple polynomial in our example serves as a model of a real-world process,
representing our best current understanding of how this process behaves.
The accuracy of this model hinges on how well a polynomial mirrors reality and how precise the parameters `a`, `b`, and `c` are.
In this section, we'll leverage ERT to enhance the parameter estimates using real-world observations.

Observations File
*****************
We have polynomial observations measured at the points 0, 2, 4, 6, and 8.
These indices align with the x values of the polynomial evaluation, although this alignment is coincidental.
The indices indicate how the observations in the file correspond to the result indices.
We have polynomial observations measured at the points 0, 2, 4, 6, and 8.
These indices align with the x values of the polynomial evaluation, although this alignment is coincidental.
The indices indicate how the observations in the file correspond to the result indices.

1. **Create the Observations File**: Create a file named ``poly_obs_data.txt`` in the projects folders with:

Expand All @@ -422,7 +422,7 @@ which refers to a file where the :ref:`GENERAL_OBSERVATION <general_observation>
Here, :ref:`GENERAL_OBSERVATION <general_observation>` initiates a set of observations and pairs them with simulation results through key-value pairs:

- **DATA**: Relates the observation to a result set.
- **INDEX_LIST**: Since we have 10 values in our results file but only 5 observations, this list tells ERT the corresponding results.
- **INDEX_LIST**: Since we have 10 values in our results file but only 5 observations, this list tells ERT the corresponding results.
If the lengths are equal, omit this.
- **RESTART**: Legacy value, must match ``REPORT_STEPS`` from the ``GEN_DATA`` line.
- **OBS_FILE**: Specifies the file containing the observations.
Expand All @@ -441,7 +441,7 @@ With the final configuration:

.. literalinclude:: with_observations/poly_final.ert

Launch ERT, choose the "Ensemble Smoother" and hit "Run Experiment".
Launch ERT, choose the "Ensemble Smoother" and hit "Run Experiment".

As simulations run, you'll notice the updated parameters yield results that better align with observations.

Expand All @@ -453,7 +453,7 @@ Different iterations can be viewed by selecting "default" and "default_smoother_

Evaluating the Updated Parameters
*********************************
Examine the improved estimates for `a`, `b`, and `c`.
Examine the improved estimates for `a`, `b`, and `c`.
Though not perfect, they're better than the initial guesses. ERT's updating algorithms have reduced uncertainty:

.. image:: with_observations/coeff_a.png
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,5 @@ GEN_KW COEFFS coeff_priors
GEN_DATA POLY_RES RESULT_FILE:poly.out
OBS_CONFIG observations

INSTALL_JOB poly_eval POLY_EVAL
INSTALL_JOB poly_eval POLY_EVAL
FORWARD_MODEL poly_eval
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,5 @@ GEN_DATA POLY_RES RESULT_FILE:poly.out

OBS_CONFIG observations

INSTALL_JOB poly_eval POLY_EVAL
INSTALL_JOB poly_eval POLY_EVAL
FORWARD_MODEL poly_eval
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@ GEN_KW COEFFS coeff_priors

GEN_DATA POLY_RES RESULT_FILE:poly.out

INSTALL_JOB poly_eval POLY_EVAL
INSTALL_JOB poly_eval POLY_EVAL
FORWARD_MODEL poly_eval
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@ NUM_REALIZATIONS 5

RUNPATH poly_out/realization-<IENS>/iter-<ITER>

INSTALL_JOB poly_eval POLY_EVAL
INSTALL_JOB poly_eval POLY_EVAL
FORWARD_MODEL poly_eval
4 changes: 2 additions & 2 deletions docs/getting_started/howto/esmda_restart.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Restarting ES-MDA
-----------------
Imagine you are conducting an experiment consisting of three iterations of ES-MDA,
Imagine you are conducting an experiment consisting of three iterations of ES-MDA,
with each iteration potentially taking hours or even days.
For the purposes of this example, assume that the "Relative weights" are set to ``4, 2, 1``.

Expand All @@ -16,7 +16,7 @@ One solution is to restart from ``default_2`` which is straightforward in ERT.
In this example, you'll pick ``default_2``.
3. Click "Run Experiment".
This creates a new ensemble which by default is called ``default_2_3``.
This ensemble will use the relative weight ``1``,
This ensemble will use the relative weight ``1``,
consistent with the last value specified in the "Relative weights" sequence: ``4, 2, 1``.

**Note:**
Expand Down
Loading
Loading