Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve docs #8

Merged
merged 5 commits into from
May 7, 2024
Merged

Improve docs #8

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a370659
Move development instructions into CONTRIBUTING.md
meshy Feb 13, 2024
01a822e
Fix references to info in README
meshy Feb 13, 2024
4ca6104
Summarise the tools available in the README
meshy Mar 1, 2024
08594a9
Use `python3` instead of `python` in instructions
meshy May 7, 2024
70fe5cb
Add missing word
meshy May 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 149 additions & 0 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
# Contributing

## Local development

### Creating a virtual environment

Ensure one of the supported Pythons (see README) is installed and used by the `python` executable:

```sh
python --version
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

python won't exist unless you've set an alias yourself or some tool has put a shim in your PATH. Should we reccommend invoking python directly with python3.12 or python3.10 etc?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say mention that it's easiest to manage python with asdf or pyvenv than to rely on a single system level python.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

easiest to manage python with asdf or pyvenv

I strongly disagree with this -- I have had all sorts of problem with both of those tools and the magic they do and have helped countless people with similar. I have never had problems with an explicit installation from python.org, as long as I don't expect my shell to read my mind.

Not the point here, though -- I don't want to get into an argument about environment setup. My only point was that python doesn't exist by default, so we might confuse users if we assume their shell setup.

All python expose a specific executable (e.g. python3.12) and it looks like they also have a python3 symlink to the same, so whichever is first in the PATH can use that. If we're going to be non-specific about minor version then I think this should at least use the real name of that symlink (python3). Similarly, we wouldn't put shell commands in docs that assumed specific oh-my-zsh plugins, even if those are what we use locally.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this.

I also don't want to get into details of shell setup, but I'm going to run on the assumption that anyone who's got as far as reading this guide will know the details of which python to use.

I'm cool with changing this to python3, but I don't think I want to go any further than that at this point.

```

Then create and activate a virtual environment. If you don't have any other way of managing virtual
environments this can be done by running:

```sh
python -m venv .venv
source .venv/bin/activate
```

You could also use [virtualenvwrapper], [direnv] or any similar tool to help manage your virtual
environments.

### Install PostgreSQL

Ensure that a supported version of PostgreSQL (see README) is installed and running on your local machine.

### Installing Python dependencies

> [!NOTE]
> You might not need to install the below requirements if you only intend to run the tests,
> because we use [tox] for the tests, and it manages the installation of dependencies.

If you only intend to run the tests with [tox], then you may only require:

```sh
pip install tox
Comment on lines +34 to +37
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we aren't installing the dependencies, are we still assuming the user has created a virtual env? Personally I would use pipx for this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to recommend any particular tool here, pipx included. I think that this file needs more revision, but that can wait for another PR. This PR should be mostly about the README.

```

Alternatively, to install all the development dependencies in your virtual environment, run:

```sh
make install
```

[direnv]: https://direnv.net
[virtualenvwrapper]: https://virtualenvwrapper.readthedocs.io/

### Testing

To start the tests with [tox], run:

```sh
make test
```

Alternatively, if you want to run the tests directly in your virtual environment,
you many run the tests with:

```sh
python -m pytest
```

### Static analysis

Run all static analysis tools with:

```sh
make lint
```

This may make changes to the local files if improvements are available.

### Managing dependencies

Package dependencies are declared in `pyproject.toml`.

- _package_ dependencies in the `dependencies` array in the `[project]` section.
- _development_ dependencies in the `dev` array in the `[project.optional-dependencies]` section.

For local development, the dependencies declared in `pyproject.toml` are pinned to specific
versions using the `requirements/development.txt` lock file.
You should not manually edit the `requirements/development.txt` lock file.

Prerequisites for installing those dependencies are tracked in the `requirements/prerequisites.txt`.


#### Adding a new dependency

To install a new Python dependency add it to the appropriate section in `pyproject.toml` and then
run:

```sh
make install
```

This will:

1. Build a new version of the `requirements/development.txt` lock file containing the newly added
package.
2. Sync your installed packages with those pinned in `requirements/development.txt`.

This will not change the pinned versions of any packages already in any requirements file unless
needed by the new packages, even if there are updated versions of those packages available.

Remember to commit your changed `requirements/development.txt` files alongside the changed
`pyproject.toml`.

#### Removing a dependency

Removing Python dependencies works exactly the same way: edit `pyproject.toml` and then run
`make install`.

#### Updating all Python packages

To update the pinned versions of all packages run:

```sh
make update
```

This will update the pinned versions of every package in the `requirements/development.txt` lock
file to the latest version which is compatible with the constraints in `pyproject.toml`.

You can then run:

```sh
make install
```

to sync your installed packages with the updated versions pinned in `requirements/development.txt`.

#### Updating individual Python packages

Upgrade a single development dependency with:

```sh
pip-compile -P $PACKAGE==$VERSION pyproject.toml --resolver=backtracking --extra=dev --output-file=requirements/development.txt
```

You can then run:

```sh
make install
```

to sync your installed packages with the updated versions pinned in `requirements/development.txt`.

[tox]: https://tox.wiki
184 changes: 46 additions & 138 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,159 +3,67 @@
Django Integrity contains tools for controlling deferred constraints
and handling `IntegrityError`s in Django projects which use PostgreSQL.

## Supported dependencies

This package is tested against:

- Python 3.10, 3.11, or 3.12.
- Django 4.1, 4.2, or 5.0.
- PostgreSQL 12 to 16
- psycopg2 and psycopg3

## Local development

### Creating a virtual environment

Ensure one of the above Pythons is installed and used by the `python` executable:

```sh
python --version
```

Then create and activate a virtual environment. If you don't have any other way of managing virtual
environments this can be done by running:

```sh
python -m venv .venv
source .venv/bin/activate
```

You could also use [virtualenvwrapper], [direnv] or any similar tool to help manage your virtual
environments.

### Install PostgreSQL

Ensure that PostgreSQL with minimum version 12 is installed and running on your local machine.

### Installing Python dependencies

> [!NOTE]
> You might not need to install the below requirements if you only intend to run the tests,
> because we use [tox] for the tests, and it manages the installation of dependencies.

If you only intend to run the tests with [tox], then you may only require:

```sh
pip install tox
```

Alternatively, to install all the development dependencies in your virtual environment, run:

```sh
make install
```

[direnv]: https://direnv.net
[virtualenvwrapper]: https://virtualenvwrapper.readthedocs.io/

### Testing
## Deferrable constraints

To start the tests with [tox], run:
Some PostgreSQL constraints can be defined as `DEFERRABLE`.
A constraint that is not deferred will be checked immediately after every command.
A deferred constraint check will be postponed until the end of the transaction.
A deferrable constraint will default to either `DEFERRED` or `IMMEDIATE`.

```sh
make test
```
The utilities in `django_integrity.constraints` can
ensure a deferred constraint is checked immediately,
or defer an immediate constraint.

Alternatively, if you want to run the tests directly in your virtual environment,
you many run the tests with:
These alter the state of constraints until the end of the current transaction:

```sh
python -m pytest
```
- `set_all_immediate(using=...)`
- `set_immedate(names=(...), using=...)`
- `set_deferred(names=(...), using=...)`

### Static analysis
To enforce a constraint immediately within some limited part of a transaction,
use the `immediate(names=(...), using=...)` context manager.

Run all static analysis tools with:
### Why do we need this?

```sh
make lint
```
This is most likely to be useful when you want to catch a foreign-key violation
(i.e.: you have inserted a row which references different row which doesn't exist).

This may make changes to the local files if improvements are available.
Django's foreign key constraints are deferred by default,
so they would normally raise an error only at the end of a transaction.
Using `try` to catch an `IntegrityError` from a foreign-key violation wouldn't work,
and you'd need to wrap the `COMMIT` instead, which is trickier.

### Managing dependencies
By making the constraint `IMMEDIATE`,
the constraint would be checked on `INSERT`,
and it would be much easier to catch.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely starting to look like a poem

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how to fix.
Might leave this until later.
Or haiku whole doc.


Package dependencies are declared in `pyproject.toml`.
More generally,
if you have custom deferrable constraint,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grammar not quite right here.

it may be useful to change the default behaviour with these tools.

- _package_ dependencies in the `dependencies` array in the `[project]` section.
- _development_ dependencies in the `dev` array in the `[project.optional-dependencies]` section.
## Refining `IntegrityError`

For local development, the dependencies declared in `pyproject.toml` are pinned to specific
versions using the `requirements/development.txt` lock file.
You should not manually edit the `requirements/development.txt` lock file.
The `refine_integrity_error` context manager in `django_integrity.conversion`
will convert an `IntegrityError` into a more specific exception
based on a mapping of rules to your custom exceptions,
and will raise the `IntegrityError` if it doesn't match.

Prerequisites for installing those dependencies are tracked in the `requirements/prerequisites.txt`.
### Why do we need this?

When a database constraint is violated,
we usually expect to see an `IntegrityError`.

#### Adding a new dependency
Sometimes we need more information about the error:
was it a unique constraint violation, or a check-constraint, or a not-null constraint?
Perhaps we ran out of 32-bit integers for our ID column?
Failing to be specific on these points could lead to bugs
where we catch an exception without realising it was not the one we expected.

To install a new Python dependency add it to the appropriate section in `pyproject.toml` and then
run:

```sh
make install
```

This will:

1. Build a new version of the `requirements/development.txt` lock file containing the newly added
package.
2. Sync your installed packages with those pinned in `requirements/development.txt`.

This will not change the pinned versions of any packages already in any requirements file unless
needed by the new packages, even if there are updated versions of those packages available.

Remember to commit your changed `requirements/development.txt` files alongside the changed
`pyproject.toml`.

#### Removing a dependency

Removing Python dependencies works exactly the same way: edit `pyproject.toml` and then run
`make install`.

#### Updating all Python packages

To update the pinned versions of all packages run:

```sh
make update
```

This will update the pinned versions of every package in the `requirements/development.txt` lock
file to the latest version which is compatible with the constraints in `pyproject.toml`.

You can then run:

```sh
make install
```

to sync your installed packages with the updated versions pinned in `requirements/development.txt`.

#### Updating individual Python packages

Upgrade a single development dependency with:

```sh
pip-compile -P $PACKAGE==$VERSION pyproject.toml --resolver=backtracking --extra=dev --output-file=requirements/development.txt
```

You can then run:

```sh
make install
```
## Supported dependencies

to sync your installed packages with the updated versions pinned in `requirements/development.txt`.
This package is tested against:

[tox]: https://tox.wiki
- Python 3.10, 3.11, or 3.12.
- Django 4.1, 4.2, or 5.0.
- PostgreSQL 12 to 16
- psycopg2 and psycopg3