Adding signals, updated docs, Transaction now depends on Session and cannot be used independently anymore

Author: maximebf

docs/engine.md

@@ -42,7 +42,18 @@ engine = Engine.from_uri("sqlite://:memory:") # sqlite instead of sqlite3 to us
 engine = Engine.from_uri("psycopg://?host=localhost&port=5432") # using the dbapi module name
 ```
 
-Once initialized, an engine can be used as a context to start sessions.
+Once initialized, an engine can be used as a context to [start sessions](sessions-and-transactions.md).
+
+## Connecting
+
+You shouldn't create connection manually when using sqlorm but use sessions.
+
+To create connections use `conn = engine.connect()`.  
+To close a connection use `engine.disconnect(conn)`.
+
+## Connection pool
 
 Connections are pooled and re-used by default. You can disabled this behavior by using `pool=False` in the engine constructor.
 `max_pool_conns` can also be used to define the maximum number of connections to start.
+
+Use `engine.disconnect_all()` to close all connections.

docs/executing.md

@@ -28,7 +28,7 @@ with engine as tx:
     tx.execute(stmt)
 ```
 
-[`SQL` utilities](#sql-utilities) to build sql statements can also be used:
+[`SQL` utilities](sql-utilities.md) to build sql statements can also be used:
 
 ```python
 from sqlorm import SQL
@@ -94,7 +94,7 @@ with engine as tx:
 > - The process of populating objects with data from the database is called "hydration".
 > - When creating objects, `__init__()` will be bypassed as well as any custom `__setattr__`.
 >
-> Learn more about sqlorm hydration process in the [mapper section](#hydrate-objects)
+> Learn more about sqlorm hydration process in the [mapper section](mapper.md#hydrate-objects)
 
 You can update existing objects by passing an object instance to the `obj` argument.
 In this case, only the first row will be used to hydrate the object. The cursor is then closed.
@@ -107,7 +107,7 @@ with engine as tx:
     assert task.title == "task title"
 ```
 
-[Models](#models) and [mappers](#mapping-any-class) can be used to customize how objects are mapped.
+[Models](models.md) and [mappers](mapper.md) can be used to customize how objects are mapped.
 `Mapper` instances can also be provided as models.
 
 ## Composite rows
@@ -183,7 +183,7 @@ with engine as tx:
     rows = tx.fetchcomposite("SELECT ... FROM posts LEFT JOIN comments ...")
 ```
 
-Using the [`SQL` utilities](#sql-utilities) makes it easier to build these kind of queries:
+Using the [`SQL` utilities](sql-utilities.md) makes it easier to build these kind of queries:
 
 ```python
 with engine as tx:

docs/index.md

@@ -10,19 +10,19 @@ Sqlorm intends to provide a solution where SQL stays front and center and where
 
 ## Getting started
 
-Create an [`Engine`](#the-engine) that will manage database connections for you:
+Create an [`Engine`](engine.md) that will manage database connections for you:
 
 ```python
 from sqlorm import Engine
 
 engine = Engine.from_uri("sqlite://:memory:")
 ```
 
-Use the engine as a [context to connect to the database](#sessions-and-transactions). A transaction object is provided to execute statements.
+Use the engine as a [context to connect to the database](sessions-and-transactions.md). A transaction object is provided to execute statements.
 If no exception is raised in the context, the transaction will be committed, rollbacked otherwise.
 
 sqlorm's `Transaction` has a similar API than DBAPI's Connection but instead of using a cursor,
-methods return [result sets](#fetching-from-the-database) that you can iterate over:
+methods return [result sets](executing.md#fetching-from-the-database) that you can iterate over:
 
 ```python
 with engine as tx:
@@ -46,7 +46,7 @@ with engine as tx:
     print(task1.title)
 ```
 
-To facilitate managing sql statements, you can create ["sql functions"](#sql-functions).
+To facilitate managing sql statements, you can create ["sql functions"](sql-functions.md).
 These are functions which only have a doc string containing the SQL statement.
 
 ```python
@@ -76,7 +76,7 @@ with engine:
 
 Note: these functions must be executed in the context of a database session (or be bound to an engine)
 
-Finally, we can create [model classes](#models) that can provide custom mapping information:
+Finally, we can create [model classes](models.md) that can provide custom mapping information:
 
 ```python
 from sqlorm import Model, PrimaryKey, create_table

docs/mapper.md

@@ -113,7 +113,7 @@ with engine as tx:
     tx.execute(mapper.delete(task))
 ```
 
-The mapper column list `mapper.columns` is an [`SQL.Cols`](#manage-list-of-sql-pieces) object which means it can be used to generate sql conditions:
+The mapper column list `mapper.columns` is an [`SQL.Cols`](sql-utilities.md#manage-list-of-sql-pieces) object which means it can be used to generate sql conditions:
 
 ```python
 with engine as tx:

docs/models.md

@@ -7,13 +7,13 @@ Any class can be used as a model. However, subclassing sqlorm's `Model` class pr
 - Auto fetching lazy attributes and relationships when accessed
 
 > [!NOTE]
-> When using classes that do not subclass `Model`, `Mapper.from_class()` is used to generate a mapping. See [Mapping any class](#mapping-any-class).
+> When using classes that do not subclass `Model`, `Mapper.from_class()` is used to generate a mapping. See [Mapping any class](mapper.md).
 
 ## Defining models
 
 Models are classes inheriting from `Model`. To define which table they represent, use the `table` class property.
 
-To define column mapping, define properties via annotations. The type used will be [converted to an sql type](#column-types).
+To define column mapping, define properties via annotations. The type used will be [converted to an sql type](sql-utilities.md#column-types).
 For more control over the mapping, you can use instantiate `Column()` objects:
 
 ```python
@@ -29,7 +29,7 @@ class Task(Model):
 
 Once columns are defined via annotations or `Column` properties, they are accessible as class and instance properties.
 
-As class properties, they can be used as a [composable piece of sql](#sql-utilities):
+As class properties, they can be used as a [composable piece of sql](sql-utilities.md):
 
 ```python
 stmt = SQL("SELECT * FROM tasks WHERE", Task.done == False)
@@ -56,7 +56,7 @@ task.title = "title"
 
 ## SQL methods on models
 
-Create SQL methods as you would [SQL functions](#sql-functions). Use `@classmethod` and `@staticmethod` decorator when needed.
+Create SQL methods as you would [SQL functions](sql-functions.md). Use `@classmethod` and `@staticmethod` decorator when needed.
 
 ```python
 class Task(Model):
@@ -140,7 +140,7 @@ However, `Model` also exposes easier to use ways to fetch data using without the
 When called out of a transaction context, a non commited transaction will automatically be started. If bound to an engine,
 these class methods can also be called out of a session context.
 
-- `query()` executes the provided statement using [`fetchhydrated()`](#fetching-composite-objects)
+- `query()` executes the provided statement using [`fetchhydrated()`](executing.md#fetching-composite-objects)
 - `find_all()` constructs a select statement based on the provided arguments and executes using `query()`
 - `find_one()` same as `find_all()` but only returns the first row
 - `get()` to find one row by primary key
@@ -160,8 +160,8 @@ with engine:
 > [!TIP]
 > You can also build select statement with auto populated column list and from clause using `Model.select_from()`.
 
-Mapped columns can easily be used as [pieces of composable sql](#sql-utilities): accessing the class attribute representing
-the column returns an [`SQL.Col` object](#manage-list-of-sql-pieces) that can be used with python operators to return
+Mapped columns can easily be used as [pieces of composable sql](sql-utilities.md): accessing the class attribute representing
+the column returns an [`SQL.Col` object](sql-utilities.md#manage-list-of-sql-pieces) that can be used with python operators to return
 sql conditions:
 
 ```python
@@ -182,6 +182,8 @@ Manipulate model objects as you would with any python objects. The following met
 - `refresh()` executes a select statement (same as `get()`) and updates the object attribute values
 - `create()` a class method to create and insert an object in one line
 
+These methods (apart from `create()`) return a boolean indicating if the operation was performed.
+
 > [!NOTE]
 > DML (Data Manipulation Language) statements are the statement that modify data in the database (insert, update and delete mostly)
 
@@ -273,7 +275,7 @@ with engine:
 > You should not disable dirty tracking when allowing unknown columns otherwise setting attributes
 > will not result in them being used in DML statements unless they are mapped.  
 > When dirty tracking is disabled and you are using unknown attributes, the only way sqlorm keeps
-> track of them is through the [`__hydrated_attrs__` attribute](#dehydrating-objects).
+> track of them is through the [`__hydrated_attrs__` attribute](mapper.md#dehydrating-objects).
 
 You can disallow unknown columns
 
@@ -445,4 +447,26 @@ with engine:
 
 ## Column types
 
+Columns can have a type which defines the SQL type and serialization/deserialization functions.
+
+!!! note
+    The goal is not to re-define sql types in python. Types are optional in your column definitions.
+    sqlorm relies mainly on the underlying DBAPI driver to do the conversion. Drivers have custom methods to
+    provide type mapping.
+
+Define types using `SQLType`:
+
+```py
+from sqlorm import SQLType
+import json
+
+JSON = SQLType("json", json.loads, json.dumps)
+
+class MyModel(Model):
+    my_json_column = Column(type=JSON)
+```
+
+The following types are already defines and importable from the `sqlorm` package: 
+`Integer`, `Decimal`, `Varchar`, `Text`, `Boolean`, `Date`, `Time`, `DateTime`, `JSON`, `Pickle`.
 
+Python types from annotations will automatically used one of these type when appropriate (see `sqlorm.types.PYTHON_TYPES_MAP`).

docs/schema.md

@@ -72,13 +72,13 @@ with engine:
 
 SQL files can contain multiple statements.
 
-In python files, use `get_current_session()` to create a transaction context:
+In python files, use `ensure_transaction()` to create a transaction context:
 
 ```python
 # 002_seed_data.py
-from sqlorm import get_current_session
+from sqlorm import ensure_transaction
 
-with get_current_session() as tx:
+with ensure_transaction() as tx:
     tx.execute("...")
 ```
 

docs/sessions-and-transactions.md

@@ -16,7 +16,7 @@ with engine as tx:
     # rollback
 ```
 
-The next section covers [how to use transactions](#executing-queries).
+The next section covers [how to use transactions](executing.md).
 
 > [!NOTE]
 > You can create transactions inside transactions with no impact (only the top-most transaction will commit)
@@ -44,8 +44,8 @@ with engine.session() as sess:
     # rollback
 ```
 
-> [!IMPORTANT]
-> sqlorm does not assume anything regarding thread safety. However, session contexts are scoped to each thread using thread locals.  
+> [!WARNING]
+> Session contexts are scoped to threads using thread locals.  
 > This means that using drivers which are not thread-safe (like sqlite) is not an issue as long as you are using the engine
 > to start sessions. Doing `with engine.session()` and `with engine:` in different threads will start different sessions.
 
@@ -83,17 +83,14 @@ count_tasks() # raises MissingEngineError
 ```
 
 > [!TIP]
-> `Session` and `Transaction` objects can be created using raw DBAPI connection objects
+> `Session` can be created using raw DBAPI connection objects
 >
 > ```python
-> from sqlorm import Session, Transaction
+> from sqlorm import Session
 > import sqlite3
 >
 > conn = sqlite3.connect(":memory:")
 >
-> with Transaction(conn) as tx:
->     # ...
->
 > with Session(conn) as sess:
 >     with sess as tx:
 >         # ...

docs/signals.md

@@ -0,0 +1,93 @@
+# Signals
+
+Signals allow you to listen and react to events emitted by sqlorm. The [blinker](https://blinker.readthedocs.io) lib is used to implement signals.
+
+## Engine
+
+The following signals exist on the `Engine` class. The sender is always the engine instance.
+
+- `connected`: receive `conn` (connection instance) and `from_pool` (bool)
+- `disconnected`: receive `conn` (connection instance) and `close_conn` (bool). This signal may not indicate a connection has been closed but only returned to the pool. Use the `close_conn` kwargs to distinguish.
+
+Example:
+
+```py
+from sqlorm import Engine
+
+@Engine.connected.connect
+def on_connected(engine, conn, from_pool):
+    # do something
+```
+
+## Session
+
+The following signals exist on the `Session` class. The sender is always the session instance.
+
+- `before_commit`
+- `after_commit`
+- `before_rollback`
+- `after_rollback`
+
+Use `session.conn` to retreive the connection object (may be None if no connection have been established, use `session.connect()` in this case).
+
+Example:
+
+```py
+from sqlorm import Session
+
+@Session.before_commit.connect
+def on_before_commit(session):
+    # do something
+```
+
+## Transaction
+
+The following signals exist on the `Transaction` class. The sender is always the transaction instance.
+
+- `before_execute`: receive `stmt` and `params`. Returning a cursor will stop sqlorm execute() and return the cursor directly
+- `before_executemany`: receive `stmt` and `seq_of_parameters`. Returning false will stop sqlorm executemany()
+
+## Model
+
+The following signals exist on the `Model` class. The sender is always the model class.
+
+- `before_query`: receive `stmt` and `params` args. can override them by returning a tuple `(stmt, params)`
+
+The following signals receive `obj` kwargs containing the model instance.
+Returning `False` from any listener will cancel the operation. Returning a value will override the statement.
+
+- `before_refresh`
+- `before_insert`
+- `before_update`
+- `before_delete`
+- `before_save` (receive a `is_new` kwargs, can only cancel operations)
+
+The following signals are sent only if an operation is performed. They receive the `obj` kwargs containing the model instance.
+
+- `after_refresh`
+- `after_insert`
+- `after_update`: receive also `updated` a boolean indicating if an update stmt was actually performed
+- `after_save`
+- `after_delete`
+
+Examples:
+
+```py
+from sqlorm import Model
+
+@Model.before_query.connect
+def on_before_query(model_class, stmt, params):
+    # do something
+
+@Model.before_query.connect_via(MyModel)
+def on_my_model_before_query(model_class, stmt, params):
+    # do something only on before_query of MyModel
+
+@Model.before_insert.connect
+def on_before_insert(model_class, obj, stmt):
+    return False # cancel the insert
+
+@Model.before_insert.connect
+def on_before_insert(model_class, obj, stmt):
+    return SQL.insert(model_class.table, {"col": "value"}) # return custom statement
+```

mkdocs.yml

@@ -12,6 +12,7 @@ nav:
   - models.md
   - sql-utilities.md
   - mapper.md
+  - signals.md
   - schema.md
   - drivers.md
 
@@ -35,6 +36,7 @@ theme:
         name: Switch to light mode
   features:
     - content.action.edit
+    - content.code.copy
     - toc.integrate
   icon:
     edit: material/pencil
@@ -46,7 +48,4 @@ markdown_extensions:
 
 plugins:
   - search
-  - callouts
-  - git-committers:
-      repository: hyperflask/sqlorm
-      branch: main
+  - callouts

pyproject.toml

@@ -10,6 +10,7 @@ packages = [{include = "sqlorm"}]
 
 [tool.poetry.dependencies]
 python = "^3.10"
+blinker = "^1.8.2"
 
 [tool.poetry.group.postgresql.dependencies]
 psycopg = {extras = ["binary"], version = "^3.1.18"}
@@ -23,7 +24,6 @@ pytest-cov = "^4.1.0"
 ruff = "^0.4.3"
 mkdocs-material = "^9.5.24"
 mkdocs-callouts = "^1.13.2"
-mkdocs-git-committers-plugin-2 = "^2.3.0"
 
 [tool.ruff]
 include = ["sqlorm/**/*.py"]