From 7ba85572934b6724c03d66111c38a0b83624c303 Mon Sep 17 00:00:00 2001 From: Victor Petrovykh Date: Tue, 11 Feb 2025 04:42:35 -0500 Subject: [PATCH] ORM docs added to guides. "Using EdgeDB with ..." section now has 4 entries corresponding to SQLAlchemy, SQLModel, Django, and Prisma. --- docs/guides/tutorials/django.rst | 284 ++++++++++++++++++++++++++ docs/guides/tutorials/index.rst | 4 + docs/guides/tutorials/prisma.rst | 213 +++++++++++++++++++ docs/guides/tutorials/sqlalchemy.rst | 293 +++++++++++++++++++++++++++ docs/guides/tutorials/sqlmodel.rst | 272 +++++++++++++++++++++++++ 5 files changed, 1066 insertions(+) create mode 100644 docs/guides/tutorials/django.rst create mode 100644 docs/guides/tutorials/prisma.rst create mode 100644 docs/guides/tutorials/sqlalchemy.rst create mode 100644 docs/guides/tutorials/sqlmodel.rst diff --git a/docs/guides/tutorials/django.rst b/docs/guides/tutorials/django.rst new file mode 100644 index 00000000000..d1319a89b9c --- /dev/null +++ b/docs/guides/tutorials/django.rst @@ -0,0 +1,284 @@ +.. _ref_guide_django: + +====== +Django +====== + +Gel supports SQL protocol for communicating with the database. This makes it possible to use it with Django by matching a Django schema to the Gel schema. You don't even need to figure out the conversion yourself because we have an automated tool for that. + +This tool becomes available when you install ``edgedb`` Python package. Once you have you Gel project setup, you can generate the Django schema with this command: + +.. code-block:: bash + + $ gel-orm django --out models.py + +The ``--out`` indicates the output file for the newly generated Django module. You will also need to include ``'gel.orm.django.gelmodels.apps.GelPGModel'`` into the ``INSTALLED_APPS`` for your Django app. + +Even though Django and Gel both view the schema as a bunch of types with fields and interconnections, there are still some differences between what Gel and Django can represent. + + +Properties +========== + +Property types must match the basic Postgres types supported by Django, so avoid any custom types as they will be skipped. Currently we support the following: + +* :eql:type:`std::uuid` - ``UUIDField`` +* :eql:type:`std::bigint` - ``DecimalField`` +* :eql:type:`std::bool` - ``BooleanField`` +* :eql:type:`std::bytes` - ``BinaryField`` +* :eql:type:`std::decimal` - ``DecimalField`` +* :eql:type:`std::float32` - ``FloatField`` +* :eql:type:`std::float64` - ``FloatField`` +* :eql:type:`std::int16` - ``SmallIntegerField`` +* :eql:type:`std::int32` - ``IntegerField`` +* :eql:type:`std::int64` - ``BigIntegerField`` +* :eql:type:`std::json` - ``JSONField`` +* :eql:type:`std::str` - ``TextField`` +* :eql:type:`std::datetime` - ``DateTimeField`` +* :eql:type:`cal::local_date` - ``DateField`` +* :eql:type:`cal::local_datetime` - ``DateTimeField`` +* :eql:type:`cal::local_time` - ``TimeField`` + +Extreme caution is needed for datetime field, the TZ aware and naive values are controlled in Django via settings (``USE_TZ``) and are mutually exclusive in the same app under default circumstances. + +Array properties are supported for all of the above types as well. + +Multi properties cannot be represented as they have no primary key at all. If you needs to reflect multi properties, consider replacing them with a single array property. + + +Links +===== + +Plain single links are reflected as a ``ForeignKey``. + +Multi links can be represented as link tables in Django schema and used as an implicit intermediary table. Creation and deletion of implicit intermediary table entries works. During creation both ``source`` and ``target`` are specified. While during deletion we rely on Gel's machinery to correctly handle deletion based on the target. + +Django is quite opinionated about the underlying SQL tables. One such important detail is that it requires a table to have a primary key (PK). Therefore, if a link has link properties we cannot reflect it at all because Django single column PK limits the ability to correctly update the link table. + +If you need to include these types of structures, you will need to make them as explicit intermediate objects connected with single links (which by default represent an N-to-1 relationship, so they are multi links in the reverse direction). + +Links with link properties can become objects in their own right: + +.. code-block:: sdl + + type User { + name: str; + # ... + } + + type UserGroup { + name: str; + # ... + + # Replace this kind of link with an explicit object + # multi link members: User; + } + + # this would replace a multi link members + type Members { + required source: UserGroup + required target: User + + # ... possibly additional payload that used + # to be link properties + } + +All links automatically generate the ``related_name`` relationships as well. The name of these back-links takes the format of ``_linkname_SourceName``, which mimics the EdgeQL version of backlinks ``.``. + +Properties +---------- + +The Gel schema declares a few properties: ``name`` for ``User`` and ``UserGroup`` as well as ``body`` for ``Post``. These get reflected as ``TextField`` in the corresponding models. As long as a property has a valid corresponding Django ``Field`` type it will be reflected in this manner. + +Links +----- + +Let's first look at the ``Post`` declaration in Gel. A ``Post`` has a link ``author`` pointing to a ``User``. So the reflected type ``Post`` has a ``ForeignKeys`` ``author`` which targets ``'User'``. + +Each reflected relationship also automatically declares a back-link via ``related_name``. The naming format is ``__``. For the ``author`` link the name of the back-link is ``_author_Post``. + +The ``User`` model has no links of its own just like in the Gel schema. + +``UserGroup`` model has a many-to-many relationship with ``User``. The model declares ``users`` as a ``ManyToManyField`` pointing to ``'User'``. The ``through`` relationship is ``UserGroupUsers``. The rules for ``related_name`` are the same as for ``ForeignKey`` and so ``_users_UserGroup`` is declared to be the back-link. + +App Settings +------------ + +In order to use these generated models in your Django app there are a couple of things that need to be added to the settings (typically found in ``settings.py``). + +First, we must add ``'gel_pg_models.apps.GelPGModel'`` to ``INSTALLED_APPS``. This will ensure that the Gel models are handled correctly, such as making ``id`` and ``gel_type_id`` read-only and managed by Gel. + +Second, we must configure the ``DATABASES`` to include the connection information for our Gel database (using PostgreSQL endpoint). + +Running ``gel instance credentials --json`` command produces something like this: + +.. code-block:: bash + + $ gel instance credentials --json + { + "host": "localhost", + "port": 10715, + "user": "admin", + "password": "h632hKRuss6i9uQeMgEvRsuQ", + "database": "main", + "branch": "main", + "tls_cert_data": "-----BEGIN CERTIFICATE----- <...>", + "tls_ca": "-----BEGIN CERTIFICATE-----<...>", + "tls_security": "default" + } + +So we can use that to create the following ``DATABASES`` entry: + +.. code-block:: python + + DATABASES = { + 'default': { + 'ENGINE': 'django.db.backends.postgresql', + 'NAME': 'main', + 'USER': 'admin', + 'PASSWORD': 'h632hKRuss6i9uQeMgEvRsuQ', + 'HOST': 'localhost', + 'PORT': '10715', + } + } diff --git a/docs/guides/tutorials/index.rst b/docs/guides/tutorials/index.rst index 351fa7cb828..ca2035a7df3 100644 --- a/docs/guides/tutorials/index.rst +++ b/docs/guides/tutorials/index.rst @@ -17,4 +17,8 @@ Using Gel with... chatgpt_bot cloudflare_workers trpc + sqlalchemy + sqlmodel + django + prisma Bun diff --git a/docs/guides/tutorials/prisma.rst b/docs/guides/tutorials/prisma.rst new file mode 100644 index 00000000000..18a10e939d5 --- /dev/null +++ b/docs/guides/tutorials/prisma.rst @@ -0,0 +1,213 @@ +.. _ref_guide_prisma: + +====== +Prisma +====== + +Gel supports SQL protocol for communicating with the database. This makes it possible to use it with Prisma by matching a Prisma schema to the Gel schema. You don't even need to figure out the conversion yourself because we have an automated tool for that. + +This tool becomes available when you install ``edgedb`` JavaScript/TypeScript package. Once you have you Gel project setup, you can generate the Prisma schema with this command: + +.. code-block:: bash + + $ npx @edgedb/generate prisma --file schema.prisma + +The ``--file`` indicates the output file for the newly generated Prisma schema. + +Even though Prisma and Gel both view the schema as a bunch of types with fields and interconnections, there are still some differences between what Gel and Prisma can represent. + + +Properties +========== + +Property types must match the basic Postgres types supported by Prisma, so avoid any custom types as they will be skipped. Currently we support the following: + +* :eql:type:`std::uuid` - ``String @db.Uuid`` +* :eql:type:`std::bigint` - ``Decimal`` +* :eql:type:`std::bool` - ``Boolean`` +* :eql:type:`std::bytes` - ``Bytes`` +* :eql:type:`std::decimal` - ``Decimal`` +* :eql:type:`std::float32` - ``Float`` +* :eql:type:`std::float64` - ``Float`` +* :eql:type:`std::int16` - ``Int`` +* :eql:type:`std::int32` - ``Int`` +* :eql:type:`std::int64` - ``BigInt`` +* :eql:type:`std::json` - ``Json`` +* :eql:type:`std::str` - ``String`` +* :eql:type:`std::datetime` - ``DateTime`` + +Array properties are supported for all of the above types as well. + +Multi properties cannot be represented as they have no primary key and therefore rows cannot be uniquely identified. That means that the schema generator will omit them from the schema. If you needs to reflect multi properties, consider replacing them with a single array property. + + +Links +===== + +Plain single links are reflected as a relation. + +Multi links get represented as a many-to-many relationship with an implicit intermediary table. + +Prisma is quite opinionated about the underlying SQL tables. It has a strict naming requirement for implicit link tables (they **must** start with an ``_``). This means that the way Gel exposes link tables is incompatible with the implicit naming requirement. So multi links and links with link properties have to reflected as explicit intermediate objects in a Prisma schema. These intermediary objects have ``source`` and ``target`` relations to the end points of the link. The link properties (if any) then become the fields of this link object. + +All links automatically generate the backward relations as well. The name of these back-links takes the format of ``bk_linkname_SourceName``, which mimics the EdgeQL version of backlinks ``._``. + +Properties +---------- + +The Gel schema declares a few properties: ``name`` for ``User`` and ``UserGroup`` as well as ``body`` for ``Post``. These get reflected as ``String`` in the corresponding models. As long as a property has a valid corresponding Prisma field type it will be reflected in this manner. + +Links +----- + +Let's first look at the ``Post`` declaration in Gel. A ``Post`` has a link ``author`` pointing to a ``User``. So the reflected type ``Post`` has a field ``author_id`` and the corresponding relation ``author``. + +Each reflected relation also automatically declares a back-link. In order to correctly map links and back-links every relation needs a name. We simply use the name of the back-link as the name of the relation. The naming format is ``bk__``. For the ``author`` link the name of the back-link is ``bk_author_Post`` and so is the name of the relation. + +We can look at the ``User`` model and find ``bk_author_Post`` relation used as a back-link of the same name. This relation is pointing back to ``Post[]``. + +``User`` model also has a many-to-many relationship with ``UserGroup``. Both ``User`` and ``UserGroup`` are connected by the ``UserGroup_users`` model. The relation names for ``UserGroup`` is the same as in the original Gel schema - ``users``. On the other hand the ``User`` model follows the back-link naming convention for this relation - ``bk_users_UserGroup``. + +Finally, ``UserGroup_users`` model has the last part of the many-to-many relationship declaration. The ``source`` relation pointing to ``UserGroup`` and the ``target`` relation pointing to ``User``. + +Connection +---------- + +In order to use these generated models in your Prisma app you need to setup the ``DATABASE_URL``. Typically this is done in the ``.env`` file. + +Running ``gel instance credentials --insecure-dsn`` command produces something like this: + +.. code-block:: bash + + $ gel instance credentials --insecure-dsn + edgedb://admin:h632hKRuss6i9uQeMgEvRsuQ@localhost:10715/main + +All we have to do is replace ``edgedb`` protocol with ``postgresql`` and add the following to ``.env``: + +.. code-block:: + + DATABASE_URL="postgresql://admin:h632hKRuss6i9uQeMgEvRsuQ@localhost:10715/main" diff --git a/docs/guides/tutorials/sqlalchemy.rst b/docs/guides/tutorials/sqlalchemy.rst new file mode 100644 index 00000000000..b260af2c7ae --- /dev/null +++ b/docs/guides/tutorials/sqlalchemy.rst @@ -0,0 +1,293 @@ +.. _ref_guide_sqlalchemy: + +========== +SQLAlchemy +========== + +Gel supports SQL protocol for communicating with the database. This makes it possible to use it with SQLAlchemy by matching a SQLAlchemy schema to the Gel schema. You don't even need to figure out the conversion yourself because we have an automated tool for that. + +This tool becomes available when you install ``edgedb`` Python package. Once you have you Gel project setup, you can generate the SQLAlchemy schema with this command: + +.. code-block:: bash + + $ gel-orm sqlalchemy --mod sqlaschema --out sqlaschema + +The ``--mod`` is required and specifies the name of the root Python module that will be generated (the name will be referenced in the generated code). The ``--out`` indicates the output directory (which will be created if it doesn't exist) for the newly generated module. + +Even though SQLAlchemy and Gel both view the schema as a bunch of types with fields and interconnections, there are still some differences between what Gel and SQLAlchemy can represent. + + +Properties +========== + +Property types must match the basic Postgres types supported by SQLAlchemy, so avoid any custom types as they will be skipped. Currently we support the following: + +* :eql:type:`std::bool` +* :eql:type:`std::str` +* :eql:type:`std::int16` +* :eql:type:`std::int32` +* :eql:type:`std::int64` +* :eql:type:`std::float32` +* :eql:type:`std::float64` +* :eql:type:`std::uuid` +* :eql:type:`std::bytes` +* :eql:type:`cal::local_date` +* :eql:type:`cal::local_time` +* :eql:type:`cal::local_datetime` +* :eql:type:`std::datetime` + +Array properties are supported for all of the above types as well. + +Multi properties do not have a uniqueness condition that would be true for every row in an exposed SQL table, so they cannot be properly reflected into a SQLAlchemy schema. That means that the schema generator will omit them from the schema. If you needs to reflect multi properties, consider replacing them with a single array property. + + +Links +===== + +Plain single links are reflected as a relationship. + +Multi links get represented as a many-to-many relationship with an implicit intermediary table. + +Links that have link properties are reflected as intermediary objects with a ``source`` and ``target`` relationships to the end points of the link. The link properties then become the fields of this link object. + +All links automatically generate the ``back_populates`` relationships as well. The name of these back-links takes the format of ``_linkname_SourceName``, which mimics the EdgeQL version of backlinks ``.__table``. + +Finally, the file containing SQLAlchemy models is ``default.py`` (named after the ``default`` Gel module). It contains ``Post``, ``User``, and ``UserGroup`` model declarations. + +Let's start with what all models have in common: ``id`` and ``gel_type_id``. They refer to the unique object ``id`` and to the ``__type__.id`` in the Gel schema. These two UUID fields are managed automatically by Gel and should not be directly modified. Effectively they are supposed to be treated as read-only fields. + +Properties +---------- + +The Gel schema declares a few properties: ``name`` for ``User`` and ``UserGroup`` as well as ``body`` for ``Post``. These get reflected as string mapped columns in the corresponding SQLAlchemy models. As long as a property has a valid corresponding SQLAlchemy type it will be reflected in this manner. + +Links +----- + +Let's first look at the ``Post`` declaration in Gel. A ``Post`` has a link ``author`` pointing to a ``User``. So the reflected class ``Post`` has a ``ForeignKey`` ``author_id`` and the corresponding relationship ``author``. + +Note that the ``author`` relationship is annotated with ``orm.Mapped['projschema.default.User']``. This annotation uses the value passed as ``--mod`` in order to correctly specify the type for the ``author`` relationship. + +Each reflected relationship also automatically declares a back-link via ``back_populates``. The naming format is ``__``. For the ``author`` link the name of the back-link is ``_author_Post``. + +We can look at the ``User`` model and find ``_author_Post`` relationship pointing back to ``'projschema.default.Post'`` and using ``author`` as the ``back_populates`` value. + +``User`` model also has a many-to-many relationship with ``UserGroup``. Since in the Gel schema that is represented by the multi link ``users`` that originates on the ``UserGroup`` type, the ``User`` end of this relationship is a back-link and it follows the back-link naming convention. The relationship is ``_users_UserGroup`` and in addition to ``back_populates`` it also declares the other endpoint as ``'projschema.default.UserGroup'`` and secondary link table (from ``_tables.py``) being used. + +Finally, ``UserGroup`` model has the other half of the many-to-many relationship declaration. It has the same name as the Gel schema: ``users``. Otherwise it mirrors the relationship declaration for ``_users_UserGroup``. diff --git a/docs/guides/tutorials/sqlmodel.rst b/docs/guides/tutorials/sqlmodel.rst new file mode 100644 index 00000000000..8d374e7813a --- /dev/null +++ b/docs/guides/tutorials/sqlmodel.rst @@ -0,0 +1,272 @@ +.. _ref_guide_sqlmodel: + +======== +SQLModel +======== + +Gel supports SQL protocol for communicating with the database. This makes it possible to use it with SQLModel by matching a SQLModel schema to the Gel schema. You don't even need to figure out the conversion yourself because we have an automated tool for that. + +This tool becomes available when you install ``edgedb`` Python package. Once you have you Gel project setup, you can generate the SQLModel schema with this command: + +.. code-block:: bash + + $ gel-orm sqlmodel --mod sqlmschema --out sqlmschema + +The ``--mod`` is required and specifies the name of the root Python module that will be generated (the name will be referenced in the generated code). The ``--out`` indicates the output directory (which will be created if it doesn't exist) for the newly generated module. + +Even though SQLModel and Gel both view the schema as a bunch of types with fields and interconnections, there are still some differences between what Gel and SQLModel can represent. + + +Properties +========== + +Property types must match the basic Postgres types supported by SQLModel, so avoid any custom types as they will be skipped. Currently we support the following: + +* :eql:type:`std::bool` +* :eql:type:`std::str` +* :eql:type:`std::int16` +* :eql:type:`std::int32` +* :eql:type:`std::int64` +* :eql:type:`std::float32` +* :eql:type:`std::float64` +* :eql:type:`std::uuid` +* :eql:type:`std::bytes` +* :eql:type:`cal::local_date` +* :eql:type:`cal::local_time` +* :eql:type:`cal::local_datetime` +* :eql:type:`std::datetime` + +Array properties are supported for all of the above types as well. + +Multi properties do not have a uniqueness condition that would be true for every row in an exposed SQL table, so they cannot be properly reflected into a SQLModel schema. That means that the schema generator will omit them from the schema. If you needs to reflect multi properties, consider replacing them with a single array property. + + +Links +===== + +Plain single links are reflected as a relationship. + +Multi links get represented as a many-to-many relationship with an implicit intermediary table. + +Links that have link properties are reflected as intermediary objects with a ``source`` and ``target`` relationships to the end points of the link. The link properties then become the fields of this link object. + +All links automatically generate the ``back_populates`` relationships as well. The name of these back-links takes the format of ``_linkname_SourceName``, which mimics the EdgeQL version of backlinks ``.__table``. + +Finally, the file containing SQLModel models is ``default.py`` (named after the ``default`` Gel module). It contains ``Post``, ``User``, and ``UserGroup`` model declarations. + +Let's start with what all models have in common: ``id`` and ``gel_type_id``. They refer to the unique object ``id`` and to the ``__type__.id`` in the Gel schema. These two UUID fields are managed automatically by Gel and should not be directly modified. Effectively they are supposed to be treated as read-only fields. + +Properties +---------- + +The Gel schema declares a few properties: ``name`` for ``User`` and ``UserGroup`` as well as ``body`` for ``Post``. These get reflected as ``str`` fields in the corresponding models. As long as a property has a valid corresponding SQLModel ``Field`` type it will be reflected in this manner. + +Links +----- + +Let's first look at the ``Post`` declaration in Gel. A ``Post`` has a link ``author`` pointing to a ``User``. So the reflected class ``Post`` has a UUID ``Field`` ``author_id`` which is a foreign key. There is also the corresponding ``author`` ``Relationship``. The target type of ``author`` is annotated to be ``'User'``. + +Each reflected ``Relationship`` also automatically declares a back-link via ``back_populates``. The naming format is ``__``. For the ``author`` link the name of the back-link is ``_author_Post``. + +We can look at the ``User`` model and find ``_author_Post`` ``Relationship`` pointing back to ``list['Post']`` and using ``author`` as the ``back_populates`` value. + +``User`` model also has a many-to-many relationship with ``UserGroup``. Since in the Gel schema that is represented by the multi link ``users`` that originates on the ``UserGroup`` type, the ``User`` end of this relationship is a back-link and it follows the back-link naming convention. The relationship is ``_users_UserGroup`` and in addition to ``back_populates`` it also declares the other endpoint as ``list['UserGroup']`` and the ``link_model`` ``UserGroup_users_table`` from ``_tables.py`` is used. + +Finally, ``UserGroup`` model has the other half of the many-to-many relationship declaration. It has the same name as the Gel schema: ``users``. Otherwise it mirrors the ``Relationship`` declaration for ``_users_UserGroup``.