Support subclassing models for default attributes with alembic migration support similar to sqlalchemy declarative base.

[bubthegreat]

NOTE: This might just be because I couldn't find any documentation on the proper way to do this!

• GINO version: current
• Python version: 3.7.x
• asyncpg version: current
• aiocontextvars version: current
• PostgreSQL version: current

Description

I want to be able to subclass GINO db.Model and have the inheritance still respect alembic migration support.

What I Did

I created a subclass factory from db.Model with the intent of adding some common configurable attributes to the model class (For FastAPI

import logging

from gino.ext.starlette import Gino


from .. import config

LOGGER = logging.getLogger(__name__)

db = Gino(
    dsn=config.DB_DSN,
    pool_min_size=config.DB_POOL_MIN_SIZE,
    pool_max_size=config.DB_POOL_MAX_SIZE,
    echo=config.DB_ECHO,
    ssl=config.DB_SSL,
    use_connection_for_request=config.DB_USE_CONNECTION_FOR_REQUEST,
    retry_limit=config.DB_RETRY_LIMIT,
    retry_interval=config.DB_RETRY_INTERVAL,
)
LOGGER.info("Database initialized: %s", db)


def CRUDModel(table, validator=None):
    class Base(db.Model):

        __tablename__ = table

        id = db.Column(db.BigInteger(), primary_key=True)

        @classmethod
        def _prefix(cls):
            return table

        @classmethod
        def _tags(cls):
            return [table]

        @classmethod
        def _validator(cls):
            return validator

    return Base

This let me add some default routers programmatically:

def add_router(app, model):

    LOGGER.info("Loading model %s.", model)

    router = APIRouter()

    @router.get("/{uid}")
    async def get_item(uid: int):
        instance = await model.get_or_404(uid)
        return instance.to_dict()

    @router.post("/")
    async def add_item(item: model._validator()):
        instance = await model.create(nickname=item.name)
        return instance.to_dict()

    @router.delete("/{uid}")
    async def delete_item(uid: int):
        instance = await model.get_or_404(uid)
        await instance.delete()
        return dict(id=uid)

    app.include_router(
        router,
        prefix=f"/{model._prefix()}",
        tags=model._tags(),
        responses={404: {"description": "Not found"}},
    )

The problem is that alembic doesn't recognize the subclass attributes, so when I try and create this subclass:

class UserValidator(BaseModel):
    first_name: str
    last_name: str


class User(CRUDModel('users', UserValidator)):

    first_name = db.Column(db.Unicode(), default="unnamed")
    last_name = db.Column(db.Unicode(), default="unnamed")

The id field from the CRUDModel Base is the only attribute that's found by alembic's automigrations generation. Either I'm subclassing this wrong, or it's not supported by the db.Model the same way that sqlalchemy allows for (Subclassing a common base class for default attributes).

[wwwjfy]

It's because Base is a model itself (with __tablename__). When User inherits it, it inherits initialized __table__ so Gino doesn't process it again.

One workaround: don't use db.Model as parent class of Base, but in User

def CRUDModel(table):
    class Base:
        __tablename__ = table

        id = db.Column(db.BigInteger(), primary_key=True)

    return Base


class User(db.Model, CRUDModel('users')):
    first_name = db.Column(db.Unicode(), default="unnamed")
    last_name = db.Column(db.Unicode(), default="unnamed")

[bubthegreat]

Sweet - that makes sense. I'll test it a bit and write some docs

…

On Sun, May 31, 2020, 6:51 AM Tony Wang ***@***.***> wrote: It's because Base is a model itself (with __tablename__). When User inherits it, it inherits initialized __table__ so Gino doesn't process it again. One workaround: don't use db.Model as parent class of Base, but in User def CRUDModel(table): class Base(): __tablename__ = table id = db.Column(db.BigInteger(), primary_key=True) return Base class User(db.Model, CRUDModel('users')): first_name = db.Column(db.Unicode(), default="unnamed") last_name = db.Column(db.Unicode(), default="unnamed") — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#687 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC5CHIOICLNUCKVAH7K6UALRUJHGLANCNFSM4NPAILBA> .

[bubthegreat]

Was just trying to solve this a bit more gracefully and found pydantic's dynamic model generation handy. I haven't played much with sqlalchemy directly, but it should be straightforward to implement recursive checking based on model relations, etc. similar to how tortoiseorm has done.

    def get_pydantic_model(cls):
        """Return a pydantic model from the GINO model definition.

        Will check for an __exclude__ property on the class to identify
        class attributes that should be excluded from the pydantic model
        generation.


        Example:

            class User(db.Model):
                __tablename__ = 'users'

                name = db.Column(db.String())
            ​
            PUser = User.get_pydantic_model()

            print(PUser.schema_json(indent=2))
            {
            "title": "User",
            "type": "object",
            "properties": {
                "name": {
                "title": "Name",
                "type": "string"
                }
            },
            "required": [
                "name"
            ]
            }
        """

        keys = [str(key) for key in cls.__dict__.keys()]
        # Assumption that may not be valid, but don't look at ones with _ in them.
        valid_keys = [key for key in keys if not key.startswith('_')]

        # Allow exclusions of model attributes from the pydantic model.
        if hasattr(cls, '__excluded__'):
            valid_keys = [key for key in valid_keys if key not in cls.__excluded__]

        # This may be unique to GINO where the python type is on a column, but
        # It would be easy enough to make a reference table for this rather than
        # pulling it directly from the model column.
        field_definitions = {}
        for key in valid_keys:
            col = getattr(cls, key)
            col_type = col.type.python_type
            # Don't forget ellipses after this, or pydantic won't create
            # the schema and validators properly.
            field_definitions[key] = (col.type.python_type,...)

        # Create our pydantic model
        pmodel = create_model(cls.__name__, **field_definitions)

        return pmodel

I forked the repo and submitted this PR to get some initial feedback on what you'd like it to support for more robust pydantic model creation. Assuming it will need things like recursive searching if the column is relational, etc.

PR for input:
#688