Fail if Related can't find an existing record.

[sjmh]

I'd like to be able to tell the Related field that it should fail if it can't find an existing parent record. This is for when a user doesn't have the ability to add parent records and I want to let them know that the record they are specifying does not exist yet.

Something like this in Related's deserialize method?

        try:
            result = self._get_existing_instance(
                self.session.query(self.related_model), value
            )
        except NoResultFound:
            # The related-object DNE in the DB, but we still want to deserialize it
            # ...perhaps we want to add it to the DB later
            if self.create:
                return self.related_model(**value)
            raise ValidationError("Related record does not exist")
        return result

At the moment, I have to accomplish this via a @validates() decorator in the schema that performs this check, like so:

class WorkflowSchema(marsh.ModelSchema):
    class Meta:
        ordered = True
        model = Workflow
        strict = True
        dump_only = ("id", "created_at", "updated_at")
        exclude = ("jobs",)

    team = Related(column="name")
    workflow_template = Related(column="name")
    parameters = fields.Dict()

    @validates("workflow_template")
    def validate_workflow_template(self, input):
        name = input.name
        try:
            WorkflowTemplate.query.filter_by(name=name).one()
        except NoResultFound:
            raise ValidationError(f"Workflow template '{name}' does not exist")

Thoughts?

[mraymond77]

I for one think this is definitely a needed feature.

[AbdealiLoKo]

FWIW, I just made a Validator function:

    def not_transient_model_object(value):
        """
        Validate if value is a model instance existing in the DB.
        i.e. Throw an error if model does not exist in DB
        """
        if isinstance(value, (list, tuple)):
            invalid_vals = sum(val not in db.session for val in value)
            if invalid_vals > 0:
                raise ValidationError('{} of the given {} values do not exist'.format(invalid_vals, len(value)))
        else:
            if value not in db.session:
                raise ValidationError('Given value does not exist')

And I use it in my metaclass that I extended (something like this):

class BaseModelSchemaMeta(type(ModelSchema)):

    @staticmethod
    def _add_related_field_validator(field_obj):
        # If `allow_creation` is not given as True - don't allow creation of an object in a relation
        if not field_obj.metadata.get('allow_creation', False):
            field_obj.validators.append(not_transient_model_object)

    def get_declared_fields(cls, klass, cls_fields, inherited_fields, dict_cls):
        declared_fields = super(type(ModelSchema), cls).get_declared_fields(
            klass, cls_fields, inherited_fields, dict_cls)
        for fname, fvalue in declared_fields.items():
            if isinstance(fvalue, Related):
                cls._add_related_field_validator(fvalue)
            elif isinstance(fvalue, fields.List) and isinstance(fvalue.container, Related):
                cls._add_related_field_validator(fvalue.container)

This way I can write the field with the kwarg allow_creation=True when I need that.
Note that the function can be used as a validate= arg too - I wanted this to be the default behaviour hence had to extend the metaclass.

It would be nice to have support for this inside the library though

[mraymond77]

@AbdealiJK could you give an example in how to implement your code? I also want creating related objects in the database disabled by default. I'm running into an error and cant figure out where the issue is. It's happening when instantiating the schema on the last line.

from marshmallow_sqlalchemy.fields import Related, RelatedList
from marshmallow_sqlalchemy.compat import with_metaclass

from rdns_app.extensions import ma, db
from rdns_app.models import GeoLocationGroup


def not_transient_model_object(value):
    """Validate if value is a model instance existing in the DB.
    i.e. Throw an error if model does not exist in DB
    """
    if isinstance(value, (list, tuple)):
        invalid_vals = sum(val not in db.session for val in value)
        if invalid_vals > 0:
            raise ValidationError('{} of the given {} values do not exist'.format(invalid_vals, len(value)))
    else:
        if value not in db.session:
            raise ValidationError('Given value does not exist')


class BaseModelSchemaMeta(type(ma.ModelSchema)):

    @staticmethod
    def _add_related_field_validator(field_obj):
        # If `allow_creation` is not given as True - don't allow creation of an object in a relation
        if not field_obj.metadata.get('allow_creation', False):
            field_obj.validators.append(not_transient_model_object)

    @classmethod
    def get_declared_fields(cls, klass, cls_fields, inherited_fields, dict_cls):
        declared_fields = super(type(ma.ModelSchema), cls).get_declared_fields(
            klass, cls_fields, inherited_fields, dict_cls)
        for fname, fvalue in declared_fields.items():
            if isinstance(fvalue, Related):
                cls._add_related_field_validator(fvalue)
            elif isinstance(fvalue, fields.List) and isinstance(fvalue.container, Related):
                cls._add_related_field_validator(fvalue.container)


class GeoLocationGroupSchema(with_metaclass(BaseModelSchemaMeta, ma.ModelSchema)):
    class Meta:
        model = GeoLocationGroup
        fields = ('name', 'geolocations')
        sqla_session = db.session
    name = ma.String(required=True)
    geolocations = RelatedList(Related(attribute="geo_locations", column='name'),
                               attribute='geo_locations', allow_creation=False)

geo_location_group_schema = GeoLocationGroupSchema()

Traceback:

  File "/Users/me/localdev/rdns/server/rdns_app/api/geolocationgroups/schemas.py", line 75, in <module>
    geo_location_group_schema = GeoLocationGroupSchema()
  File "/Users/me/.local/share/virtualenvs/server-JUnAAj2o/lib/python3.7/site-packages/marshmallow_sqlalchemy/schema.py", line 168, in __init__
    super(ModelSchema, self).__init__(*args, **kwargs)
  File "/Users/me/.local/share/virtualenvs/server-JUnAAj2o/lib/python3.7/site-packages/marshmallow/schema.py", line 369, in __init__
    self.fields = self._init_fields()
  File "/Users/me/.local/share/virtualenvs/server-JUnAAj2o/lib/python3.7/site-packages/marshmallow/schema.py", line 950, in _init_fields
    field_obj = self.declared_fields.get(field_name, ma_fields.Inferred())
AttributeError: 'NoneType' object has no attribute 'get'
...

For some reason self.declared_fields is None. Did I subclass things incorrectly?

[AbdealiLoKo]

I had written that code for a very specific marshmallow version - one of the 3.0 beta series 3.0b12 I think?
Possibly the marshmallow API has changed after that or you're using marshmallow 2.x ?

[mraymond77]

I'm using 3.0.0rc6

[mraymond77]

Sorry if that was somewhat of a hijacking of the thread. Back on subject, I still think this would be a valuable addition to the library.

[The-Gopher]

Can anyone think of a case you wouldn't want it to fail? I feel like should be a default behavior.

[sloria]

i'd be open to adding something like a create parameter that defaults to True (for backwards compatibility). if False and the related object DNE, a ValidationError is raised.

PRs welcome!