Not writing to Postgres database

[dustinmichels]

I think this library is very slick and exactly what I wanted, but unfortunately I can't get it to write computed fields to my database.

I'm using Django 2.2 and a Postgres database.

I added some ComputedTextField fields to my model as described in the documentation.

In the Django Admin console, I am able to display the computed values in both the list display & 'edit model' display, and I can see they are being computed properly, but the computed values never get saved to my database.

Do you have any ideas about what could be going on?

[brechin]

It hasn't been tested with Django 2.2 yet. I'll take a look to see if I can tell what's going on.

[dustinmichels]

Any success with this? Is it working for you?

[brechin]

I was running into trouble adding test support for various databases and updating for newer python and Django versions and then I lost track of this. Sorry about the delay.

Looking at the initial test runs, it looks like the combination of Python 3.6, Django 2.2, and Postgres passes all the tests. Have you saved/re-saved the objects you're looking at? The computed values aren't persisted until the object is saved after defining the computed field.

This caveat is not called out clearly in the docs, so I'll add that.

[brechin]

@dustinmichels I think the issue was with documentation, and I've added new content to cover the scenario. If you're experiencing other issues, please re-open with more details.

[dustinmichels]

Greetings, sorry it took me a while to reply.

I started testing things out again. The package works great for a simple test case. For my real use case I'm still having problems, though I realize my requirements are a little complex and the issues may lie mostly with the use of Django admin inlines and thus may not constitute a real "bug".

I'm going to try to describe the issue, to see if you have any ideas about what's going on and if it could be addressed within this package. But if you decide this is a specific issue outside the scope of this project, I understand!

Minimal Reproducible Example

Here is one situation I have (simplified a bit): I have a Job model and an Operation model, such that a job can have one-or-more operations (ie, job_id is a foreign key on Operation.)

I have computed fields on Job, which I want to be populated by info from its constituent Operations.

For example, say "Job 1" has two operations , "Op 1" and "Op 2." Each operation has a location_type-- say Op 1 takes place in a "house" and "Operation 2" takes place in a "field". In my context, it is reasonable to say the location_type for Job 1 is "House + Field." So my code looks something like this:

myapp/models.py

import computed_property


class Job(models.Model):
    location_type = computed_property.ComputedTextField(
        compute_from='get_location_type', null=True, blank=True)

    def get_location_type(self):
        qs = self.operations.values_list('location_type')
        return " + ".join([x[0] for x in qs])


class Operation(models.Model):
    job = models.ForeignKey(
        Job, related_name="operations", on_delete=models.CASCADE)
    location_type = models.CharField(max_length=255, null=True, blank=True)

This works as expected if I create the objects thru the Django shell. Ie, ./manage.py shell:

from myapp.models import Job, Operation

j1 = Job()
j1.save()

o1 = Operation()
o1.job_id = 1
o1.location_type = "house"
o1.save()

o2 = Operation()
o2.job_id = 1
o2.location_type = "field"
o2.save()

j1.save()
j1.location_type  # -> output: 'field + house'

I start to run into trouble when I set up Django Admin to edit the models, particularly when using inlines. I want to use inlines so that someone can attach / edit Operations from the Job edit page. The basic example is like this:

myapp/admin.py

from django.contrib import admin

from polls.models import Job, Operation


class OperationInline(admin.TabularInline):
    model = Operation
    extra = 0


@admin.register(Job)
class JobAdmin(admin.ModelAdmin):
    inlines = [OperationInline]

It produces an admin "Job" page that looks like this:

If I edit Operation details "inline" on the Job page, I have to save twice to see Job update. So I'm starting to run into some challenges with signals, but it still somewhat works.

Full setup details

# create project
django-admin startproject mysite
cd mysite
python manage.py startapp myapp

# Create a virtual environment
python3 -m venv env
source env/bin/activate  # On Windows use `env\Scripts\activate`

# install dependencies
pip install django django-computed-property psycopg2-binary

Real App

In my real app, which is quite a bit more complex, I can't get the computed fields to write to the database at all. I can't seem to reproduce this issue fully in my MRE-- but I suspect it's still related to that issue of having to save twice.

I wonder if you have any ideas about what is going on, or how to debug issues with signals. Again if this is outside the scope of this package, I totally understand.

Thanks

[brechin]

@dustinmichels Thanks for revisiting this! I'm guessing the issue with the inlines is that there's not a natural "save" of the Job after the Operations are added. This gets to the core of a TODO issues I have: #1 which I do want to tackle at some point--automatically updating an object when a relevant related object is updated. Another user referenced another library that might be of use if you need something right away.

You mention that you "can't get the computed fields to write to the database at all" in your real app, and that's confusing to me. I can't really imagine why that would happen since the database itself shouldn't really be a concern for these fields.

Is the computed field using a property or some other sort of computation? Are the issues you're having from Django admin interactions or more "normal" ORM usage? Are you using .save(update_fields) or signals in some way that might interfere with the settings? If you explicitly .save() the object is the data still not persisted to the DB?

[dustinmichels]

Thanks for replying!

Looks like if I explicitly call j1.save() the computed database fields do get updated, I just can't get them to update through the admin interface.

In my real app, I'm not only using inlines but also nested inlines and Grappelli, which adds lots of messiness to the admin interface.

I'm not messing with the signals for Job or Operation besides what those libraries might be doing.

[jam-lock]

I was having a similar issue while trying to (re-)save my models to update the pgSQL database.

While on development using sqLite I had no problem to save the models, but on the production server using pgSQL I couldn't make it update the computed field. Looking at the database rows, the computed field was always NULL.

Than I passed the argument 'update_fields' to the save function and it worked!

for instace in MyModel.objects.all().iterator():
instance.save(update_fields=["my_computed_field"])

[brechin]

@jam-lock If you're manually updating data using SQL statements in a database shell, it is not expected that it would compute and store the computed fields' values. This library depends on the Django ORM to be used to update those values when the object is saved. I would like to tackle the issue more gracefully (related issue: #1). These inlined admin interfaces are certainly common and present this issue.

[jam-lock]

In the end, I just had to override the save method of the related field

class ClientStatus(models.Model):

    STATUS_CHOICES = (
        ('--', '-----'),
        ('P', 'Em prospecção'),
        ('D', 'Diagnóstico'),
        ('R', 'Restituido'),
        ('C', 'Contratado'),
    )

    cliente = models.ForeignKey(Client, on_delete=models.CASCADE)
    status_choice = models.CharField(choices=STATUS_CHOICES, default="--", max_length=8)
    data_alteracao = models.DateField(default=timezone.now)
 
    def save(self, *args, **kwargs):
        super(ClientStatus, self).save(*args, **kwargs)
        self.cliente.save(update_fields=['status'])