DictionaryField does not implement to_python

[pembo13]

I'm not sure why, but seems to, at least sometimes, cause problems with DictionaryField, for me, specifically during a python manage.py loaddata I had to monkey-patch in a to_python to get my loaddata to work.

import ast
def hstore_dictionaryfield_to_python(self, value):
    if isinstance(value, basestring):
        if len(value) == 0: return {}
        if isinstance(value, str) and value.startswith('{'): return ast.literal_eval(value)
        if isinstance(value, unicode) and value.startswith(u'{'): return ast.literal_eval(value)
        raise ValueError('ivalid dictionary: ' + repr(value))

    return value
hstore.DictionaryField.to_python = hstore_dictionaryfield_to_python

[nemesifier]

Hi @pembo13,

would you be able to specify the steps in order to reproduce this bug?

The best thing would be a failing test case, but even a high level procedure would be good.

Thanks

[nemesifier]

@pembo13 any info?

[pembo13]

Sorry, but I didn't write a proper test, just a test project, and instructions.

Test Project

I don't see any way to attach a file to this issue, so please use this http://arthurpemberton.com/wp-content/uploads/2015/06/hstoretest.20150628.zip

Test Instructions

# 1. install requirements
pip install -U -r requirements.txt
# 2. sync database
python manage.py syncdb
# 3. create test data
python manage.py createtestdata
# 4. dump data
python manage.py dumpdata --format=xml --indent=4 core.PersonInfo > test.xml
# 5. load dumped data (error should occur here)
python manage.py loaddata test.xml
# 6. load dumped data, this time enabling match of DictionaryField in models.py, (should load fine here)
PATCHHSTORE=TRUE python manage.py loaddata test.xml

[nemesifier]

Thank you @pembo13, my question is, wouldn't it have better to include the test in django-hstore's test suite?

See: http://djangonauts.github.io/django-hstore/#_developers_guide

[pembo13]

It is probably better, but it wasn't faster for me at least.

I have written a failing test. However it fails even for simple data, so maybe the problem is worse than I thought.

    def test_dumpandloaddata(self):
        import os
        from django.core import management
        from django.utils import six

        filename = 'test_dumpdata.xml'
        new_io = six.StringIO()

        db_name = 'gamma'
        db_data = { 'a':1, 'b':['be','bee']}

        db = DataBag.objects.create(name=db_name, data=db_data)
        self.assertEqual(DataBag.objects.filter(name=db_name).count(), 1)

        management.call_command('dumpdata', 'django_hstore_tests.DataBag', **{
            'format': 'xml',
            'indent': 4,
            'stdout': new_io,
            'stderr': new_io,
            'output': filename,
        })
        self.assertTrue(os.path.isfile(filename))

        db.delete()
        self.assertEqual(DataBag.objects.filter(name=db_name).count(), 0)

        management.call_command('loaddata', filename)
        self.assertEqual(DataBag.objects.filter(name=db_name).count(), 1)

        if os.path.isfile(filename): os.remove(filename)
        return

The test fails without the patching, but passes with the patch.

Let me know if there's another step you'd like me to take.

[nemesifier]

thank you @pembo13, could you also tell me on which python and django versions you experience the issue?

[pembo13]

I believe I was on Django 1.7 when i first saw the problem. But currently:

• Python 2.7.5
• Django 1.8.2