record_outgoung_message fails if the message contains attachments with UTF-8 filenames

[alex-voodoo]

The case is simple: send an email that has an attachment that has Cyrillic filename, then try to feed it into the mailbox. Here is a snippet:

from django.core.mail import EmailMessage

def send(request):
    outgoing_message = EmailMessage('subject', 'body', '[email protected]', ['[email protected]'])
    uploaded_file = request.FILES['attachment']
    outgoing_message.attach(uploaded_file.name, uploaded_file.read(), uploaded_file.content_type)
    outgoing_message.send()
    sent_message = mailbox.record_outgoing_message(outgoing_message.message())

The outgoung message is delivered to the destination address; here are headers of an attachment in one of such test emails that I sent:

Content-Type: image/gif
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename*="utf-8''%D0%A2%D0%B0%D0%BD%D0%BA%D0%B8.gif"

However the django mailbox fails to record the message. The traceback is as follows:

File "/usr/lib/python2.7/dist-packages/django/core/handlers/exception.py", line 39, in inner
response = get_response(request)
File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python2.7/dist-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
return view_func(request, *args, **kwargs)
File "/projects/sample/app/views.py", line 100, in send
sent_message = mailbox.record_outgoing_message(outgoing_message.message())
File "/usr/local/lib/python2.7/dist-packages/django_mailbox/models.py", line 226, in record_outgoing_message
msg = self._process_message(message)
File "/usr/local/lib/python2.7/dist-packages/django_mailbox/models.py", line 354, in _process_message
message = self._get_dehydrated_message(message, msg)
File "/usr/local/lib/python2.7/dist-packages/django_mailbox/models.py", line 240, in _get_dehydrated_message
self._get_dehydrated_message(part, record)
File "/usr/local/lib/python2.7/dist-packages/django_mailbox/models.py", line 264, in _get_dehydrated_message
raw_filename = msg.get_filename()
File "/usr/lib/python2.7/email/message.py", line 687, in get_filename
return utils.collapse_rfc2231_value(filename).strip()
File "/usr/lib/python2.7/email/utils.py", line 318, in collapse_rfc2231_value
return unicode(rawval, charset, errors)
TypeError: decoding Unicode is not supported

I tried to encode the filename of the attachment when adding it to the message, but that does not work at all: such message fails to be sent. It seems to me that using the original filename 'as is' is the right way.

[coddingtonbear]

Hey there, @alex-voodoo -- thank you for taking the time to post a bug report! I'd be glad to have a look, but first, could you post a sample e-mail message?

[alex-voodoo]

Hey @coddingtonbear, I'd be happy to do so, but how? This issue only affects outgoing messages, the incoming ones are accepted just fine and stored in the DB with all Cyrillic-named attachments.

Anyway, I have attached what GMail shows as the original message—that is exactly the one that record_outgoung_message has failed on. I have obfuscated the to and from addresses in the file as well as my local IP, all the rest is untouched.

If that doesn't help, please see again my explanation above of how to reproduce the problem. I believe that all you need is to attach a file with non-Latin filename.
original_msg.txt

[ad-m]

@coddingtonbear , it looks attached message was parsed properly. https://github.com/coddingtonbear/django-mailbox/compare/cyrilic-attachment

[ad-m]

This same errors was received after:

    def test_message_outgoing_cyrilic_constructed(self):
        msg = EmailMessage('subject', 'body', '[email protected]', ['[email protected]'])

        msg.attach('\xd0\xa2\xd0\xb0\xd0\xbd\xd0\xba\xd0\xb8.gif'.decode('utf-8'),
                   self._get_email_as_text('message_with_cyrillic_attachment.eml'),
                   'image/gif')
        self.mailbox.record_outgoing_message(msg.message())

        self.assertEqual(msg.attachments.all().count(), 1)

[coddingtonbear]

I spent a half hour or so trying to understand what's going awry on this one, but I ran out of time for the moment. I'm curious to hear if you had any ideas, @ad-m?