Does not send signed emails.

[Euphorbium]

Sending encrypted mails work. I assume that all emails sent using 'secure_mail.backends.EncryptingSmtpEmailBackend' should be signed, if the to key does not exist.

[blag]

Are you specifying a true-ish value for SECURE_MAIL_SIGNING_KEY_FINGERPRINT and SECURE_MAIL_SIGNING_KEY_DATA?

[Euphorbium]

Why does SECURE_MAIL_SIGNING_KEY_DATA matter? The fingerprint points to a key that is in the keyring, that should be enough. Also, it can send encrypted messages, and both the sender and recipient keys are used in encryption I suppose.
I thought that SECURE_MAIL_SIGNING_KEY_DATA was only necessary to generate keys?

[Euphorbium]

I have checked the code. It is not possible to send signed emails with this package, why was this even in the docs? Maybe this function was removed at some point and the docs were not updated?

[blag]

Sorry it's been awhile since I've touched this code. You're correct: SECURE_MAIL_SIGNING_KEY_DATA does not matter for this; it is only used to generate keys. However, SECURE_MAIL_SIGNING_KEY_FINGERPRINT does need to be set to a true-ish value for signing to work.

I noted this in the documentation but it's really easy to overlook: this package uses the python-gnupg package from Vinay Sajip (I'll use PGPG as shorthand for this), which is different than the more widely known gnupg package by Isis Lovecruft. This can be confusing, because gnupg is an old fork of python-gnupg and has a slightly different yet very similar API. Please ensure that you have installed the correct library from PyPI. All of this is relevant below.

I have checked the code. It is not possible to send signed emails with this package, why was this even in the docs? Maybe this function was removed at some point and the docs were not updated?

This package absolutely does sign outgoing mail using the signing key specified by SECURE_MAIL_SIGNING_KEY_FINGERPRINT - IF that value is not empty. This is how it works:

1. SECURE_MAIL_SIGNING_KEY_FINGERPRINT gets converted to SIGNING_KEY_FINGERPRINT
2. SIGNING_KEY_FINGERPRINT get stuffed into the encrypt_kwargs dictionary as the value for the sign key
3. encrypt_kwargs is packed into keyword arguments and gets passed to GnuPG's encrypt function when we define our own internal encrypt function (this internal function is used because it has a pretty bulletproof signature: (cleartext, to_address_object) so idiots like me don't screw up)
4. Our internal encrypt function is called in three places: in encrypt_attachment and twice in encrypt_messages here and here

Now, about how we use python-gnupg's encrypt function. PGPG has encrypt and sign functions that rather obviously encrypt and sign content, respectively. However, the encrypt function can also take a sign parameter, the value of which should be a string specifying the fingerprint of the key to use. According to the section on Using Signing and Encryption Together:

If you want to use signing and encryption together, use the following approach:

>>> encrypted_data = gpg.encrypt(data, recipients, sign=signer_fingerprint, passphrase=signer_passphrase)

The resulting encrypted data contains the signature. When decrypting the data, upon successful decryption, signature verification is also performed (assuming the relevant public keys are available at the recipient end).

Circling back to your original assumption:

I assume that all emails sent using secure_mail.backends.EncryptingSmtpEmailBackend should be signed, if the to key does not exist.

This is an incorrect assumption, although not necessarily a bad one. This package does not sign mail to recipients without encryption keys. This package does sign mail to recipients with encryption keys however.

As to that: I'm not convinced that signing everything is really all that useful or necessary. Almost all mail nowadays is encrypted in transit, so a third party modifying mail in flight is pretty much no longer possible. You and your sender will both need to trust your SMTP gateway and the recipients' mail service to not modify the content of the email. You as the developer have a choice for who you pick as your SMTP provider, and to my knowledge none of the major ones have a reputation for modifying the content of mail in any meaningful way (eg: some add special/custom headers, but they can and do do that even if you sign and encrypt the content). You as the developer will not be able to predict or control the recipient's services, so that responsibility is really in their hands. And if they don't trust their provider to not modify their mail, they have probably already given you an encryption key. And even then, they still have to trust their mail service to not simply deny receiving mail. So no matter what, a recipient has to trust their mail service at some point.

This means that the remaining use case for sending signed but unencrypted mail is really very small. Your SMTP provider and the recipients' mail service can still trivially strip the signature and modify the contents at will - most people don't check that mail is signed at all, much less check signatures of received mail, and those that do already know what digital signing is, expect mail to be encrypted, and very likely already have encryption keys they can use.

So to me, the two main use cases are:

1. Recipient doesn't have an encryption key -> send the message unmangled
2. Recipient has an encryption key -> sign and encrypt message

Use case # 1 probably already covers 99% of users, and both together probably cover 99.5%, so adding the third use case

3. Recipient doesn't have an encryption key -> sign the message and send it

doesn't really make sense to me, but I am still open to discussion, and I'm happy to answer any further questions you may have.

Furthermore, your point about documentation is valid. At the very least, this should be documented. I was just laid off (while on vacation 😆), so I'll get to this in the next few days. I'll keep this issue open until I do. Thanks!

[blag]

@Euphorbium And since you seem to be interested in this project, if you see anything else that needs documentation or fixing up, don't hesitate to point it out! I'll have some time to devote to this later this week, so queue up anything you want me to do.

[Euphorbium]

Thanks for extensive reply. SECURE_MAIL_SIGNING_KEY_FINGERPRINT is always going to be empty, since in the docs only SECURE_MAIL_KEY_FINGERPRINT is mentioned, which is a bug.

Also, about the trust. It is not about me trusting the smtp provider. Is about the receiver NOT having to trust anybody in between me as a sender, and him as a receiver. They don't have to inspect emails to see which provider sent the email, or know anything about that. What if my receiver runs their own service? There are use cases where this is necessary, and I need to send signed unencrypted emails. It increases the trust the receiver has in my project, since he does not have to waste his trust on the middle men.

You can check my fork of this project, I've started work on SigningSmtpEmailBackend and kind of don't get what is going on in the handlers.py

[blag]

Thanks for extensive reply. SECURE_MAIL_SIGNING_KEY_FINGERPRINT is always going to be empty, since in the docs only SECURE_MAIL_KEY_FINGERPRINT is mentioned, which is a bug.

What idiot documented this crap!? Oh right, it was me. Derp. 😆

Is about the receiver NOT having to trust anybody in between me as a sender, and him as a receiver.

My point was that they do have to trust your SMTP gateway and their mail service, no matter what. The recipient has to - at minimum - trust that your SMTP provider and their mail service is actually delivering mail. If either of those didn't deliver mail then the recipient wouldn't have any way of knowing, and the signature would be worthless.

Furthermore, signing without encrypting is still trivially broken - your SMTP provider or the recipient's mail service can simply strip the signature off the message entirely, then modify the content at will. And the recipient will never know the difference. ¹

What if my receiver runs their own service?

Then they don't need signed messages to begin with - they simply use DKIM and SPF to verify that mail that claims to originate from you actually did originate from you. Done.

Furthermore, if there is a non-negligible number of users running their own mail services I will be entirely surprised and impressed. Securely running mail servers is an subtly incredibly non-trivial task.

---

About handlers.py. That code works some magic that's kinda cool (if I do say so myself).

You have to think about how encryption failures should be handled - some sites will only encrypt mail to their admins (because sometimes mail contains sensitive information), but some sites may allow normal users to upload their PGP keys. PGP keys almost always expire, or they can be too short to be useful, or they can be malicious. The point is that encryption can fail for a multitude of reasons, and different sites will need to handle this failure in different ways.

And it gets even more complicated than that - by default, Django will email the site admins when exceptions are raised. But what happens if an admin's encryption keys expires? When Django tries to mail the admin, this package will attempt to encrypt using their expired key, the encryption will fail, the default failure handlers will raise exceptions, so Django tries to email the admin, this package attempts to encrypt using the same expired key, which fails, etc., - an error loop happens.

Sometimes sites will need to notify admins whether or not the message is encrypted, but maybe sometimes they don't want to because the message contents are too sensitive - they can simply log it. ² Sites need flexibility in how to handle encryption failures, so that's why encryption failure handlers are user-configurable.

The first part of this module is pretty boring - it's simply defining default handlers for when encryption fails. The default handlers are exceptionally boring (pun very much intended): they just re-raise the exception.

def default_handle_failed_encryption(exception):
    """
    Handle failures when trying to encrypt content for messages
    """
    raise exception


def default_handle_failed_alternative_encryption(exception):
    """
    Handle failures when trying to encrypt alternative content for messages
    """
    raise exception


def default_handle_failed_attachment_encryption(exception):
    """
    Handle failures when trying to encrypt attachment content for messages
    """
    raise exception

The "second" part is defining functions that I thought might be useful for users if they write their own failure handlers:

def get_variable_from_exception(exception, variable_name):
    ...

def force_mail_admins(unencrypted_message, address):
    ...

def force_delete_key(address):
    ...

def force_send_message(unencrypted_message):
    ...

The third part is cool though:

def import_function(key):
    mod, _, function = FAILURE_HANDLERS[key].rpartition('.')
    mod = import_module(mod)
    return getattr(mod, function)

That simply imports failure handling functions from their import path string. The path strings are the values in FAILURE_HANDLERS specified by key. Users can configure secure_mail to use their own failure handlers by specifying them in the SECURE_MAIL_FAILURE_HANDLERS setting dictionary...which is not documented anywhere. I change this to be SECURE_MAIL_ENCRYPTION_FAILURE_HANDLERS before documenting it though. ³

exception_handlers = {
    'message': 'handle_failed_message_encryption',
    'alternative': 'handle_failed_alternative_encryption',
    'attachment': 'handle_failed_attachment_encryption',
}

for key, value in exception_handlers.items():
    locals()[value] = import_function(key)

This code dynamically sets the local variables handle_failed_message_encryption, handle_failed_alternative_encryption, and handle_failed_attachment_encryption to the functions imported and returned by import_function.

This means that importing secure_mail.handlers.handle_failed_message_encryption will always import the correct failure handling function, even if users have configured secure_mail to use a custom failure handler. This code is basically import redirection to enable users to specify their own handlers, while allowing our code in secure_mail/backends.py to import failure handlers from a single place.

---

¹ Unless they expect all mail from you to be signed. But encryption and digital signatures are far from ubiquitous, so I think this is a rather weak argument.
² Which reminds me, I need to log more.
³ Add that to the pile of things I need to document.

[Euphorbium]

Unless they expect all mail from you to be signed. But encryption and digital signatures are far from ubiquitous, so I think this is a rather weak argument.

Every crypto service was and is plagued by email spoofing attacks. Experienced users expect emails to be signed by now. Add this line to your website and twitter: Do not trust unsigned emails coming from <insert_email_here>. All our emails are signed. Bam, now all your users expect signed emails.

[blag]

Every crypto service was and is plagued by email spoofing attacks.

Yes, and SPF and DKIM largely prevent header/sender spoofing. Content spoofing is another thing entirely, and is only solved by encrypting the signed message - which is exactly what this package already does, not just by signing the unencrypted message. End to end encryption is good for making sure the content isn't read by anybody except the recipient, but it is also good for ensuring that the content isn't changed along the way. Signing is only useful for non-repudiation, which is different than spoofing, and only if the signature cannot be trivially stripped off the message. Simply signing without encrypting does not effectively prevent any malicious party from modifying or spoofing message content. Signing is not the silver bullet you seem to think it is.

Experienced users expect emails to be signed by now.

This is unfortunately simply untrue. And even if it was true, I can just as easily turn this around on you: experienced users expect emails to be signed and encrypted by now.

Add this line to your website and twitter: Do not trust unsigned emails coming from <insert_email_here>. All our emails are signed. Bam, now all your users expect signed emails.

How do users know to check Twitter? What if the Twitter account is hacked and that tagline is removed? How is Twitter more trustworthy than your recipient's mail service?

If users want to ensure that the content of the email is not changed in flight, they can upload their public key. Bam, now all of their emails are signed and encrypted. Mail to them is now impossible to spoof or modify along the way.

Security is really difficult to implement, and nearly impossible to implement well. This package aims to change that. But part of that is not allowing our users to do things that are trivial to work around.

I refuse to add a backend that only signs emails, because it pretends to be secure when it really actually isn't. I have given you guidance on how to implement a signing fallback in the encrypting backend, and I might be interested in that pull request. But I do not want to make it easy for users to think they are secure when they really are not, and that's exactly what a signing-only backend does.

[blag]

Closing due to inactivity, but feel free to comment and/or ask me to reopen this issue if the backend altogether stops signing mail that should be signed and encrypted.

And just to be completely clear: I will not merge in a signing-only backend, but I would accept a PR that implements a signing-only fallback in the case where encrypting mail fails.