GH-111 - more and better spam filtering inside the AS Email Service #137

scanner · 2024-02-02T08:01:12Z

Postmark gives us spam assassin.. which used to be pretty weak-sauce.. and after watching it in action is still pretty weak sauce for the kinds of spam we get.

While we will still be delivering all email's including spam to local delivery, we want the option of blocking it when sending it via forwarding.

Eventually we might also get a daily report of blocked messages on the incoming stream from postmark and present users with a "here is the spam that was filtered" system.

* updating requirements * Only log from `mail` at error or higher * Update to python3.12

… in the external pw file (#118) * GH-117 - add support for maintaing an external pwfile by email account This is still lacking the test for the code we added. * GH-117 - testing the reading and writing of external pwfile * GH-117 - update email account update task to use relative paths for mail dirs Need to write test for the two tasks. * GH-117 - test delete/add/update tasks for email accounts in external pw file

* GH-116 - I was using the wrong python redis client. * GH-116 - create default folders. Also better fix for GH-123 - catch EmailAccount.DoesNotExist. With the proper redis client we can retry tasks that fail. Since we create 9accounts at once, retry up to 10 times. Also make sure that we can lock the pwfile so that there is only one reader or writer at a time.

…-1 encoded messages, do the binary conversion ourselves. (#127)

* GH-126 - Instead of using `send_message` that can not deal with latin-1 encoded messages, do the binary conversion ourselves. * GH-126 - try ascii -> utf-8 -> latin-1

* wip * GH-126 - remember to break out of loop when encode works

* is it by message id? * GH-132 - retry on failed bounce retrieval. Bounce retrievals that failed succeeded later.. so just try again.

…les (#145) * GH-144: Add support for ‘get or create’ for creating message filter rules * GH-144: updating requirements. Re-blackening. Fix some deprecation warnings * Add import_maildelivery. Fix parsing of maildelivery lines. * Logout requires POST to a form

* GH-149 - take converting the message to binary in to our code instead of leaving it to python’s mailbox. * Update requirements

* GH-149: Move failed incoming deliveries to their own directory. * GH-149: brute force by trying different encodings. If this does not catch all the cases have to rearch for bs4.UnicodeDammit * GH-149: Update requirements.

…. Update requirements. Add more files that can be filtered on. (#154)

* GH-153: Fix so MFR list only shows MFR’s that belong to email account. Update requirements. Add more files that can be filtered on. * GH-153: Add migration for filtering more headers

* Updating requirements * updating requirements * Use fakeredis for the index test as compressor relies on being able to use the cache Also move running tests out of the docker container. * Retry failed email. This will keep retrying forever, based on the assumption that the failure was transient, and if it is not, an admin will manually deal with the messages.

scanner added 11 commits November 28, 2023 18:47

Be able to set SENTRY_PROFILES_SAMPLE_RATE from .env (#112)

e358f55

Updating requirements, python 3.12 (#115)

bea31ba

* updating requirements * Only log from `mail` at error or higher * Update to python3.12

GH-123 - catch EmailAccount.DoesNotExist and log a wrning (#124)

cceb719

GH-126 - Instead of using send_message that can not deal with latin…

37cf2ae

…-1 encoded messages, do the binary conversion ourselves. (#127)

GH-126: Try ascii -> utf-8 -> latin-1 (#128)

f30761e

* GH-126 - Instead of using `send_message` that can not deal with latin-1 encoded messages, do the binary conversion ourselves. * GH-126 - try ascii -> utf-8 -> latin-1

GH-126 - remember to break out of loop when encoding works. (#129)

43fb37a

* wip * GH-126 - remember to break out of loop when encode works

GH-130 - lowercase the incoming to email address (#131)

be1092e

GH-132 - bounce id found when retrying after delay (#135)

9f651a8

* is it by message id? * GH-132 - retry on failed bounce retrieval. Bounce retrievals that failed succeeded later.. so just try again.

GH-121-need-to-search-fields-on-admin-no-foreign-keys (#136)

f8f2e56

scanner assigned scanner and unassigned scanner Feb 2, 2024

scanner added the enhancement New feature or request label Feb 2, 2024

scanner added 9 commits February 20, 2024 20:14

GH-138-allow-file-attachments-up-to-10mb (#139)

67b1c16

GH-149: unicode errors on delivering email (#150)

258ff2b

* GH-149 - take converting the message to binary in to our code instead of leaving it to python’s mailbox. * Update requirements

GH-149: Move failed incoming deliveries to their own directory. (#151)

1388df8

GH-149: unicode error on msg to bytes (#152)

7491492

* GH-149: Move failed incoming deliveries to their own directory. * GH-149: brute force by trying different encodings. If this does not catch all the cases have to rearch for bs4.UnicodeDammit * GH-149: Update requirements.

GH-153: Fix so MFR list only shows MFR’s that belong to email account…

862d3a8

…. Update requirements. Add more files that can be filtered on. (#154)

Bug/gh 153 message filter rule endpoint not filtering correctly (#155)

5b7d284

* GH-153: Fix so MFR list only shows MFR’s that belong to email account. Update requirements. Add more files that can be filtered on. * GH-153: Add migration for filtering more headers

GH-156: Retry on FileExistsError (#157)

0488554

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GH-111 - more and better spam filtering inside the AS Email Service #137

GH-111 - more and better spam filtering inside the AS Email Service #137

scanner commented Feb 2, 2024

GH-111 - more and better spam filtering inside the AS Email Service #137

Are you sure you want to change the base?

GH-111 - more and better spam filtering inside the AS Email Service #137

Conversation

scanner commented Feb 2, 2024