Run the bot in a separate process

[link2xt]

Currently the bot is started as a thread within gunicorn process. It means that when the bot crashes, calls os._exit or needs to be restarted, gunicorn process has some downtime too.

We can still build a single Docker image, but start it twice with the same volume: once as gunicorn ..., and once as a python -m mailadm bot ... or something like this. Then gunicorn will run in a separate container with its own logs and no task except for processing new_email requests.

Pruning loop could also be a separate process, like python -m mailadm prune-loop or something like this, also running in a separate container.

This would make it easier to monitor and debug gunicorn, which provides the most important service.

[link2xt]

I see that there is some prior discussion on this and a comment in
#36 (comment)

@missytake What was the reason to run everything in a single process? Starting threads from the on_starting hook of gunicorn configuration file does not look reliable at all.

[brabo]

in redoing mailadm setup, i agree.
atm managing tokens from the cli without running a bot is a bit cumbersome. plus, if the bot is not running, the gunicorn container is not running, and even if you generate a qr on the cli, the account creation is unavailable.
also, if the gunicorn only serves to serve a single POST request, is it needed at all?

[link2xt]

if the gunicorn only serves to serve a single POST request, is it needed at all?

Flask documentation says you always need some production WSGI server: https://flask.palletsprojects.com/en/2.2.x/deploying/

[brabo]

thanks to link2xt nice update of the docs, the cli usage is a lot easier now.

[missytake]

@missytake What was the reason to run everything in a single process? Starting threads from the on_starting hook of gunicorn configuration file does not look reliable at all.

I think the main reason was that it was unclear yet how much the different parts of the system need to talk to each other. But if every part of mailadm can do mailcow API requests on its own, then #56 is the last use case I can think of where mailadm-prune and mailadm-bot need to talk to each other.

[missytake]

I think in the most urgent sense it would not be necessary to make mailadm very modular - at least when #56 is merged, it will not make much sense anymore to run mailadm without either gunicorn, mailadm-bot, mailadm-prune, or mailcow. But I agree with two advantages in making it modular:

1. reliability gains, as only gunicorn, the bot, or prune failing would keep the rest running
2. making debugging easier (not entirely sure about that though)

Am I correct in the assumption that you would still want all of the processes to make mailcow requests? So there is not one process responsible for doing mailcow requests?

I see one problem for #56 - the mailadm-prune thread is currently responsible for sending out the warnings. If mailadm-bot is running in a separate container, it can't pass the warnings to it for sending them out. But it can't run a separate deltachat core either, as mailadm-bot needs to receive all the incoming messages reliably.
-> OTOH, we could use the account credentials to just send out the expiry warnings via SMTP. It's a bit clunky, but would work I guess. We probably need to insert some headers so it looks like a chat message. And it wouldn't be updated with the deltachat core, which is a bummer.

[dumblob]

I support modularization with the goal to make it easier to:

1. Either add support for other than just mailcow projects in the future (Mail-in-a-Box, Mail-in-a-Box LDAP, syncloud.it, ...).
2. Make it easy to port (and maintain!) to other than mailcow projects.