Question about configuring workers/threads for high volume

[opardalis]

I came across this link: #697 for tweaking Postal for high volume delivery.

I implemented the mysql recommendation (max_connections=5000, innodb_buffer_pool_size= 2048M), but I am confusing about the other settings.

I added pool_size: 100 under main_db and message_db: but the link above also mentions adding this to your postal.yml:

workers:
quantity: 60
threads: 80

The default config doesn't have a workers: section, just a general: - also that change above throws errors at startup. The only supported option I found was to start postal with the --scale option, for example:

postal start --scale worker=20

I assume with the current version, the --scale method is the correct approach.

Lastly, even with those pool_size changes, mysql, and --scale worker=20 during my email campaign (total list is ~9500) I only max out at around 200 emails/min which doesn't seem that high. I'm trying to get into the 600-800 per min range. The VM is in Azure, 8-core. I could upgrade the disk tier to get higher IOPS. During the outbound delivery, htop doesn't indicate that the node is stressed. So I'm thinking the bottle neck could be mysql/disk.

[willpower232]

I'm not the expert here but have you experimented with even more scaled workers? If your CPU is maxed out or delivery speed is the same regardless of how many workers you have then you'll know what scale to have or that you need to look elsewhere

[opardalis]

Yeah, at postal start --scale worker=20 during a send event, host CPU really isn't taxed, so I'll try bumping to around 40. I also will try bumping up the disk tier in Azure, this VM only has a 30GB disk, so its IOPS is low, bumping the disk up to 512GB gets me more throughput and might help mysql to run faster.

[mattbaylor]

@opardalis Just curious if you've seen any additional performance results from your testing. Thanks!

[mredodos]

some updatE ?

[leedave]

Took me a while to find this. I don't understand why this crucial feature is so well hidden.

It is entirely missing in the documentation on the website

postal man
Does not mention these paramters at all

Most discussions here just receive incomplete and useless replies like "this question was already asked before, use --scale". I had to read through several threads until I found a link to this discussion.

In order to use something, you need to know the full and correct syntax. Please add it to the documentation.