-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update module to configure good job for background workers #10
Conversation
All the checks failing is unrelated to this PR |
b0ddbdb
to
f57bb06
Compare
Coming back to this: Our default workers for Good job might be a bit too much for most setups. I went for 4 processes (since this was the default with delayed job), but since these processes are now multi-threaded (and have 5 threads by default), that means that we can process up to 20 jobs at once. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would consider that too much even for our setup. In the worst case we're transcoding 20 files at the same time, which is a load I wouldn't want the server to have. I'd start with what we have now, but only one process for all queues instead of three. If it turns out that this results in issues, we can always adjust later.
Updated default queues to be 1 process (with the queues in order of priority) |
This PR matches accentor/api#478 and allows the user to configure both good job and delayed job. A next PR can remove the delayed job config.
I opted to have the user pass the names of queues instead of simply the amount of workers to allow more finegrained control.
The default still uses 4 workers (though now with 5 threads each). One of the workers is reserved for the most urgent items. The others will work off all queues in the order they are specified in the string.