[Idea] Consider using flower for celery monitoring

[se7entyse7en]

See link.

As we verified with the demos/pocs celery is an important part of the application that IMO should be monitored. Flower provides a webapp based tool for monitoring and administrating celery. It seems to also provide an http api to expose the functionalities.

As an idea related to this, we could also explore checking the status of the celery queue and eventually manage, increase or restart the worker pool for example. In the last demo, for example, a restart of the celery workers would have helped, maybe there's a chance we could automate as well this kind of issues.

[rpau]

It seems that Prometheus could be an alternative. https://itnext.io/prometheus-monitoring-for-celery-workers-655d089dbd59 El vie., 27 sept. 2019 11:08, Lou Marvin Caraig <[email protected]> escribió:

…

See link <https://docs.celeryproject.org/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monitor> . As we verified with the demos/pocs celery is an important part of the application that IMO should be monitored. Flower provides a webapp based tool for monitoring and administrating celery. It seems to also provide an http api to expose the functionalities. As an idea related to this, we could also explore checking the status of the celery queue and eventually manage increase or restart the worker pool for example. In the last demo, for example, a restart of the celery workers would have helped, maybe there's a chance we could automate this kind of issues. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#291?email_source=notifications&email_token=AALKFKJBCIPV7T5ZY3TNA4TQLXERVA5CNFSM4I3D2CGKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HODEZFQ>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AALKFKIDPUCE5QEEFDJSFXTQLXERVANCNFSM4I3D2CGA> .

[smacker]

Yep. I would prefer to use Prometheus to track all the metrics in the same place (celery, webserver, db, cache, bblfsh, ...).

[se7entyse7en]

Sure! If we can use it and also extend it to have a centralized view of all the services it would be great!