Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding PostgreSQL DB slow query log #6437

Merged
merged 8 commits into from
Jul 24, 2023
16 changes: 15 additions & 1 deletion source/deploy/high-scale-troubleshoot.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,18 @@ My system keeps hanging when I search for a message in Mattermost

First, check how many messages have been posted on your system, including deleted posts and posts made using automations.

Go to the **System Console > Reporting > Site Statistics** and review the **Total Posts** figure reported. If this figure is above 3,000,000 posts, we recommend deploying Elasticsearch alongside your Mattermost server for improved search performance. Follow our guides to `deploy an Elasticsearch server </scale/elasticsearch.html>`__.
Go to the **System Console > Reporting > Site Statistics** and review the **Total Posts** figure reported. If this figure is above 3,000,000 posts, we recommend deploying Elasticsearch alongside your Mattermost server for improved search performance. Follow our guides to `deploy an Elasticsearch server </scale/elasticsearch.html>`__.

How to troubleshoot server-side performance issues without Prometheus or Grafana?
-----------------------------------------------------------------------------------

Enable slow query logging in PostgreSQL and leave it enabled to gather data over time. PostgreSQL's slow query log helps you identify queries that take longer than a specified amount of time. The slow query log isn't enabled by default, and must be enabled manually.

To enable slow query logging globally, change the following line in postgresql.conf and reload the page: ``log_min_duration_statement = 5000``. When you this value to ``5000``, PostgreSQL considers queries that take longer than 5 seconds to be slow queries and logs them in the log file.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd set our suggestion to 1s for this, just to capture more. Anything longer than 1s really should be looked at.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, doing it this way requires a restart or a config reload. We have it reading as reload the page but it should be restart postgres which causes a short outage.

Additionally, we could just have them run SELECT pg_reload_conf(); to reload the config without a restart.


To enable slow query logging for a specific database, use ``ALTER DATABASE`` to change the configuration parameter for a single database. For example:

.. code-block:: none

postgres=# ALTER DATABASE test SET log_min_duration_statement = 5000;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's change test to mattermost and make a call out that the database name should be what you call it. 99% will be mattermost

ALTER DATABASE
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This actually should be removed, it's just a hanging statement if used in this way.