Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added testing note #7680

Open
wants to merge 18 commits into
base: master
Choose a base branch
from
Open

Added testing note #7680

wants to merge 18 commits into from

Conversation

cwarnermm
Copy link
Member

@cwarnermm cwarnermm commented Jan 7, 2025

Docs to-do: Add updated note to specific user count levels as needed

@cwarnermm cwarnermm added the 1: Dev Review Requires review by a core commiter label Jan 7, 2025
@cwarnermm cwarnermm added this to the v10.4.0 milestone Jan 7, 2025
Copy link

github-actions bot commented Jan 7, 2025

Newest code from mattermost has been published to preview environment for Git SHA e38d5fc

Copy link
Member

@agarciamontoro agarciamontoro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great from a technical point of view. But I'd love to get the PM point of view as well before merging, if that's possible. cc/@wiersgallak

@agarciamontoro agarciamontoro added the 1: PM Review Requires review by a product manager label Jan 7, 2025
@cwarnermm cwarnermm requested a review from sadohert January 10, 2025 14:23
@sadohert
Copy link
Contributor

Hey All... just reviewed the note. Good we caught that.

I think we need to add something else here.

When I read this:

the Elasticsearch cluster is underspecified for user counts of 30k users or more

I immediately wonder "Okay, so what should it be for those customers?" The "overspecified" case doesn't concern me really. Its the Elasticsearch underspecified.

In the absence of bandwidth to gather data-driven values for these variables, can we adapt the specifications recommended for ES >=30k to something we would consider "borderline OVERspecified", then our "Important Note" would read more like:

Hey, these were not analyzed deeply in the scope of testing for this round. Its possible lower instance specs will be sufficient. We recommend monitoring the performance of the Elasticsearch cluster to identify opportunities for reduced specifications.

@cwarnermm cwarnermm removed the request for review from wiersgallak January 10, 2025 15:09
@cwarnermm cwarnermm removed the 1: PM Review Requires review by a product manager label Jan 10, 2025
@cwarnermm
Copy link
Member Author

@agarciamontoro - Can I get your help incorporating @sadohert's input into the note details, please? I'll ensure the note displays on applicable pages in the scale section.

@agarciamontoro agarciamontoro self-requested a review January 10, 2025 16:13
@agarciamontoro
Copy link
Member

I'm onboard with Stu's idea, but coming up with such a specification without testing is... scary. Let's discuss this next week, I self-requested a review so I don't forget :)

@cwarnermm cwarnermm modified the milestone: v10.4.0 Jan 10, 2025
@cwarnermm cwarnermm mentioned this pull request Jan 13, 2025
@amyblais
Copy link
Member

@cwarnermm The base branch needs to be changed to "master" before merging this.

@cwarnermm cwarnermm changed the base branch from v10.4-documentation to master January 21, 2025 16:52
@agarciamontoro
Copy link
Member

Started a thread to discuss with the team here: https://hub.mattermost.com/xyz/pl/wdm6e34ajtdjibd4yp6d1zixky

Copy link

Newest code from mattermost has been published to preview environment for Git SHA 223a243

@agarciamontoro
Copy link
Member

That is perfect, @cwarnermm, thank you! I would only suggest to remove the last sentence, since we're trying to not get customers to run load-tests without an explicit approval from our side, since it can generate quite a high support load for us. And I may even remove the very first sentence as well, since the fact that we are recommending those specs is already proof that those specs "have been tested and are confirmed to support the specified number of users reliably".

@cwarnermm
Copy link
Member Author

@sadohert - My latest commit incorporates @agarciamontoro's feedback and is ready for your input. Thanks!

Copy link

Newest code from mattermost has been published to preview environment for Git SHA 648c8b4

Copy link
Member

@agarciamontoro agarciamontoro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @cwarnermm! I left a couple of comments below.

@@ -12,24 +12,25 @@ This page describes the Mattermost reference architecture designed for the load
.. note::
- Usage of CPU, RAM, and storage space can vary significantly based on user behavior. These hardware recommendations are based on traditional deployments and may grow or shrink depending on how active your users are.
- From Mattermost v10.4, Mattermost Enterprise customers can configure `Redis <https://redis.io/>`_ (Remote Dictionary Server) as an alternative cache backend. Using Redis can help ensure that Mattermost remains performant and efficient, even under heavy usage. See the :ref:`Redis cache backend <configure/environment-configuration-settings:redis cache backend>` configuration settings documentation for details.
- While the following specifications may be more than sufficient for some use cases, we have not extensively tested configurations with lower resource allocations for this user scale. If cost optimization is a priority, admins may choose to experiment with smaller configurations, but we recommend starting with the tested specifications to ensure system stability and performance. Keep in mind that under-provisioning can lead to degraded user experience and additional troubleshooting effort.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This note (and the same one in the other architectures) only applies to the Elasticsearch resource. Can we make that explicit converting this into a footnote or something similar?

source/scale/scaling-for-enterprise.rst Outdated Show resolved Hide resolved
+------------------------+-----------+----------------+-----------------------+
| RDS Reader | 5 | 16/128 | db.r7g.4xlarge |
+------------------------+-----------+----------------+-----------------------+
| Elasticsearch Node | 2 | 4/32 | 4 x r6g.xlarge.search |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This (and the other ones) should be:

Suggested change
| Elasticsearch Node | 2 | 4/32 | 4 x r6g.xlarge.search |
| Elasticsearch cluster | 4 | 8/64 | r6g.2xlarge.search |

Co-authored-by: Alejandro García Montoro <[email protected]>
Copy link

Newest code from mattermost has been published to preview environment for Git SHA 9b3977b

@cwarnermm
Copy link
Member Author

@agarciamontoro - Really appreciate the technical guidance on the updates needed! Thank you!

Copy link

Newest code from mattermost has been published to preview environment for Git SHA a60d622

Copy link

Newest code from mattermost has been published to preview environment for Git SHA ab9840d

Copy link

Newest code from mattermost has been published to preview environment for Git SHA ffe967e

Copy link

Newest code from mattermost has been published to preview environment for Git SHA 9c7c66d

Copy link

Newest code from mattermost has been published to preview environment for Git SHA 11707e7

Copy link

Newest code from mattermost has been published to preview environment for Git SHA 62030a3

Copy link
Member

@agarciamontoro agarciamontoro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again, @cwarnermm!

Copy link

Newest code from mattermost has been published to preview environment for Git SHA a998186

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1: Dev Review Requires review by a core commiter 2. SME Review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants