[guides] guides: Add postgres migration guidelines #6596

isacikgoz · 2023-08-17T17:20:25Z

Summary

Add the initial version (exluding plugins) of MySQL to PostgreSQL migration guide.

Ticket Link

https://mattermost.atlassian.net/browse/MM-54161

github-actions · 2023-08-17T17:27:34Z

Newest code from isacikgoz has been published to preview environment for Git SHA c220510

nab-77

@cwarnermm can we apply the new code theme to this page also?

nab-77 · 2023-08-17T17:46:08Z

source/guides/postgres-migration.rst

+
+Note that the migration guide only covers the tables for Mattermost channels, the support for other plugins such as Boards and Playbooks will be added in the future. Another exlusion we are making is in the ``db_migrations`` table which has a small difference (a typo in a single migration name) creates a diff. Since we created the Postgres schema with morph and the official mattermost source, we can consider to skip it safely. On the other hand, ``systems`` table may contain additional diffs if there was extra keys added during some of the migrations. Consider excluding ``systems`` table if you run into issues and do a manual comparison as the data in the ``systems`` table is relatively smaller in size.
+
+Notes


Recommend we move this to the top after the intro but before we get into the steps.

Agreed. This becomes our TL;DR :)

cwarnermm

This is a great docs PR, @isacikgoz!! I've provided editorial feedback inline for you to consider.

source/guides/postgres-migration.rst

cwarnermm · 2023-08-18T13:13:36Z

source/guides/postgres-migration.rst

+
+Note that the migration guide only covers the tables for Mattermost channels, the support for other plugins such as Boards and Playbooks will be added in the future. Another exlusion we are making is in the ``db_migrations`` table which has a small difference (a typo in a single migration name) creates a diff. Since we created the Postgres schema with morph and the official mattermost source, we can consider to skip it safely. On the other hand, ``systems`` table may contain additional diffs if there was extra keys added during some of the migrations. Consider excluding ``systems`` table if you run into issues and do a manual comparison as the data in the ``systems`` table is relatively smaller in size.
+
+Notes


Agreed. This becomes our TL;DR :)

cwarnermm · 2023-08-18T13:14:26Z

source/guides/postgres-migration.rst

+Notes
+-----
+
+Keep in mind that this migration guide primarily focuses on providing step-by-step instructions for the migration; however, it is essential to note that it does not encompass migration configurations for any plugins, such as Focalboard and Playbooks. If your system utilizes these plugins, we highly advise exercising patience until we incorporate the necessary configurations specifically tailored to ensure a smooth transition for those plugins as well.


This statement partially overlaps the statement above. Consider combining together to avoid content duplication.

cwarnermm · 2023-08-18T20:35:16Z

@cwarnermm can we apply the new code theme to this page also?

@nab-77 Yes! I'll refresh my memory on how that's achieved and apply it to this content.

github-actions · 2023-08-18T20:42:53Z

Newest code from cwarnermm has been published to preview environment for Git SHA ab17b6b

Co-authored-by: Carrie Warner (Mattermost) <[email protected]>

github-actions · 2023-08-19T05:56:41Z

Newest code from isacikgoz has been published to preview environment for Git SHA 3ebe2c3

source/guides/postgres-migration.rst

github-actions · 2023-08-22T07:12:50Z

Newest code from isacikgoz has been published to preview environment for Git SHA 20ca30f

nab-77

Few thoughts/ideas added.

nab-77 · 2023-08-22T08:02:30Z

source/guides/postgres-migration.rst

+.. code:: bash
+
+   pgLoader migration.load > migration.log
+


Do we want to place a note here to remind users to run the commands to re-create the indexes as noted in lines 113/114?

Sure, I can add that.

nab-77 · 2023-08-22T08:03:09Z

source/guides/postgres-migration.rst

+  :backlinks: top
+  :local:
+  :depth: 1
+


I think a known issues table might be a good addition here.

I tried to collect as much as known issues and tried to address "Schema diffs" section. I think we can add a section for known issues & troubleshooting once these gets accumulated.

nab-77 · 2023-08-22T08:03:53Z

source/guides/postgres-migration.rst

+--------------------
+
+-  Backup your MySQL data.
+-  Confirm your Mattermost version. See the **About** modal for details. 


@isacikgoz do we have a min MM version we would recommend? I would certainly like to call out being on a supported ESR.

I think we can add that, for now it looks like the minimum support version is v6.4 ESR as we require morph based schema migrations. So if they have version before v6.4 they need to upgrade it first, then conduct the migration. Shall we add that info?

@nab-77 added a note 👍

nab-77 · 2023-08-22T08:04:46Z

source/guides/postgres-migration.rst

+-  Confirm your Mattermost version. See the **About** modal for details. 
+-  Determine the migration window needed. This process requires you to stop the Mattermost Server during the migration.
+-  See the `schema-diffs <#schema-diffs>`__ section to ensure data compatibility between schemas.
+-  Prepare your PostgreSQL environment by creating a database and user. See the `database </install/prepare-mattermost-database.html>`__ documentation for details.


@isacikgoz do we have a preference for pgsql version here? (outside of ensuring its a supported version).

I think minimum version that we support is fine, but the latest supported version would be our preference as far as I'm concerned. Do you think we should provide a specific version?

nab-77 · 2023-08-22T08:05:52Z

source/guides/postgres-migration.rst

+
+Once we set the schema to desired state, we can start migrating the **data** by running ``pgLoader`` \*\*
+
+\*\* Use the following configuration for the baseline of the data migration:


@isacikgoz what are you thoughts on us providing the migration.load template for users to download/edit? Link here maybe?

The template is actually on the repo, being used by the workflow: https://github.com/mattermost/mattermost/blob/master/server/tests/template.load we can add the link here.

github-actions · 2023-08-22T09:52:14Z

Newest code from isacikgoz has been published to preview environment for Git SHA cfe431f

cwarnermm

This is such a great addition to the docs! I've provided editorial feedback inline. Approving for expediency.

source/guides/postgres-migration.rst

cwarnermm · 2023-08-22T13:37:48Z

@isacikgoz - @nab-77 had requested that the commands and code samples follow our new approach in the docs, as seen in this screenshot:

The code block text is formatted using a special class called mm-code-block:

.. code-block:: none
  :class: mm-code-block 

    your command here

github-actions · 2023-08-22T13:43:48Z

Newest code from cwarnermm has been published to preview environment for Git SHA 54e0b49

Co-authored-by: Carrie Warner (Mattermost) <[email protected]>

github-actions · 2023-08-22T15:40:59Z

Newest code from isacikgoz has been published to preview environment for Git SHA 5c82614

github-actions · 2023-08-29T08:50:28Z

Newest code from isacikgoz has been published to preview environment for Git SHA 4f040b4

nab-77

@cwarnermm I think we can push this now.

Nav location: https://docs.mattermost.com/guides/administration.html#self-hosted-administration

github-actions · 2023-08-29T16:01:43Z

Newest code from cwarnermm has been published to preview environment for Git SHA 7b7a9eb

guides: Add postgres migration guidelines

c220510

isacikgoz added 1: Dev Review Requires review by a core commiter 1: PM Review Requires review by a product manager 2: Editor Review Requires review by an editor labels Aug 17, 2023

isacikgoz requested review from agnivade, cwarnermm and nab-77 August 17, 2023 17:20

nab-77 reviewed Aug 17, 2023

View reviewed changes

cwarnermm reviewed Aug 18, 2023

View reviewed changes

cwarnermm removed the 2: Editor Review Requires review by an editor label Aug 18, 2023

Merge branch 'master' into MM-54161

ab17b6b

Apply suggestions from code review

3ebe2c3

Co-authored-by: Carrie Warner (Mattermost) <[email protected]>

agnivade reviewed Aug 21, 2023

View reviewed changes

source/guides/postgres-migration.rst Outdated Show resolved Hide resolved

source/guides/postgres-migration.rst Outdated Show resolved Hide resolved

source/guides/postgres-migration.rst Outdated Show resolved Hide resolved

isacikgoz added 2 commits August 22, 2023 10:03

reflect review comments

50229b8

Merge remote-tracking branch 'origin/master' into MM-54161

20ca30f

agnivade approved these changes Aug 22, 2023

View reviewed changes

isacikgoz requested review from cwarnermm and nab-77 August 22, 2023 07:07

nab-77 reviewed Aug 22, 2023

View reviewed changes

reflect review comments

cfe431f

cwarnermm approved these changes Aug 22, 2023

View reviewed changes

Merge branch 'master' into MM-54161

54e0b49

Apply suggestions from code review

5c82614

Co-authored-by: Carrie Warner (Mattermost) <[email protected]>

isacikgoz requested a review from nab-77 August 24, 2023 13:27

isacikgoz added 2 commits August 29, 2023 11:43

fine tune the script

da99800

Merge remote-tracking branch 'origin/master' into MM-54161

4f040b4

isacikgoz mentioned this pull request Aug 29, 2023

Add boards and playbooks guide for PostgreSQL migration #6628

Merged

nab-77 approved these changes Aug 29, 2023

View reviewed changes

Moved page and added nav pane access

7b7a9eb

cwarnermm added 3: Reviews Complete All reviewers have approved the pull request and removed 1: Dev Review Requires review by a core commiter 1: PM Review Requires review by a product manager labels Aug 29, 2023

cwarnermm merged commit a66272c into master Aug 29, 2023
3 checks passed

cwarnermm deleted the MM-54161 branch August 29, 2023 15:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[guides] guides: Add postgres migration guidelines #6596

[guides] guides: Add postgres migration guidelines #6596

isacikgoz commented Aug 17, 2023

github-actions bot commented Aug 17, 2023

nab-77 left a comment

nab-77 Aug 17, 2023

cwarnermm Aug 18, 2023

cwarnermm left a comment

cwarnermm Aug 18, 2023

cwarnermm Aug 18, 2023

cwarnermm commented Aug 18, 2023

github-actions bot commented Aug 18, 2023

github-actions bot commented Aug 19, 2023

github-actions bot commented Aug 22, 2023

nab-77 left a comment

nab-77 Aug 22, 2023

isacikgoz Aug 22, 2023

nab-77 Aug 22, 2023

isacikgoz Aug 22, 2023

nab-77 Aug 22, 2023

isacikgoz Aug 22, 2023

isacikgoz Aug 22, 2023

nab-77 Aug 22, 2023

isacikgoz Aug 22, 2023

nab-77 Aug 22, 2023

isacikgoz Aug 22, 2023

github-actions bot commented Aug 22, 2023

cwarnermm left a comment

cwarnermm commented Aug 22, 2023

github-actions bot commented Aug 22, 2023

github-actions bot commented Aug 22, 2023

github-actions bot commented Aug 29, 2023

nab-77 left a comment •

edited

Loading

github-actions bot commented Aug 29, 2023


		Note that the migration guide only covers the tables for Mattermost channels, the support for other plugins such as Boards and Playbooks will be added in the future. Another exlusion we are making is in the ``db_migrations`` table which has a small difference (a typo in a single migration name) creates a diff. Since we created the Postgres schema with morph and the official mattermost source, we can consider to skip it safely. On the other hand, ``systems`` table may contain additional diffs if there was extra keys added during some of the migrations. Consider excluding ``systems`` table if you run into issues and do a manual comparison as the data in the ``systems`` table is relatively smaller in size.

		Notes


		Once we set the schema to desired state, we can start migrating the data by running ``pgLoader`` \\

		\\ Use the following configuration for the baseline of the data migration:

[guides] guides: Add postgres migration guidelines #6596

[guides] guides: Add postgres migration guidelines #6596

Conversation

isacikgoz commented Aug 17, 2023

Summary

Ticket Link

github-actions bot commented Aug 17, 2023

nab-77 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cwarnermm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cwarnermm commented Aug 18, 2023

github-actions bot commented Aug 18, 2023

github-actions bot commented Aug 19, 2023

github-actions bot commented Aug 22, 2023

nab-77 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Aug 22, 2023

cwarnermm left a comment

Choose a reason for hiding this comment

cwarnermm commented Aug 22, 2023

github-actions bot commented Aug 22, 2023

github-actions bot commented Aug 22, 2023

github-actions bot commented Aug 29, 2023

nab-77 left a comment • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Aug 29, 2023

nab-77 left a comment •

edited

Loading