Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to Update RocketChat Helm Chart #90

Open
david9991 opened this issue Mar 6, 2023 · 7 comments
Open

Failed to Update RocketChat Helm Chart #90

david9991 opened this issue Mar 6, 2023 · 7 comments

Comments

@david9991
Copy link

Description

I attempted to update the RocketChat Helm chart from version 5.3.2 to 5.4.3, but I encountered an error message.

Error Message

The error message I received was:

rocketchat 0s Warning FailedMount pod/rocketchat-rocketchat-pre-upgrade MountVolume.SetUp failed for volume "mongodb-script" : configmap "rocketchat-rocketchat-scripts" not found
rocketchat                   0s          Warning   FailedMount          pod/rocketchat-rocketchat-pre-upgrade                Unable to attach or mount volumes: unmounted volumes=[mongodb-script], unattached volumes=[mongodb-script kube-api-access-8sqjl]: timed out waiting for the condition

Previous Version

In the previous version (v5.3.2), there were two configmaps available: rocketchat-mongodb-fix-clustermonitor-... and rocketchat-mongodb-scripts. There was no rocketchat-rocketchat-scripts.

Expected Behavior

I expected the update to proceed successfully without any errors.

Actual Behavior

The update failed due to the error message above.

Steps to Reproduce

  1. Attempt to update the RocketChat Helm chart from version 5.3.2 to 5.4.3.
  2. Observe the error message.

Possible Solution

It appears that the rocketchat-rocketchat-scripts configmap is missing. It may need to be added or updated in order for the update to proceed successfully.

Environment

  • RocketChat Helm chart v5.3.2
  • Attempting to update to v5.4.3
@masikrus
Copy link

masikrus commented Mar 15, 2023

same problem

@masikrus
Copy link

masikrus commented Mar 15, 2023

Oh... there the solutions https://github.com/RocketChat/helm-charts/tree/master/rocketchat
readme says "you need upgrade mongo your self"

just do
kubectl -n $NAMESPACE edit statefulsets.apps rocketchat-mongodb
and set first image tag 5.0.15-debian-11-r7
after pod start look to logs and wait upgrade mongo to 5.0
then exec to mongo container and set compatible versions
kubectl -n $NAMESPACE exec -it rocketchat-mongodb-0 -- bash
mongo -u $USER -p $PASSWORD
db.adminCommand( { setFeatureCompatibilityVersion: "5.0" } )
then again edit statefulsets but now set tag 6.0.5-debian-11-r0
and then again logs, wait upgrade, and set compatible version to 6.0
mongosh -u $USER -p $PASSWORD
db.adminCommand( { setFeatureCompatibilityVersion: "6.0" } )
and then apply file rocketchat-rocketchat-scripts

apiVersion: v1
kind: ConfigMap
metadata:
  name: "rocketchat-rocketchat-scripts"
  annotations:
    meta.helm.sh/release-name: rocketchat
    meta.helm.sh/release-namespace: stage
  labels:
    app.kubernetes.io/name: rocketchat
    helm.sh/chart: rocketchat-5.3.2
    app.kubernetes.io/instance: rocketchat
    app.kubernetes.io/managed-by: Helm

data:
  verifyMongodb.js: |
    const versions = [
      "4.0",
      "4.2",
      "4.4",
      "5.0",
      "6.0"
    ];
    // [0]=mongosh path, [1]=mongosh path, [2]=connection string, [3]=script path, [4]=new tag
    const newVersion = process.argv[4].split('.').splice(0, 2).join('.');
    const currentVersion = db.version().split('.').splice(0, 2).join('.');
    if (currentVersion === newVersion) quit(0);
    const featureCompatibilityVersionResp = db.adminCommand({ getParameter:
      1, featureCompatibilityVersion: 1 });
    if (!featureCompatibilityVersionResp.ok) {
      print("failed to read current feature compatibility version");
      quit(1);
    }
    const featureCompatibilityVersion = featureCompatibilityVersionResp.featureCompatibilityVersion.version;
    if (featureCompatibilityVersion !== currentVersion) {
      print("current feature compatibility version is incompatible with the mongodb version you're trying to update to");
      quit(1);
    }
    if (versions.indexOf(newVersion) - versions.indexOf(currentVersion) !== 1) {
      print("you can't skip a mongodb version while updating, please read our update documentation for more information");
      quit(1);
    }
    quit(0);

just install by helm -n $NAMESPACE apply -f $YML_FILE

And THEN! ) edit file values.yml, set
mongodb: image: tag: 6.0.5-debian-11-r0
And then you can upgrade to release v5.4.3 without problem

P.S.

  1. edit file rocketchat-rocketchat-scripts by you namespace
  2. on mongodb 6.0+ cli binary named mongosh
  3. maybe after upgrade need scale mongo statefulsets from 1 to 0 and from 0 to 1
    kubectl -n stage scale --replicas=0 statefulset rocketchat-mongodb

@david9991
Copy link
Author

To resolve the issue, I performed the following steps:

  • Dumped the old database
  • Performed a clean reinstallation
  • Restored the database from the dump to the new version

After completing these steps, my problem was resolved.

@david9991
Copy link
Author

I believe it's the same issue as described in #93.

@masikrus
Copy link

To resolve the issue, I performed the following steps:

  • Dumped the old database
  • Performed a clean reinstallation
  • Restored the database from the dump to the new version

After completing these steps, my problem was resolved.

You can't dump mongo 4.4 and restore to 5 or 6 version, dump and restore in mongodb work only in version to version

@alexkorotysh
Copy link

I have the same issue, looks like pod XXXXXX-pre-upgrade doesn't have configmap with XXXXXX-scripts before the upgrade.

@Davidffry
Copy link

Davidffry commented Sep 29, 2023

Hello, how I upgrade RocketChat :

Previous version :

  • rocketchat : v5.3.4
  • mongodb : v4.4.11

Steps :

  • Creation of ConfigMap : rocket-chat-rocketchat-scripts (enable to check the version to migration) - kubectl apply -f rocket-chat-rocketchat-scripts.yml
  • Upgrade mongoDB statefulSet to 5.0 version manually
  • Exec in MongoDB : db.adminCommand( { setFeatureCompatibilityVersion: "5.0" } )
    • Into the MongoDB pod run : mongosh -u root -p
    • Enter the password from the secret
    • Run this above command
  • Apply Helm chart for 5.4.3 (Normally we can apply to 6.4.3 but for security reason upload to 5.4.3 before)
  • Check if it work
  • It does not work : Rollback, re-run test.
  • If works,
  • Apply Helm chart for 6.3.4
  • Exec in MongoDB : db.adminCommand( { setFeatureCompatibilityVersion: "6.0" } )
    • Into the MongoDB pod run : mongosh -u root -p
    • Enter the password from the secret
    • Run to check : db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )
    • Run this above command
  • Check if it work
#rocket-chat-rocketchat-scripts.yml
apiVersion: v1
kind: ConfigMap
metadata:
  name: rocket-chat-rocketchat-scripts
  namespace: rocketchat
  labels:
    app.kubernetes.io/instance: rocket-chat
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: rocketchat
    helm.sh/chart: rocketchat-6.3.4
    k8slens-edit-resource-version: v1
  annotations:
    meta.helm.sh/release-name: rocket-chat
    meta.helm.sh/release-namespace: rocketchat
  selfLink: /api/v1/namespaces/rocketchat/configmaps/rocket-chat-rocketchat-scripts
data:
  verifyMongodb.js: >
    const versions = [
      "4.0",
      "4.2",
      "4.4",
      "5.0",
      "6.0"
    ];

    // [0]=mongosh path, [1]=mongosh path, [2]=connection string, [3]=script
    path, [4]=new tag

    const newVersion = process.argv[4].split('.').splice(0, 2).join('.');

    const currentVersion = db.version().split('.').splice(0, 2).join('.');

    if (currentVersion === newVersion) quit(0);

    const featureCompatibilityVersionResp = db.adminCommand({ getParameter:
      1, featureCompatibilityVersion: 1 });
    if (!featureCompatibilityVersionResp.ok) {
      print("failed to read current feature compatibility version");
      quit(1);
    }

    const featureCompatibilityVersion =
    featureCompatibilityVersionResp.featureCompatibilityVersion.version;

    if (featureCompatibilityVersion !== currentVersion) {
      print("current feature compatibility version is incompatible with the mongodb version you're trying to update to");
      quit(1);
    }

    if (versions.indexOf(newVersion) - versions.indexOf(currentVersion) !== 1) {
      print("you can't skip a mongodb version while updating, please read our update documentation for more information");
      quit(1);
    }

    quit(0);

Targetd Version :

  • rocketchat : v6.3.4
  • mongodb : v6.0XXXX

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants