Skip to content
This repository has been archived by the owner on Feb 8, 2024. It is now read-only.

HA provisioning on 3 node VM cluster: Manual

Jump to bottom
indrajitzagade edited this page Jun 28, 2021 · 64 revisions

Pre-requisites:

  1. Make sure that all cortx components except cortx-ha have been installed and configured on all of the 3-nodes.

  2. Install cortx-py-utils on all of the 3-nodes if not installed already.

    Install steps

  3. Install and setup Corosync/Pacamaker on all of the 3-nodes:

    3.1 Make sure /etc/hosts is reflected properly or DNS is updated to resolve host names.

    E.g.
$ cat /etc/hosts
10.230.241.214 srvnode-1

    3.2 Install EPEL Repository, corosync, pacemaker and pcsd.

    $ yum -y install epel-release
$ yum -y install corosync pacemaker pcs

    3.3 Enable pacemaker, corosync, and pcsd service

    $ systemctl enable pcsd
$ systemctl enable corosync
$ systemctl enable pacemaker

    3.4 Start pcsd service

    $ systemctl start pcsd

    3.5 Configure a password for the 'hacluster' user. Here for given wiki "admin" is used as password.

    $ echo <new-password> | passwd --stdin hacluster

Install cortx-ha on all of the 3-nodes:

  1. Add repository for Cortx 
    $ yum-config-manager --add-repo <Link to the cortx_iso>
  2. Install cortx-ha 
    $ yum -y install cortx-ha

Encrypt the pacemaker cluster user password:

Open python3 prompt on one of the 3-nodes and run following commands:

    $ from cortx.utils.security.cipher import Cipher, CipherInvalidToken
    $ key = Cipher.generate_key(cluster_id, 'cortx')

Here use the same cluster_id as it will be added to confstore template files below.

    $ encrypt_text = Cipher.encrypt(key, 'admin'.encode('ascii'))

Here 'admin' is the password that we want to use (from step 3.5 above).

    $ print(encrypt_text)
    b'gAAAAABgCSb6rPYYM_59JNS8hLAnQt-J1FniehFJ-XW6QGE2T38rSVXoE1VnUkSE5R2qh8NuAtvxs5SCQ8khTa2LmUnF2fBhYA=='

Confstore for cortx-ha 3-node VM mini-provisioning:

Update following confstore template files on all of the 3-nodes.

    1. /opt/seagate/cortx/ha/conf/ha.post-install.tmpl.3-node
    2. /opt/seagate/cortx/ha/conf/ha.prepare.tmpl.3-node

Above files do not contain any keys as of now, hence no need to update these.

    3. /opt/seagate/cortx/ha/conf/ha.config.tmpl.3-node

a) Replace TMPL_CLUSTER_ID with the cluster-id to be used. Use the same value across 3-nodes.

b) Replace TMPL_NAME with the cluster name to be used. Use the same value across 3-nodes.

c) Replace TMPL_USER with the HA cluster user used in step #3.5 i.e. "hacluster".

d) Replace TMPL_SECRET with the encrypted pacemaker cluster user password above.

e) Replace TMPL_MACHINE_ID_1 with node-1, TMPL_MACHINE_ID_2 with node-2, TMPL_MACHINE_ID_3 with node-3 machine-id of VM. Run following command to get on respective node to fetch machine-id of each VM:

    $ cat /etc/machine-id

f) Replace TMPL_PRIVATE_FQDN_1 with the node-1 hostname, TMPL_PRIVATE_FQDN_2 with the node-2 hostname and TMPL_PRIVATE_FQDN_3 with the node-3 hostname used in the step #3.1 above for a particular node.

g) Replace TMPL_NODE_ID_x, TMPL_SITE_ID_x, TMPL_RACK_ID_x and TMPL_STORAGE_SET_ID_x with the actual IDs to be used for the respective nodes.

h) Replace TMPL_TYPE with "VM".

i) Replace TMPL_VIRTUAL_HOST with vip, TMPL_INTERFACES with vip interface and TMPL_NETMASK with vip netmask. Make sure the vip used here is not already in use.

    4. /opt/seagate/cortx/ha/conf/ha.init.tmpl.3-node
    5. /opt/seagate/cortx/ha/conf/ha.test.tmpl.3-node
    6. /opt/seagate/cortx/ha/conf/ha.reset.tmpl.3-node
    7. /opt/seagate/cortx/ha/conf/ha.cleanup.tmpl.3-node

Make changes same as the ha.config.tmpl.3-node file above.

cortx-ha mini-provisioning:

Run following mini-provisioning commands on the 3-nodes as per the Cortx 3-node deployment workflow.

HA:Post_Install

$ ha_setup post_install --config "yaml:///opt/seagate/cortx/ha/conf/ha.post-install.tmpl.3-node"

HA:Prepare

$ ha_setup prepare --config "yaml:///opt/seagate/cortx/ha/conf/ha.prepare.tmpl.3-node"

HA:Config

# Note: Use --dev to avoid mgmt_vip configuration
$ ha_setup config --config "yaml:///opt/seagate/cortx/ha/conf/ha.config.tmpl.3-node"

With above commands the cortx-ha should be provisioned. Following commands are empty currently and provided just for the completeness.

HA:Init

$ ha_setup init --config "yaml:///opt/seagate/cortx/ha/conf/ha.init.tmpl.3-node"

HA:Test

$ ha_setup test --config "yaml:///opt/seagate/cortx/ha/conf/ha.test.tmpl.3-node"

HA:Reset

$ ha_setup reset --config "yaml:///opt/seagate/cortx/ha/conf/ha.reset.tmpl.3-node"

If need to revert the cortx-ha configuration then run:

HA:Cleanup

# Note: use --local option; If cleanup is required on current node only else it will assume as running in parallel on all node.
$ ha_setup cleanup --config "yaml:///opt/seagate/cortx/ha/conf/ha.cleanup.tmpl.3-node"

Starting the cortx cluster:

$ cortx cluster start --all
Clone this wiki locally