Merge pull request #759 from nuagenetworks/dev

Merge master into dev in preparation for MetroÆ v2.4.3

Documentation/DEPLOY.md

@@ -8,9 +8,9 @@ You can execute Metro Automation Engine playbooks to perform the following insta
 
 ## Prerequisites / Requirements
 
-Before deploying any components, you must have previously [set up your Nuage Metro Automation Engine Ansible environment](SETUP.md "link to SETUP documentation") and [customized the environment for your target platform](CUSTOMIZE.md "link to CUSTOMIZE documentation").
+Before deploying any components, you must have previously [set up your Nuage Metro Automation Engine Ansible environment](SETUP.md "link to SETUP documentation") and [customized your environment](CUSTOMIZE.md "link to CUSTOMIZE documentation").
 
-Make sure you have unzipped the Nuage Networks *.tar.gz files into their proper locations in the directory structure, so Metro Automation Engine can find the path of the Nuage components automatically when running commands.
+Make sure you have unzipped copies of all the Nuage Networks files you are going to need for installation or upgrade. These are generally distributed as `*.tar.gz` files that are downloaded by you from Nokia OLCS. You can unzip these files by using the nuage_unzip playbook which will place the files in subdirectories under the path specified for the `nuage_unzipped_files_dir` variable in `build_vars.yml`. You can also unzip the files manually and copy them to their proper locations by hand. For details of this process, including the subdirectory layout that Metro Automation Engine expects, see [customizing your environment](CUSTOMIZE.md "link to CUSTOMIZE documentation").
 
 ## Deploy All Components
 
@@ -55,6 +55,23 @@ Metro Automation Engine can automatically bootstrap (ZFB) a NSGV when deploying
 * Customize variables in [`zfb_vars.yml`](/zfb_vars.yml "link to zfb_vars.yml file")
 * Specify `bootstrap_method: zfb_metro,` in mynsgvs parameters in [`build_vars.yml`](/build_vars.yml "link to build_vars.yml file")
 
+## Copy QCOW2 Files before Deployment
+
+When installing or upgrading in a KVM environment, the Metro Automation Engine will copy the QCOW2 image files to the target file server during the predeploy phase. As an option, the playbook copy_qcow2_files can be used to pre-position the qcow2 files for all the components. This playbook gives the ability to copy the required images files first and then run install or upgrade later.
+
+When QCOW2 files are pre-positioned, you must add a command-line variable, 'skip_copy_qcow2', to indicate that copying QCOW2 files should be skipped. Otherwise, the QCOW2 files will be copied again. An extra-vars 'skip_copy_qcow2' needs to be passed on the command line during the deployment phase to skip copying of the image files again. For example, to pre-position the QCOW2 images, run:
+
+```
+./metro-ansible copy_qcow2_files
+```
+
+Then, to skip the image copy during the install:
+
+```
+./metro-ansible install_everything.yml --extra-vars skip_copy_qcow2=True
+```
+
+
 ## Debugging
 
 By default, ansible.cfg tells ansible to log to ./ansible.log.

Documentation/GETTING_STARTED.md

@@ -4,7 +4,7 @@
 
 1.1 [Readme](../README.md) for information on supported components  
 1.2 [Setup](SETUP.md) for setting up the Metro Automation Engine host and enabling SSH  
-1.3 [Customize](CUSTOMIZE.md) for customizing user data and files
+1.3 [Customize](CUSTOMIZE.md) for customizing user data and files  
 1.4 [Release Notes](RELEASE_NOTES.md) for information on the latest features   
 
 ## 2. Setup Metro Automation Engine Host

Documentation/RELEASE_NOTES.md

@@ -1,34 +1,23 @@
 # Metro Automation Engine Release Notes
-## Release 2.4.1
+## Release 2.4.3
 ### New Features and Enhancements
-* Support for Nuage Networks version 5.2.3
-* Add check to verify VSDs are connected to VSCs
-* Add validation for vsd hostname
-* Change remote user from ‘root’ (or nothing) to a variable
-* Add support for checking REST and JMS gateway on VSD and check VSTAT web gateway
-* Update paramiko version in two files
-* Delete all os-compute-*, osc-*, and infra-* from roles and playbooks
-* Change ‘vsc_upgrade_backup_and_prep’ to vsc_sa_upgrade_backup_and_prep’ in UPGRADE.md
-* Add parameter to specify backup location when upgrading
-* Support for master/slave VCIN.
-* Remove deprecated `include:` Ansible commands.
-* Added yum_proxy support to dns.
-* Added static route support for VNSUTIL.
-* Added new roles for installation of VRS compute nodes, vrs-vm.
-
+* add support for new cloud-init version for 5.3.2
+* add support for upgrade to version 5.3.2
+* add suppport for non-root usernames for VSD upgrade
+* add support for NuageX deployment type
+* add support for branding the VSD GUI
+* add NSG bootstrap via activation link
+* add VSD license expiration check
+* update OpenStack Compute and Plugin integration, remove need to specify vsd_ip in myoscs, add handler to restart Neutron-server, reduce time to restart Neutron-server by adding tasks to Neutron-integration idempotent, make vsd-osc-integration equivalent to os-vsd-osc-integration, move stopping of Neutron services to the vrs-oscompute-integration role, change nuage_plugin.ini to be configured to use VSD FQDN
+* improve integration with OpenStack controller, primarily by speeding up lab-installs of Nuage and OpenStack
+* add ability to customize passwords for VSD programs and services
+* add playbook to copy qcow2 files before predeploy step, add checks in predeploy step for qcow2 existence if skipCopyImages is set
 ### Resolved Issues
-* Minor correction in ‘hosts.j2’ vsr section
-* Correct SROS prompt
-* Change ‘inventory hostname’ to ‘vm_name’ for dns image path
-* Fix a failure during pip package check
-* Change ‘inventory hostname’ to ‘vm_name’ for dns image path
-* Add yum update and libguestfs-tools to ‘roles/vrs-vm-deploy/tasks/main.yml’
-* Import validate-build-vars task from common roles
-* Add name ‘nsgv_predeploy’ to ‘install_vns.yml’
-* Delete sgt-qos section of config.cfg.j2
-* Add check for DNS qcow2
-* Add guestfish from the libguestfs-tools package as a prerequisite.
-* The handle_vars playbook did not take into account custom provided build_vars_files or user_creds_file and calculated/verified the MD5 sum of the wrong files (static build_vars.yml and user_creds.yml instead of the provided values.
-* vrs-vr image directory fix.
-* Fix error on dns-predeploy when hostname and vmname are the same.
-* Fix issue with running metro-ansible without root user.
+* fix inconsistency in the way VMs were shutdown during upgrade
+* update dns zones with values from build_vars.yml and solve the firewalld issue from dns-deploy/task/main.yaml file
+* support custom group setting on ansible.log file
+* support doing MD5 checks of user input files in locations other than the current directory
+* removed redundant check for netaddr package
+* fix username for vmware-vm_shell commands in vsd-predeploy
+* fix username for executing monit_waitfor_service task in vstat-vsd-health
+* fix uri task in vstat-health to execute on localhost

Documentation/UPGRADE.md

@@ -8,7 +8,7 @@ Before upgrading any components, you must have previously [set up your Nuage Met
 
 A sample workflow for 5.1.2 to 5.2.2 upgrade. For more detailed workflow refer [Sample HA Metro workflow for an upgrade](#sample-ha-metro-workflow-for-an-upgrade)
 
-After all [Prerequisites](#prerequisites) are met, run the following set of commands in the order specified to upgrade vsd,vsc,vstat deployed in HA/Cluster mode.
+After all prerequisites are met, run the following set of commands in the order specified to upgrade vsd,vsc,vstat deployed in HA/Cluster mode.
 1. ./metro-ansible vsp_preupgrade_health -vvvv
 2. ./metro-ansible vsd_ha_upgrade_database_backup_and_decouple -vvvv
 3. ./metro-ansible vsd_ha_upgrade_shutdown_2_and_3 -vvvv
@@ -162,7 +162,7 @@ If this fails, retry.
 ./metro-ansible vsc_ha_upgrade_deploy_1 -vvvv
 ```
 
-If the step fails, you can retry. Backup plan is to manually copy a valid .tim file to the VSC to affect either the deployment (new version of tim file). (old version of tim file). If that fails, you will need to deploy a new VSC using the old version--or recover the VM from a backup. You can use Metro for the deployment (vsc_predeploy, vsc_deploy, vsc_postdeploy...).
+If the step fails, you can retry. If a retry fails, rollback is accomplished by manually copying (via scp) the .tim file, bof.cfg, and config.cfg that were backed up in step 11 to the VSC. Then reboot the VSC.
 
 13. Run VSC postdeploy on vsc_node1
 
@@ -172,7 +172,7 @@ If the step fails, you can retry. Backup plan is to manually copy a valid .tim f
 
 At this point, you have one VSC running the old version, one running the new. It is time for you to leave this procedure to execute an upgrade of your VRSs, NSGs, and so on.
 
-If this step fails, the recovery is much like that of the previous step: Manually update the tim file or a complete deploy of the old VSC followed by a retry.
+If the step fails, you can retry. If a retry fails, rollback is accomplished by manually copying (via scp) the .tim file, bof.cfg, and config.cfg that were backed up in step 11 to the VSC. Then reboot the VSC.
 
 *Upgrade VRS here!*
 
@@ -190,7 +190,7 @@ If this fails, retry.
 ./metro-ansible vsc_ha_upgrade_deploy_2 -vvvv
 ```
 
-If the step fails, you can retry. Backup plan is to manually copy a valid .tim file to the VSC to affect either the deployment. If that fails, you will need to deploy a new VSC using the old version--or recover the VM from a backup. You can use Metro for the deployment (vsc_predeploy, vsc_deploy, vsc_postdeploy...).
+If the step fails, you can retry. If a retry fails, rollback is accomplished by manually copying (via scp) the .tim file, bof.cfg, and config.cfg that were backed up in step 14 to the VSC. Then reboot the VSC.
 
 16. Run VSC postdeploy on vsc_node2
 
@@ -200,7 +200,7 @@ If the step fails, you can retry. Backup plan is to manually copy a valid .tim f
 
 At this point, you have both VSCs running the new version. It is time for you to upgrade the final VSD.
 
-If this step fails, the recovery is much like that of the previous step: Manually update the tim file or a complete deploy of the old VSC followed by a retry.
+If the step fails, you can retry. If a retry fails, rollback is accomplished by manually copying (via scp) the .tim file, bof.cfg, and config.cfg that were backed up in step 14 to the VSC. Then reboot the VSC.
 
 
 *If VSTAT nodes exist upgrade them using following procedure, if not skip to step 23 to finalize VSP upgrade*

Documentation/VAULT_ENCRYPT.md

@@ -0,0 +1,22 @@
+# Encrypting Sensitive Data with Ansible Vault  
+You can safeguard sensitive data in MetroÆ by encrypting files with Ansible's vault feature. See the steps below for instructions on how to encrypt `user_creds.yml`. More details about the vault feature can be found in [documentation](https://docs.ansible.com/ansible/2.4/vault.html) provided by Ansible.  
+### 1. Create a vault encryption passcode file  
+ Create a file containing a master passcode (example file name: myvault.txt). This passcode can be used to encode and decode all other user data files. It's generally a good idea to keep this file outside of the source code.  
+### 2. Encrypt `user_creds.yml`  
+  To encrypt `user_creds.yml` with the passcode file that you created in step one above, run the following command:  
+  ```
+  ansible-vault encrypt user_creds.yml --vault-password-file myvault.txt
+  ```     
+### 3. Pass the vault password file option  
+  While running MetroÆ commands you can supply the vault password file as an option by running the following command:
+```
+./metro-ansible the_name_of_the_playbook --vault-password-file myvault.txt
+```  
+## Questions, Feedback, and Contributing
+Ask questions and get support via email.  
+  Outside Nokia: [[email protected]](mailto:[email protected] "send email to nuage-metro project")  
+  Internal Nokia: [[email protected]](mailto:[email protected] "send email to nuage-metro project")
+
+Report bugs you find and suggest new features and enhancements via the [GitHub Issues](https://github.com/nuagenetworks/nuage-metro/issues "nuage-metro issues") feature.
+
+You may also [contribute](CONTRIBUTING.MD) to Nuage MetroÆ by submitting your own code to the project.

README.md

@@ -58,7 +58,7 @@ Destroy | removes component(s) from the infrastructure |
 Upgrade | upgrades component(s) from one release to another |
 
 ## Nomenclature  
-**Ansible Host**: The host where Metro Automation Engine runs. Ansible and the required packages are installed on this host. The Ansible Host must run el7 Linux host, e.g. Cent)S 7.* or RHEL 7.*.  
+**Ansible Host**: The host where Metro Automation Engine runs. Ansible and the required packages are installed on this host. The Ansible Host must run el7 Linux host, e.g. CentOS 7.* or RHEL 7.*.  
 **Metro Automation Engine User**: The user who runs Metro Automation Engine to deploy and upgrade components.  
 **Target Server**: The hypervisor on which one or more VSP components are installed as VMs. Each deployment may contain more than one Target Server.  
 

build.yml

@@ -28,6 +28,7 @@
     - name: Include user credential file
       include_vars: "{{ user_creds_file | default('user_creds.yml') }}"
       tags: always
+
   tasks: 
     - include_role:
         name: common
@@ -41,7 +42,7 @@
     - name: Get list of VRSs that require libnetwork to be installed on them
       delegate_to: localhost
       get_libn_host_list:
-        path: "{{ build_vars_file | default ('build_vars.yml') }}"
+        path: "{{ build_vars }}"
       register: rel_vrss
       run_once: True
   roles: