(c) Copyright 2015 Hewlett Packard Enterprise Development LP (c) Copyright 2017-2018 SUSE LLC
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
The Ardana Ansible codebase is organised into dedicated <service>-ansible repos per service, e.g. nova-ansible, cinder-ansible, etc., where each <service>-ansible contains all of the Ansible logic for the various lifecycle phases of that service, e.g. deploy, reconfigure and upgrade. For all of the core services (nova, cinder, etc.), each <service>-ansible repo is owned and managed by the relevant service team, where the additional services (database, messageq, etc.) fall under the ownership of the Ardana team.
The skeleton-ansible-repo dir provides a template and contains information required for the service teams to develop the service specific Ansible repos.
This doc contains the following sections:
- Brief overview of the overall Ardana Ansible structure with links to the separate docs within skeleton-ansible dir that provide more detail for developers working on Ardana playbooks for managing the lifecycle of an OpenStack service.
- Brief overview of the Ansible output generated by the Ardana Configuration Processor, which is consumed by the Ardana Ansible playbooks. Links to further detailed docs on this repo are also provided.
- A more detailed description of the main components of a Ardana <service>-ansible repo, including a sample service repo layout.
There is also a style guide available here: ansible-style-guide.md. Learning these rules will help reviewers approve your changes quickly.
The concept of lifecycle management and the aim of ensuring ease of cloud operation are central to this new approach. To this end, we have identified high-level lifecycle phases or "operations" that are required for the cloud as a whole and for each service, e.g. deploy, reconfigure and upgrade, where a service refers to nova, cinder, glance, etc. Each service lifecycle operation in turn uses lower-level "verbs", which are defined per service-component, where examples of a service-component are nova-api, nova-scheduler, etc. and example verbs are install, configure, start and stop.
Hence there is a hierarchy of invoked operations that may look something like:
cloud deploy
─> nova deploy
─> nova-api
─> install
─> configure
─> nova-conductor
─> install
─> configure
─> nova-post-configure
─> db_create
─> keystone_setup
─> nova-api
─> start
─> nova-conductor
─> start
...
─> glance deploy
─> glance-api
─> install
─> configure
─> glance-post-configure
─> db_create
─> keystone_setup
─> glance-api
─> start
...
The general layout of a <service>-ansible repo reflects this structure and is as follows:
- An Ansible role is defined per service-component, e.g. nova-api, nova-scheduler, etc.; additional private roles may also be defined, see detailed overview below.
- Each role comprises a set of standard verb playbooks, e.g. install.yml, configure.yml and start.yml, in addition to service-specific or private verbs.
- A top-level playbook is defined for each operation, e.g. nova-deploy.yml, nova-reconfigure.yml and nova-upgrade.yml; additional private operations may also be defined, see detailed overview below.
- The top-level operation playbooks combine and orchestrate the lower-level service-component verb playbooks in a series of plays.
Each service team is responsible for implementing the service-operation playbooks, service-component roles and their constituent verb playbooks, in addition to any private roles, operations or service-specific verb playbooks.
The skeleton-ansible-repo dir is laid out according to the general structure expected of each <service>-ansible and serves as a template for any new per-service Ansible repo being worked on in Ardana. Details on what is expected in each of the Ansible files in a Ardana Ansible repo (top-level operation playbooks, verb playbooks, vars files, etc.) are presented as separate document files in the relevant location within skeleton-ansible-repo.
Note that this documentation exercise is a work in progress, which is being updated as new Ardana features are added, so some of the documents will be empty until completed. You will also notice README.md files in otherwise empty directories. These files contain information about the use of those directories and what should exist within them.
For convenience, links to the current set of completed docs are listed here:
- Operation playbook docs:
- service-deploy.yml.md: details structure of a top-level service-deploy playbook.
- TODO: service-start.yml.md
- TODO: service-stop.yml.md
- service-cloud-configure.yml.md: details structure of a top-level service-cloud-configure playbook.
- Standard verb playbook docs:
The Configuration Processor (CP) is a key part of the Ardana framework. The CP takes as inputs a description of the desired cloud. This is a combination of:
-
A definition of the services and their inter-dependencies (for example nova-api depends on mysql and rabbitmq). This data is provided as part of Ardana and not modified by the customer. The service-component names must match the filters used in the high-level playbooks to identify the target set of nodes for a play, see examples in service-deploy.yml.md.*
-
A description of the logical architecture of the cloud (what region it has, which servcies run in which region, which are co-located, etc). This is data provided by the customer, possibly using Ardana supplied examples.
-
A description of the physical architecture of the cloud (what servers and networks to use, etc). Again this is customer supplied data which may be based on Ardana supplied examples.
The config processor combines all of the above data to produce configuration data that can be used to deploy and configure the cloud. So for example if the customer has decided to create a cloud with two regions, each of which includes nova-api it is the config processor that will produce different configuration data for nova-api in each region so that:
- nova-api is given the correct mysql endpoint to connect to
- ha-proxy in each region is given the correct set of nova-api endpoints
- nova api is given the details of the URL it needs to register in keystone
In short the role of the configuration processor is to keep the service definitions (in its inputs) and the Ansible code isolated from needing to have any built-in knowledge of the cloud topology.
To do this the CP generate a set of Ansible artefacts: hosts files, group_vars files and host_vars files:
-
The hosts files contain hostgroups for each service-component (e.g. nova-api, nova-scheduler, etc.) - this allows each service to determine the list of hosts where the service-component is to be installed.
-
The vars files contain the server specific configuration values for each service. The vars generated in host_vars and group_vars are derived from the service-component names defined in the inputs to CP, so these must match the relevant variable references used in the Ardana Ansible playbooks being described here.
More details on how the relationships between service components are expressed in CP and how this maps to the output Ansible variables will be presented in separate docs:
-
Database: details how service playbooks can connect to and administer a database for the requirements of that service, e.g. setup database and users.
-
For examples of existing playbooks using CP variables, see:
- Current set of Ardana playbooks on the service Ansible repos, e.g. keystone-ansible and db-ansible. More examples of these are coming onstream as services teams make progress with Ardana.
The remainder of this doc contains a more detailed description of the <service>-ansible repo layout.
The structure of a Ardana Ansible repo is a combination of the structure imposed by Ansible itself and the particular operation and verb playbook layout of Ardana. For further information on Ansible see http://www.ansible.com/home and in particular the documentation at http://docs.ansible.com/. At a high-level, the structure is as follows:
└── <service>-ansible
├── filter_plugins
├── library
├── <service>-<operation-1>.yml
├── <service>-<operation-2>.yml
├── <service>-<operation-3>.yml
├── _<service>-<private operation-1>.yml
├── _<service>-<private operation-2>.yml
└── roles
├── <service-component-role-1>
│ ├── defaults
│ ├── files
│ ├── handlers
│ ├── meta
│ ├── tasks
│ │ ├── <verb 1>.yml
│ │ ├── <verb 2>.yml
│ │ ├── <verb 3>.yml
│ │ ├── _<private verb 1>.yml
│ │ ├── _<private verb 2>.yml
│ │ └── .....
│ ├── templates
│ └── vars
├── <service-component-role-2>
├── <service-component-role-3>
├── ....
├── _<private-service-role-1>
│ ├── defaults
│ ├── files
│ ├── handlers
│ ├── meta
│ ├── tasks
│ │ └── .....
│ ├── templates
│ └── vars
│ └── main.yml
├── _<private-service-role-2>
└── ....
Each <service>-ansible repo contains service operation playbook files at the top level. These yaml files coordinate the service-component verbs into plays to carry out major lifecycle operations, such as deploy, reconfigure or upgrade. It is hoped that these operations will not, in the main, cross multiple services. The operation playbooks describe how the service-component role "verbs" are orchestrated, and handle any inter-role dependencies. The overarching Ardana orchestration layer (e.g. ardana-deploy.yml) will invoke the service operation playbooks. In exceptional circumstances it is possible that this overarching orchestration layer will invoke individual verbs directly.
Ardana provides playbooks that combine all service playbooks for a single operation, e.g. ardana-deploy invokes <service>-deploy for services in order to deploy a complete cloud. Each <service>-operation playbook, e.g. nova-deploy is also runnable on its own. The set of operation playbooks represent the API that is available for the operator of the cloud, both at the cloud level and the per-service level. For example, the API for managing the lifecycle of the nova service could comprise the following set of playbooks:
- nova-deploy.yml
- nova-cloud-configure.yml
- nova-reconfgure.yml
- nova-stop.yml
- nova-start.yml
- nova-upgrade.yml
The above set of example playbooks for nova can be used standalone or as part of another operation playbook. For example, the nova-deploy playbook will end by including the nova-start playbook to start all nova services. It is generally useful to group repeated sets of plays into high-level playbooks for re-use between these operation playbooks. For example, nova can use a configure playbook that has a series of plays that each invoke the configure verb for each of the nova service components:
---
- hosts: "{{ target_hosts | default('all') }}:&NOV-CND"
become: yes
roles:
- NOV-CND
tasks:
- include: roles/NOV-CND/tasks/configure.yml
- hosts: "{{ target_hosts | default('all') }}:&NOV-API"
become: yes
roles:
- NOV-API
tasks:
- include: roles/NOV-API/tasks/configure.yml
- hosts: "{{ target_hosts | default('all') }}:&NOV-SCH"
become: yes
roles:
- NOV-SCH
tasks:
- include: roles/NOV-SCH/tasks/configure.yml
.... remaining nova components.
Ardana has the concept of private operation playbooks that are not part of the API for managing the lifecycle of nova. Private operation playbooks are indicated by preceding the playbook name with a "_" character, e.g. _nova-configure.yml. This playbook can be invoked from multiple operation playbooks (e.g. nova-deploy, nova-reconfigure and nova-upgrade), but is not executed on its own.
Roles are located as sub-directories of the "roles" directory at the top level of the repo. A majority of the Ansible roles in Ardana will map to service components, e.g. nova-api, nova-scheduler, cinder-api and the convention is to give the Ansible role the same name as given to that service component in the service/*.json inputs into the CP. These roles are directly referenced in the top-level playbooks (public or private).
It is also common for a service to group a set of tasks in a role that is only directly referenced by other roles within the service; a typical example is a common role that contains tasks shared between multiple roles within the service. Such a role is termed an private role and Ardana uses the convention of preceding an private role name with a "_" character, e.g. _SWF-CMN. Private roles are not referenced in high-level operation playbooks, such as deploy or upgrade; instead, their constituent verbs are directly included by verbs in other roles, e.g. the swift object install verb:
---
- include: ../../_SWF-CMN/tasks/install.yml
- name: SWF-OBJ | install | Install Swift Object services
install_package: name=swift service={{ item.key }} state=present
with_dict: "{{ object_services }}"
which uses the install verb from an interal swift common role, and the swift object configure verb:
---
- include: ../../_SWF-CMN/tasks/configure.yml
- include: ../../_SWF-RSY/tasks/configure.yml
.... additional tasks for configuring swift object component.
which uses two private swift roles. In addition, the swift object role needs to specify a dependency on the swift common role by including the following in SWF-OBJ/meta/main.yml:
---
dependencies:
- role: _SWF-CMN
The above ensures that all variables required for the included _SWF-CMN or _SWF-RSY verbs are present when invoked from SWF-OBJ verbs.
Each of the above-mentioned roles contain a set of "verbs", which exist as individual yaml files under the "tasks" subdirectory of each role. The Ardana development team have identified a set of standard verbs that are believed to be applicable to a majority of service-components across an Openstack deployment. The expectation is that service teams will define service-component roles that include implementations of these verbs as separate yaml task files. Each verb should also include a validate step at the end, to confirm that the verb has completed successfully, or raise an error otherwise. Note, however, that some of the identified verbs may not be relevant to all service-components. Furthermore, it is recognised that service teams will also need to implement service-specific playbooks within a role that are not one of the standard verbs. The structure and naming of each service Ansible repo is ultimately under the control of the service team; this guide is attempting to enforce a convention to maintain consistency across repos and the Ansible codebase in general.
The verbs described above are invoked from a higher-level service operation playbook, such as nova-deploy.yml. Service teams can also define private verbs or task files within a role that are not intended for use outside the role, i.e. they are only invoked by other verbs within that role. The convention in Ardana is to prefix the name of these task files with a "_" character, e.g. "_join_cluster.yml" in the rabbitmq Ansible codebase (mq-ansible repo).
The "verbs" in skeleton-ansible-repo contain some text to describe the function of each of the standard verbs.
For illustration, a sample Ansible repo structure for nova is shown below (Note: this structure does not necessarily match the current nova-ansible repo state). This layout contains the following:
-
A _nova-common role, which contains verbs that are included in other role verbs.
-
A nova-post-configure role containing verbs for tasks that need to be executed on one node before starting the nova services.
-
A nova-cloud-configure role containing a verb for the setup of nova flavors.
-
A set of roles for nova service component. Each of these roles contain some or all of the standard verbs:
└── nova-ansible ├── filter_plugins ├── library ├── nova-deploy.yml ├── nova-cloud-configure.yml ├── nova-start.yml ├── nova-stop.yml ├── _nova-install.yml ├── _nova-configure.yml ├── README.md └── roles ├── _nova-common │ ├── defaults │ ├── files │ ├── handlers │ ├── meta │ ├── tasks │ │ ├── install.yml │ │ └── configure.yml │ ├── templates │ └── vars ├── nova-post-configure │ ├── defaults │ ├── files │ ├── handlers │ ├── meta │ ├── tasks │ │ ├── db_create.yml │ │ ├── db_configure.yml │ │ └── keystone_conf.yml │ ├── templates │ └── vars ├── nova-cloud-configure │ ├── defaults │ ├── files │ ├── handlers │ ├── meta │ ├── tasks │ │ └── add_flavors.yml │ ├── templates │ └── vars ├── NOV-API │ ├── defaults │ ├── files │ ├── handlers │ ├── meta │ ├── tasks │ │ ├── install.yml │ │ ├── stop.yml │ │ ├── configure.yml │ │ └── start.yml │ ├── templates │ └── vars │ └── main.yml ├── NOV-CAU │ ├── defaults │ ├── files │ ├── handlers │ ├── meta │ ├── tasks │ │ ├── install.yml │ │ ├── configure.yml │ │ ├── stop.yml │ │ └── start.yml │ ├── templates │ └── vars ├── NOV-CMP │ ├── defaults │ ├── files │ ├── handlers │ ├── meta │ ├── tasks │ │ ├── install.yml │ │ ├── configure.yml │ │ ├── start.yml │ │ └── stop.yml │ ├── templates │ └── vars ├── NOV-CND │ ├── defaults │ ├── files │ ├── handlers │ ├── meta │ ├── tasks │ │ ├── install.yml │ │ ├── configure.yml │ │ ├── start.yml │ │ └── stop.yml │ ├── templates │ └── vars ├── NOV-SCH │ ├── defaults │ ├── files │ ├── handlers │ ├── meta │ ├── tasks │ │ ├── install.yml │ │ ├── configure.yml │ │ ├── start.yml │ │ └── stop.yml │ ├── templates │ └── vars └── NOV-VNC ├── defaults ├── files ├── handlers ├── meta ├── tasks │ ├── install.yml │ ├── configure.yml │ ├── start.yml │ └── stop.yml ├── templates └── vars