Merge pull request #1 from caktus/DJAN-107_tequila_adoption

DJAN-107 tequila adoption

LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2015, Caktus Consulting Group, LLC
+Copyright (c) 2016, Caktus Consulting Group, LLC
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without modification,

README.rst

@@ -1,210 +1,114 @@
-Tequila
-=======
+tequila-django
+==============
 
-This repository holds a collection of roles for deployments using
-`Ansible <http://www.ansible.com/home>`_.  These exist primarily to
-support the `Caktus Django project template
+This repository holds an `Ansible <http://www.ansible.com/home>`_ role
+that is installable using ``ansible-galaxy``.  This role contains
+tasks used to install and set up a Django web app.  It exists
+primarily to support the `Caktus Django project template
 <https://github.com/caktus/django-project-template>`_.
 
 
 License
 -------
 
-These playbooks are released under the BSD License.  See the `LICENSE
-<https://github.com/caktus/tequila/blob/master/LICENSE>`_ file for
+This Ansible role is released under the BSD License.  See the `LICENSE
+<https://github.com/caktus/tequila-django/blob/master/LICENSE>`_ file for
 more details.
 
 
 Contributing
 ------------
 
-If you think you've found a bug or are interested in contributing to this project
-check out `tequila on Github <https://github.com/caktus/tequila>`_.
+If you think you've found a bug or are interested in contributing to
+this project check out `tequila-django on Github
+<https://github.com/caktus/tequila-django>`_.
 
 Development sponsored by `Caktus Consulting Group, LLC
 <http://www.caktusgroup.com/services>`_.
 
-How to use
-----------
 
-* Create & activate a virtualenv for your project.
-* Add a SPECIFIC tagged version of tequila to your project requirements, e.g.
-  in ``requirements.txt``::
-
-    git+https://github.com/caktus/[email protected]
-
-  This is so once you have Tequila working with the version you have pinned,
-  updates to Tequila won't be applied until you're ready to test them and
-  make sure things still work.
-
-* Install the tequila package into your virtualenv, e.g.::
-
-    pip install -r requirements.txt
-
-* In your project's top-level directory (which will be the current directory
-  when you run deploys), create an ``inventory`` directory.
-* For each environment (e.g. `staging`, `production`), create a new `Ansible
-  inventory file <http://docs.ansible.com/ansible/intro_inventory.html>`_
-  in the ``inventory`` directory, named the same as the environment
-  (with no extension).  The inventory file is an ``ini``-format file.
-
-  The purpose of the inventory file is to specify which hosts are serving which
-  roles in the deploy, *and* how to connect to them. To do this, servers should be
-  added to the groups "db", "worker", and "web", as appropriate, and variables
-  such as ``ansible_ssh_host`` and ``ansible_ssh_user`` can be set on individual
-  hosts.
-
-  Also - *very important* - there must be a group named the same as the environment,
-  that contains all hosts for that environment. That way, any variables that we
-  put into ``inventory/group_vars/<envname>`` will get applied to all the
-  environment's servers by Ansible.
-
-  Example::
-
-      # file: inventory/staging
-
-      # Put all staging servers into this group. We can also
-      # give the servers convenient "names" here, and then use
-      # ansible_ssh_host and other variables to tell Ansible how
-      # to connect to them.
-      [staging]
-      server1 ansible_ssh_host=ec2-gibberish.more.long.domain
-      server2 ansible_ssh_host=ec2-gibberish2.more.long.domain
-
-      [web]
-      server1
-
-      [db]
-      server2
-
-      [worker]
-      # no workers yet for this project
-
-  TBD: Update that list as we add more groups?  Probably better to flesh out this
-  part of the documentation in much greater detail somewhere other than this
-  README.
-
-* Put any variables that should be set for the whole project in a YAML file
-  named ``$(PWD)/inventory/group_vars/all/vars.yml``.  E.g.::
-
-      ---
-      # file: inventory/group_vars/all/vars.yml
-      project_name: our_neat_project
-      python_version: 3.4
-      less_version: 2.1.0
-      postgres_version: 9.3
-
-* Put any variables that should be set for the entire environment in a YAML file
-  named ``$(PWD)/inventory/group_vars/<envname>/vars.yml``.  E.g.::
-
-      ---
-      # file: inventory/group_vars/staging/vars.yml
-      domain: project-staging.domain.com
-      repo:
-        url: [email protected]:caktus/caktus-website.git
-        branch: develop
-
-* For any variables whose values need to be kept secret (e.g. passwords), declare
-  them in the appropriate ``$(PWD)/inventory/group_vars/<dirname>/vars.yml`` file
-  too, but set their value to
-  ``{{ secret_<varname> }}``.  E.g. if the variable is DB_PASSWORD, just put this in
-  the variable file::
-
-      DB_PASSWORD: {{ secret_DB_PASSWORD }}
-
-* Then, set the actual values of the ``secret_<varname>`` variables in a YAML file named
-  ``$(PWD)/inventory/group_vars/<dirname>/secrets.yml`` and encrypt it using the `Ansible
-  vault <http://docs.ansible.com/ansible/playbooks_vault.html>`_, e.g.::
-
-      ansible-vault edit $(PWD)/inventory/group_vars/staging/secrets.yml
-
-  (This two-step approach to secret variables is an
-  `Ansible best practice <http://docs.ansible.com/ansible/playbooks_best_practices.html#variables-and-vaults>`_).
-
-* Put the passwords for the Ansible vault in files named ``.vaultpassword-<envname>``.
-  Be *sure* that (1) they do not get added to version control, and (2) they
-  are not public (e.g. set permissions to 0600).  E.g.::
-
-      echo ".vaultpassword*" >>.gitignore
-      echo "password" >.vaultpassword-staging
-      chmod 600 .vaultpassword-staging
-
-* TODO: Add instructions here for the FIRST deploy. It might need to run
-  as root or ubuntu or whatever the initial user the server has set up
-  is.
-
-* Run ``deploy <envname>`` to update servers.  E.g.::
-
-    deploy staging
-
-  or::
-
-    deploy production
-
-Where to set variables
-----------------------
-
-Ansible supports setting variables in many places. Let's try to agree on some
-common practices for our projects:
-
-* The ``deploy`` script sets a few variables on the command line that take
-  precedence over all others.
-
-    tequila_dir: Directory where tequila was installed.  The ``roles`` subdirectory
-    of this is added to the roles path.
-
-    env_name: Name of the environment being deployed.
-
-    local_project_dir: The current directory when deploy was invoked.
-
-* Variables that are global to the project go in ``inventory/group_vars/all/vars.yml``
-  and ``inventory/group_vars/all/secrets.yml``::
-
-    ---
-    # file: inventory/group_vars/all/vars.yml
-    project_name: our_project
+Installation
+------------
 
-* Variables that apply to all servers in an environment go in
-  ``inventory/group_vars/<envname>/vars.yml`` and
-  ``inventory/group_vars/<envname>/secrets.yml``::
+Create an ``ansible.cfg`` file in your project directory to tell
+Ansible where to install your roles (optionally, set the
+``ANSIBLE_ROLES_PATH`` environment variable to do the same thing, or
+allow the roles to be installed into ``/etc/ansible/roles``) ::
 
-    ---
-    # file: inventory/group_vars/staging/vars.yml
-    domain: project-staging.example.com
+    [defaults]
+    roles_path = deployment/roles/
 
-* Variables whose values should be secret should be declared in the same
-  files as other variables, depending on their scope, but their value
-  should be set to ``{{ secret_<varname> }}``::
+Create a ``requirements.yml`` file in your project's deployment
+directory ::
 
     ---
-    # file: inventory/group_vars/staging/vars.yml
-    DB_PASSWORD: {{ secret_DB_PASSWORD }}
-
-* For each secret variable mentioned in ``inventory/group_vars/<dirname>/vars.yml``,
-  declare its actual value in ``inventory/group_vars/<dirname>/secrets.yml``.  E.g.
-  if DB_PASSWORD is set to ``{{ secret_DB_PASSWORD }}`` in
-  ``inventory/group_vars/staging/vars.yml``, then in ``inventory/group_vars/staging/secrets.yml``
-  we would expect to see::
-
-      ---
-      # file: inventory/group_vars/staging/secrets.yml
-      secret_DB_PASSWORD: "value of password"
-
-* Variables telling Ansible how to connect to a particular host go into
-  the inventory file, on the same line as the first mention of that host.
-
-TODO for this README
---------------------
-
-TODO: Add full documentation for the ``deploy`` script.
-
-TODO: Create more detailed documentation, including which groups to use and
-what variables need to be set, and lots of examples of the whole process
-
-TODO: document that setting force_ssl False will make port 80 also serve
-Django rather than redirecting to https.
-
-TODO: add hstore & postgis
-
-TODO: document that setting source_is_local True will sync the project
-files from the current directory instead of pulling them from git.
+    # file: deployment/requirements.yml
+    - src: https://github.com/caktus/tequila-django
+      version: 0.1.0
+
+Run ``ansible-galaxy`` with your requirements file ::
+
+    $ ansible-galaxy install -r deployment/requirements.yml
+
+or, alternatively, run it directly against the url ::
+
+    $ ansible-galaxy install git+https://github.com/caktus/tequila-django
+
+The project then should have access to the ``tequila-django`` role in
+its playbooks.
+
+
+Variables
+---------
+
+The following variables are made use of by the ``tequila-django``
+role:
+
+- ``project_name`` **required**
+- ``env_name`` **required**
+- ``domain`` **required**
+- ``is_web`` **default:** ``false`` (**required:** one of ``is_web`` or ``is_worker`` set to ``true``)
+- ``is_worker`` **default:** ``false``
+- ``python_version`` **default:** ``"2.7"``
+- ``root_dir`` **default:** ``"/var/www/{{ project_name }}"``
+- ``source_dir`` **default:** ``"{{ root_dir }}/src"``
+- ``venv_dir`` **default:** ``"{{ root_dir }}/env"``
+- ``ssh_dir`` **default:** ``"/home/{{ project_name }}/.ssh"``
+- ``requirements_file`` **default:** ``"{{ source_dir }}/requirements/{{ env_name }}.txt"``
+- ``requirements_extra_args`` **default:** ``""``
+- ``use_newrelic`` **default:** ``false``
+- ``new_relic_license_key`` **required if use_newrelic is true**
+- ``gunicorn_version`` **optional**
+- ``gunicorn_num_workers`` **required**
+- ``gunicorn_num_threads`` **optional** (note: gunicorn sets this at ``1`` if ``--threads=...`` is not given)
+- ``project_user`` **default:** ``"{{ project_name }}"``
+- ``project_settings`` **default:** ``"{{ project_name }}.settings.deploy"``
+- ``secret_key`` **required**
+- ``db_name`` **default:** ``"{{ project_name }}_{{ env_name }}"``
+- ``db_user`` **default:** ``"{{ project_name }}_{{ env_name }}"``
+- ``db_host`` **default:** ``'localhost'``
+- ``db_port`` **default:** ``5432``
+- ``db_password`` **required**
+- ``cache_host`` **optional**
+- ``broker_host`` **optional**
+- ``broker_password`` **optional**
+- ``celery_worker_extra_args`` **default:** ``"--loglevel=INFO"``
+- ``static_dir`` **default:** ``"{{ root_dir }}/public/static"``
+- ``media_dir`` **default:** ``"{{ root_dir }}/public/media"``
+- ``log_dir`` **default:** ``"{{ root_dir }}/log"``
+- ``repo`` **required:** dict containing url and branch
+- ``source_is_local`` **default:** ``false``
+- ``github_deploy_key`` **required if source_is_local is false**
+- ``local_project_dir`` **required if source_is_local**
+- ``global_npm_installs`` **default:** empty list
+- ``extra_env`` **default:** empty dict
+
+The ``global_npm_installs`` variable is a list of dicts, with required
+key ``name`` for the package name, and optional key ``version`` for
+the desired version to install.  This is needed for the less or sass
+packages, to make them available where supervisord and gunicorn can
+find them.
+
+The ``extra_env`` variable is a dict of keys and values that is
+desired to be injected into the environment as variables, via the
+``envfile.j2`` template.

tequila/roles/common/defaults/main.yml → defaults/main.yml

@@ -1,25 +1,21 @@
 ---
-users: []
-pg_version: 9.3
-project_user: "{{ project_name }}"
-project_settings: "{{ project_name }}.settings.deploy"
+python_version: "2.7"
 root_dir: "/var/www/{{ project_name }}"
 source_dir: "{{ root_dir }}/src"
-static_dir: "{{ root_dir }}/public/static"
-media_dir: "{{ root_dir }}/public/media"
 venv_dir: "{{ root_dir }}/env"
-log_dir: "{{ root_dir }}/log"
-public_dir: "{{ root_dir }}/public"
-ssl_dir: "{{ root_dir }}/ssl"
 ssh_dir: "/home/{{ project_name }}/.ssh"
-self_signed: true
+requirements_file: "{{ source_dir }}/requirements/{{ env_name }}.txt"
+project_user: "{{ project_name }}"
+project_settings: "{{ project_name }}.settings.deploy"
 db_name: "{{ project_name }}_{{ env_name }}"
-db_host: localhost
-db_port: 5432
 db_user: "{{ project_name }}_{{ env_name }}"
-dhparams_file: "{{ ssl_dir }}/dhparams.pem"
-
-# Set this false to turn off forcing all requests to SSL
-force_ssl: true
-
-app_minions: "{{ groups['web'] | union(groups['worker']) }}"
+db_host: 'localhost'
+db_port: 5432
+static_dir: "{{ root_dir }}/public/static"
+media_dir: "{{ root_dir }}/public/media"
+log_dir: "{{ root_dir }}/log"
+use_newrelic: false
+source_is_local: false
+is_web: false
+is_worker: false
+extra_env: {}

files/dotenv.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+# Load env vars from $PWD/.env, then exec rest of command line.
+#
+# E.g. to run "ls -h" with the settings from .env loaded as env
+# vars, do:
+#
+#   dotenv.sh ls -h
+#
+# in the directory where .env is.
+
+if [ -e .env ] ; then
+    # set -a -> all variables which are modified or created will be exported
+    set -a
+    source .env
+fi
+
+# Now exec the rest of the line, preserving quoting
+exec "$@"