-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Paul Trappitt
committed
Feb 13, 2016
1 parent
de9b678
commit 11aab6e
Showing
6 changed files
with
311 additions
and
277 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,19 +4,19 @@ | |
| [](https://aircover.co/drone-plugins/drone-ecs) | ||
| [](https://imagelayers.io/?images=plugins/drone-ecs:latest 'Get your own badge on imagelayers.io') | ||
|
|
||
| Drone plugin to deploy or update a project on AWS ECS. This has been model on the original drone-ecs plugin written in go, thanks to the drone team for their efforts and open sourcing their sofwtare. | ||
| Drone plugin to deploy or update a project on AWS ECS. This has been model on the original drone-ecs plugin written in go, thanks to the drone team for their efforts and open sourcing their sofwtare. | ||
|
|
||
| The original plugin while fully functional had not been designed or intended to be used with environments setup using cloud formation or complex ecs clusters with multiple servcies, tasks and multiple containers in task definitions. This plugin is intended to preserve existing configurations and handle the dynamic nature of cloudformation resources by using wildcards instead of fixed names. You can still use this plugin and the added config features with fixed names as well. It works exactly the same for both setups. | ||
|
|
||
| Another challenge with cloudformation is the management of enviornment variables. The are scenarios where environment variables for task deifnitions that point to other resources are set dynamically when the cloudformation script is run. Having to maintain these variable values manually in the drone.yml is not practical. | ||
| Another challenge with cloudformation is the management of enviornment variables. The are scenarios where environment variables for task deifnitions that point to other resources are set dynamically when the cloudformation script is run. Having to maintain these variable values manually in the drone.yml is not practical. | ||
|
|
||
| This plugin handles this situation by first retrieving the existing task definitions and using them as a template for the new definition. This way the configuration is preserved from the initial cloudformation deployment through subsequent drone deployments. In order to update the configuration, the plugin provides the ability to make changes to explicit parts of the configuration as required through the drone.yml file, while leaving the remaining configuration in tact. | ||
| This plugin handles this situation by first retrieving the existing task definitions and using them as a template for the new definition. This way the configuration is preserved from the initial cloudformation deployment through subsequent drone deployments. In order to update the configuration, the plugin provides the ability to make changes to explicit parts of the configuration as required through the drone.yml file, while leaving the remaining configuration in tact. | ||
|
|
||
| Full examples of all functionality are available below. | ||
|
|
||
| This plugin hasn't been extensively tested. We do have a fairly complex ecs environment which covers all the scenarios the plugin can handle. Over the coming months more and more testing will take place. | ||
| This plugin hasn't been extensively tested. We do have a fairly complex ecs environment which covers all the scenarios the plugin can handle. Over the coming months more and more testing will take place. | ||
|
|
||
| If you find it useful and want to contribute, pull requests are more than welcome. I'll do my best to address any issues as quickly as possible, but please bear in mind, we built this to address a pressing need in our business. Our core business project is our current focus, so for larger issues or feature requests, unless they are impacting us or of value to our project regrettably at the present time I can't give them much priority. | ||
| If you find it useful and want to contribute, pull requests are more than welcome. I'll do my best to address any issues as quickly as possible, but please bear in mind, we built this to address a pressing need in our business. Our core business project is our current focus, so for larger issues or feature requests, unless they are impacting us or of value to our project regrettably at the present time I can't give them much priority. | ||
|
|
||
| ## Using the plugin in drone.yml | ||
|
|
||
|
|
@@ -27,34 +27,34 @@ deploy: | |
| ecs: | ||
| image: fridaystreet/drone-ecs-node //path to plugin repo | ||
| region: ap-southeast-2 //aws region to use, currently only supports single region | ||
| access_key: $$AWS_KEY //aws access key | ||
| access_key: $$AWS_KEY //aws access key | ||
| secret_key: $$AWS_SECRET //aws secret key | ||
| image_name: registry.mydomain.com/dashboard //name of image without tag. | ||
| image_name: registry.mydomain.com/dashboard //name of image without tag. | ||
| image_tag: "1.0.$$BUILD_NUMBER" //image tag | ||
| cluster: Production-DashboardCluster //base cluster name / wildcard | ||
| family: Production-DashboardTaskDefinition //base family name / wildcard | ||
| service: Production-DashboardService //base service name / wildcard | ||
| constainer_name: dashboard //the name of the container in the definition that uses this image | ||
| constainer_name: dashboard //the name of the container in the definition that uses this image | ||
| allow_multiple_clusters: false //update services on multiple clusters if matched | ||
| allow_multiple_services: false //update multiple services on a clusters if matched | ||
| log_level: 'debug' //logging level to output | ||
| task_definition: myTaskDefFile.json //relevant path to JSON format taskDefinition file | ||
| ``` | ||
| ### Settings explained | ||
|
|
||
| Note - The following settings from drone-ecs are not used in drone-ecs-node. | ||
| Note - The following settings from drone-ecs are not used in drone-ecs-node. | ||
|
|
||
| *port_mappings | ||
| *memory | ||
| *environment_variables | ||
| *environment_variables | ||
| *NAME=VALUE | ||
|
|
||
| These settings are now handle in an ecs task definition configuration object. See below for details. | ||
| These settings are now handle in an ecs task definition configuration object. See below for details. | ||
|
|
||
| Note - Any settings that aren't listed below, operate in exactly the same way as drone-ecs http://readme.drone.io/plugins/ecs | ||
| Note - Any settings that aren't listed below, operate in exactly the same way as drone-ecs http://readme.drone.io/plugins/ecs | ||
|
|
||
| ####cluster | ||
| The cluster, service & family settings all do indexOf matches on the full Arn of the relative resource. | ||
| The cluster, service & family settings all do indexOf matches on the full Arn of the relative resource. | ||
|
|
||
| This means that is you have a cluster with an Arn of : | ||
|
|
||
|
|
@@ -80,27 +80,27 @@ Again a good example is if you have a environment where instead of using differe | |
| The family parameter operates as a wildcard as well, but it is used to match to task definitions. When the plugin matches a cluster and service, it retrieves all the current tasks within that service. From these tasks it derives the currently active task definitions. These are matched / filtered by this setting to determine which task definitions to update. | ||
|
|
||
| ####constainer_name | ||
| The container name is required in order to determine which conatiner within the task definition to update the image name for this build. This setting is intended to accomodate definitions with multiple containers. | ||
| The container name is required in order to determine which conatiner within the task definition to update the image name for this build. This setting is intended to accomodate definitions with multiple containers. | ||
|
|
||
| Note - It is a non-case sensitive exact match on the name. It does not support wildcards. | ||
|
|
||
| ####allow_multiple_clusters (optional) | ||
| This defaults to false as a safety precuation. If set to true it will allow matched definitions in multiple clusters to be updated at once. | ||
| This defaults to false as a safety precuation. If set to true it will allow matched definitions in multiple clusters to be updated at once. | ||
|
|
||
| ####allow_multiple_services (optional) | ||
| This defaults to false as a safety precuation. If set to true it will allow matched definitions in multiple services in the same cluster to be updated at once. | ||
| This defaults to false as a safety precuation. If set to true it will allow matched definitions in multiple services in the same cluster to be updated at once. | ||
|
|
||
|
|
||
| ####task_definition (optional) | ||
| The task_definition setting lets you specify a full or part ECS JSON formatted task definition to override one or all settings within the existing task defintion. | ||
| The task_definition setting lets you specify a full or part ECS JSON formatted task definition to override one or all settings within the existing task defintion. | ||
|
|
||
| The json string should be stored in a file in the workspace. ie if you had a file called taskdef.json store in the root of the workspace alongside the drone.yml file you would set the parameter to: | ||
|
|
||
| **task_definition: taskdef.json | ||
|
|
||
| The intent of this setting was to be able to supply only those settings that needed to be modified and again not needing to maintain the entire definition, where some settings may have been added dynamically by the CF deployment. | ||
|
|
||
| Some sugar has been added to the json format in order to support some of this functionality of updating / removing / adding settings in instances where the value of the parameter in the config is an array of objects. | ||
| Some sugar has been added to the json format in order to support some of this functionality of updating / removing / adding settings in instances where the value of the parameter in the config is an array of objects. | ||
|
|
||
| This is acomplished by a recursive merging process that drills down through the entire object processing each child individually. | ||
|
|
||
|
|
@@ -123,9 +123,9 @@ Output | |
|
|
||
| #####Modifying values that are part of an object in an array of objects | ||
|
|
||
| When the configurations are merged together and the plugin hits an array of objects in the provided json file, it looks to see if a 'keys' parameter exists. If it does, then it looks through all of the array items in the current config and tries to match the values of those keys in the new config to values of the keys in the old config. It's pretty much just setting up a hash of fields in order to make a uniuqe match. | ||
| When the configurations are merged together and the plugin hits an array of objects in the provided json file, it looks to see if a 'keys' parameter exists. If it does, then it looks through all of the array items in the current config and tries to match the values of those keys in the new config to values of the keys in the old config. It's pretty much just setting up a hash of fields in order to make a uniuqe match. | ||
|
|
||
| You can use one or more fields, just depends how specific you need to be in order to make the match. For environment vars for instance, no 2 will have the same name, so just specifying name would be enough. | ||
| You can use one or more fields, just depends how specific you need to be in order to make the match. For environment vars for instance, no 2 will have the same name, so just specifying name would be enough. | ||
|
|
||
| If it is able to match all of the keys for a given object in the array, then it will look to merge any other parameters in the original config that exist in the new config and are not set as keys. | ||
|
|
||
|
|
@@ -227,14 +227,14 @@ Output | |
|
|
||
|
|
||
| #####Removing items from a simple array | ||
| This is not yet implemented. | ||
| This is not yet implemented. | ||
|
|
||
| #####Examples | ||
| Lets say you have a full container definition in ecs like this | ||
| Lets say you have a full container definition in ecs like this | ||
| ``` | ||
| { | ||
| containerDefinitions: | ||
| [ | ||
| [ | ||
| { name: 'dashboard', | ||
| image: 'registry.mydomain.com/dashboard-prod:1.8.0', | ||
| cpu: 110, | ||
|
|
@@ -262,31 +262,31 @@ Lets say you have a full container definition in ecs like this | |
|
|
||
| And you want to; 1. modify the port mapping, 2. Add an ENV var, 3. delete an ENV var. All you would need to specify in your json file, is the structure and values for the items you want to modify. Like this: | ||
|
|
||
| ``` | ||
| ``` | ||
| { | ||
| containerDefinitions: | ||
| [ | ||
| [ | ||
| { | ||
| name: 'dashboard', | ||
| portMappings: | ||
| [ | ||
| { containerPort: 80, hostPort: 8070, protocol: 'udp' , keys:['containerPort', 'hostPort']} | ||
| ], | ||
| environment: [ | ||
| environment: [ | ||
| { name: 'LOGGING_SERVER', value: 'log.mydomain.com'}, | ||
| { name: 'SOMEKEY', keys: ['name'] , remove:true} | ||
| ], | ||
| keys: ['name','cpu'] | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| ``` | ||
| Let's take the containerDefintions parameter to start with. It's in the root of the task defintiion structure and it is an array of objects detailing each container. Inside the root of the first container object, our updated file has an additional parameter called 'keys' | ||
|
|
||
| The keys in the root of the first container in the array are 'name' & 'cpu'. So as name and cpu in the new configuration match a container in the old configuration, any parameters in the root of the container object in the new configuration not set as keys will be updated. In this case environment and portMappings can now be put through the merge process. | ||
|
|
||
|
|
||
| The best way to think about it from this point is that the merge process is now passed the values of the portMappings parameter and starts the whole process above over again, but with the following | ||
| The best way to think about it from this point is that the merge process is now passed the values of the portMappings parameter and starts the whole process above over again, but with the following | ||
| ``` | ||
| { | ||
| portMappings: | ||
|
|
@@ -295,7 +295,7 @@ The best way to think about it from this point is that the merge process is now | |
| ] | ||
| } | ||
| ``` | ||
| So stepping through it again, we have portMappings at the root, which is an array of objects. It iterates through each object looking for a 'keys' parameter. In this case we have one and it's asking to match containerPort & hostPort. | ||
| So stepping through it again, we have portMappings at the root, which is an array of objects. It iterates through each object looking for a 'keys' parameter. In this case we have one and it's asking to match containerPort & hostPort. | ||
|
|
||
| It pulls the portMappings array from the original ECS config and looks through the keys of each item to match containerPort: 80, hostPort: 8070. In our case there is a match and as there is another parameter 'protocol' in our new config which isn't in the keys array for this object, the original config parameter is updated to be protocol: 'udp' | ||
|
|
||
|
|
@@ -311,7 +311,7 @@ The allowed log levels are | |
|
|
||
| *info | ||
|
|
||
| outputs a guide on what's currently being done, eg Fetching clsuters from ECS. | ||
| outputs a guide on what's currently being done, eg Fetching clsuters from ECS. | ||
| *fatal | ||
|
|
||
| all ecs requests will throw an exception if there is an error | ||
|
|
@@ -328,8 +328,61 @@ The allowed log levels are | |
|
|
||
| ##Development | ||
|
|
||
| ### Install | ||
| ### Install | ||
| ``` | ||
| npm install | ||
| ``` | ||
|
|
||
| ### Example | ||
|
|
||
| (these aren't real aws keys btw) | ||
|
|
||
| ``` | ||
| node app.js -- '{"repo":{"clone_url":"git://github.com/drone/drone","owner":"drone","name":"drone","full_name":"drone/drone"},"system":{"link_url":"https://beta.drone.io"},"build":{"number":22,"status":"success","started_at":1421029603,"finished_at":1421029813,"message":"UpdatetheReadme","author":"johnsmith","author_email":"[email protected]","event":"push","branch":"master","commit":"436b7a6e2abaddfd35740527353e78a227ddcb2c","ref":"refs/heads/master"},"workspace":{"root":"/drone/src","path":"/drone/src/github.com/drone/drone"},"vargs":{"build":"","repo":"","access_key":"AHGDGJHhJ8677A2Q","secret_key":"IrlHJHGShjgsgsdgJHGSjhgsdgJJGU8JTRhSb","region":"ap-southeast-2","family":"Development-DashboardTaskDefinition","cluster":"Development-ServicesCluster","service":"Development-DashboardService","constainer_name":"dashboard","allow_multiple_clusters":false,"allow_multiple_services":false,"image_name":"registry.mydomain.com.au/dashboard-development","image_tag":"1.0.1","log_level":"debug","task_definition":"taskDef.json"}}' | ||
| ``` | ||
|
|
||
| Friendly version | ||
| ``` | ||
| {"repo": {"clone_url": "git://github.com/drone/drone","owner": "drone","name": "drone", | ||
| "full_name": "drone/drone" | ||
| }, | ||
| "system": { | ||
| "link_url": "https://beta.drone.io" | ||
| }, | ||
| "build": { | ||
| "number": 22, | ||
| "status": "success", | ||
| "started_at": 1421029603, | ||
| "finished_at": 1421029813, | ||
| "message": "Update the Readme", | ||
| "author": "johnsmith", | ||
| "author_email": "[email protected]" | ||
| "event": "push", | ||
| "branch": "master", | ||
| "commit": "436b7a6e2abaddfd35740527353e78a227ddcb2c", | ||
| "ref": "refs/heads/master" | ||
| }, | ||
| "workspace": { | ||
| "root": "/drone/src", | ||
| "path": "/drone/src/github.com/drone/drone" | ||
| }, | ||
| "vargs": { | ||
| "build": "", | ||
| "repo": "", | ||
| "access_key": "AHGDGJHhJ8677A2Q", | ||
| "secret_key": "IrlHJHGShjgsgsdgJHGSjhgsdgJJGU8JTRhSb", | ||
| "region": "ap-southeast-2", | ||
| "family": "Development-DashboardTaskDefinition", | ||
| "cluster": "Development-ServicesCluster", | ||
| "service": "Development-DashboardService", | ||
| "constainer_name": "dashboard", | ||
| "allow_multiple_clusters": false, | ||
| "allow_multiple_services": false, | ||
| "image_name": "registry.mydomain.com.au/dashboard-development", | ||
| "image_tag": "1.0.1", | ||
| "log_level": "debug", | ||
| "task_definition": "taskDef.json" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
Oops, something went wrong.