(CLOUD-295) Add RDS Support #69

petems · 2015-02-06T23:00:12Z

Rebase of #57 with spec tests added 👍

@daveseff want to have a look? 👍

garethr · 2015-02-08T19:59:00Z

lib/puppet/provider/rds_instance/v2.rb

@@ -0,0 +1,102 @@
+require_relative '../../../puppet_x/puppetlabs/aws.rb'
+require "base64"


Not sure this is used in this provider?

garethr · 2015-02-08T20:21:12Z

Thanks @petems @daveseff. I've left lots of comments, mainly small things. We'll need to expand the unit tests and add some acceptance tests to this but it's looking good. I'm happy to help out if I can.

petems · 2015-02-08T21:27:23Z

@garethr Made the changes required based on comments, I can rebase the commits later on 👍

petems · 2015-02-09T14:10:27Z

lib/puppet/type/rds_instance.rb

+  end
+
+  newproperty(:iops) do
+    desc 'The IOPS stype for the DB (minimum 1000)'


From reading the docs, by default this is 0, and that will set your RDS as standard storage rather than the Provisioned IOPS. So I'll probably remove the comment about minimum

daveseff · 2015-02-24T21:37:00Z

Any news on when this will get merged in?

petems · 2015-02-24T21:46:41Z

@daveseff Me and Gareth are going to do some acceptance test pairing soonish, so hopefully should be able to get it tied up soon! 👍

daveseff · 2015-02-24T21:49:39Z

Nice. Can I remove my fork of this then?

petems · 2015-03-09T12:26:49Z

@daveseff Yep, you can probably delete your fork, your commits are safe and sound in this branch! 👍

It can create db's now. (sort of). Added the RDS example in the README I should learn github markup. Now impliments adding rds instance to security group. Can now delete RDS instances. puppet resource rds_instance is now working. Still learning as I go. :) Figured it out. Instances no longer throw errors. puppet resource rds_instance is working fine.

* To get an MVP of RDS working, we also need to create an DB Security Group to access it

danieldreier · 2015-03-26T17:10:02Z

Thanks for your work adding RDS support.

The way I'd like to use the RDS support is to create a multi-AZ MySQL database in the production environment, with backups, maintenance window set, and a parameter group defined, in our VPC.

The only major thing that I noticed about that RDS PR is the inability to manage parameter groups, which are the only mechanism for Amazon gives for configuring MySQL settings. Drupal is so bad about making huge JOINs that tuning mysql really isn't optional - the default settings will have it constantly writing temp tables to disk and performance is significantly impacted.

The most basic use case would be simply associating an RDS instance with a parameter group, though it would really be nice to create parameter groups as well. It would be really ideal if I could copy-paste the settings from the puppetlabs-mysql module into a parameter group. It's just a hash of settings, so it seems like it should be possible.

petems · 2015-03-27T11:04:09Z

@danieldreier Just added Parameter Group type and provider, and added being able to add it to an RDS instance on creation 👍

danieldreier · 2015-03-27T18:09:22Z

awesome! That was quick. Thanks!

choonming · 2015-04-04T02:56:41Z

I was looking into the code and the API documentation on RDS. Noted that you only allow creating DB security groups with some manual work involved as per your documentation in your branch. Is it possible to add a parameter to define the VPC security group associated with the new RDS instance?

petems · 2015-04-08T12:58:58Z

@choonming Right now the AWS API is limited to just creating and deleting DB security groups. You have to do the assignment of EC2 or VPC security groups through the GUI.

petems · 2015-04-08T13:00:02Z

@choonming Actually, I'm not positive on that for VPC groups, the docs page is timing out for me right now, I'll double check that later! 👍

garethr · 2015-04-16T09:36:53Z

examples/postgres-rds-example/README.md

+
+This example creates a security group to allow Postgres RDS instance, then creates an RDS instance with that security group assigned.
+
+  puppet apply rds_security.pp


Needs a few extra leading spaces.

garethr · 2015-04-16T09:47:54Z

@peter This is looking good, apologies for taking so long to get to it. I've left a few more minor comments. On top of those we'll need:

Details in the README for the types and properties. See the examples from the 1.0 README
Unit tests covering the type property inputs (rather than just existence). This probably involves adding basic type checking as a minimum to the properties - again see lots of examples from the existing types.

choonming · 2015-04-16T10:10:26Z

@petems if we look at the RDS API documentation on ModifyDBInstance, we can specify the following:

VpcSecurityGroupIds.member.N
A list of EC2 VPC security groups to authorize on this DB instance. This change is asynchronously applied as soon as possible.
Constraints:
Must be 1 to 255 alphanumeric characters
First character must be a letter
Cannot end with a hyphen or contain two consecutive hyphens
Type: String list
Required: No

petems · 2015-04-16T10:50:12Z

@choonming Ok, I'll see if I can add that to the creation parameters 👍

Iristyle · 2015-04-22T15:39:12Z

PR references CLOUD-295 but commits all reference MODULES-1702

petems · 2015-04-22T16:53:45Z

Yep, need to rebase and rename the commits to the new ticket name.

petems · 2015-05-06T08:47:55Z

Closed in favour of new ticket name branch: #161

garethr reviewed Feb 8, 2015
View reviewed changes

petems force-pushed the MODULES-1702-add_RDS_support branch from 00d65cd to 8184af1 Compare February 8, 2015 21:15

garethr changed the title ~~MODULES-1702 Add RDS Support~~ (MODULES-1702) Add RDS Support Feb 9, 2015

garethr mentioned this pull request Feb 9, 2015

Adding support for RDS instances. #57

Closed

petems reviewed Feb 9, 2015
View reviewed changes

petems force-pushed the MODULES-1702-add_RDS_support branch 2 times, most recently from bae1f42 to d689933 Compare March 4, 2015 16:37

petems force-pushed the MODULES-1702-add_RDS_support branch 2 times, most recently from 15b02c8 to 7b26866 Compare March 9, 2015 00:57

petems and others added 4 commits March 26, 2015 08:44

(MODULES-1702) Adds rds_db_securitygroup resource

b542bee

(MODULES-1702) Add DB Security Group Resource

587c3cf

(MODULES-1702) Working RDS Creation

6122124

* To get an MVP of RDS working, we also need to create an DB Security Group to access it

petems force-pushed the MODULES-1702-add_RDS_support branch from 7b26866 to 6122124 Compare March 26, 2015 08:46

petems added 2 commits March 27, 2015 10:38

(MODULES-1702) Adds DB Parameter group T&P

2bdd273

(MODULES-1702) Adds db_parameter_group_name to RDS

a8ba7ae

garethr reviewed Apr 16, 2015
View reviewed changes

garethr changed the title ~~(MODULES-1702) Add RDS Support~~ (CLOUD-295) Add RDS Support Apr 16, 2015

PR Comment Commit (rebase later)

dcba18c

More specs, rebase this commit later

7fd9b41

petems mentioned this pull request May 6, 2015

(CLOUD-295) - Add RDS support #161

Closed

petems closed this May 6, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(CLOUD-295) Add RDS Support #69

(CLOUD-295) Add RDS Support #69

petems commented Feb 6, 2015

garethr Feb 8, 2015

garethr commented Feb 8, 2015

petems commented Feb 8, 2015

petems Feb 9, 2015

daveseff commented Feb 24, 2015

petems commented Feb 24, 2015

daveseff commented Feb 24, 2015

petems commented Mar 9, 2015

danieldreier commented Mar 26, 2015

petems commented Mar 27, 2015

danieldreier commented Mar 27, 2015

choonming commented Apr 4, 2015

petems commented Apr 8, 2015

petems commented Apr 8, 2015

garethr Apr 16, 2015

petems Apr 16, 2015

garethr commented Apr 16, 2015

choonming commented Apr 16, 2015

petems commented Apr 16, 2015

Iristyle commented Apr 22, 2015

petems commented Apr 22, 2015

petems commented May 6, 2015

		@@ -0,0 +1,102 @@
		require_relative '../../../puppet_x/puppetlabs/aws.rb'
		require "base64"


		This example creates a security group to allow Postgres RDS instance, then creates an RDS instance with that security group assigned.

		puppet apply rds_security.pp

(CLOUD-295) Add RDS Support #69

(CLOUD-295) Add RDS Support #69

Conversation

petems commented Feb 6, 2015

garethr Feb 8, 2015

Choose a reason for hiding this comment

garethr commented Feb 8, 2015

petems commented Feb 8, 2015

petems Feb 9, 2015

Choose a reason for hiding this comment

daveseff commented Feb 24, 2015

petems commented Feb 24, 2015

daveseff commented Feb 24, 2015

petems commented Mar 9, 2015

danieldreier commented Mar 26, 2015

petems commented Mar 27, 2015

danieldreier commented Mar 27, 2015

choonming commented Apr 4, 2015

petems commented Apr 8, 2015

petems commented Apr 8, 2015

garethr Apr 16, 2015

Choose a reason for hiding this comment

petems Apr 16, 2015

Choose a reason for hiding this comment

garethr commented Apr 16, 2015

choonming commented Apr 16, 2015

petems commented Apr 16, 2015

Iristyle commented Apr 22, 2015

petems commented Apr 22, 2015

petems commented May 6, 2015