Service registry lookup #1220
Comments
|
From @gabrtv on November 18, 2013 17:54 I've been working on a design for this with the Opscode guys. I hope to post a specification here shortly and get a PR together that implements a manual service registry that we can later make more dynamic (late binding, multi-tenancy, etc.) |
|
From @bacongobbler on December 1, 2013 8:37 @gabrtv, do you still have that spec hiding around somewhere for this issue? :) |
|
From @bacongobbler on December 4, 2013 23:23 Standalone Service RegistryThe service registry is a wrapper to common web services for Technical OverviewThis project will be a generic wrapper for web services, giving administrators Similar to Heroku Add-ons, customers use the service registry to provision an Target MarketThe service registry aims to target the PaaS industry, where new PaaS vendors It also targets the SaaS market, where new providers are looking to get some Administrator's SpecAn administrator of the registry can manage all of the registered gateways. Registering a GatewayAdds the gateway to the registry. The gateway is disabled by default. Unregistering a GatewayRemoves the gateway from the registry. Any addons created with this gateway Disabling a GatewayDisables a gateway, denying any calls to provision a new addon. Enabling a GatewayEnables a Gateway for use. Developer SpecA developer can manage their addons through the command line interface. You can view your current addons that are bound to your application with the Adding an AddonAdds the addon to the application, and forces a new release. Removing an AddonRemoves the addon from the application, and forces a new release. Switching PlansChanges the addon credentials within the application to reflect the upgraded (or downgraded) plan's new endpoint. Forces a new release. Opening an Addon's DashboardOpens a management page on the provider's website for the addon, giving the user administrative access to the addon. Read Documentation on an AddonOpens documentation on how to use the addon with the application. Registry EndpointsArchitecture Diagram |
|
From @gabrtv on December 4, 2013 23:59 @bacongobbler This looks great. Let's flesh out the developer UX next. For example, how does a developer actually attach a database to an existing application using the CLI? With Heroku this looks like |
|
From @bacongobbler on December 6, 2013 7:15 optionally, you should also be able to make this scriptable by specifying the |
|
From @gabrtv on December 10, 2013 0:25 Thanks for this @bacongobbler. After much chin stroking and IRC discussion, here are the next steps before we can start implementation:
Once we agree on moving forward, I suggest we start with a PostgreSQL reference implementation! |
|
From @bacongobbler on December 10, 2013 7:19 Features to discuss before this spec is considered final. These may also be cut from the final spec (as we don't need to concern ourselves with SPOF for now):
|
|
From @bacongobbler on December 15, 2013 0:33 Work in progress at https://github.com/bacongobbler/service-registry |
|
From @bacongobbler on January 7, 2014 10:23 It's very much alpha in it's current state, but I have a working proof of concept where the service registry will create and delete new MySQL databases, as well as return a URI for use. Documentation (and tests!) has still yet to be written, but you can try it out and see what kind of changes you guys would like to see. Instructions on getting started:
$ git clone git://github.com/bacongobbler/service-registry.git -b develop
$ cd service-registry
$ virtualenv venv && source venv/bin/activate
$ pip install -r requirements.txt
$ python manage.py syncdb --noinput
$ python manage.py runserver
After clicking POST to save, the entry should have been created, and the new database should be created in your local MySQL service. To verify, take a look at the $ mysql
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 13
Server version: 5.6.15 Source distribution
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| asdfqwerty |
| mysql |
| performance_schema |
+--------------------+
4 rows in set (0.00 sec)
mysql> select User from mysql.user;
+------------------+
| User |
+------------------+
| root |
| root |
| |
| |
| 8w685g6qkha3spot |
| root |
+------------------+
6 rows in set (0.00 sec)For example, my uri field is now set to $ mysql -u 8w685g6qkha3spot -pqrpsgu8yk64ig97h asdfqwerty
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 28
Server version: 5.6.15 Source distribution
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show tables;
Empty set (0.00 sec) |
|
From @tombh on January 7, 2014 10:48 So exciting! Great work :) The code's on the develop branch right? |
|
From @bacongobbler on January 7, 2014 15:42
Sorry, the instructions imply that it's currently on the develop branch. I'll be more explicit next time. You are correct! :) |
|
From @tombh on January 12, 2014 19:44 Just tried it and it worked first time out of the box ;) What's next then? Is the plan for these endpoints to live on the official Deis API? I see that there'll be new commands for the CLI client. Will the Deis cookbook officially have recipes for certain 'built-in services'? I'd like to help with this if I can. My Danabox project (formerly Crowdbox) is actually in private beta now and I think having database support is going to help encourage people to try it out. |
|
From @bacongobbler on January 12, 2014 22:30
I try to keep a green policy for master :)
The plan is to have this residing on the controller as a standalone web
By all means, please do! |
|
From @tombh on January 13, 2014 11:8 What's the reasoning for having a separate app? Sorry if this has been discussed elsewhere already. As a separate web app is it easy to piggyback off the API's cookie session from the CLI client? Is the idea to have it included in the official cookbook? And will there be recipes for built-in services in the cookbook? The new commands in the CLI client should be pretty straightforward. So should we start a new branch on Deis to pull all these things together? |
|
From @bacongobbler on January 13, 2014 16:52
Good question. I think a little background where I'm coming from is necessary... I come from a cloudfoundry background, working on Stackato. Cloudfoundry's cloud_controller_ng has service discovery built in via a message bus called NATS and by using pub/sub techniques. The service registries (named service brokers in cfv2) are all separate sinatra apps, proxied together using an asynchronous proxy server. There is one service broker for each service (i.e. MongoDB, MySQL, Postgres, etc.), so each web app knows how to handle each service. It's convoluted, but it works. It has a couple advantages this way:
flynn.io is the same, in a sense. It has multiple micro apps all formed together to make one giant PaaS system, which are all discovered using discoverd, which is another service discovery system. It helps keep different offerings available on multiple servers, and in a perfect world, makes each system fully redundant (well, the service discovery is not necessarily redundant, nor is the controller). Cloudfoundry has this issue too, where they are trying to solve the problem of having no single points of failure in the cluster (currently, the message bus and the controller are SPOFs, though there may be other components that are just as likely to be a SPOF, too). Unfortunately, Deis does not have service discovery yet (looking forward to seeing this in a future release 😉), so it'd probably make more sense to integrate this into the controller for now.
The idea is the same with how you can enable or disable formation providers. You should be able to enable or disable certain service providers with a couple changes in
Yes, a new branch would be good to put all this work back into the controller. dibs on |
|
From @tombh on January 14, 2014 13:59 That makes sense about isolating service discovery, you can't make the service-discoverer the same as one of the services that it is trying to discover. So I made @bacongobbler's dibbed 231-service-registry branch with a basic integration of the service registry into the controller. Just wondering if it's the right direction to be heading in? Sorry about the 3 commits. I forgot to remove the '.git' folder when copying in the service-registry repo and git thought it was a submodule ಠ_ಠ Still to do:
|
|
From @bacongobbler on January 14, 2014 17:27
I like your approach, but if we're following Deis' current design pattern, the service code should be integrated back into the
Tack on "add tests" to that list!
Yes. It should be the same access controls. You should only be able to add a service to an application you have permission to create new releases for.
Maybe that can be set up as a |
|
From @tombh on January 14, 2014 22:35 I did actually think complete integration was preferable. But then I wondered if it would be harder to extract out when the time came to make it standalone. Okay, so let's do it that way. So should the models and tasks be appended to Yes, tests of course! I forgot about them. Yes, same access controls. That's pretty straightforward then if it's being integrated straight into the API app. Yes, I think a |
|
From @bacongobbler on January 15, 2014 0:21
Yes.
More like a
All of the node providers/flavors are done in the same way, so yes :)
Installing MySQL can be just as easy as creating a new I think this needs a bigger discussion, as this is a particularly larger subject all on its own. As this was just a proof of concept, all communication was done locally with a base install of MySQL (you can even see the code here). Ideally, It'd be nice to have this all on a separate deis layer, with the operations engineer being able to specify what services he would like on each node (maybe he wants @gabrtv or @mboersma, any input from you guys on this topic? |
|
From @tombh on January 15, 2014 11:55
Ah, got it! So, a service layer, that definitely sounds like the right way to go. But yeah part of a bigger discussion. So there'll be CLI commands like Might I imagine there could be arguments to Definitely +1 for BTW I'm not wedded to implementing all this, I may not be the best person to do it. I'm just happy to do any grunt work. |
|
From @gabrtv on January 15, 2014 19:45 @bacongobbler @tombh When I get some time to breathe I plan to give this a more thorough review including a deep dive into the code. Here is my early feedback:
On a related, service discovery via |
|
From @tombh on January 16, 2014 15:35 @gabrtv Thanks for the feedback :) I reckon I've got a good idea now of how to flesh out a prototype for this. I really want this on Danabox, so I'm going to crack on with it today and hopefully get it deployed there for some live testing soon. BTW, point 4 and containerisation in general is sounding awesome. I don't fully understand how |
|
From @tombh on January 20, 2014 15:35 @stackedsax I was looking through the IRC logs and saw that you were interested in helping with the service registry branch. @bacongobbler's original code is here https://github.com/bacongobbler/service-registry and my WIP attempt to integrate it is here https://github.com/tombh/deis/compare/231-service-registry It's almost there. It works which is the main thing, so you can test it out. Off the top of my head you'll need to EDIT: Put wrong link to my dev branch |
|
From @bacongobbler on January 20, 2014 17:54 @tombh I've been working on some of this as well. I'll see what changes I can merge in from my own branch that I've been secretly hacking away on: https://github.com/bacongobbler/deis/compare/231-service-registry |
|
From @tombh on January 21, 2014 10:38 @bacongobbler Great. Or you could merge mine into yours. Something I've been stuck on, which I really don't know how to answer, is should (and how can) a ServiceProvider be associated with a formation/layer/node? I've also been thinking that the naming convention could be changed to further affirm the distinction from infrastructure providers. I'd suggest renaming Oh yes, I've been keeping track of the things I'm not sure on with a TODO comment that mentions '#231'. |
|
From @bacongobbler on May 5, 2014 18:2 Current status of the service registry (from #486):
|
|
From @recipher on May 6, 2014 9:19 As someone who has built a large Heroku deployment for a client, whilst using a range of 3rd party services (MongoHQ, Redis Cloud, etc), in all cases, we set up the external resource outside Heroku and just connect by manually setting the env variables. The Addons system is convenient but actually makes life more difficult sometimes by provisioning on a separate account, when sometimes you want things consolidated. In Deis' case, I don't know if being able to use an api to setup and connect to a 3rd party is particularly necessary. What is necessary is being able to (simply) provision a resource within your own cluster, and get access to the necessary connection details. It might also be handy to link the docker containers, rather than exposing ports (as an option). If someone could point me in the direction of some instructions about how to do that in the meantime, just using Docker, with, for example, Postgresql or Redis, that'd be great. I'm pretty certain I can work it out well enough (with my, presently, minimal Docker knowledge) but it'd be good to know if there was anything else needed to play nicely with Deis. Thanks. |
|
From @ianblenke on August 26, 2014 16:55 Requiring that plugins have external SaaS dependencies does seem to be the Heroku way. Being that deis is Open Source, would a self-hosting plugin scaffolding for "templated" docker orchestration of such services make sense? Being able to PaaS deploy containers in the Heroku buildpack way is a defining feature of deis at the moment. Being able to PaaS deploy containers from a Dockerfile is a unique feature of deis, but currently lacks a number of abilities that make it attractive to Docker thinking folks. Things like volume mapping and running as privileged are often required by persistence layers and orchestration tools, but do violate the security model a bit in that they can interfere with and impede deis function by knowledge of its implementation. Simple port mapping would be a great boon, both to HTTP and non-HTTP protocols, but would tend to violate security in a multi-tenant hosting platform. As immutable infrastructure for application deployment, deis is great! Adding any form of scalable persistence or CAP theory clustering inside these containers requires careful orchestration. This means directly interfacing with the deis scheduler. Dealing with that today, we use a Docker orchestration harness to independently manage the fleet deployed containers (fig, Orchard, maestro-ng, Centurion, etc). These orchestration tools are currently unaware of the deis deployed containers. There is a deis api to poll for deis changes, but nothing in the way of an event webhook trigger to take action when the deis state changes. The intersection of these two things is where this issue becomes relevant. Having a plugin harness allows for external SaaS offerings. As Heroku has shown, for velocity of implementations and innovation, there is a need for that. Having a scaffolding upon which to build Open Source "self hosting" internal services on deis itself is the golden ring here. From the perspective of orchestrated "linking" of containers, it sure would be nice to be able to a Dockerfile deployed container to dynamically EXPOSE a port on the underlying fleet machine. After that, introduce a DSL (perhaps a Deisfile or Orchestrationfile) linking it to other deployed deis apps so that ports may be exposed as environment variables the same way as a Docker linked container is exposed. As for scheduling coordination of rolls so as to not break persistence replication or CAP theory algorithms, there are many ways to accomplish that. Exposing these rules as part of a DSL (again, like a Deisfile or Orchestrationfile) may be one way. Exposing a scheduler API to allow orchestration tools sitting alongside deis to be more aware may be another. I'm not saying do these things instead of the plugin services defined here. We definitely want to be able to hook out to external SaaS platforms as innovation happens there. What I am suggesting is that we begin to contemplate how we might self-host the implementations of Open Sourced services within the fleet as Open Source self-orchestrating solutions. |
|
From @jankramer on October 30, 2014 12:28 What's the current status and priority of this issue? |
|
From @bacongobbler on October 30, 2014 16:5 @jankramer we just announced v0.15 as our stable release candidate, which means new features will not be merged any time in the immediate future. That being said, I am tentatively assigned on this after our stable release, though we'll have to revisit the spec as it's been several months. Status is still the same as it was with deis/deis#231 (comment). |
|
From @jimmychu0807 on December 11, 2014 4:6 I am also working on this problem and would like to get Deis app be able to create and connect to an in-house backing services (hosted in Deis as well). I am thinking about the request flow. Currently it is using nginx to redirect http requests to various upstream servers. When coming to other services not using http requests (say mysql on tcp requests) how would the Gateway/Router component works on the redirection? I tried nginx tcp proxy, but then realized TCP protocol doesn't support server_name directive in nginx as they have no "Host" field. Another aspect is on managing the additional exposed ports on router for different kind of services. For this, when a new kind of service is added, we will need to rebuild the router container with additional exposed port and resubmit the router unit file to run the container with additional linked ports. Am I on the right track with this thinking, or missing some aspects? Thanks. |
|
From @mboersma on January 2, 2015 15:41 This came up again on IRC, so here is a proxy +1 on this issue; it is very much wanted. |
|
From @ianblenke on March 1, 2015 7:32 This came up again this week with regard to having a plugin to support submodules (being that git archive strips the .git directory) as mentioned in #3157 |
|
From @ianblenke on March 8, 2015 18:21 Flynn has a service control plane of sorts: https://flynn.io/docs/postgres At the moment, it seems to be a certain amount of manual json to add your own persistence service layers based on docker containers: https://gist.github.com/lmars/fca0f18b4ecd507fb39a Similarly, tsuru has a services layer: http://docs.tsuru.io/en/stable/services/index.html There are some great ideas and parallel development going on in this space. Might not be a bad idea to take a quick look and understand the directions they are headed. |
|
From @bacongobbler on March 8, 2015 21:2 We've been keeping our eyes on Cloud Foundry's Service Broker as well as the corresponding API as I am intimately familiar with it, but we're open to look at what other PaaSes have done for an initial proposal. |
|
Is there anything I or other community members can do to help with this? This feature will be awesome. Thanks for all the great work! |
|
@zeiv you may want to take a look at https://github.com/kubernetes-incubator/service-catalog and sig-service-catalog if you're interested in this feature! |
|
This issue was moved to teamhephy/controller#37 |
From @mboersma on October 16, 2013 16:9
Come up with simple design for new object models / dynamic config / envvars to be Deis' service registry and attachment workflow.
Copied from original issue: deis/deis#231
The text was updated successfully, but these errors were encountered: