-
Notifications
You must be signed in to change notification settings - Fork 0
sprint 2013 01 03
Sprint 03 Jan 2013
- Previous Sprint: sprint 2012-12-20
- Next Sprint: sprint 2013-01-17
Objective: Establish process for future sprints
Review of Trello https://trello.com/board/crowbar-2-0-current-sprint/50809fb54fa7ad9c1a000f4d
- Accepted Skype -> IRC
- Database Model -> nearly done. FInal review of User
- will complete in next sprint
- Build Changes ->
- will complete in next sprint
Topics
- Rails Engines & APIs & URLs (Jan 9)
- OpenSUSE
FOSDEM event https://fosdem.org/2013/
- Feb 2-3
- possible hack meeting
- SUSE is attending, could send more people
- Dell is not currently planning to send any engineers
Possible 2/4-> week Likely something in March Please put in your timing preferencces
- James Tan (SUSE): I'll be in Asia from 6 Feb to 10 Mar (combination of vacation and working remotely), so earliest I can do is the week of 11 March onwards.
- Adam Spiers (SUSE): I should be able to make most dates work
1 Future 1 Use Case Meeting 1 Use Vagrant to create build VMs & test systems (opportunistic) 1 Review OBS Diagram: https://trello.com/card/rationalise-structure-of-obs-chef-projects/50809fb54fa7ad9c1a000f4d/61 1 Changes to SledgeHammer -> notification & coordination 1 UI Review & Discussion - SUSE has done some analysis on this 1 Dev Tool Review - latest features, pull requests, automated building. 1 Crowbar 2.0 schedule (Early in New Year) 1 How barclamps are versioned 1 Crowbar fault tolerance & recovery
Objectives
- When applying a configuration, need to identify which step failed (e.g. when applying config to individual nodes)
- Avoid search on client-side for improved consistency (see Search section below)
- Chef and Crowbar are fundamentally different. Chef is more eventual consistency driven, but Crowbar is more event driven.
- One of the main goals is to be CMDB (Chef, Puppet) agnostic.
- Create an abstraction layer
- Not gating this release on Puppet support, however. And no plans to support other CMDBs at the moment like cfEngine (at this time) but could be done by the community.
- Ability to share the upstreams
- This requires attribute injection
For the most part, we wrap 1 cookbook into 1 barclamp.
Purpose is to receive any data from external sources which is needed by Crowbar (but not all data - goal is not to replicate data from Chef, ohai etc.) There are potentially multiple external sources, although currently this is limited to Chef / ohai. As part of the inbound path, each barclamp chooses which bits of data from which sources are stored / published into Crowbar as attributes for internal consumption.
When Chef runs, or any other external source is ready to provide data, an event notification triggers the inbound path code.
When a configuration (a.k.a. proposal) is applied, per-node and per-barclamp attributes need to flow back to the configuration management system.
Use Cases for Attribtues on the outbound
- coordinate between roles within a barclamp (intra)
- e.g.: swift - it needs to find disks that are ready for use so that it can build the ring file
- coordinate between barclamps (inter)
- nova needs to find a sql database and build a connection string for it
- the sql information is managed by the SQL barclamp
Attribute driven recipes - provide information to the cookbook rather than search. This allows our cookbooks to be used by anyone as long as they set the attributes for the cookbooks. It also allows us to use community cookbooks as long as we can put the attributes in places where they are looking for it. This is emerging as a best practice on building cookbooks for community use.
Crowbar will has machinery to talk to Chef and inject the role+attributes, database+attribute into the cookbooks where they need to be. CB1.0 has a lot of assumptions about how this is setup, we are trying to make this more generic in CB2.
future topic... we discussed that there is really no difference between the inbound and outbound attributes. It is likely that we'll
Crowbar uses search to get attributes that are on nodes and other barclamps using search. While this is easy, it makes it very hard to maintain and understand.
This is a powerful feature, but we have concerns about search for several reasons
- relies on eventual consistency
- timing issues because information is missing
- relies on people putting things in expected
- may not be able to do reasonable upgrades & migrations
We are planning to allow a gradual replacement of search to attributes. We do NOT want to eliminate the use of search - there are places where it may still be required. "Avoid but don't block"
- Use case list
- what use cases are needed on the first case
- UML class diagram is needed (https://github.com/crowbar/barclamp-crowbar/blob/master/crowbar_framework/db/crowbar_cmdb_classes.dia & .png)
- needs to be updated
- Barclamp subclassing & service object models
- We've had discussions about attributes that can overlay each other. For example "SQL" overlay for "mySQL".
- Reliability & recoverability aspects of CMDB interactions
- how do we many recoving from a hardware failure
- Dealing with data that is in transistion (we have attributes that are being set to a new value but have not changed yet)
- CMDB transactions & orchestration of process management
- If I have to get an action completed, how do I know it was done
- How do I stage changes across multiple steps
Ref: http://guides.rubyonrails.org/engines.html, http://api.rubyonrails.org/classes/Rails/Engine.html
We are converting the barclamp merge into using the Rails Engines model
Challenges are that the controllers and models fold into the framework namespace.
- This creates the potential for collisions.
- CB1 required barclamps to modify the central routes file - this limited the barclamps ability to extend the API and UI
Advantages
- Barclamps will have their own namespaces without collisions and ambiguities
- Routes for Engines can be defined separately and share a mount point (that's in line w/ our API objective)
- Use the same model we use for localization
Engines appears to satisfy the objectives (API paths & independence) that we have and address the collision issues that we've seen.
- The Engine infrastructure is already in place for the application, so we already have the infrastructure needed
Current Status: Wayne is testing the a version of the "engine-ified" barclamps to see how they work.
It should be possible to convert existing barclamps over to the engines in stages (no big bang migration needed).
- The scripting will be for the initial conversion
- Need to have the barclamp install have to be updated to address this change (barclamp install already has the concept of version #s). Perhaps we should the current of barclamps to 1.9 and the engines versions to 2.0. (Rob can make this change during the Doc changes)
Log Barclamp as the first attempt to migrate
- demonstrate extending the UI
- demonstrate extending the API
Questions:
- is there still a global namespace? -> yup, plenty of access.
- how do/can engines communicate with eachother? -> helper methods for paths
- Can we pass messages through a queue between barclamps? Is there something that engines provides to make this easy? -> Nope. There's nothing stopping us from adding things like Resque to have this, if we really need it.
- Need to resolve how CSS/SASS prioritization is handled?
- For localization, can you xref i18n file alias between i18n files -> needs to be resolved
Trello Cards
- Migrate version UI Crowbar version back to 1.9
- Finalize the engine barclamps pattern
- Update the install to accomodate engine barclamp install
- Use the "test barclamp" as the first article
- Migrate the barclamps
ACTION: WE ARE MIGRATING BARCLAMPS TO ENGINES (subject to continuing review and progress)
James has been working on getting Crowbar working on OpenSUSE. This was related to getting the Ruby 1.9 fixes in
openSUSE dev VM: http://susestudio.com/a/n0rKOx/crowbar-dev (work in progress)
Currently, the focus is getting the Crowbar Rails framework working. Then he will start bringing in the Chef components.
Issues
- BDD erlang tests crash (this could be related to the app not starting -> Wayne has a fix he is planning to pull -> Yes the BDD tests crash if the Rails app is not running, but this is different. Will post details in Github issues.
- Packaging Chef11 is a block
- Still working through sledgehammer issues
Wed Jan09 @ 10am CDT. Voice & Screen Share: https://join.me/dellcrowbar
Thursday Jan03 @ 8am CDT. Voice & Screen Share: https://join.me/dellcrowbar
Agenda:
- 25 minutes review
- 10 minutes process check
- 25 minutes planning
This sprint we accomplished the following:
- migration to crowbar github organization
Changes to improve process:
- Please put out more information on the Crowbar mailing list
- IRC adoption is progressing well, but we're not quite there yet
- Use pull requests and high quality commit messages to facilitate design discussions
- People should subscribe to repos for pull request notifications
- Do these meeting times still work?
- Targeting 8am and 10am for Design Reviews
- Need to catch up on backlog of design meetings?
This sprint we are expecting to accomplish
- Blocks
- Deliverables