# Glossary
Task Tracker = An Agile task tracker with a horizontal board, like Trello or JIRA (nearly all other task/ticker trackers suck)
- To ensure that actions and history on the Task Tracker corresponds to actions and history in git
- To ensure that a deployment to an environment corresponds to an action in the Task Tracker and git
- Done means DONE! http://www.allaboutagile.com/agile-principle-7-done-means-done/
- To ensure rollback is easy
- To ensure hotfixing is easy
- To ensure we always know exactly what code is on production
- To automate as much as possible so that releases are easy, regular and stress free
- To be as simple as possible, to minimize unnecessary steps, branches, columns, etc
The board should have the following columns/sections:
- Backlog / Icebox: A long list of placeholders for work that needs to be done, not necessarily analysed, nor fleshed out
- TODO: A list of tickets that the team has decided need to be done. Hopefully some details should be fleshed out on the ticket sufficient for development to start.
- Doing: In progress tickets, in progress includes development, testing and code review.
- Ready to release: A ticket that has been developed, tested and reviewed and is ready to release (this column is redundant in fully automated CI work flows)
- Done: Released tickets
How tickets move from the Backlog to TODO is not particularly important from a development point of view, so will not be documented in detail. This could be via Sprint plannings (Scrum), or via a Kanban approach, whatever, such details are largely irrelevant to the core Agile methodology.
The only requirement is that a ticket has been reduced to the smallest self contained deliverable, that is:
-
If the ticket was released it would be callable, i.e. there exists a way the customer/user can interact with the application that would execute the corresponding code. If this is not the case, the code is dead code and should not be in production.
-
It is impossible for the ticket to be broken down into N tickets where each satisfies 1.
(Desirable, but hard) Automatically send a slack message
If you already have a ticket in Doing, but for whatever reason that ticket is pending/blocked, you should first slide it back to TODO and ensure comments on the ticket explain why it's been moved back. Otherwise:
- Create a branch from master that uses the ticket reference number as a prefix to the branch name. In Trello these are just short numbers, in JIRA these are usually a short string that includes a project ID and a ticket number. For example we may have a ticket "5 add back button", then the branch would be 5-add-back-button (observe lower kebab case convention).
- Slide the ticket from TODO to Doing and add a comment with the branch name (desirable). Ensure the ticket is assigned to you.
- Implement the ticket via ATDD, BDD, TDD cycles (TDD may not be possible for languages that don't naturally support unit tests).
- While you work on the ticket you should regularly merge master into your branch, and backup your branch by pushing it. If you don't merge master into your branch regularly then merge conflicts may become unwieldy. When you commit use the ticket reference as a prefix to the commit message.
- When you think you have finished create a pull request (you should configure git to require a certain number of reviewers, two is good to start (to stop people reviewing their own code)). By creating a pull request this should trigger a jenkins job to run a test suite, it should not be possible to merge the branch unless the suite passes. There may be some back and forth to tidy things up between the reviewer and the reviewee.
- If they are happy the code is good, they should say "I'm happy, review passed" or something on the pull request, and they should perform step 1 in the next section
(Desirable, but hard) Automatically send a slack message
(Desirable, but hard) based on the comment, automatically perform the next step:
- The aforementioned reviewer should login to Jenkins and there should be a "merge branch" / "close PR" job or similar, with a parameter that is the branch. The reviewer should run the job. This job should:
- (Desirable, but hard) Check that the user is not the assignee of the ticket
- Pull master, and merge in the branch
- (optional) does a commit with messsage "Closes #." (might need to be jenkins param for now)
- Run the test suite, this may include a suite that spins up temporary staging clusters (this step ensures that the build still passes after merge, hypothetically if the branch was up to date with master then this step would be unnecessary, but getting the PR system to dissallow merges unless the branch is up to date with master is tricky).
- If and only if the test suite passes (and maybe sonarcube), it pushes master.
- (Desirable, but hard) automatically do step 2
- Move the ticket to "Ready to Release"
(Desirable, but hard) Automatically send a slack message
(Desirable, but has caveats) A merge to master should automatically trigger this step, but this does mean a feature branch should only be completed if it is to be deployed.
This may result in the releasing of other features.
- In Jenkins click the "release" button (or "deploy") for the project, now all the following steps ought to be performed by Jenkins automatically:
- checkout/pull master
- build the artefact (which will include running the unit tests). Note for some projects that are script based "building" might just be trivial
- Run the full test suite using that artefact (including extra tests that may be slow but are important before a release)
- If the tests pass tag the commit (by bumping the major version number of the last tag) and push the tag. (Desirable) If they fail, send round an email of shame.
- Put the artefact into the artefact repo (e.g. nexus, s3, etc, something that doesn't allow overwrites)
- Then if necessary (i.e. for permanent clusters) trigger a deploy job that deploys the artefact to the prod environment
(Desirable, but hard) Automatically send a slack message
(Desirable, but hard) Automatically move all the tickets that got deployed to Done.
(Desirable, but hard) Automatically create release notes and commit them to a .md file.
Same as for Features
Same as for Features except:
In 1.: Create a branch from the most recent tag instead of master, this will ensure you are branching from code that is on production.
This differs significantly in that once the review is finished, you do NOT merge into master. The ticket may sit here waiting to be released, but essentially nothing "happens" in this transition other than the reviewer approving it.
This will only release the specific hotfix.
- In Jenkins click the "release hotfix" button (or "deploy hotfix") for the project,
- now all the steps ought to be performed by Jenkins that where done for the "release" job in Feature branches, with the key differences being:
- Jenkins pulls the hotfix branch, rather than master (so it will need to be a param to Jenkins)
- the minor version number ought to be bumped
Now at this point we have released the hotfix, but have not merged into master, so
- Manually merge master into the hotfix branch to resolve any conflicts
- Now run the "merge branch" jenkins job as mentioned in "Doing -> Ready to Release" for Features
- Move the ticket to Done
It's quite common for software developers to split a build out so that it produces multiple artefacts. There is almost always no reason to do this. Below are some of the reasons developers give, but only reason 3. can ever be considered valid.
Using multiple artefacts/builds for delineation is the wrong tool, languages like already have Package/Module Access Modifiers to do this.
Firstly this can be acheived by simply having more than one target path for the artefact. Secondly if you are scarred to deploy all your components regularly, it implies your tests are crap. Write better tests.
If you have a client side library and want to protect the IP of the code on the server side, then it makes sense to separate out client side from server side so that clients cannot get access to the IP. Usually this reason does not apply, as usually applications are not deployed on-site.
The above flow assumes that for the given git repository there exists a single entry point to all tests, and a single entry point that builds a single artefact.
A multirepo, i.e. where each project corresponds to a separate git repo, solves the above assumption somewhat simply. This means each repo has it's own corresponding Jenkins jobs. The down side of multirepos is that releases to one project sometimes break other projects, and it's often the case that a single feature requires changes to many projects. This creates a large development overhead and sometimes a manual deployment processs.
In a monorepo one needs a script that runs all the test suites, and a script that builds all the artefacts then packages them into a single monoartefact.
It may be desirable when releasing to have a means to study which artefacts have actually changed, or even to limit which artefacts are expected to change. For example suppose a release is expected to only change project A and project B but not project C, then when one performs the release they specify that they expect only A and B to change. The build checks the hashes of the artefacts to ensure this. Then the deploy job could lazily only deploy the artefacts that have changed. For temporary clusters this would be implicit since the artefact it needs would be grabbed from s3 prior to the production run.
See https://github.com/samthebest/dump/blob/master/monorepos.md for more information
It's important that when you pair:
- You both have a keyboard plugged in
- You can both easily see the monitor
- You communicate a lot, particularly the driver ought to say what they are doing
Adversarial pairing is defined by the following flow:
- Person A drives and writes a test, person B navigates
- Person B drives and makes that test pass while person A navigates
- Person B continues to drive and writes the next test while person A navigates
- Now person A drives and makes that test pass while B navigates. Go back to 1.
It's often the case tickets fall into a particular theme, or all contribute to delivering some wider goal. Use labels to organize tickets into types, like DevOps, Tech Debt, etc, and use Epics (in Trello this is just another ticket, with perhaps an Epic label) to monitor a large goal.
From experience using JIRA is unwieldy at doing Epics and Labels. Trello is the best. The nicest way to do Epics in Trello is to create a ticket, put it in TODO, create a Checklist, and each member of the checklist is simply a link to another ticket. Trello will automatically convert the hyperlink to a pretty hyperlink. Then when eventually all the small tickets are completed and their member in the checklist is checked, that Epic can be moved to done.
Given that Trello has a nice lists feature, it's easy to label lists according to releases if you so desire. That is rather than calling "Ready to release" "Ready to release".