The Product Managers’ Guide to Continuous Delivery and DevOps

by Suzie Prince on February 11, 2016

This guide is for you if ….
You’re in tech, you’re a product manager or an MBA. Your team A/B tests, feature toggles, and you have a dog in the office! Of course you understand what feature branches are, what CD is and what a DevOps culture looks like. Right? Uh … sure.

You’ve gone Agile. Engineering teams now meet with your product people every week to discuss stories and iteration2s. They collaborate well and what they’re building feels better than ever. But your customers still don’t get those features any faster. You still have to wait for the release train to leave the station. You’ve heard about companies like Etsy, Flickr and Google who deliver a 100 times a day. How do they do it?

Your development team wants to “do CD”. You’ve heard some good things but you’re also concerned about changes going to production without them being properly tested or not being able to market the changes properly. What is this CD thing?

I am here to demystify these practices, tell you just how important they are to you “on the business side” and help you get involved. It’s not that complicated, we have pictures and everything.
Let’s start with some definitions and examples
Continuous Integration (CI)
In traditional software development1 the process of integration generally took place at the end of a project after each person had completed their work. Integration generally took weeks or months and could be very painful. Continuous integration is a practice that puts the integration phase earlier in the development cycle so that building, testing and integrating code happens on a more regular basis.
CI means that one developer (hi Steve!) who writes code on his laptop at home, and another dev (hey Annie!) who codes on her desktop in the office can both write software for the same product separately, integrate their changes together in a place called the source repository. They can then build the combined software from the bits they each wrote and test that it works the way they expect.
Developers generally use a tool called the *CI Server *to do the building and the integration for them. CI requires that Steve and Annie have self-testing code. This is code that tests itself to ensure that it is working as expected, and these tests are often called unit tests. When all the unit tests pass after the code is integrated Steve and Annie will get a green build. This indicates that they have verified that their changes are successfully integrated together, and the code is working as expected by the tests. However, while the integrated code is successfully working together, it not yet ready for production because it has not been tested and verified as working in production-like environments. You can read more about what happens after CI in the Continuous Delivery section below.

To be considered to be practicing CI, Steve and Annie must check-in to the main source repository, integrate and test their code frequently and often. Usually many times an hour, but at a minimum1 once a day.
The benefit of CI is that integration becomes a non-event. Software is written and integrated all the time. Before CI, integration happened at the end of the creation process, all at once, and took an unknown amount of time; now with CI, it happens everyday, takes minutes and is just “the way we work”.
It’s most likely the your team is doing CI (or at least they believe they are). You can confirm by asking them whether they integrated code once a day 🙂 CI is the first practice that is required to do Continuous Delivery. In fact, if you’ve ever checked in help text, documentation or images, then you may have been continuously integrating!
Continuous Delivery (CD)
Let’s go back to our two developers, Steve and Annie. Continuous Delivery means that each time Steve or Annie makes changes to the code, integrates and builds the code, that they also automatically test this code on environments that are very similar to production. We call this progression of deploying to – and testing on – different environments a deployment pipeline. Often the deployment pipeline has a development environment, a test environment, and a staging environment, but these stages vary by the team, product and organisation. For example, our Mingle team has a stage called “Cupcake” which is a staging environment, and Etsy’s staging environment is called “Princess”.

In each different environment, the code that Annie or Steve wrote is tested differently. This gives more and more confidence to them, and to you, that the code will work on the production environment when the code is deployed there. Crucially, the code is only promoted to (tested on) the next environment in the deployment pipeline if

Dev and Ops before DevOPs

Devops is the new way

Useful Terminology
Checking in
The process of pushing local development code changes to the common source repository.
CI server
A tool used to build and test source code. The CI server will tell a developer if their latest code builds were successful and if they continue to pass tests.
Development environment
Where developers create, integrate, build and test code.
Deployment pipeline / pipeline
This is a set of stages that Steve and Annie’s code changes go through before they are done and ready to be delivered to production. Commonly these will be “build”, “unit test”, “functional tests”, “performance test” and “deploy”. Different automated tests will be run at different stages. Only once the code goes through the entire deployment pipeline can the software be delivered to production.
Green build
Green is an indication of success. A green version or build is one where it has passed the tests for that particular stage of the development and delivery process. Generally a build or version of the software will not be promoted to the next stage of the deployment pipeline unless it is “green”. The opposite of a green build is a red build (see below).
Incremental development
Not to be confused with iterative development (see below). Incremental development is where a little bit of the product gets built at a time until it is all complete. Pieces are added on in each increment, and those increments may be small or large. You can use CI with incremental development but it can be harder to achieve Continuous Delivery or Continuous Deployment with incremental development, as you must wait until all increments are completed to deliver value. A great illustration of the difference between Incremental and Iterative development is Jeff Paton’s Mona Lisa.
Integration
All code that is written by individuals or teams needs to be combined. We call this integration. In continuous integration, we generally mean software from individuals needs to be consolidated on a regular basis. In continuous delivery, we often mean software from different teams is integrated together to create the whole product.
Iterative development
Not to be confused with incremental development (see above). Iterative development is where a little bit of the product gets built at a time and is refined until it is complete. The product is built iteratively where the same pieces are reworked each iteration. Change is expected and planned between features in different iterations. You can use CI, Continuous Delivery or Continuous Deployment with iterative development. A great illustration of the difference between incremental and iterative development is Jeff Paton’s Mona Lisa.
Master/trunk/mainline
The “master”, “trunk” or “mainline” branch is the main branch of the source repository. Most people do trunk-based development,* *meaning that they will always integrate their changes to main line. Others do *branch-based *development when individually developers will have their own branches, or teams will have branches for different features.
Production environment
This is the place where software gets deployed or released. Customers using your product or website are most likely using this environment. Also referred to as “in production”, “in prod” or “live”.
Red build
Red is an indication of failure. A red version or build is one where it has not passed the tests for that particular stage of the development and delivery process. Generally a build or version of the software will not be promoted to the next stage of the deployment pipeline if it is “red”. The opposite of a red build is a green build (see above).
Source repository
This is where the source code lives. Steve and Annie have their own local version of the code that they are working on (meaning on their own machines), but the source repository will contain all the code after developers check in their changes to it.
Test automation
High quality test automation is needed to do continuous integration and continuous delivery. Tests are ways to check that the software will work as expected. Automated tests are tests that are coded and automatically run once code is checked into the common source repositories.
In the CI world, the unit tests are run each time the software gets integrated and built. If the tests don’t pass, that version of your software is determined as “not working”, “red” or “broken”. In some workplaces “red lights” or sad sounds occur when this happens.
In the case of a broken build, Steve or Annie (whoever committed the malfunctioning code) need to “fix it”, “make it green” or “get it working”. They can either do this by making a change to the code to fix it, or removing the prior change that broke it.
Unit tests
Unit tests are automated tests in code that test low-level, single pieces of code to ensure that they are usable and working as expected. Unit tests are considered a prerequisite for practicing CI and CD.
Further reading
Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation –http://martinfowler.com/books/continuousDelivery.html
Martin Fowler’s easy to read guide on CI – http://www.martinfowler.com/articles/continuousIntegration.html
The Phoenix Project: A Novel about IT, DevOps, and helping your business win – http://itrevolution.com/books/phoenix-project-devops-book/