The Science of Continuous Delivery

Today’s software practitioners have heard the phrases Continuous Delivery, Continuous Integration, and Continuous Deployment routinely and often abbreviated as CI/CD. I will not rehash what these are because reams of text already exist. Someone even bothered to create a website on the subject. I do remember being first exposed to CI/CD concepts while working at HP (yes, I said HP). A coworker introduced me to Cruise Control and Hudson more than a decade ago (at HP of all places). It was awesome. It was awesome because of the amount of consistency (among other intangibles) it provided to a really small development team. Hudson did most of the heavy lifting while Cruise Control connected steps together — proto-pipelines if you will.

I foolishly thought that this experience represented a basic level of normal automation pursued everywhere. Oh boy I was totally wrong about that! I can’t recall the number of customer engagements later where I had to land a temporary snowflake CI solution (Hudson then Jenkins) just to kick-start some automation — a task never encompassed within any statement of work.

I am still aghast into why the adoption rate for CI/CD is slow? I run into people talking about it CI/CD but not actually doing it. I also run into lots of excuses into why organizations will not pursue such initiatives. Well, what does science have to say about CI/CD?

Science? Yes, let use some science here.

Firstly, a research group from Finland provides an interesting accounting through a recently published case study[2]. They examined how a Continuous Delivery solution was used within a software development company with a keen focus on the Continuous Deployment the organization achieved. The publication is full of insights to be explored further. However in their concluding remarks:

The business may adhere to principles of lean startup — building a minimum viable product, continuous deployment, having actionable metrics quickly in place, and making business pivots or failing fast.[2]

This is exactly what businesses want from their software delivery organizations! The ability to respond quickly and deploy anytime. As I mentioned, there are other insights in this one publication. I personally find their notion of “post agile” provocative and intriguing.

Other researchers assume continuous delivery as normal state of affairs. In the journal publication titled Runtime Metric Meets Developer: Building Better Cloud Applications using Feedback[1], the authors propose a new agile development methodology that is only possible through the adoption of Continuous Delivery. In their new agile process called feedback-driven development, the ability to release and get real world feedback is essential to metric generation. These real world metrics provide better data points that are only possible if there is sufficient automation — ergo Continuous Delivery.

Finally, how do developers feel about CI/CD? In Exploring Peopleware in Continuous Delivery [3], the researchers performed semi-structured interviews (familiar with ethnographers) to ascertain what the developer experience was as they experienced a CI/CD enabled endeavor. As one would hope the responses were positive. The researchers state:

The practices of continuous delivery makes a more unified team that is ready to take full responsibility that the system works as expected. People are more motivated and committed when given more responsibility and freedom to use modern technologies and working techniques. Moreover, practices that improve quality and readiness to act on problems in production, also appear to lessen stress. Moreover, the longer the time that has passed since the latest production deployment, the less confident the developers feel to make the next production deployment.[3]

That was a cursory sampling of some recent science available —there is most likely a comprehensive literary review somewhere on the internet. Admittedly, I am a scientific method fan. I like case studies that use ethnographic evaluation (a qualitative method) alongside the hard numbers of quantitative research methods. I Google search and then search the ACM Digital Library. When some architect-in-name-only says “some guy at google/facebook/whomever said so in a blog post”, my eyes roll so hard I see my brain. However, for the herd followers whose decision making process requires a brand name endorsement (AKA hear say) there are more holistic works (here and here) that mention CI/CD concepts in specific contextual settings.

The CD/CI tools of the day support the cloud native micro-services future. There is Spinnaker with lots of open source community involvement. There is Concourse favored by some Pivotal pivots. Even the very popular IaaS AWS has a collection of offerings to enable CD/CI.

Whatever tooling you choose, I hope that including good ole science (in post truth Murica!) in the conversation assists those trying to convince others that Continuous Delivery is just good for business. It must be stated that in the people process technology paradigm, CI/CD is just technology to support a processes to enable people to do their jobs. It is not the end-all-be-all in an organization’s digital transformation journey.

In this article, I opened with giving a personal temporal reference to the lower bound age of CI/CD. I then introduced scholarly research on the subject followed with some tooling references. In a future post, I will address the ever sticky task of cultural change that technologist mention but don’t really deliver for customers and organizations. I have the pleasure of associating with many in the social sciences who professionally engage organizations in the change process. Most (oh hmm…all of them) have PhDs in their fields so expect some more science.

Until next time.

References

1. Jürgen Cito , Philipp Leitner , Harald C. Gall , Aryan Dadashi , Anne Keller , Andreas Roth, Runtime metric meets developer: building better cloud applications using feedback, 2015 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!), October 25–30, 2015, Pittsburgh, PA, USA [doi>10.1145/2814228.2814232]
2. Marko Leppanen, Terhi Kilmo, Tommi Mikkonen. Towards post-agile development practices through productized development infrastructure. RCoSE ’15 Proceedings of the Second International Workshop on Rapid Continuous Software Engineering. Florence, Italy — May 16–24, 2015.
3. Pauli Karpanoja, Antti Virtanen, Timo Lehtonen, Tommi Mikkonen. Exploring Peopleware in Continuous Delivery. XP ’16 Workshops Proceedings of the Scientific Workshop Proceedings of XP2016. Edinburgh, Scotland, UK — May 24–24, 2016. doi>10.1145/2962695.2962708