The great stink in software pipelines
It’s the summer of 1858. London. The River Thames is overflowing with the smell of human and industrial waste. The exceptionally hot summer months have exacerbated the problem. But this did not just happen overnight. Failure to upkeep an aging sewer system and a growing population that used it contributed to a powder keg of effluent, bringing about cholera outbreaks and shrouding the city in a smell that would not go away.
To this day, Londoners still speak of the Great Stink. Recurring cholera infections led to the dawn of the field of epidemiology, a subject in which we have all recently become amateur enthusiasts.
Fast forward to 2020 and you’ll see that modern software pipelines face a similar “Great Stink” due, in no small part, to the vast adoption of continuous integration (CI), the practice of merging all developers’ working copies into a shared mainline several times a day, and continuous delivery (CD), the ability to get changes of all types — including new features, configuration changes, bug fixes and experiments — into production, or into the hands of users, safely and quickly in a sustainable way.
While contemporary software failures won’t spread disease or emit the rancid smells of the past, they certainly reek of devastation, rendering billions of dollars lost and millions of developer hours wasted each year.
This kind of waste is antithetical to the intent of CI/CD. Everyone is employing CI/CD to accelerate software delivery; yet the ever-growing backlog of intermittent and sporadic test failures is doing the exact opposite. It’s become a growing sludge that is constantly being fed with failures faster than can be resolved. This backlog must be cleared to get CI/CD pipelines back to their full capabilities.
What value is there in a system that, in an effort to accelerate software delivery, knowingly leaves a backlog of bugs that does the exact opposite? We did not arrive at these practices by accident, and its practitioners are neither lazy nor incompetent so; how did we get here and what can we do to temper modern software development’s Great Stink?
No comments: