Author Archives: Brendan McCaffrey

Engineering Flow — Part 4

A series exploring how high-performing engineering organizations improve the flow of software delivery through DevOps, Platform Engineering, DORA Metrics, and systems thinking.

When organizations talk about CI/CD, the conversation usually starts with automation. Automated builds, automated testing, deployment pipelines, infrastructure as code, and increasingly sophisticated tooling all become part of the discussion. Eventually, we start measuring CI/CD maturity by how much of the delivery process we’ve automated.

Automation matters, but it isn’t really the point.

The real value of Continuous Integration and Continuous Delivery is reducing the risk associated with changing software. They help us make changes smaller, integrate them earlier, test them consistently, deploy them predictably, and recover quickly when something goes wrong.

When we reduce the risk of each change, teams naturally become more comfortable making changes more frequently. Faster delivery becomes an outcome of a safer delivery system rather than the goal itself.

Big Changes Create Big Risk

Consider a team that releases software once every three months. During that time, hundreds of changes accumulate. Features are added, dependencies change, database schemas evolve, configuration changes, and defects are fixed. Eventually, all of those changes have to move to production together.

Because the release carries so much change, we naturally build more controls around it. Teams coordinate schedules, testing cycles expand, change tickets are created, approvals multiply, engineers prepare rollback plans, and everyone watches production closely after the release.

None of those controls are necessarily unreasonable. They are responses to the amount of risk concentrated into a single deployment.

But organizations often try to solve that risk by adding even more controls: more testing, more approvals, and more coordination.

Continuous Delivery approaches the problem differently. Instead of asking how we can put more controls around a large change, it asks how we can make the change smaller.

Small Batches Change the Equation

Small batches are one of the most important ideas behind Continuous Delivery.

Instead of allowing weeks or months of changes to accumulate, teams integrate and deliver changes continuously. A deployment might contain a single feature, a small bug fix, or a handful of related changes rather than hundreds of unrelated modifications.

Smaller changes are easier to understand, test, and troubleshoot. If a deployment causes a problem, there are fewer potential causes to investigate. Engineers don’t have to search through months of accumulated changes to figure out what went wrong. They can start with the last small change that was deployed.

This is also why deployment frequency and production stability aren’t necessarily in conflict. Deploying more frequently can actually reduce risk when increased frequency means smaller batches.

The dangerous part isn’t necessarily how often we change production. It’s how much uncertainty we introduce each time we do.

Continuous Integration Shortens the Feedback Loop

Continuous Integration is sometimes reduced to having a build pipeline that runs when code is committed. But the pipeline itself isn’t the important part. The value comes from shortening the time between making a change and learning whether it works with everything around it.

When developers integrate frequently, automated builds and tests can identify problems while changes are still fresh. Integration conflicts, failing tests, security issues, and unexpected interactions can surface within minutes rather than days or weeks.

That timing matters because problems become harder and more expensive to resolve as they move through the delivery system. A conflict discovered while a developer is actively working on a change may take minutes to correct. The same problem discovered weeks later during a major release may require multiple teams to determine what changed and how to safely resolve it.

Continuous Integration moves that feedback earlier, when problems are generally smaller and easier to fix.

Continuous Delivery Builds Confidence

Continuous Delivery extends that principle beyond integration. The goal is to keep software in a state where it can be safely released whenever the organization chooses.

That requires more than a deployment script. Teams need confidence that automated tests will catch common failures, environments are consistent, deployments are repeatable, monitoring will identify unexpected behavior, and recovery mechanisms will work when something goes wrong.

This is where automation becomes extremely valuable. Automation doesn’t just eliminate manual work; it reduces variation.

A repeatable deployment process removes much of the uncertainty from releasing software. When the same process is used successfully over and over, deployments become predictable, teams build confidence, and releases stop feeling like high-risk events.

Repeatability creates confidence, and confidence changes behavior.

Failure Should Be Recoverable

No amount of testing can guarantee that every change will behave perfectly in production. Something will eventually fail. Trying to prevent every possible failure before deployment often creates enormous amounts of process without actually eliminating the risk.

A better delivery system assumes failures will occasionally happen and makes them easier to recover from.

That may mean automated rollback, roll-forward strategies, feature flags, or progressive delivery. Strong observability is equally important because recovery only works if teams can quickly recognize that something has gone wrong.

The objective isn’t a world where deployments never fail. It’s a world where a failed deployment doesn’t automatically become a prolonged production incident.

If a small change can be deployed, observed, and quickly reversed when necessary, the organization no longer needs to treat every deployment as an irreversible event.

Stability and Speed Aren’t Opposites

Many organizations still approach software delivery as if speed and stability exist on opposite ends of a spectrum. If we want to move faster, we have to accept more risk. If we want production to remain stable, we have to slow down.

That makes sense if moving faster means pushing larger changes through the same risky process more quickly. But that isn’t what Continuous Delivery is trying to accomplish.

Small batches reduce the impact of individual changes. Continuous Integration identifies problems earlier. Automated testing provides faster and more consistent feedback. Repeatable pipelines reduce human variation. Observability identifies problems quickly, while rollback and recovery mechanisms limit their impact.

Together, these practices allow teams to increase delivery frequency because each individual change becomes less risky.

The speed comes from confidence.

Don’t Optimize for Deployment Speed

It’s entirely possible to automate a bad delivery process.

We can build sophisticated pipelines around large batches, complicated dependencies, long approval chains, and risky releases. The process may technically be automated, but we haven’t necessarily improved the flow of software through the organization.

The better question isn’t, “How can we deploy faster?”

It’s, “How can we make each change safer?”

Reduce the size of the change. Shorten the feedback loop. Automate the tests that provide meaningful confidence. Make deployments repeatable. Detect problems quickly. Make recovery routine.

When organizations consistently reduce the risk associated with change, teams stop being afraid to deploy. Deployments become routine rather than major events. Changes become smaller, feedback becomes faster, and production becomes more stable.

Delivery speeds up, but not because we asked everyone to move faster. It speeds up because we removed much of the risk that was forcing them to move slowly in the first place.

That’s the real value of CI/CD. Faster delivery comes from reducing deployment risk, not increasing deployment speed.

Engineering Flow Series

1. Your Team Isn’t Slow. Your Delivery System Is.
   The foundation of Engineering Flow: why software delivery is constrained more by systems than by individual effort.
2. Every Dependency Is a Tax on Delivery
   why team dependencies reduce autonomy and slow delivery
3. Why Platform Engineering Exists
   How platforms enable autonomy without sacrificing governance.
4. CI/CD Isn’t About Automation. It’s About Reducing Risk. (Current Article)
   Why smaller deployments are actually safer deployments.
5. Every Handoff Is a Queue
   The hidden cost of waiting, approvals, and organizational bottlenecks.
6. Stop Measuring Output. Start Measuring Flow.
   Why engineering effectiveness starts with measuring systems, not activity.
7. DORA Metrics Aren’t KPIs
   How to use DORA Metrics to improve systems, not evaluate people.
8. CI/CD Is Easy. Culture Is Hard.
   Why trust, ownership, and feedback matter more than tooling.
9. Why Engineering Leaders Should Care About DORA Metrics
   Using engineering metrics to improve organizational performance.
10. Why Most Data Engineering Teams Aren’t Practicing DevOps
    Applying modern software delivery principles to data platforms.

If you enjoyed this article, consider subscribing for future posts in the Engineering Flow series.