I was having a drink with a friend. He took a sip, slowly placed his half full glass on the table and said, “I finally realize what I hate about DevOps. It’s just another excuse for developers to have root access in production.”
Why did he say this? Why did my friend have such a negative perception about a movement and a way of working that is transforming development and information technology?
To answer these questions and build modern security programs CISOs and security leaders must understand, support, and adopt DevOps practices.
At the heart of DevOps is the automated CI/CD pipeline.
CI stands for Continuous Integration. It’s the phase where the application is automatically built and packaged.
CD stands for either Continuous Deployment or Continuous Delivery.
In Continuous Deployment changes are automatically pushed to production. This is what my friend had a negative reaction to. In this scenario developers can check in code and push changes all the way to production, seemingly giving them root or administrative access in production (depending on your point of view).
Continuous Delivery, on the other hand, requires the operations team to pull changes and deploy them to production. The application is always ready to be deployed but a manual step is required to release changes.
The key point, because the application is ready to be deployed at any time, is that technology is no longer a limiting factor in how fast an organization can move. Choosing one approach over the other is based on compliance, regulatory, and other business drivers.
I have heard respected security leaders state that developers should not push code to production. Period. I don’t agree.
The decision of when code gets pushed to production, how frequently updates are made, and who performs this work is a business decision. Period.
Organizations want to get products and features to market faster. As a result, the DevOps CI/CD pipeline must run quickly and efficiently. This means that we can’t run security tests and scans that take weeks, days, or hours. Sometimes a scan that takes even just a few minutes at the wrong point in the pipeline can be a bottleneck.
Similarly, technology should no longer be a limiting factor on how fast we can patch critical security vulnerabilities. You’ve probably seen it before. A simple SQL Injection vulnerability is found on your web site. It takes the developer less than an hour to fix it. But, before it goes into production it has to go through QA testing, acceptance review, management review, business review, CAB approval, and a host of other process items that, depending on the organization, can take hours, days, or even weeks.
With an automated DevOps pipeline, high-performing DevOps teams have been shown to have 440x faster lead times than their lower-performing peers while also spending 50% less time remediating security issues1. Reducing the window of exposure is a huge step in mitigating business risk from security issues.
Security teams must understand the different phases of the DevOps pipeline and the appropriate processes and tools that can be injected into the Pre-Commit, Commit, Acceptance, and Production phases.
For reference, we put together a poster that lists a number of free tools that can be used in your DevOps pipeline. Click on the image to download a high resolution version.
For years the security industry has struggled to mature foundational security capabilities.
When I ask my students and clients if they have a mature and comprehensive asset inventory less than 5% answer affirmatively. Tracking via spreadsheets and de-conflicting multiple sources of data remain big challenges. With modern practices like Infrastructure as Code we have the potential to know about every system that is deployed via the DevOps pipeline.
Implementing secure configurations has also been elusive. Using configuration management tools we can now ensure that systems are deployed from the beginning with standards settings.
There is a lot of great work happening in the security orchestration and automation space. It’s exactly this type of approach that is the foundation of DevOps. But, it requires security teams to embrace modern DevOps practices such as Continuous Delivery, Infrastructure as Code, and functionality exposed via APIs to automate tedious and error prone work.
Understanding, supporting, and adopting DevOps practices helps us take steps in the direction of continuous security. Embracing DevOps ensures that you won’t miss the boat of a movement that is going to happen with or without you. And you just might see that the glass is actually half full, not half empty.
Thanks to Jaynie Bunnell for reading drafts and providing feedback on this article.
1 Per the Puppet State of DevOps Report