Mistakes we made integrating security scanning into CI/CD

It was 8AM, Slack showed 124 new unread messages and climbing. Our security scanner had broken every build pipeline. Do you want to know why? Are you curious to know the steps we took to bounce back? Do you want to learn from our mistakes?

Continuously shipping secure code is no joke. Scanning code for known vulnerabilities within CI pipeline may sound like an easy win. However, the realities of managing multiple security scanners consistently across build pipelines for diverse codebases can be quite a challenge. Maintaining visibility into scanner performance and failures is a hard requirement, as well as making scan results visible and actionable. In this talk, we will describe a series of internal events that led us to the creation of a new security framework that addresses these pain points. It is simple, yet powerful design pattern that decouples security tasks within a pipeline from the specifics of the build system. For example, it allows security teams to enable/disable scanners individually per service or globally and service owners to override properties for emergency changes without having to rely on each other. It also has hooks into Slack and Splunk for notifications, monitoring/alerting, and metrics. We will discuss what we have learned so far, and future plans.