Learning from the mistakes of others: Incident Response Edition

Join me on a guided tour of the breach of a fictitious company. Through this journey we learn how the attacker performed the breach, what the company could have done to prevent or slow down the intrusion, and what investigators require to respond effectively. Rife with real world examples and humor.

“… learning by the mistakes of others is a far simpler and less expensive process than making them all yourself.” - American Machinist, 1920. Despite being over 100 years old, this quote is still relevant to businesses trying to maintain their security today. So let’s learn from other’s mistakes!

Join me on a journey through the compromise of a fictitious company, from initial access all the way through to mission complete. We’ll take stops along the way to zoom in on how the attacker did what they did, and discuss what the victim could have done to prevent these actions from being successful. We’ll also talk about steps the victim could have taken to make their environment more “investigation ready”, and highlight that because these steps were not taken, the investigation was not conclusive. Being derived from real-world incident response engagements, you’ll literally be learning from the mistakes of others.

None of these recommendations are new or exciting, but it’s my genuine hope that by showcasing them in the context of an active breach, their value will shine, and you’ll take these lessons back to work and implement them tomorrow!

This paper was born out of a frustration that so many security people have never experienced a breach, and therefore can’t sufficiently explain the risk or justification of their recommendations to their employers or customers. Additionally, students fresh out of universities or SOC staff with 1-2 years experience have no exposure to practical breach response, even on a small scale.

I’ve performed incident response in these under prepared businesses, and I’ve been unable to tell the business conclusively what the attacker did, because the evidence doesn’t exist. That’s not even because the attacker deleted it - the most common example is that logging was never turned on in the first place (and you can’t retroactively populate logs).

In giving this talk, I’m hoping that businesses will see the large amounts of value they can get at little to no financial cost, and a relatively small cost of man hours. The recommendations included in this talk aren’t “buy splunk and ingest all of your logging”, because for a lot of businesses that’s unattainable. That doesn’t mean you can’t turn on logging and leave the logs on the endpoints, though. Many organisations have terabytes of space across their desktop fleet sitting unused. Turning on logging per endpoint, and consuming a bit of that space, costs almost nothing. It’s basically set and forget. But it can be invaluable come incident response time. Such examples are common throughout this talk.

That being said, I’ve tried to add value for the more seasoned cyber-pros in the room too. I discuss techniques for communicating technical risk to an un-receptive business, and share techniques I’ve had success with in the past.

The recommendations in this talk come from high-fidelity, reputable sources (for example, the ASD’s essential 8 is referenced), and isn’t based solely on my experience. Many of the recommendations are derived from this SANS gold paper (shameless plug, it was authored by me): https://www.sans.org/white-papers/recommendations-small-medium-sized-businesses-enabling-incident-response/.

Why me? As shown with the gold paper above, I’ve done the research. I was a Mandiant IR consultant for almost three years, where I participated in and led incident response engagements across businesses of all sizes and industries. Before that I’ve helped a company build a SOC to implement follow the sun, and worked for the emergency services (talk about a lack of budget!). I currently work at Salesforce as a cyber security incident manager, leading teams of incident responders through the murky waters of breach response. Surprisingly (or not), a number of the issues highlighted in this paper can be applied to businesses even of Salesforce’s size.