Unsafe at any clock-speed. Why we should be thinking about software liability.

In 1965 Ralph Nader published the landmark book, “Unsafe At Any Speed.” It set off not only safer cars, but also the modern product liability movement. Software has eaten the world over the same 50+ years, with very few consumer protections, yet. But there should be and that’s a good thing.

Ralph Nader’s “Unsafe At Any Speed” kicked off the modern product liability movement. No longer were consumers expected to “caveat emptor” (buyer beware), instead, manufactured goods, which are much different than a horse, a bushel of corn, or a hand woven blanket, now have an implied warranty of usability. You can’t sell irons that burn down houses when they tip over, or cars that kill their passengers at the slightest delta-v. As engineers, we’re expected to build protections into the things that we design and build so that they fail gracefully, even under extreme circumstances. As software engineers… well… not yet anyway.

I’m starting to see bits and pieces of the growing risks in the software industry: Nowadays, software can injure or kill, but according to some self-driving car manufacturers, the driver is still responsible because they are “in control” of the vehicle. The European Union’s expectations of privacy (GDPR) and the right to be forgotten are annoying us with cookie-popups, and yet, we have massive, mind bogglingly large breaches, and those responsible, after settling out of court, get to sell you a discounted subscription to some of the very same entities that are responsible for breaches in the first place.

What’s it going to take for us to change how we build and deliver “stuff.” Software has lived mostly protected by copyright law and software licenses that no one reads. Who’s responsible, especially in a multiparty, open source, as-is, integrated software environment? What happens when software becomes inseparable from product. The answer is probably the same way that the auto industry changed, alas, by losing large class action lawsuits. So far, losses have been hard to quantify (how much does your breached password cost you?), but, that will change. Also, our current common law has not adjusted well to the advances of technology, but today’s generation of judges are much more familiar with modern information tech than the retiring class that were appointed before the 1990s (30 years ago).

Your managers and owners don’t want to hear this, but maybe we should start thinking about this internally, so we don’t end up hurting people. I believe that we can make small changes today that will move us in the right direction without impacting the bottom line. Much of the engineering techniques are well known in other disciplines, maybe we can start applying them. It is certainly much easier to test-crash software!

There are no easy answers in this talk. It’s the start of a conversation with peers, consumer advocates, lawyers and our government representatives.

I’m not a lawyer, although I do have many years experience in the legal tech industry. This is a mix of “forensic engineering” that is, engineering with regard to the law, and advocacy for best practices. There will be some technical content, but almost all of it will be platform and technology independent. One of the other topics that I touch on is “Ethics in Software”, but mostly as that won’t change anything unless there’re consequences and enforcement. My talk will try to convince listeners that not only do we have a moral obligation to produce safe software, but the dangers of not doing so might cost a corporation billions. When that much money is involved…