Kill All Mutants! (Intro to Mutation Testing)

Is all your code meaningful, or is some redundant or unreachable? Are your tests strict, or do they let code get away with undesired behavior? Mutation Testing helps root out these problems. This talk explains what Mutation Testing is, how it works, how to use it, and what to do about the problems.

Would your test suite still pass if the tested code was changed? If so, there may be problems with your code, your tests, or both!

Mutation Testing reveals these cases. This talk will tell you what mutation testing is, how it works, how to use it, and its benefits, drawbacks, inner workings, and history. There will be several examples, and a list of tools for many popular languages.

You will come away equipped with a powerful technique for making sure your tests are strict and your code is meaningful!

I’ve picked Advanced level, but might also be suitable for some intermediates. Juniors might not grok the concepts, or even the underlying stuff about testing in the first place. Mainly, they need to be already comfortable with unit testing. Ideally they should also be familiar with more advanced concepts like an Abstract Syntax Tree. If you mean the level of their knowledge within the topic, then it’s more introductory. The advanced-ness is more about what to do about the problems that mutation testing finds, and that is mainly a matter of experience.

I have given this talk many times before, at both language-agnostic conferences and ones focused on various languages, in several American states and European countries, plus Australia, but not yet in Canada. The best video IMHO is https://www.youtube.com/watch?v=xBZqis7SBnw (from ACCU, the Association of C/C++ Users, Bristol, UK, 2022). It’s also the longest, at about 1:09; if you want something shorter (it goes down to 22 minutes) you can pick from my playlist at https://www.youtube.com/playlist?list=PLMrm16n64Bub8urB-bsyMyHiNPMLG7FAS

Agenda: - What is mutation testing? - Why isn’t test coverage enough? - What are its pros and cons? - How does it work (overview and details)? - Simple example (finding and fixing bad test) - Complex example (finding and fixing bad/missing test) - Complex example (finding and fixing bad/missing test, and redundant code) - FAQs as time allows - Wrapup - Q&A