Am I going to smash my phone? - A testers guide to uncovering issues via risk assessment

By Elizabeth Fiennes

Elevator Pitch

In Agile, we think of risk at a low level. Timed right, requirements can be edited quickly and code fixed on the fly.

DevOps comes with new bigger risks especially for those working on large scale releases.

In this talk, I will consider if old risk assessment methods work in this new world

Description

Let’s talk about risk
Risks have a bad reputation. Sometimes justly so. Some risks have grown up and turned into huge issues for companies.
Some risks can be positive (save time, money, effort). These are good and are risks worth taking.
The type of risks testers need to be able to assess are the negative ones with the bad consequences (implosion of product, team, reputation).

So when does a risk move from something that is acceptable to something that must be fixed?
How can we analyse how easily a risk can turn into an issue?
What’s the difference between a mitigation and a workaround?
How can testers take part in the conversations to decide which is the better option?

Consider the following risks. What could be the impact of ignoring them?
Our systems are tested end to end. We look at every possible permutation for every integration.
Some parts of our systems are vendor apps - no need to test them at all. They should “just work”.
Some of our systems are microservices - no need to test them as standalone componants. They are just tiny bits of functionality.
We cannot do any testing on our system, there is not enough time.

Aha but we have a RISK register
Some projects flaunt the use of a risk register as if the act of writing a risk down can magically stop it from developing into a full blown issue. Have a think about it, when was the last time you saw a risk register prevent something becoming an issue? From what I have seen, sometimes the keeping of the risk register becomes about finding someone to “accept the risk” rather than take any preventative actions to stop it from becoming an issue.

So what is the ask here
Under the historical way of working under Waterfall, we were encouraged to write risk based plans and approaches on the basis we could not test everything so we would just look at the risky area.
POSITIONING CLAIM: I AM NOT ADVOCATING A RETURN TO WATERFALL.
I am advocating that testers apply the same shift left thinking that we apply to code and requirements to risks. Let’s get close to our risks, understand them and, where appropriate, advocate for their removal from a place of knowledge and understanding.

So what is a risk?
A risk is something that has the potential to turn into an issue. Testers must be sure to communicate if they see a risk or an issue when reporting to their feature team. There are two key things to understand about a risk -
What is it’s liklihood? (How likely is it to occur?)
What is it’s impact? (If it does occur, what is the potential fallout?)

Let’s do some analysis
Issue: Login does not work if user has extended characters in password and alpha characters case is ignored
So password == PASSWORD == pAsSwOrD == PASSword == passWORD and so forth.
Risk: It would take an average reported 52 seconds to brute force any plain English 8 character password.
Potential Issue: Users will get really peeved off IF their accounts are hacked due to us not implementing more secure password rules. We will get negative publicity and potentially lose customers who are not even hacked
=> If you report your risks properly, you are basically making the case for the business to spend the time and effort to fix the issues you have found on the basis of what may happen if you don’t fix them.

So let’s talk about some risks and their likelihood of becoming issues:
We all have a risk threshold whether we realise it or not. It’s best descripted as the overwhelming feeling you get to shout “no” whilst clenching your bottom cheeks. We all have a risk threshold face and none of them are pretty! Let’s see if we can get you all doing your risk threshold face.

The ‘Shall I drop my phone’ game:
(Actually going to have my phone and a cushion on stage)

I am going to drop my phone (in it’s protective drop-proof case) on a cushion
I am going to drop my phone (in it’s protective drop-proof case) on the floor
I am going to drop my phone (without it’s protective drop-proof case) on a cushion
I am going to drop my phone (without it’s protective drop-proof case) on the floor

How comfortable are you at the thought of me carrying out these actions?
At what point are you feeling the need to say “NOOOOO!” That is it. That is your risk threshold warning you of a potential issue. Learn to listen to that voice. You can exercise it by looking at designs, proposals, code, integrations, environments and your team mates and thinking “What’s the risk here?”

Are risk management models the answer?
These are something to be weary of. Some of them are more invested in producing artefacts than actually understanding risk. Overreliance on these models which are a layer of abstraction outside of the team and the software can be a risk in itself. If you can see that your project is just using them to list out acceptable risk, you may need to go back and look at how you are communicating the risks you see as a tester.

Communicating risk
YOU MUST understand the root of the cause and the very end of the impact in order to be able to make the recommendation to remove or fix an issue. “Feature missing”, “feature inaccessible” or “feature not working as per requirements” is not the end of an impact – it is just the beginning. As with the impact of dropping the phone above, the risk is that my phone breaks but the impact is that I am unable to communicate with my family, I am not able to travel and I cannot see my work emails. This could lead to a loads of sub-issues as of yet unforeseen!

Notes

Besides a screen and a microphone, I have no tech needs to deliver this talk above and beyond those of most speakers.

I am Irish, educated in Cork, living in London. For conferances outside of London, I will require that travel and accomodation are covered. As part of my travel, I will look up and offer to talk at local free events if the timelines work out.

I am 20 years in testing this year and still hands on everyday. I am the designer of the internal training framework for our London office competed in collaboration with the testing team I support in London. I also received a lot of great feedback and input from the Scott Logic grad class of 2018 and 2019. Risk modelling and communications are a huge part of that training.