I often hear people talking about testing as validating and verifying requirements. But testing is so much more than that.
When we focus on verifying and validating we will most likely only find and look for those things we seek. It’s embedded in the meaning of the words themselves.
According to the Oxford Dictionaries;
Verifying is to: Make sure or demonstrate that (something) is true, accurate, or justified.
And validating means to: Check or prove the validity or accuracy of.
With this mindset we tend to focus on confirming whether the function or product works as stated. But we forget to observe what else is happening . It’s like reading your horoscope that says you will meet a blond stranger — and suddenly you are only paying attention to blond strangers.
Testing is to learn about the product through exploring and experimentation but when we focus on verification it turns into a demonstration of the product:
“Yes, I can login to the website.” “Yes, I can add some products in my shopping cart.” Does this seem familiar? However the demonstration only shows that the product can work, once, in some kind of controlled circumstance. It’s the process of verifying which leads us to think in this way.
When our brain is targeted towards verification we tend to forget to ask questions like:
What if…? What else? What happens when…? Who is this for? Why is it designed like this? When can this be used? How can this function be used differently then expected? What value does it provide? To whom?
By challenging the product or the system you are testing you will discover a lot of new things. When moving away from your confirmation bias you will increase the opportunity to find the bugs/defects you don’t want your users to find. In worst case those bugs which will cost your company a lot of money.
But we need to make sure we meet the requirements!
The most common argument I get when advocating for testing over validation and verification is “but we must make sure the requirements are fulfilled”.
Testing requires multiple information sources. The written requirements are only one input to the test analysis to decide what needs to be tested. If we want better products and software we must go beyond checking. Since we can’t test everything we must know what is the most important to test. There are many parameters which affects that decision and this is yet another skill which you need to have when testing. This is where I see many focus on a too narrow approach choosing verification over testing. What you can do instead is to focus on risk. What is the worst thing that can happen? What if we can’t proceed to checkout? Instead of checking a few specific flows — be creative and explore these flows from different perspectives.
We have both explicit and implicit requirements and we can’t write down every aspect of a requirement. Some things must be tried out before you understand it. Some things are considered so obvious that it is just not documented. There is always a possibility for interpretation and ambiguity in requirements whether they are written or spoken.
So why don’t we spend more time making sure the requirements are perfect? Perfect requirements is an illusion. Even if it would be possible to create perfect requirements we would be out of business before we got our product to our customers. The quicker we can challenge the product by interacting with the software or design, the faster our feedback loop becomes.
By challenging the product from different perspectives like different quality characteristics , different users, different context and applying different test techniques the more likely we are to find those critical bugs which in worst case can put us out of business.
So let’s step out of the verification mode and start testing!
If you are to dive into testing beyond the requirements here is a bit of reading which I can recommend to start with:
Rickard Edgren http://www.thetesteye.com/papers/TheLittleBlackBookOnTestDesign.pdf
James Bach
Testing vs Checking http://www.satisfice.com/blog/archives/856
Michael Bolton
http://www.developsense.com/articles/2005-01-TestingWithoutAMap.pdf
Anne-Marie Charrett
https://www.mavericktester.com/blog/2018/3/20/how-to-avoid-being-fooled-in-software-testing
Curiousity killed the cat 
Restricting your testing to the requirements always relies on those requirements being sound. I once spent the best part of a year working on a project only to find that the requirements had been so badly put together (by a team of consultants who, of course, were cut off from the project after they delivered their idea of what the requirements probably were) that when the product was deployed and real-life users started hitting it with real life data and scenarios, we found out just where those requirements fell short. Such as users wanting to use the product in ways no-one had anticipated, or certain data types in the code having no match to the types of data the API was configured to accept. At one stage, we had about £3 million’s worth of business stuck in the workflow because the data could not be handed off to the legacy invoicing app.
I now never assume that the requirements tell me everything about the product use cases.
Thanks for stopping by to read my blog post!
I totally agree, but we have known this for 30 years or more, so it’s frustrating to have to keep explaining it. I lay the blame almost entirely at ISTQB’s door because it’s they who have indoctrinated vast numbers of testers with the naïve concept of verification and validation.
Also, some terrible testing practices have emerged as agile development has become more common. This is not an inevitable consequence of agile development, but it’s mostly because agile teams rarely contain anyone who knows anything about testing (although they think they know everything). It’s no surprise that software quality is now getting worse from year to year.
Sadly, there is very little good quality tester training available, with Bach and Bolton’s RST course being a notable exception. The paid-for AST courses and free BBST material are also very good.
I hear you!
Thanks you for your article!
Thanks for reading it!