Manual testing is intrinsically flawed, primarily because one test cycle is almost never enough. Manual testing is performed by humans and humans have a finite capacity for repetition. When an application is first delivered, the test team will commence their efforts, but after a while will reach a point in their use cases where they can go no further until the bugs discovered to date have been rectified. The bugs are reported to the internal developers who are the external vendor and then the waiting begins.
After some time has passed an updated version of the application will be delivered. Now, what does the test team do? Do they presume that the section of the application which was tested successfully will still be OK or do they start the entire process again?
On the first re-test, we have found that most test teams are as commendably thorough as they were on the first cycle. But what about the third cycle? The fourth cycle? The twenty-fourth cycle?
At some point, even if repetition stupor hasn’t set in, the testing will focus on ensuring that the newly reported bugs have been fixed. Worryingly this means that the thoroughness of the testing is at its weakest as you approach the implementation date.
So, what can you do?
To get that conversation started it is important that we divide the test burden between the two groups who perform it. First up are the QA teams that form part of the IS function. Sometimes these teams are totally in-house while other companies choose to partially or wholly out-source this activity. Second up are the business users who perform UAT (User Acceptance Testing) and there the challenges multiply.
But let’s focus on the QA teams first.
Imagine that you’re a tester assigned to ensure that the Order Entry function of your ERP application performs in accordance with the requirements. Yes, I know, sometimes the requirements aren’t as complete as you’d like, but let’s imagine that they are. So, what happens next?
On the first screen, you are asked to select a customer that you do, and then proceed to screen to get the order details. However, at this point the application crashes. Congratulations, you have found a bug. So, you need to document and report it.
Back to the first screen and you select the first customer again and try to proceed but you are informed that the customer is currently locked in another order. Now what? You can only hope that other customers will exhibit the same behavior. The gods are smiling on you and the second customer also causes the same bug.
OK, you now know how to reproduce the bug all you have to do is go back, select another customer, take a screen print, attempt to proceed to the order details and take a screen print of the crash dialog. Open a new email, type a description of what you were doing and paste in the two screenprints as evidence.
I realize that this is about as simple a scenario as you can get but even here, and when the gods were kind to you, you still had to execute the test three times. But even in this simple example that is merely the start of the fun and games.
Would you like to make manual testing less manual?
Colin Armitage CEO Original Software
