That’s a big question, isn’t it? But a good one. Whether you’re a long-time testing professional, new to the world of software testing, or a business user looking to learn more about why you’re asked to do the things you do, understanding the process behind managing a software testing operation is handy. Perhaps you’re looking for ways to optimize your own processes. Perhaps you’ve just taken over responsibility for software testing and you’re keen to implement any sort of process. Whatever you’re here for, this blog gives you our approach to managing software testing, based on more than two decades of helping our customers do testing faster and better than they thought possible.
The three test management questions you need to answer
At Original Software we’re big fans of making things simple. So here we go: everything you need to think about when it comes to test management can be boiled down to three questions:
Considering testing from those three angles will help you build a comprehensive plan that covers all the bases. Best of all, it doesn’t matter whether your IT team uses Agile, Waterfall, Hybrid, or any other methodology. Testing happens in all forms of the software development lifecycle – you might just have to run your test management process lots of times on a small scale if you’re using a methodology like Agile, or more infrequently but for a larger volume of testing for something like Waterfall.
What do you need to test?
This is where you build your shopping list of tests that you need to run, to make sure that you deliver on the ultimate objective of any testing program: to give your organization better software while ensuring that the business can still operate.
The ways that you’ll build that list might depend on the source of the testing need you’re dealing with. Generally speaking, you’ll either need to run testing because a piece of software has updated, or because a process has changed. In each instance, you need to make sure that the software enables business users to do their work. Things to consider are:
Which processes need testing?
If an application is being updated, then you need to work out which processes rely on that application and so will be affected by the update. Remember that in today’s IT landscape, where cloud platforms, API-driven integrations and third-party applications are the norm, a change in one application can affect quite a large number of business processes.
Do any other processes need testing?
If a process is being changed, it might have knock-on consequences for other processes (for instance, if an order is now being stored in a system in a different format, can processes that rely on that order data still function?)
How big are the changes?
Testing purists would tell you that you need to test absolutely every change to any piece of software, but the truth is that it’s a judgement call each time. The only way to properly make that call is to read the documentation for every update. You may also be able to lighten the testing burden by automating some tests if the changes are small-scale.
How are you going to test it?
Let’s say you’re an Infor Cloudsuite user, and you’ve got a new update to the order entry functionality. You know that any process where a new order is created (and maybe modified too) will need testing. But you also need absolute confidence that your tests reflect what your business users actually do. Welcome to the world of test cases.
Most organizations build up a library of test cases over time, each testing a different process (or aspect of a process). That library can exist in different states, though. Some organizations have highly detailed documentation. Others have their test cases stored in the head of their testing people (or person). Where are yours?
Answering the ‘how’ question also brings up the topic of test automation. Test automation can be a valuable tool for getting high volumes of testing done. But there are some reasons to stick to manual testing, depending on the update:
- If the update changes the software significantly, it can break your automation. Most automation tools are logic-based and won’t cope if buttons move, form fields change, or the workflow in the software changes.
- If the process is changing – or even if the software just looks a lot different – users may struggle to adapt (never underestimate a user’s ability to find problems!) Manual testing ensures that you have at least some business users who know how to use the updated software that can guide the rest of the team through it and help reduce the disruption the update causes.
In our world, we see a pretty clear split between automated and manual testing, where automated scripts are used for regression testing, and manual testing is used for UAT.
When are you going to test it?
Ideally, your testing process shouldn’t be reactive, but something you can put in the diary ahead of time. If you can get there, then you’ll find that it’s easier to get your business users to commit to the testing time, devs will be ready and waiting for bugs to fix, and everything goes that much more smoothly.
Getting to this proactive planning state works a bit differently depending on the source of the testing need – the business changing processes, or a vendor updating their software.
When it comes to software updates, most of the time major updates are done on a regular schedule. Infor, for instance, does two large updates per year. And then, of course, there’s the famous Patch Tuesday. With these events in the diary, you can at least have some resources on call ahead of time, in case you need to kick off a round of testing.
If you’re testing software to make sure it can support a revised business process, your ability to be proactive will depend on how involved IT is in the decision-making process. Ideally, IT is consulted at the very start of the process and therefore has sight of any testing that’s needed with plenty of time.
Whether you’re testing proactively or reactively, the details of ‘when’ remain largely the same. You need to understand what resource is needed to run the test and fix any bugs, book the time in their diaries. Which brings us nicely on to managing the testing itself.
Doing the testing
Once you’ve understood what you’re testing, how you’re testing, and when you’re testing it, you need to get on and test it! There’s a fairly basic checklist of things to think through here:
- Allocating tests to your business users.
- Tracking their progress and chasing up anything that’s falling behind.
- Triaging feedback, identifying failures, passing those failures to developers, and then re-running tests once bugs are completed.
As with the management of any process, as things move along you need to be ready to handle unexpected setbacks. What will you do if one of your business users is sick, for instance? Can the test wait until they are better, or do you need to reallocate it to someone else to keep things moving?
As you’ll no doubt have realized, the larger-scale your testing is, the more complex this becomes.
After the testing
Once a testing cycle has finished, it’s always worth taking a moment to evaluate what happened and try to improve on it for the next cycle. The more insight you can gain into what happened during the testing process, the easier and more useful this will become.
And that leads us rather nicely on to our final message in this blog: your tools can make or break your test management process.
Choose your tools wisely
One thing that’s been clear throughout this piece is that data and insights are vital to manage the testing process properly:
- You need a comprehensive library of test cases, both manual and automated, to be sure you’ve got enough test coverage.
- You need to know when testing is likely to happen so you can line up resources properly.
- You need to be able to see how testing is going quickly and easily to make decisions when the unexpected happens.
- You need to be able to see what happened at the end of a testing cycle in order to properly optimize the next one.
The usual way organizations manage tests isn’t designed to make the management process easy. Most organizations make do with a combination of testing software, spreadsheets, emails, and screenshots of test results to make everything work. As testing volumes increase, the cracks are starting to show.
A platform for test management and beyond
The best way to solve the problem is to invest in built-for-purpose test management tools. But those tools should be integrated into the tools you actually use to do the testing itself. The Original Software testing platform, for instance, gives you one place to manage and control all your testing, which is tightly integrated with our test automation and manual testing software. As a result, test managers get a much richer level of data and insights when managing testing.
They can see when tests are complete, and drill down to look at individual pieces of feedback if they need to. They can view their manual and automated test cases side by side, and make changes to them in the same place they pass test feedback to developers. With everything in one place, the test management process becomes much easier to run and optimize, ultimately helping you deliver better results for your organization.
