Vermont Information Processing transform their QA process

With the help of Original Software solutions and a much more robust approach to software quality, VIP has completely turned around its QA process and overcome its customers’ cautious attitudes. The company has now transformed into a software testing center of excellence with happy customers keen to upgrade to new releases.

Keeping the drinks flowing

Vermont Information Processing (VIP) has been providing business solutions to beverage wholesalers since 1972. The company develops software solutions that handle every aspect of the beverage distributorship, streamlining warehouse efficiency, cutting costs, managing inventories, and forecasting sales. The solution covers everything from purchase, picking, packing, barcode tracking, profit, cost, returns, and inventory – everything a modern, efficient warehouse needs to manage the flow of stock and predict demand, coping with spikes and busy periods such as Memorial, Labor Day, and the 4th of July.

The company is based in Colchester, Vermont, and employs over 100 people. It has a proven track record of helping its customers become more efficient, proactive, and profitable. These customers rely on their systems for efficiency. VIP is well aware that any bugs or errors in the system can be a major inconvenience to the tightly run schedules its customers have perfected.

Increased complexity & faster development

A few years ago, VIP and its customers were going through a period where they experienced a lot of growing pains. The trend in the beverage industry was towards consolidation with a lot of mergers and acquisitions, resulting in fewer and larger wholesalers. These new supercompanies now found themselves with more warehouses across multiple states, more employees, distributed operations, and a lot of increased complexity. The software that they relied on to manage their business now needed to handle much more complex processes.

For example, VIP’s software held a settings file, which would have the system settings for the whole company. Suddenly, this was not sufficient, and the same settings wouldn’t work across the entire company. Tax laws would be different across different states, and each would require a different setting. It was suddenly as if they had nine separate installations, but they needed to see them all together on the same general ledger. This increased complexity meant that VIP’s customers needed new features and functionality in the software, but they also needed a much faster turnaround. Thus began a period of accelerated development for the company. They moved from having one major update release every three years to producing significant releases once a year, with numerous maintenance builds in between.

Beforehand, the testing process was essentially a beta testing period in a custom install base of around 20 customers. It would take about eight months from start to finish. It was accepted that the testing process now needed to be taken in-house. The company’s old development cycle just didn’t fit its new development schedule. Putting out new releases had become painful, not just for customers and the development team, but the support team was beginning to feel the repercussions as well.

Putting quality at the core of the business

“We went through a spell when our releases were painful to the customers, and they just didn’t want to update,” explained Bobby Erwin, Quality Control Analyst at VIP, who takes up the story. The company was now releasing new maintenance builds every two months and had a significant system release about once a year on average. These updates aim to give their customers access to the latest technology to give them a competitive advantage and improve operating efficiency.

However, customers were reluctant to go through the pain of upgrading and would go, on average, around three years before implementing an update. VIP’s programmers were performing unit tests, and the customer support team was conducting some user testing before release, but there was no formal QA department in place.

“A need for more thorough testing”

With the crunching development cycle, quality was becoming an issue for VIP. “Our customers were frustrated with us,” Erwin said. “To get a new feature, they’d also get all these other problems to boot. This would become a strain on the support team too, as they’d spend hours on the phone going through problems.” These issues made it apparent that there was a need to conduct more thorough testing of the systems and put a formal process in place that put quality right at the core of its business.

The solution

Our main business drivers were to reduce the level of risk by increasing the scope of test coverage. We needed a solution that would run on the IBM i, to test screen, database, and spool files.

Early results

VIP purchased the solutions from Original Software in May 2007, and once they’d had the initial training, it took two or three months to develop their first regression capabilities. They’ve been developing and building on these tests ever since.“Prior to this, we conducted no regression testing at all – just a month or so of acceptance testing by customer service and then the beta tests. Now we have a battery of tests we perform every time we release. It takes just 60-80 hours to run the full suite,” imparted Erwin.

“After our first training session we were benefiting from the environment protection capabilities of TestBench straight away and recording unit tests with TestDrive,” Quality Control Analyst Cheryl Arpey said. She continued: “Test environments were very hotchpotch before we had TestBench. Developers had their own data squirreled away, which inevitably became corrupted by tests and would get worse and worse until they were unusable. The developers would eventually have to go and get new databases which could be a week-long process at least. With TestBench’s environment protection functionality, we can just roll back and reset the data instantly”.

…developers would eventually have to go and get new databases, which could be a week-long process at least. With TestBench’s environment protection functionality, we can just roll back and reset the data instantly.

Reset and re-run tests in 5 minutes.

“Before TestBench, if you wanted to do the same test twice, it was manual and meant re-copying the libraries and took at least a day. It just wasn’t feasible – we wouldn’t do it. Nobody was going to run a set of invoices, reset the data and rerun it. It just wasn’t cost-effective before, but now we can do it in five minutes. That’s huge and so important to us.”

Great feedback

“TestBench allows repeatability and also comparability of tests. Beforehand you could only really test what you were looking at, but now with the database effects, you can tell exactly what happened, at what time in the test. It’s given us many more complete tests. You get great feedback in terms of results – database effects and spool files. You can see all this, and on top of that, you can compare it to how it was before – it all works together. To be able to give the programmers this level of detail is invaluable. The more information you can give them, the quicker the resolution of the issue. To be able to reproduce the problem and show them is so helpful for us,” Cheryl expanded.

After our first training session, we benefited from the environment protection capabilities of TestBench straight away and recorded unit tests with TestDrive.

This has had an incredible impact on our business.

“This has had an incredible impact on our business,” confirmed Erwin. “For example, there is a price list to verify pricing in our system, and we had a release two or three years ago where it didn’t work at all, and we didn’t know. 20 or 30 customers had the system installed. It impacted them quite significantly – not only did they have to spend much time on the phone to support, but it made operating their business very difficult. Most states have stringent restrictions on pricing laws. These businesses had regulations and time frames to meet, and we put them behind the eight ball. They weren’t able to verify their pricing was correct – a workaround would have been very time-consuming. It was a mess for the development team as they then had to identify the problem, stop everything they were doing and work on a fix. Support then had to distribute and install the fix at each customer site.”

“Fixing one problem saves 80 hours.”

With TestBench, we would have seen this easily, and it would have been far less painful to fix in-house before we released it. Fixing just one problem saves our support staff around 80 hours. In the 60-80 hours it now takes us to run the full regression suite, we would have picked up much more than just this one bug – in fact, on each run, our regression suite has turned up, on average, 10-12 similar bugs. You can easily see the return on investment in terms of time savings for all concerned.”

We are providing a much more robust and better product to the end-users

Conclusion

VIP has now completely turned around its quality control as well as its customer’s wary attitudes. The company has transformed into a software testing center of excellence with happy customers who are updating to new releases far more often than before. One indicator of this is that 66% of their customers are currently running the latest release, with 31% on the previous version. Almost nobody is on an older release.
“We are providing a much more robust and better product to the end-users,” said Bobby Erwin. “Our support and customer service teams are delighted. Our builds are just so much better now, and we can also tell them exactly how risky a release is and predict the levels of support that might be needed. Management is leaning on it heavily and is really pleased with it.”

“The regression tests help us manage our exposure: improving experience for the customer, decreasing support needs for a given update. Testing gives us a feel for how stable the build is. It’s a known entity. If the build looks good in testing, we can install it with greater confidence. If we find problems, we get a better idea of the support resources needed.” confirmed Chris Boucher, Support Manager at VIP.