Our mission in test automation
Creating and delivering an automation strategy requires business knowledge and imagination. Let your automation engineers focus on creating the strategy and designing the tests that will reflect the needs of the business. This will be much more productive than time spent in the plumbing and wiring of another code base. Our mission is to take care of all the plumbing and wiring – removing the manual labor typically required to build and maintain test automation – and convert every automation engineer into a test automation architect.
Automating automation
TestDrive is unique in its approach to test automation, providing a totally code-free approach to the mechanics of automation that allows the testers to concentrate on building tests that will validate the needs of the business. This is how TestDrive differs from QTP, Selenium, Rational, et al:
1: Framework out of the box.
Automation frameworks can be created through code in many code-based automation tools. The level of sophistication can be enhanced over time through ever more powerful programming constructs, although the maintenance of such frameworks can become unwieldy and absorb scarce resources. TestDrive provides a full automation framework out of the box
2: Separation of Navigation & Test Logic
Traditional automation vendors encapsulate both the code to control the navigation through the AUT (application under test) and the logic to test it into a single script, whereas TestDrive separates these two requirements into Play Lists and Scripts. This is more than an academic point – the TestDrive approach is fundamentally more powerful as it can ensure the testing logic survives even if the navigation must be re-implemented in response to significant changes in the AUT.
3. Full Content Analysis
TestDrive always analyses every element on every screen, making a full regression test immediately available. Code-based tools require the automation engineer to identify which properties of elements they wish to access and potentially develop technology-specific code to analyze more complex but common controls such as tables.
We had a team of people with business experience and testing skills, but no programming. We could keep that team of people intact, which was very valuable to us.”
Chris Andrews, Ageas
4. UI Annotation
The readability of test scripts and test results is key to how broad an audience can absorb them. Code-based tools rely on the names assigned to each screen element by the developers – names that are neither predictable nor necessarily accurate. TestDrive employs patented technology to assign a name to each control, table and cell; that is how it would be described by the user. For example, a coded solution might call a ‘Name’ text box as ‘txtName’ or ‘TXT0001’, whereas TestDrive will annotate it as the ‘Name Text Box’.
5. Automatic Checkpoints
Making a set of results easy to understand requires pictures and checkpoints each time the AUT has completed a logical step. This can be accomplished in code-based tools but requires significant coding outside of simple browser navigation. TestDrive automatically creates such checkpoints and captures screen images and a full content analysis across all supported technologies without user interaction.
6. Playback Timings
In simple scenarios such as browser navigations, it is straightforward for code-based tools to know when the AUT is ready for the next action. However, even minor variations, such as JavaScript running when navigation completes or AJAX communications, require significant coding. TestDrive automatically handles all such scenarios – both in the web and across all other supported technologies.
7. Unexpected Dialogs
Code-based automation tools can create routines to deal with unexpected behavior in the AUT. But by its nature, such code becomes extensive and harder to maintain as the number of potential situations increases. They are also limited by the fact that they can only handle expected dialogs rather than unexpected ones. TestDrive provides a rules-based approach that can encompass both the expected and the unexpected.
8. Database Effects
Code-based automation tools can create routines to deal with unexpected behavior in the AUT. But by its nature, code-based solutions were developed for the user interface, meaning verification of the database is normally achieved through SQL statements embedded in the test script. In contrast, TestDrive automatically captures the details of every single database effect and ensures it conforms to the defined business rules.
9. Quality Checks
Code-based tools have the potential to execute non-functional tests or quality checks, but this is only possible if they have first addressed full content analysis and checkpoints. With these problems resolved, it is possible to extend testing to include link verification, spell-checking, performance analysis, hardware impact and more. TestDrive provides all these automatically
10. Result Compares
Regression and other test results may need to be measured against a baseline and execute any verifications defined within the test script. TestDrive provides a full matching capability against any previous result set, with all differences, additions, and lost items automatically highlighted to the user. This is not achievable in a coded solution.
600 automated test scripts created in 9 months” Judy Doust, Marston’s PLC
11. Animation
The ability to replay a test without needing access to the AUT is unique to TestDrive. It allows the nature of any errors to be rapidly assimilated, gives access to test results outside the test team, and creates animated training materials, including screen bubble annotations and a spoken explanation of the application flow.
12. Self-Healing
The speed which test assets can be created in TestDrive means that you can afford to throw them away – in contrast to coded test assets that have taken weeks to create. This is the difference that makes TestDrive so powerful in an agile environment. But TestDrive goes further, offering a ‘Self-Healing’ mode where it will automatically synchronize a test script to the changes in the AUT, even if those changes are significant.
So is TestDrive Code-free?
While the code-free approach extends considerably beyond the above to encompass data-driven tests, basic test of elements, multiple machine execution, dynamically changing navigations, management of test data, email notifications, high-quality test reports and much more, it has to stop somewhere. Typically, that point is where a custom test is needed to reflect the organisation’s business rules, such as validating a credit card number. At this point, TestDrive supports Microsoft PowerShell, allowing the user to access the power of the entire .Net framework.
A Worked Example
The Fourth Coffee Company delivers coffee and fresh baked goods right to your door. They maintain a simple online ordering system to allow you to place an order. Fourth Coffee has engaged ACME Web Sites to develop, support and host their website, but they perform their own user acceptance testing and regression testing. They validate that each product is correctly listed with a price and that each item can be ordered. The site consists of three pages: the home page, an order submission page and a receipt page.
Question One
How long does it take to create an automated smoke test of selecting a single item, completing the order submission page, and receiving a receipt confirmation?
- Selenium: 30 seconds
- TestDrive: 30 seconds
Question 2
How long does it take to create an automated full regression test with the above scenario but check that every single object is the same as the baseline – size, position, content – including listing any new or missing objects?
- Selenium: 18.5 minutes. Comprising 30 seconds to create the script and 15 seconds per object to add verification points (72 objects in total).
- TestDrive: 1 minute. Comprising 30 seconds to create the script and 30 seconds to define a full-content checking rule (first time only)
Question 3
ACME Web Sites have made a couple of changes to the site. First, they have moved to a web application framework that implements dynamically generated HTML IDs and Names. Second, the checkout process has been moved to a new, secure service. However, the presentation of the website has not changed. How long does it take to modify the script to deal with the changes?
- Selenium takes 2 hours to deal with the HTML IDs and names and 5 minutes to update the URLs.
- TestDrive: 0 minutes. The script isn’t broken. 1 minute to re-synchronize the script using Self-Healing.
Total Time Saving = 2 hrs 22 mins 30 secs
