Evaluation of the test tools
First of all an appropriate test tool for the execution of UI tests should be found. In this context the requirements are analyzed in the following for the evaluation of the test tools. After having chosen an adequate test tool it should be introduced at SIV.AG. Therefore tests should be created, evaluated and documented.
QF-Test vs. GUIdancer
After the evaluation of the requirement analysis two test tools remain as possible candidates. Based on these requirements a specification and rating form was developed to find out which test tool fits the specific requirements. The two test tool candidates were examined extensively in an isolated test environment for rating the two test tool candidates. The rating form contains different features that contain several requirements. The following features are important for the evaluation:
- Easy usability
- Platform support
- Detection of bugs
- Functionality
- Report capability
- Performance and load tests
- Version control
- Planning and administration
- Pricing
- Qualification of the supplier
- Documentation
With 83% or 912 of 1,095 possible points QF-Test won through against GUIdancer (67% and 737 of 1,095 points). Decisive for the victory of QF-Test was all in all the total package of QF-Test. QF-Test is easy to use, for which the intuitive design is responsible. Also the test scripts can be recorded very easily and fast in opposite to GUIdancer. GUIdancer doesn’t use the capture method: The advantage is that tests can be created during the specification phase, but the GUIdancer’s method is very time consuming. Further crucial was that scripting languages can be used in QF-Test. For simple test cases scripting languages aren’t needed, for very complex test cases scripting languages can be very useful. Finally QF-Test was convincing in functionality, qualification and documentation. Concerning the requirements of qualification, this means maturity and market share of the product or customer assistance, QF-Test was consistently very well classified.
The whole extract from the evaluation report concerning QF-Test can be found here (German PDF only).
Master’s Thesis: Examination of automatable testing scenarios for User Interfaces of business software systems – August 2011, Christian Froh, Hochschule Wismar, Germany.
(Original German texts and citations are translated into English.)