Comparison of different GUI test tools (Open source and commercial)
Introduction
The goal of this project thesis is the selection and introduction of an UI-test tool at ALEA GmbH for the merchandise management system ALEA Commerce Suite for the automation of functional tests and for the load generation for performance tests. By these tests we can test whether the functionalities are impeccably error-free as well as if the existent client’s hardware is suitable for ALEA Commerce Suite or has to be upgraded. The present project thesis is thematically structured in two parts.
First of all an appropriate tool will be chosen for the execution of User-Interface-Tests (UITests). At first open source-UI-test tools for Java-SWT-Clients (e.g. GUIdancer or Fit) are evaluated preferentially and compared to commercial software like for example Mercury, Hewlett Peckard (HP), Loadrunner or Seque Silkperformer. If no open-source tool will fit the requirements of ALEA GmbH, we will choose an appropriate commercial tool. The following criteria must be fulfilled by the UI-test tool. It must support SWT and Eclipse-RCP-clients, macros and scripts must be being created easily, analysis tools for the tests must be available as well as multi user support for the possibility of load testing.
After having chosen the right UI-test tool for this task it should be introduced at ALEA GmbH. Therefore corresponding tests for hardware sizings and standard functional tests for ALEA Commerce Suite have to be created, evaluated and documented. Die created scripts will be used at ALEA GmbH for “general functional tests” and load tests for hardware sizing in customer’s projects.
Evaluated Tools
- Automated GUI Recorder
- FIT
- The Grinder
- Silk Performer
- Silktest
- GUI Dancer
- HP Loadrunner
- HP Quicktest Professional
- QA-Load
- Squish
- QF-Test
The decision in favor of QF-Test
Out of the range of tools that are available the decision is taken in favor of QF-Test of Quality First Software. All primary criteria are met by this tool.
QF-Test for testing Eclipse/RCP (SWT) applications
Conclusion
Due to the automation of tests ALEA Commerce Suite can be tested by QF-Test after a new construction with all implemented changes. The short entrance test with all important functionalities can be executed quickly. The tests just have to concentrate on tests that are not automated because of high adaption costs. For standard functional tests as well as for sizing Scripts were created by the test cases. Sizing has already worked successfully once in August.
The whole evaluation report can be found here (German PDF only).
Project thesis: Test automation for software with a SWT-Rich-Client (RCP), Comparison of different GUI test tools (Open source and commercial) – October 2008, Patrice Förster, Staatliche Studienakademie Thüringen, Berufsakademie Gera, ALEA GmbH, Germany.
(Original German texts and citations are translated into English.)