Testsuite for the graphic user interface
Regression tests of the surface of the editor should be executed by a capture replay tool. The central issue of an interface test is the verification if the program reacts correctly to user’s inputs and if the editor is integrated correctly in Eclipse. On this purpose first of all an overview on the tools for Java UI tests is given. Two of these tools will be chosen and evaluated. Afterwards a description of the testsuite created with the tool chosen will follow.
Conclusion and selection
In total the commercial tool seems much more comfortable and robust. The remarkably fast and competent support via email shouldn’t be neglected. This is something that you cannot expect from an open source tool and that you generally speaken not find. An exception on this are maybe the forums of huge open source products like open office or Eclipse, but the SWTBot’s community isn’t this big. Finally, in table 3.2 all relevant differences between the two tools are summarized.
The result of the evaluation and in consideration of the focus areas that are set by the task was that QF-Test should be used as tool for the UI test. This decision was made in accordance with ubigrade and due to the detected differences – especially regarding to the different linearity in test creation.
Direct comparison between SWTBot and QF-Test
| Criteria | SWTBot | QF-Test |
|---|---|---|
Capture-Replay
|
No Capture component
|
Yes
|
API knowledge necessary
|
Extensive (SWTBot.*, Hamcrest)
|
Limited (Resolvers)
|
Injection method
|
Own target platform plugin
|
Instrumented JDK, modified SWT
|
Test creation effort
|
Very high (Line-by-line and transition between test realization and SUT)
|
Low (Capture, unrecognizable components as worst case)
|
Weaknesses in test creation
|
Line-by-line scripting, nonlinear work flow
|
Less control on SUT internals
|
Execution speed
|
Fast (In situ on SUT level
|
Fast (Direct instrumentation of the JDK)
|
Strength in test execution
|
Introspection
|
Deploy neutrality
|
Setup/Teardown
|
Just on SUT-level or via ant etc.
|
Procedures, SUT can be restarted between test runs
|
Customizability
|
A customized Hamcrest-Matcher usable
|
Customized resolvers usable
|
GEF-support
|
Bad [Mer09]
|
Mature, activatable via import
|
Table 3.2: Direct comparison between SWTBot and QF-Test
The whole extract from the evaluation report concerning QF-Test can be found here (German PDF only).
Diploma thesis: Conception and realization of a protocol model editor, chapter 3: Testsuite for a Graphic User Interface - QF-Test in comparison with SWTBot – January 2010, Christoph Schmidt, TU Dresden, Germany.
(Original German texts and citations are translated into English.)
