Code coverage analysis can be used in order to analyze to which degree the source code of a program gets executed during test execution. A high percentage of executed code suggests a lower chance of undetected software bugs.
QF-Test alone does not provide a possibility to generate code coverage reports. However it is possible to combine QF-Test with Open Source Tools like JaCoCo. The integration of such tools make it possible to generate such code coverage reports for the teststeps performed by QF-Test.
How to integrate JaCoCo into a QF‑Test test run
In order to integrate JaCoCo into a QF-Test test run, you first need to extract the ‘jacocoagent.jar’ file from the lib folder of the ‘jacoco-$(version).zip’ file. This zip file can be downloaded from here.
It is easiest to integrate JaCoCo, if the Java application you are testing is a .jar or a .class file. In this case you simply need to locate the ‘Start Java SUT client’ node that is starting your Java application. Then, specify
-javaagent:
in the “parameter table” of this node.
In case you have a .exe or a .bat Java application, the above way will not work. In this case two Jython Server Scripts are needed. The first script with the content
jacocojar = "<path to jacocoagent.jar>"
destfile = "<path+filename to where jacoco should write its statistics>"
rc.setProperty("env", "JAVA_TOOL_OPTIONS", "-javaagent:%s=destfile=%s" % (jacocojar, destfile))`
needs to get inserted before the ‘Start SUT Client’ node and another more optional script with
rc.setProperty("env", "JAVA_TOOL_OPTIONS", None)
can be inserted after the ‘Wait for client to connect’ node.
Remarks
- Please note, that JaCoCo is not directly writing its report into the result file. The result file is written, when the application you are testing gets terminated!
- The JaCoCo result file is quite unreadable when opened with most text editors. So in order to analyze the statistics you probably want to use a plugin for your IDE.
- In order to gain control over the application you are testing, QF-Test is injecting some of its own classes into the application. As a result the result file does not only contain code coverage statistics for the classes of your application, but also for some QF-Test classes. Because you probably do not want to test QF-Test but your application, you can simply ignore the statistics created for QF-Test classes.
- Please be sure that the process that is executing your application has the rights to write the JaCoCo result file.
- When specifying a (Windows) path in a Jython/Groovy Script, please replace all “" characters in the path by “/”.