The TestRunListener interface can be used to execute additional actions before or after the execution of each node or in the case of any exception / error.
This actions can (for example) be used for testdocumentation or error analysis.
These actions can for example be used for test documentation or error analysis. In the following some TestRunListeners are introduced (Jython server scripts):
Variable monitoring
The following TestRunListener always logs an error when the value of a certain variable (in the example offset) changes. This is especially helpful when debugging a test suite if it is not clear where exactly the variable is assigned a certain value:
varname = "offset"
from de.qfs.apps.qftest.extensions.qftest import AbstractTestRunListener
def checkVar(state, var):
try:
val = rc.lookup(var)
if(val != state):
state = val
rc.logError("Variable '%s': %s" % (var, state))
except:
if state != "":
rc.logError("Variable '%s' nicht verfügbar" % var)
state = ""
return state
class VarChanger (AbstractTestRunListener):
def __init__(self):
self.state = None
def nodeEntered(self, event):
self.state = checkVar(self.state, varname)
def nodeExited(self, event):
self.state = checkVar(self.state, varname)
global varChanger
try:
rc.removeTestRunListener(varChanger)
except:
pass
varChanger = VarChanger()
rc.addTestRunListener(varChanger)
Alternatively, it’s easy to adapt this script so that the TestRunListener always logs an error when a certain value is assigned to the variable.
Error sound
It can happen that a user starts several tests on several computers at the same time. The following TestRunListener helps if it can happen that the test execution on a machine – for whatever reason – can come to a standstill; The TestRunListener always plays a tone if the execution of a node takes longer than a certain specified time limit. It is possible to redefine this time limit for a node using a @informUserTimeOut doctag. (Note: The sound output probably only works on Windows)
# A TestRunListener that informs the user when the current testrun has stopped for more then a certain period of time.
# some imports we need later
from de.qfs.apps.qftest.extensions.qftest import AbstractTestRunListener
from java.lang import System, Thread
from java.awt import Toolkit
import time, re
# The default time in ms after which to inform
# the user.
TIME_IN_MS_AFTER_WHICH_TO_INFORM_THE_USER = 2 * 60 * 1000
# The function that is used in order to inform the user
def informUserAction(lastActionTimestamp):
""" This function is called whenever the test-
runlistener detected the timeout. In oder words,
this function should play a sound or do something
else in order to inform the user.
:lastActionTimestamp: The timestamp of the last action.
"""
print "=== HEY %s ms passed, since we entered the last node! ===" % (System.currentTimeMillis() - lastActionTimestamp)
# output a sound on windows
Toolkit.getDefaultToolkit().getDesktopProperty("win.sound.exclamation").run()
# the thread that will inform you once the timeout is reached ...
class InformingThread(Thread):
def __init__(self):
self.updateTimeout(TIME_IN_MS_AFTER_WHICH_TO_INFORM_THE_USER)
self.stop = False
def updateTimeout(self, timeout):
self.lastAction = System.currentTimeMillis()
self.errorAfterTimestamp = self.lastAction + timeout
def run(self):
while not self.stop:
time.sleep(1)
if System.currentTimeMillis() > self.errorAfterTimestamp and not self.stop:
try: informUserAction(self.lastAction)
except: pass
# the testrunlistener that keeps track of when a new node get's enetered ...
class InformUserWhenHaveBeenIdleTestRunListener(AbstractTestRunListener):
def __init__(self):
self.thread = InformingThread()
self.thread.start()
self.myRegex = re.compile("@informUserTimeOut\\s+(\\d+)", re.DOTALL)
def nodeEntered(self, event):
timeout = 0
comment = event.getNode().comment
if comment != None and comment.find("@informUserTimeOut") != -1:
match = self.myRegex.search(comment)
if match:
timeout = int(match.group(1))
self.thread.updateTimeout(TIME_IN_MS_AFTER_WHICH_TO_INFORM_THE_USER if timeout <= 0 else timeout)
def runStopped(self, event):
self.thread.stop = True
# register the testrun listener
global informUserWhenHaveBeenIdleTestRunListener
try:
informUserWhenHaveBeenIdleTestRunListener.thread.stop = True
rc.removeTestRunListener(informUserWhenHaveBeenIdleTestRunListener)
except:
pass
informUserWhenHaveBeenIdleTestRunListener = InformUserWhenHaveBeenIdleTestRunListener()
rc.addTestRunListener(informUserWhenHaveBeenIdleTestRunListener)
Counting check nodes
The following TestRunListener counts how many check nodes where executed during test execution:
from de.qfs.apps.qftest.extensions.qftest import AbstractTestRunListener
class StatisticTestRunner (AbstractTestRunListener):
def __init__(self):
self.steps, self.checkSteps = 0, 0
def runStopped(self, event):
print "steps: %s\ncheckSteps: %s" % (self.steps, self.checkSteps)
self.steps, self.checkSteps = 0, 0 # reset counts
def nodeEntered(self, event):
self.steps += 1
if (event.getNode().getType().startswith("Check")):
self.checkSteps += 1
global statisticTestRunner
try:
rc.removeTestRunListener(statisticTestRunner)
except:
pass
statisticTestRunner = StatisticTestRunner()
rc.addTestRunListener(statisticTestRunner)
General remarks
- Similar to SUT Scripts you should avoid calling rc.callProcedure from within a TestRunListener. Although it’s often possible to call rc.callProcedure from within a TestRunListener, as this can lead to unexpected problems.
- Instead of derivating from TestRunListener you should always derivate from AbstractTestRunListener.