Large test projects may consist out of several thousands of test cases. Often executing them all would need a considerable amount of time, even if multiple machines are used in order to replay the different tests.
In order to develop such large test projects, it is often useful to store the different test sets/cases in different files. For example we may create one file that contains all tests for feature A and another file that contains all tests for feature B. Nevertheless it may still be a good idea to annotate the different tests with tags.
Tags like “feature1”, “feature2”, “fastRunningTest”, “slowTest”, “importantTest”, “oftenFailingTest” and so on make it possible to run a dedicated subset of all test cases. For example this makes it possible to run only those test cases tagged with “oftenFailingTest” and “importantTest”.
The “Condition” field of a test case/test-set node can be used to assign tags to any test case/test-set. For example if you want to assign the three tags tag1, tag2 and tag3 to a test case / test-set you can use the following syntax:
"""${default:tagsToExecute:}""" == "" or len(set(["tag1", "tag2", "tag3"]).intersection([x.strip() for x in rc.lookup("tagsToExecute", expand=False).split(",")])) >= 1
This condition will advice QF-Test to execute the test case/set when one of the following conditions are met:
- The
tagsToExecute
variable is not set or empty. - The test case/test-set is annotated with a tag that is also listed in the
tagsToExecute
variable (comma-separated list of tags).
This means in order to execute only those tests annotated with tag1, the variable tagsToExecute
needs to be set to tag1
. In order to execute those tags either annotated with tag1
or tag2
the variable tagsToExecute
needs to be set to tag1
,tag2
and in order to execute all tests annotated with tag1, tag2 or tag3 the variable tagsToExecute
needs to be set to tag1
,tag2
,tag3
.
Remarks
- When calling QF-Test from command line, it is possible to set a
tagsToExecute
variable via the -variable command line argument (e.g. -variable tagsToExecute=tag1,tag2), see
Command line arguments and exit codes - For better readability, it is allowed to add spaces before and after the comma. So in order to execute the tests annotated with
tag1
ortag2
the variabletagsToExecute
may be set to
tag1
,tag2
or
tag1
,tag2
or eventag1
,tag2
.