Dependencies in QF-Test (basics)
See here why the so called dependencies are useful for your daily work with QF-Test and you’ll learn how to implement them.
Materials for this video
Abridged Transcript
This video covers the basics of QF-Test dependencies. It explains the purpose and intention of dependencies and shows how to implement a simple dependency.
This video does not cover advanced topics such as nested dependencies, characteristic variables, or error handling. Those parts are discussed in other videos.
For the current video, I work in a specially provided test suite, which you can also download from the QF-Test website.
The test suite uses the CarConfigurator provided by QF-Test in the web version as the “system under test” (SUT).
- If you would like to work along with me during the video, you will find the suite in the commentary section.
- If you would prefer to see the result of this tutorial straight away, you will also find that suite there.
Opening the Test Suite
Now I have opened the dependency test suite I just mentioned. Let’s take a closer look at its implementation.
Here you can see a test suite implemented in a simple way:
- This is the Setup node that starts our “system under test” (the CarConfigurator)
- Here are some test cases
- Finally, a Cleanup node will close our SUT again
In this example, a single test case runs as follows:
- First, the application is reset
- Then a vehicle is selected
- Then a special model is selected
- Then further accessories are selected
- A discount is set
- Finally, the calculated final price is checked
The setup and Cleanup sequences use procedure calls. By double-clicking on a procedure call, I also get to the Procedure node of QF-Test where I can view the existing procedures of this test suite.
- The SUT package contains the procedures for starting and stopping our CarConfigurator
- The CarConfig package contains procedures needed for the test cases (these are not of interest for the time being)
Running Test Cases
Now that you are more or less familiar with our test suite, I will run test cases from it:
- I select the test set and press “Start test run”
- QF-Test now works its way through this test suite and executes the test cases with the different discount levels
We wait until the test run is complete.
The test case has run through and we can take a look at the QF-Test run log:
- In the details of the run log, we can see that first the setup was run to start the SUT
- Then the first test case was executed
- Then the application was completely closed with the Cleanup
- Then the application was restarted and the next test case was executed
For a complete restart on my system, we need approximately:
- 28 seconds for the setup
- Almost 1 second for the Cleanup work
The whole thing adds up and accounts for a large part of the total test runtime, which we can also see in the performance view.
Particularly with larger applications, constantly stopping and starting the application would be inefficient and does not represent real user behavior.
In our case, this tidying up is actually unnecessary during a normal run of a test case. The SUT has already been started and changes are reset in the test cases anyway.
However, I cannot simply remove the Cleanup node, otherwise the application would still run on the test machine forever after a test run.
In addition, an error in the SUT in an unattended test run could compromise the entire test run instead of just a single test case.
This shows the first disadvantage of the current implementation:
- The Cleanup node is executed at the end of every test case
Another disadvantage is that it is not easily possible to start just a single test case.
- For example, if a test case had thrown an exception during the last test run (due to an implementation error or a defect in the SUT), I cannot simply execute this test case again
- I first have to run the setup to start the application, or create the preconditions manually, which ultimately means executing the setup again
This is still simple in this example, but as soon as test cases become larger and more complex, the entire setup can become tedious.
We can see that we won’t get very far with these means.
Introducing Dependencies
QF-Test dependencies now enter the stage.
Dependencies ensure that the preconditions for a test case are fulfilled, if implemented correctly.
- Individual test cases can be executed without manually creating preconditions beforehand
- This works even if a completely different test case was previously executed
- Dependencies also regulate the Cleanup process and do not carry out unnecessary Cleanup work
Summary:
- Dependencies ensure a certain state
- Setup and Cleanup nodes only execute a certain sequence
Creating a Simple Dependency
- I use the test set from earlier and copy it directly below
- Rename the test case, for example, “Tests with dependencies”
-
Insert a Dependency node:
-
Menu:
Insert → Dependencies → Dependency -
Right-click:
Insert node → Dependency -
Name the dependency “SUT started”
- Add a Setup node and Cleanup node by dragging the existing nodes into the dependency
Why define the dependency at the test set level?
- Each node inherits the dependencies of its parent node by default
- In this case, each test case inherits the dependencies of the test set
- Defining it at test set level is sufficient as it automatically applies to all contained test cases
Testing the Dependency
- Ensure no SUT is running.
- Try to start a single test case in the old test set → QF-Test throws a ComponentNotFound exception
Now try the new dependency:
- Execute the first test case of the new test set
- The test case is executed without any problems, even though the precondition was not fulfilled before the test case started
What happened:
- QF-Test first looked at the Dependency when the test case was started
- The entry
"Required dependency"indicates that the test case requires"SUT started" - QF-Test jumps to the dependency and executes the Setup node
- The SUT starts automatically
Full Test Set Execution with Dependency
- Close the run log and all clients
- Execute the entire test set
Observations:
- Execution is much faster than the old test set
- The application is not closed between individual test cases
- From the run log, we see that no Cleanup node is executed between test cases
Time savings:
- Around 30 seconds per test case from the second test case onward
- Around 2 seconds on the first test case
How Dependencies Work
- Precondition setup sequences are ensured at the beginning of each test case
- QF-Test maintains a list of currently valid dependencies (
Stored dependencies) - At the beginning of the first test case, the list is empty
- When the dependency is executed, QF-Test saves the dependency name in the list
- Every new test case compares its required dependency with the stored dependencies
- If they match, the Cleanup node is skipped
Setup nodes:
- Should start the SUT only if it is not already running
- In the example, a Wait for client node checks whether the CarConfigurator client is available
- The result is stored in
IsSUTRunning - The subsequent If node starts the SUT only if necessary
Advanced Behavior
- Another test set demonstrates handling of different dependencies
- Required dependencies are compared with stored dependencies
- If a stored dependency is no longer needed, QF-Test executes the Cleanup
- This allows combining and executing test cases from multiple test sets intelligently
Reusing Dependencies
- Dependencies can be copied between test sets, but changes would require updating each copy manually
- QF-Test allows storing dependencies in the Procedures section
- New test sets or test cases can reference these stored dependencies
- The “Dependency reference” node acts as a link to the existing dependency
Conclusion
- Dependencies simplify test case execution and precondition management
- Setup nodes are executed at the start of each test case
- Cleanup nodes are executed only when necessary
- Using stored dependencies allows reuse across multiple test sets
In a follow-up video, we will cover nested dependencies and error and exception handling.
Check your own QF-Test test suites to see if dependencies are already used — implementing them is definitely worth it!