Questions about AI Testing

Case 1 - Update check

Only in the GUI and only with your consent

The update check only takes place in the graphical user interface and only with the user’s consent. It never takes place in batch or daemon mode. You can disable it with the start parameter -noupdatecheck or via the option .

Case 2 - Browser drivers

Download only in WebDriver mode

Browser drivers are only downloaded in WebDriver mode for Chrome and Edge. In the default mode (CDP), no download is required.

Case 3 - AI features

Connection only with a configured language model

AI features only establish a connection if a language model is configured. With a local model, this connection stays within your network as well.

QF-Test does not collect any usage statistics or telemetry data. License checks and the license server don’t communicate with the internet either.

Yes. The installer includes the required Java runtime environment, so nothing is downloaded afterwards.

New versions are installed alongside the existing one, so you always keep a fallback.

License updates are applied via Help › Activate license…, or you simply replace the license file.

No. The license is a signed file that you store once.

Instances running at the same time coordinate exclusively within your network:

• In the local subnet via IP multicast
• Across network boundaries via a license server that you operate yourself

There is no online activation for renewals either.

Yes. For Chrome and Edge, QF-Test uses the Chrome DevTools Protocol (CDP) by default, which doesn’t require a separate driver. We ship GeckoDriver for Firefox, and Safari comes with its own driver.

Only if you deliberately choose WebDriver mode for Chrome or Edge does QF-Test automatically download chromedriver or msedgedriver. Alternatively, you can store the drivers internally and point to them with the environment variable QFTEST_WEBDRIVERS_ROOT.

Option 1 - Configuration

Don’t add an LLM, don’t enable the MCP server

Don’t create any entry in the “LLM Configurations” table and don’t check the “Enable embedded MCP server” option. Without a configured language model, QF-Test won’t make any AI requests.

Option 2 - System option

Enable the “Block AI features” option

The system option OPT_BLOCK_AI disables all functions that make LLM calls or provide AI interfaces in one go — regardless of any other settings.

Option 3 - Installation

Remove AI libraries

On Windows, AI features can be deselected during installation. On Linux and macOS, simply delete the files lib/langchain4j.jar and lib/mcp.jar from the installation directory.

All three options take permanent effect and can be combined — for example, in environments where a company policy prohibits the use of AI features entirely.

The AI can interact with the SUT using MCP tools provided by QF-Test. It uses the same component recognition and simulates user interactions in the same way.

When using AI in QF-Test, all actions, MCP tool uses, and responses from the AI agent are documented in the log. This enables full traceability of what the agent did and how it arrived at its results.

Autonomous exploration is designed for test environments. Irreversible actions, accidental deletions, or access to systems outside the test environment must be controlled by the test environment itself — not by the AI agent. This applies equally to any AI agent.

Depth limits can be specified in the test description and are generally respected in practice. However, there is currently no mechanism to technically enforce them. The agent can be stopped at any time if undesired behavior is observed during exploration.

There is currently no reliable way to prevent the AI from treating the current application state as expected behavior. However, this is not the primary purpose of the feature — it is meant to give QA teams a structured starting point for the test structure, which they then review and refine.

For stable tests, it is recommended to replace the generated AI nodes with traditional QF-Test nodes wherever possible.

The prompt sent to the AI agent consists of two parts: a base prompt (defined in the library test suite qfs-ai.qft, overridable in your own test suite such as user-ai.qft) and your test description. This description can be as detailed as you like.

You can include tickets and test plans, acceptance criteria, business rules, or references to existing tests in the prompt. As with any AI application, there is no guarantee that all details will be fully taken into account — the more precise and complete your description, the better the result.

Note that overloading the model’s context window can lead to worse results. Keep your descriptions detailed, but focused.

AI test generation in QF-Test is inherently non-deterministic: multiple runs of the same generation process produce different results. This is a fundamental characteristic of large language models.

If the test generation prompt remains the same across all runs, the structure and approach of the generated test suites will generally be similar — even if specific steps or wording differ. Treat each run as a draft to be reviewed and refined by your QA team — not as a finished test package.