QF-Test
Manual

AI26.4.3 Scripting with AI

The ai module is available in QF-Test scripts without any import and enables direct communication with a configured LLM. Typical use cases are dynamic text evaluation, generating test data at runtime, or AI-controlled interaction with an application using MCP tools.

26.4.3.1 Sending LLM Requests

"Object ask(String message, String configName, Double temperature, Double topP, Integer topK, Double frequencyPenalty, Double presencePenalty, Integer maxOutputTokens, List<String> stopSequences, Integer timeoutMs) throws TestException" sends a text message to a configured LLM and returns its response. Without specifying a configuration, the default configuration is used. 

// Simple request with default configuration
answer = ai.ask("Briefly explain what a regression test is")
rc.logMessage(answer)

// Request with explicit configuration
answer = ai.ask("Are you reachable?", "my-model")
Example 26.1:  Simple LLM requests (Groovy script)

"Object askWithTools(String message, String configName, ...) throws TestException" works identically but additionally allows the model to call the predefined MCP tools of QF-Test. This way the agent can interact with the application independently without you having to specify each individual step. 

ai.askWithTools("Open the application, log in as 'admin', and check whether the dashboard is displayed")
Example 26.2:  LLM request with MCP tool use (Groovy script)

26.4.3.2 Setting the AI Configuration in a Script

"void setDefaultConfig(String provider, String baseUrl, String apiKey, String modelName, String displayName="Default")" allows the active AI configuration to be changed at runtime without opening the options dialog. This is useful when different models are to be used for different tasks within the same test suite. "void resetDefaultConfig()" resets to the first configured configuration afterwards. 

// Set configuration at runtime
ai.setDefaultConfig("OpenAIGeneric", "https://api.openai.com/v1", api_key, "gpt-4o")
answer = ai.ask("Generate five test case names for a login test")

// Reset to the configured default configuration
ai.resetDefaultConfig()
Example 26.3:  Setting AI configuration at runtime (Groovy script)

"List<String> getConfigNames()" returns a list of all configuration names entered in the options dialog.

26.4.3.3 Retrieving Prompts

Procedures defined as MCP prompts in qfs-ai.qft or your own user-ai.qft can be retrieved as ready-to-use prompt text with "String getPrompt(String promptName, Map<String, Object> promptArguments)". The result can be passed directly to "Object ask(String message, String configName, Double temperature, Double topP, Integer topK, Double frequencyPenalty, Double presencePenalty, Integer maxOutputTokens, List<String> stopSequences, Integer timeoutMs) throws TestException". 

// Retrieve predefined prompt template and use it
prompt = ai.getPrompt("plan_tests", ["url": "https://example.com"])
testplan = ai.ask(prompt)
rc.logMessage(testplan)
Example 26.4:  Retrieving a prompt and sending it to the LLM (Groovy script)

"List<List<Object>> listPrompts()" returns an overview of all available prompts with name, title, and description.

26.4.3.4 Registering Custom Models

"void addCustomModel(final String name, final Object responseFunction)" lets you register your own model that communicates via any interface - for example a command-line tool or an external AI system. The registered model is then available under its name just like any other AI configuration.

The callback function receives not only the message but also optional parameters and a toolBridge object. The available QF-Test tools can be retrieved via toolBridge.listTools(); they can be called with toolBridge.callTool(name, params). Only if the custom model forwards the available tools to the AI system and processes their calls - or alternatively uses an external client connected directly to the QF-Test MCP server - can it be used for agentic features such as test generation. 

// Register an external CLI tool as an AI model
ai.addCustomModel("MyModel", (msg, parameters, toolBridge) -> {
    def cmd = ["my-ai-shell-command", msg]
    def proc = cmd.execute()
    proc.outputStream.close()
    proc.waitFor()
    return proc.in.text
})

// Use the registered model
answer = ai.ask("Tell a joke!", "MyModel")

// Remove the model again
ai.removeCustomModel("MyModel")
Example 26.5:  Registering a custom model (Groovy script)

The complete API reference for the ai module can be found under "The ai module".