QF-Test
Manual

AI52.3 The ai module

10.0+The ai module lets you access the LLM integration of QF-Test from scripts.

 
 
Object ask(String message, String configName, Double temperature, Double topP, Integer topK, Double frequencyPenalty, Double presencePenalty, Integer maxOutputTokens, List<String> stopSequences, Integer timeoutMs) throws TestException

Sends a message to a configured LLM and returns the response. 

print(ai.ask("Are you there?", "my-llm"))
                print(ai.ask(
                "Let's try something more complex",
                "my-llm",
                temperature=0.7,
                topK=4,
                topP=0.0,
                presencePenalty=1.0,
                frequencyPenalty=0.9,
                maxOutputTokens=1000,
                stopSequences=['stop!'],
                timeoutMs=10000,
                ))
Example 52.5:  Usage of ai.ask (Jython script)
Parameters
message The message to send to the LLM.
configurationName The name of a LLM configuration as set in LLM Configurations in the QF-Test options
temperature Controls the randomness of the generated text.
topK Controls how many of the most likely tokens should be considered.
topP Controls which tokens should be considered.
presencePenalty Prevents the llm from reusing words.
frequencyPenalty Prevents the llm from repeating words.
maxOutputTokens Maximum number of tokens that can be generated in the response.
stopSequences Prevent LLM from generating more text after that string appears.
timeoutMs How long to wait for a response from the LLM, in milliseconds.
Returns The answer from the LLM. The return type depends on the response content: a String for text-only responses, an ImageRep when the LLM returns a single image without text, and a List of String and/or ImageRep elements when the response contains multiple content parts.
 
Object askWithTools(String message, String configName, ...) throws TestException

Executes the same LLM request as "Object ask(String message, String configName, Double temperature, Double topP, Integer topK, Double frequencyPenalty, Double presencePenalty, Integer maxOutputTokens, List<String> stopSequences, Integer timeoutMs) throws TestException", but allows the AI system to use the Built-in MCP Tools of QF-Test. 

print(ai.askWithTools("Click on the button to close the application"))
Example 52.6:  Usage of ai.askWithTools (Jython script)
Parameters
message The message to send to the LLM.
configurationName The name of a LLM configuration as set in LLM Configurations in the QF-Test options
Returns The answer from the LLM. The return type depends on the response content: a String for text-only responses, an ImageRep when the LLM returns a single image without text, and a List of String and/or ImageRep elements when the response contains multiple content parts.
 
void addCustomModel(final String name, final Object responseFunction)

Adds a custom model to the list of defined AI configurations

Note For a custom model to be used for agentic features such as test generation, the callback function must forward the QF-Test tools provided via toolBridge to the AI system and process their calls. Alternatively, the custom model can use an external client connected directly to the QF-Test MCP server. 

if ("myLLM" not in ai.getConfigNames()) {
                ai.addCustomModel("myLLM", { msg, parameters, toolBridge ->
                qf.logMessage("Input for the LLM: " + msg)
                return "Unfortunately, I cannot help you."
                })
                }
Example 52.7:  Usage of ai.addCustomModel (Jython script)
Parameters
name The AI configuration name under which the custom model can be referenced
responseFunction A function which receives the query with three arguments:
msg
The chat message to process.
parameters
(optional) The request parameters.
toolBridge
(optional) If tool access is allowed, this bridge object allows to get a list of available tools (toolBridge.listTools()) and to call the tools (via toolBridge.callTool(toolName, toolParameterMap)). Otherwise, the argument is null.
 
void removeCustomModel(final String name)

Removes the custom model from the list of defined models

Parameters
name The name under which the custom model was registered
 
List<String> getConfigNames()

Returns a list of all available AI configuration names

Returns A List of all available AI configuration names
 
void setDefaultConfig(String provider, String baseUrl, String apiKey, String modelName, String displayName="Default")

Define a default LLM configuration to use instead of the first one configured in Options > Artificial Intelligence.

Parameters
provider Currently available provider types: OpenAIGeneric, Anthropic, Gemini and Ollama.
baseUrl The base API URL endpoint to the provider. Usually ends with /v1 oder similar.
apiKey The API key to submit to the provider.
modelName The name of the model to use, like gpt-4o or gemini-2.0-flash
displayName How QF-Test will identify the configuration in logs and error messages, defaults to Default
 
void resetDefaultConfig()

Reset the default LLM configuration previously set via ai.setDefaultConfig to use the first one defined in Options > Artificial Intelligence.

 
String getPrompt(String promptName, Map<String, Object> promptArguments)

Get an MCP prompt by name with the given arguments and return the result as a single String. This prompt can be used with ask or askWithTools.

Parameters
promptName The name of the prompt.
promptArguments The arguments of the prompt
Returns The joined text messages of the prompt.
 
List<List<Object>> listPrompts()

List available MCP prompts. Each entry is a map with keys: name (String), title (String), description (String), arguments (Map<String, Object>).

Returns A List of all available MCP prompts.