Get X-Ray vision: Integrate QF-Test results with Q12-TMT and Xray

Q12-TMT and Xray Test Management are popular test management tools. In this special webinar, we will show you how to use the latest features in QF-Test to synchronize your test results with Q12-TMT and Xray. This way, you can combine QF-Test’s detailed reports with the strengths of these tools. The webinar will also include a brief introduction to Q12-TMT by our parent company, mgm technology partners.

Transcript

Chapter 1: Introduction

Once again, a very warm welcome to our special webinar. Today, we want to work together to answer the following questions:

• How do I keep track of all these application tests?
• How do I know what failed, when it failed, and how it failed?
• And is it actually a serious problem or not?

The key term here is, of course, test management systems.

Today, we will focus especially on two systems with which QF-Test has become closely integrated since the latest version:

• the well-known Xray system
• and the powerful Q12-TMT

Joining us today are two recognized experts on the topic:

• Gregor Schmidt, the creator of QF-Test, who even after 25 years is still the driving force behind QF-Test development
• and Thomas Rätzel from Leipzig, who has been a QA team lead at MGM for almost ten years and will introduce us to Q12-TMT

I myself am Pascal Bihler, also a developer at QF-Test, and I’ll be answering your questions alongside the presentation.

So let’s get started. Gregor, I’m already looking forward to it.

Gregor Schmidt

Thank you very much, Pascal, for the wonderful introduction.

The integration of test management tools into QF-Test is not a new topic. On the slide here on the right, you can see some earlier integrations, some of which have existed for a very long time, such as:

• HP ALM
• imbus TestBench

Recently, however, there has been increasing demand for a more direct integration with Xray.

Xray is currently probably the market leader based on Atlassian Jira — with all the advantages and disadvantages that come with it.

Our first approach was to investigate whether we could rewrite the JUnit reports so they could be uploaded directly into Xray. This is fundamentally possible, but very limited because it only works for this one exact use case.

At the same time, Q12-TMT was developed over the past few years. The tool is not yet very well known, but it comes from our parent company MGM Technology Partners and is already heavily used in many projects there — naturally also together with QF-Test.

As a result, there was also strong demand for tighter integration there.

Planned Features

For both test management tools, we are also working on:

• Import/export of complete test sets
• Hierarchies of test cases
• Synchronization between systems

These features are not yet ready for release, but they will come in future versions.

What we implemented first — because this was where the greatest demand existed — was:

• Uploading test results into the respective test management tools.

So this doesn’t remain too abstract, my colleague Thomas will now present Q12-TMT from a user perspective.

By the way, the concepts are very similar to Xray. Technically, we follow an almost identical approach for both systems — only the actual requests differ.

Thomas, over to you.

Chapter 2: Q12-TMT Demo

What is Q12-TMT?

Q12-TMT stands for: Test Management Tool

The tool is based on our in-house low-code platform R12.

Why did we develop our own testing tool?

Quite simply:

In the past, we worked with other tools, but certain features were missing. So we said: we are a software solutions company and capable of developing our own tool that maps our requirements exactly.

Basic Structure of TMT

TMT is relatively intuitive to use once you understand the basic structure.

Here we see an opened project. TMT supports multiple projects and different test management areas.

For this webinar, we are using the well-known Car Configurator as an example.

Project Overview

In the overview, you can see things such as:

• Number of test cases
• Processing status
• Versions
• Additional project information

On the left side are various navigation sections.

One important area is test automation.

For each test case, you can define whether it is:

• already automated
• still to be automated
• intentionally manual

This effectively creates a small task board for automation engineers.

Test Cases and Test Runs

The two most important sections are:

Test Suites

This is where you define:

• which test cases exist
• how they are organized

Test Runs

Here you can see:

• which tests were executed
• results
• statuses
• start times
• unique IDs

Graphical Analysis

Particularly interesting:

TMT offers graphical evaluations.

Using the so-called “fireworks diagram,” you can identify:

• error clusters
• concentrations of issues
• open problem areas

Colors directly indicate:

• Green = successful
• Red = failed
• additional status values

Structuring the Tests

The application can be structured however you like, for example:

• by UI areas
• functionally
• by components

In the example:

• Vehicles
• Specials
• Accessories

The structure is completely flexible and can be adapted to the respective testing strategy.

Structure of a Test Case

A test case typically contains:

• Title
• Status
• Responsible persons
• Automation status
• Description
• Jira links
• Risk assessment
• Preconditions
• Test steps
• Expected results

Additionally, formatting options are available for improved readability.

History & Versioning

A central feature of TMT is versioning.

This means:

• new versions can be created from existing ones
• histories are preserved
• test cases continue across versions
• changes are stored in a traceable way

Example:

• Version 1.8
• new version 2.0

Version Delta

A particularly important concept is the so-called version delta.

This means that new test cases are marked, and modified test cases are marked as well. This allows TMT to automatically recognize which test cases changed in a version. This becomes extremely helpful later during test execution.

Creating Test Runs

A new test run can be created very easily.

Afterward, test cases can be added:

Options:

• automatically via version delta
• manually via test suites
• specifically by automation status

Test Execution

During execution, the tester can:

• check off individual steps
• document results
• set statuses
• continue later

Additionally, tasks can be assigned directly to specific people.

Previous Workflow

Up until now, the workflow looked like this:

1. Manage test cases in TMT
2. Link them with QF-Test via IDs
3. Transfer results back manually

Naturally, this was relatively time-consuming.

That’s why today’s focus is on how this process can be automated in the future.

Chapter 3: The QA Toolchain from a Single Source

A major advantage of Q12-TMT: it does not only run in the cloud.

There are:

• a cloud variant
• and an on-premise installation on your own servers

This allows customers to decide flexibly depending on their requirements.

In addition, support requests can be handled jointly through the familiar QF-Test support team.

Target Vision

The vision is a seamless workflow:

QF-Test → TMT/Xray → Risk-based analysis

This creates a complete QA toolchain.

Chapter 4: Uploading Test Results to TMT/Xray

Requirements

For this to work, you need:

1. automated tests
2. at least one test case with a linking ID

For example:

• @TMTID
• @XID

Linking the Systems

The connection is established via so-called doctags.

Example:

@TMTID=2416
@XID=Xray-12

The following applies:

• multiple QF-Test test cases can be assigned to one management test case
• or vice versa

So this is an m:n relationship.

Chapter 5: Xray Integration

Configuration

For both systems, connection data must be configured:

TMT

• Login
• Project
• Subproject
• Version
• Mapping

Xray

• Jira access
• Cloud/Data Center configuration
• Ticket types
• IDs

Test Execution

During the test run, errors are intentionally generated so that results become visible.

In the log, you can then see examples such as:

• Exceptions
• Failed checks
• Components not found

HTML Reports

An HTML report is required for the upload.

This contains:

• Screenshots
• Error messages
• Logs
• Details

Upload to TMT and Xray

After the upload, the following are created automatically:

In TMT

• new test runs
• result overviews
• error statuses
• histories

In Xray

• test executions
• evidences (screenshots)
• comments
• report links

Differences in Status Values

Example in TMT:

• Error → “Failed”
• Exception → “Blocked”

The mapping can be customized individually.

History Tracking

An important point:

When tests are executed again:

• history is preserved
• new results overwrite the current status
• old results remain traceable

Batch Upload

In practice, of course, people do not work with individual reports.

Typical dimensions: 30,000 to 40,000 test cases per nightly run.

Therefore, QF-Test also supports:

• batch uploads
• command-line calls
• automated processing

Advantages of Batch Mode

• multiple reports simultaneously
• flexible configuration
• encrypted secrets
• option files
• full automation

Chapter 6: Summary

To conclude, here are the key points once again:

What was demonstrated?

• Linking QF-Test with TMT and Xray
• Uploading automated test results
• History tracking
• Batch processing
• Screenshots and reports directly in test management
• Integration of manual and automated tests

Result

This creates a central platform for:

• Test management
• Automation
• Analysis
• Risk assessment
• Histories

Many thanks once again to Thomas and Gregor for the detailed presentation — and best of luck with QF-Test and the test management tool of your choice.