QF-Test
Manual

10.0+28.1 Q12-TMT

Q12-TMT is a tool for professional software test management in complex IT projects, created by mgm technology partners. Tight integration with QF-Test is an ongoing process. Already available is the ability to import test results from QF-Test into TMT. Bidirectional import/export of test cases is on the agenda.

28.1.1 Integration concepts

Association and correlation of test cases between QF-Test and a test management tool is a concept that goes beyond TMT. In short, an arbitrary but management-tool-related doctag is chosen and specified in the option Doctag for identifying test cases in TMT. For TMT, the default doctag is @tmtid. For each QF-Test Test case node that implements a TMT test case, the corresponding @tmtid doctag should be added to its Comment attribute with the ID of the TMT test case as the value, e.g., @tmtid C_36. There is an N:M relationship between QF-Test and TMT test cases, meaning that a QF-Test test case can contain multiple @tmtid doctags, and several QF-Test test cases can contain the same doctag.

For close integration with TMT, it can also be useful to associate QF-Test test suites via their Test suite nodes with TMT test suites (e.g., @tmtid S_11), Test set nodes with TMT folders (e.g., @tmtid F_36), and Test step nodes with TMT test steps (e.g., @tmtid TS_C_36_1). The latter is an artificial concept. Since TMT test steps don't have their own ID, they are represented via the containing test case plus their 1-based numeric index. If such additional @tmtid doctags are used, the option Optional regex that TMT test cases must match must be set to C_.* so that QF-Test can identify TMT test cases correctly.

Test runs in TMT define a set of tasks to be performed, either manually or automatically or, in some cases, both. To that end, test cases are added to the test run, and the result for each test case is set after it has been executed. QF-Test can create a test run on the fly when uploading data.

28.1.2 Uploading test results to TMT

When uploading test results to TMT, some decisions need to be made:

The following sections explain how to configure QF-Test to address these questions.

28.1.2.1 Settings and preconditions

For interaction with TMT, several options must be configured. These can be set in the QF-Test options dialog or provided via command line arguments when running in batch mode:

For the client secret and password, it is strongly recommended to use the encrypted form ${decrypt:...} instead of plain text. Please see TMT settings for details on all TMT-related options.

28.1.2.2 Interactive upload from the QF-Test UI

In addition to the batch mode workflow described below, you can upload test results to TMT directly from the QF-Test UI after an interactive test run.

To upload results from the QF-Test UI, proceed as follows:

  1. After a test run, open the run log in QF-Test.
  2. Click the Create report toolbar button or select »File«-»Create report...«. In the report creation dialog, check the TMT option in the export section to enable uploading to TMT.
  3. After confirming the report dialog, the Upload results to TMT dialog appears. This dialog exposes all relevant settings, including connection parameters, project, version, and target test run. Options already configured in the QF-Test options dialog are pre-filled automatically.
  4. Review and adjust the settings as needed, then confirm the dialog. QF-Test creates the report and uploads the test results to the specified TMT test run. Progress is shown in the status bar during the upload.
  5. Once the upload completes, a result dialog informs you whether the upload was successful, along with any messages from the upload process.

The settings entered in the upload dialog are remembered for the rest of the session and pre-filled the next time you create a report with TMT upload enabled.

Note The target test run must be specified in the upload dialog. Enter an existing test run ID (e.g., R_123) to add results to an existing run, or new (optionally followed by :<title>) to create a new run.

28.1.2.3 Command line parameters

Uploading of test results happens as a side effect of report creation. To enable TMT processing, use the -report-tmt argument when creating a report or running tests in batch mode. For general information about report creation, see "Create a report from run logs".

The following command line arguments are available for TMT integration. Most of these correspond to the options listed above:

-tmt-url <url>
Specifies the base URL of the TMT server to connect to.
-tmt-clientid <clientid>
Specifies the client ID for authenticating with the TMT API. Defaults to uaa-auth-client.
-tmt-secret <secret>
Specifies the client secret for authenticating with the TMT API.
-tmt-user <user>
Specifies the username for authenticating with TMT.
-tmt-password <password>
Specifies the password for the specified TMT user.
-tmt-project <project>
Specifies the ID of the TMT project to upload results to.
-tmt-subproject <subproject>
Specifies the ID of the TMT subproject. Leave empty if the project does not use subprojects.
-tmt-version <version>
Specifies the version of the TMT project to which test results are uploaded.
-tmt-testrun <testrun>
Specifies the ID of the TMT test run to upload results to. This argument is required. Use new to create a new test run with a default title, or new:<title> to specify a custom title.
-tmt-extend
Whether to automatically add test cases to an existing test run if they are not yet part of it. Default is true. Use -tmt-extend=false to disable.

Note Passwords and client secrets on the command line are insecure. As an alternative, you can configure the options Client secret for TMT and TMT password in the QF-Test options dialog, but in all cases, the encrypted form using ${decrypt:...} is recommended.

28.1.2.4 Example

The following example demonstrates a typical batch call that runs a test suite and uploads the results to a new TMT test run: 

qftest -batch -genreport -report ReportDir \
    -report-tmt -tmt-url "https://tmt.example.com" \
    -tmt-project "P_42" -tmt-suproject "SP_6" -tmt-version "1.1.0" \
    -tmt-testrun "new:Nightly Build RC_2602_1" \
    SmokeTestSuite.qzp
Example 28.1:  Uploading test results to TMT

In this example, the client secret, username, and password are assumed to be configured in the QF-Test options. If you prefer to pass them on the command line, add the -tmt-secret <secret>, -tmt-user <user>, and -tmt-password <password> arguments accordingly. Conversely, the -tmt-project <project>, -tmt-subproject <subproject>, and -tmt-version <version> arguments may be omitted if the corresponding options are configured.

To upload results to an existing test run instead of creating a new one, replace -tmt-testrun "new:..." with the ID of the test run, e.g., -tmt-testrun "T_123". In this case, set -tmt-extend=false if you do not want test cases to be added to the test run automatically.

28.1.2.5 Using an options file

Specifying all TMT-related arguments on the command line can be unwieldy and may bloat your batch scripts. As an alternative, you can place these arguments in an options file and reference it via the -options <file> argument. This approach keeps your command line concise and allows you to manage TMT settings separately.

Create a file named tmt.options with the following content: 

tmt-url=https://tmt.example.com
tmt-secret=${decrypt:ylAr5/4nu...}
tmt-user=automation
tmt-password=${decrypt:QhtrY7w...}
tmt-project=P_42
tmt-subproject=SP_6
tmt-version=1.1.0
Example 28.2:  Example tmt.options file

Then reference this file in your batch call: 

qftest -batch -genreport -report ReportDir \
    -options tmt.options -report-tmt \
    -tmt-testrun "new:Nightly Build RC_2602_1" \
    SmokeTestSuite.qzp
Example 28.3:  Batch call using an options file

Arguments from the command line override those from the options file, so you can still specify the test run and other run-specific arguments directly. You can also use multiple options files to separate credentials from project settings, for example. The value for -options <file> must be a path relative to the working directory of the QF-Test batch run or an absolute path.

Note Store options files containing credentials securely and exclude them from version control.