QF-Test
Manual

10.1+28.2 Xray

Xray is a popular test management solution that integrates directly with JIRA. It allows teams to plan, track, and manage software testing within their existing JIRA environment. QF-Test provides tight integration with Xray, enabling efficient synchronization of test cases and test results.

28.2.1 Integration concepts

Association and correlation of test cases between QF-Test and Xray follows the same concept used for other test management tools. A doctag is chosen and specified in the option Doctag for identifying test cases in Xray. For Xray, the default doctag is @xrayid. For each QF-Test Test case node that implements an Xray test case, the corresponding @xrayid doctag should be added to its Comment attribute with the key of the Xray test case as the value, e.g., @xrayid PROJ-123. There is an N:M relationship between QF-Test and Xray test cases, meaning that a QF-Test test case can contain multiple @xrayid doctags, and several QF-Test test cases can contain the same doctag.

Test executions in Xray are represented as JIRA issues that define a set of tests to be performed. Test cases are added to the test execution, and the result for each test case is recorded after it has been executed. QF-Test can create a test execution on the fly when uploading data.

28.2.2 Uploading test results to Xray

When uploading test results to Xray, some decisions need to be made:

The following sections explain how to configure QF-Test to address these questions.

28.2.2.1 Settings and preconditions

QF-Test supports both Xray Cloud and Xray Data Center (DC). The variant is selected automatically based on the Base URL for Xray (Cloud only) option: if the base URL is empty, DC mode is used; if it is set to the Xray Cloud API endpoint, Cloud mode is used.

The following options are required for both variants. They can be set in the QF-Test options dialog or provided via command line arguments when running in batch mode:

The following options are required for Xray Cloud only:

For the client secret and JIRA API token, it is strongly recommended to use the encrypted form ${decrypt:...} instead of plain text. Please see Xray settings for details on all Xray-related options.

28.2.2.2 Interactive upload from the QF-Test UI

In addition to the batch mode workflow described below, you can upload test results to Xray directly from the QF-Test UI after an interactive test run.

To upload results from the QF-Test UI, proceed as follows:

  1. After a test run, open the run log in QF-Test.
  2. Click the Create report toolbar button or select »File«-»Create report...«. In the report creation dialog, check the Xray option in the export section to enable uploading to Xray.
  3. After confirming the report dialog, the Upload results to Xray dialog appears. This dialog exposes all relevant settings, including connection parameters, credentials, project key, and target test execution. Options already configured in the QF-Test options dialog are pre-filled automatically.
  4. Review and adjust the settings as needed, then confirm the dialog. QF-Test creates the report and uploads the test results to the specified Xray test execution. Progress is shown in the status bar during the upload.
  5. Once the upload completes, a result dialog informs you whether the upload was successful, along with any messages from the upload process.

The settings entered in the upload dialog are remembered for the rest of the session and pre-filled the next time you create a report with Xray upload enabled.

Note The target test execution must be specified in the upload dialog. Enter an existing JIRA issue key (e.g., PROJ-456) to add results to an existing test execution, or new (optionally followed by :<summary>) to create a new one.

28.2.2.3 Command line parameters

Uploading of test results happens as a side effect of report creation. To enable Xray processing, use the -report-xray argument when creating a report or running tests in batch mode. For general information about report creation, see "Create a report from run logs".

The following command line arguments are available for Xray integration. Most of these correspond to the options listed above:

-xray-project <project>
Specifies the JIRA project key for Xray test executions.
-xray-jiraurl <url>
Specifies the base URL of the JIRA instance that contains Xray.
-xray-jiratoken <token>
Specifies the JIRA API token (Cloud) or Personal Access Token (Data Center) for authentication. For Data Center, the token is used directly as a Bearer token and no username is required.
-xray-customfields <json>
Specifies default values for custom fields in Xray JIRA tickets. The format is a comma-separated list suitable for a JSON array.
-xray-url <url> (Cloud only)
Specifies the base URL of the Xray Cloud API endpoint. Setting this argument selects Cloud mode; omit it (or leave the corresponding option empty) for Data Center.
-xray-clientid <clientid> (Cloud only)
Specifies the client ID for authenticating with the Xray API.
-xray-secret <secret> (Cloud only)
Specifies the client secret for authenticating with the Xray API.
-xray-jirauser <user> (Cloud only)
Specifies the JIRA username for authentication with JIRA.
-xray-testexecution <id>
Specifies the ID of the Xray test execution to upload results to. This argument is required. Use new to create a new test execution with a default summary, or new:<summary> to specify a custom summary.
-xray-extend
Whether to automatically add test cases to an existing test execution if they are not yet part of it. Default is true. Use -xray-extend=false to disable.

Note Client secrets and API tokens on the command line are insecure. As an alternative, you can configure the options Xray client secret (Cloud only) and JIRA API token for Xray in the QF-Test options dialog, but in all cases, the encrypted form using ${decrypt:...} is recommended.

28.2.2.4 Example

The following example demonstrates a typical batch call using Xray Cloud that runs a test suite and uploads the results to a new test execution: 

qftest -batch -genreport -report ReportDir \
    -report-xray -xray-url "https://xray.cloud.getxray.app" \
    -xray-jiraurl "https://mycompany.atlassian.net" \
    -xray-testexecution "new:Nightly Build RC_2602_1" \
    SmokeTestSuite.qzp
Example 28.4:  Uploading test results to Xray Cloud

In this example, the client ID, client secret, JIRA username, and JIRA API token are assumed to be configured in the QF-Test options. If you prefer to pass them on the command line, add the -xray-clientid <clientid>, -xray-secret <secret>, -xray-jirauser <user>, and -xray-jiratoken <token> arguments accordingly.

For Xray Data Center, omit -xray-url <url> and replace the JIRA API token with a Personal Access Token. No -xray-jirauser <user> is needed: 

qftest -batch -genreport -report ReportDir \
    -report-xray \
    -xray-jiraurl "https://jira.mycompany.com" \
    -xray-jiratoken "${decrypt:QhtrY7w...}" \
    -xray-testexecution "new:Nightly Build RC_2602_1" \
    SmokeTestSuite.qzp
Example 28.5:  Uploading test results to Xray Data Center

To upload results to an existing test execution instead of creating a new one, replace -xray-testexecution "new:..." with the JIRA key of the test execution, e.g., -xray-testexecution "PROJ-456". In this case, set -xray-extend=false if you do not want test cases to be added to the test execution automatically.

28.2.2.5 Using an options file

Specifying all Xray-related arguments on the command line can be unwieldy and may bloat your batch scripts. As an alternative, you can place these arguments in an options file and reference it via the -options <file> argument. This approach keeps your command line concise and allows you to manage Xray settings separately.

For Xray Cloud, create a file named xray.options with the following content: 

xray-url=https://xray.cloud.getxray.app
xray-clientid=ABC123DEF456...
xray-secret=${decrypt:ylAr5/4nu...}
xray-jiraurl=https://mycompany.atlassian.net
xray-jirauser=automation
xray-jiratoken=${decrypt:QhtrY7w...}
xray-project=PROJ
Example 28.6:  Example xray.options file (Cloud)

For Xray Data Center, the file is shorter because no Xray API URL, client credentials, or JIRA username are needed: 

xray-jiraurl=https://jira.mycompany.com
xray-jiratoken=${decrypt:QhtrY7w...}
xray-project=PROJ
Example 28.7:  Example xray.options file (Data Center)

Then reference this file in your batch call: 

qftest -batch -genreport -report ReportDir \
    -options xray.options -report-xray \
    -xray-testexecution "new:Nightly Build RC_2602_1" \
    SmokeTestSuite.qzp
Example 28.8:  Batch call using an options file

Arguments from the command line override those from the options file, so you can still specify the test execution and other run-specific arguments directly. You can also use multiple options files to separate credentials from project settings, for example. The value for -options <file> must be a path relative to the working directory of the QF-Test batch run or an absolute path.

Note Store options files containing credentials securely and exclude them from version control.