Interpret run results

Relevant for: GUI tests and components,API testing, and business process tests and flows

This task describes navigate and interpret the information presented in the run results. This can help you quickly understand the results and errors in your test to isolate and solve problems in your application.

Set run result reporting options

Before you begin a test run, set options and properties to change the information included with the test results:

Option UI location Description
Instruct UFT to automatically open the run results Run Sessions pane (General tab > Run Sessions node) Select the View results when run session ends option.
Select the format of the run results Run sessions pane (General tab > Run Sessions node)

Select either the HTML Report or Run Results Viewer Report option.

Note: If you are running a business component, the HTML report option is supported only from ALM versions 12.53 and higher.

Take screen captures of steps Screen Capture pane (GUI Testing tab > Screen Capture pane)

Select the Save still image captures to results option. You can specify to save screen captures for:

  • Always (all steps)

  • For errors

  • For errors and warnings

Capture movies of the run session Screen Capture pane (GUI Testing tab > Screen capture pane)

Select the Save movie to results option. You can specify to capture the movie:

  • Always (all steps)

  • For errors

  • For errors and warnings

Save HTTP request/response body with the run results Properties pane for an HTTP request activity in an API test (Properties pane > General tab)

Set values to one or both of the following properties:

  • Save request body with this extension

  • Save response body with this extension

Run results with the request / response body are saved in the <Results>\Report\Data folder

Back to top

View step details

In the run results, you can see details for each test step. Do the following:

  1. In the lower part of the run results, display the Test Flow. The run results displays a full list of all steps included in the test run.

  2. From the step tree, select the step whose results you want to view. A summary of the details for that step are displayed.

  3. View the details for the step, including:

    • A description of the step, including the operation performed

    • The test object used in the test step and the test object description

    • A screen capture of the application with the test object highlighted (if you selected the option)

    • Call stack information, if the test step reported an error

Back to top

Analyze errors

In addition to reporting the results of every test step, the run results also display a special section detailing the errors and warnings that occurred during the test run. You can view these errors without the test flow to determine the root cause of the error:

  1. In the lower part of the run results, display the Errors List.

  2. Select an error from the list of errors. A summary of the error details are displayed.

    Tip: If you are in the Test Flow view, you can also use the Previous Error and Next Error buttons to quickly jump to the next error in the list.

  3. Use the available details to isolate the cause of the error including:

    • A description of the error

    • The test object being used in the step

    • The properties of the test object

    • The call stack of the current step

Back to top

Analyze checkpoint results

For each checkpoint step, view information about the checkpoint:

For a checkpoint that succeeded
  1. In the lower part of the run results, display the Test Flow.

  2. Select the checkpoint step from the step list. A summary of the step details is displayed.

  3. Use the information in the summary to view the checkpoint, including:

    • The properties checked in the checkpoint

    • The test object used in the checkpoint

    • The description properties of the test object used in the step

For a checkpoint that failed
  1. In the lower part of the run results, display the Errors.

  2. Select the failed checkpoint from the list. A summary of the error is displayed.

  3. Use the information in the summary to find the source of the error, including:

    • The properties checked in the checkpoint

    • The expected and actual values of the checkpoint

    • The test object used in the checkpoint

    • The description properties of the test object used in the step

Back to top

View the included data sources

If your test uses a data source, this data source is attached to the test as an external file. This enables you to see exactly which data was used for this test run.

The location of the data differs depending on the test type and the type of data source:

Test type Data source type Location
GUI test or component Data table

A link named Data displayed in the See More section above the Test Flow and Details area of the run results. The data table is opened as an Excel file.

You can also find the Excel file in the run results folder: <run results folder>\Report\Default.xls

API test
  • Excel

  • Local table

  • XML

  • Database

A link named Data displayed in the See More section above the Test Flow and Details area of the run results. This link opens a new browser page in which the specific data sources for the test are displayed. You can click on the name of a data source to view it as an external file.

Note: The actual run-time values used in each step (taken from the data source) are displayed in the Details section for each test step.

Note: The run results do not sync the selected step in the Test Flow with the data source.

Back to top

View the call stack to isolate errors in the test flow

When you have an error in your test, you can use the Call Stack to determine exactly where this error occurs. This helps you isolate the specific line in the test that contains the order.

  1. In the lower section of the run results, display the Errors.

  2. From the list of errors and warnings, select an error. The summary of the error is displayed

  3. In the error details, find the section containing the StackTrace. This section displays the following:

    • The section of the test containing the error (an action, function library, etc.)

    • The specific function containing the error (if the error occurred in the context of a function call)

    • The full script line in which the error occurred

    • The line number of the error in the relevant document

Back to top

View the step properties capture for an API test step

When you run an API test, each test step requires specific property values to run the step. In the run results, you can view the properties and the values used in this specific test run:

  1. In the lower section of the run results, display the Test Flow.

  2. In the Test Flow, select the step you want to view. A summary of the test step is displayed.

  3. In the summary, view the Captured Data section of the summary.

    If the Captured Data contains links to external resources (for example a Web service Request/Response), you can click the link in the Captured Data and a floating window displays the detailed data.

Back to top

View custom messages sent to the run results

For GUI tests

Use the Reporter object to send custom messages to the run results. These messages appear in the Test Flow in the step in which you inserted the statement.

For API tests

When you send a custom message to the run results from an step's event handler, it is displayed as part of the step's captured data, which includes the properties and values used for a specific test run.

  1. In the lower section of the run results, display the Test Flow.

  2. In the Test Flow, select the step you want to view. The details of the test step are displayed in the right pane.

  3. In the details, view the Captured Data. In the captured data, you can see the custom field added in the data grid.

Back to top

Send the run results by email

In the tab displaying the run results, right-click the tab name and select Send by Email. A mail message opens in your default mail application with the run results attached.

Note: On computers running Windows 7, after you open the mail message window, you cannot use any user interface items in the UFT window until the mail window is closed.

Back to top