Execute Plugins

tmt

Warning

Please, be aware that the documentation below is a work in progress. We are working on fixing it, adding missing bits and generally making it better. Also, it was originaly used for command line help only, therefore the formatting is often suboptional.

Use the internal tmt executor to execute tests

The internal tmt executor runs tests on the guest one by one, shows testing progress and supports interactive debugging as well. Test result is based on the script exit code (for shell tests) or the results file (for beakerlib tests).

Configuration

duration:

The maximal time allowed for the test to run.

Default: 1h

exit-first: true|false

Stop execution after the first test failure.

Default: false

interactive: true|false

Run tests in interactive mode, i.e. with input and output streams shared with tmt itself. This allows input to be passed to tests via stdin, e.g. responding to password prompts. Test output in this mode is not captured, and duration has no effect.

Default: false

no-progress-bar: true|false

Disable interactive progress bar showing the current test.

Default: false

script: SCRIPT

Shell script to be executed as a test.

Default: not set

Common Keys

name:

The name of the step phase.

order:

Order in which the phase should be handled.

Default: 50

summary:

Concise summary describing purpose of the phase.

Default: not set

where: GUEST|ROLE

Run this phase on given guest or guests with the given role only.

Default: not set

upgrade

Warning

Please, be aware that the documentation below is a work in progress. We are working on fixing it, adding missing bits and generally making it better. Also, it was originaly used for command line help only, therefore the formatting is often suboptional.

Perform system upgrade during testing.

The upgrade executor runs the discovered tests (using the internal executor), then performs a set of upgrade tasks from a remote repository, and finally, re-runs the tests on the upgraded guest.

The IN_PLACE_UPGRADE environment variable is set during the test execution to differentiate between the stages of the test. It is set to old during the first execution and new during the second execution. Test names are prefixed with this value to make the names unique.

The upgrade tasks performing the actual system upgrade are taken from a remote repository either based on an upgrade path (e.g. fedora35to36) or filters. The upgrade path must correspond to a plan name in the remote repository whose discover step selects tests (upgrade tasks) performing the upgrade. Currently, selection of upgrade tasks in the remote repository can be done using both fmf and shell discover method. The supported keys in discover are:

  • ref

  • filter

  • exclude

  • tests

  • test

The environment variables defined in the remote upgrade path plan are passed to the upgrade tasks when they are executed. An example of an upgrade path plan (in the remote repository):

discover: # Selects appropriate upgrade tasks (L1 tests)
    how: fmf
    filter: "tag:fedora"
environment: # This is passed to upgrade tasks
    SOURCE: 35
    TARGET: 36
execute:
    how: tmt

If no upgrade path is specified in the plan, the tests (upgrade tasks) are selected based on the configuration of the upgrade plugin (e.g. based on the filter in its configuration).

If these two possible ways of specifying upgrade tasks are combined, the remote discover plan is used but its options are overridden with the values specified locally.

The same options and config keys and values can be used as in the internal executor.

Minimal execute config example with an upgrade path:

execute:
    how: upgrade
    url: https://github.com/teemtee/upgrade
    upgrade-path: /paths/fedora35to36

Execute config example without an upgrade path:

execute:
    how: upgrade
    url: https://github.com/teemtee/upgrade
    filter: "tag:fedora"

Configuration

duration:

The maximal time allowed for the test to run.

Default: 1h

exclude: REGEXP

Exclude a regular expression from search result.

Default: not set

exit-first: true|false

Stop execution after the first test failure.

Default: false

filter: FILTERS

Include only tests matching the filter.

Default: not set

interactive: true|false

Run tests in interactive mode, i.e. with input and output streams shared with tmt itself. This allows input to be passed to tests via stdin, e.g. responding to password prompts. Test output in this mode is not captured, and duration has no effect.

Default: false

no-progress-bar: true|false

Disable interactive progress bar showing the current test.

Default: false

ref: REVISION

Branch, tag or commit specifying the git revision.

Default: not set

script: SCRIPT

Shell script to be executed as a test.

Default: not set

test: NAMES

Select tests by name.

Default: not set

upgrade-path: PLAN_NAME

Upgrade path corresponding to a plan name in the repository with upgrade tasks.

Default: not set

url: REPOSITORY

URL of the git repository with upgrade tasks.

Default: not set

Common Keys

name:

The name of the step phase.

order:

Order in which the phase should be handled.

Default: 50

summary:

Concise summary describing purpose of the phase.

Default: not set

where: GUEST|ROLE

Run this phase on given guest or guests with the given role only.

Default: not set