Discover Plugins

Common Keys

The following keys are accepted by all plugins of the discover step.

name:

The name of the step phase.

order:

Order in which the phase should be handled.

Default: 50

summary:

Concise summary describing purpose of the phase.

Default: not set

when: RULE

If specified, phase is run only if any rule matches plan context.

Default: not set

where: GUEST|ROLE

Run this phase on given guest or guests with the given role only.

Default: not set

fmf

Warning

Please, be aware that the documentation below is a work in progress. We are working on fixing it, adding missing bits and generally making it better. Also, it was originally used for command line help only, therefore the formatting is often suboptimal.

Discover available tests from fmf metadata.

By default all available tests from the current repository are used so the minimal configuration looks like this:

discover:
    how: fmf

Full config example:

discover:
    how: fmf
    url: https://github.com/teemtee/tmt
    ref: main
    path: /fmf/root
    test: /tests/basic
    filter: 'tier: 1'

If no ref is provided, the default branch from the origin is used.

The following keys are used to limit the test discovery:

test - list of test names or regular expressions used to select tests

link - select tests using the link keys

filter - apply advanced filter based on test metadata attributes

exclude - exclude tests which match a regular expression

prune - copy only immediate directories of executed tests and their required files

For DistGit repo one can download sources and use code from them in the tests. Sources are extracted into $TMT_SOURCE_DIR path, patches are applied by default. See options to install build dependencies or to just download sources without applying patches. To apply patches, special prepare phase with order 60 is added, and prepare step has to be enabled for it to run.

It can be used together with ref, path and url, however ref is not possible without using url.

discover:
    how: fmf
    dist-git-source: true

Selecting tests containing specified link is possible using link key accepting RELATION:TARGET format of values. Regular expressions are supported for both relation and target part of the value. Relation can be omitted to target match any relation.

discover:
    how: fmf
    link: verifies:.*issue/850$

It is also possible to limit tests only to those that have changed in git since a given revision. This can be particularly useful when testing changes to tests themselves (e.g. in a pull request CI).

Related keys: modified-only, modified-url, modified-ref

Example to compare local repo against upstream main branch:

discover:
    how: fmf
    modified-only: True
    modified-url: https://github.com/teemtee/tmt
    modified-ref: reference/main

Note that internally the modified tests are appended to the list specified via test, so those tests will also be selected even if not modified.

Use the adjust-tests key to modify the discovered tests’ metadata directly from the plan. For example, extend the test duration for slow hardware or modify the list of required packages when you do not have write access to the remote test repository. The value should follow the adjust rules syntax.

The following example adds an avc check for each discovered test, doubles its duration and replaces each occurrence of the word python3.11 in the list of required packages.

discover:
    how: fmf
    adjust-tests:
      - check+:
          - how: avc
      - duration+: '*2'
        because: Slow system under test
        when: arch == i286
      - require~:
          - '/python3.11/python3.12/'

Configuration

See also Common Keys accepted by the plugin.

adjust-tests:

Modify metadata of discovered tests from the plan itself. Use the same format as for adjust rules.

Default: not set

dist-git-download-only: true|false

Just download the sources. No rpmbuild -bp, nor installation of require or buildddeps happens.

Default: false

dist-git-extract:

What to copy from extracted sources, globbing is supported. Defaults to the top fmf root if it is present, otherwise top directory (shortcut “/”).

Default: not set

dist-git-init: true|false

Set to true to initialize fmf root inside extracted sources at dist-git-extract location or top directory. To be used when the sources contain fmf files (for example tests) but do not have an associated fmf root.

Default: false

dist-git-install-builddeps: true|false

Install package build dependencies according to the specfile.

Default: false

dist-git-merge: true|false

Set to true to combine fmf root from the sources and fmf root from the plan. It allows to have plans and tests defined in the DistGit repo which use tests and other resources from the downloaded sources. Any plans in extracted sources will not be processed.

Default: false

dist-git-remove-fmf-root: true|false

Remove fmf root from extracted source (top one or selected by copy-path, happens before dist-git-extract.

Default: false

dist-git-require: PACKAGE

Additional required package to be present before sources are prepared. The rpm-build package itself is installed automatically.

Default: not set

dist-git-source: true|false

Download DistGit sources and rpmbuild -bp them (can be skipped).

Default: false

dist-git-type: fedora|centos|redhat

Use the provided DistGit handler instead of the auto detection. Useful when running from forked repositories.

Default: not set

exclude: REGEXP

Exclude tests matching given regular expression.

Default: not set

filter: FILTERS

Apply advanced filter based on test metadata attributes. See pydoc fmf.filter for more info.

Default: not set

fmf-id: true|false

Only print fmf identifiers of discovered tests to the standard output and exit.

Default: false

link: RELATION:TARGET

Select tests using the link keys. Values must be in the form of RELATION:TARGET, tests containing at least one of them are selected. Regular expressions are supported for both relation and target. Relation part can be omitted to match all relations.

Default: not set

modified-only: true|false

Set to true if you want to filter modified tests only. The test is modified if its name starts with the name of any directory modified since modified-ref.

Default: false

modified-ref: REVISION

The branch, tag or commit specifying the reference git revision (if not provided, the default branch is used). Note that you need to specify reference/<branch> to compare to a branch from the repository specified in modified-url.

Default: not set

modified-url: REPOSITORY

An additional remote repository to be used as the reference for comparison. Will be fetched as a reference remote in the test dir.

Default: not set

path: ROOT

Path to the metadata tree root. Must be relative to the git repository root if url was provided, absolute local filesystem path otherwise. By default . is used.

Default: not set

prune: true|false

Copy only immediate directories of executed tests and their required files.

Default: false

ref: REVISION

Branch, tag or commit specifying the desired git revision. Defaults to the remote repository’s default branch if url was set or to the current HEAD of the current repository.

Additionally, one can set ref dynamically. This is possible using a special file in tmt format stored in the default branch of a tests repository. This special file should contain rules assigning attribute ref in an adjust block, for example depending on a test run context.

Dynamic ref assignment is enabled whenever a test plan reference has the format ref: @FILEPATH.

Default: not set

repository:

Default: not set

revision:

Default: not set

sync-repo: true|false

Force the sync of the whole git repo. By default, the repo is copied only if the used options require it.

Default: false

test: NAMES

List of test names or regular expressions used to select tests by name. Duplicate test names are allowed to enable repetitive test execution, preserving the listed test order.

Default: not set

upgrade-path:

Default: not set

url: REPOSITORY

Git repository containing the metadata tree. Current git repository used by default.

Default: not set

shell

Warning

Please, be aware that the documentation below is a work in progress. We are working on fixing it, adding missing bits and generally making it better. Also, it was originally used for command line help only, therefore the formatting is often suboptimal.

Use provided list of shell script tests.

List of test cases to be executed can be defined manually directly in the plan as a list of dictionaries containing test name and actual test script. It is also possible to define here any other test metadata such as the duration or a path to the test. The default duration for tests defined directly in the discover step is 1h.

Example config:

discover:
    how: shell
    tests:
      - name: /help/main
        test: tmt --help
      - name: /help/test
        test: tmt test --help
      - name: /help/smoke
        test: ./smoke.sh
        path: /tests/shell

For DistGit repo one can download sources and use code from them in the tests. Sources are extracted into $TMT_SOURCE_DIR path, patches are applied by default. See options to install build dependencies or to just download sources without applying patches. To apply patches, special prepare phase with order 60 is added, and prepare step has to be enabled for it to run.

discover:
    how: shell
    dist-git-source: true
    tests:
      - name: /upstream
        test: cd $TMT_SOURCE_DIR/*/tests && make test

To clone a remote repository and use it as a source specify url. It accepts also ref to checkout provided reference. Dynamic reference feature is supported as well.

discover:
    how: shell
    url: https://github.com/teemtee/tmt.git
    ref: "1.18.0"
    tests:
      - name: first test
        test: ./script-from-the-repo.sh

Configuration

See also Common Keys accepted by the plugin.

dist-git-download-only: true|false

Just download the sources. No rpmbuild -bp, nor installation of require or buildddeps happens.

Default: false

dist-git-install-builddeps: true|false

Install package build dependencies according to the specfile.

Default: false

dist-git-require: PACKAGE

Additional required package to be present before sources are prepared. The rpm-build package itself is installed automatically.

Default: not set

dist-git-source: true|false

Download DistGit sources and rpmbuild -bp them (can be skipped).

Default: false

dist-git-type: fedora|centos|redhat

Use the provided DistGit handler instead of the auto detection. Useful when running from forked repositories.

Default: not set

keep-git-metadata: true|false

By default the .git directory is removed to save disk space. Set to true to sync the git metadata to guest as well. Implicit if dist-git-source is used.

Default: false

ref: REVISION

Branch, tag or commit specifying the desired git revision. Defaults to the remote repository’s default branch.

Default: not set

tests:

Default: not set

url: REPOSITORY

URL of the git repository with tests to be fetched.

Default: not set