Context
Definition of the environment context
As a user I want to define a context so that test, plan or story metadata can be adjusted to it accordingly.
This specification defines all standard context dimensions which can be used together with the adjust attribute to modify object metadata for particular context. For a detailed description of the concept itself see the fmf context documentation.
The context is usually defined from the command line using the
--context
option. All dimensions are optional. It is
possible to define your own dimensions to describe the
context, just make sure the dimension
name does not conflict with reserved names.
Some of the context dimensions can be detected (e.g. distro from the provision step config). Each plan can provide its own context which will override detected values. Finally, the context definition provided directly on the command line overrides detected and defined dimensions.
Examples:
tmt --context distro=fedora-33 run
tmt --context product=rhscl run
tmt --context trigger=code run
tmt -c distro=fedora test show
tmt -c distro=fedora plan show
Implemented by /tmt/base.py
dimension
Supported context dimensions
As a tester I want to clearly specify dimensions, such as ‘distro’ or ‘trigger’, so that I can well describe the context in which a test is running.
The following dimensions are reserved for storing dedicated information as described below:
- distro
The operating system distribution on which the application runs (fedora, fedora-33, centos, centos-8, centos-8.4, centos-stream, centos-stream-9, rhel, rhel-8, rhel-8.4).
- variant
The distribution variant (Client, Desktop, Server Workstation, Silverblue, CoreOS).
- arch
The guest architecture (aarch64, armhfp, i386, ppc64, ppc64le, s390x, x86_64).
- component
Name of the relevant component, should match the source package name (bash, php, httpd).
- collection
The Red Hat Software Collection name (python27, rh-python34, rh-mariadb100, httpd24).
- module
Module name with an optional stream specification (mariadb, mariadb:10.5, httpd, httpd:2.4, perl, perl:5.32, ruby, ruby:2.7).
- initiator
Name of the service, pipeline or tool which initiated the testing or special value
human
for manual testing. See the initiator section for details.- trigger
The event which triggered testing, see the trigger section for the full list of supported values.
Examples:
context:
distro: fedora-33
variant: Workstation
arch: x86_64
context:
product: rhscl
collection: httpd24
Status: implemented
Implemented by /tmt/base.py
initiator
Definition of the initiator dimension
As a user I want to run a custom set of tests based on the service, pipeline or tool which initiated the testing.
Sometimes it is useful to choose a custom set of tests for particular automation service or when tests are run manually. The following values are defined:
- human
Test execution was initiated manually by a human.
- packit
For jobs initiated by the Packit service.
- fedora-ci
Testing in the Fedora CI pipeline. To be implemented.
- centos-ci
Testing in the CentOS CI pipeline. To be implemented.
- rhel-ci
Testing in the RHEL CI pipeline. To be implemented.
- ewa
Errata Workflow Automation jobs. To be implemented.
Each service, pipeline or tool is responsible to set the value
to its name. For manual testing --context initiator=human
is expected on the command line.
Examples:
# Enable the plan only when run manually
summary: Tests requiring special environment
discover:
how: fmf
filter: tag:special
execute:
how: tmt
adjust:
enabled: false
when: initiator != human
Status: implemented
Implemented by /tmt/base.py
trigger
Definition of the trigger dimension
As a user I want to run a custom set of tests for different events which trigger the testing.
Testing can be triggered by various events and a different set
of tests is usually executed for each of them. The trigger
dimension supports the following values:
- commit
Project source code has changed. This can be either a pull/merge request or a new commit pushed to a branch.
- build
There has been a new package built in koji or brew and is available for testing.
- update
A new bodhi update or errata advisory with one or more builds has been created or updated.
- compose
There is a new compose ready for integration testing.
This context dimension will usually be provided from the command line.
Examples:
summary: Full test coverage (takes three days)
discover:
how: fmf
execute:
how: tmt
adjust:
enabled: false
when: trigger == code
Status: implemented
Implemented by /tmt/base.py