Core¶
I want to have common core attributes used consistently across all metadata levels.
adjust¶
Adjust metadata based on the given context
As a user I want to adjust test, plan or story metadata based on the given context such as product, distribution, architecture or trigger.
The adjust
attribute allows to modify arbitrary object
metadata based on the context in which they are to be used.
For example, tests may be relevant only for given environment,
the set of required packages may differ across distributions
or there might be a different set of plans executed for pull
requests versus for the full compose integration testing.
Note
The context currently has to be specified explicitly, see the Context documentation for details. In the future, tmt will detect (at least some of) the needed information from the environment.
The value can be a dictionary or a list of dictionaries which represent individual rules to be applied. Each rule contains metadata to be merged into the original object. The following three keys are reserved for rule evaluation:
- when
The condition to be evaluated in order to decide if the metadata should be merged. In the expression, you can use any defined context dimension. See the full condition syntax for details about supported operators. This is a required key.
- continue
By default, all provided rules are evaluated. When set to
false
, the first successful rule finishes the evaluation and the rest is ignored.- because
An optional comment with justification of the adjustment. Should be a plain string.
Note
This covers and extends the original concept of Test Case Relevancy which is now obsoleted.
Examples:
# Disable a test for older distros
enabled: true
adjust:
enabled: false
when: distro < fedora-33
because: the feature was added in Fedora 33
# Adjust the required package name
require: procps-ng
adjust:
- require: procps
when: distro == centos-6
# Extend the environment variables, use multiple rules
adjust:
- environment+:
SH: bash
when: component == bash
continue: true
- when: distro < centos-6
enabled: false
# Install the fresh pytest from pip on older distros
adjust:
prepare+:
- how: shell
name: fresh-pytest
order: 90
script: 'python3 -m pip install -U pytest'
when: distro < rhel-8
Status: implemented and verified
description¶
Detailed description of the object
Multiline string
describing all important aspects of the object. Usually spans across several paragraphs. For detailed examples using a dedicated attributes ‘examples’ should be considered.
Status: implemented
enabled¶
Allow disabling individual tests, plans or stories
As a developer or tester I want selected tests or plans to be skipped during test execution.
When a test or a plan is broken or it is not relevant for given Context it can be disabled so that it’s skipped during the execution. For stories, this attribute might be used to mark stories which should be skipped when generating the documentation.
Should be a boolean
. The default value is true
.
Examples:
# Mark as disabled
enabled: false
# Disable for older distros
enabled: true
adjust:
enabled: false
when: distro < fedora-33
because: the feature was added in Fedora 33
# List only enabled tests
tmt tests ls --filter enabled:true
Status: implemented and verified
link¶
Link related objects
As a user I want to link related tests, stories or external issues so that I can easily find out what the test, plan or story is addressing.
The core attribute link
is used to track relevant objects
such as user stories or external issues verified by the given
test, source code implementing particular story, blocking
issues or parent-child relations.
The value can be a single link or a list of links. There are
two options how to specify the target
of the reference:
- string
This can be either a
name
of the fmf node within the same metadata tree, a filepath
from the tree root to the application or documentation source code or aURL
to an external issue tracker.- dict
In order to reference remote fmf objects use a dictionary with the full fmf identifier.
By default, the link between the two objects is interpreted in
a generic way as relates
. To explicitly define a different
connection to the target reference, use a dictionary and set
the key to one of the supported relations:
- verifies
This object (a test or a plan) verifies functionality defined in the target (a story, an issue, a bug).
- verified-by
The functionality (described in a story, an issue or a bug) is verified by the target (a test or a plan).
- implements
This object (source code) implements functionality defined in the target (a story describing the feature).
- implemented-by
The functionality (described in a story) is implemented by the target (source code).
- documents
This object (a guide or examples) documents the target (a story).
- documented-by
The functionality (described in a story) is documented by the target (a guide or examples).
- blocks
This object is blocking the target.
- blocked-by
This object is blocked by the target.
- duplicates
This object is a duplicate of the target.
- duplicated-by
This object is duplicated by the target.
- parent
This object is a child of the target parent.
- child
This object is a parent of the target child.
- relates
This object relates to the provided target. This is the default relation.
An optional key note
can be used to add an arbitrary
comment describing the relation.
Examples:
# A related issue
link: https://github.com/teemtee/tmt/issues/461
# A test verifying a story
link:
verifies: /stories/cli/init/base
# A test verifying a story and a bug
link:
- verifies: /stories/cli/init/base
- verifies: https://bugzilla.redhat.com/show_bug.cgi?id=1234
# An implemented story covered by both tests and docs
link:
- implemented-by: /tmt/cli.py
- verified-by: /tests/init/base
- documented-by: /docs/guide.rst
# A story blocked by a remote story with optional note
link:
blocked-by:
url: https://github.com/teemtee/fmf
name: /stories/select/filter/regexp
note: Need to get the regexp filter working first.
Status: implemented
order¶
Order in which the object should be handled
Sometimes it is important in which order objects are processed. For example when running tests we need to execute some tests first or when exporting stories we need to arrange content in the right sequence to create a meaningful documentation. Should be an integer
, negative values are allowed as well. The default value is 50
.
Examples:
order
Status: implemented and verified
summary¶
Concise summary describing purpose of the object
Should be a one-line string
with up to 50 characters. It is challenging to be both concise and descriptive, but that is what a well-written summary should do.
Status: implemented