Classes

Class Structure

Here’s the overview of the essential classes used in the tmt project. It should help you to get quickly started and better understand the relation between the individual classes.

Basic

The Common class is the parent of most of the available classes, it provides common methods for logging, running commands and workdir handling. The Core class together with its child classes Test, Plan and Story cover the Metadata Specification:

Common
├── Core
│   ├── Plan
│   ├── Story
│   └── Test
├── Clean
├── Guest
├── Phase
├── Run
├── Status
├── Step
└── Tree

Phases

Actions performed during a normal step and plugins for individual step:

Phase
├── Action
│   ├── Login
│   └── Reboot
└── BasePlugin
    ├── GuestlessPlugin
    │   ├── DiscoverPlugin
    │   │   ├── DiscoverFmf
    │   │   └── DiscoverShell
    │   ├── ProvisionPlugin
    │   │   ├── ProvisionArtemis
    │   │   ├── ProvisionConnect
    │   │   ├── ProvisionLocal
    │   │   ├── ProvisionPodman
    │   │   └── ProvisionTestcloud
    │   └── ReportPlugin
    │       ├── ReportDisplay
    │       ├── ReportHtml
    │       ├── ReportJUnit
    │       └── ReportPolarion
    └── Plugin
        ├── ExecutePlugin
        │   └── ExecuteInternal
        │       └── ExecuteUpgrade
        ├── FinishPlugin
        │   ├── FinishAnsible
        │   └── FinishShell
        └── PreparePlugin
            ├── PrepareAnsible
            │   └── FinishAnsible
            ├── PrepareInstall
            ├── PrepareMultihost
            └── PrepareShell

Steps

A brief overview of all test steps:

Step
├── Discover
├── Provision
├── Prepare
├── Execute
├── Finish
└── Report

Containers used for storing configuration data for individual step plugins:

DataContainer
└── SpecBasedContainer, SerializableContainer
    ├── FmfId
    │   └── RequireFmfId
    ├── Link
    ├── Links
    └── StepData
        ├── DiscoverStepData
        │   ├── DiscoverFmfStepData
        │   └── DiscoverShellData
        ├── ExecuteStepData
        │   ├── ExecuteInternalData
        │   └── ExecuteUpgradeData
        ├── FinishStepData
        │   └── FinishShellData
        ├── PrepareStepData
        │   ├── PrepareAnsibleData
        │   ├── PrepareInstallData
        │   ├── PrepareMultihostData
        │   └── PrepareShellData
        ├── ProvisionStepData
        │   ├── ProvisionArtemisData
        │   ├── ProvisionConnectData
        │   ├── ProvisionLocalData
        │   ├── ProvisionPodmanData
        │   └── ProvisionTestcloudData
        └── ReportStepData
            ├── ReportHtmlData
            ├── ReportJUnitData
            └── ReportPolarionData

Guests

Guests provisioned for test execution:

Guest
├── GuestContainer
├── GuestLocal
└── GuestSsh
    ├── GuestArtemis
    └── GuestTestcloud

Data related to provisioned guests:

GuestData
├── GuestSshData
│   ├── ArtemisGuestData
│   ├── ConnectGuestData
│   └── TestcloudGuestData
└── PodmanGuestData

Attributes

Object hierarchy is following: Run -> Plans -> Steps -> Plugins, where the Run is on the top of this hierarchy. The objects have the parent attribute, that is pointing to the parent in which the current instance is contained.

The node attribute of Test, Plan and Story instances references the original leaf node of the fmf metadata tree from which the respective test, plan or story have been created.

In a similar way, the tree property of the Tree instance points to the original fmf.Tree from which it was initialized.

Class Conversions

Various internal objects and classes often need to be converted from their Python nature to data that can be saved, loaded or exported in different form. To facilitate these conversions, three families of helper methods are provided, each with its own set of use cases.

to_spec/to_minimal_spec/from_spec

This family of methods works with tmt specification, i.e. raw user-provided data coming from fmf files describing plans, tests, stories, or from command-line options. from_spec() shall be called to spawn objects representing the user input, while to_spec() should produce output one could find in fmf files.

The default implementation comes from tmt.utils.SpecBasedContainer class, all classes based on user input data should include this class among their bases.

to_minimal_spec performs the identical operation as to_spec, but its result should not include keys that are optional and not set, while to_spec should always include all keys, even when set to default values or not set at all.

# Create an fmf id object from raw data
fmf_id = tmt.base.FmfId.from_spec({'url': ..., 'ref': ...})

to_serialized/from_serialized/unserialize

This family of methods is aiming at runtime objects that may be saved into and loaded from tmt working files, i.e. files tmt uses to store a state in its workdir, like step.yaml or guests.yaml.

Third member of this family, unserialize, is similar to from_serialized - both create an object from its serialized form, only unserialize is capable of detecting the class to instantiate while for using from_serialized, one must already know which class to work with. unserialize then uses from_serialized under the hood to do the heavy lifting when correct class is identified.

The default implementation comes from tmt.utils.SerializableContainer class, all classes that are being saved and loaded during tmt run should include this class among their bases.

See https://en.wikipedia.org/wiki/Serialization for more details on the concept of serialization.

 # tmt.steps.discover.shell.DiscoverShellData wishes to unserialize its
 # `tests` a list of `TestDescription` objects rather than a list of
 # dictionaries (the default implementation).
 @classmethod
 def from_serialized(cls, serialized: Dict[str, Any]) -> 'DiscoverShellData':
     obj = super().from_serialized(serialized)

     obj.tests = [TestDescription.from_serialized(
         serialized_test) for serialized_test in serialized['tests']]

     return obj

# A step saving its state...
content: Dict[str, Any] = {
    'status': self.status(),
    'data': [datum.to_serialized() for datum in self.data]
}
self.write('step.yaml', tmt.utils.dict_to_yaml(content))

# ... and loading it back.
# Note the use of unserialize(): step data may have been serialized from
# various different classes (derived from tmt.steps.provision.Guest),
# and unserialize() will detect the correct class.
raw_step_data: Dict[Any, Any] = tmt.utils.yaml_to_dict(self.read('step.yaml'))
self.data = [
    StepData.unserialize(raw_datum) for raw_datum in raw_step_data['data']
]

to_dict/to_minimal_dict

Very special helper methods: its use cases are not related to any input or output data, and most of the time, when in need of iterating over object’s keys and/or values, one can use keys(), values() or items() methods. They are used as sources of data for serialization and validation, but they usually have no use outside of default implementations.

Warning

If you think of using to_dict(), please, think again and be sure you know what are you doing. Despite its output being sometimes perfectly compatible with output of to_serialized() or to_spec(), it is not generaly true, and using it instead of proper methods may lead to unexpected exceptions.

The same applies to to_minimal_dict().

# tmt.base.FmfId's specification is basically just a mapping,
# therefore `to_dict()` is good enough to produce a specification.
def to_spec(self) -> Dict[str, Any]:
     return self.to_dict()

Essential Classes

Test Management Tool

class tmt.Clean(parent: Optional[Common] = None, name: Optional[str] = None, workdir: Optional[Union[typing_extensions.Literal[True], str]] = None, context: Optional[Context] = None)

A class for cleaning up workdirs, guests or images

guests() bool

Clean guests of runs

images() bool

Clean images of provision plugins

runs() bool

Clean workdirs of runs

class tmt.Guest(data: GuestData, name: Optional[str] = None, parent: Optional[Common] = None)

Guest provisioned for test execution

A base class for guest-like classes. Provides some of the basic methods and functionality, but note some of the methods are left intentionally empty. These do not have valid implementation on this level, and it’s up to Guest subclasses to provide one working in their respective infrastructure.

The following keys are expected in the ‘data’ container:

role ....... guest role in the multihost scenario
guest ...... name, hostname or ip address

These are by default imported into instance attributes.

ansible(playbook: str, extra_args: Optional[str] = None) None

Prepare guest using ansible playbook

details() None

Show guest details such as distro and kernel

execute(command: Union[str, List[str]], test_session: bool = False, **kwargs: Any) CommandOutput

Execute command on the guest

command … string or list of command arguments (required) env ……. dictionary with environment variables cwd ……. working directory to be entered before execution

If the command is provided as a list, it will be space-joined. If necessary, quote escaping has to be handled by the caller.

guest: Optional[str]
load(data: GuestData) None

Load guest data into object attributes for easy access

Called during guest object initialization. Takes care of storing all supported keys (see class attribute _keys for the list) from provided data to the guest object attributes. Child classes can extend it to make additional guest attributes easily available.

Data dictionary can contain guest information from both command line options / L2 metadata / user configuration and wake up data stored by the save() method below.

localhost = False
pull(source: Optional[str] = None, destination: Optional[str] = None, options: Optional[List[str]] = None, extend_options: Optional[List[str]] = None) None

Pull files from the guest

push(source: Optional[str] = None, destination: Optional[str] = None, options: Optional[List[str]] = None) None

Push files to the guest

reboot(hard: bool = False, command: Optional[str] = None, timeout: Optional[int] = None) bool

Reboot the guest, return True if successful

Parameter ‘hard’ set to True means that guest should be rebooted by way which is not clean in sense that data can be lost. When set to False reboot should be done gracefully.

Use the ‘command’ parameter to specify a custom reboot command instead of the default ‘reboot’.

Parameter ‘timeout’ can be used to specify time (in seconds) to wait for the guest to come back up after rebooting.

reconnect(timeout: Optional[int] = None, tick: float = 5, tick_increase: float = 1.0) bool

Ensure the connection to the guest is working

The default timeout is 5 minutes. Custom number of seconds can be provided in the timeout parameter. This may be useful when long operations (such as system upgrade) are performed.

remove() None

Remove the guest

Completely remove all guest instance data so that it does not consume any disk resources.

classmethod requires() List[str]

No extra requires needed

role: Optional[str]
save() GuestData

Save guest data for future wake up

Export all essential guest data into a dictionary which will be stored in the guests.yaml file for possible future wake up of the guest. Everything needed to attach to a running instance should be added into the data dictionary by child classes.

start() None

Start the guest

Get a new guest instance running. This should include preparing any configuration necessary to get it started. Called after load() is completed so all guest data should be available.

stop() None

Stop the guest

Shut down a running guest instance so that it does not consume any memory or cpu resources. If needed, perform any actions necessary to store the instance status to disk.

wake() None

Wake up the guest

Perform any actions necessary after step wake up to be able to attach to a running guest instance and execute commands. Called after load() is completed so all guest data should be prepared.

class tmt.GuestSsh(data: GuestData, name: Optional[str] = None, parent: Optional[Common] = None)

Guest provisioned for test execution, capable of accepting SSH connections

The following keys are expected in the ‘data’ dictionary:

role ....... guest role in the multihost scenario (inherited)
guest ...... hostname or ip address (inherited)
port ....... port to connect to
user ....... user name to log in
key ........ path to the private key (str or list)
password ... password

These are by default imported into instance attributes.

ansible(playbook: str, extra_args: Optional[str] = None) None

Prepare guest using ansible playbook

execute(command: Union[str, List[str]], test_session: bool = False, **kwargs: Any) CommandOutput

Execute command on the guest

command … string or list of command arguments (required) env ……. dictionary with environment variables cwd ……. working directory to be entered before execution

If the command is provided as a list, it will be space-joined. If necessary, quote escaping has to be handled by the caller.

key: List[str]
password: Optional[str]
port: Optional[int]
pull(source: Optional[str] = None, destination: Optional[str] = None, options: Optional[List[str]] = None, extend_options: Optional[List[str]] = None) None

Pull files from the guest

By default the whole plan workdir is synced from the same location on the guest. Use the ‘source’ and ‘destination’ to sync custom location, the ‘options’ parameter to modify default options DEFAULT_RSYNC_PULL_OPTIONS and ‘extend_options’ to extend them (e.g. by exclude).

push(source: Optional[str] = None, destination: Optional[str] = None, options: Optional[List[str]] = None) None

Push files to the guest

By default the whole plan workdir is synced to the same location on the guest. Use the ‘source’ and ‘destination’ to sync custom location and the ‘options’ parametr to modify default options which are ‘-Rrz –links –safe-links –delete’.

reboot(hard: bool = False, command: Optional[str] = None, timeout: Optional[int] = None, tick: float = 30.0, tick_increase: float = 1.0) bool

Reboot the guest, return True if reconnect was successful

Parameter ‘hard’ set to True means that guest should be rebooted by way which is not clean in sense that data can be lost. When set to False reboot should be done gracefully.

Use the ‘command’ parameter to specify a custom reboot command instead of the default ‘reboot’.

remove() None

Remove the guest

Completely remove all guest instance data so that it does not consume any disk resources.

classmethod requires() List[str]

No extra requires needed

stop() None

Stop the guest

Shut down a running guest instance so that it does not consume any memory or cpu resources. If needed, perform any actions necessary to store the instance status to disk.

user: Optional[str]
class tmt.Plan(*, node: Tree, run: Optional[Run] = None, skip_validation: bool = False, raise_on_validation_error: bool = False, **kwargs: Any)

Plan object (L2 Metadata)

context: Dict[str, List[str]] = {}
static create(name: str, template: str, path: str, force: bool = False, dry: Optional[bool] = None) None

Create a new plan

static edit_template(raw_content: str) str

Edit the default template with custom values

property environment: Dict[str, str]

Return combined environment from plan data and command line

export(*, format_: typing_extensions.Literal[<ExportFormat.DICT: 'dict'>], keys: ~typing.Optional[~typing.List[str]] = None) Dict[str, Any]
export(*, format_: typing_extensions.Literal[<ExportFormat.YAML: 'yaml'>], keys: ~typing.Optional[~typing.List[str]] = None) str
export(*, format_: typing_extensions.Literal[<ExportFormat.RST: 'rst'>], keys: ~typing.Optional[~typing.List[str]] = None) str
export(*, format_: typing_extensions.Literal[<ExportFormat.NITRATE: 'nitrate'>], keys: ~typing.Optional[~typing.List[str]] = None) None
export(*, format_: typing_extensions.Literal[<ExportFormat.POLARION: 'polarion'>], keys: ~typing.Optional[~typing.List[str]] = None) None

Export plan data into requested format

Supported formats are ‘yaml’ and ‘dict’.

extra_L2_keys = ['context', 'environment', 'environment-file', 'gate']
gate: List[str] = []
go() None

Execute the plan

import_plan() Optional[Plan]

Import plan from a remote repository, return a Plan instance

property is_remote_plan_reference: bool

Check whether the plan is a remote plan reference

lint() bool

Check plan against the L2 metadata specification

Return whether the plan is valid.

static overview(tree: Tree) None

Show overview of available plans

show() None

Show plan details

step_names(enabled: bool = True, disabled: bool = False, skip: Optional[List[str]] = None) Generator[str, None, None]

Iterate over enabled / all step names.

Yields step names of all enabled steps by default. Use ‘disabled=True’ to iterate over all. Use ‘skip’ to pass the list of steps to be skipped.

steps(enabled: bool = True, disabled: bool = False, skip: Optional[List[str]] = None) Generator[Step, None, None]

Iterate over enabled / all steps

Yields instances of all enabled steps by default. Use ‘disabled=True’ to iterate over all. Use ‘skip’ to pass the list of steps to be skipped.

class tmt.Result(data: ResultData, name: Optional[str] = None, test: Optional[Test] = None)

Test result

Required parameter ‘data’ needs to be type of ResultData. Required parameter ‘test’ or ‘name’ should contain a test reference.

duration: Optional[str] = None
static failures(log: Optional[str], msg_type: str = 'FAIL') str

Filter stdout and get only messages with certain type

classmethod from_serialized(serialized: Dict[str, Any]) Result

Convert from a serialized form loaded from a file.

See https://tmt.readthedocs.io/en/stable/classes.html#class-conversions for more details.

See to_serialized() for its counterpart.

ids: Dict[str, Optional[str]]
log: Union[List[Any], Dict[Any, Any]]
name: str
note: Optional[str] = None
result: ResultOutcome
show() str

Return a nicely colored result with test name (and note)

static summary(results: List[Result]) str

Prepare a nice human summary of provided results

to_serialized() Dict[str, Any]

Convert to a form suitable for saving in a file.

See https://tmt.readthedocs.io/en/stable/classes.html#class-conversions for more details.

See from_serialized() for its counterpart.

static total(results: List[Result]) Dict[ResultOutcome, int]

Return dictionary with total stats for given results

class tmt.Run(id_: Optional[str] = None, tree: Optional[Tree] = None, context: Optional[Context] = None)

Test run, a container of plans

property environment: Dict[str, str]

Return environment combined from wake up and command line

finish() None

Check overall results, return appropriate exit code

follow() None

Periodically check for new lines in the log.

go() None

Go and do test steps for selected plans

load() None

Load list of selected plans and enabled steps

load_from_workdir() None

Load the run from its workdir, do not require the root in run.yaml to exist. Doest not load the fmf tree.

Use only when the data in workdir is sufficient (e.g. tmt clean and status only require the steps to be loaded and their status).

property plans: List[Plan]

Test plans for execution

save() None

Save list of selected plans and enabled steps

tree: Optional[Tree]
class tmt.Status(parent: Optional[CommonDerivedType] = None, name: Optional[str] = None, workdir: Optional[Union[typing_extensions.Literal[True], str]] = None, context: Optional[Context] = None, relative_indent: int = 1, **kwargs: Any)

Status of tmt work directories.

FIRST_COL_LEN = 11
LONGEST_STEP = 'provision'
static colorize_column(content: str) str

Add color to a status column

static get_overall_plan_status(plan: Plan) str

Examines the plan status (find the last done step)

classmethod pad_with_spaces(string: str) str

Append spaces to string to properly align the first column

plan_matches_filters(plan: Plan) bool

Check if the given plan matches filters from the command line

print_header() None

Print the header of the status table based on verbosity

print_plans_status(run: Run) None

Display the status of each plan of the given run

print_run_status(run: Run) None

Display the overall status of the run

print_verbose_status(run: Run) None

Display the status of each step of the given run

process_run(run: Run) None

Display the status of the given run based on verbosity

run_matches_filters(run: Run) bool

Check if the given run matches filters from the command line

show() None

Display the current status

class tmt.Story(*, node: Tree, skip_validation: bool = False, raise_on_validation_error: bool = False, **kwargs: Any)

User story object

KEYS_SHOW_ORDER: List[str] = ['summary', 'title', 'story', 'id', 'priority', 'description', 'example', 'enabled', 'order', 'tag', 'tier', 'link']
coverage(code: bool, test: bool, docs: bool) Tuple[bool, bool, bool]

Show story coverage

static create(name: str, template: str, path: str, force: bool = False, dry: Optional[bool] = None) None

Create a new story

property documented: List[Link]

Return links to relevant documentation

example: List[str] = []
export(*, format_: typing_extensions.Literal[<ExportFormat.RST: 'rst'>, <ExportFormat.YAML: 'yaml'>], include_title: bool = True) str
export(*, format_: typing_extensions.Literal[<ExportFormat.DICT: 'dict'>], include_title: bool = True) Dict[str, Any]
export(*, format_: typing_extensions.Literal[<ExportFormat.NITRATE: 'nitrate'>, <ExportFormat.POLARION: 'polarion'>], include_title: bool = True) None

Export story data into requested format

property implemented: List[Link]

Return links to relevant source code

lint() bool

Check story against the L3 metadata specification.

Return whether the story is valid.

static overview(tree: Tree) None

Show overview of available stories

priority: Optional[StoryPriority] = None
show() None

Show story details

story: str
title: Optional[str] = None
property verified: List[Link]

Return links to relevant test coverage

class tmt.Test(*, node: Tree, skip_validation: bool = False, raise_on_validation_error: bool = False, **kwargs: Any)

Test object (L1 Metadata)

KEYS_SHOW_ORDER: List[str] = ['summary', 'description', 'contact', 'component', 'id', 'test', 'path', 'framework', 'manual', 'require', 'recommend', 'environment', 'duration', 'enabled', 'order', 'result', 'tag', 'tier', 'link']
component: List[str] = []
contact: List[str] = []
static create(name: str, template: str, path: str, force: bool = False, dry: Optional[bool] = None) None

Create a new test

duration: str = '5m'
environment: Dict[str, str] = {}
export(*, format_: typing_extensions.Literal[<ExportFormat.DICT: 'dict'>], keys: ~typing.Optional[~typing.List[str]] = None) Dict[str, Any]
export(*, format_: typing_extensions.Literal[<ExportFormat.YAML: 'yaml'>], keys: ~typing.Optional[~typing.List[str]] = None) str
export(*, format_: typing_extensions.Literal[<ExportFormat.RST: 'rst'>], keys: ~typing.Optional[~typing.List[str]] = None) str
export(*, format_: typing_extensions.Literal[<ExportFormat.NITRATE: 'nitrate'>], keys: ~typing.Optional[~typing.List[str]] = None) None
export(*, format_: typing_extensions.Literal[<ExportFormat.POLARION: 'polarion'>], keys: ~typing.Optional[~typing.List[str]] = None) None

Export test data into requested format

In addition to ‘yaml’ and ‘dict’ it supports also a special format ‘execute’ used by the execute step which returns (test-name, test-data) tuples.

framework: str = 'shell'
classmethod from_dict(mapping: Dict[str, Any], name: str, skip_validation: bool = False, raise_on_validation_error: bool = False, **kwargs: Any) Test

Initialize test data from a dictionary.

Useful when data describing a test are stored in a mapping instead of an fmf node.

lint() bool

Check test against the L1 metadata specification.

Return whether the test is valid.

manual: bool = False
static overview(tree: Tree) None

Show overview of available tests

path: Optional[str] = None
real_duration: Optional[str] = None
recommend: List[Union[RequireSimple, RequireFmfId]] = []
require: List[Union[RequireSimple, RequireFmfId]] = []
result: str = 'respect'
returncode: Optional[int] = None
show() None

Show test details

test: str
class tmt.Tree(path: str = '.', tree: Optional[Tree] = None, context: Optional[Dict[str, List[str]]] = None)

Test Metadata Tree

static init(path: str, template: str, force: bool) None

Initialize a new tmt tree, optionally with a template

plans(keys: Optional[List[str]] = None, names: Optional[List[str]] = None, filters: Optional[List[str]] = None, conditions: Optional[List[str]] = None, run: Optional[Run] = None, links: Optional[List[LinkNeedle]] = None, excludes: Optional[List[str]] = None) List[Plan]

Search available plans

property root: Optional[str]

Metadata root

stories(keys: Optional[List[str]] = None, names: Optional[List[str]] = None, filters: Optional[List[str]] = None, conditions: Optional[List[str]] = None, whole: bool = False, links: Optional[List[LinkNeedle]] = None, excludes: Optional[List[str]] = None) List[Story]

Search available stories

tests(keys: Optional[List[str]] = None, names: Optional[List[str]] = None, filters: Optional[List[str]] = None, conditions: Optional[List[str]] = None, unique: bool = True, links: Optional[List[LinkNeedle]] = None, excludes: Optional[List[str]] = None) List[Test]

Search available tests

property tree: Tree

Initialize tree only when accessed