Test checks Plugins

Common Keys

The following keys are accepted by all plugins of the test-checks step.

enabled

Whether the check is enabled or not.

Default: true

Environment variable: TMT_PLUGIN_TEST-CHECKS_AVC_ENABLED

In plan metadata:

enabled: true|false

On command-line:

--enabled
export TMT_PLUGIN_TEST-CHECKS_AVC_ENABLED=1|0

result

How to interpret the check result.

Default: respect

Environment variable: TMT_PLUGIN_TEST-CHECKS_AVC_RESULT

In plan metadata:

result: info|respect|xfail

On command-line:

--result 'info|respect|xfail'
export TMT_PLUGIN_TEST-CHECKS_AVC_RESULT='info|respect|xfail'

avc

Check for SELinux AVC denials raised during the test.

The check collects SELinux AVC denials from the audit log, gathers details about them, and together with versions of the selinux-policy and related packages stores them in a report file after the test.

check:
  - name: avc

Note

To work correctly, the check requires SELinux to be enabled on the guest, and auditd must be running. Without SELinux, the check will turn into no-op, reporting skip result, and without auditd, the check will discover no AVC denials, reporting pass.

If the test manipulates auditd or SELinux in general, the check may report unexpected results.

Added in version 1.28.

Configuration

See also Common Keys accepted by the plugin.

delay-before-report

How many seconds to wait before running ausearch after the test. Increasing it may help when events do reach logs fast enough for ausearch report them.

Default: 5

Environment variable: TMT_PLUGIN_TEST-CHECKS_AVC_DELAY_BEFORE_REPORT

In plan metadata:

delay-before-report: SECONDS

On command-line:

--delay-before-report SECONDS
export TMT_PLUGIN_TEST-CHECKS_AVC_DELAY_BEFORE_REPORT=SECONDS

test-method

Which method to use when calling ausearch to report new AVC denials. With checkpoint, native --checkpoint option of ausearch is used, while timestamp will depend on --ts option and a date/time recorded before the test.

Default: timestamp

Environment variable: TMT_PLUGIN_TEST-CHECKS_AVC_TEST_METHOD

In plan metadata:

test-method: timestamp|checkpoint

On command-line:

--test-method 'timestamp|checkpoint'
export TMT_PLUGIN_TEST-CHECKS_AVC_TEST_METHOD='timestamp|checkpoint'

coredump

Warning

Please, be aware that the documentation below is a work in progress. We are working on fixing it, adding missing bits and generally making it better. Also, it was originally used for command line help only, therefore the formatting is often suboptimal.

Check for system crashes using coredump.

The check monitors for any crashes caught by systemd-coredump during test execution. This includes segmentation faults and other crashes that produce core dumps. By default, any crash will cause the test to fail.

Example config with optional ignore patterns:

check:
  - how: coredump
    ignore-pattern:
      - 'Process.*\(sleep\).*dumped core'  # Ignore sleep crashes
      - 'Package: ddcutil/2.1.2-2.fc41'      # Ignore dumps of a specific package

The patterns are matched against the full coredumpctl info output, which includes fields like Process, Command Line, Signal, etc. You can use ‘coredumpctl info’ to see the available fields and their format.

Common pattern examples:

Ignore all crashes from a specific process: ‘Process.*(specific-process)’
Ignore crashes with a specific signal: ‘Signal: .*(SIGSEGV)’
Ignore crashes from a specific package: ‘Package: package-name/.*’
Ignore crashes during a specific command: ‘Command Line: .*specific-command-pattern.*’

Added in version 1.46.

Configuration

See also Common Keys accepted by the plugin.

ignore-pattern

Optional list of regular expressions to ignore in crash reports. If a crash report matches any of these patterns, it will be ignored and not cause a failure. Any other crashes will still cause the test to fail. If no patterns are specified, any crash will cause a failure.

Default: not set

Environment variable: TMT_PLUGIN_TEST-CHECKS_COREDUMP_IGNORE_PATTERN

In plan metadata:

ignore-pattern: PATTERN

On command-line:

--ignore-pattern PATTERN
export TMT_PLUGIN_TEST-CHECKS_COREDUMP_IGNORE_PATTERN=PATTERN

is-availability-reason

Default: not set

Environment variable: TMT_PLUGIN_TEST-CHECKS_COREDUMP_IS_AVAILABILITY_REASON

In plan metadata:
is-availability-reason:
On command-line:
--is-availability-reason ...
export TMT_PLUGIN_TEST-CHECKS_COREDUMP_IS_AVAILABILITY_REASON=...

is-available

Default: not set

Environment variable: TMT_PLUGIN_TEST-CHECKS_COREDUMP_IS_AVAILABLE

In plan metadata:
is-available:
On command-line:
--is-available ...
export TMT_PLUGIN_TEST-CHECKS_COREDUMP_IS_AVAILABLE=...

dmesg

Save the content of kernel ring buffer (aka “console”) into a file.

The check saves one file before the test, and then again when test finishes.

check:
  - how: dmesg

Check will identify patterns that signal kernel crashes and core dumps, and when detected, it will report as failed result. It is possible to define custom patterns:

check:
  - how: dmesg
    failure-pattern:
      # These are default patterns
      - 'Call Trace:
      - '\ssegfault\s'

      # More patterns to look for
      - '\[Firmware Bug\]'

Added in version 1.28.

Changed in version 1.33: failure-pattern has been added.

Configuration

See also Common Keys accepted by the plugin.

failure-pattern

List of regular expressions to look for in dmesg output. If any of patterns is found, dmesg check will report fail result.

Default: Call Trace:, \ssegfault\s Environment variable: TMT_PLUGIN_TEST-CHECKS_DMESG_FAILURE_PATTERN

In plan metadata:

failure-pattern: PATTERN

On command-line:

--failure-pattern PATTERN
export TMT_PLUGIN_TEST-CHECKS_DMESG_FAILURE_PATTERN=PATTERN

internal/abort

Warning

Please, be aware that the documentation below is a work in progress. We are working on fixing it, adding missing bits and generally making it better. Also, it was originally used for command line help only, therefore the formatting is often suboptimal.

Check for test aborts during execution.

This check fails when tests are aborted before completion.

Added in version 1.50.

internal/interrupt

Warning

Please, be aware that the documentation below is a work in progress. We are working on fixing it, adding missing bits and generally making it better. Also, it was originally used for command line help only, therefore the formatting is often suboptimal.

Check for signal interruptions during test execution.

This check fails when tests are interrupted by SIGINT or SIGTERM signals.

Added in version 1.50.

internal/invocation

Warning

Please, be aware that the documentation below is a work in progress. We are working on fixing it, adding missing bits and generally making it better. Also, it was originally used for command line help only, therefore the formatting is often suboptimal.

Check for uncategorized invocation errors during test execution.

This check fails when tests encounter errors that are not covered by more specific checks.

Added in version 1.50.

internal/permission

Warning

Please, be aware that the documentation below is a work in progress. We are working on fixing it, adding missing bits and generally making it better. Also, it was originally used for command line help only, therefore the formatting is often suboptimal.

Check for permission issues during execution.

This check fails when tests encounter permission-related errors.

Added in version 1.50.

internal/timeout

Warning

Please, be aware that the documentation below is a work in progress. We are working on fixing it, adding missing bits and generally making it better. Also, it was originally used for command line help only, therefore the formatting is often suboptimal.

Check for test timeouts during execution.

This check fails when tests exceed their maximum allowed duration.

Added in version 1.50.

watchdog

Take various actions when guest becomes unresponsive.

Watchdog runs selected probes every now and then, and when a given number of probes fail, watchdog would run one or more of the predefined actions.

Check comes with two probes, “ping” and “SSH ping”, and single action, “reboot”.

“ping” uses the classic ICMP echo to check whether the guest is still up and running,
“SSH ping” tries to establish SSH connection,
“reboot” action issues a hard reboot of the guest.

Warning

Be aware that this feature may be limited depending on how the guest was provisioned. See Hard reboot.

Each probe has a “budget” of allowed failures, and when it runs out, the action is taken. A successful probe replenishes its budget to the original level.

Multiple probes can be enabled at the same time, for the action to happen it’s enough if just one of them runs out of its budget.

check:
  - how: watchdog
    ping: true
    reboot: true

check:
  - how: watchdog

    # Use only SSH ping.
    ping: false
    ssh-ping: true

    # Try every 5 minutes, allow 7 failed attempts, and reboot
    # the guest when we run out of attempts.
    interval: 300
    reboot: true
    ssh-ping-threshold: 7

Added in version 1.32.

Configuration

See also Common Keys accepted by the plugin.

interval

How often should the watchdog run.

Default: 60

Environment variable: TMT_PLUGIN_TEST-CHECKS_WATCHDOG_INTERVAL

In plan metadata:

interval: SECONDS

On command-line:

--interval SECONDS
export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_INTERVAL=SECONDS

ping

If enabled, watchdog would probe guest’s responsiveness with ICMP packets.

Default: false

Environment variable: TMT_PLUGIN_TEST-CHECKS_WATCHDOG_PING

In plan metadata:

ping: true|false

On command-line:

--ping
export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_PING=1|0

ping-packets

How many ICMP packates to send as one probe.

Default: 1

Environment variable: TMT_PLUGIN_TEST-CHECKS_WATCHDOG_PING_PACKETS

In plan metadata:

ping-packets: N

On command-line:

--ping-packets N
export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_PING_PACKETS=N

ping-threshold

How many failed ping probes before taking any further action.

Default: 10

Environment variable: TMT_PLUGIN_TEST-CHECKS_WATCHDOG_PING_THRESHOLD

In plan metadata:

ping-threshold: N

On command-line:

--ping-threshold N
export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_PING_THRESHOLD=N

reboot

If enabled, watchdog would reboot the guest after enough failed probes.

Default: false

Environment variable: TMT_PLUGIN_TEST-CHECKS_WATCHDOG_REBOOT

In plan metadata:

reboot: true|false

On command-line:

--reboot
export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_REBOOT=1|0

ssh-ping

If enabled, watchdog would probe guest’s responsiveness by connecting to its SSH port.

Default: false

Environment variable: TMT_PLUGIN_TEST-CHECKS_WATCHDOG_SSH_PING

In plan metadata:

ssh-ping: true|false

On command-line:

--ssh-ping
export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_SSH_PING=1|0

ssh-ping-threshold

How many failed SSH connections before taking any further action.

Default: 10

Environment variable: TMT_PLUGIN_TEST-CHECKS_WATCHDOG_SSH_PING_THRESHOLD

In plan metadata:

ssh-ping-threshold: N

On command-line:

--ssh-ping-threshold N
export TMT_PLUGIN_TEST-CHECKS_WATCHDOG_SSH_PING_THRESHOLD=N