framework: Add aborting option when a monitored error is detected

This adds a policy which advises when the user should reboot the system
to avoid noisy test results due to system becoming unstable, for
instance, and therefore continues testing successfully.

To do this, a new module is proposed. A class Monitoring is used for
managing the monitoring rules. Two types of rules, MonitoringFile and
MonitoringLinuxDmesg, derived from the abstract class MonitoringBase,
have been implemented. The first allow to track a pattern on standard
files or locked files. The second, derived from dmesg.LinuxDmesg, will
track a pattern on the dmesg.

The monitoring rules must be defined in piglit.conf at the section
monitored-errors. If one of the regex is found, Piglit will raise a
PiglitAbort exception, stop the test execution -terminating test thread
pool- and exit with code 3.

Then test execution resume, after rebooting the system or not, is done
like usually with command line parameter "resume".

To call it, use command line parameter: --abort-on-monitored-error

This option implies --no-concurrent

This include also a set of unit tests for this module.

Reviewed-by: Dylan Baker <dylanx.c.baker@intel.com>

1 files changed, 29 insertions, 0 deletions

diff --git a/piglit.conf.example b/piglit.conf.example
index b95ff89f4..b555129b1 100644
--- a/piglit.conf.example
+++ b/piglit.conf.example
@@ -162,3 +162,32 @@ run_test=./%(test_name)s
 [expected-crashes]
 ; Like expected-failures, but specifies that a test is expected to
 ; crash.
+
+[monitored-errors]
+; Set the monitoring rules for checking when the system need to be rebooted.
+; Piglit must be launched with --abort-on-monitored-error
+;
+; For each activated monitoring rule a section must be created in this file that
+; contains the type of monitoring (dmesg, file or locked_file).
+; Depending on the type, the parameter 'parameters' is a filename or a list of
+; options. The regex is the pattern that causes Piglit aborting when it's found.
+; Examples :
+;
+;i915_error_state
+;kmsg_monitoring
+;dmesg_error
+;
+;[i915_error_state]
+;type=file
+;parameters=/sys/kernel/debug/dri/0/i915_error_state
+;regex=^((?!no error state collected).)*$
+;
+;[kmsg_monitoring]
+;type=locked_file
+;parameters=/dev/kmsg
+;regex=\*ERROR\* ring create req|\*ERROR\* Failed to reset chip
+;
+;[dmesg_error]
+;type=dmesg
+;parameters=--level emerg,alert,crit,err,warn,notice
+;regex=\*ERROR\* ring create req|\*ERROR\* Failed to reset chip|BUG:|Oops:|turning off the locking correctness validator