diff options
| author | Mark Janes <mark.a.janes@intel.com> | 2014-09-23 15:05:19 -0700 |
|---|---|---|
| committer | Dylan Baker <dylanx.c.baker@intel.com> | 2014-09-25 14:34:47 -0700 |
| commit | 1921e13dcdca4fcb7cbaa400692aebf3ad55f646 (patch) | |
| tree | 549edb7fc488976201f0eb036c0ef0a32aaa7b69 /piglit.conf.example | |
| parent | 9031648adb03e6e0600b4fb69f7848e55af35c73 (diff) | |
Filter expected failures from JUnit output
Piglit test results are difficult to analyze with simpler JUnit
visualizers like those provided by Jenkins. There are hundreds of
failures, but the engineer is typically interested only in NEW
failures. Jenkins JUnit rules typically expect 100% pass rate, or
some static threshold of failures.
Specifying test names in the [expected-failures] or [expected-crashes]
sections of the config file enables JUnit reporting of just the test
failures which represent new regressions.
Test names are expected in the dotted JUnit format (eg:
"piglit.spec.ARB_fragment_program.fp-formats") and are case
insensitive.
Reviewed-by: Dylan Baker <dylanx.c.baker@intel.com>
Diffstat (limited to 'piglit.conf.example')
| -rw-r--r-- | piglit.conf.example | 12 |
1 files changed, 12 insertions, 0 deletions
diff --git a/piglit.conf.example b/piglit.conf.example index 9cb7dd175..a864c0190 100644 --- a/piglit.conf.example +++ b/piglit.conf.example @@ -71,3 +71,15 @@ run_test=./%(test_name)s ; Options can be found running piglit run -h and reading the section for ; -b/--backend ;backend=json + +[expected-failures] +; Provide a list of test names that are expected to fail. These tests +; will be listed as passing in JUnit output when they fail. Any +; unexpected result (pass/crash) will be listed as a failure. The +; test name must be written in the JUnit format ('.' instead of '/'). +; Special characters for config file format ('=' and ':') must be +; replaced with '.' + +[expected-crashes] +; Like expected-failures, but specifies that a test is expected to +; crash.
\ No newline at end of file |