| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
If xz is present, and the piglit command's output is redirected, then
xz will think its output is redirected. This will cause xz to try to
read data from stdin to compress.
Instead we can run xz with --help to cause it to print help
information if the xz executable is present, and prevent xz from
trying to compress data from stdin.
Since we don't want to see 'xz --help' output from piglit, we now need
to redirect both stdout and stderr to /dev/null.
This allows us to pipe the output of piglit summary when using the shell
xz compressor.
v2 (Dylan):
- send both stdout and stderr directly to /dev/null
- Fix xz detection in the xz shell path.
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
|
|
This patch changes binary_check to work when a binary is present, but
doesn't work for some reason.
The example here is in the json content tests, where it looks for
glxinfo, but if X isn't running glxinfo provides no output, which will
cause a test to fail. This isn't useful, so instead of using the *nix
command 'which', call the command and allow an expected returncode to be
passed, if the returncode isn't that returncode, then skip.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This was a design mistake from the start, subclassing the Test class and
overriding the run() method achieves the same result, without the need
for this hook
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Currently if piglit updates a result to a new version, then tries to
load from the same directory the first result will be
'results.json.old', and piglit will interpret the '.old' as a
compression extension, and die in a fire.
With this patch when searching a folder (but not when pointing directly
at a file) piglit will ignore any files ending in '.old', which will
correct the problem.
A unit test is also included.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This makes the code a little cleaner and easier to read.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This modifies a unit test to catch that a test name ends with '.' when
it shouldn't, which in turn demonstrates a bug in the xz shell path,
where it adds an extra '.' to the end of a filename. This patch fixes
that bug as well.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
|
|
This adds a test and fixes it, so that when specifying a filename like
'foo.json..gz' it will work.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
|
|
This adds a test for overwriting an xz file in the non-backports.lzma
path, and a fix for overwriting an existing xz file.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
|
|
When compressing, if there is an error it will be caught and handled by
the framework, with a nice error message. Don't allow xz to spam stderr
as well.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
|
|
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Liam Middlebrook <lmiddlebrook@nvidia.com>
Acked-by: Dylan Baker <baker.dylan.c@gmail.com>
|
|
This reverts commit 472e97211b1e01054613f2c93b682227e12a7017.
This breaks our jenkins setup in non obvious ways. Revert it until a
better solution is found.
|
|
This reverts commit 992bbcec39c335892c2c618fe3923a6a0ce6ffa4.
|
|
With the previous change there is only one user of checkDir, which was
in need of refactor if it was going to be generally useful.
So instead, just remove it, the replacement is less code and it's
obvious what it does.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
A current source of confusion right now, is to run piglit, and setting a
results directory that already contains a partial result in it. If the
new run is shorter than the run it's overwriting, then when it
accumulates all of the tests together into a single file at the end of
the run, these extra tests from the previous run will be included.
To prevent that, this patch clears the contents of the directory (at an
implementation level it actually deletes and recreates the directory),
so that there is a clean working directory to work with.
Obviously this doesn't affect the resume path.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Shares the setup and teardown functions for locking compression modes
between the various backend test modules.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
There is currently a bug in piglit (demonstrated by the previous patch),
that shows that compression is hard set before piglit.conf is loaded in
run, and not affected by any subsequent re-reads of piglit.conf. This is
problematic, resulting in the compression mode being either
PIGLIT_COMPRESSION or the default value, bz2.
This patch corrects that by removing constant values and replacing them
with a getting function.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Currently the compression code has no way to handle a change in
piglit.conf (which incidentally it needs to, since that happens
immediately in run mode). This results in the default mode (bz2) being
picked when a PIGLIT_COMPRESSION environment variable isn't specified,
even when a piglit.conf value is.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This adds the option of using a python module, rather than calling out
to the shell for xz support.
v4: - add this patch
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This requires that there is an xz binary installed, and accessible,
obviously.
v4: - add this patch
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This adds bzip2 compression support to piglit. This produces a file that
is ~5% the size of the original uncompressed json file.
v4: - add more tests
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This adds support to compress results with gzip compression. This
reduces the size of json results significantly (from 21M to 1.6M when
running the quick profile (which is about 7% of the uncompressed size).
v4: - add additional tests
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This creates a framework for compressing results as part of the
FileBackend class. This allows for the simple addition of compression
formats (gz is easily possible in python2, xz in python3), but does not
actually implement any compression. zip and bz2 compression are also
possible, although they'll require a little more code.
This patch implements a framework, and tests for that framework.
The goal is that other compression methods can easily be added to piglit
simply by setting a few values in framework/backends/compression, and
then everything will just work.
This will allow junit results that are compressed to be read, but
doesn't add support to junit for compressing results, since junit is
mainly intended for consumption by jenkins, which doesn't understand
compression.
v2: - replace tests decorator set_compression with use of
utils.set_piglit_conf. This is better because it uses a generic
tools, and because set_compression is somewhat misleading of a
name.
v3: - backport changes from my attempt to port this to python3. There
are significant differences between python2 and python3 compression
handling because of the bytes/str/unicode differences between the
two versions. There are also code cleanups from that work here
- Make a better effort to protect the unit tests from the
environment around them. The goal is that the tests control the
compression options 100%, with no chance of piglit.conf or
environment variables affecting the tests.
- reduce code duplication in unit tests
- add additional unit tests as made possible by refactoring changes
v4: - add additional unit tests
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This decorator sets environment variables, then runs the test, then
restores them.
If the value is stored as __DONOTRESTORE__, the idea is that it should
be deleted, otherwise it should be restored. But, before this patch it
would set the value to __DONOTRESTORE__ if the value was
__DONOTRESTORE__, and the key wasn't in os.environ.
This fixes that.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Timothy Arceri <t_arceri@yahoo.com.au>
|
|
In both cases these should have been changed to PiglitConfig instead,
which is mostly the same except for a couple of convenience methods.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Timothy Arceri <t_arceri@yahoo.com.au>
|
|
Because python 3 uses unicode by default, this is required to get python
3 to work; for python 2 this has no effect on the generated values.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Liam Middlebrook <lmiddlebrook@nvidia.com>
Reviewed-by: Timothy Arceri <t_arceri@yahoo.com.au>
|
|
This is necessary in Python 3 to put a Status in a dictionary or set,
but is optional in python 2. It doesn't do any harm implementing it now,
and will easy the python 3 transition
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Timothy Arceri <t_arceri@yahoo.com.au>
|
|
Python2.7 supports both the legacy next method, and the future looking
next() function. We'd rather be future looking so use the next function.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Timothy Arceri <t_arceri@yahoo.com.au>
|
|
Apparently python's os.path.isfile doesn't consider special files like
one in /proc/self/fd/ to be a file. So instead of testing with 'isfile',
test with 'not isdir'.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
|
|
When an uncaught exception reaches the top level exception handler, and
PIGLIT_DEBUG is true, then the exception will be re-raised. The problem
is that currently the exception is re-raised as 'raise e', which raise
the correct exception, but does so from the wrong context, making the
back-trace completely useless. By re-raising by simply using the 'raise'
keyword causes the exception to be passed on 'as-is', making the back
trace useful for debugging.
Trivial.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Incomplete results do not include a time or return code, so use a blank
string as the default value for these keys.
Signed-off-by: Thomas Wood <thomas.wood@intel.com>
Reviewed-by: Dylan Baker <baker.dylan.c@gmail.com>
|
|
Currently if one specifies a non-existant module (ie: piglit run
deqp_gles4 out), then an uncaught ImportError will be raised. This patch
fixes that by catching the exception and re-raising a PiglitFatalError.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
JUnit has no concept of "warning". It supports the following
statuses:
- skip
- success
- fail
- error
A test which emits a warning is more accurately represented as
"success" in JUnit.
v2: Continue to report failure for "dmesg-warn", which is more serious
than "warn". (from Ilia Mirkin)
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Dylan Baker <baker.dylan.c@gmail.com>
|
|
The order of the statuses in the docstring was incorrect, this patch
corrects the order to be correct. Also adds the missing incomplete
status.
Trivial.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Currently we blindly pass deqp extra args to each test case, and only to
the test case.
There are two cases where this is problematic:
1) if the extra args need to be passed to the deqp command that
generates the test list (it isn't before this patch)
2) if there are any --deqp-case options passed into the Test derived
classes this will cause the tests to fail (conflicting options)
Both of these are resolved by this patch.
Reviewed-by: Mark Janes <mark.a.janes@intel.com>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
When extra_args aren't set in piglit.conf and the envrionment variable
isn't set either, there is a bug that causes the deqp tests to not run.
This patch corrects that by providing a mechanism for setting an
appropriate fallback value.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Programs that terminate via MSVCRT's abort(), including failed
assertions, return code 3, not a negative number.
This is particularly pertinent now that the use of the standard assert
macro has increased in Mesa (over gallium's assert macro, which would
trap the debugger with INT3, and return a negative exception number.)
Reviewed-by: Dylan Baker <baker.dylan.c@gmail.com>
|
|
When two json results have the same "name" value in the root of the json
dictionary, piglit will try to create two directories with the same
name. In this event an OSError will be raised to the top level exception
handler, resulting in a bug warning.
This patch raises an expected error, giving the user advice on resolving
the issue.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Ben Widawsky <ben@bwidawsk.net>
|
|
The correct way to get error messages from exceptions is to use str()
or unicode() on them, not by reading the message attribute. This is even
more relevant as we look toward python 3, where exceptions don't have a
message attribute at all.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This option has never been tested against anything except Piglit tests,
and probably isn't useful anyway, since we probably don't care if
integrated tests are leaking.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
DEQP provides a large number of test profiles, but currently we can only
use the gles3 profile. It would be useful to get at the gles31 profile
and the gles2 profile at least, if not also the egl profile.
This also adds unit tests for the shared parts of deqp.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
file is corrupt
If a complete result is corrupt, currently an exception will be raised
and caught at the root (this will be an Exception, but a
PiglitFatalError), this will result in a bug message.
With this patch the corrupt file will be identified as well as the error
presented by the json module about the underlying file.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This patch adds an option to not retry incomplete tests when resuming
a test run. This is especially useful when a failing test causes
a crash or reboot. Currently, that same test runs again when
attempting to resume the test run, resulting in the same crash or
reboot.
Signed-off-by: Mike Mason <michael.w.mason@intel.com>
Reviewed-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Use an exception rather than passing booleans. This removes a bunch of
biolerplate code and makes things easier.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Largely this just replaces a local exception with a generic one.
Signed-off-by: Dylan Bake <dylanx.c.baker@intel.com>
|
|
Use piglit exceptions for aborting and erroring rather than custom
classes.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Don't raise Exception.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This patch largely replaces an existing TestDictError class with a
generic exeption for exceptions.
It also simplifies some error handles in the load_test_profile function.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This allows more exceptions to be caught at the top level exception
handling, requiring fewer try/except blocks in the middle of the code
that exit. This should make the code a little easier to read.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
