| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
six.moves.zip is itertools.izip on python 2, and builtins.zip in python
3. Which are the same lazy function
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Acked-by: Jose Fonseca <jfonseca@vmware.com>
|
|
This is another shim for something that moved between python3 and
python2.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Acked-by: Jose Fonseca <jfonseca@vmware.com>
|
|
This module is a compatibility shim for python 3.x and 2.x
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Acked-by: Jose Fonseca <jfonseca@vmware.com>
|
|
This is xrange on python 2.x and range on 3.x
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Acked-by: Jose Fonseca <jfonseca@vmware.com>
|
|
These are the three python3 like behaviors that piglit should rely on.
The only other applicable future import is unicode_literals. Although my
plan is to use unicode_literals, that will actually cause behavioral
changes in some cases, where these cause minimal changes to the code.
Piglit will not be targeting < 3.2, they are old, unsupported, and have
fewer features than 2.7.
Piglit now has division (using / as floating division, and // as integer
division), print as a function (rather than a statement), and
absolute import, which changes the way import works when there's a
conflict between a local import and a system wide one. Absolute import
makes more sense, and copies the behavior of python 3
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Acked-by: Jose Fonseca <jfonseca@vmware.com>
|
|
This variable allows timeouts to be disabled, which can be useful for
debugging tests.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Previously a timeout would be marked as pass, but that makes no sense.
This marks it as an error.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Subprocess32 provides a backport of python 3.2's subprocess module,
which has a timeout parameter for Popen.communicate. When the timeout
runs out then an exception is raised, and when that exception is caught
we can kill the process.
This is fairly similar to the way the current timeout mechanism works,
except that the current mechanism is not thread safe. Since one of the
major features of piglit is that it offer's processes isolated
concurrency of tests, it makes sense to make the effort to provide a
timeout mechanism that supports concurrency. Unfortunately there isn't a
good cross platform mechanism for this in python 2.x, even with
subprocess 32 only *nix systems are supported, not windows.
The big advantage of this is it allows consistent behavior between
python 2.x and 3.x as we look toward the future and the possibility of
using a hybrid approach to transition to python 3.x while maintaining
2.x support until it's not needed.
This patch look pretty substantial. It's not as big as it looks, since
it adds some fairly big tests.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Exceptions were not reaching it.
Reviewed-by: Dylan Baker <baker.dylan.c@gmail.com>
|
|
Currently the framework is pretty confusing. There's 'test' which is a
package containing test classes, then there's 'tests' which contains
unit tests for the framework. This is obviously not optimal. Beyond that
the unittests get installed, and that isn't really necessary.
As a bonus this removes the unit tests from coverage reports, which is a
nice plus.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This is more groundwork to move the unit tests out of the framework
directory.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Setting PIGLIT_NO_FAST_SKIP will disable the fast skipping mechanism
altogether.
It does this by shadowing FastSkipMixin at import time. This method is
robust and cheap, but makes testing difficult.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This commit was generated with the following sed commands:
sed -i -e 's@from framework.tests import utils@from . import utils@g'
sed -i -e 's@import framework.tests.utils as utils@from . import utils@g'
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Use a path relative to the unit test. This is groundwork for moving the
unit tests out of the framework folder.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Vinson Lee <vlee@freedesktop.org>
|
|
This adds a new environment variable, PIGLIT_FORCE_GLSLPARSER_DESKTOP,
which forces glsl_parser_test.py to use "glslparsertest" for GLES tests
(instead of "glslparsertest_gles2").
This could be used to force testing ES_<ver>_compatibility extensions
even when OpenGL ES tests are being built.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This patch fixes the behavior of glsl_parser_test in cases other that
OpenGL and OpenGL ES are available. This means that if OpenGL ES isn't
available then OpenGL ES shaders will be passed to the regular version
of glslparsertest, which can run them with an ARB_ES<ver>_compatibility
extension. When a test is for desktop OpenGL, but only OpenGL ES is
available, then the test will skip in the python layer.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
v3: - Rebase on top of now upstream fast-skipping patches
- When using glslparsertest to run GLES tests add
ARB_ES<ver>_compatibility to gl_required, to integrate with fast
skipping.
- Fix some tests. These tests are pretty fragile and probably just
need to be reworked.
- small refactors, make use of the _is_gles_version helper
- Split skip conditions into a private method
v4: - Fix bug where ARB_ES<ver>_compatibility skip requirement would be
overwritten if the test had other extension requirements (and
update tests to catch this bug)
- Rename some constants to be more clear
- Remove use of flag for skipping.
|
|
There are two versions of glslparsertest, an undecorated one for desktop
OpenGL, and a '_gles2' suffixed one for OpenGL ES. Piglit should pass 3.1
and 3.2 to the '_gles2' version like it does for GLES 2 and GLES 3.0
This also extends the test generator to cover these new cases (and a few
it should have been covering before). It now covers 1.00, 3.00, 3.10,
3.20, 3.00 es, 3.10 es, and 3.20 es.
v5: - fix the wording of a comment
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This information can be used on linux in conjunction with scripts to
associate information in dmesg with information in piglit results.
Adds the pid information to both the json and junit backends, and adds
tests for both.
This information is not published into any of the summary modes on
purpose. It's usefulness is limited to the system on which the test was
ran, and only while a log of dmesg exists.
v2: - remove extra newline in junit stderr (Mark)
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Mark Janes <mark.a.janes@intel.com>
|
|
This slightly refactors these test classes, fixing time time tests, and
making the to_json test not do a dict -> test result -> dict conversion,
skipping the first dict stage.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Mark Janes <mark.a.janes@intel.com>
|
|
This extends the version handling methods of the WflInfo class to handle
two kinds of errors. First, it handles getting a patch version in OpenGL
and GLSL version strings. Second, it handles getting back
WFLINFO_GL_ERROR, such as is returned on the nvidia blob driver because
of the patch version.
v2: - update tests to cover more tests (Jose)
- update gl_extensions to not return set(['WFLINFO_GL_ERROR']). This
wouldn't break anything, but it wasn't strictly correct.
Reviewed-by: Jose Fonseca <jfonseca@vmware.com>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Previously a stack trace would be suppressed in all instances unless
the PIGLIT_DEBUG environment variable was truthy. This was suboptimal
for a number of reasons.
This patch removes that functionality, now only a subset of piglit
specific exceptions that are meant to be suppressed are. All other
exceptions are directly raised, stopping the runner immediately. Other
piglit specific exceptions have been extended to have custom error
messages rather than relying on the handler to add them.
v2: - remove unit tests that no longer apply
Reviewed-by: Jose Fonseca <jfonseca@vmware.com> (v1)
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
I'm going to admit I'm a bit puzzled how this could have slipped through
without being caught (I'm guessing an unrelated change uncovered this).
But basically if a test doesn't raise an exception, and the returncode
is > 0, it should mark the test with a status of "fail", but it doesn't.
Instead the default status "notrun" is passed to the logger, which it
doesn't support, and an exception is raised. This results in the test
being reported as "incomplete", since the thread asserts before it
writes the new result.
This patch corrects that problem.
This fixes the following test on i965 from mesa master:
spec/arb_compute_shader/indirect-compute: incomplete -> fail
bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=93340
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Mark Janes <mark.a.janes@intel.com>
|
|
I hit this assert debugging a problem, and found the lack of a message
detailing the assert annoying.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
|
|
Since division isn't used in the module this is a trivial change.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Trivial.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Vinson Lee <vlee@freedesktop.org>
|
|
This exception predates the exceptions module. The only instance of this
error is used in a place where piglit should stop because it cannot
continue, which is what PiglitFatalError is for.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Currently this class uses the abc decorators, but its type isn't
actually abc.ABCMeta, which makes the abc decorators no-ops. This fixes
that.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Trivial.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Vinson Lee <vlee@freedesktop.org>
|
|
This module reworks the dmesg testing to be much more robust, largely by
taking advantage of the mock module. This allows us test test dmesg
without actually calling dmesg, which eliminates the need for root
privileges, and allowing all tests to run on all platforms.
These tests also don't probe at the internal bits of the class, instead
they probe at the public API.
This significantly simplifies the tests, increases coverage, and
improves the quality of the tests in many cases, while making some of
the tests less fragile but less specific in other cases.
v2: - add tests for dmesg wrapping. Tests for this functionality where
present in the replaced implementation.
- remove '--attr=\!privileged' from tox.ini. There are now no
privileged tests, so no need to exclude them
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This adds extra information to determine the intel gt slice.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Chad Versace <chad.versace@linux.intel.com>
|
|
This adds a new "summary feature" command to piglit which creates a HTML table
with the feature x DUT status (useful for multiple features, multiple DUTs).
Another use case is the feature status for subsequent test results (depending
on the meaning of that test result - DUT or build)
A feature readiness is defined by the piglit regexp which selects the tests
relevant for that feature and the acceptance percentage threshold (pass rate).
It requires an input json file containing the list of features, in the
following format (this is just an example):
{
"glsl" : {
"include_tests" : "glsl",
"exclude_tests" : "",
"target_rate" : 90
},
"arb" : {
"include_tests" : "arb_gpu",
"exclude_tests" : "",
"target_rate" : 10
}
}
v3:
Changed json rate to int instead of string
Applied other review comments
v2:
Apply review comments (Dylan, Thomas)
Fixed 2nd round of review comments
Signed-off-by: Gabriel Feceoru <gabriel.feceoru@intel.com>
Reviewed-by: Dylan Baker <baker.dylan.c@gmail.com>
|
|
This patch allows passing '/' separated tests into the
-t/--include-tests -x/--exclude-tests options. This becomes
particularly important to fix since the verbose logger now prints '/'
separated names, as does the console logger.
v2: - fix typos
Reviewed-by: Glenn Kennard <glenn.kennard@gmail.com>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This stops the logger from using '@', and instead prints '/'.
Reviewed-by: Glenn Kennard <glenn.kennard@gmail.com>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This adds a function to grouptools that replaces grouptools.SEPARATOR
with '/'. This function is meant for use when printing to the console,
since most people find '/' easier to read than whatever is being used
internally to separate a group, and because consistence counts.
The implementation differs from that originally used in console_.py,
this implementation (tested on python 2.7.10) is roughly twice as fast.
v2: - fix typos
Reviewed-by Glenn Kennard <glenn.kennard@gmail.com>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
trivial.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Dylan Baker <baker.dylan.c@gmail.com>
|
|
On the glslparser profile I see a roughly ~15 second speedup on my HSW
with this patch.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Running the glslparser profile this results in a ~50 second reduction in
run time on my HSW
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This adds support to ShaderTest to scrape the GLSL requirements, and
provide them to the FasSkipMixin.
This reduces run time by ~15 seconds on my HSW.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This implements a similar mechanism to check GL or GLES version support
on the current driver.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This hooks up the new mixin from the previous patch in ShaderTest (the
class used to run shader_runner). This patch gives me a ~40 second
reduction in runtime for shader.py (a profile that runs only shader_test
files).
My setup is as follows:
HSW i7-4500U (4 logical cores)
running piglit with "-c -p gbm"
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This adds support to the FastSKipMixin for checking GLSL and GLSL ES
versions.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This patch adds support to the FasSkipMixin for skipping on GL and GLES
versions. It does this by querying wflinfo for the highest supported
This skipping only supports checking that the requirement is <= to the
maximum GL(ES) provided. There are some instances of tests that have
requirements like "GL <= 2.1", and this patch doesn't cover case. These
cases are pretty rare, so it doesn't seem like a path worth optimizing
for anyway.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This Mixin provides a way for OpenGL tests to skip very fast. Currently
it only applies to GL extensions, but will be extended to cover GLSL
version requirements and GL version requirements (and ES)>
This is split into a separate module because it's going to grow into a
fairly large amount of code (mostly around querying wflinfo).
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This changes the behavior of the dEQP integration such that a status
that is < 0 on unix or (< 0 || == 3) on windows will be a crash rather
than a fail. A status > 0 (except 3 on windows) will still be marked
fail.
This makes use of the helper function from framework/test/base.py
rather than calling super() (which would still call the helper) because
we don't want to get the warn status that we would also inherit.
v2: - Update tests to use actual deqp ouput.
- Fix status loop, v1 had a bug that would cause the loop to not
exit when it needed to, but it would pass the simplified tests.
v3: - rename test helper function per Jason.
- Simplify a massive if/elif statement in said test helper function
Tested-by: Jason Ekstrand <jason.ekstrand@intel.com> (v2)
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
If users wish to clear their dmesg before a piglit run, the current code
will emit a warning. It is possible to query the information about the
running kernel, and most distros do package that, so default to that.
The live kernel config is more future proof than regex parsing of the
timestamp (although I'd be shocked if either ever stopped working), and
it works for the empty dmesg case.
v2: Just return if we found a timestamp in config.gz
Use with for opening the file (Dylan)
Add comment (Dylan)
Reviewed-by: Dylan Baker <dylanx.c.baker@intel.com>
Signed-off-by: Ben Widawsky <benjamin.widawsky@intel.com>
|
|
This is the plunge to change over from the old options model to the new
one. After this change Options is a global value, and no more passing or
creation of new instances is done (except for testing).
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This changes the behavior of the Options class. Instead of being an
instance that's passed around, it's now treated as a global variable (or
as close to one as python gets to having globals), anyone who needs it
goes and gets it out of the options namespace and uses it.
This patch does a lot of refactoring work that is needed to ensure that
Options continues to work as it did before (where it was mostly
initialized and then passed around), This includes a significant amount
of testing for the new code around Options.
v2: - fix spelling in comments and docstrings
- Fix _ReList docstring, which was _ReListDescriptor's docstring
- Fix _ReList.__compile to ensure RegexObject.flags was correct
- Add docstring to _ReList.__compile
- Add __delete__ to _ReListDescriptor as NotImplementedError, this
is just a completeness issue
- Add/update tests for these changes
- Remove duplicate addition of 'mock' to tox.ini
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This adds a pretty extensive set of tests for oglconform.py, with the
goal of making further cleanups and refactors easier and not introducing
regressions.
This adds a dependency of mock for the unittests. No production code
needs this dependency, so this wont affect most (any?) piglit users.
Mock allows for better testing, in fact, it allows oglconform to be
tested even when it's not installed.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
