| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
The resume path duplicates the 2 lines of code, so the same fix is
required there.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This fixes trying to load .tmp files that don't get moved during an
interrupted run.
bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=101503
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Hector Edmundo Ramirez Gomez <hector.edmundox.ramirez.gomez@intel.com>
|
|
This is currently expected to fail.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This reworks the fixture to not use autouse, so that other tests can be
added to the class without getting the setup it doesn't use. This also
uses a class scope fixture instead of an instance scope one.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
|
|
This allows the command to be overwritten or modified after
instantiation, which is useful for adding additional arguments in a
profile.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
|
|
This move is going to allow us to supplement the TestDict with a
different class that can be used instead that loads xml.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This allows a significant amount of cleanup to happen. It allows
removing attributes from the global OPTIONS, removing tests that no
longer apply, and because of the split run method it allows more values
to simply be passed.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This patch changes the way that the tests from TestProfile are
processed. Rather than having a process_test_list method that modifies
TestProfile.test_list, a new itertests() method creates an iterator that
yields tests that are not excluded by filters.
This saves a bit of code, increases the memory usage, and reduces
surprise.
It would be nice if there wasn't a need to create a concrete test list,
but there wouldn't be any way to get the number of tests otherwise.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Since these are also just special cases of filters for the standard
TestProfile filtering mechanism, and they have a lot of unique classes.
This is just a waste, the same can be achieved with a much simpler class
structure.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Rather than putting this in a global variable, just add a filter for
this in the runner. Far simpler, and removes more globals.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Because we can copy profiles, we don't need to merge them to run more
than one of them. Instead we can simply have a list of profiles, and run
them one by one. One side effect of this is that tests will be run one
profile at a time, so if running with out the -1/--no-concurrency or
-c/--all-concurrent options tests will run in a sort of zipper pattern:
<p1 concurrent>, <p1 non-concurrent>, <p2 concurrent>, etc.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This will allow a profile to be copied and "subclassed" without
affecting the original profile. This will allow a long-standing bug that
made it impossible to run two subclasses of all.py (say shader.py and
glslparser.py) at the same time, since they would both try to modify the
all.py profile in incompatible ways.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
There are a couple of reasons for doing this. First, profile is a big
complex mess that does entirely too much, and this helps with that.
Second, there are bugs in the way two profiles run at the same time
work, and this is going to fix that.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This adds a few simple tests for the feature completeness summary
object. Nothing to strenuous but much better than nothing.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Adds this new generator to the list of generators to test. Trivial.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
If it's not present make it an empty list.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Brian Paul <brianp@vmware.com>
|
|
The JUnit loader was never updated to handle PID's as lists rather than
a single int. This patch corrects that and allows JUnit results to be
loaded again.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Mark Janes <mark.a.janes@intel.com>
|
|
Currently piglit doesn't consistently deserialize JSON into it's
internal representation, leaving some bits as dictionaries or lists,
which breaks the summary code. This patch corrects that.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Mark Janes <mark.a.janes@intel.com>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
|
|
This changes the way updates are done in the backend, instead of
converting to a TestrunResult immediately, all of the transformations
are done to the JSON data in it's rawest form, ie, as dicts and lists,
and then transform to a TestrunResult (and children) after that.
This makes the loading code more robust and simpler, since it's
decoupled from the representation, making the transformations easier to
test and manage.
Part of this change is fixing the .to_json and .from_dict methods, many
of which "worked" because their shortcomings were papered over by using
json.load with a custom decoder. This patch fixes them to actually work
correctly. Despite my best attempts I couldn't decouple this work for
this patch because of the close coupling of the JSON loading code and
the class representations.
There are a number of fixups to the tests in this patch, since a number
of issues were being covered by the TestrunResult.to_json() method
filling it missing values.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Most of these are pretty old at this point (over a year), and it seems
unlikely anyone still has results in those formats. The biggest problem
with them is that some of them (especially the early ones) are complex
transformations with a lot of validation needed, and some of the middle
ones rely on the class representations to do the transformations, which
is what this series is trying to change, since the tight coupling makes
changes to the schema difficult, or nearly impossible in other cases.
If there is compelling reason I could push support back a bit more, but
I really want to drop some of the versions we support since it
simplifies the code considerably.
This also drops support for the older "main" file, and only supports
results.json and results.json.<compression>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This adds a class based on the ReducedProcessMixin that allows the
python layer to understand the output of shader_runner when it runs more
than one test per process.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This Mixin makes writing classes for handling tests that run multiple
tests in a single process simpler. It does this through the use of the
subtest feature. It makes it possible to implement two new methods, and
an aware interpret_result method and have support for this feature,
including a cherry-like resume feature that starts again after a test
crashes, without rerunning the crashed test.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This change makes the TestResult.pid field into a list of ints instead
of a list. This will be used by the multiple test per process tests to
ensure that all PIDs are recorded when the cherry features is invoked.
This is groundwork for later patches in the series.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This splits the FastSkipMixin into two classes. One that actually does
the fast skipping, and one that provides a mixin for Test.
This split will allow the class to be used in the traditional way, but
also in shader_test.MultiShaderTest.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Which was broken by the last commit touching the logger, as were several
unittests that continued passing because of a dirty environment.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Ensure that it converts bytes to str and also collects correctly.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
For the apitrace test runner, I need to be able to store known-good
images for various drivers, which means I need some way to group
drivers. Not all Mesa drivers we care about are covered here, but
this is a sample to get things started.
v2: Apply feedback from Dylan, add classification for my Skylake.
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
|
|
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
It turns out that the backends coincidently put certain fields in, but
the new jsonstreams backend does not automatically add these fields, it
must be passed them. (This is the behavior of piglit when not testing).
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
We generate this on the fly if it's not there and add it.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This class handles subtests by making a test with subtests into a
testsuite element, and then makes each subtest a testcase element.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Acked-by: Jose Fonseca <jfonseca@vmware.com> (v1)
Reviewed-by: Mark Janes <mark.a.janes@intel.com>
Tested-by: Mark Janes <mark.a.janes@intel.com>
|
|
This new class is attached as an attribute to the JUnitWriter class, in
such a way that it has the same interface, and some attributes are moved
to this new class since they're only needed there. This has the
advantage of making the JUnitBackend class simpler, and makes the
writing easier to test, it will also be used in a follow up patch to
implement a different class that supports subtests.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Acked-by: Jose Fonseca <jfonseca@vmware.com> (v1)
Reviewed-by: Mark Janes <mark.a.janes@intel.com>
Tested-by: Mark Janes <mark.a.janes@intel.com>
|
|
The signature of the stub version was not updated when the real version
was, which means setting the environment variable will raise an
exception.
Also adds some tests for this problem.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Marek Olšák <marek.olsak@amd.com>
|
|
This patch generalizes the mustpass list functionality in deqp_gles3.py.
While the idea is the same, this implementation is considerably
different than the version in deqp_gles3, since it bypasses calling the
binary to generate a test list altogether, and simply replaces the
iter_deqp_test_cases function by providing output that the make_profile
function can consume.
The end result is that this is simpler (though more LOC are added since
this actually has tests), and can be applied to gles2 and gles31.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
In the past piglit has added or removed options from that part of the
results without caring too much. The reason for this is that these
values are really meant for developers to manually inspect, and for
resuming. We don't support resuming a run on a different version of
piglit than it started (and that's crazy, we never will), so allowing
these values to change arbitrarily is fine.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Currently the python layer tries to dispatch shader_tests to the right
binary (gles2, gles3 or gl). But the implementation is pretty dumb. It
basically assumes either >= or == are the only operators used, and makes
wrong assignments for < or <=, which are allowed. Is also only
understands 2.0 and 3.0 as GLES versions, which is problematic.
This version uses the closes equivalent of a match statement that python
has, to try to pick the right GLES version, otherwise it uses the GL
version. It also shares this picking mechanism with the Fast Skip
mechanism, so they always have the same result.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
cc: mark.a.janes@intel.com
cc: currojerez@riseup.net
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
|
|
When using the GL binary to run GLES tests the appropriate ES
compatibility extension is added to the fast skipping requirements, but
not to the command line options. It needs to be added to the command
line options too, so that it will be skipped properly when not using the
fast-skipping layer.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Currently if a C style comment has a blank line in it:
/*
* my comment
*
* more
*/
The parser will choke and die. That's not good, so fix it.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This Mixin is currently broken because it doesn't handle warn statuses,
but it really needs to handle them because the underlying Test class is
going to turn a number of statuses it expects to be 'pass' into 'warn'
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This is currently being set to True, but valid options are "some",
"all", or "none".
This fixes the JSON validity unit test.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This deletes the quite extensive nose framework that was built up in
piglit, and the tox dependencies. These have all been replaced with a
combination of mock and pytest features instead.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This module is really a want for a schema and enforcement for that
schema, since there is in fact schema for JSON and several python
libraries to enforce said schema, it would be much better to use use one
of those, since that also makes the output format reproducible and
portable.
This adds the schema and the proper test, which currently this test
xfails because bugs.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
