| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This is required to get the crash and warn status.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This adds crash and warn statuses back and should have been handled as
part of commit 947b212a39304, but was not.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This changes the changes category to include every case were the first
status is not the same as the previous status, including
<status> <-> notrun, except when <status> is skip.
This fixes the failing unit tests from the previous commit.
v2: - don't include skip <-> notrun (Ilia)
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
|
|
changes
This test changes the behavior of the changes group to include enabled
and disabled tests, as well as fixes and regressions.
This adds 3 expected failures to the piglit framework unit tests.
v2: - dont add skip <-> notrun (Ilia)
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
These weren't updated when the API to the Log classes __init__ method
was changed.
Pass the extra argument needed to make these tests pass
Trivial
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Add an HTTP logger so that it is possible to monitor the progress and
status of a test run remotely through a web server. The web server runs
on port 8080 by default and can be configured to use a different port by
setting the "port" key in the "http" section of the configuration file.
The server responds to requests for /summary, which provides a summary
of the current state in JSON format.
v2: Add appropriate locking when reading state data for the web server
Stop the server after the request for the final results
Remove the full results page
Add a configuration option to set the server port (Dylan Baker)
v3: Use the new fallback parameter of safe_get (Dylan Baker)
Convert the test result status back to string
v4: Add newline after BaseHTTPServer imports (Dylan Baker)
Signed-off-by: Thomas Wood <thomas.wood@intel.com>
Reviewed-by: Dylan Baker <baker.dylan.c@gmail.com>
|
|
Declare the state and state lock variables at the same time so that the
same lock can be always used when accessing the state variable.
v2: fix dummy logger
Signed-off-by: Thomas Wood <thomas.wood@intel.com>
Reviewed-by: Dylan Baker <baker.dylan.c@gmail.com>
|
|
The gist of the problem is that we handle lookup for subtests in the all
group correctly, but when we went to compare between multiple results we
don't.
This patch makes use of the get_result from the previous patch to
simplify test name lookups, and fix the non-all case. This also fixes
the unit tests introduced in the previous patch.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Marek Olšák <marek.olsak@amd.com>
|
|
There is currently a bug in summary.Names that means that subtests are
added to the 'all' page (index.html), but not to the subset pages
(fixes.html, problems.html, etc). These demonstrate that problem.
This introduces 7 expected failures to the piglit framework unittests.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This provides a single source of getting results from tests or subtests,
and hides the magic of differentiating the two. This will make it easier
to work with subtests in the Summary module, since we can easily know
that we've checked subtests.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Marek Olšák <marek.olsak@amd.com>
|
|
TestResult already puts restrictions on what values it can store, so we
don't need to ensure that it can still be serialized, it will fail in
some other test non-specifically long before this test is able to
identify problems.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This fixes the privileged dmesg tests to actually work (it also reduces
the number of tests to one, since all of the tests actually tested the
same code path.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This function was unused and broken, so just remove it.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Everything is hooked up, except writing it. Restoring, processing, all
of that still works, just not writing it to the json, so fix that.
cc: <imirkin@alum.mit.edu>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This calculates the group totals on load (this is stored in the json),
which is required to get proper totals information.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Mark Janes <mark.a.janes@intel.com>
|
|
This allows time to be fully restored, but will fallback to the more
basic time element for restoration if this isn't parsable.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Mark Janes <mark.a.janes@intel.com>
|
|
This adds a dictionary assert that is similar to assert_dict_equal, but
it calls dict() on each object before comparing them. This allows
dictionary like objects (like Subtest) to be compared with the powerful
nose tools.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This marks the timeout tests as slow (because they are). This is nice
for excluding things.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This is a more standard way to mark tests out as special case, and not
rolling our own code is good.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Adds a message to test_parse_listfile_tilde.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This adds start and end time to the junit stderr. It would be nice to
have someplace better to put this, but junit doesn't allow extra tags in
the xml, and there are plans to use this data for post processing tasks.
Reviewed-by: Mark Janes <mark.a.janes@intel.com>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This makes use of the time TimeAttribute for handling times in results,
both for tests and for runs.
This change requires bumping the json version, and a large chunk of the
changes in this patch are for that change.
This could be split into two patches, but that would require making two
sequential bumps to the json results version, and I felt that the larger
patch was better than making two version bumps.
Reviewed-by: Mark Janes <mark.a.janes@intel.com>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This makes the test that tries to run a version 1 result to <current
version> more robust by always trying to update to whatever the latest
is.
Reviewed-by: Mark Janes <mark.a.janes@intel.com>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This class will provide a new interface for time, one that stores the
start and stop times, rather than the total time. It then provides
methods for getting the total and the delta (as used in the summary).
This commit adds the new class and tests for the class, but doesn't
start plugging it in yet.
The goal of this change is to allow post-processing to determine which
tests were running in parallel.
Reviewed-by: Mark Janes <mark.a.janes@intel.com>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
No one wants to see this, it isn't useful, and it produces a bunch of
clutter, so just send it to /dev/null (or your OS's equivalent).
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Like command, this somehow didn't get moved to the new TestResult class.
Generally this isn't a problem since this is a very uncommon path, but
in some cases this does get hit, and it would cause an exception in the
runner threads. Python has very bad exception handling in threads,
namely it doesn't have any; this results in an exception being raised
and remaining uncaught, terminating the thread, which in turn results in
the test result being incomplete, even though it should have been crash.
This patch fixes that situation.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Tested-by: Mark Janes <mark.a.janes@intel.com>
Reviewed-by: Mark Janes <mark.a.janes@intel.com>
|
|
It's like glxinfo or wglinfo, but for OpenCL.
Reviewed-by: Serge Martin <edb+piglit@sigluy.net>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This fixes the lists of regressions and fixes to not include skip or
notrun statuses.
This fixes a regression introduced in the recent summary rework of
piglit.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This was an oversight, it should have always been written to the json,
but was missed in the to_json method.
cc: Dave Airlie <airlied@gmail.com>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Sub-classing built-in types in python is an error-prone leap, and many
corner cases came out to bit us in this instance. By wrapping a dict
inside of another class we can get the desired results, it means more
code, but it also means less corner cases.
Also change the equality test of two unit tests, since Subtests isn't a
dictionary anymore assert_dict_equal wont work anymore.
cc: Mark Janes <mark.a.janes@intel.com>
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This is in keeping with the way that configparser.get() works in python3
(although there are some corner case differences since in python3 it
uses keyword only arguments, which don't exist in python2).
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
Reviewed-by: Thomas Wood <thomas.wood@intel.com>
|
|
This simplifies the code considerably, and removes the need to manually
calculate the group totals (since from_dict handles that for us)
Signed-off-by: <dylanx.c.baker@intel.com>
|
|
If we aggregate a result the metadata.json will be left (just in case),
and this code would ignore the results.json.<compression> and regenerate
the results each time. That's expensive and useless.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This creates 3 functions, each of which are simpler to understand and
do less than the giant function did previously.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This splits the code that generates the testname out of the main console
function, it also adds tests.
|
|
Splits the code that generates the summary out into a helper functions,
and adds tests.
These tests have identified bugs already.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This will allows us to break up the html and console generators into
smaller more manageable functions, while easily maintaining a pleasant
interface.
This split will allow for better unit testing, but keep the API clean
and simple.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Adds tests for functions that have existed for some time, but haven't
had tests.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This is a pretty hot function at this point, so making it faster is
better.
This works by pulling the dictionary that is used in place of a switch
statement out of the function, and into the module. This presumably
changes the lifetime of the dictionary making lookups faster.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This patch makes two changes:
First, it replaces the license header with a differently formatted, and
very slightly differently worded, version. This version is taken from:
https://opensource.org/licenses/MIT
and is used in most of the piglit framework.
Second, after some archeology I've added copyright assignment to
contributers back to 2013 (at which point the summary module was
completely rewritten).
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This adds a basic docstring to the summary module.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This reduces the amount of code considerably, and doesn't add too much
complexity to the template. The result is easier to maintain, and that
the code used is better broken up.
This version is also slightly faster (as measured with two results),
moving from ~1.8 seconds to ~1.2
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Gain a little code reuse and unit tests.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This removes the last vestiges of the Summary object, which wasn't doing
too much other than holding a few constants and turning result files
into a Results object. With this html is a stand alone function making
use of the Results class and it's members, rather than the Summary
class.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This allows us to set the options for all of our templates in one place,
once. This reduces code duplication considerably.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
Instead of building data structures full of this data that will only be
used once, calculate the data on the fly, this reduces memory
consumption and simplifies the code a lot
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
This makes it much easier for a human to parse the output.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
|
The summary object is a giant, bloated, awful thing. Using all of the
groundwork we've put together in the previous patches we can now
starting ripping it to pieces, pulling out the parts worth saving, and
converting them to use the new data structures, and throwing away a lot
of awful code in the process.
As a side effect, the list of tests will now include subtests.
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
|
