summaryrefslogtreecommitdiff
path: root/unittests
AgeCommit message (Collapse)AuthorFilesLines
2016-05-16unittests: Add a test case to ensure subtests don't mask crashesDylan Baker1-0/+13
Currently a test can crash, but if some subtests have a better status then the status will be the worst of the subtests. This isn't correct behavior because 'crash' is a framework generated status, not a status that the runner itself generates. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> reviewed-by: Brian Paul <brianp@vmware.com> Tested-by: Brian Paul <brianp@vmware.com>
2016-05-05unittests: make error messages for generator tests betterDylan Baker1-9/+11
This replaces the simple "it failed" message with the actual stderr output when the test fails. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
2016-05-03framework/profile: don't replace TestDict instance with regular dictDylan Baker1-17/+16
Currently when the test dictionary is filtered the TestDict instance is replaced by a built-in-dict instance. This isn't ideal for a number of reasons, obviously the silent change in type isn't good. This is also groundwork for other patches in this series. This also fixes a number of unit tests that I'm not exactly sure why they passed before this test. They were doing some invalid things, mainly mangling types. They also happened to be cloning data after destructively editing the data. They are fixed now. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Brian Paul <brianp@vmware.com> Tested-by: Brian Paul <brianp@vmware.com> Tested-by: Gabriel Feceoru <gabriel.feceoru@intel.com>
2016-05-03framework/profile: Convert TestDict to a collections.MutableMappingDylan Baker1-3/+19
MutableMapping is an abstract base class that makes writing dict-like objects easier, by providing some of the boilerplate code. This is the recommended way to make a dict-like object, as opposed to subclassing dict. This is because subclassing built-ins is perilous, sometimes they call their protocol methods (or magic methods), sometimes they don't. This change uncovered such a bug, TestDict.update would silently overwrite other tests when calling update, without there being an code to explicitly do that. This patch additionally fixes the test that trips the bug and adds a test the explicitly cover it. This is groundwork for the next patch. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Brian Paul <brianp@vmware.com> Tested-by: Brian Paul <brianp@vmware.com> Tested-by: Gabriel Feceoru <gabriel.feceoru@intel.com>
2016-04-16unittests: Fix spelling mistakesEric Engestrom14-28/+28
Signed-off-by: Eric Engestrom <eric@engestrom.ch> Reviewed-by: Dylan Baker <baker.dylan.c@gmail.com> Signed-off-by: Vinson Lee <vlee@freedesktop.org>
2016-04-13framework/glslparsertest: Don't add exclude extensions to the required listDylan Baker1-0/+16
glslparsertest has a syntax for the 'require_extensions' option that is "!<extension_name>", when this is specified it means "This extensions is not supported". There is a bug in the fast skip code in glsl_parser_test.py that causes it to add these extensions to the fast skip required extensions list, which is obviously wrong. This patch adds a test for this behavior as well as fixing the behavior to work correctly. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Reviewed-by: Mark Janes <mark.a.janes@intel.com>
2016-04-13unittests/opengl_tests.py: Add some additional testsDylan Baker1-0/+32
These tests verify that the fast-skip code works when random junk is inserted into the output of wflinfo. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
2016-04-13unittests: clean up after the generator testsDylan Baker1-0/+1
This currently leaves a cl and spec folder laying around. That's not good. This patch uses a helper from utils to generate everything in a temporary directory, and then throw that directory away afterwards. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
2016-04-13unittests: move generator tests to unittestsDylan Baker1-0/+94
This is excluded in tox for the non-generator profile. This allows us to share the utils directory easily, and puts *all* of the tests in one directory. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
2016-04-13unittests: Fix tests that leave temporary files laying aroundDylan Baker2-13/+5
These are a bit more complicated, and the solution is more difficult. Basically the problem is that the temporary file gets renamed out from under the test, and the name of the file is a generated and unknown. To get rid of these I've wrapped the shared private method in the utils.test_in_tempdir, and extended that decorator to return a value. This is kind of a hack, and all of this should be fixed by mocking out things rather than using files. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
2016-04-13unittests/utils.py: Don't leave a bunch of temp files aroundDylan Baker1-8/+12
Currently unit tests leave a bunch of temporary files and directories around. This is due to the fact that functions implemented using contextlib.contextmanager don't automatically handle exceptions like context managers implemented as by classes do. Using try/finally solves the problem. There are a couple of temporary directories still left behind, but those are a different problem to be handled by a different patch. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
2016-04-13generators: Add a minimum version helper.Dylan Baker1-0/+174
This adds another GLSL related helper, a factory for getting minimum versions based on stages. This is able to sort all of the stages for both OpenGL and OpenGL ES. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Ilia Mirkin <imirkin@alum.mit.edu>
2016-04-13generators: Add helper classes for GLSL version numbersDylan Baker2-0/+299
This adds a new module in the generated_tests/modules directory, which contains three classes, GLSLVersion, GLSLESVersion, and Version. Version is a factory that caches other versions and makes GLSLVersion and GLSLESVersion instances on demand. The goal of these classes is to provide a simple, unified method for dealing with GLSL version numbers, which is something that a lot of generators need to do. To that end it provides rich comparisons against each other and against ints and floats. The hope is that other generator writers could leverage this work in their generators to simplify things. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Ilia Mirkin <imirkin@alum.mit.edu>
2016-03-15deqp-integration: Handle ResourceError.Dylan Baker1-21/+29
This is a previously unseen error type generated on by the Vulkan CTS on DEQP. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Reviewed-and-Tested-by: Mark Janes <mark.a.janes@intel.com>
2016-02-11python: function with six version 1.5.2Dylan Baker3-6/+32
CMake actually marks that we require six 1.4.0, however, I can't find any packages anywhere for 1.4.0, and the lowest version I've seen requested is 1.5.2. This fixes requirements for working with six 1.5.2, and sets tox to use 1.5.2 (and a suitable version of mock). Primarily there are a few things we're using that are not available: six.moves.getcwd, six.viewvalues, six.python_2_unicode_compatible. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Tested-by: Brian Paul <brianp@vmware.com>
2016-02-08options_tests.py: Fix test that happened to work on python 2Dylan Baker1-2/+2
On python 2 re.compile sets no flags, so asking if the flags bitfield == re.IGNORECASE works. For python 3 this isn't the case (the default it returns for me is 100010), so the test needs to correctly check that re.IGNORECASE is in the bitfield. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08unittests/opengl_tests.py: stop mock leakageDylan Baker1-9/+17
For some reason in python 3 (all versions) some of the mock patchers in this opengl test leak out into other modules. This doesn't happen in python 2. I'm not sure why this fixes the issue, but it does. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08unittests/options_tests.py: fix mocking for python 3.xDylan Baker1-1/+2
By default the builtins don't appear in the python 3 namespace, and create=True must be set to make this work. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08framework/test/glsl_parser_test.py: stop supporting bytesDylan Baker2-8/+8
This makes the GLSLParserTest class only support str (unicode in python 2), and not bytes (str in python 2). Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08log_tests.py: fix incompatible seek for python 3Dylan Baker1-1/+1
In python2 one can seek to negatives (-1 is the end of the file, for example). In python 2.6+ (and exclusively in python 3) one instead uses the second argument of seek to say whether to start at the start or end of the file, and then an exact point. So file.seek(-1) -> file.seek(0, 2) Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08framework/test/base.py: fix compare to None in python 3Dylan Baker1-0/+1
Also fix unittest that relied on compare to None having python 2 behavior. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08framework/results.py: fix StringDescriptor handling of bytesDylan Baker1-7/+7
This actually needs to handle bytes, since it gets assigned from subprocess directly. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08status_tests.py: drop bytes tests, status doesn't support bytes anymoreDylan Baker1-4/+3
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08dmesg.py: Fix str/bytes/unicode for python 2/3 compatibilityDylan Baker1-9/+10
Mostly this involves marking str (unicode) and bytes explicitly and not mixing the two. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08unittest/utils.py: fix tempfile modes for python3Dylan Baker1-2/+5
By default in python 3 write modes for tempfile are bytes, but piglit uses unicode. This fixes that. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08compression.py: use the proper read and write modes for python versionsDylan Baker1-3/+5
In python 2 the write mode should be 'w', and the read 'r', but in python 3 they need to be 'wt', and 'rt' respectively. Python 3 also has an lzma module built in, which vastly simplifies supporting xz. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08opengl.py: fix bytes returned from wflinfo.Dylan Baker1-64/+64
Convert to unicode, this requires reworking the tests to ensure that they provide bytes instances in the right places, or they trip up on the decode method call, which in python 3 is only available on str (unicode), and not on bytes. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08framework,unittests: use __future__ unicode_literalsDylan Baker32-41/+102
Use unicode_literals from __future__. This makes undecorated strings (those not using and b or u prefix) into unicode instead of bytes in python 2. This means that bytes strings need to have a b prefix now. This also fixes a couple of unittests that broke during the transition. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08unittests/results_tests.py: Add traceback testsDylan Baker1-0/+10
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08base_tests.py: Add tests for an exception in Test.executeDylan Baker3-0/+53
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08base_tests.py: make tests for timeout run on python 3.xDylan Baker1-9/+15
These tests are always valid on python 3, but only valid on python 2 if subprocess32 is installed. One unittest on python3 raises a warning. I've tracked it back to somewhere in the python stdlib. I think that for some reason a file is being unlinked twice, but I'm having trouble tracking it down, it works fine under python 2.7, and I think it's safe to ignore. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08unittests: try to import mock from unittestDylan Baker11-11/+55
As of python 3.3 mock is included in python's stdlib. Try to import the included version, then the external package version if that fails. This patch was generated with the following sed command: sed -i -e 's@import mock@try:\n from unittest import mock\nexcept ImportError:\n import mock\n@g' unittests/*py Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08python: use six.{iter,view}{items,keys,values}Dylan Baker5-8/+13
This was initially generated via the following sed command: sed -i \ -e 's@in \(.*\)\.iter\(values,keys,items\)()@in six.iter\2(\1)@g' \ -e 's@in \(.*\..*\)\.iter\(values,keys,items\)()@in six.iter\2(\1)@g' Then cleaned up by hand, including changes for view*. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08python: use six unicode/bytes/str handlingDylan Baker2-1/+4
This is not feature complete for python 3.x in and of itself, but it gets started by using six functions rather than str and unicode. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08python: use six.moves.cStringIODylan Baker2-4/+4
This is the fast C based version on python 2, and is adapative on python3, importing the C based version if it's available and falling back if necessary. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08unittests: use six.moves.getcwdDylan Baker3-5/+8
Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08python: use six.moves.rangeDylan Baker1-1/+2
This is xrange on python 2.x and range on 3.x Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-02-08python: use future print, division, and absolute_importDylan Baker23-23/+23
These are the three python3 like behaviors that piglit should rely on. The only other applicable future import is unicode_literals. Although my plan is to use unicode_literals, that will actually cause behavioral changes in some cases, where these cause minimal changes to the code. Piglit will not be targeting < 3.2, they are old, unsupported, and have fewer features than 2.7. Piglit now has division (using / as floating division, and // as integer division), print as a function (rather than a statement), and absolute import, which changes the way import works when there's a conflict between a local import and a system wide one. Absolute import makes more sense, and copies the behavior of python 3 Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com> Acked-by: Jose Fonseca <jfonseca@vmware.com>
2016-01-21framework/test/base.py: use subprocess32 for timeouts.Dylan Baker1-3/+116
Subprocess32 provides a backport of python 3.2's subprocess module, which has a timeout parameter for Popen.communicate. When the timeout runs out then an exception is raised, and when that exception is caught we can kill the process. This is fairly similar to the way the current timeout mechanism works, except that the current mechanism is not thread safe. Since one of the major features of piglit is that it offer's processes isolated concurrency of tests, it makes sense to make the effort to provide a timeout mechanism that supports concurrency. Unfortunately there isn't a good cross platform mechanism for this in python 2.x, even with subprocess 32 only *nix systems are supported, not windows. The big advantage of this is it allows consistent behavior between python 2.x and 3.x as we look toward the future and the possibility of using a hybrid approach to transition to python 3.x while maintaining 2.x support until it's not needed. This patch look pretty substantial. It's not as big as it looks, since it adds some fairly big tests. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
2016-01-21framework/tests/base_tests.py: use utils helper to simplify testDylan Baker1-12/+8
This just saves some code duplication. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
2016-01-21framework/tests/base_tests.py: fix descriptions of two timeout testsDylan Baker1-2/+2
These tests claimed to test the timeout mechanism, but what they're really testing is the status changing ability of the timeout mechanism. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
2016-01-21framework/tests: add helper for checking for 3rd party modulesDylan Baker2-2/+10
This little helper skips a test if a module isn't available. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
2016-01-08unittests: move framework/tests to unittestsDylan Baker34-0/+8486
Currently the framework is pretty confusing. There's 'test' which is a package containing test classes, then there's 'tests' which contains unit tests for the framework. This is obviously not optimal. Beyond that the unittests get installed, and that isn't really necessary. As a bonus this removes the unit tests from coverage reports, which is a nice plus. Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-24 17:17:30 +0000