summaryrefslogtreecommitdiff
path: root/framework
AgeCommit message (Collapse)AuthorFilesLines
2018-11-09framework: Add support for serializing VkRunner testsNeil Roberts1-1/+3
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2018-11-09framework/test: Use relative paths in the filename for VkRunnerTestNeil Roberts1-4/+4
Previously the filename was the full path to the test. This would break if the filename was serialized and installed to a different location. The code is based on what is done for ASMParserTest. However I have a suspicion that this won’t work for running generated tests from an out-of-tree build. In that case the path no longer contains the path to the build directory. If this really is a problem it will need to be fixed for ASMParserTest too so it could be done in a separate patch. In the meantime there aren’t any generated VkRunner tests yet so it’s not really a problem. Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2018-11-09framework/test: Set the filename property on VkRunnerTestNeil Roberts1-0/+2
This will be retrieved when generating the XML profile for the VkRunner tests. Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2018-11-08python: add explicit utf-8 encodingEric Engestrom39-0/+39
I went with `# coding=utf-8` as it was the most common format in the existing code: 24 # coding=utf-8 21 ## coding=utf-8 19 # encoding=utf-8 9 ## encoding=utf-8 6 # -*- coding: utf-8 -*- I added it as the first line after the shebang, if any. Suggested-by: Dylan Baker <dylan@pnwbakers.com> Signed-off-by: Eric Engestrom <eric@engestrom.ch> Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2018-11-08framework: fix python2Eric Engestrom1-1/+1
The non-ascii char broke the python2 CI: https://travis-ci.org/mesa3d/piglit/jobs/452170869 Fixes: 221d924689e3bb2796ce "framework: Add a vulkan tests profile" Cc: Neil Roberts <nroberts@igalia.com> Cc: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Signed-off-by: Eric Engestrom <eric@engestrom.ch> Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2018-11-07framework: Add a vulkan tests profileNeil Roberts1-0/+23
This searches for files named *.vk_shader_test in the tests/vulkan directory and runs them with VkRunner. VkRunner is executed as an external dependency. It is found either with the vkrunner:bin config option, by setting the PIGLIT_VKRUNNER_BINARY environment variable, or just in the search path. v2: Move VkShaderTest to piglit_test.py and rename to VkRunnerTest. Add future imports. Remove unused import. v3: Support the PIGLIT_VKRUNNER_BINARY variable to specify the location of VkRunner. v4: Add documentation to the README. Add an option in piglit.conf to set the binary location. (Suggested by Samuel Iglesias) Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2018-09-20framework/wflinfo: delete deprecated interfaceDylan Baker2-52/+0
This is the old interface which is now not depended on by any tests. v2: - remove shader_test interfaces as well
2018-09-20framework/test: Convert tests to new fast-skip interfaceDylan Baker3-173/+150
This converts the two test types that use the fast-skipping mechanism (glsl parser and shader) to use the new interfaces. This has been verified by running with PIGLIT_NO_FAST_SKIP=1 and without, and sans 1 test (that is fixed later in the series) the results are the same.
2018-09-20wflinfo: Reimplement wflinfo separating the various API'sDylan Baker1-139/+181
Currently fast skipping is implemented such that it assumes there's a single version of ES, a single version of desktop, and all extensions are shared between them. This has basically worked because 1) there are basically no gles1 tests, and 2) piglit didn't have compat profile. But worked and correct are two different things. With the addition of compat profiles it's time to re-evaluate how fast skipping works. Namely we need to have different attributes for ES1, ES1+, core, compat, and I've added on for "legacy" (pre-profile), since waffle supports that. This maintains legacy interfaces so that existing code continues to work. v2: - Fix versions < 3.1 on implementations without core profile
2018-09-20framework/shader_test: GL_NUM_PROGRAM_BINARY_FORMATS >= 1 is not an extensionDylan Baker1-2/+4
So don't treat it like one. Stops one test from being fast-skipped that shouldn't be.
2018-09-20framework/test: Fix whitespaceDylan Baker1-0/+1
2018-08-21framework: add --glsl optionAlejandro Piñeiro3-2/+15
So when executing shader tests, they will be executed with -glsl. This is the force GLSL mode, that is only relevant if the shader test includes SPIR-V shaders. So for example: ./piglit run tests/shader.py -t ARB_gl_spirv --glsl results/results Will try to run all the shader tests for ARB_gl_spirv using GLSL instead of the default (for that extension) SPIR-V. Acked-by: Timothy Arceri <tarceri@itsqueeze.com>
2018-08-07framework/test/shader_test.py: Remove redundant workaround for python 3.3 mockRhys Kidd1-3/+1
As python 3.3 is deprecated and no longer supported by piglit, remove redundant workaround for lack of readlines() support with mock in python 3.3. Signed-off-by: Rhys Kidd <rhyskidd@gmail.com> Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2018-07-24summary/html: generalize info outputDylan Baker1-4/+1
So that random keys show up somewhat nicely.
2018-07-24JSON: move info utilities out of rootDylan Baker4-14/+23
And into a generic sub element in the output. This makes information like glxinfo and clinfo not a hard requirement and will allow any sort of system information to be encoded.
2018-07-20framework: deepcopy TestProfile's filters when copying itAndres Gomez1-1/+4
This avoids modifications in the original filters object. Detected through Travis CI. Fixed also a couple of typos. Fixes: 1f0f2ad25 ("framework: Add filters class") Cc: Dylan Baker <dylanx.c.baker@intel.com> Cc: Michel Dänzer <michel.daenzer@amd.com> Signed-off-by: Andres Gomez <agomez@igalia.com> Reviewed-by: Dylan Baker <dylan@pnwbakers.com> Reviewed-by: Rhys Kidd <rhyskidd@gmail.com>
2018-07-18framework: Make error about mixed GLES and GL more imformativeDylan Baker1-1/+2
This just adds a helpful message about what directory the conflicting shaders are in.
2018-06-29framework: add a --timeout parameterDylan Baker1-0/+11
This allows an timeout (in integers) to be set for each test. If the tests run over this alotted time they'll be killed and the status will be set to timeout. This only work on Unix-like machines running with python 3.x or with 2.7 and subprocess32 installed. It can be made to work for windows, either by bumping the python version or by writing some windows specific code for killing processes. v2: - Add the word "Default" to help message. Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2018-06-22deqp: Add Mesa version overrides for khr_gl* testlist generation.Eric Anholt1-1/+5
The runner will assertion fail if it can't create an appropriate context, so just force the version so that we can get our testlists generated. Acked-by: Dylan Baker <dylan@pnwbakers.com>
2018-05-30framework/resume: properly parse the no_retry optionNicolai Hähnle1-1/+4
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2018-05-30framework: allow specifying the number of jobs for concurrencyNicolai Hähnle3-4/+25
The default remains the same: number of CPUs. But on systems with lots of cores but comparatively little (V)RAM it can make sense to reduce the number of jobs to avoid random failures caused by out-of-memory conditions. Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2018-05-30framework/html: guard against errors writing individual test resultsNicolai Hähnle1-7/+14
v2: check ENAMETOOLONG Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2018-05-24glslparsertest: support compatibility profile testsMarek Olšák1-0/+5
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
2018-05-16profile: add the idea of process-isolated xml fileDylan Baker1-1/+16
This adds a new name.no_process.xml (or .meta.xml) file that is loaded when --process-isolation=false is used. Tested-by: Michel Dänzer <michel.daenzer@amd.com>
2018-05-09framework: make use of Filter.run in Profile classesDylan Baker1-16/+21
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
2018-05-09framework: Add filters classDylan Baker1-3/+39
This class works like a list, but with an extra method that allows it to "reset" a filter. This will allow class based filters that need to be reset to have deterministic behavior to do so. Tested-by: Michel Dänzer <michel.daenzer@amd.com>
2018-05-09framework: ensure that all tests are run before exitingDylan Baker1-2/+2
This is a bug that has existed for a long time, and I'm not really sure how we haven't hit this before honestly, I think that the reason this has worked is that we've always provided a concrete type to to pool.imap. The problem is that pool.imap is an iterator. If you don't walk the results then the program might exit before finishing. This seems to be especially true when running without -c or -1. Instead we'll write the loop, using pool.apply_async does basically the same thing (without the ability to use chunks), but with out exiting early. Tested-by: Michel Dänzer <michel.daenzer@amd.com>
2018-05-09framework: fix running with mixed concurrency (neither -c or -1)Dylan Baker1-2/+5
test_list is an iterator, you can't walk an iterator more than once. To solve this we use itertools.tee, which creates a shared buffer for the iterators to draw from. This fixes errors like: [000/480] Traceback (most recent call last): File "./piglit", line 178, in <module> main() File "./piglit", line 174, in main sys.exit(runner(args)) File "/home/user/piglit/framework/exceptions.py", line 51, in _inner func(*args, **kwargs) File "/home/user/piglit/framework/programs/run.py", line 370, in run backend.finalize({'time_elapsed': time_elapsed.to_json()}) File "/home/user/piglit/framework/backends/json.py", line 163, in finalize assert data['tests'] AssertionError Thanks Tomi for figuring out what was wrong with the original patch! CC: Tomi Sarvela <tomi.p.sarvela@intel.com> Tested-by: Michel Dänzer <michel.daenzer@amd.com> Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
2018-05-02framework: handle UnicodeDecodeErrorMarek Olšák1-0/+3
This happens due to LLVM printing colored text into stdout/stderr on error. Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2018-05-02framework: fix grouptools.commonprefix when len(args) == 1Nicolai Hähnle1-1/+1
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2018-05-01profile: use gz to compress profilesDylan Baker1-5/+5
This results in substantially smaller profiles and doesn't seem to affect runtime. v2: - install xml and xml.gz files. This is needed so that meta profiles will be installed. Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01fix glslparser test for out of tree buildsDylan Baker1-6/+13
Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01tests/glsl_parser_test.py: fix is_skip for serialized profilesDylan Baker1-10/+10
Currently is_skip() relies on runtime detection of which glslparsertest binaries are built, but we can't assume that at build time. Instead always assign the appropriate binary, and then check for the existence of that binary at run time. Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01shader_tests: correctly generate xml during out of tree buildsDylan Baker1-4/+10
Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework/profile: Add support for meta profilesDylan Baker1-2/+62
MetaProfiles are a single profile composed of more than one profile. This is designed to allow the old all-inclusive names to continue to exist without having to keep duplicates of each profile around. A metaprofile simply declares that it needs profiles a, b, and c and then runs those. This allows an "all" meta profile to include opengl, shader, and glslparser, for example. A metaprofile is allowed to include XML or python based profiles. Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01profile: allow forcing python or xml loadingDylan Baker1-6/+15
This will be used during build time to allow profiles that modify other profiles to load XML instead of rebuilding. Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01profile: Add support for loading xml based profilesDylan Baker1-0/+108
Which includes de-serialization on demand. This will vastly reduce the amount of resident memory that is necessary to hold the tests since test instances are made on demand, and can be garbage collected as soon as the run in complete. Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework/test: make CLProgramTester take relative pathsDylan Baker1-1/+1
Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework/test: make BuiltInConstantTest files relativeDylan Baker1-0/+5
Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework/test: use relative paths for GLSLParser testsDylan Baker1-3/+10
Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework/test: Make shader paths relativeDylan Baker1-6/+10
Because they need to be relative at build time for serialization. Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework/test: Add class for cl-program-testerDylan Baker1-10/+25
These tests need some special handling for serializing, since they take file paths as arguments. Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework/test/piglit_test: make cl_concurrency always a booleanDylan Baker1-2/+2
Just a nice cleanup. Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework/test: Split multishader tooDylan Baker1-20/+43
Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework: do the same for shader testDylan Baker1-20/+26
Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework: use a class method for building test via parsingDylan Baker1-3/+9
Since the serialized tests wont need this path, build a generic constructor that this special constructor can leverage. Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework/test: add a test class for built-in constantsDylan Baker1-0/+5
This has the same problem as asmparser tests. Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework: Use custom class for ASM parser testsDylan Baker1-0/+14
This is going to make serialization easier, since asmparser tests take a file argument, which must be relative at build time, but absolute at run time. Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework/profile: Add a __len__ method to TestProfileDylan Baker1-7/+7
This exposes a standard interface for getting the number of tests in a profile, which is itself nice. It will also allow us to encapsulate the differences between the various profiles added in this series. Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
2018-05-01framework/test: expose required and excluded platformsDylan Baker1-6/+6
These need to be exposed for serialization. Tested-by: Rafael Antognolli <rafael.antognolli@intel.com>
generated by cgit v1.2.3 (git 2.39.1) at 2024-10-16 10:32:38 +0000