summaryrefslogtreecommitdiff
path: root/generated_tests
AgeCommit message (Collapse)AuthorFilesLines
2012-06-13uniform-initializer: Don't rely on Python 2.7José Fonseca1-1/+1
zero-length formats "{}{}" not supported on Python 2.6 Trivial.
2012-06-12Fix gen_uniform_initializer_tests.py for out-of-tree builds.Paul Berry1-2/+9
Previously, the script opened template files relative to the current working directory. This failed for out-of-tree builds because the script is run from within the build tree, not the source tree. This patch fixes the problem by using Python's built-in __file__ variable to determine the path of the source file, and then looking for the template files relative to it. Signed-off-by: Paul Berry <stereotype441@gmail.com>
2012-06-12glsl-1.20 / glsl-1.30: Add tests for uniforms with initializers modified by ↵Ian Romanick4-0/+81
GL API In these tests the uniforms are initialized with one set of values in the shader source, but the values are changed using the GL API. The shaders expect the values set by the GL API. v2: Use '{0}{1}' formatting instead of '{}{}'. The latter is added in Python 2.7, but piglit only requires Python 2.6. This was pointed out by Jordan Justen. v3: Clean up the generation of the if-statement based on feedback from Paul Berry. Signed-off-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-06-12glsl-1.20 / glsl-1.30: Add tests for uniforms with initializers in another stageIan Romanick4-0/+75
These are similar to the first tests except the uniform is initialized in one stage (vertex or fragment) but used in the other stage (vice-versa). v2: Use '{0}{1}' formatting instead of '{}{}'. The latter is added in Python 2.7, but piglit only requires Python 2.6. This was pointed out by Jordan Justen. v3: Clean up the generation of the if-statement based on feedback from Paul Berry. Signed-off-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-06-12glsl-1.20 / glsl-1.30: Add tests for uniforms with initializers from a constantIan Romanick4-0/+75
These tests are similar to the previous tests except the uniforms are initialized using a variable declared as const. Since any compile-time constant expression can be used to initialize a uniform, this should also work. v2: Use '{0}{1}' formatting instead of '{}{}'. The latter is added in Python 2.7, but piglit only requires Python 2.6. This was pointed out by Jordan Justen. v3: Clean up the generation of the if-statement based on feedback from Paul Berry. Signed-off-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-06-12glsl-1.20 / glsl-1.30: Add tests for uniforms with initializersIan Romanick4-1/+264
These first tests simply set the value of an uniform with an initializer in the shader source and verifies that the uniform has that value when the shader is executed. v2: Use '{0}{1}' formatting instead of '{}{}'. The latter is added in Python 2.7, but piglit only requires Python 2.6. This was pointed out by Jordan Justen. v3: Clean up the generation of the if-statement based on feedback from Paul Berry. Also clean up the list types used to generate tests. Signed-off-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-05-31generated_tests: Add positional argument specifiers.Vinson Lee2-4/+4
Fix build failure with Python 2.6 introduced at commit 8460edff15defcfd8e26c87989cf9c961a4b2434. Signed-off-by: Vinson Lee <vlee@freedesktop.org> Reviewed-by: Brian Paul <brianp@vmware.com>
2012-05-22builtin tests: Make glsl_version be a number instead of a string.Eric Anholt3-135/+135
Note that it's not a float number -- we go with the integer like is used in #version.
2012-05-22generated tests: Fix the "#version 110" skipping I broke.Eric Anholt1-1/+1
The float changes I did for a bunch of the >= comparisons were not fully done here, and the comparison was always false, so we got a bunch of extra #version 110 statements.
2012-05-01generated_tests: Add testing for GLSL 1.40 inverse().Eric Anholt1-1/+6
2012-05-01generated_tests: Add support for GLSL 1.40 tests using VBOs.Eric Anholt1-17/+54
2012-03-20glsl-1.30: Add tests for mix(gentype, gentype, genbtype).Eric Anholt1-0/+2
We missed testing this in the GLSL 1.30 work. v2: Fix mistaken generation of mix(vec, vec, bool) variants, and adjust the lambda to Ken's tastes.
2012-01-13glsl-1.10: Verify pre- and post-inc/dec cannot be l-valuesIan Romanick2-1/+168
These generators create two kinds of tests. The first verifies that pre- and post-increment/decrement operations cannot be assignment targets. These look like: uniform float u; varying vec4 v; void main() { float t = u; ++t = float(v.x); gl_FragColor = vec4(t, v.yzw); } The second kind of test verifies that pre- and post-increment/decrement operations cannot be used for function 'out' parameters. These look like: uniform ivec4 u; attribute vec4 v; void f(out ivec4 p) { p = ivec4(v); } void main() { ivec4 t = u; f(t--); gl_Position = vec4(t); } Function 'inout' parameters are not explicitly tested. A test is generated for both vertex and fragment targets for each int and float vector or scalar datatype. Matrix types and GLSL 1.30 unsigned types are not tested. Signed-off-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2011-10-27Add tests of interpolation qualifier behavior.Paul Berry2-1/+359
This patch adds 160 generated tests to verify correct behavior of the GLSL 1.30 interpolation qualifiers under all combinations of the following circumstances: - When the interpolation qualifier is 'flat', 'noperspective', 'smooth', or not specified. - When the variable in question is 'gl_FrontColor', 'gl_BackColor', 'gl_FrontSecondaryColor', 'gl_BackSecondaryColor', or a non-built-in vertex shader output. - When the GL shade model (set with the glShadeModel command) is 'GL_SMOOTH' or 'GL_FLAT'. - When the triangle is clipped using gl_ClipDistance, using gl_ClipVertex, using fixed clip planes, or not clipped at all. Note: in a perfect world, we wouldn't need to test all 160 combinations of these possibilities (e.g. the GL shade model should only have an effect when no interpolation qualifier is specified, and clipping should have no effect whatsoever on interpolation). However, based on what I know about how interpolation currently works in Mesa's i965 driver, it seems prudent to err on the side of caution and test all 160 possible combinations. The tests all operate by painting a triangle on the screen using a frustum projection, where the three vertices are colored red, green, and blue, and each vertex has a different Z coordinate (so that we can test proper perspective interpolation). Then they use "relative probe rgba" on various points in the interior of the triangle to verify that the colors have been interpolated correctly. I've validated the tests using nVidia's proprietary Linux driver, and they all pass except for the tests that attempt to override the default interpolation behavior of one of the built-in gl_Color variables. In those cases, the nVidia driver favors the interpolation mode that is set in GL state. GLSL 1.30 section 4.3.7 ("Interpolation") explicitly states that the interpolation behavior of these variables may be overridden, and that "When an interpolation qualifier is used, it overrides settings established through the OpenGL API." So this seems pretty clearly to be due to a bug in the nVidia driver. As an aid in review, here is an example of one of the tests created by the test generator script. This one is called "interpolation-noperspective-gl_FrontColor-flat-fixed.shader_test": [require] GLSL >= 1.30 [vertex shader] in vec4 vertex; in vec4 input_data; noperspective out vec4 gl_FrontColor; void main() { gl_Position = gl_ModelViewProjectionMatrix * vertex; gl_FrontColor = input_data; } [fragment shader] noperspective in vec4 gl_Color; void main() { gl_FragColor = gl_Color; } [vertex data] vertex/float/3 input_data/float/4 0.0 2.0 -2.0 0.0 1.0 0.0 1.0 -1.0 -1.0 -1.0 1.0 0.0 0.0 1.0 3.0 -3.0 -3.0 0.0 0.0 1.0 1.0 [test] frustum -1.75 1.75 -1.75 1.75 1.75 3.0 clear color 0.0 0.0 0.0 0.0 clear enable GL_VERTEX_PROGRAM_TWO_SIDE shade model flat draw arrays GL_TRIANGLES 0 3 relative probe rgba (0.444444444444, 0.222222222222) (0.0, 0.0, 0.0, 0.0) relative probe rgba (0.636363636364, 0.181818181818) (0.272727272727, 0.181818181818, 0.545454545455, 1.0) relative probe rgba (0.769230769231, 0.153846153846) (0.153846153846, 0.153846153846, 0.692307692308, 1.0) relative probe rgba (0.866666666667, 0.133333333333) (0.0666666666667, 0.133333333333, 0.8, 1.0) relative probe rgba (0.5, 0.4) (0.0, 0.0, 0.0, 0.0) relative probe rgba (0.666666666667, 0.333333333333) (0.166666666667, 0.333333333333, 0.5, 1.0) relative probe rgba (0.785714285714, 0.285714285714) (0.0714285714286, 0.285714285714, 0.642857142857, 1.0) relative probe rgba (0.545454545455, 0.545454545455) (0.181818181818, 0.545454545455, 0.272727272727, 1.0) relative probe rgba (0.692307692308, 0.461538461538) (0.0769230769231, 0.461538461538, 0.461538461538, 1.0) relative probe rgba (0.583333333333, 0.666666666667) (0.0833333333333, 0.666666666667, 0.25, 1.0) Reviewed-by: Eric Anholt <eric@anholt.net>
2011-09-08generated_tests: work around bug in Windows numpy implementation.Paul Berry1-9/+18
The windows implementation of numpy seems to have multiple independent copies of the int32 and uint32 types, as illustrated by this interactive session: Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> x = numpy.int32(5) >>> y = numpy.abs(x) >>> print type(x) <type 'numpy.int32'> >>> print type(y) <type 'numpy.int32'> >>> print type(x) == type(y) False >>> exit() On Linux, the last "print" yields True, which is clearly the correct behavior. This duplication of int32 and uint32 classes was preventing builtin_function.py's glsl_type_of() function from being able to reliably determine the type of its argument, causing an assertion failure while building piglit. This patch works around the problem by creating tuples INT32_TYPES and UINT32_TYPES which contain the duplicate types. These tuples are used instead of referring directly to numpy.int32 and numpy.uint32. On platforms that do not have this bug, the tuples are "oneples" (one element each) so there is no change in behavior. I've verified that on Linux, this change does not affect the generated tests. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=40697 Tested-by: Vinson Lee <vlee@vmware.com>
2011-09-07Fix generated "op-mod" tests to actually test the modulus operator.Paul Berry1-1/+1
Due to a copy-paste bug on my part, the generated "op-mod" tests actually weren't testing the "%" operator at all. They were testing the "/" operator (and were thus identical to the "op-div" tests). This patch fixes the tests to actually test the "%" operator.
2011-09-07generated tests: add large uints division test for GLSL 1.3Christoph Bumiller1-0/+2
At least some of these divisions should fail on implementations that naively use 32 bit float division (without refinement). Reviewed-by: Paul Berry <stereotype441@gmail.com>
2011-08-18generated tests: Add tests for builtin functions introduced in GLSL 1.30Paul Berry1-0/+32
GLSL 1.30 added support for the following new built-in functions: sinh, cosh, tanh, asinh, acosh, atanh, round, roundEven, trunc, modf, isnan, and isinf. This patch adds tests for all of the new built-in functions except modf, isnan, and isinf. modf will need to be tested separately because it has an out parameter; isnan and isinf will need to be tested separately because producing infinite and NaN values will require extra work in the shader. Reviewed-by: Chad Versace <chad@chad-versace.us>
2011-08-18generated tests: Test operators that were introduced in GLSL 1.30Paul Berry1-0/+114
GLSL 1.30 adds support for the following operators: %, ~, <<, >>, &, ^, and |. This patch adds autogenerated tests for testing the functionality of all these operators. Reviewed-by: Chad Versace <chad@chad-versace.us>
2011-08-18generated tests: Add uint and int tests for GLSL 1.30.Paul Berry2-34/+66
GLSL 1.30 allows int and ivecN types to be passed to the functions abs, sign, min, max, and clamp. In addition, it adds new unsigned integer types uint and uvecN, which may be passed to min, max, clamp, lessThan, lessThanEqual, greaterThan, greaterThanEqual, equal, notEqual, and built-in operators. This patch adds auto-generated tests for all of these new integral built-in functions and operations. Reviewed-by: Chad Versace <chad@chad-versace.us>
2011-08-18generated tests: Add builtin tests using if.Paul Berry1-17/+110
For builtin functions and operators returning bool, added generated tests that use the return values in if statements like this: if (a < b) gl_FragColor = vec4(1.0, 1.0, 0.0, 1.0); else gl_FragColor = vec4(0.0, 0.0, 1.0, 1.0); in addition to the existing tests that convert the return values to floats, like this: bool result = a < b; gl_FragColor = vec4(result, 0.0, 0.0, 0.0); Since implementations may produce substantially different GPU code for these two cases. Reviewed-by: Chad Versace <chad@chad-versace.us>
2011-08-18generated tests: Added tests for the 'notEqual' builtin.Paul Berry1-0/+1
This function has been available since as early GLSL 1.10, it was simply forgotten from the initial implementation of generated tests. Reviewed-by: Chad Versace <chad@chad-versace.us>
2011-08-18generated tests: Expand the range of ints and ivecs used in testing.Paul Berry1-2/+6
This patch adds negative and zero values to the set of ints used to test lessThan, lessThanEqual, greaterThan, greaterThanEqual, and equal. It also adds ivecs whose elements are all positive to the tests for operators--these will come in handy when we add tests for %, which is not defined for negative inputs. Reviewed-by: Chad Versace <chad@chad-versace.us>
2011-08-18generated tests: Make test generation happen in a deterministic order.Paul Berry2-2/+2
This patch makes gen_builtin_uniform_tests.py and gen_constant_array_size_tests.py build tests in a deterministic order, so that when diffing their outputs from one revision to the next, it is easier to see what changed. There is no change in the actual tests generated, just in the *.list files that are created as a side effect. Reviewed-by: Chad Versace <chad@chad-versace.us>
2011-08-15Add operator tests to auto-generated built-in function tests.Paul Berry3-69/+266
This patch adds tests for the following built-in GLSL operators: - op-add, op-sub, op-mult, op-div: binary arithemetic operators on floats, vecs, mats, ints, and ivecs. - op-uplus, op-neg: unary arithmetic operators on floats, vecs, mats, ints, and ivecs. - op-gt, op-lt, op-ge, op-le: comparison operators on ints and floats. - op-eq, op-ne: equality and inequality comparison on any GLSL 1.20 type. - op-and, op-or, op-xor: logical operations on bools. - op-not: unary logical not on bools. - op-selection: trinary selection operator (x?y:z), where the first argument is a bool and the second and third arguments are any GLSL 1.20 type. Note that implicit type conversions are not tested. So, for instance, int * ivec is tested, but float * ivec is not. This was in an effort to avoid generating an outrageous number of tests. Note also that the "shortcut" behavior of logical and/or and trinary selection is not tested. These tests leverage the same test generation framework used to test built-in functions, so the tests exercise vertex shaders, fragment shaders, and constant folding. All in all 1332 tests are added, in the subtrees spec/glsl-1.{10,20}/{compiler,execution}/built-in-functions. Reviewed-by: Eric Anholt <eric@anholt.net>
2011-08-11Add constant array size builtin tests.Paul Berry2-1/+237
This patch adds a new set of generated tests which parallels those introduced by commt cccc419e (Add comprehensive tests of builtin functions with uniform input.) The new tests validate the correctness of built-in functions when used to compute array sizes. The tests work by creating arrays whose size is 1 if the result of applying the built-in function is within tolerance of the correct answer, and -1 if not. Since negative array sizes are prohibited, any improperly computed values will generate a compile error. Though it may seem absurd to compute an array size based on the result of calling a built-in function, it's worth testing for two reasons: (1) It is explicitly allowed in GLSL versions since 1.20. (2) Since array sizes need to be computed at compile time, this is a convenient way of testing that the GLSL compiler properly implements constant folding of built-in functions. Indeed, the original motivation for these tests was to validate bug fixes to Mesa's constant folding logic. Here is an example of one of the generated tests (this test is exp2-vec2.vert): /* [config] * expect_result: pass * glsl_version: 1.20 * [end config] * * Check that the following test vectors are constant folded correctly: * exp2(vec2(-2.0, -0.66666669)) => vec2(0.25, 0.62996054) * exp2(vec2(0.66666669, 2.0)) => vec2(1.587401, 4.0) */ void main() { float[distance(exp2(vec2(-2.0, -0.66666669)), vec2(0.25, 0.62996054)) <= 6.7775386e-06 ? 1 : -1] array0; float[distance(exp2(vec2(0.66666669, 2.0)), vec2(1.587401, 4.0)) <= 4.3034684e-05 ? 1 : -1] array1; gl_Position = vec4(array0.length() + array1.length()); }
2011-08-11Refactor cmake script to generate tests for later reuse.Paul Berry1-9/+27
This patch creates a function, piglit_make_generated_tests (in generated_tests/CMakeLists.txt), to encapsulate the logic that builds a custom command. The new function can be reused to generate tests using other python scripts.
2011-08-11Increase the range of values used to test atan(x) and atan(x,y)Paul Berry1-2/+8
This patch ensures that Piglit tests both atan(x) and atan(x,y) for both very small and very large inputs. Previously, Piglit only tested the atan() functions over a small range of input values, causing it to miss a bug in Mesa (which has since been fixed--see Mesa commit b1b4ea0b).
2011-08-10Generated tests: unify code for testing exp2 across all numpy versions.Paul Berry1-5/+5
Versions of numpy previous to 1.3.0 (such as the one that ships with Snow Leopard, which is 1.2.1) didn't include an exp2() function, so commit a5180ddc worked around the problem by emulating it using power(2, x) when the numpy version was less than 1.3.0. There is no real advantage to using exp2() over power(2, x) (other than to save a few keystrokes), but there is a maintenance advantage to being as consistent as possible across numpy versions. So this patch changes Piglit to use power(2, x) regardless of the numpy version.
2011-08-10Fix autogeneration of builtin uniform tests on Windows.Paul Berry2-18/+7
This patch modifies the invocation of CMake's add_custom_command() function so that instead of specifying all 462 output files explicitly, it instead specifies a single file, builtin_uniform_tests.list, which receives a list of all the files generated. It is safe not to specify the individual output files since there are no other build steps that depend on them, so there is no danger of the build step happening at the wrong time. This prevents CMake from trying to output an outrageously long diagnostic message listing all the files generated by the custom command, a message that was too long to work with the NMake back-end used on Windows. Bugzilla:https://bugs.freedesktop.org/show_bug.cgi?id=39873
2011-08-09Fix test generation on Arch Linux.Paul Berry1-2/+2
On Arch Linux, "python" means Python 3.x, and Python 2.x is called "python2". The test generation code requires Python 2.x. In many other distributions, "python" refers to Python 2.x. This patch adds logic to the toplevel CMakeLists that checks both "python2" and "python" to see which one is Python 2.x, and once it is found, sets the cmake variable ${python} to point to it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-05cmake: Fix build with CMake 2.6.Vinson Lee1-2/+2
2011-08-05generated_tests: Add numpy.exp2 workaround for versions below 1.3.0.Vinson Lee1-1/+5
numpy.exp2 became available starting with numpy-1.3.0. Mac OS X 10.6.8 (Snow Leopard) has numpy-1.2.1.
2011-08-05Add tolerance to auto-generated built-in function tests.Paul Berry2-78/+329
Previously, the auto-generated tests converted their outputs to pixel values and tested them against the expected values using shader_runner's "probe rgba" command--as a result, the built-in functions only tested to a tolerance of 1 part in 255. This patch changes the auto-generated tests so that the expected value is checked inside the shader itself, to an explicit tolerance. Unfortunately, the GLSL and OpenGL specs are somewhat ambiguous as to how accurate the built-in functions need to be. Section 2.1.1 of the OpenGL 2.1 spec, for instance, says that "individual results of floating point operations are accurate to about 1 part in 10^5". However, it's not clear whether a built-in function is intended to constitute a single "operation" in this context. And in experimenting with the systems available to me (Mesa on Intel i965, both IronLake and SandyBridge, and an nVidia system running nVidia's proprietary Linux driver), I've found that trig functions in particular fail to meet this strict requirement. Considering how trig functions are typically used in shaders (e.g. calculating lighting angles), it seems like 1 part in 10^5 is an unreasonably tight limit. So I've settled for the time being on the following compromise: - Trig functions are tested to a tolerance of 1 part in 10^3 relative to the output of the built-in function, or an absolute tolerance of 10^-4, whichever is larger. - The cross product is tested to a tolerance of 1 part in 10^5, relative to the product of the magnitudes of the input vectors. This avoids an unreasonably tight tolerance in cases where the terms of the cross product cancel out, yielding a small result. - All other functions are tested to a tolerance of 1 part in 10^5 relative to the output of the built-in function. To avoid additional sources of error due to floating-point conversions, all test vectors are generated as 32-bit floating-point values. As an aid in review, here is the generated test for the exp() built-in: [require] GLSL >= 1.10 [vertex shader] varying vec4 color; uniform float arg0; uniform float tolerance; uniform float expected; void main() { gl_Position = gl_Vertex; float result = exp(arg0); color = distance(result, expected) <= tolerance ? vec4(0.0, 1.0, 0.0, 1.0) : vec4(1.0, 0.0, 0.0, 1.0); } [fragment shader] varying vec4 color; void main() { gl_FragColor = color; } [test] uniform float arg0 -2.0 uniform float expected 0.13533528 uniform float tolerance 1.3533528e-06 draw rect -1 -1 2 2 probe rgba 0 0 0.0 1.0 0.0 1.0 uniform float arg0 -0.66666669 uniform float expected 0.51341712 uniform float tolerance 5.1341713e-06 draw rect -1 -1 2 2 probe rgba 1 0 0.0 1.0 0.0 1.0 uniform float arg0 0.66666669 uniform float expected 1.9477341 uniform float tolerance 1.9477342e-05 draw rect -1 -1 2 2 probe rgba 2 0 0.0 1.0 0.0 1.0 uniform float arg0 2.0 uniform float expected 7.3890562 uniform float tolerance 7.3890566e-05 draw rect -1 -1 2 2 probe rgba 3 0 0.0 1.0 0.0 1.0
2011-08-05Add comprehensive tests of builtin functions with uniform input.Paul Berry4-0/+1014
This patch adds 462 tests to piglit, which comprehensively test the behavior of GLSL built-in functions on vertex and fragment shaders. The test vectors are auto-generated in Python, by using the numpy library to simulate the behavior of the built-in functions on a variety of inputs (in builtin_function.py), and then producing shader_runner tests to verify the expected behavior (in gen_builtin_uniform_tests.py). Test generation happens as part of the piglit build process. To run the test generation step by itself, use "make gen-tests" from the root of the piglit tree (after configuring using cmake). The generated tests may be found in generated_tests/spec. The auto-generation code depends on Python 2.6 and on the numpy library (www.numpy.org). I've updated the README to reflect these dependencies, and added code to CMakeLists.txt to check for them at configure time. (Note: other parts of Piglit already depend on Python 2.6, but the dependency wasn't previously explicit).