deqp: (V5)Add option to run subset of external dEQP-GLES3

Google have already added a subset of dEQP into Android CTS test,
and we believe this part of dEQP have higher priority than the rest
of dEQP test cases.
The case list is stored at some xml files. Such as:
com.drawelements.deqp.gles3.xml.
It's git repo lives in the Android tree at
[https://android.googlesource.com/platform/external/ \
deqp/+/master/android/cts/com.drawelements.deqp.gles3.xml]

This patch is based on Chad's patch(Add test profile for external
dEQP-GLES3 tests), and add an option to run the subset of dEQP-GLES3.
The only differnce for the running method is: you need to set the
environment variable PIGLIT_DEQP_MUSTPASS or the piglit.conf
option deqp-gles3.mustpasslist,then it will run the subset of dEQP
follow the test case list. If not set, it will still run the whole
dEQP.

Tested on Intel HSW. There are 45866 test cases for the whole dEQP
and 37354 test cases for the subset of dEQP using Android CTS test
case list.

V2:
1.Update the link of get test case list file.
2.Using xml.etree.cElementTree module instead of re module.
3.Fix some Piglit style issue

V3:
1.Update the output issue.
2.Fix some Piglit style issue.

V4:
1.Fix some Piglit style issue.

V5:
1.Fix some Piglit style issue.

v6 (Dylan):
- Remove trailing whitespace

Signed-off-by: Wang Shuo <shuo.wang@intel.com>
Reviewed-by: Dylan Baker <dylanx.c.baker@intel.com>

2 files changed, 41 insertions, 0 deletions

diff --git a/piglit.conf.example b/piglit.conf.example
index b3869c273..c30eef417 100644
--- a/piglit.conf.example
+++ b/piglit.conf.example
@@ -44,6 +44,12 @@ testB
 ; option is not required. The environment variable PIGLIT_DEQP_GLES3_EXTRA_ARGS
 ; overrides the value set here.
 ;extra_args=--deqp-visibility hidden
+;
+; Path to the test case list of CTS for deqp-gles3. You can also set this with
+; the environment variable PIGLIT_DEQP_MUSTPASS. Piglit will run the subset of
+; dEQP-GLES3 tests if and only if this option is set.
+;mustpasslist= \
+; /android/platform/external/deqp/android/cts/com.drawelements.deqp.gles3.xml
 
 ; Section for specific oclconform test.  One of these sections is required for
 ; each test list in the oclconform section and must be called:
diff --git a/tests/deqp_gles3.py b/tests/deqp_gles3.py
index b1bb33b73..9b048f5a0 100644
--- a/tests/deqp_gles3.py
+++ b/tests/deqp_gles3.py
@@ -21,6 +21,7 @@
 import ConfigParser
 import os
 import subprocess
+import xml.etree.cElementTree as ET
 
 # Piglit modules
 import framework
@@ -28,6 +29,34 @@ from framework.profile import Test, TestProfile
 
 __all__ = ['profile']
 
+
+def get_test_case(root, root_group, outputfile):
+    """Parser the test case list of Google Android CTS,
+    and store the test case list to dEQP-GLES3-cases.txt
+    """
+    for child in root:
+        root_group.append(child.get('name'))
+        if child.tag == "Test":
+            outputfile.write('TEST: {}\n'.format('.'.join(root_group)))
+            del root_group[-1]
+        else:
+            get_test_case(child, root_group, outputfile)
+            del root_group[-1]
+
+
+def load_test_hierarchy(input, output):
+    """Google have added a subset of dEQP to CTS test, the case list is stored
+    at some xml files. Such as: com.drawelements.deqp.gles3.xml This function
+    is used to parser the file, and generate a new dEQP-GLES3-cases.txt which
+    only contain the subset of dEQP.
+    """
+    tree = ET.parse(input)
+    root = tree.getroot()
+    root_group = []
+    with open(output, 'w') as f:
+        get_test_case(root, root_group, f)
+
+
 def get_option(env_varname, config_option):
     """Query the given environment variable and then piglit.conf for the
     option. Return None if the option is unset.
@@ -50,6 +79,10 @@ def get_option(env_varname, config_option):
 # Path to the deqp-gles3 executable.
 DEQP_GLES3_EXE = get_option('PIGLIT_DEQP_GLES3_EXE', ('deqp-gles3', 'exe'))
 
+# Path to the xml file which contained the case list of the subset of dEQP
+# and marked as must pass in CTS.
+DEQP_MUSTPASS = get_option('PIGLIT_DEQP_MUSTPASS', ('deqp-gles3', 'mustpasslist'))
+
 DEQP_GLES3_EXTRA_ARGS = get_option('PIGLIT_DEQP_GLES3_EXTRA_ARGS',
                                    ('deqp-gles3', 'extra_args'))
 
@@ -72,6 +105,8 @@ def gen_caselist_txt():
     subprocess.check_call(
         [DEQP_GLES3_EXE, '--deqp-runmode=txt-caselist'],
         cwd=basedir)
+    if DEQP_MUSTPASS is not None:
+        load_test_hierarchy(DEQP_MUSTPASS, caselist_path)
     assert os.path.exists(caselist_path)
     return caselist_path