1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
|
XTS(miscmansuffix)
==================
Name
----
XTS - The X Test Suite
Description
-----------
This is a revamped version of X Test Suite (XTS) which removes some of
the ugliness of building and running the tests.
Downloading
-----------
Currently xts is only available from a git repository:
$ git clone https://gitlab.freedesktop.org/xorg/test/xts.git
Eventually there will be a tarball release when it is stabilized.
Requirements
------------
Required libraries: libX11, libXau, libXext, libXi, libXtst
Required utilities: xdpyinfo, xset, perl, bdftopcf, mkfontdir
If you are using packages outside of the standard system directories,
make sure to set PATH and PKG_CONFIG_PATH appropriately. See
pkg-config(1) for details on the latter.
Build and Install
-----------------
The build and install steps are now straightforward:
$ ./autogen.sh # using a git checkout
$ ./configure # using a tarball
$ make
$ make install
It is not necessary to install the suite. Tests may be run from the build
directories.
Running the Tests
-----------------
The tests can be run from the build directory or from the installed
package. The check.sh script sets up the environment to run tests from
the build tree.
$ ./check.sh # run all the tests
$ ./check.sh XI # run just the XI test scenario
The "make run-tests" framework can also be used from the build tree.
$ make run-tests # run all the tests
$ make run-tests scenario=XI # run just the XI test scenario
$ make test-XI # shortcut for the previous
A wrapper script, xts-run, is the easiest way to run the installed tests.
$ xts-run # run all the tests
$ xts-run XI # run just the XI test scenario
By default, the test output will go to xts-results/<date> in the current
directory. This can be changed with the xts-run -o parameter. The output
contains the full journal of all test output plus individual logs for
each test program. Journal files from completed runs can be analyzed by
the xts-report tool (formerly vswrpt):
$ xts-report -f /xts/output/dir/journal
Running a Single Test
---------------------
The pt script can be used to run a single set of tests or a single test case
only. For example, to run only the XGrabButton tests, run
$ pt ./XGrabButton
The logs for the test are saved in a file <testcase>.log, for the example
above the log file would be ./XGrabButton.log.
The pt script sets up some environment variables, based on TET_ROOT and
other settings. To start the debugger automatically after the environment is
set up, run:
$ pt -gdb ./XGrabButton
To run only a single test-case inside one test set, use the -i flag.
$ pt -i 10 ./XGrabButton
This command only runs test case 10 (t010() in the source).
Bugs
----
Some tests provide different results when run as a set versus run
one-by-one with the pt script. This has been observed with
Xlib13/XGrabButton where the modifier state is not reset correctly.
When debugging tests, it is recommended that failing tests are run
separately to verify the result.
XTS was only recently resurrected after a period of neglect. Tests have not
been verified. At this point, XTS results should only be used to notice
differences between server versions. A failing test may not necessarily
indicate a bug in the server and a successful test may not necessarily
indicate correct functioning of the server.
See Also
--------
https://www.x.org/wiki/BuildingXtest/
|
