XorgTesting.mdwn


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

# Using the X Test Suite

The X Test Suite is the test suite for core rendering in the X Server.  For other X testing tools, see [[http://www.freedesktop.org/wiki/Software/XTesting/]]

## Build xts5

This is the tree with all the actual test code.  It has a framework for running tests and comparing results, but I don't recommend using them.  You can learn more about some of those tools in the README if you want.

    # git clone https://gitlab.freedesktop.org/xorg/test/xts.git
    # cd xts
    # ./autogen.sh
    # make
    # cd ..

## Build xtsttopng

This tool takes the output logs of tests that generate image comparisons, remaps the pixel values to visibly different colors, and generates pngs for the two (-0 being the rendered image and -1 being the reference image).

Note that these images don't necessarily represent what was *actually* rendered, since the reference image is generated automatically, and for tests that don't care about some area of the drawable, it'll smash the areas it doesn't care about to a standard value in both the rendered and reference image.

    # git clone https://gitlab.freedesktop.org/xorg/test/xtsttopng.git
    # cd xtsttopng
    # ./autogen.sh
    # make
    # sudo make install

## Get piglit

    # git clone https://gitlab.freedesktop.org/mesa/piglit.git
    # cd piglit
    # cp piglit.conf.example piglit.conf
    # $EDITOR piglit.conf

And set [xts] path=~/source/xts (or wherever you download the xts repository)

Now, here's an example of using this to do some tests comparing two rendering modes in Xephyr.  Xlib9 is the set of core rendering tests (not counting the window movement code, which also does rendering and is covered in another chapter).

    # Xephyr :5 -noreset -ac -screen 1024x768 &
    # DISPLAY=:5 ./piglit-run.py -t Xlib9 tests/xts.py results/xts
    # killall Xephyr
    # Xephyr :5 -noreset -ac -screen 1024x768 -glamor &
    # DISPLAY=:5 ./piglit-run.py -t Xlib9 tests/xts.py results/xts-glamor
    # ./piglit summary html summary/mysum results/xts results/xts-glamor
    # firefox summary/mysum/index.html

Take a look at changes.html for a common way of looking at piglit results as a developer (I don't care if things are still broken, I want to know if I broke or fixed new things).