From 402349fd7e464b18882e531f05668125a4a9d73b Mon Sep 17 00:00:00 2001 From: Allen Akin Date: Wed, 15 Dec 1999 01:24:19 +0000 Subject: Initial revision --- doc/html/build.html | 72 ++++++++++++++ doc/html/buildgnu.html | 156 +++++++++++++++++++++++++++++++ doc/html/buildwin.html | 86 +++++++++++++++++ doc/html/changes.html | 141 ++++++++++++++++++++++++++++ doc/html/cleanup.html | 59 ++++++++++++ doc/html/index.html | 35 +++++++ doc/html/newfeat.html | 107 +++++++++++++++++++++ doc/html/newtest.html | 198 +++++++++++++++++++++++++++++++++++++++ doc/html/next.html | 60 ++++++++++++ doc/html/overview.html | 247 ++++++++++++++++++++++++++++++++++++++++++++++++ doc/html/port.html | 57 ++++++++++++ doc/html/repo.html | 48 ++++++++++ doc/html/run.html | 248 +++++++++++++++++++++++++++++++++++++++++++++++++ doc/html/whatis.html | 102 ++++++++++++++++++++ 14 files changed, 1616 insertions(+) create mode 100644 doc/html/build.html create mode 100644 doc/html/buildgnu.html create mode 100644 doc/html/buildwin.html create mode 100644 doc/html/changes.html create mode 100644 doc/html/cleanup.html create mode 100644 doc/html/index.html create mode 100644 doc/html/newfeat.html create mode 100644 doc/html/newtest.html create mode 100644 doc/html/next.html create mode 100644 doc/html/overview.html create mode 100644 doc/html/port.html create mode 100644 doc/html/repo.html create mode 100644 doc/html/run.html create mode 100644 doc/html/whatis.html (limited to 'doc') diff --git a/doc/html/build.html b/doc/html/build.html new file mode 100644 index 0000000..dbfbbf7 --- /dev/null +++ b/doc/html/build.html @@ -0,0 +1,72 @@ + + + + + How to build glean + + + + + + + +

How do I build glean?

+ +

+glean is intended to run on a variety of operating systems. +It has been tested on Linux-based GNU systems with the XFree86 +X11 server, and on Microsoft Windows. + +

+To help port glean to other +systems, please see Where do we go from here?. + +

Other software you'll need before you can build glean

+ +

+glean stores images in the TIFF format, because it's one of +the few standard formats that's capable of handling images with 32-bit +and floating-point color values. The best widely-available library for +handling TIFF is Sam Leffler's +libtiff. Your operating system may include this library already +(most Linux distributions do); if not, you'll need to obtain a copy of +the source and compile it according to the instructions contained therein. + +

+Some of the tools in the glean suite use Mark +Kilgard's GLUT library. Again, your OS (or your MESA distribution) may already include +GLUT, but if not, you'll need to grab a copy and install it. + +

+Once you've completed those tasks, choose one of the following: + +

+ + +

+ +

+ + + + diff --git a/doc/html/buildgnu.html b/doc/html/buildgnu.html new file mode 100644 index 0000000..005e64b --- /dev/null +++ b/doc/html/buildgnu.html @@ -0,0 +1,156 @@ + + + + + Building glean for GNU/Linux + + + + + + + +

Building glean for GNU/Linux

+ +

+The present versions of glean Makefiles depend on features of GNU make. +(In particular, the automatic re-scanning of all dependency files after +any have changed.) If your version of make is known to be significantly +incompatible with GNU make, then you will need to acquire a copy of +GNU make or modify glean's Makefiles. + +

+glean makes extensive use of the C++ Standard Template Library. +If your C++ compiler fails to compile glean, you may want to try +egcs or the latest version of gcc. +glean was developed with egcs 1.1.2; earlier versions are missing +some features that glean uses, such as namespaces. + + +

Obtain the source code

+ +

+Obtain the gzipped tar file containing glean from + +ftp://ftp.mesa3d.org/mesa/contrib/. + +

+Unpack the tar file: +

+    tar xfz glean_1_0.tgz
+    cd glean_1_0
+

+This will create a subdirectory named glean_1_0 and +make it your current directory. + +

Set the GLEAN_ROOT environment variable

+ +

+To compile or run glean, you will need to set the environment +variable GLEAN_ROOT to the full pathname of the glean +source directory. +If you're following along, this is your current directory, so +this command should do the trick for users of csh and its descendants: +

+    setenv GLEAN_ROOT `pwd`
+

+and this command should work for users of sh and its descendants: +

+    GLEAN_ROOT=`pwd`; export GLEAN_ROOT
+

+Be sure to set the environment variable each time you want to compile +or run glean. Perhaps the easiest way to ensure this is to +set it in your shell's startup script. + +

Set Makefile options

+ +

+Edit $GLEAN_ROOT/make/common.mak and modify the configuration variables +according to your preferences. Some of the variables that are most +likely to need customization are: + +

CONFIG: + This variable selects the operating system and window system for which + glean will be compiled. The OS option must be either __UNIX__ + or __MS__. The window-system option must be either __X11__ or __WIN__. + (Yes, there is some ambiguity between the OS and the window system in + Microsoft's case. However, there are X11 products that run under + Windows, so the combination of __MS__ and __X11__ is possible.) + By default, this variable is set to "-D__UNIX__ -D__X11__". +
CC: + This variable specifies the location of the C++ compiler. + By default it is "g++". +
XINC, GLINC, GLUTINC, TIFFINC: + These variables specify the directories containing include files for + X, OpenGL, GLUT, and libtiff, respectively. + The defaults are /usr/include/X11, /usr/local/include, /usr/local/include, + and /usr/include, respectively. +
XLIB, GLLIB, GLUTLIB, TIFFLIB: + These variables specify the directories containing libraries + (libX11, libGL, libglut, and libtiff, respectively). + The defaults are /usr/X11R6/lib, /usr/local/lib, /usr/local/lib, and + /usr/lib, respectively. +
_INC, _OPT, _DBG, _WARN, _LIBDIR, _LIB, _PROF: + These variables define C++ command-line options for include-file + directories, optimization, debugging, warnings, library directories, + libraries, and profiling. These apply universally, to all glean + compilations. Individual Makefiles can make local additions to + these defaults by defining corresponding variables whose names + do not have a leading underscore. Note that it is possible to + configure your system so that the order of include-file + or library-file directories is significant; for example, by having + two versions of the same include file, one in /usr/include and one + in /usr/local/include. In such cases, you'll need to make sure + that the _INC and _LIB variables specify directories in precisely + the order you require. +

+ +

Build glean

+ +

+When you've finished editing the Makefile, simply type +

+    cd src
+    make install
+

+This will build glean and the other tools in the suite, and install +them in $GLEAN_ROOT/bin. + + +

+ +

+ + + + diff --git a/doc/html/buildwin.html b/doc/html/buildwin.html new file mode 100644 index 0000000..89428fa --- /dev/null +++ b/doc/html/buildwin.html @@ -0,0 +1,86 @@ + + + + + + + Building glean under Windows + + + +

+Building glean under Windows

+Note : this document asumes that you are using Microsoft Visual C++ 6 (VC6). +

+Other software you'll need

+Currently glean doesn't build with the STL included with VC6. glean +has been verified to build with the STLport +implementation. +
+

+Obtain the source code

+Obtain the .zip file containing glean from +ftp://ftp.mesa3d.org/mesa/contrib/. +

Unzip the file: +

    pkunzip -d glean_1_0.zip c:\
+    cd glean_1_0

+This will create a subdirectory named glean_1_0 and make it your +current directory. +

+Set the GLEAN_ROOT environment variable

+To compile or run glean, you will need to set the environment variable +GLEAN_ROOT to the full pathname of the glean source directory. +Open a dos prompt and type +

    set GLEAN_ROOT=c:\glean_1_0

+Be sure to set the environment variable each time you want to compile or +run glean. +
+

+Set the VC6 environment variables

+To use the command-line tools, you will have to set some environment variables. +There is a batch file in VC6's bin directory called vcvars32.bat that +contains the necessary commands. Just execute this in the same dos +prompt as the previous step and you'll be set to go. +
+

+Set Makefile options

+Edit GLEAN_ROOT\make\common.win and modify the configuration variables +according to your preferences. Some of the variables that are most likely +to need customization are: +

STL_DIR +
This variables specifies +the directory containing include files of the STLport. +

LIBTIFF_INC_DIR +
LIBTIFF_LIB_DIR +
These variables specify +the directories that contain the libtiff include files and library resp. +
+

+Build glean

+When you've finished editing the Makefile, simply type +

    cd src
+    nmake /f makefile.win

+This will build glean and the other tools in the suite, and install +them in $GLEAN_ROOT/bin. + +

+ +

+ + + + diff --git a/doc/html/changes.html b/doc/html/changes.html new file mode 100644 index 0000000..7b60379 --- /dev/null +++ b/doc/html/changes.html @@ -0,0 +1,141 @@ + + + + + Recent changes in glean + + + + + + + +

What has changed recently?

+ +

Changes from Version 0.6 to Version 1.0

+ +

+Added hints to set window position as well as size, so that users +of window managers without automatic window placement don't have +to click to create each test window. + +

Changes from Version 0.5 to Version 0.6

+ +

+Replaced *.dsw and *.dsp files with nmake +makefiles (named common.win and makefile.win) +for building on Windows. + +

+Added Johan Smet's documentation for building glean on Windows. This +necessitated reworking the documentation structure somewhat. + +

+Fixed a storage allocation bug in the Windows version of showvis. + +

+Fixed a couple of problems in libs/dsurf/dsconfig.cpp related to the +visual_rating extension that caused compilation errors on some +systems. Thanks to Brian Paul for catching these. + +

+Brian also reported some bizarre link errors. These occurred because +there were multiple gl.h files installed on his system, and the +makefiles searched different header file directories for different +glean components. This has been worked around by changing common.mak +and the individual makefiles to search for header files and libraries +in the same order for all components. This is not a universal +solution; if your system has a very unusual configuration of library +or header directories, you may need to customize the makefiles. + +

Changes from Version 0.4 to Version 0.5

+ +

+Incorporated the second set of Johan Smet's changes +to port glean to Microsoft Windows. + +

Changes from Version 0.3 to Version 0.4

+ +

+Added the ability to include or exclude specific tests from a run. + +

+ +A recent change in the behavior of the C++ string +getline() function in the GNU libraries caused +glean's test comparison operation to appear to hang. The new +behavior appears to be correct, but it was necessary to add a +workaround to glean so that it would run properly on both old +and new versions of the library. + +

Changes from Version 0.2 to Version 0.3

+ +

+Incorporated the first set of Johan Smet's changes +to port glean to Microsoft Windows. These included: + +

+ VC6 still uses old-style scoping rules for variables declared + in the init part of ``for'' statements, so in some cases new + blocks or new variables were introduced to avoid collisions. + +
+ VC6 has trouble with some styles of initialization in + declarations; for example, +
```
+        WindowSystem& ws(env->winSys);
+	
```
+ had to be changed to +
```
+        WindowSystem& ws = env->winSys;
+	
```
+ +
+ Several items in the std namespace (notably the stream + functions) weren't scoped correctly. These were all fixed + with ``using namespace std'' or explicit qualification, as + needed. + +
+ VC6 can't handle the template functions used for image packing + and unpacking, so those were replaced. +

+ +

Changes from Version 0.1 to Version 0.2

+ +

+In lib/dsurf, bit counts for green and blue channels were reversed. +This caused incorrect drawing-surface configuration reports for normal +color channels and accumulation buffer channels, and also confused +code that determined errors in color values. + +

+In lib/dsurf, the expression for computing bits-in-error was in error. :-) + +

+Switched from egcs to g++ in common.mak. + + +

+ +

+ + + + diff --git a/doc/html/cleanup.html b/doc/html/cleanup.html new file mode 100644 index 0000000..14f0875 --- /dev/null +++ b/doc/html/cleanup.html @@ -0,0 +1,59 @@ + + + + + cleaning up loose ends in glean + + + + + + + +

Cleaning up the loose ends

+ +

+Many loose-end projects are marked by comments containing "XXX". +Any of those you'd like to tackle are fair game. + +

+Some other things that would be nice: +

Use const more consistently throughout the code. +
Use string in preference to char* where possible. +
Add the concept of test categories, assign each test to one or + more of them, and add a way to run all the tests in a given + category. +
Push the Test class hierarchy down one more level of detail. + (I actually did this, factoring out everything that's common to + classes like BasicTest, but it turned out to be more awkward to + use than just cutting and pasting code, so I backed out the changes. + After a few more tests are written, it should be more clear how to do + this correctly.) +
Automated checks for memory leaks. +
GLX 1.3 support. +

+ + +

+ +

+ + + + diff --git a/doc/html/index.html b/doc/html/index.html new file mode 100644 index 0000000..e60ac95 --- /dev/null +++ b/doc/html/index.html @@ -0,0 +1,35 @@ + + + + + glean doc + + + + + + + +

glean: An OpenGL Test and + Benchmarking Suite

+ +

+Welcome! This is the glean documentation home page. + +

+ +

+Warning: This software is a work in progress. +Despite the best efforts of the contributors, there may be discrepancies +between the documentation and the code. + + + + diff --git a/doc/html/newfeat.html b/doc/html/newfeat.html new file mode 100644 index 0000000..c2a48e7 --- /dev/null +++ b/doc/html/newfeat.html @@ -0,0 +1,107 @@ + + + + + adding new infrastructure features to glean + + + + + + + +

Adding new infrastructure features to glean

+ +

+(Thanks to Neal Tringham for most of these.) + +

On all systems

+ +It might be useful to add a command-line option that would force +glean to overwrite an existing database, which is something +that it refuses to do now. Probably the best way to implement this +would be to recursively delete the old results directory before +creating a new one, since the set of tests may have changed in +unpredictable ways. This code would be system-dependent, so distinct +versions would be needed for all operating systems. + +

On Windows

+ +

Non-default color depths

+ +glean and the other tools in the suite run only at the default +screen color depth. It would be useful to add a command-line option +that switches to a new depth, because some graphics cards accelerate +only some color depths. Note that the usual method for doing this +(ChangeDisplaySettings) can't be guaranteed to work on +Windows 95, and using DirectDraw to change the depth causes some +drivers to crash. However, it would be valuable on Windows 98 and +perhaps on Windows 2000. Documenting the need to switch color depths +manually on Windows 95 (and Linux!) would cover the other cases. + +

Supporting old standalone 3D cards

+ +glean uses the normal ICD mechanism to access the OpenGL +driver. This means that graphics on older standalone cards (such as +3Dfx Voodoo 1 and Voodoo 2) will never be accelerated, and the test +results will be valueless on those cards. Probably the best solution +is to use the upcoming GLSetup API, which allows the proper driver to +be loaded. If this is done, we'll need to make sure that tests which +create more than one window are not run on the older cards. + +

+A related issue is that GLUT also doesn't support standalone cards, +so the tools which use it need to be modified to use the +GLSetup API or some toolkit other than GLUT. + +

+Note that there are further issues with supporting old 3Dfx cards. +Neal Tringham reports: + +

+ + The problem is that even if the driver's been installed, the 3dfx + card might have been taken out later, leaving a driver which + faults as soon as you load the 3dfxvgl dll. As far as I know, + the only truly safe way to deal with this is to check for the + presence of the Glide dlls, load glide if they're present, and + then use the Glide API (grGetNumBoards or something with a + vaguely similar name) to actually detect the presence of the + hardware. If its there, you should be able to load 3dfxvgl.dll + safely. + +

+ +

+Perhaps the GLSetup implementation will cover this. + +

Supporting multi-card systems

+ +For systems with multiple graphics cards, it would be nice to have a +command-line option that specifies the card to be tested. This could +be similar to the display option used on X11-based +systems. + + +

+ +

+ + + + diff --git a/doc/html/newtest.html b/doc/html/newtest.html new file mode 100644 index 0000000..eafcdda --- /dev/null +++ b/doc/html/newtest.html @@ -0,0 +1,198 @@ + + + + + adding new tests to glean + + + + + + + +

Adding new tests to glean

+ +

+On this page, we'll consider some general testing principles and +suggest areas in which new tests are needed. +When you're ready to implement a new test, check out the +overview of glean internals. + +

Guidelines for writing glean tests

+ +

+To define a new test, first create a new test class derived from +Test. Most of the time you'll want to write tests +that run on all drawing surface configurations and are portable +(in the sense that they don't use any OS- or window-system-dependent +data structures exposed in the Environment, WindowSystem, +and DrawingSurface classes). Starting from BasicTest may +be easiest in those cases. + +

+Once you've created a new test class, create one or more global +objects of that class. The constructor for Test ensures that +all such objects are automatically linked onto the master list of +tests. (Creating test objects dynamically isn't recommended, but if +you can make a good argument for it we can figure out the rules needed to +make it work.) Every test must have a good description; it's very much +worthwhile to add text that will help the user understand the results +or debug the problem if something goes wrong. + +

+The first workhorse method in any test class is run(). +Here are some things to remember about it: +

+ Every run must produce at least one PASS, FAIL, or NOTE message in the + log. +
+ In the case of failure, never just report failure and assume the user + will go add debugging code to the source to figure out exactly what failure + occurred. Ideally, provide enough information so that a driver + developer could start debugging his code without having to look at + the glean source first. +
+ Each test is producing two kinds of result: The result that goes into + the results database, and the result that remains in memory, accessible + through the test object. Eventually there will be tests that refer + back to the results of previous tests! (This is the reason for the + hasRun mechanism in the Test class; one test may be + forced to run another as a prerequisite, but we never need to run a + test twice.) +
+ Try to keep most test runs short. Eventually there will be a lot of + tests, and some machines will have a lot of drawing surface + configurations, and runtime will add up. +
+ Whenever possible, avoid tests that just generate images for the + user to check, rather than determining for themselves whether they + pass or fail. Even experts find it hard to tell which of two + complicated images is correct, and when some subtle part of the + OpenGL driver changes (like the precision with which colors are + computed) the user might be faced with comparing hundreds or + thousands of images for all existing tests. +
+ Prefer statistical summaries of error to simple pass/fail results. + They provide a lot more useful information, and in some cases they + make writing test comparisons much easier and more effective. +

+ +

+The second major method is compare(). Some things to keep in mind: +

+ When two test results are the same (whatever that means for the + particular test in question), don't report that they are the same + unless the user has requested verbose output. Ideally, if two + test runs are identical, glean should report nothing. +
+ Subject to the constraint in the previous item, every comparison + must yield at least one SAME or DIFF message in the log. +
+ As with run(), statistical summaries are preferable to simple + pass/fail assessments. +

+ +

New tests

+ +

+Here are some suggestions for tests that could be added to glean. +(These are not listed in any particular order; any of them would be +worthwhile.) + +

+ Quake3 Arena. +
+ We could use a set of tests that check everything John Carmack mentioned + in + http://www.quake3arena.com/news/glopt.html. +
+ Default state check. +
+ The OpenGL spec defines initial values for all state variables, and + many programs (including glean!) depend on those initial + values being correct. It would be great to have a test that actually + verified them. +
+ Enumerant coverage check. +
+ It would be good to verify that each OpenGL command actually accepts + all the argument values that it's supposed to accept, without generating + an error. Conversely, it would be nice to know that errors are + generated appropriately when illegal arguments are used. +
+ Regression tests. +
+ (This is especially relevant to everyone working on Mesa.) + Regression tests simply check that a previously-discovered bug or + performance problem doesn't reappear after later changes to an OpenGL + driver. Regression tests are really valuable, because bugs tend to + appear in the parts of the driver that are the most difficult to + implement correctly. Therefore, writing tests in response to each bug + report tends to cluster tests in precisely the area where they're most + needed. As a general principle, whenever you report a bug or fix a bug, + you could write a glean test to make sure that bug will never + reappear. +
+ Stress tests. +
+ Stress tests assess the robustness of an OpenGL implementation. Some + possibilities are: create and download many texture objects, then + switch rapidly from one to another; perform many buffer swaps in + rapid succession; execute ten thousand glClear() operations and see + whether the hardware FIFO times out; and create a large number of windows + and rendering contexts and randomly bind arbitrary pairs of them. +
+ Limits tests. +
+ Every OpenGL implementation has limits (sometimes enforced by the graphics + hardware) and these often determine how applications must be written. + For example, it would be good to know the maximum acceptable texture + size for a variety of texture formats; the largest image that can + be passed to glDrawPixels(); the maximum number of vertices that can + be placed in a triangle strip or vertex array; and the maximum number of + windows or rendering contexts that can be created. +
+ Window-system binding tests. +
+ Most tests concentrate on OpenGL itself, but an area that must not be + overlooked is the window system binding (GLX or WGL). It would be useful + to test the window system binding's extension string and all the basic + operations like context creation, copy, and deletion. Sharing texture + objects and display lists is important to test. +
+ Core OpenGL correctness tests. +
+ Most OpenGL implementations have been checked with the OpenGL + conformance tests. However, the conformance tests are lax in some + areas due to historical reasons. One way to address this would be + to add some correctness tests to glean. Check the contents + page of the OpenGL spec; each heading is a good suggestion for a + test. +

+ + +

+ +

+ + + + diff --git a/doc/html/next.html b/doc/html/next.html new file mode 100644 index 0000000..5849046 --- /dev/null +++ b/doc/html/next.html @@ -0,0 +1,60 @@ + + + + + next steps for using and enhancing glean + + + + + + + +

Where do we go from here?

+ +

+This release of glean is just a beginning. With a little +luck, and if the open-source development process works well, +glean will eventually become a comprehensive tool helping consumers +to choose the best OpenGL-based products and developers to create +them. Here's what we need to do next. + +

+ +

+Please consider volunteering for some part of the jobs above. +These projects are great candidates for open-source methods: they +won't take too much of your time, you can work independently, and +everything you contribute will have great leverage to help make sure 3D +graphics is Done Right. Contact +glean@pobox.com if you can help. + + +

+ +

+ + + + diff --git a/doc/html/overview.html b/doc/html/overview.html new file mode 100644 index 0000000..c72191c --- /dev/null +++ b/doc/html/overview.html @@ -0,0 +1,247 @@ + + + + + overview of glean internals + + + + + + + +

Overview of glean internals

+ +

Directory structure

+ +

+Starting at $GLEAN_ROOT, you'll see bin, +include, and lib directories. +These are initially empty, and are filled by the Makefile in +src. + +

+$GLEAN_ROOT also contains documentation in doc, +common Makefile templates in make, sample result files in +results, and source code in src. +The Makefile templates are particularly interesting, since they +automate nearly all of the process of defining Makefiles. + +

+cd into src, and take a look around. +The main Makefile resides here. There are also subdirectories for +glean itself (glean), a wrapper for OpenGL header +files that covers up some Windows-specific problems (glh), +libraries that are used by multiple tools (libs), and +code for tools like showvis (tools). The Makefile +ensures that everything in the subdirectories is built in the correct +order. + +

Libraries

+ +

+There are several libraries providing services that you may want when +you start writing tests. These particular libraries are of sufficiently +general usefulness that they're broken out into separate directories. +(As we'll see later, some other services are local to the glean +subdirectory.) +

+ dsurf +
+ This library provides a portable interface to drawing surface + configuration information (pixel format descriptors under Windows, + and extended X11 Visuals under X). It also provides the filtering + capability that's used by showvis and glean to + select configurations that match particular criteria. +
+ image +
+ The basic abstraction provided by this library is an in-memory + image structure that maps almost directly onto the image format + used by OpenGL. The library also offers utilities that read and + write TIFF files, tools to convert image data to a common format, + and convenience routines for invoking OpenGL operations like + reading and drawing images. Finally, there is a "registration" + utility that compares two images and attempts to align one on top + of the other in the position at which the images are most similar. +
+ lex +
+ This is a simple lexical analyzer. libdsurf uses it when + parsing drawing surface configuration criteria, and when constructing + a drawing surface configuration from a canonical description stored + in a results file. +
+ rand +
+ This set of classes provides uniform random values in several different + ranges and formats, in an OS-independent fashion. (This allows results + generated by glean on one OS to be compared to results on the same + hardware running under another OS.) It's based on a linear congruential + generator discussed in Knuth Vol. 2 and Numerical Recipes (2nd ed.). +
+ stats +
+ These classes generate simple statistical measures (min, max, mean, + standard deviation, etc.). +
+ timer +
+ This class provides timing services for benchmarking. It's capable of + calibrating measurement overhead, repeating benchmarks until a target + runtime is achieved, etc. +

+ +

glean itself

+ +

+glean consists of a relatively simple main program, a set of +classes providing commonly-needed services, and the tests themselves. + +

+The main program essentially just parses command-line options, creates +the test environment (see below) and invokes each test in turn. The +list of tests is generated automatically by test class constructors, +so there's no need to maintain one independently. It should be fairly +straightforward to replace the current main program with a graphical +front-end, if anyone cares to do so. + +

+The main service classes are as follows: + +

+ Options +
+ This class collects all option information (the verbosity level, + names of the test results databases, etc.) in one place. Doing + so allows option setting to be separated from option interpretation. +
+ Environment +
+ This class encapsulates information about the test environment: + the options, an output stream for the test log, the window system, + etc. It is passed as an argument to each test. +
+ WindowSystem +
+ This class abstracts information about the window system, so that + most glean tests need not know the system under which they're + running. Its main task is to maintain lists of drawing surface + configurations, drawing surfaces, and OpenGL rendering contexts. + However, it does expose window-system-specific data (such as the + X11 Display pointer) so that window-system-specific tests can be + written. (Such tests must use conditional compilation to protect + references to non-portable function calls and data structures, + though.) +
+ DrawingSurface and Window +
+ A drawing surface is a canvas on which OpenGL rendering can take + place. Each drawing surface has a configuration that specifies + the buffers (color, depth, stencil, etc.) associated with the + surface. + The prototypical drawing surface is the window, which adds size + information and buffer-swap capability that's used by most tests. + Eventually we will need to add more drawing surfaces to glean, + for example, the PBuffers supported under GLX version 1.3. +
+ RenderingContext +
+ A rendering context encapsulates all OpenGL state information + (except for the contents of the drawing surface). In general, + in order to render, you must create a drawing surface, create + a rendering context, and then bind the two (using a service + provided by the WindowSystem class). Then you can use + ordinary OpenGL commands freely. +

+ +

+A few general utilities are also available: +

+ useScreenCoords() (glutils.h) +
+ Sets up the modelview and projection matrices so that OpenGL + object coordinates correspond directly to screen pixel + coordinates. (Note: The origin of the coordinate system is + at the lower left corner of the window.) This is convenient + for most tests of rasterization operations. +
+ RandomMesh2D (geomutil.h) +
+ Creates a 2D array of points that fit within a given bounding + rectangle, but are displaced by random amounts from the lattice + points within it. This is especially useful for benchmarking, + because it yields randomly sized and oriented triangles whose + average area is nevertheless accurately known. +

+ +

+To create a new test, you need do little more than create a new +.cpp file in the glean subdirectory. (The Makefile +will handle its dependencies and build it automatically.) +All tests are derived from the base class Test. For truly +new types of tests you'll need to derive a new class, but for many +common types of tests you can start by copying a test that already +exists. Here are some examples: + +

+ BasicTest (tbasic.cpp) +
+ This is a good outline for any + simple test that runs in one window with one rendering context, + generating a result for each drawing surface configuration that + meets specified criteria. If your test is of this type, you may + be able to copy tbasic.cpp and modify just the functions + runOne(), compareOne(), put(), and get(). + It may be instructive to compare tbasic.cpp with + tgetstr.cpp, which is derived in just that way. +
+ TexBindPerf (tchgperf.cpp) +
+ This test demonstrates how to measure performance, by benchmarking + texture binding time. It's a bit more complicated than + BasicTest, but the structure is basically the same, and + most of the work is concentrated in the four functions mentioned + above. +
+ RGBTriStripTest (trgbtris.cpp) +
+ This shows how to save and compare TIFF images. While it's not a + good idea to make too many tests dependent on image comparisons + (because they're not particularly robust), there are cases like + regression testing where image comparison may be a good approach. +

+ +

+That's pretty much all there is to it. The glean framework +provides most of the mechanisms needed to develop simple tests. +Now it's time to create a good set of tests and thereby make a new +tool that's valuable to the entire OpenGL community. + + +

+ +

+ + + + diff --git a/doc/html/port.html b/doc/html/port.html new file mode 100644 index 0000000..7252b65 --- /dev/null +++ b/doc/html/port.html @@ -0,0 +1,57 @@ + + + + + porting glean to new environments + + + + + + + +

Porting glean to new environments

+ +

+Thanks to Johan Smet, +we have a working version of glean for Microsoft Windows, +using Visual C++ version 6. + +

+Porting glean to BeOS and MacOS would be well worthwhile. + +

+Porting glean to other UNIX variants should be straightforward, but I +haven't looked into it. + +

+In all of these cases, most of the work is in the window system +code (src/libs/dsurf, src/glean/winsys.cpp, src/glean/dsurf.cpp). +Some other things are needed in src/libs/timer and src/glean/environ.cpp. +It's always possible that differences in compilers will result in a +need for more pervasive changes; some problems of this type did arise +during the port to Windows. + + +

+ +

+ + + + diff --git a/doc/html/repo.html b/doc/html/repo.html new file mode 100644 index 0000000..a112980 --- /dev/null +++ b/doc/html/repo.html @@ -0,0 +1,48 @@ + + + + + repository for glean results + + + + + + + +

A repository of glean results

+ +

+After glean becomes a bit more comprehensive, it would be nice +to accumulate results and make them accessible through a +website. For example, if you're considering buying a graphics card, +or you want to survey all cards to see whether you can count on using +a given feature, this website would be the place to go. +Creating an index for this database would be an interesting challenge; +at the least you'd have to deal with glean version, CPU type, +graphics card type, OS type, OS version, and driver version; maybe +graphics chip revisions, too. + + +

+ +

+ + + + diff --git a/doc/html/run.html b/doc/html/run.html new file mode 100644 index 0000000..66d76a8 --- /dev/null +++ b/doc/html/run.html @@ -0,0 +1,248 @@ + + + + + How to run glean + + + + + + + +

How do I run glean?

+ +

+Note: These instructions assume you're running glean on a +GNU/Linux system. We hope the translation to running glean on a +Windows system is straightforward. If not, we're always happy +to receive additional documentation for the distribution. :-) + +

+glean runs in two modes: Gathering information about your +OpenGL implementation (run mode) and comparing results of two +previous runs (compare mode). After using glean in +either mode, you may also find it helpful to evaluate results with +other tools in the suite. + +

Generating test results

+ +

+To analyze the OpenGL implementation on your machine, first choose +a directory name and then run glean, instructing it to store +results in that directory: +

+    $GLEAN_ROOT/bin/glean -r myogl
+

+The results directory must not exist prior to running glean; +glean will not overwrite existing information. + +

+As glean runs, it produces a log containing information like +the following: +

+blendFunc:  FAIL rgb8, db, z16, s8, accrgba16, win+pmap, id 34
+	source factor = GL_SRC_ALPHA, dest factor = GL_ONE_MINUS_SRC_ALPHA
+	Readback had 1.00002 bits in error; blending had 1.43898 bits in error.
+
+texBindPerf:  PASS rgb8, db, z16, s8, accrgba16, win+pmap, id 34
+	Approximate texture binding time = 10.8911 microseconds.
+	Range of valid measurements = [9.34022, 16.2041]
+
+getString:  PASS rgb8, db, z16, s8, accrgba16, win+pmap, id 34
+

+Each test produces output including its name, a description of the +format of the window in which it is run, and a brief summary of its +results. Each summary always includes one of the words PASS, FAIL, +or NOTE; the meaning of the first two is obvious, and NOTE simply +means that something is worthy of the reader's attention. (Some +tests don't explicitly pass or fail; these generate only NOTEs.) + +

+A more verbose log, including descriptions of each test, +will be generated if you supply the -v option on the +glean command line. + +

+To run just a particular set of tests, or to exclude some tests from the +usual run of all tests, you can use the -t +option on the command line: +

+    $GLEAN_ROOT/bin/glean -r mysubset -t "basic+getString"
+    $GLEAN_ROOT/bin/glean -r mysubset -t "-blendFunc-texBindPerf"
+

+ +

+Some of the tests in glean (such as the texBindPerf test +shown in the example above) involve benchmarking OpenGL operations. +To obtain the most accurate, repeatable results for these tests, there +should be no other processes running on your machine, and you should run +glean at higher than normal priority. However, glean +makes multiple sample runs and attempts to stabilize the system before +taking measurements, in order to get sensible results even if you +don't take those precautions. + +

Comparing test results

+ +

+To compare two previous runs, invoke glean with the -c +(compare) option: +

+    $GLEAN_ROOT/bin/glean -c myogl anotherogl
+
+    texBindPerf:  DIFF rgb8, db, z16, s8, accrgba16, win+pmap, id 34
+	    myogl appears to have higher performance.
+
+    getString:  DIFF rgb8, db, z16, s8, accrgba16, win+pmap, id 34
+	    Extensions in anotherogl but not in myogl:
+		    GL_ARB_multitexture
+	    Extensions in myogl but not in anotherogl:
+		    GL_EXT_shared_texture_palette
+	    Extensions in both myogl and in anotherogl:
+		    GL_EXT_abgr
+		    GL_EXT_blend_color
+		    GL_EXT_blend_logic_op
+		    GL_EXT_blend_minmax
+		    GL_EXT_blend_subtract
+		    GL_EXT_multitexture
+		    GL_EXT_paletted_texture
+		    GL_EXT_point_parameters
+		    GL_EXT_polygon_offset
+		    GL_EXT_rescale_normal
+		    GL_EXT_texture3D
+		    GL_EXT_texture_object
+		    GL_EXT_vertex_array
+		    GL_MESA_resize_buffers
+		    GL_MESA_window_pos
+		    GL_SGIS_multitexture
+		    GL_SGIS_texture_edge_clamp
+

+glean will note any significant differences between the two +runs, displaying the test name, the keyword DIFF, the format of the +window used for the test, and an explanation of the differences. +Again, you can generate a more extensive log by specifying the +-v option on the command line. In addition to extra +explanatory information, glean will also list tests without +significant differences, marking them with the notation "SAME". + +

+Some sample results files can be found in $GLEAN_ROOT/results. + +

Looking at test results in detail

+ +

+When you run glean with the -r option, glean +uses the name of each test to create a subdirectory within the main +test result directory you specified on the command line. + +

+Within these subdirectories you'll find a text file named +results containing the machine-readable form of the test +results. The precise format of the data in the results file varies +from test to test; you'll need to look at the source code to understand +it fully. + +

+One special case is that of TIFF images generated by a test. These +also appear in the test's subdirectory, and have names of the form +iNNN.tif, where NNN represents a three-digit +number. The glean log will explain which test result is +associated with which image number. + +

+You can use the showtiff tool to display an image: +

+    $GLEAN_ROOT/bin/showtiff myogl/rgbTriStrip/i001.tif
+

+Use the right-hand mouse button to pop up a menu of available commands. + +

+You can also use the difftiff tool to compare two images: +

+    $GLEAN_ROOT/bin/difftiff myogl/rgbTriStrip/i001.tif \
+        anotherogl/rgbTriStrip/i001.tif
+

+Again, pressing the right-hand mouse button will pop up a command menu. +difftiff allows you to display either image, a difference image +(using selectable difference threshold and color), or a difference image +overlayed on one of the two base images. This makes it easier to spot +subtle differences between images generated by two different OpenGL +implementations. + +

Drawing surface configurations and showvis

+ +

+In current OpenGL implementations, one creates a drawing surface +(usually a window) by selecting a configuration that describes +the depth of the color buffers, whether a Z buffer is present, etc. +from those configurations that the implementation supports. +You can list the drawing surface configurations that are available on +your system with showvis: +

+$GLEAN_ROOT/bin/showvis
+
+rgb8, db, z16, s8, accrgba16, win+pmap, id 34
+

+In this particular case, there is only one supported configuration. +The color portion of the framebuffer is 24 bits deep (r, g, and b each +have 8 bits) and it's double-buffered. There is a 16-bit depth (Z) buffer, +an 8-bit stencil buffer, and an RGBA accumulation buffer with 16 bits per +color component. The configuration can be used for creating windows and +pixmaps. It has an X11 Visual ID of 34. + +

+By default, showvis lists all the configurations your system supports. +However, it's possible to ask showvis about configurations with +specific characteristics. You do this by providing command-line options +that are logical expressions using C syntax and involving variables that +are related to attributes of drawing surface configurations. You may +also request that the configurations be listed in increasing or decreasing +order of any particular attribute. Some examples: +

+    $GLEAN_ROOT/bin/showvis db
+

+Lists all drawing surface configurations that support double buffering. +

+    $GLEAN_ROOT/bin/showvis "window && rgb && r == g && g == b, min z"
+

+Lists all drawing surface configurations for RGB windows where the three +color channels have equal size, sorted with smallest Z first. + +

+You may also use these filter expressions to restrict glean to +a particular set of drawing surface configurations. For example, +

+    $GLEAN_ROOT/bin/glean -r rgbWindows --visuals "rgb && window"
+

+would run only those glean tests that can be run on RGB windows. +Color index windows, tests that only run on pixmaps, etc., would be +omitted. + +

+For more information on filter expressions, try executing: +

+    $GLEAN_ROOT/bin/showvis --help
+

+ +

+ + + + diff --git a/doc/html/whatis.html b/doc/html/whatis.html new file mode 100644 index 0000000..91dd940 --- /dev/null +++ b/doc/html/whatis.html @@ -0,0 +1,102 @@ + + + + + What is glean? + + + + + + + +

What is glean?

+ +

+glean is a suite of tools for evaluating the quality of an +OpenGL implementation and diagnosing any problems that are discovered. +glean also has the ability to compare two OpenGL implementations and +highlight the differences between them. + +

+Who should care about glean? + +

Anyone who buys or reviews graphics cards. +
+ glean gives you the ability to compare performance, features, + and image quality of different graphics cards. Since the source code + is freely available, you can customize glean to add tests for + the things that are most important to you. + +
Software developers. +
+ With glean you can learn which features really work for a + given combination of hardware and driver. You can also find the + timing for basic drawing operations, mode changes, texture + downloads, etc. so that you can maximize the performance of your + application. + +
OpenGL driver developers. +
+ glean can increase your confidence that your driver software + is correct and efficient. It also allows you to compare a new release + to a previous release, to make sure that there are no regressions in + quality or performance. +

+ +

+glean is provided under an "X11-style" copyright, which permits +it and its derivatives to be used at no cost for both commercial and +non-commercial projects. + +

Acknowledgments

+ +

+Development of the original glean framework and diagnosis tools +was supported by +Precision Insight, Inc. + +

+Johan Smet took on the Herculean +job of porting glean to Microsoft Windows. The importance of +this work cannot be understated! In addition to making glean +useful to a great many more people, it enlarges the pool of developers +that can contribute to glean. + +

+I'm also grateful to Brian Paul +and VA Linux for providing +glean's first home on the Web. + +

+Cass Everitt suggested several improvements +in the ways glean uses C++. + +

+Currently Allen Akin is coordinating enhancements to glean. +Please address glean-related correspondence to +glean@pobox.com. + +

+ +

+ + + + -- cgit v1.2.3