| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
For this we extend the boilerplate get_image() routines to extract a
single page out of a paginated document and then proceed to manually
check each page of the fallback-resolution test.
(Well that's the theory, in practice SVG doesn't support multiple pages
and so we just generate a new surface for each resolution. But the
infrastructure is in place so that we can automate other tests,
e.g. test/multi-pages.)
|
|
SVG 1.2 is sufficiently different from 1.1 that it has separate code
paths within cairo-svg-surface and so justifies its own boilerplate
target.
|
|
One possibility for a read failure whilst converting the image is if the
external utility crashed. This information is important for the test suite
as knowing input that causes the converter to crash is just as vital as
identifying a crash within the library.
|
|
Compare the current output against a previous run to determine if there
has been any change since last time, and only run through imagediff if
there has been. For the vector surfaces, we can check the vector output
first and potentially skip the rasterisation. On my machine this reduces
the time for a second run from 6 minutes to 2m30s. As most of the time,
most test output will remain unchanged, so this seems to be a big win. On
unix systems, hard linking is used to reduce the amount of storage space
required - others will see about a three-fold increase in the amount of
disk used. The directory continues to be a stress test for file selectors.
In order to reduce the changes between runs, the current time is no longer
written to the PNG files (justified by that it only exists as a debugging
aid) and the boilerplate tweaks the PS surface so that the creation date
is fixed. To fully realise the benefits here, we need to strip the
creation time from all the reference images...
The biggest problem with using the caches is that different runs of the
test suite can go through different code paths, introducing potential
Heisenbergs. If you suspect that caching is interfering with the test
results, use 'make -C test clean-caches check'.
|
|
In order to achieve substantial speed improvements the external conversion
utilities are rewritten as a daemon that communicates with the test suite
over a local socket. This is faster as it avoids the libtool and dynamic
linker overhead for each invocation, the caches persist between tests and
we no longer require a round trip through libpng.
The daemon is started automatically by the test suite and if communication
cannot be established then it falls back to using a pipe to a normal
conversion utility. The daemon will then persist for 60 seconds waiting
for further connections.
Of course any memory leak (stares at poppler) is exacerbated.
|
|
If the external conversion utility was killed by a signal (e.g. the user
sent SIGINT), raise that signal within our process as well. This means
that a crash inside poppler or rsvg will be flagged as a crash inside the
test suite, and makes interrupting the test suite far more responsive.
|
|
As Behdad suggested, we can dramatically speed up the test suite by
short-circuiting the write to a png file, only to then immediately read it
back in. So for the raster based surfaces, we avoid the round-trip through
libpng by implementing a new boilerplate method to directly extract the image
buffer from the test result. A secondary speedup is achieved by caching the
most recent reference image.
|
|
Construct the test name to pass to the boilerplate creation routines such
that it uniquely identifies the test in terms of test, target, content and
pass (similar, offset, thread). This allows the vector targets to create
output different output files for each test, whereas before, later tests
would overwrite existing files making debugging more difficult.
|
|
In order to run under memfault, the framework is first extended to handle
running concurrent tests - i.e. multi-threading. (Not that this is a
requirement for memfault, instead it shares a common goal of storing
per-test data). To that end all the global data is moved into a per-test
context and the targets are adjusted to avoid overlap on shared, global
resources (such as output files and frame buffers). In order to preserve
the simplicity of the standard draw routines, the context is not passed
explicitly as a parameter to the routines, but is instead attached to the
cairo_t via the user_data.
For the masochist, to enable the tests to be run across multiple threads
simply set the environment variable CAIRO_TEST_NUM_THREADS to the desired
number.
In the long run, we can hope the need for memfault (runtime testing of
error paths) will be mitigated by static analysis. A promising candidate
for this task would appear to be http://hal.cs.berkeley.edu/cil/.
|
|
in favor of cairo_boilerplate_svg_surface_force_fallbacks.
|
|
|
|
|
