summaryrefslogtreecommitdiff
path: root/test/cairo-test.c
AgeCommit message (Collapse)AuthorFilesLines
2009-09-09[test] Compare a failure against the image outputChris Wilson1-0/+46
If a backend fails in exactly the same way as the image, then we can safely assume that the failure is systematic and not an error in the backend, so change the result to XFAIL.
2009-09-03[test] Allow tests to XFAIL by putting the cairo_t into an error state.M Joonas Pihlaja1-7/+7
The test runner was extra strict about never letting a test put the cairo_t into an error state, and never would it check for the expectedness status of the failure. This patch moves the check for a test being an XFAIL above the check on the cairo_t's final status.
2009-08-31[test] Use HAVE_FLOCKFILE instead of _POSIX_C_SOURCE.M Joonas Pihlaja1-3/+2
The _POSIX_C_SOURCE 2001.. #define requires C99 mode and clang on Solaris is strict about such things. Use configure tests for flockfile() instead.
2009-07-28[test] Wrap yet another call to get_image_surface() inside a timeout.Chris Wilson1-0/+3
I missed this call to get_image_surface() that is now being hit having restored the reference image for dash-infinite-loop.
2009-07-28[test] Add timeouts around finishing and converting surfaces to images.Chris Wilson1-4/+10
In order to catch infinite loops whilst replaying and converting vector surfaces to images (via external renderers) we need to also install alarms around the calls to finish() and get_image().
2009-07-24[test] Trap SIGFPEChris Wilson1-0/+7
Note the crash if we hit a floating-point exception.
2009-07-23[drm] Add an accelerated image surface.Chris Wilson1-1/+1
Use the DRM interface to h/w accelerate composition on image surfaces. The purpose of the backend is simply to explore what such a hardware interface might look like and what benefits we might expect. The use case that might justify writing such custom backends are embedded devices running a drm compositor like wayland - which would, for example, allow one to write applications that seamlessly integrated accelerated, dynamic, high quality 2D graphics using Cairo with advanced interaction (e.g. smooth animations in the UI) driven by a clutter framework... In this first step we introduce the fundamental wrapping of GEM for intel and radeon chipsets, and, for comparison, gallium. No acceleration, all we do is use buffer objects (that is use the kernel memory manager) to allocate images and simply use the fallback mechanism. This provides a suitable base to start writing chip specific drivers.
2009-07-23Remove clip handling from generic surface layer.Chris Wilson1-3/+15
Handling clip as part of the surface state, as opposed to being part of the operation state, is cumbersome and a hindrance to providing true proxy surface support. For example, the clip must be copied from the surface onto the fallback image, but this was forgotten causing undue hassle in each backend. Another example is the contortion the meta surface endures to ensure the clip is correctly recorded. By contrast passing the clip along with the operation is quite simple and enables us to write generic handlers for providing surface wrappers. (And in the future, we should be able to write more esoteric wrappers, e.g. automatic 2x FSAA, trivially.) In brief, instead of the surface automatically applying the clip before calling the backend, the backend can call into a generic helper to apply clipping. For raster surfaces, clip regions are handled automatically as part of the composite interface. For vector surfaces, a clip helper is introduced to replay and callback into an intersect_clip_path() function as necessary. Whilst this is not primarily a performance related change (the change should just move the computation of the clip from the moment it is applied by the user to the moment it is required by the backend), it is important to track any potential regression: ppc: Speedups ======== image-rgba evolution-20090607-0 1026085.22 0.18% -> 672972.07 0.77%: 1.52x speedup ▌ image-rgba evolution-20090618-0 680579.98 0.12% -> 573237.66 0.16%: 1.19x speedup ▎ image-rgba swfdec-fill-rate-4xaa-0 460296.92 0.36% -> 407464.63 0.42%: 1.13x speedup ▏ image-rgba swfdec-fill-rate-2xaa-0 128431.95 0.47% -> 115051.86 0.42%: 1.12x speedup ▏ Slowdowns ========= image-rgba firefox-periodic-table-0 56837.61 0.78% -> 66055.17 3.20%: 1.09x slowdown ▏
2009-07-20[test] Fallback to comparing the base image referencesChris Wilson1-68/+163
After looking at backend specific images, check against the base image reference. This is useful to fallback surfaces like xlib-fallback, which should look closer to the image backend than the xlib backend.
2009-07-20[test] Misidentification of XFAIL as NEWChris Wilson1-1/+1
cut'n'paste error compared the image against the known failure instead of any recorded new failure, when checking for NEW fails.
2009-07-17Add OpenVG backend.Chris Wilson1-1/+2
Based on the work by Øyvind Kolås and Pierre Tardy -- many thanks to Pierre for pushing this backend for inclusion as well as testing and reviewing my initial patch. And many more thanks to pippin for writing the backend in the first place! Hacked and chopped by myself into a suitable basis for a backend. Quite a few issues remain open, but would seem to be ready for testing on suitable hardware.
2009-07-13[test] Track XFAIL using expected results stored as xfail.pngChris Wilson1-127/+278
Instead of tagging the sources, which is insensitive to changes, track the known failure modes by recording the current fail as an xfail.png reference. (We also introduce a new.png to track a fresh error, so that they are not lost in the noise of the old XFAILs and hopefully do not cause everyone to fret). As we have removed the XFAIL tagging we find, surprise surprise, that some tests are now working -- so review all the reference images (as also some .ref.png now should be .xfail.png). Note: I've only checked image,pdf,ps,svg. The test surfaces report some failures that probably need to addressed in source. I've not correct the changes for win32 and quartz. Nor fixed up the experimental backends.
2009-06-28[test] Timeout support for testsChris Wilson1-0/+18
Enforce that each test must render within 60 seconds or be considered to have hit an infinite loop and be reported as a CRASH. The timeout value is adjustable via CAIRO_TEST_TIMEOUT -- a value of 0 will disable.
2009-06-27[test] Add group-unalignedChris Wilson1-1/+1
Test case for: Bug 22441 -- Unexpected shift with push_group and pop_group https://bugs.freedesktop.org/show_bug.cgi?id=22441 This is a test that demonstrates the error in the pdf backend when using groups on surfaces with non-integer sizes. In order to create such a surface, we need to update the boilerplate to use doubles instead of integers when specifying the surface size.
2009-06-21[test] Fix a typo in the _POSIX_SOURCE version number.M Joonas Pihlaja1-1/+1
I hope POSIX isn't around in 20000 AD!
2009-06-15Expose _cairo_null_surface_create() via a test surfaceChris Wilson1-4/+10
Using a null surface is a convenient method to measure the overhead of the performance testing framework, so export it although as a test-surface so that it will only be available in development builds and not pollute distributed libraries.
2009-06-07[configure] Check for FcInit()Chris Wilson1-0/+2
2009-05-15[memfault] Update macros to avoid namescape collision with memcheckChris Wilson1-31/+31
Basing the macro names of the memfault skin lead to a namespace collision with memcheck. After updating the headers, update cairo's usage to match.
2009-04-20[test] Call FcInit() manually.Chris Wilson1-0/+6
Pre-initialise fontconfig whilst memfault is disabled to avoid a lot of expensive, redundant testing of FcInit() throughout the test suite.
2009-04-16[test] Disable signal handling under valgrindChris Wilson1-1/+1
Capturing and handling the signals prevents valgrind from providing backtraces for fatal signals -- which is often more useful.
2009-03-31[test] Improve fault injection coverageChris Wilson1-0/+35
In order to exercise the meta-surfaces, we need to inject faults into cairo_surface_finish().
2009-01-29[test] Compile again without memfault.Chris Wilson1-10/+16
Hide the valgrind macro when not using memfault.
2009-01-29[test] Suppress suppressed memfault reportChris Wilson1-2/+9
Check to see if there are any *unsuppressed* memfaults before declaring unreported faults.
2009-01-02[test] Free test nameChris Wilson1-0/+3
Adding the missing free for the converted test name.
2008-11-05[test] Mark targets with is_meta?Chris Wilson1-1/+12
Allow tests to skip targets based on whether they are a meta surface or not.
2008-11-02Specify a background color for XFAIL textAdrian Johnson1-1/+1
To improve readability on terminals with a white background.
2008-10-31[test] Use '.' as the field separator in the namesChris Wilson1-20/+48
We frequently use '-' within the test name or format name and so we encounter confusion as '-' is also used as the field separator. At times this has caused a new test to break an old test because the new test would match one of the old test's target specific reference images. So switch everything over to use '.' between fields (test name, target, format, subtest, etc.).
2008-10-31[test] Build test suite into single binary.Chris Wilson1-261/+333
Avoid calling libtool to link every single test case, by building just one binary from all the sources. This binary is then given the task of choosing tests to run (based on user selection and individual test requirement), forking each test into its own process and accumulating the results.
2008-10-30[test] Reenable device offset testing.Chris Wilson1-1/+1
Restore device offset testing that was temporarily disable in the run up to the 1.8 release.
2008-10-18[test/get-clip] Check the status on the secondary context.Chris Wilson1-1/+13
As we solely use a secondary context, we must manually report NO_MEMORY errors whilst running under memfault.
2008-10-17[test] Add support for memfault.Chris Wilson1-58/+213
Add the core support to cairo-test for running the test-suite under a malloc fault injector. This commit contains the adjustments to cairo_test_run() to repeat the test if it detects a failure due to fault injection and complains if it detects unreported faults or memory leaks.
2008-10-11[test] Use _POSIX_C_SOURCE for flockfile.Chris Wilson1-2/+3
From bug 18010 (https://bugs.freedesktop.org/show_bug.cgi?id=18010), in order to make flockfile() available we need to set _POSIX_C_SOURCE and according to the man page, the appropriate feature check is _POSIX_THREAD_SAFE_FUNCTIONS.
2008-09-28Automate error checking for fallback-resolution.Chris Wilson1-18/+18
For this we extend the boilerplate get_image() routines to extract a single page out of a paginated document and then proceed to manually check each page of the fallback-resolution test. (Well that's the theory, in practice SVG doesn't support multiple pages and so we just generate a new surface for each resolution. But the infrastructure is in place so that we can automate other tests, e.g. test/multi-pages.)
2008-09-27[cairo-test] Fix cairo_test_file_is_older().Chris Wilson1-4/+7
It did not even compile, so no longer it wasn't deleting cached results if you updated the reference images.
2008-09-25[test] Write the OUTPUT filename to the log if we match the vector surface.Chris Wilson1-0/+3
If the vector surface matches the output from last time, then the rasterisation is skipped - but we need to write the expected OUTPUT filename to the log so that the image is referenced from index.html.
2008-09-24Remove the device-offset testing (-25 cases) for the release.Carl Worth1-1/+1
The reasoning behind the -25 testing is that we want to ensure that cairo provides translation invariance. However, for many vector backends we use external rasterizers that don't necessarily provide that translation invariance. So this testing makes a bunch of failures appear that we don't really care about, (and we don't even have a mechanism to turn them off with custom reference images). For the release, I'm just turning this off. After the release, I plan to turn this back on, and then we could fix this by ensuring that the vector output itself is unaffected by a device offset, or by moving away from external rasterizers, (see Chris's micro-gs work to test PostScript with cairo-based rasterization).
2008-09-15[test] Fixup potential use of uninitialized test_image.Chris Wilson1-2/+0
In the churn of reordering the tests, some of the error paths were left unaltered - causing potential invalid dereferences.
2008-09-11[test] Check for updated reference images.Chris Wilson1-20/+63
Delete the results of previous runs if the reference images are more recent. There's still potential error if the conversion utility or its required libraries are modified...
2008-09-11[test] Fix order of deleting images vs checks.Chris Wilson1-3/+2
The early-checks assume that the images are still available - so it helps not to unlink them before comparing results to the previous run.
2008-09-04[test] Do not include "xmalloc.h" directlyBehdad Esfahbod1-1/+0
Not needed, it's renamed and included by boilerplate directly.
2008-09-02Move _GNU_SOURCE declarations to where it's usedBehdad Esfahbod1-0/+2
Such that we don't rely on more GNU extensions accidentally.
2008-09-02[cairo-test] Fixup a few memleaks on failure.Chris Wilson1-0/+5
Free the test image before jumping to UNWIND_CAIRO.
2008-08-26[test] Ensure that the output is always saved.Chris Wilson1-27/+27
Do the saving of the output first before checking for various failures with respect to the reference images.
2008-08-21[test] Explicity save a fail image.Chris Wilson1-11/+11
Be explicit about handling cached FAIL images, instead of relying on the sequences of failed matches as the files are an external resource and we can not guarantee their individual accessibility. Note this also changes the filename, so you may want to run: $ find -name '*-last.*' -print | xargs rm after this checkout.
2008-08-20[test] Cache last output and compare next time.Chris Wilson1-21/+165
Compare the current output against a previous run to determine if there has been any change since last time, and only run through imagediff if there has been. For the vector surfaces, we can check the vector output first and potentially skip the rasterisation. On my machine this reduces the time for a second run from 6 minutes to 2m30s. As most of the time, most test output will remain unchanged, so this seems to be a big win. On unix systems, hard linking is used to reduce the amount of storage space required - others will see about a three-fold increase in the amount of disk used. The directory continues to be a stress test for file selectors. In order to reduce the changes between runs, the current time is no longer written to the PNG files (justified by that it only exists as a debugging aid) and the boilerplate tweaks the PS surface so that the creation date is fixed. To fully realise the benefits here, we need to strip the creation time from all the reference images... The biggest problem with using the caches is that different runs of the test suite can go through different code paths, introducing potential Heisenbergs. If you suspect that caching is interfering with the test results, use 'make -C test clean-caches check'.
2008-08-19[test] Destroy test ouput image on failure.Chris Wilson1-0/+1
Add the missing cairo_surface_destroy(). Sigh, less haste more speed.
2008-08-19[test] Always write the test output to a png.Chris Wilson1-7/+9
Always write the test output to a png so that it can be copied and used as a reference image with CAIRO_REF_DIR.
2008-08-19[test] Pass extended output name to ref_name_for_test().Chris Wilson1-8/+10
Having included some extra details in the test output PNG filename, we need to pass the extra information to cairo_ref_name_for_test_target_format() in order to find the match.
2008-08-18[test] Avoid redundant writes/reads of test surfaces via png.Chris Wilson1-28/+132
As Behdad suggested, we can dramatically speed up the test suite by short-circuiting the write to a png file, only to then immediately read it back in. So for the raster based surfaces, we avoid the round-trip through libpng by implementing a new boilerplate method to directly extract the image buffer from the test result. A secondary speedup is achieved by caching the most recent reference image.
2008-08-13[test] Pass a 'complete' name to create_surface().Chris Wilson1-38/+31
Construct the test name to pass to the boilerplate creation routines such that it uniquely identifies the test in terms of test, target, content and pass (similar, offset, thread). This allows the vector targets to create output different output files for each test, whereas before, later tests would overwrite existing files making debugging more difficult.