summaryrefslogtreecommitdiff
path: root/ezbench.sh
AgeCommit message (Collapse)AuthorFilesLines
2015-08-31core: update the headerMartin Peres1-3/+5
2015-08-24core: use --format=fuller when generating the patchMartin Peres1-1/+1
2015-08-18core: Improve comparison with uninitialized earlier run dataChris Wilson1-11/+6
Check against an undefined PrevTest result so that a failed read of the last run does not print out garbage to the console.
2015-08-15core: Add -B option (inverse of -b) to allow excluding arbitrary testsChris Wilson1-3/+16
As we have multiple different versions of a test, for example disabling rendering in order to purely measure the CPU overhead, we want an easy method to exclude those from a round of testing.
2015-08-14core: fix the feature allowing to pass a list of commitsMartin Peres1-11/+12
2015-08-13core: Allow tests to invert the ratioChris Wilson1-1/+8
Not all tests are fps (or operations per second) and instead measure duration. For these, we need to invert the ratio between tests.
2015-08-13core: Make the tests folder an optional argumentChris Wilson1-44/+49
By default keep using ezBenchDir/tests.d/, but allow the user to override. Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
2015-08-13core: Adjust geometric mean awk script for older versionsChris Wilson1-1/+1
Older versions of gawk do not have the '**' operator so to perform the equivalent pow() we need to use the logarithm. Note that we pass in a single line and need to iterate over fields in that line as opposed to summing over multiple lines.
2015-08-13core: save the commits as we goMartin Peres1-0/+2
2015-07-30core: add a new variable containing the benchmark's nameMartin Peres1-0/+2
2015-07-30core: add global pre/post hooks for all the benchmarksMartin Peres1-0/+2
2015-07-28Provide a conf.d for splitting up the callback definitionsChris Wilson1-1/+4
2015-07-28Use LC_LOCALE=C for numeric interchangeChris Wilson1-1/+1
Otherwise those of us in non-US regions get warning messages about unknown locale!
2015-07-28core: fix the geometric mean computationMartin Peres1-1/+1
2015-07-28core: force the usage of the us locale for numeric operationsMartin Peres1-0/+3
2015-07-27Run the ezbench_pre_hook before querying the testsChris Wilson1-3/+3
This way I can put my machine setup inside the pre-hook for the run.
2015-07-27Detect test run failure and highlight itChris Wilson1-1/+8
If the run fails to return any values or reported an error, keep the error logs and highlight the failure in the console.
2015-07-27Seed the output results with the final commit from the previous roundChris Wilson1-3/+14
When we split the testing over multiple runs, it is nice for the pretty console output if we lookup the final result and use that for our diff.
2015-07-27Only source the .test files onceChris Wilson1-45/+43
As the .test scripts grow more complex, we only want to do one pass through them.
2015-07-27Allow .test to define multiple tests at onceChris Wilson1-13/+17
2015-07-27Allow the user to provide an optional run name to concatenate resultsChris Wilson1-4/+7
v2: ()Martin Peres) - add the documentation in show-help
2015-07-27Allow multiple -b optionsChris Wilson1-1/+1
Append the next -b selection onto the list of benchmarks.
2015-07-27Provide a conf.d for splitting up the callback definitionsChris Wilson1-1/+2
2015-07-27Allow arbitrary git commit ids and ranges to be specified on the commandlineChris Wilson1-5/+18
2015-07-27Don't add a 0 result to the list for geometric mean processingChris Wilson1-1/+1
2015-07-27Remove extra unguarded compile_pre_hookChris Wilson1-2/+0
Just a typo, nothing to see here.
2015-07-23core: use the long version for the year for the log dir (2015 instead of 15)Martin Peres1-1/+1
2015-07-22Allow individual tests to provide a hook to produce a global statisticChris Wilson1-10/+16
For the slightly more complex tests that do not just measure a constant workload and so need something different than a mean average, let them provide a hook to convert their output into a single statistic useful for tracking differences between commits. The full output is still available for detailed analysis.
2015-07-22Allow the user to disable compilationChris Wilson1-17/+33
v2: (Martin Peres) - reuse the -m option instead - document the behaviour - make sure the compilation time is not included in the estimation This is a small step towards making the benchmarking two phase in order to allow for say benchmarking the kernel over many commits (where we need to reboot between commits).
2015-07-22Save the per-run FPS values where we amalgamateChris Wilson1-3/+4
v2: (Martin Peres) - make benchmarks only output one fps reading per round - fix gen_report to ignore the per-run values
2015-07-22Use globstar to enable subdirectories in tests.dChris Wilson1-2/+7
2015-07-21Be flexible in choosing where to benchmark uptoChris Wilson1-4/+9
Allow the user to pass in the target commit. This is just a first step to allow the user to parse in arbitrary git ranges.
2015-07-21Make the FPS difference computations robust against divide-by-zeroChris Wilson1-8/+8
2015-07-21Reset the FPS summation between runsChris Wilson1-0/+1
If we don't remember to reset the FPS counter, the geometric mean gradually averages the result of the entire test run - rendering its comparison very confusing.
2015-07-20core: dump the commit list as we go to enable offline analysisMartin Peres1-0/+5
2015-07-20core: switch to the finish date instead of just the execution timeMartin Peres1-1/+2
2015-07-20core: general style fixesMartin Peres1-11/+12
2015-07-20Finish each commit by showing the geometric meanChris Wilson1-0/+25
The geometric mean helps to summarise all of the individual tests into a single traffic.
2015-07-20Abort if we fail to stash the current dirty stateChris Wilson1-0/+5
We never want to lose work-in-progress, so abort if we cannot stash it away.
2015-07-20Add some color to the fps diff outputChris Wilson1-12/+30
I could say that the color provide a quick guide as to how the runs are fairing, but really it just makes the output pretty. Just as important is to just the text alignment so it becomes easier to scan through the numbers.
2015-07-20Place the repo check before setting up the termination handlerChris Wilson1-12/+12
If we don't have a git repository we exit early and have nothing to restore.
2015-07-20Replace git log parsing with git plumbingChris Wilson1-7/+4
Instead of piping 'git log' output through a text pipeline, just use the git plumbing commands directly to generate the commit lists.
2015-07-20Try benchmarking any work-in-progressChris Wilson1-4/+13
2015-07-20Preserve local modificationsChris Wilson1-0/+5
Next challenge will be to benchmark them as well
2015-07-20Reverse computation of perf changesChris Wilson1-1/+1
An increase from 6000 to 15000 fps should be reported as an 150% improvement not, a mere, 60%.
2015-07-20core: redirect the tests' output to an error fileMartin Peres1-4/+10
2015-07-20Use 'git show --format' to generate oneline commit identifierChris Wilson1-2/+1
Using the format for git show we can do the name generation in a single step, throw in a bit of color and avoid including an extra character into the commit id when copy-pasting.
2015-07-17core: save the logs in different foldersMartin Peres1-5/+5
2015-07-16core: let the tests make sure that they do not print anything in stderrMartin Peres1-1/+1
This makes it easier to debug what the heck is going wrong when it does :)
2015-07-16core: add support for hooksMartin Peres1-1/+24