summaryrefslogtreecommitdiff
path: root/tko/parse.py
AgeCommit message (Collapse)AuthorFilesLines
2009-05-21Send the job dir, not the individual tests, to site_parsembligh1-3/+4
Signed-off-by: Rachel Kroll <rkroll@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3156 592f7852-d20e-0410-864c-8624ca9c26a4
2009-05-13Add post-parse site hooks (parse -P to trigger, default = off)mbligh1-0/+15
Make scheduler call tko/parse with -P Signed-off-by: Rachel Kroll <rkroll@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3123 592f7852-d20e-0410-864c-8624ca9c26a4
2009-04-22fix a simple bug in the parser -- in one code path, old_tests never gets ↵showard1-0/+1
bound, but it gets referenced later anyway Signed-off-by: Steve Howard <showard@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3033 592f7852-d20e-0410-864c-8624ca9c26a4
2009-04-22Explicitly check for empty results when looking for old tests. Otherwise ↵showard1-0/+2
we'll generate invalid SQL in the subsequent query. Signed-off-by: Steve Howard <showard@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3032 592f7852-d20e-0410-864c-8624ca9c26a4
2009-04-21Fix a bug in the parser when dealing with test labels. The finaljadmanski1-2/+50
reparse drops the existing test entries and replaces them with new ones, so that leaves behind a bunch of orphaned labels in the database and basically kills all the lables you've added. So instead we add some code to move the existing labels over the new entries that replace them (making a best effort to match up "new" and "old" entries). Add foreign keys to the test_labels_tests table. This also requires compacting test_labels_tests into an unsigned int(10), since that's what the tests.test_idx field is. Risk: Low Visibility: Fix up the parser's handling of test labels, and add foreign keys that should've been there from the start (but were dropped because of MyISAM vs InnoDB issues). Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3022 592f7852-d20e-0410-864c-8624ca9c26a4
2009-03-11Fix bug in parse_path where results from multi-machine synchronous jobs are ↵mbligh1-7/+16
ignored. This patch fixes a bug in parse_path where results from multi-machine synchronous jobs are not parsed correctly and do not show up in the database. Synchronous jobs keep result info in results/groupN/status.log. The current parser algorithm parses results in results/groupN/<subdirs>. This patch fixes that by adding parsing results/groupN in addition to the subdirs. Thanks to John Admanski for explaining the fix. Signed-off-by: Nikhil Rao <ncrao@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@2865 592f7852-d20e-0410-864c-8624ca9c26a4
2009-01-30Add some more slack to the parser when dealing with multi-machinejadmanski1-1/+5
jobs. If the job isn't neatly broken down into machine-specific subjobs, just accept it and use the set of hostnames as the "hostname". This should allow us to do meaningful parsing of multi-machine test results, even if things aren't as intelligent as we'd like. Risk: Low Visibility: Doesn't affect the parsing of anything that the parser can currently handle; allows us to do SOME parsing of generic multi-machine results. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@2720 592f7852-d20e-0410-864c-8624ca9c26a4
2009-01-30Fixed a typo in trying to bitwise OR non blocking locking which probablymbligh1-2/+2
made that the parse "-n" argument never worked. Signed-off-by: Mihai Rusu <dizzy@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@2703 592f7852-d20e-0410-864c-8624ca9c26a4
2008-12-03The parser should be using job.read_keyval to find job keyvals, notjadmanski1-11/+4
looking for the keyval itself. Risk: Low Visibility: Parsing multi-machine jobs with the standalone script will work again. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@2543 592f7852-d20e-0410-864c-8624ca9c26a4
2008-12-03Refactor the core pidfile code out of autoserv and into a common libjadmanski1-33/+48
module, and add a similar pidfile option to the parser. Risk: Low Visibility: Adds an (opt-in) .parser_execute pidfile to the parser. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@2538 592f7852-d20e-0410-864c-8624ca9c26a4
2008-11-11Fix the parsing script so that if you have multiple levels ofjadmanski1-14/+26
nesting before you reach the actual job directories, it still correctly parses them. Risk: Low Visibility: Only affects the "reparse entire results dir" mode of the parse.py script. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@2394 592f7852-d20e-0410-864c-8624ca9c26a4
2008-06-20Fix a bug that will keep server sync jobs from being properly parsed.jadmanski1-1/+1
The original intention of the code was always to parse individual machine directories when a .machines file exists. Risk: Low Visibility: Server-side sync jobs that are actually broken down into individual machine jobs will now get their results parsed. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1727 592f7852-d20e-0410-864c-8624ca9c26a4
2008-06-17Fix errors found by lint.mbligh1-1/+1
Signed-off-by: Jeremy Orlow <jorlow@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1712 592f7852-d20e-0410-864c-8624ca9c26a4
2008-06-06Convert all python code to use four-space indents instead of eight-space tabs.jadmanski1-173/+173
Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1658 592f7852-d20e-0410-864c-8624ca9c26a4
2008-05-29Risk: Mediumjadmanski1-3/+5
Visibility: Should eliminate missing results errors due to the final reparse failing because of intermittent db issues. Adds an autoretry to the transactions used for the final reparse. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1567 592f7852-d20e-0410-864c-8624ca9c26a4
2008-05-14Add the version 1 of the TKO parser, modify server_job to use thisjadmanski1-15/+26
version, and modify parse.py to look up the status version number in order to instantiate the correct parser version. Basically, it's an implementation of the parser that follows the spec as outlined in http://test.kernel.org/autotest/DraftParserSpecification. I did that by implementing the "version 1" parser, as opposed to the existing "version 0" parser, and it also adds some code to autotest itself to log the fact that the status logs being written out follow the "version 1" specification, so that when re-parsing existing results older logs will still be parsed using the old (rather ad-hoc and difficult to follow) algorithm. The implementation is fairly similar to the existing version 0 implementation; it still uses all the same files for gathering results, but there are the core changes: - instead of grabbing kernel information from build.log, it gets embedded into the status logs associated with reboots - if a group is lacking a proper "END" section it implicitly assumes it was somehow aborted - reboots are wrapped in a group - a "JOB" result is logged even if nothing bad happens (rather than only logging it when something bad happens) - you can have arbitrarily large amounts of group nesting Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1511 592f7852-d20e-0410-864c-8624ca9c26a4
2008-04-17Convert the parser into an embeddable library. This includes a majormbligh1-410/+182
re-organization of the existing code. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1447 592f7852-d20e-0410-864c-8624ca9c26a4
2008-03-28Change the parser to never ignore reboot.* lines due to their indent.mbligh1-1/+2
Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1377 592f7852-d20e-0410-864c-8624ca9c26a4
2008-03-26Once the parser hits the end job marker, start looking for top-levelmbligh1-0/+4
events again. From: <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1366 592f7852-d20e-0410-864c-8624ca9c26a4
2008-03-25Add kernel information to reboot.verify logging, and add support tombligh1-3/+8
the parser to use it as a source of kernel version information. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1359 592f7852-d20e-0410-864c-8624ca9c26a4
2008-03-05Properly indent console WARN messages that appear inside of START-ENDmbligh1-9/+48
groups. Also, modify the parser to interpret the status of a group as being the "worst" status message logged in that group, not just whatever status is associated with the END message. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1299 592f7852-d20e-0410-864c-8624ca9c26a4
2008-02-27Parse is improperly handling the new numbered boots, occasionallymbligh1-1/+1
treated a reboot line as a normal test instead of properly tracking whether or not the machine is currently rebooting. This leads to cases where the job is mistakenly reported as aborted because of a failed reboot due to the reboot.verify line being misinterpreted. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1265 592f7852-d20e-0410-864c-8624ca9c26a4
2008-02-21split patched kernels into their own kernel rows V2apw1-1/+1
When we have tests on a kernel such as 2.6.24 and against 2.6.24+patch they are lumped together in the same results row. This is not optimal as commonly you are patching those kernels in order to fix a problem and want to see those separately. Ideally we would have a row for the virgin 2.6.24 results and a separate row for each different patch combination. The current structure simply displays the contents of the kernels table 'printable' field when displaying the matrix. That would necessitate encoding the base name and all applied patches into this field. Additionally if we desire to have tooltips, or links to individual patches the HTML encoding for that would need to be packed into this field. This is not desirable. This patch set modifies the meaning of the kernels table printable field, making it more of a "sortable" field. This field must be unique for all kernels, and also approximatly sortable into "kernel release" order. For kernels without patches, this is simply the base kernel version as normal. For patched kernels it is the base kernel version plus the kernel index number. This combination effectively allows the kernel to be sorted by base release, then by date. Now that the printable field is mearly a sortable identifier, it is no longer suitable for display. Thus this patch also introduces a new map abstraction for data fields. It supplies a mapper for the kernel field which uses the sortable kernel identifier to locate the base kernel and the full list of patches. For display purposes we want to see the kernel version plus the names of all the patches, all without allowing making the kernel column excessively wide. So the supplied mapper displays the base kernel version, plus truncated patch names (but with tooltips for the full names). Which ends up looking something like this: 2.6.24-mm1 p112 +add-new-string-functions- +x86-amd-thermal-interrupt This mapping does introduce a single additional query over the patches table (which is pretty small) before the matrix can be produced, only when the kernel field is displayed. Signed-off-by: Andy Whitcroft <apw@shadowen.org> git-svn-id: svn://test.kernel.org/autotest/trunk@1261 592f7852-d20e-0410-864c-8624ca9c26a4
2008-02-21tko: boot errors should track boot numbermbligh1-2/+3
Recently boot numbering was introduced. However, boot failures were missed. These should also be reported against the current boot number. Signed-off-by: Andy Whitcroft <apw@shadowen.org> git-svn-id: svn://test.kernel.org/autotest/trunk@1256 592f7852-d20e-0410-864c-8624ca9c26a4
2008-02-19Strip the optional fields (e.g. timestamps) out of the reason fieldsmbligh1-13/+27
when parsing test results. Also, add a test_finished_time field for storing the timestamp in the database. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1248 592f7852-d20e-0410-864c-8624ca9c26a4
2008-02-19Add job queued, started and finished timestamps to the tko db andmbligh1-1/+11
modify the parsing code to find these timestamps in the job keyval file and insert them into the db. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1246 592f7852-d20e-0410-864c-8624ca9c26a4
2008-02-12Label different boots differently so we don't have duplicate 'test'mbligh1-1/+3
names in the tko db. Signed-off-by: Jeremy Orlow <jorlow@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1234 592f7852-d20e-0410-864c-8624ca9c26a4
2008-01-10Fix up parsing to be a bit more robust, and change the warning_monitormbligh1-2/+9
not to print STATUS Signed-off-by: Martin Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1141 592f7852-d20e-0410-864c-8624ca9c26a4
2008-01-10Fix parser to recognize job abortsmbligh1-4/+13
Signed-off-by: Martin Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1140 592f7852-d20e-0410-864c-8624ca9c26a4
2008-01-03Fix parse to not get confused by reboots outside of the mainmbligh1-4/+5
job block - we were catching the start, but not the end, and thinking there was an unfinished reboot. Treat both the same Signed-off-by: Martin Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1106 592f7852-d20e-0410-864c-8624ca9c26a4
2007-12-19Fix up parse so it can reparse a job - allows continuous statusmbligh1-3/+9
Signed-off-by: Martin Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1087 592f7852-d20e-0410-864c-8624ca9c26a4
2007-12-13tko: add a new ALERT state and load it from the ALERT recordsapw1-0/+9
Use the ALERT records (triggered by kernel BUG's etc) in the status file to mark in progress tests as having triggered the alert. Signed-off-by: Andy Whitcroft <apw@shadowen.org> git-svn-id: svn://test.kernel.org/autotest/trunk@1070 592f7852-d20e-0410-864c-8624ca9c26a4
2007-12-03Some databases do not like numbers to have newlines on the end. Clean upmbligh1-0/+1
then values before recording them. Signed-off-by: Andy Whitcroft <apw@shadowen.org> git-svn-id: svn://test.kernel.org/autotest/trunk@1016 592f7852-d20e-0410-864c-8624ca9c26a4
2007-12-03When a boot is initiated in a job, but the machine fails to come up wembligh1-0/+10
do not run boot validation. Unless boot validation is run the boot is not recorded in the test results; which only include the build. Track the boot initiation and where the status file completes without such an indication record a boot failure. Signed-off-by: Andy Whitcroft <apw@shadowen.org> git-svn-id: svn://test.kernel.org/autotest/trunk@1014 592f7852-d20e-0410-864c-8624ca9c26a4
2007-11-30Add code to deal with machine owners and their displaymbligh1-2/+3
Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1010 592f7852-d20e-0410-864c-8624ca9c26a4
2007-11-29Fix up parse for new status formatmbligh1-10/+37
Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1006 592f7852-d20e-0410-864c-8624ca9c26a4
2007-11-26Fix up crappy keval handling in parsembligh1-19/+32
Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@979 592f7852-d20e-0410-864c-8624ca9c26a4
2007-11-24Skip job-level events in parse (don't pretend they're tests)mbligh1-0/+3
This avoids us getting tests that look like '----' in the results pages Signed-off-by: Martin Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@962 592f7852-d20e-0410-864c-8624ca9c26a4
2007-11-24Fix spacing in parse.py and remove a few old regexpsmbligh1-28/+24
Signed-off-by: Martin Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@961 592f7852-d20e-0410-864c-8624ca9c26a4
2007-11-24Kludge parse.py to understand new reboot format.mbligh1-10/+12
A better fix is slightly tricky, as we don't know easily which reboot matches up to which test. Signed-off-by: Martin Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@957 592f7852-d20e-0410-864c-8624ca9c26a4
2007-11-05Add email reporting to failing tests when parsing resultsmbligh1-2/+42
* added failtest for testing this (and anything else that needs a failing test ) * Improved readability in parse script output * Added getopt() for parse and added -m option for sending mail on failures * pulled mail* functions in from mirror/mirror * Get a realuser value in autoserv for run tests * Fix jobkeyval value From: David McMahon <djmm@google.com> Signed-off-by: Martin Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@910 592f7852-d20e-0410-864c-8624ca9c26a4
2007-11-05Fix parsing - subdir is often None, so move keyval2 checking insidembligh1-9/+10
the if clause Signed-off-by: Martin Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@882 592f7852-d20e-0410-864c-8624ca9c26a4
2007-10-25Make test parser insert the test version into the tko db.mbligh1-0/+8
From: David McMahon <djmm@google.com> Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@842 592f7852-d20e-0410-864c-8624ca9c26a4
2007-10-17DB schema/parse change to store user name and user assigned labels for tests.mbligh1-0/+14
From: radha@google.com git-svn-id: svn://test.kernel.org/autotest/trunk@813 592f7852-d20e-0410-864c-8624ca9c26a4
2007-09-30Fix up parsing for no kernelmbligh1-20/+24
Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@722 592f7852-d20e-0410-864c-8624ca9c26a4
2007-09-26Update tko to parse latest status file formatmbligh1-9/+57
Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@691 592f7852-d20e-0410-864c-8624ca9c26a4
2007-09-20add iterations from keyval properly when parsingmbligh1-6/+7
Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@679 592f7852-d20e-0410-864c-8624ca9c26a4
2007-09-14tko fixupsmbligh1-4/+3
git-svn-id: svn://test.kernel.org/autotest/trunk@669 592f7852-d20e-0410-864c-8624ca9c26a4
2007-09-14tko updatesmbligh1-0/+2
git-svn-id: svn://test.kernel.org/autotest/trunk@667 592f7852-d20e-0410-864c-8624ca9c26a4
2007-09-13Fix up parsing bug - load all testsmbligh1-5/+10
Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@664 592f7852-d20e-0410-864c-8624ca9c26a4
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-25 12:30:14 +0000