summaryrefslogtreecommitdiff
path: root/tko
AgeCommit message (Collapse)AuthorFilesLines
2011-06-25tko.parsers: Support multiline, tabbed reasons. Improve parser robustness.HEADmasterlmr3-22/+45
Allowing reasons with multiple lines and tabs provides more flexibility in error messages. The parser is also more robust, as any invalid fields are moved into the reason and can be fixed offline without impacting the rest of the system. Signed-off-by: Dale Curtis <dalecurtis@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@5452 592f7852-d20e-0410-864c-8624ca9c26a4
2011-03-02Results parser: Handle errors parsing lines more properlylmr1-1/+7
Use a better error message to help debug what is going on. I don't see how that regex could fail off the top of my head, even an empty string should match and return '' and '' for the groups. (update: What we're seeing after this is that somehow the parser is getting a "line" that contains newlines at times; not sure how) Signed-off-by: Gregory Smith <gps@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@5274 592f7852-d20e-0410-864c-8624ca9c26a4
2010-11-27Perf metrics with negative values never get into tko databaselmr2-11/+14
Perf metrics with negative values never get into tko database. Generalize syntax of perf value to allow any float literal. Signed-off-by: Duane Sand <duanes@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4963 592f7852-d20e-0410-864c-8624ca9c26a4
2010-11-08Massive permission and indentation fix on the autotest treelmr2-0/+0
Once again I went through the autotest tree with my automated script to check all autotest source, and discovered several places that needed permission and indentation (including trailing whitespace) fixes. I have checked the generated patch and indeed it looks OK. Will commit this shortly, as preparation for autotest 0.13. Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4907 592f7852-d20e-0410-864c-8624ca9c26a4
2010-09-14Add support to the parser and server job for logging the result of an exceptionjadmanski1-2/+7
when it is raised within the server control file and not inside of a job. Currently these exceptions are only logged into the debug/ logs and not into the status log (and thus not into the database). It adds support by logging an INFO message with a special job_abort_reason field when a server job fails with an exception, and the parser then special-cases these info lines (much like it already does with kernel version INFO lines) and updates the SERVER_JOB entry. Unfortunately this can't easily be done with a "normal" status log entry since when a failure occurs outside of a test the special SERVER_JOB entry is the only place where this reason information actually belongs, but the parser has no existing mechanism for modifying this entry (since it's automatically generated by the parser, rather than being generated in response to a specific log). Adding this (relatively small) special case seemed more practical than either adding yet another special entry or trying to perform a major restructuring of the parser. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4785 592f7852-d20e-0410-864c-8624ca9c26a4
2010-08-05Fix some table names in tko/parse.pyjamesren1-5/+5
Signed-off-by: James Ren <jamesren@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4746 592f7852-d20e-0410-864c-8624ca9c26a4
2010-07-20Disregarding patches in the job serializing process because they are never ↵jamesren3-67/+5
used in our system anymore. Patches still exist in tko. OK to submit to non-main branch. Signed-off-by: Darren Kuo <darrenkuo@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4730 592f7852-d20e-0410-864c-8624ca9c26a4
2010-06-16Added more fields to support all the field in the test view.jamesren5-23/+75
Signed-off-by: Darren Kuo <darrenkuo@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4624 592f7852-d20e-0410-864c-8624ca9c26a4
2010-06-15Fix test fields names and removed some write out to binary code.jamesren1-5/+3
Signed-off-by: Darren Kuo <darrenkuo@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4622 592f7852-d20e-0410-864c-8624ca9c26a4
2010-06-15Adding site-specific hook for exporting data.jamesren1-1/+13
Signed-off-by: Darren Kuo <darrenkuo@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4621 592f7852-d20e-0410-864c-8624ca9c26a4
2010-06-10Fixed tko.proto to have inner message 'classes' to serialize correctly. ↵jamesren3-46/+50
Fixed Status setup in job_serializer_unittest. OK to submit to non-main branch. Signed-off-by: Darren Kuo <darrenkuo@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4599 592f7852-d20e-0410-864c-8624ca9c26a4
2010-06-10Interface between tko parse and whatever the other client is for shipping ↵jamesren4-0/+846
out data. OK to submit to non-main branch. Signed-off-by: Darren Kuo <darrenkuo@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4598 592f7852-d20e-0410-864c-8624ca9c26a4
2010-06-10Fixed a minor bug in version_0.py. The constructor was missing an argument. ↵jamesren1-2/+2
The number of arguments to the version_0.test constructor was inconsistent with that of test constructor in version_1 and model. Signed-off-by: Darren Kuo <darrenkuo@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4597 592f7852-d20e-0410-864c-8624ca9c26a4
2010-05-04Fix the recent CLIENT_JOB changes in the parser to not conflict withjadmanski1-6/+7
the SERVER_JOB changes, and to properly tie the RUNNING entry to the final CLIENT_JOB entry. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4464 592f7852-d20e-0410-864c-8624ca9c26a4
2010-04-30Fix the parser to report a "RUNNING" status for CLIENT_JOB.* entriesjadmanski1-0/+10
in TKO, analogous to the same entry for SERVER_JOB. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4454 592f7852-d20e-0410-864c-8624ca9c26a4
2010-04-15More fixes to hard-coded values in tko/display.pyjamesren1-9/+10
Signed-off-by: James Ren <jamesren@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4418 592f7852-d20e-0410-864c-8624ca9c26a4
2010-04-13Fix display of embedded results table on job details tabjamesren1-6/+9
Signed-off-by: James Ren <jamesren@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4408 592f7852-d20e-0410-864c-8624ca9c26a4
2010-03-23Rebase the AUTOTEST_WEB database to schema version 51. Also default thejamesren1-175/+2
migration manager to the AUTOTEST_WEB database, since the TKO database has been removed. Signed-off-by: James Ren <jamesren@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4317 592f7852-d20e-0410-864c-8624ca9c26a4
2010-02-04Autotest: Porting all hash operations to use utils.hashlmr2-5/+6
Replace all functions that evaluate md5 and sha1 hexdigests with the function utils.hash(). Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4204 592f7852-d20e-0410-864c-8624ca9c26a4
2010-01-26Table name in tko database have been changed. So update table name inmbligh1-1/+1
tko/compose_query.cgi. This patch will fix error "Table 'tko.kernels' doesn't exist:" in 'Results database' page. Signed-off-by: Feng Yang <fyang@redhat.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4172 592f7852-d20e-0410-864c-8624ca9c26a4
2010-01-15Support for job keyvalsshoward4-41/+64
* can be passed as an argument to create_job, stored in AFE DB * scheduler reads them from the AFE DB and writes them to the job-level keyval file before the job starts * parser reads them from the keyval file and writes them to the TKO DB in a new table Since the field name "key" happens to be a MySQL keyword, I went ahead and made db.py support proper quoting of field names. Evetually it'd be really nice to deprecate db.py and use Django models exclusively, but that is a far-off dream. Still lacking support in the AFE and TKO web clients and CLIs, at least the TKO part will be coming soon Signed-off-by: Steve Howard <showard@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4123 592f7852-d20e-0410-864c-8624ca9c26a4
2010-01-12Merge the two databases and the two Django projects. Note that the TKO ↵showard2-8/+8
migrations are still present, as is the migration code for TKO in database/migrate.py. These pieces of code are preserved because they are required by this change to perform the merge. A separate change will be submitted in the future to remove those references. WARNING: This change will move all tables in TKO into the AUTOTEST_WEB database. Custom TKO tables that Autotest does not know about will NOT be moved, and will be lost after this change. Any scripts that run directly against the TKO database will cease functioning until they are modified to run against AUTOTEST_WEB. Additionally, if you were importing any .py files from autotest_lib.new_tko.tko.*, they have been moved to autotest_lib.frontend.tko.* Notes: You will need to manually create two files after applying this patch touch <autotest_dir>/frontend/tko/__init__.py touch <autotest_dir>/tko/migrations/__init__.py Optionally, you can also remove the entire <autotest_dir>/new_tko/ directory, as it is no longer needed. Signed-off-by: James Ren <jamesren@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4106 592f7852-d20e-0410-864c-8624ca9c26a4
2010-01-11Massive permission fixlmr5-0/+0
Fix permissions for all the development tree Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4094 592f7852-d20e-0410-864c-8624ca9c26a4
2010-01-11Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>mbligh1-1/+1
git-svn-id: svn://test.kernel.org/autotest/trunk@4089 592f7852-d20e-0410-864c-8624ca9c26a4
2010-01-05Add a new Archiving stage to the scheduler, which runs after Parsing. This ↵mbligh1-13/+0
stage is responsible for copying results to the results server in a drone setup, a task currently performed directly by the scheduler, and allows for site-specific archiving functionality, replacing the site_parse functionality. It does this by running autoserv with a special control file (scheduler/archive_results.control.srv), which loads and runs code from the new scheduler.archive_results module. The implementation was mostly straightfoward, as the archiving stage is fully analogous to the parser stage. I did make a couple of refactorings: * factored out the parser throttling code into a common superclass that the ArchiveResultsTask could share * added some generic flags to Autoserv to duplicate special-case functionality we'd added for the --collect-crashinfo option -- namely, specifying a different pidfile name and specifying that autoserv should allow (and even expect) an existing results directory. in the future, i think it'd be more elegant to make crashinfo collection run using a special control file (as archiving works), rather than a hard-coded command-line option. * moved call to server_job.init_parser() out of the constructor, since this was an easy source of exceptions that wouldn't get logged. Note I believe some of the functional test changes slipped into my previous change there, which is why that looks smaller than you'd expect. Signed-off-by: Steve Howard <showard@google.com> ==== (deleted) //depot/google_vendor_src_branch/autotest/tko/site_parse.py ==== git-svn-id: svn://test.kernel.org/autotest/trunk@4070 592f7852-d20e-0410-864c-8624ca9c26a4
2009-12-23Rename the tables in the databases, by prefixing the app name. This isshoward11-194/+441
in preparation for merging the two databases and the two Django projects into one. Note that this renames *all* standard Autotest DB tables in both the autotest_web and tko databases. If you have scripts written directly against these databases, *they will break*. If your scripts access the RPC interfaces, they should continue to work. Another patch will be along within the next few weeks to actually move the TKO tables into the autotest_web database. From: James Ren <jamesren@google.com> Signed-off-by: Steve Howard <showard@google.com> Rename the tables in the databases, by prefixing the app name. This is in preparation for merging the two databases and the two Django projects into one. Note that this renames *all* standard Autotest DB tables in both the autotest_web and tko databases. If you have scripts written directly against these databases, *they will break*. If your scripts access the RPC interfaces, they should continue to work. From: James Ren <jamesren@google.com> Signed-off-by: Steve Howard <showard@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4040 592f7852-d20e-0410-864c-8624ca9c26a4
2009-12-04The reparse of existing job is done by a delete and then an insert, thisshoward2-58/+45
introduces a new job_idx/test_idx for the same job and voids the old ones, so users won't be able to use job_idx/test_idx to refer to job/test as their index might change. This cl makes reparse to keep the same job_idx and test_idx for existing jobs. Also fixes a bug where user created test attributes are being deleted inadvertently during reparse. Signed-off-by: Jiqing Tang <jiqingtang@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3992 592f7852-d20e-0410-864c-8624ca9c26a4
2009-12-02Update retrieve_logs to call the site_find_repository even if the results ↵mbligh1-19/+22
server is localhost. There is no reason why we can't have an additional remote repo with a localhost results server. Signed-off-by: Scott Zawalski <scottz@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3989 592f7852-d20e-0410-864c-8624ca9c26a4
2009-10-14update crackerjack URLmbligh1-1/+1
Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3857 592f7852-d20e-0410-864c-8624ca9c26a4
2009-09-11fix jsonp_fetcher, it was broken by changes to ServiceHandler in previous ↵showard1-6/+6
patch to implement RPC logging Signed-off-by: Steve Howard <showard@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3706 592f7852-d20e-0410-864c-8624ca9c26a4
2009-09-08Added afe_job_id column to the jobs table in TKO. Changed link in AFEshoward2-1/+85
View Job tab to use afe_job_id instead of job_tag. Signed-off-by: James Ren <jamesren@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3665 592f7852-d20e-0410-864c-8624ca9c26a4
2009-08-31TKO migration to ensure the "invalidated" label exists.showard1-0/+15
Signed-off-by: Steve Howard <showard@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3629 592f7852-d20e-0410-864c-8624ca9c26a4
2009-08-27Make a bunch of stuff executablembligh10-0/+0
git-svn-id: svn://test.kernel.org/autotest/trunk@3621 592f7852-d20e-0410-864c-8624ca9c26a4
2009-08-24New versions of twoway container benchmark graphs.mbligh1-338/+748
Remove site-specific details from nightly.py. Replace uplevel vars of nested functions by class object. Signed-off-by: Duane Sand <duanes@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3593 592f7852-d20e-0410-864c-8624ca9c26a4
2009-08-20Make the parser read and use the machine platform from host_keyvals, if ↵showard4-40/+125
available. Goodbye machine_transfer! Also made it handle multimachine jobs (use a comma-separated, sorted list of distinct platforms from the individual machines) and atomic group jobs (use the same atomic group override value that was used for machine name, when applicable). Signed-off-by: Steve Howard <showard@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3583 592f7852-d20e-0410-864c-8624ca9c26a4
2009-08-11Delete stale benchmark graphs.mbligh2-153/+0
Signed-off-by: Duane Sand <duanes@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3523 592f7852-d20e-0410-864c-8624ca9c26a4
2009-08-03This is the result of a batch reindent.py across our tree.mbligh4-34/+32
As Martin pointed out, we ought to be more careful and create a pre-svn commit script to avoid inserting trash in the tree, meanwhile, this is a good start to cleanup things Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3487 592f7852-d20e-0410-864c-8624ca9c26a4
2009-07-28Update retrieve logs to allow for a more dynamic site_retrieve_logs that canmbligh1-5/+17
return a host. Rename retrieve_hosts site function to site_retrieve_hosts Add host:port support to get_full_url Signed-off-by: Scott Zawalski <scottz@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3470 592f7852-d20e-0410-864c-8624ca9c26a4
2009-07-20Change the parser handling of perf keyvals to bypass any non-numeric values.jadmanski3-19/+55
MySQL will interpret those as zero when we really don't want to include values like NaN at all. This also adds some unit tests for this function and refactors the function. Risk: Low Visibility: Non-numeric perf values will no longer get written into the DB. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3424 592f7852-d20e-0410-864c-8624ca9c26a4
2009-07-11For files that won't be directly executed, the #! is unneccessary and canmbligh4-5/+0
confuse some tools that attempt to detect what the file is. Best to just not have them there. Signed-off-by: Jesse Keating <jkeating@redhat.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3407 592f7852-d20e-0410-864c-8624ca9c26a4
2009-07-11Update parser unittests to use autotempmbligh2-11/+19
Signed-off-by: Scott Zawalski <scottz@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3397 592f7852-d20e-0410-864c-8624ca9c26a4
2009-07-01Make some changes to the parser so that when multiple reasons arejadmanski3-0/+48
logged not only are duplicates discarded, but also reason strings which are subsets of one another. So for example if a test logs X and then X,Y and then X,Y,Z then the final reason will only include X,Y,Z. Risk: Low Visibility: Should reduce the amount of duplication in final reasons for multi-reason failures. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3346 592f7852-d20e-0410-864c-8624ca9c26a4
2009-06-22Not all distros put a symlink in for the python version. Howevermbligh3-3/+3
all should have a /usr/bin/python target. There are still a couple files that hardcode a /usr/lib/python2.4 path, but that's a harder problem to solve and outside the scope of this patch. Signed-off-by: Jesse Keating <jkeating@redhat.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3319 592f7852-d20e-0410-864c-8624ca9c26a4
2009-06-22Dashboard for Kernel Qual Performance benchmarks.mbligh1-1/+2
Signed-off-by: Duane Sand <duanes@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3318 592f7852-d20e-0410-864c-8624ca9c26a4
2009-06-22Revise dashboard of Kernel Qual Container benchmarks for nightly dev kernels.mbligh1-35/+71
Signed-off-by: Duane Sand <duanes@google.com> ==== (deleted) //depot/google_vendor_src_branch/autotest/tko/site_kq_cont_nightly.cgi ==== git-svn-id: svn://test.kernel.org/autotest/trunk@3317 592f7852-d20e-0410-864c-8624ca9c26a4
2009-06-22Move client/bin/kernel_versions to common_lib; it is used by tko.mbligh4-7/+6
Signed-off-by: Duane Sand <duanes@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3316 592f7852-d20e-0410-864c-8624ca9c26a4
2009-06-10If a group doesn't define a subdir, and it executes "inner" tests thatjadmanski1-0/+2
do, apply that same subdir to the outer group instead of simply using nothing. Risk: Low Visibility: Changes some subdir handling for groups. Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3241 592f7852-d20e-0410-864c-8624ca9c26a4
2009-05-26Always use the host_group_name even if a sysinfo/hostname file existsmbligh2-1/+25
when the keyvals hostname is a list of machines. Fixes the version_0 host_group_name unittests to stub out find_hostname so that local files are not read. This also verifies that find_hostname is not called when a host_group_name is present. Signed-off-by: Gregory Smith <gps@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3178 592f7852-d20e-0410-864c-8624ca9c26a4
2009-05-21Send the job dir, not the individual tests, to site_parsembligh1-3/+4
Signed-off-by: Rachel Kroll <rkroll@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3156 592f7852-d20e-0410-864c-8624ca9c26a4
2009-05-13Add post-parse site hooks (parse -P to trigger, default = off)mbligh1-0/+15
Make scheduler call tko/parse with -P Signed-off-by: Rachel Kroll <rkroll@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3123 592f7852-d20e-0410-864c-8624ca9c26a4
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-23 08:24:45 +0000