summaryrefslogtreecommitdiff
path: root/mirror
AgeCommit message (Collapse)AuthorFilesLines
2011-05-09mirror.config-sample: Fixing undefined variable error more properlylmr1-2/+3
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com> git-svn-id: svn://test.kernel.org/autotest/trunk@5360 592f7852-d20e-0410-864c-8624ca9c26a4
2011-05-06mirror.config-sample: Fixing undefined variable errorlmr1-2/+2
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com> git-svn-id: svn://test.kernel.org/autotest/trunk@5359 592f7852-d20e-0410-864c-8624ca9c26a4
2010-05-27Added a directory contents based source for "mirror" and updated mbligh2-27/+161
unittests. Signed-off-by: Mihai Rusu <dizzy@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4565 592f7852-d20e-0410-864c-8624ca9c26a4
2010-03-24Fixing the unittest that had wrong lists and only worked because we zipped ↵mbligh1-0/+2
the lists to compare them Signed-off-by: Jean-Marc Eurin <jmeurin@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4334 592f7852-d20e-0410-864c-8624ca9c26a4
2010-03-11Update source_unittest to specify the timezone so file checks pass properly ↵mbligh1-1/+11
when scraping rsync output. Signed-off-by: Scott Zawalski <scottz@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4295 592f7852-d20e-0410-864c-8624ca9c26a4
2010-01-11Massive permission fixlmr1-0/+0
Fix permissions for all the development tree Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com> git-svn-id: svn://test.kernel.org/autotest/trunk@4094 592f7852-d20e-0410-864c-8624ca9c26a4
2009-11-21The server.frontend wrapper of the RPC interface offers object syntax tombligh2-8/+14
access the dictionary entries returned by the RPC call so it breaks when using dictionary syntax on those. Fixed and updated unittests. Signed-off-by: Mihai Rusu <dizzy@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3964 592f7852-d20e-0410-864c-8624ca9c26a4
2009-11-21Added mirror.trigger.map_action optional upload_kernel_config flag tombligh2-18/+66
have it ask the control file generation RPC generate server side code to download and send to the client the kernel config file and make it aware of the new location. This allows to have clients receive kernel config files that they cannot access otherwise. It requires that the tests given to map_action are server side tests. Signed-off-by: Mihai Rusu <dizzy@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3963 592f7852-d20e-0410-864c-8624ca9c26a4
2009-11-06Changed mirror.trigger.map_action to use generate_control_file RPCmbligh3-99/+73
instead of doing its own generation logic. Updated sample config file and unittests. Signed-off-by: Mihai Rusu <dizzy@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3910 592f7852-d20e-0410-864c-8624ca9c26a4
2009-10-02Added support to mirror.trigger.map_action to tell it a username to usembligh1-4/+4
when connecting to the frontend (which will be set as the owner for the job). Signed-off-by: Mihai Rusu <dizzy@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3782 592f7852-d20e-0410-864c-8624ca9c26a4
2009-08-27Make a bunch of stuff executablembligh3-0/+0
git-svn-id: svn://test.kernel.org/autotest/trunk@3621 592f7852-d20e-0410-864c-8624ca9c26a4
2009-08-24Fixed 2 mirror script bugs:mbligh1-4/+4
- create a set when filtering the kernel files so that duplicates are removed - update the known kernels db before running the triggers not after because triggers are allowed to run for a long time and if one runs mirror while another mirror is already running the second one will again find the same new kernel files that the first one found Signed-off-by: Mihai Rusu <dizzy@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3598 592f7852-d20e-0410-864c-8624ca9c26a4
2009-08-24Improved reliability when saving "mirror" dict_database contents bymbligh2-21/+93
employing a fcntl.flock() based process synchronization to protect against 2 or more processes updating the db at the same time. Also switched to using a temporary file to protect against write failures (like disk full) situations from corrupting the existent data. Updated unittest. Signed-off-by: Mihai Rusu <dizzy@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3596 592f7852-d20e-0410-864c-8624ca9c26a4
2009-07-29Rewritten most of the mirror script around a new configuration file mbligh11-1269/+1140
written in Python code syntax to allow for much greater configuration flexibility. Also introduced the concept of a persistent database to store the list of known kernels (written an implementation that uses cPickle), the concept of a "source" object that provides the current list of kernels (written implementations for rsync and urllib2/HTML sources). Exposed the actions configured with the trigger object to the config file so anyone can configure whatever actions they want taken to handle the new kernel versions. Provided sample config file. When setting up the mirror script on a new installation you will probably want (after creating a config file for it) to run it first in "dry run" mode to initialize the database with the known kernels reported by the source at that moment (otherwise it will think all those kernels are new and will want to run whatever actions you configured on all of them). Signed-off-by: Mihai Rusu <dizzy@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3477 592f7852-d20e-0410-864c-8624ca9c26a4
2009-07-11For files that won't be directly executed, the #! is unneccessary and canmbligh1-1/+0
confuse some tools that attempt to detect what the file is. Best to just not have them there. Signed-off-by: Jesse Keating <jkeating@redhat.com> git-svn-id: svn://test.kernel.org/autotest/trunk@3407 592f7852-d20e-0410-864c-8624ca9c26a4
2009-03-25Updated mirror/mirror script:mbligh1-34/+30
- removed builtin support for kernel patches as this is already handled by kernelextract tool (and it wasn't used by mirror anyways) - removed "queuedir" command line argument and updated it to use server.frontend API for creating the jobs on the server - grouped jobs by control-file/kernel-config tuple to avoid scheduling a job per machine when not necessary Signed-off-by: Mihai Rusu <dizzy@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@2931 592f7852-d20e-0410-864c-8624ca9c26a4
2009-03-19clean up whitespace in mirrormbligh1-19/+12
Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@2909 592f7852-d20e-0410-864c-8624ca9c26a4
2008-07-31Adding a couple of __init__.py files that didn't make it in.showard1-0/+0
git-svn-id: svn://test.kernel.org/autotest/trunk@1933 592f7852-d20e-0410-864c-8624ca9c26a4
2008-07-11provides coverage of the rsync utilities.mbligh2-3/+93
Signed-off-by: Travis Miller <raphtee@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1824 592f7852-d20e-0410-864c-8624ca9c26a4
2008-06-06Convert all python code to use four-space indents instead of eight-space tabs.jadmanski3-265/+265
Signed-off-by: John Admanski <jadmanski@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1658 592f7852-d20e-0410-864c-8624ca9c26a4
2008-05-16Remove all places where we are hardcoding python2.4 on thembligh2-66/+62
scripts Risk: Medium/High Summary: After going trough the autotest tree, removing all places where we have statements like #!/usr/bin/python2.4 As they're unnecessary for those who have python > 2.4. Still, we want to make sure we're running on something > 2.4 for those scripts, so I hooked the scripts to the common lib infrastructure when needed. I tried to be as careful as possible to make sure all scripts worked after my modifications, I could even spot an import problem on server/standalone_profiler.py and fix it. Also, while I was there, I fixed several inconsistencies on the mirror/mirror script. Signed-off-by: Lucas Meneghel Rodrigues <lucasmr@br.ibm.com> git-svn-id: svn://test.kernel.org/autotest/trunk@1527 592f7852-d20e-0410-864c-8624ca9c26a4
2007-09-30prepend kernel name instead of queue name to results dirmbligh1-1/+1
Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@719 592f7852-d20e-0410-864c-8624ca9c26a4
2007-09-26Update mirror/trigger engines.mbligh2-44/+312
From: Ryan Stutsman <stutsman@google.com> Ok, final rework to this mirror script. From here bugs should be easy to fix as patches. This version should address all the concerns you outlined in your prior looks at the mirror script. The major addition of this version is automatic smart kernel config selection for builds. It works like this: you pass a directory to mirror via the -f flag like so: mirror -f /home/testing/config -c /home/testing/client_test_mapping /home/testing/mirror /home/testing/queue Now, when a new kernel release is found it will test that kernel on each machine listed in the client_test_mapping file with each of the control files listed on that line. For example, if client_test_mapping contains: machine1 ~/basic-tests ~/do-something.srv machine2 ~/basic-tests It will queue the AutoTest control file ~/basic-tests on both machines and the ~/do-something.srv Autoserv file just on machine1. The Autoserv file support should make it quite easy to do similar testing with RPMs, for example, though the mirroring urls and regexps would probably have to modified or configurable. In doing this for AutoTest control files it will wrap it with the code necessary to build and boot the kernel version it mirrored. If there is a file in /home/testing/config with the name machine1, for example, it will use that to determine which kernel build config should used. If there where a /home/testing/config/machine1 like this: 2.6.0 /boot/config-2.6.18 2.6.19 /boot/config-2.6.21 Then for any kernel 2.6.0 or later it will build it with /boot/config-2.6.18 (path on the *client* machine). For any kernel 2.6.19 or later it will use /boot/config-2.6.21. If no file exists for that machine it will simply use /boot/config. Also, if the kernel version to be tested it older than any config in the mapping it will automatically use the oldest in the file (2.6.0 == /boot/config-2.6.18) in this case. The /home/testing/mirror arg tells it to mirror the kernel.org releases to that directory and /home/testing/queue contains the queues that correspond to the names given in client_test_mapping. Hopefully this all makes sense. I'll be watching the list to see if there are questions or bugs to work out with the script. I have tested it somewhat and it seems to work as expected. git-svn-id: svn://test.kernel.org/autotest/trunk@692 592f7852-d20e-0410-864c-8624ca9c26a4
2006-11-12Add the -rc patches from testing/ directorymbligh1-0/+1
git-svn-id: svn://test.kernel.org/autotest/trunk@373 592f7852-d20e-0410-864c-8624ca9c26a4
2006-10-13Change explicit Python 2.4 references to generic /usr/bin/pythonmbligh3-3/+3
git-svn-id: svn://test.kernel.org/autotest/trunk@344 592f7852-d20e-0410-864c-8624ca9c26a4
2006-10-08Fix the other os.mkdir to os.makedirs()mbligh1-1/+1
Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@306 592f7852-d20e-0410-864c-8624ca9c26a4
2006-10-07change from os.mkdir to os.makedirs in rsync.pymbligh1-1/+1
Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@303 592f7852-d20e-0410-864c-8624ca9c26a4
2006-10-07Move mirror.py to mirrormbligh1-0/+0
git-svn-id: svn://test.kernel.org/autotest/trunk@302 592f7852-d20e-0410-864c-8624ca9c26a4
2006-10-07Stop mirroring broken-out/ directories, and fix quoting of trigger scan mbligh2-4/+4
file git-svn-id: svn://test.kernel.org/autotest/trunk@301 592f7852-d20e-0410-864c-8624ca9c26a4
2006-10-07Fix double-scan to use simpler grouping instead.mbligh1-25/+17
Signed-off-by: Martin J. Bligh <mbligh@google.com> git-svn-id: svn://test.kernel.org/autotest/trunk@300 592f7852-d20e-0410-864c-8624ca9c26a4
2006-10-07Add mirror functions, intended to mirror and monitor kernel.orgmbligh4-0/+986
git-svn-id: svn://test.kernel.org/autotest/trunk@299 592f7852-d20e-0410-864c-8624ca9c26a4
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-04 13:05:28 +0000