| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Under the django backend, Django 1.3 does not have OperationalError
and ProgrammingError anymore (as those are specific to the MySQL
backend). So in this case, let's mock OperationalError and
ProgrammingError with DatabaseError, which is the base DB error
on Django.
This commit fixes autotest_lib.scheduler.monitor_db_functional_test
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
git-svn-id: svn://test.kernel.org/autotest/trunk@5438 592f7852-d20e-0410-864c-8624ca9c26a4
|
|
object creation.
Signed-off-by: Scott Zawalski <scottz@google.com>
git-svn-id: svn://test.kernel.org/autotest/trunk@4283 592f7852-d20e-0410-864c-8624ca9c26a4
|
|
"django" database_connection backend and having the scheduler use it. This eliminates the duplicate connection that the scheduler was setting up -- now it uses only a single connection (the Django one).
Signed-off-by: Steve Howard <showard@google.com>
git-svn-id: svn://test.kernel.org/autotest/trunk@4000 592f7852-d20e-0410-864c-8624ca9c26a4
|
|
monitor_db.py file holistically, made possible by the fact that monitor_db.py is already isolated from all direct system access through drone_manager (this was a necessary separation for distributed scheduling). by mocking out the entire drone_manager, as well as other major dependencies (email manager, global config), and filling a test database, we can allow the dispatcher to execute normally and allow it to interact with all the other code in monitor_db. at the end, we can check the state of the database and the drone_manager, and (probably most importantly, given the usual failure mode of the scheduler) we can ensure no exceptions get raised from monitor_db.
right now, the test is very minimal. it's able to walk through the process of executing a normal, default job with nothing unusual arising. it checks very little, other than ensuring that the scheduler doesn't die from any exceptions. over time it can be extended to test many more cases and check many more things. this is just a start.
as a side note, i added a "django" backend to database_connection.py which just uses the Django connection wrapper. this is preferable because Django has lots of nice translations already in place. for example, SQLite returns datetime columns as strings, but Django's wrapper automatically translates them into real datetime objects. we could probably just use this in lots of places, such as the frontend and scheduler unit tests. but it was necessary here.
Signed-off-by: Steve Howard <showard@google.com>
git-svn-id: svn://test.kernel.org/autotest/trunk@3801 592f7852-d20e-0410-864c-8624ca9c26a4
|
|
Scheduling a job on an atomic group means that all of the Ready machines
(up to a maximum specified in the atomic group) in a single label associated
with that atomic group will be used to run the job.
The job synch_count becomes a minimum when scheduling on an atomic group.
Both HostQueueEntrys and Labels may have an AtomicGroup associated with
them:
* A HostQueueEntry with an AtomicGroup acts to schedule a job on
all Ready machines of a single Label associated with that AtomicGroup.
* A Label with an AtomicGroup means that any Hosts bearing that Label
may only be scheduled together as a group with other hosts of that Label
to satisify a Job's HostQueueEntry bearing the same AtomicGroup.
Such Hosts will never be scheduled as normal metahosts.
Future patches are coming that will add the ability to schedule jobs using
this feature to the RPC interface, CLI and GUI.
Signed-off-by: Gregory Smith <gps@google.com>
git-svn-id: svn://test.kernel.org/autotest/trunk@2878 592f7852-d20e-0410-864c-8624ca9c26a4
|
|
git-svn-id: svn://test.kernel.org/autotest/trunk@2415 592f7852-d20e-0410-864c-8624ca9c26a4
|
|
* get rid of monitor_db.DatabaseConn, and make monitor_db use the new DatabaseConnection
* modify some queries in monitor_db that weren't SQLite-compatible (SQLite doesn't support TRUE and FALSE literals)
* add frontend/django_test_utils.py, which contains utilities to
* setup a django environment (something manage.py normally does for you)
* replace the configured DB with a SQLite one, either in-memory or on disk
* run syncdb on the test DB
* backup and restore the test DB, handy because then we can syncdb once, save the fresh DB, and quickly restore it between unittests without having to run syncdb again (syncdb is terribly slow for whatever reason)
* modify monitor_db_unittest to use these methods to set up a temporary SQLite DB, run syncdb on it, and test against it
* replace much of the data modification code in monitor_db_unittest with use of the django models. The INSERTs were very problematic with SQLite because syncdb doesn't set database defaults, but using the models solves that (django inserts the defaults itself). using the models is much cleaner anyway as you can see. it was just difficult to do before, but now that we've got the infrastructure to setup the environment anyway, it's easy. this is a good model for how we can make the scheduler use the django models eventually.
* reorder fields of Label model to match actual DB ordering; this is necessary since monitor_db depends on field ordering
* add defaults to some fields in AFE models that should've had them
* make DatabaseConnection.get_test_database support SQLite in files, which gives us persistence that is necessary and handy in the scheduler unittest
* add a fix to _SqliteBackend for pysqlite2 crappiness
The following are extras that weren't strictly necessary to get things working:
* add a debug feature to DatabaseConnection to print all queries
* add an execute_script method to DatabaseConnection (it was duplicated in migrate and monitor_db_unittest)
* rename "arguments" to "parameters" in _GenericBackend.execute, to match the DB-API names
* get rid of some debug code that was left in monitor_db, and one unnecessary statement
Signed-off-by: Steve Howard <showard@google.com>
git-svn-id: svn://test.kernel.org/autotest/trunk@2252 592f7852-d20e-0410-864c-8624ca9c26a4
|
|
be used by migrate, scheduler, parser (eventually), and maybe others. This will consolidate the multiple database wrappers we have throughout the code and allow us to swap in SQLite for MySQL for unit testing purposes.
-add database/ directory for database libraries. migrate.py will move here soon.
-add database_connection.py under server_common, a basic database wrapper supporting both MySQL and SQLite. PostgreSQL should be an easy future addition (any library supporting Python DB-API should be trivial to add). DatabaseConnection also supports graceful handling of dropped connections.
-add unittest for DatabaseConnection
-change migrate.py to use common DatabaseConnection. Scheduler will be changed to use it in a coming CL and in the future hopefully the TKO parser will be able to use it as well.
-change migrate_unittest.py to use SQLite.
Signed-off-by: Steve Howard <showard@google.com>
git-svn-id: svn://test.kernel.org/autotest/trunk@2234 592f7852-d20e-0410-864c-8624ca9c26a4
|
