Insanity, a GStreamer QA System
-------------------------------

The source code and documentation contained within is LGPL.

This work was sponsored by Nokia.

Philosophy
----------

   Insanity is a collection of several parts:

   * the insanity python module (insanity/) containing:
     * base classes for tests and scenarios
     * logic for running the tests
     * database support for storing results
     * base classes for applications
   * A series of tests and scenarios (tests/)
   * Some command-line tools (bin/)
   * A django web interface (web/)

Provided tools
--------------
  To run all these tools, you need to have the location of the
  insanity module in your python path.
    Ex : PYTHONPATH=/full/path/to/top/directory:$PYTHONPATH

 * dumpresults.py : provides visualisation of the database contents
 * compare.py : Compares two test runs
 * grouper.py : Groups the results of a testrun for easier reporting.


Inline documentation
--------------------

  The source code is commented with python-style documentation.

  You can get an html version by using epydoc and the following
command:

  epydoc -o epydoc --docformat plaintext -v insanity

Requirements
------------
  Python >= 2.5 *OR* Python >= 2.4 with the sqlite python module
  DBus and python bindings
  Django 1.3 (for the web frontend only)

Database support
----------------

  Currently the data is stored in a sqlite file.
  The default filename is 'testrun.db' in the current directory.

Django web frontend
-------------------

  Django web frontend is integrated with the GTK+/DBus parts so it
can configure and run tests as well as view reports.

  The frontend is provided in the web/ directory of the source tree,
and is installed to /usr/share/insanity/web/ by default.

To configure the Django frontend:

  * modify the database settings in settings.py according to your
    local settings;

  * configure the test folders in settings.py (see below for syntax)

  The hybrid django+gtk server needs to be run using the 'daemon'
management command:

  > python /usr/share/insanity/web/manage.py daemon

  The daemon command behaves like 'runserver --noreload', and can take
any additional arguments that runserver can. So, for example, IP and
port to bind on can be specified:

  > python manage.py daemon 0.0.0.0:8000

  To stop the app gracefully, press Ctrl-C or send a SIGINT signal to the
process.


Configuring test folders for web frontend
-----------------------------------------

  The settings.py file defines a Python dict specifying folders with test
media and their options.

  For each item in the dictionary, the key is a full path to the folder
containing the test media. The value is a dictionary containing string
keys and option-dependant values:

 * name -> human-readable folder name representation (string)

 * extra-arguments -> a dictionary of extra arguments ot pass to the tests
   running in this folder

  If it is expected that some test fail for some test file in the folder, the
'expected_failures' extra argument can be passed in. The value of the argument
is a list of patterns.

  Each pattern is a Python dict with at least 'checkitem' key, whose value is
a check item name that's expected to fail. Other items represent arguments
that were passed to this test (ie. each item needs to match the test argument)
in order for the pattern to match the test.

  For example, 'uri' can be passed with the name of the media file that's
known to cause the specific expected failure.

Example configuration:

  INSANITY_TEST_FOLDERS = {
      '/path/to/test-media/': {
          'name': 'Test Media Folder',
          'extra-arguments': {
              'expected_failures': [ # patterns of checkitem/arguments to match
                  {
                      'checkitem': 'is-media-type', # required
                      'uri': 'file:///path/to/test/media/not-a-media-file.zip'
                  }
              ]
          }
      }
  }

Web frontend through Apache
---------------------------

To run insanity through Apache, you'll need to install mod_wsgi and add the
following to your apache configuration.

  WSGIDaemonProcess insanity user=buildbot group=buildbot threads=25
  WSGIProcessGroup insanity

  Alias /media/ /usr/local/share/insanity/web/site_media

  <Directory /usr/local/share/insanity/web/site_media>
      Order deny,allow
      Allow from all
  </Directory>

  WSGIScriptAlias / /usr/local/share/insanity/web/wsgi.py

  <Directory /usr/local/share/insanity/web/>
    <Files wsgi.py>
      Order deny,allow
      Allow from all
    </Files>
  </Directory>

Change 'buildbot' to any valid username and group you want to use. Change
'/usr/local/' to match your installation prefix. You may need to change
DATA_PATH in @prefix@/share/insanity/web/settings.py to a location where
the selected user can write. Use 'manage.py syncdb' to initialized the
the database, as one would normally with other Django webapp.