1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
|
Insanity, a GStreamer QA System
-------------------------------
The source code and documentation contained within is LGPL.
This work was sponsored by Nokia.
Philosophy
----------
Insanity is a collection of several parts:
* the insanity python module (insanity/) containing:
* base classes for tests and scenarios
* logic for running the tests
* database support for storing results
* base classes for applications
* A series of tests and scenarios (tests/)
* Some command-line tools (bin/)
* A django web interface (web/)
Provided tools
--------------
To run all these tools, you need to have the location of the
insanity module in your python path.
Ex : PYTHONPATH=/full/path/to/top/directory:$PYTHONPATH
* dumpresults.py : provides visualisation of the database contents
* compare.py : Compares two test runs
* grouper.py : Groups the results of a testrun for easier reporting.
Inline documentation
--------------------
The source code is commented with python-style documentation.
You can get an html version by using epydoc and the following
command:
epydoc -o epydoc --docformat plaintext -v insanity
Requirements
------------
Python >= 2.5 *OR* Python >= 2.4 with the sqlite python module
DBus and python bindings
Django 1.3 (for the web frontend only)
Database support
----------------
Currently the data is stored in a sqlite file.
The default filename is 'testrun.db' in the current directory.
Django web frontend
-------------------
Django web frontend is integrated with the GTK+/DBus parts so it
can configure and run tests as well as view reports.
The frontend is provided in the web/ directory of the source tree,
and is installed to /usr/share/insanity/web/ by default.
To configure the Django frontend:
* modify the database settings in settings.py according to your
local settings;
* configure the test folders in settings.py (see below for syntax)
The hybrid django+gtk server needs to be run using the 'daemon'
management command:
> python /usr/share/insanity/web/manage.py daemon
The daemon command behaves like 'runserver --noreload', and can take
any additional arguments that runserver can. So, for example, IP and
port to bind on can be specified:
> python manage.py daemon 0.0.0.0:8000
To stop the app gracefully, press Ctrl-C or send a SIGINT signal to the
process.
Configuring test folders for web frontend
-----------------------------------------
The settings.py file defines a Python dict specifying folders with test
media and their options.
For each item in the dictionary, the key is a full path to the folder
containing the test media. The value is a dictionary containing string
keys and option-dependant values:
* name -> human-readable folder name representation (string)
* extra-arguments -> a dictionary of extra arguments ot pass to the tests
running in this folder
If it is expected that some test fail for some test file in the folder, the
'expected_failures' extra argument can be passed in. The value of the argument
is a list of patterns.
Each pattern is a Python dict with at least 'checkitem' key, whose value is
a check item name that's expected to fail. Other items represent arguments
that were passed to this test (ie. each item needs to match the test argument)
in order for the pattern to match the test.
For example, 'uri' can be passed with the name of the media file that's
known to cause the specific expected failure.
Example configuration:
INSANITY_TEST_FOLDERS = {
'/path/to/test-media/': {
'name': 'Test Media Folder',
'extra-arguments': {
'expected_failures': [ # patterns of checkitem/arguments to match
{
'checkitem': 'is-media-type', # required
'uri': 'file:///path/to/test/media/not-a-media-file.zip'
}
]
}
}
}
Web frontend through Apache
---------------------------
To run insanity through Apache, you'll need to install mod_wsgi and add the
following to your apache configuration.
WSGIDaemonProcess insanity user=buildbot group=buildbot threads=25
WSGIProcessGroup insanity
Alias /media/ /usr/local/share/insanity/web/site_media
<Directory /usr/local/share/insanity/web/site_media>
Order deny,allow
Allow from all
</Directory>
WSGIScriptAlias / /usr/local/share/insanity/web/wsgi.py
<Directory /usr/local/share/insanity/web/>
<Files wsgi.py>
Order deny,allow
Allow from all
</Files>
</Directory>
Change 'buildbot' to any valid username and group you want to use. Change
'/usr/local/' to match your installation prefix. You may need to change
DATA_PATH in @prefix@/share/insanity/web/settings.py to a location where
the selected user can write. Use 'manage.py syncdb' to initialized the
the database, as one would normally with other Django webapp.
|
