1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
|
= TODO =
== logging ==
- cache more stuff (ghostpad resolving, right now it differs a bit for
transmission and element stats)
== wrap more methods ==
- gst_pad_query (we have gst_element_query)
== more statistics ==
- gst_pad_{set,get}_caps? or just check caps when buffers are processed
- detailed checks
- track caps changes per elements
- check if they are real changes (new pointer and different content)
- gst_element_post_message
- we could handle state-changes specifically
- have time on the x axis
- have states on the y axis
- one plot per element
- count total unique buffers, events, messages and queries
- right now we counts buffer and event transfers (per pad/element)
- just using a hashmap is not enough, as the memory can be reused
- events now have a seqence number
- also have states per event/query-type
- just use a Hashmap for event/query type
- glibs g_slice in debug builds has g_slice_debug_tree_statistics
== graphs ==
- condense the multiplot
- we should plot all pad-graphs of one element in one plot
- pad-logs are named pad_<element-name>_<pad_name>.log
- we could ev. group by element-name
- right now its ambigous is there are '_' in pad or element names
- also helps overlaying events and messages
- plot stream-time (we could plot buffer timestamps on pads)
- normally just a rising line
- should show seeking
- use java sequence tool to draw message, query, event graph?
== log pre processing ==
- splitlog would ideally add a order number to the log so that we have a sorted
multiplot, like instead of pad_mydecoder_src.log and pad_mydecoder_sink.log
we would have pad_05_mydecoder_src.log and pad_05_mydecoder_sink.log
- then we can plot all pad_XX_*.log in one graph and also plot then in right
order (sources at the top)
- if we can log the preffered order at gst_preload_done, then we can maybe also
reorder in gsttl_plot.sh
- convert timestamps to a double (e.g. sec:ms)
== sanity checks ==
- enable/disable per env-var
- caps changes (see above "wrap more methods")
GSTTL_CHK_CAPS
1) keep last caps and compare with caps on next buffer
2) gst_caps_is_equal(GST_PAD_CAPS(pad),GST_BUFFER_CAPS(buffer))
- continous timestamps (except DISCONT) - like identity checks
GSTTL_CHK_TIMESTAMPS
- track buffers
- can we identify elements that don't use pad-alloc but should?
- can we identify elements that could use sub-buffers?
== bandwidth meassurements ==
- for raw data we know the datarate and thus we can show that as a line in the
graph, this can use same scale as bytes-recv/sent, so its easy to see how much
the link is saturated.
- for other elements we'd like to know what kind of performance is expected on
the link too, e.g. bitrate on demux ! queue ! decoder and encoder ! queue ! muxer
- can we listen to GST_TAG_BITRATE on those elements?
- some codecs put a bitrate into caps (fluwmadec)
== plugin api ==
- we'd like to have external plugins that can add statistics
- we collect statistics in 4 scopes:
- global, thread, element, pad
- plugins would need to
- register hooks:
- global (called from _log_common)
- one for each overridden function
- register statistic buffers
- allocate sub-struct in global/thread/element/pad stats
- at run time we tracelib just calls the hooks
- open items
- how can a plugin call _log_entry?
- we could pass a struct with function pointers to plugin_init
- do we need a hook for the global stats that are print on exit?
== random ideas ==
- queue fill levels
- latencies
- more rusage fields
- check clock_gettime with CLOCK_PROCESS_CPUTIME_ID and CLOCK_THREAD_CPUTIME_ID
(experimental code in git now)
== dynamic graphing ==
gsttlui is work-in-progress of a UI where we can inspect a running pipeline:
- parse topology log events in gsttlui
- generate the graph structure using graphviz libraries
- plot the layout using goocanvas
- ideas:
- show threads:
- we need to calculate the hull around all pads involved in a thread
- for that we define boxes around pads at e.g. 110% size, links would be
trapezoids (also need interla links in elements)
- now we need a union of all those boxes (intersections on the links)
http://www.geometrictools.com/SampleMathematics/Boolean2D/Boolean2D.html
http://paulbourke.net/geometry/insidepoly/
== future ideas ==
- check if we can update properties and redraw easily
- state changes on elements and pads
- think how we can show buffers, events, queries and messages
- it would be neat to show threads by underlaying all elements+pads in each
thread with a cloud
- union of x% bigger rectangles around the pads + rectangles around the links
- the main process and each thread cloud could have some statistic
- cpu usage
- data transfer
== design ==
+------+ +------+
| elem | | elem |
| src--sink |
+------+ +------+
+------------------+
| bin |
| +------+ | +------+
| | elem | | | elem |
| | src--prx--gho--sink |
| +------+ | +------+
+------------------+
+------------------+
| bin |
+------+ | +------+ |
| elem | | | elem | |
| src--gho--prx--sink | |
+------+ | +------+ |
+------------------+
+------------------+ +------------------+
| bin | | bin |
| +------+ | | +------+ |
| | elem | | | | elem | |
| | src--prx--gho--gho--prx--sink | |
| +------+ | | +------+ |
+------------------+ +------------------+
+------------------------------+
| bin |
| +------------------+ |
| | bin | |
| | +------+ | | +------+
| | | elem | | | | elem |
| | | src--prx--gho--prx--gho--sink |
| | +------+ | | +------+
| +------------------+ |
+------------------------------+
e->e b->e e->b b->b
gst_pad_push src,sink src,gho,sink src,gho,sink src,gho,gho,sink
gst_pad_pull_range src,sink sink,gho,src sink,gho,src sink,gho,gho,sink
gst_pad_send_event
gst_pad_get_caps
gst_pad_set_caps
- proxy-pads of ghostpads are not properly handled
- could/should we always account all data activity on proxy-pads to its parent
ghostpad?
== bugs ==
- we seem to not fully catch the removal of proxy-pads
- the playbin2 example has pad_link_full/pad_unlink pairs with a proxypad,
but the proxypad is never appearing for anything else
(e.g. decodepad3_proxypad4)
- we don't need to catch request_pads separately
|
