= TODO = == logging == - cache more stuff (ghostpad resolving, right now it differs a bit for transmission and element stats) == wrap more methods == - gst_pad_query (we have gst_element_query) == more statistics == - gst_pad_{set,get}_caps? or just check caps when buffers are processed - detailed checks - track caps changes per elements - check if they are real changes (new pointer and different content) - gst_element_post_message - we could handle state-changes specifically - have time on the x axis - have states on the y axis - one plot per element - count total unique buffers, events, messages and queries - right now we counts buffer and event transfers (per pad/element) - just using a hashmap is not enough, as the memory can be reused - events now have a seqence number - also have states per event/query-type - just use a Hashmap for event/query type - glibs g_slice in debug builds has g_slice_debug_tree_statistics == graphs == - condense the multiplot - we should plot all pad-graphs of one element in one plot - pad-logs are named pad__.log - we could ev. group by element-name - right now its ambigous is there are '_' in pad or element names - also helps overlaying events and messages - plot stream-time (we could plot buffer timestamps on pads) - normally just a rising line - should show seeking - use java sequence tool to draw message, query, event graph? == log pre processing == - splitlog would ideally add a order number to the log so that we have a sorted multiplot, like instead of pad_mydecoder_src.log and pad_mydecoder_sink.log we would have pad_05_mydecoder_src.log and pad_05_mydecoder_sink.log - then we can plot all pad_XX_*.log in one graph and also plot then in right order (sources at the top) - if we can log the preffered order at gst_preload_done, then we can maybe also reorder in gsttl_plot.sh - convert timestamps to a double (e.g. sec:ms) == sanity checks == - enable/disable per env-var - caps changes (see above "wrap more methods") GSTTL_CHK_CAPS 1) keep last caps and compare with caps on next buffer 2) gst_caps_is_equal(GST_PAD_CAPS(pad),GST_BUFFER_CAPS(buffer)) - continous timestamps (except DISCONT) - like identity checks GSTTL_CHK_TIMESTAMPS - track buffers - can we identify elements that don't use pad-alloc but should? - can we identify elements that could use sub-buffers? == bandwidth meassurements == - for raw data we know the datarate and thus we can show that as a line in the graph, this can use same scale as bytes-recv/sent, so its easy to see how much the link is saturated. - for other elements we'd like to know what kind of performance is expected on the link too, e.g. bitrate on demux ! queue ! decoder and encoder ! queue ! muxer - can we listen to GST_TAG_BITRATE on those elements? - some codecs put a bitrate into caps (fluwmadec) == plugin api == - we'd like to have external plugins that can add statistics - we collect statistics in 4 scopes: - global, thread, element, pad - plugins would need to - register hooks: - global (called from _log_common) - one for each overridden function - register statistic buffers - allocate sub-struct in global/thread/element/pad stats - at run time we tracelib just calls the hooks - open items - how can a plugin call _log_entry? - we could pass a struct with function pointers to plugin_init - do we need a hook for the global stats that are print on exit? == random ideas == - queue fill levels - latencies - more rusage fields - check clock_gettime with CLOCK_PROCESS_CPUTIME_ID and CLOCK_THREAD_CPUTIME_ID (experimental code in git now) == dynamic graphing == gsttlui is work-in-progress of a UI where we can inspect a running pipeline: - parse topology log events in gsttlui - generate the graph structure using graphviz libraries - plot the layout using goocanvas - ideas: - show threads: - we need to calculate the hull around all pads involved in a thread - for that we define boxes around pads at e.g. 110% size, links would be trapezoids (also need interla links in elements) - now we need a union of all those boxes (intersections on the links) http://www.geometrictools.com/SampleMathematics/Boolean2D/Boolean2D.html http://paulbourke.net/geometry/insidepoly/ == future ideas == - check if we can update properties and redraw easily - state changes on elements and pads - think how we can show buffers, events, queries and messages - it would be neat to show threads by underlaying all elements+pads in each thread with a cloud - union of x% bigger rectangles around the pads + rectangles around the links - the main process and each thread cloud could have some statistic - cpu usage - data transfer == design == +------+ +------+ | elem | | elem | | src--sink | +------+ +------+ +------------------+ | bin | | +------+ | +------+ | | elem | | | elem | | | src--prx--gho--sink | | +------+ | +------+ +------------------+ +------------------+ | bin | +------+ | +------+ | | elem | | | elem | | | src--gho--prx--sink | | +------+ | +------+ | +------------------+ +------------------+ +------------------+ | bin | | bin | | +------+ | | +------+ | | | elem | | | | elem | | | | src--prx--gho--gho--prx--sink | | | +------+ | | +------+ | +------------------+ +------------------+ +------------------------------+ | bin | | +------------------+ | | | bin | | | | +------+ | | +------+ | | | elem | | | | elem | | | | src--prx--gho--prx--gho--sink | | | +------+ | | +------+ | +------------------+ | +------------------------------+ e->e b->e e->b b->b gst_pad_push src,sink src,gho,sink src,gho,sink src,gho,gho,sink gst_pad_pull_range src,sink sink,gho,src sink,gho,src sink,gho,gho,sink gst_pad_send_event gst_pad_get_caps gst_pad_set_caps - proxy-pads of ghostpads are not properly handled - could/should we always account all data activity on proxy-pads to its parent ghostpad? == bugs == - we seem to not fully catch the removal of proxy-pads - the playbin2 example has pad_link_full/pad_unlink pairs with a proxypad, but the proxypad is never appearing for anything else (e.g. decodepad3_proxypad4) - we don't need to catch request_pads separately