diff options
author | Wim Taymans <wim.taymans@collabora.co.uk> | 2012-01-14 19:16:01 +0100 |
---|---|---|
committer | Wim Taymans <wim.taymans@collabora.co.uk> | 2012-02-09 11:48:02 +0100 |
commit | 3d4a1a98b556d250a810c9c522333694503ec801 (patch) | |
tree | 0987ffa0d95141ed26e4cafa8b8f8b1905d646c9 | |
parent | 552d726c3a016dfd9eb2d2ad5d3c575434017139 (diff) |
docs: update and improve docs
-rw-r--r-- | docs/design/draft-metadata.txt | 9 | ||||
-rw-r--r-- | docs/design/part-TODO.txt | 4 | ||||
-rw-r--r-- | docs/design/part-bufferpool.txt | 2 | ||||
-rw-r--r-- | docs/design/part-element-transform.txt | 2 | ||||
-rw-r--r-- | docs/design/part-events.txt | 32 | ||||
-rw-r--r-- | docs/design/part-latency.txt | 25 | ||||
-rw-r--r-- | docs/design/part-meta.txt | 4 | ||||
-rw-r--r-- | docs/design/part-negotiation.txt | 120 | ||||
-rw-r--r-- | docs/design/part-probes.txt | 24 | ||||
-rw-r--r-- | docs/design/part-query.txt | 101 | ||||
-rw-r--r-- | docs/design/part-segments.txt | 2 | ||||
-rw-r--r-- | docs/design/part-streams.txt | 2 | ||||
-rw-r--r-- | gst/gstquery.c | 4 |
13 files changed, 223 insertions, 108 deletions
diff --git a/docs/design/draft-metadata.txt b/docs/design/draft-metadata.txt index ffc4df659..aa8407581 100644 --- a/docs/design/draft-metadata.txt +++ b/docs/design/draft-metadata.txt @@ -96,17 +96,16 @@ http://wiki.creativecommons.org/Tracker_CC_Indexing Current Metadata handling ~~~~~~~~~~~~~~~~~~~~~~~~~ -When reading files, demuxers or parsers extract the metadata. It will be sent as -a GST_MESSAGE_TAG to the GstBus and GST_EVENT_TAG to downstream elements. +When reading files, demuxers or parsers extract the metadata. It will be sent +a GST_EVENT_TAG to downstream elements. When a sink element receives a tag +event, it will post a GST_MESSAGE_TAG message on the bus with the contents of +the tag event. Elements receiving GST_EVENT_TAG events can mangle them, mux them into the buffers they send or just pass them through. Usually is muxers that will format the tag data into the form required by the format they mux. Such elements would also implement the GstTagSetter interface to receive tags from the application. - GstMessage(tag) over GstBus to application - ^ - | +----------+ | demux | sink src --> GstEvent(tag) over GstPad to downstream element diff --git a/docs/design/part-TODO.txt b/docs/design/part-TODO.txt index 9cb8f0cff..e37015820 100644 --- a/docs/design/part-TODO.txt +++ b/docs/design/part-TODO.txt @@ -51,7 +51,7 @@ API/ABI and another a push, the push might be busy while the block callback is done. * maybe this name is overloaded. We need to look at some more use cases before trying to fix this. - FIXED in 0.11 with BLOCKING probes. Not everything is implemented yet, though. + FIXED in 0.11 with BLOCKING probes. - rethink the way we do upstream renegotiation. Currently it's done with pad_alloc but this has many issues such as only being able to suggest 1 format @@ -61,7 +61,7 @@ API/ABI This could, for example, easily be triggered with a RENEGOTIATE event. FIXED in 0.11 with RECONFIGURE events. -- Remove the result format value in queries. +- Remove the result format value in queries. FIXED in 0.11 - Try to minimize the amount of acceptcaps calls when pushing buffers around. The element pushing the buffer usually negotiated already and decided on the diff --git a/docs/design/part-bufferpool.txt b/docs/design/part-bufferpool.txt index b4254dc4e..5bd4b8cdb 100644 --- a/docs/design/part-bufferpool.txt +++ b/docs/design/part-bufferpool.txt @@ -147,7 +147,7 @@ Allocating from pool will block when all buffers are outstanding until a buffer is returned to the pool. This behaviour can be changed by specifying the GST_BUFFER_POOL_FLAG_DONTWAIT flag in the parameters. With this flag set, - allocation will return GST_FLOW_UNEXPECTED when the pool is empty. + allocation will return GST_FLOW_EOS when the pool is empty. Renegotiation diff --git a/docs/design/part-element-transform.txt b/docs/design/part-element-transform.txt index e3c11b86f..2afe8c160 100644 --- a/docs/design/part-element-transform.txt +++ b/docs/design/part-element-transform.txt @@ -273,7 +273,7 @@ Allocation ~~~~~~~~~~ After the transform element is configured with caps, a bufferpool needs to be -negotiated to perform the allocation of buffers. We habe 2 cases: +negotiated to perform the allocation of buffers. We have 2 cases: - The element is operating in passthrough we don't need to allocate a buffer in the transform element. diff --git a/docs/design/part-events.txt b/docs/design/part-events.txt index 319369823..10c9f4a21 100644 --- a/docs/design/part-events.txt +++ b/docs/design/part-events.txt @@ -34,27 +34,23 @@ Different types of events exist to implement various functionalities. src pads -------- -A gst_pad_push_event() on a srcpad will first store the event in the sticky -array before sending the event to the peer pad. If there is no peer pad, the -gst_pad_push_event() function returns NOT_LINKED. +A gst_pad_push_event() on a srcpad will first store the sticky event in the +sticky array before sending the event to the peer pad. If there is no peer pad +and the event was not stored in the sticky array, FALSE is returned. -Note that the behaviour is not influenced by a flushing pad. +Flushing pads will refuse the events and will not store the sticky events. -FLUSH_START and FLUSH_STOP events are dropped on blocked pads. sink pads --------- -A gst_pad_send_event() on a sinkpad will check the new event against the -existing event. If they are different, the new event is stored as a pending -event. If the events are the same, nothing changes. +A gst_pad_send_event() on a sinkpad will call the event function on the pad. If +the event function returns success, the sticky event is stored in the sticky +event array and the event is marked for update. -When the pad is flushing, the _send_event() function returns WRONG_STATE -immediately. +When the pad is flushing, the _send_event() function returns FALSE immediately. -The event function is then called for all pending events. If the function -returns success, the pending event is copied to the active events, else the -pending event is removed and the current active event is unchanged. +When the next data item is pushed, the pending events are pushed first. This ensures that the event function is never called for flushing pads and that the sticky array only contains events for which the event function returned @@ -64,9 +60,9 @@ success. pad link -------- -When linking pads, all the sticky events from the srcpad are copied to the -pending array on the sinkpad. The pending events will be sent to the event -function of the sinkpad on the next event or buffer. +When linking pads, the srcpad sticky events are marked for update when they are +different from the sinkpad events. The next buffer push will push the events to +the sinkpad. FLUSH_START/STOP @@ -112,7 +108,7 @@ The EOS event can only be sent on a sinkpad. It is typically emited by the source element when it has finished sending data. This event is mainly sent in the streaming thread but can also be sent from the application thread. -An EOS event sent on a srcpad returns GST_FLOW_UNEXPECTED. +An EOS event sent on a srcpad returns GST_FLOW_EOS. The downstream element should forward the EOS event to its downstream peer elements. This way the event will eventually reach the sinks which should @@ -131,7 +127,7 @@ GStreamer core will take the STREAM_LOCK. Sometimes the EOS event is generated by another element than the source, for example a demuxer element can generate an EOS event before the source element. This is not a problem, the demuxer does not send an EOS event to the upstream -element but returns GST_FLOW_UNEXPECTED, causing the source element to stop +element but returns GST_FLOW_EOS, causing the source element to stop sending data. An element that sends EOS on a pad should stop sending data on that pad. Source diff --git a/docs/design/part-latency.txt b/docs/design/part-latency.txt index 7b58b1012..315c75657 100644 --- a/docs/design/part-latency.txt +++ b/docs/design/part-latency.txt @@ -220,6 +220,31 @@ separate async state change thread (like the one currently used for going from PAUSED->PLAYING in a non-live pipeline). +Query +~~~~~ + +The pipeline latency is queried with the LATENCY query. + + (out) "live", G_TYPE_BOOLEAN (default FALSE) + - if a live element is found upstream + + (out) "min-latency", G_TYPE_UINT64 (default 0) + - the minimum latency in the pipeline + + (out) "max-latency", G_TYPE_UINT64 (default 0) + - the maximum latency in the pipeline + + +Event +~~~~~ + +The latency in the pipeline is configured with the LATENCY event, which contains +the following fields: + + "latency", G_TYPE_UINT64 + - the configured latency in the pipeline + + Latency compensation ~~~~~~~~~~~~~~~~~~~~ diff --git a/docs/design/part-meta.txt b/docs/design/part-meta.txt index 797c3157d..ac09e1ffd 100644 --- a/docs/design/part-meta.txt +++ b/docs/design/part-meta.txt @@ -61,6 +61,7 @@ GstMeta A GstMeta is a structure as follows: struct _GstMeta { + GstMetaFlags flags; const GstMetaInfo *info; /* tag and info for the meta item */ }; @@ -166,7 +167,8 @@ The complete buffer with metadata could, for example, look as follows: GstMiniObject | GType (GstBuffer) | | refcount, flags, copy/disp/free | +-------------------------------------+ -GstBuffer | caps, parent, pool | +GstBuffer | pool,pts,dts,duration,offsets | + | <private data> | +.....................................+ | next ---+ +- | info ------> GstMetaInfo diff --git a/docs/design/part-negotiation.txt b/docs/design/part-negotiation.txt index 2a5f4bee1..89ae38bad 100644 --- a/docs/design/part-negotiation.txt +++ b/docs/design/part-negotiation.txt @@ -8,10 +8,65 @@ pipeline that have information to those parts of the pipeline that are flexible, constrained by those parts of the pipeline that are not flexible. + +Basic rules +~~~~~~~~~~~ + +The simple rules must be followed: + + 1) downstream suggests formats + 2) upstream decides on format + +There are 4 queries/events used in caps negotiation: + + 1) GST_QUERY_CAPS : get possible formats + 2) GST_QUERY_ACCEPT_CAPS : check if format is possible + 3) GST_EVENT_CAPS : configure format (downstream) + 4) GST_EVENT_RECONFIGURE : inform upstream of possibly new caps + + +Queries +------- + +A pad can ask the peer pad for its supported GstCaps. It does this with +the CAPS query. The list of supported caps can be used to choose an +appropriate GstCaps for the data transfer. + + (in) "filter", GST_TYPE_CAPS (default NULL) + - a GstCaps to filter the results against + + (out) "caps", GST_TYPE_CAPS (default NULL) + - the result caps + + +A pad can ask the peer pad if it supports a given caps. It does this with +the ACCEPT_CAPS query. + + (in) "caps", GST_TYPE_CAPS + - a GstCaps to check + + (out) "result", G_TYPE_BOOLEAN (default FALSE) + - TRUE if the caps are accepted + + +Events +~~~~~~ + +When a media format is negotiated, peer elements are notified of the GstCaps +with the CAPS event. The caps must be fixed. + + "caps", GST_TYPE_CAPS + - the negotiated GstCaps + + +Operation +~~~~~~~~~ + GStreamer's two scheduling modes, push mode and pull mode, lend themselves to different mechanisms to achieve this goal. As it is more common we describe push mode negotiation first. + Push-mode negotiation ~~~~~~~~~~~~~~~~~~~~~ @@ -31,9 +86,9 @@ The basics of negotiation are as follows: - An element should reconfigure itself to the new format received as a CAPS event before processing the following buffers. If the data type in the - caps event is not acceptable, the element should refuse the buffer by - returning an appropriate GST_FLOW_NOT_NEGOTIATED return value from the - chain function. + caps event is not acceptable, the element should refuse the event. The + element should also refuse the next buffers by returning an appropriate + GST_FLOW_NOT_NEGOTIATED return value from the chain function. - Downstream elements can request a format change of the stream by sending a RECONFIGURE event upstream. Upstream elements will renegotiate a new format @@ -43,11 +98,18 @@ The general flow for a source pad starting the negotiation. src sink | | - | accepts? | - type A |---------------->| - | yes | - |< - - - - - - - -| - | | + | querycaps? | + |---------------->| + | caps | + select caps |< - - - - - - - -| + from the | | + candidates | | + | |-. + | accepts? | | + type A |---------------->| | optional + | yes | | + |< - - - - - - - -| | + | |-' | send_event() | send CAPS |---------------->| Receive type A, reconfigure to event A | | process type A. @@ -61,9 +123,9 @@ The general flow for a source pad starting the negotiation. [element wants to create a buffer] if not format # see what we can do - ourcaps = gst_pad_get_caps (srcpad) + ourcaps = gst_pad_query_caps (srcpad) # see what the peer can do filtered against our caps - candidates = gst_pad_peer_get_caps (srcpad, ourcaps) + candidates = gst_pad_peer_query_caps (srcpad, ourcaps) foreach candidate in candidates # make sure the caps is fixed @@ -100,7 +162,7 @@ The general flow for a sink pad starting a renegotiation. mark .-|<----------------| send RECONFIGURE event renegotiate| | | '>| | - | get_caps() | + | querycaps() | renegotiate |---------------->| | suggest B | |< - - - - - - - -| @@ -121,7 +183,7 @@ videotestsrc ! xvimagesink 1) Who decides what format to use? - src pad always decides, by convention. sinkpad can suggest a format - by putting it high in the getcaps function GstCaps. + by putting it high in the caps query result GstCaps. - since the src decides, it can always choose something that it can do, so this step can only fail if the sinkpad stated it could accept something while later on it couldn't. @@ -143,11 +205,11 @@ videotestsrc ! xvimagesink - src receives the RECONFIGURE event and marks renegotiation - On the next buffer push, the source renegotiates the caps and the bufferpool. The sink will put the new new prefered format high in the list - of caps it returns from its getcaps function. + of caps it returns from its caps query. videotestsrc ! queue ! xvimagesink - - queue proxies all accept and getcaps to the other peer pad. + - queue proxies all accept and caps queries to the other peer pad. - queue proxies the bufferpool - queue proxies the RECONFIGURE event - queue stores CAPS event in the queue. This means that the queue can contain @@ -230,15 +292,15 @@ The sink determines that the upstream elements support pull based scheduling by doing a SCHEDULING query. The sink initiates the negotiation process by intersecting the results -of gst_pad_get_caps() on its sink pad and its peer src pad. This is the +of gst_pad_query_caps() on its sink pad and its peer src pad. This is the operation performed by gst_pad_get_allowed_caps(). In the simple -passthrough case, the peer pad's getcaps() function should return the +passthrough case, the peer pad's caps query should return the intersection of calling get_allowed_caps() on all of its sink pads. In this way the sink element knows the capabilities of the entire pipeline. The sink element then fixates the resulting caps, if necessary, -resulting in the flow caps. From now on, the getcaps function -of the sinkpad will only return these fixed caps meaning that upstream elements +resulting in the flow caps. From now on, the caps query of the sinkpad +will only return these fixed caps meaning that upstream elements will only be able to produce this format. If the sink element could not set caps on its sink pad, it should post @@ -255,3 +317,25 @@ function. The state will commit to PAUSED when the first buffer is received in the sink. This is needed to provide a consistent API to the applications that expect ASYNC return values from sinks but it also allows us to perform the remainder of the negotiation outside of the context of the pulling thread. + + +Patterns +~~~~~~~~ + +We can identify 3 patterns in negotiation: + + 1) Fixed : Can't choose the output format + - Caps encoded in the stream + - A video/audio decoder + - usually uses gst_pad_use_fixed_caps() + + 2) Passthrough + - Caps not modified + - can do caps transform based on element property + - videobox + + 3) Dynamic : can choose output format + - A converter element + - depends on downstream caps, needs to do a CAPS query to find + transform. + - usually prefers to use the identity transform diff --git a/docs/design/part-probes.txt b/docs/design/part-probes.txt index 185770906..f8a4a1efb 100644 --- a/docs/design/part-probes.txt +++ b/docs/design/part-probes.txt @@ -15,8 +15,8 @@ identify the following types: This is essential to be able to implement dynamic relinking of elements without breaking the dataflow. - - be notified when data or events are pushed or sent on a pad. It should also - be possible to inspect and modify the data. + - be notified when data, events or queries are pushed or sent on a pad. It + should also be possible to inspect and modify the data. - be able to drop, pass and block on data based on the result of the callback. @@ -52,14 +52,17 @@ Overview GST_PAD_PROBE_TYPE_BLOCK = (1 << 1), /* flags to select datatypes */ - GST_PAD_PROBE_TYPE_BUFFER = (1 << 2), - GST_PAD_PROBE_TYPE_BUFFER_LIST = (1 << 3), - GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM = (1 << 4), - GST_PAD_PROBE_TYPE_EVENT_UPSTREAM = (1 << 5), + GST_PAD_PROBE_TYPE_BUFFER = (1 << 4), + GST_PAD_PROBE_TYPE_BUFFER_LIST = (1 << 5), + GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM = (1 << 6), + GST_PAD_PROBE_TYPE_EVENT_UPSTREAM = (1 << 7), + GST_PAD_PROBE_TYPE_EVENT_FLUSH = (1 << 8), + GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM = (1 << 9), + GST_PAD_PROBE_TYPE_QUERY_UPSTREAM = (1 << 10), /* flags to select scheduling mode */ - GST_PAD_PROBE_TYPE_PUSH = (1 << 6), - GST_PAD_PROBE_TYPE_PULL = (1 << 7), + GST_PAD_PROBE_TYPE_PUSH = (1 << 12), + GST_PAD_PROBE_TYPE_PULL = (1 << 13), } GstPadProbeType; @@ -111,7 +114,7 @@ Blocking probes the callback will be called as soon as the pad becomes idle in the streaming thread. - The IDLE probe in useful to perform dynamic linking, it allows to wait for for + The IDLE probe is useful to perform dynamic linking, it allows to wait for for a safe moment when an unlink/link operation can be done. Since the probe is a blocking probe, it will also make sure that the pad stays idle until the probe is removed. @@ -279,3 +282,6 @@ Query probes have the GST_PAD_PROBE_TYPE_QUERY_* flag set in the callbacks. < - - - - - - O | | | +For queries, the PUSH ProbeType is set when the query is traveling to the object +that will answer the query and the PULL type is set when the query contains the +answer. diff --git a/docs/design/part-query.txt b/docs/design/part-query.txt index 4973c21cc..b81773ec8 100644 --- a/docs/design/part-query.txt +++ b/docs/design/part-query.txt @@ -1,44 +1,12 @@ Query ----- -NOTE: this is implemented as proposed - Purpose ~~~~~~~ Queries are used to get information about the stream. A query is started on a specific pad and travels up or downstream. -Types of queries -~~~~~~~~~~~~~~~~ - - - get length of stream - - get position in stream - - get seeking capabilities - - get latency - - convert one value to another - - query supported formats - - query internal links. - -Current implementation -~~~~~~~~~~~~~~~~~~~~~~ - - The current implementation of query requires pads to implement the - following functions: - - gboolean (*GstPadConvertFunction) (GstPad *pad, - GstFormat src_format, gint64 src_value, - GstFormat *dest_format, gint64 *dest_value); - gboolean (*GstPadQueryFunction) (GstPad *pad, GstQueryType type, - GstFormat *format, gint64 *value); - GList* (*GstPadIntLinkFunction) (GstPad *pad); - const GstFormat* (*GstPadFormatsFunction) (GstPad *pad); - const GstEventMask* (*GstPadEventMaskFunction) (GstPad *pad); - const GstQueryType* (*GstPadQueryTypeFunction) (GstPad *pad); - - Most of these functions are not very extensible in particular, - the queryfunction can only return one value. - Requirements ~~~~~~~~~~~~ @@ -47,29 +15,23 @@ Requirements - one pad function to perform the query - extensible queries. -Proposition -~~~~~~~~~~~ +Implementation +~~~~~~~~~~~~~~ - - define GstQuery extending GstMiniObject and containing a GstStructure (see GstMessage) - - define standard query types (see proposed types) - - define methods to create a parse the results in the GstQuery. + - GstQuery extends GstMiniObject and contains a GstStructure (see GstMessage) + - some standard query types are defined below + - methods to create and parse the results in the GstQuery. - define pad method: - gboolean (*GstPadQueryFunction) (GstPad *pad, GstQuery *query); + gboolean (*GstPadQueryFunction) (GstPad *pad, + GstObject *parent, + GstQuery *query); pad returns result in query structure and TRUE as result or FALSE when query is not supported. -Proposed types -~~~~~~~~~~~~~~ - - - GST_QUERY_SEEKING: - - get info on how seeking can be done - - getrange, with/without offset/size - - ranges where seeking is efficient (for caching network sources) - - flags describing seeking behaviour (forward, backward, segments, - play backwards, ...) +Query types +~~~~~~~~~~~ - GST_QUERY_POSITION: @@ -81,7 +43,19 @@ Proposed types - GST_QUERY_LATENCY: - get amount of latency introduced in the pipeline. + get amount of latency introduced in the pipeline. (See part-latency.txt) + + - GST_QUERY_RATE: + + get the current playback rate of the pipeline + + - GST_QUERY_SEEKING: + + get info on how seeking can be done + - getrange, with/without offset/size + - ranges where seeking is efficient (for caching network sources) + - flags describing seeking behaviour (forward, backward, segments, + play backwards, ...) - GST_QUERY_SEGMENT: @@ -95,3 +69,32 @@ Proposed types return list of supported formats that can be used for GST_QUERY_CONVERT. + - GST_QUERY_BUFFERING: + + query available media for efficient seeking (See part-buffering.txt) + + - GST_QUERY_CUSTOM: + + a custom query, the name of the query defines the properties of the query. + + - GST_QUERY_URI: + + query the uri of the source or sink element + + - GST_QUERY_ALLOCATION: + + the buffer allocation properties (See part-bufferpool.txt) + + - GST_QUERY_SCHEDULING: + + the scheduling properties (See part-scheduling.txt) + + - GST_QUERY_ACCEPT_CAPS: + + check if caps are supported (See part-negotiation.txt) + + - GST_QUERY_CAPS: + + get the possible caps (See part-negotiation.txt) + + diff --git a/docs/design/part-segments.txt b/docs/design/part-segments.txt index e05ec5b9e..21926aed8 100644 --- a/docs/design/part-segments.txt +++ b/docs/design/part-segments.txt @@ -90,7 +90,7 @@ Use case: FLUSHING seek segment, the decoder knows it should not send the P frame. Avidemux stops sending data after pushing a frame with timestamp 5 and - returns GST_FLOW_UNEXPECTED from the chain function to make the upstream + returns GST_FLOW_EOS from the chain function to make the upstream elements perform the EOS logic. diff --git a/docs/design/part-streams.txt b/docs/design/part-streams.txt index 119921d3c..ebc33d255 100644 --- a/docs/design/part-streams.txt +++ b/docs/design/part-streams.txt @@ -64,7 +64,7 @@ Typical stream 3) EOS - marks the end of data, nothing is to be expected after EOS, elements - should refuse more data and return GST_FLOW_UNEXPECTED. A FLUSH_STOP + should refuse more data and return GST_FLOW_EOS. A FLUSH_STOP event clears the EOS state of an element. diff --git a/gst/gstquery.c b/gst/gstquery.c index d3659bf40..b617b9e27 100644 --- a/gst/gstquery.c +++ b/gst/gstquery.c @@ -566,8 +566,8 @@ gst_query_new_latency (void) * gst_query_set_latency: * @query: a #GstQuery * @live: if there is a live element upstream - * @min_latency: the minimal latency of the live element - * @max_latency: the maximal latency of the live element + * @min_latency: the minimal latency of the upstream elements + * @max_latency: the maximal latency of the upstream elements * * Answer a latency query by setting the requested values in the given format. * |