diff options
| author | Wim Taymans <wim.taymans@collabora.co.uk> | 2011-02-22 14:11:59 +0100 |
|---|---|---|
| committer | Wim Taymans <wim.taymans@collabora.co.uk> | 2011-02-22 14:11:59 +0100 |
| commit | 238b9a57cc022aec7ebd52d908ac5e24d76c3183 (patch) | |
| tree | 9563e2353a2c427a40fae0e7de0ed73c2348c402 /docs | |
| parent | 90fff577f0661c1ec7a0f2c324d5b69d1d1d040e (diff) | |
| parent | cc5edeefcd7772ac645fa0b8b91cfe7be201d78a (diff) | |
Merge branch 'master' into 0.11
Conflicts:
configure.ac
gst/gstelement.c
gst/gstelement.h
gst/gstpad.c
gst/gstutils.c
libs/gst/base/Makefile.am
libs/gst/check/Makefile.am
libs/gst/controller/Makefile.am
libs/gst/dataprotocol/Makefile.am
libs/gst/net/Makefile.am
win32/common/libgstreamer.def
Diffstat (limited to 'docs')
45 files changed, 1096 insertions, 55 deletions
diff --git a/docs/design/draft-bufferpool.txt b/docs/design/draft-bufferpool.txt new file mode 100644 index 000000000..72d3a617a --- /dev/null +++ b/docs/design/draft-bufferpool.txt @@ -0,0 +1,415 @@ +Bufferpool +---------- + +This document details a possible design for how buffers can be allocated +and managed in pools. + +Bufferpools should increase performance by reducing allocation overhead and +improving possibilities to implement zero-copy memory transfer. + + +Current Situation +----------------- + + - elements can choose to implement a pool of buffers. These pools + can contain buffers for both source and sink pad buffers. + + - elements can provide buffers to upstream elements when the upstream element + requests a buffer with gst_pad_alloc_buffer(). + + - The buffer pool can preallocate a certain amount of buffers to avoid + runtime allocation. pad_alloc_buffer() is allowed to block the upstream + element until buffers are recycled in the pool. + + - the pad_alloc_buffer function call can be passed downstream to the sink + that actually will perform the allocation. A fallback option exists to use + a default memory bufferpool whe there is no alloc_buffer function installed + on a pad. + + - Upstream renegotiation is performed by making the pad_alloc_buffer function + return a buffer with new caps. + + +Problems +-------- + + - There is currently no helper base class to implement efficient buffer pools + meaning that each element has to implement its own version. + + - There is no negotiation between elements about their buffer requirements. + Upstream elements that decide to use pad_alloc_buffer() can find that the + buffer they received is not appropriate at all. The most common problem + is that the buffers don't have the right alignment or insufficient padding. + + - There is no negotiation of minimum and maximum amounts of preallocated + buffers. In order to not avoid deadlocks, this means that buffer pool + implementations should be able to allocate unlimited amounts of buffers and + are never allowed to block in pad_alloc_buffer() + + +Requirements +------------ + + - maintain and reuse a list of buffers in a reusable base GstBufferPool + object + + - negotiate allocation configuration between source and sink pad. + - have minimum and maximum amount of buffers with the option of + preallocating buffers. + - alignment and padding support + - arbitrary extra options + + - integrate with dynamic caps renegotiation + + - dynamically change bufferpool configuration based on pipeline changes. + + - allow the application to control buffer allocation + + +GstBufferPool +------------- + + The bufferpool object manages a list of buffers with the same properties such + as size, padding and alignment. + + The bufferpool has two states: flushing and non-flushing. In the flushing + state, the bufferpool can be configured with the required allocation + preferences. In the non-flushing state, buffers can be retrieved from and + returned to the pool. + + The default implementation of the bufferpool is able to allocate buffers + from main memory with arbitrary alignment and padding/prefix. + + Custom implementations of the bufferpool can override the allocation and + free algorithms of the buffers from the pool. This should allow for + different allocation strategies such as using shared memory or hardware + mapped memory. + + The bufferpool object is also used to perform the negotiation of configuration + between elements. + + +GstPad +------ + + The GstPad has a method to query a GstBufferPool and its configuration. + + GstBufferPool * gst_pad_query_bufferpool (GstPad * pad); + + This function can return a handle to a bufferpool object or NULL when no + bufferpool object can be provided by the pad. + + This function should return a bufferpool object with the + GstBufferPoolConfig set to the desired parameters of the buffers that will be + handled by the given pad. This function can only be called on a sinkpad and + will usually be called by the peer srcpad with the convenience method: + + GstBufferPool * gst_pad_peer_query_bufferpool (GstPad * pad); + + + There is also a new function to configure a bufferpool on a pad and its peer + pad: + + gboolean gst_pad_set_bufferpool (GstPad * pad, GstBufferPool *pool); + + This function is to inform a pad and its peer pad that a bufferpool should + be used for allocation (on source pads) and that bufferpool is used by the + upstream element (on sinkpads). + + The currently configured bufferpool can be retrieved with: + + GstBufferPool * gst_pad_get_bufferpool (GstPad * pad); + + New functions exist to configure these bufferpool functions on pads: + gst_pad_set_querybufferpool_function and gst_pad_set_setbufferpool_function. + + +negotiating pool and config +--------------------------- + +Since upstream needs to allocate buffers from a buffer pool, it should first +negotiate a buffer pool with the downstream element. We propose a simple +scheme where a sink can propose a bufferpool and some configuration and where +the source can choose to use this allocator or use its own. + +The algorithm for doing this is rougly like this: + + + /* srcpad knows media type and size of buffers and is ready to + * prepare an output buffer but has no pool yet */ + + /* first get the pool from the downstream peer */ + pool = gst_pad_peer_query_bufferpool (srcpad); + + if (pool != NULL) { + GstBufferPoolConfig config; + + /* clear the pool so that we can reconfigure it */ + gst_buffer_pool_set_flushing (pool, TRUE); + + do { + /* get the config */ + gst_buffer_pool_get_config (pool, &config); + + /* check and modify the config to match our requirements */ + if (!tweak_config (&config)) { + /* we can't tweak the config any more, exit and fail */ + gst_object_unref (pool); + pool = NULL; + break; + } + } + /* update the config */ + while (!gst_buffer_pool_set_config (pool, &config)); + + /* we managed to update the config, all is fine now */ + /* set the pool to non-flushing to make it allocate things */ + gst_buffer_pool_set_flushing (pool, FALSE); + } + + if (pool == NULL) { + /* still no pool, we create one ourself with our ideal config */ + pool = gst_buffer_pool_new (...); + } + + /* now set the pool on this pad and the peer pad */ + gst_pad_set_bufferpool (pad, pool); + + +Negotiation is the same for both push and pull mode. In the case of pull +mode scheduling, the srcpad will perform the negotiation of the pool +when it receives the first pull request. + + +Allocating from pool +-------------------- + + Buffers are allocated from the pool of a pad: + + res = gst_buffer_pool_acquire_buffer (pool, &buffer, ¶ms); + + convenience functions to automatically get the pool from a pad can be made: + + res = gst_pad_acquire_buffer (pad, &buffer, ¶ms); + + Buffers are refcounted in te usual way. When the refcount of the buffer + reaches 0, the buffer is automatically returned to the pool. This is achieved + by setting and reffing the pool as a new buffer member. + + Since all the buffers allocated from the pool keep a reference to the pool, + when nothing else is holding a refcount to the pool, it will be finalized + when all the buffers from the pool are unreffed. By setting the pool to + the flushing state we can drain all buffers from the pool. + + +Renegotiation +------------- + +Renegotiation of the bufferpool might need to be performed when the +configuration of the pool changes. Changes can be in the buffer size (because +of a caps change), alignment or number of buffers. + +* downstream + + When the upstream element wants to negotiate a new format, it might need + to renegotiate a new bufferpool configuration with the downstream element. + This can, for example, happen when the buffer size changes. + + We can not just reconfigure the existing bufferpool because there might + still be outstanding buffers from the pool in the pipeline. Therefore we + need to create a new bufferpool for the new configuration while we let the + old pool drain. + + Implementations can choose to reuse the same bufferpool object and wait for + the drain to finish before reconfiguring the pool. + + The element that wants to renegotiate a new bufferpool uses exactly the same + algorithm as when it first started. + +* upstream + + When a downstream element wants to negotiate a new format, it will send a + RECONFIGURE event upstream. This instructs upstream to renegotiate both + the format and the bufferpool when needed. + + A pipeline reconfiguration is when new elements are added or removed from + the pipeline or when the topology of the pipeline changes. Pipeline + reconfiguration also triggers possible renegotiation of the bufferpool and + caps. + + A RECONFIGURE event tags each pad it travels on as needing reconfiguration. + The next buffer allocation will then require the renegotiation or + reconfiguration of a pool. + + If downstream has specified a RENEGOTIATE flag, it must be prepared to + received NOT_NEGOTIATED results when allocating buffers, which instructs + it to start caps and bufferpool renegotiation. When using this flag, + upstream can more quickly react to downstream format or size changes. + + +Shutting down +------------- + + In push mode, a source pad is responsible for setting the pool to the + flushing state when streaming stops. The flush will unblock any pending + allocations so that the element can shut down. + + In pull mode, the sink element should set the pool to the flushing state when + shutting down so that the peer _get_range() function can unblock. + + In the flushing state, all the buffers that are returned to the pool will + be automatically freed by the pool and new allocations will fail. + + + + +Use cases +--------- + +1) videotestsrc ! xvimagesink + + Before videotestsrc can output a buffer, it needs to negotiate caps and + a bufferpool with the downstream peer pad. + + First it will negotiate a suitable format with downstream according to the + normal rules. + + Then it does gst_pad_peer_query_bufferpool() which triggers the querybufferpool + function installed on the xvimagesink pad. This bufferpool is currently in + the flushing state and thus has no buffers allocated. + + videotestsrc gets the configuration of the bufferpool object. This + configuration lists the desired configuration of the xvimagesink, which can + have specific alignment and/or min/max amount of buffers. + + videotestsrc updates the configuration of the bufferpool, it will likely + set the min buffers to 1 and the size of the desired buffers. It then + updates the bufferpool configuration with the new properties. + + When the configuration is successfully updated, videotestsrc sets the + bufferpool on its source pad, this will also set the pool on the peer + sinkpad. + + It then sets the bufferpool to the non-flushing state. This preallocates + the buffers in the pool (if needed). This operation can fail when there + is not enough memory available. Since the bufferpool is provided by + xvimagesink, it will allocate buffers backed by an XvImage and pointing + to shared memory with the X server. + + If the bufferpool is successfully activated, videotestsrc can acquire a + buffer from the pool, set the caps on it, fill in the data and push it + out to xvimagesink. + + xvimagesink can know that the buffer originated from its pool by following + the pool member or checking the specific GType of it GstBuffer subclass. + It might need to get the parent buffer first in case of subbuffers. + + when shutting down, videotestsrc will set the pool to the flushing state, + this will cause further allocations to fail and currently allocated buffers + to be freed. videotestsrc will then free the pool and stop streaming. + + +2) videotestsrc ! queue ! myvideosink + + In this second use case we have a videosink that can at most allocate + 3 video buffers. + + Again videotestsrc will have to negotiate a bufferpool with the peer + element. For this it will perform gst_pad_peer_query_bufferpool() which + queue will proxy to its downstream peer element. + + The bufferpool returned from myvideosink will have a max_buffers set to 3. + queue and videotestsrc can operate with this upper limit because none of + those elements require more than that amount of buffers for temporary + storage. + + The bufferpool of myvideosink will then be configured with the size of the + buffers for the negotiated format and according to the padding and alignment + rules. When videotestsrc sets the pool to non-flushing, the 3 video + buffers will be preallocated in the pool. + + The pool will then be configured on the src of videotestsrc and the + sinkpad of the queue. The queue will proxy the setbufferpool method to + its srcpad, which finally configures the pool all the way to the sink. + + videotestsrc acquires a buffer from the configured pool on its srcpad and + pushes this into the queue. When the videotestsrc has acquired and pushed + 3 frames, the next call to gst_buffer_pool_acquire_buffer() will block + (assuming the GST_BUFFER_POOL_FLAG_WAIT is specified). + + When the queue has pushed out a buffer and the sink has rendered it, the + refcount of the buffer reaches 0 and the buffer is recycled in the pool. + This will wake up the videotestsrc that was blocked, waiting for more + buffers and will make it produce the next buffer. + + In this setup, there are at most 3 buffers active in the pipeline and + the videotestsrc is rate limited by the rate at which buffers are recycled + in the bufferpool. + + When shutting down, videotestsrc will first set the bufferpool on the srcpad + to flushing. This causes any pending (blocked) acquire to return with a + WRONG_STATE result and causes the streaming thread to pause. + + +3) .. ! myvideodecoder ! queue ! fakesink + + In this case, the myvideodecoder requires buffers to be aligned to 128 + bytes and padded with 4096 bytes. The pipeline starts out with the + decoder linked to a fakesink but we will then dynamically change the + sink to one that can provide a bufferpool. + + When it negotiates the size with the downstream element fakesink, it will + receive a NULL bufferpool because fakesink does not provide a bufferpool. + It will then select it own custom bufferpool to start the datatransfer. + + At some point we block the queue srcpad, unlink the queue from the + fakesink, link a new sink, set the new sink to the PLAYING state and send + the right newsegment event to the sink. Linking the new sink would + automatically send a RENEGOTIATE event upstream and, through queue, inform + myvideodecoder that it should renegotiate its bufferpool because downstream + has been reconfigured. + + Before pushing the next buffer, myvideodecoder would renegotiate a new + bufferpool. To do this, it performs the usual bufferpool negotiation + algorithm. If it can obtain and configure a new bufferpool from downstream, + it sets its own (old) pool to flushing and unrefs it. This will eventually + drain and unref the old bufferpool. + + The new bufferpool is set as the new bufferpool for the srcpad and sinkpad + of the queue and set to the non-flushing state. + + +4) .. ! myvideodecoder ! queue ! myvideosink + + myvideodecoder has negotiated a bufferpool with the downstream myvideosink + to handle buffers of size 320x240. It has now detected a change in the + video format and need to renegotiate to a resolution of 640x480. This + requires it to negotiate a new bufferpool with a larger buffersize. + + When myvideodecoder needs to get the bigger buffer, it starts the + negotiation of a new bufferpool. It queries a bufferpool from downstream, + reconfigures it with the new configuration (which includes the bigger buffer + size), it sets the bufferpool to non-flushing and sets the bufferpool as + the new pool for the srcpad and its peer. This automatically flushes the + old pool and unrefs it, which causes the old format to drain. + + It then uses the new bufferpool for allocating new buffers of the new + dimension. + + If at some point, the decoder wants to switch to a lower resolution again, + it can choose to use the current pool (which has buffers that are larger + than the required size) or it can choose to renegotiate a new bufferpool. + + +5) .. ! myvideodecoder ! videoscale ! myvideosink + + myvideosink is providing a bufferpool for upstream elements and wants to + change the resolution. + + myvideosink sends a RENEGOTIATE event upstream to notify upstream that a + new format is desirable. upstream elements try to negotiate a new format + and bufferpool before pushing out a new buffer. The old bufferpools are + drained in the regular way. + + + diff --git a/docs/design/part-MT-refcounting.txt b/docs/design/part-MT-refcounting.txt index a6e652935..888607231 100644 --- a/docs/design/part-MT-refcounting.txt +++ b/docs/design/part-MT-refcounting.txt @@ -55,7 +55,7 @@ Shared data structures and writability: The advantage of this method is that no reader/writers locks are needed. all threads can concurrently read but writes happen locally on a new copy. In most cases _get_writable() can avoid a real copy because the calling method is the - only one holding a reference, wich makes read/writes very cheap. + only one holding a reference, which makes read/write very cheap. The drawback is that sometimes 1 needless copy can be done. This would happen when N threads call _get_writable() at the same time, all seeing that N @@ -193,7 +193,7 @@ Objects disposed it has to unset itself as the parent of the object before disposing itself, else the child object holds a parent pointer to invalid memory. - The responsibilites for an object that sinks other objects are summarised as: + The responsibilities for an object that sinks other objects are summarised as: - taking ownership of the object - call _object_set_parent() to set itself as the object parent, this call diff --git a/docs/design/part-block.txt b/docs/design/part-block.txt index 5a1cf8068..6a028b18f 100644 --- a/docs/design/part-block.txt +++ b/docs/design/part-block.txt @@ -38,7 +38,7 @@ is called or the sync block returned) no data is flowing in elem2.sink. In this situation, the streaming thread is blocked on a GCond and is waiting to be unblocked. -When sending a flushing seek upstream on elem1.src, the FLUSH_START and +When sending a flushing seek upstream on elem1.src, the FLUSH_START event will temporary unblock the streaming thread and make all pad functions that triggers a block (_push/_alloc_buffer/_push_event/_pull_range) return GST_FLOW_WRONG_STATE. This will then eventually pause the streaming thread diff --git a/docs/design/part-buffering.txt b/docs/design/part-buffering.txt index eb9d807bd..b53fedb6b 100644 --- a/docs/design/part-buffering.txt +++ b/docs/design/part-buffering.txt @@ -44,7 +44,8 @@ Some use cases: posted, which instructs the application to continue playback. - When during playback, the low watermark is hit, the queue will start posting BUFFERING messages again, making the application PAUSE the pipeline again - until the high watermark is hit again. + until the high watermark is hit again. This is called the rebuffering + stage. - during playback, the queue level will fluctuate between the high and the low watermark as a way to compensate for network irregularities. @@ -247,3 +248,61 @@ A GstBaseSrc in push mode replies to the BUFFERING query with: "stop" = current position "estimated-total" = -1 + +Buffering strategies +~~~~~~~~~~~~~~~~~~~~ + + Buffering strategies are specific implementations based on the buffering + message and query described above. + + Most strategies have to balance buffering time versus maximal playback + experience. + + * simple buffering + + NON-live pipelines are kept in the paused state while buffering messages with + a percent < 100% are received. + + This buffering strategy relies on the buffer size and low/high watermarks of + the element. It can work with a fixed size buffer in memory or on disk. + + The size of the buffer is usually expressed in a fixed amount of time units + and the estimated bitrate of the upstream source is used to convert this time + to bytes. + + All GStreamer applications must implement this strategy. Failure to do so + will result in starvation at the sink. + + * no-rebuffer strategy + + This strategy tries to buffer as much data as possible so that playback can + continue without any further rebuffering. + + This strategy is initially similar to simple buffering, the difference is in + deciding on the condition to continue playback. When a 100% buffering message + has been received, the application will not yet start the playback but it will + start a periodic buffering query, which will return the estimated amount of + buffering time left. When the estimated time left is less than the remaining + playback time, playback can continue. + + This strategy requires a unlimited buffer size in memory or on disk, such as + provided by elements that implement the incremental download buffering mode. + + Usually, the application can choose to start playback even before the + remaining buffer time elapsed in order to more quickly start the playback at + the expense of a possible rebuffering phase. + + * Incremental rebuffering + + The application implements the simple buffering strategy but with each + rebuffering phase, it increases the size of the buffer. + + This strategy has quick, fixed time startup times but incrementally longer + rebuffering times if the network is slower than the media bitrate. + + + + + + + diff --git a/docs/design/part-bufferlist.txt b/docs/design/part-bufferlist.txt index 91674509a..d8857814f 100644 --- a/docs/design/part-bufferlist.txt +++ b/docs/design/part-bufferlist.txt @@ -14,7 +14,7 @@ organized in a list) to be treated as a multiple groups of GstBuffers. This allo for the following extra functionality: - A logical GstBuffer (called a group) can consist of disjoint memory each with - their own copy/free and metadata. Logicallty the group should be treated as + their own copy/free and metadata. Logically the group should be treated as one single GstBuffer. - Multiple groups can be put into one bufferlist. This allows for a single method call to pass multiple (logical) buffers downstream. diff --git a/docs/design/part-clocks.txt b/docs/design/part-clocks.txt index 5f080c5d9..2345e4d61 100644 --- a/docs/design/part-clocks.txt +++ b/docs/design/part-clocks.txt @@ -5,7 +5,7 @@ The GstClock returns a monotonically increasing time with the method _get_time(). Its accuracy and base time depends on the specific clock implementation but time is always expessed in nanoseconds. Since the baseline of the clock is undefined, the clock time returned is not -meaningfull in itself, what matters are the deltas between two clock +meaningful in itself, what matters are the deltas between two clock times. The time reported by the clock is called the absolute_time. diff --git a/docs/design/part-element-sink.txt b/docs/design/part-element-sink.txt index edc8dd4d5..b04f1419e 100644 --- a/docs/design/part-element-sink.txt +++ b/docs/design/part-element-sink.txt @@ -3,7 +3,7 @@ Sink elements Sink elements consume data. They normally have no source pads. -typical sink elements include: +Typical sink elements include: - audio/video renderers - network sinks diff --git a/docs/design/part-gstbin.txt b/docs/design/part-gstbin.txt index 6cd911f24..bf486f86d 100644 --- a/docs/design/part-gstbin.txt +++ b/docs/design/part-gstbin.txt @@ -35,7 +35,7 @@ operations on itself to all of its children. This includes: - state changes - index get/set - - clock gst/set + - clock get/set The state change distribution is the most complex and is explained in part-states.txt. @@ -107,7 +107,7 @@ Subclassing Subclasses of GstBin are free to implement their own add/remove implementations. It is a good idea to update the GList of children so that the _iterate() functions -can still be used if the custom bin allows acces to its children. +can still be used if the custom bin allows access to its children. Any bin subclass can also implement a custom message handler by overriding the default message handler. diff --git a/docs/design/part-gstghostpad.txt b/docs/design/part-gstghostpad.txt index 919677405..b83523b9f 100644 --- a/docs/design/part-gstghostpad.txt +++ b/docs/design/part-gstghostpad.txt @@ -55,7 +55,7 @@ Some design requirements +----* internal | (------------) - Which we will abreviate to: + Which we will abbreviate to: (- X --------) | | diff --git a/docs/design/part-overview.txt b/docs/design/part-overview.txt index d45e8b07a..e29215097 100644 --- a/docs/design/part-overview.txt +++ b/docs/design/part-overview.txt @@ -53,7 +53,7 @@ The task of the application is to construct a pipeline as above using existing elements. This is further explained in the pipeline building topic. The application does not have to manage any of the complexities of the -actual dataflow/decoding/conversions/synchronsiation etc. but only calls high +actual dataflow/decoding/conversions/synchronisation etc. but only calls high level functions on the pipeline object such as PLAY/PAUSE/STOP. The application also receives messages and notifications from the pipeline such @@ -249,7 +249,7 @@ Dataflow and events Parallel to the dataflow is a flow of events. Unlike the buffers, events can pass both upstream and downstream. Some events only travel upstream others only downstream. -the events are used to denote special conditions in the dataflow such as EOS or +The events are used to denote special conditions in the dataflow such as EOS or to inform plugins of special events such as flushing or seeking. Some events must be serialized with the buffer flow, others don't. Serialized @@ -491,7 +491,7 @@ element performs the following steps. 5) send NEWSEGMENT event to inform all elements of the new position and to complete the seek. -In step 1) all dowstream elements have to return from any blocking operations +In step 1) all downstream elements have to return from any blocking operations and have to refuse any further buffers or events different from a FLUSH done. The first step ensures that the streaming thread eventually unblocks and that diff --git a/docs/design/part-preroll.txt b/docs/design/part-preroll.txt index 264492d0b..4ecd78f1c 100644 --- a/docs/design/part-preroll.txt +++ b/docs/design/part-preroll.txt @@ -6,7 +6,7 @@ has been queued on the input pad or pads. This process is called prerolling and is needed to fill the pipeline with buffers so that the transition to PLAYING goes as fast as possible with no visual delay for the user. -Preroll is also crucial in maintaining correct audio and video synchronsation +Preroll is also crucial in maintaining correct audio and video synchronisation and ensuring that no buffers are dropped in the sinks. After receiving a buffer (or EOS) on a pad the chain/event function should diff --git a/docs/design/part-progress.txt b/docs/design/part-progress.txt new file mode 100644 index 000000000..bb8fe6e1e --- /dev/null +++ b/docs/design/part-progress.txt @@ -0,0 +1,233 @@ +Progress Reporting +------------------ + +This document describes the design and use cases for the progress reporting +messages. + +PROGRESS messages are posted on the bus to inform the application about the +progress of asynchonous operations in the pipeline. This should not be confused +with asynchronous state changes. + +We accomodate for the following requirements: + + - Application is informed when an async operation starts and completes. + - It should be possible for the application to generically detect common + operations and incorporate their progress into the GUI. + - Applications can cancel pending operations by doing regular state changes. + - Applications should be abe able to wait for completion of async operations. + +We allow for the following scenarios: + + - Elements want to inform the application about asynchronous DNS lookups and + pending network requests. This includes starting and completing the lookup. + - Elements opening devices and resources assynchronously. + - Applications having more freedom to implement timeout and cancelation of + operations that currently block the state changes or happen invisibly behind + the scenes. + + +Rationale +~~~~~~~~~ + +The main reason for adding these extra progress notifications is twofold: + + 1) to give the application more information of what is going on + + When there are well defined progress information codes, applications + can let the user know about the status of the progress. We anticipate to + have at least DNS resolving and server connections and requests be well + defined. + + 2) To make the state changes non-blocking and cancelable. + + Currently state changes such as going to the READY or PAUSED state often do + blocking calls such as resolving DNS or connecting to a remote server. These + operations often block the main thread and are often not cancelable, causing + application lockups. + + We would like to make the state change function, instead, start a separate + thread that performs the blocking operations in a cancelable way. When going + back to the NULL state, all pending operations would be canceled immediately. + + For downward state changes, we want to let the application implement its own + timeout mechanism. For example: when stopping an RTSP stream, the clients + needs to send a TEARDOWN request to the server. This can however take an + unlimited amount of time in case of network problems. We want to give the + application an opportunity to wait (and timeout) for the completion of the + async operation before setting the element to the final NULL state. + +Progress updates are very similar to buffering messages in the same way that the +application can decide to wait for the completion of the buffering process +before performing the next state change. It might make sense to implement +buffering with the progress messages in the future. + + +Async state changes +~~~~~~~~~~~~~~~~~~~ + +GStreamer currently has a GST_STATE_CHANGE_ASYNC return value to note to the +application that a state change is happening assynchronously. + +The main purpose of this return value is to make the pipeline wait for preroll +and delay a future (upwards) state changes until the sinks are prerolled. + +In the case of async operations on source, this will automatically force sinks +to stay async because they will not preroll before the source can produce data. + +The fact that other asynchonous operations happen behind the scnes is irrelevant +for the prerolling process so it is not implemented with the ASYNC state change +return value in order to not complicate the state changes and mix concepts. + + +Use cases +~~~~~~~~~ + + * RTSP client (but also HTTP, MMS, ...) + + When the client goes from the READY to the PAUSED state, it opens a socket, + performs a DNS lookup, retieves the SDP and negotiates the streams. All these + operations currently block the state change function for an undefinite amount + of time and while they are blocking cannot be canceled. + + Instead, a thread would be started to perform these operations assynchronously + and the state change would complete with the usual NO_PREROLL return value. + Before starting the thread a PROGRESS message would be posted to mark the + start of the async operation. + + As the DNS lookup completes and the connection is established, PROGRESS messages + are posted on the bus to inform the application of the progress. When + something fails, an error is posted and a PROGRESS CANCELED message is posted. + The application can then stop the pipeline. + + If there are no errors and the setup of the streams completed successfully, a + PROGRESS COMPLETED is posted on the bus. The thread then goes to sleep and the + assynchronous operation completed. + + The RTSP protocol requires to send a TEARDOWN request to the server + before closing the connection and destroying the socket. A state change to the + READY state will issue the TEARDOWN request in the background and notify the + application of this pending request with a PROGRESS message. + + The application might want to only go to the NULL state after it got confirmation + that the TEARDOWN request completed or it might choose to go to NULL after a + timeout. It might also be possible that the application just want to close the + socket as fast as possible without waiting for completion of the TEARDOWN request. + + * Network performance measuring + + DNS lookup and connection times can be measured by calculating the elapsed + time between the various PROGRESS messages. + + + +Messages +~~~~~~~~ + + A new PROGRESS message will be created. + The following fields will be contained in the message: + + - "type", GST_TYPE_PROGRESS_TYPE + + - a set of types to define the type of progress + + GST_PROGRESS_TYPE_START: A new task is started in the background + GST_PROGRESS_TYPE_CONTINUE: The previous tasks completed and a new + one continues. This is done so that the application can follow + a set of continuous tasks and react to COMPLETE only when the + element completely finished. + GST_PROGRESS_TYPE_CANCELED: A task is canceled by the user. + GST_PROGRESS_TYPE_ERROR: A task stopped because of an error. In case of + an error, an error message will have been posted before. + GST_PROGRESS_TYPE_COMPLETE: A task completed successfully. + + - "code", G_TYPE_STRING + + A generic extensible string that can be used to programatically determine the + action that is in progress. Some standard predefined codes will be + defined. + + - "text", G_TYPE_STRING + + A user visible string detailing the action. + + - "percent", G_TYPE_INT between 0 and 100 + + Progress of the action as a percentage, the following values are allowed: + - GST_PROGRESS_TYPE_START always has a 0% value. + - GST_PROGRESS_TYPE_CONTINUE have a value between 0 and 100 + - GST_PROGRESS_TYPE_CANCELED, GST_PROGRESS_TYPE_ERROR and + GST_PROGRESS_TYPE_COMPLETE always have a 100% value. + + - "timeout", G_TYPE_INT in milliseconds + + The timeout of the async operation. -1 if unknown/unlimited.. + This field can be interesting to the application when it wants to display + some sort of progress indication. + + - .... + + Depending on the code, more fields can be put here. + + +Implementation +~~~~~~~~~~~~~~ + +Elements should not do blocking operations from the state change function. +Instead, elements should post an appropriate progress message with the right +code and of type GST_PROGRESS_TYPE_START and then start a thread to perform +the blocking calls in a cancelable manner. + +It is highly recommended to only start async operations from the READY to PAUSED +state and onwards and not from the NULL to READY state. The reason for this is +that streaming threads are usually started in the READY to PAUSED state and that +the current NULL to READY state change is used to perform a blocking check for +the presence of devices. + +The progress message needs to be posted from the state change function so that +the application can immediately take appropriate action after setting the state. + +The threads will usually perform many blocking calls with different codes +in a row, a client might first do a DNS query and then continue with +establishing a connection to the server. For this purpose the +GST_PROGRESS_TYPE_CONTINUE must be used. + +Usually, the thread used to perform the blocking operations can be used to +implement the streaming threads when needed. + +Upon downward state changes, operations that are busy in the thread are canceled +and GST_PROGRESS_TYPE_CANCELED is posted. + +The application can know about pending tasks because they received the +GST_PROGRESS_TYPE_START messages that didn't complete with a +GST_PROGRESS_TYPE_COMPLETE message, got canceled with a +GST_PROGRESS_TYPE_CANCELED or errored with GST_PROGRESS_TYPE_ERROR. +Applications should be able to choose if they wait for the pending +operation or cancel them. + +If an async operation fails, an error message is posted first before the +GST_PROGRESS_TYPE_ERROR progress message. + + +Categories +~~~~~~~~~~ + + We want to propose some standard codes here: + + "open" : A resource is being opened + "close" : A resource is being closed + + "name-lookup" : A DNS lookup. + + "connect" : A socket connection is established + + "disconnect" : a socket connection is closed + + "request" : A request is sent to a server and we are waiting for a + reply. This message is posted right before the request is sent + and completed when the reply has arrived completely. + + "mount" : A volume is being mounted + + "unmount" : A volume is being unmounted + + More codes can be posted by elements and can be made official later. diff --git a/docs/design/part-push-pull.txt b/docs/design/part-push-pull.txt index 29e415d3f..549f8d700 100644 --- a/docs/design/part-push-pull.txt +++ b/docs/design/part-push-pull.txt @@ -8,7 +8,7 @@ driving force in the pipeline as it initiates data transport. It is also possible for an element to pull data from an upstream element. The downstream element does this by calling gst_pad_pull_range() on one -of its sinkpads. In this mode, the upstream element is the driving force +of its sinkpads. In this mode, the downstream element is the driving force in the pipeline as it initiates data transfer. It is important that the elements are in the correct state to handle a diff --git a/docs/design/part-qos.txt b/docs/design/part-qos.txt index 5cb7a87e8..a1a07f5cd 100644 --- a/docs/design/part-qos.txt +++ b/docs/design/part-qos.txt @@ -16,6 +16,10 @@ made: in the sinks. - long term rate corrections based on trends observed in the sinks. +It is also possible for the application to artificially introduce delay +between synchronized buffers, this is called throttling. It can be used +to reduce the framerate, for example. + Sources of quality problems ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -23,6 +27,7 @@ Sources of quality problems - High CPU load - Network problems - Other resource problems such as disk load, memory bottlenecks etc. + - application level throttling QoS event @@ -31,6 +36,19 @@ QoS event The QoS event is generated by an element that synchronizes against the clock. It travels upstream and contains the following fields: + - type, GST_TYPE_QOS_TYPE: + The type of the QoS event, we have the following types and the default type + is GST_QOS_TYPE_UNDERFLOW: + + GST_QOS_TYPE_OVERFLOW: an element is receiving buffers too fast and can't + keep up processing them. Upstream should reduce the + rate. + GST_QOS_TYPE_UNDERFLOW: an element is receiving buffers too slowly and has + to drop them because they are too late. Upstream should + increase the processing rate. + GST_QOS_TYPE_THROTTLE: the application is asking to add extra delay between + buffers, upstream is allowed to drop buffers + - timestamp, G_TYPE_UINT64: The timestamp on the buffer that generated the QoS event. These timestamps are expressed in total running_time in the sink so that the value is ever @@ -39,7 +57,10 @@ travels upstream and contains the following fields: - jitter, G_TYPE_INT64: The difference of that timestamp against the current clock time. Negative values mean the timestamp was on time. Positive values indicate the - timestamp was late by that amount. + timestamp was late by that amount. When buffers are received in time and + throttling is not enabled, the QoS type field is set to OVERFLOW. + When throttling, the jitter contains the throttling delay added by the + application and the type is set to THROTTLE. - proportion, G_TYPE_DOUBLE: Long term prediction of the ideal rate relative to normal rate to get @@ -175,7 +196,7 @@ Values DR1 > 1.0 mean that the upstream element cannot produce buffers of duration D1 in real-time. It is exactly DR1 that tells the amount of speedup we require from upstream to regain real-time performance. -An element that is not receiving enough data is said to be starved. +An element that is not receiving enough data is said to be underflowed. Element measurements @@ -185,7 +206,7 @@ In addition to the measurements of the datarate of the upstream element, a typical element must also measure its own performance. Global pipeline performance problems can indeed also be caused by the element itself when it receives too much data it cannot process in time. The element is then said to -be flooded. +be overflowed. Short term correction @@ -259,6 +280,24 @@ as given by the proportion member. Failure to do so will certainly lead to more dropped frames and a generally worse QoS. +Throttling +---------- + +In throttle mode, the time distance between buffers is kept to a configurable +throttle interval. This means that effectively the buffer rate is limited +to 1 buffer per throttle interval. This can be used to limit the framerate, +for example. + +When an element is configured in throttling mode (this is usually only +implemented on sinks) it should produce QoS events upstream with the jitter +field set to the throttle interval. This should instruct upstream elements to +skip or drop the remaining buffers in the configured throttle interval. + +The proportion field is set to the desired slowdown needed to get the +desired throttle interval. Implementations can use the QoS Throttle type, +the proportion and the jitter member to tune their implementations. + + QoS strategies -------------- @@ -299,7 +338,7 @@ calculate a proportion. The processing time in system time is compared to render time to decide if the majority of the time is spend upstream or in the sink itself. This value -is used to decide flood or starvation. +is used to decide overflow or underflow. The number of rendered and dropped buffers is used to query stats on the sink. @@ -316,6 +355,11 @@ the human eye. A QoS message is posted whenever a (part of a) buffer is dropped. +In throttle mode, the sink sends QoS event upstream with the timestamp set to +the running_time of the latest buffer and the jitter set to the throttle interval. +If the throttled buffer is late, the lateness is subtracted from the throttle +interval in order to keep the desired throttle interval. + GstBaseTransform ---------------- @@ -380,7 +424,7 @@ Sources The QoS events only apply to push based sources since pull based sources are entirely controlled by another downstream element. -Sources can receive a flood or starvation event that can be used to switch to +Sources can receive a overflow or underflow event that can be used to switch to less demanding source material. In case of a network stream, a switch could be done to a lower or higher quality stream or additional enhancement layers could be used or ignored. diff --git a/docs/design/part-scheduling.txt b/docs/design/part-scheduling.txt index 3b8a00928..3e1734095 100644 --- a/docs/design/part-scheduling.txt +++ b/docs/design/part-scheduling.txt @@ -33,7 +33,7 @@ push function to push the result to the peer sinkpad. Deciding the scheduling mode ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -When tha pad is activated, the _activate() function is called. The pad can then +When a pad is activated, the _activate() function is called. The pad can then choose to activate itself in push or pull mode depending on upstream capabilities. @@ -43,13 +43,13 @@ activate function for the pad. The chain function ~~~~~~~~~~~~~~~~~~ -The chain function will be called when a upstream element perform a _push() on the pad. +The chain function will be called when a upstream element performs a _push() on the pad. The upstream element can be another chain based element or a pushing source. The getrange function ~~~~~~~~~~~~~~~~~~~~~ -The getrange function is called when a peer pad perform a _pull_range() on the pad. This +The getrange function is called when a peer pad performs a _pull_range() on the pad. This downstream pad can be a pulling element or another _pull_range() based element. Plug-in techniques diff --git a/docs/design/part-seeking.txt b/docs/design/part-seeking.txt index 699a8212b..8fddb91f3 100644 --- a/docs/design/part-seeking.txt +++ b/docs/design/part-seeking.txt @@ -11,15 +11,15 @@ pipeline. Sending the seek event to a bin will by default forward the event to all sinks in the bin. When performing a seek, the start and stop values of the segment can be -specified as absoulte positions or relative to the currently configured +specified as absolute positions or relative to the currently configured playback segment. Note that it is not possible to seek relative to the current playback position. To seek relative to the current playback position, one must query the position first and then perform an absolute seek to the desired position. -Feedback of the seek operation can be immediatly using the GST_SEEK_FLAG_FLUSH +Feedback of the seek operation can be immediately using the GST_SEEK_FLAG_FLUSH flag. With this flag, all pending data in the pipeline is discarded and playback -starts from the new position immediatly. +starts from the new position immediately. When the FLUSH flag is not set, the seek will be queued and executed as soon as possible, which might be after all queues are emptied. @@ -47,7 +47,7 @@ application has some time to issue a new seek to make the transition seamless. Typically the allowed delay is defined by the buffer sizes of the sinks as well as the size of any queues in the pipeline. -The seek can also change the playback speed of the configured segment. +The seek can also change the playback speed of the configured segment. A speed of 1.0 is normal speed, 2.0 is double speed. Negative values mean backward playback. @@ -83,8 +83,8 @@ FLUSH seeking ^^^^^^^^^^^^^ This is the most common way of performing a seek in a playback application. -The application issues a seek on the pipeline and the new media is immediatly -played after the seek calls returns. +The application issues a seek on the pipeline and the new media is immediately +played after the seek call returns. seeking without FLUSH diff --git a/docs/design/part-segments.txt b/docs/design/part-segments.txt index 2dfbbb068..d5b7e7682 100644 --- a/docs/design/part-segments.txt +++ b/docs/design/part-segments.txt @@ -80,7 +80,7 @@ Use case: FLUSHING seek synchronisation and position reporting. Since after a flushing seek the stream_time is reset to 0, the new buffer - will be rendered immediatly after the seek and the current_position will be + will be rendered immediately after the seek and the current_position will be the stream_time of the seek that was performed. The stop time is important when the video format contains B frames. The diff --git a/docs/design/part-states.txt b/docs/design/part-states.txt index ef1bbe5bc..a4fec2940 100644 --- a/docs/design/part-states.txt +++ b/docs/design/part-states.txt @@ -48,7 +48,7 @@ the following state changes are possible: PAUSED -> PLAYING - Most elements ignore this state change. - The pipeline selects a clock and distributes this to all the children - before setting them to PLAYING. This means that it is only alowed to + before setting them to PLAYING. This means that it is only allowed to synchronize on the clock in the PLAYING state. - The pipeline uses the clock and the running_time to calculate the base_time. The base_time is distributed to all children when performing the state @@ -113,7 +113,7 @@ _set_state(), called the STATE_LOCK. Setting state on elements ~~~~~~~~~~~~~~~~~~~~~~~~~ -The state of an element can be changed with _element_set_state(). When chaning +The state of an element can be changed with _element_set_state(). When changing the state of an element all intermediate states will also be set on the element until the final desired state is set. @@ -125,7 +125,7 @@ The _set_state() function can return 3 possible values: GST_STATE_SUCCESS: The state change is completed successfully. GST_STATE_ASYNC: The state change will complete later on. This can happen - When the element needs a long time to perform the state + when the element needs a long time to perform the state change or for sinks that need to receive the first buffer before they can complete the state change (preroll). @@ -143,7 +143,7 @@ When setting the state of an element, the STATE_PENDING is set to the required state. Then the state change function of the element is called and the result of that function is used to update the STATE and STATE_RETURN fields, STATE_NEXT, STATE_PENDING and STATE_RETURN fields. If the function returned ASYNC, this result -is immediatly returned to the caller. +is immediately returned to the caller. Getting state of elements diff --git a/docs/design/part-streams.txt b/docs/design/part-streams.txt index 0a4fbb5a3..1274ae01c 100644 --- a/docs/design/part-streams.txt +++ b/docs/design/part-streams.txt @@ -24,9 +24,9 @@ Typical stream ~~~~~~~~~~~~~~ A typical stream starts with a newsegment event that marks the - buffer timestamp range. After that buffers are send one after the + buffer timestamp range. After that buffers are sent one after the other. After the last buffer an EOS marks the end of the stream. No - more buffer are to be processed after the EOS event. + more buffers are to be processed after the EOS event. +--+ +-++-+ +-+ +---+ |NS| |B||B| ... |B| |EOS| diff --git a/docs/design/part-synchronisation.txt b/docs/design/part-synchronisation.txt index 2c3185b54..07d87c8a1 100644 --- a/docs/design/part-synchronisation.txt +++ b/docs/design/part-synchronisation.txt @@ -55,7 +55,7 @@ to PAUSED and restores this time based on the current absolute_time when going back to PLAYING. This allows for both clocks that progress when in the PAUSED state (systemclock) and clocks that don't (audioclock). -The clock and pipeline now provides a running_time to all elements that want to +The clock and pipeline now provide a running_time to all elements that want to perform synchronisation. Indeed, the running time can be observed in each element (during the PLAYING state) as: diff --git a/docs/gst/gstreamer-docs.sgml b/docs/gst/gstreamer-docs.sgml index b961835cc..ca969a141 100644 --- a/docs/gst/gstreamer-docs.sgml +++ b/docs/gst/gstreamer-docs.sgml @@ -57,6 +57,7 @@ Windows. It is released under the GNU Library General Public License </para> <xi:include href="xml/gst.xml" /> + <xi:include href="xml/gstatomicqueue.xml" /> <xi:include href="xml/gstbin.xml" /> <xi:include href="xml/gstbuffer.xml" /> <xi:include href="xml/gstbufferlist.xml" /> diff --git a/docs/gst/gstreamer-sections.txt b/docs/gst/gstreamer-sections.txt index 9df58068b..0fbaf2eb1 100644 --- a/docs/gst/gstreamer-sections.txt +++ b/docs/gst/gstreamer-sections.txt @@ -39,6 +39,22 @@ GstPluginLoaderFuncs </SECTION> <SECTION> +<FILE>gstatomicqueue</FILE> +<TITLE>GstAtomicQueue</TITLE> +GstAtomicQueue +gst_atomic_queue_new + +gst_atomic_queue_ref +gst_atomic_queue_unref + +gst_atomic_queue_push +gst_atomic_queue_peek +gst_atomic_queue_pop + +gst_atomic_queue_length +</SECTION> + +<SECTION> <FILE>gstbin</FILE> <TITLE>GstBin</TITLE> GstBin @@ -400,6 +416,7 @@ gst_clock_get_time gst_clock_new_single_shot_id gst_clock_new_periodic_id gst_clock_single_shot_id_reinit +gst_clock_periodic_id_reinit gst_clock_get_internal_time gst_clock_adjust_unlocked gst_clock_unadjust_unlocked @@ -522,6 +539,7 @@ gst_element_get_compatible_pad gst_element_get_compatible_pad_template gst_element_get_request_pad gst_element_get_static_pad +gst_element_request_pad gst_element_no_more_pads gst_element_release_request_pad gst_element_remove_pad @@ -785,8 +803,11 @@ gst_event_parse_tag gst_event_new_buffer_size gst_event_parse_buffer_size +GstQOSType gst_event_new_qos +gst_event_new_qos_full gst_event_parse_qos +gst_event_parse_qos_full GstSeekType GstSeekFlags @@ -815,6 +836,7 @@ GST_TYPE_EVENT GST_TYPE_EVENT_TYPE GST_TYPE_SEEK_TYPE GST_TYPE_SEEK_FLAGS +GST_TYPE_QOS_TYPE GST_TYPE_EVENT_TYPE_FLAGS <SUBSECTION Private> GST_EVENT_TYPE_SHIFT @@ -823,6 +845,7 @@ gst_event_get_type gst_event_type_get_type gst_seek_type_get_type gst_seek_flags_get_type +gst_qos_type_get_type gst_event_type_flags_get_type </SECTION> @@ -1232,6 +1255,10 @@ gst_message_parse_stream_status gst_message_set_stream_status_object gst_message_get_stream_status_object +GstProgressType +gst_message_new_progress +gst_message_parse_progress + <SUBSECTION Standard> GstMessageClass GST_MESSAGE @@ -1244,11 +1271,13 @@ GST_MESSAGE_GET_CLASS GST_TYPE_MESSAGE_TYPE GST_TYPE_STRUCTURE_CHANGE_TYPE GST_TYPE_STREAM_STATUS_TYPE +GST_TYPE_PROGRESS_TYPE <SUBSECTION Private> gst_message_get_type gst_message_type_get_type gst_structure_change_type_get_type gst_stream_status_type_get_type +gst_progress_type_get_type GST_MESSAGE_COND GST_MESSAGE_GET_LOCK GST_MESSAGE_LOCK @@ -2181,6 +2210,7 @@ GST_TAG_ISRC GST_TAG_ORGANIZATION GST_TAG_COPYRIGHT GST_TAG_COPYRIGHT_URI +GST_TAG_ENCODED_BY GST_TAG_COMPOSER GST_TAG_CONTACT GST_TAG_LICENSE @@ -2677,6 +2707,7 @@ GST_TYPE_ARRAY gst_value_list_append_value gst_value_list_prepend_value gst_value_list_concat +gst_value_list_merge gst_value_list_get_size gst_value_list_get_value diff --git a/docs/libs/gstreamer-libs-sections.txt b/docs/libs/gstreamer-libs-sections.txt index 553ad6e59..6f3a7bfe7 100644 --- a/docs/libs/gstreamer-libs-sections.txt +++ b/docs/libs/gstreamer-libs-sections.txt @@ -275,6 +275,8 @@ gst_base_sink_is_last_buffer_enabled gst_base_sink_get_last_buffer gst_base_sink_set_blocksize gst_base_sink_get_blocksize +gst_base_sink_get_throttle_time +gst_base_sink_set_throttle_time GST_BASE_SINK_PAD diff --git a/docs/manual/advanced-threads.xml b/docs/manual/advanced-threads.xml index dc6aea501..218606e8d 100644 --- a/docs/manual/advanced-threads.xml +++ b/docs/manual/advanced-threads.xml @@ -63,9 +63,9 @@ means to make data throughput between threads threadsafe, and it can also act as a buffer. Queues have several <classname>GObject</classname> properties to be configured for specific uses. For example, you can set - lower and upper tresholds for the element. If there's less data than - the lower treshold (default: disabled), it will block output. If - there's more data than the upper treshold, it will block input or + lower and upper thresholds for the element. If there's less data than + the lower threshold (default: disabled), it will block output. If + there's more data than the upper threshold, it will block input or (if configured to do so) drop data. </para> <para> diff --git a/docs/manual/basics-bins.xml b/docs/manual/basics-bins.xml index 733c466c0..fa09c8c73 100644 --- a/docs/manual/basics-bins.xml +++ b/docs/manual/basics-bins.xml @@ -136,6 +136,10 @@ main (int argc, } </programlisting> <para> + (This is a silly example of course, there already exists a much more + powerful and versatile custom bin like this: the playbin2 element.) + </para> + <para> Custom bins can be created with a plugin or an XML description. You will find more information about creating custom bin in the <ulink type="http" @@ -143,10 +147,31 @@ main (int argc, Writers Guide</ulink>. </para> <para> - Examples of such custom bins are the playbin and decodebin elements from<ulink + Examples of such custom bins are the playbin2 and uridecodebin elements from<ulink type="http" url="http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-plugins/html/index.html"> gst-plugins-base</ulink>. </para> </sect1> + <sect1 id="section-bin-state-change-handling"> + <title>Bins manage states of their children</title> + <para> + Bins manage the state of all elements contained in them. If you set + a bin (or a pipeline, which is a special top-level type of bin) to + a certain target state using <function>gst_element_set_state ()</function>, + it will make sure all elements contained within it will also be set + to this state. This means it's usually only necessary to set the state + of the top-level pipeline to start up the pipeline or shut it down. + </para> + <para> + Note, however, that if elements are added to a bin or pipeline that's + already running, , e.g. from within a "pad-added" or "new-decoded-pad" + signal callback, its state will not automatically be brought in line with + the current state or target state of the bin or pipeline it was added to. + Instead, you have to need to set it to the desired target state yourself + using <function>gst_element_set_state ()</function> or + <function>gst_element_sync_state_with_parent ()</function> when adding + elements to an already-running pipeline. + </para> + </sect1> </chapter> diff --git a/docs/manual/basics-elements.xml b/docs/manual/basics-elements.xml index 201cb25d3..257ec68fd 100644 --- a/docs/manual/basics-elements.xml +++ b/docs/manual/basics-elements.xml @@ -554,5 +554,16 @@ main (int argc, url="&URLAPI;GstBus.html"><classname>GstBus</classname></ulink>. See <xref linkend="chapter-bus"/> for details. </para> + <para> + When you set a bin or pipeline to a certain target state, it will usually + propagate the state change to all elements within the bin or pipeline + automatically, so it's usually only necessary to set the state of the + top-level pipeline to start up the pipeline or shut it down. However, + when adding elements dynamically to an already-running pipeline, e.g. + from within a "pad-added" or "new-decoded-pad" signal callback, you + need to set it to the desired target state yourself using + <function>gst_element_set_state ()</function> or + <function>gst_element_sync_state_with_parent ()</function>. + </para> </sect1> </chapter> diff --git a/docs/manual/basics-pads.xml b/docs/manual/basics-pads.xml index bdd7c18e3..f2501d6ad 100644 --- a/docs/manual/basics-pads.xml +++ b/docs/manual/basics-pads.xml @@ -118,6 +118,14 @@ main (int argc, <!-- example-begin pad.c d --> } <!-- example-end pad.c d --></programlisting> + <para> + It is not uncommon to add elements to the pipeline only from within + the "pad-added" or "new-decoded-pad" callback. If you do this, don't + forget to set the state of the newly-added elements to the target + state of the pipeline using + <function>gst_element_set_state ()</function> or + <function>gst_element_sync_state_with_parent ()</function>. + </para> </sect2> <sect2 id="section-pads-request"> diff --git a/docs/manuals.mak b/docs/manuals.mak index 353c44c23..ebf65e4ef 100644 --- a/docs/manuals.mak +++ b/docs/manuals.mak @@ -89,6 +89,7 @@ $(BUILDDIR)/$(MAIN): $(XML) $(CSS) $(EXTRA_SRC) # we should switch to xsltproc # docbook2html aka jade can't add the encoding easily to the html meta +# (but we are lazy and just abuse sed to add it) # jw -f docbook -b html -d pwg.dsl -o ../html -V '%use-id-as-filename%' $(MAIN) # this is a starting point # xsltproc --nonet /usr/share/xml/docbook/stylesheet/nwalsh/html/docbook.xsl pwg.xml @@ -98,7 +99,8 @@ html/index.html: $(BUILDDIR)/$(MAIN) $(PNG_BUILT) $(FIG_SRC) @echo "*** Generating HTML output ***" @-mkdir -p html @cp -f $(srcdir)/../image-png $(BUILDDIR)/image.entities - @cd $(BUILDDIR) && docbook2html -o ../html -V '%use-id-as-filename%' $(MAIN) + @cd $(BUILDDIR) && SP_ENCODING="UTF-8" docbook2html -o ../html -V '%use-id-as-filename%' $(MAIN) + @$(SED) -i -e 's/\(^CONTENT.*\)\(.>\)/\1;charset=UTF-8\2/' html/*html @test "x$(CSS)" != "x" && \ echo "Copying .css files: $(CSS)" && \ cp $(srcdir)/$(CSS) html @@ -111,7 +113,7 @@ $(DOC).ps: $(BUILDDIR)/$(MAIN) $(EPS_BUILT) $(PNG_SRC) $(FIG_SRC) @$(MAKE) check-local @echo "*** Generating PS output ***" @cp -f $(srcdir)/../image-eps $(BUILDDIR)/image.entities - cd $(BUILDDIR) && docbook2ps -o .. $(MAIN) + cd $(BUILDDIR) && SP_ENCODING="UTF-8" docbook2ps -o .. $(MAIN) # export LC_PAPER=$(PAPER_LOCALE) && cd $(BUILDDIR) && xmlto ps -o .. $(MAIN) $(DOC).pdf: $(DOC).ps diff --git a/docs/plugins/Makefile.am b/docs/plugins/Makefile.am index 0053e1e76..0e44c57f2 100644 --- a/docs/plugins/Makefile.am +++ b/docs/plugins/Makefile.am @@ -86,11 +86,14 @@ EXTRA_HFILES = \ $(top_srcdir)/plugins/elements/gstfilesrc.h \ $(top_srcdir)/plugins/elements/gstfilesink.h \ $(top_srcdir)/plugins/elements/gstidentity.h \ + $(top_srcdir)/plugins/elements/gstinputselector.h \ $(top_srcdir)/plugins/elements/gstmultiqueue.h \ + $(top_srcdir)/plugins/elements/gstoutputselector.h \ $(top_srcdir)/plugins/elements/gstqueue.h \ $(top_srcdir)/plugins/elements/gstqueue2.h \ $(top_srcdir)/plugins/elements/gsttypefindelement.h \ - $(top_srcdir)/plugins/elements/gsttee.h + $(top_srcdir)/plugins/elements/gsttee.h \ + $(top_srcdir)/plugins/elements/gstvalve.h # Images to copy into HTML directory. HTML_IMAGES = diff --git a/docs/plugins/gstreamer-plugins-docs.sgml b/docs/plugins/gstreamer-plugins-docs.sgml index a86fc8205..3d6a15cb6 100644 --- a/docs/plugins/gstreamer-plugins-docs.sgml +++ b/docs/plugins/gstreamer-plugins-docs.sgml @@ -25,11 +25,14 @@ <xi:include href="xml/element-filesrc.xml" /> <xi:include href="xml/element-filesink.xml" /> <xi:include href="xml/element-identity.xml" /> + <xi:include href="xml/element-input-selector.xml" /> <xi:include href="xml/element-multiqueue.xml" /> + <xi:include href="xml/element-output-selector.xml" /> <xi:include href="xml/element-queue.xml" /> <xi:include href="xml/element-queue2.xml" /> <xi:include href="xml/element-tee.xml" /> <xi:include href="xml/element-typefind.xml" /> + <xi:include href="xml/element-valve.xml" /> </chapter> <chapter> diff --git a/docs/plugins/gstreamer-plugins-sections.txt b/docs/plugins/gstreamer-plugins-sections.txt index 6f80cc532..185411214 100644 --- a/docs/plugins/gstreamer-plugins-sections.txt +++ b/docs/plugins/gstreamer-plugins-sections.txt @@ -71,6 +71,7 @@ GstFileSrc <SUBSECTION Standard> GstFileSrcClass GST_FILE_SRC +GST_FILE_SRC_CAST GST_IS_FILE_SRC GST_TYPE_FILE_SRC GST_FILE_SRC_CLASS @@ -125,6 +126,26 @@ gst_identity_get_type </SECTION> <SECTION> +<FILE>element-input-selector</FILE> +<TITLE>input-selector</TITLE> +GstInputSelector +<SUBSECTION Standard> +GstInputSelectorClass +GST_INPUT_SELECTOR +GST_INPUT_SELECTOR_CLASS +GST_IS_INPUT_SELECTOR +GST_IS_INPUT_SELECTOR_CLASS +GST_TYPE_INPUT_SELECTOR +gst_input_selector_get_type +GST_INPUT_SELECTOR_BROADCAST +GST_INPUT_SELECTOR_GET_COND +GST_INPUT_SELECTOR_GET_LOCK +GST_INPUT_SELECTOR_LOCK +GST_INPUT_SELECTOR_UNLOCK +GST_INPUT_SELECTOR_WAIT +</SECTION> + +<SECTION> <FILE>element-queue</FILE> <TITLE>queue</TITLE> GstQueue @@ -155,6 +176,7 @@ GST_IS_QUEUE2_CLASS GST_TYPE_QUEUE2 <SUBSECTION Private> GstQueue2Size +GstQueue2Range gst_queue2_get_type </SECTION> @@ -174,6 +196,20 @@ gst_multi_queue_get_type </SECTION> <SECTION> +<FILE>element-output-selector</FILE> +<TITLE>output-selector</TITLE> +GstOutputSelector +<SUBSECTION Standard> +GstOutputSelectorClass +GST_OUTPUT_SELECTOR +GST_OUTPUT_SELECTOR_CLASS +GST_IS_OUTPUT_SELECTOR +GST_IS_OUTPUT_SELECTOR_CLASS +GST_TYPE_OUTPUT_SELECTOR +gst_output_selector_get_type +</SECTION> + +<SECTION> <FILE>element-typefind</FILE> <TITLE>typefind</TITLE> GstTypeFindElement @@ -206,3 +242,18 @@ GST_IS_TEE_CLASS gst_tee_get_type </SECTION> +<SECTION> +<FILE>element-valve</FILE> +<TITLE>valve</TITLE> +GstValve +<SUBSECTION Standard> +GstValveClass +GST_VALVE +GST_IS_VALVE +GST_TYPE_VALVE +GST_VALVE_CLASS +GST_IS_VALVE_CLASS +<SUBSECTION Private> +gst_valve_get_type +</SECTION> + diff --git a/docs/plugins/gstreamer-plugins.args b/docs/plugins/gstreamer-plugins.args index 6bc594b9d..9fa076ba2 100644 --- a/docs/plugins/gstreamer-plugins.args +++ b/docs/plugins/gstreamer-plugins.args @@ -998,3 +998,73 @@ <DEFAULT>0</DEFAULT> </ARG> +<ARG> +<NAME>GstValve::drop</NAME> +<TYPE>gboolean</TYPE> +<RANGE></RANGE> +<FLAGS>rw</FLAGS> +<NICK>Drop buffers and events</NICK> +<BLURB>Whether to drop buffers and events or let them through.</BLURB> +<DEFAULT>FALSE</DEFAULT> +</ARG> + +<ARG> +<NAME>GstOutputSelector::active-pad</NAME> +<TYPE>GstPad*</TYPE> +<RANGE></RANGE> +<FLAGS>rw</FLAGS> +<NICK>Active pad</NICK> +<BLURB>Currently active src pad.</BLURB> +<DEFAULT></DEFAULT> +</ARG> + +<ARG> +<NAME>GstOutputSelector::resend-latest</NAME> +<TYPE>gboolean</TYPE> +<RANGE></RANGE> +<FLAGS>rw</FLAGS> +<NICK>Resend latest buffer</NICK> +<BLURB>Resend latest buffer after a switch to a new pad.</BLURB> +<DEFAULT>FALSE</DEFAULT> +</ARG> + +<ARG> +<NAME>GstOutputSelector::pad-negotiation-mode</NAME> +<TYPE>GstOutputSelectorPadNegotiationMode</TYPE> +<RANGE></RANGE> +<FLAGS>rw</FLAGS> +<NICK>Pad negotiation mode</NICK> +<BLURB>The mode to be used for pad negotiation.</BLURB> +<DEFAULT>All</DEFAULT> +</ARG> + +<ARG> +<NAME>GstInputSelector::active-pad</NAME> +<TYPE>GstPad*</TYPE> +<RANGE></RANGE> +<FLAGS>rw</FLAGS> +<NICK>Active pad</NICK> +<BLURB>The currently active sink pad.</BLURB> +<DEFAULT></DEFAULT> +</ARG> + +<ARG> +<NAME>GstInputSelector::n-pads</NAME> +<TYPE>guint</TYPE> +<RANGE></RANGE> +<FLAGS>r</FLAGS> +<NICK>Number of Pads</NICK> +<BLURB>The number of sink pads.</BLURB> +<DEFAULT>0</DEFAULT> +</ARG> + +<ARG> +<NAME>GstInputSelector::select-all</NAME> +<TYPE>gboolean</TYPE> +<RANGE></RANGE> +<FLAGS>rw</FLAGS> +<NICK>Select all mode</NICK> +<BLURB>Forwards data from all input pads.</BLURB> +<DEFAULT>FALSE</DEFAULT> +</ARG> + diff --git a/docs/plugins/gstreamer-plugins.hierarchy b/docs/plugins/gstreamer-plugins.hierarchy index 6d424b5e9..d6a33d1a2 100644 --- a/docs/plugins/gstreamer-plugins.hierarchy +++ b/docs/plugins/gstreamer-plugins.hierarchy @@ -17,11 +17,14 @@ GObject GstIdentity GstBin GstPipeline + GstInputSelector GstMultiQueue + GstOutputSelector GstQueue GstQueue2 GstTee GstTypeFindElement + GstValve GstIndex GstFileIndex GstMemIndex diff --git a/docs/plugins/gstreamer-plugins.signals b/docs/plugins/gstreamer-plugins.signals index 265feee66..636d2e19a 100644 --- a/docs/plugins/gstreamer-plugins.signals +++ b/docs/plugins/gstreamer-plugins.signals @@ -107,3 +107,20 @@ GstMultiQueue *gstmultiqueue GstMultiQueue *gstmultiqueue </SIGNAL> +<SIGNAL> +<NAME>GstInputSelector::block</NAME> +<RETURNS>gint64</RETURNS> +<FLAGS>la</FLAGS> +GstInputSelector *gstinputselector +</SIGNAL> + +<SIGNAL> +<NAME>GstInputSelector::switch</NAME> +<RETURNS>void</RETURNS> +<FLAGS>l</FLAGS> +GstInputSelector *gstinputselector +GstPad *arg1 +gint64 arg2 +gint64 arg3 +</SIGNAL> + diff --git a/docs/plugins/inspect/plugin-coreelements.xml b/docs/plugins/inspect/plugin-coreelements.xml index 261a6ac1c..10036790c 100644 --- a/docs/plugins/inspect/plugin-coreelements.xml +++ b/docs/plugins/inspect/plugin-coreelements.xml @@ -3,7 +3,7 @@ <description>standard GStreamer elements</description> <filename>../../plugins/elements/.libs/libgstcoreelements.so</filename> <basename>libgstcoreelements.so</basename> - <version>0.10.31.1</version> + <version>0.10.32.1</version> <license>LGPL</license> <source>gstreamer</source> <package>GStreamer git</package> @@ -142,6 +142,27 @@ </pads> </element> <element> + <name>input-selector</name> + <longname>Input selector</longname> + <class>Generic</class> + <description>N-to-1 input stream selector</description> + <author>Julien Moutte <julien@moutte.net>, Jan Schmidt <thaytan@mad.scientist.com>, Wim Taymans <wim.taymans@gmail.com></author> + <pads> + <caps> + <name>sink%d</name> + <direction>sink</direction> + <presence>request</presence> + <details>ANY</details> + </caps> + <caps> + <name>src</name> + <direction>source</direction> + <presence>always</presence> + <details>ANY</details> + </caps> + </pads> + </element> + <element> <name>multiqueue</name> <longname>MultiQueue</longname> <class>Generic</class> @@ -163,6 +184,27 @@ </pads> </element> <element> + <name>output-selector</name> + <longname>Output selector</longname> + <class>Generic</class> + <description>1-to-N output stream selector</description> + <author>Stefan Kost <stefan.kost@nokia.com></author> + <pads> + <caps> + <name>sink</name> + <direction>sink</direction> + <presence>always</presence> + <details>ANY</details> + </caps> + <caps> + <name>src%d</name> + <direction>source</direction> + <presence>request</presence> + <details>ANY</details> + </caps> + </pads> + </element> + <element> <name>queue</name> <longname>Queue</longname> <class>Generic</class> @@ -246,5 +288,26 @@ </caps> </pads> </element> + <element> + <name>valve</name> + <longname>Valve element</longname> + <class>Filter</class> + <description>Drops buffers and events or lets them through</description> + <author>Olivier Crete <olivier.crete@collabora.co.uk></author> + <pads> + <caps> + <name>sink</name> + <direction>sink</direction> + <presence>always</presence> + <details>ANY</details> + </caps> + <caps> + <name>src</name> + <direction>source</direction> + <presence>always</presence> + <details>ANY</details> + </caps> + </pads> + </element> </elements> </plugin>
\ No newline at end of file diff --git a/docs/plugins/inspect/plugin-coreindexers.xml b/docs/plugins/inspect/plugin-coreindexers.xml index 4028dfa9a..29f92f5d5 100644 --- a/docs/plugins/inspect/plugin-coreindexers.xml +++ b/docs/plugins/inspect/plugin-coreindexers.xml @@ -3,7 +3,7 @@ <description>GStreamer core indexers</description> <filename>../../plugins/indexers/.libs/libgstcoreindexers.so</filename> <basename>libgstcoreindexers.so</basename> - <version>0.10.31.1</version> + <version>0.10.32.1</version> <license>LGPL</license> <source>gstreamer</source> <package>GStreamer git</package> diff --git a/docs/pwg/building-signals.xml b/docs/pwg/building-signals.xml index 3070eb2b5..73be168e9 100644 --- a/docs/pwg/building-signals.xml +++ b/docs/pwg/building-signals.xml @@ -10,7 +10,7 @@ for application-element interaction, signals are probably not what you're looking for. In many cases, however, signals can be very useful. See the <ulink type="http" - url="http://www.le-hacker.org/papers/gobject/index.html">GObject + url="http://library.gnome.org/devel/gobject/stable/">GObject documentation</ulink> for all internals about signals. </para> </chapter> diff --git a/docs/pwg/pwg.xml b/docs/pwg/pwg.xml index e57f0caf4..9a97b69b0 100644 --- a/docs/pwg/pwg.xml +++ b/docs/pwg/pwg.xml @@ -1,4 +1,4 @@ -<?xml version='1.0'?> +<?xml version='1.0' encoding='utf-8'?> <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [ <!ENTITY % image-entities SYSTEM "image.entities"> diff --git a/docs/random/caps b/docs/random/caps index 415a74a5a..bbcca9b7c 100644 --- a/docs/random/caps +++ b/docs/random/caps @@ -104,7 +104,7 @@ factory: The static autoplugger has to be carefull when connecting the mpg123 element with the audiosink because it is theoretically possible that the mpg123 element outputs raw audio with a rate that cannot be -handled by the audiosink (ex. 4000KHz). In the absense of another +handled by the audiosink (ex. 4000KHz). In the absence of another audiosink with more capabilities, the autoplugging of this simple pipeline will not be possible and would fail. diff --git a/docs/random/ensonic/draft-bufferpools.txt b/docs/random/ensonic/draft-bufferpools.txt index 617fe18e1..e03669ba2 100644 --- a/docs/random/ensonic/draft-bufferpools.txt +++ b/docs/random/ensonic/draft-bufferpools.txt @@ -66,7 +66,7 @@ for the negotiated format. The bufferpool could be a helper GObject (like we use GstAdapter). If would manage a collection of GstBuffers. For each buffer t tracks wheter its in use or -available. The bufferpool in gst-plgin-good/sys/v4l2/gstv4l2bufferpool might be +available. The bufferpool in gst-plugin-good/sys/v4l2/gstv4l2bufferpool might be a starting point. diff --git a/docs/random/omega/TODO-0.1.0 b/docs/random/omega/TODO-0.1.0 index d7c1619e3..498eb36fb 100644 --- a/docs/random/omega/TODO-0.1.0 +++ b/docs/random/omega/TODO-0.1.0 @@ -9,7 +9,7 @@ Add interface to control the level of output, both compile- and run-time (done) Make sure the build is capable of being run without any debugging noise -Make sure that the absense of any of the optional libraries will not be fatal +Make sure that the absence of any of the optional libraries will not be fatal Make sure all the old plugins don't build normally, via some configure option? diff --git a/docs/random/thomasvs/capturing b/docs/random/thomasvs/capturing index 9832c677c..806e8fae3 100644 --- a/docs/random/thomasvs/capturing +++ b/docs/random/thomasvs/capturing @@ -60,7 +60,7 @@ ELEMENTS (v4lsrc, alsasrc, osssrc) - a second element can be responsible for making the stream time-contiguous. (ie, T1 + D1 = T2 for all buffers). This way they are made - acceptible for gapless presentation (which is useful for audio). + acceptable for gapless presentation (which is useful for audio). - The element treats the incoming stream as data-contiguous but not necessarily time-contiguous. - If the timestamps are contiguous as well, then everything is fine and diff --git a/docs/random/wtay/events b/docs/random/wtay/events index 5498a68fa..c49f6fde7 100644 --- a/docs/random/wtay/events +++ b/docs/random/wtay/events @@ -11,7 +11,7 @@ and properties are still used for plugin-app communication). Events will be generated by either a plugin or the app. It should be possible for a plugin to generate an event on one of its pads and it -should be possible for an app to insert an event on an abitrary pad in +should be possible for an app to insert an event on an arbitrary pad in the pipeline. diff --git a/docs/random/wtay/events3 b/docs/random/wtay/events3 index 493d67383..09962236d 100644 --- a/docs/random/wtay/events3 +++ b/docs/random/wtay/events3 @@ -11,7 +11,7 @@ and properties are still used for plugin-app communication). Events will be generated by either a plugin or the app. It should be possible for a plugin to generate an event on one of its pads and it -should be possible for an app to insert an event on an abitrary pad or +should be possible for an app to insert an event on an arbitrary pad or element in the pipeline. diff --git a/docs/slides/outline b/docs/slides/outline index 5808dbb01..10fc8c0da 100644 --- a/docs/slides/outline +++ b/docs/slides/outline @@ -3,7 +3,7 @@ Introduction (1) GStreamer is a library and set of tools to build arbitrary, reconfigurable filter graphs. It derives from the OGI Pipeline and DirectShow (docs, no experience), and is in its second - generation (first was completed/abandonded *in* Atlanta on the + generation (first was completed/abandoned *in* Atlanta on the way to the Linux Expo). 0.1.0 release is scheduled for Oct 31, 1999. Will cover Background, Goals, Design, and Futures |