1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
|
<chapter id="chapter-advanced-clock">
<title>Clocking</title>
<para>
When playing complex media, each sound and video sample must be played in a
specific order at a specific time. For this purpose, GStreamer provides a
synchronization mechanism.
</para>
<sect1 id="section-clocks" xreflabel="Clocks">
<title>Clocks</title>
<para>
Time in &GStreamer; is defined as the value returned from a particular
<classname>GstClock</classname> object from the method
<function>gst_clock_get_time ()</function>.
</para>
<para>
In a typical computer, there are many sources that can be used as a
time source, e.g., the system time, soundcards, CPU performance
counters, ... For this reason, there are many
<classname>GstClock</classname> implementations available in &GStreamer;.
The clock time doesn't always start from 0 or from some known value.
Some clocks start counting from some known start date, other clocks start
counting since last reboot, etc...
</para>
<para>
As clocks return an absolute measure of time, they are not usually used
directly. Instead, differences between two clock times are used to
measure elapsed time according to a clock.
</para>
</sect1>
<sect1 id="section-clock-time-types" xreflabel="Clock running-time">
<title> Clock running-time </title>
<para>
A clock returns the <emphasis role="strong">absolute-time</emphasis>
according to that clock with <function>gst_clock_get_time ()</function>.
From the absolute-time is a <emphasis role="strong">running-time</emphasis>
calculated, which is simply the difference between a previous snapshot
of the absolute-time called the <emphasis role="strong">base-time</emphasis>.
So:
</para>
<para>
running-time = absolute-time - base-time
</para>
<para>
A &GStreamer; <classname>GstPipeline</classname> object maintains a
<classname>GstClock</classname> object and a base-time when it goes
to the PLAYING state. The pipeline gives a handle to the selected
<classname>GstClock</classname> to each element in the pipeline along
with selected base-time. The pipeline will select a base-time in such
a way that the running-time reflects the total time spent in the
PLAYING state. As a result, when the pipeline is PAUSED, the
running-time stands still.
</para>
<para>
Because all objects in the pipeline have the same clock and base-time,
they can thus all calculate the running-time according to the pipeline
clock.
</para>
</sect1>
<sect1 id="section-buffer-time-types" xreflabel="Buffer running-time">
<title> Buffer running-time </title>
<para>
To calculate a buffer running-time, we need a buffer timestamp and
the SEGMENT event that preceeded the buffer. First we can convert
the SEGMENT event into a <classname>GstSegment</classname> object
and then we can use the
<function>gst_segment_to_running_time ()</function> function to
perform the calculation of the buffer running-time.
</para>
<para>
Synchronization is now a matter of making sure that a buffer with a
certain running-time is played when the clock reaches the same
running-time. Usually this task is done by sink elements. Sink also
have to take into account the latency configured in the pipeline and
add this to the buffer running-time before synchronizing to the
pipeline clock.
</para>
</sect1>
<sect1 id="section-clock-obligations-of-each-element" xreflabel="Obligations
of each element">
<title>
Obligations of each element.
</title>
<para>
Let us clarify the contract between GStreamer and each element in the
pipeline.
</para>
<sect2>
<title>Non-live source elements </title>
<para>
Non-live source elements must place a timestamp in each buffer that
they deliver when this is possible. They must choose the timestamps
and the values of the SEGMENT event in such a way that the
running-time of the buffer starts from 0.
</para>
<para>
Some sources, such as filesrc, is not able to generate timestamps
on all buffers. It can and must however create a timestamp on the
first buffer (with a running-time of 0).
</para>
<para>
The source then pushes out the SEGMENT event followed by the
timestamped buffers.
</para>
</sect2>
<sect2>
<title>Live source elements </title>
<para>
Live source elements must place a timestamp in each buffer that
they deliver. They must choose the timestamps and the values of the
SEGMENT event in such a way that the running-time of the buffer
matches exactly the running-time of the pipeline clock when the first
byte in the buffer was captured.
</para>
</sect2>
<sect2>
<title>Parser/Decoder/Encoder elements </title>
<para>
Parser/Decoder elements must use the incoming timestamps and transfer
those to the resulting output buffers. They are allowed to interpolate
or reconstruct timestamps on missing input buffers when they can.
</para>
</sect2>
<sect2>
<title>Demuxer elements </title>
<para>
Demuxer elements can usually set the timestamps stored inside the media
file onto the outgoing buffers. They need to make sure that outgoing
buffers that are to be played at the same time have the same
running-time. Demuxers also need to take into account the incoming
timestamps on buffers and use that to calculate an offset on the outgoing
buffer timestamps.
</para>
</sect2>
<sect2>
<title>Muxer elements</title>
<para>
Muxer elements should use the incoming buffer running-time to mux the
different streams together. They should copy the incoming running-time
to the outgoing buffers.
</para>
</sect2>
<sect2>
<title>Sink elements</title>
<para>
If the element is intended to emit samples at a specific time (real time
playing), the element should require a clock, and thus implement the
method <function>set_clock</function>.
</para>
<para>
The sink should then make sure that the sample with running-time is played
exactly when the pipeline clock reaches that running-time + latency.
Some elements might use the clock API such as
<function>gst_clock_id_wait()</function>
to perform this action. Other sinks might need to use other means of
scheduling timely playback of the data.
</para>
</sect2>
</sect1>
</chapter>
|
