When playing complex media, each sound and video sample must be played in a specific order at a specific time. For this purpose, GStreamer provides a synchronization mechanism.
GStreamer provides support for the following use cases:
Non-live sources with access faster than playback rate. This is the case where one is reading media from a file and playing it back in a synchronized fashion. In this case, multiple streams need to be synchronized, like audio, video and subtitles.
Capture and synchronized muxing/mixing of media from multiple live sources. This is a typical use case where you record audio and video from a microphone/camera and mux it into a file for storage.
Streaming from (slow) network streams with buffering. This is the typical web streaming case where you access content from a streaming server with http.
Capture from live source and and playback to live source with configurable latency. This is used when, for example, capture from a camera, apply an effect and display the result. It is also used when streaming low latency content over a network with UDP.
Simultaneous live capture and playback from prerecorded content. This is used in audio recording cases where you play a previously recorded audio and record new samples, the purpose is to have the new audio perfectly in sync with the previously recorded data.
GStreamer uses a GstClock
object, buffer
timestamps and a SEGMENT event to synchronize streams in a pipeline
as we will see in the next sections.
In a typical computer, there are many sources that can be used as a
time source, e.g., the system time, soundcards, CPU performance
counters, ... For this reason, there are many
GstClock
implementations available in GStreamer.
The clock time doesn't always start from 0 or from some known value.
Some clocks start counting from some known start date, other clocks start
counting since last reboot, etc...
A GstClock
returns the
absolute-time
according to that clock with gst_clock_get_time ()
.
The absolute-time (or clock time) of a clock is monotonically increasing.
From the absolute-time is a running-time
calculated, which is simply the difference between a previous snapshot
of the absolute-time called the base-time.
So:
running-time = absolute-time - base-time
A GStreamer GstPipeline
object maintains a
GstClock
object and a base-time when it goes
to the PLAYING state. The pipeline gives a handle to the selected
GstClock
to each element in the pipeline along
with selected base-time. The pipeline will select a base-time in such
a way that the running-time reflects the total time spent in the
PLAYING state. As a result, when the pipeline is PAUSED, the
running-time stands still.
Because all objects in the pipeline have the same clock and base-time, they can thus all calculate the running-time according to the pipeline clock.