14.2. Caps negotiation use cases

In what follows we will look at some use cases for push-mode scheduling. The pull-mode scheduling negotiation phase is discussed in Section 14.5 and is actually similar as we will see.

Since the sink pads only suggest formats and the source pads need to decide, the most complicated work is done in the source pads. We can identify 3 caps negotiation use cases for the source pads:

14.2.1. Fixed negotiation

In this case, the source pad can only produce a fixed format. Usually this format is encoded inside the media. No downstream element can ask for a different format, the only way that the source pad will renegotiate is when the element decides to change the caps itself.

Elements that could implement fixed caps (on their source pads) are, in general, all elements that are not renegotiable. Examples include:

gst_pad_use_fixed_caps() is used on the source pad with fixed caps. As long as the pad is not negotiated, the default CAPS query will return the caps presented in the padtemplate. As soon as the pad is negotiated, the CAPS query will return the negotiated caps (and nothing else). These are the relevant code snippets for fixed caps source pads.


[..]
  pad = gst_pad_new_from_static_template (..);
  gst_pad_use_fixed_caps (pad);
[..]

      

The fixed caps can then be set on the pad by calling gst_pad_set_caps ().


[..]
    caps = gst_caps_new_simple ("audio/x-raw",
        "format", G_TYPE_STRING, GST_AUDIO_NE(F32),
        "rate", G_TYPE_INT, <samplerate>,
        "channels", G_TYPE_INT, <num-channels>, NULL);
    if (!gst_pad_set_caps (pad, caps)) {
      GST_ELEMENT_ERROR (element, CORE, NEGOTIATION, (NULL),
          ("Some debug information here"));
      return GST_FLOW_ERROR;
    }
[..]

      

These types of elements also don't have a relation between the input format and the output format, the input caps simply don't contain the information needed to produce the output caps.

All other elements that need to be configured for the format should implement full caps negotiation, which will be explained in the next few sections.

14.2.2. Transform negotiation

In this negotiation technique, there is a fixed transform between the element input caps and the output caps. This transformation could be parameterized by element properties but not by the content of the stream (see Section 14.2.1 for that use-case).

The caps that the element can accept depend on the (fixed transformation) downstream caps. The caps that the element can produce depend on the (fixed transformation of) the upstream caps.

This type of element can usually set caps on its source pad from the _event() function on the sink pad when it received the CAPS event. This means that the caps transform function transforms a fixed caps into another fixed caps. Examples of elements include:

Below is an example of a negotiation steps of a typical transform element. In the sink pad CAPS event handler, we compute the caps for the source pad and set those.


  [...]

static gboolean
gst_my_filter_setcaps (GstMyFilter *filter,
		       GstCaps *caps)
{
  GstStructure *structure;
  int rate, channels;
  gboolean ret;
  GstCaps *outcaps;

  structure = gst_caps_get_structure (caps, 0);
  ret = gst_structure_get_int (structure, "rate", &rate);
  ret = ret && gst_structure_get_int (structure, "channels", &channels);
  if (!ret)
    return FALSE;

  outcaps = gst_caps_new_simple ("audio/x-raw",
      "format", G_TYPE_STRING, GST_AUDIO_NE(S16),
      "rate", G_TYPE_INT, samplerate,
      "channels", G_TYPE_INT, channels, NULL);
  ret = gst_pad_set_caps (filter->srcpad, outcaps);
  gst_caps_unref (outcaps);

  return ret;
}

static gboolean
gst_my_filter_sink_event (GstPad    *pad,
		          GstObject *parent,
		          GstEvent  *event)
{
  gboolean ret;
  GstMyFilter *filter = GST_MY_FILTER (parent);

  switch (GST_EVENT_TYPE (event)) {
    case GST_EVENT_CAPS:
    {
      GstCaps *caps;

      gst_event_parse_caps (event, &caps);
      ret = gst_my_filter_setcaps (filter, caps);
      break;
    }
    default:
      ret = gst_pad_event_default (pad, parent, event);
      break;
  }
  return ret;
}

  [...]

      

14.2.3. Dynamic negotiation

A last negotiation method is the most complex and powerful dynamic negotiation.

Like with the transform negotiation in Section 14.2.2, dynamic negotiation will perform a transformation on the downstream/upstream caps. Unlike the transform negotiation, this transform will convert fixed caps to unfixed caps. This means that the sink pad input caps can be converted into unfixed (multiple) formats. The source pad will have to choose a format from all the possibilities. It would usually like to choose a format that requires the least amount of effort to produce but it does not have to be. The selection of the format should also depend on the caps that can be accepted downstream (see a QUERY_CAPS function in Implementing a CAPS query function).

A typical flow goes like this:

Examples of this type of elements include:

Let's look at the example of an element that can convert between samplerates, so where input and output samplerate don't have to be the same:


static gboolean
gst_my_filter_setcaps (GstMyFilter *filter,
		       GstCaps *caps)
{
  if (gst_pad_set_caps (filter->sinkpad, caps)) {
    filter->passthrough = TRUE;
  } else {
    GstCaps *othercaps, *newcaps;
    GstStructure *s = gst_caps_get_structure (caps, 0), *others;

    /* no passthrough, setup internal conversion */
    gst_structure_get_int (s, "channels", &filter->channels);
    othercaps = gst_pad_get_allowed_caps (filter->srcpad);
    others = gst_caps_get_structure (othercaps, 0);
    gst_structure_set (others,
      "channels", G_TYPE_INT, filter->channels, NULL);

    /* now, the samplerate value can optionally have multiple values, so
     * we "fixate" it, which means that one fixed value is chosen */
    newcaps = gst_caps_copy_nth (othercaps, 0);
    gst_caps_unref (othercaps);
    gst_pad_fixate_caps (filter->srcpad, newcaps);
    if (!gst_pad_set_caps (filter->srcpad, newcaps))
      return FALSE;

    /* we are now set up, configure internally */
    filter->passthrough = FALSE;
    gst_structure_get_int (s, "rate", &filter->from_samplerate);
    others = gst_caps_get_structure (newcaps, 0);
    gst_structure_get_int (others, "rate", &filter->to_samplerate);
  }

  return TRUE;
}

static gboolean
gst_my_filter_sink_event (GstPad    *pad,
		          GstObject *parent,
		          GstEvent  *event)
{
  gboolean ret;
  GstMyFilter *filter = GST_MY_FILTER (parent);

  switch (GST_EVENT_TYPE (event)) {
    case GST_EVENT_CAPS:
    {
      GstCaps *caps;

      gst_event_parse_caps (event, &caps);
      ret = gst_my_filter_setcaps (filter, caps);
      break;
    }
    default:
      ret = gst_pad_event_default (pad, parent, event);
      break;
  }
  return ret;
}

static GstFlowReturn
gst_my_filter_chain (GstPad    *pad,
		     GstObject *parent,
		     GstBuffer *buf)
{
  GstMyFilter *filter = GST_MY_FILTER (parent);
  GstBuffer *out;

  /* push on if in passthrough mode */
  if (filter->passthrough)
    return gst_pad_push (filter->srcpad, buf);

  /* convert, push */
  out = gst_my_filter_convert (filter, buf);
  gst_buffer_unref (buf);

  return gst_pad_push (filter->srcpad, out);
}