GStreamer Application

The GStreamer Application (aka GST App) in the SiMA SDK is a reference multimedia application designed to demonstrate and validate the integration of the GStreamer framework with the underlying SiMA System-on-Chip (SoC) hardware. It serves as a practical example for building and deploying GStreamer-based media pipelines that leverage hardware-accelerated components provided by the SDK.

The GST App typically showcases a range of capabilities including camera capture, video encoding/decoding, image scaling, display rendering, and real-time streaming, all implemented using GStreamer pipelines. It utilizes SiMA-optimized GStreamer plugins to interface directly with hardware blocks such as video decoders, encoders, image signal processors (ISPs), and AI accelerators, ensuring efficient and low-latency processing.

Developers can use the GST App to test various pipeline configurations, evaluate performance, and customize pipelines for target applications such as surveillance, machine vision, or embedded AI. The app is often modular, configurable via command-line options or configuration files, and supports both live and file-based media input/output.

Features

  • Ability to parse GStreamer strings and build pipelines.

  • Ability to start and stop a pipeline.

  • Support for all messages from a pipeline to be posted to the GStreamer Bus. Additionally, the application reads appropriate message of interest from the GStreamer Bus and decides to take further action, if needed.

  • Ability to communicate through the MQTT Broker within the device.

    Messages handled by GST App

    • GST_MESSAGE_EOS

    • GST_MESSAGE_ERROR

    • GST_MESSAGE_CUSTOM_SIMA_EVENT

    • GST_MESSAGE_APPLICATION

Components

  • Pipeline Builder - Builds a programmatic representation of a gstreamer pipeline using the gst-string.

    Uses the API, gst_parse_launch.

  • MQTT interface - Subscribes and publishes to MQTT broker topics. This enables the GST App to both send and receive both data and control messages. The interface uses the mosquitto.h library.

  • GStreamer Bus Listener - Continuously listen to the GST Bus to read and parse messages.

    Uses the API, gst_bus_timed_pop_filtered.

  • Message Parser - Parses the incoming custom message to retrieve the KPIs.

  • Manifest Parser - A json parser that parses the manifest.json file to retrieve details such as pipeline_id, plugins, configs, and so on.

  • KPI Synchronization - A caching mechanism that collates KPI information for each frame_id.

Usage

The GST App executable expects 2 command-line parameters while running; gst-string, and manifest-json.

/path/to/gst_app --manifest-json="/json/path" --gst-string="appsrc ! appsink"