Run Demos

Before building your first pipeline, try the canned demo first. SiMa provides two distinct demo experiences:

Edgematic (🔥 Beta) is a state-of-the-art web-based development platform for Edge AI applications. Users can quickly experience the demo by simply dragging and dropping a prebuilt pipeline with a few mouse clicks.
Run a demo pipeline on the DevKit with the help of Palette, using a companion Ubuntu machine to source the video and render the MLA accelerated YoloV7 processing results.

Follow the instructions below to explore our demos. This also serves as a great way to verify that your environment is set up correctly.

Note

Currently Edgematic is in the beta program through a free trial on Amazon Web Services (AWS) Marketplace, to obtain access, click this link. You will need an AWS account in order to sign up.

Log into Edgematic, create a new project called demo, on the right hand side Catalog tab, find yolo_v5_ethernet application under SiMa and drag it into the Canvas. Then hit the play button the top right corner of the page.

If you have access to a DevKit, then running this demo helps you understand the general workflow of things. The following diagram illustrates the demo setup. An Ubuntu machine connects to the DevKit over the network and is configured for RTSP (Real-Time Streaming Protocol) video streaming to and from the DevKit, which handles accelerated ML tasks.

Additionally, you need to install Palette on the host machine to build and deploy the app. Palette can be installed on the same machine running the video streaming processes.

Note

This is a demo for the standalone mode. For more information on how to setup network in the standalone mode, refer to this page.

Step 4. Edit the Demo App GStreamer Pipeline

You need to edit the sample app configuration file application.json located inside the Palette container. You can use either vim or nano to edit the file.

user@palette-container-id:~$ cd /usr/local/simaai/app_zoo/Gstreamer/YoloV7
user@palette-container-id:/usr/local/simaai/app_zoo/Gstreamer/YoloV7$ vim application.json

The gst element in this file defines the gStreamer launch command, inside this command there are 3 variables need to be replaced:

<RTSP_SRC>: Replace to rtsp://<ubuntu_host_ip>:8554/mystream
<HOST_IP>: Replace to the Ubuntu host IP address
<PORT> : Replace with the Ubuntu host port where the third terminal from the previous step indicates the listening port for the video rendering process.

Click to view the explainations of the gst launch command in the application.json file

The gst launch command is used to construct and execute multimedia processing and machine learning pipeline executed in SiMa MLA. It enables real-time streaming, decoding, processing, and rendering of data streams using various GStreamer plugins. Common use cases include playing media files, capturing video from a camera, streaming RTSP sources, and applying machine learning inference on video frames. By chaining different elements, gst launch allows developers to create flexible and efficient multimedia applications.

"gst": "rtspsrc location=rtsp://<HOST_IP>:8554/mystream !\
        rtph264depay wait-for-keyframe=true ! h264parse ! 'video/x-h264, parsed=true, stream-format=(string)byte-stream, alignment=(string)au, width=(int)[1,4096], height=(int)[1,4096]' !\
        simaaidecoder sima-allocator-type=2 name='decoder' next-element='CVU' !\
        tee name=source ! 'video/x-raw' !\
        simaaiprocesscvu_new name=simaai_preprocess num-buffers=5 !\
        simaaiprocessmla_new name=simaai_process_mla num-buffers=5 !\
        simaaiprocesscvu_new name=simaai_postprocess num-buffers=5 !\
        nmsyolov5_new name=simaai_nms_yolov5 orig-img-width=1280 orig-img-height=720 ! \
        overlay. source. ! 'video/x-raw' ! \
        simaai-overlay2_new name=overlay render-info='input::decoder,bboxy::simaai_nms_yolov5' labels-file='/data/simaai/applications/YoloV7/share/overlay_new/labels.txt' !\
        simaaiencoder enc-bitrate=4000 name=encoder !\
        h264parse !\
        rtph264pay !\
        udpsink host=<HOST_IP> port=PORT"

RTSP Source:

rtspsrc location=rtsp://<HOST_IP>:8554/mystream: Retrieves the RTSP video stream from the Ubuntu host.

H.264 Stream Handling:

rtph264depay wait-for-keyframe=true: Depayloads the H.264 stream while ensuring it starts with a keyframe.
h264parse: Parses the H.264 stream.
'video/x-h264, parsed=true, stream-format=(string)byte-stream, alignment=(string)au, width=(int)[1,4096], height=(int)[1,4096]': Specifies video format constraints.

Decoding:

simaaidecoder sima-allocator-type=2 name='decoder' next-element='CVU': Uses SiMa AI’s hardware decoder and allocates memory.

Processing & Object Detection:

tee name=source: Duplicates the video stream for parallel processing.
simaaiprocesscvu_new name=simaai_preprocess num-buffers=5: Prepares frames for processing.
simaaiprocessmla_new name=simaai_process_mla num-buffers=5: Runs inference using the Machine Learning Accelerator (MLA).
simaaiprocesscvu_new name=simaai_postprocess num-buffers=5: Post-processes the model output.
nmsyolov5_new name=simaai_nms_yolov5 orig-img-width=1280 orig-img-height=720: Applies Non-Maximum Suppression (NMS) for YOLOv5 object detection.

Overlaying Bounding Boxes:

overlay. source. ! 'video/x-raw': Merges the original and processed video streams.
simaai-overlay2_new name=overlay render-info='input::decoder,bboxy::simaai_nms_yolov5' labels-file='/data/simaai/applications/YoloV7_Eth/share/overlay_new/labels.txt':
- Overlays detected objects on the video.
- Uses the provided labels file for object classification.

Encoding & Transmission:

simaaiencoder enc-bitrate=4000 name=encoder: Encodes the processed video stream.
h264parse: Parses the encoded H.264 stream.
rtph264pay: Packs the stream into RTP format.
udpsink host=<HOST_IP> port=<PORT>: Sends the processed video stream over UDP to the specified host.