Ethernet Tutorial

This tutorial demonstrates how to use the Ethernet Pipeline for real-time object detection with YOLOv8 models. You will learn how to connect to live RTSP streams and stream results over UDP, enabling edge AI deployments on the SiMa.ai MLSoC platform.

Note

This section covers building ethernet pipelines with PePPi (Python).

Features:

• Real-time object detection on RTSP streams

• YOLOv8 model optimized for edge inference

• UDP streaming output for visualization

• High-performance edge AI deployment

Purpose

This application is designed to:

• Read video from an RTSP stream using rtspsrc

• Run detection using the YOLOv8 model on SiMa MLSoC

• Annotate frames with bounding boxes and class labels

• Stream the output frames over UDP for real-time visualization

This setup is ideal for evaluating high-performance object detection in edge AI deployments.

Configuration Overview

The application is driven by project.yaml. The parameters below describe its structure.

Input/Output Configuration

Parameter	Description	Example
source.name	Input source type	"rtspsrc"
source.value	RTSP stream URL	"<RTSP_URL>"
udp_host	Destination IP for UDP stream	"<HOST_IP>"
port	Destination port for UDP stream	"<PORT_NUM>"
pipeline	Processing pipeline name	"YoloV8"

Model Configuration

Parameter	Description	Value
name	Model identifier	"YOLO"
targz	Compressed model archive path	"<targz_path>"
label_file	Path to label file	"labels.txt"
normalize	Apply input normalization	true
channel_mean	Input channel mean values	[0.0, 0.0, 0.0]
channel_stddev	Input channel stddev values	[1.0, 1.0, 1.0]
padding_type	Padding type during preprocessing	"CENTER"
aspect_ratio	Maintain input aspect ratio	true
topk	Max number of detections per frame	10
detection_threshold	Score threshold for valid detections	0.7
nms_iou_threshold	IOU threshold for non-max suppression	0.3
decode_type	Detection decoding strategy	"yolo"
num_classes	Number of classes the model can detect	87

Project Configuration

project.yaml

source:
  name: "rtspsrc"
  value: "<RTSP_URL>"
udp_host: "<HOST_IP>"
port: "<PORT_NUM>"
pipeline: "YoloV8"

Models:
  - name: "YOLO"
    targz: "<targz_path>"
    label_file: "labels.txt"
    normalize: true
    channel_mean: [0.0, 0.0, 0.0]
    channel_stddev: [1.0, 1.0, 1.0]
    padding_type: "CENTER"
    aspect_ratio: true
    topk: 10
    detection_threshold: 0.7
    nms_iou_threshold: 0.3
    decode_type: "yolo"
    num_classes: 87

Script Behavior

The Python script executes the following steps:

1. Loads project.yaml

2. Initializes a VideoReader for RTSP input and a VideoWriter for UDP output

3. Sets up the YOLOv8 model using a MLSoCSession configured with SiMa MLSoC

4. In a loop: - Reads an input frame - Optionally dumps it to disk for debugging (/tmp/nv12.out) - Runs the model - Renders detection results onto the frame - Streams the annotated frame via UDP

Note

The pipeline is optimized for real-time processing with minimal latency for edge AI applications.

Model Details

Property	Details
Model File	YoloV8 optimzied Model for SiMa devices
Model Type	YOLOv8
Input Format	NV12
Normalization	Yes (mean: [0.0, 0.0, 0.0], stddev: [1.0, 1.0, 1.0])
Detection Threshold	0.7
NMS IOU Threshold	0.3
Output	Up to 10 detections per frame
Classes	87 object categories

Download Model

MLSOC (Gen1) Download

Download MLSOC Model

Modalix (Gen2) Download

Download Modalix Model

Pipeline Architecture

Stage	Description
RTSP Input	Receives live video stream from RTSP source
Frame Processing	Converts and preprocesses frames for model input
YOLOv8 Inference	Runs object detection on SiMa MLSoC
Post-processing	Applies NMS and threshold filtering
Annotation	Renders bounding boxes and labels on frames
UDP Output	Streams annotated frames for real-time visualization

Build and Deploy

Follow the standard PePPi build and deploy process:

1. Prerequisites:

   1. Make sure the device is connected via SDK for deployment of project to device. Please follow the setup device section and follow the same.
   
   sima-user@docker-image-id:/home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie$ mpk device connect -t root@192.168.135.104
   ℹ Connecting to root@192.168.135.104...
   🔗 Connection established to 192.168.135.104 .
   ℹ Fetching Device Plugin Version data file from : 192.168.135.104 ...
   ✔ Successfully fetched and updated Plugin Version Data file from : 192.168.135.104.
   

2. Create the project directory:

   sima-user@docker-image-id:/home/docker/sima-cli/workspace$ mkdir ethernet_pipeline
   sima-user@docker-image-id:/home/docker/sima-cli/workspace$ cd ethernet_pipeline/
   

3. Copy the YOLOv8 Pipeline from SDK to project directory:

   sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline$ cp -r /usr/local/simaai/app_zoo/Peppi/YoloV8/ .
   sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline$ cd YoloV8/
   sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8$ ls
   labels.txt  main.py  project.yaml  README.md
   

4. Copy downloaded model tar.gz file to project directory:

   sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8$ cp path/to/yolov8_mpk.tar.gz .
   sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8$ ls
   labels.txt  main.py  project.yaml  README.md yolov8_mpk.tar.gz
   

5. Update Config parameters and Yaml:

   source:
     name: "rtspsrc"
     value:  "rtsp://<host_ip/rtsp_url>:8554/mystream"
   udp_host: "<host_ip>"
   port: "8005"
   pipeline: YoloV8
   
   Models:
   - name: "YOLO"
     targz: "yolov8_mpk.tar.gz"
     label_file: labels.txt
     normalize: true
     channel_mean: [0.0, 0.0, 0.0]
     channel_stddev: [1.0, 1.0, 1.0]
     padding_type: "CENTER"
     aspect_ratio: true
     topk: 10
     detection_threshold: 0.7
     nms_iou_threshold: 0.3
     decode_type: "yolo"
     num_classes: 87
   

6. Create the pipeline package:

   sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8$ mpk create --peppi -s . -d . --main-file=main.py --yaml-file=project.yaml
   ℹ Generating requirements.txt file...
   ✔ Generated requirements.txt.
   ℹ Dowloading required packages...
   ✔ Dowloaded required packages.
   ℹ Building Rpm...
   ✔ Rpm built successfully.
   ℹ Creating mpk file...
   ✔ Mpk file created successfully at
   /home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8/project.mpk
   

7. Deploy the pipeline package:

   sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8$ mpk deploy -f project.mpk
   ℹ Checking if App YoloV8 Plugin Version Index File
   /home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8/YoloV8_plugin_version_index.json exists...
   ❗ App YoloV8 : File doesn't exist at
   /home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8/YoloV8_plugin_version_index.json. Please
   check the path/file.
   ❔ Proceed by Skipping Plugin Version Check? [y/n]: y
   ‼ User chose to proceed with App YoloV8 deployment, Skipping Plugin Version Check! Performance could
   be sub-optimal!
   🚀 Sending MPK to 192.168.135.104...
   Transfer Progress for project.mpk:  100.00%
   🏁 MPK sent successfully!
   Installing MPK... ━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  25%
   
   After Deployment - STATUS -->
   
   sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8$ mpk deploy -f project.mpk
   ℹ Checking if App YoloV8 Plugin Version Index File
   /home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8/YoloV8_plugin_version_index.json exists...
   ❗ App YoloV8 : File doesn't exist at
   /home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8/YoloV8_plugin_version_index.json. Please
   check the path/file.
   ❔ Proceed by Skipping Plugin Version Check? [y/n]: y
   ‼ User chose to proceed with App YoloV8 deployment, Skipping Plugin Version Check! Performance could
   be sub-optimal!
   🚀 Sending MPK to 192.168.135.104...
   Transfer Progress for project.mpk:  100.00%
   🏁 MPK sent successfully!
   ✔ MPK Deployed! ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
   ✔ MPK Deployment is successful for project.mpk.
   

8. Visualize the result:

   On Host machine below commands need to be run -
   
   GST_DEBUG=0 gst-launch-1.0 udpsrc port=8005 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! 'video/x-h264,stream-format=byte-stream,alignment=au' ! avdec_h264 ! fpsdisplaysink
   
   ** udpsrc port needs to be configured as per the project.yaml port configuration. **
   

9. To create a rtsp stream - Use the below commands :

   1. docker run --rm -it --network=host bluenviron/mediamt
   2. ffmpeg -re -stream_loop -1 -i <input_video_feed>.mp4 -c copy -f rtsp rtsp://<host_ip>:8554/mystream
   

Performance Considerations

Aspect	Consideration
Network Latency	RTSP stream quality and network stability affect input performance
Model Optimization	YOLOv8 model is optimized for edge inference on SiMa MLSoC
UDP Streaming	Output streaming performance depends on network bandwidth
Real-time Processing	Pipeline designed for minimal latency edge AI applications
Memory Usage	Efficient memory management for continuous stream processing

Warning

Ensure stable network connectivity for both RTSP input and UDP output streaming.