.. _peppi-ethernet-tutorial: Ethernet Tutorial ================= This tutorial demonstrates how to use the **Ethernet Pipeline** for real-time object detection with YOLOv8 models. You will learn how to connect to live RTSP streams and stream results over UDP, enabling edge AI deployments on the SiMa.ai MLSoC platform. .. note:: This section covers building ethernet pipelines with PePPi (Python). **Features:** - Real-time object detection on RTSP streams - YOLOv8 model optimized for edge inference - UDP streaming output for visualization - High-performance edge AI deployment Purpose ------- This application is designed to: - Read video from an RTSP stream using ``rtspsrc`` - Run detection using the YOLOv8 model on SiMa MLSoC - Annotate frames with bounding boxes and class labels - Stream the output frames over UDP for real-time visualization This setup is ideal for evaluating high-performance object detection in edge AI deployments. Configuration Overview --------------------- The application is driven by ``project.yaml``. The parameters below describe its structure. Input/Output Configuration ^^^^^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :widths: 20 50 30 :header-rows: 1 :class: narrow-table * - Parameter - Description - Example * - source.name - Input source type - ``"rtspsrc"`` * - source.value - RTSP stream URL - ``""`` * - udp_host - Destination IP for UDP stream - ``""`` * - port - Destination port for UDP stream - ``""`` * - pipeline - Processing pipeline name - ``"YoloV8"`` Model Configuration ^^^^^^^^^^^^^^^^^^ .. list-table:: :widths: 25 50 25 :header-rows: 1 :class: narrow-table * - Parameter - Description - Value * - name - Model identifier - ``"YOLO"`` * - targz - Compressed model archive path - ``""`` * - label_file - Path to label file - ``"labels.txt"`` * - normalize - Apply input normalization - ``true`` * - channel_mean - Input channel mean values - ``[0.0, 0.0, 0.0]`` * - channel_stddev - Input channel stddev values - ``[1.0, 1.0, 1.0]`` * - padding_type - Padding type during preprocessing - ``"CENTER"`` * - aspect_ratio - Maintain input aspect ratio - ``true`` * - topk - Max number of detections per frame - ``10`` * - detection_threshold - Score threshold for valid detections - ``0.7`` * - nms_iou_threshold - IOU threshold for non-max suppression - ``0.3`` * - decode_type - Detection decoding strategy - ``"yolo"`` * - num_classes - Number of classes the model can detect - ``87`` Project Configuration -------------------- project.yaml ^^^^^^^^^^^ .. code-block:: yaml :class: code-narrow :linenos: source: name: "rtspsrc" value: "" udp_host: "" port: "" pipeline: "YoloV8" Models: - name: "YOLO" targz: "" label_file: "labels.txt" normalize: true channel_mean: [0.0, 0.0, 0.0] channel_stddev: [1.0, 1.0, 1.0] padding_type: "CENTER" aspect_ratio: true topk: 10 detection_threshold: 0.7 nms_iou_threshold: 0.3 decode_type: "yolo" num_classes: 87 Script Behavior -------------- The Python script executes the following steps: 1. Loads ``project.yaml`` 2. Initializes a ``VideoReader`` for RTSP input and a ``VideoWriter`` for UDP output 3. Sets up the YOLOv8 model using a ``MLSoCSession`` configured with SiMa MLSoC 4. In a loop: - Reads an input frame - Optionally dumps it to disk for debugging (``/tmp/nv12.out``) - Runs the model - Renders detection results onto the frame - Streams the annotated frame via UDP .. note:: The pipeline is optimized for real-time processing with minimal latency for edge AI applications. Model Details ------------ .. list-table:: :widths: 30 70 :header-rows: 1 :class: narrow-table * - Property - Details * - Model File - YoloV8 optimzied Model for SiMa devices * - Model Type - YOLOv8 * - Input Format - NV12 * - Normalization - Yes (mean: ``[0.0, 0.0, 0.0]``, stddev: ``[1.0, 1.0, 1.0]``) * - Detection Threshold - 0.7 * - NMS IOU Threshold - 0.3 * - Output - Up to 10 detections per frame * - Classes - 87 object categories Download Model -------------- MLSOC (Gen1) Download .. button-link:: https://docs.sima.ai/pkg_download/SDK1.7.0/SDK1.7.0/model_zoo/davinci/yolov8_mpk.tar.gz :color: primary :shadow: Download MLSOC Model Modalix (Gen2) Download .. button-link:: https://docs.sima.ai/pkg_download/SDK1.7.0/model_zoo/modalix/yolov8_mpk.tar.gz :color: primary :shadow: Download Modalix Model Pipeline Architecture -------------------- .. list-table:: :widths: 20 80 :header-rows: 1 :class: narrow-table * - Stage - Description * - RTSP Input - Receives live video stream from RTSP source * - Frame Processing - Converts and preprocesses frames for model input * - YOLOv8 Inference - Runs object detection on SiMa MLSoC * - Post-processing - Applies NMS and threshold filtering * - Annotation - Renders bounding boxes and labels on frames * - UDP Output - Streams annotated frames for real-time visualization Build and Deploy ---------------- Follow the standard PePPi build and deploy process: #. **Prerequisites:** .. code-block:: console :class: code-narrow 1. Make sure the device is connected via SDK for deployment of project to device. Please follow the setup device section and follow the same. sima-user@docker-image-id:/home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie$ mpk device connect -t root@192.168.135.104 ℹ Connecting to root@192.168.135.104... 🔗 Connection established to 192.168.135.104 . ℹ Fetching Device Plugin Version data file from : 192.168.135.104 ... ✔ Successfully fetched and updated Plugin Version Data file from : 192.168.135.104. #. **Create the project directory:** .. code-block:: console :class: code-narrow sima-user@docker-image-id:/home/docker/sima-cli/workspace$ mkdir ethernet_pipeline sima-user@docker-image-id:/home/docker/sima-cli/workspace$ cd ethernet_pipeline/ #. **Copy the YOLOv8 Pipeline from SDK to project directory:** .. code-block:: console :class: code-narrow sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline$ cp -r /usr/local/simaai/app_zoo/Peppi/YoloV8/ . sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline$ cd YoloV8/ sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8$ ls labels.txt main.py project.yaml README.md #. **Copy downloaded model tar.gz file to project directory:** .. code-block:: console :class: code-narrow sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8$ cp path/to/yolov8_mpk.tar.gz . sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8$ ls labels.txt main.py project.yaml README.md yolov8_mpk.tar.gz #. **Update Config parameters and Yaml:** .. code-block:: console :class: code-narrow source: name: "rtspsrc" value: "rtsp://:8554/mystream" udp_host: "" port: "8005" pipeline: YoloV8 Models: - name: "YOLO" targz: "yolov8_mpk.tar.gz" label_file: labels.txt normalize: true channel_mean: [0.0, 0.0, 0.0] channel_stddev: [1.0, 1.0, 1.0] padding_type: "CENTER" aspect_ratio: true topk: 10 detection_threshold: 0.7 nms_iou_threshold: 0.3 decode_type: "yolo" num_classes: 87 #. **Create the pipeline package:** .. code-block:: console :class: code-narrow sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8$ mpk create --peppi -s . -d . --main-file=main.py --yaml-file=project.yaml ℹ Generating requirements.txt file... ✔ Generated requirements.txt. ℹ Dowloading required packages... ✔ Dowloaded required packages. ℹ Building Rpm... ✔ Rpm built successfully. ℹ Creating mpk file... ✔ Mpk file created successfully at /home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8/project.mpk #. **Deploy the pipeline package:** .. code-block:: console :class: code-narrow sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8$ mpk deploy -f project.mpk ℹ Checking if App YoloV8 Plugin Version Index File /home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8/YoloV8_plugin_version_index.json exists... ❗ App YoloV8 : File doesn't exist at /home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8/YoloV8_plugin_version_index.json. Please check the path/file. ❔ Proceed by Skipping Plugin Version Check? [y/n]: y ‼ User chose to proceed with App YoloV8 deployment, Skipping Plugin Version Check! Performance could be sub-optimal! 🚀 Sending MPK to 192.168.135.104... Transfer Progress for project.mpk: 100.00% 🏁 MPK sent successfully! Installing MPK... ━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 25% After Deployment - STATUS --> sima-user@docker-image-id:/home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8$ mpk deploy -f project.mpk ℹ Checking if App YoloV8 Plugin Version Index File /home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8/YoloV8_plugin_version_index.json exists... ❗ App YoloV8 : File doesn't exist at /home/docker/sima-cli/workspace/ethernet_pipeline/YoloV8/YoloV8_plugin_version_index.json. Please check the path/file. ❔ Proceed by Skipping Plugin Version Check? [y/n]: y ‼ User chose to proceed with App YoloV8 deployment, Skipping Plugin Version Check! Performance could be sub-optimal! 🚀 Sending MPK to 192.168.135.104... Transfer Progress for project.mpk: 100.00% 🏁 MPK sent successfully! ✔ MPK Deployed! ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% ✔ MPK Deployment is successful for project.mpk. #. **Visualize the result:** .. code-block:: console :class: code-narrow On Host machine below commands need to be run - GST_DEBUG=0 gst-launch-1.0 udpsrc port=8005 ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! 'video/x-h264,stream-format=byte-stream,alignment=au' ! avdec_h264 ! fpsdisplaysink ** udpsrc port needs to be configured as per the project.yaml port configuration. ** #. **To create a rtsp stream - Use the below commands :** .. code-block:: console :class: code-narrow 1. docker run --rm -it --network=host bluenviron/mediamt 2. ffmpeg -re -stream_loop -1 -i .mp4 -c copy -f rtsp rtsp://:8554/mystream Performance Considerations ------------------------- .. list-table:: :widths: 30 70 :header-rows: 1 :class: narrow-table * - Aspect - Consideration * - Network Latency - RTSP stream quality and network stability affect input performance * - Model Optimization - YOLOv8 model is optimized for edge inference on SiMa MLSoC * - UDP Streaming - Output streaming performance depends on network bandwidth * - Real-time Processing - Pipeline designed for minimal latency edge AI applications * - Memory Usage - Efficient memory management for continuous stream processing .. warning:: Ensure stable network connectivity for both RTSP input and UDP output streaming. .. toctree:: :maxdepth: 2 :caption: Ethernet Pipeline