.. _peppi-pcie-tutorial: PCIe Tutorial ============= This tutorial walks you through using the **PCIe Pipeline** to enable high-performance video processing over the PCIe interface. You will learn how to run real-time object detection with YOLOv7 models on the SiMa.ai MLSoC platform. .. note:: This section covers building PCIe pipelines with PePPi (Python). **Prerequisites:** - Make sure ``sima_hos_pcie_pkg`` is installed on host for proper communication - Follow :ref:`PCIe setup instructions ` to configure PCIe on the host machine - PCIe pipeline is not supported on Modalix Early Access Kit and Modalix DevKit as they don't have PCIe interface Purpose ------- This application is designed to: - Ingest video input over PCIe from a host or connected capture device - Run YOLOv7-based object detection on each frame - Annotate frames with bounding boxes and labels - Stream the output back to the host or a connected interface (e.g., display, recording module) This setup targets integration and benchmarking scenarios for PCIe-connected systems. Configuration Overview --------------------- The runtime parameters are defined in ``project.yaml``. The following tables provide a detailed breakdown of the configuration. Input Configuration ^^^^^^^^^^^^^^^^^^ .. list-table:: :widths: 20 50 30 :header-rows: 1 :class: narrow-table * - Parameter - Description - Example * - source.name - Input type used - ``"pcie"`` * - source.value - Identifier string for PCIe input - ``"PCIE"`` * - udp_host - Not used for PCIe setup - ``""`` * - port - Not used for PCIe setup - ``""`` * - pipeline - Inference pipeline identifier - ``"YoloV7-Pcie"`` Model Configuration ^^^^^^^^^^^^^^^^^^ .. list-table:: :widths: 25 50 25 :header-rows: 1 :class: narrow-table * - Parameter - Description - Value * - name - Model identifier - ``"yolov7"`` * - targz - Path to the YOLOv7 model archive - ``""`` * - label_file - Path to the label file - ``"labels.txt"`` * - normalize - Enable input normalization - ``true`` * - channel_mean - Per-channel mean values - ``[0.0, 0.0, 0.0]`` * - channel_stddev - Per-channel stddev values - ``[1.0, 1.0, 1.0]`` * - padding_type - Padding strategy during preprocessing - ``"CENTER"`` * - aspect_ratio - Preserve input aspect ratio - ``true`` * - topk - Maximum number of detections per frame - ``10`` * - detection_threshold - Confidence threshold for object detection - ``0.7`` * - decode_type - Postprocessing decode strategy - ``"yolo"`` Project Configuration -------------------- project.yaml ^^^^^^^^^^^ .. code-block:: yaml :class: code-narrow :linenos: source: name: "pcie" value: "PCIE" udp_host: "" port: "" pipeline: "YoloV7-Pcie" Models: - name: "yolov7" targz: "" label_file: "labels.txt" normalize: true channel_mean: [0.0, 0.0, 0.0] channel_stddev: [1.0, 1.0, 1.0] padding_type: "CENTER" aspect_ratio: true topk: 10 detection_threshold: 0.7 decode_type: "yolo" Script Behavior -------------- The main script performs the following: 1. Loads the configuration from ``project.yaml`` 2. Initializes the PCIe ``VideoReader`` and ``VideoWriter`` 3. Sets the expected input and output resolution to **1280×720** 4. Loads and configures the YOLOv7 model session on SiMa's MLSoC 5. Continuously: - Reads frames from PCIe input - Runs detection inference - Annotates the output frame - Writes the result using PCIe streaming with metadata passthrough .. warning:: Make sure the PCIe input source delivers frames at **1280x720 resolution**, or update the script accordingly. Model Details ------------ .. list-table:: :widths: 30 70 :header-rows: 1 :class: narrow-table * - Property - Details * - Model File - Tiny YoloV7 optimzied for SiMa devices * - Model Type - YOLOv7 Tiny * - Normalization - Yes (mean: ``[0.0, 0.0, 0.0]``, stddev: ``[1.0, 1.0, 1.0]``) * - Detection Threshold - 0.7 * - Output - Top 10 detections per frame * - Decoding Strategy - YOLO Download Model -------------- MLSOC (Gen1) Download .. button-link:: https://docs.sima.ai/pkg_downloads/SDK1.7.0/SDK1.7.0/model_zoo/davinci/yolov7-tiny-opt_mpk.tar.gz :color: primary :shadow: Download MLSOC Model MOdalix (Gen2) Download .. button-link:: https://docs.sima.ai/pkg_downloads/SDK1.7.0/model_zoo/modalix/yolov7-tiny-opt_mpk.tar.gz :color: primary :shadow: Download Modalix Model Host Pipeline Setup ------------------ Run the Host side pipeline with the following command: .. code-block:: console :class: code-narrow GST_DEBUG=3 gst-launch-1.0 -v rtspsrc location= ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw,format=NV12 ! pciehost queuedepth=30 fps=30 showfps=1 card-number=1 ! queue ! videoparse format=nv12 width=1280 height=720 ! videoconvert ! autovideosink Build and Deploy ---------------- Follow the standard PePPi build and deploy process: #. **Prerequisite:** .. code-block:: console :class: code-narrow 1. Make sure the device is connected via SDK for deployment of project to device. Please follow the setup device section and follow the same. sima-user@docker-image-id:/home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie$ mpk device connect -t root@192.168.135.104 ℹ Connecting to root@192.168.135.104... 🔗 Connection established to 192.168.135.104 . ℹ Fetching Device Plugin Version data file from : 192.168.135.104 ... ✔ Successfully fetched and updated Plugin Version Data file from : 192.168.135.104. #. **Create project directory:** .. code-block:: console :class: code-narrow sima-user@docker-image-id:/home/docker/sima-cli/workspace$ mkdir pcie_pipeline sima-user@docker-image-id:/home/docker/sima-cli/workspace$ cd pcie_pipeline/ #. **Copy the PCIe pipeline from SDK to workspace project directory:** .. code-block:: console :class: code-narrow sima-user@docker-image-id:/home/docker/sima-cli/workspace/pcie_pipeline$ cp -r /usr/local/simaai/app_zoo/Peppi/YoloV7_pcie/ . sima-user@docker-image-id:/home/docker/sima-cli/workspace/pcie_pipeline$ cd YoloV7_pcie/ sima-user@docker-image-id:/home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie$ ls labels.txt main.py project.yaml README.md #. **Copy downloaded model tar.gz file to project directory:** .. code-block:: console :class: code-narrow sima-user@docker-image-id:/home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie$ cp path/to/yolov7-tiny-opt_mpk.tar.gz . sima-user@docker-image-id:/home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie$ ls labels.txt main.py project.yaml README.md yolov7-tiny-opt_mpk.tar.gz #. **Update Config parameters and Yaml:** .. code-block:: console :class: code-narrow source: name: "pcie" value: "PCIE" udp_host: "" port: "" pipeline: "YoloV7-Pcie" Models: - name: "yolov7" targz: "yolov7-tiny-opt_mpk.tar.gz" label_file: labels.txt normalize: true channel_mean: [0.0, 0.0, 0.0] channel_stddev: [1.0, 1.0, 1.0] padding_type: "CENTER" aspect_ratio: true topk: 10 detection_threshold: 0.7 decode_type: "yolo" #. **Create the pipeline package:** .. code-block:: console :class: code-narrow sima-user@docker-image-id:/home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie$ mpk create --peppi -s . -d . --main-file=main.py --yaml-file=project.yaml ℹ Generating requirements.txt file... ✔ Generated requirements.txt. ℹ Dowloading required packages... ✔ Dowloaded required packages. ℹ Building Rpm... ✔ Rpm built successfully. ℹ Creating mpk file... ✔ Mpk file created successfully at /home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie/project.mpk #. **Deploy the pipeline package:** .. code-block:: console :class: code-narrow sima-user@docker-image-id:/home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie$ mpk deploy -f project.mpk ℹ Checking if App YoloV7-Pcie Plugin Version Index File /home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie/YoloV7-Pcie_plugin_version_index.json exists... ❗ App YoloV7-Pcie : File doesn't exist at /home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie/YoloV7-Pcie_plugin_version_index.json. Please check the path/file. ❔ Proceed by Skipping Plugin Version Check? [y/n]: y ‼ User chose to proceed with App YoloV7-Pcie deployment, Skipping Plugin Version Check! Performance could be sub-optimal! 🚀 Sending MPK to 192.168.135.104... Transfer Progress for project.mpk: 100.00% 🏁 MPK sent successfully! Installing MPK... ━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 25% After Deployment - STATUS --> sima-user@docker-image-id:/home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie$ mpk deploy -f project.mpk ℹ Checking if App YoloV7-Pcie Plugin Version Index File /home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie/YoloV7-Pcie_plugin_version_index.json exists... ❗ App YoloV7-Pcie : File doesn't exist at /home/docker/sima-cli/workspace/pcie_pipeline/YoloV7_pcie/YoloV7-Pcie_plugin_version_index.json. Please check the path/file. ❔ Proceed by Skipping Plugin Version Check? [y/n]: y ‼ User chose to proceed with App YoloV7-Pcie deployment, Skipping Plugin Version Check! Performance could be sub-optimal! 🚀 Sending MPK to 192.168.135.104... Transfer Progress for project.mpk: 100.00% 🏁 MPK sent successfully! ✔ MPK Deployed! ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% ✔ MPK Deployment is successful for project.mpk. #. **Visualize the results:** .. code-block:: console :class: code-narrow On Host machine below commands need to be run - GST_DEBUG=3 gst-launch-1.0 -v rtspsrc location= ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw,format=NV12 ! pciehost queuedepth=30 fps=30 showfps=1 card-number=1 ! queue ! videoparse format=nv12 width=1280 height=720 ! videoconvert ! autovideosink ** card-number parameter in pciehost plugin needs to be set accordingly. 0 if only 1 card is available ese "i" based on the card to process the input frames/data. ** #. **To create a rtsp stream - Use the below commands:** .. code-block:: console :class: code-narrow 1. docker run --rm -it --network=host bluenviron/mediamt 2. ffmpeg -re -stream_loop -1 -i .mp4 -c copy -f rtsp rtsp://:8554/mystream .. toctree:: :maxdepth: 2 :caption: PCIe Pipeline