simaaiprocesscvu

This section describes the GStreamer plugin named simaaiprocesscvu.

Brief Description

This plugin is used to run a graph on the CVU cores. This plugin has to be used for all CVU (EV74) operations in the pipeline. The user must not change the plugin source code in any way. Only configuration changes are needed to start using this plugin.

Under the hood, this plugin sends a request for processing input data to the specific EV74 graph/kernel. See the list of available EV74 graphs (ADD LINK) that can be used by the pipeline when incorporating this plugin.

Config JSON Parameters

Description

The Computer Vision Unit (CVU) on SiMa.ai’s MLSoC consists of a four-core Synopsys ARC EV74 video processor supporting up to 600 16-bit GOPS. The MLSoC Firmware comes with a library of optimized graphs accessible via Python or GStreamer plugins, enabling complex signal processing tasks in your application.

The library of graphs available to your application depends on the FW version you are running.

simaaiprocesscvu plugin is responsible for executing graphs available in the CVU Graphs library provided by SiMa.ai.

Sample Config JSON

The overall structure for the config json for simaaiprocesscvu is similar. But the parameters differ based on the graph being executed. Example config json for SIMA_GENERIC_PREPROC is as follows.

{
  "version": 0.1,
  "node_name": "ev-gen-preproc",
  "simaai__params": {
      "params": 15,
      "index": 0,
      "dump_data": 1,
      "cpu": 1,
      "next_cpu": 2,
      "graph_id": 200,
      "no_of_outbuf": 2,
      "ibufname": "allegrodec",
      "out_sz": 1572864,
      "img_height": 720,
      "img_width": 1280,
      "debug": 0,
      "tile_width": 32,
      "tile_height": 86,
      "input_width": 1280,
      "input_height": 720,
      "output_width": 512,
      "output_height": 512,
      "scaled_width": 512,
      "scaled_height": 288,
      "batch_size": 1,
      "normalize": 0,
      "rgb_interleaved": 1,
      "aspect_ratio": 1,
      "input_depth": 3,
      "output_depth": 3,
      "quant_scale": 53.59502780503762,
      "quant_zp": -14,
      "mean_r": 0.485,
      "mean_g": 0.456,
      "mean_b": 0.406,
      "std_dev_r": 0.229,
      "std_dev_g": 0.224,
      "std_dev_b": 0.225,
      "input_type": 0,
      "scaling_type": 1,
      "output_type": 0,
      "padding_type": 0,
      "offset": 786432
  }
}

Example Usage

# Run depended application
/path/to/depended/app/ev_cfg_your_app_name -r /path/to/config/folder/ -p preprocess.json

# Run pipeline
gst-launch-1.0 --gst-plugin-path=/path/to/folder/with/simaaiprocesscvu/library simaaisrc location=input_image.out node-name="allegrodec" mem-target=1 ! simaaiprocesscvu source-node-name="allegrodec" buffers-list="allegrodec" config="/path/to/config.json" ! fakesink

! simaaiprocesscvu config="/path/to/config/json/process_generic_preproc.json" buffers-list="simaaidecoder1" source-node-name="simaaidecoder1" !

In the given example, the simaaiprocesscvu is taking in the input from simaaidecoder1.

source-node-name: Plugin (node) name which output is used as an input for simaaiprocesscvu
buffers-list: name of input buffers used by simaaiprocesscvu

`gst-inspect-1.0` Output

Plugin Details:
Name                     simaaiprocesscvu
Description              GStreamer SiMa.ai Process CVU Plugin
simaaiprocesscvu: SiMa.ai simaaiprocesscvu element
Filename                 libgstsimaaiprocesscvu.so
Version                  1.18.16
License                  LGPL
Source module            gst-plugins-sima
Binary package           GStreamer SiMa.ai Topk2 Plug-in
Origin URL               https://bitbucket.org/sima-ai/gst-plugins-sima

simaaiprocesscvu: SiMa.AI Process Cvu Plugin

1 features:
+-- 1 elements

Note

Please refer to simaaiprocesscvu’s README file for more information.