simaaiboxdecode

The simaaiboxdecode GStreamer plugin processes tensors into valid object detections. It detesselates and dequantizes tensors on the fly, identifies cells of interest, classifies objects, converts box regressions to model-frame coordinates, applies NMS and configurable top-K filtering, and rescales boxes to the original frame size.

Properties

The plugin supports the following properties:

Property	Type	Default	Description
config	string	“/mnt/host/simaaiboxdecode.json”	Path to the JSON configuration file for the simaaiboxdecode.
emit-signals	boolean	FALSE	Send signals.
latency	int64	0	Additional latency in live mode to allow upstream to take longer to produce buffers for the current position (in nanoseconds).
min-upstream-latency	int64	0	Overrides initial minimum latency if dynamically added sources have higher latency. (ns)
name	string	“simaaisimaaiboxdecode0”	The name of the object.
parent	GstObject		The parent of the object.
silent	boolean	TRUE	Produce verbose output.
sima-allocator-type	int	0	1 - no segment API, 2 - segment API support.
start-time	int64	18446744073709551615	Start time to use if start-time-selection=set.
start-time-selection	Enum	0, “zero”	Decides which start time is output.
transmit	boolean	FALSE	Transmit KPI Messages.

Usage

The plugin can be integrated into a GStreamer pipeline as follows:

gst-launch-1.0 simaaisrc location=input.bin ! simaaiboxdecode config=/path/to/config.json ! fakesink

Replace /path/to/config.json with the actual path to your JSON configuration file. input.bin should be replaced with the path to your input file containing the encoded data. The fakesink element can be replaced with another element suitable for processing the decoded data.

Configuration

The configuration is divided into three blocks:

1. GenerixBoxDecode library parameters

   For the full list of parameters, refer to the library README.
2. Caps block

   For a detailed description of the Caps block, refer to the Caps Negotiation Library README.
3. Plugin parameters

   These are common across all aggregator template base plugins:

   • node_name – the name of the current node (used as the output buffer name).

   • memory – defines output memory options:

     • cpu – CPU where this plugin will run (affects only memory allocation).

     • next_cpu – CPU where the next plugin will run (affects only memory allocation).

   • system – defines plugin system settings:

     • out_buf_queue – size of the BufferPool.

     • dump_data – dumps output buffers to /tmp/{name-of-the-object}-{frame_id}.out.

   • buffers – defines input/output buffers:

     • input – an array of objects specifying the input buffer name and size.

     • output – defines the output buffer size.

Actual examples of all 3 blocks can be found in example config.json.

Configuration file example

{
  "version": 0.1,
  "node_name": "simaai_boxdecode",
  "memory": {
    "cpu": 0,
    "next_cpu": 1
  },
  "system": {
    "out_buf_queue": 1,
    "debug": 0,
    "dump_data": 0
  },
  "buffers": {
    "input": [
      {
        "name": "simaai_process_mla",
        "size": 16000
      }
    ],
    "output": {
      "size": 580
    }
  },
  "decode_type" : "detr",
  "topk" : 24,
  "original_width": 1280,
  "original_height": 720,
  "model_width" : 640,
  "model_height" : 480,
  "num_classes" : 92,
  "detection_threshold" : 0.9,
  "nms_iou_threshold" : 0,
  "num_in_tensor": 2,
  "input_width": [
    100,
    100
  ],
  "input_height": [
    1,
    1
  ],
  "input_depth": [
    92,
    4
  ],
  "slice_width": [
    50,
    100
  ],
  "slice_height": [
    1,
    1
  ],
  "slice_depth": [
    92,
    4
  ],
  "dq_scale": [
    6.950103398907683,
    512.0
  ],
  "dq_zp": [
    37,
    -127357
  ],
  "data_type": [
    "INT8",
    "INT32"
  ],
  "caps": {
    "sink_pads": [
      {
        "media_type": "application/vnd.simaai.tensor",
        "params": [
          {
            "name": "format",
            "type": "string",
            "values": "MLA",
            "json_field": null
          },
          {
            "name": "data_type",
            "type": "string",
            "values": "(INT8, INT16, INT32), (INT8, INT16, INT32)",
            "json_field": "data_type"
          },
          {
            "name": "width",
            "type": "int",
            "values": "(1 - 4096), (1 - 4096)",
            "json_field": "input_width"
          },
          {
            "name": "height",
            "type": "int",
            "values": "(1 - 4096), (1 - 4096)",
            "json_field": "input_height"
          },
          {
            "name": "depth",
            "type": "int",
            "values": "(1 - 4096), (1 - 4096)",
            "json_field": "input_depth"
          },
          {
            "name": "slice_width",
            "type": "int",
            "values": "(1 - 4096), (1 - 4096)",
            "json_field": "slice_width"
          },
          {
            "name": "slice_height",
            "type": "int",
            "values": "(1 - 4096), (1 - 4096)",
            "json_field": "slice_height"
          },
          {
            "name": "slice_depth",
            "type": "int",
            "values": "(1 - 4096), (1 - 4096)",
            "json_field": "slice_depth"
          }
        ]
      }
    ],
    "src_pads": [
      {
        "media_type": "application/vnd.simaai.tensor",
        "params": [
          {
            "name": "format",
            "type": "string",
            "values": "BBOX",
            "json_field": null
          }
        ]
      }
    ]
  }
}

Installation

To install the simaaiboxdecode plugin, copy the plugin library file from /usr/local/simaai/plugin_zoo/gst-simaai-plugins-base/gst/ into your project’s plugins/ directory.