topk

This section describes the GStreamer plugin named topk.

Brief Description

A top-k retrieval algorithm returns the k best answers of a query according to a given ranking. This plugin wraps these algorithms such that it can be used in GStreamer-based pipelines.

/* SliceHeigt & SliceWidth  => configured Tile sizes in the json                                  */
/* FrameHeight & FrameWidth => Input Tensor Height & Width configured n the json                  */
/* TileHeight & TileWidth   => Actul tile sizes when applied on the Frame                         */
/* What is an EVEN and an ODD tile ?                                                              */
/* EVEN, ODD1, ODD2 tiles representation.                                                         */
/* EVEN -> Tiles where TileHeight(TH) and TileWidth(TW) are same as SliceWidth & SliceHeight.     */
/* ODD1 -> Tiles where Tilewidth is lesser than SliceWidth and TileHeight is same as SliceHeight .*/
/* ODD2 -> Tiles where TileHeight is less than OutputTileHeight.                                  */
/* ODD3 -> Tiles where both TileHeight and TileWidth are lesser than SliceHeight & SliceWIdth.    */
/* -----------------------------------------------------------------------------------------------*/
/* | <---------FrameWidth---------->|                                                             */
/* |________________________________|___                                                          */
/* |       |       |       |        |  ^                                                          */
/* | EVEN  | EVEN  | EVEN  |  ODD1  |  |                                                          */
/* |_______|_______|_______|________|  |                                                          */
/* |       |       |       |        |  |                                                          */
/* | EVEN  | EVEN  | EVEN  |  ODD1  | FrameHeight                                                 */
/* |_______|_______|_______|________|  |                                                          */
/* |       |       |       |        |  |                                                          */
/* | ODD2  | ODD2  | ODD2  |  ODD3  |  |                                                          */
/* |_______|_______|_______|________|__v_                                                         */
/*                                                                                                */
/*  ----------------------------------------------------------------------------------------------*/

The topk plugin is implemented as an AGGREGATOR which means it expects few input buffers. Current implementation works only with two input buffers: - model output tensor - heatmaps.

Config JSON Parameters

{
    "version" : 0.1,
    "node_name" : "a65-topk",
    "simaai__params" : {
        "params" : 13, <========================================= NOT USED
        "index" : 4, <=========================================== NOT USED
        "cpu" : 0, <============================================= NOT USED
        "next_cpu" : 1, <======================================== CPU type used for memory allocation. 1 means EV74
        "op_id" : 2, <=========================================== NOT USED
        "orig_img_width" : 1280, <=============================== original image width
        "orig_img_height" : 720, <=============================== original image height
        "frame_height" : 128, <================================== frame height
        "frame_width" : 128, <=================================== frame width
        "heat_map_depth" : 8, <================================== heatmap depth
        "n_request" : 50, <====================================== N of requests, in other words - max num of bounding boxes returned by TopK algorithm
        "inpath" : "/root/centernet_output.raw", <=============== DEFAULT ONE
        "ibufname" : "ev-cnet-postproc", <======================= NOT USED
        "in_sz" : 0, <=========================================== NOT USED
        "out_sz" : 580, <======================================== output buffer size. Calculated as n_request * sizeof(bbox_t) + sizeof(int32_t)
        "no_of_outbuf" : 5, <==================================== num of output buffers created by plugin. Must be greater than 0
        "out_type" : 2, <======================================== NOT USED
        "tile_width" : 16, <===================================== tile height tessellation
        "tile_height" : 11, <==================================== tile height tessellation
        "threshold" : 0.5, <===================================== threshold value
        "qscale" : [ 108.68913716853953, 1.6627093312937011 ], <= Quant scale for regression and width_height
        "qzp" : [ -26, -126 ], <================================= Quant zero-point for regression and width_height
        "debug" : 0, <=========================================== NOT USED
        "dump_data" : 0 <======================================== Dump output buffer into file at /tmp
    }
}

Apart from the configuration file, a user must specify the model details as command line arguments for the plugin.

  1. model-name="centernet"

  2. model-info="heatmap=128:128:16:4,offset=128:128:16:1,size_w_h=128:128:16:1"

    • This parameter encodes model details so that we can extract proper data from the model output.

    • heatmap part specifies the 16-bit Q15, {heatmapDepth} channels.

    • offset part specifies 8-bit integer, 2 channels.

    • size_w_h is not used but still has to be specified.

Example Usage

! topk2_agg config="/path/to/process_topk.json" model-name="centernet" model-info="heatmap=128:128:16:4,offset=128:128:16:1,size_w_h=128:128:16:1" name=topk !

gst-inspect-1.0 Output

 1Plugin Details:
 2Name                     simaaitopk
 3Description              GStreamer SiMa.ai Topk Plugin
 4Filename                 ./libgstsimaaitopk2.so
 5Version                  1.18.16
 6License                  LGPL
 7Source module            gst-plugins-sima
 8Binary package           GStreamer SiMa.ai Topk2 Plug-in
 9Origin URL               https://bitbucket.org/sima-ai/gst-plugins-sima
10
11topk2_agg: SiMa.AI Topk Plugin
12
131 features:
14+-- 1 elements

Note

See the topk’s README file for more information.