Edgematic Features

Machine Learning Models

Edgematic enables model evaluation and AI application development using both SiMa-provided models and user-uploaded models, which may have different licenses and training datasets. Users can upload models from their device or via a pre-signed S3 link, making them available for evaluation within Edgematic.

Edgematic supports two types of models: - .onnx models: These models are quantized and compiled using random data.

Limitation: The compiled model should not be used for application development as it is not optimized for accuracy.

Use Case: Useful for gathering KPI metrics such as FPS and FPS/W.

Pre-compiled .tar.gz SiMa models:
- Recommendation: Before using it in Edgematic, accuracy should be verified to ensure reliable application development.

Note

This video describes how to upload your pre-compiled FP32 model (.onnx) into Edgematic, either from your device or from the S3 link. Once you upload your model it displays on the right panel under User defined models in the system.

GStreamer Pipelines

GStreamer is an open-source multimedia framework that allows the creation of pipelines, which are sequences of processing elements connected to handle media streams such as video and audio. In a GStreamer pipeline, elements like decoders, encoders, filters, and AI inference modules process data in a structured flow. Edgematic simplifies pipeline development by providing a drag-and-drop interface, enabling users to seamlessly construct and modify pipelines without requiring in-depth coding knowledge. Users can integrate pre-built components, custom plugins, and AI models, optimizing data flow and inference efficiency. Edgematic also provides performance metrics such as FPS and FPS/W, helping users evaluate pipeline efficiency before deploying AI applications on edge devices.

Note

This video demonstrates how to run a pipeline and also stop a pipeline.

GStreamer Plugins

Edgematic can use plugins for building an application and generate the necessary business logic, read input, and generate outputs. GStreamer plugins can be of two types, catalog plugins or custom plugins. The catalog plugins are plugins that are general purpose or have been created by SiMa.ai for specific pipelines. The plugins contained in the catalog have been optimized for the pipeline in which they were used.

Note

This video demonstrates how to add plugins to a pipeline, using the drag-and-drop feature, already open in the canvas. It also shows the available plugins for use in pipeline development.

Note

The custom plugins have a section where you have to add your logic into the plugin. In this section you can do any type of logic that you wish and it will be run once per frame coming in.To create a custom plugin right click the name of the project directory in the left panel. Click Create Custom Plugin, add a plugin name and choose the programming language. The options are Python and C++. These plugins are aggregator plugins. Therefore they can have multiple inputs but only a single output. If you want to have multiple outputs just make it part of the same output and if you want to divide those outputs tee the output and parse that input in different plugins.

This video demonstrates how to create a plugin in Python using the <strong>Custom Plugin</strong> option in Edgematic. That is, a user can not only define an existing plugin in Python but also create custom plugins in Python.

Applications

Note

This video describes how to launch an application from the Application Catalog on the right panel of the Edgematic window. Edgematic’s drag-and-drop capability in launching applications leads to greater user experience.

Note

Within the run function you must add your logic to be run in each frame:

plugin_name = "my_custom_plugin"   #define PLUGIN_NAME HERE
out_size = int(1280 * 720 * 1.5)  # outsize of plugin in bytes
class MyPlugin(AggregatorTemplate):
    def __init__(self):
        print(f"Out Size for {plugin_name}: {out_size}")
        super(MyPlugin, self).__init__(plugin_name=plugin_name, out_size=out_size, next_metaparser=False)


def run(self, input_buffers: List[SimaaiPythonBuffer], output_buffer: bytes) -> None:
    # Read model output
    model_output_buffer = np.frombuffer(input_buffers[0].data, dtype=np.float32)
    image = np.frombuffer(input_buffers[1].data, dtype=np.int8).reshape(1280, 720, 3)
    # ...
    # Code Logic ...
    # ...
    # Set model output
    output_buffer[:self.out_size] = np.concatenate([y_.flatten(), uv_.flatten()]).tobytes()

This is an example of a Python code, you can see that to parse the input you must the function np.frombuffer since the input are bytes coming in and specify the type, we recommend you to also shape the input if it has a specific shape. For example 1280, 720, 3 for the image in this case.

For the output, add everything to a numpy array, flatten all the output tensors, and convert them to bytes and assign them to the output buffer. Before doing so, we recommend to verify that the out_size specified before the run function matches the size of the output plugin size.

KPI Measurement

KPI (Key Performance Indicator) measurement in the context of AI model evaluation refers to the process of assessing key metrics that determine the efficiency and performance of a model running on hardware. In Edgematic, KPI measurement includes metrics such as Frames Per Second (FPS), which indicates how many frames the model can process per second, and FPS per Watt (FPS/W), which evaluates energy efficiency by measuring how many frames are processed per unit of power consumption. These KPIs help users compare models, optimize performance, and ensure efficient deployment of AI applications on edge devices.

Note

This video demonstrates how to measure the performance metrics of Frames-per-Second (FPS) for an application which is currently running in the main window of Edgematic.