Overview of Hardware and Software

Overview of SiMa.ai’s MLSoC

In this section, we will cover an overview of the MLSoC HW components, and how they are accessed through SiMa.ai’s software stack. We will focus on the main components needed for this exercise - more advanced concepts will be covered in the advanced section.

../../../_images/mlsoc_diagram.png

For more information on the MLSoC, please refer to the MLSoC product brief here: https://sima.ai/mlsoc/

Overview of SiMa.ai’s application SW stack

../../../_images/simplified_sw_stack.jpg

Palette provides various GStreamer plugins found in GStreamer Plugins to allow developers to construct a GStreamer pipeline that can access the various heterogenous compute engines on the MLSoC (HW encode/decode, DSP, MLA, etc.). In this guide we will learn how to use the plugins in order to build an application that optimally utilizes the MLSoC. We will also learn how to develop custom plugins.

Conclusion and next steps

In this section, we:

  • Learned about the various IPs that the MLSoC has to offer.

  • Learned about the SiMa.ai GStreamer SW application stack at a high level.

Next let’s take a look at a typical application, and how it may map to the MLSoC given our new hardware and software understanding.