Software Overview
Palette™ is SiMa’s flagship software platform that supports the entire product development cycle and the complete AI/ML journey beginning from the developer’s chosen ML model, sourced from any framework, all the way to deployment of the end-to-end accelerated application. Once installed on the host development machine, Palette software runs seamlessly inside of a Docker container providing access to all the APIs, frameworks, libraries and CLI tools to build, deploy, and debug complex AI/ML applications on SiMa.ai’s MLSoC. This section provides an overview of Palette and all of its components.
Note
Palette™, and all its components, run fully on a host development machine within the Docker container (including all compilers mentioned in this section).
Palette Components
ModelSDK
The ModelSDK enables developers to load, quantize, test and compile previously trained models from many ML frameworks such as PyTorch, Tenserflow, and ONNX to target SiMa.ai’s Machine Learning Accelerator (MLA) at runtime.
Palette CLI / MPK
The MPK toolchain is used by the Palette CLI (Command Line Interface) engine to allow users to build, deploy, and manage applications on the SiMa devices of all form factors. SiMa.ai allows developers to package and deploy applications in a bundle called an MLSoC Application Package (MPK). An MPK file is a zip file of a packaged application with the extension .mpk that contains all resources and information about how to deploy an application on SiMa.ai’s MLSoC.
An application comprises various elements such as executable code, runtime graphs, and images, requiring multiple resources for launch. These components are packaged and deployed to the the MLSoC, enabling the installation and execution of the application as necessary.
In addition to application deployment, the MPK CLI tools can also perform device management, application management, system upgrades, and debug.
GStreamer Plugin Zoo
The developers are focused on creating complex applications that can perform in real-time and provide high efficiency using GStreamer on SiMa.ai’s MLSoC. To make development simpler using GStreamer, SiMa.ai provides a set of pre-written plugins that can access all hardware IPs on the platform and other commonly used functionality that developers can use out-of-the-box or can modify and re-compile to meet their needs.
MLA Compiler
The Palette software’s ModelSDK includes SiMa’s MLA Compiler which will optimize and target the MLA for any loaded and quantized neural network.
Debugging and Logging
Palette™ includes tools that facilitate debugging and logging from the host machine. In addition to having gdb debugging capabilities, Palette also includes a debug component that allows remote debugging of deployed applications running on the MLSoC. You can also pull or stream logs from the MLSoC to the host machine running the Palette software.
Arm Cross-Compiler
Palette™ includes the full cross compilation toolchain within the Docker Engine container in order for developers to have the ability to cross-compile from their x86 based systems and target the Arm A65 on the MLSoC.
Development Flow
A typical machine learning application development flow consists of the following high-level steps:
Development Step |
Palette Component(s) |
|---|---|
Palette Setup |
Palette install package and Docker |
Setup MLSoC Developer Board |
Palette and MLSoC Developer Board |
Run model loading and compilation examples |
ModelSDK, MLA Compiler (included inside the ModelSDK) and Palette examples |
Load, quantize, test and compile custom neural network model from ML framework (PyTorch, TF, …) |
ModelSDK, MLA Compiler (included inside the ModelSDK) |
Testing model on board |
Accelerator mode and example scripts |
Begin application development using GStreamer by running examples |
Palette GStreamer examples, MPK tools, Arm Cross-Compiler |
Begin application development |
MPK tools, Arm Cross-Compiler, GStreamer Plugin Zoo |
Deploy, debug and manage application |
MPK tools, Arm Cross-Compiler, GStreamer Plugin Zoo, Debugging and Logging |
Debug on device (not through Palette tools) |
GDB and PDB through ssh |