.. _Palette Software Overview:

Software Overview
#################

|var_palette_sw_diagram|

|var_palette_tm| is SiMa’s flagship software platform that supports the entire product 
development cycle and the complete AI/ML journey beginning from the developer's chosen ML model,   
sourced from any framework, all the way to deployment of the end-to-end accelerated 
application. Once installed on the host development machine, Palette software runs seamlessly 
inside of a Docker container providing access to all the APIs, frameworks, libraries and CLI tools 
to build, deploy, and debug complex AI/ML applications on SiMa.ai's MLSoC. This section provides 
an overview of Palette and all of its components.

.. note:: 
   |var_palette_tm|, and all its components, run fully on a host development machine within the
   Docker container (inlcuding all compilers mentioned in this section).

Palette Components
******************

ModelSDK
---------
The |var_modelsdk| enables developers to load, quantize, test and compile previously trained models 
from many ML frameworks such as PyTorch, Tenserflow, and ONNX to target SiMa.ai's Machine Learning 
Accelerator (MLA) at runtime.

Palette CLI / MPK
-----------------
The MPK toolchain is used by the Palette CLI (Command Line Interface) engine to allow users to build,
deploy, and manage applications on the SiMa devices of all form factors. SiMa.ai allows developers to package and deploy applications in a bundle called an *MLSoC
Application Package* (MPK). An MPK file is a zip file of a packaged application with the extension
`.mpk` that contains all resources and information about how to deploy an application on SiMa.ai's
MLSoC.

An application comprises various elements such as executable code, runtime graphs, and images, 
requiring multiple resources for launch. These components are packaged and deployed to the 
the MLSoC, enabling the installation and execution of the application as necessary.

In addition to application deployment, the MPK CLI tools can also perform device management,
application management, system upgrades, and debug.

GStreamer PluginZoo
-------------------
The developers are focused on creating complex applications that can perform in real-time and provide
high efficiency using GStreamer on SiMa.ai's MLSoC. To make development simpler using GStreamer,
SiMa.ai provides a set of pre-written plugins that can access all hardware IPs on the platform and 
other commonly used functionality that developers can use out-of-the-box or can modify and 
re-compile to meet their needs.

MLA Compiler
------------
The Palette software's |var_modelsdk| includes SiMa's |var_ml_compiler| which will optimize and target the MLA 
for any loaded and quantized neural network.

Debugging and Logging
---------------------
|var_palette_tm| includes tools that facilitate debugging and logging from the host machine. In
addition to having gdb debugging capabilities, Palette also includes a debug component that allows
remote debugging of deployed applications running on the MLSoC. You can also pull or stream logs 
from the MLSoC to the host machine running the Palette software. 

Arm Cross-Compiler
------------------
|var_palette_tm| includes the full cross compilation toolchain within the Docker Engine container in order
for developers to have the ability to cross-compile from their x86 based systems and target the Arm
A65 on the MLSoC.

Development Flow
****************

A typical machine learning application development flow consists of the following high-level steps:

.. list-table:: Development Flow with Palette Software
   :header-rows: 1
   :align: center

   * - **Development Step**
     - **Palette Component(s)**
   * - Palette Setup
     - Palette install package and Docker
   * - Setup |var_mlsoc_dev_kit|
     - Palette and |var_mlsoc_dev_kit|
   * - Run model loading and compilation examples
     - |var_modelsdk|, |var_ml_compiler| (included inside the |var_modelsdk|) and Palette examples
   * - Load, quantize, test and compile custom neural network model from ML framework 
       (PyTorch, TF, ...)
     - |var_modelsdk|, |var_ml_compiler| (included inside the |var_modelsdk|)
   * - Testing model on board
     - Accelerator mode and example scripts
   * - Begin application development using GStreamer by running examples
     - Palette GStreamer examples, |var_mpk_tools|, |var_arm_cross_compiler|
   * - Begin application development
     - |var_mpk_tools|, |var_arm_cross_compiler|, |var_gstreamer_zoo|
   * - Deploy, debug and manage application
     - |var_mpk_tools|, |var_arm_cross_compiler|, |var_gstreamer_zoo|, |var_debug_and_logs|
   * - Debug on device (not through Palette tools)
     - GDB and PDB through ssh

..
    Parameters and resources

.. |var_mlsoc_dev_kit| replace:: MLSoC Developer Board
.. |var_palette_tm| replace:: Palette™
.. |var_modelsdk| replace:: ModelSDK
.. |var_ml_compiler| replace:: MLA Compiler
.. |var_gstreamer_zoo| replace:: GStreamer Plugin Zoo
.. |var_mpk_tools| replace:: MPK tools
.. |var_arm_cross_compiler| replace:: Arm Cross-Compiler
.. |var_debug_and_logs| replace:: Debugging and Logging

.. |var_palette_sw_diagram| image:: media/palette_sw_overview.png
    :alt: Palette SW components
    :align: middle