afe.backends.mpk.defines ======================== .. py:module:: afe.backends.mpk.defines Classes ------- .. autoapisummary:: afe.backends.mpk.defines.AFEMPKData afe.backends.mpk.defines.TensorTypeMPKData afe.backends.mpk.defines.InOutNodesMPKData afe.backends.mpk.defines.ModelInputMPKData afe.backends.mpk.defines.ConfigParamsMPKData afe.backends.mpk.defines.EV74ConfigParamsMPKData afe.backends.mpk.defines.APUConfigParamsMPKData afe.backends.mpk.defines.MLAConfigParamsMPKData afe.backends.mpk.defines.PluginInputNodeMPKData afe.backends.mpk.defines.PluginResourcesMPKData afe.backends.mpk.defines.PluginMPKData afe.backends.mpk.defines.AwesomeNetMPKData Module Contents --------------- .. py:class:: AFEMPKData Provides data that is needed for reconstructing the MPK JSON file. The data is convertible to JSON format. All changes to these data structures must adhere to changes in MPK file specification. .. py:class:: TensorTypeMPKData Provides the information about a tensor type. .. attribute:: scalar The tensor's scalar element type, as a string that numpy can interpret as a type. .. attribute:: shape The tensor's shape. .. py:attribute:: scalar :type: str .. py:attribute:: shape :type: list[int] .. py:class:: InOutNodesMPKData Provides the information about a node. .. attribute:: name Node name which uniquely identifies the node over the scope of one model file. .. attribute:: type Data type. From AFE's perspective, the type of all nodes is "buffer". .. attribute:: size Node's data size in bytes. .. py:attribute:: name :type: str .. py:attribute:: type :type: str :value: 'buffer' .. py:attribute:: size :type: int :value: 0 .. py:class:: ModelInputMPKData Provides the information about a Model input node. .. attribute:: name Node name which uniquely identifies the node over the scope of one model file. .. attribute:: type Data type. From AFE's perspective, the type of all nodes is "buffer". .. attribute:: size Node's data size in bytes. .. attribute:: input_range Input range of floating point tensors. .. py:attribute:: name :type: str .. py:attribute:: type :type: str :value: 'buffer' .. py:attribute:: size :type: int :value: 0 .. py:attribute:: input_range :type: list[float] | None :value: None .. py:class:: ConfigParamsMPKData Provides the configuration parameters for the plugin. Specification depends on the plugin's processor type. .. py:class:: EV74ConfigParamsMPKData Configuration parameters for EV74 plugin. .. attribute:: desired_batch_size Batch size requested by user. .. attribute:: actual_batch_size Batch size used in code generation. .. attribute:: kernel EV74 function. .. attribute:: params Parameters of the EV74 function. .. py:attribute:: desired_batch_size :type: int .. py:attribute:: actual_batch_size :type: int .. py:attribute:: kernel :type: str .. py:attribute:: params :type: dict[str, Any] .. py:class:: APUConfigParamsMPKData Configuration parameters for APU plugin. .. attribute:: input_names Names of the parameters of the APU code's entry point function. These names may be used when passing parameters at runtime. Must have the same length as input_types. .. attribute:: input_types Shapes of input tensors, as they should be interpreted by the APU code. .. attribute:: output_types Shapes of output tensors, as they should be interpreted by the APU code. .. py:attribute:: input_names :type: list[str] .. py:attribute:: input_types :type: list[TensorTypeMPKData] .. py:attribute:: output_types :type: list[TensorTypeMPKData] .. py:class:: MLAConfigParamsMPKData Configuration parameters for MLA plugin. .. attribute:: desired_batch_size Batch size requested by user. .. attribute:: actual_batch_size Batch size used in code generation. .. attribute:: number_of_quads_to_user Number of quads used in MLA Production Compiler. Must be 4, other values might be supported in the future. .. py:attribute:: desired_batch_size :type: int :value: 1 .. py:attribute:: actual_batch_size :type: int :value: 1 .. py:attribute:: number_of_quads_to_user :type: int :value: 4 .. py:class:: PluginInputNodeMPKData Provides the information of the input node for plugin. .. attribute:: name Node name which uniquely identifies the node over the scope of one model file. .. attribute:: size Size of the node's data in bytes. .. py:attribute:: name :type: str .. py:attribute:: size :type: int .. py:class:: PluginResourcesMPKData Provides information about resources used by plugin. .. attribute:: executable Name of the generated executable file. The file format depends on .. attribute:: which backend the resource belongs to. .. py:attribute:: executable :type: str .. py:class:: PluginMPKData Provides data containing all information needed to generate MPK JSON data for a single plugin. .. attribute:: name Plugin name. .. attribute:: sequence Plugin's position in the model's execution sequence. .. attribute:: processor Execution processor for the plugin. It can be either "MLA", "EV74" or "A65". .. attribute:: config_params Configuration parameters for the plugin. .. attribute:: input_nodes JSON objects providing information on the inputs for this plugin. .. attribute:: output_nodes JSON objects providing information on the outputs for this plugin. .. attribute:: type Plugin type. From the AFE's perspective, it always has the value "sgpProcess". .. attribute:: resources JSON object providing information about resources used by this plugin. .. py:attribute:: name :type: str .. py:attribute:: sequence :type: int .. py:attribute:: processor :type: str .. py:attribute:: config_params :type: ConfigParamsMPKData .. py:attribute:: input_nodes :type: list[PluginInputNodeMPKData] .. py:attribute:: output_nodes :type: list[InOutNodesMPKData] .. py:attribute:: type :type: str :value: 'sgpProcess' .. py:attribute:: resources :type: PluginResourcesMPKData | None :value: None .. py:class:: AwesomeNetMPKData Provides data containing all information needed to generate the MPK JSON data for a single model. .. attribute:: name Model name. .. attribute:: sequence Model's position in an execution sequence. .. attribute:: model_path Original model path. .. attribute:: model_checksum Original model checksum. .. attribute:: input_nodes An array of JSON objects providing information on the pipeline's input nodes. .. attribute:: plugins An array of JSON objects, each of which define an invocation of a plugin either on MLA, EV74 or A65. .. py:attribute:: name :type: str .. py:attribute:: model_path :type: str .. py:attribute:: model_checksum :type: str .. py:attribute:: sequence :type: int :value: 1 .. py:attribute:: model_sdk_version :type: str .. py:attribute:: input_nodes :type: list[ModelInputMPKData] :value: [] .. py:attribute:: plugins :type: list[PluginMPKData] :value: [] .. py:method:: get_actual_batch_size() -> int | None Returns actual batch size from plugins data. Returns None if all plugins are assigned to A65. Display warnings if plugins have different actual batch sizes. .. py:method:: get_plugins_backend_distribution() -> dict Returns number of plugins assigned to MLA, EV74 and A65.