afe.backends.mpk.defines

Classes

AFEMPKData	Provides data that is needed for reconstructing the MPK JSON file.
TensorTypeMPKData	Provides the information about a tensor type.
InOutNodesMPKData	Provides the information about a node.
ModelInputMPKData	Provides the information about a Model input node.
ConfigParamsMPKData	Provides the configuration parameters for the plugin.
EV74ConfigParamsMPKData	Configuration parameters for EV74 plugin.
APUConfigParamsMPKData	Configuration parameters for APU plugin.
MLAConfigParamsMPKData	Configuration parameters for MLA plugin.
PluginInputNodeMPKData	Provides the information of the input node for plugin.
PluginResourcesMPKData	Provides information about resources used by plugin.
PluginMPKData	Provides data containing all information needed to generate MPK JSON data for a single plugin.
AwesomeNetMPKData	Provides data containing all information needed to generate the MPK JSON data for a single model.

Module Contents

class afe.backends.mpk.defines.AFEMPKData

Provides data that is needed for reconstructing the MPK JSON file. The data is convertible to JSON format. All changes to these data structures must adhere to changes in MPK file specification.

class afe.backends.mpk.defines.TensorTypeMPKData

Provides the information about a tensor type.

Parameters:

• scalar – The tensor’s scalar element type, as a string that numpy can interpret as a type.

• shape – The tensor’s shape.

scalar: str

shape: List[int]

class afe.backends.mpk.defines.InOutNodesMPKData

Provides the information about a node.

Parameters:

• name – Node name which uniquely identifies the node over the scope of one model file.

• type – Data type. From AFE’s perspective, the type of all nodes is “buffer”.

• size – Node’s data size in bytes.

name: str

type: str = 'buffer'

size: int = 0

class afe.backends.mpk.defines.ModelInputMPKData

Provides the information about a Model input node.

Parameters:

• name – Node name which uniquely identifies the node over the scope of one model file.

• type – Data type. From AFE’s perspective, the type of all nodes is “buffer”.

• size – Node’s data size in bytes.

• input_range – Input range of floating point tensors.

name: str

type: str = 'buffer'

size: int = 0

input_range: List[float] | None = None

class afe.backends.mpk.defines.ConfigParamsMPKData

Provides the configuration parameters for the plugin. Specification depends on the plugin’s processor type.

class afe.backends.mpk.defines.EV74ConfigParamsMPKData

Configuration parameters for EV74 plugin.

Parameters:

• desired_batch_size – Batch size requested by user.

• actual_batch_size – Batch size used in code generation.

• kernel – EV74 function.

• params – Parameters of the EV74 function.

desired_batch_size: int

actual_batch_size: int

kernel: str

params: Dict[str, Any]

class afe.backends.mpk.defines.APUConfigParamsMPKData

Configuration parameters for APU plugin. :param input_names: Names of the parameters of the APU code’s entry point function. These

names may be used when passing parameters at runtime. Must have the same length as input_types.

Parameters:

• input_types – Shapes of input tensors, as they should be interpreted by the APU code.

• output_types – Shapes of output tensors, as they should be interpreted by the APU code.

input_names: List[str]

input_types: List[TensorTypeMPKData]

output_types: List[TensorTypeMPKData]

class afe.backends.mpk.defines.MLAConfigParamsMPKData

Configuration parameters for MLA plugin.

Parameters:

• desired_batch_size – Batch size requested by user.

• actual_batch_size – Batch size used in code generation.

• number_of_quads_to_user – Number of quads used in MLA Production Compiler. Must be 4, other values might be supported in the future.

desired_batch_size: int = 1

actual_batch_size: int = 1

number_of_quads_to_user: int = 4

class afe.backends.mpk.defines.PluginInputNodeMPKData

Provides the information of the input node for plugin.

Parameters:

• name – Node name which uniquely identifies the node over the scope of one model file.

• size – Size of the node’s data in bytes.

name: str

size: int

class afe.backends.mpk.defines.PluginResourcesMPKData

Provides information about resources used by plugin.

Parameters:

executable – Name of the generated executable file. The file format depends on which backend the resource belongs to.

executable: str

class afe.backends.mpk.defines.PluginMPKData

Provides data containing all information needed to generate MPK JSON data for a single plugin.

Parameters:

• name – Plugin name.

• sequence – Plugin’s position in the model’s execution sequence.

• processor – Execution processor for the plugin. It can be either “MLA”, “EV74” or “A65”.

• config_params – Configuration parameters for the plugin.

• input_nodes – JSON objects providing information on the inputs for this plugin.

• output_nodes – JSON objects providing information on the outputs for this plugin.

• type – Plugin type. From the AFE’s perspective, it always has the value “sgpProcess”.

• resources – JSON object providing information about resources used by this plugin.

name: str

sequence: int

processor: str

config_params: ConfigParamsMPKData

input_nodes: List[PluginInputNodeMPKData]

output_nodes: List[InOutNodesMPKData]

type: str = 'sgpProcess'

resources: PluginResourcesMPKData | None = None

class afe.backends.mpk.defines.AwesomeNetMPKData

Provides data containing all information needed to generate the MPK JSON data for a single model.

Parameters:

• name – Model name.

• sequence – Model’s position in an execution sequence.

• input_nodes – An array of JSON objects providing information on the pipeline’s input nodes.

Param:

plugins: An array of JSON objects, each of which define an invocation of a plugin either on MLA, EV74 or A65.

name: str

sequence: int = 1

model_sdk_version: str

input_nodes: List[ModelInputMPKData] = []

plugins: List[PluginMPKData] = []

get_actual_batch_size() → int | None

Returns actual batch size from plugins data. Returns None if all plugins are assigned to A65.

Display warnings if plugins have different actual batch sizes.

get_plugins_backend_distribution() → Dict

Returns number of plugins assigned to MLA, EV74 and A65.