afe.backends.mpk.defines

Classes

AFEMPKData	Provides data that is needed for reconstructing the MPK JSON file.
TensorTypeMPKData	Provides the information about a tensor type.
InOutNodesMPKData	Provides the information about a node.
ModelInputMPKData	Provides the information about a Model input node.
ConfigParamsMPKData	Provides the configuration parameters for the plugin.
EV74ConfigParamsMPKData	Configuration parameters for EV74 plugin.
APUConfigParamsMPKData	Configuration parameters for APU plugin.
MLAConfigParamsMPKData	Configuration parameters for MLA plugin.
PluginInputNodeMPKData	Provides the information of the input node for plugin.
PluginResourcesMPKData	Provides information about resources used by plugin.
PluginMPKData	Provides data containing all information needed to generate MPK JSON data for a single plugin.
AwesomeNetMPKData	Provides data containing all information needed to generate the MPK JSON data for a single model.

Module Contents

class afe.backends.mpk.defines.AFEMPKData

Provides data that is needed for reconstructing the MPK JSON file. The data is convertible to JSON format. All changes to these data structures must adhere to changes in MPK file specification.

class afe.backends.mpk.defines.TensorTypeMPKData

Provides the information about a tensor type.

scalar

The tensor’s scalar element type, as a string that numpy can interpret as a type.

shape

The tensor’s shape.

scalar: str

shape: list[int]

class afe.backends.mpk.defines.InOutNodesMPKData

Provides the information about a node.

name

Node name which uniquely identifies the node over the scope of one model file.

type

Data type. From AFE’s perspective, the type of all nodes is “buffer”.

size

Node’s data size in bytes.

name: str

type: str = 'buffer'

size: int = 0

class afe.backends.mpk.defines.ModelInputMPKData

Provides the information about a Model input node.

name

Node name which uniquely identifies the node over the scope of one model file.

type

Data type. From AFE’s perspective, the type of all nodes is “buffer”.

size

Node’s data size in bytes.

input_range

Input range of floating point tensors.

name: str

type: str = 'buffer'

size: int = 0

input_range: list[float] | None = None

class afe.backends.mpk.defines.ConfigParamsMPKData

Provides the configuration parameters for the plugin. Specification depends on the plugin’s processor type.

class afe.backends.mpk.defines.EV74ConfigParamsMPKData

Configuration parameters for EV74 plugin.

desired_batch_size

Batch size requested by user.

actual_batch_size

Batch size used in code generation.

kernel

EV74 function.

params

Parameters of the EV74 function.

desired_batch_size: int

actual_batch_size: int

kernel: str

params: dict[str, Any]

class afe.backends.mpk.defines.APUConfigParamsMPKData

Configuration parameters for APU plugin.

input_names

Names of the parameters of the APU code’s entry point function. These names may be used when passing parameters at runtime. Must have the same length as input_types.

input_types

Shapes of input tensors, as they should be interpreted by the APU code.

output_types

Shapes of output tensors, as they should be interpreted by the APU code.

input_names: list[str]

input_types: list[TensorTypeMPKData]

output_types: list[TensorTypeMPKData]

class afe.backends.mpk.defines.MLAConfigParamsMPKData

Configuration parameters for MLA plugin.

desired_batch_size

Batch size requested by user.

actual_batch_size

Batch size used in code generation.

number_of_quads_to_user

Number of quads used in MLA Production Compiler. Must be 4, other values might be supported in the future.

desired_batch_size: int = 1

actual_batch_size: int = 1

number_of_quads_to_user: int = 4

class afe.backends.mpk.defines.PluginInputNodeMPKData

Provides the information of the input node for plugin.

name

Node name which uniquely identifies the node over the scope of one model file.

size

Size of the node’s data in bytes.

name: str

size: int

class afe.backends.mpk.defines.PluginResourcesMPKData

Provides information about resources used by plugin.

executable

Name of the generated executable file. The file format depends on

which backend the resource belongs to.

executable: str

class afe.backends.mpk.defines.PluginMPKData

Provides data containing all information needed to generate MPK JSON data for a single plugin.

name

Plugin name.

sequence

Plugin’s position in the model’s execution sequence.

processor

Execution processor for the plugin. It can be either “MLA”, “EV74” or “A65”.

config_params

Configuration parameters for the plugin.

input_nodes

JSON objects providing information on the inputs for this plugin.

output_nodes

JSON objects providing information on the outputs for this plugin.

type

Plugin type. From the AFE’s perspective, it always has the value “sgpProcess”.

resources

JSON object providing information about resources used by this plugin.

name: str

sequence: int

processor: str

config_params: ConfigParamsMPKData

input_nodes: list[PluginInputNodeMPKData]

output_nodes: list[InOutNodesMPKData]

type: str = 'sgpProcess'

resources: PluginResourcesMPKData | None = None

class afe.backends.mpk.defines.AwesomeNetMPKData

Provides data containing all information needed to generate the MPK JSON data for a single model.

name

Model name.

sequence

Model’s position in an execution sequence.

model_path

Original model path.

model_checksum

Original model checksum.

input_nodes

An array of JSON objects providing information on the pipeline’s input nodes.

plugins

An array of JSON objects, each of which define an invocation of a plugin either on MLA, EV74 or A65.

name: str

model_path: str

model_checksum: str

sequence: int = 1

model_sdk_version: str

input_nodes: list[ModelInputMPKData] = []

plugins: list[PluginMPKData] = []

get_actual_batch_size() → int | None

Returns actual batch size from plugins data. Returns None if all plugins are assigned to A65.

Display warnings if plugins have different actual batch sizes.

get_plugins_backend_distribution() → dict

Returns number of plugins assigned to MLA, EV74 and A65.