afe.core.graph_manager

Attributes

`IR_MODEL_FILE_SUFFIX`
`FP32_AWESOMENET_SUFFIX`

Functions

`load_ir_module_from_file`(→ afe._tvm._defines.TVMIRModule)	Loads the Relay IRModule from file. The file path and name are derived from
`load_awesome_net_from_file`(→ afe.ir.net.AwesomeNet)	Loads the AwesomeNet from file. The file path and name are derived from
`load_quantized_awesome_net_from_file`(...)	Loads the quantized AwesomeNet from file. The file path and name are derived from
`transform_ir_module`(→ afe._tvm._defines.TVMIRModule)	Transforms the Relay IRModule using the list of TVM Relay transforms.
`run_quantization_aware_partition_selector`(→ Dict[int, ...)	Runs the quantization-aware partitioning of the input Relay IRModule.
`partition_ir_module`(→ afe._tvm._defines.TVMIRModule)	Partition the Relay IRModule according to the given TransformerConfigs.
`translate_irmod_to_sima_ir`(→ afe.ir.net.AwesomeNet)	Converts a TVM IRModule to an AwesomeNet in the following steps:
`transform_and_partition_ir_module`(...)	Transforms and partitions the Relay IRModule according to the given TransformerConfigs.
`transform_irmod_to_awesomenet`(...)	Transforms and partitions the Relay IRModule according to the given TransformerConfigs
`run_end_to_end_processing`(→ afe.ir.net.AwesomeNet)	Runs end-to-end processing of a Relay IRModule which consists of following steps:

Module Contents

afe.core.graph_manager.IR_MODEL_FILE_SUFFIX: str = '_ir_model.json'[source]

afe.core.graph_manager.FP32_AWESOMENET_SUFFIX: str = '_fp32_awesomenet'[source]

afe.core.graph_manager.load_ir_module_from_file(model_configs: afe.core.configs.ModelConfigs) → afe._tvm._defines.TVMIRModule[source]

Loads the Relay IRModule from file. The file path and name are derived from model_configs argument.

Parameters:: model_configs – ModelConfigs. Specifies the file path and the model name.
Returns:: tvm_def.TVMIRModule. A Relay IRModule loaded from file.

afe.core.graph_manager.load_awesome_net_from_file(model_configs: afe.core.configs.ModelConfigs) → afe.ir.net.AwesomeNet[source]

Loads the AwesomeNet from file. The file path and name are derived from model_configs argument.

Parameters:: model_configs – ModelConfigs. Specifies the file path and the model name.
Returns:: AwesomeNet. An AwesomeNet loaded from file.

afe.core.graph_manager.load_quantized_awesome_net_from_file(model_configs: afe.core.configs.ModelConfigs) → afe.ir.net.AwesomeNet[source]

Loads the quantized AwesomeNet from file. The file path and name are derived from model_configs argument.

Parameters:: model_configs – ModelConfigs. Specifies the file path and the model name.
Returns:: AwesomeNet. A quantized AwesomeNet loaded from file.

afe.core.graph_manager.transform_ir_module(mod: afe._tvm._defines.TVMIRModule, transforms: List[tvm.transform.Pass]) → afe._tvm._defines.TVMIRModule[source]

Transforms the Relay IRModule using the list of TVM Relay transforms. Note that the partitioning transformation step is done separately, due to a possibility that quantization-aware partitioning might be run beforehand.

Parameters:

mod – tvm_def.TVMIRModule. A Relay IRModule that is being transformed.
transforms – List[tvm.transform.Pass]. Defines the list of transform passes that will be applied by transformer.

Returns:

tvm_def.TVMIRModule. Transformed Relay IRModule.

afe.core.graph_manager.run_quantization_aware_partition_selector(mod: afe._tvm._defines.TVMIRModule, configs: afe.core.configs.AfeProcessingConfigs, calibration_generator: sima_utils.data.data_generator.DataGenerator, graph_evaluator: afe.core.evaluate_networks.GraphEvaluator) → Dict[int, afe.backends.Backend][source]

Runs the quantization-aware partitioning of the input Relay IRModule. Selects the nodes that should be run in higher precision and updates the TransformerConfig’s indices_to_backend_dict.

Steps in QAP are as follows:

Map the expressions in TVM Relay IRModule to targeted backend.

2. Translate the input Relay IRModule to AwesomeNet taking into account the expressions to backend mapping. Expressions mapped to non-MLA backend are translated to AwesomeNodes consisting of ExternalOps. 3. Analyze the performance of floating-point AwesomeNet for reference. 4. Calibrate the network. 5. Execute loop which quantizes the network, analyzes its performance and, if the performance is not sufficient, finds the node with the highest quantization error and fixes it to floating-point. 6. Updates the indices_to_backend_dict with list of nodes fixed to floating point from step 5.

Parameters:

mod – Input IRModule
configs – AfeProcessingConfigs. Contains processing configuration information.
calibration_generator – DataGenerator. Used to generate data used in calibration.
graph_evaluator – GraphEvaluator. Used to perform graph evaluation.

:return Dict[int, Backend]. Dictionary containing mapping of node indices to target Backend.

afe.core.graph_manager.partition_ir_module(mod: afe._tvm._defines.TVMIRModule, configs: afe.core.configs.AfeProcessingConfigs, calibration_generator: sima_utils.data.data_generator.DataGenerator | None, graph_evaluator: afe.core.evaluate_networks.GraphEvaluator | None) → afe._tvm._defines.TVMIRModule[source]

Partition the Relay IRModule according to the given TransformerConfigs. The resulting Relay IRModule consists of a series of functions that contain subgraphs of operations, each being annotated on any one of the backends.

Parameters:

mod – tvm_def.TVMIRModule. A Relay IRModule that is being partitioned.
configs – AfeProcessingConfigs. Contains configuration information controlling the full end-to-end processing flow.
calibration_generator – Optional[DataGenerator]. A DataGenerator used in calibration step. Needs to be provided if the quantization-aware partitioning is enabled.
graph_evaluator – Optional[GraphEvaluator]. An object used in graph evaluation. Needs to be provided if quantization-aware partitioning is enabled.

Returns:

tvm_def.TVMIRModule. A partitioned Relay IRModule.

afe.core.graph_manager.translate_irmod_to_sima_ir(mod: afe._tvm._defines.TVMIRModule, params: afe.tvm_converter.parameters.TVMConverterParams, dump_to_files: bool = False, model_name: str | None = None, output_directory: str | None = None, expr_to_backend_dict: dict[afe._tvm._defines.TVMRelayExpr, list[afe.backends.Backend]] | None = None, *, output_labels: list[str] | None = None, model_path: str | None = None) → afe.ir.net.AwesomeNet[source]

Converts a TVM IRModule to an AwesomeNet in the following steps:

Create a Relay expression to name map. The map contains graph metadata that can be used to translate to a different representation

Use the Relay expression to name map to create an AwesomeNet

Parameters:

mod – tvm_def.TVMIRModule. A Relay IRModule being translated.
params – Parameters that affect how Relay IR is translated to SiMa IR.
dump_to_files – bool. If set to True, resulting AwesomeNet will be saved to file.
model_name – Optional[str]. Default is None. Name of the model used to generate file name. Needs to be provided if dump_to_files is set to True.
output_directory – Optional[str]. Default is None. The path to the directory where files should be stored. Needs to be provided if dump_to_files is set to True.
expr_to_backend_dict – Optional[Dict[tvm_def.TVMRelayExpr, List[Backend]]]. Default is None. If given, represents mapping of expressions to Backend on which the expression is to be executed.
output_labels – Names of the network outputs. These names are only used in debugging output.
model_path – Original model path.

Returns:

AwesomeNet. A SimaIR representation of a model.

afe.core.graph_manager.transform_and_partition_ir_module(mod: afe._tvm._defines.TVMIRModule, configs: afe.core.configs.AfeProcessingConfigs, calibration_generator: sima_utils.data.data_generator.DataGenerator | None = None, graph_evaluator: afe.core.evaluate_networks.GraphEvaluator | None = None, dump_to_files: bool = False) → afe._tvm._defines.TVMIRModule[source]

Transforms and partitions the Relay IRModule according to the given TransformerConfigs.

Parameters:

mod – tvm_def.TVMIRModule. An input Relay IRModule.
configs – AfeProcessingConfigs. Contains processing configuration information.
calibration_generator – Optional[DataGenerator]. Default is None. DataGenerator used in calibration step. Needs to be provided if quantization-aware partitioning is enabled.
graph_evaluator – Optional[GraphEvaluator]. Default is None. An object used in graph evaluation. Needs to be provided if quantization-aware partitioning is enabled.
dump_to_files – bool. Default is False. If set to True, intermediate and output Relay IRModules will be saved to appropriate files.

Returns:

afe.core.graph_manager.transform_irmod_to_awesomenet(mod: afe._tvm._defines.TVMIRModule, configs: afe.core.configs.AfeProcessingConfigs, calibration_generator: sima_utils.data.data_generator.DataGenerator | None = None, graph_evaluator: afe.core.evaluate_networks.GraphEvaluator | None = None, dump_to_files: bool = False, return_irmod: bool = False, *, output_labels: list[str] | None = None, model_path: str | None = None) → afe.ir.net.AwesomeNet | Tuple[afe.ir.net.AwesomeNet, afe._tvm._defines.TVMIRModule][source]

Transforms and partitions the Relay IRModule according to the given TransformerConfigs and translates it into a SimaIR AwesomeNet.

Parameters:

mod – tvm_def.TVMIRModule. An input Relay IRModule.
configs – AfeProcessingConfigs. Contains processing configuration information.
calibration_generator – Optional[DataGenerator]. Default is None. DataGenerator used in calibration step. Needs to be provided if quantization-aware partitioning is enabled.
graph_evaluator – Optional[GraphEvaluator]. Default is None. An object used in graph evaluation. Needs to be provided if quantization-aware partitioning is enabled.
dump_to_files – bool. Default is False. If set to True, configuration and intermediate model representations (Relay IRModule and AwesomeNets) will be saved to appropriate files.
return_irmod – bool. Default is False. Whether to return the transformed TVM IRModule.
output_labels – Names of the network outputs. These names are only used in debugging output.
model_path – Original model path.

Returns:

A Tuple consisting of generated AwesomeNet and transformed TVM IRModule if return_irmod parameter: is set to True, otherwise only generated AwesomeNet.

afe.core.graph_manager.run_end_to_end_processing(mod: afe._tvm._defines.TVMIRModule, configs: afe.core.configs.AfeProcessingConfigs, calibration_generator: sima_utils.data.data_generator.DataGenerator, graph_evaluator: afe.core.evaluate_networks.GraphEvaluator | None, dump_to_files: bool = False) → afe.ir.net.AwesomeNet[source]

Runs end-to-end processing of a Relay IRModule which consists of following steps:

1. Transform Relay IRModule according to information contained in TransformerConfigs that is a part of AfeProcessingConfigs. 2. Partition Relay IRModule according to information contained in TransformerConfigs that is a part of AfeProcessingConfigs. 3. Translate the Relay IRModule to SimaIR AwesomeNet. 4. Calibrate the SimaIR AwesomeNet. 5. Quantize the SimaIR AwesomeNet.

Parameters:

mod – tvm_def.TVMIRModule. An input Relay IRModule.
configs – AfeProcessingConfigs. Contains the configuration information which control the full end-to-end processing flow.
calibration_generator – DataGenerator. Object used to generate data used in calibration.
graph_evaluator – Optional[GraphEvaluator]. Default is None. Object used for graph evaluation. Needs to be provided if quantization-aware partitioning is enabled.
dump_to_files – bool. Default is False. If set to True, all configuration and intermediate graph representations will be written to appropriate files.

Returns:

AwesomeNet. Final result of processing which is partitioned, calibrated and quantized AwesomeNet.