afe.core.graph_manager ====================== .. py:module:: afe.core.graph_manager Attributes ---------- .. autoapisummary:: afe.core.graph_manager.IR_MODEL_FILE_SUFFIX afe.core.graph_manager.FP32_AWESOMENET_SUFFIX Functions --------- .. autoapisummary:: afe.core.graph_manager.load_ir_module_from_file afe.core.graph_manager.load_awesome_net_from_file afe.core.graph_manager.load_quantized_awesome_net_from_file afe.core.graph_manager.transform_ir_module afe.core.graph_manager.run_quantization_aware_partition_selector afe.core.graph_manager.partition_ir_module afe.core.graph_manager.translate_irmod_to_sima_ir afe.core.graph_manager.transform_and_partition_ir_module afe.core.graph_manager.transform_irmod_to_awesomenet afe.core.graph_manager.run_end_to_end_processing Module Contents --------------- .. py:data:: IR_MODEL_FILE_SUFFIX :type: str :value: '_ir_model.json' .. py:data:: FP32_AWESOMENET_SUFFIX :type: str :value: '_fp32_awesomenet' .. py:function:: load_ir_module_from_file(model_configs: afe.core.configs.ModelConfigs) -> afe._tvm._defines.TVMIRModule Loads the Relay IRModule from file. The file path and name are derived from model_configs argument. :param model_configs: ModelConfigs. Specifies the file path and the model name. :return: tvm_def.TVMIRModule. A Relay IRModule loaded from file. .. py:function:: load_awesome_net_from_file(model_configs: afe.core.configs.ModelConfigs) -> afe.ir.net.AwesomeNet Loads the AwesomeNet from file. The file path and name are derived from model_configs argument. :param model_configs: ModelConfigs. Specifies the file path and the model name. :return: AwesomeNet. An AwesomeNet loaded from file. .. py:function:: load_quantized_awesome_net_from_file(model_configs: afe.core.configs.ModelConfigs) -> afe.ir.net.AwesomeNet Loads the quantized AwesomeNet from file. The file path and name are derived from model_configs argument. :param model_configs: ModelConfigs. Specifies the file path and the model name. :return: AwesomeNet. A quantized AwesomeNet loaded from file. .. py:function:: transform_ir_module(mod: afe._tvm._defines.TVMIRModule, transforms: List[tvm.transform.Pass]) -> afe._tvm._defines.TVMIRModule Transforms the Relay IRModule using the list of TVM Relay transforms. Note that the partitioning transformation step is done separately, due to a possibility that quantization-aware partitioning might be run beforehand. :param mod: tvm_def.TVMIRModule. A Relay IRModule that is being transformed. :param transforms: List[tvm.transform.Pass]. Defines the list of transform passes that will be applied by transformer. :return: tvm_def.TVMIRModule. Transformed Relay IRModule. .. py:function:: run_quantization_aware_partition_selector(mod: afe._tvm._defines.TVMIRModule, configs: afe.core.configs.AfeProcessingConfigs, calibration_generator: sima_utils.data.data_generator.DataGenerator, graph_evaluator: afe.core.evaluate_networks.GraphEvaluator) -> Dict[int, afe.backends.Backend] Runs the quantization-aware partitioning of the input Relay IRModule. Selects the nodes that should be run in higher precision and updates the TransformerConfig's indices_to_backend_dict. Steps in QAP are as follows: 1. Map the expressions in TVM Relay IRModule to targeted backend. 2. Translate the input Relay IRModule to AwesomeNet taking into account the expressions to backend mapping. Expressions mapped to non-MLA backend are translated to AwesomeNodes consisting of ExternalOps. 3. Analyze the performance of floating-point AwesomeNet for reference. 4. Calibrate the network. 5. Execute loop which quantizes the network, analyzes its performance and, if the performance is not sufficient, finds the node with the highest quantization error and fixes it to floating-point. 6. Updates the indices_to_backend_dict with list of nodes fixed to floating point from step 5. :param mod: Input IRModule :param configs: AfeProcessingConfigs. Contains processing configuration information. :param calibration_generator: DataGenerator. Used to generate data used in calibration. :param graph_evaluator: GraphEvaluator. Used to perform graph evaluation. :return Dict[int, Backend]. Dictionary containing mapping of node indices to target Backend. .. py:function:: partition_ir_module(mod: afe._tvm._defines.TVMIRModule, configs: afe.core.configs.AfeProcessingConfigs, calibration_generator: Optional[sima_utils.data.data_generator.DataGenerator], graph_evaluator: Optional[afe.core.evaluate_networks.GraphEvaluator]) -> afe._tvm._defines.TVMIRModule Partition the Relay IRModule according to the given TransformerConfigs. The resulting Relay IRModule consists of a series of functions that contain subgraphs of operations, each being annotated on any one of the backends. :param mod: tvm_def.TVMIRModule. A Relay IRModule that is being partitioned. :param configs: AfeProcessingConfigs. Contains configuration information controlling the full end-to-end processing flow. :param calibration_generator: Optional[DataGenerator]. A DataGenerator used in calibration step. Needs to be provided if the quantization-aware partitioning is enabled. :param graph_evaluator: Optional[GraphEvaluator]. An object used in graph evaluation. Needs to be provided if quantization-aware partitioning is enabled. :return: tvm_def.TVMIRModule. A partitioned Relay IRModule. .. py:function:: translate_irmod_to_sima_ir(mod: afe._tvm._defines.TVMIRModule, params: afe.tvm_converter.parameters.TVMConverterParams, dump_to_files: bool = False, model_name: str | None = None, output_directory: str | None = None, expr_to_backend_dict: dict[afe._tvm._defines.TVMRelayExpr, list[afe.backends.Backend]] | None = None, *, output_labels: list[str] | None = None, model_path: str | None = None) -> afe.ir.net.AwesomeNet Converts a TVM IRModule to an AwesomeNet in the following steps: 1. Create a Relay expression to name map. The map contains graph metadata that can be used to translate to a different representation 2. Use the Relay expression to name map to create an AwesomeNet :param mod: tvm_def.TVMIRModule. A Relay IRModule being translated. :param params: Parameters that affect how Relay IR is translated to SiMa IR. :param dump_to_files: bool. If set to True, resulting AwesomeNet will be saved to file. :param model_name: Optional[str]. Default is None. Name of the model used to generate file name. Needs to be provided if dump_to_files is set to True. :param output_directory: Optional[str]. Default is None. The path to the directory where files should be stored. Needs to be provided if dump_to_files is set to True. :param expr_to_backend_dict: Optional[Dict[tvm_def.TVMRelayExpr, List[Backend]]]. Default is None. If given, represents mapping of expressions to Backend on which the expression is to be executed. :param output_labels: Names of the network outputs. These names are only used in debugging output. :param model_path: Original model path. :returns: AwesomeNet. A SimaIR representation of a model. .. py:function:: transform_and_partition_ir_module(mod: afe._tvm._defines.TVMIRModule, configs: afe.core.configs.AfeProcessingConfigs, calibration_generator: Optional[sima_utils.data.data_generator.DataGenerator] = None, graph_evaluator: Optional[afe.core.evaluate_networks.GraphEvaluator] = None, dump_to_files: bool = False) -> afe._tvm._defines.TVMIRModule Transforms and partitions the Relay IRModule according to the given TransformerConfigs. :param mod: tvm_def.TVMIRModule. An input Relay IRModule. :param configs: AfeProcessingConfigs. Contains processing configuration information. :param calibration_generator: Optional[DataGenerator]. Default is None. DataGenerator used in calibration step. Needs to be provided if quantization-aware partitioning is enabled. :param graph_evaluator: Optional[GraphEvaluator]. Default is None. An object used in graph evaluation. Needs to be provided if quantization-aware partitioning is enabled. :param dump_to_files: bool. Default is False. If set to True, intermediate and output Relay IRModules will be saved to appropriate files. :return: .. py:function:: transform_irmod_to_awesomenet(mod: afe._tvm._defines.TVMIRModule, configs: afe.core.configs.AfeProcessingConfigs, calibration_generator: sima_utils.data.data_generator.DataGenerator | None = None, graph_evaluator: afe.core.evaluate_networks.GraphEvaluator | None = None, dump_to_files: bool = False, return_irmod: bool = False, *, output_labels: list[str] | None = None, model_path: str | None = None) -> Union[afe.ir.net.AwesomeNet, Tuple[afe.ir.net.AwesomeNet, afe._tvm._defines.TVMIRModule]] Transforms and partitions the Relay IRModule according to the given TransformerConfigs and translates it into a SimaIR AwesomeNet. :param mod: tvm_def.TVMIRModule. An input Relay IRModule. :param configs: AfeProcessingConfigs. Contains processing configuration information. :param calibration_generator: Optional[DataGenerator]. Default is None. DataGenerator used in calibration step. Needs to be provided if quantization-aware partitioning is enabled. :param graph_evaluator: Optional[GraphEvaluator]. Default is None. An object used in graph evaluation. Needs to be provided if quantization-aware partitioning is enabled. :param dump_to_files: bool. Default is False. If set to True, configuration and intermediate model representations (Relay IRModule and AwesomeNets) will be saved to appropriate files. :param return_irmod: bool. Default is False. Whether to return the transformed TVM IRModule. :param output_labels: Names of the network outputs. These names are only used in debugging output. :param model_path: Original model path. :returns: A Tuple consisting of generated AwesomeNet and transformed TVM IRModule if return_irmod parameter is set to True, otherwise only generated AwesomeNet. .. py:function:: run_end_to_end_processing(mod: afe._tvm._defines.TVMIRModule, configs: afe.core.configs.AfeProcessingConfigs, calibration_generator: sima_utils.data.data_generator.DataGenerator, graph_evaluator: Optional[afe.core.evaluate_networks.GraphEvaluator], dump_to_files: bool = False) -> afe.ir.net.AwesomeNet Runs end-to-end processing of a Relay IRModule which consists of following steps: 1. Transform Relay IRModule according to information contained in TransformerConfigs that is a part of AfeProcessingConfigs. 2. Partition Relay IRModule according to information contained in TransformerConfigs that is a part of AfeProcessingConfigs. 3. Translate the Relay IRModule to SimaIR AwesomeNet. 4. Calibrate the SimaIR AwesomeNet. 5. Quantize the SimaIR AwesomeNet. :param mod: tvm_def.TVMIRModule. An input Relay IRModule. :param configs: AfeProcessingConfigs. Contains the configuration information which control the full end-to-end processing flow. :param calibration_generator: DataGenerator. Object used to generate data used in calibration. :param graph_evaluator: Optional[GraphEvaluator]. Default is None. Object used for graph evaluation. Needs to be provided if quantization-aware partitioning is enabled. :param dump_to_files: bool. Default is False. If set to True, all configuration and intermediate graph representations will be written to appropriate files. :return: AwesomeNet. Final result of processing which is partitioned, calibrated and quantized AwesomeNet.