afe.driver.passes ================= .. py:module:: afe.driver.passes .. autoapi-nested-parse:: Compiler passes. These functions wrap IR transforms into compiler passes to be called by driver or API functions. Driver code should construct a compilation pipeline in CompileStep, then run it. Attributes ---------- .. autoapisummary:: afe.driver.passes.InputDict afe.driver.passes.MIXED_PRECISION_SEARCH_LIMIT Classes ------- .. autoapisummary:: afe.driver.passes.BinarySearchState Functions --------- .. autoapisummary:: afe.driver.passes.import_model afe.driver.passes.tvm_transformations afe.driver.passes.import_and_transform afe.driver.passes.update_quantization_configs afe.driver.passes.calibration afe.driver.passes.quantization afe.driver.passes.equalization afe.driver.passes.calibration_quantization afe.driver.passes.evaluation afe.driver.passes.binary_search afe.driver.passes.noise_analysis afe.driver.passes.noise_based_mixed_precision_quantization afe.driver.passes.dump_diagnostic_files afe.driver.passes.dump_diagnostic_files_after Module Contents --------------- .. py:data:: InputDict .. py:data:: MIXED_PRECISION_SEARCH_LIMIT :value: 20 .. py:function:: import_model(config: afe.load.importers.general_importer.ImporterParams) -> afe.driver.compile_step.CompileStep[Tuple[afe._tvm._defines.TVMIRModule, Optional[List[str]]]] Create a compiler pass to import a model :param config: configuration for import of a model :return: A compiler step to import a model. It returns the imported TVM module and the module's output names. Output names are only included if the source model has output names. .. py:function:: tvm_transformations(*, layout: Optional[str] = 'NCHW', index_to_backend_dict: Optional[Dict[int, afe.backends.Backend]] = None, is_quantized: bool = False, name: str, framework: Optional[str] = None) -> Callable[[afe._tvm._defines.TVMIRModule], afe.driver.compile_step.CompileStep[afe._tvm._defines.TVMIRModule]] Create a compiler pass to run TVM transformations on a model. The TVM transformations include ConvertLayout. :param layout: Data layout of activation tensors in the input model. :param index_to_backend_dict: Assignment of nodes to backends. Assignments given here override the partitioning algorithm's decision. :param is_quantized: Whether the input is quantized. If quantized, partitioning will decide whether a given operator can execute on MLA. If not quantized, it will decide whether a given operator can be quantized and then execute on MLA. :param name: Name of the model. :return: Compiler pass that transforms Relay IR. .. py:function:: import_and_transform(config: afe.load.importers.general_importer.ImporterParams, *, name: str, index_to_backend_dict: Optional[Dict[int, afe.backends.Backend]] = None, is_quantized: bool = False) -> afe.driver.compile_step.CompileStep[afe._tvm._defines.TVMIRModule] Create a compiler pass to import and run TVM transformations on a model. See import_model and tvm_transformations for parameter documentation. The returned compile step does the same processing as the old load_*_model functions. .. py:function:: update_quantization_configs(quantization_config: afe.core.configs.QuantizationConfigs, *, custom_quantization_configs: Optional[Dict[afe.ir.net.NodeName, Dict[str, Any]]] = None) -> Callable[[afe.ir.net.AwesomeNet], afe.driver.compile_step.CompileStep[afe.ir.net.AwesomeNet]] Create a compiler pass that records quantization parameters on each node in a network. The compiler pass modifies the network. :param quantization_config: Global configuration parameters for quantization. These will be inserted on all nodes, but will not override previously inserted parameters. :param custom_quantization_configs: Dictionary to override quantization settings for specific nodes. This parameter may only be used in tests. Where custom_quantization_configs[node_name][field_name] = value, it will set the given node's given QuantizationConfigs field to the given value. For example, passing the value {"MLA_1/conv2d_add_84": {"output_int32": True}} will override the configuration of the node named "MLA_1/conv2d_add_84" by setting its output_int32 field to True. :return: Compiler pass to update parameters. The pass mutates and returns its input. .. py:function:: calibration(calibration_config: afe.core.configs.CalibrationConfigs) -> Callable[[afe.ir.net.AwesomeNet, Iterable[InputDict]], afe.driver.compile_step.CompileStep[afe.ir.net.AwesomeNet]] Create a compiler pass to calibrate a network. :param calibration_config: Configuration for calibration. :return: A compiler pass to calibrate a network. The pass mutates and returns its input. .. py:function:: quantization(input_dataset: Optional[Iterable[InputDict]]) -> Callable[[afe.ir.net.AwesomeNet], afe.driver.compile_step.CompileStep[afe.ir.net.AwesomeNet]] Create a compiler pass to quantize a network. Quantization configuration is set by UpdateQuantizationConfigs transform the calibration pass. :return: A compiler pass to quantize a network. The pass mutates and returns its input. .. py:function:: equalization(calibration_config: afe.core.configs.CalibrationConfigs, quantization_config: afe.core.configs.QuantizationConfigs) -> Callable[[afe.ir.net.AwesomeNet, Iterable[InputDict]], afe.driver.compile_step.CompileStep[afe.ir.net.AwesomeNet]] Run SmoothQuant and/or channel equalization if they are enabled in config. This pass should run before quantization. .. py:function:: calibration_quantization(config: afe.core.configs.OptimizationConfigs, *, custom_quantization_configs: Optional[Dict[afe.ir.net.NodeName, Dict[str, Any]]] = None) -> Callable[[afe.ir.net.AwesomeNet, Iterable[InputDict]], afe.driver.compile_step.CompileStep[afe.ir.net.AwesomeNet]] Run calibration and quantization. Quantization-related optimizations that can run at the same time are included here. :param config: Parameters for calibration and quantization. :param custom_quantization_configs: Dictionary to override quantization settings for specific nodes. This parameter may only be used in tests. :return: A compiler pass to calibrate and quantize a network. The pass mutates and returns its input. .. py:function:: evaluation(criterion: afe.driver.statistic.Statistic[Tuple[List[numpy.ndarray], afe.apis.compilation_job_base.GroundTruth], _A], *, fast_mode: bool = False) -> Callable[[afe.ir.net.AwesomeNet, Iterable[Tuple[afe.apis.defines.InputValues, afe.apis.compilation_job_base.GroundTruth]]], afe.driver.compile_step.CompileStep[_A]] Execute a model on an input set and compute an aggregate result from the model's outputs. The primary use case of this function is to estimate a model's accuracy. In this case, criterion computes an accuracy metric for the model over the data set. :param criterion: Function to compute on the model's output and auxiliary data. :param fast_mode: Whether to execute in fast mode. :return: A compiler pass that takes an AwesomeNet and data source, runs the model, and returns the function's result. .. py:class:: BinarySearchState State of binary search. :param lo: Low bound of search range. This is the highest index that was found to not satisfy the search condition. :param hi: High bound of search range. This is the lowest index that was found to satisfy the search condition. :param hi_value: Value associated with high bound. This will be returned if the high value is selected when the search finishes. :param iteration: Iteration of search, starting from zero. Used for deciding to stop early. .. py:attribute:: lo :type: int .. py:attribute:: hi :type: int .. py:attribute:: hi_value :type: _A .. py:attribute:: iteration :type: int .. py:function:: binary_search(get_result: Callable[[int], afe.driver.compile_step.CompileStep[tuple[bool, _A]]], lo: int, hi: int, limit: int, *, procedure_name: Optional[str] = None) -> afe.driver.compile_step.CompileStep[Optional[_A]] Do a binary search for the smallest integer n in the range [lo, hi] such that get_result(n) returns True in its bool result. Return the second result of get_result for the best n that was found, or None if every call returned False. The search always tests lo and hi, so if one of these returns True then the search will find a satisfactory n. The search assumes that get_result is monotonic, that is, there's some n such that get_result(i) returns False for all i <= n and returns True for all i > n. If it is not monotonic, it may not find the optimal n. :param get_result: How to evaluate the search at a given value of n. When it runs, it returns a success flag and caller-specific data. :param lo: Lowest value of n to evaluate :param hi: Highest value of n to evaluate :param limit: Maximum number of binary search steps to perform :param procedure_name: Name of the procedure to be printed in progress messages to the console. If None, do not print progress messages. :return: .. py:function:: noise_analysis() -> Callable[[afe.ir.net.AwesomeNet, afe.ir.net.AwesomeNet, Iterable[InputDict]], afe.driver.compile_step.CompileStep[Dict[afe.core.graph_analyzer.utils.Metric, afe.core.graph_analyzer.graph_analyzer.AnalyzedResultDict]]] Analyze noise that is introduced by quantization. :return: A compiler pass to analyze noise. The pass takes as parameters an un-quantized net, a quantized net derived from it, and the evaluation input. It executes both nets on the evaluation inputs and compares the values at each layer to estimate quantization noise. It returns the analysis results. .. py:function:: noise_based_mixed_precision_quantization(config: afe.core.configs.OptimizationConfigs, criterion: afe.driver.statistic.Statistic[Tuple[List[numpy.ndarray], afe.apis.compilation_job_base.GroundTruth], float], *, target_accuracy: float, custom_quantization_configs: Optional[Dict[afe.ir.net.NodeName, Dict[str, Any]]] = None, max_iterations: int = MIXED_PRECISION_SEARCH_LIMIT, fast_mode: bool = True) -> Callable[[afe.ir.net.AwesomeNet, Iterable[InputDict], Iterable[InputDict], Iterable[tuple[afe.apis.defines.InputValues, afe.apis.compilation_job_base.GroundTruth]]], afe.driver.compile_step.CompileStep[afe.ir.net.AwesomeNet]] Do mixed-precision quantization using noise analysis to choose precision. It will first quantize the model with int8 precision and measure its quantization noise on the analysis dataset. Then it will try to minimize the number of int16 nodes that achieve the target accuracy. It raises an exception if it cannot achieve the target accuracy. A node achieves the target accuracy if evaluating the node using 'criterion' returns a number that is at least 'target_accuracy'. :param config: Parameters for calibration and quantization. The quantization precision must be int8. :param criterion: Method of evaluating accuracy on a data set. :param target_accuracy: Desired accuracy of network. :param custom_quantization_configs: Dictionary to override quantization settings for specific nodes. This parameter may only be used in tests. :param max_iterations: Maximum number of binary search steps to perform. :param fast_mode: Whether to use fast mode when executing the network. :return: A compiler pass that does mixed-precision quantization on a floating-point model, calibration dataset, analysis dataset, and evaluation dataset. .. py:function:: dump_diagnostic_files(model_config: afe.core.configs.ModelConfigs, opt_config: afe.core.configs.OptimizationConfigs, *, prefix: str = '', suffix: str = '') -> Callable[[afe.ir.net.AwesomeNet], afe.driver.compile_step.CompileStep[None]] Save intermediate compilation results to files for diagnostic purposes. The following files are saved in model_config.output_directory. The model is saved to {prefix}{model_config.name}{suffix}.yaml and {prefix}{model_config.name}{suffix}npz. The configuration parameters are saved to {model_config.model_name}.yaml. :param model_config: Model testing configuration :param opt_config: Optimization parameters :param prefix: Prefix to attach to filenames :param suffix: Suffix to attach to filenames :return: A compiler pass that dumps diagnostic files .. py:function:: dump_diagnostic_files_after(step: afe.driver.compile_step.CompileStep[afe.ir.net.AwesomeNet], model_config: afe.core.configs.ModelConfigs, opt_config: afe.core.configs.OptimizationConfigs, *, condition: bool = True, prefix: str = '', suffix: str = '') -> afe.driver.compile_step.CompileStep[afe.ir.net.AwesomeNet] Run a compiler pass, then save intermediate compilation results to files for diagnostic purposes. See dump_diagnostic_files for details. :param step: Compilation step to run before dumping results :param model_config: Model testing configuration :param opt_config: Optimization parameters :param prefix: Prefix to attach to filenames :param suffix: Suffix to attach to filenames :param condition: Whether to dump files. If False, only the compilation step runs. :return: Compilation step extended to dump files at the end