afe.ir.defines ============== .. py:module:: afe.ir.defines Attributes ---------- .. autoapisummary:: afe.ir.defines.AwesomeDataLayout afe.ir.defines.AwesomeDataLayout5D afe.ir.defines.AwesomeConvWeightLayout afe.ir.defines.AwesomeConvWeightLayout5D afe.ir.defines.AwesomeDepthwiseConvWeightLayout afe.ir.defines.AwesomeDepthwiseConvWeightLayout5D afe.ir.defines.AwesomeTransposeConvWeightLayout5D afe.ir.defines.NoneType afe.ir.defines.NodeName afe.ir.defines.InputName afe.ir.defines.TensorFormat afe.ir.defines.InputShape afe.ir.defines.ConvPad afe.ir.defines.AwesomePad2D afe.ir.defines.AwesomeStrides2D afe.ir.defines.AwesomeDilation2D afe.ir.defines.AwesomePoolSize2D afe.ir.defines.AwesomePad3D afe.ir.defines.AwesomeStrides3D afe.ir.defines.AwesomeDilation3D afe.ir.defines.AwesomePoolSize3D afe.ir.defines.AwesomePad afe.ir.defines.AwesomeStrides afe.ir.defines.AwesomeDilation afe.ir.defines.AwesomePoolSize afe.ir.defines.Float afe.ir.defines.QuantizedTensor afe.ir.defines.QuantizedTensorNew afe.ir.defines.QuantizedTensorInt16 afe.ir.defines.QuantizedParam Classes ------- .. autoapisummary:: afe.ir.defines.Status afe.ir.defines.DataValue afe.ir.defines.TensorValue afe.ir.defines.TupleValue afe.ir.defines.DataIndex afe.ir.defines.TensorIndex afe.ir.defines.TupleIndex afe.ir.defines.NodeAssociatedValue afe.ir.defines.RequantizationMode afe.ir.defines.Quantization afe.ir.defines.RequantMethod afe.ir.defines.QuantizationCast afe.ir.defines.IdentityCast afe.ir.defines.QuantCast afe.ir.defines.DequantCast afe.ir.defines.RequantCast afe.ir.defines.ConvertCast afe.ir.defines.TupleCast afe.ir.defines.InputsQuantCast afe.ir.defines.QuantizationCasts afe.ir.defines.LayerStats afe.ir.defines.NodeReporter afe.ir.defines.LogNodeReporter afe.ir.defines.BiasCorrectionType Functions --------- .. autoapisummary:: afe.ir.defines.foreach_data_value afe.ir.defines.data_value_elements afe.ir.defines.get_expected_tensor_value afe.ir.defines.get_expected_tuple_values afe.ir.defines.reduce_data_value afe.ir.defines.map_data_value afe.ir.defines.zip_data_value afe.ir.defines.reconstruct_data_value afe.ir.defines.index_data_value Module Contents --------------- .. py:data:: AwesomeDataLayout :value: 'NHWC' .. py:data:: AwesomeDataLayout5D :value: 'NDHWC' .. py:data:: AwesomeConvWeightLayout :value: 'HWIO' .. py:data:: AwesomeConvWeightLayout5D :value: 'DHWIO' .. py:data:: AwesomeDepthwiseConvWeightLayout :value: 'HWOI' .. py:data:: AwesomeDepthwiseConvWeightLayout5D :value: 'DHWOI' .. py:data:: AwesomeTransposeConvWeightLayout5D :value: 'DHWOI' .. py:data:: NoneType .. py:data:: NodeName .. py:data:: InputName .. py:data:: TensorFormat .. py:data:: InputShape .. py:data:: ConvPad .. py:data:: AwesomePad2D .. py:data:: AwesomeStrides2D .. py:data:: AwesomeDilation2D .. py:data:: AwesomePoolSize2D .. py:data:: AwesomePad3D .. py:data:: AwesomeStrides3D .. py:data:: AwesomeDilation3D .. py:data:: AwesomePoolSize3D .. py:data:: AwesomePad .. py:data:: AwesomeStrides .. py:data:: AwesomeDilation .. py:data:: AwesomePoolSize .. py:data:: Float .. py:data:: QuantizedTensor .. py:data:: QuantizedTensorNew .. py:data:: QuantizedTensorInt16 .. py:data:: QuantizedParam .. py:class:: Status Status for AwesomeNode RELAY: Right after parsing from TVM Relay IR module CALIBRATED: Calibrated SIMA_QUANTIZED: SiMa Quantized BACKEND_IR_LOWERED: After lowering MLA subgraphs to SiMa BackendIR BACKEND_IR_COMPILED: After compilation using compile_awesomenet .. py:attribute:: RELAY :value: 'RELAY' .. py:attribute:: CALIBRATED :value: 'CALIBRATED' .. py:attribute:: SIMA_QUANTIZED :value: 'SIMA_QUANTIZED' .. py:attribute:: BACKEND_IR_LOWERED :value: 'BACKEND_IR_LOWERED' .. py:attribute:: BACKEND_IR_COMPILED :value: 'BACKEND_IR_COMPILED' .. py:class:: DataValue An abstract value in a network. The type parameter represents the data type that stands in for a tensor value. .. py:class:: TensorValue An abstract value associated with a tensor in a network. .. py:attribute:: value :type: _TENSOR .. py:class:: TupleValue An abstract value associated with a tuple in a network. An abstract value is associated with each element of the tuple. .. py:attribute:: elements :type: List[DataValue[_TENSOR]] .. py:function:: foreach_data_value(f: Callable[[_TENSOR], None], v: DataValue[_TENSOR]) -> None Apply a function to each tensor value in a DataValue. :param f: Function to apply :param v: DataValue to traverse .. py:function:: data_value_elements(v: DataValue[_TENSOR]) -> List[_TENSOR] Get all tensor values in a DataValue. Since the DataValue structure is ignored, this function is only suitable when it doesn't matter where the tensor values are located inside the DataValue. .. py:function:: get_expected_tensor_value(v: DataValue[_TENSOR]) -> _TENSOR Get a value from DataValue while expecting that the type of s DataValue is a TensorValue. .. py:function:: get_expected_tuple_values(v: DataValue[_TENSOR]) -> List[_TENSOR] Get a list of values from DataValue while expecting that the type of s DataValue is non-nested TupleValue. .. py:function:: reduce_data_value(f: Callable[[_A, _TENSOR], _A], v: DataValue[_TENSOR], initial: _A) -> _A Combine all values in a DataValue using the given function. :param f: Combining function :param v: DataValue to traverse :param initial: Initial value of result :return: Combined value .. py:function:: map_data_value(f: Callable[[_A], _B], v: DataValue[_A]) -> DataValue[_B] Transform each tensor value in a DataValue according to the given function, and return the results as a new DataValue. :param f: Function to apply :param v: DataValue to transform :return: DataValue with all tensor values transformed .. py:function:: zip_data_value(f: Callable[[_A, _B], _C], x: DataValue[_A], y: DataValue[_B]) -> DataValue[_C] Apply f to each pair of tensor values at the same positions in x and y, which must have the same shape. Return the results as a new DataValue having the same shape as x and y. :param f: Function to apply :param x: DataValue to transform :param y: DataValue to transform :return: Transformed data .. py:function:: reconstruct_data_value(values: List[_TENSOR]) -> DataValue[_TENSOR] Convert a list to a DataValue, using heuristics to guess the data structure. This function is provided for compatibility with existing code that does not keep track of the data structure. If the list has one item, it's treated as representing a single tensor. If it has many items, it's treated as representing a tuple of tensors. :param values: Values to interpret as a DataValue .. py:class:: DataIndex The position of an A within a DataValue[A]. This is an algebraic data type. .. py:class:: TensorIndex Identifies the single value in a TensorValue. .. py:class:: TupleIndex Identifies a position in a TupleValue. .. py:attribute:: index :type: int .. py:attribute:: nested_index :type: DataIndex .. py:function:: index_data_value(v: DataValue[_TENSOR], i: DataIndex) -> _TENSOR Get the value at the given index. .. py:class:: NodeAssociatedValue A set of abstract values associated with a network node's inputs and outputs. Input values are held in an ordered dictionary mapping strings to data values. Inputs can be examined positionally or by name. The output value is a single data value. .. py:attribute:: inputs :type: Dict[Union[NodeName, InputName], DataValue[_TENSOR]] .. py:attribute:: output :type: DataValue[_TENSOR] .. py:class:: RequantizationMode A way of doing quantized arithmetic. Different modes make different arithmetic simplifications embodying different speed accuracy tradeoffs. It is expected that TFLite-style quantization would give better accuracy while Sima-style quantization will run faster. The requantiaztion mode only applies to convolution operators. .. py:attribute:: sima .. py:attribute:: tflite .. py:class:: Quantization A quantization scale. It represents an encoding of real numbers r as integers q where: L = -2^(bits-1) (integer range lower bound) U = 2^(bits-1)-1 (integer range upper bound) q_unbounded = round((r * scale) + zero_point) (linear mapping to representable range) q = max(L, min(U, q_unbounded)) (clip to range) Fields min_val and max_val give the range of floating-point values that are represented, for instance the range that was selected by calibration. This range must be representable within the integer range, that is, L <= round((min_val * scale) + zero_point) <= round((max_val * scale) + zero_point) <= U Often it spans the entire range from L to U. It may be smaller if the range was expanded due to constraints on the quantized representation, such as when using symmetric quantization for a numeric range that is not symmetric. If a larger numeric range was clipped when quantizing, min_val and max_val still describe the representable range and not the original range. When a tensor contains only zero, scale is set to 0. and min_val = max_val = 0. The default values represent quantization of the floating-point range [-128, 127] using the integer range [-128, 127]. .. py:attribute:: scale :type: float :value: 1.0 .. py:attribute:: zero_point :type: int :value: 0 .. py:attribute:: bits :type: int :value: 8 .. py:attribute:: min_val :type: float :value: -128.0 .. py:attribute:: max_val :type: float :value: 127.0 .. py:method:: representable(scale: float, zero_point: int, bits: int) -> Quantization :staticmethod: Create a quantization scale that includes the entire representable integer range. See Quantization for documentation of the parameters. For zero tensors, scale is 0. and min_val = max_val = 0. :param scale: Quantization scale. :param zero_point: Quantization zero point. :param bits: Quantization bits. :return: Quantization scale constructed from the given parameters. .. py:class:: RequantMethod A requantization method as defined in ml_kernels. This enum is used to select which type of requantization to use when a network is quantized. .. py:attribute:: fractional_zero .. py:attribute:: arith_folded .. py:attribute:: scaled_fz .. py:class:: QuantizationCast A quantization-related conversion on data. When the algorithm detects that a conversion needs to be inserted in a model graph, it's recorded using this class. This is an algebraic data type. .. py:class:: IdentityCast A conversion that does nothing. It represents the case where no conversion is needed. .. py:class:: QuantCast A quantization cast. It represents a cast of a tensor having the given shape from float32 to int8 or int32 by computing round(r * scale + zero_point). .. py:attribute:: shape :type: Tuple[int, Ellipsis] .. py:attribute:: scale :type: float .. py:attribute:: zero_point :type: int .. py:attribute:: num_bits :type: int .. py:attribute:: out_type :type: afe.ir.tensor_type.ScalarType .. py:class:: DequantCast A quantization cast. It represents a cast of a tensor having the given shape from an integer type to float32 by computing (q - zero_point) / scale. :param shape: Shape of tensor to dequantize :param scale: Quantization scale :param zero_point: Quantization zero point :param input_dtype: Input data type. The valid Numpy data types are: np.int8, np.int16, or np.int32. .. py:attribute:: shape :type: Tuple[int, Ellipsis] .. py:attribute:: scale :type: float .. py:attribute:: zero_point :type: int .. py:attribute:: input_dtype :type: numpy.dtype .. py:attribute:: output_dtype :type: numpy.dtype .. py:class:: RequantCast A quantization cast. It represents a cast of a tensor having the given shape from an int32 type to int16/int8. :param shape: Shape of a tensor :param in_scale: Input quantization scale :param in_zero_point: Input quantization zero point :param out_scale: Output quantization scale :param out_zero_point: Output quantization zero point :param input_32_bit: If True, the input type is int32. If False, the input type is int16. :param output_type: Output data type, can be int16 or int8 :param requantization_type: Type of requantization to use. If arith_folded is used, then the requantization will use only a shift; the scales and zero points must be related by a power of 2 factor to minimize rounding error. .. py:attribute:: shape :type: Tuple[int, Ellipsis] .. py:attribute:: in_scale :type: float .. py:attribute:: in_zero_point :type: int .. py:attribute:: out_scale :type: float .. py:attribute:: out_zero_point :type: int .. py:attribute:: min_val :type: float .. py:attribute:: max_val :type: float .. py:attribute:: input_32_bit :type: bool .. py:attribute:: output_16_bit :type: bool .. py:attribute:: requant_method :type: RequantMethod .. py:method:: get_input_quantization() -> Quantization .. py:method:: get_output_quantization() -> Quantization .. py:class:: ConvertCast A numeric conversion. It represents a conversion from one numeric type to the nearest approximation in another numeric type. :param shape: Shape of a tensor :param in_type: Scalar type of input :param out_type: Scalar type of output .. py:attribute:: shape :type: Tuple[int, Ellipsis] .. py:attribute:: in_type :type: afe.ir.tensor_type.ScalarType .. py:attribute:: out_type :type: afe.ir.tensor_type.ScalarType .. py:class:: TupleCast A tuple cast. It applies a cast to each element of the tuple. .. py:attribute:: elements :type: List[QuantizationCast] .. py:class:: InputsQuantCast A set of quantization casts to apply to a node's inputs. The dict has an entry for each input. .. py:attribute:: casts :type: Dict[InputName, QuantizationCast] .. py:method:: does_nothing() -> bool Return true if this cast does nothing. .. py:class:: QuantizationCasts A set of quantization casts to apply to a model. The casts are collected during a traversal of the model, then applied after the traversal is finished. Field `casts` holds the casts to apply to node inputs. If a node does not need casts, it is omitted. .. py:attribute:: casts :type: Dict[NodeName, InputsQuantCast] .. py:method:: insert(node: NodeName, cast: InputsQuantCast) .. py:class:: LayerStats Layer statistics. For each MLA node, quantization error is calculated, that information is than forwarded to .sima.json file, and it can be viewed in Netron. :param metric: Metric that is used for calculating error value. :param error_value: Error value. .. py:attribute:: metric :type: str .. py:attribute:: error_value :type: float .. py:class:: NodeReporter A node reporter to display information or warning messages about a node during transformations .. py:method:: info(msg: str) :abstractmethod: .. py:method:: debug(msg: str) :abstractmethod: .. py:method:: warn(msg: str) :abstractmethod: .. py:class:: LogNodeReporter(node_name: NodeName) A node reporter to display information or warning messages about a node during transformations :param node_name: Name of the node .. py:attribute:: node_name :type: NodeName .. py:method:: info(msg: str) .. py:method:: debug(msg: str) .. py:method:: warn(msg: str) .. py:class:: BiasCorrectionType A bias correction method for convolution. REGULAR: Bias correction using input mean estimated during calibration ITERATIVE: Bias correction using input mean estimated by executing the quantized model with a set of calibration inputs NONE: No bias correction .. py:attribute:: REGULAR :value: 'REGULAR' .. py:attribute:: ITERATIVE :value: 'ITERATIVE' .. py:attribute:: NONE :value: 'NONE'