afe.ir.attributes

Attributes

`DEFAULT_PER_CHANNEL`
`ASYMMETRY`
`dummy_quant_result_tensor_type`
`ACTIVATION_ATTRS`
`QUANT_ACTIVATION_ATTRS`

Classes

`ObservedDistribution`	A value distribution that was observed during calibration. This value
`QuantResultTensorType`	The result of running the quantization transformation on a tensor.
`AwesomeCalibAttrs`	Calibration attributes
`AwesomeQuantAttrBase`	Base class of quantized operator attributes. This class is used for instance
`PlaceholderQuantAttrs`	Properties of a quantized placeholder.
`ConstantQuantAttrs`
`ConcatQuantAttrs`	Contains quantization attributes for concatenate quantization.
`MultiplyQuantAttrs`	param lhs_input_shape: Lhs input shape
`LeakyReluQuantAttrs`	The slope for quantized_intput < zero_point is (alpha >> right_shift)
`AwesomeAttributes`	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
`PlaceholderAttrs`	Properties of a placeholder.
`ConstantAttrs`
`MultiplyAttrs`	Attributes of a multiply operator.
`ConvAttrs`	Attributes of a convolution operation.
`PoolAttrs`
`MaxPoolAttrs`
`AvgPoolAttrs`
`VarianceAttrs`
`ReluAttrs`
`AddAttrs`	Attributes of an add operator.
`SubtractAttrs`	Attributes of a subtract operator.
`BiasAddAttrs`
`ConstantMultiplyAddAttrs`	Attributes representing the computation (ac + bd) for scalar constants c and d.
`ReduceAttrs`	Attributes used inside reduce operators such as mean and sum.
`MeanQuantAttrs`	Contains quantization attributes for mean quantization.
`ReduceQuantAttrs`	Contains quantization attributes for reduce op quantization.
`ArgMaxAttrs`
`LayerNormAttrs`
`RMSNormAttrs`
`InstanceNormAttrs`	Instance Normalization operator attributes.
`SoftmaxAttrs`
`PadAttrs`
`LRNAttrs`
`ClipAttrs`	Attributes for Clip operation. Clip operation is always merged into a composite operator.
`ExtmAttrs`	Attributes for extremum op, can be min or max op depending on the max boolean.
`ExtmQuantAttrs`	Contains quantization attributes for max and min quantization.
`ProdAttrs`
`FullAttrs`
`TileAttrs`
`UpsamplingAttrs`
`ImageResize2DAttrs`
`GridSampleAttrs`	Attributes of GridSample operator.
`MaximumAttrs`	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
`MinimumAttrs`	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
`TensorManipulationBaseAttrs`	Do nothing. Used for better structuring data structure
`TupleAttrs`	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
`TupleGetItemAttrs`
`SqueezeAttrs`
`ConcatenateAttrs`
`TransposeAttrs`
`DepthToSpaceAttrs`	Attributes of DepthToSpace operator
`ReshapeAttrs`
`ExpandDimsAttrs`
`BatchFlattenAttrs`	Attributes of a batch flatten operation.
`SplitAttrs`
`TakeAttrs`
`StridedSliceAttrs`
`LayoutTransformAttrs`
`BroadcastToAttrs`	Do nothing. Used for better structuring data structure
`TessellationTransformAttrs`
`DetessellationTransformAttrs`
`PackTransformAttrs`
`UnpackTransformAttrs`
`NormalizationTransformAttrs`
`QuantizationTransformAttrs`
`DequantizationTransformAttrs`
`ResizeTransformAttrs`
`ChromaUpsampleTransformAttrs`
`YuvRgbConversionTransformAttrs`
`BgrRgbConversionTransformAttrs`
`SigmoidTransformAttrs`
`NmsMaxpoolTransformAttrs`
`CastAttrs`
`UDFAttrs`	Common attributes for UDF functions:
`LeakyReluAttrs`
`SwishAttrs`	Common attributes for UDF functions:
`PReluAttrs`
`ClipQuantAttrs`	Attributes for Clip operation. Clip operation is always merged into a composite operator.
`ReluQuantAttrs`	Base class of quantized operator attributes. This class is used for instance
`AddActivationAttrs`	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
`ConvAddActivationAttrs`	Attributes of a fused convolution operator consisting of convolution, optional bias-add,
`TupleConcatenateAttrs`	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
`ExternalAttrs`
`QNNQuantizeAttrs`	Further reference: tvm/src/relay/qnn/op/quantize.cc
`QNNDequantizeAttrs`	Further reference: tvm/src/relay/qnn/op/dequantize.cc
`RequantizeAttrs`	Further reference: tvm/src/relay/qnn/op/requantize.cc
`CustomOpAttrs`	Custom Op AwesomeAttributes
`AddQuantAttrs`	Attributes for quantized AddActivationOp.
`SubtractQuantAttrs`	param attrs: SubtractAttrs class holding SubtractOp parameters
`ConvQuantAttrs`	Used for all variants of convolution.
`UpsamplingQuantAttrs`
`ImageResize2DQuantAttrs`
`LRNQuantAttrs`
`LayerNormQuantAttrs`
`InstanceNormQuantAttrs`	Quantized Instance Normalization operator attributes.
`RMSNormQuantAttrs`
`SoftmaxQuantAttrs`
`RequantizeQuantAttrs`	Base class of quantized operator attributes. This class is used for instance
`ConcatQuantAttrs`	Contains quantization attributes for concatenate quantization.
`CustomOpQuantAttrs`	Contains quantization attributes for custom operation quantization.
`PoolQuantAttrs`	Contains quantization attributes for pool quantization.
`VarianceQuantAttrs`
`UDFQuantAttrs`	Base class of quantized operator attributes. This class is used for instance
`DivideAttrs`	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
`DivideQuantAttrs`	Base class of quantized operator attributes. This class is used for instance
`HardSigmoidAttrs`	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
`LeakyReluCompositeQuantAttrs`	Contains quantization attributes for both UDF and breakdown LeakyRelu quantization.
`PReluQuantAttrs`	The slope for quantized_intput < zero_point is (alpha >> right_shift)
`PowerAttrs`	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
`ArgMaxQuantAttrs`	Base class of quantized operator attributes. This class is used for instance
`BatchMatmulAttrs`	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
`BatchMatmulQuantAttrs`	Base class of quantized operator attributes. This class is used for instance
`SliceConcatAttrs`	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
`SliceConcatQuantAttrs`	Base class of quantized operator attributes. This class is used for instance
`BroadcastToQuantAttrs`	Base class of quantized operator attributes. This class is used for instance

Functions

`get_quant_result_scale_with_dummy`(...)	Get the quantization scale; if there is none, return a dummy value.
`get_data_value_quant_result_scale_with_dummy`(...)	Run get_quant_result_scale_with_dummy on the contents of a DataValue.
`get_dict_quant_result_scale_with_dummy`(...)	Run get_quant_result_scale_with_dummy on the contents of a dict of DataValue.
`update_quant_result_quantization`(...)	Insert the given quantization into t, replacing existing quantization values in t.
`update_quant_result_type`(...)	Ensure that t's type matches new_type by replacing dummy types with data from t and
`set_quant_result_type_batch_size`(...)	Modifies DataValue of QuantResultTensorType with given batch size.
`is_dummy_type`(→ bool)
`convolution_output_shape`(→ tuple[int, Ellipsis])	Get the shape of a convolution's output tensor based on its attributes.

Module Contents

afe.ir.attributes.DEFAULT_PER_CHANNEL = False

afe.ir.attributes.ASYMMETRY = True

class afe.ir.attributes.ObservedDistribution(observer: afe.ir.node_observer.NodeObserver)

A value distribution that was observed during calibration. This value distribution can be used to decide how to quantize a tensor.

calculate_quantization(qrange: Tuple[int, int]) → afe.ir.defines.DataValue[afe.ir.defines.Quantization]

Choose a quantization to use for representing the observed value distribution using the given integer range.

Parameters:: qrange – Integer range to quantize for. The range must be representable by an 8-bit or 16-bit signed integer.
Returns:: Selected quantizations.

has_per_channel_properties(): Return True if the observer has per-channel information, False otherwise.

get_min_max() → Tuple[float, float]

Get range (min and max) of observed values. Min-max range does not need to include zero.

Returns:: Tuple of min and max values.

get_mean()

class afe.ir.attributes.QuantResultTensorType

The result of running the quantization transformation on a tensor. It has the tensor’s type and quantization, as they are after the transformation. Only tensors that are quantized by the transformation have a quantization.

Parameters:

type – The tensor’s type after transformation. It has the same shape as before the transformation. Its scalar type may be different.
quant – The tensor’s quantization, if it was quantized by the quantization transformation. None otherwise. Floating-point tensors do not have a quantization. Integer tensors do not have a quantization if they were already integer before the quantization transformation.
requant_method – The method that should be used for requantizing this tensor’s value when requantization is needed. This field must be None iff quant is None.

type: afe.ir.tensor_type.TensorType

quant: afe.ir.defines.Quantization | None

requant_method: afe.ir.defines.RequantMethod | None

static from_type(type: afe.ir.tensor_type.TensorType) → QuantResultTensorType: Make a QuantResultTensorType that only has type information.

static from_quant(quant: afe.ir.defines.Quantization | None) → QuantResultTensorType: Make a QuantResultTensorType from a Quantization using dummy type information. This is a temporary method that should be removed when support for QuantResultTensorType is finished.

afe.ir.attributes.get_quant_result_scale_with_dummy(t: QuantResultTensorType) → afe.ir.defines.Quantization

Get the quantization scale; if there is none, return a dummy value.

The dummy value is a temporary solution that should be removed when support for QuantResultTensorType is finished.

afe.ir.attributes.get_data_value_quant_result_scale_with_dummy(t: afe.ir.defines.DataValue[QuantResultTensorType]) → afe.ir.defines.DataValue[afe.ir.defines.Quantization]: Run get_quant_result_scale_with_dummy on the contents of a DataValue.

afe.ir.attributes.get_dict_quant_result_scale_with_dummy(t: Dict[afe.ir.defines.InputName, afe.ir.defines.DataValue[QuantResultTensorType]]) → Dict[afe.ir.defines.InputName, afe.ir.defines.DataValue[afe.ir.defines.Quantization]]: Run get_quant_result_scale_with_dummy on the contents of a dict of DataValue.

afe.ir.attributes.update_quant_result_quantization(t: afe.ir.defines.DataValue[QuantResultTensorType], new_quant: afe.ir.defines.DataValue[afe.ir.defines.Quantization]) → afe.ir.defines.DataValue[QuantResultTensorType]

Insert the given quantization into t, replacing existing quantization values in t.

Parameters:

t – Quantization result type to modify
new_quant – Quantization values

Returns:

A copy of t with values from new_type inserted

afe.ir.attributes.update_quant_result_type(t: afe.ir.defines.DataValue[QuantResultTensorType], new_type: afe.ir.defines.DataValue[afe.ir.tensor_type.TensorType]) → afe.ir.defines.DataValue[QuantResultTensorType]

Ensure that t’s type matches new_type by replacing dummy types with data from t and checking non-dummy types.

This function’s purpose is to save the type into t while developing QuantResultTensorType, then to check consistency after it is developed.

Parameters:

t – Quantization result type, which may contain dummy types
new_type – Type that should be the same as the type in t

Returns:

A copy of t with values from new_type inserted to replace any dummy types

afe.ir.attributes.set_quant_result_type_batch_size(t: afe.ir.defines.DataValue[QuantResultTensorType], batch_size: int) → afe.ir.defines.DataValue[QuantResultTensorType]

Modifies DataValue of QuantResultTensorType with given batch size.

Parameters:

t – DataValue[QuantResultTensorType]. Value to be modified.
batch_size – int. Batch size value to be used in constructing new QuantResultTensorType DataValue.

Returns:

DataValue[QuantResultTensorType]. QuantResultTensorType with its type’s shape field modified to use batch_size.

afe.ir.attributes.dummy_quant_result_tensor_type

afe.ir.attributes.is_dummy_type(t: afe.ir.tensor_type.TensorType) → bool

class afe.ir.attributes.AwesomeCalibAttrs

Calibration attributes :param observer: Observer used during calibration of the node. If the node does not use

calibration data for calculation of quantization parameters, observer will not be created and its value will be None.

Parameters:

intermediate_observers – Observers used for quantization of intermediate results.
quant – Quantization scale of the output. It is assigned during quantization.
input_quant – Quantization scale of each input. During quantization, it is first assigned the type and quantization scale that were determined at the nodes that compute the inputs. Then, when the node is quantized, it is assigned the types and quantization scales of inputs that the node accepts.

observer: afe.ir.node_observer.NodeObserver | None = None

intermediate_observers: Dict[str, afe.ir.node_observer.NodeObserver] | None = None

quant: afe.ir.defines.DataValue[QuantResultTensorType]

input_quant: Dict[afe.ir.defines.InputName, afe.ir.defines.DataValue[QuantResultTensorType]]

precomputed_quant: afe.ir.defines.Quantization | None = None

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.AwesomeQuantAttrBase

Base class of quantized operator attributes. This class is used for instance checking only.

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size. Should be implemented separately inside each class that inherits from AwesomeQuantAttrBase.

class afe.ir.attributes.PlaceholderQuantAttrs

Properties of a quantized placeholder.

Parameters:

type – Type of the placeholder’s output.
quantization – Quantization of the placeholder, if it was quantized by the Quantize compiler pass.

type: afe.ir.tensor_type.TensorType

quantization: afe.ir.defines.Quantization | None

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConstantQuantAttrs

Parameters:: quant_data – Quantized tensor value

quant_data: numpy.ndarray

class afe.ir.attributes.ConcatQuantAttrs

Contains quantization attributes for concatenate quantization.

Parameters:

input_scale_corrections – Quantized scale correction for each inputs.
input_zp_corrections – Quantized zero point correction for each inputs.
right_shift – Number of bits in right shift during requantize at inference time.
layer_bits – Number of bits used for quantizing the tensor.
axis – The axis along which the tensors are concatenated.
node_zps – Zero points(s) of the quantized output tensors(s)
input_scales – Quantized scale for eash inputs.
node_scales – Using the max input_scales as the concatenate output scale of the quantized output tensors(s).

input_scale_corrections: List[int] = []

input_zp_corrections: List[int] = []

right_shifts: List[int] = []

layer_bits: List[int] = [8]

axis: int | None = None

input_scales: List[float | List[float]] | None = []

node_scales: List[float] | None = []

node_zps: List[int] | None = None

rounding_type: ml_kernels.math_helpers.RoundType

class afe.ir.attributes.MultiplyQuantAttrs

param lhs_input_shape: Lhs input shape param rhs_input_shape: Rhs input shape :param input_int16: If True, the inputs have int16 type. If false, the inputs have int8 type. :param intrinsic_shift: Right-shift to apply before requantization. param requant Requantization parameters param lhs_zero_point: Zero point of the left-hand side input. param rhs_zero_point: Zero point of the right-hand side input. param layer_bits: Number of bits used to quantize output tensor.

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

input_int16: bool

intrinsic_shift: int

requant: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

lhs_zero_point: int = 0

rhs_zero_point: int = 0

layer_bits: int = 8

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LeakyReluQuantAttrs

The slope for quantized_intput < zero_point is (alpha >> right_shift)

alpha: int

input_shape: Tuple[int, Ellipsis]

right_shift: int

zero_point: int

bits: int

rounding_type: ml_kernels.math_helpers.RoundType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.AwesomeAttributes

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size. Should be implemented separately inside each class that inherits from AwesomeAttributes.

class afe.ir.attributes.PlaceholderAttrs

Properties of a placeholder.

type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConstantAttrs

Parameters:: data – Tensor value before quantization

data: numpy.ndarray

class afe.ir.attributes.MultiplyAttrs

Attributes of a multiply operator.

Parameters:

scalar_type – Type of input and output. Must be a floating-point type.
lhs_input_shape – Shape of first input.
rhs_input_shape – Shape of second input.

scalar_type: afe.ir.tensor_type.ScalarType

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

afe.ir.attributes.convolution_output_shape(conv_attrs: ConvAttrs) → tuple[int, Ellipsis]: Get the shape of a convolution’s output tensor based on its attributes.

class afe.ir.attributes.ConvAttrs

Attributes of a convolution operation.

The attributes describe a convolution with input and output activations in NWC, NHWC, or NDHWC layout and weights in WIGO, HWIGO, or DHWIGO layout.

The dimension order for 1, 2, or 3 spatial dimensions respectively is W, HW, or DHW.

Parameters:

stride – Stride in each spatial dimension
dilation – Dilation in each spatial dimension
padding – Padding in each spatial dimension. The padding in each dimension is a tuple holding the padding width at the beginning and end of the dimension.
output_padding – Padding of the output tensor in each spatial dimension for transposed convolution. If it is not a transposed convolution, all padding values must be zero. If it is a transposed convolution, the first element of the padding must be zero.
is_transposed – Whether it is a transposed convolution.
weight_shape – Shape of the weight tensor in spatial dimensions ++ IGO layout, for example HWIGO. IGO is an abbreviation for “input channels, groups, output channels”.
input_spatial_shape – Shape of the input tensor in spatial dimensions.
batch_size – Batch size.
input_type – Scalar type of the convolution’s input tensor. This type is ignored for quantized convolutions.

stride: tuple[int, Ellipsis]

dilation: tuple[int, Ellipsis]

padding: tuple[tuple[int, int], Ellipsis]

output_padding: tuple[tuple[int, int], Ellipsis]

is_transposed: bool

weight_shape: tuple[int, Ellipsis]

input_spatial_shape: tuple[int, Ellipsis]

batch_size: int

input_type: afe.ir.tensor_type.ScalarType

property groups: int: Get the number of convolution groups.

property channels: int: Get the number of convolution output channels.

property input_channels: int: Get the number of convolution input channels.

property kernel_size: tuple[int, Ellipsis]: Get the shape of the convolution kernel in the spatial dimensions.

property num_spatial_dimensions: int: Get the number of spatial dimensions for this convolution.

property input_shape: tuple[int, Ellipsis]: Get the shape of the convolution’s input tensor in NWC, NHWC, or NDHWC layout.

property output_shape: tuple[int, Ellipsis]: Get the shape of the convolution’s output tensor in NWC, NHWC, or NDHWC layout.

property is_depthwise_one_channel: bool: Return true if this convolution is a depthwise convolution with equal number of input and output channels.

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size. Should be implemented separately inside each class that inherits from AwesomeAttributes.

class afe.ir.attributes.PoolAttrs

Parameters:

ceil_mode – Used to take ceil or floor when computing the output shape
out_layout – Layout of the output. This can be an empty str if layout is the same as data_layout.
layout – Uses the letters NHWC for BatchNumber, Height, Width, Channels
padding – ((pad_top, pad_bot), …) along the dimensions of NHWC according to layout
pool_size – Size of pooling
strides – Strides
dilation – Dilation along the dimensions of NHWC according to data_layout
scalar_type – Data type of the input and output.

ceil_mode: int

out_layout: str

layout: str

padding: afe.ir.defines.AwesomePad

pool_size: afe.ir.defines.AwesomePoolSize

strides: afe.ir.defines.AwesomeStrides

dilation: afe.ir.defines.AwesomeDilation

input_shape: afe.ir.defines.AwesomePoolSize

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.MaxPoolAttrs

Parameters:

ceil_mode – Used to take ceil or floor when computing the output shape
out_layout – Layout of the output. This can be an empty str if layout is the same as data_layout.
layout – Uses the letters NHWC for BatchNumber, Height, Width, Channels
padding – ((pad_top, pad_bot), …) along the dimensions of NHWC according to layout
pool_size – Size of pooling
strides – Strides
dilation – Dilation along the dimensions of NHWC according to data_layout
scalar_type – Data type of the input and output.

class afe.ir.attributes.AvgPoolAttrs

Parameters:: count_include_pad – If true, include padding to compute the average.

count_include_pad: bool

class afe.ir.attributes.VarianceAttrs

input_data_shape: Shape of the input tensor.

mean_shape: Shape of the mean input tensor.

scalar_type: Scalar type of the input and output.

axis: The axes to sum over when computing mean.

input_data_shape: tuple[int, Ellipsis]

mean_shape: tuple[int, Ellipsis]

scalar_type: afe.ir.tensor_type.ScalarType

axis: tuple[int, Ellipsis]

class afe.ir.attributes.ReluAttrs

Parameters:

scalar_type – Type of input and output.
input_shape – Shape of input.

scalar_type: afe.ir.tensor_type.ScalarType

input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.AddAttrs

Attributes of an add operator.

Parameters:

scalar_type – Type of input and output. Must be a floating-point type.
lhs_input_shape – Shape of first input.
rhs_input_shape – Shape of second input.

scalar_type: afe.ir.tensor_type.ScalarType

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.SubtractAttrs

Attributes of a subtract operator.

Parameters:

scalar_type – Type of input and output. Must be a floating-point type.
lhs_input_shape – Shape of first input.
rhs_input_shape – Shape of second input.

scalar_type: afe.ir.tensor_type.ScalarType

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.BiasAddAttrs

Parameters:

input_shape – The shape of the input activation tensor
axis – The axis to add the bias

input_shape: Tuple[int, Ellipsis]

axis: int

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConstantMultiplyAddAttrs(scalar_type: afe.ir.tensor_type.ScalarType, lhs_input_shape: Tuple[int, Ellipsis], rhs_input_shape: Tuple[int, Ellipsis], in1_const_attrs: ConstantAttrs, in2_const_attrs: ConstantAttrs | None = None)

Attributes representing the computation (a*c + b*d) for scalar constants c and d.

scalar_type: afe.ir.tensor_type.ScalarType

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

in1_const_attrs: ConstantAttrs

in2_const_attrs: ConstantAttrs | None

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ReduceAttrs

Attributes used inside reduce operators such as mean and sum.

Parameters:

axis – Axis or axes along which a reduction operation is performed.
exclude – If exclude is true, we use the axes that are NOT in the axis field.
keepdims – If set to true the axes reduces are left with a size of 1.
shape – Shape of the input tensor.
scalar_type – Type of input and output. Must be a floating-point type.

axis: list[int] | tuple[int]

exclude: bool

keepdims: bool

shape: tuple[int]

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.MeanQuantAttrs

Contains quantization attributes for mean quantization.

Parameters:

attrs – ReduceAttrs used in mean operator.
node_scales – Scales(s) of the quantized output tensors(s).
node_zps – Zero points(s) of the quantized output tensors(s).

attrs: ReduceAttrs

node_scales: float = 1.0

node_zps: int = 0

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ReduceQuantAttrs

Contains quantization attributes for reduce op quantization.

attrs: ReduceAttrs used in reduce operator.

requant: Requantization parameters.

attrs: ReduceAttrs

requant: ml_kernels.requantization.BaseRequantization | None = None

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ArgMaxAttrs

Parameters:

axis – Axis or axes along which a mean operation is performed.
exclude – If exclude is true, we use the axes that are NOT in the axis field
keepdims – If set to true the axes reduces are left with a size of 1
select_last_index – Whether to select the last index or the first index if the max element appears in multiple indices.
shape – Shape of input tensor
result_scalar_type – Type of numbers in result tensor. It must be either ScalarType.int32 or the same as the input tensor’s type.
input_scalar_type – Type of input values. It must be either ScalarType.float32 or ScalarType.int8.

axis: List[int]

exclude: int

keepdims: int

shape: Tuple[int, Ellipsis]

select_last_index: bool

result_scalar_type: afe.ir.tensor_type.ScalarType

input_scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LayerNormAttrs

Parameters:

axis – The axis to sum over when computing mean.
input_shape – Shape of input tensor.
epsilon – The epsilon value to use to avoid division by zero.
scalar_type – Type of input and output.

axis: int | tuple[int, int]

input_shape: tuple[int, Ellipsis]

epsilon: float

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.RMSNormAttrs

Parameters:

input_shape – Shape of input tensor.
epsilon – The epsilon value to use to avoid division by zero.
scalar_type – Type of input and output.

input_shape: Tuple[int, Ellipsis]

epsilon: float

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.InstanceNormAttrs

Instance Normalization operator attributes.

axis: The axes to sum over when computing mean.

input_data_shape: Shape of the input tensor.

mean_shape: Shape of the mean input tensor.

variance_shape: Shape of the variance input tensor.

epsilon: The epsilon value to use to avoid division by zero.

scalar_type: Type of input and output.

axis: tuple[int, int]

input_data_shape: tuple[int, Ellipsis]

mean_shape: tuple[int, Ellipsis]

variance_shape: tuple[int, Ellipsis]

epsilon: float

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.SoftmaxAttrs

Parameters:

axis – The axis to sum over when computing softmax
input_shape – Shape of input tensor
scalar_type – Type of input and output

axis: int

input_shape: Tuple[int, Ellipsis]

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.PadAttrs

Parameters:

pad_mode – ‘constant’, ‘edge’, ‘reflect’
pad_width – padding along each input dimension N in the format of (before_N, after_N)

pad_mode: str

pad_width: afe.ir.defines.AwesomePad

input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LRNAttrs

Parameters:

alpha – The scaling parameter.
axis – Input data layout channel axis. Default value is 1 for NCHW format
beta – The exponent parameter.
bias – The offset parameter to avoid dividing by 0.
size – The size of the local region to be considered for normalization.
shape – Shape of input tensor

# NOTES FOR TENSORFLOW # TVM defines size as size_tvm = (depth_radius_tf * 2) + 1 # TVM defines alpha as alpha_tvm = alpha_tf * size_tf

alpha: float

axis: int

beta: float

bias: float

size: int

shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ClipAttrs

Attributes for Clip operation. Clip operation is always merged into a composite operator. Same class is used in floating-point and quantized version.

Parameters:

a_min – min value of clip
a_max – max calue of clip
shape – Shape of input tensor

a_min: float

a_max: float

shape: Tuple[int, Ellipsis]

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ExtmAttrs

Attributes for extremum op, can be min or max op depending on the max boolean.

Parameters:

axis – Axis or axes along which a mean operation is performed.
exclude – If exclude is true, we use the axes that are NOT in the axis field
keepdims – If set to true the axes reduces are left with a size of 1
shape – Shape of input tensor
scalar_type – Type of input and output
max – If true the operation is max, if false the operation is min.

axis: List[int]

exclude: bool

keepdims: bool

shape: Tuple[int, Ellipsis]

scalar_type: afe.ir.tensor_type.ScalarType

max: bool

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ExtmQuantAttrs

Contains quantization attributes for max and min quantization.

Parameters:

attrs – ExtmAttrs used in Extm operator.
requant – Requantization parameters.
rounding_type – Rounding type for quantization.

attrs: ExtmAttrs

requant: ml_kernels.requantization.BaseRequantization | None = None

rounding_type: ml_kernels.math_helpers.RoundType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ProdAttrs

Parameters:

axis – Axis or axes along which a mean operation is performed.
exclude – If exclude is true, we use the axes that are NOT in the axis field
keepdims – If set to true the axes reduces are left with a size of 1

axis: list[int] | tuple[int]

exclude: int

keepdims: int

shape: tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.FullAttrs

Parameters:

shape – The shape of the target.
dtype – The data type of the target.

shape: List[int]

dtype: str

class afe.ir.attributes.TileAttrs

Parameters:: reps – The number of times repeating the tensor data.

reps: List[int]

class afe.ir.attributes.UpsamplingAttrs

Parameters:

input_shape – Shape of the input tensor.
scale_h – The scale factor for height upsampling.
scale_w – The scale factor for width upsampling.
layout – Layout of the input.
method – Scale method to used [nearest_neighbor, bilinear, bicubic].
align_corners – Whether to keep corners in proper place.
scalar_type – Data type.

input_shape: Tuple[int, Ellipsis]

scale_h: int

scale_w: int

layout: str

method: str

align_corners: bool

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ImageResize2DAttrs

Parameters:

size – The out size to which the image will be resized.
roi – The region of interest for cropping the input image. Expected to be of size 4 and format [start_h, start_w, end_h, end_w]. Only used if coordinate transformation_mode is ‘tf_crop_and_resize’.
layout – Layout of the input.
method – Scale method to used [nearest_neighbor, linear, bicubic].
coordinate_transformation_mode – Describes how to transform the coordinate in the resized tensor to the coordinate in the original tensor. Refer to the ONNX Resize operator specification for details. [half_pixel, align_corners, asymmetric]
rounding_method – (string, optional) - Indicates how to find the “nearest” pixel in nearest_neighbor method [round, floor, ceil]
cubic_alpha – (float) – Spline Coefficient for Bicubic Interpolation
cubic_exclude – (int) – Flag to exclude exterior of the image during bicubic interpolation
extrapolation_value – Fill value to use when roi is outside of the image.
out_dtype – Type to return. If left None returns the same type as input.

size: List[int]

roi: Tuple[float]

layout: str

method: str

coordinate_transformation_mode: str

rounding_method: str

cubic_alpha: float

cubic_exclude: int

extrapolation_value: float

out_dtype: str

input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.GridSampleAttrs

Attributes of GridSample operator.

input_shape: Shape of the input tensor. grid_shape: Shape of the grid tensor. method: Interpolation method to use [“nearest”, “bilinear”, “bicubic”]. padding_mode: padding mode [“zeros”, “border”, “reflection”]. align_corners: Whether to align the corners in interpolation. scalar_type: Data type.

input_shape: Tuple[int, Ellipsis]

grid_shape: Tuple[int, Ellipsis]

method: str

padding_mode: str

align_corners: bool

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.MaximumAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.MinimumAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.TensorManipulationBaseAttrs: Do nothing. Used for better structuring data structure

class afe.ir.attributes.TupleAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

input_types: List[afe.ir.tensor_type.TensorType]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.TupleGetItemAttrs

Parameters:

input_types – List of input tensor types
index – The index of the tuple_value we return

input_types: List[afe.ir.tensor_type.TensorType]

index: int

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.SqueezeAttrs

Parameters:

axis – Set of axes to remove
input_shape – Shape of input tensor
input_type – Data type of input tensor

axis: List[int]

input_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConcatenateAttrs

Parameters:

scalar_type – Scalar tyoe of the output.
axis – The axis along which the tensors are concatenated.
input_types – List of input tensor types.

scalar_type: afe.ir.tensor_type.ScalarType

axis: int

input_types: List[afe.ir.tensor_type.TensorType]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.TransposeAttrs

Parameters:

axes – The target axes order, reverse order if not specified.
input_shape – Shape of input tensor
input_type – Data type of input tensor

axes: List[int]

input_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.DepthToSpaceAttrs

Attributes of DepthToSpace operator

block_size: Bolck size that is shifted from channels to height and width mode: DCR for depth-column-row order re-arrangement, CRD for column-row-depth order input_shape: Shape of input tensor input_type: Data type of input tensor

block_size: int

mode: str

input_shape: tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ReshapeAttrs

Parameters:

input_shape – Shape of input tensor
dtype – Data type
newshape – The new shape.

input_shape: Tuple[int, Ellipsis]

dtype: afe.ir.tensor_type.ScalarType

newshape: List[int]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ExpandDimsAttrs

Parameters:

axis – The axis that is expanded
num_newaxis – The number of axes to be inserted. Should be >= 0
input_shape – Shape of input tensor
input_type – Data type of input tensor

axis: int

num_newaxis: int

input_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.BatchFlattenAttrs

Attributes of a batch flatten operation.

Parameters:

input_shape – Shape of input tensor.
dtype – Data type.

input_shape: Tuple[int, Ellipsis]

dtype: afe.ir.tensor_type.ScalarType

class afe.ir.attributes.SplitAttrs

Parameters:: indices_or_sections – Indices or sections to split into. Accepts an int or a tuple

If indices_or_sections is an integer, the input will be divided equally along given axis. If such a split is not possible, an error is raised. If indices_or_sections is a tuple of sorted integers, the entries indicate where along axis the array is split. :param axis: The axis over which to split. :param input_shape: Shape of input tensor :param input_type: Data type of input tensor

indices_or_sections: int | Tuple[int, Ellipsis]

axis: int

input_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.TakeAttrs

Parameters:

axis – The axis over which to select values. By default, the flattened input array is used.
mode – Specifies how out-of-bound indices will behave [clip, wrap, fast]. clip: clip to the range (default). wrap: wrap around the indices. fast: no clip or wrap around (user must make sure indices are in-bound).

axis: int

batch_dims: int

mode: str

input_shape: Tuple[int, Ellipsis]

indices_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.StridedSliceAttrs

Parameters:

begin – The indices to begin with in the slicing.
end – Indices indicating end of the slice.
strides – Specifies the stride values, it can be negative in that case, the input tensor will be reversed in that particular axis.
axes – Tuple[int] or List[int], optional. Axes along which slicing is applied. When it is specified, the length of begin, end, strides, and axes must be equal. Moreover, begin, end, strides, and axes must be static (cannot be relay.Expr). Axes argument for dynamic parameter slicing is not supported yet.
slice_mode –
The slice mode [end, size]. end: The ending indices for the slice [default]. size: The input strides will be ignored, input end in this mode indicates

the size of a slice starting at the location specified by begin. If end[i] is -1, all remaining elements in that dimension are included in the slice.
input_shape – Shape of input tensor
input_type – Data type of input tensor

begin: List[int]

end: List[int]

strides: List[int]

axes: Tuple[int] | List[int] | None

slice_mode: str

input_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LayoutTransformAttrs

Parameters:

input_type – Shape and data type of input tensor
src_layout – The source layout. (e.g NCHW)
dst_layout – The destination layout. (e.g. NCHW16c)
implicitly_removable – Whether this transform can be removed from the beginning or end of a model. If the field is True, transform can be removed in order to convert a model’s input and output data layout to NHWC (although the removal is done regardless of the actual data layout). Does not affect other optimizations that change or remove layout_transform operators.

input_type: afe.ir.tensor_type.TensorType

src_layout: str

dst_layout: str

implicitly_removable: bool = False

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.BroadcastToAttrs

Do nothing. Used for better structuring data structure

input_type: afe.ir.tensor_type.TensorType

output_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.TessellationTransformAttrs

Parameters:

slice_shape – Shape of slice in tessellation
align_c16 – Flag to force 16-channel alignment in tessellation
cblock – Flag to interleave the 16-channel blocks in tessellation
frame_type – Tensor type before tessellation

slice_shape: Sequence[int]

align_c16: bool

cblock: bool

frame_type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.DetessellationTransformAttrs

Parameters:

slice_shape – Shape of slice in de-tessellation
align_c16 – Flag to indicate that channels are aligned to 16 in tessellated slice.
cblock – Flag to indicate that tesseallated slice is interleaved by channel blocks.
frame_type – Tensor type after de-tessellation
input_shape – Shape of input tensor

slice_shape: Sequence[int]

align_c16: bool

cblock: bool

frame_type: afe.ir.tensor_type.TensorType

input_shape: Tuple[int]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.PackTransformAttrs

Parameters:

input_shapes – List of input tensor shape tuples
result_scalar_type – Type of numbers in result tensor, must be ScalarType.int8

input_shapes: List[afe.ir.tensor_type.TensorType]

result_scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.UnpackTransformAttrs

Parameters:

input_shape – Shape of input tensor
tensor_types – List of target tensor types to unpack

input_shape: Tuple[int]

tensor_types: List[afe.ir.tensor_type.TensorType]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.NormalizationTransformAttrs

Parameters:

channel_params – The list of tuples for (divisor, mean, standard deviation)
input_type – Type and shape of input tensor

channel_params: List[Tuple[float, float, float]]

input_type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.QuantizationTransformAttrs

Parameters:

channel_params – The list of tuples for (quant_scale, zero_point) Length of the list can be 1 for per tensor quantization or equal to the number of channels of input_shape for per channel quantization.
input_shape – Shape of input tensor, expected to be channel last
num_bits – The number of bits used for quantization
rounding – The rounding type for quantization
output_data_type – If number of bits is 8 data type is int8 otherwise int32.

channel_params: List[Tuple[float, int]]

input_shape: Tuple[int, int, int, int]

num_bits: int | None = 8

rounding: ml_kernels.math_helpers.RoundType

output_data_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.DequantizationTransformAttrs

Parameters:

channel_params – The list of tuples for (quant_scale, zero_point) Length of the list can be 1 for per tensor quantization or equal to the number of channels of input_shape for per channel quantization.
input_type – Type and shape of input tensor, expected to be channel last.
output_type – Type of output tensor.

channel_params: List[Tuple[float, int]]

input_type: afe.ir.tensor_type.TensorType

output_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ResizeTransformAttrs

Parameters:

target_height – Target height of resized tensor
target_width – Target width of resized tensor
keep_aspect – Boolean flag to keep aspect ratio
deposit_location – Enum to indicate deposit position of resized image
method – Enum to indicate supported interpolation methods
input_type – Input tensor shape and scalar type

target_height: int

target_width: int

keep_aspect: bool

deposit_location: afe.apis.defines.ResizeDepositLocation

method: afe.apis.defines.ResizeMethod

input_type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ChromaUpsampleTransformAttrs

Parameters:

frame_height – Height of full sampling frame
frame_width – Width of full sampling frame
yuv_sampling – Chroma sampling Enum

frame_height: int

frame_width: int

yuv_sampling: afe.apis.defines.ChromaSampling

input_type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.YuvRgbConversionTransformAttrs

Parameters:

conversion – Direction of conversion between YUV and RGB
std – Standard for color space conversion

conversion: afe.apis.defines.ColorConversion

std: afe.apis.defines.ColorSpaceStandard

input_shape: Tuple[int, int, int, int]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.BgrRgbConversionTransformAttrs

Parameters:: conversion – Direction of conversion between BGR and RGB

conversion: afe.apis.defines.ColorConversion

input_shape: Tuple[int, int, int, int]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.SigmoidTransformAttrs

Parameters:: save_int16 – Boolean flag to save output as 16-bit fixed point

save_int16: bool

input_shape: Tuple[int, int, int, int]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.NmsMaxpoolTransformAttrs

Parameters:: kernel – Size of pooling kernel

kernel: int

input_type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.CastAttrs

Parameters:

out_dtype – The data type of the target.
input_shape – Shape of input tensor.

out_dtype: str

input_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.UDFAttrs

Common attributes for UDF functions:

Sqrt
Rsqrt
Sigmoid
Exp
Tanh
log, log2, log10

input_shape: Tuple[int, Ellipsis]

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LeakyReluAttrs

Parameters:: alpha – The slope for the small gradient when x < 0

alpha: float

class afe.ir.attributes.SwishAttrs

Common attributes for UDF functions:

Sqrt
Rsqrt
Sigmoid
Exp
Tanh
log, log2, log10

class afe.ir.attributes.PReluAttrs

Parameters:

scalar_type – Type of input and output. Must be a floating-point type.
axis – The axis channel dimension is specified.
alpha – The slope for the small gradient when x < 0 (constant tensor)
input_shape – Shape of input.

scalar_type: afe.ir.tensor_type.ScalarType

axis: int

alpha: numpy.ndarray

input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ClipQuantAttrs

Attributes for Clip operation. Clip operation is always merged into a composite operator. Same class is used in floating-point and quantized version.

Parameters:

a_min – min value of clip
a_max – max calue of clip
shape – Shape of input tensor

a_min: int

a_max: int

shape: Tuple[int, Ellipsis]

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ReluQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

input_shape: Tuple[int, Ellipsis]

zero_point: int = 0

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

afe.ir.attributes.ACTIVATION_ATTRS

afe.ir.attributes.QUANT_ACTIVATION_ATTRS

class afe.ir.attributes.AddActivationAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

add_attrs: AddAttrs

activ_attrs: ACTIVATION_ATTRS | None = None

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConvAddActivationAttrs

Attributes of a fused convolution operator consisting of convolution, optional bias-add, and optional activation function.

weights_attrs: ConstantAttrs

conv_attrs: ConvAttrs

bias_attrs: ConstantAttrs | None = None

add_attrs: AddAttrs | BiasAddAttrs | None = None

activ_attrs: ACTIVATION_ATTRS | None = None

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.TupleConcatenateAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

tuple_attrs: TupleAttrs

concat_attrs: ConcatenateAttrs

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ExternalAttrs

Parameters:

external_input_list – Parameter names of the external code. This list must be equal to list(node_type.keys()).
node_type – The external operation’s type.
backend – The build target.
irmod_str – The TVM IRModule of the external code saved in string form. Code representations in other fields are derived from this one. It has batch size 1, regardless of batch_size.
operations – A list of strings that detail the ops that are contained within the IRModule.
_graph_module – Lazily compiled executable representation of the external code. This module is used for executing this node on the compilation host.
batch_size – The batch size that this node handles.

external_input_list: List[str]

node_type: afe.ir.tensor_type.NodeType

backend: str

irmod_str: str

operations: List[str]

batch_size: int = 1

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

property graph_module: afe._tvm._defines.TVMGraphModule

Get the operator’s code as a TVM module that can run on the compilation host.

Returns:: TVM Graph module

class afe.ir.attributes.QNNQuantizeAttrs

Further reference: tvm/src/relay/qnn/op/quantize.cc :param out_dtype: Specifies the output data type. :param axis: The channel axis for quantization. :param input_type: Tensor input type.

out_dtype: str

axis: int

input_type: afe.ir.tensor_type.TensorType

output_scale: numpy.ndarray

output_zero_point: numpy.ndarray

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.QNNDequantizeAttrs

Further reference: tvm/src/relay/qnn/op/dequantize.cc :param axis: The channel axis for quantization.

axis: int

input_type: afe.ir.tensor_type.TensorType

input_scale: numpy.ndarray

input_zero_point: numpy.ndarray

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.RequantizeAttrs

Further reference: tvm/src/relay/qnn/op/requantize.cc :param axis: The channel axis for quantization. This axis only apply to the input :param rounding: Defines the rounding direction when the value is midway

between two representable values.

Parameters:

compute_dtype – Specifies the data type used during requantize. Supported options: “int64”, “float32”, “float64”
out_dtype – Specifies the output data type.

axis: int

rounding: str

compute_dtype: str

out_dtype: str

input_type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.CustomOpAttrs

Custom Op AwesomeAttributes :param custom_op_attrs: Union[str, Dic[str, Union[str, bool]]]. Custom op attrs

in either str format or a dictionary

Parameters:

c_code_in_dtypes – Optional[List[str]]. Input tensors’ dtypes. This attribute will be assigned at the runtime
c_code_in_shapes – Optional[List[Tuple[int, …]]]. Input tensors’ shapes This attribute will be assigned at the runtime
function – Optional[OperatorFunction]. Compiled custom op C function This attribute will be assigned at the runtime
args_list – Optional[Any]. A list of arguments for the custom op C function. This attribute will be assigned at the runtime

output_types: List[afe.ir.tensor_type.TensorType]

custom_op_attrs: str | Dict[str, str | bool]

c_code_in_dtypes: List[str] | None = None

c_code_in_shapes: List[Tuple[int, Ellipsis]] | None = None

function: ml_kernels.c_function_call_helpers.OperatorFunction | None = None

args_list: Any | None = None

class afe.ir.attributes.AddQuantAttrs

Attributes for quantized AddActivationOp.

param lhs_scale: Scale correction applied to the left-hand side input. param rhs_scale: Scale correction applied to the right-hand side input. :param input_int16: If True, the inputs have int16 type. If false, the inputs have int8 type. :param requant: Requantization to perform on the output. :param relu_zero_point: Zero point of the output for relu activation. Ignored if

relu is not used.

param layer_bits: Number of bits used to quantize output tensor. param activ_attrs: Activation attributes used in Add composite operators.

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

input_int16: bool

requant: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

relu_zero_point: int = 0

lhs_scale: int = 1

rhs_scale: int = 1

layer_bits: int = 8

activ_attrs: QUANT_ACTIVATION_ATTRS | None = None

property node_scales: List[float]

property node_zps: List[int]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.SubtractQuantAttrs

param attrs: SubtractAttrs class holding SubtractOp parameters :param input_int16: If True, the inputs have int16 type. If False, the inputs have int8 type. param lhs_scale: Scale correction applied to the left-hand side input. param rhs_scale: Scale correction applied to the right-hand side input. param layer_bits: Number of bits used to quantize output tensor.

attrs: SubtractAttrs

input_int16: bool

requant: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

lhs_scale: int = 1

rhs_scale: int = 1

layer_bits: int = 8

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConvQuantAttrs

Used for all variants of convolution. The attributes describe the following sequence of operators (some are optional). Relu and clip are mutually exclusive.

convolution
bias_add
requantize
relu/clip

Due to limitations of how the backend is implemented, we cannot allow the combination zero_point != 0 and isinstance(activ_attrs, ReluAttrs) and isinstance(requant, ArithFoldedRequantization). The quantizer must conform to this restriction.

Parameters:

conv_attrs – Attributes of the convolution operator.
weight_quant_data – Quantized weights data.
scale – Scale of the convolution operation.
zero_point – Zero point of the quantized output tensor.
input_zp – Zero point of input to the convolution.
bias_quant_data – Quantized bias data.
weight_bits – Number of bits used to quantize the weights.
bits – Number of bits used for quantization.
per_channel – If true, each output channel of the weights will have an independent scale.
activ_attrs – Activation attributes.
requant – Requantization to do after convolve and add.
input_int16 – Whether the input tensor has int16 type. If true, then the operator will execute using the 15-bit convolution algorithm.
msb_left_shift – Whether the 15-bit convolution algorithm will left-shift the MSB (effectively right-shifting the full product by 1). If false, it will right-shift the LSB (effectively right-shifting the full product by 8). Ignored if input_int16 is False.
c_block_size – Size of one c block if block quantization is used. None otherwise.

conv_attrs: ConvAttrs

weight_quant_data: numpy.ndarray

requant: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

scale: float = 1.0

zero_point: int = 0

input_zp: int = 0

bias_quant_data: numpy.ndarray | None = None

per_channel: bool = False

activ_attrs: QUANT_ACTIVATION_ATTRS | None = None

input_int16: bool = False

msb_left_shift: bool | numpy.ndarray = True

c_block_size: int | None = None

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.UpsamplingQuantAttrs

Parameters:

input_zp
rounding_type

upsampling_attrs: UpsamplingAttrs

input_zp: int = 0

input_scale: float = 1.0

rounding_type: ml_kernels.math_helpers.RoundType

input_int16: bool = False

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ImageResize2DQuantAttrs

Parameters:

input_zp
rounding_type
requant – Requantization to perform on the output.
input_int16 – If True, the inputs have int16 type. If False, the inputs have int8 type.

image_resize2d_attrs: ImageResize2DAttrs

input_int16: bool = False

input_zp: int = 0

input_scale: float = 1.0

rounding_type: ml_kernels.math_helpers.RoundType

requant: ml_kernels.requantization.BaseRequantization | None = None

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LRNQuantAttrs

Parameters:

axis – Input data layout channel axis. Default value is 1 for NCHW format
size – The size of the local region to be considered for normalization.
lut_scale – The scale for quantization of LUT input .
lut_zp_corr – The zp correction for quantization of LUT input .
lut_sh – The shift for quantization of LUT input .
output_scale – The scale for quantization of output.
output_zp_corr – The zp correction for quantization of output.
output_sh – The shift for quantization of output.

# NOTES FOR TENSORFLOW # TVM defines size as size_tvm = (depth_radius_tf * 2) + 1 # TVM defines alpha as alpha_tvm = alpha_tf * size_tf

axis: int

size: int

shape: Tuple[int, Ellipsis]

input_zp: int

lut_scale: int

lut_zp_corr: int

lut_sh: int

output_scale: int

output_zp_corr: int

output_sh: int

lookup_table: numpy.ndarray

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LayerNormQuantAttrs

Parameters:

axis – Indicates the dimension along which LayerNorm will be performed.
input_shape – Input shape.
lookup_table_rsqrt – Look-up table f(x) = 1 / sqrt(x + epsilon).
zp_rsqrt – Output zero point of the Rsqrt LUT.
requant_mean – Requantization parameters for input mean (integer inputs only).
requant_lut_input – Requantization parameters for Rsqrt LUT input.
requant_output – Requantization of final output.

axis: int

input_shape: Tuple[int, Ellipsis]

zp_rsqrt: int

lookup_table_rsqrt: numpy.ndarray

requant_mean: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

requant_lut_input: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

requant_output: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.InstanceNormQuantAttrs

Quantized Instance Normalization operator attributes.

attrs: InstanceNorm attributes.

lut_rsqrt: Look-up table f(x) = 1 / sqrt(x + epsilon).

zp_rsqrt: Output zero point of the Rsqrt LUT.

requant_out: Requantization of the output.

attrs: InstanceNormAttrs

lut_rsqrt: numpy.ndarray

zp_rsqrt: int

requant_out: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

class afe.ir.attributes.RMSNormQuantAttrs

Parameters:

input_shape – Input shape.
zp_ifm – Input tensor zero point.
lookup_table_rsqrt – Look-up table f(x) = 1 / sqrt(x + epsilon).
zp_rsqrt – Output zero point of the Rsqrt LUT.
requant_lut_input – Requantization parameters for Rsqrt LUT input.
requant_output – Requantization of final output.
lut_input_pre_shift – LUT input requantization pre-shift value.
output_pre_shift – Output requantization pre-shift value.
enable_lut_int16 – If True, quantize LUT to int16 otherwise to int8.

input_shape: Tuple[int, Ellipsis]

zp_ifm: int

lookup_table_rsqrt: numpy.ndarray

zp_rsqrt: int

requant_lut_input: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

requant_output: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

lut_input_pre_shift: int

output_pre_shift: int

enable_lut_int16: bool

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.SoftmaxQuantAttrs

Parameters:

axis – Input data layout channel axis.
input_shape – Input shape.
exp_zp – Exp zero point.
rec_zp – Rec zero point.
requant_lut – Requantization parameters for quantization of reciprocal LUT input.
requant_output – Requantization parameters for output.
lookup_table_exp – LUT for exponential function.
lookup_table_rec – LUT for reciprocal function.
enable_int16 – Whether int8 or int16 quantization is used.
lut_input_pre_shift – LUT input requantization pre-shift value (int16 only).
output_pre_shift – Output requantization pre-shift value (int16 only).

axis: int

input_shape: Tuple[int, Ellipsis]

exp_zp: int

rec_zp: int

requant_lut: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

requant_output: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

lookup_table_exp: numpy.ndarray

lookup_table_rec: numpy.ndarray

enable_int16: bool

lut_input_pre_shift: int | None = None

output_pre_shift: int | None = None

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.RequantizeQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

attrs: RequantizeAttrs

requant: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConcatQuantAttrs

Contains quantization attributes for concatenate quantization.

Parameters:

attrs – ConcatenateAttrs holding ConcatenateOp parameters.
requants – Requantization parameters
layer_bits – Number of bits used for quantizing the tensor.
input_scales – Quantized scale for eash inputs.
node_scales – Using the max input_scales as the concatenate output scale of the quantized output tensors(s).
node_zps – Zero points(s) of the quantized output tensors(s)

attrs: ConcatenateAttrs

requants: List[ml_kernels.requantization.BaseRequantization[numpy.ndarray]]

layer_bits: List[int] = [8]

input_scales: List[float | List[float]] = []

node_scales: List[float] = []

node_zps: List[int] = None

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.CustomOpQuantAttrs

Contains quantization attributes for custom operation quantization.

Parameters:

custom_op_attrs – CustomOp attributes.
layer_bits – Number of bits used for quantizing the tensor.
node_zps – Zero points(s) of the quantized output tensors(s)
node_scales – Output scale of the quantized output tensors(s).
input_zps – Quantized zero points correction each input.
input_scales – Quantized scales for each input.

custom_op_attrs: CustomOpAttrs

layer_bits: List[int] = [8]

node_zps: List[int] = []

node_scales: List[float] = []

input_zps: List[int | List[int]] = []

input_scales: List[float | List[float]] = []

class afe.ir.attributes.PoolQuantAttrs

Contains quantization attributes for pool quantization.

Parameters:

pool_attrs – Pool attrs class holding MaxPool/AvgPoll operator parameters. Its scalar type does not determine the scalar type for the quantized operator.
pad_value – Padding value.
rounding_type – RoundType.

pool_attrs: PoolAttrs

pad_value: float | int

rounding_type: ml_kernels.math_helpers.RoundType

input_int16: bool

requant: ml_kernels.requantization.BaseRequantization | None = None

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.VarianceQuantAttrs

attrs: Variance attributes.

requant: Requantiation of the intermediate values.

requant_var: Requantization of the Variance operator final output.

attrs: VarianceAttrs

requant: ml_kernels.requantization.BaseRequantization

requant_var: ml_kernels.requantization.BaseRequantization

class afe.ir.attributes.UDFQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

attrs: UDFAttrs

input_signed: bool = False

output_signed: bool = False

lookup_table: numpy.ndarray | None = None

input_int16: bool = False

requant: ml_kernels.requantization.BaseRequantization | None = None

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.DivideAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

udf_attrs: UDFAttrs

multiply_attrs: MultiplyAttrs

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.DivideQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

udf_attrs: UDFQuantAttrs

multiply_attrs: MultiplyQuantAttrs

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.HardSigmoidAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

input_shape: tuple[int, Ellipsis]

scalar_type: afe.ir.tensor_type.ScalarType

alpha: float

beta: float

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LeakyReluCompositeQuantAttrs

Contains quantization attributes for both UDF and breakdown LeakyRelu quantization. :param attrs: LeakyRelu attributes class holding LeakyReluOp parameters. :param leaky_relu_uses_udf: bool. If True, use UDF version in quantization. Otherwise, use breakdown version. :param leaky_relu_quant_attrs: Contains quantization parameters for breakdown version if breakdown version is used. :param udf_quant_attrs: Contains quantization parameters for UDF version if UDF version is used.

attrs: LeakyReluAttrs

leaky_relu_uses_udf: bool = True

leaky_relu_quant_attrs: LeakyReluQuantAttrs | None = None

udf_quant_attrs: UDFQuantAttrs | None = None

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.PReluQuantAttrs

The slope for quantized_intput < zero_point is (alpha >> right_shift)

axis: int

input_shape: Tuple[int, Ellipsis]

quant_alpha: numpy.ndarray

alpha_shift: int = 0

data_zero_point: int = 0

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.PowerAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ArgMaxQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

attrs: ArgMaxAttrs

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.BatchMatmulAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

transpose_b: bool

input_shapes: List[Tuple[int, Ellipsis]]

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.

get_output_shape() → Tuple[int, Ellipsis]

class afe.ir.attributes.BatchMatmulQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

attrs: BatchMatmulAttrs

lhs_zp: int

rhs_zp: int

requant: ml_kernels.requantization.BaseRequantization

intrinsic_shift: int

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size. Should be implemented separately inside each class that inherits from AwesomeQuantAttrBase.

class afe.ir.attributes.SliceConcatAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

slice_attrs: List[StridedSliceAttrs]

tuple_concat_attrs: TupleConcatenateAttrs

class afe.ir.attributes.SliceConcatQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

slice_attrs: List[StridedSliceAttrs]

tuple_concat_attrs: ConcatQuantAttrs

class afe.ir.attributes.BroadcastToQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

input_type: afe.ir.tensor_type.TensorType

output_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int): Modify internal parameters’ shapes for the given batch size.