afe.ir.attributes

Attributes

DEFAULT_PER_CHANNEL
ASYMMETRY
dummy_quant_result_tensor_type
ACTIVATION_ATTRS
QUANT_ACTIVATION_ATTRS

Classes

ObservedDistribution	A value distribution that was observed during calibration. This value
QuantResultTensorType	The result of running the quantization transformation on a tensor.
AwesomeCalibAttrs	Calibration attributes
AwesomeQuantAttrBase	Base class of quantized operator attributes. This class is used for instance
PlaceholderQuantAttrs	Properties of a quantized placeholder.
ConstantQuantAttrs
ConcatQuantAttrs	Contains quantization attributes for concatenate quantization.
MultiplyQuantAttrs	param lhs_input_shape: Lhs input shape
LeakyReluQuantAttrs	The slope for quantized_intput < zero_point is (alpha >> right_shift)
AwesomeAttributes	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
PlaceholderAttrs	Properties of a placeholder.
ConstantAttrs
MultiplyAttrs	Attributes of a multiply operator.
ConvAttrs	Attributes of a convolution operation.
PoolAttrs
MaxPoolAttrs
AvgPoolAttrs
VarianceAttrs
AdaptiveAvgPool2DAttrs
ReluAttrs
AddAttrs	Attributes of an add operator.
SubtractAttrs	Attributes of a subtract operator.
BiasAddAttrs
ConstantMultiplyAddAttrs	Attributes representing the computation (a*c + b*d) for scalar constants c and d.
MeanAttrs
MeanQuantAttrs	Contains quantization attributes for mean quantization.
ArgMaxAttrs
LayerNormAttrs
RMSNormAttrs
InstanceNormAttrs	Instance Normalization operator attributes.
SoftmaxAttrs
PadAttrs
LRNAttrs
ClipAttrs	Attributes for Clip operation. Clip operation is always merged into a composite operator.
ExtmAttrs	Attributes for extremum op, can be min or max op depending on the max boolean.
SumAttrs
ProdAttrs
FullAttrs
TileAttrs
UpsamplingAttrs
ImageResize2DAttrs
GridSampleAttrs	Attributes of GridSample operator.
MaximumAttrs	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
MinimumAttrs	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
TensorManipulationBaseAttrs	Do nothing. Used for better structuring data structure
TupleAttrs	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
TupleGetItemAttrs
SqueezeAttrs
ConcatenateAttrs
TransposeAttrs
DepthToSpaceAttrs	Attributes of DepthToSpace operator
ReshapeAttrs
ExpandDimsAttrs
BatchFlattenAttrs	Do nothing. Used for better structuring data structure
SplitAttrs
TakeAttrs
StridedSliceAttrs
LayoutTransformAttrs
BroadcastToAttrs	Do nothing. Used for better structuring data structure
TessellationTransformAttrs
DetessellationTransformAttrs
PackTransformAttrs
UnpackTransformAttrs
NormalizationTransformAttrs
QuantizationTransformAttrs
DequantizationTransformAttrs
ResizeTransformAttrs
ChromaUpsampleTransformAttrs
YuvRgbConversionTransformAttrs
BgrRgbConversionTransformAttrs
SigmoidTransformAttrs
NmsMaxpoolTransformAttrs
CastAttrs
UDFAttrs	Common attributes for UDF functions:
LeakyReluAttrs
SwishAttrs	Common attributes for UDF functions:
PReluAttrs
ClipQuantAttrs	Attributes for Clip operation. Clip operation is always merged into a composite operator.
ReluQuantAttrs	Base class of quantized operator attributes. This class is used for instance
AddActivationAttrs	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
ConvAddActivationAttrs	Attributes of a fused convolution operator consisting of convolution, optional bias-add,
TupleConcatenateAttrs	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
ExternalAttrs
QNNQuantizeAttrs	Further reference: tvm/src/relay/qnn/op/quantize.cc
QNNDequantizeAttrs	Further reference: tvm/src/relay/qnn/op/dequantize.cc
RequantizeAttrs	Further reference: tvm/src/relay/qnn/op/requantize.cc
CustomOpAttrs	Custom Op AwesomeAttributes
AddQuantAttrs	Attributes for quantized AddActivationOp.
SubtractQuantAttrs	param attrs: SubtractAttrs class holding SubtractOp parameters
ConvQuantAttrs	Used for all variants of convolution.
UpsamplingQuantAttrs
ImageResize2DQuantAttrs
LRNQuantAttrs
LayerNormQuantAttrs
InstanceNormQuantAttrs	Quantized Instance Normalization operator attributes.
RMSNormQuantAttrs
SoftmaxQuantAttrs
RequantizeQuantAttrs	Base class of quantized operator attributes. This class is used for instance
ConcatQuantAttrs	Contains quantization attributes for concatenate quantization.
CustomOpQuantAttrs	Contains quantization attributes for custom operation quantization.
PoolQuantAttrs	Contains quantization attributes for pool quantization.
VarianceQuantAttrs
UDFQuantAttrs	Base class of quantized operator attributes. This class is used for instance
DivideAttrs	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
DivideQuantAttrs	Base class of quantized operator attributes. This class is used for instance
LeakyReluCompositeQuantAttrs	Contains quantization attributes for both UDF and breakdown LeakyRelu quantization.
PReluQuantAttrs	The slope for quantized_intput < zero_point is (alpha >> right_shift)
PowerAttrs	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
ArgMaxQuantAttrs	Base class of quantized operator attributes. This class is used for instance
BatchMatmulAttrs	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
BatchMatmulQuantAttrs	Base class of quantized operator attributes. This class is used for instance
SliceConcatAttrs	A class that stores attributes necessary for the execution of its associated AwesomeOperation.
SliceConcatQuantAttrs	Base class of quantized operator attributes. This class is used for instance
BroadcastToQuantAttrs	Base class of quantized operator attributes. This class is used for instance

Functions

get_quant_result_scale_with_dummy(...)	Get the quantization scale; if there is none, return a dummy value.
get_data_value_quant_result_scale_with_dummy(...)	Run get_quant_result_scale_with_dummy on the contents of a DataValue.
get_dict_quant_result_scale_with_dummy(...)	Run get_quant_result_scale_with_dummy on the contents of a dict of DataValue.
update_quant_result_quantization(...)	Insert the given quantization into t, replacing existing quantization values in t.
update_quant_result_type(...)	Ensure that t's type matches new_type by replacing dummy types with data from t and
set_quant_result_type_batch_size(...)	Modifies DataValue of QuantResultTensorType with given batch size.
is_dummy_type(→ bool)
convolution_output_shape(→ tuple[int, Ellipsis])	Get the shape of a convolution's output tensor based on its attributes.

Module Contents

afe.ir.attributes.DEFAULT_PER_CHANNEL = False

afe.ir.attributes.ASYMMETRY = True

class afe.ir.attributes.ObservedDistribution(observer: afe.ir.node_observer.NodeObserver)

A value distribution that was observed during calibration. This value distribution can be used to decide how to quantize a tensor.

calculate_quantization(qrange: Tuple[int, int]) → afe.ir.defines.DataValue[afe.ir.defines.Quantization]

Choose a quantization to use for representing the observed value distribution using the given integer range.

Parameters:

qrange – Integer range to quantize for. The range must be representable by an 8-bit or 16-bit signed integer.

Returns:

Selected quantizations.

get_min_max() → Tuple[float, float]

Get range (min and max) of observed values. Min-max range does not need to include zero.

Returns:

Tuple of min and max values.

get_mean()

class afe.ir.attributes.QuantResultTensorType

The result of running the quantization transformation on a tensor. It has the tensor’s type and quantization, as they are after the transformation. Only tensors that are quantized by the transformation have a quantization.

Parameters:

• type – The tensor’s type after transformation. It has the same shape as before the transformation. Its scalar type may be different.

• quant – The tensor’s quantization, if it was quantized by the quantization transformation. None otherwise. Floating-point tensors do not have a quantization. Integer tensors do not have a quantization if they were already integer before the quantization transformation.

• requant_method – The method that should be used for requantizing this tensor’s value when requantization is needed. This field must be None iff quant is None.

type: afe.ir.tensor_type.TensorType

quant: afe.ir.defines.Quantization | None

requant_method: afe.ir.defines.RequantMethod | None

static from_type(type: afe.ir.tensor_type.TensorType) → QuantResultTensorType

Make a QuantResultTensorType that only has type information.

static from_quant(quant: afe.ir.defines.Quantization | None) → QuantResultTensorType

Make a QuantResultTensorType from a Quantization using dummy type information. This is a temporary method that should be removed when support for QuantResultTensorType is finished.

afe.ir.attributes.get_quant_result_scale_with_dummy(t: QuantResultTensorType) → afe.ir.defines.Quantization

Get the quantization scale; if there is none, return a dummy value.

The dummy value is a temporary solution that should be removed when support for QuantResultTensorType is finished.

afe.ir.attributes.get_data_value_quant_result_scale_with_dummy(t: afe.ir.defines.DataValue[QuantResultTensorType]) → afe.ir.defines.DataValue[afe.ir.defines.Quantization]

Run get_quant_result_scale_with_dummy on the contents of a DataValue.

afe.ir.attributes.get_dict_quant_result_scale_with_dummy(t: Dict[afe.ir.defines.InputName, afe.ir.defines.DataValue[QuantResultTensorType]]) → Dict[afe.ir.defines.InputName, afe.ir.defines.DataValue[afe.ir.defines.Quantization]]

Run get_quant_result_scale_with_dummy on the contents of a dict of DataValue.

afe.ir.attributes.update_quant_result_quantization(t: afe.ir.defines.DataValue[QuantResultTensorType], new_quant: afe.ir.defines.DataValue[afe.ir.defines.Quantization]) → afe.ir.defines.DataValue[QuantResultTensorType]

Insert the given quantization into t, replacing existing quantization values in t.

Parameters:

• t – Quantization result type to modify

• new_quant – Quantization values

Returns:

A copy of t with values from new_type inserted

afe.ir.attributes.update_quant_result_type(t: afe.ir.defines.DataValue[QuantResultTensorType], new_type: afe.ir.defines.DataValue[afe.ir.tensor_type.TensorType]) → afe.ir.defines.DataValue[QuantResultTensorType]

Ensure that t’s type matches new_type by replacing dummy types with data from t and checking non-dummy types.

This function’s purpose is to save the type into t while developing QuantResultTensorType, then to check consistency after it is developed.

Parameters:

• t – Quantization result type, which may contain dummy types

• new_type – Type that should be the same as the type in t

Returns:

A copy of t with values from new_type inserted to replace any dummy types

afe.ir.attributes.set_quant_result_type_batch_size(t: afe.ir.defines.DataValue[QuantResultTensorType], batch_size: int) → afe.ir.defines.DataValue[QuantResultTensorType]

Modifies DataValue of QuantResultTensorType with given batch size.

Parameters:

• t – DataValue[QuantResultTensorType]. Value to be modified.

• batch_size – int. Batch size value to be used in constructing new QuantResultTensorType DataValue.

Returns:

DataValue[QuantResultTensorType]. QuantResultTensorType with its type’s shape field modified to use batch_size.

afe.ir.attributes.dummy_quant_result_tensor_type

afe.ir.attributes.is_dummy_type(t: afe.ir.tensor_type.TensorType) → bool

class afe.ir.attributes.AwesomeCalibAttrs

Calibration attributes :param observer: Observer used during calibration of the node. If the node does not use

calibration data for calculation of quantization parameters, observer will not be created and its value will be None.

Parameters:

• intermediate_observers – Observers used for quantization of intermediate results.

• quant – Quantization scale of the output. It is assigned during quantization.

• input_quant – Quantization scale of each input. During quantization, it is first assigned the type and quantization scale that were determined at the nodes that compute the inputs. Then, when the node is quantized, it is assigned the types and quantization scales of inputs that the node accepts.

observer: afe.ir.node_observer.NodeObserver | None = None

intermediate_observers: Dict[str, afe.ir.node_observer.NodeObserver] | None = None

quant: afe.ir.defines.DataValue[QuantResultTensorType]

input_quant: Dict[afe.ir.defines.InputName, afe.ir.defines.DataValue[QuantResultTensorType]]

precomputed_quant: afe.ir.defines.Quantization | None = None

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.AwesomeQuantAttrBase

Base class of quantized operator attributes. This class is used for instance checking only.

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size. Should be implemented separately inside each class that inherits from AwesomeQuantAttrBase.

class afe.ir.attributes.PlaceholderQuantAttrs

Properties of a quantized placeholder.

Parameters:

• type – Type of the placeholder’s output.

• quantization – Quantization of the placeholder, if it was quantized by the Quantize compiler pass.

type: afe.ir.tensor_type.TensorType

quantization: afe.ir.defines.Quantization | None

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConstantQuantAttrs

Parameters:

quant_data – Quantized tensor value

quant_data: numpy.ndarray

class afe.ir.attributes.ConcatQuantAttrs

Contains quantization attributes for concatenate quantization.

Parameters:

• input_scale_corrections – Quantized scale correction for each inputs.

• input_zp_corrections – Quantized zero point correction for each inputs.

• right_shift – Number of bits in right shift during requantize at inference time.

• layer_bits – Number of bits used for quantizing the tensor.

• axis – The axis along which the tensors are concatenated.

• node_zps – Zero points(s) of the quantized output tensors(s)

• input_scales – Quantized scale for eash inputs.

• node_scales – Using the max input_scales as the concatenate output scale of the quantized output tensors(s).

input_scale_corrections: List[int] = []

input_zp_corrections: List[int] = []

right_shifts: List[int] = []

layer_bits: List[int] = [8]

axis: int | None = None

input_scales: List[float | List[float]] | None = []

node_scales: List[float] | None = []

node_zps: List[int] | None = None

rounding_type: ml_kernels.math_helpers.RoundType

class afe.ir.attributes.MultiplyQuantAttrs

param lhs_input_shape: Lhs input shape param rhs_input_shape: Rhs input shape :param input_int16: If True, the inputs have int16 type. If false, the inputs have int8 type. :param intrinsic_shift: Right-shift to apply before requantization. param requant Requantization parameters param lhs_zero_point: Zero point of the left-hand side input. param rhs_zero_point: Zero point of the right-hand side input. param layer_bits: Number of bits used to quantize output tensor.

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

input_int16: bool

intrinsic_shift: int

requant: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

lhs_zero_point: int = 0

rhs_zero_point: int = 0

layer_bits: int = 8

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LeakyReluQuantAttrs

The slope for quantized_intput < zero_point is (alpha >> right_shift)

alpha: int

input_shape: Tuple[int, Ellipsis]

right_shift: int

zero_point: int

bits: int

rounding_type: ml_kernels.math_helpers.RoundType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.AwesomeAttributes

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size. Should be implemented separately inside each class that inherits from AwesomeAttributes.

class afe.ir.attributes.PlaceholderAttrs

Properties of a placeholder.

type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConstantAttrs

Parameters:

data – Tensor value before quantization

data: numpy.ndarray

class afe.ir.attributes.MultiplyAttrs

Attributes of a multiply operator.

Parameters:

• scalar_type – Type of input and output. Must be a floating-point type.

• lhs_input_shape – Shape of first input.

• rhs_input_shape – Shape of second input.

scalar_type: afe.ir.tensor_type.ScalarType

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

afe.ir.attributes.convolution_output_shape(conv_attrs: ConvAttrs) → tuple[int, Ellipsis]

Get the shape of a convolution’s output tensor based on its attributes.

class afe.ir.attributes.ConvAttrs

Attributes of a convolution operation.

The attributes describe a convolution with input and output activations in NWC, NHWC, or NDHWC layout and weights in WIGO, HWIGO, or DHWIGO layout.

The dimension order for 1, 2, or 3 spatial dimensions respectively is W, HW, or DHW.

Parameters:

• stride – Stride in each spatial dimension

• dilation – Dilation in each spatial dimension

• padding – Padding in each spatial dimension. The padding in each dimension is a tuple holding the padding width at the beginning and end of the dimension.

• output_padding – Padding of the output tensor in each spatial dimension for transposed convolution. If it is not a transposed convolution, all padding values must be zero. If it is a transposed convolution, the first element of the padding must be zero.

• is_transposed – Whether it is a transposed convolution.

• weight_shape – Shape of the weight tensor in spatial dimensions ++ IGO layout, for example HWIGO. IGO is an abbreviation for “input channels, groups, output channels”.

• input_spatial_shape – Shape of the input tensor in spatial dimensions.

• batch_size – Batch size.

• input_type – Scalar type of the convolution’s input tensor. This type is ignored for quantized convolutions.

stride: tuple[int, Ellipsis]

dilation: tuple[int, Ellipsis]

padding: tuple[tuple[int, int], Ellipsis]

output_padding: tuple[tuple[int, int], Ellipsis]

is_transposed: bool

weight_shape: tuple[int, Ellipsis]

input_spatial_shape: tuple[int, Ellipsis]

batch_size: int

input_type: afe.ir.tensor_type.ScalarType

property groups: int

Get the number of convolution groups.

property channels: int

Get the number of convolution output channels.

property input_channels: int

Get the number of convolution input channels.

property kernel_size: tuple[int, Ellipsis]

Get the shape of the convolution kernel in the spatial dimensions.

property num_spatial_dimensions: int

Get the number of spatial dimensions for this convolution.

property input_shape: tuple[int, Ellipsis]

Get the shape of the convolution’s input tensor in NWC, NHWC, or NDHWC layout.

property output_shape: tuple[int, Ellipsis]

Get the shape of the convolution’s output tensor in NWC, NHWC, or NDHWC layout.

property is_depthwise_one_channel: bool

Return true if this convolution is a depthwise convolution with equal number of input and output channels.

class afe.ir.attributes.PoolAttrs

Parameters:

• ceil_mode – Used to take ceil or floor when computing the output shape

• out_layout – Layout of the output. This can be an empty str if layout is the same as data_layout.

• layout – Uses the letters NHWC for BatchNumber, Height, Width, Channels

• padding – ((pad_top, pad_bot), …) along the dimensions of NHWC according to layout

• pool_size – Size of pooling

• strides – Strides

• dilation – Dilation along the dimensions of NHWC according to data_layout

• scalar_type – Data type of the input and output.

ceil_mode: int

out_layout: str

layout: str

padding: afe.ir.defines.AwesomePad

pool_size: afe.ir.defines.AwesomePoolSize

strides: afe.ir.defines.AwesomeStrides

dilation: afe.ir.defines.AwesomeDilation

input_shape: afe.ir.defines.AwesomePoolSize

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.MaxPoolAttrs

Parameters:

• ceil_mode – Used to take ceil or floor when computing the output shape

• out_layout – Layout of the output. This can be an empty str if layout is the same as data_layout.

• layout – Uses the letters NHWC for BatchNumber, Height, Width, Channels

• padding – ((pad_top, pad_bot), …) along the dimensions of NHWC according to layout

• pool_size – Size of pooling

• strides – Strides

• dilation – Dilation along the dimensions of NHWC according to data_layout

• scalar_type – Data type of the input and output.

class afe.ir.attributes.AvgPoolAttrs

Parameters:

count_include_pad – If true, include padding to compute the average.

count_include_pad: bool

class afe.ir.attributes.VarianceAttrs

input_data_shape

Shape of the input tensor.

mean_shape

Shape of the mean input tensor.

scalar_type

Scalar type of the input and output.

axis

The axes to sum over when computing mean.

input_data_shape: tuple[int, Ellipsis]

mean_shape: tuple[int, Ellipsis]

scalar_type: afe.ir.tensor_type.ScalarType

axis: tuple[int, Ellipsis]

class afe.ir.attributes.AdaptiveAvgPool2DAttrs

Parameters:

• output_size – tuple of int. optional Output height and width.

• out_layout – Layout of the output. This can be an empty str if layout is the same as data_layout.

• layout – Layout of the input.

output_size: Tuple[int, Ellipsis]

out_layout: str

layout: str

class afe.ir.attributes.ReluAttrs

Parameters:

• scalar_type – Type of input and output.

• input_shape – Shape of input.

scalar_type: afe.ir.tensor_type.ScalarType

input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.AddAttrs

Attributes of an add operator.

Parameters:

• scalar_type – Type of input and output. Must be a floating-point type.

• lhs_input_shape – Shape of first input.

• rhs_input_shape – Shape of second input.

scalar_type: afe.ir.tensor_type.ScalarType

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.SubtractAttrs

Attributes of a subtract operator.

Parameters:

• scalar_type – Type of input and output. Must be a floating-point type.

• lhs_input_shape – Shape of first input.

• rhs_input_shape – Shape of second input.

scalar_type: afe.ir.tensor_type.ScalarType

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.BiasAddAttrs

Parameters:

• input_shape – The shape of the input activation tensor

• axis – The axis to add the bias

input_shape: Tuple[int, Ellipsis]

axis: int

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConstantMultiplyAddAttrs(scalar_type: afe.ir.tensor_type.ScalarType, lhs_input_shape: Tuple[int, Ellipsis], rhs_input_shape: Tuple[int, Ellipsis], in1_const_attrs: ConstantAttrs, in2_const_attrs: ConstantAttrs | None = None)

Attributes representing the computation (a*c + b*d) for scalar constants c and d.

scalar_type: afe.ir.tensor_type.ScalarType

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

in1_const_attrs: ConstantAttrs

in2_const_attrs: ConstantAttrs | None

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.MeanAttrs

Parameters:

• axis – Axis or axes along which a mean operation is performed.

• exclude – If exclude is true, we use the axes that are NOT in the axis field

• keepdims – If set to true the axes reduces are left with a size of 1

axis: List[int]

exclude: bool

keepdims: bool

shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.MeanQuantAttrs

Contains quantization attributes for mean quantization.

Parameters:

• attrs – MeanAttrs used in mean operator.

• node_scales – Scales(s) of the quantized output tensors(s).

• node_zps – Zero points(s) of the quantized output tensors(s).

attrs: MeanAttrs

node_scales: float = 1.0

node_zps: int = 0

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ArgMaxAttrs

Parameters:

• axis – Axis or axes along which a mean operation is performed.

• exclude – If exclude is true, we use the axes that are NOT in the axis field

• keepdims – If set to true the axes reduces are left with a size of 1

• select_last_index – Whether to select the last index or the first index if the max element appears in multiple indices.

• shape – Shape of input tensor

• result_scalar_type – Type of numbers in result tensor. It must be either ScalarType.int32 or the same as the input tensor’s type.

• input_scalar_type – Type of input values. It must be either ScalarType.float32 or ScalarType.int8.

axis: List[int]

exclude: int

keepdims: int

shape: Tuple[int, Ellipsis]

select_last_index: bool

result_scalar_type: afe.ir.tensor_type.ScalarType

input_scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LayerNormAttrs

Parameters:

• axis – The axis to sum over when computing mean.

• input_shape – Shape of input tensor.

• epsilon – The epsilon value to use to avoid division by zero.

• scalar_type – Type of input and output.

axis: int | tuple[int, int]

input_shape: tuple[int, Ellipsis]

epsilon: float

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.RMSNormAttrs

Parameters:

• input_shape – Shape of input tensor.

• epsilon – The epsilon value to use to avoid division by zero.

• scalar_type – Type of input and output.

input_shape: Tuple[int, Ellipsis]

epsilon: float

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.InstanceNormAttrs

Instance Normalization operator attributes.

axis

The axes to sum over when computing mean.

input_data_shape

Shape of the input tensor.

mean_shape

Shape of the mean input tensor.

variance_shape

Shape of the variance input tensor.

epsilon

The epsilon value to use to avoid division by zero.

scalar_type

Type of input and output.

axis: tuple[int, int]

input_data_shape: tuple[int, Ellipsis]

mean_shape: tuple[int, Ellipsis]

variance_shape: tuple[int, Ellipsis]

epsilon: float

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.SoftmaxAttrs

Parameters:

• axis – The axis to sum over when computing softmax

• input_shape – Shape of input tensor

• scalar_type – Type of input and output

axis: int

input_shape: Tuple[int, Ellipsis]

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.PadAttrs

Parameters:

• pad_mode – ‘constant’, ‘edge’, ‘reflect’

• pad_width – padding along each input dimension N in the format of (before_N, after_N)

pad_mode: str

pad_width: afe.ir.defines.AwesomePad

input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LRNAttrs

Parameters:

• alpha – The scaling parameter.

• axis – Input data layout channel axis. Default value is 1 for NCHW format

• beta – The exponent parameter.

• bias – The offset parameter to avoid dividing by 0.

• size – The size of the local region to be considered for normalization.

• shape – Shape of input tensor

# NOTES FOR TENSORFLOW # TVM defines size as size_tvm = (depth_radius_tf * 2) + 1 # TVM defines alpha as alpha_tvm = alpha_tf * size_tf

alpha: float

axis: int

beta: float

bias: float

size: int

shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ClipAttrs

Attributes for Clip operation. Clip operation is always merged into a composite operator. Same class is used in floating-point and quantized version.

Parameters:

• a_min – min value of clip

• a_max – max calue of clip

• shape – Shape of input tensor

a_min: float

a_max: float

shape: Tuple[int, Ellipsis]

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ExtmAttrs

Attributes for extremum op, can be min or max op depending on the max boolean.

Parameters:

• axis – Axis or axes along which a mean operation is performed.

• exclude – If exclude is true, we use the axes that are NOT in the axis field

• keepdims – If set to true the axes reduces are left with a size of 1

• max – If true the operation is max, if false the operation is min.

axis: List[int]

exclude: int

keepdims: int

shape: Tuple[int, Ellipsis]

max: bool

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.SumAttrs

Parameters:

• axis – Axis or axes along which a mean operation is performed.

• exclude – If exclude is true, we use the axes that are NOT in the axis field

• keepdims – If set to true the axes reduces are left with a size of 1

• num_element – Number of element to be summed. This attribute is not a default TVM attribute. It will be assigned during any floating point inference and used in quantization.

axis: List[int]

exclude: int

keepdims: int

shape: Tuple[int, Ellipsis]

num_element: int

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ProdAttrs

Parameters:

• axis – Axis or axes along which a mean operation is performed.

• exclude – If exclude is true, we use the axes that are NOT in the axis field

• keepdims – If set to true the axes reduces are left with a size of 1

axis: List[int]

exclude: int

keepdims: int

shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.FullAttrs

Parameters:

• shape – The shape of the target.

• dtype – The data type of the target.

shape: List[int]

dtype: str

class afe.ir.attributes.TileAttrs

Parameters:

reps – The number of times repeating the tensor data.

reps: List[int]

class afe.ir.attributes.UpsamplingAttrs

Parameters:

• input_shape – Shape of the input tensor.

• scale_h – The scale factor for height upsampling.

• scale_w – The scale factor for width upsampling.

• layout – Layout of the input.

• method – Scale method to used [nearest_neighbor, bilinear, bicubic].

• align_corners – Whether to keep corners in proper place.

• scalar_type – Data type.

input_shape: Tuple[int, Ellipsis]

scale_h: int

scale_w: int

layout: str

method: str

align_corners: bool

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ImageResize2DAttrs

Parameters:

• size – The out size to which the image will be resized.

• roi – The region of interest for cropping the input image. Expected to be of size 4 and format [start_h, start_w, end_h, end_w]. Only used if coordinate transformation_mode is ‘tf_crop_and_resize’.

• layout – Layout of the input.

• method – Scale method to used [nearest_neighbor, linear, bicubic].

• coordinate_transformation_mode – Describes how to transform the coordinate in the resized tensor to the coordinate in the original tensor. Refer to the ONNX Resize operator specification for details. [half_pixel, align_corners, asymmetric]

• rounding_method – (string, optional) - Indicates how to find the “nearest” pixel in nearest_neighbor method [round, floor, ceil]

• cubic_alpha – (float) – Spline Coefficient for Bicubic Interpolation

• cubic_exclude – (int) – Flag to exclude exterior of the image during bicubic interpolation

• extrapolation_value – Fill value to use when roi is outside of the image.

• out_dtype – Type to return. If left None returns the same type as input.

size: List[int]

roi: Tuple[float]

layout: str

method: str

coordinate_transformation_mode: str

rounding_method: str

cubic_alpha: float

cubic_exclude: int

extrapolation_value: float

out_dtype: str

input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.GridSampleAttrs

Attributes of GridSample operator.

input_shape: Shape of the input tensor. grid_shape: Shape of the grid tensor. method: Interpolation method to use [“nearest”, “bilinear”, “bicubic”]. padding_mode: padding mode [“zeros”, “border”, “reflection”]. align_corners: Whether to align the corners in interpolation. scalar_type: Data type.

input_shape: Tuple[int, Ellipsis]

grid_shape: Tuple[int, Ellipsis]

method: str

padding_mode: str

align_corners: bool

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.MaximumAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.MinimumAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.TensorManipulationBaseAttrs

Do nothing. Used for better structuring data structure

class afe.ir.attributes.TupleAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

input_types: List[afe.ir.tensor_type.TensorType]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.TupleGetItemAttrs

Parameters:

• input_types – List of input tensor types

• index – The index of the tuple_value we return

input_types: List[afe.ir.tensor_type.TensorType]

index: int

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.SqueezeAttrs

Parameters:

• axis – Set of axes to remove

• input_shape – Shape of input tensor

• input_type – Data type of input tensor

axis: List[int]

input_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConcatenateAttrs

Parameters:

• scalar_type – Scalar tyoe of the output.

• axis – The axis along which the tensors are concatenated.

• input_types – List of input tensor types.

scalar_type: afe.ir.tensor_type.ScalarType

axis: int

input_types: List[afe.ir.tensor_type.TensorType]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.TransposeAttrs

Parameters:

• axes – The target axes order, reverse order if not specified.

• input_shape – Shape of input tensor

• input_type – Data type of input tensor

axes: List[int]

input_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.DepthToSpaceAttrs

Attributes of DepthToSpace operator

block_size: Bolck size that is shifted from channels to height and width mode: DCR for depth-column-row order re-arrangement, CRD for column-row-depth order input_shape: Shape of input tensor input_type: Data type of input tensor

block_size: int

mode: str

input_shape: tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ReshapeAttrs

Parameters:

• input_shape – Shape of input tensor

• dtype – Data type

• newshape – The new shape.

input_shape: Tuple[int, Ellipsis]

dtype: afe.ir.tensor_type.ScalarType

newshape: List[int]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ExpandDimsAttrs

Parameters:

• axis – The axis that is expanded

• num_newaxis – The number of axes to be inserted. Should be >= 0

• input_shape – Shape of input tensor

• input_type – Data type of input tensor

axis: int

num_newaxis: int

input_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.BatchFlattenAttrs

Do nothing. Used for better structuring data structure

class afe.ir.attributes.SplitAttrs

Parameters:

indices_or_sections – Indices or sections to split into. Accepts an int or a tuple

If indices_or_sections is an integer, the input will be divided equally along given axis. If such a split is not possible, an error is raised. If indices_or_sections is a tuple of sorted integers, the entries indicate where along axis the array is split. :param axis: The axis over which to split. :param input_shape: Shape of input tensor :param input_type: Data type of input tensor

indices_or_sections: int | Tuple[int, Ellipsis]

axis: int

input_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.TakeAttrs

Parameters:

• axis – The axis over which to select values. By default, the flattened input array is used.

• mode – Specifies how out-of-bound indices will behave [clip, wrap, fast]. clip: clip to the range (default). wrap: wrap around the indices. fast: no clip or wrap around (user must make sure indices are in-bound).

axis: int

batch_dims: int

mode: str

input_shape: Tuple[int, Ellipsis]

indices_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.StridedSliceAttrs

Parameters:

• begin – The indices to begin with in the slicing.

• end – Indices indicating end of the slice.

• strides – Specifies the stride values, it can be negative in that case, the input tensor will be reversed in that particular axis.

• axes – Tuple[int] or List[int], optional. Axes along which slicing is applied. When it is specified, the length of begin, end, strides, and axes must be equal. Moreover, begin, end, strides, and axes must be static (cannot be relay.Expr). Axes argument for dynamic parameter slicing is not supported yet.

• slice_mode –

  The slice mode [end, size]. end: The ending indices for the slice [default]. size: The input strides will be ignored, input end in this mode indicates

  the size of a slice starting at the location specified by begin. If end[i] is -1, all remaining elements in that dimension are included in the slice.

• input_shape – Shape of input tensor

• input_type – Data type of input tensor

begin: List[int]

end: List[int]

strides: List[int]

axes: Tuple[int] | List[int] | None

slice_mode: str

input_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LayoutTransformAttrs

Parameters:

• input_type – Shape and data type of input tensor

• src_layout – The source layout. (e.g NCHW)

• dst_layout – The destination layout. (e.g. NCHW16c)

• implicitly_removable – Whether this transform can be removed from the beginning or end of a model. If the field is True, transform can be removed in order to convert a model’s input and output data layout to NHWC (although the removal is done regardless of the actual data layout). Does not affect other optimizations that change or remove layout_transform operators.

input_type: afe.ir.tensor_type.TensorType

src_layout: str

dst_layout: str

implicitly_removable: bool = False

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.BroadcastToAttrs

Do nothing. Used for better structuring data structure

input_type: afe.ir.tensor_type.TensorType

output_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.TessellationTransformAttrs

Parameters:

• slice_shape – Shape of slice in tessellation

• align_c16 – Flag to force 16-channel alignment in tessellation

• cblock – Flag to interleave the 16-channel blocks in tessellation

• frame_type – Tensor type before tessellation

slice_shape: Sequence[int]

align_c16: bool

cblock: bool

frame_type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.DetessellationTransformAttrs

Parameters:

• slice_shape – Shape of slice in de-tessellation

• align_c16 – Flag to indicate that channels are aligned to 16 in tessellated slice.

• cblock – Flag to indicate that tesseallated slice is interleaved by channel blocks.

• frame_type – Tensor type after de-tessellation

• input_shape – Shape of input tensor

slice_shape: Sequence[int]

align_c16: bool

cblock: bool

frame_type: afe.ir.tensor_type.TensorType

input_shape: Tuple[int]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.PackTransformAttrs

Parameters:

• input_shapes – List of input tensor shape tuples

• result_scalar_type – Type of numbers in result tensor, must be ScalarType.int8

input_shapes: List[afe.ir.tensor_type.TensorType]

result_scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.UnpackTransformAttrs

Parameters:

• input_shape – Shape of input tensor

• tensor_types – List of target tensor types to unpack

input_shape: Tuple[int]

tensor_types: List[afe.ir.tensor_type.TensorType]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.NormalizationTransformAttrs

Parameters:

• channel_params – The list of tuples for (divisor, mean, standard deviation)

• input_type – Type and shape of input tensor

channel_params: List[Tuple[float, float, float]]

input_type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.QuantizationTransformAttrs

Parameters:

• channel_params – The list of tuples for (quant_scale, zero_point) Length of the list can be 1 for per tensor quantization or equal to the number of channels of input_shape for per channel quantization.

• input_shape – Shape of input tensor, expected to be channel last

• num_bits – The number of bits used for quantization

• rounding – The rounding type for quantization

• output_data_type – If number of bits is 8 data type is int8 otherwise int32.

channel_params: List[Tuple[float, int]]

input_shape: Tuple[int, int, int, int]

num_bits: int | None = 8

rounding: ml_kernels.math_helpers.RoundType

output_data_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.DequantizationTransformAttrs

Parameters:

• channel_params – The list of tuples for (quant_scale, zero_point) Length of the list can be 1 for per tensor quantization or equal to the number of channels of input_shape for per channel quantization.

• input_type – Type and shape of input tensor, expected to be channel last.

• output_type – Type of output tensor.

channel_params: List[Tuple[float, int]]

input_type: afe.ir.tensor_type.TensorType

output_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ResizeTransformAttrs

Parameters:

• target_height – Target height of resized tensor

• target_width – Target width of resized tensor

• keep_aspect – Boolean flag to keep aspect ratio

• deposit_location – Enum to indicate deposit position of resized image

• method – Enum to indicate supported interpolation methods

• input_type – Input tensor shape and scalar type

target_height: int

target_width: int

keep_aspect: bool

deposit_location: afe.apis.defines.ResizeDepositLocation

method: afe.apis.defines.ResizeMethod

input_type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ChromaUpsampleTransformAttrs

Parameters:

• frame_height – Height of full sampling frame

• frame_width – Width of full sampling frame

• yuv_sampling – Chroma sampling Enum

frame_height: int

frame_width: int

yuv_sampling: afe.apis.defines.ChromaSampling

input_type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.YuvRgbConversionTransformAttrs

Parameters:

• conversion – Direction of conversion between YUV and RGB

• std – Standard for color space conversion

conversion: afe.apis.defines.ColorConversion

std: afe.apis.defines.ColorSpaceStandard

input_shape: Tuple[int, int, int, int]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.BgrRgbConversionTransformAttrs

Parameters:

conversion – Direction of conversion between BGR and RGB

conversion: afe.apis.defines.ColorConversion

input_shape: Tuple[int, int, int, int]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.SigmoidTransformAttrs

Parameters:

save_int16 – Boolean flag to save output as 16-bit fixed point

save_int16: bool

input_shape: Tuple[int, int, int, int]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.NmsMaxpoolTransformAttrs

Parameters:

kernel – Size of pooling kernel

kernel: int

input_type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.CastAttrs

Parameters:

• out_dtype – The data type of the target.

• input_shape – Shape of input tensor.

out_dtype: str

input_shape: Tuple[int, Ellipsis]

input_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.UDFAttrs

Common attributes for UDF functions:

• Sqrt

• Rsqrt

• Sigmoid

• Exp

• Tanh

• log, log2, log10

input_shape: Tuple[int, Ellipsis]

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LeakyReluAttrs

Parameters:

alpha – The slope for the small gradient when x < 0

alpha: float

class afe.ir.attributes.SwishAttrs

Common attributes for UDF functions:

• Sqrt

• Rsqrt

• Sigmoid

• Exp

• Tanh

• log, log2, log10

class afe.ir.attributes.PReluAttrs

Parameters:

• scalar_type – Type of input and output. Must be a floating-point type.

• axis – The axis channel dimension is specified.

• alpha – The slope for the small gradient when x < 0 (constant tensor)

• input_shape – Shape of input.

scalar_type: afe.ir.tensor_type.ScalarType

axis: int

alpha: numpy.ndarray

input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ClipQuantAttrs

Attributes for Clip operation. Clip operation is always merged into a composite operator. Same class is used in floating-point and quantized version.

Parameters:

• a_min – min value of clip

• a_max – max calue of clip

• shape – Shape of input tensor

a_min: int

a_max: int

shape: Tuple[int, Ellipsis]

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ReluQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

input_shape: Tuple[int, Ellipsis]

zero_point: int = 0

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

afe.ir.attributes.ACTIVATION_ATTRS

afe.ir.attributes.QUANT_ACTIVATION_ATTRS

class afe.ir.attributes.AddActivationAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

add_attrs: AddAttrs

activ_attrs: ACTIVATION_ATTRS | None = None

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConvAddActivationAttrs

Attributes of a fused convolution operator consisting of convolution, optional bias-add, and optional activation function.

weights_attrs: ConstantAttrs

conv_attrs: ConvAttrs

bias_attrs: ConstantAttrs | None = None

add_attrs: AddAttrs | BiasAddAttrs | None = None

activ_attrs: ACTIVATION_ATTRS | None = None

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.TupleConcatenateAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

tuple_attrs: TupleAttrs

concat_attrs: ConcatenateAttrs

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ExternalAttrs

Parameters:

• external_input_list – Parameter names of the external code. This list must be equal to list(node_type.keys()).

• node_type – The external operation’s type.

• backend – The build target.

• irmod_str – The TVM IRModule of the external code saved in string form. Code representations in other fields are derived from this one. It has batch size 1, regardless of batch_size.

• operations – A list of strings that detail the ops that are contained within the IRModule.

• _graph_module – Lazily compiled executable representation of the external code. This module is used for executing this node on the compilation host.

• batch_size – The batch size that this node handles.

external_input_list: List[str]

node_type: afe.ir.tensor_type.NodeType

backend: str

irmod_str: str

operations: List[str]

batch_size: int = 1

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

property graph_module: afe._tvm._defines.TVMGraphModule

Get the operator’s code as a TVM module that can run on the compilation host.

Returns:

TVM Graph module

class afe.ir.attributes.QNNQuantizeAttrs

Further reference: tvm/src/relay/qnn/op/quantize.cc :param out_dtype: Specifies the output data type. :param axis: The channel axis for quantization. :param input_type: Tensor input type.

out_dtype: str

axis: int

input_type: afe.ir.tensor_type.TensorType

output_scale: numpy.ndarray

output_zero_point: numpy.ndarray

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.QNNDequantizeAttrs

Further reference: tvm/src/relay/qnn/op/dequantize.cc :param axis: The channel axis for quantization.

axis: int

input_type: afe.ir.tensor_type.TensorType

input_scale: numpy.ndarray

input_zero_point: numpy.ndarray

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.RequantizeAttrs

Further reference: tvm/src/relay/qnn/op/requantize.cc :param axis: The channel axis for quantization. This axis only apply to the input :param rounding: Defines the rounding direction when the value is midway

between two representable values.

Parameters:

• compute_dtype – Specifies the data type used during requantize. Supported options: “int64”, “float32”, “float64”

• out_dtype – Specifies the output data type.

axis: int

rounding: str

compute_dtype: str

out_dtype: str

input_type: afe.ir.tensor_type.TensorType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.CustomOpAttrs

Custom Op AwesomeAttributes :param custom_op_attrs: Union[str, Dic[str, Union[str, bool]]]. Custom op attrs

in either str format or a dictionary

Parameters:

• c_code_in_dtypes – Optional[List[str]]. Input tensors’ dtypes. This attribute will be assigned at the runtime

• c_code_in_shapes – Optional[List[Tuple[int, …]]]. Input tensors’ shapes This attribute will be assigned at the runtime

• function – Optional[OperatorFunction]. Compiled custom op C function This attribute will be assigned at the runtime

• args_list – Optional[Any]. A list of arguments for the custom op C function. This attribute will be assigned at the runtime

output_types: List[afe.ir.tensor_type.TensorType]

custom_op_attrs: str | Dict[str, str | bool]

c_code_in_dtypes: List[str] | None = None

c_code_in_shapes: List[Tuple[int, Ellipsis]] | None = None

function: ml_kernels.c_function_call_helpers.OperatorFunction | None = None

args_list: Any | None = None

class afe.ir.attributes.AddQuantAttrs

Attributes for quantized AddActivationOp.

param lhs_scale: Scale correction applied to the left-hand side input. param rhs_scale: Scale correction applied to the right-hand side input. :param input_int16: If True, the inputs have int16 type. If false, the inputs have int8 type. :param requant: Requantization to perform on the output. :param relu_zero_point: Zero point of the output for relu activation. Ignored if

relu is not used.

param layer_bits: Number of bits used to quantize output tensor. param activ_attrs: Activation attributes used in Add composite operators.

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

input_int16: bool

requant: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

relu_zero_point: int = 0

lhs_scale: int = 1

rhs_scale: int = 1

layer_bits: int = 8

activ_attrs: QUANT_ACTIVATION_ATTRS | None = None

property node_scales: List[float]

property node_zps: List[int]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.SubtractQuantAttrs

param attrs: SubtractAttrs class holding SubtractOp parameters :param input_int16: If True, the inputs have int16 type. If False, the inputs have int8 type. param lhs_scale: Scale correction applied to the left-hand side input. param rhs_scale: Scale correction applied to the right-hand side input. param layer_bits: Number of bits used to quantize output tensor.

attrs: SubtractAttrs

input_int16: bool

requant: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

lhs_scale: int = 1

rhs_scale: int = 1

layer_bits: int = 8

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConvQuantAttrs

Used for all variants of convolution. The attributes describe the following sequence of operators (some are optional). Relu and clip are mutually exclusive.

1. convolution

2. bias_add

3. requantize

4. relu/clip

Due to limitations of how the backend is implemented, we cannot allow the combination zero_point != 0 and isinstance(activ_attrs, ReluAttrs) and isinstance(requant, ArithFoldedRequantization). The quantizer must conform to this restriction.

Parameters:

• conv_attrs – Attributes of the convolution operator.

• weight_quant_data – Quantized weights data.

• scale – Scale of the convolution operation.

• zero_point – Zero point of the quantized output tensor.

• input_zp – Zero point of input to the convolution.

• bias_quant_data – Quantized bias data.

• weight_bits – Number of bits used to quantize the weights.

• bits – Number of bits used for quantization.

• per_channel – If true, each output channel of the weights will have an independent scale.

• activ_attrs – Activation attributes.

• requant – Requantization to do after convolve and add.

• input_int16 – Whether the input tensor has int16 type. If true, then the operator will execute using the 15-bit convolution algorithm.

• msb_left_shift – Whether the 15-bit convolution algorithm will left-shift the MSB (effectively right-shifting the full product by 1). If false, it will right-shift the LSB (effectively right-shifting the full product by 8). Ignored if input_int16 is False.

conv_attrs: ConvAttrs

weight_quant_data: numpy.ndarray

requant: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

scale: float = 1.0

zero_point: int = 0

input_zp: int = 0

bias_quant_data: numpy.ndarray | None = None

per_channel: bool = False

activ_attrs: QUANT_ACTIVATION_ATTRS | None = None

input_int16: bool = False

msb_left_shift: bool | numpy.ndarray = True

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.UpsamplingQuantAttrs

Parameters:

• input_zp

• rounding_type

upsampling_attrs: UpsamplingAttrs

input_zp: int = 0

input_scale: float = 1.0

rounding_type: ml_kernels.math_helpers.RoundType

input_int16: bool = False

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ImageResize2DQuantAttrs

Parameters:

• input_zp

• rounding_type

• requant – Requantization to perform on the output.

• input_int16 – If True, the inputs have int16 type. If False, the inputs have int8 type.

image_resize2d_attrs: ImageResize2DAttrs

input_int16: bool = False

input_zp: int = 0

input_scale: float = 1.0

rounding_type: ml_kernels.math_helpers.RoundType

requant: ml_kernels.requantization.BaseRequantization | None = None

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LRNQuantAttrs

Parameters:

• axis – Input data layout channel axis. Default value is 1 for NCHW format

• size – The size of the local region to be considered for normalization.

• lut_scale – The scale for quantization of LUT input .

• lut_zp_corr – The zp correction for quantization of LUT input .

• lut_sh – The shift for quantization of LUT input .

• output_scale – The scale for quantization of output.

• output_zp_corr – The zp correction for quantization of output.

• output_sh – The shift for quantization of output.

# NOTES FOR TENSORFLOW # TVM defines size as size_tvm = (depth_radius_tf * 2) + 1 # TVM defines alpha as alpha_tvm = alpha_tf * size_tf

axis: int

size: int

shape: Tuple[int, Ellipsis]

input_zp: int

lut_scale: int

lut_zp_corr: int

lut_sh: int

output_scale: int

output_zp_corr: int

output_sh: int

lookup_table: numpy.ndarray

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LayerNormQuantAttrs

Parameters:

• axis – Indicates the dimension along which LayerNorm will be performed.

• input_shape – Input shape.

• lookup_table_rsqrt – Look-up table f(x) = 1 / sqrt(x + epsilon).

• zp_rsqrt – Output zero point of the Rsqrt LUT.

• requant_mean – Requantization parameters for input mean (integer inputs only).

• requant_lut_input – Requantization parameters for Rsqrt LUT input.

• requant_output – Requantization of final output.

axis: int

input_shape: Tuple[int, Ellipsis]

zp_rsqrt: int

lookup_table_rsqrt: numpy.ndarray

requant_mean: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

requant_lut_input: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

requant_output: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.InstanceNormQuantAttrs

Quantized Instance Normalization operator attributes.

attrs

InstanceNorm attributes.

lut_rsqrt

Look-up table f(x) = 1 / sqrt(x + epsilon).

zp_rsqrt

Output zero point of the Rsqrt LUT.

requant_out

Requantization of the output.

attrs: InstanceNormAttrs

lut_rsqrt: numpy.ndarray

zp_rsqrt: int

requant_out: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

class afe.ir.attributes.RMSNormQuantAttrs

Parameters:

• input_shape – Input shape.

• zp_ifm – Input tensor zero point.

• lookup_table_rsqrt – Look-up table f(x) = 1 / sqrt(x + epsilon).

• zp_rsqrt – Output zero point of the Rsqrt LUT.

• requant_lut_input – Requantization parameters for Rsqrt LUT input.

• requant_output – Requantization of final output.

• lut_input_pre_shift – LUT input requantization pre-shift value.

• output_pre_shift – Output requantization pre-shift value.

• enable_lut_int16 – If True, quantize LUT to int16 otherwise to int8.

input_shape: Tuple[int, Ellipsis]

zp_ifm: int

lookup_table_rsqrt: numpy.ndarray

zp_rsqrt: int

requant_lut_input: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

requant_output: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

lut_input_pre_shift: int

output_pre_shift: int

enable_lut_int16: bool

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.SoftmaxQuantAttrs

Parameters:

• axis – Input data layout channel axis.

• input_shape – Input shape.

• exp_zp – Exp zero point.

• rec_zp – Rec zero point.

• requant_lut – Requantization parameters for quantization of reciprocal LUT input.

• requant_output – Requantization parameters for output.

• lookup_table_exp – LUT for exponential function.

• lookup_table_rec – LUT for reciprocal function.

• enable_int16 – Whether int8 or int16 quantization is used.

• lut_input_pre_shift – LUT input requantization pre-shift value (int16 only).

• output_pre_shift – Output requantization pre-shift value (int16 only).

axis: int

input_shape: Tuple[int, Ellipsis]

exp_zp: int

rec_zp: int

requant_lut: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

requant_output: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

lookup_table_exp: numpy.ndarray

lookup_table_rec: numpy.ndarray

enable_int16: bool

lut_input_pre_shift: int | None = None

output_pre_shift: int | None = None

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.RequantizeQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

attrs: RequantizeAttrs

requant: ml_kernels.requantization.BaseRequantization[numpy.ndarray]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ConcatQuantAttrs

Contains quantization attributes for concatenate quantization.

Parameters:

• attrs – ConcatenateAttrs holding ConcatenateOp parameters.

• requants – Requantization parameters

• layer_bits – Number of bits used for quantizing the tensor.

• input_scales – Quantized scale for eash inputs.

• node_scales – Using the max input_scales as the concatenate output scale of the quantized output tensors(s).

• node_zps – Zero points(s) of the quantized output tensors(s)

attrs: ConcatenateAttrs

requants: List[ml_kernels.requantization.BaseRequantization[numpy.ndarray]]

layer_bits: List[int] = [8]

input_scales: List[float | List[float]] = []

node_scales: List[float] = []

node_zps: List[int] = None

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.CustomOpQuantAttrs

Contains quantization attributes for custom operation quantization.

Parameters:

• custom_op_attrs – CustomOp attributes.

• layer_bits – Number of bits used for quantizing the tensor.

• node_zps – Zero points(s) of the quantized output tensors(s)

• node_scales – Output scale of the quantized output tensors(s).

• input_zps – Quantized zero points correction each input.

• input_scales – Quantized scales for each input.

custom_op_attrs: CustomOpAttrs

layer_bits: List[int] = [8]

node_zps: List[int] = []

node_scales: List[float] = []

input_zps: List[int | List[int]] = []

input_scales: List[float | List[float]] = []

class afe.ir.attributes.PoolQuantAttrs

Contains quantization attributes for pool quantization.

Parameters:

• pool_attrs – Pool attrs class holding MaxPool/AvgPoll operator parameters. Its scalar type does not determine the scalar type for the quantized operator.

• pad_value – Padding value.

• rounding_type – RoundType.

pool_attrs: PoolAttrs

pad_value: float | int

rounding_type: ml_kernels.math_helpers.RoundType

input_int16: bool

requant: ml_kernels.requantization.BaseRequantization | None = None

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.VarianceQuantAttrs

attrs

Variance attributes.

requant

Requantiation of the intermediate values.

requant_var

Requantization of the Variance operator final output.

attrs: VarianceAttrs

requant: ml_kernels.requantization.BaseRequantization

requant_var: ml_kernels.requantization.BaseRequantization

class afe.ir.attributes.UDFQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

attrs: UDFAttrs

input_signed: bool = False

output_signed: bool = False

lookup_table: numpy.ndarray | None = None

input_int16: bool = False

requant: ml_kernels.requantization.BaseRequantization | None = None

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.DivideAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

udf_attrs: UDFAttrs

multiply_attrs: MultiplyAttrs

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.DivideQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

udf_attrs: UDFQuantAttrs

multiply_attrs: MultiplyQuantAttrs

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.LeakyReluCompositeQuantAttrs

Contains quantization attributes for both UDF and breakdown LeakyRelu quantization. :param attrs: LeakyRelu attributes class holding LeakyReluOp parameters. :param leaky_relu_uses_udf: bool. If True, use UDF version in quantization. Otherwise, use breakdown version. :param leaky_relu_quant_attrs: Contains quantization parameters for breakdown version if breakdown version is used. :param udf_quant_attrs: Contains quantization parameters for UDF version if UDF version is used.

attrs: LeakyReluAttrs

leaky_relu_uses_udf: bool = True

leaky_relu_quant_attrs: LeakyReluQuantAttrs | None = None

udf_quant_attrs: UDFQuantAttrs | None = None

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.PReluQuantAttrs

The slope for quantized_intput < zero_point is (alpha >> right_shift)

axis: int

input_shape: Tuple[int, Ellipsis]

quant_alpha: numpy.ndarray

alpha_shift: int = 0

data_zero_point: int = 0

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.PowerAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

lhs_input_shape: Tuple[int, Ellipsis]

rhs_input_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.ArgMaxQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

attrs: ArgMaxAttrs

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

class afe.ir.attributes.BatchMatmulAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

transpose_b: bool

input_shapes: List[Tuple[int, Ellipsis]]

scalar_type: afe.ir.tensor_type.ScalarType

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.

get_output_shape() → Tuple[int, Ellipsis]

class afe.ir.attributes.BatchMatmulQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

attrs: BatchMatmulAttrs

lhs_zp: int

rhs_zp: int

requant: ml_kernels.requantization.BaseRequantization

intrinsic_shift: int

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size. Should be implemented separately inside each class that inherits from AwesomeQuantAttrBase.

class afe.ir.attributes.SliceConcatAttrs

A class that stores attributes necessary for the execution of its associated AwesomeOperation. Subclasses should include all additional attributes in their __init__ functions and call back to the AwesomeAttributes __init__ function to include the default attributes

slice_attrs: List[StridedSliceAttrs]

tuple_concat_attrs: TupleConcatenateAttrs

class afe.ir.attributes.SliceConcatQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

slice_attrs: List[StridedSliceAttrs]

tuple_concat_attrs: ConcatQuantAttrs

class afe.ir.attributes.BroadcastToQuantAttrs

Base class of quantized operator attributes. This class is used for instance checking only.

input_type: afe.ir.tensor_type.TensorType

output_shape: Tuple[int, Ellipsis]

set_batch_size(batch_size: int)

Modify internal parameters’ shapes for the given batch size.