afe.core.evaluate_networks

Classes

PerformanceAnalyzerProxy	Proxy class used to provide interface of DatasetPerformanceAnalyzer.
ComposeDatasetCompareFunction	ComposeDatasetCompareFunction(f, a) returns a dataset performance analyzer that
GraphEvaluatorLogger	Used for printing the progress and results of graph evaluation.
GraphEvaluator	Wrapper class encapsulating objects used in graph evaluation.

Functions

checked_zip(→ Iterator[Tuple[_A, _B]])

Zip together two iterables that must have the same length. The returned

Module Contents

afe.core.evaluate_networks.checked_zip(x: Iterable[_A], y: Iterable[_B]) → Iterator[Tuple[_A, _B]]

Zip together two iterables that must have the same length. The returned iterator behaves like zip, except that it raises an exception if one iterator is longer than the other.

Parameters:

• x – First iterable

• y – Second iterable

Returns:

Iterable of pairs of values taken from x and y

class afe.core.evaluate_networks.PerformanceAnalyzerProxy(dataset_performance_analyzer: Any)

Proxy class used to provide interface of DatasetPerformanceAnalyzer. DatasetPerformanceAnalyzer is used by GraphEvaluator to determine network performance by comparing network outputs to ground truth data.

Attribute

attribute _dataset_performance_analyzer:

Any. DatasetPerformanceAnalyzer object which performs the performance analysis. It must provide the same interface as this proxy class.

reset() → None

Clears memory of past network performance data.

compare(net_out: List[numpy.ndarray], gt_data: _GroundTruth) → str | None

Compares the network output and the ground truth output and saves the performance information within the class. Returns a message detailing the performance for the given output. The message can also include the overall performance when taking into account the records from previous compare() results.

analyze_results() → str

Returns a message of the overall performance of the network from all previous compare() results.

property performance: float

Returns a number from 0 to 1 indicating the network’s performance.

class afe.core.evaluate_networks.ComposeDatasetCompareFunction(dataset_performance_analyzer: Any, transform_outputs: Callable[[List[numpy.ndarray], _GroundTruth], Tuple[List[numpy.ndarray], _GroundTruth2]])

ComposeDatasetCompareFunction(f, a) returns a dataset performance analyzer that behaves like a, except that its compare function is equivalent to

def compare(out, gt):

out2, gt2 = f(out, gt) return a.compare(out2, gt2)

See DataSetPerformanceAnalyzerProxy for method documentation.

reset() → None

compare(net_out: List[numpy.ndarray], gt_data: _GroundTruth) → str | None

analyze_results() → str

property performance: float

class afe.core.evaluate_networks.GraphEvaluatorLogger(verbose: bool, log_filename: str | None)

Used for printing the progress and results of graph evaluation.

Attribute

attribute _verbose:

bool. Whether to print out the progress and results. If set to False, logging will be disabled.

attribute _log_file:

Optional[IO]. The IO object used to keep the graph evaluation logs, if any.

print_progressbar(current_step: int, total_steps: int, analysis_str: str)

Prints the progressbar if logging is enabled.

Parameters:

• current_step – int. The current step in graph evaluation process.

• total_steps – int. The total number of steps in evaluation process.

• analysis_str – str. The output of the current step in evaluation.

print_analysis_str(analysis_str: str)

Prints out the analysis string.

Parameters:

analysis_str – str. The output of the current step in evaluation.

print_analysis_summary(get_analysis_summary: Callable[[], str])

Prints out the summary of graph evaluation process.

Parameters:

get_analysis_summary – Callable. The function used to get the results of the evaluation.

print_error_message()

Prints out the warning message in case evaluation didn’t yield any results.

class afe.core.evaluate_networks.GraphEvaluator(input_generator: Iterable[Dict[afe.ir.defines.NodeName, numpy.ndarray]], ground_truth_data_generator: Iterable[_GroundTruth], performance_analyzer: Any, sample_count_hint: int, transpose_output: bool = False)

Wrapper class encapsulating objects used in graph evaluation.

Attribute

attribute input_generator:

DataGenerator. Used to generate input data used in graph evaluation.

:attribute ground_truth_data_generator. DataGenerator. Used to generate ground truth

outputs used in graph evaluation.

:attribute dataset_performance_analyzer. PerformanceAnalyzerProxy. Used to perform

graph evaluation.

attribute sample_count_hint:

Optional[int]. Number of samples in the input, used for progress reporting. Does not affect the number of samples actually processed from the input. If None, the number of samples in the input is unknown and progress is not shown.

attribute transpose_output:

bool. Whether to transpose output from NHWC to NCHW layout.

input_generator: Iterable[Dict[afe.ir.defines.NodeName, numpy.ndarray]]

ground_truth_data_generator: Iterable[_GroundTruth]

dataset_performance_analyzer: PerformanceAnalyzerProxy[_GroundTruth]

sample_count_hint: int | None

transpose_output: bool

evaluate(run_func: Callable[[Dict[afe.ir.defines.NodeName, numpy.ndarray]], List[numpy.ndarray]], verbose: bool = False, analysis_log_filename: str | None = None) → float

Perform evaluation of the network performance. If an exception is raised while performing the evaluation the performance is set to zero.

Parameters:

• run_func – Callable[[Dict[NodeName, np.ndarray], List[np.ndarray]]. Function which takes input dataset and generates inference results.

• verbose – bool. Default is False. If set to True, print out the evaluation results.

• analysis_log_filename – Optional[str]. Default is None. If given, represents the file to which the evaluation results shall be logged.

Returns:

float. A number from 0 to 1 indicating the network’s performance.