Load a Model

load_model(self, device, in_shape_list, out_shape_list, metadata, model_path=None, model_hdl=None)

Description

This method loads a model onto the connected device from a file path or using a model handle dictionary prepares the input and output tensors and sets up the metadata.

If model_path is not provided, it assumes the application is already deployed and running on the device and does not deploy any pipeline.

If model_path is provided, it launches the application on the device. This function is blocking but has a timeout in case of an error.

  • Possible failures include:

    • Issue with provided mpk, or incorrect mpk path

    • Memory allocation failure at the device driver

    • MLSoC fails to respond or deploy

  • If an Exception is raised:

    • Reset the MLSoC target by invoking reset(self, device). The reset API is a blocking call, once the function returns, re-enumerate to ensure the GUID is found upon reboot, connect using the same GUID, and retry loading the model.

    • If this does not work, then the device has run into an unrecoverable error, and power cycling the machine is the next step.

    • Dmesg or logging will provide further information.

Parameters

  • device (DevicePointer): The device pointer

  • in_shape_list (list): List of input tensor shapes

  • out_shape_list (list): List of output tensor shapes

  • metadata (list): Metadata associated with the model

  • model_path (str, optional): Path to the model. Default is None. If provided, deploys and launches the application on the device.

  • model_hdl (dict, optional): Model handle dictionary containing the model definitions. Default is None.

Returns

  • ModelReference: A reference to the loaded mod

Raises

  • ValueError: If device and self.dev_ptr do not point to the same device

  • Exception: If shapes of in or out tensors is None

Usage

 1 in_shape_list = [(1, 224, 224, 3)]
 2 out_shape_list = [(1, 1000)]
 3 metadata = ["meta1", "meta2"]
 4 model_path = "/path/to/model"
 5 model_ref = device_interface.load_model(device_ptr, in_shape_list, out_shape_list, metadata,
 6 model_path)
 7 # Example without model path
 8 model_hdl = {
 9 "in_tensors": 1,
10 "out_tensors": 1,
11 "in_batch_sz": 1,
12 "out_batch_sz": 1,
13 "in_shape": [(1, 224, 224, 3)],
14 "out_shape": [(1, 1000)],
15}
16 model_ref = device_interface.load_model(device_ptr, in_shape_list, out_shape_list, metadata,
17 model_hdl=model_hdl)