SDK API

Overview

AXCL API is divided into two parts: the Runtime API and the Native API. The Runtime API is an independent set of APIs, currently only including Memory for memory management and Engine API for driving the Axera(TM) NPU to work. When the AXCL API is used in the form of a compute card without using codec functions, all computation tasks can be completed using only the Runtime API. When codec functions are required, you need to understand the Native API and the related contents of the FFMPEG module.

runtime

Using the Runtime API allows the NPU to be called on the host system to perform computing functions. The Memory API can allocate and free memory space on both the host and the compute card, while the Engine API can complete all NPU functionalities from model initialization and IO settings to inference.

runtime

axclInit

axclError axclInit(const char *config);

Description:

System initialization, synchronous interface.

Parameters:

• config [IN]: Specifies the path to the JSON configuration file.
  • Users can configure system parameters through the JSON configuration file. Currently, log level settings are supported. Format refer to FAQ.
  • NULL or a non-existent JSON file can be passed, in which case the system uses the default configuration.

Limitations:

• Must be paired with axclFinalize for system cleanup.
• This interface must be called before invoking any AXCL interface-based application.
• Can be called only once within a single process.

axclFinailze

axclError axclFinalize();

Description:

System de-initialization, releasing AXCL resources within the process, synchronous interface.

Limitations:

• Must be paired with axclInit.
• Before the application process exits, this interface should be explicitly called for de-initialization.
• For C++ applications, it is not recommended to call this in the destructor. Otherwise, the process may exit abnormally upon termination due to an uncertain destruction order of singletons.

axclrtGetVersion

axclError axclrtGetVersion(int32_t *major, int32_t *minor, int32_t *patch);

Description:

Query the system version number, synchronous interface.

Parameters:

• major [OUT]: Major version number.
• minor [OUT]: Minor version number.
• patch [OUT]: Patch version number.

Limitations:

No special restrictions.

axclrtGetSocName

const char *axclrtGetSocName();

Description:

Query the current chip SOC name string, synchronous interface.

Limitations:

No special restrictions.

axclrtSetDevice

axclError axclrtSetDevice(int32_t deviceId);

Description:

Specify the device in the current process or thread, while implicitly creating a default Context, synchronous interface.

Parameters:

• deviceId [IN]: Device ID.

Limitations:

• This interface implicitly creates a default Context internally, which is automatically released by the system in axclrtResetDevice and must not be explicitly destroyed using axclrtDestroyContext.
• In multiple threads within the same process, if the deviceId specified in this interface is the same, the implicitly created Context will also be the same.
• Must be paired with axclrtResetDevice to release the device resources used by this process; internally uses reference counting to allow multiple calls, and only releases resources when the reference count reaches 0.
• In a multi-device scenario, you can switch devices within the process using this interface or axclrtSetCurrentContext.

axclrtResetDevice

axclError axclrtResetDevice(int32_t deviceId);

Description:

Reset the device and release resources on the device, including both implicitly and explicitly created Contexts, synchronous interface.

Parameters:

• deviceId [IN]: Device ID.

Limitations:

• For Contexts explicitly created using axclrtCreateContext, it is recommended to explicitly destroy them using axclrtDestroyContext before calling this interface to release device resources.
• Must be paired with axclrtSetDevice; the system will automatically reclaim default Context resources.
• Internally uses reference counting to allow multiple calls, and only releases resources when the reference count reaches 0.
• When the application process exits, make sure axclrtResetDevice is called, especially after handling exception signals; otherwise, this may cause a C++ terminated abort exception.

axclrtGetDevice

axclError axclrtGetDevice(int32_t *deviceId);

Description:

Get the current device ID in use, synchronous interface.

Parameters:

• deviceId [OUT]: Device ID.

Limitations:

• If neither axclrtSetDevice nor axclrtCreateContext has been called to specify a device, this interface will return an error.

axclrtGetDeviceCount

axclError axclrtGetDeviceCount(uint32_t *count);

Description:

Get the total number of connected devices, synchronous interface.

Parameters:

• count [OUT]: Number of devices.

Limitations:

No special restrictions.

axclrtGetDeviceList

axclError axclrtGetDeviceList(axclrtDeviceList *deviceList);

Description:

Get the IDs of all connected devices, synchronous interface.

Parameters:

• deviceList [OUT]: Information of all connected device IDs.

Limitations:

No special restrictions.

axclrtSynchronizeDevice

axclError axclrtSynchronizeDevice();

Description:

Synchronously execute all tasks on the current device, synchronous interface.

Limitations:

At least one device must be active.

axclrtGetDeviceProperties

axclError axclrtGetDeviceProperties(int32_t deviceId, axclrtDeviceProperties *properties);

Description:

Retrieve information such as device UID, CPU usage, NPU usage, and memory, synchronous interface.

Limitations:

No special restrictions.

axclrtCreateContext

axclError axclrtCreateContext(axclrtContext *context, int32_t deviceId);

Description:

Explicitly create a Context in the current thread, synchronous interface.

Parameters:

• context [OUT]: Handle for the created Context.
• deviceId [IN]: Device ID.

Limitations:

• If a user-created sub-thread needs to call the AXCL API, it must explicitly create a Context using this interface or bind one using axclrtSetCurrentContext.
• If the specified device has not been activated, this interface will activate it first internally.
• Call axclrtDestroyContext to explicitly release Context resources.
• Multiple threads can share the same Context (bound by axclrtSetCurrentContext), but task execution depends on the system’s thread scheduling order. Users must manage and maintain execution synchronization order between threads themselves. For multi-threading, it is recommended to create a dedicated Context for each thread to improve maintainability.

axclrtDestroyContext

axclError axclrtDestroyContext(axclrtContext context);

Description:

Explicitly destroy a Context, synchronous interface.

Parameters:

• context [IN]: Handle for the created Context.

Limitations:

• Only Context resources created using axclrtCreateContext can be destroyed.

axclrtSetCurrentContext

axclError axclrtSetCurrentContext(axclrtContext context);

Description:

Bind the Context for a thread to run with, synchronous interface.

Parameters:

• context [IN]: Context handle.

Limitations:

• If this interface is called multiple times to bind a Context to a thread, the last binding takes precedence.
• If the device corresponding to the bound Context has been reset using axclrtResetDevice, the Context cannot be set for the thread, otherwise exceptions will occur.
• It is recommended to use the Context in the same thread where it was created. If axclrtCreateContext is called in Thread A and the Context is used in Thread B, the user must ensure the task execution order under the same Context is maintained between the two threads.

axclrtGetCurrentContext

axclError axclrtGetCurrentContext(axclrtContext *context);

Description:

Get the Context handle bound to the thread, synchronous interface.

Parameters:

• context [OUT]: The current Context handle.

Limitations:

• The calling thread must have previously bound a Context using axclrtSetCurrentContext or created one using axclrtCreateContext before it can be retrieved.
• If axclrtSetCurrentContext has been called multiple times, the Context retrieved will be the one set most recently.

memory

axclrtMalloc

axclError axclrtMalloc(void **devPtr, size_t size, axclrtMemMallocPolicy policy);

Description:

Allocate non-CACHED physical memory on the device side. The allocated memory pointer is returned via devPtr, synchronous interface.

Parameters:

• devPtr [OUT]: Pointer to the allocated device-side physical memory.
• size [IN]: Size of the allocated memory in bytes.
• policy [IN]: Memory allocation policy, currently unused.

Limitations:

• Allocates continuous physical memory from the device-side CMM memory pool.
• Allocates non-CACHED memory, no consistency handling is required.
• Use axclrtFree to release memory.
• Frequent allocation and release will impact performance; it is recommended to use pre-allocation or secondary memory management to avoid frequent operations.

axclrtMallocCached

axclError axclrtMallocCached(void **devPtr, size_t size, axclrtMemMallocPolicy policy);

Description:

Allocate CACHED physical memory on the device side. The allocated memory pointer is returned via devPtr, synchronous interface.

Parameters:

• devPtr [OUT]: Pointer to the allocated device-side physical memory.
• size [IN]: Size of the allocated memory in bytes.
• policy [IN]: Memory allocation policy, currently unused.

Limitations:

• Allocates continuous physical memory from the device-side CMM memory pool.
• Allocates CACHED memory, and the user must handle data consistency.
• Use axclrtFree to release memory.
• Frequent allocation and release will impact performance; it is recommended to use pre-allocation or secondary memory management to avoid frequent operations.

axclrtFree

axclError axclrtFree(void *devPtr);

Description:

Release memory allocated on the device side, synchronous interface.

Parameters:

• devPtr [IN]: Device memory to be released.

Limitations:

• Can only release memory allocated using axclrtMalloc or axclrtMallocCached.

axclrtMemFlush

axclError axclrtMemFlush(void *devPtr, size_t size);

Description:

Flush the data in the cache to DDR and invalidate the contents in the cache, synchronous interface.

Parameters:

• devPtr [IN]: Starting address pointer of the DDR memory to be flushed.
• size [IN]: Size of the DDR memory to be flushed, in bytes.

Limitations:

No special restrictions.

axclrtMemInvalidate

axclError axclrtMemInvalidate(void *devPtr, size_t size);

Description:

Invalidate the data in the cache, synchronous interface.

Parameters:

• devPtr [IN]: Starting address pointer of the DDR memory whose cache data will be invalidated.
• size [IN]: Size of the DDR memory, in bytes.

Limitations:

• No special restrictions.

axclrtMallocHost

axclError axclrtMallocHost(void **hostPtr, size_t size);

Description:

Allocate virtual memory on the HOST, synchronous interface.

Parameters:

• hostPtr [OUT]: Start address of the allocated memory.
• size [IN]: Size of the allocated memory, in bytes.

Limitations:

• Memory allocated with axclrtMallocHost must be released with axclrtFreeHost.
• Frequent memory allocations and releases can impact performance; it is recommended to use pre-allocation or secondary memory management to avoid frequent operations.
• Memory can also be allocated on the HOST using the standard malloc interface, but axclrtMallocHost is recommended.

axclrtFreeHost

axclError axclrtFreeHost(void *hostPtr);

Description:

Release memory allocated by axclrtMallocHost, synchronous interface.

Parameters:

• hostPtr [IN]: Start address of the memory to be released.

Limitations:

• Can only release HOST memory allocated by axclrtMallocHost.

axclrtMemset

axclError axclrtMemset(void *devPtr, uint8_t value, size_t count);

Description:

Only used to initialize device-side memory allocated by axclrtMalloc or axclrtMallocCached, synchronous interface.

Parameters:

• devPtr [IN]: Start address of the device-side memory to initialize.
• value [IN]: Value to set.
• count [IN]: Length of the memory to initialize, in bytes.

Limitations:

• Only memory allocated by axclrtMalloc or axclrtMallocCached can be initialized.
• Device-side memory allocated by axclrtMallocCached requires a call to axclrtMemInvalidate after initialization to maintain consistency.
• For HOST memory allocated by axclrtMallocHost, please use the standard memset function for initialization.

axclrtMemcpy

axclError axclrtMemcpy(void *dstPtr, const void *srcPtr, size_t count, axclrtMemcpyKind kind);

Description:

Performs synchronous memory copy operations between HOST memory, between HOST and DEVICE memory, and within DEVICE memory.

Parameters:

• devPtr [IN]: Destination memory address pointer.
• srcPtr [IN]: Source memory address pointer.
• count [IN]: Length of memory to be copied, in bytes.
• kind [IN]: Type of memory copy.
  • [AXCL_MEMCPY_HOST_TO_HOST]: Memory copy within HOST memory.
  • [AXCL_MEMCPY_HOST_TO_DEVICE]: Copy from HOST virtual memory to DEVICE memory.
  • [AXCL_MEMCPY_DEVICE_TO_HOST]: Copy from DEVICE memory to HOST virtual memory.
  • [AXCL_MEMCPY_DEVICE_TO_DEVICE]: Memory copy within DEVICE.
  • [AXCL_MEMCPY_HOST_PHY_TO_DEVICE]: Copy from HOST continuous physical memory to DEVICE.
  • [AXCL_MEMCPY_DEVICE_TO_HOST_PHY]: Copy from DEVICE to HOST continuous physical memory.

Limitations:

• The source and destination memory must comply with the requirements of kind.

axclrtMemcmp

axclError axclrtMemcmp(const void *devPtr1, const void *devPtr2, size_t count);

Description:

Performs memory comparison within the DEVICE, synchronous interface.

Parameters:

• devPtr1 [IN]: Pointer to device memory address #1.
• devPtr2 [IN]: Pointer to device memory address #2.
• count [IN]: Length to compare, in bytes.

Limitations:

• Only supports comparison of device memory; returns AXCL_SUCC(0) only if the memory contents are the same.

engine

axclrtEngineInit

axclError axclrtEngineInit(axclrtEngineVNpuKind npuKind);

Description:

Initializes the Runtime Engine. Must be called before using the Runtime Engine.

Parameters:

• npuKind [IN]: Specifies the type of VNPU to initialize.

Limitations:

After using the Runtime Engine, the user must call axclrtEngineFinalize to clean up the Runtime Engine.

axclrtEngineGetVNpuKind

axclError axclrtEngineGetVNpuKind(axclrtEngineVNpuKind *npuKind);

Description:

Retrieves the initialized VNPU type of the Runtime Engine.

Parameters:

• npuKind [OUT]: Returns the VNPU type.

Limitations:

The user must call axclrtEngineInit to initialize the Runtime Engine before calling this function.

axclrtEngineFinalize

axclError axclrtEngineFinalize();

Description:

Cleans up the Runtime Engine. Must be called after completing all operations.

Limitations:

The user must call axclrtEngineInit before calling this function.

axclrtEngineLoadFromFile

axclError axclrtEngineLoadFromFile(const char *modelPath, uint64_t *modelId);

Description:

Loads model data from a file and creates a model ID.

Parameters:

• modelPath [IN]: Path to the offline model file.
• modelId [OUT]: The model ID generated after loading, used as an identifier for subsequent operations.

Limitations:

The user must call axclrtEngineInit to initialize the Runtime Engine before calling this function.

axclrtEngineLoadFromMem

axclError axclrtEngineLoadFromMem(const void *model, uint64_t modelSize, uint64_t *modelId);

Description:

Loads offline model data from memory, with memory for model execution managed internally by the system.

Parameters:

• model [IN]: Model data stored in memory.
• modelSize [IN]: Size of the model data.
• modelId [OUT]: The model ID generated after loading, used as an identifier for subsequent operations.

Limitations:

• The model memory must be device memory, which the user must manage and release.

axclrtEngineUnload

axclError axclrtEngineUnload(uint64_t modelId);

Description:

This function is used to unload a model with the specified model ID.

Parameters:

• modelId [IN]: The model ID to unload.

Limitations:

No special restrictions.

axclrtEngineGetModelCompilerVersion

const char* axclrtEngineGetModelCompilerVersion(uint64_t modelId);

Description:

This function retrieves the version of the toolchain used to build the model.

Parameters:

• modelId [IN]: Model ID.

Limitations:

No special restrictions.

axclrtEngineSetAffinity

axclError axclrtEngineSetAffinity(uint64_t modelId, axclrtEngineSet set);

Description:

This function is used to set the model’s NPU affinity.

Parameters:

• modelId [IN]: Model ID.
• set [OUT]: Affinity set to apply.

Limitations:

Cannot be zero, and the mask bits set must not exceed the affinity range.

axclrtEngineGetAffinity

axclError axclrtEngineGetAffinity(uint64_t modelId, axclrtEngineSet *set);

Description:

This function retrieves the model’s NPU affinity.

Parameters:

• modelId [IN]: Model ID.
• set [OUT]: Returned affinity set.

Limitations:

No special restrictions.

axclrtEngineGetUsage

axclError axclrtEngineGetUsage(const char *modelPath, int64_t *sysSize, int64_t *cmmSize);

Description:

This function retrieves the required system memory size and CMM memory size for executing a model based on its file.

Parameters:

• modelPath [IN]: Model path used to obtain memory information.
• sysSize [OUT]: Working system memory size required to run the model.
• cmmSize [OUT]: CMM memory size required to run the model.

Limitations:

No special restrictions.

axclrtEngineGetUsageFromMem

axclError axclrtEngineGetUsageFromMem(const void *model, uint64_t modelSize, int64_t *sysSize, int64_t *cmmSize);

Description:

This function retrieves the required system memory size and CMM memory size for executing a model based on its data in memory.

Parameters:

• model [IN]: User-managed model memory.
• modelSize [IN]: Size of the model data.
• sysSize [OUT]: Working system memory size required to run the model.
• cmmSize [OUT]: CMM memory size required to run the model.

Limitations:

Model memory must reside in device memory, which the user must manage and release.

axclrtEngineGetUsageFromModelId

axclError axclrtEngineGetUsageFromModelId(uint64_t modelId, int64_t *sysSize, int64_t *cmmSize);

Description:

This function retrieves the required system memory size and CMM memory size for executing a model based on its ID.

Parameters:

• modelId [IN]: Model ID.
• sysSize [OUT]: Working system memory size required to run the model.
• cmmSize [OUT]: CMM memory size required to run the model.

Limitations:

No special restrictions.

axclrtEngineGetModelType

axclError axclrtEngineGetModelType(const char *modelPath, axclrtEngineModelKind *modelType);

Description:

This function retrieves the model type based on its file.

Parameters:

• modelPath [IN]: Model path used to retrieve model type.
• modelType [OUT]: Returned model type.

Limitations:

No special restrictions.

axclrtEngineGetModelTypeFromMem

axclError axclrtEngineGetModelTypeFromMem(const void *model, uint64_t modelSize, axclrtEngineModelKind *modelType);

Description:

This function retrieves the model type based on model data in memory.

Parameters:

• model [IN]: User-managed model memory.
• modelSize [IN]: Size of the model data.
• modelType [OUT]: Returned model type.

Limitations:

Model memory must reside in device memory, which the user must manage and release.

axclrtEngineGetModelTypeFromModelId

axclError axclrtEngineGetModelTypeFromModelId(uint64_t modelId, axclrtEngineModelKind *modelType);

Description:

This function retrieves the model type based on the model ID.

Parameters:

• modelId [IN]: Model ID.
• modelType [OUT]: Returned model type.

Limitations:

No special restrictions.

axclrtEngineGetIOInfo

axclError axclrtEngineGetIOInfo(uint64_t modelId, axclrtEngineIOInfo *ioInfo);

Description:

This function retrieves the model IO information based on the model ID.

Parameters:

• modelId [IN]: Model ID.
• ioInfo [OUT]: Returned axclrtEngineIOInfo pointer.

Limitations:

The user should call axclrtEngineDestroyIOInfo to release axclrtEngineIOInfo before the model ID is destroyed.

axclrtEngineDestroyIOInfo

axclError axclrtEngineDestroyIOInfo(axclrtEngineIOInfo ioInfo);

Description:

This function destroys data of type axclrtEngineIOInfo.

Parameters:

• ioInfo [IN]: axclrtEngineIOInfo pointer.

Limitations:

No special restrictions.

axclrtEngineGetShapeGroupsCount

axclError axclrtEngineGetShapeGroupsCount(axclrtEngineIOInfo ioInfo, int32_t *count);

Description:

This function retrieves the number of IO shape groups.

Parameters:

• ioInfo [IN]: axclrtEngineIOInfo pointer.
• count [OUT]: Number of shape groups.

Limitations:

The Pulsar2 toolchain can specify multiple shapes during model conversion. Standard models typically have only one shape, so calling this function for normally converted models is unnecessary.

axclrtEngineGetNumInputs

uint32_t axclrtEngineGetNumInputs(axclrtEngineIOInfo ioInfo);

Description:

This function retrieves the number of model inputs from axclrtEngineIOInfo.

Parameters:

• ioInfo [IN]: axclrtEngineIOInfo pointer.

Limitations:

No special restrictions.

axclrtEngineGetNumOutputs

uint32_t axclrtEngineGetNumOutputs(axclrtEngineIOInfo ioInfo);

Description:

This function retrieves the number of model outputs from axclrtEngineIOInfo.

Parameters:

• ioInfo [IN]: axclrtEngineIOInfo pointer.

Limitations:

No special restrictions.

axclrtEngineGetInputSizeByIndex

uint64_t axclrtEngineGetInputSizeByIndex(axclrtEngineIOInfo ioInfo, uint32_t group, uint32_t index);

Description:

This function retrieves the size of a specific input based on axclrtEngineIOInfo.

Parameters:

• ioInfo [IN]: axclrtEngineIOInfo pointer.
• group [IN]: Input shape group index.
• index [IN]: Index of the input whose size is to be retrieved, starting from 0.

Limitations:

No special restrictions.

axclrtEngineGetOutputSizeByIndex

uint64_t axclrtEngineGetOutputSizeByIndex(axclrtEngineIOInfo ioInfo, uint32_t group, uint32_t index);

Description:

This function retrieves the size of a specific output based on axclrtEngineIOInfo.

Parameters:

• ioInfo [IN]: axclrtEngineIOInfo pointer.
• group [IN]: Output shape group index.
• index [IN]: Index of the output whose size is to be retrieved, starting from 0.

Limitations:

No special restrictions.

axclrtEngineGetInputNameByIndex

const char *axclrtEngineGetInputNameByIndex(axclrtEngineIOInfo ioInfo, uint32_t index);

Description:

This function retrieves the name of a specific input.

Parameters:

• ioInfo [IN]: axclrtEngineIOInfo pointer.
• index [IN]: Input IO index.

Limitations:

The returned input tensor name has the same lifecycle as ioInfo.

axclrtEngineGetOutputNameByIndex

const char *axclrtEngineGetOutputNameByIndex(axclrtEngineIOInfo ioInfo, uint32_t index);

Description:

This function retrieves the name of a specific output.

Parameters:

• ioInfo [IN]: axclrtEngineIOInfo pointer.
• index [IN]: Output IO index.

Limitations:

The returned output tensor name has the same lifecycle as ioInfo.

axclrtEngineGetInputIndexByName

int32_t axclrtEngineGetInputIndexByName(axclrtEngineIOInfo ioInfo, const char *name);

Description:

This function retrieves the input index based on the name of the input tensor.

Parameters:

• ioInfo [IN]: Model description.
• name [IN]: Input tensor name.

Limitations:

No special restrictions.

axclrtEngineGetOutputIndexByName

int32_t axclrtEngineGetOutputIndexByName(axclrtEngineIOInfo ioInfo, const char *name);

Description:

This function retrieves the output index based on the name of the output tensor.

Parameters:

• ioInfo [IN]: Model description.
• name [IN]: Output tensor name.

Limitations:

No special restrictions.

axclrtEngineGetInputDims

axclError axclrtEngineGetInputDims(axclrtEngineIOInfo ioInfo, uint32_t group, uint32_t index, axclrtEngineIODims *dims);

Description:

This function retrieves the dimension information of a specified input.

Parameters:

• ioInfo [IN]: axclrtEngineIOInfo pointer.
• group [IN]: Input shape group index.
• index [IN]: Input tensor index.
• dims [OUT]: Returned dimension information.

Limitations:

The storage space for axclrtEngineIODims is allocated by the user, and should be released before axclrtEngineIOInfo is destroyed.

axclrtEngineGetOutputDims

axclError axclrtEngineGetOutputDims(axclrtEngineIOInfo ioInfo, uint32_t group, uint32_t index, axclrtEngineIODims *dims);

Description:

This function retrieves the dimension information of a specified output.

Parameters:

• ioInfo [IN]: axclrtEngineIOInfo pointer.
• group [IN]: Output shape group index.
• index [IN]: Output tensor index.
• dims [OUT]: Returned dimension information.

Limitations:

The storage space for axclrtEngineIODims is allocated by the user, and should be released before axclrtEngineIOInfo is destroyed.

axclrtEngineCreateIO

axclError axclrtEngineCreateIO(axclrtEngineIOInfo ioInfo, axclrtEngineIO *io);

Description:

This function creates data of type axclrtEngineIO.

Parameters:

• ioInfo [IN]: axclrtEngineIOInfo pointer.
• io [OUT]: Created axclrtEngineIO pointer.

Limitations:

The user should call axclrtEngineDestroyIO to release axclrtEngineIO before the model ID is destroyed.

axclrtEngineDestroyIO

axclError axclrtEngineDestroyIO(axclrtEngineIO io);

Description:

This function destroys data of type axclrtEngineIO.

Parameters:

• io [IN]: The axclrtEngineIO pointer to be destroyed.

Limitations:

No special restrictions.

axclrtEngineSetInputBufferByIndex

axclError axclrtEngineSetInputBufferByIndex(axclrtEngineIO io, uint32_t index, const void *dataBuffer, uint64_t size);

Description:

This function sets the input data buffer by IO index.

Parameters:

• io [IN]: Address of the axclrtEngineIO data buffer.
• index [IN]: Input tensor index.
• dataBuffer [IN]: Address of the data buffer to add.
• size [IN]: Size of the data buffer.

Limitations:

The data buffer must be in device memory, and the user is responsible for managing and releasing it.

axclrtEngineSetOutputBufferByIndex

axclError axclrtEngineSetOutputBufferByIndex(axclrtEngineIO io, uint32_t index, const void *dataBuffer, uint64_t size);

Description:

This function sets the output data buffer by IO index.

Parameters:

• io [IN]: Address of the axclrtEngineIO data buffer.
• index [IN]: Output tensor index.
• dataBuffer [IN]: Address of the data buffer to add.
• size [IN]: Size of the data buffer.

Limitations:

The data buffer must be in device memory, and the user is responsible for managing and releasing it.

axclrtEngineSetInputBufferByName

axclError axclrtEngineSetInputBufferByName(axclrtEngineIO io, const char *name, const void *dataBuffer, uint64_t size);

Description:

This function sets the input data buffer by IO name.

Parameters:

• io [IN]: Address of the axclrtEngineIO data buffer.
• name [IN]: Input tensor name.
• dataBuffer [IN]: Address of the data buffer to add.
• size [IN]: Size of the data buffer.

Limitations:

The data buffer must be in device memory, and the user is responsible for managing and releasing it.

axclrtEngineSetOutputBufferByName

axclError axclrtEngineSetOutputBufferByName(axclrtEngineIO io, const char *name, const void *dataBuffer, uint64_t size);

Description:

This function sets the output data buffer by IO name.

Parameters:

• io [IN]: Address of the axclrtEngineIO data buffer.
• name [IN]: Output tensor name.
• dataBuffer [IN]: Address of the data buffer to add.
• size [IN]: Size of the data buffer.

Limitations:

The data buffer must be in device memory, and the user is responsible for managing and releasing it.

axclrtEngineGetInputBufferByIndex

axclError axclrtEngineGetInputBufferByIndex(axclrtEngineIO io, uint32_t index, void **dataBuffer, uint64_t *size);

Description:

This function retrieves the input data buffer by IO index.

Parameters:

• io [IN]: Address of the axclrtEngineIO data buffer.
• index [IN]: Input tensor index.
• dataBuffer [OUT]: Address of the data buffer.
• size [IN]: Size of the data buffer.

Limitations:

The data buffer must be in device memory, and the user is responsible for managing and releasing it.

axclrtEngineGetOutputBufferByIndex

axclError axclrtEngineGetOutputBufferByIndex(axclrtEngineIO io, uint32_t index, void **dataBuffer, uint64_t *size);

Description:

This function retrieves the output data buffer by IO index.

Parameters:

• io [IN]: Address of the axclrtEngineIO data buffer.
• index [IN]: Output tensor index.
• dataBuffer [OUT]: Address of the data buffer.
• size [IN]: Size of the data buffer.

Limitations:

The data buffer must be in device memory, and the user is responsible for managing and releasing it.

axclrtEngineGetInputBufferByName

axclError axclrtEngineGetInputBufferByName(axclrtEngineIO io, const char *name, void **dataBuffer, uint64_t *size);

Description:

This function retrieves the input data buffer by IO name.

Parameters:

• io [IN]: Address of the axclrtEngineIO data buffer.
• name [IN]: Input tensor name.
• dataBuffer [OUT]: Address of the data buffer.

Limitations:

The data buffer must be in device memory, and the user is responsible for managing and releasing it.

axclrtEngineGetOutputBufferByName

axclError axclrtEngineGetOutputBufferByName(axclrtEngineIO io, const char *name, void **dataBuffer, uint64_t *size);

Description:

This function retrieves the output data buffer by IO name.

Parameters:

• io [IN]: Address of the axclrtEngineIO data buffer.
• name [IN]: Output tensor name.
• dataBuffer [OUT]: Address of the data buffer.

Limitations:

The data buffer must be in device memory, and the user is responsible for managing and releasing it.

axclrtEngineSetDynamicBatchSize

axclError axclrtEngineSetDynamicBatchSize(axclrtEngineIO io, uint32_t batchSize);

Description:

This function sets the number of images processed at one time in a dynamic batching scenario.

Parameters:

• io [IN]: Model inference IO.
• batchSize [IN]: Number of images processed at one time.

Limitations:

No special restrictions.

axclrtEngineCreateContext

axclError axclrtEngineCreateContext(uint64_t modelId, uint64_t *contextId);

Description:

This function creates a model runtime context for the given model ID.

Parameters:

• modelId [IN]: Model ID.
• contextId [OUT]: Created context ID.

Limitations:

A model ID can have multiple runtime contexts, each running with its own settings and memory space.

axclrtEngineExecute

axclError axclrtEngineExecute(uint64_t modelId, uint64_t contextId, uint32_t group, axclrtEngineIO io);

Description:

This function performs synchronous inference for the model until the inference result is returned.

Parameters:

• modelId [IN]: Model ID.
• contextId [IN]: Model inference context.
• group [IN]: Model shape group index.
• io [IN]: Model inference IO.

Limitations:

No special restrictions.

axclrtEngineExecuteAsync

axclError axclrtEngineExecuteAsync(uint64_t modelId, uint64_t contextId, uint32_t group, axclrtEngineIO io, axclrtStream stream);

Description:

This function performs asynchronous inference for the model.

Parameters:

• modelId [IN]: Model ID.
• contextId [IN]: Model inference context.
• group [IN]: Model shape group index.
• io [IN]: Model inference IO.
• stream [IN]: Stream.

Limitations:

No special restrictions.

native

• The AXCL NATIVE module supports the SYS, VDEC, VENC, IVPS, DMADIM, ENGINE, and IVE modules.

• The AXCL NATIVE API parameters are exactly the same as those of the AX SDK API, the only difference being that the function names change from the original AX prefix to AXCL.
  Example:

  AX_S32 AXCL_SYS_Init(AX_VOID);
  AX_S32 AXCL_SYS_Deinit(AX_VOID);
  
  /* CMM API */
  AX_S32 AXCL_SYS_MemAlloc(AX_U64 *phyaddr, AX_VOID **pviraddr, AX_U32 size, AX_U32 align, const AX_S8 *token);
  AX_S32 AXCL_SYS_MemAllocCached(AX_U64 *phyaddr, AX_VOID **pviraddr, AX_U32 size, AX_U32 align, const AX_S8 *token);
  AX_S32 AXCL_SYS_MemFree(AX_U64 phyaddr, AX_VOID *pviraddr);
  
  ...
  
  AX_S32 AXCL_VDEC_Init(const AX_VDEC_MOD_ATTR_T *pstModAttr);
  AX_S32 AXCL_VDEC_Deinit(AX_VOID);
  
  AX_S32 AXCL_VDEC_ExtractStreamHeaderInfo(const AX_VDEC_STREAM_T *pstStreamBuf, AX_PAYLOAD_TYPE_E enVideoType,
                                           AX_VDEC_BITSTREAM_INFO_T *pstBitStreamInfo);
  
  AX_S32 AXCL_VDEC_CreateGrp(AX_VDEC_GRP VdGrp, const AX_VDEC_GRP_ATTR_T *pstGrpAttr);
  AX_S32 AXCL_VDEC_CreateGrpEx(AX_VDEC_GRP *VdGrp, const AX_VDEC_GRP_ATTR_T *pstGrpAttr);
  AX_S32 AXCL_VDEC_DestroyGrp(AX_VDEC_GRP VdGrp);
  
  ...

  AX_S32 AXCL_SYS_Init(AX_VOID); AX_S32 AXCL_SYS_Deinit(AX_VOID); /* CMM API */ AX_S32 AXCL_SYS_MemAlloc(AX_U64 *phyaddr, AX_VOID **pviraddr, AX_U32 size, AX_U32 align, const AX_S8 *token); AX_S32 AXCL_SYS_MemAllocCached(AX_U64 *phyaddr, AX_VOID **pviraddr, AX_U32 size, AX_U32 align, const AX_S8 *token); AX_S32 AXCL_SYS_MemFree(AX_U64 phyaddr, AX_VOID *pviraddr); ... AX_S32 AXCL_VDEC_Init(const AX_VDEC_MOD_ATTR_T *pstModAttr); AX_S32 AXCL_VDEC_Deinit(AX_VOID); AX_S32 AXCL_VDEC_ExtractStreamHeaderInfo(const AX_VDEC_STREAM_T *pstStreamBuf, AX_PAYLOAD_TYPE_E enVideoType, AX_VDEC_BITSTREAM_INFO_T *pstBitStreamInfo); AX_S32 AXCL_VDEC_CreateGrp(AX_VDEC_GRP VdGrp, const AX_VDEC_GRP_ATTR_T *pstGrpAttr); AX_S32 AXCL_VDEC_CreateGrpEx(AX_VDEC_GRP *VdGrp, const AX_VDEC_GRP_ATTR_T *pstGrpAttr); AX_S32 AXCL_VDEC_DestroyGrp(AX_VDEC_GRP VdGrp); ...

• Please refer to the AX SDK API documentation, such as "AX SYS API Documentation.docx", "AX VDEC API Documentation.docx", etc.

• The dynamic library .so naming has changed from the original libax_xxx.so to libaxcl_xxx.so. The comparison table is as follows:

• Some AX SDK APIs are not supported in AXCL NATIVE. The specific list is as follows:

PPL

architecture

libaxcl_ppl.so is a highly integrated module that implements typical pipeline (PPL). The architecture diagram is as follows:

| ----------------------------- |
| application                   |
| ----------------------------- |
| libaxcl_ppl.so                |
| ----------------------------- |
| libaxcl_lite.so               |
| ----------------------------- |
| axcl sdk                      |
| ----------------------------- |
| pcie driver                   |
| ----------------------------- |

transcode

Refer to the source code of axcl_transcode_sample which located in path: axcl/sample/ppl/transode.

API

axcl_ppl_init

axcl runtime system and ppl initialization.

prototype

axclError axcl_ppl_init(const axcl_ppl_init_param* param);

parameter

typedef enum {
    AXCL_LITE_NONE = 0,
    AXCL_LITE_VDEC = (1 << 0),
    AXCL_LITE_VENC = (1 << 1),
    AXCL_LITE_IVPS = (1 << 2),
    AXCL_LITE_JDEC = (1 << 3),
    AXCL_LITE_JENC = (1 << 4),
    AXCL_LITE_DEFAULT = (AXCL_LITE_VDEC | AXCL_LITE_VENC | AXCL_LITE_IVPS),
    AXCL_LITE_MODULE_BUTT
} AXCLITE_MODULE_E;

typedef struct {
    const char *json; /* axcl.json path */
    AX_S32 device;
    AX_U32 modules;
    AX_U32 max_vdec_grp;
    AX_U32 max_venc_thd;
} axcl_ppl_init_param;

IMPORTANT:

• AXCL_PPL_TRANSCODE: modules = AXCL_LITE_DEFAULT
• max_vdec_grp should be equal to or greater than the total number of VDEC groups across all processes. For example, if 16 processes are launched, with each process having one decoder, max_vdec_grp should be set to at least 16.
• max_venc_thrd usually be configured as same as the total VENC channels of each process.

return

0 (AXCL_SUCC) if success, otherwise failure.

axcl_ppl_deinit

axcl runtime system and ppl deinitialization.

prototype

axclError axcl_ppl_deinit();

return

0 (AXCL_SUCC) if success, otherwise failure.

axcl_ppl_create

create ppl.

prototype

axclError axcl_ppl_create(axcl_ppl* ppl, const axcl_ppl_param* param);

parameter

typedef enum {
    AXCL_PPL_TRANSCODE = 0, /* VDEC -> IVPS ->VENC */
    AXCL_PPL_BUTT
} axcl_ppl_type;

typedef struct {
    axcl_ppl_type ppl;
    void *param;
} axcl_ppl_param;

AXCL_PPL_TRANSCODE parameters as shown as follows:

typedef AX_VENC_STREAM_T axcl_ppl_encoded_stream;
typedef void (*axcl_ppl_encoded_stream_callback_func)(axcl_ppl ppl, const axcl_ppl_encoded_stream *stream, AX_U64 userdata);

typedef struct {
    axcl_ppl_transcode_vdec_attr vdec;
    axcl_ppl_transcode_venc_attr venc;
    axcl_ppl_encoded_stream_callback_func cb;
    AX_U64 userdata;
} axcl_ppl_transcode_param;

typedef struct {
    AX_PAYLOAD_TYPE_E payload;
    AX_U32 width;
    AX_U32 height;
    AX_VDEC_OUTPUT_ORDER_E output_order;
    AX_VDEC_DISPLAY_MODE_E display_mode;
} axcl_ppl_transcode_vdec_attr;

Important:

• output_order:
  • If decode sequence is same as display sequence such as IP stream, recommend to AX_VDEC_OUTPUT_ORDER_DEC to save memory.
  • If decode sequence is different to display sequence such as IPB stream, set AX_VDEC_OUTPUT_ORDER_DISP.
• display_mode
  • AX_VDEC_DISPLAY_MODE_PREVIEW: preview mode which frame dropping is allowed typically for RTSP stream... etc.
  • AX_VDEC_DISPLAY_MODE_PLAYBACK: playback mode which frame dropping is not forbidden typically for local stream file.

typedef struct {
    AX_PAYLOAD_TYPE_E payload;
    AX_U32 width;
    AX_U32 height;
    AX_VENC_PROFILE_E profile;
    AX_VENC_LEVEL_E level;
    AX_VENC_TIER_E tile;
    AX_VENC_RC_ATTR_T rc;
    AX_VENC_GOP_ATTR_T gop;
} axcl_ppl_transcode_venc_attr;

return

0 (AXCL_SUCC) if success, otherwise failure.

axcl_ppl_destroy

destroy ppl.

prototype

axclError axcl_ppl_destroy(axcl_ppl ppl)

parameter

return

0 (AXCL_SUCC) if success, otherwise failure.

axcl_ppl_start

start ppl.

prototype

axclError axcl_ppl_start(axcl_ppl ppl);

parameter

return

0 (AXCL_SUCC) if success, otherwise failure.

axcl_ppl_stop

stop ppl.

prototype

axclError axcl_ppl_stop(axcl_ppl ppl);

parameter

return

0 (AXCL_SUCC) if success, otherwise failure.

axcl_ppl_send_stream

send nalu frame to ppl.

prototype

axclError axcl_ppl_send_stream(axcl_ppl ppl, const axcl_ppl_input_stream* stream, AX_S32 timeout);

parameter

typedef struct {
    AX_U8 *nalu;
    AX_U32 nalu_len;
    AX_U64 pts;
    AX_U64 userdata;
} axcl_ppl_input_stream;

return

0 (AXCL_SUCC) if success, otherwise failure.

axcl_ppl_get_attr

get attribute of ppl.

prototype

axclError axcl_ppl_get_attr(axcl_ppl ppl, const char* name, void* attr);

parameter

/**
 *            name                                     attr type        default
 *  axcl.ppl.id                             [R  ]       int32_t                            increment +1 for each axcl_ppl_create
 *
 *  axcl.ppl.transcode.vdec.grp             [R  ]       int32_t                            allocated by ax_vdec.ko
 *  axcl.ppl.transcode.ivps.grp             [R  ]       int32_t                            allocated by ax_ivps.ko
 *  axcl.ppl.transcode.venc.chn             [R  ]       int32_t                            allocated by ax_venc.ko
 *
 *  the following attributes take effect BEFORE the axcl_ppl_create function is called:
 *  axcl.ppl.transcode.vdec.blk.cnt         [R/W]       uint32_t          8                depend on stream DPB size and decode mode
 *  axcl.ppl.transcode.vdec.out.depth       [R/W]       uint32_t          4                out fifo depth
 *  axcl.ppl.transcode.ivps.in.depth        [R/W]       uint32_t          4                in fifo depth
 *  axcl.ppl.transcode.ivps.out.depth       [R  ]       uint32_t          0                out fifo depth
 *  axcl.ppl.transcode.ivps.blk.cnt         [R/W]       uint32_t          4
 *  axcl.ppl.transcode.ivps.engine          [R/W]       uint32_t   AX_IVPS_ENGINE_VPP      AX_IVPS_ENGINE_VPP|AX_IVPS_ENGINE_VGP|AX_IVPS_ENGINE_TDP
 *  axcl.ppl.transcode.venc.in.depth        [R/W]       uint32_t          4                in fifo depth
 *  axcl.ppl.transcode.venc.out.depth       [R/W]       uint32_t          4                out fifo depth
 */

return

0 (AXCL_SUCC) if success, otherwise failure.

axcl_ppl_set_attr

set attribute of ppl.

prototype

axclError axcl_ppl_set_attr(axcl_ppl ppl, const char* name, const void* attr);

parameter

return

0 (AXCL_SUCC) if success, otherwise failure.

Error Codes

Definition

typedef int32_t axclError;

typedef enum {
    AXCL_SUCC                   = 0x00,
    AXCL_FAIL                   = 0x01,
    AXCL_ERR_UNKNOWN            = AXCL_FAIL,
    AXCL_ERR_NULL_POINTER       = 0x02,
    AXCL_ERR_ILLEGAL_PARAM      = 0x03,
    AXCL_ERR_UNSUPPORT          = 0x04,
    AXCL_ERR_TIMEOUT            = 0x05,
    AXCL_ERR_BUSY               = 0x06,
    AXCL_ERR_NO_MEMORY          = 0x07,
    AXCL_ERR_ENCODE             = 0x08,
    AXCL_ERR_DECODE             = 0x09,
    AXCL_ERR_UNEXPECT_RESPONSE  = 0x0A,
    AXCL_ERR_OPEN               = 0x0B,
    AXCL_ERR_EXECUTE_FAIL       = 0x0C,

    AXCL_ERR_BUTT               = 0x7F
} AXCL_ERROR_E;

#define AX_ID_AXCL           (0x30)

/* module */
#define AXCL_RUNTIME         (0x00)
#define AXCL_NATIVE          (0x01)
#define AXCL_LITE            (0x02)

/* runtime sub module */
#define AXCL_RUNTIME_DEVICE  (0x01)
#define AXCL_RUNTIME_CONTEXT (0x02)
#define AXCL_RUNTIME_STREAM  (0x03)
#define AXCL_RUNTIME_TASK    (0x04)
#define AXCL_RUNTIME_MEMORY  (0x05)
#define AXCL_RUNTIME_CONFIG  (0x06)
#define AXCL_RUNTIME_ENGINE  (0x07)
#define AXCL_RUNTIME_SYSTEM  (0x08)

/**
 * |---------------------------------------------------------|
 * | |   MODULE    |  AX_ID_AXCL | SUB_MODULE  |    ERR_ID   |
 * |1|--- 7bits ---|--- 8bits ---|--- 8bits ---|--- 8bits ---|
 **/
#define AXCL_DEF_ERR(module, sub, errid) \
    ((axclError)((0x80000000L) | (((module) & 0x7F) << 24) | ((AX_ID_AXCL) << 16 ) | ((sub) << 8) | (errid)))

#define AXCL_DEF_RUNTIME_ERR(sub, errid)    AXCL_DEF_ERR(AXCL_RUNTIME, (sub), (errid))
#define AXCL_DEF_NATIVE_ERR(sub,  errid)    AXCL_DEF_ERR(AXCL_NATIVE,  (sub), (errid))
#define AXCL_DEF_LITE_ERR(sub,    errid)    AXCL_DEF_ERR(AXCL_LITE,    (sub), (errid))

Note:

Error codes are divided into two types: AXCL Runtime Library and AX NATIVE SDK error codes, distinguished by the third byte of axclError.
If the third byte is equal to AX_ID_AXCL (0x30), it indicates an AXCL Runtime Library error code; otherwise, it indicates a Device-side AX NATIVE SDK module error code passed through to the HOST side.

• AXCL Runtime Library error codes: Refer to the axcl_rt_xxx.h header file.
• Device NATIVE SDK error codes are passed directly to the HOST side: Refer to the "AX Software Error Code Documentation".

Parsing

Visit AXCL Error Code Parser to parse error codes online.