Missing onnx library for ZhouyiExecutionProvider

#1

Hi,

I am trying to run this onnx model on NPU execution provider, it keeps failing with some missing library:

code:
import onnxruntime as ort
print(ort.get_available_providers())

onnx_model_path = “/home/radxa/onnx_repo/no_wrapper_maykeye.onnx”
#/home/radxa/onnx_repo

Create session options

sess_options = ort.SessionOptions()
sess_options.intra_op_num_threads = 55 # limit CPU threads to 1
sess_options.inter_op_num_threads = 55 # limit parallel executions to 1

session = ort.InferenceSession(onnx_model_path, sess_options=sess_options, providers=[“ZhouyiExecutionProvider”])

Missing Library:

[ERROR][init:141]Cannot find layerlib under path: ./operator:./:./lib
[ERROR][compass_runtime_midware_init:34]Cannot get valid operators library!

Thanks

#2

I think u missed to set the environment for ort.
it is described in the doc

#3

Thanks, are you referring to this statement in the docs:

Additionally, OnnxRuntime can be used to run end-to-end examples locally on an x86 host or on the O6 using the CPU:

python3 inference_onnx.py

#4

You can get a noe SDK release from support.cixtech.com（apply through EBP）, which includes a doc explaining how to use NPU for inference.

For C++ API, you should set env through:
$ export LD_LIBRARY_PATH=/usr/share/cix/lib/onnxruntime:$LD_LIBRARY_PATH
$ export OPERATOR_PATH=/usr/share/cix/lib/onnxruntime/operator/

For Python API, you should pip install onnxruntime_zhouyi.whl first:
$ pip3 install /usr/share/cix/pypi/onnxruntime_zhouyi-xxxx-linux_aarch64.whl
and set env:
$ export OPERATOR_PATH=/usr/share/cix/lib/onnxruntime/operator/
and add the following to the python file:
from ZhouyiOperators import operators

1 Like