Pytorch squeezenet SSD running on NPU

#1

Hi,

Did any one successfully run the sq-ssd-lite from this repository on NPU using rknn_toolkit?

I am able to get to the point of building the model successfully but failed at Init running environment. The following is the error message:

–> config model
done
–> Loading model
./sqnet.pt ********************
W Warning: Axis may need to be adjusted according to original model shape.
W Warning: Axis may need to be adjusted according to original model shape.
W Unhandle status: the input shape of reshape layer unsqueeze_at_priors.1_171_159 is not 4-D
W Warning: Axis may need to be adjusted according to original model shape.
W Warning: Axis may need to be adjusted according to original model shape.
W Warning: Axis may need to be adjusted according to original model shape.
W Warning: Axis may need to be adjusted according to original model shape.
W Unhandle status: the input shape of reshape layer unsqueeze_at_priors.1_171_159 is not 4-D
W Warning: Axis may need to be adjusted according to original model shape.
W Warning: Axis may need to be adjusted according to original model shape.
done
–> Building model
W The target_platform is not set in config, using default target platform rk1808.
W The channel_mean_value is deprecated. Please use mean_values and std_values to replace it.
done
–> Export RKNN model
done
–> Init runtime environment
I NPUTransfer: Starting NPU Transfer Client, Transfer version 2.1.0 (b5861e7@2020-11-23T11:50:51)
E RKNNAPI: rknn_init, msg_load_ack fail, ack = 1(ACK_FAIL), expect 0(ACK_SUCC)!
E RKNNAPI: ==============================================
E RKNNAPI: RKNN VERSION:
E RKNNAPI: API: 1.6.0 (79320de build: 2020-12-29 10:57:09)
E RKNNAPI: DRV: 1.6.0 (159d2d3 build: 2021-01-12 15:23:09)
E RKNNAPI: ==============================================
E Catch exception when init runtime!
E Traceback (most recent call last):
E File “rknn/api/rknn_base.py”, line 844, in rknn.api.rknn_base.RKNNBase.init_runtime
E File “rknn/api/rknn_runtime.py”, line 340, in rknn.api.rknn_runtime.RKNNRuntime.build_graph
E Exception: RKNN init failed. error code: RKNN_ERR_MODEL_INVALID
Init runtime environment failed

Am I missing something? Please help. Thanks a lot!

#2

Tell me about the line init_runtime

#3

Hi @setq,

The attached message in my original post is all the message I got.

The code for init_runtime is as following:

init runtime environment

print('--> Init runtime environment')
ret = rknn.init_runtime()
if ret != 0:
	print('Init runtime environment failed')
	exit(ret)
print('done')

You may download sqnet.pt and sqnet.rknn from here.

Thank you very much for your help.

#4

Try to change to

ret = rknn.init_runtime(target='rk3399pro'）
#5

Hi @setq

Thank you for your help.

After I change the line to

ret = rknn.init_runtime(target=‘rk3399pro’)

I got the following error message:

–> Loading model
./sqnet.pt ********************
W Warning: Axis may need to be adjusted according to original model shape.
W Warning: Axis may need to be adjusted according to original model shape.
W Unhandle status: the input shape of reshape layer unsqueeze_at_priors.1_171_159 is not 4-D
W Warning: Axis may need to be adjusted according to original model shape.
W Warning: Axis may need to be adjusted according to original model shape.
W Warning: Axis may need to be adjusted according to original model shape.
W Warning: Axis may need to be adjusted according to original model shape.
W Unhandle status: the input shape of reshape layer unsqueeze_at_priors.1_171_159 is not 4-D
W Warning: Axis may need to be adjusted according to original model shape.
W Warning: Axis may need to be adjusted according to original model shape.
done
–> Building model
W The target_platform is not set in config, using default target platform rk1808.
W The channel_mean_value is deprecated. Please use mean_values and std_values to replace it.
done
–> Export RKNN model
done
–> Init runtime environment
E Only support ntb mode on ARM64 platform. But can not find device with ntb mode.
E Catch exception when init runtime!
E Traceback (most recent call last):
E File “rknn/api/rknn_base.py”, line 815, in rknn.api.rknn_base.RKNNBase.init_runtime
E File “rknn/api/rknn_runtime.py”, line 170, in rknn.api.rknn_runtime.RKNNRuntime.init
E File “rknn/api/rknn_platform_utils.py”, line 307, in rknn.api.rknn_platform_utils.start_ntp_or_adb
E Exception: Init runtime environment failed!
E Current device id is: None
E Devices connected:
E [‘96b99f614d50171c’]
Init runtime environment failed

From pytorch to rknn
#6

Hi @setq

I tried the same squeezenet SSD on PC (ubuntu 18.04) rknn toolkit simulator and got different error:

–> config model
done
–> Loading model
./sqnet.pt ********************
E Catch exception when loading pytorch model: ./sqnet.pt!
E Traceback (most recent call last):
E File “rknn/api/rknn_base.py”, line 339, in rknn.api.rknn_base.RKNNBase.load_pytorch
E File “rknn/base/RKNNlib/RK_nn.py”, line 146, in rknn.base.RKNNlib.RK_nn.RKnn.load_pytorch
E File “rknn/base/RKNNlib/app/importer/import_pytorch.py”, line 128, in rknn.base.RKNNlib.app.importer.import_pytorch.ImportPytorch.run
E File “rknn/base/RKNNlib/converter/convert_pytorch_new.py”, line 2250, in rknn.base.RKNNlib.converter.convert_pytorch_new.convert_pytorch.load
E File “rknn/base/RKNNlib/converter/convert_pytorch_new.py”, line 2055, in rknn.base.RKNNlib.converter.convert_pytorch_new.PyTorchOpConverter.report_missing_conversion
E NotImplementedError: The following operators are not implemented: [‘aten::to’]
Load pytorch model failed!

It seem that simulator is worse than running rknn on rk3399pro.
What shall I do to move forward? What is the “NotImplementedError: The following operators are not implemented: [‘aten::to’]”? Do we need to install different version of torch? currently in my system, torch=1.5.1 installed.
Thanks for your help.