Does rknn toolkit support conv3d for pytorch?

#1

Does rknn support conv3d for pytorch?

If it does, what would be the rknn.config? same as conv2d as below?

rknn.config(channel_mean_value=‘123.675 116.28 103.53 58.395’, reorder_channel=‘0 1 2’)

I have a network with the following export_pytorch_model() :

 def export_pytorch_model():
	#net = models.resnet18(pretrained=True)
	net = create_squeezenet3d_ssd3d_lite(len(class_names), is_test=False) 
	net.load(model_path)
	net.eval()
	trace_model = torch.jit.trace(net, torch.Tensor(1,3,2,180,360))
	trace_model.save('./sqnet.pt')

It loads the original pytorch model successfully, but when rknn try to load_pytorch the ./sqnet.pt as below:

input_size_list = [[3,2,180,360]] #for conv3d input [C,D,H,W]

ret = rknn.load_pytorch(model=model, input_size_list=input_size_list)

it failed with following error message:

–> config model
done
loading: ./models/saved/r14-sq3d-ssd3d-lite-Epoch-2280-Loss-3.2486252784729004.pth
–> Loading model
./sqnet.pt ********************
W Channels(360) of input node: input.4 > 128, mean/std values will be set to default 0/1.
W Please do pre-processing manually before inference.
E convolution output_padding only suport [0,0] for now, but meet [0,0]
W ----------------Warning(2)----------------
E Catch exception when loading pytorch model: ./sqnet.pt!
E Traceback (most recent call last):
E File “rknn/api/rknn_base.py”, line 339, in rknn.api.rknn_base.RKNNBase.load_pytorch
E File “rknn/base/RKNNlib/RK_nn.py”, line 146, in rknn.base.RKNNlib.RK_nn.RKnn.load_pytorch
E File “rknn/base/RKNNlib/app/importer/import_pytorch.py”, line 128, in rknn.base.RKNNlib.app.importer.import_pytorch.ImportPytorch.run
E File “rknn/base/RKNNlib/converter/convert_pytorch_new.py”, line 2255, in rknn.base.RKNNlib.converter.convert_pytorch_new.convert_pytorch.load
E File “rknn/base/RKNNlib/converter/convert_pytorch_new.py”, line 2370, in rknn.base.RKNNlib.converter.convert_pytorch_new.convert_pytorch.parse_nets
E File “rknn/base/RKNNlib/converter/convert_pytorch_new.py”, line 2059, in rknn.base.RKNNlib.converter.convert_pytorch_new.PyTorchOpConverter.convert_operators
E File “rknn/base/RKNNlib/converter/convert_pytorch_new.py”, line 648, in rknn.base.RKNNlib.converter.convert_pytorch_new.PyTorchOpConverter.convolution
E File “rknn/api/rknn_log.py”, line 312, in rknn.api.rknn_log.RKNNLog.e
E ValueError: convolution output_padding only suport [0,0] for now, but meet [0,0]
Load pytorch model failed!

Question: how could I print out layer by layer from rknn to check where does the error occur? or fundamentally rknn does not support conv3d, so no need to debug? Please help. Thanks a lot.

#2

I made a single layer network:

def export_pytorch_model():
	net = Conv3d(3, 64, kernel_size=[2,3,3], stride=[1,2,2])
	net.eval()
	trace_model = torch.jit.trace(net, torch.Tensor(1,3,2,180,360))
	trace_model.save('./sqnet.pt')

Then follow the rknn api as below:

input_size_list = [[3,2,180,360]] #stacked 3D images
# Create RKNN object
rknn = RKNN()
# pre-process config
print('--> config model')
rknn.config(channel_mean_value='123.675 116.28 103.53 58.395', reorder_channel='0 1 2') 
print('done')

if not os.path.exists('./sqnet.rknn'):
	export_pytorch_model()
	model = './sqnet.pt'
	# Load pytorch model
	print('--> Loading model')
	ret = rknn.load_pytorch(model=model, input_size_list=input_size_list)
	if ret != 0:
		print('Load pytorch model failed!')
		exit(ret)
	print('done')

When run the above code, I got following error message:
–> config model
done
–> Loading model
./sqnet.pt ********************
W Channels(360) of input node: input.1 > 128, mean/std values will be set to default 0/1.
W Please do pre-processing manually before inference.
E convolution output_padding only suport [0,0] for now, but meet [0,0]
W ----------------Warning(2)----------------
E Catch exception when loading pytorch model: ./sqnet.pt!
E Traceback (most recent call last):
E File “rknn/api/rknn_base.py”, line 339, in rknn.api.rknn_base.RKNNBase.load_pytorch
E File “rknn/base/RKNNlib/RK_nn.py”, line 146, in rknn.base.RKNNlib.RK_nn.RKnn.load_pytorch
E File “rknn/base/RKNNlib/app/importer/import_pytorch.py”, line 128, in rknn.base.RKNNlib.app.importer.import_pytorch.ImportPytorch.run
E File “rknn/base/RKNNlib/converter/convert_pytorch_new.py”, line 2255, in rknn.base.RKNNlib.converter.convert_pytorch_new.convert_pytorch.load
E File “rknn/base/RKNNlib/converter/convert_pytorch_new.py”, line 2370, in rknn.base.RKNNlib.converter.convert_pytorch_new.convert_pytorch.parse_nets
E File “rknn/base/RKNNlib/converter/convert_pytorch_new.py”, line 2059, in rknn.base.RKNNlib.converter.convert_pytorch_new.PyTorchOpConverter.convert_operators
E File “rknn/base/RKNNlib/converter/convert_pytorch_new.py”, line 648, in rknn.base.RKNNlib.converter.convert_pytorch_new.PyTorchOpConverter.convolution
E File “rknn/api/rknn_log.py”, line 312, in rknn.api.rknn_log.RKNNLog.e
E ValueError: convolution output_padding only suport [0,0] for now, but meet [0,0]
Load pytorch model failed!

Question: what can I do to fix this [0,0] padding issue? Thanks for your help.