Fogwise Airbox: Issues with Memory Allocation for NPU on Radxa Airbox running SD3

#1

Hello everyone,

I’m having trouble allocating memory for the NPU on my Radxa Airbox when following the instructions on Radxa’s SD3 model deployment guide. Additionally, I have installed the libkcapi1 library using the sudo apt install libkcapi1 command.

To resolve the memory allocation issues, I have tried various memory sharing configurations, including those recommended in the memory allocation guide on Radxa’s website. I also made sure that no container is started in Casa OS that allocates NPU mem.

which is the output when I try to load the model:

$ python3 gr.py
/mnt/NVME/projekte/SD3-Medium-TPU/python_demo/.venv/lib/python3.8/site-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (2.2.2) or chardet (3.0.4) doesn’t match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn’t match a supported "
Running on local URL: http://0.0.0.0:8999

To create a public link, set share=True in launch().
read file models/mmdit.bmodel, size 100000, duration 0.000309, speed 0.301399 GB/s
[BMRT][bmcpu_setup:406] INFO:cpu_lib ‘libcpuop.so’ is loaded.
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1084] INFO:Loading bmodel from [models/mmdit.bmodel]. Thanks for your patience…
[BMRT][load_bmodel:1023] INFO:pre net num: 0, load net num: 26
[BMRT][bmcpu_setup:406] INFO:cpu_lib ‘libcpuop.so’ is loaded.
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1084] INFO:Loading bmodel from [models/clip_g.bmodel]. Thanks for your patience…
[BMRT][load_bmodel:1023] INFO:pre net num: 0, load net num: 14
[BMRT][bmcpu_setup:406] INFO:cpu_lib ‘libcpuop.so’ is loaded.
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1084] INFO:Loading bmodel from [models/clip_l.bmodel]. Thanks for your patience…
[BMRT][load_bmodel:1023] INFO:pre net num: 0, load net num: 34
[BMRT][bmcpu_setup:406] INFO:cpu_lib ‘libcpuop.so’ is loaded.
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1084] INFO:Loading bmodel from [models/t5.bmodel]. Thanks for your patience…
[BMRT][load_bmodel:1023] INFO:pre net num: 0, load net num: 26
[BMRT][bmcpu_setup:406] INFO:cpu_lib ‘libcpuop.so’ is loaded.
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1084] INFO:Loading bmodel from [models/vae_decoder.bmodel]. Thanks for your patience…
[BMRT][load_bmodel:1023] INFO:pre net num: 0, load net num: 1
[bmlib_memory][error] bm_alloc_gmem failed, dev_id = 0, size = 0x82100000
[BM_CHECK][error] BM_CHECK_RET fail /workspace/aim-bm1684x-sdk/libsophon/bmlib/src/bmlib_memory.cpp: bm_malloc_device_byte_heap_mask: 705
[BMRT][must_alloc_device_mem:1820] FATAL:device mem alloc failed: size=2182086656[0x82100000] type_len=1 status=5 desc=neuron_mem0
Error: /data/ssd/sd3tpu-main/src/sd3.cc:135

I would appreciate any advice or suggestions from the community on how to properly allocate memory for the NPU or troubleshoot this issue. Thanks in advance for your help!

Best regards,
Matthias

#2

Hi, @M_Kraft

SD3 recommend TPU Memory: NPU->7615MB, VPU->2360MB, VPP->2360MB

best,
Morgan

#3

Hello @Morgan , thank you very much for the quick response.

The models are now loading properly but I can’t create an image. the image generation seems to run through, but the error occurs at 100%. The Gradio webui just shows an error and in the SSH terminal there is a connection reset. Thank you in advance for your help

Unbenannt
Unbenannt.png1911×915 136 KB

Full SSH LOG:
(.venv) admin@Airbox:/mnt/NVME/projekte/SD3-Medium-TPU/python_demo$ python3 gr.py
/mnt/NVME/projekte/SD3-Medium-TPU/python_demo/.venv/lib/python3.8/site-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (2.2.2) or chardet (3.0.4) doesn’t match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn’t match a supported "
Running on local URL: http://0.0.0.0:8999

To create a public link, set share=True in launch().
read file models/mmdit.bmodel, size 100000, duration 0.000309, speed 0.301399 GB/s
[BMRT][bmcpu_setup:406] INFO:cpu_lib ‘libcpuop.so’ is loaded.
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1084] INFO:Loading bmodel from [models/mmdit.bmodel]. Thanks for your patience…
[BMRT][load_bmodel:1023] INFO:pre net num: 0, load net num: 26
[BMRT][bmcpu_setup:406] INFO:cpu_lib ‘libcpuop.so’ is loaded.
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1084] INFO:Loading bmodel from [models/clip_g.bmodel]. Thanks for your patience…
[BMRT][load_bmodel:1023] INFO:pre net num: 0, load net num: 14
[BMRT][bmcpu_setup:406] INFO:cpu_lib ‘libcpuop.so’ is loaded.
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1084] INFO:Loading bmodel from [models/clip_l.bmodel]. Thanks for your patience…
[BMRT][load_bmodel:1023] INFO:pre net num: 0, load net num: 34
[BMRT][bmcpu_setup:406] INFO:cpu_lib ‘libcpuop.so’ is loaded.
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1084] INFO:Loading bmodel from [models/t5.bmodel]. Thanks for your patience…
[BMRT][load_bmodel:1023] INFO:pre net num: 0, load net num: 26
[BMRT][bmcpu_setup:406] INFO:cpu_lib ‘libcpuop.so’ is loaded.
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init
[BMRT][load_bmodel:1084] INFO:Loading bmodel from [models/vae_decoder.bmodel]. Thanks for your patience…
[BMRT][load_bmodel:1023] INFO:pre net num: 0, load net num: 1
You set add_prefix_space. The tokenizer needs to be converted from the slow tokenizers
set seed to: 3

100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [01:02<00:00, 2.23s/it]
client_loop: send disconnect: Connection reset

#4

@M_Kraft

Hi, Could you confirm that the AirBox is disconnected from SSH due to an unexpected reboot?

If so, it’s a power supply issue, if you have another 20V~3.25A PC power supply, it may help you to solve this issue.

We’ve received some feedback about unexpected power reboots when running SD3, we’re tracking down the issue and if it’s confirmed to be a problem with our power supply, the radxa aftermarket team will contact you.

best,
Morgan

#5

Hi @Morgan , thanks for the help. Yes it is definitely a problem with the power supply I am using. The Radxa power supply is not yet available in Germany. With cheap power supplies from Amazon I had the problem that the Fogwise Airbox does not boot at all so I will wait until the original Radxa Power DC 65W is available in Germany.

Best regards,
Matthias