RK3588 with USB3 Machine Vision (non-V4L2) Camera BayerGB8

Jagan · May 29, 2025, 6:34pm

I’m trying to use USB3 Vision Camera in 5B (6.1.84, Bookworm) with USB3.0 slot. Which is non-V4L2 based so we need to use aravissrc. Technically it is Bayer pixel camera where we need to use ‘bayer2rgb’ pipeline in gstreamer.

The same camera and pipeline are working in X86 Laptop (ThinkPad E14) but 5B it trigger CPU0 crossing 100% and hang.

gst-launch-1.0 -v aravissrc do-timestamp=true ! video/x-bayer,format=gbrg,width=2448,height=2048 ! bayer2rgb ! videoconvert n-threads=4 ! fpsdisplaysink sync=false text-overlay=true video-sink="kmssink"
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0: display-width = 2560
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0: display-height = 1440
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0: sync = false
/GstPipeline:pipeline0/GstAravis:aravis0: camera = (null)
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstAravis:aravis0: stream = (null)
/GstPipeline:pipeline0/GstAravis:aravis0.GstPad:src: caps = video/x-bayer, format=(string)gbrg, width=(int)2448, height=(int)2048, framerate=(fraction)20/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-bayer, format=(string)gbrg, width=(int)2448, height=(int)2048, framerate=(fraction)20/1
/GstPipeline:pipeline0/GstBayer2RGB:bayer2rgb0.GstPad:src: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)20/1, format=(string)BGRA
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)20/1, format=(string)BGRA
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)20/1, format=(string)BGRA
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay.GstPad:src: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)20/1, format=(string)BGRA
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)20/1, format=(string)BGRA
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay.GstPad:video_sink: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)20/1, format=(string)BGRA
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)20/1, format=(string)BGRA
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)20/1, format=(string)BGRA
/GstPipeline:pipeline0/GstBayer2RGB:bayer2rgb0.GstPad:sink: caps = video/x-bayer, format=(string)gbrg, width=(int)2448, height=(int)2048, framerate=(fraction)20/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-bayer, format=(string)gbrg, width=(int)2448, height=(int)2048, framerate=(fraction)20/1
Redistribute latency...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0: sync = false

Even the hardware pipeline also have similar issues.

$ GST_VIDEO_CONVERT_USE_RGA=1 gst-launch-1.0 -e -v \
  aravissrc ! \
  video/x-bayer,format=gbrg,width=2448,height=2048 ! \
  bayer2rgb ! \
  videoconvert ! \
  mpph265enc qp-init=30 qp-min=28 qp-max=36 ! \
  h265parse ! \
  splitmuxsink location="test-%05d.mkv" max-size-time=60000000000 muxer=matroskamux

Any clue, anyone uses?

Jagan · May 30, 2025, 6:20am

Look like an USB issue from sensor or aravissrc (didn’t see any log from kernel)

Setting pipeline to PAUSED ...
[19:06:27.334] ð interface> Found 1 USB3Vision device (among 11 USB devices)
[19:06:27.356] ð device> [UvDevice::new] GUID    = 04B400000209
[19:06:27.358] ð device> [UvDevice::new] USB 3.0
[19:06:27.358] ð device> [UvDevice::new] Using control endpoint 2, interface 0
[19:06:27.358] ð device> [UvDevice::new] Using data endpoint 1, interface 1
[19:06:27.358] ð device> Get genicam
[19:06:27.358] ð device> MANUFACTURER_NAME =        'Hifly Technology'
[19:06:27.359] ð device> MAX_DEVICE_RESPONSE_TIME = 0x000001f4
[19:06:27.359] ð device> DEVICE_CAPABILITY        = 0x0000000000000a09
[19:06:27.359] ð device> SRBM_ADDRESS =             0x0000000002000000
[19:06:27.359] ð device> MANIFEST_TABLE_ADDRESS =   0x0000000004000000
[19:06:27.360] ð device> U3VCP_CAPABILITY =         0x0000000000000001
[19:06:27.360] ð device> MAX_CMD_TRANSFER =         0x00004000
[19:06:27.360] ð device> MAX_ACK_TRANSFER =         0x00004000
[19:06:27.360] ð device> SIRM_OFFSET =              0x0000000003000000
[19:06:27.362] ð device> SIRM_INFO =                0x00000000
[19:06:27.362] ð device> SIRM_CONTROL =             0x00000000
[19:06:27.362] ð device> SIRM_REQ_PAYLOAD_SIZE =    0x00000000004c8000
[19:06:27.362] ð device> SIRM_REQ_LEADER_SIZE =     0x00000040
[19:06:27.362] ð device> SIRM_REQ_TRAILER_SIZE =    0x00000020
[19:06:27.362] ð device> SIRM_MAX_LEADER_SIZE =     0x00000400
[19:06:27.362] ð device> SIRM_PAYLOAD_SIZE =        0x00004000
[19:06:27.362] ðž device> SIRM_PAYLOAD_COUNT =       0x00000132
[19:06:27.363] ð device> SIRM_TRANSFER1_SIZE =      0x00000000
[19:06:27.363] ð device> SIRM_TRANSFER2_SIZE =      0x00000000
[19:06:27.363] ð device> SIRM_MAX_TRAILER_SIZE =    0x00000400
[19:06:27.363] ð device> MANIFEST_N_ENTRIES =       0x0000000000000001
[19:06:27.363] ð device> MANIFEST ENTRY
                         00000000 00 00 00 01 00 04 01 01 00 00 00 05 00 00 00 00  ................
                         00000010 c8 2b 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .+..............
                         00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
                         00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[19:06:27.363] ð device> genicam address =          0x0000000005000000
[19:06:27.363] ð device> genicam size    =          0x0000000000002bc8
[19:06:27.365] ð device> zip file =                 M3S518.xml
[19:06:27.387] ð misc> [PixelFormat::to_gst_caps_string] 0x010c0055 not found
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0: display-width = 2560
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0: display-height = 1440
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0: sync = false
/GstPipeline:pipeline0/GstAravis:aravis0: camera = (null)
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[19:06:27.396] ðž stream> SIRM_INFO             = 0x00000000
[19:06:27.396] ð stream> SIRM_REQ_PAYLOAD_SIZE = 0x00000000004c8000
[19:06:27.396] ð stream> SIRM_REQ_LEADER_SIZE  = 0x00000040
[19:06:27.396] ð stream> SIRM_REQ_TRAILER_SIZE = 0x00000020
[19:06:27.396] ð stream> Required alignment    = 1
[19:06:27.397] ð stream> SIRM_PAYLOAD_SIZE     = 0x00100000
[19:06:27.397] ð stream> SIRM_PAYLOAD_COUNT    = 0x00000004
[19:06:27.397] ð stream> SIRM_TRANSFER1_SIZE   = 0x000c8000
[19:06:27.397] ð stream> SIRM_TRANSFER2_SIZE   = 0x00000000
[19:06:27.397] ð stream> SIRM_MAX_LEADER_SIZE  = 0x00000040
[19:06:27.397] ð stream> SIRM_MAX_TRAILER_SIZE = 0x00000020
[19:06:27.398] ð stream-thread> Start async USB3Vision stream thread
/GstPipeline:pipeline0/GstAravis:aravis0: stream = (null)
/GstPipeline:pipeline0/GstAravis:aravis0.GstPad:src: caps = video/x-bayer, format=(string)gbrg, width=(int)2448, height=(int)2048, framerate=(fraction)10/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-bayer, format=(string)gbrg, width=(int)2448, height=(int)2048, framerate=(fraction)10/1
/GstPipeline:pipeline0/GstBayer2RGB:bayer2rgb0.GstPad:src: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)10/1, format=(string)BGRA
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)10/1, format=(string)BGRA
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)10/1, format=(string)BGRA
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay.GstPad:src: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)10/1, format=(string)BGRA
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)10/1, format=(string)BGRA
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay.GstPad:video_sink: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)10/1, format=(string)BGRA
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)10/1, format=(string)BGRA
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw, width=(int)2448, height=(int)2048, framerate=(fraction)10/1, format=(string)BGRA
/GstPipeline:pipeline0/GstBayer2RGB:bayer2rgb0.GstPad:sink: caps = video/x-bayer, format=(string)gbrg, width=(int)2448, height=(int)2048, framerate=(fraction)10/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-bayer, format=(string)gbrg, width=(int)2448, height=(int)2048, framerate=(fraction)10/1
0:00:00.0 / 99:99:99.Redistribute latency...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0: sync = false
0:00:00.5 / 99:99:99.0:00:00.6 / 99:99:99.[19:06:28.233] ð stream-thread> Payload transfer failed (LIBUSB_TRANSFER_STALL)
[19:06:28.233] ð stream-thread> Trailer transfer failed (LIBUSB_TRANSFER_ERROR)
[19:06:28.233] ð stream-thread> Leader transfer failed (LIBUSB_TRANSFER_ERROR)
[19:06:28.234] ð stream-thread> Payload transfer failed (LIBUSB_TRANSFER_ERROR)
[19:06:28.235] ð stream-thread> Payload transfer failed (LIBUSB_TRANSFER_ERROR)
[19:06:28.235] ð stream-thread> Payload transfer failed (LIBUSB_TRANSFER_ERROR)
[19:06:28.236] ð stream-thread> Payload transfer failed (LIBUSB_TRANSFER_ERROR)