I am afraid yes, for now. The current upstream for RK3308 has major issues with I2S - DMA communication - see Rock Pi S - I2S0 - anyone getting clean 96/192kHz? and RockPi S + CS42448 I2S/I2C DAC setup - #39 by Pavel_Hofman . I tested bsp 6.6 (latest) and 6.1 (stable), no difference. The android kernel linux-6.1-stan-rkr1 works perfectly in this regard. There must be something different.
Actually I have also tested your CM3 (RK3566) and CM5 (RK3588S) with the bsp kernel (they all use the rockchip-i2s-tdm and pl330 drivers), and also hit the I2S - DMA issues, no matter how much I tried to increase the pl330 DMAC clock, AHBUS clock, nothing made a difference. I also tried increasing the DMA buffer and maxburst size as described in RockPi S + CS42448 I2S/I2C DAC setup , no difference. Installing your 6.1 android kernel fixed it right away on RK3308 (not tested on the other SoCs)