Board: rock-5b V1.41
nvme ssd: PM9A1 256G
CMD: uname -a
Linux rock-5b 5.10.69-rockchip-rk3588 #trunk SMP Fri Sep 2 12:29:57 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
CMD: df -h
Filesystem Size Used Avail Use% Mounted on
tmpfs 1.6G 9.6M 1.6G 1% /run
/dev/mmcblk1p1 112G 5.4G 106G 5% /
tmpfs 7.7G 0 7.7G 0% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 7.7G 0 7.7G 0% /tmp
/dev/zram1 47M 2.7M 41M 7% /var/log
tmpfs 1.6G 0 1.6G 0% /run/user/0
/dev/nvme0n1p1 234G 4.1G 218G 2% /mnt/tmp
CMD: fio --name=write --ioengine=libaio --iodepth=4 --rw=write --bs=1M --direct=1 --size=2G --numjobs=30 --runtime=60 --group_reporting --filename=/mnt/tmp/test
write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=4
...
fio-3.28
Starting 30 processes
Jobs: 29 (f=28): [W(3),_(1),W(16),f(1),W(9)][92.3%][w=2524MiB/s][w=2524 IOPS][eta 00m:02s]
write: (groupid=0, jobs=30): err= 0: pid=2597: Fri Sep 2 23:21:23 2022
write: IOPS=2488, BW=2488MiB/s (2609MB/s)(60.0GiB/24692msec); 0 zone resets
slat (usec): min=83, max=50873, avg=273.06, stdev=1188.22
clat (usec): min=1504, max=906958, avg=47127.47, stdev=22880.66
lat (usec): min=1671, max=907128, avg=47401.68, stdev=22927.36
clat percentiles (msec):
| 1.00th=[ 13], 5.00th=[ 39], 10.00th=[ 43], 20.00th=[ 44],
| 30.00th=[ 45], 40.00th=[ 46], 50.00th=[ 47], 60.00th=[ 48],
| 70.00th=[ 49], 80.00th=[ 50], 90.00th=[ 52], 95.00th=[ 53],
| 99.00th=[ 77], 99.50th=[ 87], 99.90th=[ 460], 99.95th=[ 676],
| 99.99th=[ 860]
bw ( MiB/s): min= 849, max= 3977, per=100.00%, avg=2526.07, stdev=13.84, samples=1428
iops : min= 845, max= 3969, avg=2520.00, stdev=13.84, samples=1428
lat (msec) : 2=0.01%, 4=0.01%, 10=0.83%, 20=0.81%, 50=79.72%
lat (msec) : 100=18.26%, 250=0.22%, 500=0.04%, 750=0.07%, 1000=0.03%
cpu : usr=1.00%, sys=1.25%, ctx=61745, majf=0, minf=529
IO depths : 1=0.1%, 2=0.1%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,61440,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=4
Run status group 0 (all jobs):
WRITE: bw=2488MiB/s (2609MB/s), 2488MiB/s-2488MiB/s (2609MB/s-2609MB/s), io=60.0GiB (64.4GB), run=24692-24692msec
Disk stats (read/write):
nvme0n1: ios=0/122167, merge=0/5, ticks=0/5761076, in_queue=5761243, util=99.58%
CMD: fio --name=read --ioengine=libaio --iodepth=4 --rw=read --bs=1M --direct=1 --size=2G --numjobs=30 --runtime=60 --group_reporting --filename=/mnt/tmp/test
read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=4
...
fio-3.28
Starting 30 processes
Jobs: 29 (f=29): [R(1),_(1),R(28)][85.0%][r=6451MiB/s][r=6451 IOPS][eta 00m:03s]
read: (groupid=0, jobs=30): err= 0: pid=2564: Fri Sep 2 23:20:45 2022
read: IOPS=3789, BW=3789MiB/s (3973MB/s)(60.0GiB/16214msec)
slat (usec): min=45, max=66869, avg=351.89, stdev=1712.08
clat (nsec): min=291, max=91404k, avg=31076643.76, stdev=14981997.30
lat (usec): min=119, max=91815, avg=31429.80, stdev=14814.58
clat percentiles (usec):
| 1.00th=[ 141], 5.00th=[ 750], 10.00th=[ 898], 20.00th=[21103],
| 30.00th=[37487], 40.00th=[37487], 50.00th=[38011], 60.00th=[38011],
| 70.00th=[38011], 80.00th=[38536], 90.00th=[38536], 95.00th=[41157],
| 99.00th=[50594], 99.50th=[52167], 99.90th=[54264], 99.95th=[55313],
| 99.99th=[67634]
bw ( MiB/s): min= 2491, max=12891, per=89.81%, avg=3403.08, stdev=67.08, samples=954
iops : min= 2490, max=12885, avg=3398.65, stdev=67.07, samples=954
lat (nsec) : 500=0.18%, 750=0.20%, 1000=0.08%
lat (usec) : 2=0.26%, 4=0.13%, 10=0.05%, 20=0.01%, 250=0.43%
lat (usec) : 500=0.96%, 750=2.56%, 1000=10.54%
lat (msec) : 2=3.21%, 4=0.07%, 10=0.29%, 20=0.78%, 50=78.98%
lat (msec) : 100=1.28%
cpu : usr=0.18%, sys=3.38%, ctx=47584, majf=0, minf=31384
IO depths : 1=0.1%, 2=0.1%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=61440,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=4
Run status group 0 (all jobs):
READ: bw=3789MiB/s (3973MB/s), 3789MiB/s-3789MiB/s (3973MB/s-3973MB/s), io=60.0GiB (64.4GB), run=16214-16214msec
Disk stats (read/write):
nvme0n1: ios=97800/2, merge=0/1, ticks=3745843/8, in_queue=3745856, util=97.63%
CMD: fio --name=randwrite --ioengine=libaio --iodepth=4 --rw=randwrite --bs=4K --direct=1 --size=2G --numjobs=30 --runtime=60 --group_reporting --filename=/mnt/tmp/test
randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
...
fio-3.28
Starting 30 processes
Jobs: 15 (f=15): [_(1),w(1),_(3),w(1),_(1),w(2),_(1),w(2),_(1),w(3),_(2),w(2),_(1),w(1),_(1),w(2),_(2),w(1),_(2)][25.4%][w=1738MiB/s][w=445k IOPS][eta 01m:2Jobs: 15 (f=15): [_(1),w(1),_(3),w(1),_(1),w(2),_(1),w(2),_(1),w(3),_(2),w(2),_(1),w(1),_(1),w(2),_(2),w(1),_(2)][34.1%][w=1722MiB/s][w=441k IOPS][eta 01m:0Jobs: 15 (f=15): [_(1),w(1),_(3),w(1),_(1),w(2),_(1),w(2),_(1),w(3),_(2),w(2),_(1),w(1),_(1),w(2),_(2),w(1),_(2)][34.8%][w=1729MiB/s][w=443k IOPS][eta 01m:0Jobs: 14 (f=13): [_(1),w(1),_(3),w(1),_(1),w(2),_(1),f(1),w(1),_(1),w(3),_(2),w(2),_(1),w(1),_(1),w(1),_(3),w(1),_(2)][35.5%][w=1722MiB/s][w=441k IOPS][eta Jobs: 9 (f=9): [_(1),w(1),_(3),w(1),_(2),w(1),_(2),w(1),_(1),w(1),_(1),w(1),_(3),w(1),_(1),w(1),_(5),w(1),_(2)][38.2%][w=1768MiB/s][w=453k IOPS][eta 00m:55sJobs: 5 (f=5): [_(1),w(1),_(6),w(1),_(4),w(1),_(5),w(1),_(1),w(1),_(8)][53.8%][w=1606MiB/s][w=411k IOPS][eta 00m:30s]
Jobs: 2 (f=2): [_(1),w(1),_(19),w(1),_(8)][92.5%][w=998MiB/s][w=255k IOPS][eta 00m:03s]
randwrite: (groupid=0, jobs=30): err= 0: pid=2633: Fri Sep 2 23:22:16 2022
write: IOPS=421k, BW=1646MiB/s (1726MB/s)(60.0GiB/37332msec); 0 zone resets
slat (usec): min=2, max=28682, avg=13.80, stdev=36.08
clat (nsec): min=583, max=49662k, avg=207342.51, stdev=500564.46
lat (usec): min=16, max=49695, avg=221.56, stdev=504.35
clat percentiles (usec):
| 1.00th=[ 21], 5.00th=[ 25], 10.00th=[ 29], 20.00th=[ 31],
| 30.00th=[ 35], 40.00th=[ 37], 50.00th=[ 40], 60.00th=[ 50],
| 70.00th=[ 85], 80.00th=[ 129], 90.00th=[ 281], 95.00th=[ 1565],
| 99.00th=[ 2180], 99.50th=[ 2376], 99.90th=[ 3130], 99.95th=[ 3326],
| 99.99th=[ 6390]
bw ( MiB/s): min= 198, max= 9778, per=100.00%, avg=2225.66, stdev=89.50, samples=1741
iops : min=50787, max=2503312, avg=569768.11, stdev=22912.17, samples=1741
lat (nsec) : 750=0.01%, 1000=0.01%
lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.82%, 50=59.14%
lat (usec) : 100=14.02%, 250=15.44%, 500=1.31%, 750=0.46%, 1000=0.88%
lat (msec) : 2=6.22%, 4=1.69%, 10=0.02%, 20=0.01%, 50=0.01%
cpu : usr=4.42%, sys=18.35%, ctx=11816563, majf=0, minf=550
IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,15728640,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=4
Run status group 0 (all jobs):
WRITE: bw=1646MiB/s (1726MB/s), 1646MiB/s-1646MiB/s (1726MB/s-1726MB/s), io=60.0GiB (64.4GB), run=37332-37332msec
Disk stats (read/write):
nvme0n1: ios=0/15703763, merge=0/7, ticks=0/363049, in_queue=363060, util=100.00%
CMD: fio --name=randread --ioengine=libaio --iodepth=4 --rw=randread --bs=4K --direct=1 --size=2G --numjobs=30 --runtime=60 --group_reporting --filename=/mnt/tmp/test
randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=4
...
fio-3.28
Starting 30 processes
Jobs: 10 (f=9): [r(1),_(3),r(2),_(3),r(2),_(5),r(1),_(3),r(1),_(1),r(1),_(3),r(1),f(1),_(2)][97.1%][r=1683MiB/s][r=431k IOPS][eta 00m:01s]
randread: (groupid=0, jobs=30): err= 0: pid=2665: Fri Sep 2 23:24:21 2022
read: IOPS=460k, BW=1798MiB/s (1885MB/s)(60.0GiB/34170msec)
slat (usec): min=2, max=30890, avg=15.46, stdev=65.42
clat (nsec): min=874, max=37093k, avg=236124.02, stdev=339283.04
lat (usec): min=18, max=37120, avg=252.11, stdev=351.62
clat percentiles (usec):
| 1.00th=[ 51], 5.00th=[ 67], 10.00th=[ 79], 20.00th=[ 101],
| 30.00th=[ 127], 40.00th=[ 157], 50.00th=[ 186], 60.00th=[ 217],
| 70.00th=[ 258], 80.00th=[ 322], 90.00th=[ 441], 95.00th=[ 553],
| 99.00th=[ 889], 99.50th=[ 1123], 99.90th=[ 2245], 99.95th=[ 4228],
| 99.99th=[16909]
bw ( MiB/s): min= 1130, max= 2732, per=100.00%, avg=1824.44, stdev=11.34, samples=1995
iops : min=289440, max=699442, avg=467056.43, stdev=2903.53, samples=1995
lat (nsec) : 1000=0.01%
lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.19%, 50=0.68%
lat (usec) : 100=18.87%, 250=48.75%, 500=24.47%, 750=5.26%, 1000=1.08%
lat (msec) : 2=0.57%, 4=0.07%, 10=0.03%, 20=0.02%, 50=0.01%
cpu : usr=6.62%, sys=20.16%, ctx=5503081, majf=0, minf=696
IO depths : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=15728640,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=4
Run status group 0 (all jobs):
READ: bw=1798MiB/s (1885MB/s), 1798MiB/s-1798MiB/s (1885MB/s-1885MB/s), io=60.0GiB (64.4GB), run=34170-34170msec
Disk stats (read/write):
nvme0n1: ios=15722156/2, merge=0/1, ticks=1668570/5, in_queue=1668575, util=100.00%