NVME not detected

I use a Samsung 970EVO witrh 500Gbytes and tested the performance today. I think it does not yet run at full speed:

sudo hdparm -Tt --direct /dev/nvme0n1

/dev/nvme0n1:
Timing O_DIRECT cached reads: 614 MB in 2.00 seconds = 306.61 MB/sec
Timing O_DIRECT disk reads: 920 MB in 3.00 seconds = 306.39 MB/sec

carsten@rockpi4b:~$ sysbench --test=fileio --file-total-size=2G --file-test-mode=rndrw --init-rng=on --max-time=300 --max-requests=0 run
sysbench 0.4.12: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1
Initializing random number generator from timer.

Extra file open flags: 0
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Time limit exceeded, exiting…
Done.

Operations performed: 297300 Read, 198200 Write, 634191 Other = 1129691 Total
Read 4.5364Gb Written 3.0243Gb Total transferred 7.5607Gb (25.807Mb/sec)
1651.66 Requests/sec executed

Test execution summary:
total time: 300.0009s
total number of events: 495500
total time taken by event execution: 31.3085
per-request statistics:
min: 0.01ms
avg: 0.06ms
max: 15.18ms
approx. 95 percentile: 0.18ms

Threads fairness:
events (avg/stddev): 495500.0000/0.00
execution time (avg/stddev): 31.3085/0.00

Hi !
There are my results :

rockpi@rockpi4b:~$ sudo hdparm -Tt --direct /dev/nvme0n1
[sudo] password for rockpi:

/dev/nvme0n1:
 Timing O_DIRECT cached reads:   1020 MB in  2.00 seconds = 510.00 MB/sec
 Timing O_DIRECT disk reads: 1382 MB in  3.00 seconds = 460.60 MB/sec

But for me the best app to test the speed of my NVME is iozone (NVME = WD Black):

rockpi@rockpi4b:~$ sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
        Iozone: Performance Test of File I/O
                Version $Revision: 3.429 $
                Compiled for 64 bit mode.
                Build: linux

        Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                     Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                     Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                     Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                     Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                     Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                     Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                     Vangel Bojaxhi, Ben England, Vikentsi Lapa.

        Run began: Tue Mar 12 08:34:56 2019

        Include fsync in write timing
        O_DIRECT feature enabled
        Auto Mode
        File size set to 102400 kB
        Record Size 4 kB
        Record Size 16 kB
        Record Size 512 kB
        Record Size 1024 kB
        Record Size 16384 kB
        Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
        Output is in kBytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 kBytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    59325    84009    76919    77801    40893    79823
          102400      16   130702   227024   213902   216338   120339   226025
          102400     512   557596   590522   521821   522062   500256   576589
          102400    1024   629537   641508   583415   600946   550795   644552
          102400   16384   661427   694791   659492   659860   660166   681809

iozone test complete.

It’s a little bit slow compare to my NanopiPCT4 (XPG NVME) but the result can be compare to my RockPro64 (Samsung Evo 970 NVME).

This is the result I got on my RockPro64 with my Samsung Evo 970 :

rock@rockpro64:~$ sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
[sudo] password for rock:
Iozone: Performance Test of File I/O
Version $Revision: 3.429 $
Compiled for 64 bit mode.
Build: linux

    Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                 Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                 Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                 Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                 Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                 Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                 Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                 Vangel Bojaxhi, Ben England, Vikentsi Lapa.

    Run began: Tue Mar 12 08:48:01 2019

    Include fsync in write timing
    O_DIRECT feature enabled
    Auto Mode
    File size set to 102400 kB
    Record Size 4 kB
    Record Size 16 kB
    Record Size 512 kB
    Record Size 1024 kB
    Record Size 16384 kB
    Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
    Output is in kBytes/sec
    Time Resolution = 0.000001 seconds.
    Processor cache size set to 1024 kBytes.
    Processor cache line size set to 32 bytes.
    File stride size set to 17 * record size.
                                                          random    random     bkwd    record    stride
          kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
      102400       4    82368   117913   145635   146582    52602    84380
      102400      16   166460   238068   269713   271330   168286   235563
      102400     512   590845   619712   572332   572707   557272   621562
      102400    1024   616225   633981   578821   578426   572636   638544
      102400   16384   676638   695671   657252   657771   656296   695662

iozone test complete.

Hi !
Ok, I did the little recipe writed here KingSpec 128GB M.2 NVMe not found Debian+Ubuntu

and now the speed of my NVME increase a lot !!

rockpi@rockpi4b:~$ sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
[sudo] password for rockpi:
Iozone: Performance Test of File I/O
Version $Revision: 3.429 $
Compiled for 64 bit mode.
Build: linux

    Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                 Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                 Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                 Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                 Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                 Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                 Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                 Vangel Bojaxhi, Ben England, Vikentsi Lapa.

    Run began: Tue Mar 12 15:10:40 2019

    Include fsync in write timing
    O_DIRECT feature enabled
    Auto Mode
    File size set to 102400 kB
    Record Size 4 kB
    Record Size 16 kB
    Record Size 512 kB
    Record Size 1024 kB
    Record Size 16384 kB
    Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
    Output is in kBytes/sec
    Time Resolution = 0.000001 seconds.
    Processor cache size set to 1024 kBytes.
    Processor cache line size set to 32 bytes.
    File stride size set to 17 * record size.
                                                          random    random     bkwd    record    stride
          kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
      102400       4    60903   103967    79528    82953    41992    85719
      102400      16   142777   267881   247469   248952   123519   261772
      102400     512   781774   910028   769780   770132   725685   878638
      102400    1024   971169  1026856   920207   906010   788857  1059701
      102400   16384  1125377  1226484  1208797  1208035  1205309  1164655

iozone test complete.

Hi,

I did the same, but my results only improved at bit:

before:
sudo hdparm -Tt --direct /dev/nvme0n1

/dev/nvme0n1:
Timing O_DIRECT cached reads: 614 MB in 2.00 seconds = 306.61 MB/sec
Timing O_DIRECT disk reads: 920 MB in 3.00 seconds = 306.39 MB/sec

after:
udo hdparm -Tt --direct /dev/nvme0n1

/dev/nvme0n1:
Timing O_DIRECT cached reads: 950 MB in 2.00 seconds = 474.42 MB/sec
Timing O_DIRECT disk reads: 1316 MB in 3.00 seconds = 438.23 MB/sec

How can I install iozone?

Carsten

Hello !
To install iozone

sudo apt install iozone3

Cool, thx. I did not spoit ist is iozone3

here are my results (option -I dis give me an error, so I did not use it):

root@rockpi4b:/tmp# iozone -e -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
Iozone: Performance Test of File I/O
Version $Revision: 3.429 $
Compiled for 64 bit mode.
Build: linux

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
             Vangel Bojaxhi, Ben England, Vikentsi Lapa.

Run began: Wed Mar 13 12:59:10 2019

Include fsync in write timing
Auto Mode
File size set to 102400 kB
Record Size 4 kB
Record Size 16 kB
Record Size 512 kB
Record Size 1024 kB
Record Size 16384 kB
Command line used: iozone -e -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
Output is in kBytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 kBytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
                                                          random    random     bkwd    record    stride                                    
          kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
      102400       4  1129045  1373941  1648500  1692926  1208573  1254668                                                          
      102400      16  1299375  1589864  1835551  1931747  1421671  1461232                                                          
      102400     512   985505  1355517  1283321  1452912  1359262  1572724                                                          
      102400    1024   977517  1402778  1265571  1337655  1282953  1455931                                                          
      102400   16384  1005955  1362626  1320420  1384915  1314086  1402176

Hi Mike,

very good that you got that done. But what did you do in detail? I stand with a Samsung 960 nvme ssd facing the same problem?!?
Best thanks in advance !

Telekomiker

Also tried to connect the nvme directly to the board with the same issue

Which nvme ssd are you using? @Lafades

Yesterday arrived my new ROCK PI 4A v1.4 4GB with heatsink and m2 extension board.
I bought blindly Intel 660p M.2 1 TB SSD NVMe (111€) not verifing compatibility…
Not detected in ROCK PI 4 directly or via extension board.
Tested with Armbian, Debian and Ubuntu.
Powered by QC charger.
Tomorrow i expect PD charger. I hope its just a power issue.

PS: NVMe disk works in my old PC thru cheap PCI Express board.

It works! With issues.
After 3 days of tinkering and using PD charger up to 60W .
Its does not recognize same as in M.2 NVMe Intel 1TB 660P Not reconized on Ubuntu Server
BUT after few minutes, i did restart and there it was.


Its very peculiar behaviour. After cool down nothing again (Power down for longer time). After warm up its detected again.

Posting here because I’m in the same situation… Intel 660p not being detected. If anyone figures out a reliable method, please do post it!

I guess it’s because 660P is a little too slow at cold start, like some microSD cards having similar issue RP4 will skip it at cold start (though it’s accessible once started from eMMC) but just fine when hot reboot. It could be fixable in the firmware.

No, i did not find solution how to detect it on cold boot.

Im using PD2 power brick, armbian from radxa download page and extension board as on picture:
RockPiExt

@ O635789
Its not just after restart. I need to wait few minutes (to warm up?) and then restart.
If it were normal mini PC, I would return it. But I consider it “tinkering” board, so I’m willing to accept this peculiar behaviour.

PS: I believe I saw other RK3399 board(s) with this drive working on other forums.

Interesting. For now I’ll just use a standard ssd in a USB enclosure until a stable solution appears.

Are you referring that, you need to:

  1. unplug the power adapter
  2. plug the power and cold boot rockpi4
  3. wait a few minutes, nvme not detected
  4. restart in desktop or type reboot
  5. nvme detected

Is that correct?

Yes, that’s pretty much accurate. That was my first success with this drive under Debian.
Just reboot is not enough after cold start. I need to wait few minutes. Then I can restart as many times as I want.

Now I’m using Armbian 24/7 headless. Because I do have manual mounting in fstab, I’ll try to connesct thru ssh. If i’m not sucessfull I unplug USB-C from power adapter and immediatly plug back - and it works.

rockpi@rockpi4a:~$ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
mmcblk0     179:0    0   3.7G  0 disk
└─mmcblk0p1 179:1    0   3.5G  0 part /
zram0       254:0    0    50M  0 disk /var/log
nvme0n1     259:0    0 953.9G  0 disk
└─nvme0n1p1 259:1    0 953.9G  0 part /mnt/m2
rockpi@rockpi4a:~$ sudo hdparm -t --direct /dev/nvme0n1

/dev/nvme0n1:
 Timing O_DIRECT disk reads: 2030 MB in  3.00 seconds = 676.54 MB/sec

Nope.
“Cold start” with “warm electronics” is ok, it will detect drive correctly. It needs to be powered off for longer time to not detect this nvme

I think we have solved this issue. Please update the latest u-boot.

sudo apt-get install rockpi4b-rk-u-boot-latest
sudo /usr/local/sbin/rockpi4b_upgrade_bootloader.sh

and type YES.

Now you don’t need to warm it up.