As per the title, a little while ago I tested a range of microSD cards covering most brands and hopefully, it’s OK to share those results here on the forums!
If anyone has any questions, feel free to ask away…
Thanks,
Bret
I tested 25+ microSD cards in the 5B if it helps anyone!
2 Likes
Can you please run an sbc-bench -S with this SD card inserted?
1 Like
does this mean that the sd card controller is limited to 75mb/s?
Sure!
58.9GB "SK Hynix USD00" UHS SDR104 SDXC card as /dev/mmcblk1: date 03/2022, manfid/oemid: 0x0000ad/0x4c53, hw/fw rev: 0x1/0x0Nope since I measured ~90 MB/sec: https://github.com/ThomasKaiser/Knowledge/blob/master/articles/Quick_Preview_of_ROCK_5B.md#sd-card
The clock can be defined via DT as explained there and at least in the beginning Radxa shipped their OS images with some serious downclocking for reasons unknown to me.
I’ll be looking to do some new testing in regard to storage when I publish a standalone 5B benchmark piece so I’ll keep this in mind and play around with the different clocks. I’ve unfortunately lost the supporting data since performing these tests so I don’t recall the exact image/settings used so I need to re-do them ideally anyway!
Thanks for Your hard work, but with Rock5 much more interesting would be nvme tests, for now sdxc uhs-1 is about 90MB/s, eMMC is about 370MB/s and nvme has about 2700MB/s o my cheap sn570 (and upt to 3700MB/s with some sequential reads). It’s just no reason to use SD card there 
This may be more handy for ROCK 5a, but still eMMC is that option there.
UHS-1 (SDR104) is limited to 104MB/s theoretically. Still it’s much less than everything else on this board.
1 Like
Yup, no doubt - however whilst I have 4 NVMe drives available to me, only 1 is “spare” and used for testing, the rest are in live machines. Sadly I’m not a big review house with loads of $$$ laying around, nor am I big enough for companies to want to send me these items 
The Crucial P3 Plus is the only drive I have laying around so that’s the only data I can add to that, sadly.
I notice now though that I don’t have the eMMC results as I only received the module after I posted this piece. I’ll update to include that this week.
And measuring sequential transfer speeds is somewhat silly anyway. Use case first! And which use case benefits from high sequential transfer speeds with SD card? None. It’s all about random I/O if the rootfs resides on SD card which ofc also benefits from UHS-1 (SDR104).
Sure, just worth to know up limit to see if result is ok.
90MB/s then is quite ok for sd card (if You want to use it BTW
It’s just a random number unless we really start to think about how this number has been generated.
People always forget that a specific benchmark score is not the result of ‘only testing the hardware’ but a combination of hardware + software + settings.
When I test an SD card I use a block size of up to 16MB and a single task accessing the storage: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
EDIT: And i switch all cpufreq governors from their defaults to performance and of course let the benchmark be executed on a big cores since unless you do this you’re benchmarking mostly settings. There is one such setting affecting especially random I/O performance: https://github.com/radxa/kernel/commit/55f540ce97a3d19330abea8a0afc0052ab2644ef#commitcomment-79484235
This suggestion has been adopted by Radxa few weeks ago so w/o tuning settings prior to benchmarking you get better scores on recent Radxa provided OS images compared to e.g. Armbian (or lame Armbian derivates like DietPi)
As can be seen by the link I referenced before block size and access patterns matter. And at least I’ve no idea how Bret generated his numbers.
If Bret used only 1M block size his numbers are of course lower than mine even if his cards are faster. If he does a common mistake in the SBC world and used hdparm then he was testing with just 128K block size which is laughable small by today’s standards and especially by today’s real-world access patterns if it’s about sequential transfer speeds.
If we both would use a test utilizing 8 tasks in parallel (switching from iozone to fio for example) maybe we would be able to generate numbers above 100 MB/s.
Alrighty, so yup, there are likely some less-than-ideal situations happening in the tests and I’ve already made those changes in the benchmark “suite” that I’m using for all new boards that I test. Initially, though, I’d used the same tests as Jeff Geerling for his Raspberry Pi microSD card benchmarks (including hdparm) so people could compare them, though in hindsight, due to the reasons you’ve mentioned, I realised this wasn’t ideal and I started using fio more in the storage parts of my tests but I was still using smaller block sizes. I’m very grateful for your examples, though and as I’ve not yet published any reviews using the new suite, I’ll look at running some differently configured tests.
I’m still learning a lot as I go along with this testing and I’ve used your own tools/documentation as part of that. I’ve also been trying to balance the information provided when it comes to these “Best microSD card for X” posts so it’s somewhat digestible to people that may not be as knowledgeable and just want to see some straightforward-ish numbers. Saying that, though, I can still improve the way I get those 4 metrics so they’re a little more “real-world”, or I can look to include a couple of options and clear explanations about why.
Going back to the ROCK 5B and the Amazon SD card specifically here, I’ve used your iozone example (governors were already set to performance and iozone was pinned to a big core in the cluster with a slightly higher clock and no other processes accessing it) and indeed, we’re seeing some different numbers.
random random
kB reclen write rewrite read reread read write
102400 4 5963 5767 18522 18554 12898 5214
102400 16 22148 22326 45633 45130 35920 21461
102400 512 64374 63502 81649 81687 81319 64840
102400 1024 74159 75150 84088 84177 83641 74888
102400 16384 78382 73972 89669 89345 89180 74348
I’ll look to update the testing here and include that in the standalone board reviews as well as go back through older posts to update these whilst trying not to overload them.
Thanks again, if you want me to try anything else then let me know 
Yep, all consumers usually want is a bunch of numbers (presented as nice graphs of course) and ‘more is better’ or ‘less is better’ to avoid having to use their brain when making buying decisions.
But even those people deserve better since the average buyer of an SD card to be used in a SBC needs to be aware of a few pieces of information instead of staring only at graphs:
- Counterfeit flash memory is still a problem. Always ensure that you’re buying somewhere where a ‘no questions asked’ return/refund policy exists since it’s somewhat probable that you’ll end up with fake products (applies not only to SD cards but to all sorts of flash memory, SSDs included).
- As such product rankings of this sort are questionable by definition since even if the honest reviewer tested e.g. brand A to be the best, the person reading the review and deciding to base his own purchase decision on this might end up buying a fake A product. This actually happened and happens again and again: when for example Samsung entered the SD card market and countless reviews attested them to outperform their competition soon after the counterfeit crown went from Kingston to Samsung (the counterfeiters ofc printed the most popular logo on their fake products)
- Another problem is somewhat related: different production batches might perform differently and this applies especially to vendors who just buy whatever raw NAND flash available on the stock market to be combined with whatever FTL (flash translation layer) controller flying around. This affects all but a few manufacturers like Samsung, SanDisk, Kioxa (those have both knowledge and factories to produce their own flash and controllers) since all the others just do this: combine someone else’s flash chips + controllers and print your name on it (some like Kingston, Adata or Lexar then often ‘brand’ the controller firmware which identifies then as Kingston, Adata or Lexar even if it’s in reality a Phison, TI or Silicon Power controller). But even those vendors that produce their own NAND flash and their own controllers are affected by a similar effect: see my tests with SanDisk A1 and A2 rated cards from half a decade ago where the older and ‘lower’ specced A1 card outperformed the newer A2 card.
- Use case first! With the rootfs on an SD card almost all that matters is random I/O while sequential transfer speeds are (close to) irrelevant. Well, we all know that every consumer out there is trained to look at the wrong metrics… and that’s why it’s important to repeat this over and over again!
- Choose the right spec: over half a decade ago some of us did the painful job to test through various brands (me also starting with @geerlingguy’s iozone parameters just adding 16M block size since more appropriate) to measure which vendor currently provides the cards with highest random I/O. But fortunately that’s not necessary any more since SD Association defined Application Performance Classes A1 and A2.
So what an SD card buyer wanting to push this thing into any SBC really needs to know is only the following:
- forget about the brand but choose the seller wisely (‘no questions asked’ return/refund policy)
- choose A1 or A2 rated cards only (and if you’re also after higher sustained sequential transfer speeds then pay attention to the video ratings, e.g. choose something where next to the A1/A2 logo also a V30 logo is printed since such a card is not only fit for random I/O but has also to provide 30MB/s sustained write speeds)
- don’t trust in benchmarks somewhere on the net for these simple reasons: the card you’ll end up buying might be a counterfeit card or from a different batch (e.g. ‘Amazon Basics’ acquiring not only FTL controllers and flash from ‘SK Hynix’ but also from other vendors so your own card and the one from the reviewer have nothing in common except the logos printed onto the card’s surface.
- ALWAYS test the card you buy immediately! ALWAYS! That’s the only way to check for counterfeit flash products. With an SBC running Linux it’s as easy as using f3. 1st check with f3probe, if the card already fails with this return immediately to the seller. If this 1st check passes the 2nd check has to be made with f3write/f3read. This also tells you real sequential performance of the device you bought and is the only way to ensure you’ve not bought fake flash product.
And I guess that pretty much describes the dilemma: consumers love these lists of useless numbers presented as nice graphs, they look only at the irrelevant metrics and forget that the product they’re about to buy might have nothing in common with the one tested/benchmarked other than some logos. IMO these consumers would be better served by some educational sentences no-one likes to hear than those product listings
The note on the bait/switch with flash and overall quality on these Amazon cards is something I’ve been a little worried about and I did touch on this in a couple of my pieces, as well as a standalone piece. So far, I’ve ordered 64GB cards from Amazon UK 18 months apart and from Amazon DE around 12 months after the 1st UK purchase. I also picked up a 128GB card to see whether there were any differences there and I can see SK Hynix chips in all of them, with different manufacturing dates over a year apart. I’m due to make another purchase in a few months so if at any point I start seeing different data, I’ll be going back to update the posts.
I haven’t mentioned (to the best of my knowledge) how to check for fake memory in any of my pieces, though I have touched on the potential downfalls of Amazon’s inventory system and dishonest players ruining the pool by mixing in counterfeit cards. It’s all good information though and I’ll be sure to keep this in mind to try and highlight moving forward.
I’m also wanting to do some endurance testing but I need to try and write something that does it reliably whilst reporting useful data that I can then take out. I’m open to any ideas you may have on that front!
I don’t know where you get those ultra fast speed on your Nvme. Using f3read / f3write i get 650 MB/s / 150 MB/s (average). Can you run it on your Nvme for real life benchmark, so it is possible to compare? I don’t see anyone doing this, we need real life values and results. I would encourage anyone with Nvme to run these tests and create a single thread to write down their results.
One of my Nvme is slow but is certainly way faster than eMMC.
SD card, eMMC, and Nvme get slower and slower while filling up, that’s what i call real life.
Reading and writing a few MB is indeed lightning fast.
One thing to note: The kernel provided by Rockchip is v5.10 of the Android fork. At least in mainline 5.10, (not sure about the Android fork), of the features required to properly drive A2-spec SD cards, only command queuing is supported, but not write caching, which was apparently implemented only in 5.15 (see https://forum.odroid.com/viewtopic.php?t=44076). Now, it’s possible that Rockchip or Google has backported write caching support, but I would still be cautious about random read/write tests on an A2 card with a 5.10 kernel.
EDIT: Turns out command queue support is implemented, but only enabled for eMMC devices, not SD, even in recent kernels.
That’s something to consider, yup, though there are only so many disclaimers you can put out there and this is what is available to users of the board. If it’s not available in the Rockchip BSP kernel then this is all I can test.
At first, I thought about bending over backwards to fix everything in my general board reviews but ultimately I’ve decided that if it’s “fixable” and the manufacturer/vendor doesn’t want to include that in their own images then they’ll be tested in the way that they’re provided to users and I’ll touch on the modifications needed in a separate post later. Sorry, went on a bit of a tangent here 
Completely agree, but just You need to get some idea what card should be able to do compared to different cards. Good example for that are some good 4k action cams that won’t record in 4k unless card passes write test (and this is perfectly understandable).
Atto disk benchmark on windows shows that with nice graph, block size from few kilobytes to megabytes, both read and write. Replicating something like this in sbc-bench would be awesome! maybe with modified several things, less block size beside that I always liked when those
Yes, and least important today is sd card durability. There are some products that are marked as bit better but still card will just fail some day.
Totally agree. I’ve got some fake cards and they look identical even with small details, but they just don’t work in good camera with high speed writes. For SD fake cards are usually about 1/4 speed at write. I also got some that were much smaller and because of that not usable, correcting their controller to report right size was not easy, but I managed to get them to work.
Those were just some quick tests, not to get highest amount but just to get idea what is the difference on those two (eMMC and few nvmes). As Thomas said number alone has no meaning, but it’s worth to just check if everything is ok, like on ROCK 4 when adding pcie2-gen will effectively double measured speed. Here it’s same thing - I won’t say You will get this number constantly, but You will easily get idea about difference to other storage options
Defrag still in mind?
Sure, it’s just much faster than anyother storage option so easy to said that nvme is at least 8x better than eMMC which is about 4x faster than sd card, also much durable. (let’s not argue about what x one is better than another, even 2x is much faster for me)
Of course still You can connect slow card and get fraction of speed. That is why worth to benchmarking.
Having said all that I still install my os on an emmc module-it just works with no bs.
1 Like
Well, with “Linux defaults” on everything except Armbian or DietPi the card will be simply logged to death. The problem is called ‘Write Amplification’ and those Linux defaults (ext4 default commit interval and systemd logging behaviour) will result in your flash memory wearing out way earlier than necessary. And this does not only affect SD cards but all sorts of flash memory, as such eMMC and SSDs too.
The fix is easy: https://github.com/ThomasKaiser/Knowledge/blob/master/articles/Quick_Review_of_RPi_Zero_2_W.md#sd-card-endurance (though that doesn’t cover TRIM but the way I use SD cards I don’t have to care anyway).
Unlike SD cards and eMMC with SSDs majority of them at least reports the FTL controller’s idea of how worn out the flash memory would be and this is something sbc-bench now reports with -S and -r modes:
sbc-bench.sh -S
* 238.5GB "KXG50ZNV256G NVMe TOSHIBA 256GB" SSD as /dev/nvme0:
Speed 8GT/s (ok), Width x4 (ok), 13% worn out, unhealthy drive
temp: 67°C
* 14.5GB "Foresee NCard" HS200 eMMC 5.0 card as /dev/mmcblk0:
date 08/2016, manfid/oemid: 0x000088/0x0103, hw/fw rev: 0x0/0x0100000000000000
* Macronix MX25U12835F 16MB SPI NOR flash, drivers in use: spi-nor/rockchip-sfc
BTW: thanks to @Googulator’s link to ODROID forum I learned that with recent hdparm versions the block size must have been increased a lot: https://forum.odroid.com/viewtopic.php?f=205&t=44025
At least in the past hdparm always used hardcoded 128K and this resulted in rather low sequential transfer speeds reported. As always: benchmarking the benchmark should be the first step but in the SBC world this never happens…