Yup, no doubt - however whilst I have 4 NVMe drives available to me, only 1 is “spare” and used for testing, the rest are in live machines. Sadly I’m not a big review house with loads of $$$ laying around, nor am I big enough for companies to want to send me these items The Crucial P3 Plus is the only drive I have laying around so that’s the only data I can add to that, sadly.
I notice now though that I don’t have the eMMC results as I only received the module after I posted this piece. I’ll update to include that this week.
And measuring sequential transfer speeds is somewhat silly anyway. Use case first! And which use case benefits from high sequential transfer speeds with SD card? None. It’s all about random I/O if the rootfs resides on SD card which ofc also benefits from UHS-1 (SDR104).
It’s just a random number unless we really start to think about how this number has been generated.
People always forget that a specific benchmark score is not the result of ‘only testing the hardware’ but a combination of hardware + software + settings.
This suggestion has been adopted by Radxa few weeks ago so w/o tuning settings prior to benchmarking you get better scores on recent Radxa provided OS images compared to e.g. Armbian (or lame Armbian derivates like DietPi)
As can be seen by the link I referenced before block size and access patterns matter. And at least I’ve no idea how Bret generated his numbers.
If Bret used only 1M block size his numbers are of course lower than mine even if his cards are faster. If he does a common mistake in the SBC world and used hdparm then he was testing with just 128K block size which is laughable small by today’s standards and especially by today’s real-world access patterns if it’s about sequential transfer speeds.
If we both would use a test utilizing 8 tasks in parallel (switching from iozone to fio for example) maybe we would be able to generate numbers above 100 MB/s.
Alrighty, so yup, there are likely some less-than-ideal situations happening in the tests and I’ve already made those changes in the benchmark “suite” that I’m using for all new boards that I test. Initially, though, I’d used the same tests as Jeff Geerling for his Raspberry Pi microSD card benchmarks (including hdparm) so people could compare them, though in hindsight, due to the reasons you’ve mentioned, I realised this wasn’t ideal and I started using fio more in the storage parts of my tests but I was still using smaller block sizes. I’m very grateful for your examples, though and as I’ve not yet published any reviews using the new suite, I’ll look at running some differently configured tests.
I’m still learning a lot as I go along with this testing and I’ve used your own tools/documentation as part of that. I’ve also been trying to balance the information provided when it comes to these “Best microSD card for X” posts so it’s somewhat digestible to people that may not be as knowledgeable and just want to see some straightforward-ish numbers. Saying that, though, I can still improve the way I get those 4 metrics so they’re a little more “real-world”, or I can look to include a couple of options and clear explanations about why.
Going back to the ROCK 5B and the Amazon SD card specifically here, I’ve used your iozone example (governors were already set to performance and iozone was pinned to a big core in the cluster with a slightly higher clock and no other processes accessing it) and indeed, we’re seeing some different numbers.
I’ll look to update the testing here and include that in the standalone board reviews as well as go back through older posts to update these whilst trying not to overload them.
Thanks again, if you want me to try anything else then let me know
Yep, all consumers usually want is a bunch of numbers (presented as nice graphs of course) and ‘more is better’ or ‘less is better’ to avoid having to use their brain when making buying decisions.
But even those people deserve better since the average buyer of an SD card to be used in a SBC needs to be aware of a few pieces of information instead of staring only at graphs:
Counterfeit flash memory is still a problem. Always ensure that you’re buying somewhere where a ‘no questions asked’ return/refund policy exists since it’s somewhat probable that you’ll end up with fake products (applies not only to SD cards but to all sorts of flash memory, SSDs included).
As such product rankings of this sort are questionable by definition since even if the honest reviewer tested e.g. brand A to be the best, the person reading the review and deciding to base his own purchase decision on this might end up buying a fake A product. This actually happened and happens again and again: when for example Samsung entered the SD card market and countless reviews attested them to outperform their competition soon after the counterfeit crown went from Kingston to Samsung (the counterfeiters ofc printed the most popular logo on their fake products)
Another problem is somewhat related: different production batches might perform differently and this applies especially to vendors who just buy whatever raw NAND flash available on the stock market to be combined with whatever FTL (flash translation layer) controller flying around. This affects all but a few manufacturers like Samsung, SanDisk, Kioxa (those have both knowledge and factories to produce their own flash and controllers) since all the others just do this: combine someone else’s flash chips + controllers and print your name on it (some like Kingston, Adata or Lexar then often ‘brand’ the controller firmware which identifies then as Kingston, Adata or Lexar even if it’s in reality a Phison, TI or Silicon Power controller). But even those vendors that produce their own NAND flash and their own controllers are affected by a similar effect: see my tests with SanDisk A1 and A2 rated cards from half a decade ago where the older and ‘lower’ specced A1 card outperformed the newer A2 card.
Use case first! With the rootfs on an SD card almost all that matters is random I/O while sequential transfer speeds are (close to) irrelevant. Well, we all know that every consumer out there is trained to look at the wrong metrics… and that’s why it’s important to repeat this over and over again!
Choose the right spec: over half a decade ago some of us did the painful job to test through various brands (me also starting with @geerlingguy’s iozone parameters just adding 16M block size since more appropriate) to measure which vendor currently provides the cards with highest random I/O. But fortunately that’s not necessary any more since SD Association defined Application Performance Classes A1 and A2.
So what an SD card buyer wanting to push this thing into any SBC really needs to know is only the following:
forget about the brand but choose the seller wisely (‘no questions asked’ return/refund policy)
choose A1 or A2 rated cards only (and if you’re also after higher sustained sequential transfer speeds then pay attention to the video ratings, e.g. choose something where next to the A1/A2 logo also a V30 logo is printed since such a card is not only fit for random I/O but has also to provide 30MB/s sustained write speeds)
don’t trust in benchmarks somewhere on the net for these simple reasons: the card you’ll end up buying might be a counterfeit card or from a different batch (e.g. ‘Amazon Basics’ acquiring not only FTL controllers and flash from ‘SK Hynix’ but also from other vendors so your own card and the one from the reviewer have nothing in common except the logos printed onto the card’s surface.
ALWAYS test the card you buy immediately! ALWAYS! That’s the only way to check for counterfeit flash products. With an SBC running Linux it’s as easy as using f3. 1st check with f3probe, if the card already fails with this return immediately to the seller. If this 1st check passes the 2nd check has to be made with f3write/f3read. This also tells you real sequential performance of the device you bought and is the only way to ensure you’ve not bought fake flash product.
And I guess that pretty much describes the dilemma: consumers love these lists of useless numbers presented as nice graphs, they look only at the irrelevant metrics and forget that the product they’re about to buy might have nothing in common with the one tested/benchmarked other than some logos. IMO these consumers would be better served by some educational sentences no-one likes to hear than those product listings
The note on the bait/switch with flash and overall quality on these Amazon cards is something I’ve been a little worried about and I did touch on this in a couple of my pieces, as well as a standalone piece. So far, I’ve ordered 64GB cards from Amazon UK 18 months apart and from Amazon DE around 12 months after the 1st UK purchase. I also picked up a 128GB card to see whether there were any differences there and I can see SK Hynix chips in all of them, with different manufacturing dates over a year apart. I’m due to make another purchase in a few months so if at any point I start seeing different data, I’ll be going back to update the posts.
I haven’t mentioned (to the best of my knowledge) how to check for fake memory in any of my pieces, though I have touched on the potential downfalls of Amazon’s inventory system and dishonest players ruining the pool by mixing in counterfeit cards. It’s all good information though and I’ll be sure to keep this in mind to try and highlight moving forward.
I’m also wanting to do some endurance testing but I need to try and write something that does it reliably whilst reporting useful data that I can then take out. I’m open to any ideas you may have on that front!
I don’t know where you get those ultra fast speed on your Nvme. Using f3read / f3write i get 650 MB/s / 150 MB/s (average). Can you run it on your Nvme for real life benchmark, so it is possible to compare? I don’t see anyone doing this, we need real life values and results. I would encourage anyone with Nvme to run these tests and create a single thread to write down their results.
One of my Nvme is slow but is certainly way faster than eMMC.
SD card, eMMC, and Nvme get slower and slower while filling up, that’s what i call real life.
Reading and writing a few MB is indeed lightning fast.
One thing to note: The kernel provided by Rockchip is v5.10 of the Android fork. At least in mainline 5.10, (not sure about the Android fork), of the features required to properly drive A2-spec SD cards, only command queuing is supported, but not write caching, which was apparently implemented only in 5.15 (see https://forum.odroid.com/viewtopic.php?t=44076). Now, it’s possible that Rockchip or Google has backported write caching support, but I would still be cautious about random read/write tests on an A2 card with a 5.10 kernel.
EDIT: Turns out command queue support is implemented, but only enabled for eMMC devices, not SD, even in recent kernels.
That’s something to consider, yup, though there are only so many disclaimers you can put out there and this is what is available to users of the board. If it’s not available in the Rockchip BSP kernel then this is all I can test.
At first, I thought about bending over backwards to fix everything in my general board reviews but ultimately I’ve decided that if it’s “fixable” and the manufacturer/vendor doesn’t want to include that in their own images then they’ll be tested in the way that they’re provided to users and I’ll touch on the modifications needed in a separate post later. Sorry, went on a bit of a tangent here
Completely agree, but just You need to get some idea what card should be able to do compared to different cards. Good example for that are some good 4k action cams that won’t record in 4k unless card passes write test (and this is perfectly understandable).
Atto disk benchmark on windows shows that with nice graph, block size from few kilobytes to megabytes, both read and write. Replicating something like this in sbc-bench would be awesome! maybe with modified several things, less block size beside that I always liked when those
Yes, and least important today is sd card durability. There are some products that are marked as bit better but still card will just fail some day.
Totally agree. I’ve got some fake cards and they look identical even with small details, but they just don’t work in good camera with high speed writes. For SD fake cards are usually about 1/4 speed at write. I also got some that were much smaller and because of that not usable, correcting their controller to report right size was not easy, but I managed to get them to work.
Those were just some quick tests, not to get highest amount but just to get idea what is the difference on those two (eMMC and few nvmes). As Thomas said number alone has no meaning, but it’s worth to just check if everything is ok, like on ROCK 4 when adding pcie2-gen will effectively double measured speed. Here it’s same thing - I won’t say You will get this number constantly, but You will easily get idea about difference to other storage options
Defrag still in mind?
Sure, it’s just much faster than anyother storage option so easy to said that nvme is at least 8x better than eMMC which is about 4x faster than sd card, also much durable. (let’s not argue about what x one is better than another, even 2x is much faster for me)
Of course still You can connect slow card and get fraction of speed. That is why worth to benchmarking.
Well, with “Linux defaults” on everything except Armbian or DietPi the card will be simply logged to death. The problem is called ‘Write Amplification’ and those Linux defaults (ext4 default commit interval and systemd logging behaviour) will result in your flash memory wearing out way earlier than necessary. And this does not only affect SD cards but all sorts of flash memory, as such eMMC and SSDs too.
Unlike SD cards and eMMC with SSDs majority of them at least reports the FTL controller’s idea of how worn out the flash memory would be and this is something sbc-bench now reports with -S and -r modes:
At least in the past hdparm always used hardcoded 128K and this resulted in rather low sequential transfer speeds reported. As always: benchmarking the benchmark should be the first step but in the SBC world this never happens…
No, no. Not that bad…
I just wanted to know what benchmark or quick test you ran to compare to my Nvme(s).
Running a complete f3 test and filling up the SD card, Nvme or eMMC gives an accurate picture.
But i don’t advise running it on eMMC, i lost 2 eMMC (32 GB) in one month (each). It was running on an early dev board, maybe there was some kind of overvolting and they fixed it on later versions.
TRIM is unfortunately another feature supported by recent SD cards, but not the Linux kernel. (The feature itself is present, but locked to eMMC only, and never enabled on SD cards.)
BTW, for command queuing, the situation is a bit worse: not only is it only enabled for eMMC, but also only for hosts that can manage the queue entirely in hardware. The Linux kernel itself will never issue CMD44/45 queued requests from software.
Yeah, a lot missing on the software side of things wrt SD cards and Linux. As for TRIM I forgot something important in my above list how to ‘correctly buy SD cards to be used with SBC’: that’s after verifying capacity with f3write/f3read to always use SD Association’s ‘SD Formatter’ (unfortunately not available for FOSS operating systems) to ‘format’ cards again since this implements CMD38 to tell the card’s controller the whole storage being empty again.
Or do you know something better and also working from within Linux (since the last time I visited this area was 6 years ago).
The samsung A2 Pro Plus and A2 EVO are strangely slower and not A2 spec if you test them with the Samsung USB adapter that comes in the pack. (Which I purchased just to make sure).
It’s probably not a UHS issue, rather, the Samsung adapter probably also doesn’t implement queuing and caching, required for proper performance on A2 cards.
The Crucial P3 Plus is the only drive I have laying around so that’s the only data I can add to that, sadly.
