Blog‎ > ‎RaspPhotoPipe - En‎ > ‎

Storage Expansion - Part II

posted 23 Mar 2013, 12:53 by Flip Pipe   [ updated 23 Mar 2013, 13:40 ]
As I wrote in February, one of the purposes is to expand the CF card of the camera. After some weeks of experiences finally I choose something for my project.
  • Three USB 3.0 Flash Drive in RAID 5.

This decision was made after about six weeks of investigations to reach this statement before.

What type of USB pen drive to use

My first and probably most obvious decision was to use USB 2.0 pen drives, and there are two kinds of this: UDP and COB (Chip on Board).

A description about the differences can be found here, but it don't say about life time and throughput. I try some help from StackExchange but without luck so far.

At this point the only comparison I've was two usb pens. Using Disk Utility from Ubuntu I got some results:


With this results and because they are smaller, I focus in UDP.

Where to Buy

My first thought was buy it directly from China and from the manufactures to be cheaper and to buy it with any casing.

I find a site of business to business:

I've contact about 6, all answer to my questions, but in the end, the MOQ (minimum order quantity) was 50 unities, and at the best I will use 6 in a more elaborated RAID configuration.

One of the manufactures, point me the wholesale version of Alibaba:

But, without casing I didn't find any product, to be cheap, the MOQ was 10 unities.

So lets check the more conventional stores.

After some hours in MyMemory comparing flash drives, I choose a Kingston DataTraveler USB 3, 32 GB.

"The price is identical to USB 2, even being a lousy USB 3, should at least have the same performance as USB 2." This was my thoughts to buy it... and buy 6 of them.

By curiosity, MyMemory send me each of them in a different package. Not very ambient friendly.

When it arrives, I've done some tests with Disk Utility, and for me was a little disappointing.


As you can see, reading has a good throughput, but writing it is as good as my UDP USB 2.0 flash drive.

Now, with 6 flash drives, I will not buy others. But there is a peak of hope... like the first peak in writing. Probably, that peak can be enough for in the overall RAID give some gain.

What RAID level to choose

I will not explain the differences between RAID level because they are well documented in the web.

RAID 0 - Gives the best write performance and best price per GB, but it is not usable because if one pen fail, all RAID will fail, but will test it for comparison.

RAID 5 - Good price per GB, but it lacks write performance, but how much?

RAID 50 and 05 - At a little higher cost of price, but joining the striping with mirroring, probably will get bester performance than RAID 5.

RAID 10 and 01 - The most expensive in consideration of price, but it will have a great performance... well lets test it.

To do the testing, I've connect the 6 pens to the computer and slice each in 6 partitions

With it, I've created 6 RAID drives, each one with 6 partitions, one of each pen and in the end have this scenario:

The first 3 are RAID levels directly supported by mdadm, but the other three are combinations with the RAID levels because they are not directly supported by mdadm.

With this configurations I get this total size of the volumes

And at this point, the winner is RAID 5, followed by their brothers 50 and 05.

Next point was test the writing throughput. To do that, I've done two tests... one with dd coping 4G from /dev/zero, and other more realistic for their use, created 7 directories with 3 sets of pictures, one in raw and other in jpeg. Then with rsync, copied it from memory and between copy of each directory I wait 5 seconds. The last test is closest I remember to simulate the real use. All the tests were done three times to check the consistency.

And the firsts values with dd were disappointing.

All RAIDs with stripping (level 5) the results where miserable. During the tests, I've collected some data how was the writing in each partition with dstat. You have that data and some more graphics in the files of this post. That information allow the creation of this graphic

But how it will compare with a test more similar with the reality.

Any conclusions with this tests... I notice while taking the note of this values, they are very similar during the sets and the rounds of the tests. As you can see, there are many cell with values ~107MB/s, ~35MB/s and ~15MB/s in all RAID levels.

With this results, RAID 5 have more chances of been choose, but why this results. Probably some kind of RAID memory cache before the data be sent to the physical drive.

I've tried to search this question googling a bit, but I find something more useful.
There is a parameter in RAID 5 which affects the performance: stripe_cache_size

In Stack Exchange there are one answer

In this blog it has a graphic how the value of this parameter affects the write performance

The why... I didn't find...

From the default value of 256 I've change to 8192 the redo the tests with RAID 5... check the differences

Write speed during time...

WOW... great difference... with RAID 5 tuned, there is no more questions which RAID level I will use in my RaspPhotoPipe.

But all this tests were done in my desktop, not in the Raspberry PI. So the next post storage will be configuring and fine tuning RAID 5 in RaspPhotoPipe.

Flip Pipe,
23 Mar 2013, 12:53
Flip Pipe,
23 Mar 2013, 12:53
Flip Pipe,
23 Mar 2013, 12:53
Flip Pipe,
23 Mar 2013, 12:53
Flip Pipe,
23 Mar 2013, 12:53
Flip Pipe,
23 Mar 2013, 12:53
Flip Pipe,
23 Mar 2013, 12:53