[RndTbl] RAID5 rebuild performance

Sean Cody sean at tinfoilhat.ca
Thu May 19 21:38:32 CDT 2011


Instead if /dev/zero use another deduce to read blindly.  There is no guarantee /dev/zero will not produce a sparse file.

Generate a realistic data file from a sufferer volume (say the boot volume
or a well used/full USB key or drive).  Better data set, better entropy to avoid caching ruining the results.

I'm interested in the results too!

-- 
Sean (mobile)

On 2011-05-19, at 9:27 PM, "Adam Thompson" <athompso at athompso.net> wrote:

> Well, I did say there was more than one variable involved!  25MB/sec is a bit slow, but I don’t know how efficiently that LSI chip is at managing bus contention, or how efficient the kernel driver for that chip is.  I do know from experience that 8-disk arrays are well into the land of diminishing returns from a speed perspective: RAID-1 on two disks or RAID-10 on four seems to be the sweet spot for speed.
> 
>  
> 
> Once your rebuild has finished, I would recommend doing some throughput tests (both reading and writing) on the array; something perhaps like:
> 
> # sync; time sh –c ‘dd if=/dev/zero of=/mnt/raidarray/BIGFILE.ZERO bs=1M count=1024;sync’
> 
> followed by
> 
> # time dd if=/dev/md0 of=/dev/null bs=1M count=1024
> 
> Those are both very naïve approaches, but should give you a feel for the maximum read and write speeds of your array.  I strongly suspect those numbers will be much higher than the raid re-sync rate, again, mostly due to write-flush barriers in the md code.
> 
>  
> 
> I’m interested in knowing how this ends, personally… please let us know.
> 
>  
> 
> -Adam
> 
>  
> 
> (P.S. does anyone know how to avoid top-posting in Outlook 2010?)
> 
>  
> 
>  
> 
> From: roundtable-bounces at muug.mb.ca [mailto:roundtable-bounces at muug.mb.ca] On Behalf Of Kevin McGregor
> Sent: Thursday, May 19, 2011 14:52
> To: Continuation of Round Table discussion
> Subject: Re: [RndTbl] RAID5 rebuild performance
> 
>  
> 
> raid6: using algorithm sse2x2 (2883 MB/s). So, 1% of that is reasonable? :-) Oh well, I guess I can wait until tomorrow for the rebuild to finish.
> 
> On Thu, May 19, 2011 at 2:12 PM, Adam Thompson <athompso at athompso.net> wrote:
> 
> Look back in dmesg output for the RAID module speed tests, notice which one was selected, and divide that number by 2.  There's your theoretical bottleneck on the CPU.
> Take the minimum sustained disk/channel/controller throughput, factor in interrupt latency, device driver efficiency, etc. and make a rough guess as to the overall throughput.
> Consider that md code seems to have a lot of write barriers for safety -  so even a rebuild will aspens much of its time waiting for the disk to sync().
> All in all, I think your numbers are probably reasonable.
> -Adam
> 
> 
> 
> Kevin McGregor <kevin.a.mcgregor at gmail.com> wrote:
> 
> >I installed Ubuntu Server 10.04.2 LTS AMD64 on a HP ProLiant ML370 G3 (4 x
> >dual-core/hyperthreaded Xeon 2.66 GHz, 8 GB RAM) and I used the on-board
> >SCSI controller to manage 8 x 300 GB 15K RPM SCSI drives in a software RAID
> >5 set up as a 7-drive array with 1 hot-spare drive. All drives are the exact
> >same model with the same firmware version.
> >
> >It's currently rebuilding the array (because I just created the array) and
> >/proc/mdstat is reporting "finish=165.7min speed=25856K/sec". Does that
> >sound "right" in the sense that it's the right order of magnitude? I though
> >it should be higher, but I haven't set up such an array before, so I don't
> >have anything to compare it to.
> >
> >If it's slow, does anyone have a suggestion for speeding it up?
> >
> >Kevin
> >
> >_______________________________________________
> >Roundtable mailing list
> >Roundtable at muug.mb.ca
> >http://www.muug.mb.ca/mailman/listinfo/roundtable
> 
> _______________________________________________
> Roundtable mailing list
> Roundtable at muug.mb.ca
> http://www.muug.mb.ca/mailman/listinfo/roundtable
> 
>  
> 
> _______________________________________________
> Roundtable mailing list
> Roundtable at muug.mb.ca
> http://www.muug.mb.ca/mailman/listinfo/roundtable
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.muug.mb.ca/pipermail/roundtable/attachments/20110519/d9641762/attachment-0001.html 


More information about the Roundtable mailing list