Well, I did say there was more than one variable involved! 25MB/sec is a bit slow, but I don’t know how efficiently that LSI chip is at managing bus contention, or how efficient the kernel driver for that chip is. I do know from experience that 8-disk arrays are well into the land of diminishing returns from a speed perspective: RAID-1 on two disks or RAID-10 on four seems to be the sweet spot for speed.
Once your rebuild has finished, I would recommend doing some throughput tests (both reading and writing) on the array; something perhaps like:
# sync; time sh –c ‘dd if=/dev/zero of=/mnt/raidarray/BIGFILE.ZERO bs=1M count=1024;sync’
followed by
# time dd if=/dev/md0 of=/dev/null bs=1M count=1024
Those are both very naïve approaches, but should give you a feel for the maximum read and write speeds of your array. I strongly suspect those numbers will be much higher than the raid re-sync rate, again, mostly due to write-flush barriers in the md code.
I’m interested in knowing how this ends, personally… please let us know.
-Adam
(P.S. does anyone know how to avoid top-posting in Outlook 2010?)
From: roundtable-bounces@muug.mb.ca [mailto:roundtable-bounces@muug.mb.ca] On Behalf Of Kevin McGregor Sent: Thursday, May 19, 2011 14:52 To: Continuation of Round Table discussion Subject: Re: [RndTbl] RAID5 rebuild performance
raid6: using algorithm sse2x2 (2883 MB/s). So, 1% of that is reasonable? :-) Oh well, I guess I can wait until tomorrow for the rebuild to finish.
On Thu, May 19, 2011 at 2:12 PM, Adam Thompson athompso@athompso.net wrote:
Look back in dmesg output for the RAID module speed tests, notice which one was selected, and divide that number by 2. There's your theoretical bottleneck on the CPU. Take the minimum sustained disk/channel/controller throughput, factor in interrupt latency, device driver efficiency, etc. and make a rough guess as to the overall throughput. Consider that md code seems to have a lot of write barriers for safety - so even a rebuild will aspens much of its time waiting for the disk to sync(). All in all, I think your numbers are probably reasonable. -Adam
Kevin McGregor kevin.a.mcgregor@gmail.com wrote:
I installed Ubuntu Server 10.04.2 LTS AMD64 on a HP ProLiant ML370 G3 (4 x dual-core/hyperthreaded Xeon 2.66 GHz, 8 GB RAM) and I used the on-board SCSI controller to manage 8 x 300 GB 15K RPM SCSI drives in a software RAID 5 set up as a 7-drive array with 1 hot-spare drive. All drives are the exact same model with the same firmware version.
It's currently rebuilding the array (because I just created the array) and /proc/mdstat is reporting "finish=165.7min speed=25856K/sec". Does that sound "right" in the sense that it's the right order of magnitude? I though it should be higher, but I haven't set up such an array before, so I don't have anything to compare it to.
If it's slow, does anyone have a suggestion for speeding it up?
Kevin
Roundtable mailing list Roundtable@muug.mb.ca http://www.muug.mb.ca/mailman/listinfo/roundtable
_______________________________________________ Roundtable mailing list Roundtable@muug.mb.ca http://www.muug.mb.ca/mailman/listinfo/roundtable