ZFS I/O strangeness
I have a ZFS RAIDZ3 filesystem under OpenSolaris (actually a Nexenta build so a GNU userland on top of OpenSolaris kernel). "dd if=filename of=/dev/null bs=8k" gives me ~66Mbytes/sec. "cat filename | dd of=/dev/null bs=8k" gives me ~142Mbytes/sec. WTF? Why would using cat *increase* the overall speed? The only thing I can think of is readahead, but why would cat cause that to happen while dd doesn't? -Adam Thompson athompso@athompso.net
I'd expect the readahead to be the same for the two commands, as it should all happen behind the scenes at the filesystem level. Or does the C library, or the cat command itself, do its own readahead? More likely cat is using a different input buffer size than 8k and that is resulting in the extra efficiency. I'd try different bs= values on the first dd command to see if that helps. On 03/03/2012 12:52 PM, Adam Thompson wrote:
I have a ZFS RAIDZ3 filesystem under OpenSolaris (actually a Nexenta build so a GNU userland on top of OpenSolaris kernel).
"dd if=filename of=/dev/null bs=8k" gives me ~66Mbytes/sec. "cat filename | dd of=/dev/null bs=8k" gives me ~142Mbytes/sec.
WTF? Why would using cat *increase* the overall speed? The only thing I can think of is readahead, but why would cat cause that to happen while dd doesn't?
-Adam Thompson athompso@athompso.net
_______________________________________________ Roundtable mailing list Roundtable@muug.mb.ca http://www.muug.mb.ca/mailman/listinfo/roundtable
-- Gilles R. Detillieux E-mail:<grdetil@scrc.umanitoba.ca> Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/ Dept. Physiology, U. of Manitoba Winnipeg, MB R3E 0J9 (Canada)
On 2012-03-03 Adam Thompson wrote:
I have a ZFS RAIDZ3 filesystem under OpenSolaris (actually a Nexenta build so a GNU userland on top of OpenSolaris kernel).
"dd if=filename of=/dev/null bs=8k" gives me ~66Mbytes/sec. "cat filename | dd of=/dev/null bs=8k" gives me ~142Mbytes/sec.
Adam, you said you get optimal speed at 8M but your examples shows 8k! The above makes perfect sense as cat will use ibs>8k and dd is using 8k. I bet you'd get 142 if you did for the dd ibs=1M obs=8k, which is probably what the cat line is doing behind the scenes. Methinks your problem was a typo.
participants (3)
-
Adam Thompson -
Gilles Detillieux -
Trevor Cordes