I have a ZFS RAIDZ3 filesystem under OpenSolaris (actually a Nexenta build so a GNU userland on top of OpenSolaris kernel).
"dd if=filename of=/dev/null bs=8k" gives me ~66Mbytes/sec. "cat filename | dd of=/dev/null bs=8k" gives me ~142Mbytes/sec.
WTF? Why would using cat *increase* the overall speed? The only thing I can think of is readahead, but why would cat cause that to happen while dd doesn't?
-Adam Thompson athompso@athompso.net
I'd expect the readahead to be the same for the two commands, as it should all happen behind the scenes at the filesystem level. Or does the C library, or the cat command itself, do its own readahead? More likely cat is using a different input buffer size than 8k and that is resulting in the extra efficiency. I'd try different bs= values on the first dd command to see if that helps.
On 03/03/2012 12:52 PM, Adam Thompson wrote:
I have a ZFS RAIDZ3 filesystem under OpenSolaris (actually a Nexenta build so a GNU userland on top of OpenSolaris kernel).
"dd if=filename of=/dev/null bs=8k" gives me ~66Mbytes/sec. "cat filename | dd of=/dev/null bs=8k" gives me ~142Mbytes/sec.
WTF? Why would using cat *increase* the overall speed? The only thing I can think of is readahead, but why would cat cause that to happen while dd doesn't?
-Adam Thompson athompso@athompso.net
Roundtable mailing list Roundtable@muug.mb.ca http://www.muug.mb.ca/mailman/listinfo/roundtable
On 2012-03-03 Adam Thompson wrote:
I have a ZFS RAIDZ3 filesystem under OpenSolaris (actually a Nexenta build so a GNU userland on top of OpenSolaris kernel).
"dd if=filename of=/dev/null bs=8k" gives me ~66Mbytes/sec. "cat filename | dd of=/dev/null bs=8k" gives me ~142Mbytes/sec.
Adam, you said you get optimal speed at 8M but your examples shows 8k! The above makes perfect sense as cat will use ibs>8k and dd is using 8k. I bet you'd get 142 if you did for the dd ibs=1M obs=8k, which is probably what the cat line is doing behind the scenes. Methinks your problem was a typo.
-
Adam Thompson -
Gilles Detillieux -
Trevor Cordes