September 2014 - Roundtable

do you have a Seagate external USB3 2TB from 2013-2014?
by Trevor Cordes 11 Sep '14

11 Sep '14

Out of curiosity, if you do and wouldn't mind trading up to a new unit (same size or bigger) at nominal (or no) cost, with imaging to the new drive done for free (if you require, or you can do yourself and DBAN), email me the part # printed on your unit's case. I'm not guaranteeing I'll do this deal, I'm just seeing if anyone has one and might be interested. Guaranteed it would be a win-win for a couple of MUUG members :-)

1 0

linux md RAID6 + XFS + add 1 drive
by Trevor Cordes 10 Sep '14

10 Sep '14

Just some observations, tangential ramblings from a recent experience: For the first time in a long time I ran out of space on my home file server. It was a 8 x 2TB drive linux software (md) RAID6, 12TB usable, with a tweaked XFS on top. 99.5% full and running out of things I can painlessly delete. Yesterday, I added a 2TB 7200rpm WD SE ("enterprise") drive that is actually quite affordable. A very solid, heavy drive. Built like a SCSI. Crossed my fingers and did a md RAID6 grow/reshape, which is quite easy to do, but took 24 hours 10 mins to complete. And I did it while the fs live and mounted and nfs exported! (For my sanity I did turn off some heavy-data daemons.) Hooray, no errors! Read up on XFS grows and did, while mounted, an xfs_grow -d and (surprisingly?) it *instantly* returned and df showed the array magically was 2TB bigger. Hooray #2. Had to adjust my mdadm.conf (I manually specify some parameters) and luckily remembered to also do a dracut --force (Fedora system) for next boot. Had to adjust fstab to tweak sunit/swidth (s==stripe) to have the remount use a new, sane, value for performance. I'm a bit confused after reading the sparse docs and the nearly non-existent googles of people doing precisely this: md grow + xfs grow. Of course, it's not imperative, unless you care about performance. It appears sunit is not changeable unless you increase it by a factor of a power of 2. For instance, doubling your drives would allow you to double your sunit. That kind of sucks. Some articles I read said you should set sunit = RaidChunkSize * 2, which mine is set to. But reading the XFS mans in more detail it strikes me as incorrect, unless you have only 2 usable-data drievs (ie 4 drive RAID6 or 3 drive RAID5). I'm not sure who to believe. Regardless, if my reading of the XFS docs is correct, it looks like my sunit was never configured properly, and is set to perfectly match 8 data drives, not my old 6 and certainly not my new 7. I may, in short order, get a 10th drive for my array, and run perf tests before and after to see if sunit should indeed be a multiple of usable-drives or not. swidth is easier. Just make it sunits * usabledrives, and it appears it can be any valid multiple, changed at will. swidth was set correctly for my old setup and is now set correctly for my new setup. Bottom line is, if my new reading of sunit is correct, and it needs to be tailored to usable-disk-count and can only be changed by factors of 2, then md+XFS is de facto limited to only ever doubling your usable-disk count, and NOT allowing of adding 1 disk here, 1 disk there, as I first thought. *IF* you care about performance, that is. Nothing stops you from doing any of this, but if your FS striping does not match your RAID striping you could see insane performance decreases. Neat: Filesystem 1K-blocks Used Available Use% Mounted on before: /dev/md4 11716644864 11499740512 216904352 99% /data after : /dev/md4 13669767872 11499740704 2170027168 85% /data Final note: After hearing ZFS (on Linux at least) cannot grow by 1 disk when using RAID5 or 6, and after nearly 10 years using XFS on md on huge arrays, I say give some hero cookies to md/XFS. It's withstood some pretty strange events on my server, and has never blown up. If I'm wrong about the sunit being a problem, then md/XFS is a great option for those who want to gradually add space as they need it.

2 3

HGST
by Trevor Cordes 10 Sep '14

10 Sep '14

As per an after-meeting conversation: HGST was indeed bought out by WD. I have a vague recollection of that news but I think I repressed the memory for sanity reasons. Wyatt and I were just discussing how it's funny that hard drive prices really haven't changed in 2 years (except for the 3+ TB size) and... boom, "there's your problem". Now there are only 2 companies (of note) competing worldwide in spinning rust. Can you spell duopoly? Now it all becomes clear...

1 0

Re: [RndTbl] hard drive failure curve (John Donovan)
by JD 09 Sep '14

09 Sep '14

I am glad a visual was put, for the meaning of 'Bathtub Curve'. I am no stats guy, but I can dig a Bell Curve, nine times out of ten, and I groove to a good Fletcher-Munson Curve with regularity. The idea of a Bathtub Curve makes me feel .. bubbly. However, when it comes to interpreting Hard Disk Drive Mortality Rates, I think there are some more factors to be considered. I've sold at least a thousand, and worked on almost as many more drives. Mostly from the latter case, I have come to form some impressions, having seen quite a lot of what most storage users would call 'very private' information. So I end up having an opinion or two, for this. Ok. Three. It's come up in meetings before, but bears repeating: Google released findings saying those common-off-the-shelf IDE drives they'd been running (!droves of drives!) did NOT fail (any differently) when their environs let them run rather hot. So, now, if the much dreaded Heat is not our bugbear, what gives with hard drives dying without our permission? We are not children, here, so let's skip past the usual physical interferences, like shock, vibration, infra-Gaussian warbling, ball-pein hammers and gamma particle storms. I believe the real answer is easily explained. Hard disk drives are marvels of precision machining, spartan firmware, inspiring ASIC chippery and wildly fast signal processing when those bits zing by under the 'flying' heads. These drives are conceived in unusually clean rooms. Some say classy. I say immaculate. And their nascence in mangers of such purity does not prepare them for life on the streets. I choose to believe that most hard disk drives feel a bit excited when they are first written with their very own system image: "Yes. I'll remember that for you. Yes. Yes." .. etc. At this stage, all is as intended. Soon enough, our earnest little friends are further coddled into various systems, for shipment worldwide. Away from their siblings, they are now out on their own, and are still eager to remember & retrieve lots more for us, in mere milliseconds! Then the box opens and it's up & onto the new owner's desktop! Happyhappy! Joyjoy! But alas: As if it's not disappointing enough for them to finally learn which operating system the majority of these drives have to host, add to that, that a certain number of them will soon be asked to store untold horrors; filth and crud which their owners call 'data'. Bad recipes. Panda porn. Political opinions. Twittertwattwaddle. Fwd'd Fwds .. yucky stuff. Blech. Think about it. (or maybe .. don't). Marvel as we may, it is _us_ who kill them: Hard drives die of abject depression, chronic bewilderment, and disgust.

6 8

cloud drives
by Dan Martin 06 Sep '14

06 Sep '14

I first looked at cloud drives as a solution to lack of space on my laptop hard drive. It turns out, however, that most of the major services keep a copy on the local drive. In fact, photos from my phone, downloaded by DropBox, were taking up more room. I thought Google Drive might be a good bet, with lots of free space and the option to block syncing of some data to certain devices. However, it doesn't seem to sync properly. Sometimes it syncs forever, but cannot seem to get the job done. Last time, it said it was completed but when I checked the data (GnuCash accounting files) it was inconsistent. Now, it has been syncing a single item for over an hour. DropBox seems to be working, so I will use it as extra backup and to keep some data synced (Unison has served me well for the sync part). Has anyone else had trouble with Google Drive? I have like everything Google up until now. -- Dan Martin, MD GP Hospital Practitioner Computer Scientist ummar143(a)shaw.ca (204) 831-1746 answering machine always on

2 1

Re: [RndTbl] Roundtable Digest, Vol 117, Issue 2
by Jean Jubinville 04 Sep '14

04 Sep '14

I agree with the cars analogy, in a more specific way... Ok showing my true colours here but really, we're all nerds of one stripe or another aren't we? Disclaimer: I'm not the guy in the video, but am putting together a very similar type of setup on my farm. check out: KenSF3wOtIc If these engines get a good start in life, they can run continuously for 30+years; sometimes oil changes are done by refilling at the same time as draining out the bottom /while the engine is running/. I have also operated and maintained forklifts built in the '60s, operating in scrapyards, abused and overloaded for their entire working lifetime and still in good enough condition to serve the daily needs of the company-50 years running! The Point: In our flaky bits and byte world, I must admit that I am amazed at the durability of some of our last century's 'moving parts' inventions. The longevity of these machines isn't great for economic stimulation, but the lifespan is amazing if pride doesn't factor in, and you have no need to live with the latest. Nowhere in this does a bathtub factor in, but the old iron: there is longevity I can appreciate, feel, and hear. p.s.: mechanic's insider knowledge!: subaru boxer engines WILL give at least 500,000 km of reliable service, with regular oil change(just do the filter every 2-3 oil changes and add sheet metal to rust holes that may pop up in the doors). J.J. On Thu, Sep 4, 2014 at 12:00 PM, <roundtable-request(a)muug.mb.ca> wrote: > Send Roundtable mailing list submissions to > roundtable(a)muug.mb.ca > > To subscribe or unsubscribe via the World Wide Web, visit > http://www.muug.mb.ca/mailman/listinfo/roundtable > or, via email, send a message with subject or body 'help' to > roundtable-request(a)muug.mb.ca > > You can reach the person managing the list at > roundtable-owner(a)muug.mb.ca > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Roundtable digest..." > > Today's Topics: > > 1. hard drive failure curve (Trevor Cordes) > 2. Re: hard drive failure curve (Adam Thompson) > 3. Re: hard drive failure curve (Trevor Cordes) > 4. Re: hard drive failure curve (Adam Thompson) > > > ---------- Forwarded message ---------- > From: Trevor Cordes <trevor(a)tecnopolis.ca> > To: MUUG RndTbl <roundtable(a)muug.mb.ca> > Cc: > Date: Wed, 3 Sep 2014 14:10:53 -0500 > Subject: [RndTbl] hard drive failure curve > OK, so you know how many (most?) computer components' failure rates > follow the "bathtub curve": > > http://users.ece.cmu.edu/~koopman/des_s99/sw_reliability/Image1.gif > > They either die quickly due to egregious manufacturing faults, or live > a long time (near 0% failure rate), and then all start dying as they get > really old. > > The bathtub curve occurs in tons of places, like with a simple light > bulb. > > As per my experience (probably dealing with over 1000 drives > personally), and google's studies, as well as anecdotal information, > hard drives are different. The failure rate seems to be bathtub-ish. > The initial failure rate is fairly bathtub. But the useful-life period > is at a much higher level and seems to grow slightly, meaning instead > of dropping to near-zero failures during useful life, they drop to > around 15% failure rate each year, mostly independent of age. Then, > in the old-age phase they seem to ramp up much slower towards death. > It's not a hockey-stick right edge, it's more a constant gentle hill > slope. We all have seen many drives that are still running after 8, > 10, 20 years. In fact, it seems more of a survival of the fittest > system (maybe like turtles?), where once they live to a certain age, > many go on to live far beyond the average. > > Anyhow, my question is, what else in the non-computer world has a > failure graph like hard drives? Something easy for a neophyte to grasp. > Normal bathtubs have lightbulbs. What analogy can be used for hard > drives? I haven't been able to think of any! > > Surely there must be one?? > > > > ---------- Forwarded message ---------- > From: Adam Thompson <athompso(a)athompso.net> > To: Continuation of Round Table discussion <roundtable(a)muug.mb.ca>, Trevor Cordes <trevor(a)tecnopolis.ca> > Cc: > Date: Wed, 03 Sep 2014 14:53:21 -0500 > Subject: Re: [RndTbl] hard drive failure curve > The obvious analogy that for obvious reasons* won't have occurred to you: cars. If you don't get a lemon, it's probably good for a while, but eventually starts to break down. > Also, historically, humans - at least in ages and places where infant mortality is/was a significant drag on population growth. > > -Adam > > *for those of you who don't know Trevor well, his 25(?)-year-old car is lovingly & painstakingly maintained in near-new condition. > > On September 3, 2014 2:10:53 PM CDT, Trevor Cordes <trevor(a)tecnopolis.ca> wrote: >> >> OK, so you know how many (most?) computer components' failure rates >> follow the "bathtub curve": >> >> http://users.ece.cmu.edu/~koopman/des_s99/sw_reliability/Image1.gif >> >> They either die quickly due to egregious manufacturing faults, or live >> a long time (near 0% failure rate), and then all start dying as they get >> really old. >> >> The bathtub curve occurs in tons of places, like with a simple light >> bulb. >> >> As per my experience (probably dealing with over 1000 drives >> personally), and google's studies, as well as anecdotal information, >> hard drives are different. The failure rate seems to be bathtub-ish. >> The initial failure rate is fairly bathtub. But the useful-life period >> is at a much higher level and seems to grow slightly, meaning instead >> of dropping to near-zero failures during useful life, they dr! >> op to >> around 15% failure rate each year, mostly independent of age. Then, >> in the old-age phase they seem to ramp up much slower towards death. >> It's not a hockey-stick right edge, it's more a constant gentle hill >> slope. We all have seen many drives that are still running after 8, >> 10, 20 years. In fact, it seems more of a survival of the fittest >> system (maybe like turtles?), where once they live to a certain age, >> many go on to live far beyond the average. >> >> Anyhow, my question is, what else in the non-computer world has a >> failure graph like hard drives? Something easy for a neophyte to grasp. >> Normal bathtubs have lightbulbs. What analogy can be used for hard >> drives? I haven't been able to think of any! >> >> Surely there must be one?? >> ________________________________ >> >> Roundtable mailing list >> Roundtable(a)muug.mb.ca >> http://www.muug.mb.ca/mailman/listinfo/roundtable > > > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. > > > ---------- Forwarded message ---------- > From: Trevor Cordes <trevor(a)tecnopolis.ca> > To: MUUG RndTbl <roundtable(a)muug.mb.ca> > Cc: > Date: Thu, 4 Sep 2014 02:23:21 -0500 > Subject: Re: [RndTbl] hard drive failure curve > On 2014-09-03 Adam Thompson wrote: >> The obvious analogy that for obvious reasons* won't have occurred to >> you: cars. If you don't get a lemon, it's probably good for a while, >> but eventually starts to break down. Also, historically, humans - at >> least in ages and places where infant mortality is/was a significant >> drag on population growth. > > The Cars example is imperfect, because, as you said "it's good for a > while", but with hard drives I find that the near-zero-failure part of > the bathtub, if it even exists for HDDs, is abominably short, like 6 > months. Most quality cars these days (ones ranked high in Consumer > Reports reliability) should be nearly trouble-free for 5 years. > > Here's my interpretation of the HD curve (monospace required): > > | > | ____________________________ > \ __________/ > \ _________/ > \ / > \__/ > > The main point being that after (maybe) a short reprieve, the failure > probability immediately ramps up to and sits at an unacceptable number > (like 15%) nearly forever, with but a tiny increment after each > additional year. Whether there is a bathtub end after 10-20 years, is > up for debate. (Brad and I have Atari ST drives from 30 years ago that > still operate, for example.) > > Surely if you ASCII'd a modern car graph it wouldn't quite fit? > > Your human being analogy is probably much closer to what I'm looking > for, but that one definitely has an abrupt bathtub hockey-stick at the > right hand side :-) With humans, make it past the first tricky year > and you probably have a small chance that stays relatively static for > dozens of years of contracting a terminal illness or getting hit by a > bus. Not as straight-forward as a "light bulb" example as I was > looking for (as it requires some thought and reflection) but pretty > good. > > Surely, though, in the world of consumer items something else must be > just like hard drives? > > > > ---------- Forwarded message ---------- > From: Adam Thompson <athompso(a)athompso.net> > To: roundtable(a)muug.mb.ca > Cc: > Date: Thu, 04 Sep 2014 02:41:02 -0500 > Subject: Re: [RndTbl] hard drive failure curve > On 14-09-04 02:23 AM, Trevor Cordes wrote: >> >> The Cars example is imperfect, because, as you said "it's good for a >> while", > > > Yup. Closest mass good I could think of offhand. > >> Surely if you ASCII'd a modern car graph it wouldn't quite fit? > > > The key difference is that you can repair and maintain a car, whereas a HDD (or SSD, for that matter) is either alive and well, alive and dying, or dead - and there's nothing you can do about it. > >> Your human being analogy is probably much closer to what I'm looking >> for, but that one definitely has an abrupt bathtub hockey-stick at the >> right hand side :-) > > > Yessss... although not so abrupt, at various points in history. > >> Surely, though, in the world of consumer items something else must be >> just like hard drives? > > > Not that I can think of. You have to combine a) non-negligible failure rate, with b) extremely tight tolerances, with c) variable quality control on (b), to get a similar result. Outside the computing field, I can't think of anything [other than cars] that has as much complexity, as "finicky" as 10,000rpm spinning platters - AND is common enough that everyone will understand it. > > > -- > -Adam Thompson > athompso(a)athompso.net > > > > _______________________________________________ > Roundtable mailing list > Roundtable(a)muug.mb.ca > http://www.muug.mb.ca/mailman/listinfo/roundtable >

1 0

hard drive failure curve
by Trevor Cordes 04 Sep '14

04 Sep '14

OK, so you know how many (most?) computer components' failure rates follow the "bathtub curve": http://users.ece.cmu.edu/~koopman/des_s99/sw_reliability/Image1.gif They either die quickly due to egregious manufacturing faults, or live a long time (near 0% failure rate), and then all start dying as they get really old. The bathtub curve occurs in tons of places, like with a simple light bulb. As per my experience (probably dealing with over 1000 drives personally), and google's studies, as well as anecdotal information, hard drives are different. The failure rate seems to be bathtub-ish. The initial failure rate is fairly bathtub. But the useful-life period is at a much higher level and seems to grow slightly, meaning instead of dropping to near-zero failures during useful life, they drop to around 15% failure rate each year, mostly independent of age. Then, in the old-age phase they seem to ramp up much slower towards death. It's not a hockey-stick right edge, it's more a constant gentle hill slope. We all have seen many drives that are still running after 8, 10, 20 years. In fact, it seems more of a survival of the fittest system (maybe like turtles?), where once they live to a certain age, many go on to live far beyond the average. Anyhow, my question is, what else in the non-computer world has a failure graph like hard drives? Something easy for a neophyte to grasp. Normal bathtubs have lightbulbs. What analogy can be used for hard drives? I haven't been able to think of any! Surely there must be one??

4 5

[Muug] MUUG Meeting, Sept 9, 7:30pm -- Sage
by Gilbert E. Detillieux 02 Sep '14

02 Sep '14

The Manitoba UNIX User Group (MUUG) will be holding its next monthly meeting on Tuesday, September 9. The meeting topic for this month is as follows: Sage Sage is a free open-source program that allows (in the style of Mathematica or Maple) various types of symbolic manipulation. As described on the web page, Sage is built out of nearly 100 open-source packages and features a unified interface. It can be used to study elementary and advanced, pure and applied mathematics. This includes a huge range of mathematics, including basic algebra, calculus, elementary to very advanced number theory, cryptography, numerical computation, commutative algebra, group theory, combinatorics, graph theory, exact linear algebra and much more. It combines various software packages and seamlessly integrates their functionality into a common experience. It is well-suited for education and research. The user interface is a notebook in a web browser or the command line. Using the notebook, Sage connects either locally to your own Sage installation or to a Sage server on the network. Inside the Sage notebook you can create embedded graphics, beautifully typeset mathematical expressions, add and delete input, and share your work across the network. Michael Doob will be our presenter. Before the break, as this month's RTFM topic, Rob Keizer will talk about the tmux(1) command. The group holds its general meetings at 7:30pm on the second Tuesday of every month from September to June. (There are no meetings in July and August.) Meetings are open to the general public; you don't have to be a MUUG member to attend. ***************************************************************** Please note our *NEW* meeting location for this month: Room 1L08, Lockhart Hall, University of Winnipeg, entrance on Ellice Ave. between Spence St. and Balmoral St. Parking is available on the surrounding streets and in the lots on nearby streets. Look for signage once you're at the building, or ask a security guard. ***************************************************************** For more information about MUUG, and its monthly meetings, check out their Web server: http://www.muug.mb.ca/ Help us promote this month's meeting, by putting this poster up on your workplace bulletin board or other suitable public message board: http://www.muug.mb.ca/meetings/MUUGmeeting.pdf -- Gilbert E. Detillieux E-mail: <gedetil(a)muug.mb.ca> Manitoba UNIX User Group Web: http://www.muug.mb.ca/ PO Box 130 St-Boniface Phone: (204)474-8161 Winnipeg MB CANADA R2H 3B4 Fax: (204)474-7609

1 0