Can someone here recommend some specific USB3 to Ethernet hardware adapters, to make a reliable NAS drive. I want this architecture because the external USB3 drive will frequently need to be disconnected from NAS usage and connected locally to another computer via USB (for certain data updates).
Hartmut - +1-204-339-8331
I don't understand - the drive and the NIC are orthogonal to each other. Based on your email, an eSATA port might make sense, but I'm not seeing how an USB NIC helps you at all. Please clarify? -Adam
Get Outlook for Androidhttps://aka.ms/AAb9ysg ________________________________ From: Roundtable roundtable-bounces@muug.ca on behalf of Hartmut W Sager hwsager@marityme.net Sent: Monday, January 23, 2023 2:14:57 AM To: MUUG - Round Table roundtable@muug.ca Subject: [RndTbl] USB3 to Ethernet, to make NAS
Can someone here recommend some specific USB3 to Ethernet hardware adapters, to make a reliable NAS drive. I want this architecture because the external USB3 drive will frequently need to be disconnected from NAS usage and connected locally to another computer via USB (for certain data updates).
Hartmut - +1-204-339-8331
Also, I'm afraid I have to point out that "USB to Ethernet adapter" and "reliable" don't belong in the same sentence to begin with.
As to specific recommendations, no. Virtually all USB Ethernet adapters for sale today use the Class driver, and are implemented using one of about 3 or 4 chipsets - and made using one of about 4 or 5 reference designs. The only variability is (a) whether the adapter has other co-existing functions [e.g. a hub], which does not IMHO have any specific correlation with quality, and/or (b) the warranty and ease of contacting tech support.
Mind you, all the same goes for USB-SATA adapters.
There's still some room to differentiate on USB-C to NVMe adapters, but even that market is rapidly coalescing and consolidating.
-Adam
Get Outlook for Androidhttps://aka.ms/AAb9ysg ________________________________ From: Roundtable roundtable-bounces@muug.ca on behalf of Adam Thompson athompso@athompso.net Sent: Monday, January 23, 2023 7:42:43 AM To: Continuation of Round Table discussion roundtable@muug.ca Subject: Re: [RndTbl] USB3 to Ethernet, to make NAS
I don't understand - the drive and the NIC are orthogonal to each other. Based on your email, an eSATA port might make sense, but I'm not seeing how an USB NIC helps you at all. Please clarify? -Adam
Get Outlook for Androidhttps://aka.ms/AAb9ysg ________________________________ From: Roundtable roundtable-bounces@muug.ca on behalf of Hartmut W Sager hwsager@marityme.net Sent: Monday, January 23, 2023 2:14:57 AM To: MUUG - Round Table roundtable@muug.ca Subject: [RndTbl] USB3 to Ethernet, to make NAS
Can someone here recommend some specific USB3 to Ethernet hardware adapters, to make a reliable NAS drive. I want this architecture because the external USB3 drive will frequently need to be disconnected from NAS usage and connected locally to another computer via USB (for certain data updates).
Hartmut - +1-204-339-8331
+1 to what Adam is saying, sounds like we might be solving the wrong problem.
I'm going to make the following assumptions:
- You're trying to do this using a USB portable hard drive like these https://www.memoryexpress.com/Category/HardDrives?FilterID=5dc914c6-0785-94bb-49a7-804c5c40a606 - You want to use the USB hard drive as a NAS most of the time - You want the flexibility to connect the hard drive directly to a computer sometimes - Your USB portable drive isn't using USB-C
Some recommended solutions:
- Many home wifi routers include a USB port which can be used to make USB storage available on the network. For example, Mikrotik hEX https://mikrotik.com/product/RB960PGS. If your router has this feature, this would be a quick solution - If you have an existing home server or PC that's always powered on, plugging in your USB drive and configuring the drive to be shared over the network might be OK as well - If you don't have existing hardware that can be used, then you could consider buying a single board computer https://www.memoryexpress.com/Category/SingleBoardComputers or refurb PC https://www.memoryexpress.com/Category/DesktopComputers?FilterID=0e757706-0903-d8ff-8186-d297993b5869 to convert it into a NAS or use an off the shelf NAS unit https://www.memoryexpress.com/Category/NetworkNAS
Cursed solution:
- This product category seems to have died out, but you used to be able to buy converters that could turn USB hard drives into standalone NAS units - https://www.aliexpress.com/item/1762899257.html - Just search for "Mini WAN NET Giga NAS FTP Dongle GIGA NAS FAT 32 / NTFS USB 2.0 Network Storage Adapter Dongle for FTP Server" - The performance and reliability was probably terrible, but it's still neat
On Mon, Jan 23, 2023 at 7:50 AM Adam Thompson athompso@athompso.net wrote:
Also, I'm afraid I have to point out that "USB to Ethernet adapter" and "reliable" don't belong in the same sentence to begin with.
As to specific recommendations, no. Virtually all USB Ethernet adapters for sale today use the Class driver, and are implemented using one of about 3 or 4 chipsets - and made using one of about 4 or 5 reference designs. The only variability is (a) whether the adapter has other co-existing functions [e.g. a hub], which does not IMHO have any specific correlation with quality, and/or (b) the warranty and ease of contacting tech support.
Mind you, all the same goes for USB-SATA adapters.
There's still some room to differentiate on USB-C to NVMe adapters, but even that market is rapidly coalescing and consolidating.
-Adam
Get Outlook for Android https://aka.ms/AAb9ysg
*From:* Roundtable roundtable-bounces@muug.ca on behalf of Adam Thompson athompso@athompso.net *Sent:* Monday, January 23, 2023 7:42:43 AM *To:* Continuation of Round Table discussion roundtable@muug.ca *Subject:* Re: [RndTbl] USB3 to Ethernet, to make NAS
I don't understand - the drive and the NIC are orthogonal to each other. Based on your email, an eSATA port might make sense, but I'm not seeing how an USB NIC helps you at all. Please clarify? -Adam
Get Outlook for Android https://aka.ms/AAb9ysg
*From:* Roundtable roundtable-bounces@muug.ca on behalf of Hartmut W Sager hwsager@marityme.net *Sent:* Monday, January 23, 2023 2:14:57 AM *To:* MUUG - Round Table roundtable@muug.ca *Subject:* [RndTbl] USB3 to Ethernet, to make NAS
Can someone here recommend some specific USB3 to Ethernet hardware adapters, to make a reliable NAS drive. I want this architecture because the external USB3 drive will frequently need to be disconnected from NAS usage and connected locally to another computer via USB (for certain data updates).
Hartmut - +1-204-339-8331 _______________________________________________ Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable
On 2023-01-23 Hartmut W Sager wrote:
Can someone here recommend some specific USB3 to Ethernet hardware adapters, to make a reliable NAS drive. I want this architecture because the external USB3 drive will frequently need to be disconnected from NAS usage and connected locally to another computer via USB (for certain data updates).
If your plan is to do:
USB-HDD <-> USB-LAN
It cannot be done. There has to be a "smart" device, i.e. computer of some sort, that speaks both USB-HDD and USB-LAN in between the two. The only way it would work is if you found some magical product that is "USB-LAN-can-speak-usb-hdd-protocols". I'm not sure I've ever seen such a thing?
Ah...
On 2023-01-23 Chris Audet wrote:
- https://www.aliexpress.com/item/1762899257.html
- Just search for "Mini WAN NET Giga NAS FTP Dongle GIGA NAS FAT
32 / NTFS USB 2.0 Network Storage Adapter Dongle for FTP Server"
What Chris found is basically the magical product what you'd be looking at (nice find!). But USB2 only will be dog slow. Anyhow, it's just a little SBC computer doing the work for you as described above. Probably with Linux. You could do the same thing with a Pi or other mini solution. I think these types of devices died out many years ago, and not sure if they ever made a USB3 one.
I will warn that external USB HDDs are (usually) the worst quality garbage drives the companies can produce. Especially Seagate. They often explicitly state FOR BRIEF USE ONLY, i.e. not always-on in a NAS. They aren't lying. The motors/bearings are designed to fail after a tiny number of hours. WD might be slightly better.
If you do want to do such a thing, you might want to make sure you RAID-1 the system. Not sure how you do that with your plug-in-via-usb-for-updates requirement... then again, why not plug it into your "update computer" via Gb Ethernet? Either Gb Ethernet or USB3 will probably max out the rust speed, so nothing is gained via USB.
Whenever I need to do a "moveable" HD solution, I use the "5.25 bay to 3.5 sata rust drive external bay" things where there's a bay(s) you put in computer(s) and drive caddies that slot into them that you can put your drive(s) in. They are cheap ($40ish?) and work well and protect the drive better than external USB drive cases do. You'd then need to build a "real" NAS box that has a 5.25" bay for it. (Still have the problem of re-syncing any RAID though.) I'd just build something with a small size but-with-5.25"-bays case and some old mobo. Pis won't help though, as they don't have SATA. You need an Atom or ITX or NuC type thing. Then run FreeNAS or just roll your own with your fave distro.
Depending on the NAS software, the need for RAID might be subsumed by the use of snapshots or daily/hourly backups, in either case stored on a second non-mobile drive. I hadn't thought of the drive quality and fragility issue, but I can confirm Trevor's right. And even if you buy the best HDD money can buy and are using (e.g.) a USB-SATA adapter with it, all that moving around WILL reduce the drive's lifespan.
Hartmut, can you elaborate on why you think you need to connect this drive to a PC directly, in the first place? There may be a better solution for your problem.
-Adam
-----Original Message----- From: Roundtable roundtable-bounces@muug.ca On Behalf Of Trevor Cordes Sent: Monday, January 23, 2023 9:57 PM To: Hartmut W Sager hwsager@marityme.net Cc: Continuation of Round Table discussion roundtable@muug.ca Subject: Re: [RndTbl] USB3 to Ethernet, to make NAS
On 2023-01-23 Hartmut W Sager wrote:
Can someone here recommend some specific USB3 to Ethernet hardware adapters, to make a reliable NAS drive. I want this architecture because the external USB3 drive will frequently need to be disconnected from NAS usage and connected locally to another computer via USB (for certain data updates).
If your plan is to do:
USB-HDD <-> USB-LAN
It cannot be done. There has to be a "smart" device, i.e. computer of some sort, that speaks both USB-HDD and USB-LAN in between the two. The only way it would work is if you found some magical product that is "USB-LAN-can-speak-usb-hdd-protocols". I'm not sure I've ever seen such a thing?
Ah...
On 2023-01-23 Chris Audet wrote:
- https://www.aliexpress.com/item/1762899257.html
- Just search for "Mini WAN NET Giga NAS FTP Dongle GIGA NAS FAT
32 / NTFS USB 2.0 Network Storage Adapter Dongle for FTP Server"
What Chris found is basically the magical product what you'd be looking at (nice find!). But USB2 only will be dog slow. Anyhow, it's just a little SBC computer doing the work for you as described above. Probably with Linux. You could do the same thing with a Pi or other mini solution. I think these types of devices died out many years ago, and not sure if they ever made a USB3 one.
I will warn that external USB HDDs are (usually) the worst quality garbage drives the companies can produce. Especially Seagate. They often explicitly state FOR BRIEF USE ONLY, i.e. not always-on in a NAS. They aren't lying. The motors/bearings are designed to fail after a tiny number of hours. WD might be slightly better.
If you do want to do such a thing, you might want to make sure you RAID-1 the system. Not sure how you do that with your plug-in-via-usb-for-updates requirement... then again, why not plug it into your "update computer" via Gb Ethernet? Either Gb Ethernet or USB3 will probably max out the rust speed, so nothing is gained via USB.
Whenever I need to do a "moveable" HD solution, I use the "5.25 bay to 3.5 sata rust drive external bay" things where there's a bay(s) you put in computer(s) and drive caddies that slot into them that you can put your drive(s) in. They are cheap ($40ish?) and work well and protect the drive better than external USB drive cases do. You'd then need to build a "real" NAS box that has a 5.25" bay for it. (Still have the problem of re-syncing any RAID though.) I'd just build something with a small size but-with-5.25"-bays case and some old mobo. Pis won't help though, as they don't have SATA. You need an Atom or ITX or NuC type thing. Then run FreeNAS or just roll your own with your fave distro. _______________________________________________ Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable
Hartmut, can you elaborate on why you think you need to connect this drive to a PC directly, in the first place? There may be a better solution for your problem.
Thank you to all the folks here who responded. I hope to produce a detailed reply during the coming night, so please be patient, and "hold your further fire". :) As some of you know, my neurology limits how quickly and how thoroughly I can respond.
Hartmut
Now my detailed response. I initially asked:
Can someone here recommend some specific USB3 to Ethernet hardware adapters, to make a reliable NAS drive. I want this architecture because the external USB3 drive will frequently need to be disconnected from NAS usage and connected locally to another computer via USB (for certain data updates).
When I initially received several "interesting" responses, I already realized I had better consult with Bill Reid before I stick my foot any further into my mouth. :) Thank you, *Bill*, for the detailed phone discussion we had on Monday evening.
When I referred to "USB3 to Ethernet hardware adapters", I completely forgot that this could so easily refer to those USB adapters that simply provide an Ethernet port (or an additional Ethernet port) where needed on a computer/laptop/whatever. *Adam*, I'm sure you at least, and maybe others, got confused by this. (I love your "orthogonal" comment, though I never did mention a NIC in my query.)
*No, I meant* an intelligent device/box/adapter that would have some NAS server logic in it. *Chris Audet*, you got me 100% right.
*Applicable tagline:* I know what I meant to say; I said this instead. :)
Adam> Also, I'm afraid I have to point out that "USB to Ethernet adapter" and "reliable" don't belong in the same sentence to begin with.
LOL. Good one! And I agree.
Vijay> I have some USB-2 Gigabit Ethernet adapters you can try if you like. I found it to be quite reliable for the following use cases — I used two of them on a ThinkPad as a firewall (Shaw gigabit and MTS). It worked quite well to create an HA VPN. I also have used it to test virtual Ethernet performance (vether on OpenBSD).
Thank you, *Vijay*. I'll keep these in mind, though they are USB2 (or did you mean USB3 *to* Gigabit Ethernet?).
Chris Audet> I'm going to make the following assumptions:
- You're trying to do this using a USB portable hard drive like these
- You want to use the USB hard drive as a NAS most of the time
- You want the flexibility to connect the hard drive directly to a computer sometimes
- Your USB portable drive isn't using USB-C
*Chris*, you nailed it 100% on all 4 assumptions!
Chris Audet> Some recommended solutions:
- Many home wifi routers include a USB port which can be used to make USB storage available on the network. For example, Mikrotik hEX. If your router has this feature, this would be a quick solution
Both I and the person (Lynn) that I'm doing this for have the *Shaw Hitron CGNM-2250* cable modem/router combo, and, upon me looking closely several months ago, it does have two such USB ports (blue inserts, so maybe even USB3). However, even a *cusadmin login* into the modem has nothing in the menus for these ports. Web research did reveal that those ports can accept NAS, but configuring requires a secret URL into that part of the Hitron, and there are a few digits in the URL that vary across these modems, depending on carrier (Shaw), firmware version, etc, and the phase of the moon.
I'd rather not pursue this possibility (even if it's USB3), in case the modem ever fails and Shaw can't give the same model anymore. They already don't like issuing this model, preferring their new consumer "Mars lander" contraption.
- If you have an existing home server or PC that's always powered on, plugging in your USB drive and configuring the drive to be shared over the network might be OK as well
I had considered, and still am considering, that solution. My computers are always on, but Lynn's computer is not (so far). This does remain an attractive solution though.
- If you don't have existing hardware that can be used, then you could consider buying a single board computer or refurb PC to convert it into a NAS or use an off the shelf NAS unit
An SBC is attractive, except that the learning curve doesn't work for me at this time. A refurb PC is too bulky for me and Lynn (we both have very small residential premises). Off-the-shelf NAS doesn't (?) give USB access, and we already put good money into this high-end external USB3 unit. It's a Seagate high-MTBF 3.5-inch 8TB drive in a 5.25-inch case (with AC power adapter of course).
Cursed solution:
LOL!
Trevor> It cannot be done. There has to be a "smart" device, ..... Ah, Chris found .....
Thanks *Trevor*, for your very detailed reply. You too got fooled by my poor terminology in my very first sentence. And in any case, even after "Chris found", USB2 is a deal-breaker.
Trevor> I will warn that external USB HDDs are (usually) the worst quality garbage drives the companies can produce. Especially Seagate. They often explicitly state FOR BRIEF USE ONLY, i.e. not always-on in a NAS. They aren't lying. The motors/bearings are designed to fail after a tiny number of hours. WD might be slightly better.
Oh I know, and I totally agree. I often give this "junk HDD" lecture to others. And WD is no better - those two companies must have used RAID to synchronize their junk in real time.
Adam> ..... the use of snapshots or daily/hourly backups, in either case stored on a second non-mobile drive.
It's a thought, but nah, see further down. This isn't really a snapshots/backups scenario.
Adam> I hadn't thought of the drive quality and fragility issue, but I can confirm Trevor's right. And even if you buy the best HDD money can buy and are using (e.g.) a USB-SATA adapter with it, all that moving around WILL reduce the drive's lifespan.
*Adam*, agreed - see my comment to Trevor above. But, we do have a high-end drive, and the movement won't be all too frequent, and will be done with care in small premises (i.e., not big movements). And depending on exact location details, we might even be able to just plug/unplug and move the USB3-type-A connect at the other end.
(Ironic aside: No 5.25-inch or 3.5-inch HDD (of the high-MTBF kind) in daily use has ever failed me over the decades, while *every* 2.5-inch HDD I've ever had in daily use has failed me.)
Adam> Hartmut, can you elaborate on why you think you need to connect this drive to a PC directly, in the first place? There may be a better solution for your problem.
*Adam*, I'll give an initial answer here now, but this might be fruitful to expand on in further forum posts in this thread.
This 8TB drive shall contain Lynn's life collection of photos, music, and a few other things, adding up to about 5TB at this time, which seems to rule out reasonably-priced cloud storage and access (like the Dropbox I subscribe to). The music and photos need to be accessible to her wireless Android devices (plural) and her computer, at least while she is at home, and the access from her wireless Android devices would also be desirable while out-and-about.
The whole collection is still in need of much re-organizing, which needs to be done on an ongoing basis, as time permits, from her desktop computer, and once in a while even at my place from my computers (we currently live in the same building). The re-organizing consists of identifying and deleting duplicates across the whole directory tree, and also lots of grafting, pruning, merging, splitting, and renaming of folders.
Whew! I hope that sums it up.
Again, in true MUUG tradition, I appreciate all the thoughtful input I've received.
Hartmut
On Mon 23 Jan 2023 at 23:14:19 -06:00, Hartmut W Sager hwsager@marityme.net wrote:
Hartmut, can you elaborate on why you think you need to connect this drive to a PC directly, in the first place? There may be a better solution for your problem.
Thank you to all the folks here who responded. I hope to produce a detailed reply during the coming night, so please be patient, and "hold your further fire". :) As some of you know, my neurology limits how quickly and how thoroughly I can respond.
Hartmut
For several years I ran a Pi3B+ with a USB drive and stock Linux + SAMBA for sharing as a NAS device that saved video from a security camera and it worked great. If I was setting this up today I'd go with a Raspberry Pi4 for better USB & network speed and open source NAS software for easier management.
The only negative is the Pi (all versions) don't have steller network performance compared to a commercial NAS, but it should still be more than adequate for home use.
There are tons of "how-to" and documentation on how to do this, but here is a youtube video:
https://www.youtube.com/watch?v=gyMpI8csWis
My final comment is, no matter what else, just make sure that everything isn't stored on a single drive. That is a guarantee that you will lose everything eventually.
Here is a mistake I made: I personally was backing up everything from a home PC to an external USB drive including large tar.gz archives of some old systems I had. Then one day the PC hard drive failed and I replaced it. I was too lazy to copy everything back over from the USB drive to the PC and sure enough a year or so later the USB drive also failed... Fortunately it didn't die completely, just errored on certain files which included one of the big tar.gz files which wasn't anything I really cared about but lesson learned.
For pictures I backup the originals onto another drive, but I also replicate a lower quality version to the cloud which is free and unlimited so worse case I'll always have a copy.
John
On Tue, Jan 24, 2023 at 5:25 PM Vijay Sankar vsankar@foretell.ca wrote:
The usb gigabit adapters are just USB-2. Not USB-3. They are old, and probably not of any relevance to your situation as Adam and Chris clearly articulated. But you’re welcome to have them if it is of any use.
Sent from my iPhone
On Jan 24, 2023, at 07:13, Hartmut W Sager hwsager@marityme.net wrote:
Now my detailed response. I initially asked:
Can someone here recommend some specific USB3 to Ethernet hardware adapters, to make a reliable NAS drive. I want this architecture because the external USB3 drive will frequently need to be disconnected from NAS usage and connected locally to another computer via USB (for certain data updates).
When I initially received several "interesting" responses, I already realized I had better consult with Bill Reid before I stick my foot any further into my mouth. :) Thank you, *Bill*, for the detailed phone discussion we had on Monday evening.
When I referred to "USB3 to Ethernet hardware adapters", I completely forgot that this could so easily refer to those USB adapters that simply provide an Ethernet port (or an additional Ethernet port) where needed on a computer/laptop/whatever. *Adam*, I'm sure you at least, and maybe others, got confused by this. (I love your "orthogonal" comment, though I never did mention a NIC in my query.)
*No, I meant* an intelligent device/box/adapter that would have some NAS server logic in it. *Chris Audet*, you got me 100% right.
*Applicable tagline:* I know what I meant to say; I said this instead. :)
Adam> Also, I'm afraid I have to point out that "USB to Ethernet adapter" and "reliable" don't belong in the same sentence to begin with.
LOL. Good one! And I agree.
Vijay> I have some USB-2 Gigabit Ethernet adapters you can try if you like. I found it to be quite reliable for the following use cases — I used two of them on a ThinkPad as a firewall (Shaw gigabit and MTS). It worked quite well to create an HA VPN. I also have used it to test virtual Ethernet performance (vether on OpenBSD).
Thank you, *Vijay*. I'll keep these in mind, though they are USB2 (or did you mean USB3 *to* Gigabit Ethernet?).
Chris Audet> I'm going to make the following assumptions:
- You're trying to do this using a USB portable hard drive like these
- You want to use the USB hard drive as a NAS most of the time
- You want the flexibility to connect the hard drive directly to a
computer sometimes
- Your USB portable drive isn't using USB-C
*Chris*, you nailed it 100% on all 4 assumptions!
Chris Audet> Some recommended solutions:
- Many home wifi routers include a USB port which can be used to make USB
storage available on the network. For example, Mikrotik hEX. If your router has this feature, this would be a quick solution
Both I and the person (Lynn) that I'm doing this for have the *Shaw Hitron CGNM-2250* cable modem/router combo, and, upon me looking closely several months ago, it does have two such USB ports (blue inserts, so maybe even USB3). However, even a *cusadmin login* into the modem has nothing in the menus for these ports. Web research did reveal that those ports can accept NAS, but configuring requires a secret URL into that part of the Hitron, and there are a few digits in the URL that vary across these modems, depending on carrier (Shaw), firmware version, etc, and the phase of the moon.
I'd rather not pursue this possibility (even if it's USB3), in case the modem ever fails and Shaw can't give the same model anymore. They already don't like issuing this model, preferring their new consumer "Mars lander" contraption.
- If you have an existing home server or PC that's always powered on,
plugging in your USB drive and configuring the drive to be shared over the network might be OK as well
I had considered, and still am considering, that solution. My computers are always on, but Lynn's computer is not (so far). This does remain an attractive solution though.
- If you don't have existing hardware that can be used, then you could
consider buying a single board computer or refurb PC to convert it into a NAS or use an off the shelf NAS unit
An SBC is attractive, except that the learning curve doesn't work for me at this time. A refurb PC is too bulky for me and Lynn (we both have very small residential premises). Off-the-shelf NAS doesn't (?) give USB access, and we already put good money into this high-end external USB3 unit. It's a Seagate high-MTBF 3.5-inch 8TB drive in a 5.25-inch case (with AC power adapter of course).
Cursed solution:
LOL!
Trevor> It cannot be done. There has to be a "smart" device, ..... Ah, Chris found .....
Thanks *Trevor*, for your very detailed reply. You too got fooled by my poor terminology in my very first sentence. And in any case, even after "Chris found", USB2 is a deal-breaker.
Trevor> I will warn that external USB HDDs are (usually) the worst quality garbage drives the companies can produce. Especially Seagate. They often explicitly state FOR BRIEF USE ONLY, i.e. not always-on in a NAS. They aren't lying. The motors/bearings are designed to fail after a tiny number of hours. WD might be slightly better.
Oh I know, and I totally agree. I often give this "junk HDD" lecture to others. And WD is no better - those two companies must have used RAID to synchronize their junk in real time.
Adam> ..... the use of snapshots or daily/hourly backups, in either case stored on a second non-mobile drive.
It's a thought, but nah, see further down. This isn't really a snapshots/backups scenario.
Adam> I hadn't thought of the drive quality and fragility issue, but I can confirm Trevor's right. And even if you buy the best HDD money can buy and are using (e.g.) a USB-SATA adapter with it, all that moving around WILL reduce the drive's lifespan.
*Adam*, agreed - see my comment to Trevor above. But, we do have a high-end drive, and the movement won't be all too frequent, and will be done with care in small premises (i.e., not big movements). And depending on exact location details, we might even be able to just plug/unplug and move the USB3-type-A connect at the other end.
(Ironic aside: No 5.25-inch or 3.5-inch HDD (of the high-MTBF kind) in daily use has ever failed me over the decades, while *every* 2.5-inch HDD I've ever had in daily use has failed me.)
Adam> Hartmut, can you elaborate on why you think you need to connect this drive to a PC directly, in the first place? There may be a better solution for your problem.
*Adam*, I'll give an initial answer here now, but this might be fruitful to expand on in further forum posts in this thread.
This 8TB drive shall contain Lynn's life collection of photos, music, and a few other things, adding up to about 5TB at this time, which seems to rule out reasonably-priced cloud storage and access (like the Dropbox I subscribe to). The music and photos need to be accessible to her wireless Android devices (plural) and her computer, at least while she is at home, and the access from her wireless Android devices would also be desirable while out-and-about.
The whole collection is still in need of much re-organizing, which needs to be done on an ongoing basis, as time permits, from her desktop computer, and once in a while even at my place from my computers (we currently live in the same building). The re-organizing consists of identifying and deleting duplicates across the whole directory tree, and also lots of grafting, pruning, merging, splitting, and renaming of folders.
Whew! I hope that sums it up.
Again, in true MUUG tradition, I appreciate all the thoughtful input I've received.
Hartmut
On Mon 23 Jan 2023 at 23:14:19 -06:00, Hartmut W Sager < hwsager@marityme.net> wrote:
Hartmut, can you elaborate on why you think you need to connect this
drive to a PC directly, in the first place? There may be a better solution for your problem.
Thank you to all the folks here who responded. I hope to produce a detailed reply during the coming night, so please be patient, and "hold your further fire". :) As some of you know, my neurology limits how quickly and how thoroughly I can respond.
Hartmut
Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable
Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable
John Lange> If I was setting this up today I'd go with a Raspberry Pi4
Thanks for your detailed Pi comments. I would sure consider this, if I were in the right headspace.
John Lange> My final comment is, no matter what else, just make sure that everything isn't stored on a single drive. That is a guarantee that you will lose everything eventually.
Agreed totally! I failed to mention, both Lynn and I have the same model of external USB3 8TB HDD, and mine can be part of a backup regimen, at least initially. Not sufficient though, and we're still working on what additional backup/syncing solution we might use (famous clouds are too expensive for, and/or don't offer, 5TB to accommodate her current data). At the moment, her collection of old HDD's where the data all came from are a feeble backup too, but we have no intention of using those HDD's once we're into the whole re-organization.
John Lange> Then one day the PC hard drive failed and .....
Agreed wholeheartedly again.
John Lange> For pictures I backup the originals onto another drive, but I also replicate a lower quality version to the cloud which is free and unlimited so worse case I'll always have a copy.
Good point! A lower quality picture in hand is worth much more than all the pictures in the (lost) bush.
Trevor> Whatever the Seagate MTBF says, take it with a grain of salt. Look carefully at the box/instructions/specs to see if it says "not for always-on use".
Indeed, I did conflate high-MTBF with "good for always-on". Now if we do go for the extra computer or SBC on a LAN, we have noticed that this external USB3 8TB HDD connected to her current desktop computer does spin-down quite soon after last access, and her access pattern would be a few hours of almost continuous access followed by no access for the rest of the 24-hour day.
Trevor> ..... make sure the NAS uses NTFS .....
Yes!
Trevor> John and Adam are right: back the thing up. Buy a 2nd 8TB USB drive, the cheapest one you can find
See somewhere above; for the moment, we have that 2nd HDD (mine), but we may get her something additional. But question: Are the cheap ones "reliable" if we operate such a HDD for very few hours, or are they junky all around, and prone to failure even with low usage? I suspect the latter, and if so, I'll avoid the "cheapest one".
Trevor> Just do a byte-level copy of the whole thing, or a file-level backup, every few months
I *always* do file-level backups/copies. I never want the reading of backups/copies to be dependent on any software. Of course, if I have to image a partition, that's a different matter.
Trevor> Also, John's Pi solution might be ideal if you don't want to wing it, and even better if someone makes some nice NAS OS images that you can just plop onto it?? For your use case, performance limitations won't be a big deal.
If we go Pi, I presume I could engage one of you MUUG folks here for payment?
Hartmut
There are several of us that can help you with setting up a NAS, for varying amounts of pay.
Re. Cloud backup cost, have you priced out BsckBlaze's B2 (sic) and regular services? I think they have a referral program that gets you a discount, too. I back up my 4 TB NAS and it's remarkably cheap.
-Adam
Get Outlook for Androidhttps://aka.ms/AAb9ysg ________________________________ From: Roundtable roundtable-bounces@muug.ca on behalf of Hartmut W Sager hwsager@marityme.net Sent: Thursday, January 26, 2023 4:32:07 AM To: MUUG - Round Table roundtable@muug.ca Subject: Re: [RndTbl] USB3 to Ethernet, to make NAS
John Lange> If I was setting this up today I'd go with a Raspberry Pi4
Thanks for your detailed Pi comments. I would sure consider this, if I were in the right headspace.
John Lange> My final comment is, no matter what else, just make sure that everything isn't stored on a single drive. That is a guarantee that you will lose everything eventually.
Agreed totally! I failed to mention, both Lynn and I have the same model of external USB3 8TB HDD, and mine can be part of a backup regimen, at least initially. Not sufficient though, and we're still working on what additional backup/syncing solution we might use (famous clouds are too expensive for, and/or don't offer, 5TB to accommodate her current data). At the moment, her collection of old HDD's where the data all came from are a feeble backup too, but we have no intention of using those HDD's once we're into the whole re-organization.
John Lange> Then one day the PC hard drive failed and .....
Agreed wholeheartedly again.
John Lange> For pictures I backup the originals onto another drive, but I also replicate a lower quality version to the cloud which is free and unlimited so worse case I'll always have a copy.
Good point! A lower quality picture in hand is worth much more than all the pictures in the (lost) bush.
Trevor> Whatever the Seagate MTBF says, take it with a grain of salt. Look carefully at the box/instructions/specs to see if it says "not for always-on use".
Indeed, I did conflate high-MTBF with "good for always-on". Now if we do go for the extra computer or SBC on a LAN, we have noticed that this external USB3 8TB HDD connected to her current desktop computer does spin-down quite soon after last access, and her access pattern would be a few hours of almost continuous access followed by no access for the rest of the 24-hour day.
Trevor> ..... make sure the NAS uses NTFS .....
Yes!
Trevor> John and Adam are right: back the thing up. Buy a 2nd 8TB USB drive, the cheapest one you can find
See somewhere above; for the moment, we have that 2nd HDD (mine), but we may get her something additional. But question: Are the cheap ones "reliable" if we operate such a HDD for very few hours, or are they junky all around, and prone to failure even with low usage? I suspect the latter, and if so, I'll avoid the "cheapest one".
Trevor> Just do a byte-level copy of the whole thing, or a file-level backup, every few months
I always do file-level backups/copies. I never want the reading of backups/copies to be dependent on any software. Of course, if I have to image a partition, that's a different matter.
Trevor> Also, John's Pi solution might be ideal if you don't want to wing it, and even better if someone makes some nice NAS OS images that you can just plop onto it?? For your use case, performance limitations won't be a big deal.
If we go Pi, I presume I could engage one of you MUUG folks here for payment?
Hartmut
Thank you very much, Adam, for that tip-off. I've never heard of them before, but I'm sure studying their Website now. I'm mainly looking at B2, and it is indeed remarkably cheap, and their ongoing download charge per GB works out to a very small amount for my estimated volume in our use case.
Hartmut
On Thu 26 Jan 2023 at 04:41:03 -06:00, Adam Thompson athompso@athompso.net wrote:
There are several of us that can help you with setting up a NAS, for varying amounts of pay.
Re. Cloud backup cost, have you priced out BsckBlaze's B2 (sic) and regular services? I think they have a referral program that gets you a discount, too. I back up my 4 TB NAS and it's remarkably cheap.
-Adam
Let me know if you want to sign up, I think i can generatea referral code that gives you (and me, full disclosure!) a discount. -Adam
Get Outlook for Androidhttps://aka.ms/AAb9ysg ________________________________ From: Roundtable roundtable-bounces@muug.ca on behalf of Hartmut W Sager hwsager@marityme.net Sent: Friday, January 27, 2023 8:51:17 AM To: MUUG - Round Table roundtable@muug.ca Subject: Re: [RndTbl] USB3 to Ethernet, to make NAS
Thank you very much, Adam, for that tip-off. I've never heard of them before, but I'm sure studying their Website now. I'm mainly looking at B2, and it is indeed remarkably cheap, and their ongoing download charge per GB works out to a very small amount for my estimated volume in our use case.
Hartmut
On Thu 26 Jan 2023 at 04:41:03 -06:00, Adam Thompson <athompso@athompso.netmailto:athompso@athompso.net> wrote: There are several of us that can help you with setting up a NAS, for varying amounts of pay.
Re. Cloud backup cost, have you priced out BsckBlaze's B2 (sic) and regular services? I think they have a referral program that gets you a discount, too. I back up my 4 TB NAS and it's remarkably cheap.
-Adam
I'll definitely contact you and try for the referral thing if/when I sign up. And, since I'm a cloud sync/backup junkie, that might be a lot sooner than the "next few months" my overall decisions on the whole project might take.
Indeed, such a cloud sync/backup junkie am I, that I just re-visited another interesting (cloud-to-cloud) sync product
https://www.multcloud.com/ https://www.multcloud.com/support-clouds/backblaze.html
and was delighted to find that they do list Backblaze in their 30+ clouds that they can work with.
Hartmut
On Fri 27 Jan 2023 at 08:55:23 -06:00, Adam Thompson athompso@athompso.net wrote:
Let me know if you want to sign up, I think i can generatea referral code that gives you (and me, full disclosure!) a discount. -Adam
On 2023-01-27 Hartmut W Sager wrote:
Thank you very much, Adam, for that tip-off. I've never heard of them before, but I'm sure studying their Website now. I'm mainly looking at B2, and it is indeed remarkably cheap
People who know me know I *hate* cloud. If this is the cheapest cloud out there, then it's still too much money *if you are a techie who knows what they are doing and likes futzing with hardware*. (For average Joes, ya ok.)
Your 5TB use case would appear to equal $25/mo at this place. You can often find 5TB USB3 drives on blowout for under $100. So in 4 months you're paying more than the DIY backup solution. The cloud solution will cost you that fee *forever*.
And with the cloud, all your base belong to them (and NSA, and other .gov with or without a warrant). And you must trust they are having the appropriate level of redundancy/protection, and it's not just some dude with their own external USB3 drive on a Pi!! Will you bet your life on the data being there when you need it? What's your recourse? Almost all places will just refund the fee you paid (if that)... can't sue for the value of lost data.
Who would not be better served with more back-ups including offsite redundancies?
For good reason many of us have our own vehicles, housing, computers...?
However, but a casual read of these exchanges on having one's own NAS is clearly not for the faint hearted. i would also imagine there many others who like Trevor are not found of so-called third party cloud services with all the risks he articulated.
Wondering what interest there might be in doing another variant on a Muug type project but specific to creating a Muug type back-up service and mirrored for increased saftey. One system with huge memory would be but a tiny fraction of many of us doing our own. I'm not suggesting our people with know how do this as a labour of love but as a kind of community minded coop.
Looking ahead to security needs, I would however suggest data entry security be as tight as access, cause one would not want a rip-van-Winkle poison pill virus to be added for potential nefarious wake-up if and when ever the back-up was used as a restore and have a trogan nested within.
Eduard On 28/01/2023 3:09 AM, Trevor Cordes wrote:
People who know me know I *hate* cloud. If this is the cheapest cloud out there, then it's still too much money *if you are a techie who knows what they are doing and likes futzing with hardware*. (For average Joes, ya ok.)
Your 5TB use case would appear to equal $25/mo at this place. You can often find 5TB USB3 drives on blowout for under $100. So in 4 months you're paying more than the DIY backup solution. The cloud solution will cost you that fee *forever*.
And with the cloud, all your base belong to them (and NSA, and other .gov with or without a warrant). And you must trust they are having the appropriate level of redundancy/protection, and it's not just some dude with their own external USB3 drive on a Pi!! Will you bet your life on the data being there when you need it? What's your recourse? Almost all places will just refund the fee you paid (if that)... can't sue for the value of lost data. _______________________________________________ Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable
While I'm no longer on the board, this - and similar ideas - have been floated several times in the past.
The capsule summary is, that with MUUG running such a service even on a strict cost-recovery-only, volunteer-only basis, we would not be able to meet, never mind beat, any of the commercial providers on cost or quality. Economies of scale are a massive part of the business model, and we would have all the invariant costs, with none of the incremental revenue to amortize them.
The last time I ran the numbers for my own reasons (last year), what I pay Backblaze ~$10/mo for would cost roughly ~$200/month for a local operation, if operated on a "hobby" scale. That's not a typo: scale allows them to reduce total end-user costs by 20-fold and still (presumably) make a profit. Even more extreme ratios apply to public or private cloud hosting.
If encrypting your data locally and only then uploading it doesn't meet your privacy requirements, you probably shouldn't be connected to the internet at all... Also, see Tarsnap, an online backup service for the EXTREMELY privacy-conscious.
-Adam
Get Outlook for Androidhttps://aka.ms/AAb9ysg ________________________________ From: Roundtable roundtable-bounces@muug.ca on behalf of eh@eduardhiebert.com eh@eduardhiebert.com Sent: Thursday, February 9, 2023 9:32:15 PM To: Continuation of Round Table discussion roundtable@muug.ca Subject: Re: [RndTbl] Is there an Muug like community based appetite for alternative to USB3 to Ethernet, to make NAS
Who would not be better served with more back-ups including offsite redundancies?
For good reason many of us have our own vehicles, housing, computers...?
However, but a casual read of these exchanges on having one's own NAS is clearly not for the faint hearted. i would also imagine there many others who like Trevor are not found of so-called third party cloud services with all the risks he articulated.
Wondering what interest there might be in doing another variant on a Muug type project but specific to creating a Muug type back-up service and mirrored for increased saftey. One system with huge memory would be but a tiny fraction of many of us doing our own. I'm not suggesting our people with know how do this as a labour of love but as a kind of community minded coop.
Looking ahead to security needs, I would however suggest data entry security be as tight as access, cause one would not want a rip-van-Winkle poison pill virus to be added for potential nefarious wake-up if and when ever the back-up was used as a restore and have a trogan nested within.
Eduard On 28/01/2023 3:09 AM, Trevor Cordes wrote:
People who know me know I *hate* cloud. If this is the cheapest cloud out there, then it's still too much money *if you are a techie who knows what they are doing and likes futzing with hardware*. (For average Joes, ya ok.)
Your 5TB use case would appear to equal $25/mo at this place. You can often find 5TB USB3 drives on blowout for under $100. So in 4 months you're paying more than the DIY backup solution. The cloud solution will cost you that fee *forever*.
And with the cloud, all your base belong to them (and NSA, and other .gov with or without a warrant). And you must trust they are having the appropriate level of redundancy/protection, and it's not just some dude with their own external USB3 drive on a Pi!! Will you bet your life on the data being there when you need it? What's your recourse? Almost all places will just refund the fee you paid (if that)... can't sue for the value of lost data. _______________________________________________ Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable
_______________________________________________ Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable
On 2023-02-09 eh@eduardhiebert.com wrote:
Wondering what interest there might be in doing another variant on a Muug type project but specific to creating a Muug type back-up service and mirrored for increased saftey. One system with huge memory would be but a tiny fraction of many of us doing our own. I'm not suggesting our people with know how do this as a labour of love but as a kind of community minded coop.
It's not a bad idea, but there are a couple of sticking points that make it a bit tough for MUUG:
1. Liability for what members put on it. What if they put naughty stuff? Not sure tiny MUUG would get the same protection the big providers do. Even if ultimately protected, any legal fight alone would be disaster.
2. Liability if we lose the data.
3. Liability if the server gets hacked and data is leaked.
4. Trust and need-to-know access levels on the server for board members (the admins).
5. Hard to police that only members are using it. (Pw sharing, etc.)
6. Don't want to irk Les.
On the pro side, we do have decent space left still, and it would make a nice benefit for members. My guess is it's not going to happen, unless lawyers and govs cease to exist tomorrow. :-) But we will certainly discuss it at a board meeting as a member suggestion.
On 2023-02-10 Adam Thompson wrote:
The last time I ran the numbers for my own reasons (last year), what I pay Backblaze ~$10/mo for would cost roughly ~$200/month for a local operation, if operated on a "hobby" scale. That's not a typo: scale allows them to reduce total end-user costs by 20-fold and still (presumably) make a profit. Even more extreme ratios apply to public or private cloud hosting.
Well, my personal business has run a backup service for local people / businesses for 22 years. And I do it for gobs under $200/mo. But yes, it's over $10 mo: then again it's a service, not a DIY self-managed cloud thing, and includes a free managed firewall/router setup (minus hardware)! My point is, there are affordable options that still keep your data out of Silicon Valley/NSA, even if you don't want to DIY.
If someone wanted to do the same idea as a co-op thing, maybe that could work. It might better fit a venue like Skullspace though?
If encrypting your data locally and only then uploading it doesn't meet your privacy requirements, you probably shouldn't be connected to the internet at all... Also, see Tarsnap, an online backup service for the EXTREMELY privacy-conscious.
Ah, you're assuming SSL and <insert gov-endorsed encryption alg here> don't all have NSA backdoors. You're not thinking like a paranoid, Adam! ;-)
Personally, my own data is on RAID-6 and is backed-up periodically to encrypted optical media and stored off-site using my own custom software. I consider my data very safe, as it would take a volcano sprouting at Portage & Main or a nuke of all of the Wpg area to hose my data. :-)
Although with fingers crossed I was hoping for better. Nevertheless the facts are always friendly and I thank each of you for your detailed reply. Thanks!
The idea of using encryption as part of backups with third parties was a novel new safety application for me but raises concerns like the guy who bought cryptocurrency but lost a huge sum because you know what he forgot? :)
Eduard
On 2023-02-09 22:08, Trevor Cordes wrote:
On 2023-02-09 eh@eduardhiebert.com wrote:
Wondering what interest there might be in doing another variant on a Muug type project but specific to creating a Muug type back-up service and mirrored for increased saftey. One system with huge memory would be but a tiny fraction of many of us doing our own. I'm not suggesting our people with know how do this as a labour of love but as a kind of community minded coop.
It's not a bad idea, but there are a couple of sticking points that make it a bit tough for MUUG:
- Liability for what members put on it. What if they put naughty
stuff? Not sure tiny MUUG would get the same protection the big providers do. Even if ultimately protected, any legal fight alone would be disaster.
Liability if we lose the data.
Liability if the server gets hacked and data is leaked.
Trust and need-to-know access levels on the server for board members
(the admins).
Hard to police that only members are using it. (Pw sharing, etc.)
Don't want to irk Les.
On the pro side, we do have decent space left still, and it would make a nice benefit for members. My guess is it's not going to happen, unless lawyers and govs cease to exist tomorrow. :-) But we will certainly discuss it at a board meeting as a member suggestion.
On 2023-02-10 Adam Thompson wrote:
The last time I ran the numbers for my own reasons (last year), what I pay Backblaze ~$10/mo for would cost roughly ~$200/month for a local operation, if operated on a "hobby" scale. That's not a typo: scale allows them to reduce total end-user costs by 20-fold and still (presumably) make a profit. Even more extreme ratios apply to public or private cloud hosting.
Well, my personal business has run a backup service for local people / businesses for 22 years. And I do it for gobs under $200/mo. But yes, it's over $10 mo: then again it's a service, not a DIY self-managed cloud thing, and includes a free managed firewall/router setup (minus hardware)! My point is, there are affordable options that still keep your data out of Silicon Valley/NSA, even if you don't want to DIY.
If someone wanted to do the same idea as a co-op thing, maybe that could work. It might better fit a venue like Skullspace though?
If encrypting your data locally and only then uploading it doesn't meet your privacy requirements, you probably shouldn't be connected to the internet at all... Also, see Tarsnap, an online backup service for the EXTREMELY privacy-conscious.
Ah, you're assuming SSL and <insert gov-endorsed encryption alg here> don't all have NSA backdoors. You're not thinking like a paranoid, Adam! ;-)
Personally, my own data is on RAID-6 and is backed-up periodically to encrypted optical media and stored off-site using my own custom software. I consider my data very safe, as it would take a volcano sprouting at Portage & Main or a nuke of all of the Wpg area to hose my data. :-)
@Eduard In cases like this I think it's helpful to refer back to the MUUG mission statement: https://muug.ca/pub/bylaws/bylaws-nov2011.pdf
The objectives of the group shall be (Paraphrased)
- Promote free exchange of information and practice of open systems tech in MB - Enhance professional efficiency and effectiveness of its members - Encourage cooperation among industry, gov, edu, and other special interest groups
The way I've always interpreted the mission is that the primary objectives are community building, and to support local users by providing a platform to share learnings, and receive feedback on whatever they might be working on.
I'm not opposed to expanding the scope of services offered at some point, but since the server admin team are all volunteers I'd be concerned about keeping the amount of ongoing maintenance at a sustainable level. Going down the service provider route isn't impossible, but I'd consider it a significant change of direction for the club 🙂
*Quick aside*: I've been using *Deja-Dup https://wiki.gnome.org/Apps/DejaDup* since switching to desktop Linux mid-2022. This app rocks, it's very simple to set up, and thanks to using *Duplicity https://en.wikipedia.org/wiki/Duplicity_(software)* under the hood it allows you to encrypt all backups using a password. Even though my backups are uploaded to Microsoft Onedrive, the actual contents of the backups are not visible. Would 100% recommend giving it a look if you're evaluating cheap offsite backup solutions.
*Quick aside 2*: look up tilde communities https://tildeverse.org/ for an example of a Linux group that's more focused on providing services than community building. Very interesting topic, but a little out of the scope of this discussion so I'll leave it at that for now.
On Sun, Feb 12, 2023 at 8:24 PM eh@eduardhiebert.com wrote:
Although with fingers crossed I was hoping for better. Nevertheless the facts are always friendly and I thank each of you for your detailed reply. Thanks!
The idea of using encryption as part of backups with third parties was a novel new safety application for me but raises concerns like the guy who bought cryptocurrency but lost a huge sum because you know what he forgot? :)
Eduard
On 2023-02-09 22:08, Trevor Cordes wrote:
On 2023-02-09 eh@eduardhiebert.com wrote:
Wondering what interest there might be in doing another variant on a Muug type project but specific to creating a Muug type back-up service and mirrored for increased saftey. One system with huge memory would be but a tiny fraction of many of us doing our own. I'm not suggesting our people with know how do this as a labour of love but as a kind of community minded coop.
It's not a bad idea, but there are a couple of sticking points that make it a bit tough for MUUG:
- Liability for what members put on it. What if they put naughty
stuff? Not sure tiny MUUG would get the same protection the big providers do. Even if ultimately protected, any legal fight alone would be disaster.
Liability if we lose the data.
Liability if the server gets hacked and data is leaked.
Trust and need-to-know access levels on the server for board members
(the admins).
Hard to police that only members are using it. (Pw sharing, etc.)
Don't want to irk Les.
On the pro side, we do have decent space left still, and it would make a nice benefit for members. My guess is it's not going to happen, unless lawyers and govs cease to exist tomorrow. :-) But we will certainly discuss it at a board meeting as a member suggestion.
On 2023-02-10 Adam Thompson wrote:
The last time I ran the numbers for my own reasons (last year), what I pay Backblaze ~$10/mo for would cost roughly ~$200/month for a local operation, if operated on a "hobby" scale. That's not a typo: scale allows them to reduce total end-user costs by 20-fold and still (presumably) make a profit. Even more extreme ratios apply to public or private cloud hosting.
Well, my personal business has run a backup service for local people / businesses for 22 years. And I do it for gobs under $200/mo. But yes, it's over $10 mo: then again it's a service, not a DIY self-managed cloud thing, and includes a free managed firewall/router setup (minus hardware)! My point is, there are affordable options that still keep your data out of Silicon Valley/NSA, even if you don't want to DIY.
If someone wanted to do the same idea as a co-op thing, maybe that could work. It might better fit a venue like Skullspace though?
If encrypting your data locally and only then uploading it doesn't meet your privacy requirements, you probably shouldn't be connected to the internet at all... Also, see Tarsnap, an online backup service for the EXTREMELY privacy-conscious.
Ah, you're assuming SSL and <insert gov-endorsed encryption alg here> don't all have NSA backdoors. You're not thinking like a paranoid, Adam! ;-)
Personally, my own data is on RAID-6 and is backed-up periodically to encrypted optical media and stored off-site using my own custom software. I consider my data very safe, as it would take a volcano sprouting at Portage & Main or a nuke of all of the Wpg area to hose my data. :-)
Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable
Another option is to find someone who will host your backup server at their home. You could do the same for them. With encrypted backups, that might cover most personal use cases. With many internet plans these days, there is bandwidth to spare.
FYI I'm using TrueNAS Core (previously known as FreeNAS) at both ends.
On Mon, Feb 13, 2023 at 12:35 PM Chris Audet cj.audet@gmail.com wrote:
@Eduard In cases like this I think it's helpful to refer back to the MUUG mission statement: https://muug.ca/pub/bylaws/bylaws-nov2011.pdf
The objectives of the group shall be (Paraphrased)
- Promote free exchange of information and practice of open systems
tech in MB
- Enhance professional efficiency and effectiveness of its members
- Encourage cooperation among industry, gov, edu, and other special
interest groups
The way I've always interpreted the mission is that the primary objectives are community building, and to support local users by providing a platform to share learnings, and receive feedback on whatever they might be working on.
I'm not opposed to expanding the scope of services offered at some point, but since the server admin team are all volunteers I'd be concerned about keeping the amount of ongoing maintenance at a sustainable level. Going down the service provider route isn't impossible, but I'd consider it a significant change of direction for the club 🙂
*Quick aside*: I've been using *Deja-Dup https://wiki.gnome.org/Apps/DejaDup* since switching to desktop Linux mid-2022. This app rocks, it's very simple to set up, and thanks to using *Duplicity https://en.wikipedia.org/wiki/Duplicity_(software)* under the hood it allows you to encrypt all backups using a password. Even though my backups are uploaded to Microsoft Onedrive, the actual contents of the backups are not visible. Would 100% recommend giving it a look if you're evaluating cheap offsite backup solutions.
*Quick aside 2*: look up tilde communities https://tildeverse.org/ for an example of a Linux group that's more focused on providing services than community building. Very interesting topic, but a little out of the scope of this discussion so I'll leave it at that for now.
On Sun, Feb 12, 2023 at 8:24 PM eh@eduardhiebert.com wrote:
Although with fingers crossed I was hoping for better. Nevertheless the facts are always friendly and I thank each of you for your detailed reply. Thanks!
The idea of using encryption as part of backups with third parties was a novel new safety application for me but raises concerns like the guy who bought cryptocurrency but lost a huge sum because you know what he forgot? :)
Eduard
On 2023-02-09 22:08, Trevor Cordes wrote:
On 2023-02-09 eh@eduardhiebert.com wrote:
Wondering what interest there might be in doing another variant on a Muug type project but specific to creating a Muug type back-up service and mirrored for increased saftey. One system with huge memory would be but a tiny fraction of many of us doing our own. I'm not suggesting our people with know how do this as a labour of love but as a kind of community minded coop.
It's not a bad idea, but there are a couple of sticking points that make it a bit tough for MUUG:
- Liability for what members put on it. What if they put naughty
stuff? Not sure tiny MUUG would get the same protection the big providers do. Even if ultimately protected, any legal fight alone would be disaster.
Liability if we lose the data.
Liability if the server gets hacked and data is leaked.
Trust and need-to-know access levels on the server for board members
(the admins).
Hard to police that only members are using it. (Pw sharing, etc.)
Don't want to irk Les.
On the pro side, we do have decent space left still, and it would make a nice benefit for members. My guess is it's not going to happen, unless lawyers and govs cease to exist tomorrow. :-) But we will certainly discuss it at a board meeting as a member suggestion.
On 2023-02-10 Adam Thompson wrote:
The last time I ran the numbers for my own reasons (last year), what I pay Backblaze ~$10/mo for would cost roughly ~$200/month for a local operation, if operated on a "hobby" scale. That's not a typo: scale allows them to reduce total end-user costs by 20-fold and still (presumably) make a profit. Even more extreme ratios apply to public or private cloud hosting.
Well, my personal business has run a backup service for local people / businesses for 22 years. And I do it for gobs under $200/mo. But yes, it's over $10 mo: then again it's a service, not a DIY self-managed cloud thing, and includes a free managed firewall/router setup (minus hardware)! My point is, there are affordable options that still keep your data out of Silicon Valley/NSA, even if you don't want to DIY.
If someone wanted to do the same idea as a co-op thing, maybe that could work. It might better fit a venue like Skullspace though?
If encrypting your data locally and only then uploading it doesn't meet your privacy requirements, you probably shouldn't be connected to the internet at all... Also, see Tarsnap, an online backup service for the EXTREMELY privacy-conscious.
Ah, you're assuming SSL and <insert gov-endorsed encryption alg here> don't all have NSA backdoors. You're not thinking like a paranoid, Adam! ;-)
Personally, my own data is on RAID-6 and is backed-up periodically to encrypted optical media and stored off-site using my own custom software. I consider my data very safe, as it would take a volcano sprouting at Portage & Main or a nuke of all of the Wpg area to hose my data. :-)
Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable
Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable
On 2023-01-26 Hartmut W Sager wrote:
Indeed, I did conflate high-MTBF with "good for always-on". Now if we do go for the extra computer or SBC on a LAN, we have noticed that this external USB3 8TB HDD connected to her current desktop computer does spin-down quite soon after last access, and her access pattern would be a few hours of almost continuous access followed by no access for the rest of the 24-hour day.
Yes, that access pattern would help prolong its life, as long as the OS isn't doing random/periodic weirdness that wakes it up every few mins or hours. That would actually be harder on it. You won't know until you make your rig and see how it behaves.
See somewhere above; for the moment, we have that 2nd HDD (mine), but we may get her something additional. But question: Are the cheap ones "reliable" if we operate such a HDD for very few hours, or are they junky all around, and prone to failure even with low usage? I suspect the latter, and if so, I'll avoid the "cheapest one".
What I propose would mean buying a dirt cheap USB for periodic backups: as in plug it in, copy all the contents from the main drive (wait 10 hours), then unplug the cheap one and put it in a closet. Wait a few weeks/months and rinse/repeat. That is most USB drives' design use case and most should have decent longevity when used in that manner.
If we go Pi, I presume I could engage one of you MUUG folks here for payment?
If you wanted to hire someone, best to spec out what you expect the box to do in terms of protocols supported and any extra s/w required (such as a media sharing server). I'm great at normal boxes with Fedora, others here are (much) better at Pi. Maybe if someone is interested in helping with the job they can private email you. You could also put out an official request on the jobs@muug.ca list. I would assume someone could set you up very quickly, as I'm sure there are prebuilt tools for all of this: it can't be that rare a use case. Good luck!
On 2023-01-26 Hartmut W Sager wrote: Indeed, I did conflate high-MTBF with "good for always-on". Now if we do go for the extra computer or SBC on a LAN, we have noticed that this external USB3 8TB HDD connected to her current desktop computer does spin-down quite soon after last access, and her access pattern would be a few hours of almost continuous access followed by no access for the rest of the 24-hour day.
On Fri 27 Jan 2023 at 02:24:02 -06:00, Trevor Cordes trevor@tecnopolis.ca wrote: Yes, that access pattern would help prolong its life, as long as the OS isn't doing random/periodic weirdness that wakes it up every few mins or hours. That would actually be harder on it. You won't know until you make your rig and see how it behaves.
Very good point! Such OS behaviour would indeed destroy a fundamental premise here.
If we go Pi, I presume I could engage one of you MUUG folks here for payment?
If you wanted to hire someone, best to spec out what you expect the box to do ..... Maybe if someone is interested in helping with the job they can private email you. You could also put out an official request on the jobs@muug.ca list.
Excellent thoughts. I will probably take the next few months, though, before I finalize these matters.
Hartmut
A couple of final thoughts now that we know the situation:
1. Make sure whatever NAS "device" you settle on for Lynn's side formats the HDD with a fs that your computer can understand. If you only have Windows, then make sure the NAS uses NTFS. If you have Linux, then the world's your oyster, and I would opt for a Linux fs, but NTFS will be ok with ntfs3g.
2. Whatever the Seagate MTBF says, take it with a grain of salt. Look carefully at the box/instructions/specs to see if it says "not for always-on use". If you have some funky special-grade device, I'd be curious to know the model #.
3. John and Adam are right: back the thing up. Buy a 2nd 8TB USB drive, the cheapest one you can find (often there are sales that are very attractive on external drives). Just do a byte-level copy of the whole thing, or a file-level backup, every few months (i.e. no RAID). The NAS device you settle on might have an option to do this for you. I will virtually guarantee you your drive will die before you get a hankering to replace it, even if you haven't personally experienced such a thing.
The very worst thing to lose is the drive full of irreplaceable family photos.
Also, John's Pi solution might be ideal if you don't want to wing it, and even better if someone makes some nice NAS OS images that you can just plop onto it?? For your use case, performance limitations won't be a big deal.