Product idea: RAID Backup Enclosures

Cory Doctorow at Boing Boing links to an article at TechCrunch that lists Better and Cheaper Online File Storage as a product that needs to be made. However, Ben Laurie does the sums on online storage as a useful backup medium, and found them not exactly compelling (e.g. 100GB of data will take 75 days to upload over an 128Kbps link).

I tend to agree. An online host isn’t great as a backup host, since, in my experience, there are two types of backups required:

  • The important small files (for example: encrypted password lists, my address book, my ~/bin directory)
  • The massive big filesets (for example: MP3s, photos)

The first kind of fileset is amenable to an online backup-storage service, at first glance. However — in my opinion you’re better off going the whole hog for these files, and using the distributed, versioned backup method of putting it in a good networked revision control system, and checking it out everywhere, so you can also make changes and check in from any host; otherwise, you face the perils of syncing up a single backup from multiple “writers”, without conflicts. So far, none of the online file storage services offer SVN as an access method, so a shell account at a colo server still seems more useful on that count.

The second kind of fileset, as Ben notes, will take donkey’s years to upload and sync as a backup mechanism; and the economics are hardly compelling for the service provider.

I think I prefer Brad Templeton’s idea to deal with large-data backups —

I propose a software RAID-5, done over a LAN with 3 to 5 drives scattered over several machines on the LAN.

Slow as hell, of course, having to read and write your data out over the LAN even at 100mbits. Gigabit would obviously be better. But what is it we have that’s taking up all this disk space ? it?s video, music and photos. Things which, if just being played back, don?t need to be accessed very fast. If you’re not editing video or music, in particular, you can handle having it on a very slow device. (Photos are a bigger issue, as they do sometimes need fast access when building thumbnails etc.)

This could even be done among neighbours over 802.11g, with suitable encryption. In theory.

As a commenter notes, Linux has support for this already, in the form of software RAID and the network block device.

So: take an external IDE enclosure, add a GumStix board running Linux with software RAID, LVM, and nbd, and add wifi. Then add DAV, SMB and NFS export of the disk, and some decent UI code to organise the volumes into a single exported RAID volume (hopefully automatically!), and it’d be a pretty compelling product, in my opinion!

(hey Craig! I said GumStix! ;)

This entry was posted in Uncategorized and tagged , , , , , , , , . Bookmark the permalink. Both comments and trackbacks are currently closed.

12 Comments

  1. Posted November 24, 2005 at 03:24 | Permalink

    Full disclosure I do this for a living, benefit materially, all opinions are mine and mine alone, not in any way connected to any device named here unless I explicitly say so, blah, blah, blah…

    “But what is it we have that’s taking up all this disk space ? it?s video, music and photos. Things which, if just being played back, don?t need to be accessed very fast. If you’re not editing video or music, in particular, you can handle having it on a very slow device. (Photos are a bigger issue, as they do sometimes need fast access when building thumbnails etc.)”

    I say that’s not backup, it’s archiving. Backups are for fast restores of operational data, getting you moving as quickly as possible after an event or failure, not near line (slow) file access. He’s not talking about backup anymore he’s talking about data classification and Storage Resource Management. Migrating data between different tiers of storage. Yes if you want to speed up your backups you should be pulling stuff off the backup target and moving it into an active archive.

    There are consumer level devices on the market in that space, such as Storage Central http://www.netgear.com/promotions/2005/storage_central.php , which can serve as easy to use RAID protected NAS target, it’s just a bloody shame that in Storage Central’s case it requires a Windows only driver to communicate with it.

  2. Posted November 24, 2005 at 05:03 | Permalink

    wow, that Storage Central box does look cool; halfway there. Mind you, it’s NetGear — no bloody way I’ll ever touch any of their crap again after suffering the MR814 router.

  3. Posted November 24, 2005 at 11:12 | Permalink

    Oh I totally agree about NetGear, but the software for that particular product is OEM’d from a startup called Zetera, so it’s bound to show up in another vendors offering at some stage.

    Maybe then someone will deliver OS X/Linux support (Translation: Tell Zetera to write the required code), and I could roll it out at home.

  4. Posted December 5, 2005 at 23:48 | Permalink

    heh. Zetera are just down the road from me, here in Irvine! Must be a Maxtor spin-off.

    Bit of a turd of a review here — some atrocious feedback in the comments. That’s not a good sign, at all.

  5. Posted December 6, 2005 at 04:08 | Permalink

    Ha. I knew the proprietary IP stuff, which is kludged UDP if I recall, was a pain in the ass but I didn’t bloody well know that there wasn’t any insight into or management of the unit.

    That’s a data roach motel right there. I’ll stick to RAID 1/RAID 10 and NFS/SMB for my home NAS needs.

  6. Posted December 6, 2005 at 04:16 | Permalink

    Speak of the devil, and my RSS reader, Zetra appear to be on a [press binge.]((http://www.eweek.com/article2/0,1759,1894001,00.asp?kc=EWRSS03119TX1K0000594) I know I’m a vested interest since I work for the largest information storage & storage management corporation in the world, but there’s no way in hell Z-SAN is going to fly in the face of current standards like iSCSI.

  7. Posted December 27, 2005 at 18:45 | Permalink

    dunno if its of interest but http://www.24sevenbackup.ie are doing an online backup product.

    at the moment i am giving it a whirl as a freebie for a year. i’ll let you know how it goes when i start getting some serious usage of it.

    rwb

  8. Posted December 27, 2005 at 20:08 | Permalink

    Hey Justin, it’s a sign from god. Throw out all that smelly socialist Linux crap and run Windows so you can back it up. ;-p

  9. Posted December 29, 2005 at 22:11 | Permalink

    I know, I know, but I just keep finding more NAS appliances.

    http://www.infrant.com/productsReadyNASX6.htm

  10. Posted May 5, 2006 at 04:20 | Permalink

    Been there…..

    After going through the mill with norton ghost / Acronis / Verital…… I ended up using the online option. Safer, automatic, email reporting and instant retrieval. I priced around and got the feeling that most of them were only agents and were using standard software. I ended up signing with backupanytime.com They weren’t any cheaper but they support just about everything including mac and linux both of which i use and have little knoledge of (seperately…lol) Have it going now three months and I’ve tested it and the support. All round – excellent.

  11. Posted January 10, 2007 at 14:03 | Permalink

    We signed up with backupanytime.com and yes it works perfectly. They introduced us to their sister company spamseal.com so we could ensure we were not paying to backup spam mail. We took this with a pinch of salt as we have tried many so called antispam solutions but guess what, we tried it and it worked — perfectly — we get no spam now and we have registered on some crazy sites to check it. Nice one guys.

  12. Posted January 10, 2007 at 14:19 | Permalink

    Spam warning: “Jason” is advertising his own sites.

    whois irishmedical.com: Domain name: irishmedical.com

    Registrant Contact: O’Neill e-business John O’Neill ([email protected]) +353.868101404 Fax: 22 Portacarron, Knocknacarra Galway, Galway – IE

    whois backupanytime.com: O’Neill e-business NA NA (NA) NA Fax: 22 Portacarron, Knocknacarra Galway, GALWAY – IE

    whois spamseal.com: O’Neill e-business John O’Neill ([email protected]) +353.868101404 Fax: 22 Portacarron, Knocknacarra Galway, Galway – IE

    Author : Jason (IP: 86.43.99.210 , 86.43.99.210) E-mail : [email protected] URI : http://www.irishmedical.com

    It looks like “B.T.” is similarly a spammer, probably the same guy. Nice try, spammer….