taint.org: Justin Mason’s Weblog

• Dabble DB : looks like a web-based version of good old Filemaker
  (tags: dabble-db databases web db spreadsheets analytics groupware)

• retrocomputing hackers reminisce about the 1541 diskette drive : skip to the comment thread, it’s fantastic. I used to know lots of this stuff (via Donncha)
  (tags: via:donncha commodore-64 hacks 1541 disks copy-protection drm hardware hacking history retro c64)

• MPLC racketeering Irish playschools : the ‘Motion Picture Licensing Company’ sent a letter to 2,500 Irish playschools, demanding a fee of EUR3 per child to cover license fees for the kids watching DVDs. However, it seems they themselves hadn’t registered as required by law, so were acting illegally in issuing demands… oh the irony
  (tags: mplc racketeering law ireland dvds movies mpaa licensing copyright legal children kindergarten playschools)

• Komplett’s new Dublin pick-up point is open : routing around the Irish couriers and An Post’s brokenness by allowing customers to pick up their items directly. a shame this is necessary
  (tags: an-post delivery couriers dublin ireland komplett components pc hardware)

Tags: 1541, an-post, analytics, c64, children, commodore-64, components, copy-protection, copyright, couriers, dabble-db, databases, db, delivery, disks, drm, dublin, dvds, groupware, hacking, hacks, hardware, history, ireland, kindergarten, komplett, law, legal, licensing, movies, mpaa, mplc, pc, playschools, racketeering, retro, spreadsheets, via:donncha, web

• Ubuntu and Debian AMIs for Amazon EC2 : they look well-maintained
  (tags: amis debian ubuntu ec2 aws)

• Making an AMI use EBS as the root filesystem : using some nasty pivot_root hackery
  (tags: root filesystems aws ec2 amazon ebs disks linux)

• Richard Hulse: Clearing the Cache on Matrix : IPC::DirQueue used in the field as part of a cache-invalidation system for a NZ radio station
  (tags: ipc-dirqueue cache-revalidation users user-stories)

Tags: amazon, amis, aws, cache-revalidation, debian, disks, ebs, ec2, filesystems, ipc-dirqueue, linux, root, ubuntu, user-stories, users

• Four Tweaks for Using Linux with Solid State Drives : good tips (via Jeremy)
  (tags: for:hughescr linux ssd disks optimization performance tmpfs kernel)

• VW Should Bring Back The Microbus And Make It Electric : and a pony! (via fergusb)
  (tags: want electric-vehicles cars volkswagen techcrunch over-ambitious and-a-pony)

Tags: and-a-pony, cars, disks, electric-vehicles, for:hughescr, kernel, linux, optimization, over-ambitious, performance, ssd, techcrunch, tmpfs, volkswagen, want

• 30% Of Internet Users Admit To Buying From Spam : ugh. we need a reminder of the Boulder Pledge. Mind you, Marshal have put out what appear to be inaccurate figures in the past regarding the Rustock botnet, so apply a pinch of salt
  (tags: marshal spam boulder-pledge commerce shopping statistics via:techdirt)

• more details on EC2 Elastic Block Store : achieves 70MB/s on an m1.small instance; ‘performance exceeds what we’ve seen for filesystems striped across the four local drives of x-large instances’. pretty good for a network filesystem, although not great compared to fast local SATA disks. also: snapshots are incremental and perform nicely compared to local S3 copy-and-upload
  (tags: ebs amazon performance benchmarks s3 ec2 disks snapshotting filesystems)

• Amazon Elastic Block Store (EBS) : ‘Prior to Amazon EBS, block storage within an Amazon EC2 instance was tied to the instance itself so that when the instance was terminated, the data within the instance was lost. Now with Amazon EBS, users can chose to allocate storage volumes that persist reliably and independently from Amazon EC2 instances.’ — can even snapshot to S3
  (tags: amazon ebs ec2 aws s3 cloud-computing hosting)

Tags: amazon, aws, benchmarks, boulder-pledge, cloud-computing, commerce, disks, ebs, ec2, filesystems, hosting, marshal, performance, s3, shopping, snapshotting, spam, statistics, via:techdirt

• non-PC devices increasing browser share : .5 – 1.5% of visitors to Warner music sites are now coming from games consoles and smartphones. bad news for Flash sites (via Torrez)
  (tags: flash web browsers via:torrez warner-music os)

• full(4): always full device : ‘Writes to the /dev/full device will fail with an ENOSPC error. This can be used to test how a program handles disk-full errors.’ – that’s nifty. I can’t believe I’m still finding useful new UNIX features after 18 years
  (tags: devices unix linux testing disks errors edge-cases enospc manual-pages)

Tags: browsers, devices, disks, edge-cases, enospc, errors, flash, linux, manual-pages, os, testing, unix, via:torrez, warner-music, web

Green Karma – Carbon-offset your colo box must-read post from Chris. If you run a colo box, you should think about offsetting the ~2 tonnes of CO2 output it generates per year

sorenragsdale: Building a Cheap ZFS Server good set of details on MrN’s new ZFS-based home disk server

Tags: carbon, carbon-dioxide, co2, colo, disks, drobo, environment, filesystems, green, hacking, hardware, home, hosting, offsetting, readynas, servers, solaris, sysadmin, via:iloaf, zfs

I’ve been running into problems recently (since several months ago at least), with USB hardware on my Thinkpad T40 running Ubuntu Hoary Dapper; in particular, every time I plug in my iPod or one of my USB hard disks nowadays, I get this:

[5008549.187000] usb 4-3: USB disconnect, address 14
[5008550.143000] usb 4-3: new high speed USB device using ehci_hcd and address 18
[5008552.643000] usb 4-3: new high speed USB device using ehci_hcd and address 27
[5008557.393000] usb 4-3: new high speed USB device using ehci_hcd and address 43
[5008557.893000] usb 4-3: new high speed USB device using ehci_hcd and address 44
[5008558.643000] usb 4-3: new high speed USB device using ehci_hcd and address 46
[5008558.895000] ehci_hcd 0000:00:1d.7: port 3 reset error -110
[5008558.896000] hub 4-0:1.0: hub_port_status failed (err = -32)
[5008559.893000] usb 4-3: new high speed USB device using ehci_hcd and address 48
[5008562.643000] usb 4-3: new high speed USB device using ehci_hcd and address 58
[5008563.143000] usb 4-3: new high speed USB device using ehci_hcd and address 59
[5008563.643000] usb 4-3: new high speed USB device using ehci_hcd and address 60
[5008570.143000] usb 4-3: new high speed USB device using ehci_hcd and address 85

This repeats ad infinitum until the USB device is disconnected.

I had this down as a hardware issue (since it started happening just after warranty expiration ;), but some accidental googling revealed several other cases – and a workaround:

sudo modprobe -r ehci-hcd

Run that repeatedly, each time replugging the device and monitoring dmesg via watch -n 1 ‘dmesg | tail’ in a window, until the device is finally recognised as a USB hard disk. It generally seems to take 3 or 4 attempts, in my experience.

This LKML thread suggests hardware changes can cause it, but this hardware hasn’t changed in years. Annoying.

Anyway, this is ongoing. This tip seems to help, but it might be just treating a symptom, I don’t know — just posting for google and posterity… and to moan, of course :(

Tags: bugs, disks, ehci-hcd, hardware, ipod, linux, storage, usb

Keith Dawson sent this on — an interview with Jim Gray, head of Microsoft’s Bay Area Research Center and winner of the ACM Turing Award, talking about new transmission systems for truly massive data collections. Very interesting:

[One] option is to send whole computers. …. We’re now into the 2-terabyte realm, so we can’t actually send a single disk; we need to send a bunch of disks. It’s convenient to send them packaged inside a metal box that just happens to have a processor in it. I know this sounds crazy — but you get an NFS or CIFS server and most people can just plug the thing into the wall and into the network and then copy the data.

Dave Patterson, interviewer: What’s the difference in cost between sending a disk and sending a computer?

JG: If I were to send you only one disk, the cost would be double – something like $400 to send you a computer versus $200 to send you a disk. But I am sending bricks holding more than a terabyte of data — and the disks are more than 50 percent of the system cost. Presumably, these bricks circulate and don’t get consumed by one use.

DP: Are you sending them a whole PC?

JG: Yes, an Athlon with a Gigabit Ethernet interface, a gigabyte of RAM, and seven 300-GB disks — all for about $3,000.

DP: It’s your capital cost to implement the Jim Gray version of “Netflicks.” (jm: sic)

JG: Right. We built more than 20 of these boxes we call TeraScale SneakerNet boxes. Three of them are in circulation. We have a dozen doing TeraServer work; we have about eight in our lab for video archives, backups, and so on. It’s real convenient to have 40 TB of storage to work with if you are a database guy. Remember the old days and the original eight-inch floppy disks? These are just much bigger.

DP: “Sneaker net” was when you used your sneakers to transport data?

JG: In the old days, sneaker net was the notion that you would pull out floppy disks, run across the room in your sneakers, and plug the floppy into another machine. This is just TeraScale SneakerNet. You write your terabytes onto this thing and ship it out to your pals. Some of our pals are extremely well connected — they are part of Internet 2, Virtual Business Networks (VBNs), and the Next Generation Internet (NGI). Even so, it takes them a long time to copy a gigabyte. Copy a terabyte? It takes them a very, very long time across the networks they have.

Tags: bandwidth, disks, fedex, jim-gray, ms-research, shipping, sneakernet, storage, terabytes, ups

Cory Doctorow at Boing Boing links to an article at TechCrunch that lists Better and Cheaper Online File Storage as a product that needs to be made. However, Ben Laurie does the sums on online storage as a useful backup medium, and found them not exactly compelling (e.g. 100GB of data will take 75 days to upload over an 128Kbps link).

I tend to agree. An online host isn’t great as a backup host, since, in my experience, there are two types of backups required:

• The important small files (for example: encrypted password lists, my address book, my ~/bin directory)
• The massive big filesets (for example: MP3s, photos)

The first kind of fileset is amenable to an online backup-storage service, at first glance. However — in my opinion you’re better off going the whole hog for these files, and using the distributed, versioned backup method of putting it in a good networked revision control system, and checking it out everywhere, so you can also make changes and check in from any host; otherwise, you face the perils of syncing up a single backup from multiple “writers”, without conflicts. So far, none of the online file storage services offer SVN as an access method, so a shell account at a colo server still seems more useful on that count.

The second kind of fileset, as Ben notes, will take donkey’s years to upload and sync as a backup mechanism; and the economics are hardly compelling for the service provider.

I think I prefer Brad Templeton’s idea to deal with large-data backups –

I propose a software RAID-5, done over a LAN with 3 to 5 drives scattered over several machines on the LAN.

Slow as hell, of course, having to read and write your data out over the LAN even at 100mbits. Gigabit would obviously be better. But what is it we have that’s taking up all this disk space ? it?s video, music and photos. Things which, if just being played back, don?t need to be accessed very fast. If you’re not editing video or music, in particular, you can handle having it on a very slow device. (Photos are a bigger issue, as they do sometimes need fast access when building thumbnails etc.)

This could even be done among neighbours over 802.11g, with suitable encryption. In theory.

As a commenter notes, Linux has support for this already, in the form of software RAID and the network block device.

So: take an external IDE enclosure, add a GumStix board running Linux with software RAID, LVM, and nbd, and add wifi. Then add DAV, SMB and NFS export of the disk, and some decent UI code to organise the volumes into a single exported RAID volume (hopefully automatically!), and it’d be a pretty compelling product, in my opinion!

(hey Craig! I said GumStix! ;)

Tags: backups, disks, gumstix, hardware, linux, lvm, nbd, product, raid