taint.org: Justin Mason’s Weblog

Recently, there’s been a bit of discussion online about whether or not it makes sense for companies to host server infrastructure at Amazon EC2, or on traditional colo infrastructure. Generally, these discussions have focussed on one main selling point of EC2: its elasticity, the ability to horizontally scale the number of server instances at a moment’s notice.

If you’re in a position to gain from elasticity, that’s great. But it is still worth noting that even if you aren’t in that position, there’s another good reason to host at an EC2-like cloud; if you want to deploy another copy of the app, either from a different version-control branch (dev vs staging vs production deployments), or to run separate apps with customizations for different customers. These aren’t scaling an existing app up, they’re creating new copies of the app, and EC2 works nicely to do this.

If you can deploy a set of servers with one click from a source code branch, this is entirely viable and quite useful.

Another reason: EC2-to-S3 traffic is extremely fast and cheap compared to external-to-S3. So if you’re hosting your data on S3, EC2 is a great way to crunch on it efficiently. Update: Walter observed this too on the backend for his Twitter Mosaic service.

Tags: amazon, aws, cloud-computing, ec2, hosting, s3, servers

• Symantec buys MessageLabs : hope this works out well for Matt and the guys
  (tags: messagelabs anti-spam consolidation buyouts mergers symantec)

• There is NOTHING “overplayed” about privacy concerns : great rant opposing Symantec CEO’s wittering about how online privacy is a lost cause
  (tags: privacy online internet web symantec rants)

• S3rsync : ‘Full Rsync layer to Amazon S3 storage’. preserves file perms, symlinks, partial files. commercial, charged at 5 cents per hour of rsync time used
  (tags: rsync s3 amazon aws storage backup sync)

• why you shouldn’t believe companies when they play down data leaks : in 2006, T-Mobile lost data on 17m customers, but noted that they could find no evidence of misuse or compromise, so made no public announcement — until last weekend, when _Der Spiegel_ discovered this data for sale online. so much for keeping it quiet. another argument for compulsory breach disclosure
  (tags: der-spiegel t-mobile deutsche-telekom privacy security breaches breach-disclosure via:pogowasright germany leaks)

• macvim – Google Code : ‘a port of the text editor Vim to Mac OS X that is meant to look better and integrate more seamlessly with the Mac than the older Carbon port of Vim’ (thanks MinHee Hong!)
  (tags: vim macos mac editors porting tools applications)

Tags: amazon, anti-spam, applications, aws, backup, breach-disclosure, breaches, buyouts, consolidation, der-spiegel, deutsche-telekom, editors, germany, internet, leaks, mac, macos, mergers, messagelabs, online, porting, privacy, rants, rsync, s3, security, storage, symantec, sync, t-mobile, tools, via:pogowasright, vim, web

• Addictions, Think Amazon, not Google : Brian “Krow” Aker: ‘Google’s AppEngine is much closer to [...] “Digitial Sharecropping” [...] S3 and EC2 have little tie in to them. You can end up with a physical addiction to the services but the mental addiction to a framework does not exist. S3 is just storage and EC2 for most is just a hosted Linux image.’ +1!
  (tags: google gae aws ec2 amazon hosting s3)

• The Porterhouse to get a shiny new bottling line : hooray, great news for Irish beer drinkers sick of Guinness
  (tags: porterhouse beer ireland brewing)

Tags: amazon, aws, beer, brewing, ec2, gae, google, hosting, ireland, porterhouse, s3

• baltic-avenue: An open source clone of S3 : built on top of Google App Engine. interesting hack!
  (tags: gae s3 aws amazon baltic-avenue google)

Tags: amazon, aws, baltic-avenue, gae, google, s3

• 30% Of Internet Users Admit To Buying From Spam : ugh. we need a reminder of the Boulder Pledge. Mind you, Marshal have put out what appear to be inaccurate figures in the past regarding the Rustock botnet, so apply a pinch of salt
  (tags: marshal spam boulder-pledge commerce shopping statistics via:techdirt)

• more details on EC2 Elastic Block Store : achieves 70MB/s on an m1.small instance; ‘performance exceeds what we’ve seen for filesystems striped across the four local drives of x-large instances’. pretty good for a network filesystem, although not great compared to fast local SATA disks. also: snapshots are incremental and perform nicely compared to local S3 copy-and-upload
  (tags: ebs amazon performance benchmarks s3 ec2 disks snapshotting filesystems)

• Amazon Elastic Block Store (EBS) : ‘Prior to Amazon EBS, block storage within an Amazon EC2 instance was tied to the instance itself so that when the instance was terminated, the data within the instance was lost. Now with Amazon EBS, users can chose to allocate storage volumes that persist reliably and independently from Amazon EC2 instances.’ — can even snapshot to S3
  (tags: amazon ebs ec2 aws s3 cloud-computing hosting)

Tags: amazon, aws, benchmarks, boulder-pledge, cloud-computing, commerce, disks, ebs, ec2, filesystems, hosting, marshal, performance, s3, shopping, snapshotting, spam, statistics, via:techdirt

• Hacking Mifare Transport Cards : notable mainly because Schneier calls MIFARE Classic’s crypto ‘terrible’ and ‘kindergarten cryptography’. ‘Anyone with any security experience would be embarrassed to put his name to the design.’ ZING
  (tags: ouch mifare cracks security zing crypto-cards smartcards oyster nxp)

• some good AWS tips : e.g. “upload files to S3 in lexically-sorted order”, apparently it’s faster. who knew!? (Sorry Joshua, gave you a bad tip y’day in that case)
  (tags: amazon aws s3 sqs sdb web-services hosting for:joshua)

• The Coreflood Report : fascinating Joe Stewart post-mortem of a server run by a Russian malware group targeting online banking; one apparent Miami-based victim was defrauded of $90k, and it appears that the group would have had access to a combined $2.5m in all victims’ accounts
  (tags: coreflood trojans joe-stewart secureworks malware banking online-banking autoproxy joe-lopez)

• using “data=writeback” on your Ubuntu filesystems : aha! This is the root cause of the crippling Firefox 3/VIM slowness; Ubuntu and Debian use a crappy ext3 option which sacrifices speed for correctness, by effectively turning every fsync() into a sync(). Here’s how to disable it
  (tags: fsync sync unix linux ext3 ubuntu debian firefox vim grub annoying sysadmin)

Tags: amazon, annoying, autoproxy, aws, banking, coreflood, cracks, crypto-cards, debian, ext3, firefox, for:joshua, fsync, grub, hosting, joe-lopez, joe-stewart, linux, malware, mifare, nxp, online-banking, ouch, oyster, s3, sdb, secureworks, security, smartcards, sqs, sync, sysadmin, trojans, ubuntu, unix, vim, web-services, zing

Mike Culver, Amazon’s “Web Services Evangelist”, will be in Dublin next week to evangelize about the goodness that is Amazon S3, EC2, SQS and so on. It seems he’ll be talking at the following locations:

• in the Auditorium of the Digital Exchange, Crane Street, Dublin 8 on Tuesday October 30th, 3-5pm; here’s a flyer the Amazonites have been passing around. (upcoming.org page)

• according to Damien, later that evening, he’s in the Westin Hotel on Westmoreland St., D2, starting at 7pm; note, it seems you need to book places at this, see Damien’s post.

• and again at the Irish Linux User’s Group on Thursday November 1st at 19:30 in the Irish Computer Society in Dublin (map).

I guess these are all going to be same talk, bar the Q&A ;)

There was some kind of an ICTE get-together mooted for Friday 2nd.

Also, the ILUG annual general meeting is scheduled on the following Saturday, 3rd November, also at the ICS. Gareth Eason notes ‘we’re hoping to start at 3pm sharp, with talks from Dave Wilson (HEAnet), Frank Duignan, John Looney (Google), and others, followed by a relaxing wind-down in the Schoolhouse pub later on.’ (upcoming.org page)

Hopefully I’ll get to at least one of the AWS talks (probably the Digital Exchange one) and the ILUG AGM… busy week!

Tags: amazon, aws, dublin, ec2, icte, ilug, ireland, linux, mike-culver, s3, social, talks

Like a few other anti-spammers, I found myself under a hitherto-unprecedented level of spam blowback this weekend. Disappointingly, there are still thousands of SMTP servers configured to send bounce messages in response to spam.

Even with the anti-bounce ruleset for SpamAssassin, the volume was so great that our creaky old server had a lot of difficulty keeping up — once the messages got to SpamAssassin, the load issues had already been created. Also, Postfix’s anti-spam features really weren’t designed to deal with blowback.

While attempting to take some shortcuts in the setup on our server to deal with this, a great idea occurred to me — why not come up with an app that uses Amazon EC2 to flexibly provision enough server power and bandwidth to pre-filter the SMTP traffic for an MX under attack?

I’m basically thinking of qpsmtpd, with SpamAssassin and/or other antispam blobs active, running in an Amazon EC2 server image. Multiple images can be brought up, and added to the attacked domain’s MX record at an equal priority, to take load off the main (overloaded) MX.

Now to cogitate a little — details to follow…

Tags: amazon, anti-spam, aws, ec2, qpsmtpd, s3, smtp