I woke up this morning to hear speculation on RTE Radio as to how Eircom’s DDOS woes were possibly being caused by the Russian mob, of all things. This absurd speculation is not helped by lines in statements like this:
‘The company blamed the problems on “an unusual and irregular volume of internet traffic” directed at its website, which affected the systems and servers that provide access to the internet for its customers.’
I’m speculating, too, but it seems a lot more likely to me that this isn’t just a DDOS, and someone — possibly just a lone Irish teenager — is running an attempted DNS cache-poisoning attack. Here’s why.
Last week, there were two features of the attack in reports: DDOS levels of traffic and incorrect pages coming up for some popular websites. To operate a Kaminsky DNS cache-poisoning attack requires buckets of packets — easily perceivable as DDOS levels. This level of traffic would be the first noticeable symptom on Eircom’s network management consoles, so it’d be easy to jump to the conclusion that a simple DDOS attack was the root cause.
This week, there’s just the DDOS levels of traffic. No cache poisoning effects have been reported. This would be consistent with Eircom’s engineers getting the finger out over the weekend, and upgrading the NSes to a non-vulnerable version. ;)
Once the attacker(s) realise this, they’ll probably stop the attack.
It’s not even a good attack for a bad guy to make, by the way. Given the timing, right after major press about a North Korean DDOS on US servers. it’s extremely high-profile, and made the news in several national newspapers (albeit in rather inept fashion). If someone wanted to make money from an attack, a massive-scale packet flood indistinguishable from a DDOS against the nation’s largest ISP is not exactly a subtle way to do it.
In the meantime, apparently OpenDNS have really seen the effects, with mass switchover of Eircom’s customers to the OpenDNS resolvers. Probably just as well…
11 Comments
It’s odd in part because it implies that the servers provided for Eircom customers aren’t firewalled off from externally originating traffic. There seems to me to be no reason for this except inertia.
From my standpoint, the switchover to opendns is actually an unfortunate consequence of this attack. As I’ve mentioned in a comment here before, using opendns is usually a bad idea. They have a habit of trapping NXDOMAIN, they sometimes http proxy traffic to sites and, most importantly, their servers aren’t geographically distributed enough to give optimal endpoints for big services. Since recursive DNS resolvers don’t pass along the originating IP, geo-IP based balancing relies on the recursive resolver’s IP. OpenDNS don’t have servers that are local to Ireland and so you get bad endpoints for akamai, google et al.
To see this, do ‘dig @208.67.222.222 http://www.google.com‘ and compare with the result from any more independent source.
But what other recourse than OpenDNS does a punter have?
I made a screencast on resetting Eircom DNS for my XP friends: http://www.flickr.com/photos/irisheyes/3719985673/
Eircom’s DNS servers were unusable long before this little incident.
“Normal operations” for users involve a 3-5 second pause while the steam engines on port 53 wake themselves.
It’s a little amusing and a little depressing that even blogger types here with 2Mbps DSL lines have apparently never used a connection with a proper DNS resolver before. I can’t think of any other reason they’d put up with the at-least-weekly DNS server outages (no doubt Support told them to reboot their routers) and crappy latency.
@John — yeah, the DNS resolvers have long been the weakest link for consumer home broadband ISPs in my experience. Eircom aren’t alone in this; a lot of ISPs are lousy at it. I had shitty results from Comcast when I was living in the US for example.
Here’s an old report on the topic:
http://blog.opendns.com/2006/08/17/cnet-reports-isps-arent-very-good-at-dns/
e.g., pretty shocking stat — ‘Verizon drops 3.14% of all DSL subscribers’ DNS requests’. oh dear.
Good news for OpenDNS though, I suppose!
The Kaminsky attack takes seconds or less, against servers that didn’t randomise QID. It is against DNS servers that randomise the QID that the attack turns into a DDoS level of packets attack.
Further in blog entry here, as the trackback/pingback doesn’t seem to have made it.
I also thought it was cache poisoning, all attempts to access major sites went to bing searches for those sites
Even against completely random QID, the Kaminsky attack never needed to reach “DoS” levels — think in the range of 300 packets or less on average.
Alan, I thought it was dependent on the DNS server software in use.
one thing that this should have taught the irish registrar domainregistry.ie is that they need to secure what they can so when is .ie going to be signed for dnssec ?
this combined with DKIM means that the world is a little more secure and things like this are less likely to affect ireland…
regards
John Jones http://www.johnjones.me.uk
The Kaminsky attack allows attackers to have essentially infinite attempts to poison an NS for some given domain. Prior to that, it seemed attempts were rate-limited by the TTL in valid answers.
On servers where the Query-ID response-cookie is predictable across queries (e.g. older BIND), the attack takes few packets (the amount needed scales with the busyness of the server). Servers where QID are not predictable across packets take longer, but it’s been shown it can be done within 10 hours. Cite is on my blog entry.
Just received the following from Eircom. Kaminsky discovered his bug when, mid-2008?