taint.org: Justin Mason’s Weblog

According to this Wired story, Google reckons spammers are giving up on spam:

a remarkable trend is underfoot, according to Brad Taylor, a staff software engineer at Google: The number of spam attempts — that is, the number of junk messages sent out by spammers — is flat, and may even be declining for the first time in years.

Actually, this is a wilful misunderstanding of what the Googler in question really said, which was that ‘attempts to spam Gmail users have been leveling off over the last year and more recently, even declining slightly’. In other words, they didn’t make an observation about the state of the spam problem on an internet-wide basis — just about the “local” situation as it pertains to Gmail. Bad reporting there, Wired.

But, in passing…

David Berlind at ZDNet recently blogged a rather grumpy response to InfoWorld coverage of CEAS 2007. He raised a very important point:

If I could say something to the author of that story, it would be that so long as any anti-spam solution is not deployed universally throughout the Internet’s e-mail system (in other words, so long as some anti-spam tech is not a standard), that anti-spam solution actually makes the spam problem worse. You read that right. Worse. Proprietary anti-spam solutions make the global spam problem worse. They are digging us deeper into the hole that the Internet is already in because everyone who makes those solutions is under the false belief that “s/he who is finally successful at filtering out all spam while allowing the legitimate mail in wins.”

Google’s blog post is a case in point: ‘we’re keeping more spam out of your inbox than ever before, so more and more, you can use Gmail for things you enjoy without even realizing that the spam filter is there most of the time.’

That’s great — but it doesn’t help anyone except Gmail. It’s a myopic view of the spam problem, and David’s point stands.

(I disagree with his later conclusion that the only way forward is for Google, MS, AOL and Yahoo! to get together and ‘commit to jointly supporting the same technical solutions’ — when the usual BigCos get together, they tend to focus on their own priorities. Take what happened back in 2005 with nofollow for blog-spam — while it helped the search giants with their own overriding priority, which was to tweak their algorithms to filter out the spam on the search results page, it did nothing to slow the spam flood itself, which has continued unabated.)

We need more open-source, and open-data, anti-spam work.

Tags: anti-spam, aol, ceas, david-berlind, filtering, google, microsoft, nofollow, spam, wired, yahoo

Things have really been heating up recently around the AOL/Goodmail “pay to send” CertifiedMail scheme — the EFF and a host of other groups have launched dearaol.com, stating:

This system would create a two-tiered Internet in which affluent mass emailers could pay AOL a fee that amounts to an “email tax” for every email sent, in return for a guarantee that such messages would bypass spam filters and go directly to AOL members’ inboxes. Those who did not pay the “email tax” would increasingly be left behind with unreliable service. Your customers expect that your first obligation is to deliver all of their wanted mail, and this plan is a step away from that obligation.

While I dislike this proposal, too, as far as I can tell, AOL actually have pretty reasonable intentions with this program — nowhere near as bad as the DearAOL.com site makes out.

However, they’re doing a really really crappy job of getting this information out there, or committing to reasonable limits on the program, such as announcing that they will use it only for transactional emails, as Yahoo! have done.

I’d strongly recommend reading Carl Hutzler’s posting on the subject. Carl was AOL’s head of anti-spam operations until last year, so he really knows what he’s talking about, and he lays it out clearly — a lot more clearly than any corporate statements from AOL do. His blog contains a fair bit more on the subject, too.

But seriously — why isn’t there a press release on the AOL site about this scheme? Some front-channel communication about now might be useful, I’d suggest, before things really get hairy — this crapstorm is coming about partly because AOL’s comments are all filtering out in drips and drabs via third parties, and (AOLers say) are being misconstrued and misrepresented in the process. It’s a classic case of missing the cluetrain.

I’d also really encourage the EFF people to tone done the rhetoric; statements like “senders will have no guarantee that their emails will be delivered” is scare-mongering, given that SMTP email already provides no such guarantee.

Update: wow, MoveOn went really overboard — “threatening the Internet as we know it … The very existence of online civic participation and the free Internet as we know it are under attack.” OMG the sky is falling!

Side Issue: The Spam Definition

Also, another note to EFF: defining spam as “whatever you don’t want to read” is a terrible mistake to make. That confuses a good, clear, enforceable and automatable definition of spam – unsolicited bulk email – and makes it effectively unenforceable by law, unpoliceable by ISPs, impossible to detect automatically, and incompatible with existing, effective EU and Australian legislation.

Listen to your own Chairman of the Board; he’s right on this count.

PS: any luck fixing up the non-confirmed signups issue? Last time I checked I could still subscribe any address to the EFF Action Alerts without a cross-check, which is not a good thing.

Tags: anti-spam, aol, defining-spam, eff, goodmail, pay-to-send, spam, ube

AOL and Yahoo! have been making a lot of headlines with their plans to reduce their whitelist-management workload — and make a little pay-to-send money on the side — with a deal with Goodmail.

Now Spamhaus have gone on the record against the plan:

On Monday, Richard Cox, chief information officer at antispam organization Spamhaus, said that “an e-mail charge will destroy the spirit of the Internet.”

“The Internet has become what it is because of freedom of communication. Open discussion is what gives it value. There should be no cost for particular services, and e-mail should be free and accessible to all. This will disenfranchise people.”

Tags: anti-spam, aol, goodmail, pay-to-send, spam, spamhaus, whitelisting

Spam: I’m quoted in
New Scientist! w00t!

SlashDot picked it up pretty quickly. One comment there misses the point, though:

This is interesting and promising technology. But like all antispam techniques, spammers will find a way around it. Once spammers get a copy of the software, they can create and test countermeasures in the comfort of their own sleazy lairs.

It’s worth talking about this. Newsflash: spammers have no difficulty testing their spam against closed-source spam filters, even when they can’t ‘get a copy’ and test them in ‘their sleazy lairs’.

How do they do it? Easy — just set up an account at a site that uses that filter (AOL, Yahoo!, Hotmail, and GMail, it’s pretty obvious how to do that; for other closed-source filters, find an ISP that uses it). Then send ‘test mails’ repeatedly to that account, and apply trial and error to see what gets past the filter and what doesn’t. Eventually, they figure out what works for that filter, and what doesn’t.

How did I figure this out? Well, I came across the manual for the Send-Safe ratware on-line. It noted that the ‘hashbuster’ randomisation technique, which we in the SpamAssassin team had long assumed was intended to block hash matches by DCC, Pyzor and Razor, was in fact intended to block AOL’s implementation of that system. The open source ones weren’t even mentioned.

Update: found it — from their FAQ:

Mime Encoded content

If you want to get into AOL… use it.

MIME encoders allow you to send documents written within a specific application through email without causing readability or formatting problems. For example, you can send a letter created in MSWord with and be certain that it arrives at its destination in the same format by encoding it with MIME first. The recipient then decodes it back into the original MSWord format.

That isn’t why we use it though.

We use it to cause ‘uniqueness’.

When you put a rotate tag at the beginning of a MIME encoded email, it causes everything after that point (including checksums) to be ‘different’ in every message.

Why is that that important?

Because it throws off filters that look for many copies of the same message to nuke.

Tags: account, aol, copy, email, filter, format, mime, msword, point, spam

Given that something like 8.13% of of the hosts that have sent non-spam mail to me do not have reverse DNS information recorded, the fact that AOL have just switched this on as a requirement will be interesting:

: jm ftp 1019...; dig aol.com mx
aol.com.                3559    IN      MX      15 mailin-01.mx.aol.com.
mailin-01.mx.aol.com.   92      IN      A       152.163.224.26
...
: jm ftp 1020...; telnet 152.163.224.26 25
Trying 152.163.224.26...
Connected to 152.163.224.26.
Escape character is '^]'.
220-rly-za01.mx.aol.com ESMTP mail_relay_in-za1.6; Thu, 22 May 2003
15:09:54 -0400
220-America Online (AOL) and its affiliated companies do not
220-     authorize the use of its proprietary computers and computer
220-     networks to accept, transmit, or distribute unsolicited bulk
220-     e-mail sent from the internet.  Effective immediately:  AOL 
220-     may no longer accept connections from IP addresses which 
220      have no reverse-DNS (PTR record) assigned.
^]
telnet> q
Connection closed.

Tags: aol, com, dns, fact, ftp, information, mail, requirement, something, telnet

Lycos: AOL reports to Members on Its Efforts to Fight Spam. ‘Members Now Reporting 4.1 Million Junk E-Mails Daily To AOL’ …. ‘AOL announced that its proprietary anti-spam filtering technology is blocking up to 780 million pieces of junk mail every day from reaching member e-mail inboxes, which amounts to an average of 22 blocked spam e-mails per account daily.’

Of course, they don’t say how much mail overall arrives at AOL, but I’d hazard a guess it’s not much over 1,300 million messages per day based on those figures.

Tags: aol, daily, day, junk, lycos, mail, million, reporting, spam, technology

AOL patents instant messaging (/.). ‘Specifically, any technology that provides ‘a network that allows multiple users to see when other users are present and then to communicate with them’ is covered.’

The CNet story which /. references points out that the patent was filed in 1997 — but that’s still 6 years after I wrote a similar perl script on the Maths Department UNIX machines in TCD. There’s a myriad of similar apps, of the same vintage, too.

The thing I find amazing is this, however — the AOL patent actually cites prior art in its References section, namely the xhtalk README file, dated 1992. There’s nothing different between xhtalk and AOL Instant Messenger apart from the protocol and the look and feel, and those aren’t key to the patent.

The US patent office really needs to start reading the patent applications before granting them.

Tags: aol, cnet, maths, network, patent, perl, script, story, technology, xhtalk

Things are getting crazy in the fight against spam: it seems AOL blocked access (for two weeks) to its mailserver from Telia.com, one of Sweden’s biggest ISPs (if not the biggest), due to spam.

Attached is an unauthorized translation of an article in the Swedish IDG paper Computer Sweden (web edition, Oct 24), provided by Claes Tullbrink.

Until a (previous) article was published, noting this ban, AOL had not succeeded in contacting Telia to talk about it. Amazing stuff.

Date: Thu, 24 Oct 2002 14:51:19 +0200
From: Claes Tullbrink (spam-protected)
Subject: Telia.com not blocked by AOL any longer

Computer Sweden (in Swedish, password may be required after today):

http://computersweden.idg.se/ArticlePages/200210/24/20021024131806_CS539/20021024131806_CS539.dbp.asp

Oct 24, pm.

For more then two weeks mail from Telia.com was blocked by AOL.

Jocelyn Cole, AOL UK, confirmed the block, which was due to big amounts of spam sent from Telia domains to AOL. The block is now removed, and AOL is cooperating with Telia to find a long term solution to decrease the amount of spam sent from Telia, to protect AOL customers.

Press officer Jan Sjöberg, Telia, says it was the article that solved the issue: a Telia contact person name was mentioned in the article, and it seems
that AOL had read the articles [and *so* and in no other way knew who they could contact? CT]

Jan Sjöberg is still not sure how the block was related to spam: due to spam, reports of spam or a customer’s open mail relay. Telia will investigate. [proxies was not mentioned. I don’t know if “reports of spam” relates to refusing to accept plain mail reports sent to (spam-protected)

Claes

Tags: aol, article, block, claes, com, computer, oct, spam, telia, tullbrink

C|Net reports:

Two weeks ago, six top financial institutions met privately with AOL Time Warner, Microsoft, IBM and other leading corporate instant messaging providers and urged them to build communications networks that interoperate. …. The meeting, which took place at Merrill Lynch’s New York offices, was among the first convened by the Instant Messaging Standards Board (IMSB), a newly created consortium led by financial services firms Lehman Brothers, J.P. Morgan Chase, Merrill Lynch, Morgan Stanley Dean Witter, UBS and Deutsche Bank.

Holy shit, that’s a lot of gorillas! (via Doc).

Tags: aol, ibm, interoperate, lynch, meeting, merrill, microsoft, morgan, time, warner