Skip to content

Month: May 2007

DSPAM acquired by Sensory Networks

Published May 31, 2007

whoa, didn’t see that coming. Quoting Jonathan Zdziarski via jgc’s newsletter:

…The [DSPAM] project had grown to a point where it would take others – with enough free time – to bring DSPAM to the next level as a widely accepted enterprise-class solution, and [I] decided that it would be in the best interest of the project to entrust it to someone with the technical knowhow and dedication to reach these goals. Many of you are aware of my work in the past with Sensory Networks in developing a hardware-accelerated version of DSPAM (capable of supporting multi-megabit speeds in large carrier environments). I’ve spent a considerable amount of time with SN’s team over the past several years and when we initially discussed working together, they had shown to be very excited and motivated about the project.

After careful consideration and many discussions at length, I decided to allow Sensory Networks to acquire the rights to the project, and continue development on it with their own team. SN has displayed a strong commitment to the open source community and has been working closely with other leading projects such as Snort, Clam Antivirus, and SpamAssassin. They assured me that the project will remain open-source and available to all, and at the same time the project will receive exposure in commercial environments it has not seen before, as many of you have been asking for. We’ve now completed the acquisition for the project, and I’d like to encourage you to support them in helping them move forward as it grows into new areas.

More details at zdziarski.com.

Dealing with backscatter, revisited

Published May 30, 2007

Back in January, I wrote about how I deal with email backscatter nowadays. Since then, I’ve made a notable tweak.

This is that I no longer reject “null-sender” traffic during the SMTP transaction. It turned out that it broke Exim’s implementation of Sender Address Verification, which performs the SAV check using a MAIL FROM of <>, rendering it indistinguishable from a bounce during the SMTP transaction.

Now, I’ve complained about SAV, but I have to be pragmatic anyway (Postel’s law and all that!) — so it was better to just allow other sites to perform SAV lookups against our server, and fix the anti-bounce stuff some other way.

The new method (below) does this, by allowing null-sender SMTP traffic just fine; it detects bounces in Postfix if they arrive via SMTP in RFC-3464 format, and bounces that slip past are then dealt with in a more CPU-intensive manner using the SpamAssassin “VBounce” ruleset (which is part of the now-released SpamAssassin 3.2.0, btw).

This increases the load, since some bounces cannot be rejected at MAIL FROM time now, and instead we have to wait ’til DATA — but CPU hasn’t been a problem recently, so this is ok.

Here are the updated instructions:

In Postfix

In my Postfix configuration, on the machine that acts as MX for my domains — edit ‘/etc/postfix/header_checks’, and add these lines:

/^Content-Type: multipart\/report; report-type=delivery-status\;/  REJECT no third-party DSNs
/^Content-Type: message\/delivery-status; /     REJECT no third-party DSNs

Edit ‘/etc/postfix/main.cf’, and ensure it contains:

header_checks = regexp:/etc/postfix/header_checks

Then run:

sudo /etc/init.d/postfix restart

This catches most of the bounces — RFC-3464-format Delivery-Status-Notification messages from other mail servers.

In SpamAssassin

As before, install the Virus-bounce ruleset and set it up. This will catch challenge-response mails, “out of office” noise, “virus scanner detected blah” crap, and bounce mails generated by really broken groupware MTAs — the stuff that gets past the Postfix front-line.

Dead laptop time

Published May 28, 2007

Argh. My Thinkpad’s power socket must have received a knock during the move. It no longer works with either of the two power bricks I have here — so it looks like it’s time to either (a) buy a soldering iron and some screwdrivers (incl Torx ones?) or (b) renew my IBM warranty service and send it in for some fixing :(

Bad timing.

Update: oh look, it’s working again! phew. I guess I should probably set aside some time for warranty service here anyway though…

Back

Published May 25, 2007

Hey — I’m back, rested and full of tasty, tasty Niçois and Provencal cuisine.

I got back just in time to vote, for what good that did with Bertie’s gang leading strongly in the current counts… argh!

For what it’s worth, I gave Patricia McKenna a preference, in the end. I was reminded that she’d been entirely on our side on software patents during her time as an MEP — so credit where it’s due, there; on top of that, a vote for the Greens is better than a vote going to Sinn Fein, after all, no matter what. ;)

Carbon offsetting

Published May 10, 2007

I’m off to Nice on vacation for two weeks, starting tomorrow — back on May 25th. See ya then!

In the meantime, and appropriately enough given that jet fuel I’ll be consuming, here’s some interesting stuff from my mate Eoin on carbon offsetting…

‘It’s a fecking minefield to figure out. There are many conflicting standards, some of which sound impressive but are useless in reality.

Steer clear of tree planting, especially outside Europe; even a well-run forestry in Europe will take decades to make any difference.

The best quality-mark appears to be the CDM Gold Standard. The Gold Standard is a recent introduction, a response to the weak, conflicting Kyoto standards and many ad hoc government ones. Gold Standard specifically excludes tree plantatations.

The following operators are the only ones I found that are Gold Standarded and also pass the bullshit smell test (which is far more stringent ;-) thanks to all who supplied links etc. — eoin

  • My Climate — Seem good. run out of Switzerland. Professional vibe. Mainly projects in the developing world.
  • Atmosfair — like the swiss one except smaller and German. Again, seems professional, their projects page in particular reads well. Doing a German schools project as well as developing world ones.
  • Climate Friendly — Aussies. Mainly wind power, in Oz & NZ. Again seem good, have been around for a few years. Website is decent if a bit all over the place.
  • Sustainable Travel International — more an eco-holidays travel agent than offsetting per se. Useful bookmark.
  • Puretrust.org.uk — These guys seem good. Interesting business model. They buy high quality carbon credits, from mainly Gold Standard providers, and retire these credits. Permanent retirement, I think, though this wasn’t 100% clear on their site. So they both support the providers directly by doing business with them, and also jack up the market price by reducing supply. This supply choke isn’t something that the rest of them do, at first glance anyway. Clever idea. As the market price gets higher it will put pressure on companies to reduce their emissions, not just buy their way out of it.’

Now it’s worth noting that this is the state of play as of May 2007; it’ll definitely change pretty quickly as time goes on. Good info, though.

Eircom broadband — it’s never easy

Published May 9, 2007

Argh, it’s never easy.

After this post, the consensus was that nowadays, Eircom have a pretty good quality of service for their DSL offerings, taking both price and service into account. I was happy enough to go with that, so I ordered their “Eircom broadband always on 2MB and Eircom talktime anytime bundle”, back around the middle of April.

I had a great call with the sales agent, Hazel. Everything went swimmingly, we were all set for the modem to be delivered and the service to be up and running in 10 working days — by May 1st April 30th. I asked for an order reference number and she said I didn’t need one, it was all handled in their system. Great!

Unfortunately it seems the call centre staff never got that quality-of-service memo.

Come May 1st, there was no sign of the modem, so I rang Eircom’s order line to see how things were going. To my horror, the staff I talked to told me that there was no record of my previous order, or call… it was as if that call had never taken place at all. No part of the order had even started.

As a result, I’ve had to reorder from scratch. The previous 10 working days we’ve waited counts for nothing. (The agents lie through their teeth about this, though — one agent says they’ll send it out in the “next 3-5 days”, the next agent insists that we have to wait the full 10 days, and the next says somewhere in between — anything to get us off the line within 4 minutes.)

This is bad news, since we’re waiting on the broadband to move in — since I work from home, we can’t move in until we have a good ‘net connection.

We can’t even make a complaint to Eircom about this fuckup, because they refuse to take complaints without the original order number to reference — the one that “Hazel” told me wasn’t needed anymore. Now that’s bureaucracy. Attempts at escalation just wound up with a dead end, where supervisors had no names and had left the office at 10am anyway. >:(

Best of all, their online complaints system now takes a maximum message length of 400 characters, so you can’t even provide a detailed written complaint online anymore. (That is, not unless you submit the complaint in 15 separate parts…)

What a fiasco.

So we now have to wait until May the 15th. We’ve submitted the complaint via the aforementioned 15 parts, and postally; if they don’t take action on those, we’ll complain to Comreg (and let’s see what that’s worth).

But here’s a question — assuming they fail to deliver the second order within time this time around, can we cancel at that stage? There’s a minimum contract length of 6 months, but since the service hasn’t been delivered, I would hope that hasn’t started yet. The terms and conditions document says:

“Ready for Service date” (otherwise “RFS date”) means the date on which eircom establishes the Facility for the Customer.

3.1 This Agreement shall commence on the Ready for Service date and shall be for the Initial Period. Provided that this Agreement has not been terminated in accordance with its terms or in accordance with the Regulations, this Agreement shall thereafter automatically renew for successive six-month periods. For the purposes of this clause 3, a six-month period will be calculated from the anniversary of the RFS date.

3.2 The Customer may cancel its order for the Facility at any time prior to the RFS date. In the event of such cancellation by the Customer it shall be obliged to return any Kit, which may have been provided to it by eircom. Any Kit shall be returned to eircom by posting it to the freepost address detailed in the welcome pack. In the event of any Kit not being returned to eircom within fourteen (14) days of the cancellation of the Order for the Facility, the Customer shall be charged by eircom and shall pay to eircom such sum as is set out in the Regulations as being the charge payable in respect of the non-return of any Kit.

So I guess as long as the facility — the ADSL line — is not up and running, I’m clear to cancel, right? It’s a little worrying that the “facility” doesn’t include the “kit” — ie. the broadband modem, though; if they fuck up sending out the modem, but the line is up, am I liable for 200 Euros?

In terms of who are viable options to switch to — in my opinion it’s got to be fixed wireless, since everyone else now would have to go via Eircom’s exchanges anyway, and be delayed there. So — Irish Broadband. I know they had some pretty massive problems 2 or 3 years ago, but recently I’ve been hearing good things about them, Boards.ie has some reasonably good-sounding recent experiences, and half of my new neighbours (srsly!) are using them with great results. Anyone got recent news about how useful they are with service quality and install speed for their Breeze product in the D9/D11 area?

Alternatively, Ripwave might make a reasonable stop-gap option? 120 euros is the minimum fee (6 months at 18.95 per month), which is better than the money I’m paying now to live in two houses…

Alternatively anyone know an Eircom engineer in D9/D11 that can nip over to the exchange and plug in my connection on the DSLAM? ;)

Moin Moin attachment spam

Published May 8, 2007

Here’s a new trick used by the web spammers — attachments on a Moin Moin wiki. The taint.org/wk RecentChanges list illustrates it well:

2007-05-07  set bookmark
[UPDATED]       UserPreferences         04:17   Info    ?StepStep [1-21]        
  #01 Upload of attachment 'big-cocks.html'.
  #02 Upload of attachment 'big-cock.html'.
  #03 Upload of attachment 'big-boobs.html'.
  #04 Upload of attachment 'big-ass.html'.
  #05 Upload of attachment 'bdsm.html'.
  #06 Upload of attachment 'bbw.html'.
  #07 Upload of attachment 'bang-bros.html'.
  #08 Upload of attachment 'bangbros.html'.
  #09 Upload of attachment 'baby.html'.
  #10 Upload of attachment 'asian-porn.html'.
  #11 Upload of attachment 'asian-girls.html'.
  #12 Upload of attachment 'anime-porn.html'.
  #13 Upload of attachment 'anime-girls.html'.
  #14 Upload of attachment 'angelina-jolie.html '.
  #15 Upload of attachment 'amature.html'.
  #16 Upload of attachment 'amatuer.html'.
  #17 Upload of attachment 'adult-videos.html'.
  #18 Upload of attachment 'adult-stories.html' .
  #19 Upload of attachment 'adult-games.html'.
  #20 Upload of attachment '69.html'.
  #21 Upload of attachment '3d.html'.

Great. Lots of spam. This first started appearing on Feb 27 2007, in a multi-upload attack on a single page (“FindPage”), from IP address 212.26.129.162; then reoccurred on Apr 27 and May 7 from the (insecure open proxy) proxy.drevlanka.ru.

Annoyingly my “subscribe to wiki changes” patch doesn’t catch this — these aren’t gatewayed through as “changes” via mail for review. I need to fix that in my copious free time. :(

Also, the RecentChanges RSS feed doesn’t list them, although the HTML form does.

So unfortunately, the only way I can see to block this is either to review by visiting the RecentChanges page in a web browser regularly (how retro!), and delete them retrospectively, or simply to turn off attachments entirely — which is what I’ve done, by editing “wikiconfig.py” and adding:

    actions_excluded = ['AttachFile']

It looks like quite a few other wikis around the web are running into the issue too :(

SpamAssassin 3.2.0!

Published May 2, 2007

W00t! SpamAssassin 3.2.0 has finally gone gold!

This release is a big one — it’s the first major release since 3.1.0, back in September 2005, just over a year and a half ago. Here is the release announcement mail, containing a list of major changes since version 3.1.8. There are a few major new features that I feel worth picking out in more detail and editorialising about:

sa-compile

This is a biggie. This new script takes the active SpamAssassin ruleset, and uses code contributed by Matt Sergeant to produce input for re2c. re2c in turn compiles the ruleset into a deterministic finite automaton, which can match multiple regular expressions in parallel. That’s not all, though; re2c then compiles that DFA into C code — which is then compiled into native object code. SpamAssassin will then load that object code and use it to replace the slower perl regexp tests, if it’s available at scan-time.

Now, it’s been a long time since SpamAssassin’s ruleset consisted mainly of rudimentary regular expressions matched against the body text — a good portion of SpamAssassin’s ruleset these days operates against headers, performs network lookups, analyzes URLs extracted from the body, uses the more advanced features supported by Perl’s NFA regexp engine, or so on. But even given that, the effects of ‘sa-compile’ seem to average between a 15% and 25% speedup, in my testing. That’s good ;)

Many of the commercial versions of SpamAssassin include their own body-rule speedups — but this is the first time anything similar has made it into the open source code.

Short-circuiting

Another good one for performance. There are some rules that you can reasonably assume will never hit nonspam or spam mail in a well-configured setup. For example, a hit on “ALL_TRUSTED” should mean that the message never traversed an untrusted network, therefore it cannot be spam, so why bother applying the expensive tests? It should be reasonable to “short-circuit” and immediately return a “ham” score for that mail.

This new plugin implements that algorithm — and efficiently, too, which historically has been the hard part!

I’ve been using this for a while with a ruleset like this one — in my experience, it’s cut overall CPU time spent scanning mail by 20%.

It is pretty flexible, too — there’s lot of tweakage that can be done with this functionality to suit your own setup.

Reduced memory footprint

One aim of this release has been to reduce the memory usage of SpamAssassin; the core code now uses less RAM than 3.1.x does, when tested with the same ruleset. (Unfortunately we’ve added lots more rules in the interim, so it’s a bit of a wash overall. ;)

The VBounce anti-bounce ruleset

Detects spurious bounce messages sent by broken mail systems in response to spam or viruses. More info about that here.

Apache-spamd

apache-spamd implements spamd as a mod_perl module. This was contributed by Radoslaw Zielinski, as a Google Summer of Code project last year. Thanks Radoslaw!

There are plenty more new, useful features and rules — these are just the top ones, in my opinion. Pretty cool stuff!

Patricia McKenna and MMR, again

Published May 1, 2007

Great! Patricia McKenna just called around, canvassing our area — and just got a serious telling off from the wife ;)

Catherine — unsurprisingly, given that she’s a zoology Ph.D — was fantastic, hitting every key point of the issue: that we’re both long-time Green voters who’ve been forced to not vote Green this time around, due to this MMR issue and the anti-science/pro-hokum angle it represents.

Interestingly, she claimed that her stance on MMR was always her own point of view, and that it wasn’t party policy — and that it was mentioned on the party website was a rumour put about by the PDs.

While it turns out that Dr. Ruairi Hanley, the author of this letter to the Indo is indeed a PD (didn’t realise that!), Treasa at Winds and Breezes also noted it appearing on the Green Party site, as follows:

Questioning the Benefits of Immunisation

There are significant question marks about the effectiveness of mass immunisation programs. We would launch a major study of the benefits of these programs looking at all aspects of health

So Treasa — are you a stealth PD rumour-monger? ;)

Worth noting that at no time did McKenna reassure C that her policy would not become government policy if the Greens were elected… as an elected representative, surely her own policies would influence the government’s thinking?

Screenclick devolve again

Published May 1, 2007

After a short period where things were looking up, Screenclick have once again reverted to type, by ditching the lovely simple Netflix-style queue they seemed to be using, and instead instituting some new kind of bizarre homebrew wierdness.

It looks like a queue, with a line-by-line listing of movies — but then beside each title, there are 3 radio buttons: “High”, “Medium”, and “Low”.

The instructions run as follows:

All titles are sorted in alphabetical order within their priority group
  • – High: Please deliver these titles as soon as possible
  • – Medium: Please deliver these titles as they become available
  • – Low: I don’t mind when you send these titles

So what — does this mean that if I put a title in as “High”, I’m going to receive it next, or not, or what? and what’s with the alphabetical order? WTF is going on? argh.

Anyway, I just got out “Amores Perros”, presumably due to this alphabetical ordering thing. not what I wanted at all. What a mess.

A week of Bertiespam

Published May 1, 2007

We’re in the run-up to a general election here in Ireland, and I live in Bertie’s constituency. For the past year or so, things have been pretty quiet, but in the last week there’s been a sudden flurry of activity and direct postal mail from Bertie’s office — and from many departments of local government, too:

Mon Apr 23:

  • Fianna Fail: “Fianna Fail delivers on education in Dublin Central”, tabloid newspaper.

  • direct from the office of Bertie: a photocopied letter from the Environmental Health Officers of Dublin City Council about the standards of rented houses “in my area”.

Tues Apr 24:

  • HSE: “Parents Who Listen, Protect” leaflet, a full-colour glossy handbook “on building good communication in families and communities” “as part of a national initiative on child protection”.

  • Dept of Environment: a leaflet on the “National Climate Change Strategy, 2007-2012, Main Points”. Printed on recycled paper, naturally ;)

Fri Apr 27:

  • Fianna Fail Senator Cyprian Brady: “dear resident, please vote for me” — one-page full-colour glossy.

  • Spring 2007 “Central News”, “Official Voice of Fianna Fail in Dublin Central”, a 16-page tabloid newspaper, featuring stories like “Smithfield: the Temple Bar of the Northside” (like Temple Bar, but with more winos and Children’s Court, and less stuff!)

Mon Apr 30:

  • HSE: “Need a doctor urgently? Call D-DOC out-of-hours GP service”, full-colour glossy leaflet.

  • from Bertie: Evening of Election Letter. “Good evening constituents” etc.

It’s a veritable flood of full-colour glossies! Could be worse, I suppose — I hear the PDs have been blanketing selected Dublin constituencies in free books. However I suspect grimy Dublin 7 is a little off their list (see “winos”, above).

It’s worth noting that a good half of this flood (which I’ve coined Bertiespam to describe) isn’t from Bertie’s constituency office — it’s from government departments like the HSE and the Department of Environment. It’s funny that we hadn’t heard a peep from them all year, then once an election looms — “here come the voters! look busy!” ;)

What bertiespam have you been getting?