taint.org: Justin Mason’s Weblog

Bluetooth As a Laptop Sensor in Cambridge, England.

I link-blogged this yesterday, where it got picked up by Waxy, and thence to Boing Boing — where some readers are reportedly considering it doubtful. Craig also expressed some skepticism. However, I think it’s for real.

Check out the comments section of Schneier’s post — there’s a few notable points:

• Some Bluetooth-equipped laptops will indeed wake from suspend to respond to BT signals.

• Davi Ottenheimer reports that the current Bluetooth spec offers “always-on discoverability” as a feature. (Obviously the protocol designers let usability triumph over security on that count.)

• Many cellphones are equipped with Bluetooth, and can therefore be used to detect other ‘discoverable’ BT devices in range.

• Walking around a UK hotel car park, while pressing buttons on a mobile phone, would be likely to appear innocuous — I know I’ve done it myself on several occasions. ;)

Finally — this isn’t the first time the problem has been noted. The same problem was reported at Disney World, in the US:

Here’s the interesting part: every break-in in the past month (in the Disney parking lots) had involved a laptop with internal bluetooth. Apparently if you just suspend the laptop the bluetooth device will still acknowledge certain requests, allowing the thief to target only cars containing these laptops.

Mind you, perhaps this is a ‘chinese whispers’ case of the Disney World thefts being amplified. Perhaps it was noted as happening in Disney World, reported in an ‘emerging threats’ forum where the Cambridgeshire cop heard it, and he then picked it up as something worth warning the public about, without knowing for sure that it was happening locally.

Update: aha. An observant commenter on Bruce Schneier’s post has hit on a possibly good reason why laptops implement wake-on-Bluetooth:

On my PowerBook, the default Bluetooth settings were “Discoverable” and “Wake-on-Bluetooth” — the latter so that a Bluetooth keyboard or mouse can wake the computer up after it has gone to sleep.

Tags: bluetooth, boing, disney, laptop, problem, sensor, thence, waxy, world, yesterday

Perl: double-encoding is a frequent problem when dealing with UTF-8 text, where a UTF-8 string is treated as (typically) ISO Latin-1, and is re-encoded.

utf8lint is a quick hack script which uses perl’s Encode module to detect this. Feed it your data on STDIN, and it’ll flag lines that contain text which may be doubly-encoded UTF-8, in a lintish way.

Tags: encode, iso, latin, lint, perl, problem, script, string, text, utf

Web: Almost every project and organisation has, at some stage, bemoaned having stale data on their website, and wished there was a better way to keep it up to date; or wished their FAQ was more complete; or wished they had the time to HTML-ize all their know-how and get it up there.

Well, here’s what we did in SpamAssassin to deal with this problem. (Seeing as I’ve talked about this three times in the past month, I’ll write it up here so I can just point at the URL next time!)

First off, we experimented with having the site checked into CVS, FAQ-o-matic, and the Python FAQ software (which was pretty good). All were OK, but very specific in format, using the traditional question-answer FAQ layout — that’s good for FAQs, but not so good for a lot of other stuff — and keeping it updated was still limited to a small group, therefore the info got stale again.

So we moved to a Wiki. Here’s my tips for Wiki-izing your website so that the end results are better than what went in.

Use good wiki software: unusable software will be a pain to use, and the info will still go stale. We used Moin Moin - http://moin.sourceforge.net/ - partly because I like Python (it’s nearly perl! ;), it can produce RSS, and it was pretty easy to install.

Don’t worry: people won’t vandalise it (much). It turns out that vandalism and people throwing up crappy info isn’t a serious problem at all. You should increase the barrier, in the following ways:

Require user accounts: set the security policy so that a user account must be set up before editing is possible. This means you won’t get wiki-spammed, and also has the side effect of imposing a pretty big barrier to casual vandals.

Send changes to a list: set all changes to be mailed to a mailing list as diffs. This is the most important tip. If you already have a mailing list with the knowledgeable part of the community on it, use that list — because they’re the ones who’ll be able to recognise if erroneous info is put up, and will be annoyed about this enough to bother fixing it. There’s a bonus side-effect of this; even if some people didn’t like the wiki to start with, they’ll eventually be needled into using it by wanting to fix stuff they perceive as wrong. And then they get sucked in ;)

Use diff for the mailed changes: Moin by default will only send out change messages saying ’something changed on this page!’. That’s not good enough, unfortunately — you want to mail out what the new text looks like, and highlight exactly where the change happened. Moin can do this nicely, with this patch, which adds a mail_commits_address, where all diffs on every page are sent, using the normal diff mechanism.

Ensure the wiki software can revert quickly: If someone does make a bad change, Moin supports one-click reversion of the page to what it was beforehand. That’s great for dealing with spam, or clueless vandalism.

Keep one or two static pages: If you’re worried about some script kiddie thinking that defacing a wiki makes them look cool, then keep one or two of the primary user-facing pages as static data. For example, take a look at the link-bar at the top of http://spamassassin.apache.org/ ; five of the ten links are to static pages, the other five are now wiki-ized. In particular, our front page and our downloads page are both static, but our docs are predominantly Wiki’d.

Publicize Mozex: most techie groups will have techie users, and we hate using browser text-boxes to edit text. Mozex — http://mozex.mozdev.org/ — saves the day here — it’s a godsend.

Shepherd new changes: in the early stages, you want one or two people who tidy up changes from Wiki newbies, as they go in. They need to keep it looking pretty, and perform Refactoring of stuff that could be laid out better or should become multiple pages. Eventually, others will get the hang of that (and do a much better job than you do ;).

That’s the lot. Most of these are to, essentially, migrate aspects of your already-existing and already-working community into this new outlet. In our experience, it’s worked really well — our Wiki is now the most reliable source of info about SpamAssassin, and is extensive and up-to-date.

Tags: faq, info, list, problem, python, software, time, use, web, website, wiki

Ireland: Here’s a hot UL that’s floating around the irish web right now —

In a British program about Samuel L Jackson and Colin Farrell’s lastest movie SWAT presented by British presenter, Kate Thornton, the following exchange occured:

• Thornton: What was it like working with Colin (Farrell), cos he
  • is just so hot in the U.K. right now?
• Jackson: He’s pretty hot in the U.S. too.
• Thornton: Yeah, but he is one of our own.
• Jackson: Isn’t he from Ireland?
• Thornton: Yeah, but we can claim him cos Ireland is beside us.
• Jackson: You see that’s your problem right there. You British keep claiming people that don’t belong to you. We had that problem here in America too, it was called slavery.

… yeah, right. ;)

(Update: Actually, believe it or not, that’s more or less how it really went. Here’s the transcript.)

Some commentary at
TheReggaeBoyz.com (quote: ‘I NEARLY DEAD TO RASS!!!!’) and Kuro5hin.

It looks like the TV programme does exist; no scripts online, unfortunately, so we’ll never figure out if this one really happened, I think.

IMO, it’s made up for sure. That last line is just a little too harsh for a primetime schmooze-a-gram, at the very least. Plus, it’s the kind of thing only an Irishman would give a shit about — the perpetual adoption of Irish celebs and worthies by the UK media is a continual source of irritation for the Irish — as Dervala puts it:

‘No, Oscar Wilde was ours. You put him in jail, though. And Shaw was ours. And Yeats. And Johnny Rotten.’

Tags: exchange, ireland, irish, movie, presenter, problem, program, right, slavery, swat, web

Patents: There’s a good discussion over at Joi Ito’s weblog on software patents.

Unfortunately, there’s a persistent, and popular, fallacy that crops up quite frequently in these discussions, and does so here in the comments:

‘much of the processing of patents has been, to use understatement, deficient. An invention that is ’silly or obvious’ will likely not pass the approrpiate legal test - if this test is applied by people who understand the inventive technology …. while I agree with most of your observations about deficiencies, I fail to see the logic in your solution (to simply outlaw these kinds of inventions).’

So, what the commenter is saying is that the patenting of software and business methods would be acceptable, if only the ‘inventive bar’ was raised so that trivial patents were not granted.

The problem with this is that:

• it ignores the fundamental problem with these kinds of patents, which is

  • that they patent ideas instead of physical inventions.

    A parallel would be to allow the patenting of plot-lines in fiction, meter in poetry, or combinations of ingredients and cooking methods in recipes. These are all ideas, transformed into output ‘products’ by performing them as input on a set of hardware (books, cooking equipment), in the same way as software patents and business method patents are abstract ideas that operate on input, generating output, when implemented on a CPU. So, should they be patentable, too?

    Patenting of physical designs is fundamentally different from patenting of abstract ideas in one key way. Physical designs must function correctly under real-world physics, and this requires extensive up-front design and prototyping, before they can be turned into mass-produced products.

    Abstract ideas can be developed mentally, and the up-front work required before the idea can be put down on paper is trivial by comparison.

    Consider these EPO patents: EP0807891 (Sun’s ’shopping cart’ patent) or EP0689133 (Adobe’s ‘tabbed palette window’ patent). The up-front work required to devise these applications is trivial to anyone with a rudimentary knowledge of UI design; the hard part appears to be writing the legalese, and I understand the patent lawyers take care of that part. ;)

    Compare with US patent D0450164, a design patent for a Dyson washing machine. The level of detail, and extensive specifications, is massive, and it’s clear a lot of work had gone into the process before the patent application was filed.

  • In addition, the commenter assumes that extensive prior art searches really do take place. From what I’ve heard from patent applicants, and from what I’ve observed in the range of granted software patents, this is cursory at best, and generally performed by the patent lawyer and the examiner, not the applicant themselves.

    I’ve even observed a few patents where prior art, cited in the patent, implemented exactly what was claimed!

Tags: business, cooking, discussion, input, output, patent, patents, problem, software, test, way

Politics: Bruce Sterling’s speech at SXSW ‘04. It’s excellent. He covers climate change, globalization, the Bush administration’s Lysenkoism, the spam problem, WMDs, and the Spanish election. Now I want to move to Austin ;)

Tags: administration, bush, change, climate, globalization, lysenkoism, politics, problem, spam, speech, sxsw

Security: Educated Guesswork forwards a great illustration of real-world security-measure subversion.

Public places with relatively unattended and un-secured toilet facilities, like train stations, have historically had a problem with intravenous drug users using the cubicles to inject. So about 10 years ago, some bright spark came up with the idea of lighting these places with ultraviolet lights, under which the blue blood in someone’s veins cannot be seen.

Apparently, this works — or at least worked until recently, when the IV drug users figured out an ingenious circumvention technique – highlight your veins beforehand using a UV marker. In normal lighting, the ink is invisible — but once in the UV-lit area, it shows up, apparently better than the veins show up under normal lighting anyway!

As EKR says: ‘remember, folks, your opponent will change his behavior to oppose you. That’s why he’s called your opponent.’

Health: An oldie from 1998. City Limits: 7 1/2 Days. An undercover investigative reporter gets incarcerated as a mental patient in Brooklyn — for a lot longer than he planned. Horrific.

Life: yesterday, I saw Mohammed Ali in the flesh. I was totally star-struck.

Tags: drug, educated, guesswork, illustration, lighting, opponent, problem, security, subversion, toilet, train

Here’s a hot UL that’s floating around the irish web right now —

In a British program about Samuel L Jackson and Colin Farrell’s lastest movie SWAT presented by British presenter, Kate Thornton, the following exchange occured:

Thornton: What was it like working with Colin (Farrell), cos he is just so hot in the U.K. right now?

Jackson: He’s pretty hot in the U.S. too.

Thornton: Yeah, but he is one of our own.

Jackson: Isn’t he from Ireland?

Thornton: Yeah, but we can claim him cos Ireland is beside us.

Jackson: You see that’s your problem right there. You British keep claiming people that don’t belong to you. We had that problem here in America too, it was called slavery.

… yeah, right. ;)

(Update: Actually, believe it or not, that’s more or less how it really went. Here’s the transcript.)

Some commentary at
TheReggaeBoyz.com (quote: ‘I NEARLY DEAD TO RASS!!!!’) and Kuro5hin.

It looks like the TV programme does exist; no scripts online, unfortunately, so we’ll never figure out if this one really happened, I think.

IMO, it’s made up for sure. That last line is just a little too harsh for a primetime schmooze-a-gram, at the very least. Plus, it’s the kind of thing only an Irishman would give a shit about — the perpetual adoption of Irish celebs and worthies by the UK media is a continual source of irritation for the Irish — as Dervala puts it:

‘No, Oscar Wilde was ours. You put him in jail, though. And Shaw was ours. And Yeats. And Johnny Rotten.’

Tags: exchange, irish, movie, presenter, problem, program, right, slavery, swat, web

Tech: Old-timers may remember the Commodore PET’s halt-and-catch-fire POKE instruction.

Turns out modern LCDs may have a similar problem that can cause permanent damage from software activity, according to this posting to Red Hat’s fedora-list.

Update: Yoz mails, noting another case: the perils of reviewing games. Ouch.

Tags: activity, damage, hat, instruction, lcds, old-timers, poke, problem, red, software, tech

Old-timers may remember the Commodore PET’s halt-and-catch-fire POKE instruction.

Turns out modern LCDs may have a similar problem that can cause permanent damage from software activity, according to this posting to Red Hat’s fedora-list.

Update: Yoz mails, noting another case: the perils of reviewing games. Ouch.

Tags: activity, damage, hat, instruction, lcds, old-timers, poke, problem, red, software

Hmm. The Irish branch of the BSA claims that the percentage of illegal software used in Ireland stood at 42 percent for 2002.

Does that mean 42 percent of all software running in Ireland is a pirate copy? I wonder if anyone ever audits this figures — qui custodiet etc. The article continues:

‘The lack of improvement on Ireland’s piracy rate can be attributed to the proliferation of Internet piracy … and a certain amount of apathy,’ said Julian McMenamin, BSA Ireland chairman. ‘But whatever the excuse, a piracy rate of 42 percent is appalling.’

Is internet piracy really that widespread in the workplace? I can imagine your typical 15-year-old firing up KaZaa or whatever, but is this really likely for your typical Irish IT staffer? I doubt it, to be honest.

And given Ireland’s broadband woes, it’d probably be cheaper to hop in the car, drive to the local Compustore, and buy it over the counter, just to avoid paying those ISDN charges. ;)

Just to contradict the ‘internet piracy’ statement, in this story, McMenamin then states that ‘a particular problem in Ireland was small and medium-sized companies purchasing a licence that is too small, or not upgrading their licence as their companies grow. An offending company might typically have a 10-user licence but allow 50 people to use the software.’

This is a hell of a lot more likely than the ’scary internet’ bugbear, but I would still find it very hard to imagine that this is a uniquely Irish problem that could account for Ireland supposedly having nearly twice the rate of piracy of the UK (42% in Ireland vs. the UK’s 26%).

While taking a look at the BSA site, I note that their automated tool, GASP, now exposes the illegal distribution and storage of copyrighted music, apparently: ‘new features in GASP v6.5 include reports that show a complete listing of MP3 files on audited systems’.

I wonder if my collection of MP3s downloaded from eMusic.com, and ripped from albums I bought, would set off its alarms. Given that the MP3 format has no way to differentiate between pirated and non-pirated music, I would imagine so. False positives a go-go!

Patents: Eamon O’Tuathail’s call for action against software patents. The vote of the Legal Affairs Committee of the EP was yesterday, apparently. Wonder how it went…

Tags: bsa, gasp, internet, licence, percent, piracy, problem, rate, software, wonder

Lost from the Baghdad museum: truth (Guardian). hmmm! It seems we’ve been had:

(In April, it was widely reported that) 100,000-plus priceless items were looted (from the Baghdad museum) either under the very noses of the Yanks, or by the Yanks themselves. And the only problem with it is that it’s nonsense. It isn’t true. It’s made up. It’s bollocks.

Incredible — it seems (a) the museum was looted — to a degree; the vast majority of ‘missing’ items had actually been moved into safe storage, and ‘most of the serious looting was an inside job’.

And (b) the academics and journalists who reported ‘170,000 items … stolen or destroyed’ had been led by the nose by Dr Donny George, the museum’s director of research. It just wasn’t true:

Over the past six weeks it has gradually become clear that most of the objects which had been on display in the museum galleries were removed before the war. Some of the most valuable went into bank vaults, where they were discovered last week. Eight thousand more have been found in 179 boxes hidden ‘in a secret vault’. And several of the larger and most remarked items seem to have been spirited away long before the Americans arrived in Baghdad.

George is now quoted as saying that that items lost could represent ‘a small percentage’ of the collection and blamed shoddy reporting for the exaggeration. ‘There was a mistake,’ he said. ‘Someone asked us what is the number of pieces in the whole collection. We said over 170,000, and they took that as the number lost. Reporters came in and saw empty shelves and reached the conclusion that all was gone. But before the war we evacuated all of the small pieces and emptied the showcases except for fragile or heavy material that was difficult to move.’

This indictment of world journalism has caused some surprise to those who listened to George and others speak at the British Museum meeting. One art historian, Dr Tom Flynn, now speaks of his ‘great bewilderment’. ‘Donny George himself had ample opportunity to clarify to the best of (his) knowledge the extent of the looting and the likely number of missing objects,’ says Flynn. ‘Is it not a little strange that quite so many journalists went away with the wrong impression, while Mr George made little or not attempt to clarify the context of the figure of 170,000 which he repeated with such regularity and gusto before, during, and after that meeting.’ To Flynn it is also odd that George didn’t seem to know that pieces had been taken into hiding or evacuated. ‘There is a queasy subtext here if you bother to seek it out,’ he suggests.

Tags: april, collection, guardian, hmmm, museum, number, problem, truth, war, yanks

Yoz does a great job rounding up some Plan For Spam links. First off, he links to a great essay, Shooting The Messenger, which nicely rebuts the idea that to deal with spam, we need an SMTPng. Recommended. (He goes a bit overboard with some hard-ass filtering recommendations at the end IMO, though…)

Secondly, Yoz links to a couple more posts. The first is a friendly-fire incident involving the SpamCop DNS blacklists, illustrating the dangers of peer-to-peer ‘this is spam’ reporting. There’s a related issue with the SpamCop DNSBL, in that it’s over-sensitive; one report can sometimes be enough to get a site BLed, which is not good. The problems with SpamCop’s hair-trigger thresholds are well-documented, and — hopefully — Julian will fix them soon.

The second is a mail from John Gilmore to Politech. He says ‘a simple rule for anti-spam measures that preserves non-spammers’ freedom to communicate is: No anti-spam measure should ever block a non-spam message. But there isn’t a single anti-spam organization that actually follows this rule.’

Wrong. That’s exactly the SpamAssassin angle. If the user says it’s not spam, it’s not spam — and we have to figure out a way to get our scoring system to return that result, if at all possible. And yes, it gets it wrong about 0.1% of the time — and that’s why we never tell users to block, bounce or delete spam if at all possible; just mark it ‘possible spam’ and divert to another folder, and always let a human take a look to verify that decision.

Given the nature of the spam problem, and the nuisance it poses to virtually everybody trying to use email, that’s the best that can be done at this point.

And yes, something has to be done. Spam is a massive problem. If it’s not dealt with somehow, and kept out of our day-to-day inboxes, people will stop using mail. Before spam filters became ubiquitous, I talked to many casual internet users who (a) closed down their email address every 6 months to escape the flood, or (b) gave up reading their mail because of it. (And why did spam filters become ubiquitous?)

It comes down to: what’s better for the internet — a mislabelled email in your ’spam bucket’ folder — or no email at all?

Tags: email, essay, job, mail, plan, problem, rule, spam, spamcop, yoz

BBC: How does Dyson make water go uphill? A very cool hack from a Dyson engineer for the Chelsea Flower Show — an M. C. Escher-influenced water feature which gives the illusion that the water is flowing uphill.

A set of four glass ramps positioned in a square clearly show water travelling up each of them before it pours off the top, only to start again at the bottom of the next ramp.

It is a sight which defies logic, and has become probably the most memorable image of this year’s show.

Mr Dyson says his inspiration was a drawing by the Dutch artist MC Escher (he of Gothic palaces where soldiers are eternally walking upstairs, and of patterns where birds turn into fish).

Privacy: Danny forwards this post which discusses what the poster calls the ‘little elves’ problem. Very good point and contains this great real-world example:

Peter Wright in ‘Spycatcher’ … describes one of the problems arising out of the Berlin Tunnel Operation thus: ‘So much raw intelligence was flowing out from the East that it was literally swamping the resources available to transcribe (and translate) and analyse it. MI6 had a special transcription center set up in Earl’s Court, but they were still transcribing material seven years later when they discovered that George Blake had betrayed the Tunnel to the Russians from the outset’.

Funnily enough, I have the same problem — a lack of processing power to deal with the raw incoming volume — with my spamtraps from time to time. Now I can describe it in terms of ‘little elves’.

Patents: W3C announce patent policy. They’ve decided on Royalty-Free as a requirement, good news. TimBL’s comments on the decision:

Many participants in the original development of the Web knew that they might have sought patents on the work they contributed to W3C, and that they might have tried to secure exclusive access to these innovations or charge licensing fees for their use. However, those who contributed to building the Web in its first decade made the business decision that they, and the entire world, would benefit most by contributing to standards that could be implemented ubiquitously, without royalty payments.

This decision on the W3C Patent Policy coincides almost exactly with the tenth anniversary of CERN’s decision to provide unencumbered access to the basic Web protocols and software developed there, even before the creation of W3C. In fact, the success of technical work at the World Wide Web Consortium depended significantly on that decision by CERN. The decision to base the Web on royalty-free standards from the beginning has been vital to its success until now. The open platform of royalty-free standards enabled software companies to profit by selling new products with powerful features, enabled e-commerce companies to profit from services that on this foundation, and brought social benefits in the non-commercial realm beyond simple economic valuation. By adopting this Patent Policy with its commitment to royalty-free standards for the future, we are laying the foundation for another decade of technical innovation, economic growth, and social advancement.

Quite. I remember seeing Mosaic for the first time — my first thought was ‘wow, it’s like those commercial hypertext systems, but it’s free’. Initially, the free-ness was a lot more important than the network transparency it also offered.

There had already been several commercial hypertext systems, with expensive licensing terms. I’d only ever seen them bundled with other products (like the AIX documentation viewer) or used in kiosk systems.

They pretty much foundered when HTTP and HTML became available. But there’s no question to my mind that if CERN had made HTTP/HTML a commercial, licensed, or royalty-paying proposition, we wouldn’t even be talking about the web (or should I say the ‘WWW’?) nowadays.

Tags: bbc, engineer, escher-influenced, feature, problem, show, time, tunnel, water, web

TidBITS weighs in. They cover the issues very well, and also have noticed the problem that arises when a C-R system decides to challenge e-commerce notifications — like your air travel e-tickets, for example.

Found at Gary Robinson’s blog, where he also links a couple of taint.org items, cheers Gary ;)

Also, from /.: the House of Lords debates the etymology of ’spam’. Quite funny:

Lady Saltoun of Abernethy: My Lords, do the Government have any plans to restrict unsolicited faxes? My fax paper is always being wasted by people who send me faxes I do not want. I do not know whether they could be called ‘corned beef’ or something, but I have had enough of them.

Plus another anti-spam Senate bill, from Rep. W.J. ‘Billy’ Tauzin (R-La.) and F. James Sensenbrenner Jr. (R-Wis.). This one is apparently riddled with loopholes: ‘this is yet another bill . . . attempting to get rid of the porn and the scams, but really clearing the way for legitimate companies to spam,” said John Mozena, co-founder of … CAUCE.’

Tags: air, bill, blog, example, found, problem, spam, system, tidbits, travel

Declan McCullagh: A modest proposal to end spam. Good article on Larry Lessig’s ’spam bounties’ proposal.

Lofgren’s plan won’t give everyone who gets spammed new rights to sue (although spam victims may already may have some rights under state antispam or other laws). Instead, it states that people sending unsolicited commercial e-mail must label it with ‘ADV:’ in the subject line or run the risk of being sued by the Federal Trade Commission. If you are the first to report an unlabeled spam-o-gram to the government, you will get a bounty of ‘not less than 20 percent’ of the fine the spammer pays, assuming it can ever be collected.

There are problems with this. As far as I know, the FTC is not having a problem collecting spam — the figures I’ve seen (can’t recall them right now) indicate that they get hundreds of megs a day. (Even the SpamAssassin.org spamtraps get over 100Mb a day.)

The difficulty is chasing down the perpetrator, and prosecuting. That takes law-enforcement manpower, and that’s just not there right now — because, let’s face it, spam is not a serious offence like rape or murder.

Anyway, Declan says that the major problem is that the spammers are offshore:

For one thing, an increasing percentage of it comes from overseas, and you can be certain that offshore bulk mailers will gleefully thumb their noses at Congress. Ken Schneider, chief technical officer of antispam company Brightmail, estimates that 30 percent to 50 percent of the spam his company tracks comes from outside the United States. ‘It’s a big number,’ Schneider said. ‘It’s a global economy, and spammers are certainly taking advantage of it.’

This is a frequent misapprehension. This is not the case. It’s true that much spam is relayed through machines in Asia and South America, but the originators — the people who are writing the spam and sending it to compromised relay machines and proxies — are US-based. In fact, a vast quantity of ‘em seem to be based in Florida. (This is the thing about country-code blacklists. In reality, if we could track a message all the way back to the origin, a state-code blacklist for FL would probably work much better ;)

In other news from the same article:

… Sen. Chuck Schumer, D-N.Y., is expected to introduce a bill this week to create an national ‘do not e-mail’ list–an idea that the New Democrats touted earlier this month.

OK, while I’m here, let’s debunk ‘do no mail’ lists too. ;) ‘Do not call’ lists work well for telephones, since you typically have only one phone number. But for email:

• one can have thousands of valid email addresses forwarding to you (I do). There’s a variety of methods to address even one user, for example ‘foo@jmason.org‘, ‘foo@jmason.org.’, ‘foo%jmason.org@localhost‘, ‘foo@212.17.35.15‘, ‘foo@dogma.slashnull.org‘ will all reach me. That’s not even considering ‘role’ addresses, like ’sales@’, ‘info@’, etc., or single-use addresses set up for particular transactions, like ‘onetime-buying-stuff-from-ecommerce-site-example@jmason.org‘.

• mailing lists and ‘exploders’ are widespread, and frequently spammed.

• ‘do not mail’ lists are hard to implement, since they may be vulnerable to scraping (if naively done) or dictionary attacks (less naive).

In summary, I’m not confident a ‘do not mail’ list could actually be operable.

Finally — The SBL’s answer to the EMarketersAmerica.org SLAPP lawsuit.

Tags: antispam, article, company, day, lofgren, percent, plan, problem, proposal, spam

The Open Proxy Problem, a PowerPoint/PDF presentation shown at the Internet2 Members Meeting of April 9th 2003, by Joe St Sauver, Ph.D (Director, User Services and Network Applications University of Oregon Computing Center).

Well worth a read if you’re interested in network security or spam. Joe’s done an astonishing job of researching every angle of the issue, from historical comparisons to ‘blue boxes’ circa 1971, the status of proxy servers to the Chinese government, and even a statistical analysis of proxy DNSBL overlap. (BTW, did you know that the New York Times was broken into via an open proxy?)

Tags: april, internet, network, open, pdf, powerpoint, presentation, problem, proxy, sauver

I’ve just fixed my desktop machine (had to buy a new CPU, unfortunately, after the old one died during shipping).

I then upgraded to Red Hat 9 (woo, very nice), switched to KDE for my desktop, and took a look at software suspend (because the machine is too noisy to leave on permanently in the corner of the living room).

However, the latter won’t work with my video card; instead, the machine reboots continually when resuming from suspend. Problem.

A bit of thinking about the problem came up with a nifty solution… I’d heard of folks using a VNC server for their main desktop, in order to connect to it from any machine they found themselves near, and not be ‘tethered’ to one particular desktop machine. The same system also means I can run my desktop with a virtual display, and just ‘connect’ to this from the real one. Then, when I want to suspend, I can just kill off the X server, suspend, and start up a new one after resume.

If you’re curious about how to do this, read on…

From: Justin Mason
Subject: setting up a VNC desktop

Software suspend won’t work with my video card; instead, the machine reboots continually when resuming from suspend. Problem.

A bit of thinking about the problem came up with a nifty solution… I’d heard of folks using a VNC server for their main desktop, in order to connect to it from any machine they found themselves near, and not be ‘tethered’ to one particular desktop machine. The same system also means I can run my desktop with a virtual display, and just ‘connect’ to this from the real one. Then, when I want to suspend, I can just kill off the ‘hardware’ X server, suspend, and start up a new one after resume.

First, install xf4vnc. This gives you a VNC server that can use the ‘Render’ extension, and therefore display anti-aliased text efficiently. Installation of this is a bit of a manual job, unfortunately, since the author hasn’t actually packaged it in any way. Not too hard though; just 3 copy commands; I don’t think you actually need any files apart from the two in the xf4vnc-linux-i386 group.

Create a file called ~/.xserverrc containing:

:: /usr/local/bin/Xvnc-xf4vnc -depth 16 -geometry 1152×864 -deferupdate 10 :0

Best to make the depth and geometry match your current display.

Next, create a script called ~/bin/x containing:

:: #!/bin/sh
:: X :1 &
:: sleep 4
:: vncviewer -compresslevel 0 -quality 9 -fullscreen -display :1 localhost:0

(ie. start an X display on :1, then display vncviewer to that display.) Don’t forget to make it executable with chmod.

Now, close your current X desktop, return to the console, and run startx to start a new one. This won’t display; instead, it’ll run GNOME/KDE/whatever using a virtual framebuffer. CTRL-Z and bg that process.

Run the x script. It’ll connect to your virtual desktop. That’s it!

You can now hit CTRL-ALT-Backspace to your heart’s content. When your display is killed, the applications and desktop remain untouched. When you rerun the x script, it’ll reconnect and nothing will have changed apart from the mouse pointer position. In fact, I just restarted my X server halfway through that sentence ;)

Have fun!

Tags: bit, desktop, display, machine, problem, resume, server, software, solution, vnc

Tim Bray’s weblog is a great read; I’ve added it to my daily list. Today, he’s provided a fantastic article about the drugs problem in Vancouver’s Downtown Eastside.

Dublin has historically had a serious of up-and-down swings with a heroin problem; at one stage, it was one of the worst in Europe. It improved quite a lot during the 90’s, but it’s going downhill again, apparently; maybe the legislators need to read this article.

(The big problem as far as I can see is that treatment centres are horrifically underfunded, it being a lot easier, and — while not cheaper – at least already budgeted for, to ship the junkies off to prison. Business as usual. Of course, while they’re there, they’re (a) off the streets (out of sight, out of mind), and (b) learning all the latest criminal techniques, and getting well hooked on all the cheap heroin in there.)

(BTW did you know that one reason heroin is massively popular in prisons, is due to drug-testing? Apparently, marijuana can be detected a month after use, whereas heroin is undetectable 48 hours afterwards. So prison drug-testing regimes indirectly encourage heroin use. Oops!)

Linux: Linux Journal: report from LinuxWorld Ireland. Sounds like a great talk from maddog and Michael Meeks. And if you look carefully at the photo on that article page, you can see Proinnsias in the background!

Mind you, I would probably have just done my ‘incomprehensible question about software patents’ schtick with the IBM guy again…

What with this and GUADEC coming to Dublin, I’m missing all the good piss-ups^Wevents it seems ;)

Tags: article, bray, heroin, linux, lot, prison, problem, tim, use, weblog

BBC news chiefs have met to discuss the increasing problem of misinformation coming out of Iraq as staff concern grows at the series of premature claims and counter claims by military sources. ‘By last Sunday the southern Iraqi seaport of Umm Qasr had been reported ‘taken’ nine times’ … ‘We’re getting more truth out of Baghdad than the Pentagon at the moment’.

Date: Fri, 28 Mar 2003 09:05:27 +0000
From: “Tim Chapman” (spam-protected)
To: forteana (spam-protected)
Subject: Fun with disinformation

http://media.guardian.co.uk/broadcast/story/0,7493,924169,00.html

BBC chiefs stress need to attribute war sources

Claims and counter-claims in the media

Ciar Byrne Friday March 28, 2003

BBC news chiefs have met to discuss the increasing problem of misinformation coming out of Iraq as staff concern grows at the series of premature claims and counter claims by military sources.

As a result the corporation has reinforced the message to correspondents that they must clearly attribute information to the military when it has not been backed up by another source.

“There’s been a discussion about attribution and it’s been reinforced with people that we do have to attribute military information,” said a BBC spokeswoman.

“We have to be very careful in the midst of a conflict like this one to be very sure when we’re reporting something we’ve not seen with our own eyes that we attribute it,” she added.

On nearly every day of the war so far there have been reports that could be seen as favourable to coalition forces, which have later turned out to be inaccurate.

Earlier this week there was confusion over whether there had been an uprising in the key southern city of Basra. A British forces spokesman, Group Captain Al Lockwood, said on Thursday there had been a “popular uprising”, but this was denied by Iraqi authorities.

By last Sunday the southern Iraqi seaport of Umm Qasr had been reported “taken” nine times, while reports of the discovery of a chemical weapons factory in An Najaf have not been confirmed - just two more examples of the confusion over what is coming out of military sources.

“We’re absolutely sick and tired of putting things out and finding they’re not true. The misinformation in this war is far and away worse than any conflict I’ve covered, including the first Gulf war and Kosovo,” said a senior BBC news source.

“On Saturday we were told they’d taken Basra and Nassiriya and then subsequently found out neither were true. We’re getting more truth out of Baghdad than the Pentagon at the moment. Not because Baghdad is putting out pure and morally correct information but because they’re less savvy about it, I think.

“I don’t know whether they (the Pentagon) are putting out flyers in the hope that we’ll run them first and ask questions later or whether they genuinely don’t know what’s going on - I rather suspect the latter.”

Earlier this week the BBC’s director of news, Richard Sambrook, admitted it was proving difficult for journalists in Iraq to distinguish truth from false reports, and that the pressures facing reporters on 24-hour news channels had led to premature or inaccurate stories.

Veteran war correspondent Martin Bell has called for 24-hour news channels to “curb their excitability” and warned against unsubstantiated reports which may help the allied cause, but later turn out to be false.

The Times journalist Janine di Giovanni has also said that the demands of real-time television, combined with the restrictions placed on reporters in Baghdad by the Iraqis and the difficulties of getting to the front line are making it virtually impossible for journalists to cover the war properly.

Tags: bbc, concern, information, misinformation, news, problem, seaport, series, staff, war

Yahoo: al-Jazeera website redirected:

The hacker was able to gain control of the domain name by asking domain
seller Network Solutions for the account password on official al-Jazeera
stationery, said an industry source speaking on condition of anonymity.
A spokesman for Network Solutions’ parent company declined to comment on
how the hacker was able to hijack the domain name, but said the company
had fixed the problem and was trying to track the impostor down.
‘We followed our procedures, in this particular instance someone was able
to get around those procedures,’ said Brian O’Shaughnessy, a spokesman
for Internet security firm VeriSign.

They fixed the problem? Surely this is exactly what happened with the sex.com domain several years ago?

Tags: al, company, domain, hacker, name, network, problem, spokesman, website, yahoo

Son of Star Wars leaves drivers stranded (Guardian). Interesting collision between military and civvie radio technology.

The upgrading of the security and surveillance systems at (RAF Fylingdales base in Yorkshire, which is planned to be used as a UK base for new US ‘Star Wars’ projects) … is knocking out the electrical systems of expensive cars. … High power radar pulses trigger the immobilising devices of many makes of cars and motorcycles - BMW, Mercedes and Jeep among them. Many have had to be towed out of range of the base before they can be restarted.

Wing Commander Chris Knapman, of RAF Fylingdales, said it was not up to the base to resolve the problem. ‘We have had the frequencies we use for a very long time,’ he said. ‘They are allocated to commercial, military and government users, and the allocation is very tightly controlled. As far as we are concerned, the radars are working on frequencies which are well known, and most car manufacturers take that into account.’

A spokesman for Jeep said: ‘The problem is that the government gives manufacturers such a narrow band to operate in - so the radio wave (sic) we use for our key fob is severely restricted.’

Tags: base, fylingdales, government, jeep, military, problem, radio, raf, son, star

A great tale of systems wierdness, via 0xdeadbeef:

‘We’re having a problem sending email out of the department.’ ‘What’s the problem?’ I asked. ‘We can’t send mail more than 500 miles,’ the chairman explained.

Date: Tue, 26 Nov 2002 14:57:40 -0800
From: (spam-protected) (glen mccready)
To: (spam-protected)
Subject: The case of the 500-mile email.

>Forwarded-by: Nev Dull (spam-protected)
>Forwarded-by: Kirk McKusick (spam-protected)
>From: Trey Harris (spam-protected)

Here’s a problem that *sounded* impossible… I almost regret posting the story to a wide audience, because it makes a great tale over drinks at a conference. :-) The story is slightly altered in order to protect the guilty, elide over irrelevant and boring details, and generally make the whole thing more entertaining.

I was working in a job running the campus email system some years ago when I got a call from the chairman of the statistics department.

“We’re having a problem sending email out of the department.”

“What’s the problem?” I asked.

“We can’t send mail more than 500 miles,” the chairman explained.

I choked on my latte. “Come again?”

“We can’t send mail farther than 500 miles from here,” he repeated. “A little bit more, actually. Call it 520 miles. But no farther.”

“Um… Email really doesn’t work that way, generally,” I said, trying to keep panic out of my voice. One doesn’t display panic when speaking to a department chairman, even of a relatively impoverished department like statistics. “What makes you think you can’t send mail more than 500 miles?”

“It’s not what I *think*,” the chairman replied testily. “You see, when we first noticed this happening, a few days ago–”

“You waited a few DAYS?” I interrupted, a tremor tinging my voice. “And you couldn’t send email this whole time?”

“We could send email. Just not more than–”

“–500 miles, yes,” I finished for him, “I got that. But why didn’t you call earlier?”

“Well, we hadn’t collected enough data to be sure of what was going on until just now.” Right. This is the chairman of *statistics*. “Anyway, I asked one of the geostatisticians to look into it–”

“Geostatisticians…”

“–yes, and she’s produced a map showing the radius within which we can send email to be slightly more than 500 miles. There are a number of destinations within that radius that we can’t reach, either, or reach sporadically, but we can never email farther than this radius.”

“I see,” I said, and put my head in my hands. “When did this start? A few days ago, you said, but did anything change in your systems at that time?”

“Well, the consultant came in and patched our server and rebooted it. But I called him, and he said he didn’t touch the mail system.”

“Okay, let me take a look, and I’ll call you back,” I said, scarcely believing that I was playing along. It wasn’t April Fool’s Day. I tried to remember if someone owed me a practical joke.

I logged into their department’s server, and sent a few test mails. This was in the Research Triangle of North Carolina, and a test mail to my own account was delivered without a hitch. Ditto for one sent to Richmond, and Atlanta, and Washington. Another to Princeton (400 miles) worked.

But then I tried to send an email to Memphis (600 miles). It failed. Boston, failed. Detroit, failed. I got out my address book and started trying to narrow this down. New York (420 miles) worked, but Providence (580 miles) failed.

I was beginning to wonder if I had lost my sanity. I tried emailing a friend who lived in North Carolina, but whose ISP was in Seattle. Thankfully, it failed. If the problem had had to do with the geography of the human recipient and not his mail server, I think I would have broken down in tears.

Having established that — unbelievably — the problem as reported was true, and repeatable, I took a look at the sendmail.cf file. It looked fairly normal. In fact, it looked familiar.

I diffed it against the sendmail.cf in my home directory. It hadn’t been altered — it was a sendmail.cf I had written. And I was fairly certain I hadn’t enabled the “FAIL_MAIL_OVER_500_MILES” option. At a loss, I telnetted into the SMTP port. The server happily responded with a SunOS sendmail banner.

Wait a minute… a SunOS sendmail banner? At the time, Sun was still shipping Sendmail 5 with its operating system, even though Sendmail 8 was fairly mature. Being a good system administrator, I had standardized on Sendmail 8. And also being a good system administrator, I had written a sendmail.cf that used the nice long self-documenting option and variable names available in Sendmail 8 rather than the cryptic punctuation-mark codes that had been used in Sendmail 5.

The pieces fell into place, all at once, and I again choked on the dregs of my now-cold latte. When the consultant had “patched the server,” he had apparently upgraded the version of SunOS, and in so doing *downgraded* Sendmail. The upgrade helpfully left the sendmail.cf alone, even though it was now the wrong version.

It so happens that Sendmail 5 — at least, the version that Sun shipped, which had some tweaks — could deal with the Sendmail 8 sendmail.cf, as most of the rules had at that point remained unaltered. But the new long configuration options — those it saw as junk, and skipped. And the sendmail binary had no defaults compiled in for most of these, so, finding no suitable settings in the sendmail.cf file, they were set to zero.

One of the settings that was set to zero was the timeout to connect to the remote SMTP server. Some experimentation established that on this particular machine with its typical load, a zero timeout would abort a connect call in slightly over three milliseconds.

An odd feature of our campus network at the time was that it was 100% switched. An outgoing packet wouldn’t incur a router delay until hitting the POP and reaching a router on the far side. So time to connect to a lightly-loaded remote host on a nearby network would actually largely be governed by the speed of light distance to the destination rather than by incidental router delays.

Feeling slightly giddy, I typed into my shell:

$ units 1311 units, 63 prefixes

You have: 3 millilightseconds You want: miles

• 558.84719 / 0.0017893979

“500 miles, or a little bit more.”

Trey Harris – I’m looking for work. If you need a SAGE Level IV with 10 years Perl, tool development, training, and architecture experience, please email me at (spam-protected) I’m willing to relocate for the right opportunity.

Tags: chairman, department, email, forwarded-by, mail, panic, problem, story, tale, voice

Joel on Software now features a great new article on what he calls “Leaky Abstractions”. Some snippets:

• Even though network libraries like NFS and SMB let you treat files on remote machines “as if” they were local, sometimes the connection becomes very slow or goes down, and the file stops acting like it was local, and as a programmer you have to write code to deal with this. The abstraction of “remote file is the same as local file” leaks. …

(jm: the ‘transparent does not always mean good’ problem)

• Something as simple as iterating over a large two-dimensional array can have radically different performance if you do it horizontally rather than vertically, depending on the “grain of the wood” — one direction may result in vastly more page faults than the other direction, and page faults are slow. Even assembly programmers are supposed to be allowed to pretend that they have a big flat address space, but virtual memory means it’s really just an abstraction, which leaks when there’s a page fault and certain memory fetches take way more many nanoseconds than other memory fetches.

(jm: the ‘why objects are not always the way to do it’ problem)

And finally, he ends with a killer:

Ten years ago, we might have imagined that new programming paradigms would have made programming easier by now. Indeed, the abstractions we’ve created over the years do allow us to deal with new orders of complexity in software development that we didn’t have to deal with ten or fifteen years ago, like GUI programming and network programming. And while these great tools, like modern OO forms-based languages, let us get a lot of work done incredibly quickly, suddenly one day we need to figure out a problem where the abstraction leaked, and it takes 2 weeks. And when you need to hire a programmer to do mostly VB programming, it’s not good enough to hire a VB programmer, because they will get completely stuck in tar every time the VB abstraction leaks.

Well said! Read the article!

Tags: abstraction, direction, file, memory, page, problem, programmer, programming, the, way

Gamasutra reports from GDC Europe. It’s good to see Systemic Game Design is getting a lot more attention these days as CPU power increases on consoles, instead of the random 3D graphics tweakery that predominates on the PC platform. Systemic game design is defined here as follows:

“Instead of hard-coding lots of features into the game .. the systemic paradigm tries to create global patterns which provide emergent gameplay, and the ability to create alternative strategies using the level’s resources. … In this way a player can come up with new ideas to solve problems by combining items in ways that perhaps even the level designers hadn’t considered. This improves the sense of immersion and freedom, while emphasizing player’s self-expression capabilities through the game. … An example of a systemic game is GTA3, where each mission can be solved in dozens of ways, as compared to old lock-and-key adventure games, where player expression and alternative strategies were basically non-existent. In a systemic game world, the player can use different methods to solve a problem. In a non-systemic game world, you must guess how the game designer wanted you to solve the problem, even if that way does not feel very intuitive, nor fun.”

Mmm. Grand Theft Auto 3. PS: GTA3 can also be found on my Amazon wishlist ;)

Tags: design, gamasutra, game, gta, level, player, problem, systemic, way, world

Hamlet vs. ISDN:

Technician 1: My name be John. What problem do you have?

Hamlet: A heart so full of woe to shame the gods.

My father dead. My mother newly wed

To mine own uncle who hath stole my crown.

But worst of all, like demon born of Hell,

Connection’s lost; I hath no ISDN.

Tags: father, hamlet, hath, heart, isdn, name, problem, shame, technician, woe