PRNGs and Groove Theory

Urban Dead is a new browser-based MMORPG that’s become popular recently. I’m not planning to talk about the game itself, at least not until I’ve played it a bit!, but there’s something worth noting here — a cheat called Groove Theory:

Groove Theory was a cheat for Urban Dead that claimed to exploit an apparent lack [sic] of a random number generator in the game, [so] that performing an action exactly eight seconds after a successful action would also be successful.

Kevan, the Urban Dead developer, confirmed that Groove Theory did indeed work, and made this comment after fixing the bug:

There is some pattern to the random numbers, playing around with them; “srand(time())” actually brings back some pretty terrible patterns, and an eight-second wait will catch some of these.

So — here’s my guess as to how this happened.

It appears that Urban Dead is implemented as a CGI script. I’ll speculate that somewhere near the top of the script, there’s a line of code along the lines of srand(time()), as Kevan mentioned. With a sufficiently fast network connection, and a sufficiently unloaded server, you can be reasonably sure that hitting “Refresh” will cause that srand call to be executed on the server within a fraction of a second of your button-press. In other words, through careful timing, the remote user can force the pseudo-random-number generator used to implement rand() into a desired set of states!

As this perl script demonstrates, the output from perl’s rand() is perfectly periodic in its low bits on a modern Linux machine, if constantly reseeded using srand()the demo script’s output decrements from 3 to 0 by 1 every 2 seconds, then repeats the cycle, indefinitely.

I don’t know if Urban Dead is a perl script, PHP, or whatever; but whatever language it’s written in, I’d guess that language uses the same PRNG implementation as perl is using on my Linux box.

As it turns out, this PRNG failing is pretty well-documented in the manual page for rand(3):

on older rand() implementations, and on current implementations on different systems, the lower-order bits are much less random than the higher-order bits. Do not use this function in applications intended to be portable when good randomness is needed.

That manual page also quotes Numerical Recipes in C: The Art of Scientific Computing (William H. Press, Brian P. Flannery, Saul A. Teukolsky, William T. Vetterling; New York: Cambridge University Press, 1992 (2nd ed., p. 277)) as noting:

“If you want to generate a random integer between 1 and 10, you should always do it by using high-order bits, as in

j=1+(int) (10.0*rand()/(RAND_MAX+1.0));

and never by anything resembling

j=1+(rand() % 10);

(which uses lower-order bits).”

I think Groove Theory demonstrates this nicely!

Update: I need to be clearer here.

Most of the Groove Theory issue is caused by the repeated use of srand(). If the script could be seeded once, instead of at every request, or if the seed data came from a secure, non-predictable source like /dev/random, things would be a lot safer.

However, the behaviour of rand() is still an issue though, due to how it’s implemented. The classic UNIX rand() uses the srand() seed directly, to entirely replace the linear congruential PRNG’s state; on top of that, the arithmentic used means that the low-order bits have an extremely obvious, repeating, simple pattern, mapping directly to that seed’s value. This is what gives Groove Theory its practicability by a human, without computer aid; with a more complex algorithm, it’d still be guessable with the aid of a computer, but with the simple PRNG, it’s guessable, unaided.

Update 2: as noted in a comment, Linux glibc’s rand(3) is apparently quite good at producing decent numbers. However, perl’s rand() function doesn’t use that; it uses drand48(), which in glibc is still a linear congruential PRNG and displays the ‘low randomness in low-order bits’ behaviour.

Tags: , , , , , , , ,

Comments (8)

Faster string search alternative to Boyer-Moore: BloomAV

An interesting technique, from the ClamAV development list — using Bloom filters to speed up string searching. This kind of thing works well when you’ve got 1 input stream, and a multitude of simple patterns that you want to match against the stream. Bloom filters are a hashing-based technique to perform extremely fast and memory-efficient, but false-positive-prone, binary lookups.

The mailing list posting (’Faster string search alternative to Boyer-Moore‘) gives some benchmarks from the developers’ testing, along with the core (GPL-licensed) code:

Regular signatures (28,326) :

  • Extended Boyer-Moore: 11 MB/s

  • BloomAV 1-byte: 89 MB/s

  • BloomAV 4-bytes: 122 MB/s

Some implementation details:

the (implementation) we chose is a simple bit array of (256 K * 8) bits. The filter is at first initialized to all zeros. Then, for every virus signature we load, we take the first 7 bytes, and hash it with four very fast hash functions. The corresponding four bits in the bloom filter are set to 1s.

Our intuition is that if the filter is small enough to fit in the CPU cache, we should be able to avoid memory accesses that cost around 200 CPU cycles each.

Also, in followup discussion, the following paper was mentioned: A paper describing hardware-level Bloom filters in the Snort IDS — S. Dharmapurikar, P. Krishnamurthy, T. Sproull, and J. W. Lockwood, “Deep packet inspection using parallel Bloom filters,” in Hot Interconnects, (Stanford, CA), pp. 44–51, Aug. 2003.

This system is dubbed ‘BloomAV’. Pretty cool. It’s unclear if the ClamAV developers were keen to incorporate it, though, but it does point at interesting new techniques for spam signatures.

Tags: , , , , , ,

Comments

Happy Birthday to the RISKS Forum!

Tech: One of the first online periodicals I started reading regularly, when I first got access to USENET back in 1989 or so, was comp.risks – Peter G. Neumann’s RISKS Forum. Since then, I’ve been reading it religiously, in various formats over the years.

It appears that RISKS has just celebrated its 20th anniversary.

Every couple of weeks it provides a hefty dose of computing reality to counter the dreams of architecture astronauts and the more tech-worshipping members of our society, who fail to realise that just because something uses high technology, doesn’t necessarily make it safer.

I got to meet PGN a couple of weeks ago at CEAS, and I was happy to be able to give my thanks — RISKS has been very influential on my code and my outlook on computing and technology.

Nowadays, with remote code execution exploits for e-voting machines floating about, and National Cyber-Security Czars, I’d say RISKS is needed more than ever. Long may it continue!

Tags: , , , , , , , , , ,

Comments

Continuations in perl

Code: Ugo Cei: Building Interactive Web Programs with Continuations quoting Phil Windley:

This leads to the question: what if I could write programs for the Web that were ’structured’ in the programming sense of that word? The result would be Web programs that were more natural to write and easy to read. You’d no longer have to maintain the state of your program outside the language and the data could be kept in variables, where it belongs. The answer is: you can.

I hate the ’save all state’ model imposed by developing for the web, and have been hoping for a way to do this for a while — and now I know what it’s called ;)

It seems Seaside is the leading continuations-based web-app framework, using Smalltalk, and (as Ugo noted) Apache Cocoon has it too, but there’s a whole load more. Can you tell I haven’t been following web-app development techniques much recently?

Never mind those other languages, though — Continuity looks promising as a Perl framework based around continuations. Perl 6 will reportedly have native continuation support, and Dan Sugalski gives a good write-up of how they’re implemented and their ramifications there.

Tags: , , , , , , , , , ,

Comments

Web-browser style history for the command line

Code: Here’s something I came up with recently — it’s actually an evolution of the idea of pushd and popd, as included in BASH. To quote the POD docs:

cdhistory is a perl script used to implement web-browser style “history” for UNIX shells; as you use the cd command to explore the filesystem, your moves are remembered, and you can go “back” through history, and “forward” again, as you like.

Download the perl script here.

Tags: , , , , , , , , , ,

Comments

Why implanted ID chips are bad for privacy

Security: The RFID vendors are clearly on a roll, with all manner of uses being proposed. The most recent story is that VeriChip plans to implant them subdermally in hospital patients.

The company line is that it’s privacy-safe, since it doesn’t expose health records per se — just the patient’s ID number. However, that’s missing the point, in my opinion.

RFID chips will broadcast their ID whenever they are within range of a compatible scanner, and the range (in this case) is several feet – although the story notes that their readers used to track farmed salmon work from 10-12 feet, and the Schmoo Group guys I met last month had no doubts that a high-powered directional antenna like their wi-fi sniper rifle could extend that. There’s no encryption, or handshaking, in these chips, it sounds like.

There’s no mention if the chip is removed after you leave hospital; some comments about the idea behind this is that it may help if you’re involved in an accident, and want your info available to healthcare users, in which case you’d have the chip implanted and broadcasting at other times, in other places, as well.

So, if you’ve got one of these implanted, it’ll broadcast a unique code to readers in range at all times. If an attacker can scan while you’re nearby, and picks up that code, they know that it’s you, and you only. They only have to match that ID code to a visual identification once, and henceforth you can be tracked by that ID code.

There’s a possibility that they’ll fix this, by upping the CPU power and incorporating some decent public-key encryption — but then you need a PKI big enough to track every implanted citizen in the entire country, and the costs will go up and up. I’d find that doubtful. (Mind you, they seem to assume that having a centralized secure database of medical records is a fait accompli in most of the articles anyway, so…)

Tags: , , , , , , , , , ,

Comments

MS Using Apache Software

Apache: Not content with distributing GPL’d software, Microsoft are now taking another step into the open-source world by shipping Apache-licensed code.

Not quite as big a deal as the GPL — but still, another interesting milestone.

Tags: , , , , , , , , , ,

Comments

Ahmed Chalabi and Iran’s encryption

Security: some crypto drama.

Ahmad Chalabi apparently told the Iranian government that the NSA had broken their secret code, according to ‘US intelligence officials’: NYTimes: Chalabi Reportedly Told Iran That U.S. Had Code. This story is still running — Bruce Schneier has just posted his expert opinion, as has Ross Anderson. As I noted on Eric Rescorla’s weblog, here’s my (non-expert) theory ;)

It’s known that the Iranians used Crypto AG equipment up until about 1992, and it’s been widely reported that Crypto AG’s systems were backdoored by the NSA and traffic routinely decrypted. (also, Baltimore Sun story, 1995)

Reportedly, the Anglo-Irish discussions of the 1985 were a rather one-sided affair, because the Irish government used Crypto AG machines to communicate between their Embassy in London and Dublin, and intercepts of their reports were fed back to the UK government.

In addition, according to this article (backup), the NSA also provided Iraq with intercepts of Iranian secret traffic, while Iraq was a US ally — which could explain why Chalabi would have known about it.

It also speculates as to how it was done:

‘Knowledgeable sources indicate that the Crypto AG enciphering process, developed in cooperation with the NSA and the German company Siemans, involved secretly embedding the decryption key in the cipher text. Those who knew where to look could monitor the encrypted communication, then extract the decryption key that was also part of the transmission, and recover the plain text message. Decryption of a message by a knowledgeable third party was not any more difficult than it was for the intended receiver. (More than one method was used. Sometimes the algorithm was simply deficient, with built-in exploitable weaknesses.)’

So my opinion is that Chalabi’s claim was very old news from the 80’s and early 90’s — which pretty much fits in with the rest of his tip-offs to everyone else ;)

Tags: , , , , , , , , , ,

Comments

MS’ latest patent

Patents: Oh, come on. USPTO: task list window for use in an integrated development environment. Here’s claim 1:

  1. A computer-implemented method for managing development-related tasks, the method comprising:

    during an interactive code development session, evaluating source code to determine whether a comment token is present;

    in response to determining that the source code contains a comment token, inserting a task into a task list; and

    in response to completion of a task, modifying the task list during the interactive code development session to indicate that the task has been completed.

There’s 74 more claims that are about up to that standard, including the usual ‘an input module connected to the knee-bone’ mumbo-jumbo that means it ‘isn’t a software patent’.

This is just quite simply absurd. Are we really supposed to believe that nobody had thought of what is essentially a list of tickboxes, displaying the output of ‘grep TODO *.c’, before March 6, 2000? You have got to be kidding. This /. comment suggests that Delphi 5 (released 1999) did it.

(update: looks like there was a provisional patent application, so that may have to be Mar 5 1999.)

William Chiles, Anders Hejlsberg, Randy Kimmerly and Peter Loforte should be ashamed of themselves for filing this joke. And the USPTO examiner who granted it should be fired.

(PS: a factoid from the slashdot comments: IBM receives (note: not even ‘files for’) nearly 10 patents every day.)

Tags: , , , , , , , , , ,

Comments

Bloom Filters

Code: A very good intro to Bloom Filters at perl.com by Maciej Ceglowski.

Strikes me as something that might be very applicable to the SpamAssassin auto-whitelist…

Tags: , , , , , , ,

Comments

Closed-Source Runtimes

Open Source: A good entry at sourcefrog.net describing some reasons people are driven to use open source — the closed-source component library one, in particular, drives me nuts.

I’ve run into this in the past — here’s an example I can point to. That’s a fixed version of Java 1.0’s java.util.StreamTokenizer class, to fix a bug where space cannot be treated as a special character. (Hopefully it’s now obsolete, seeing as I wrote that 9 years ago!)

Note that I probably do not have permission to use and redistribute that class. Also note that the bug fix I submitted to Java 1.0 probably never made it into the code, because I was an individual user and not a major corporate client. The bug may have been fixed independently, however, given that StreamTokenizer still exists, but I doubt my fix ever got near the dev team. (However it still means I can say I fixed a bug in James Gosling’s code ;)

Invariably, getting access to source, and being allowed to fix bugs in it, is a key issue — and one that continually drives developers to open source/free software libraries. RMS has been saying this for years, of course.

Music: A massive selection of links to mp3 blogs. gabba > Pod looks very interesting… they even had a copy of Egyptian Empire’s Horn Track recently, one of my favourites.

Tags: , , , , , , , , , ,

Comments

sleep(1) in Berkeley DB?

Code: Berkeley DB, the de-facto std for open-source high-performance database files on UNIX, is displaying some odd behaviour — it appears to be sleeping for 1 second inside the database library code, under load, for some versions of libdb. If you’re curious, there’s More info here.

Tags: , , , , , , , , ,

Comments

Debugging Thoughts

Software: Nelson Minar: Primitive Debugging. Nelson quotes Kernighan, ‘The most effective debugging tool is still careful thought, coupled with judiciously placed print statements’, and assents from a viewpoint a quarter of a century later. Strange but true; I find this also. Why is that?

IMO, it’s all usability problems.

  • debuggers are labour-intensive. To print or explore a complex data
    • structure requires lots of typing, or liberal cut and paste from a side window with your debugger commands ready to go. DDD does a very good job of helping with this, since it’s built around a data display model.
    • It’s easy to make a mistake that requires a full restart. If you’re single-stepping through some code, hit a loop, and want to skip several steps, you might select ‘continue until loop exit’ — then find that you’ve gone too far. What can you do? Restart from scratch.

      There is a fix for this — backstepping. However, so far it seems to be only available in research models; I don’t know of any deployed debuggers that support this.

Even given a good debugger, I find myself throwing in a printf() every time. By now, my brain’s hard-wired to debug using printf.

(More correctly, my own equivalent, a ‘JMDB’ statement. This is a little bit of usability sugar; I’ve defined that in my editor as a language-dependent macro to output a ‘JMD’ string — so I can find it easily in code and output — and the file and line numbers, along with whatever data I want to log.)

It’s too late to save me ;)

Tags: , , , , , , , , , ,

Comments

Subversion

Code: Rod writes: ‘I have had a bunch of fun today, gleefully playing with a new source-control package. I truly lead a sad life.’

I’d guess that was our fault, moving SpamAssassin CVS to subversion at apache.org ;) Happy to oblige, Rod!

If I wasn’t so jet-lagged (still!), suffering from a cold, and busy with the day job, I’d be having that fun myself; SVN is very, very, very nice from what little I can tell so far. Only time will tell if it can beat the lovely Perforce, though, the virtues of which I have extolled on many occasions (earning myself a freebie T-shirt in the process, payola!).

But yeah, SVN looks really cool.

Tags: , , , , , , , , , ,

Comments

How Not To Use OOP

Code: OOP over the top: a hilarious dissection of some of the most monstrous ‘how to rewrite OO-style’ I have ever seen — take a 15-line if/elseif/else clause and rewrite as a thoroughly over-engineered unmaintainable 7-class, 15-method disaster, using the Singleton and Factory patterns. The rewrite in the original article is intended seriously, as far as I can tell.

As the xmldatabases.org article says: ‘this is really a general problem with OO development. Fancy object oriented architectures have become the goal and this article maybe makes that point more clearly then anything I could ever say. It’s representative of the thinking from a few years ago (written in 2000), and shows us just how much damage we now have to undo. It basically says that the simple solution that just works is wrong and will be unmaintainable. Maybe that’s true, maybe it’s not, nowhere does the article consider the question of whether or not that code actually needs to be that generic. It simply says that the simple solution is bad and that the seven class monster they came up with is the right solution. Talk about doing a disservice to students trying to learn how to build solid computer systems.’

(Found via sourcefrog.net – Martin Pool’s weblog, great for Linux and code bits).

WebMake: linux.com: An introduction to building sites with WebMake. W00t! Let’s hope nobody asks any questions while I’m away for xmas ;)

Tags: , , , , , , , , ,

Comments

Windows/Linux Biculturalism

Software: Joel on Biculturalism: ‘What are the cultural differences between Unix and Windows programmers? There are many details and subtleties, but for the most part it comes down to one thing: Unix culture values code which is useful to other programmers, while Windows culture values code which is useful to non-programmers.’

I’m not sure I agree; I’ve met lots of Windows programmers who take what Joel calls the ‘UNIX’ orientation, and even a few Unix people who are as user-oriented in their coding as what Joel calls the ‘Windows’ way.

But, talking of the Unix/Win divide — it seems that Ward Cunningham, inventor of the Wiki, is joining MS, who have something called SharePoint Team Services, an editable-web group sharing system as part of Front Page.

If you ever wanted to see an illustration of a Windows-Unix divide in the web age, it sounds like this is it: Wiki has quick-and-dirty links in FuglyBouncingCaps, is text-heavy, has obscure text markup formats, has little in the way of roles, access control, or a workflow model, and has some odd magic pages that live in the same namespace as everything else despite being different.

SharePoint, by contrast, is integrated with everything in Office, is a great success where the MS Kool-Aid is viewed as tasty, uses role-based security and a workflow, and seems to be generally reviled elsewhere.

No better illustration. The only thing that could improve that would be if SharePoint has a talking paperclip I’ve missed.

Tags: , , , , , , , , , ,

Comments

More on the ACT EVACS E-Voting System

Voting: Nathan Cochrane mailed in some great tidbits about the ACT EVACS e-voting system. (thanks!)

First off, this Debian-news posting notes some snippets from an Age article by Nathan; Here’s some longer excerpts. It features some great quotes: ‘the only platform that provided robustness and voter confidence was GNU Debian Linux, with all source code released under the General Public License (GPL).’

And this one:

‘Classical voting systems, notably the Australian paper ballot, are designed precisely on such anti-trust grounds,’ Jones said. ‘We simply assume from the start that each and every participant in the system is a partisan with a vested interest in doing everything possible to help his or her favorite candidates.’

He said paper and pencil voting systems, such as that first used in Victoria in 1858, meet this test. Electronic voting does not.

This letter to LWN notes: ‘You might be interested to know that some of the work on this project is being done by ‘big name’ open source people, including Andrew Tridgell (aka Mr Samba), Dave Gibson (orionoco wireless LAN driver), Martin Pool (apache), and Rusty Russell (netfilter and other gross kernel hacks)’, and links to the code’s CVS repository!

It seems those guys performed the work on behalf of a Canberra open-source consultancy group, Software Improvements; Here’s the product brochure.

This posting to iRights gives a few more details.

It all looks like an excellent job all ’round, as far as I can see.

Tags: , , , , , , , , ,

Comments

More on the ACT EVACS E-Voting System

Nathan Cochrane mailed in some great tidbits about the ACT EVACS e-voting system. (thanks!)

First off, this Debian-news posting notes some snippets from an Age article by Nathan; Here’s some longer excerpts. It features some great quotes: ‘the only platform that provided robustness and voter confidence was GNU Debian Linux, with all source code released under the General Public License (GPL).’

And this one:

‘Classical voting systems, notably the Australian paper ballot, are designed precisely on such anti-trust grounds,’ Jones said. ‘We simply assume from the start that each and every participant in the system is a partisan with a vested interest in doing everything possible to help his or her favorite candidates.’

He said paper and pencil voting systems, such as that first used in Victoria in 1858, meet this test. Electronic voting does not.

This letter to LWN notes: ‘You might be interested to know that some of the work on this project is being done by ‘big name’ open source people, including Andrew Tridgell (aka Mr Samba), Dave Gibson (orionoco wireless LAN driver), Martin Pool (apache), and Rusty Russell (netfilter and other gross kernel hacks)’, and links to the code’s CVS repository!

It seems those guys performed the work on behalf of a Canberra open-source consultancy group, Software Improvements; Here’s the product brochure.

This posting to iRights gives a few more details.

It all looks like an excellent job all ’round, as far as I can see.

Tags: , , , , , , , , ,

Comments

E-Voting: ACT’s open-source e-voting system

Voting: I’ve pointed to this before, but I use taint.org partly as a searchable database of annotated bookmarks, so — for reference — here’s the Australian Capital Territory’s EVACS system, an entire, open-source e-voting system:

EVACS is the computer system that provides for electronic voting and electronic counting for ACT Legislative Assembly elections. It provides for counting according to the Hare-Clark electoral system rules set out in the Electoral Act 1992.

EVACS was written using Linux open source software to ensure appropriate transparency. A copy of the source code is available in a zip file (127 kb). The source code for the casual vacancy module is in a separate file (38 kb). For more information contact Software Improvements.

Still not perfect — it uses electronic ballot stations, instead of paper ballots — but it does support paper ballots. And it’s open source; note the keyword above — ‘appropriate transparency‘. They said it, not me ;)

Tags: , , , , , , , , , ,

Comments

Using a Web of Trust to stop spam

Spam: Been thinking about a distributed ‘web of trust’ approach to fighting spam.

Combine those with another key point — that we do not need PKI, crypto, or any other changes to identify senders in current SMTP — and it could be done today, I think.

Why we don’t need crypto to identify an SMTP sender

Every email message delivered via SMTP across the internet will contain these headers:

  • the From line
  • one or more Received headers

Traditionally, whitelisting uses just the From line, which is vulnerable to spoofing. SpamAssassin used this up to version 2.3x. Spammers started spoofing mails where ‘From’ was the same as ‘To’, and since most people had themselves in the whitelist, that worked. boo.

In 2.3x or 2.4x, we added code to extract the IP addresses from the Received headers, and use a combined token — ( from_address, ip_address ) — as the sender’s address.

(In fact, we use just the top 24 bits of each IP to deal with situations like DHCP or dialup pools, where a relay may get a different IP every now and again. That’s close enough, at least.)

This is much harder to forge without doing a full-scale TCP spoofing attack; which is why the SpamAssassin auto-whitelist generally works well.

So basically, to identify someone strongly enough to provide a spam fix in plain old vanilla current SMTP, gen up a string containing their ‘From’ address, along with all the /24 masks of the IP addresses found in the ‘Received’ headers.

Remove your relays’ IP addresses, and you have an unspoofable ID for that person’s SMTP traffic. Any spammer who wants to spoof that, will have to compromise their mail server (or a server in the same /24). That’s not cost-effective for spamming.

Note that whitelisting based on that is effectively what the SpamAssassin auto-whitelist does. But for that to be more useful than the AWL, it has to extend over the internet to those people your friends haven’t corresponded with yet; ie. it’s got to be distributed.

(If you would like to comment on this scheme, I’d prefer if you could post comments at this QuickTopic forum.)

Tags: , , , , , , , , ,

Comments

Using a Web of Trust to stop spam

Been thinking about a distributed ‘web of trust’ approach to fighting spam.

Combine those with another key point — that we do not need PKI, crypto, or any other changes to identify senders in current SMTP — and it could be done today, I think.

Why we don’t need crypto to identify an SMTP sender

Every email message delivered via SMTP across the internet will contain these headers:

  • the From line
  • one or more Received headers

Traditionally, whitelisting uses just the From line, which is vulnerable to spoofing. SpamAssassin used this up to version 2.3x. Spammers started spoofing mails where ‘From’ was the same as ‘To’, and since most people had themselves in the whitelist, that worked. boo.

In 2.3x or 2.4x, we added code to extract the IP addresses from the Received headers, and use a combined token — ( from_address, ip_address ) — as the sender’s address.

(In fact, we use just the top 24 bits of each IP to deal with situations like DHCP or dialup pools, where a relay may get a different IP every now and again. That’s close enough, at least.)

This is much harder to forge without doing a full-scale TCP spoofing attack; which is why the SpamAssassin auto-whitelist generally works well.

So basically, to identify someone strongly enough to provide a spam fix in plain old vanilla current SMTP, gen up a string containing their ‘From’ address, along with all the /24 masks of the IP addresses found in the ‘Received’ headers.

Remove your relays’ IP addresses, and you have an unspoofable ID for that person’s SMTP traffic. Any spammer who wants to spoof that, will have to compromise their mail server (or a server in the same /24). That’s not cost-effective for spamming.

Note that whitelisting based on that is effectively what the SpamAssassin auto-whitelist does. But for that to be more useful than the AWL, it has to extend over the internet to those people your friends haven’t corresponded with yet; ie. it’s got to be distributed.

(If you would like to comment on this scheme, I’d prefer if you could post comments at this QuickTopic forum.)

Tags: , , , , , , , , ,

Comments

Decent C String APIs

meanwhile, back in C-land…

strlcpy() - a replacement for strcpy() and strncpy(), with some very nice performance figures.

I usually use snprintf() to do this, but even that has differint semantics between platforms which needs workarounds. Plus the perf numbers regarding strlcpy() are nice. Plus it’s BSD-licensed. (Found via Linux Weekly News.)

In passing, it’s worth noting that strncpy() imposes a pretty hefty performance hit (4x - 10x in tests there), due to a wierd specified behaviour; it NULs out unused parts of the buffer! ouch.

See also MS’ strsafe APIs. However, the code for that is available only on Windows, which makes it pretty much useless for most C code I’d be writing, and they note ‘performance hits’.

Tags: , , , , , , , , ,

Comments

More SCO: the Vegas show in full

a must-read: Bruce Perens posts and then demolishes the Las Vegas slideshow comprehensively, demonstrating that one of the code snippets SCO showed did in fact date from 1973, not 1979; and the other snippet was a clean-room reimplementation based on the published specification for the Berkeley Packet Filter, and the SCO code most likely came from the BSD-licensed implementation.

That raises two points: 1. the SCO ‘pattern-recognition team’ need to go back to Google school; 2. why didn’t the SCO implementation of the BPF code maintain the legal copyright attribution text it was supposed to include, so they would have noticed this when out ‘recognising’ ‘patterns’?

I’m looking forward to this getting to court eventually…

Tags: , , , , , , , , ,

Comments

The Irish 419 scam

FROM: UNIVERSAL STAKES LOTTERY, IRELAND. (forwarded by Rick Kleffel on the forteana list)

SCOvEveryone: so SCO showed some ‘evidence’ of code-copying from SCO to Linux — problem is, it’s code from UNIX v7, written around 1978/79; the code was released in BSD UNIX, rereleased by SCO/Caldera themselves under a BSD license later, and versions appear in textbooks under public domain. In other words, the SCO ‘pattern analysis’ team who found this ‘copied code’ didn’t realise that this source had been released long ago — even by their own company, no less. ho hum, good luck prosecuting based on that. next!

Blogs: Malte, one of the SpamAssassin dev team, now has a weblog too — and with a better translation of the ‘W32.Blaster caused the blackout’ theory too. ;)

From: “James” (spam-protected)
Date: Mon Aug 18, 2003 4:15:40 AM US/Pacific
To: (spam-protected)
Subject: Congratulation! ( Please acknowledge this mail asap)

FROM: UNIVERSAL STAKES LOTTERY
IRELAND. REF NUMBER: 014/060/532 BATCH NUMBER: 762901-PCD03

Sir/Madam,

We are pleased to inform you of the result of the Lottery Winners International programs held on the 3rd of July, 2003. Your e-mail address attached to ticket number 27522465896-6453 with serial number 3772-554 drew lucky numbers 7-14-18-23-31-45 which consequently won in the 2nd category, you have therefore been approved for a lump sum pay out of 2,000,000 (EUROS ) (TWO MILLION EUROS)

CONGRATULATIONS!!!

For security purpose and clarity, we advise that you keep your winning information confidential until your claims have been processed and your money remitted to you. This is part of our security protocol to avoid double claiming and unwarranted abuse of this program by some participants. All participants were selected through a computer ballot system drawn from over 20,000 companies and 30,000,000 individual email addresses and names from all over the world. This promotional program takes place every year. This lottery was promoted and sponsored by eminent personalities like the Sultan of Brunei. We look forward to your active participation in our next year USD50 million slot. You are requested to contact our clearance office to assist you with the claim and transfer of your winnings fund into your instructed account by acknowledging the receipt of this mail with the email address below.

Email address: (spam-protected)

Note that, all winnings must be claimed not later than one month. After this date all unclaimed funds will be null and void.

Please note in order to avoid unnecessary delays and complications, remember to quote your reference number and batch numbers in all correspondence. Furthermore, should there be any change of address do inform our agent as soon as possible. Congratulations once more and thank you for being part of our promotional program. NOTE: YOU ARE AUTOMATICALLY DISQUALIFIED IF YOU ARE BELOW 18 YEARS OF
AGE.

Sincerely yours,

James Clark.

(Lottery Coordinator)

Tags: , , , , , , , , ,

Comments

Linux and MS: WinCE now customizable

Whoa: ‘This spring, Microsoft dropped the price of Windows CE and completely opened its embedded operating system to developers, allowing them for the first time to not only view and modify CE, but also sell products that incorporated the customized code.’

Really? So WinCE developers can modify and then rebuild and sell WinCE with code changes? That’s a big deal. It’s kind of unavoidable, though. That close to the metal is virtually impossible without source IMO.

Tags: , , , , , , , , ,

Comments

more on SCO v. IBM: ‘All your base are belong to us’

Ben forwards a link to this Byte article, SCO: All your base are belong to us. His commentary:

One day I’ll have a blogtastic dalymount.com, but for the moment, have you seen this priceless interview, in which SCO goes over the edge into complete barking insanity?

‘We believe that UNIX System V provided the basic building blocks for all subsequent computer operating systems, and that they all tend to be derived from UNIX System V (and therefore are claimed as SCO’s intellectual property).’

His emphasis. But let’s face it, he’s emphasising the right part ;)

So they now think they are owed money by every modern OS: that includes FreeBSD, Windows, Apple, presumably QNX, etc. etc. Linux was just the easiest one to start with, since the source is available and IBM (with their deep pockets) are closely allied with it. MS have already paid up for a SCO license, although many commentators see this as a means to support SCO in their anti-UNIX lawsuits.

In more detail, SCO claim to have full IP rights to several major components of any high-spec OS:

  • JFS (Journalling File System).
  • NUMA (Non Uniform Memory Access).
  • RCU (Read-Copy Update).
  • SMP (Symmetrical Multi-Processing).

Let’s pick one there: RCU in Linux seems to have originated (at a glance) from code developed by Sequent for their DYNIX/ptx UNIX, which was an AT&T UNIX System V-based OS. Sequent ran into trouble, and were bought out by IBM. Later, patches to implement RCU were submitted by IBM from Sequent’s code.

SCO now owns the AT&T UNIX System V IP; therefore SCO owns the RCU code in Linux — even though Sequent developed it independently, on top of the System V base, as far as I can see. Hey, that’s even more ‘viral’ than the GPL — at least the GPL tells you in advance what mistakes you’d have to make for this to happen! ;)

In other words, it seems their POV is that, if any code came anywhere near other code that may have been part of the original AT&T codebase, it’s now tainted with SCO’s own ‘viral license’. Absolutely insane.

It’s unclear exactly how ‘all subsequent computer operating systems’ also infringe this viral license, but SCO reckon they do.

In the meantime, they don’t seem to have realized that these kinds of over-broad claims are not looked on favourably under EU law; while they make cartooney threats in the US, they open themselves up to all sorts of anti-trust-type claims elsewhere in the world. But then, at this stage I don’t think they plan to actually offer any products, or operate as a software company, so they probably don’t really care about that.

To really muddy the waters, an ex-SCO employee has recently made allegations that SCO copied code from the GPL’d Linux kernel into their UnixWare product.

Ah, fireworks. Anyway.

For a kinder, gentler form of total insanity, check out the guidelines for forming ‘inexplicable mobs’ in Manhattan — via bb. Totally cool.

Tags: , , , , , , , , ,

Comments

SCO’s strong-arm tactics

In case you missed it — SCO’s letter to Linux customers. Executive summary:

  • open-source code development methodology bashing, to start with
  • SCO will ’suspend their own Linux-related activities’, whatever they were
  • all users of Linux are vaguely threatened in a ‘cartooney’ fashion
  • ‘Similar to analogous efforts underway in the music industry, we are prepared to take all actions necessary to stop the ongoing violation of our intellectual property or other rights.’

Classy! And a bonus good point from a comment on this LJ article: ‘According to this article, SCO Linux 4.0 contains version 2.4.19 of the Linux kernel. … By the act of distributing the Linux 2.4.19 kernel, SCO has irrevocably released any and all of their intellectual property present in the 2.4.19 kernel under the (terms of the) GPL.’

Tags: , , , , , , , , ,

Comments

SARS genome decoding ‘couldn’t have been done without mail’

just got back from a super-quick booze-soaked weekend visit to Ben in SF. It was so good to visit a city once again, and get the opportunity to paint the town red, hit the bars, eat in plentiful cheap restaurants, and generally enjoy city life (which I’ve been missing massively since the move from Dublin). But now back in post-suburban Irvine to cope with the hangover.

Also got to meet up with Komal, one of my co-workers up there — which was cool. Unfortunately it was a super-speedy weekend whistle-stop tour though, so having a good social meet-up with all the guys will have to wait until the next visit. ;)

Net: ‘The Canadian scientists who broke the genetic code for SARS … say they couldn’t have done it without the Internet. … The key to that collaboration was ordinary e-mail‘.

It also turns out the ProMED mailing list was the central point at which SARS reports were collated in the early stages, even despite evasion and cover-up by the Chinese state.

So there you go — as usual, SMTP is the killer app — or in this case, a life-saving app! All the more reason to figure out ways to deal with spam and return SMTP to its top spot in the protocol pantheon.

Good thing the FTC Spam Forum went so well, then. Sounds like there was unprecedented agreement between the non-spam folks, clear understanding of the issues by quite a few of the Washington denizens, and maybe even some good footage of the other side digging holes for themselves.

Health: US, Asian Airlines Disagree on SARS. Me, I just wish the airlines would stop being so bloody cheap, and bring in more fresh air rather than recirculating. ;)

Date: Sun, 04 May 2003 12:20:16 -0400
From: STEPHEN JONES (spam-protected)
To: (spam-protected) (spam-protected)
Subject: Internet is a good thing says Steve Jones clone

Internet played a key role in decoding SARS genome, scientists say

DENNIS BUECKERT

OTTAWA (CP) - The Canadian scientists who broke the genetic code for SARS just weeks after the disease appeared say they couldn’t have done it without the Internet.

Scientists from the Michael Smith Genome Sciences Centre of the B.C. Cancer Agency say their achievement relied on rapid communication with scientists around the world. The key to that collaboration was ordinary e-mail, said Steven Jones of the Vancouver-based research agency in a teleconference Thursday sponsored by Science magazine.

“Within a day of us having a press release announcing our participation in the sequencing we had an amazing amount of e-mail from scientists all around the world,” Jones said.

As soon as the sequence was decoded, the B.C. researchers posted it on the Internet.

“People were, within minutes of that, able to download the sequence and analyse it in their own laboratories and their own computers,” Jones said.

“The Internet has had a profound impact on how this data has been shared and how scientists have collaborated.”

A short time later, researchers at the Atlanta Centers for Disease Control published the sequence of a coronavirus taken from another SARS patient.

The genetic coding for the two viruses were virtually identical, boosting confidence that the coronavirus was in fact the causal agent.

Now both sequences are posted on the World Wide Web for the benefit of researchers in many countries racing to find a reliable test for SARS, and a vaccine to prevent it.

Scientists say the speed of the decoding was amazing.

The first reports of the new disease came from China in November, and on March 13 cases were reported in Toronto and Vancouver. The sequences were posted on the net on April 15.

By contrast, it took years to identify the agents behind diseases like AIDS and hepatitis C.

Mel Crajdon of the B.C. Centre for Disease Control said all evidence points to the coronavirus as being the cause of SARS, despite some seemingly contradictory findings.

Earlier this week Frank Plummer, who heads the National Microbiology Laboratory in Winnipeg, said he was puzzled by the number of people who show evidence of the SARS coronavirus but not symptoms of the disease.

Crajdon suggested the apparent anomaly is due to imperfect understanding of how the disease presents itself, as well as lack of reliable tests for the presence of the virus.

“I’m not surprised by the results that have been obtained to date and I think that they will rapidly improve,” he said.

More than 5,400 cases of SARS have been diagnosed worldwide, with at least 394 deaths. In Canada, there have been 23 deaths, all in the Toronto area.

  • - -

On the Net:

SARS sequences: http://sciencemag.org/features/data/sars

SARS data: http://aaas.org

SARS Comments: http://eurekalert.org

Tags: , , , , , , , , ,

Comments

(Untitled)

The Enigma story, and the misattributions of credit:

In U-571, Hollywood gave the credit for the Enigma code-cracking heroics of World War Two to the Americans. In the British thriller Enigma, out today, the praise is given to the English. Now, if a protest from the Polish embassy in London is to be believed, it was the Poles that done it after all.

From what I’ve read, the Polish cryptographers are certainly missing out on a lot of the credit they’re rightly due.

Date: Fri, 28 Sep 2001 12:29:35 +0100
From: “Tim Chapman” (spam-protected)
To: forteana (spam-protected)
Subject: UK accused of movie history revisionism

http://film.guardian.co.uk/News_Story/Exclusive/0,4029,559785,00.html

Enigma deepens as Poles claim code-cracking breakthrough

Friday September 28, 2001

In U-571, Hollywood gave the credit for the Enigma code-cracking heroics of world war two to the Americans. In the British thriller Enigma, out today, the praise is given to the English. Now, if a protest from the Polish embassy in London is to be believed, it was the Poles that done it after all. The statement claims that Polish intelligence experts captured the Enigma machine on which the Germans conducted all their most secret cipher traffic before the war had even begun, and later presented this to the Allied forces. The statement quotes a Professor M.R.D. Foot as claiming that: “The most important service the Poles ever rendered to the anti-Nazi cause was something they did before the war had even begun.” An accompanying missive from the Federation of Poles in Great Britain adds that: “Mathematicians of the Polish Intelligence Service were the first to
break the Enigma code. In July 1939 passed over to British Intelligence a copy of the Enigma machine and the fruits of their work done in breaking the code in the years 1932-1939. This work greatly assisted the Bletchley Park code breakers and contributed to the Allied victory in world war two.” The Polish authorities are particularly annoyed with Enigma’s depiction of a traitorous Polish officer at Bletchley Park, the wartime headquarters of code-cracking intelligence, who works as a spy for the Nazis. The statement insists that no Pole ever worked at Bletchley Park. “Obviously we feel that this is a gratuitous slur on Poles who fought side by side with their British allies.”

———————— Yahoo! Groups Sponsor ———————~–> FREE COLLEGE MONEY CLICK HERE to search 600,000 scholarships! http://us.click.yahoo.com/ujOgTC/4m7CAA/ySSFAA/7gSolB/TM ———————————————————————~->

To unsubscribe from this group, send an email to: (spam-protected)

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

Tags: , , , , , , , , ,

Comments