A fishy Challenge-Response press release

I have a Google News notification set up for mentions of “SpamAssassin”, which is how I came across this press release on PRNewsWire:

Study: Challenge-Response Surpasses Other Anti-Spam Technologies in Performance, User Satisfaction and Reliability; Worst Performing are Filter-based ISP Solutions

NORTHBOROUGH, Mass., July 17 /PRNewswire/ — Brockmann & Company, a research and consulting firm, today released findings from its independent, self-funded “Spam Index Report– Comparing Real-World Performance of Anti-Spam Technologies.”

The study evaluated eight anti-spam technologies from the three main technology classes — filters, real-time black list services and challenge- response servers. The technologies were evaluated using the Spam Index, a new method in anti-spam performance measurement that leverages users’ real-world experiences.

[…] The report finds that the best performing anti-spam technology is challenge-response, based on that technology’s lowest average Spam Index score of 160.

[…] Filter – Open Source software-(Spam Index: 388): This technology is frequently configured to work in conjunction with PC email client filters. The server adds * * SPAM * * to the subject line so that the client filter can move the message into the junk folder. This class of software includes projects such as ASSP, Mail Washer and SpamAssassin, among others.

The “Spam Index” is a proprietary measurement of spam filtering, created by Brockmann and Company. A lower “Spam Index” score is better, apparently, so C/R wins! (Funny that. The author, Peter Brockmann, seems to have some kind of relationship with C/R vendor Sendio, being quoted in Sendio press releases like this one and this one, and providing a testimonial on the Sendio.com front page.)

However — there’s a fundamental flaw with that “Spam Index” measurement, though; it’s designed to make C/R look good. Here’s how it’s supposed to work. Take these four measurements:

  • Average number of spam messages each day x 20 (to get approximate number per work-month)
  • Average minutes spent dealing with spam each day x 20 (to get approximate minutes per work-month)
  • Number of resend requests last month
  • Number of trapped messages last month

Then sum them, and that gives you a “Spam Index”.

First off, let’s translate that into conventional spam filter accuracy terms. The ‘minutes spent dealing with spam each day’ measures false negatives, since having to ‘deal with’ (ie delete) spam means that the spam got past the filter into the user’s inbox. The ‘number of trapped messages’ means, presumably, both true positives — spam marked correctly as spam — and false positives — nonspam marked incorrectly as spam. The ‘number of resend requests last month’ also measures false positives, although it will vastly underestimate them.

Now, here’s the first problem. The “Spam Index” therefore considers a false negative as about as important as a false positive. However, in real terms, if a user’s legit mail is lost by a spam filter, that’s a much bigger failure than letting some more spam through. When measuring filters, you have to consider false positives as much more serious! (In fact, when we test SpamAssassin, we consider FPs to be 50 times more costly than a false negative.)

Here’s the second problem. Spam is sent using forged sender info, so if a spammer’s mail is challenged by a Challenge/Response filter, the challenge will be sent to one of:

  • (a) an address that doesn’t exist, and be discarded (this is fine); or
  • (b) to an invalid address on an innocent third-party system (wasting that system’s resources); or
  • (c) to an innocent third-party user on an innocent third-party system (wasting that system’s resources and, worst of all, the user’s time).

The “Spam Index” doesn’t measure the latter two failure cases in any way, so C/R isn’t penalised for that kind of abusive traffic it generates.

Also, if a good, nonspam mail is challenged, either

  • (a) the sender will receive the challenge and take the time to jump through the necessary hoops to get their mail delivered (“visit this web page, type in this CAPTCHA, click on this button” etc.); or
  • (b) they’ll receive the challenge, and not bother jumping through hoops (maybe they don’t consider the mail that important); or
  • (c) they’ll not be able to act on the challenge at all (for example, if an automated mail is challenged).

Again, the “Spam Index” doesn’t measure the latter two failure cases.

In other words, the situations where C/R fails are ignored. Is it any wonder C/R wins when the criteria are skewed to make that happen?

This entry was posted in Uncategorized and tagged , , , , , , . Bookmark the permalink. Both comments and trackbacks are currently closed.


  1. Posted July 19, 2007 at 14:28 | Permalink

    Related to innocent 3rd parties receiving unsolicited C/R messages, spamcop.net will accept C/R messages as spam evidence when received and submitted by the innocent 3rd parties. This can result in the system using C/R to be IP blacklisted, the consequence of means their legitimate outbound mail will probably be blocked or receive a high score from SpamAssassin. The C/R system will then have to get themselves delisted and in order to do that, will probably have to give up C/R and prove they don’t send spurious messages.

    Personally I never answer C/R messages and rank them right up there as spam with all those anti-virus notices telling you that your mail to Alice is infected, but you don’t know who the f..k is Alice.

  2. Posted July 19, 2007 at 14:32 | Permalink

    “Comparing Real-World Performance of Anti-Spam Technologies.”

    Whats wrong with good old precision and recall? They’ve been used for decades to compare real-world performance for retrieval and classification systems. These kind of made-up evaluations really annoy me! I’m guessing that if they could show better P/R then they wouldn’t have needed to make up a new evaluation metric.

  3. Manni Heumann
    Posted July 19, 2007 at 14:54 | Permalink

    I love it how they cluster the anti-spam solutions by technology and then manage to come up with “Filter – Open Source software”. Huh?

    That press release did one bit of good: I finally looked up the word “to leverage” in a dictionary. I’ve often seen it; always in marketing speak. Problem is: I still have no idea how they use it here.

  4. Posted July 19, 2007 at 15:04 | Permalink

    Aidan: yep! Precision and recall, in anti-spam, are basically FP% and FN% inverted:


  5. Posted July 19, 2007 at 15:42 | Permalink

    In fact, when we test SpamAssassin, we consider FPs to be 50 times more costly than a false negative.

    I would agree with this assessment. In Exchange Hosted Services, we get some complaints amount about the amout of spam hitting people’s inboxes. However, when somebody puts in a bad rule that causes excessive false positives, we really hear about it. Over and over again, from multiple sources.

  6. Posted July 19, 2007 at 17:02 | Permalink

    You missed another failure case:. Should the moronic solution require you to confirm on a site blocked by one of the big corp-url-filtering lists such as secure computing. Or the one I always hit… the moronic app is on a non-default port blocked by any corp-firewall.

  7. Seth
    Posted July 19, 2007 at 19:38 | Permalink

    I always answer challenges, when they’re for mail I didn’t send.

    If someone else wants me to filter his spam for him correctly he should start by offering to pay me.

  8. Posted July 19, 2007 at 19:40 | Permalink

    Thanks for the analysis. We plan to do more of these user-oriented studies and can tune the Spam Index to be more weighted on the f-p front at that time.

    The basic idea was to take the debate to the user’s inbox, give them a tool to see how they compare with others, and to score the performance of anti-spam technologies. It is undeniable that C-R scored better from a users’ perspective – fewer inappropriate, irrelevant, anonymous, bulk, spam messages in the inbox, and less time spent dealing with spam. Also, 50% more C-R users said they were ‘very satisifed’ with the email experience than hosted processors, the next highest scoring technology.

    Correction on a few points:

    The ‘minutes spent dealing with spam each day’ measures false negatives, since having to ‘deal with’ (ie delete) spam means that the spam got past the filter into the user’s inbox.

    This element also includes the time the user spends searching the junk folder for good email, so this would also measure false-positives.

    Number of trapped messages last month

    This should read number of trapped good messages last month, in which case it further weights the false-positive. In the calculator at http://www.brockmann.com and in the actual report we have made this a lot clearer.

    The concern about the volume of challenge messages is irrelevant from the perspective of the user. Business users (the folks in the study) will respond to the challenge especially if there’s an economic consequence of not doing so, like business that didn’t appear, and especially if they originated the message (which is precisely the point). So ignoring challenges might work for you, but in many people’s lines of business that’s unacceptable. Some C-R solutions work to minimize the # of challenges by doing grey listing, header checking and anti-virus before the challenge is sent, and even monitoring the outbound message stream. In future studies, we plan to dive into the use of email as a first-contact method in buyer-initiated dialogs with vendors.

    The author, Peter Brockmann, seems to have some kind of relationship with C/R vendor Sendio…

    How exactly are my comments about Sendio product all that different than Gartner’s comments or Yankee’s comments in other vendors’ press releases?

    Thanks for the analysis and comments on our tool.

  9. Posted July 19, 2007 at 20:28 | Permalink

    If a C-R user writes to [email protected], and it gets forwarded to [email protected],com, then Joe gets challenged when he writes back. Meanwhile, he’s gone home, someone else is taking the “[email protected]” mail, and example.com’s mail server probably spam-filtered the challenge anyway. So the user blames example.com’s “lousy support” for the problems caused by his or her own C-R system.

    I see lots of “I just started using C-R, it’s great” posts, but no “I’ve been using C-R for years and it’s great” posts. C-R is something that you try and give up on. Or, in my case, watch other people try and give up on.

    Maybe I should start putting this in .muttrc…

    my_hdr: X-Challenge-Response: refused

  10. Seth
    Posted July 19, 2007 at 21:21 | Permalink

    The concern about the volume of challenge messages is irrelevant from the perspective of the user.

    At least up until the recipients of those challenges start blocking the user for spamming them.

  11. Posted July 20, 2007 at 13:02 | Permalink

    Peter: thanks for commenting.

    The concern about the volume of challenge messages is irrelevant from the perspective of the user. Business users (the folks in the study) will respond to the challenge especially if there’s an economic consequence of not doing so, like business that didn’t appear, and especially if they originated the message (which is precisely the point). So ignoring challenges might work for you, but in many people’s lines of business that’s unacceptable.

    That argument works for some situations, but there are others where the business benefits of a transaction go only one way — therefore it fails. I’ll give you a common situation where I ignore challenges, which would be a loss of important mail for a business user, to demonstrate this.

    Occasionally, a user on a mailing list will ask for help with something, and I’ll respond to them with the solution they need — only to then receive a challenge from their C/R filter. Now, I never confirm those challenges, so the response they were hoping to receive, is lost.

    Here’s the thing: the message is important for them to receive, possibly with economic consequences (downtime costs money etc.), but it’s not important for me to ensure they receive it. I’ve already spent my time to help them with their problem for free, there’s no ‘economic consequence’ to me, and I’m damned if I’m going to spend time filtering their mail for them. ;)

    (btw, whitelisting the To: addresses of outbound mail wouldn’t help here, since the to addr was a mailing list, not my personal address.)

    Anyway, my overall point is that focussing the debate on the “user’s inbox” ignores the overall picture, including everyone else’s mailbox, which is where C/R fails.

  12. Brian McNett
    Posted July 20, 2007 at 17:42 | Permalink

    Anyway, my overall point is that focussing the debate on the “user’s inbox” ignores the overall picture, including everyone else’s mailbox, which is where C/R fails.

    This is a point which MANY so-called anti-spam businesses completely neglect. Few of them consider the effect their product has on the vast majority of people who are NOT using it, and whether those effects are out of proportion to the benefits to the user.

    As an anti-spam method, C/R fails this test on multiple levels.

    Only the low adoption rate of C/R has prevented it from becoming a problem for ISP abuse desks (and hence not yet resulted in terminations of C/R users for network abuse). No doubt the efforts of popularizers such as Mr. Brockman (I’m unable to see his study as anything other than a blatant attempt to popularize C/R), will result in an increase in adoption. One can only hope that the subsequent termination of a C/R user for AUP violations will put a stake through the heart of this bad idea.

  13. David
    Posted July 23, 2007 at 21:39 | Permalink

    A simple change to challenge response system incorporated to the basic SMTP operation would virtually eliminate spam. The current RFC 2821 for SMTP servers allows spammers to spoof their return IP address on the Internet with a modified client. The SMTP server should generated a small random number and use it as a session ID that has to precede the body of the Email message coming from the Client. This forces the Client to be at the real return IP address that the SMTP server sees for the Client in order to add the session ID to its message. This Real return IP address, forwarded along in the header of the message will allow effective blacklisting of the spamming IP address / URL by traditional anti spam software. The Downside is all SMTP servers and Email clients would need an update or patch to implement this.

  14. Seth
    Posted July 23, 2007 at 21:44 | Permalink

    IP spoofing has not been feasible this century. The IP address of the machine sending me spam is always available.

  15. Posted July 27, 2007 at 18:32 | Permalink

    More at richijennings.com, including the mysterious Wikipedia connection.

  16. mouss
    Posted July 29, 2007 at 16:47 | Permalink

    Thanks Justin.

    you can add one case to the last list: (d) the challenge is sent to an innocent who gets annoyed and validated the challenge. Once this is done, the forged sender addres is whitelisted by the CR system, and its users will get more spam.

  17. Posted July 30, 2007 at 08:13 | Permalink

    It seems to me that the major problem is that it’s too easy to forge the “from” address. If you were unable to forge the “from” address, then at least you stop the problem where a C/R system would end up becoming a spammer itself. You’re stiill left with the problem of people not bothering to respond to the challenge, but if your business doesn’t care about that, then C/R would be ideal… that may be a big “if” though :-)

  18. Posted July 30, 2007 at 10:38 | Permalink

    Dean —

    yes, that would help. I’d certainly be friendlier to a C/R implementation that only challenged SPF, Domain Keys or DKIM-certified mail messages, and no others. As far as I know, none of them do this.

  19. Posted July 30, 2007 at 10:57 | Permalink

    Justin, a few of them are doing SPF lookups now. But it doesn’t really help the problem — bots can simply choose a correct sender to forge, based on which network they’re on.

    For example, if they’re on an IP belonging to example.net, they can forge [email protected] — a SPF lookup would succeed. This may require using the network’s smarthost, but several bots are doing this now, to combat port25 blocking.

    The good news is that there are some C/R vendors that are now quietly backing away from C/R — Sendio is one. These vendors claim to be adding layers of “conventional” spam filtering techniques, to avoid sending inappropriate challenges. I spoke to another such vendor recently (but under NDA, so I can’t reveal who). I’ve also heard that BoxBe has placed conventional spam filtering in front of its C/R engine, in order to minimize backscatter.

  20. Seth Breidbart
    Posted July 30, 2007 at 18:26 | Permalink

    If I get C/R blowback because a bot on a user of my ISP forged my account, then I could complain to my ISP who would have more incentive than usual to take action, because it’s their customer complaining.

    If a bot on my own system caused me to get C/R challenges, then I deserve them.

  21. Posted July 31, 2007 at 00:44 | Permalink

    Seth: that would my thought also. All the more incentive to secure your own network. Besides, it should be much easier to filter spam on a smarthost than on individual clients — that is, if a client is using the smarthost to send spam, you might see lots of messages “from” different people, but the same IP.

    It may be more work for the smarthost (who may not otherwise filter outbound emails) but the filtering would not be so complex. Just looking at the “from” and IP address; you would not nessecarily have to filter the actual content.

  22. Posted July 31, 2007 at 05:03 | Permalink

    I agree with Dean Harding since due to the nature of internet spam, in most cases there is no reliable way to detect if the sender really sent an email or if it was forged. Thus very often C/R emails are sent to people who have no intent to communicate with the recipient and never have sent an email.

    This kind of false C/R email is called C/R Spam and results often in blacklisting the sending mail server. This can not be avoided.

    C/R Systems are also highly support intensive because of the frequent requests of users (Senders and Recipients) why the sender has received the challenge or why the recipient did not receive the email.

    Many people consider C/R messages as Spam (or simply do not understand them) and delete them, instead of answering the challenge and are therefore never be able to send email to the C/R protected recipient.

    It’s also usual that you send a message and just go out to do something else. Sometimes the message you sent stay queued for some minutes and then again takes some more minutes until you get back the C/R and I guess in some situations you just can’t wait for it.

    Imagine a friday afternoon that you just go home after sending a message. Some C/R products are very amusing since they can delete your message after some hours when the C/R is not responded and the part I love is that some of them have the option to just delivery it to recipient when the C/R was not responded.

    It seems that I still have problems to understand why a good network administrator would even consider to use such system – and I’m not even talking about the waste of resources like bandwidth and machines usage – I guess it’s not necessary.

  23. John Foster
    Posted August 13, 2007 at 23:04 | Permalink

    I’ve been a C/R subscriber since 2003 for a personal email account and a part time business I run. From a C/R subscriber’s point of view it is an awesome system. I typically get one Nigerian scammer every 3-4 months and that’s it. Since SPAMbots won’t be able to respond to a CAPTCHA in the reply, those are never let through. The main benefit of a C/R system is not having to deal with 100 junk emails on a WAP phone or a PDA. Junk email is annoying enough on the desktop. Getting tazered by your Blackberry vibrating for no good reason is even more annoying.

    The ISP filter that GoDaddy uses is garbage and flags everything as SPAM even on its lowest setting. The advantage of C/R is that real humans can move themselves from the junk folder to your Inbox by filling out a CAPTCHA, meaning if they really want to talk to you then they will jump through the hoops.

    Hosted solutions such as MXLogic, Postini, and Message Labs will do better than something stand alone such as on the desktop (Outlook or Thunderbird filters or commercial addon’s like McAfee), or a single site SpamAssassin with no training, but for almost the same monthly price you will get less SPAM with a C/R system. For the same price the 96%-99% accuracy of a hosted system is unacceptable.

    I’ll stick with my C/R subscription since it works as advertised. I don’t consider it to be spamming since challenge emails are not commercial in nature and if you can find a lawyer that can successfully argue email is spam based on the recipients opinion and what mood they are in, you might want to have them read CAN-SPAM and the various state statutes to get a legal definition of SPAM. But first if you really believe that your opinion of what is/isn’t spam is more mighty than the law, you and I need to talk about a bridge I have in Brooklyn for sale.

  24. Seth Breidbart
    Posted August 13, 2007 at 23:17 | Permalink

    John Foster, if I get a challenge from you because some spammer forged my address, you will get the spam. You will also be reported as a spammer. If that results in email from your system being blocked, that’s what happens to spammers.

    CAN-SPAM doesn’t use or define the word “spam” (uppercase is a trademark of Hormel). I call unsolicited challenges spam because they are. I don’t care that you don’t consider your C/R challenges to be spam; read the Laws of Spam, every spammer defines spam as “not what I do”.

    If you feel my reports of your spamming are false, you have every right to complain. Nobody else has any obligation to listen to your complaints.

  25. Posted August 13, 2007 at 23:30 | Permalink

    So, John, do you deny that challenges to forged spam cause backscatter?

    Do you deny that sending backscatter will get your email server blacklisted?

    Do you deny that undelivered challenges cause a false positive problem for C/R users?

    A “yes” to any of these gets you an invitation to buy “London Bridge,” like Robert P. McCulloch — the joker in Lake Havasu City, AZ ;-)

  26. Posted August 13, 2007 at 23:42 | Permalink

    John, you don’t mention what kind of a business you run. Let me guess–super-sensitive car alarms that go off when the 6am garbage truck arrives?

  27. Posted August 14, 2007 at 01:33 | Permalink

    Hey Seth: While you’re at it accept my challenges too please. I get so few interesting spam (none actually) these days, it might do good to see the odd one once in a while… :-D

    John, your experience is consistent with my research about user experiences with email at http://www.brockmann.com. Some of the solutions that I’ve seen from Sendio and Vanquish and Reflexion to name 3 really work hard to avoid sending challenges, relying on other techniques to avoid spam first – outbound message flow management, greylisting, AV, Bayesian filters that score the goodness of a suspect message or intelligent addressing. It seems in practice that C/R is a useful, last resort method.

    Of course, every one of these commercial solutions have mechanisms for protected users to override outstanding challenges and accept messages and senders regardless of whether they responded to the challenge or not. Likewise, they can block any user with equal ease.

    Richi – you’re going to have to come up with another line of questioning. In the great volume of spam that users and ISPs are bombarded with, responsible challenge backsplatter (responsible meaning those solutions (like these three) that avoid sending challenges willy nilly) is such a small proportion of the junk the risk of blacklisting, I would think is quite low.

  28. John Foster
    Posted August 14, 2007 at 02:04 | Permalink

    Awesome, a bridge in London! Touché Rich!

    And for Don, I do IT security consulting, typically for government agencies and I’m also a DJ on the weekends. I don’t accept solicitations for business. If I have the bandwidth I will write a proposal and submit it to the entity that posted the RFP on their website. If I’m looking to DJ at a night club somewhere I will contact a venue or local booking promoters if the location and time is convenient to my schedule. Most of the time promoters pick a time or a city that just doesn’t fit so if I’m traveling I’ll find the promoters myself. In short there’s no reason to email me unless I’ve contacted you first or exchanged business cards with you, but I wouldn’t consider first contact spam :)

    Though all this talk of C/R, Hosted Filtering, and stand alone filtering is interesting, I’ll throw the topic of semi-closed systems such as Facebook and MySpace out to you guys. I have several friends who run their entire business (granted it’s entertainment business) from MySpace. They’ve totally checked out of the SMTP world. Good, Bad, Ugly?

  29. Posted August 14, 2007 at 04:32 | Permalink


    I’m on the anti-spam business for quite some time. We have really no problems to develop some C/R feature to our product. However, we will never do it for a very simple reason: C/R is a bad idea. To be quite frank with you, that kind of feature wouldn’t take more then a week to be fully implemented but there is not reason to do it. I fully respect your thoughts about C/R but I don’t think it will change my mind about it. In addition, I’m wondering about the traffic you should have on your server after all, you receive everything people send to you which alone is already a bad idea. A good system would save quite some bandwidth and CPU usage. And last but not least, if I get some tons of C/R spam (or whatever you call it) from your server be quite sure that you will blacklisted. ;)

  30. Manni
    Posted August 14, 2007 at 06:49 | Permalink
    I don’t consider it to be spamming since challenge emails are not commercial in nature and if you can find a lawyer that can successfully argue email is spam […] you might want to have them read CAN-SPAM and the various state statutes to get a legal definition of SPAM.

    Spam is not primarily a legal issue. That’s why spam fighters (and email users in general) don’t care too much about any definitions is US law. By your logic, you could argue that spam didn’t exist before CANSPAM was in effect. Spam is anything that ends up in my inbox that is unwanted.

  31. Posted August 14, 2007 at 14:39 | Permalink

    “Joh”, the background to the London Bridge comment was that Robert P. McCulloch thought he was buying Tower Bridge (the iconic, bascule drawbridge in London). John Rennie’s London Bridge was an altogether more plain affair.

    McCulloch had it shipped in pieces to Arizona where it was reassembled. Both he and Ivan Luckin—the chap who sold it to him—deny that it was bought in error… but they would say that, wouldn’t they?

    For extra credit: the Fergie Ferg connection.

  32. Posted August 14, 2007 at 14:48 | Permalink

    Peter, YES. As I said before, it seems like a very positive development that several vendors are backing away from C/R, now that they realize the (ahem) limitations of the technology.

    As I said earlier, I also have a client (via Ferris Research) who is well known as a C/R vendor who seems to be doing an excellent job of implementing state-of-the-art spam filtering in front of C/R to prevent the backscatter problem. Unfortunately, I still can’t disclose the name of the client.

    I eagerly await the day when my spamtraps and “real” email accounts stop receiving C/R backscatter. Hopefully the reformed vendors will make it easy for their customer to upgrade to the new versions, eh?

  33. Posted September 16, 2007 at 18:31 | Permalink

    Richi: I don’t think it’s a matter of

    backing away from C/R

    but a matter of tuning the solution to make it more useful and less wasteful of enterprise resources (storage and bandwidth, for example). Sendio has implemented silverlisting to force first time senders to attempt a resend at some incremental time, which stops plenty of message attempts since the spammer just moves on to the next target. Vanquish uses a reverse Bayesian approach to assess the probability of it being a good message. Reflexion Networks uses on-the-fly addressing to give users very fine controls of their unique addresses.

    These C/R implementations (and I’m sure there are others) are far more mature than you and Justin assume. All three make C/R an option for deployment and manage outbound flows to make sure that the people that users send to are automatically added to the accept list.

    My report about user’s experience (The Spam Index Report) points loud and clear that C/R users experience are more positive than users of any other anti-spam technology.

  34. Seth
    Posted September 17, 2007 at 01:59 | Permalink

    If using C/R is good for C/R users, and bad for the victims of the spam emitted by C/R users, how are C/R users different from other spammers? They get some value for themselves at the cost of taking it from other people without those other people’s consent.

    (This does not apply to C/R by 55x message, which does not backscatter innocent victims of forgery.)

  35. Posted September 20, 2007 at 15:18 | Permalink

    Seth, I’m still waiting for you to accept a backscattered message from me like you promised …. ;-)

    I think the ‘Backspatter’ argument is a red herring (smelly fish to distract people from the truth). Here’s the logic:

    1. My research shows that people are getting 11.5 spams a day on average despite the best efforts of spam filters. And if they’re 95% successful at removing spam, that means that their email inbox is a target for 11.5/0.05 = 230 spam/user/day.

    2. A recent study just completed (not yet published) shows that C/R users represent 5.6% of business email users. Well behaved C/R systems send out only 2-6% challenges with about 1% going to legitimate first time senders. The question is of the 2-6% how many are actually forged? My address hasn’t been forged by a spammer except to send a message to me from me.

    If the forged address is count is low (likely) say 1% then the probability of getting a backscatter message is 0.0000224 or 1 in 44,643 email.

    1. At the rate of 230 spam a day, that would be about once every 194 days. Of course honeypot operators are likely to be more vulnerable than others.

    So, although one can argue that C/R is unfair to forged address users, but from this math, it is trivially unfair to them and at the same time both correctly and completely unfair to spammers. I’d suggest that that is a very reasonable side-effect of the technology.

  36. Seth
    Posted September 21, 2007 at 00:51 | Permalink

    If you think me getting some spam from your C/R spam engine is “a very reasonable side-effect” then I’ll merely point out that you getting kicked off the Internet for spamming me (if you have a legitimate provider; else your provider getting widely blocked for allowing spamming) is an even better side-effect.

    Stealing from me is wrong. Period. I don’t care what your analysis claims.

    I’ve been getting dozens of backscatter (bounce, not C/R) messages per day for some time now. They wouldn’t be any less objectionable if they were C/R.

  37. Posted September 22, 2007 at 21:30 | Permalink

    A couple of comments were lost in the server move of this blog, so here they are, cut and pasted for posterity:

    from Chris:

    @peter: My address gets forged a lot ~4000 times a day by the look of my logs and when I get joe-jobbed that spikes at once server crippling levels. In both situations I don’t want CR’s.

    2-6% eh… To be honest you’re saying your CR solutions output is in proportion to your own spam preventions effectiveness. Gah!

    IMHO you can stick all the CR systems that aren’t part of a mailing list COI process where the sun don’t shine!

    And one from me:

    ‘If the forged address is count is low (likely) say 1% then the probability of getting a backscatter message is 0.0000224 or 1 in 44,643 email. At the rate of 230 spam a day, that would be about once every 194 days. Of course honeypot operators are likely to be more vulnerable than others.’

    Peter, I’d be happy to forward on all the backscatter mails I receive, to you, if you like. there’s a lot more than that ;)