Jeff asked:
What kinds of percentage of spam message header domains are showing up on SURBLs? I would imagine the hit rates might not be too high, so there may be a processing cost/benefit issue.
...and...
I'm puzzled why there would be FPs. Are hammers forging spam domains in their headers? That would seem bizarre if so.
I have to correct something... I misspoke. I **used** to use SURBLs for checking headers. I had forgotten that I had stopped doing so a few months ago because (1) too many FPs (for my admitted strict standards), and (2) I made enough great improvements in others parts of my filtering that I felt I could back off on the SURBL-checking of headers.
(I was just too tired to think straight about this in my last e-mail).
But, let me mention that the overall FP rate is still very, very low. It was like 1/200 FPs, or less. (but I'm guessing)
Most often, if a FP occurred, it was because an IP address used in a spammer's URL would, for whatever reason, also appear in the headers of legit messages.
Also, have you ever seen those e-mails where some guy e-mails ALL 90 of his friends using outlook? Every once in a while, such an e-mail would pass through my server where one of these friends would be an employee of a spamming organization... thus triggering the FP. Of course, these tended to be the more marginally listed domains of SURBL... not the Russian pill spammers, but it still happened on rare occasion.
I recall catching about 50 extra spams a day on my 10K messages/day server by checking the header against SURBL. Statistically, not that much, but every 1/2 percent counts for something and these were ones which, at that time, wouldn't have been caught otherwise.
From a processing perspective, I don't think it is that big a deal. What I
found to be really slow (that I also used to do but no longer do) is to convert domains to IPs and check these against spamhaus. The problem here is that some domains take a LONG time to convert to IP because of delays on that domain's DNS server. This method also caught about 50 extra spams per day... but at too high a processing cost.
I don't think that processing SURBL against headers was a big processing drain... but the FPs were too high for my very strict tastes. Still, it is a VERY good indicator of spam and might work well if integrated into a scoring system and not outright blocked for that alone.
Rob McEwen