Spamhaus says:
From: Rob McEwen rob@pvsys.com To: "'SURBL Discussion list'" discuss@lists.surbl.org Date: Saturday, August 14, 2004, 1:52:26 PM Subject: [SURBL-Discuss] FP Pattern for sbl-xbl.spamhaus.org
RE: FP Pattern for sbl-xbl.spamhaus.org
For a while now, my philosophy has been to use sbl-xbl.spamhaus.org to block at the connection level and not even allow these messages onto my server. Much of the remaining spam filtering is then done by SURBL-checking. However, more recently, I been testing samples of sbl-xbl.spamhaus.org blocked messages and I've noticed two things.
(1) more false positives than I would want to see (though still a very tiny, tiny percentage overall) get blocked by sbl-xbl.spamhaus.org
...and...
(2) those that ARE legitimate tend to be cases where a mistake was made and, by the next day (or later that same day), the offending IP is removed from sbl-xbl.spamhaus.org
Hmmm, this can happen. Also depends on the volume of mail he processes.
I'd be interested in if this is due to SBL or XBL hits. Both can produce FP's, but it brings up a debate as to what's an FP. If an IP has been found sending viruses or spam and is auto-listed by the XBL system (normally due to a compromised box), but that IP also sends non-bad email, it's not a false listing. We error on the side of stopping the 100,000 viruses being sent to users worldwide than to let the fewer legit emails pass.
However, I must admit, I'm drawing sweeping conclusions from very little sampling of data. Therefore, don't take my word for it...
If it's SBL, if he has a cousin in the Chinanet-CQ IP space, or in parts of Brazil Telecom's space, he'll probably see many more FPs as we do have large SBL listings. Also, if he gets email from people hosting on the cheap-spam-friendly networks like New Horizon, CET Networks or OC3, they get all IP space listed.
Rather, is this consistent with anyone else's experience with sbl-xbl.spamhaus.org? The reason I mention this is that, if my initial conclusions are true, there would then be a strong argument for "holding" sbl-xbl.spamhaus.org blocked mail and giving it a "second try" some hours later.
He sure could, many people do greylist with DNSBLs.
Also, if this is true, does anyone have a "feel" for exactly how long "bad" data stays on sbl-xbl.spamhaus.org before it gets removed?
No way to know. Most XBL listed IPs can be "self-removed", SBL mistakes that generate FPs are normally found out by us pretty quickly as either the blocked users or people who use us and like he, check their logs, let us know.
(Recognizing, of course, that SpamHaus is probably the most reliable and respected free RBL in existence and they rarely make mistakes in the first place).
:-)
Any thoughts or suggestions? Has anyone examined their sbl-xbl.spamhaus.org blocked messages lately?
On large ISP type corpses, we still have a tiny fraction of true FPs.
That being said, some places (with little volume or huge cheap bandwidth and lots of CPU) will just use SBL+XBL as a part of a SpamAssassin type formula. They won't toss on it being BL'd, but will want some other spammy trait to push it over the spam-score.
Rob McEwen
Thanks Jeff for the feedback from SpamHaus.
Also, I do agree with the philosophy that a little collateral damage from legitimate sources is O.K. if the network originally sending the spam is a known, flagrant, and unrepentant spam source. (How else are they going to be motivated to clean up their act?)
Given Raymond and other's suggestions, I have set up a new test. I now filter at the MTA level using list.dsbl.org. My next level filters using SURBL (on the message's body content only, of course). My third level is to SURBL filter on the Client's server IP address. Although, I haven't yet seen any traction on this one yet... I know... not "by the book" :)
Finally, (...and this is the point of this message...) my fourth level is to filter on the ClientIP using each SpamHaus feed separately. This fourth level then saves all blocked mail to its own folder and I'm manually checking these daily. (This is not as time consuming as it sounds because these are only those e-mails which made it past both DSBL and SURBL filtering, but then got blocked by SpamHaus).
Since I started this only yesterday, I have 26 SBLs and 68 XBLs, with ZERO FPs.
Of course, I have other levels of filtering (linguistic/heuristic) after these I've described...
I'll report back in about a week (or two) regarding how many spams/FPs SpamHaus has caught by then using this setup I described.
Rob McEwen
At 14:38 2004-08-15 -0400, Rob McEwen wrote:
Given Raymond and other's suggestions, I have set up a new test. I now filter at the MTA level using list.dsbl.org.
My personal experience is actually that list.dsbl.org generates more serious FPs than either SBL or XBL/CBL. They have a history of occasionally listing outgoing ISP mail servers and de-listing of those taking a couple of days. I have never seen that happen on XBL/CBL.
Finally, (...and this is the point of this message...) my fourth level is to filter on the ClientIP using each SpamHaus feed separately.
You might consider using SBL, CBL and OPM separately, rather than SBL and XBL separately. In my experience, there is/was a certain delay between entries being added/removed to/from the origin lists CBL and OPM, and those entries being mirrored to XBL. With newly introduced trojaned home computers, that slight delay might mean spam getting through or not.
Patrik
Hi Patrik,
Given Raymond and other's suggestions, I have set up a new test. I now filter at the MTA level using list.dsbl.org.
My personal experience is actually that list.dsbl.org generates more serious FPs than either SBL or XBL/CBL. They have a history of occasionally listing outgoing ISP mail servers and de-listing of those taking a couple of days. I have never seen that happen on XBL/CBL.
Pardon me ? Sorry, thats bull. DSBL only lists if a message gets confirmed Could you please subscribe how the above scenario would happen? Are you sure you are talking about DSBL? Not some other list? Please check the way how DSBL lists messages, its not like people add blocks manually or something. Its all done with a confirmation round. I really think you are mistaken and mean another RBL...
The only way a ISP mailserver gets added if its running a open proxy itself.
Do you have examples of this ?
I closely watch the admin and announce list of DSBL, i cannot recally anything like you are saying. You must be confused?
Bye, Raymond.
At 21:16 2004-08-15 +0200, Raymond Dijkxhoorn wrote:
Pardon me ? Sorry, thats bull. DSBL only lists if a message gets confirmed Could you please subscribe how the above scenario would happen? Are you sure you are talking about DSBL? Not some other list? Please check the way how DSBL lists messages, its not like people add blocks manually or something. Its all done with a confirmation round. I really think you are mistaken and mean another RBL...
Nope, I mean dsbl single hop.
The only way a ISP mailserver gets added if its running a open proxy itself.
DSBL doesn't just list open proxies. It also lists open and unsecure relays, as well as anything else that can be made to send certain messages to a certain address... Many ISP mail servers *are* open relays - for their customers. Many ISPs don't require that their customers authenticate to relay (which is bad, but still common...). As long as your client IP is inside the ISP customer range, you are allowed to relay. Viruses installing proxies that relay through the default outgoing mail server rather than direct to MX is one potential for false positives in dsbl. As is dsbl "trusted users" making mistakes...
Do you have examples of this ? I closely watch the admin and announce list of DSBL, i cannot recally anything like you are saying. You must be confused?
Nope, I'm not confused. I've had to whitelist outgoing mail servers of major major Swedish ISPs after they got listed on dsbl. Those servers where not open proxies or globally open relays.
Unless you believe that the dsbl "trusted users" can be 100% trusted to not make mistakes, or that there are no computers infected with viruses that install proxies configured to relay throgh the default outgoing mail server rather than direct to MX, or any other unforseen new issue, there is a risk of FPs in dsbl.
While the DSBL FAQ is correct in this being "very unlikely", it still happens. As some Googling will show. The dsbl process is not fool-proof.
I'm not bringing this up as a general critique of dsbl. I like dsbl and I still use it, but no longer for blocking directly at the MTA level, and only after first whitelisting local ISPs that don't act as quickly on getting listed in dsbl as they should.
Patrik
Hi!
Nope, I'm not confused. I've had to whitelist outgoing mail servers of major major Swedish ISPs after they got listed on dsbl. Those servers where not open proxies or globally open relays.
Unless you believe that the dsbl "trusted users" can be 100% trusted to not make mistakes, or that there are no computers infected with viruses that install proxies configured to relay throgh the default outgoing mail server rather than direct to MX, or any other unforseen new issue, there is a risk of FPs in dsbl.
Didnt experience that, was that recently ? Anyway, sure, everybody can make mistakes, but so far i didnt see many in DSBL actually...
Guess you were 'lucky'
Bye, Raymond.
On Sun, 15 Aug 2004, Patrik Nilsson wrote:
Nope, I'm not confused. I've had to whitelist outgoing mail servers of major major Swedish ISPs after they got listed on dsbl. Those servers where not open proxies or globally open relays.
If it was because of open proxies or relays on client IP addresses, those servers really should have ended up on multihop.dsbl.org, NOT list.dsbl.org.
If you can tell me the IP address(es) in question, I'll take a look at what happened. If one of the trusted testers maliciously put the wrong tested IP address in the DSBL message, it's time to demote an account to untrusted...
cheers,
Rik (currently inactive DSBL founder)
On Sunday, August 15, 2004, 11:38:11 AM, Rob McEwen wrote:
Thanks Jeff for the feedback from SpamHaus.
Also, I do agree with the philosophy that a little collateral damage from legitimate sources is O.K. if the network originally sending the spam is a known, flagrant, and unrepentant spam source. (How else are they going to be motivated to clean up their act?)
FWIW this is one thing about Spamhaus and other RBLs that I don't like. I don't believe in punishing innocent IP addresses this way in order to pressure ISPs. But we should not start flamewars about RBLs here.
Given Raymond and other's suggestions, I have set up a new test. I now filter at the MTA level using list.dsbl.org. My next level filters using SURBL (on the message's body content only, of course). My third level is to SURBL filter on the Client's server IP address. Although, I haven't yet seen any traction on this one yet... I know... not "by the book" :)
Finally, (...and this is the point of this message...) my fourth level is to filter on the ClientIP using each SpamHaus feed separately. This fourth level then saves all blocked mail to its own folder and I'm manually checking these daily. (This is not as time consuming as it sounds because these are only those e-mails which made it past both DSBL and SURBL filtering, but then got blocked by SpamHaus).
Since I started this only yesterday, I have 26 SBLs and 68 XBLs, with ZERO FPs.
Of course, I have other levels of filtering (linguistic/heuristic) after these I've described...
I'll report back in about a week (or two) regarding how many spams/FPs SpamHaus has caught by then using this setup I described.
Rob McEwen
Sounds like a promising approach. Please let us know how it works out.
If you're using SpamAssassin to do all this, you may want to consider adding it to the SA Wiki for others to see.
Jeff C.
I said:
Also, I do agree with the philosophy that a little collateral damage from legitimate sources is O.K. if the network originally sending the spam is a known, flagrant, and unrepentant spam source. (How else are they going to be motivated to clean up their act?)
Jeff responded:
FWIW this is one thing about Spamhaus and other RBLs that I don't like. I don't believe in punishing innocent IP addresses this way in order to pressure ISPs.
When I read Jeff's comment, I realized that I worded my original statement in a way that could be taken differently that what I intended. To be sure, I abhor the practices described in this article:
http://www.nwfusion.com/research/2001/0910feat.html
But, at the same time, I don't have a problem when out of every 1,000 e-mails coming from a source like Munged-terra.es, 1 legitimate e-mail gets blocked along with 999 spam e-mails.
But we should not start flamewars about RBLs here.
Sorry, I didn't mean to start such a ruckus. I have a proposal. I'll re-configure my filter so that it only blocks those IPs at the MTA level which are listed on both (1) DSBL (...AND...) (2) listed on at least one of the two SpamHaus lists.
If a message does not fit this criteria, then I'll allow it through and (next) filter out messages via SURBL.
After SURBL filtering, out of the remaining messages, I'll then re-check them using EACH of the following three lists:
(1) list.dsbl.org (2) xbl.spamhaus.org (3) sbl.spamhaus.org
(Remember, this will already EXCLUDE those things which are on BOTH list.dsbl.org and sbl-xbl.spamhaus.org. It will also exclude stuff that was block by standard SURBL. Therefore, hopefully, what is left over won't be too huge to analyze.)
Messages then block by any of these three lists will be saved to a folder corresponding to that list.
After about a week of this, I'll zip each of these folders of messages and e-mail the zipped files to Jeff, Raymond, Patrik, and anyone else interested. (I have to be careful here for privacy issues). I'll also provide my own stats for what I judged to be FPs vs. total spams for each folder.
Certainly, this won't be a perfect test because my base of users is not as large as an ISP, for example. But it would be interesting, don't you think?
This way, we can then let the data speak for itself.
How does that sound?
Rob McEwen
-
Jeff Chan -
Patrik Nilsson -
Raymond Dijkxhoorn -
Rik van Riel -
Rob McEwen