[SURBL-Discuss] Probable new data source: DNS queries
jeffc at surbl.org
Wed Nov 10 14:57:46 CET 2004
On Wednesday, November 10, 2004, 5:25:43 AM, Rob McEwen wrote:
> 1st, if you are converting domains to IPs and then checking these IPs
> against spamhaus, you may have to make sure your system can whitelist the
> domains **before** conversion to IP since the IPs can change without notice.
Interesting. I was going to whitelist after detection.
Whitelisting first would prevent some processing.
Note that we're not proposing making a list of IP addresses.
The output is still mostly a list of domains.
> 2nd, SpamHaus keeps listing the following:
> msn.click-url.com, (& variations)
> (These show up FREQUENTLY in hams, so I'd Whitelist these up front. They
> seem to go in an out of SpamHaus intermittently.)
> FOR EXAMPLE:
> msn.click-url.com = 184.108.40.206
> ...points to...
click-url.com is already manually whitelisted so it would not be
on our version of the lists. We would likely apply the SURBL
whitelisting to these lists.
> 3rd, in fact, SpamHaus is going to list a lot of greymarketers that
> shouldn't be listed in SURBL (flowgo, euniverse, etc)
That is one area where we disagee with Spamhaus, and we've
whitelisted most of those since they appear in legitimate
newsletters, etc. However our whitelists of those domains
may not be complete.
> 4th, most of the FPs I find in SpamHaus are XBL listings where the data
> source for that particular FP was http://cbl.abuseat.org/
> CBL catches a LOT of spam... but it also periodically will list the
> mailserver for respected IPS where that ISP had one user who send out a
> bunch of spam and then CBL listed the IP address of that server.
> Unfortunately, this creates a lot of collateral damage. Recently, I
> experienced this with one of my clients's customer's BellSouth E-mail
> services. (I don't know the ratio of XBL stuff via CBL versus XBL stuff from
> other sources. I'd be curious to know this.)
Queries into our DNS servers almost never match domains that
resolve into XBL, which makes sense since those are mostly zombies.
However a domain list of XBL hits may be a useful early warning of
spammers starting to use zombies for hosting, DNS, etc, which
fortunately they haven't done much yet. In practical terms the
XBL hits are so few now as to be a non-issue.
(Really I just included XBL for completeness; SBL is generally
more relevant for URIs, which is why it's what's used by uridnsbl
in SapmAssasisn by default. uridnsbl was probably designed with
SBL in mind. If we do this, the SBL and XBL lists would be
> Jeff, very likely, (I have a feeling) I've misunderstood your original
> intended use of SpamHaus? But maybe this information will be helpful anyway?
> I would definitely recommend NOT using the strategy I've described as an
> **automatic** way to get listed in SURBL. This would defeat MOST of the hard
> work we've done to minimize FPs. But, on the other hand, there are many
> great possibilities here for using this as a tool for evaluating URIs or as
> a honeypot for queuing URIs for evaluation where the URI wasn't already in
> Rob McEwen
The reason for looking at this is a way to avoid the DNS
resolution on wild URI domains that urbdnsbl does in SA 3.
This process is an approximation of what uridnsbl does
with sbl. I suspect that uridnsbl gets some false
positives similar to what you notice in your own
processing. Presumably uridnsbl is scored lower than
SURBLs because of the FPs, and a SURBL version of the
sbl data should probably also be scored lower than other
SURBLs for similar reasons. Our whitelists would tend
to reduce the FP rate somewhat, if applied, which seems
I share your concerns about FPs, but since we're doing
something very similar to what uridnsbl does but with
much less DNS overhead, the same concerns apply to FPs
with uridnsbl, it's just that this new way of doing
things would be much faster.
We have not turned this data into lists yet, but the reasons
for considering it are as I describe: to bypass the very
time consuming name resolution that urndnsbl does against
domains in wild messages. It's meant to be a potential
speedup/replacement for uridnsbl.
We should definitely discuss this more, and I'd like to
hear from the SA developers.
"If it appears in hams, then don't list it."
More information about the Discuss