-----Original Message----- From: Daniel Quinlan [mailto:quinlan@pathname.com] Sent: Thursday, May 05, 2005 1:19 AM To: discuss@lists.surbl.org Cc: jeffc@surbl.org; dev@spamassassin.apache.org Subject: registrar boundary inconsistencies
I ran SURBL (well, a copy a few weeks old) through the split_domains() function in SpamAssassin to see which listings contained both a host+domain rather than just domain from the perspective of SpamAssassin. Those listings would be missed by the URIBL module.
These are reversed for easier reading, but basically, it works like this:
if this is listed:
com.50megs.brisisbri com.50megs.cddvdmp3 com.50megs.slashbackman
were these then in SURBL:
brisisbri.50megs.com cddvdmp3.50megs.com slashbackman.50megs.com
However, the URIDNSBL plugin would catch none of those unless 50megs.com was listed (it's not) since 50megs.com is the domain as far as SpamAssassin is concerned. However, it would catch them if 50megs.com was in SURBL in addition or instead of those hostname.domain combinations.
Here is the data. We (SURBL or SpamAssassin) need to do one of these actions for each of these listings and SURBL probably has more to say about it (initially, at least) since it's your database.
- change the domain code in SA to consider the domain a registry like eu.org or demon.co.uk (let us know and we'll change our
code as long as it makes sense ;-). This means we don't expect blacklist the entire "registry".
SURBL (or your data provider) blacklists the entire domain
remove the hostname.domain listings ... why bother if nothing's going to hit them
Daniel
I vote for changing the domain code to recognise these domains. Blacklisting the entire domain can have too many problems. Removing the whole thing would let spammers game these domains.
I imagine that SA would need updating a lot for more domains like this. Each release. Unless of course there was some data cf file that we could just update at SARE? SImply a list of these type of domains, so they aren't hard coded?
anyway, I hope you devs are having a great Cinco De Mayo!!
--Chris
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Chris Santerre writes:
- change the domain code in SA to consider the domain a registry like
eu.org or demon.co.uk (let us know and we'll change our code as long as it makes sense ;-). This means we don't expect blacklist the entire "registry".
SURBL (or your data provider) blacklists the entire domain
remove the hostname.domain listings ... why bother if nothing's
going to hit them
Daniel
I vote for changing the domain code to recognise these domains. Blacklisting the entire domain can have too many problems. Removing the whole thing would let spammers game these domains.
heh, I did say this would happen last year ;) I also think we should consider these private registries equivalent to TLD registries, as I said back then. Here's the bug -- it's still open:
http://bugzilla.spamassassin.org/show_bug.cgi?id=3549
- --j.
Chris Santerre csanterre@MerchantsOverseas.com writes:
I vote for changing the domain code to recognise these domains. Blacklisting the entire domain can have too many problems. Removing the whole thing would let spammers game these domains.
Well, that's easy to SAY, but we need to know:
- which are really registrars (so some hosts will be bad and some will not be) - which are just spammers using different hostnames on their domain
We can't just add them willy-nilly.
Daniel
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
John Gardiner Myers writes:
Daniel Quinlan wrote:
We can't just add them willy-nilly.
Why not? Treat them like .us -- do two queries.
we don't currently do that. but that may be a good option, actually! allow url_to_domain to return >1 datum, and query all of them.
In the case of .us, and these private registrars, return 2 domains, "foo.eu.org" and "eu.org", or "foo.state.us" and "bar.foo.state.us".
- --j.
On Thursday, May 5, 2005, 11:41:11 AM, Justin Mason wrote:
John Gardiner Myers writes:
Daniel Quinlan wrote:
We can't just add them willy-nilly.
Why not? Treat them like .us -- do two queries.
we don't currently do that. but that may be a good option, actually! allow url_to_domain to return >1 datum, and query all of them.
In the case of .us, and these private registrars, return 2 domains, "foo.eu.org" and "eu.org", or "foo.state.us" and "bar.foo.state.us".
- --j.
That's one approach that could work: check at both 2 and 3 levels. Likely we would never list the registrars, but their spamming customer subdomains would get listed and caught. So one of the two should match. Of course the problem is that generates two DNS queries where one would be ideal.
We currently have a (partial) list of entities that register subdomains, with two-level-tlds list, but does not include all private registries.
http://spamcheck.freeapp.net/two-level-tlds
If it would help, we could expand it to include private registries that register subdomains, in addition to cctld second level domains. SURBL-using applications would still need to include that list and know to process on the next level (the subdomain) as outlined at:
http://www.surbl.org/implementation.html
That list does currently have some non-country code domains like:
eu.org au.com br.com cn.com de.com de.net eu.com [...]
Is SpamAssassin using that list? If so, it it nearly sufficient to make this judgement about what level to check on? Can we improve it just by adding more private registries?
Jeff C. -- Don't harm innocent bystanders.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Jeff Chan writes:
On Thursday, May 5, 2005, 11:41:11 AM, Justin Mason wrote: That list does currently have some non-country code domains like:
eu.org au.com br.com cn.com de.com de.net eu.com [...]
Is SpamAssassin using that list? If so, it it nearly sufficient to make this judgement about what level to check on? Can we improve it just by adding more private registries?
Yes. Except in the bug I posted earlier, I was about the only person who was +1 on that idea I think ;)
- --j.
-
Chris Santerre -
Daniel Quinlan -
Jeff Chan -
jm＠jmason.org -
John Gardiner Myers