[SURBL-Discuss] Re: RFC: sex site domain SURBL

Marc Kool M.Kool at vioro.nl
Tue Jul 20 16:27:52 CEST 2004


Hi Jeff,

Jeff Chan wrote:
> Doing a little preliminary checking of this particular dataset
> leads me to wonder a little how appropirate it might be for
> SURBLs.  In particular I found over a hundred whitelist hits of
> sites like aol.com, att.net, btopenworld.com, budweiser.com,
> clara.net, cnet.com, comcast.net, he.net, lsu.edu, match.com,
> mindspring.com, msn.com, rr.com, sina.com, texas.net, tripod.com,
> umich.edu, victoriassecret.com, washington.edu, etc.: 
> 
>   http://spamcheck.freeapp.net/adult.domains.whitelist-hits

I did a quick check on a few domains and I do not share your conclusion.

# grep aol.com domains
adultaol.com
register.oscar.aol.com
sex-aol.com
sexonaol.com
usaol.com

# grep att.net domains
adultonly.home.att.net
borderjumper.home.att.net
brookeb.home.att.net
chrisd054.home.att.net
dating.home.att.net
divinenews.home.att.net
lilcindy.home.att.net
livevids.home.att.net
livevids2.home.att.net
livevids3.home.att.net
livevids4.home.att.net
models.home.att.net
models2.home.att.net
personals.home.att.net
pvelasquez.home.att.net
sasha69.home.att.net
sex-ads.home.att.net
sexworld.home.att.net
xxxmovies.home.att.net

# grep -w au.com domains
aotoys.au.com
condoms.au.com
freeporn.au.com
hornytoad.au.com
muff.au.com

So aol.com and att.net and au.com are not in the database and not blacklisted.
no subdomain of aol.com is in the blacklist.
For au.com and att.net there are only adult subdomains in the blacklist.  This is ok.

> that's after excluding the adult/urls list which had about 300
> whitelist hits, including more hosting providers like terra.es,
> etc.  Recall that our whitelists are not too complete, so there
> may be other legitimate domains that are included.  We can't be
> blocking on aol.com, cnet.com, msn.com, etc.
> 
> Clearly some of these (shared hosting) sites may have been used
> to host sex content, but since RBLs are domain-based, and SURBLs
> are registrar-domain-based, I'm having some doubts about how
> useful this particular data source might be for SURBL use.
> 
>   ftp://ftp.univ-tlse1.fr/pub/reseau/cache/squidguard_contrib/adult.tar.gz
> 
> Perhaps there are other lists of sex domains that are more
> selective?
> 
> Jeff C.

The domain terra.es is also not in the domains list.
It is indeed in the url list, e.g.
personal.telefonica.terra.es/web/sex
terra.es/personal2/amateursexual
etc.

I think that the only database that can be used by SURBL is the domains database
and that the url database is not suitable to be used by SURBL since URLs are difficult
to translate to a DNS query string.

I assume that something went wrong when you verified the quality of the database.
If you have any questions you can also contact me off list.

-Marc


More information about the Discuss mailing list