[SURBL-Discuss] RE: ANNOUNCE: Mail::SpamAsssassin::SpamCopURI 0.11

[Simon Byrnand]

(I've moved this message from the SA list to the SURBL list where it's more 
relevant and wont get lost in the noise....)

At 06:14 22/04/2004, Dallas L. Engelken wrote:

> >
> > > I have just released SpamCopURI version 0.11.  This fixes a
> > few bugs
> > > that had been reported and adds open redirect resolution.
> > [...]
> >
>
>Just installed it...  Can you tell me what is up with this.
>
>@400000004086b7c400ac051c debug: Query failed for
>thegolfchannel.com.ws.surbl.org
>@400000004086b7c400ad2244 debug: querying for
>www.thegolfchannel.com.ws.surbl.org
>@400000004086b7c400ad262c
>@400000004086b7c400d251cc debug: Query failed for
>www.thegolfchannel.com.ws.surbl.org
>@400000004086b7c400d74b3c debug: querying for
>thegolfchannel.com.ws.surbl.org
>@400000004086b7c400d7530c
>@400000004086b7c400f8d144 debug: Query failed for
>thegolfchannel.com.ws.surbl.org
>@400000004086b7c400f9ea84 debug: querying for
>www.thegolfchannel.com.ws.surbl.org
>@400000004086b7c400f9f254
>@400000004086b7c4011e6e2c debug: Query failed for
>www.thegolfchannel.com.ws.surbl.org
>@400000004086b7c40123d8e4 debug: querying for
>thegolfchannel.com.ws.surbl.org
>@400000004086b7c40123e0b4
>@400000004086b7c4014c5814 debug: Query failed for
>thegolfchannel.com.ws.surbl.org
>@400000004086b7c4014d7924 debug: querying for
>www.thegolfchannel.com.ws.surbl.org
>@400000004086b7c4014d7d0c
>@400000004086b7c401729524 debug: Query failed for
>www.thegolfchannel.com.ws.surbl.org
>@400000004086b7c401777724 debug: querying for
>thegolfchannel.com.ws.surbl.org
>@400000004086b7c401777ef4
>@400000004086b7c401993f94 debug: Query failed for
>thegolfchannel.com.ws.surbl.org
>@400000004086b7c4019a648c debug: querying for
>www.thegolfchannel.com.ws.surbl.org
>@400000004086b7c4019a6c5c
>@400000004086b7c401bec124 debug: Query failed for
>www.thegolfchannel.com.ws.surbl.org
>@400000004086b7c401c3a324 debug: querying for
>thegolfchannel.com.ws.surbl.org
>
>Like 20 some times it tried to query before it finally stopped.  Does
>query failed actually mean 'failed' or there was no A record found?  If
>I send a test from the command line on a message that contains a uri on
>both lists, it works fine.
>
>[root at localhost service]# echo -e 'From: dallase\n\n<a
>href="http://8006hosting.com">click here</A>' | spamc
>         ...
>         *  3.0 SC_URI_RBL Contains a URL listed in the SC SURBL
>blocklist
>         *  2.5 WS_URI_RBL Contains a URL listed in the WS SURBL
>blocklist
>         ...
>
>Do I need a new DNS::Resolver or is this normal behavior?

I'm seeing the same thing with SpamCopURI-0.12 as well, I don't remember 
whether I was seeing that with 0.10 though. I've seen cases where one 
message is causing 20 or more lookings for the "same" dns record.

I think I've worked out what is happening. Basically each different 
variation of a subdomain URL found in a message is causing a seperate 
lookup, even though the base domains that are actually being looked up are 
the same. For example I made a test message that looked like this:

http://serbserb.testdomain.co.nz/blah
http://sebserbr.testdomain.co.nz/blah
http://bsertbse.testdomain.co.nz/blah
http://srtnsrtn.testdomain.co.nz/blah
http://nrtnsrtn.testdomain.co.nz/blah
http://saerbsee.testdomain.co.nz/blah
http://rtndrtsn.testdomain.co.nz/blah
http://nrtndrtn.testdomain.co.nz/blah
http://sdfgserg.testdomain.co.nz/blah
http://bcvcvbcx.testdomain.co.nz/blah
http://ergsergh.testdomain.co.nz/blah
http://qwertybe.testdomain.co.nz/blah
http://lphtrhtr.testdomain.co.nz/blah
http://bxdfbgnf.testdomain.co.nz/blah
http://ergerger.testdomain.co.nz/blah
http://cbxcvbxc.testdomain.co.nz/blah
http://tyjftyjt.testdomain.co.nz/blah
http://awefawfe.testdomain.co.nz/blah
http://awefawef.testdomain.co.nz/blah
http://awefawef.testdomain.co.nz/blah

Where there is a randomized subdomain in front of the actual domain. Many 
spams with lots of image links (ones selling printer cartridges, etc etc) 
effectively do this. (Each URL refers to a randomized subdomain)

Each URL above generated a dns lookup for testdomain.co.nz.sc.surbl.org and 
co.nz.sc.surbl.org, so a total of 40 dns lookups just for the sc list. I'm 
also using ws and be lists too, so thats a total of 120 dns lookups 
generated by an email with 20 randomized URLs :(

Luckily local dns caching largely offsets the problem but it would be good 
to avoid in the first place. Somehow as each URL is stripped down, a list 
of stripped names needs to be created with duplicates removed before doing 
the DNS queries.... extra coding I guess...

Regards,
Simon