[I replied on the SA list also]
On Wednesday, April 21, 2004, 6:14:51 PM, Simon Byrnand wrote:
(I've moved this message from the SA list to the SURBL list where it's more relevant and wont get lost in the noise....)
At 06:14 22/04/2004, Dallas L. Engelken wrote:
I have just released SpamCopURI version 0.11. This fixes a
few bugs
that had been reported and adds open redirect resolution.
[...]
Just installed it... Can you tell me what is up with this.
@400000004086b7c400ac051c debug: Query failed for thegolfchannel.com.ws.surbl.org @400000004086b7c400ad2244 debug: querying for www.thegolfchannel.com.ws.surbl.org @400000004086b7c400ad262c
...
@400000004086b7c401bec124 debug: Query failed for www.thegolfchannel.com.ws.surbl.org @400000004086b7c401c3a324 debug: querying for thegolfchannel.com.ws.surbl.org
Like 20 some times it tried to query before it finally stopped. Does query failed actually mean 'failed' or there was no A record found? If I send a test from the command line on a message that contains a uri on both lists, it works fine.
[root@localhost service]# echo -e 'From: dallase\n\n<a href="http://8006hosting.com">click here</A>' | spamc ... * 3.0 SC_URI_RBL Contains a URL listed in the SC SURBL blocklist * 2.5 WS_URI_RBL Contains a URL listed in the WS SURBL blocklist ...
Do I need a new DNS::Resolver or is this normal behavior?
I'm seeing the same thing with SpamCopURI-0.12 as well, I don't remember whether I was seeing that with 0.10 though. I've seen cases where one message is causing 20 or more lookings for the "same" dns record.
I think I've worked out what is happening. Basically each different variation of a subdomain URL found in a message is causing a seperate lookup, even though the base domains that are actually being looked up are the same. For example I made a test message that looked like this:
http://serbserb.testdomain.co.nz/blah http://sebserbr.testdomain.co.nz/blah
...
http://awefawfe.testdomain.co.nz/blah http://awefawef.testdomain.co.nz/blah http://awefawef.testdomain.co.nz/blah
Where there is a randomized subdomain in front of the actual domain. Many spams with lots of image links (ones selling printer cartridges, etc etc) effectively do this. (Each URL refers to a randomized subdomain)
Each URL above generated a dns lookup for testdomain.co.nz.sc.surbl.org and co.nz.sc.surbl.org, so a total of 40 dns lookups just for the sc list. I'm also using ws and be lists too, so thats a total of 120 dns lookups generated by an email with 20 randomized URLs :(
Luckily local dns caching largely offsets the problem but it would be good to avoid in the first place. Somehow as each URL is stripped down, a list of stripped names needs to be created with duplicates removed before doing the DNS queries.... extra coding I guess...
Regards, Simon
Looks like it's probably normal failure to resolve an A record which means the domain is not on the list:
% nslookup thegolfchannel.com.ws.surbl.org
*** localhost.freeapp.net can't find thegolfchannel.com.ws.surbl.org: Non-existent host/domain
Let me ask Eric if there's a way he can eliminate duplicate DNS queries. Perhaps that went away when he deprecated the use of Storable in SpamCopURI.
Simon, you're right that DNS caching means this doesn't hurt much in terms of performance. (It may even be faster than trying to store these in SA to prevent duplication.)
Jeff C.