At 04:56 PM 9/9/2004, Chris Santerre wrote:
So is there a way to use the IP info in a good way? Could SA or SURBL do a quick ping of the URL and match against a URL? This would allow us to simply list 1 IP instead of all these domains.
Chris, SA 3.0 appears to already support checking DNS blacklisting of URLs based on resolved IP. (as well as surbl-style based on domain name). So theoretically, SURBL could open up a separate list based on IP's (i.e.: multi.dnsbl.surbl.org)
Take a look at the example where it checks the resolved IP of a URL against the SBL (an IP based list):
uridnsbl URIBL_SBL sbl.spamhaus.org. TXT header URIBL_SBL eval:check_uridnsbl('URIBL_SBL') describe URIBL_SBL Contains a URL listed in the SBL blocklist tflags URIBL_SBL net
and from URIDNSBL.pm:
This works by analysing message text and HTML for URLs, extracting the domain names from those, querying their NS records in DNS, resolving the hostnames used therein, and querying various DNS blocklists for those IP addresses. This is quite effective.
SYNOPSIS
loadplugin Mail::SpamAssassin::Plugin::URIDNSBL uridnsbl URIBL_SBLXBL sbl-xbl.spamhaus.org. TXT
On Thu, 9 Sep 2004, Matt Kettler wrote:
If it's blacklisting based on resolved ip, it should probably be noted that there are a couple of caveats:
1) Spammers can set up multiple ip addresses to an A record. Whatever does the reporting should check all A records, from the top down. i.e. query each NS multiple times to make sure it's not being round-robined or reported differently from multiple DNS servers.
2) I can easily forsee spammers doing a wildcard subdomain as an effort to thwart this, if we're doing nslookups.
3) It's a common case that spammers use disposable landing sites, such as the forwarding services offered by tinyurl, zoneedit, and the like, or will put an HTTP redirect on a hotmail or geocities page. Should those be exempt from this, since they have a fair number of legitimate domains as well?
-Dan
At 04:56 PM 9/9/2004, Chris Santerre wrote:
So is there a way to use the IP info in a good way? Could SA or SURBL do a quick ping of the URL and match against a URL? This would allow us to simply list 1 IP instead of all these domains.
Chris, SA 3.0 appears to already support checking DNS blacklisting of URLs based on resolved IP. (as well as surbl-style based on domain name). So theoretically, SURBL could open up a separate list based on IP's (i.e.: multi.dnsbl.surbl.org)
Take a look at the example where it checks the resolved IP of a URL against the SBL (an IP based list):
uridnsbl URIBL_SBL sbl.spamhaus.org. TXT header URIBL_SBL eval:check_uridnsbl('URIBL_SBL') describe URIBL_SBL Contains a URL listed in the SBLblocklist tflags URIBL_SBL net
and from URIDNSBL.pm:
This works by analysing message text and HTML for URLs, extractingthe domain names from those, querying their NS records in DNS, resolving the hostnames used therein, and querying various DNS blocklists for those IP addresses. This is quite effective.
SYNOPSIS loadplugin Mail::SpamAssassin::Plugin::URIDNSBL uridnsbl URIBL_SBLXBL sbl-xbl.spamhaus.org. TXT
--
"I hate Windows"
-Tigerwolf, Anthrocon 2004
--------Dan Mahoney-------- Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---------------------------
Hi!
- Spammers can set up multiple ip addresses to an A record. Whatever does
the reporting should check all A records, from the top down. i.e. query each NS multiple times to make sure it's not being round-robined or reported differently from multiple DNS servers.
- I can easily forsee spammers doing a wildcard subdomain as an effort to
thwart this, if we're doing nslookups.
- It's a common case that spammers use disposable landing sites, such as the
forwarding services offered by tinyurl, zoneedit, and the like, or will put an HTTP redirect on a hotmail or geocities page. Should those be exempt from this, since they have a fair number of legitimate domains as well?
Did you actually have a look on the sata provided at the start of this thread ? Sure, it COULD be different, but somehow, it isnt.
Thats why we posted the data in the first place, a lot of spam is boosted inside via the exact same way. We can ignore that, and say they will migitate, but if we never react they will never migitate either.
Bye, Raymond.
On Thu, 9 Sep 2004, Raymond Dijkxhoorn wrote:
Hi!
- Spammers can set up multiple ip addresses to an A record. Whatever
does the reporting should check all A records, from the top down. i.e. query each NS multiple times to make sure it's not being round-robined or reported differently from multiple DNS servers.
- I can easily forsee spammers doing a wildcard subdomain as an effort to
thwart this, if we're doing nslookups.
- It's a common case that spammers use disposable landing sites, such as
the forwarding services offered by tinyurl, zoneedit, and the like, or will put an HTTP redirect on a hotmail or geocities page. Should those be exempt from this, since they have a fair number of legitimate domains as well?
Did you actually have a look on the sata provided at the start of this thread ? Sure, it COULD be different, but somehow, it isnt.
Yes, I did. But I'm trying to think ahead of current practice, by what's considered a GOOD practice to keep a site up, and what's bad. I'm not saying they're all doing it now, but I've *seen* them have another server ready to go when I yank ether (invariably, they migrate the ip by hand, to prevent everything being yanked at onces).
-Dan
Thats why we posted the data in the first place, a lot of spam is boosted inside via the exact same way. We can ignore that, and say they will migitate, but if we never react they will never migitate either.
Bye, Raymond.
--
"Man, this is such a trip"
-Dan Mahoney, October 25, 1997
--------Dan Mahoney-------- Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---------------------------
Hi!
Did you actually have a look on the sata provided at the start of this thread ? Sure, it COULD be different, but somehow, it isnt.
Yes, I did. But I'm trying to think ahead of current practice, by what's considered a GOOD practice to keep a site up, and what's bad. I'm not saying they're all doing it now, but I've *seen* them have another server ready to go when I yank ether (invariably, they migrate the ip by hand, to prevent everything being yanked at onces).
But are you surprised if i tell you that the 4 or 5 sites (IPs) i listed are responsible for 28% of our incomming spam volume, on a pretty large site. Thats around 400.000 spams of those guys alone. If we can get them out, anyhow, thats a big win. Those crap domains are really everywhere, so i cant imagine anyone is not seeing those on spamchecks... :)
Bye, Raymond.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Dan Mahoney, System Admin writes:
On Thu, 9 Sep 2004, Matt Kettler wrote:
If it's blacklisting based on resolved ip, it should probably be noted that there are a couple of caveats:
- Spammers can set up multiple ip addresses to an A record. Whatever
does the reporting should check all A records, from the top down. i.e. query each NS multiple times to make sure it's not being round-robined or reported differently from multiple DNS servers.
- I can easily forsee spammers doing a wildcard subdomain as an effort to
thwart this, if we're doing nslookups.
they already do. this also opens a list-washing hole, as a hidden link to <a href=http://myaddress-rot13-encoded.spammer.com/> will be resolved, indicating to the spammer that some software at the remote end is resolving all links in the message.
If OTOH you choose not to use the exact hostname parts of hrefs to avoid this, instead just resolving "www.spammer.com", they can then ensure that spammer.com and www.spammer.com do not resolve to hostnames and spam using links to notwww.spammer.com/payload.html instead.
- --j.
- It's a common case that spammers use disposable landing sites, such as
the forwarding services offered by tinyurl, zoneedit, and the like, or will put an HTTP redirect on a hotmail or geocities page. Should those be exempt from this, since they have a fair number of legitimate domains as well?
-Dan
At 04:56 PM 9/9/2004, Chris Santerre wrote:
So is there a way to use the IP info in a good way? Could SA or SURBL do a quick ping of the URL and match against a URL? This would allow us to simply list 1 IP instead of all these domains.
Chris, SA 3.0 appears to already support checking DNS blacklisting of URLs based on resolved IP. (as well as surbl-style based on domain name). So theoretically, SURBL could open up a separate list based on IP's (i.e.: multi.dnsbl.surbl.org)
Take a look at the example where it checks the resolved IP of a URL against the SBL (an IP based list):
uridnsbl URIBL_SBL sbl.spamhaus.org. TXT header URIBL_SBL eval:check_uridnsbl('URIBL_SBL') describe URIBL_SBL Contains a URL listed in the SBLblocklist tflags URIBL_SBL net
and from URIDNSBL.pm:
This works by analysing message text and HTML for URLs, extractingthe domain names from those, querying their NS records in DNS, resolving the hostnames used therein, and querying various DNS blocklists for those IP addresses. This is quite effective.
SYNOPSIS loadplugin Mail::SpamAssassin::Plugin::URIDNSBL uridnsbl URIBL_SBLXBL sbl-xbl.spamhaus.org. TXT--
"I hate Windows"
-Tigerwolf, Anthrocon 2004
--------Dan Mahoney-------- Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org
Hi!
- Spammers can set up multiple ip addresses to an A record. Whatever
does the reporting should check all A records, from the top down. i.e. query each NS multiple times to make sure it's not being round-robined or reported differently from multiple DNS servers.
- I can easily forsee spammers doing a wildcard subdomain as an effort to
thwart this, if we're doing nslookups.
they already do. this also opens a list-washing hole, as a hidden link to <a href=http://myaddress-rot13-encoded.spammer.com/> will be resolved, indicating to the spammer that some software at the remote end is resolving all links in the message.
SURBL only takes the domain, so thats fine, its only a little feaky for your nameserver, but then again, SA does rely on DNS a lot, so thats now news :)
If OTOH you choose not to use the exact hostname parts of hrefs to avoid this, instead just resolving "www.spammer.com", they can then ensure that spammer.com and www.spammer.com do not resolve to hostnames and spam using links to notwww.spammer.com/payload.html instead.
Very true.
Bye, Raymond.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Raymond Dijkxhoorn writes:
- Spammers can set up multiple ip addresses to an A record. Whatever
does the reporting should check all A records, from the top down. i.e. query each NS multiple times to make sure it's not being round-robined or reported differently from multiple DNS servers.
- I can easily forsee spammers doing a wildcard subdomain as an effort to
thwart this, if we're doing nslookups.
they already do. this also opens a list-washing hole, as a hidden link to <a href=http://myaddress-rot13-encoded.spammer.com/> will be resolved, indicating to the spammer that some software at the remote end is resolving all links in the message.
SURBL only takes the domain, so thats fine, its only a little feaky for your nameserver, but then again, SA does rely on DNS a lot, so thats now news :)
Yeah. I was referring to the proposal to lookup IP addresses for href hostnames directly (instead of looking up the NS'es.)
- --j.
If OTOH you choose not to use the exact hostname parts of hrefs to avoid this, instead just resolving "www.spammer.com", they can then ensure that spammer.com and www.spammer.com do not resolve to hostnames and spam using links to notwww.spammer.com/payload.html instead.
Very true.
Bye, Raymond.
On Thursday, September 9, 2004, 3:19:49 PM, Justin Mason wrote:
Raymond Dijkxhoorn writes:
- Spammers can set up multiple ip addresses to an A record. Whatever
does the reporting should check all A records, from the top down. i.e. query each NS multiple times to make sure it's not being round-robined or reported differently from multiple DNS servers.
- I can easily forsee spammers doing a wildcard subdomain as an effort to
thwart this, if we're doing nslookups.
they already do. this also opens a list-washing hole, as a hidden link to <a href=http://myaddress-rot13-encoded.spammer.com/> will be resolved, indicating to the spammer that some software at the remote end is resolving all links in the message.
SURBL only takes the domain, so thats fine, its only a little feaky for your nameserver, but then again, SA does rely on DNS a lot, so thats now news :)
Yeah. I was referring to the proposal to lookup IP addresses for href hostnames directly (instead of looking up the NS'es.)
Yep. Resolving domain names found in spam URIs is slow (especially if timeouts are hit, which can take like what, 20 seconds per domain) and it opens the door to confirming for the spammers which recipient addresses got through. It's a good way for spammers to build a confirmed recipient list.
That's another reason we don't do it with SURBLs.
Jeff C.
[ Whew! CC trimmed :-) ]
Jeff Chan wrote to Justin Mason:
Yeah. I was referring to the proposal to lookup IP addresses for href hostnames directly (instead of looking up the NS'es.)
Yep. Resolving domain names found in spam URIs is slow
Aha. Key word = "domain names".
All the world's a host. Spammers are already using random subdomains in their emails, and there is absolutely *no* guarantee whatsoever that these subdomains resolve to the same IP(s) as the registrar domain (or even as the rest of the subdomains). It's basic DNS, and, in this case, it means we're basically screwed before we start. :-)
There *may* be some benefit to the idea, but I'm betting it would be extremely short-term, because spammers would too easily thwart it by pointing their TLDs A record to somewhere else.
Unless we started keeping more host information...but then we're effectively DoSsed by the sheer number of subdomains in use. There are a few ways I could think to greatly optimize that, but, so far, I don't see a big win.
- Ryan
On Thursday, September 9, 2004, 3:57:46 PM, Ryan Thompson wrote:
Jeff Chan wrote to Justin Mason:
Yeah. I was referring to the proposal to lookup IP addresses for href hostnames directly (instead of looking up the NS'es.)
Yep. Resolving domain names found in spam URIs is slow
Aha. Key word = "domain names".
All the world's a host. Spammers are already using random subdomains in their emails, and there is absolutely *no* guarantee whatsoever that these subdomains resolve to the same IP(s) as the registrar domain (or even as the rest of the subdomains). It's basic DNS, and, in this case, it means we're basically screwed before we start. :-)
There *may* be some benefit to the idea, but I'm betting it would be extremely short-term, because spammers would too easily thwart it by pointing their TLDs A record to somewhere else.
Unless we started keeping more host information...but then we're effectively DoSsed by the sheer number of subdomains in use. There are a few ways I could think to greatly optimize that, but, so far, I don't see a big win.
On the other hand, we can resolve (FQDNs) on the data side (not the client side) and see if any IPs consistently emerge. If so we can certainly use them. If not, they're noise and get ignored automatically.
Given that hosting and redirection are not zombied (yet), the pool of hosting IPs available to bad guys is probably sufficiently small and concentrated to be potentially useful. (It's not the same as the pool of *sending* IPs, which is as large as can be zombied.)
Yeah, yeah, I know it sounds like I'm arguing against my prior position against IPs, but I'm not. I'm just putting a finer point on it.
Jeff C.
on Thu, Sep 09, 2004 at 04:57:46PM -0600, Ryan Thompson wrote:
[ Whew! CC trimmed :-) ]
Jeff Chan wrote to Justin Mason:
Yeah. I was referring to the proposal to lookup IP addresses for href hostnames directly (instead of looking up the NS'es.)
Yep. Resolving domain names found in spam URIs is slow
Aha. Key word = "domain names".
All the world's a host. Spammers are already using random subdomains in their emails, and there is absolutely *no* guarantee whatsoever that these subdomains resolve to the same IP(s) as the registrar domain (or even as the rest of the subdomains). It's basic DNS, and, in this case, it means we're basically screwed before we start. :-)
It's wildcard DNS if anything - the "random" bits are added to allow for tracking.
exhibit #1: from a real spam: schampeo@cayenne:1009 $ ns www.illusiontantrumillsexhaledtarpaper.shjkss.d.dd.f.ff.k.gerswe.gatsrsa.com Server: 216.27.21.209 Address: 216.27.21.209#53
Non-authoritative answer: Name: www.illusiontantrumillsexhaledtarpaper.shjkss.d.dd.f.ff.k.gerswe.gatsrsa.com Address: 222.55.10.3
exhibit #2: take a guess: schampeo@cayenne:1010 $ ns www.spammersdeservenothinglessthanlongslowpainfuldeath.shjkss.d.dd.f.ff.k.gerswe.gatsrsa.com Server: 216.27.21.209 Address: 216.27.21.209#53
Non-authoritative answer: Name: www.spammersdeservenothinglessthanlongslowpainfuldeath.shjkss.d.dd.f.ff.k.gerswe.gatsrsa.com Address: 222.55.10.3
I wouldn't worry that much about it.
On Thursday, September 9, 2004, 2:48:51 PM, System Dan Mahoney wrote:
On Thu, 9 Sep 2004, Matt Kettler wrote:
If it's blacklisting based on resolved ip, it should probably be noted that there are a couple of caveats:
- Spammers can set up multiple ip addresses to an A record. Whatever
does the reporting should check all A records, from the top down. i.e. query each NS multiple times to make sure it's not being round-robined or reported differently from multiple DNS servers.
Good point.
- I can easily forsee spammers doing a wildcard subdomain as an effort to
thwart this, if we're doing nslookups.
Code using SURBLs attempts reduce domains to the base (registrar) domains before comparing to SURBLs. In other words we ignore the subdomains, host portion, etc.
http://www.surbl.org/faq.html#random
- It's a common case that spammers use disposable landing sites, such as
the forwarding services offered by tinyurl, zoneedit, and the like, or will put an HTTP redirect on a hotmail or geocities page. Should those be exempt from this, since they have a fair number of legitimate domains as well?
Please see:
http://www.surbl.org/faq.html#redirect
and the rest of the FAQ. :-)
Jeff C.
-
Dan Mahoney, System Admin -
Jeff Chan -
jm＠jmason.org -
Matt Kettler -
Raymond Dijkxhoorn -
Ryan Thompson -
Steven Champeon