On Tuesday, July 20, 2004, 4:55:01 PM, Marc Kool wrote:
Jeff Chan wrote:
The main reason we did this was to defeat the "random subdomain" spammers who generate random subdomains to try to defeat simple URI pattern matching or to key their spams to confirm the recipient addresses. Examples might be "abc1.xyz.spammerdomain.com" and "abc2.xyz.spammerdomain.com". Those we want to reduce to just "spammerdomain.com" since the randomized/keyed versions may occur only once and the sc.surbl.org data engine tries to increase the likelyhood of inclusion in the list with an increasing number of reports.
It may be useful to read about the sc.surbl.org data:
Yep, the reasons why this is done are clear but are not flawless. There are ISPs myisp.net that give customers a subdomain: e.g. myspamsite.myisp.net which can not be included in SURBL. I also assume that the percentage of these type of domains is not so big...
Yes, I think they are rare because a legitimate ISP would not want a major spam site on their domain, even a subdomain, for damage to their reputation, etc. Any ISP that would willingly host a spam site on a subdomain of their own domain I think we would consider rogue ISPs which I would not feel too bad about blocking entirely. But few ISPs seem to put themselves into this position, which is perhaps why big spammers use so many custom domains.
I think you're right; I can't really think of many examples of this actually happening, so our design compromise perhaps seems reasonable. :-)
[...]
For the record: my originals proposal would make sex.surbl.org more of a squidguard-based list than a surbl-based list.
Right, which is fine. Please see my next message for some proposed solutions to this.
One of the reasons to propose sex.surbl.org was the fact that SURBL list lag behind reality. In July I received 156 spams of which 16 were not detected by SA+SOME_SARE_RULES+OWN_RULES+SURBL because the SURBL lists were not updates fast enough (the 16 spams were marked as spam at a later time because then SURBL marked them and the SA rating went up). This is not meant to criticize anybody, just to put a fact.
I observed that many spams from new domains
- share IP addresses
- automatically forward you to a known sex site (in the squidguard database)
and proposed sex.surbl.org
There will always be some lag, but once caught, SURBLs have the potential to limit the spread of the spams, at least ones with the same URIs mentioned repeately.
Note that the next version of the sc data engine will cut this lag quite dramatically, especially for those resolving to frequently appearing spammer IP blocks. For more info on the proposed next version of this data engine, please see:
http://www.surbl.org/faq.html#numbered
However, I see some value for the squidguard adult database to be used by software behind spamtraps: if an URI is retrieved and redirects you to a known sex site, the URI can be added automatically (= fast) to a SURBL list.
Marc
I agree RBLs are a convenient and fast way to get data out. It takes good advantage of the existing DNS infrastructure.
Jeff C.