Discuss July 2004

discuss@lists.surbl.org

65 participants
151 discussions

Fwd: Re: More on SpamCopURI busting tests
by Jeff Chan 19 Jul '04

19 Jul '04

This is a forwarded message From: Bob McClure Jr <robertmcclure(a)earthlink.net> To: Jeff Chan <jeffc(a)surbl.org> Date: Monday, July 19, 2004, 1:15:27 PM Subject: More on SpamCopURI busting tests ===8<==============Original message text=============== On Mon, Jul 19, 2004 at 01:06:48PM -0700, Jeff Chan wrote: > On Monday, July 19, 2004, 12:53:57 PM, Bob Jr wrote: > > I have just posted this on the SourceForge page for SpamCopURI: > > > Here's another data point. About a month ago I installed it on my > > workstation and it passed all tests. URI version is 1.28. > > > Today, I tried installing it on three other machines, and all required > > a new URI and got v1.31, and SpamCopURI busted the same three tests. > > > Just to test the relavance of that, I upgraded URI on my workstation via CPAN, and then > > manually did the "perl Makefile.PL", "make", and "make test" on > > SpamCopURI, and sure enough, the three tests busted. So apparently > > something happened in URI between v1.28 and v.131 that's causing > > SpamCopURI to bust the tests. > > Thanks for the additional data Bob. Eric Kolve is aware of some > of the net tests failing during installation and is working on > a SpamCopURI install that will skip those tests if appropriate. > (He mentions outbound port 80 being blocked as the source of the > issue.) Hmm. I disagree. For three of the machines, I _know_ there is no port 80 blocking. > It might be useful if you could show us which tests are failing, > but I assume it's the same redirection-handling ones that other > people have mentioned. Yup. Same set: t/blacklist........ok t/dnsrbl...........ok t/extract_urls.....ok t/mailto...........ok t/open_redirect....# Failed test (t/open_redirect.t at line 37) t/open_redirect....NOK 1# Failed test (t/open_redirect.t at line 44) t/open_redirect....NOK 2# Failed test (t/open_redirect.t at line 52) t/open_redirect....ok 4/5# Looks like you failed 3 tests of 5. t/open_redirect....dubious Test returned status 3 (wstat 768, 0x300) Scalar found where operator expected at (eval 152) line 1, near "'int' $__val" (Missing operator before $__val?) DIED. FAILED tests 1-3 Failed 3/5 tests, 40.00% okay t/spamcopuri.......ok t/whitelist........ok > Jeff C. > -- > Jeff Chan > mailto:jeffc@surbl.org > http://www.surbl.org/ Cheers, -- Bob McClure, Jr. Bobcat Open Systems, Inc. robertmcclure(a)earthlink.net http://www.bobcatos.com People we like the least need our love the most. ===8<===========End of original message text===========

1 0

Re: include IP lookups in SURBL lists
by Jeff Chan 19 Jul '04

19 Jul '04

OK I'm going to respond to several ideas in this thread in a single reply. It may help to go back and review some of the thread messages. 1. Regarding adding resolved IP addresses to SURBLs: Not gonna happen. FP potential is way too high. A single (false) entry resolving to a legitimate large shared web hosting server could block hundreds or more legitimate sites. 2. However the next version of sc.surbl.org data engine will be a hybrid name/number system where: A. the domains will get resolved internally, B. the resulting IPs will get sorted into (CIDR) bins, C. any fresh domain report that happens to resolve into one of those bins will inherit the count of hits in the bins (perhaps modulo some function), and most likely any fresh spam domains resolving into a well-populated bin will get listed on the first report instead of the tenth as sc does now. We could even raise the threshold to decrease FPs or change to a "top 500" or "top 1000" list. So that should short circuit most the lag in detection for domains resolving to persistent spammer IPs for the sc data. The resulting lists will still be mostly domains. We probably won't let the internal IPs out, at least not in the existing SURBLs. Perhaps we could turn them into a separate list which could be scored lower. But our focus will remain on domains because they are highly specific and don't require the time- consuming step of name resolution. (Name resolution is no problem on a small box, but on big mail systems it can make content checking impractical. Resolved IPs also have some of the potential problems already mentioned, most importantly FPs.) 3. The outblaze data already has a "recentness of domain registration factor" of 90 days. It also includes extensive spam traps. The combination appears to catch many spammer URI domains pretty quickly and with a low FP rate. So it already somewhat incorporates John Hardin's idea of catching recently registered domains, with the added factor that they actually got caught spamming. Outblaze's traps apparently are pretty well engineered, given the relatively low FP rate. BTW, there's a longer discussion of this question in the FAQ: http://www.surbl.org/faq.html#numbered "Are there plans to offer an RBL list with the domain names resolved into IP addresses?" Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

2 1

[SURBL-Discuss] Re: sex.surbl.org (was: Re: include IP lookups in SURBL lists)
by Jeff Chan 19 Jul '04

19 Jul '04

On Monday, July 12, 2004, 4:06:44 PM, Marc Kool wrote: > Chris Santerre wrote: >>>To start another thread: >>>I am a contributor to the free URL database that can be used >>>by squidguard and dansguardian >>>with a strong focus on sex sites (with 397000 domains). >>>Although not every sex site sends spam, a mail administrator >>>may want to implement >>>a local policy to block emails that refer to sex sites (I would :-) >>>Does anybody wants/needs/likes sex.surbl.org ??? >>>For a list like sex.surbl.org a feature to include IP >>>addresses is a benefit >>>since 57% percent of the sex domains have 10 or more domains >>>resolving to a single IP address. >> Mark this is a GREAT idea. I'm BCC'ing Jeff on it to make sure he sees this. >> --Chris > I received some questions, so for those who don't know it yet: > the free URL database that mainly includes a list of adult domains can be found on > http://cri.univ-tlse1.fr/documentations/cache/squidguard_en.html#contrib > (correction to the above number: 466057 adult domains are listed on July 13 2004). > If you want to pay for it, you may want to go to http://www.urlblacklist.com > -Marc Thanks for sharing this idea and a free list of the relevant domains. I agree it's an interesting idea, though we've been focussed on spammer's URI domains so far, so it's a bit of a tangent. At nearly 500k record we'd definitely want to do this with rbldnsd. BIND is already getting pretty slow with lists approaching 100k records. For comparison, sbl.spamhaus.org has about 5k records, xbl has about 1.7 million records, list.dsbl.org has nearly 4 million records. multi.surbl.org, our largest production list has about 45k records. How does anyone else feel about turning this list of sex sites into a SURBL? Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

4 5

exacttarget-MUNGED.com
by Rob McEwen 19 Jul '04

19 Jul '04

First, I love SURBL... I have integrated it into my server's mail filtering system. (You may recall weeks back my asking for help on this. I found a solution! My previous filter was upgraded by its author to work with SURBL.) The purpose of THIS e-mail is to ask about a situation where a legitimate opt-in newsletter has been blocked. This was a NewsMax.com newsletter that I, in fact, specifically requested. The offending domain found in this newsletter is the following: exacttarget-MUNGED.com Maybe exacttarget-MUNGED.com plays clean sometimes (as the case of this particular newsletter) and plays dirty other times. If that is the case, then I don't have a problem with this domain name getting blocked... but if not please double check this and consider whitelisting them. Thanks. Rob McEwen rob(a)pvsys.com

2 1

test
by Chris Santerre 19 Jul '04

19 Jul '04

Sorry, it said I was disabled. Just checking. Chris Santerre System Admin and SARE Ninja http://www.rulesemporium.com http://www.surbl.org 'It is not the strongest of the species that survives, not the most intelligent, but the one most responsive to change.' Charles Darwin

3 2

test
by Chris Santerre 19 Jul '04

19 Jul '04

Sorry I got booted from the list due to bounces. Time to go look at my server logs :) Chris Santerre System Admin and SARE Ninja http://www.rulesemporium.com http://www.surbl.org 'It is not the strongest of the species that survives, not the most intelligent, but the one most responsive to change.' Charles Darwin

1 0

{Spam?} SBL with message body URIs
by Patrik Nilsson 19 Jul '04

19 Jul '04

At Thu Jul 15 03:44:25 CEST 2004, Jeff Chan wrote: >On Thursday, July 15, 2004, 2:21:49 AM, Robert Brooks wrote: > > William Stearns wrote: > >> TOP SPAM RULES FIRED > >> ------------------------------------------------------------ > >> RANK RULE NAME COUNT PERCENT > >> ------------------------------------------------------------ > >> -> 1 URIBL_WS_SURBL 13057 5.36% > >> -> 2 URIBL_SBL 12907 5.30% > > > does sbl.spamhaus.org work with Mail-SpamAssassin-SpamCopURI? The few > spam > > (uri) domains I've checked don't seem to return records. > >sbl is not really intended to be used with message body URI >checkers like SpamCopURI or urirhsbl, but it may get a few >hits since I think Spamhaus may include a few spam URI domains >in SBL. But the results will probably not be too useful or >productive, since it's not an intended use of sbl.spamhaus.org. Actually, having done some tests using uridnsbl under SA 3 as well as manual checks, I would say that SBL is an excellent tool for catching spam domains in message body URIs. I don't think everyone is aware of what uridnsbl, as an alternative to urirhsbl/urirhssub, actually does, so I'll try to explain it. First - SBL does not just list IPs used by known spammers to relay mail. It lists any ips used by known spammers, for whatever purpose. That includes web sites as well as, and most importantly, dns servers. uridnsbl checks the ns records for domains in URIs, resolves those ns records to ip adresses, and then checks those IP adresses in SBL (by default - you can add/change what RBLs it checks). If any of the name servers for a domain is listed in SBL, you get a rule hit. Spammers does not change their dns servers nearly as often as they change domains. This means that most of all the new domains that spammers introduce hit the uridnsbl SBL rule immediately, even if the domain hasn't been reported to any blacklist yet. I picked the 10 most recently reported domains to the SC blocklist and manually checked what dns servers they used, and if the IPs for those dns servers where already listed in SBL. For 9 out of 10, they where. Data included below. This doesn't mean that we should list resolved IPs in SURBL lists. Since there is already good data in SBL, there is no reason to. But - I think it would be a good idea to encourage SURBL implementations to include functionality similar to uridnsbl in addition to regular urirhsbl-style SURBL list checks. For me, it's the main reason why I plan to update all servers to SA 3 ASAP, as it's not possible to do this with SA 2.63 and SpamCopURI. Also - as long as you only check the ns records for a domain, rather than going further and resolving the host name in the URI, there isn't any need to fear "keyed domain name" address verification by spammers of the type discussed in the SURBL FAQ. /patrik -------------------------------------------------------------------------- 2004-07-18 09:08 digestion5594rneds.us digestion5594rneds.us nameserver = NS3.AIRMARAMBA.biz Name: NS3.AIRMARAMBA.biz Address: 61.250.93.207 SBL listed - http://www.spamhaus.org/query/bl?ip=61.250.93.207 digestion5594rneds.us nameserver = NS2.AUDI56SEW.biz Name: NS2.AUDI56SEW.biz Address: 221.143.42.30 SBL listed - http://www.spamhaus.org/query/bl?ip=221.143.42.30 -------------------------------------------------------------------------- 2004-07-18 09:10 acdfiaj.info acdfiaj.info nameserver = second.muchaagua.info Name: second.muchaagua.info Address: 221.139.2.84 SBL listed - http://www.spamhaus.org/query/bl?ip=221.139.2.84 acdfiaj.info nameserver = first.muchaagua.info Name: first.muchaagua.info Address: 221.143.42.209 SBL listed: http://www.spamhaus.org/query/bl?ip=221.143.42.209 acdfiaj.info nameserver = third.muchaagua.info Name: third.muchaagua.info Address: 61.128.198.11 SBL listed - http://www.spamhaus.org/query/bl?ip=61.128.198.11 -------------------------------------------------------------------------- 2004-07-18 09:24 pro-svcs.com pro-svcs.com nameserver = ns2.3070.biz ns2.3070.biz internet address = 202.104.237.173 SBL listed - http://www.spamhaus.org/query/bl?ip=202.104.237.173 pro-svcs.com nameserver = ns3.3070.biz ns3.3070.biz internet address = 200.153.20.31 SBL listed - http://www.spamhaus.org/query/bl?ip=200.153.20.31 pro-svcs.com nameserver = ns1.3070.biz ns1.3070.biz internet address = 200.40.40.1 NOT SBL listed. -------------------------------------------------------------------------- 2004-07-18 10:35 tophgh.com tophgh.com nameserver = ns2.dns.com.cn Name: ns2.dns.com.cn Address: 218.244.47.6 NOT SBL listed. tophgh.com nameserver = ns1.dns.com.cn Name: ns1.dns.com.cn Address: 218.244.47.5 NOT SBL listed. -------------------------------------------------------------------------- 2004-07-18 11:26 fox621dryg.us fox621dryg.us nameserver = NS2.AUDI56SEW.biz Name: NS2.AUDI56SEW.biz Address: 221.143.42.30 SBL listed - http://www.spamhaus.org/query/bl?ip=221.143.42.30 fox621dryg.us nameserver = NS3.AIRMARAMBA.biz Name: NS3.AIRMARAMBA.bi Address: 61.250.93.207 SBL listed - http://www.spamhaus.org/query/bl?ip=61.250.93.207 -------------------------------------------------------------------------- 2004-07-18 12:08 polishebertikas.org polishebertikas.org nameserver = ns1.kaleinc-dns-server.org Name: ns1.kaleinc-dns-server.org Address: 201.3.240.234 SBL listed - http://www.spamhaus.org/query/bl?ip=201.3.240.234 polishebertikas.org nameserver = ns1.kaleinc-dns-server2.org Name: ns1.kaleinc-dns-server2.org Address: 201.3.240.234 SBL listed - http://www.spamhaus.org/query/bl?ip=201.3.240.234 polishebertikas.org nameserver = ns1.koleyfore.org Name: ns1.koleyfore.org Address: 211.158.15.58 SBL listed - http://www.spamhaus.org/query/bl?ip=211.158.15.58 -------------------------------------------------------------------------- 2004-07-18 12:20 greenpill.info greenpill.info nameserver = ns1.greenpill.info ns1.greenpill.info internet address = 219.148.49.244 SBL listed - http://www.spamhaus.org/query/bl?ip=219.148.49.244 greenpill.info nameserver = ns2.greenpill.info ns2.greenpill.info internet address = 219.148.49.245 SBL listed - http://www.spamhaus.org/query/bl?ip=219.148.49.245 -------------------------------------------------------------------------- 2004-07-18 13:20 medsparadise.info medsparadise.info nameserver = ns2.medsparadise.info ns1.medsparadise.info internet address = 219.148.49.244 SBL listed - http://www.spamhaus.org/query/bl?ip=219.148.49.244 medsparadise.info nameserver = ns1.medsparadise.info ns2.medsparadise.info internet address = 219.148.49.245 SBL listed - http://www.spamhaus.org/query/bl?ip=219.148.49.245 -------------------------------------------------------------------------- 2004-07-18 14:24 misogynist2527dryg.biz misogynist2527dryg.biz nameserver = www.misogynist2527dryg.biz Name: misogynist2527dryg.biz Address: 200.193.29.211 Aliases: www.misogynist2527dryg.biz SBL listed - http://www.spamhaus.org/query/bl?ip=200.193.29.211 -------------------------------------------------------------------------- 2004-07-18 14:32 hedhoncho.net hedhoncho.net nameserver = ns2.3070.biz ns2.3070.biz internet address = 202.104.237.173 http://www.spamhaus.org/query/bl?ip=202.104.237.173 hedhoncho.net nameserver = ns3.3070.biz ns3.3070.biz internet address = 200.153.20.31 http://www.spamhaus.org/query/bl?ip=200.153.20.31 hedhoncho.net nameserver = ns1.3070.biz ns1.3070.biz internet address = 200.40.40.1 NOT SBL listed. --------------------------------------------------------------------------

3 8

SURBL querying engine?
by Shashank Tripathi 19 Jul '04

19 Jul '04

Hello I am looking for some simple PHP based functionality that will allow me to query the SURBL database for a domain name and confirm if it is a spamvertised one or not. Is there any engine for this? I would like this functionality on hosted domains as well, which means setting up a DNS resolution etc may be impractical. All I need is some online location to which I can send an input (a URL, or just the domain name) and get back a "YES" or a "NO" indicating whether this domain name is clean or not. Thanks for any ideas! Shanx

3 3

6dos data into ds.surbl.org
by Jeff Chan 19 Jul '04

19 Jul '04

Chris Santerre asked on the spamassassin-users list whether ds.surbl.org was being updated with the 6dos data. The answer is no, it's static now based on that snapshot Chris gave me a couple weeks ago. To be honest, the 6dos data seems too over-inclusive to me for production use, but perhaps Chris or Rich could help me figure out something closer to a live feed as we beta test it. Jeff C.

2 2

Fwd: Re: ob.surbl.org FP
by Jeff Chan 18 Jul '04

18 Jul '04

Brian Godette noticed on spamassassin-users that kerrypicksedwards.com showed up on ob.surbl.org last week ago. That started a fairly long thread, to which I responded below. From: Jeff Chan To: spamassassin-users Date: Saturday, July 17, 2004, 11:22:23 PM Subject: ob.surbl.org FP OK here are some of my responses to the themes in this thread: 1. I've whitelisted: kerrypicksedwards.com, johnkerry.com, bush's site and some other candidate sites. (By whitelisting I mean preventing these domains from getting onto SURBLs. There is no whitening or score improvement of your incoming messages with the SURBL whitelists.) 2. Some people seem more comfortable sending non-commercial spam out. It's still spam, but in their religious or political fervor they think they're justified, which of course they are not. And any site that has unconfirmed opt-in is just begging for this kind of abuse by zealots (and opponents) also. 3. Justin is right that's it's important to keep domains of sites that might get mentioned in legitimate messages (ham) out of SURBLs, otherwise false positives are possible. In this sense CAN-SPAM is irrelevant since any partially whitehat sites need to not get listed. In this sense the standards for inclusion in SURBLs need to be higher than sender IP or domain RBLs. Blocking messages based on URIs has potentially much larger effect on mail in general than blocking a specific zombie or rogue mail server. The focus of SURBLs therefore should be on the hard core professional criminal spammers' domains first. 4. The outblaze data is based on their spam traps, so they *are* getting spam from these when they should not (for example outside the U.S.). The whitelists take precedence over any input data feeds like Outblaze, so they will prevent list inclusion. 5. Outblaze has another interesting idea in that only domains less than 90 days old are listed. The idea is that spammers burn through domains quickly so the really recently registered ones are more likely to be spammers'. This is only for domains that actually get delivered into their spam traps, and it's probably a good idea. 6. Outblaze may do some additional processing before we get their data, but in general it appears to be quite good. We iterated a bit by sharing whitelist hits, etc. so we could focus their feed on the most often spamming URI domains. So it may take more than a single trap hit to get onto their list, and in addition to the 90 day newness of registration factor, their data seems quite good in general. Jeff C.

1 0

← Newer
1
...
5
6
7
8
9
10
11
...
16
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss July 2004