- Discuss - Mailman 3

Spamcop conflict of interest ?
by Simon Byrnand 02 May '04

02 May '04

Hmm....just noticed these comments in a recent slashdot article. Being slashdot it's hard to know whether to take it seriously or not ;-) http://yro.slashdot.org/comments.pl?sid=105978&cid=9022567 Regards, Simon

5 8

Re: better TLD handling
by Jeff Chan 02 May '04

02 May '04

On Friday, April 30, 2004, 7:12:10 PM, Justin Mason wrote: > ... is now in SpamAssassin SVN trunk. it uses quite a lot of data > from the SURBL list of TLDs and their subdomains, btw, thanks! ;) Hehe, I assumed *you* guys had the ccTLD magic bullet. ;-) Will check it out! (I hate to ask but is there any kind of direct URL access into SVN so I don't need to install the entire structure?) An open request to everyone: if you know of or can find any registrar lists of ccTLDs that are not already in our list: http://spamcheck.freeapp.net/two-level-tlds then please forward them to me for addition. I think we're lacking places like Deutschland .de and Italia .it, and probably some smaller ones also. N.B. found at http://www.denic.de/de/domains/index.html and added: moebel.de glueckwunsch.de buecher.de boerse.de kueche.de buero.de fluege.de kuechen.de aerzte.de reisebuero.de Surely there are many more. (Those look strange without Umlauten... :-) This is for country code TLDs which should themselves be whitelisted. Doing so allows us to check the third levels after them. Therefore we want: category.country but not domain.country where an abstract category is like doctors, a city name, etc. where a potential spammer would register at a lower level like spamming.doctor.be or spamming-subdomain.london.uk . We don't want to whitelist spammer.uk so please don't include registered base domains that terminate at the second level. TIA for any help with this, Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

2 2

Problems with uri/_foreign
by Matt Yackley 01 May '04

01 May '04

Hi all, I'm hoping that someone can whack me with a clue-stick here. I'm running into problems getting SURBL up and running. System: RedHat 9 SA 2.63 Amavisd-new SpamCopURI 0.14 I downloaded and ran through the install of the SpamCopURI 0.14 package, put my rules in place, and restarted my services. Here is the error that I receive, each time a messae passes through the system: Failed to run SPAMCOP_URI_RBL SpamAssassin test, skipping: (Can't locate URI/_foreign.pm in @INC (@INC contains: lib /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl) at /usr/lib/perl5/site_perl/5.8.0/URI.pm line 56. ) 1. I re-copied the three .pm files from the source, like the install docs advised. 2. The _foreign.pm file does exist at /usr/lib/perl5/site_perl/5.8.0/uri/_foreign.pm 3. CPAN claims that URI is up to date 4. A quick google search and search of the SURBL archives don't seem to offer much info on this error I'm sure that I'm missing something silly, so any pointers would be great. Thanks, Matt

2 4

Good news/bad news on .us ccTLDs
by Jeff Chan 01 May '04

01 May '04

1. Found a list of .us reserved domains at: http://www.nic.us/registrars/fcfs/dotus_reservedlist_v3.zip 2. The list has 50k domains at 1 MB of text uncompressed, so it will probably be somewhat impractical to use on the client side. 3. It only needs to be processed on the data side to make sure none of the .us ccTLDs gets into the SRUBLs, and that's now in place. 4. There are very many more reserved domains as described at: http://www.nic.us/faqs/index.html#list_reserved_names as: > Where can I see a list of reserved .US names? > See http://www.neustar.us/registrars/. In addition to the > specific names contained in this reserved name file, the > following categories of names are also reserved: > > * All numbers five digits and higher > * All numbers in the format five digits-four digits (zip codes) > * All telephone numbers including toll-free numbers > * Tagged domain namesall labels with hyphens in the third > and fourth character positions (e.g. "bq--kn2n4h4b") 5. It's very unlikely that any of these geographic or reserved domains will become a major source of spam hosting. I don't envision city or country registrars handing subdomains under reserved domains to random spammers. I had a heck of a time trying to get the late, lamented Jon Postel to let the City of Mountain View change theirs from mtview or mtnview to mountainview.... I know things are different now, but I still doubt reserved subdomains are going to be handed out like Mardi Gras candy or anything like that. :-) 6. Note that none of this prevents spammerdomain.us from getting onto the blocklists. That will still happen as it should. Jeff C.

1 0

Re: better TLD handling
by Daniel Quinlan 01 May '04

01 May '04

It's lib/Mail/SpamAssassin/Util/RegistrarBoundaries.pm -- Daniel Quinlan anti-spam (SpamAssassin), Linux, http://www.pathname.com/~quinlan/ and open source consulting

1 0

[SURBL-Discuss] Re: Don't want to give ham lists for creating whitelists
by Daniel Quinlan 01 May '04

01 May '04

For what it's worth, most of the URLs hitting SURBL in my ham are spam URLs anyway. On Saturday, 140 messages hit BE or WS SURBL out of 14971 total ham in my corpus. Retesting those 140 messages now, only 105 messages hit, so you've probably removed some domains since Friday. Hopefully, because they're not spamming domains. In those 105 messages, there were about 123 hits (40 when looking for unique domain-rule pairs) on 34 different domains as follows: count domain rule 80 xxxxx.xxx URIBL_WS_SURBL 3 xxxxxxxxxx.xxx URIBL_WS_SURBL 2 xxxxxxxxx.xxx URIBL_WS_SURBL 2 xxxxxxxxxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxx.xxx URIBL_BE_SURBL 1 xxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxx.xxx URIBL_BE_SURBL 1 xxxxxxxxxxxxxxx.xxxx URIBL_WS_SURBL 1 xxxxxxxxxxxxxxx.xxxx URIBL_BE_SURBL 1 xxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxx.xxx URIBL_BE_SURBL 1 xxxxxxxxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxxxxxxxx.xxx URIBL_BE_SURBL 1 xxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxx.xxx URIBL_WS_SURBL 1 xxxx.xxx URIBL_BE_SURBL 1 xxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxx.xxx URIBL_BE_SURBL 1 xxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxxxx.xxx URIBL_WS_SURBL 1 x-xxxxxx.xxx URIBL_BE_SURBL 1 xxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xx-xxxxxx-xxxxxxxxxx.xxx URIBL_WS_SURBL Most of them look like spam domains. -- Daniel Quinlan anti-spam (SpamAssassin), Linux, http://www.pathname.com/~quinlan/ and open source consulting

2 1

[SURBL-Discuss] Re: Don't want to give ham lists for creating whitelists
by Daniel Quinlan 01 May '04

01 May '04

Jeff Chan <jeffc(a)surbl.org> writes: > Your arguments make perfect sense in terms of SA development > and use, but hopefully my reasons for asking also make some > sense. Not really once you consider than most of the listed URLs in our ham are actually spam domains being discussed in the context of anti-spam efforts. Looking at URLs in the body is never going to have a perfect S/O ratio -- even non-developers discuss spam, forward funny spam, innocently link to interesting content on primarily spamvertized sites, etc. > I guess it's time to stop asking. :-( There are plenty of ways for you to get whitelist URLs without relying on our corpora. -- Daniel Quinlan anti-spam (SpamAssassin), Linux, http://www.pathname.com/~quinlan/ and open source consulting

2 1

[SURBL-Discuss] Don't want to give ham lists for creating whitelists
by Daniel Quinlan 01 May '04

01 May '04

(was: Want your ham lists for creating whitelists (Was: Re: [SURBL-Discuss] This ROCKS!)) Jeff Chan <jeffc(a)surbl.org> writes: > Is there any way we can get the message body URI domains (or even raw > messages) from some of the ham lists? This is a really really really really really really bad idea. > The intent is not to "cheat" the S/O scores, but to compile a good, > hand checked whitelist of legitimate message body domains. *I don't > really care where they come from*, but we have this nice, untapped > source of them just sitting there, and a glaring need for some > legitimate domains to keep off the blocklists.... I'm very concerned that if people who plan on submitting their corpus results into the SpamAssassin mass-check process for 3.0 will end up with no ham hits whereas your average user will have some. Cheating is not the word I would use, but it would completely completely throw off our GA process by making it look like SURBL cannot ever issue a false positive, screwing over non-developers. Even if you only get results from people who are never going to submitting corpus results, you're going to end up with a very heavy whitelist bias towards technical users, penalizing non-technical users who will have the rule scored too high compared to what it should be. > I'm results oriented. If there are some good hand-checked ham lists, > I see no practical reason why we should not use them to generate > message body domain whitelists. See above. It would be much better if you came up with other methods for generating a whitelist besides using our benchmark data which is comparatively tiny compared to the world of ham. One other note: just because a URL comes out of ham does *not* mean it should be listed in SURBL. I expect spam URLs to be present in some ham, discussion of spam mostly, so there should always be a small false positive rate (and the rules should be appropriately scored). Daniel -- Daniel Quinlan anti-spam (SpamAssassin), Linux, http://www.pathname.com/~quinlan/ and open source consulting

2 1

Re: [SURBL-Discuss] This ROCKS!
by Daniel Quinlan 01 May '04

01 May '04

Chris Santerre <csanterre(a)MerchantsOverseas.com> writes: > Hey there guys! This was the crazy idea I was discussing with Doc. I wished > for a realtime form of DB or flat file to be updated continuously on rule > hits. No grepping thru logs or anything. Simply when an email is sent thru > SA, whatever rules hit, increase a counter in a db or flat file for that > rule. Seperate db or flat file for ham and spam. This gives live stats on a > system. No grep'n going on. Just a counter per rule. That might give you a good hit rate, but it won't give you an accurate S/O number. > This is to be used on some advanced rule writing we want to work on. It also > alows an admin to see what might not be worth keeping around. Allowing them > to remove poor performers and increase system speed. Sort of like http://www.pathname.com/~corpus/DETAILS.new ? The corpora have to be sorted by humans to be accurate and runs need to be synchronized so everyone tests the same rules so runs only happen once a day, which is fast enough. We've been doing this for well over a year and it works great. If only we had more active developers working on rules... Daniel -- Daniel Quinlan anti-spam (SpamAssassin), Linux, http://www.pathname.com/~quinlan/ and open source consulting

2 1

This ROCKS!
by Doc Schneider 30 Apr '04

30 Apr '04

Just wanted to say how much this software and spam fighting tool really Rock and rolls them spams into neat little balls and tosses them into never never land. I am on my days off and hope to get some type of stats gathering ditty going. But not sure what stats folks want to see. I had some conversations with Chris "The Big Evil One" and he had some ideas. Am just not sure what direction to pursue. I did join the dev list for SA and sort of got talked into checking the docs for SA 3.0 which I'm slowly working on. Just checking them for proper API coverage in the different modules it is using. So my question to you folks is what sort of statistics would you like to see? Number of rules hit, even if a ham (I always get hungry when talking about ham and spam but I digress) or just total number and the actual rules that are being hit for all spam? And isn't there something already in SA that does these rules and hits? corporra(sic and too lazy to look). Anyway let me know what you all would like and I can whip something up. -- -Doc (The D - Ninja)

3 4

RE: [SURBL-Discuss] This ROCKS!
by Chris Santerre 30 Apr '04

30 Apr '04

>-----Original Message----- >From: jm(a)jmason.org [mailto:jm@jmason.org] >Sent: Friday, April 30, 2004 1:10 PM >To: SURBL Discussion list >Cc: SpamAssassin-dev(a)incubator.apache.org >Subject: Re: [SURBL-Discuss] This ROCKS! > > >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > > >Doc Schneider writes: >>I am on my days off and hope to get some type of stats >gathering ditty >>going. But not sure what stats folks want to see. I had some >>conversations with Chris "The Big Evil One" and he had some ideas. >>Am just not sure what direction to pursue. >>I did join the dev list for SA and sort of got talked into >checking the >>docs for SA 3.0 which I'm slowly working on. Just checking them for >>proper API coverage in the different modules it is using. >> >>So my question to you folks is what sort of statistics would >you like to >>see? Number of rules hit, even if a ham (I always get hungry when >>talking about ham and spam but I digress) or just total >number and the >>actual rules that are being hit for all spam? And isn't there >something >>already in SA that does these rules and hits? corporra(sic >and too lazy >>to look). > >Hi Doc -- cc'ing sa-dev, since it's really an SpamAssassin thing >rather than a SURBL thing ;) > >I'm not sure if you mean measuring rule accuracy in advance to >pick good >scores, or reporting stats after the fact for sysadmins. > >For the first one, read: > > http://wiki.apache.org/spamassassin/MassCheck > http://wiki.apache.org/spamassassin/HitFrequencies > >For the second: > >We recently added some additional stats output to spamd in SpamAssassin >3.0.0. This should improve the accessibility of info about rules >being used during scanning for tools to summarise. > >- --j. Hey there guys! This was the crazy idea I was discussing with Doc. I wished for a realtime form of DB or flat file to be updated continuously on rule hits. No grepping thru logs or anything. Simply when an email is sent thru SA, whatever rules hit, increase a counter in a db or flat file for that rule. Seperate db or flat file for ham and spam. This gives live stats on a system. No grep'n going on. Just a counter per rule. This is to be used on some advanced rule writing we want to work on. It also alows an admin to see what might not be worth keeping around. Allowing them to remove poor performers and increase system speed. HTH Chris Santerre System Admin and SARE Ninja http://www.rulesemporium.com 'It is not the strongest of the species that survives, not the most intelligent, but the one most responsive to change.' Charles Darwin Charles Darwin

1 0

query failed
by Grant Baxter 29 Apr '04

29 Apr '04

Just installed spamcop uri .14. I ran spamd in debug mode just to make sure everything is alright. For every email I've seen so far I see a bunch of "query failed yahoo.com.sc.surbl.org; query failed ying.com.sc.surbl.org; query failed usda.gov.sc.surbl.org; query failed yahoogroups.com.sc.surbl.org". However, I received one email that shows a hit on "SPAMCOP_URI_RBL " with a value of 3. Did I do something wrong when I set this up? Or is this normal behavior? TIA, grant

2 1

[SURBL-Discuss] RE: Add "$DATASET dnset @" directive to SURBL rbldnsd zone files?
by Dallas L. Engelken 28 Apr '04

28 Apr '04

> > > >> Can we please add a $DATASET definition to rbldnsd zone > >files for sc,ws, > >> and be? > >> If this gets added, rbldnsd operators would need to change their invocation of rbldnsd from zone:dnset:file to zone:combined:file To run a single zone... Because dnset does not expect to set $DATASET... Or if they want to combine lists into a single zone, use zone:combined:befile,scfile,wsfile how about making a duplicate of each set available with the $DATASET line added... So you have something like this, [root@rhn-1 surbl]# ll total 1040 -rw-r--r-- 1 root root 112368 Apr 28 10:22 be.surbl.org.dataset.rbldnsd -rw-r--r-- 1 root root 112268 Apr 27 06:13 be.surbl.org.rbldnsd -rw-r--r-- 1 root root 13957 Apr 28 10:22 mw.surbl.org.dataset.rbldnsd -rw-r--r-- 1 root root 13857 Apr 24 00:15 mw.surbl.org.rbldnsd -rw-r--r-- 1 root root 185420 Apr 28 10:22 sa.surbl.org.dataset.rbldnsd -rw-r--r-- 1 root root 185320 Apr 27 06:12 sa.surbl.org.rbldnsd -rw-r--r-- 1 root root 9416 Apr 28 10:22 sc.surbl.org.dataset.rbldnsd -rw-r--r-- 1 root root 9316 Apr 27 23:47 sc.surbl.org.rbldnsd -rw-r--r-- 1 root root 185420 Apr 28 10:22 ws.surbl.org.dataset.rbldnsd -rw-r--r-- 1 root root 185320 Apr 27 06:12 ws.surbl.org.rbldnsd That would allow you to combine zones -> uribl.yourdomain.com:combined:be.surbl.org.dataset.rbldnsd,sc.surbl.org. dataset.rbldnsd,ws.surbl.org.dataset.rbldnsd Or still run each individual zone by -> be.yourdomain.com:dnset:be.surbl.org.rbldnsd -> ws.yourdomain.com:dnset:ws.surbl.org.rbldnsd -> sc.yourdomain.com:dnset:sc.surbl.org.rbldnsd And not create any issues, besides having to maintain duplicate zone files. Dallas

1 0

[SURBL-Discuss] RE: Add "$DATASET dnset @" directive to SURBL rbldnsd zone files?
by Chris Santerre 28 Apr '04

28 Apr '04

>-----Original Message----- >From: Jeff Chan [mailto:jeffc@surbl.org] >Sent: Tuesday, April 27, 2004 10:07 PM >To: SURBL Discuss >Cc: SpamAssassin Users >Subject: RFC: Add "$DATASET dnset @" directive to SURBL rbldnsd zone >files? > > >Dallas Engleken of SARE has suggested that we add a >dataset directive to our rbldnsd zone files: > >> Can we please add a $DATASET definition to rbldnsd zone >files for sc,ws, >> and be? >> >> Ie.. On the 3rd line after $NS and $SOA, add a line labeled, >> >> $DATASET dnset @ >> >> It will not break anything currently set up, but it will >give those of >> use that use the 'combined' type with multi files in rbldnsd >(called via >> uribl.surbl.org:combined:sc,be,ws) to merge ws, sc, and be >together to >> create a single query. > >Does anyone have any comments on this, good, bad or otherwise? >Do other RBLs do it? Is it safe? > >Here's the man page entry: :-) > >> man rbldnsd >> >> combined >> This is a special dataset that stores no data >by itself but acts >> like a container for several other datasets of >any type except >> of combined type itself. The data file >contains an optional >> common section, where various specials are >recognized like $NS, >> $SOA, $TTL (see above), and a series of >sections, each of which >> defines one (nested) dataset and several >subzones of the base >> zone, for which this dataset should be >consulted. New (nested) >> dataset starts with a line >> $DATASET type subzone subzone... >> and all subsequent lines up to the end of >current file or to >> next $DATASET line are interpreted as a part >of dataset of type >> type. Note that combined datasets cannot be >nested. Every subâ >> zone will always be relative to the base zone >name specified on >> command line. If subzone specified as >single character "@", >> dataset will be connected to the base zone itself. >> >> This dataset type aims to simplify subzone >maintenance, in order >> to be able to include several subzones in one >file for easy data >> transfer, atomic operations and to be able >to modify list of >> subzones on remote secondary nameservers. >> >> Note that $NS and $SOA values applies to the >base zone only, >> regardless of the placement in the file. >Unlike the $TTL values >> and $n substitutions, which may be both global >and local for a >> given (subâ)dataset. > >Thumbs up or thumbs down? ;-) > >Jeff C. >-- But we are combining them anyway? Well WS and BE combined. So it would be WS and SC. I see no reason not to try it. --Chris

1 0

Re: second and third level domains - again!
by John Fawcett 28 Apr '04

28 Apr '04

Jeff Chan wrote on Mon Apr 26 03:58:58 CEST 2004 > The underlying principle as I see it is that most > major site will have functional anti-abuse and anti-spam > policies, so either a base domain is good or bad. I > know that seems simplistic, but it's easy and fast to > implement AND it seems to match reality pretty well. There may come a point where a "big" domain starts to appear in spam despite an outward image of "antispam" and the choice which the current infrastructure offers is to block all of it or none of it. Blocking all of it may produce too many FPs whereas blocking a subset may be acceptable. At the moment there haven't been any tough decisions to take on listing or whitelisting. Everything has been very clear cut. It's not guaranteed to stay that way. I don't know how real a risk this is, but I was worried about an infrastructure which effectively ties our hands on this point. (Once surbl is deployed in many different client software, I suppose it will be hard to change anything about the "public interface" to the data....) > That's an interesting idea. Basically you want to > signal redirection to higher domain levels with a > special result for levels that should never get > checked like co.uk. > That might be doable, but it would require extra > logic on the client side as you note. That already > sounds more complex than I like, though I see what > you're getting at. Better to control what goes into > the data (i.e. never let the TLD itself co.uk in), > and make sure the client is following similar rules. What I was proposing is a *change* to client processing logic, but not added *complexity*. rather a simplification. What I mean is that currently the client has to contain - processing logic - data on the ccTLDs (in the long run there will be multiple versions of the ccTLD data implemented in various client versions at any one time. Users will have to upgrade the client software to keep current with domain data). Using an A record in surbl to indicate "this domain is not listed but a subdomain is. Try again" would mean the client just follows a simple processing rule. It doesn't need to know anything about specific domain data. The logic would be something like this. lets assume that "url" is found in an email. level=2 more_info_result = '127.0.0.255' listed_result = '127.0.0.2' do { result = query_dns ( extract_domain (url, level)) level = level +1 }while (result == more_info_result) if (result == listed_result) { score it } > We will always catch bigspammer.co.uk with the current > strategy. I think that some of the ccTLDs have a mixed assignment strategy. This means that they should sometimes be checked at the 2nd level sometime at the 3rd level. The current logic always checks at a single predefined level. The two example I saw were: .fr and .ca Currently we check .ca at the third level, but it is possible to register a second level domain at .ca which we never catch so bigspammer.ca will get through. We check .fr at the second level however there are many "standard" second level domains (like .nom.fr) which means we probably want to be checking these ones at the third level. (Translation: any bigspammer.nom.fr domain is imune to the current strategy unless we want to upset everyone who has a nom.fr domain by listing that). > I know a lot of what I argue for above seems simplistic > when a more complex solution could have more interesting > results, but very often the simpler solutions are better, > especially in terms of resource consumption. You're right to argue for the most simple solution. (In fact there are probably simpler solutions than the one I suggested!). My concern was to avoid inflexibility in the infrastructure and automatic immunity for various classes of domains. John

4 5

access.db -> Bind zone file converter
by David Hooton 28 Apr '04

28 Apr '04

Hi All, We're wanting to move a bunch of our rather huge sendmail access files into bind compliant zone files. Does anyone have a tool that can do this in a slightly more automated way than our current options? We're also looking for a tool to convert a domain list into our own surbl zone (for the moment private, but we may release it later) again any pointers greatly appreciated :-) Regards, David Hooton Senior Partner Platform Networks www.platformnetworks.net ======================================================================== Pain free spam & virus protection by: www.mailsecurity.net.au Forward undetected SPAM to: spam(a)mailsecurity.net.au ========================================================================

4 6

[SURBL-Discuss] RFC: Add "$DATASET dnset @" directive to SURBL rbldnsd zone files?
by Jeff Chan 28 Apr '04

28 Apr '04

Dallas Engleken of SARE has suggested that we add a dataset directive to our rbldnsd zone files: > Can we please add a $DATASET definition to rbldnsd zone files for sc,ws, > and be? > > Ie.. On the 3rd line after $NS and $SOA, add a line labeled, > > $DATASET dnset @ > > It will not break anything currently set up, but it will give those of > use that use the 'combined' type with multi files in rbldnsd (called via > uribl.surbl.org:combined:sc,be,ws) to merge ws, sc, and be together to > create a single query. Does anyone have any comments on this, good, bad or otherwise? Do other RBLs do it? Is it safe? Here's the man page entry: :-) > man rbldnsd > > combined > This is a special dataset that stores no data by itself but acts > like a container for several other datasets of any type except > of combined type itself. The data file contains an optional > common section, where various specials are recognized like $NS, > $SOA, $TTL (see above), and a series of sections, each of which > defines one (nested) dataset and several subzones of the base > zone, for which this dataset should be consulted. New (nested) > dataset starts with a line > $DATASET type subzone subzone... > and all subsequent lines up to the end of current file or to > next $DATASET line are interpreted as a part of dataset of type > type. Note that combined datasets cannot be nested. Every subâ > zone will always be relative to the base zone name specified on > command line. If subzone specified as single character "@", > dataset will be connected to the base zone itself. > > This dataset type aims to simplify subzone maintenance, in order > to be able to include several subzones in one file for easy data > transfer, atomic operations and to be able to modify list of > subzones on remote secondary nameservers. > > Note that $NS and $SOA values applies to the base zone only, > regardless of the placement in the file. Unlike the $TTL values > and $n substitutions, which may be both global and local for a > given (subâ)dataset. Thumbs up or thumbs down? ;-) Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

3 6

another redirector ?
by Simon Byrnand 28 Apr '04

28 Apr '04

Just spotted the following redirected URL in a spam. Doesn't look like it will be getting caught yet with the current redirector rules: http://images.google.ca/imgres?imgurl=gmib.free.fr/viagra.jpg&imgrefurl… Using images.google.ca as a redirector ? Thats a new one.... I'm not game to click on the link to see where it goes though... its from the same spammer that was blatently abusing the yahoo redirectors and msn ones... Is this a sign that the current system used in SpamCopURI (checking HTTP responses of specifically mentioned redirectors) is just going to play catchup all the time ? Regards, Simon

6 7

url parsing
by Eric Kolve 28 Apr '04

28 Apr '04

I have decided to add some URL parsing to SpamCopURI. I figure it will be kind of fun and should help catch a few of these edge cases that have popped up. If a few people could email me example redirects you have seen (you may want to tar.gz them up so they don't get filtered), that would helpful in making this more robust. I have examples for g.msn.com, ads.msn.com, rd.yahoo.com, and images.google.ca I am mainly looking for stuff like the URL Simon sent where the spam URL is hex encoded and is buried beneath three or four redirects. thanks, --eric

1 0

Re: [SURBL-Discuss] Simon's complex redirection
by Patrik Nilsson 28 Apr '04

28 Apr '04

At 10:55 2004-04-28 +1200, Simon Byrnand wrote: > > It's double-encoded. We can catch that easily. But first, my question -- > > does this *work* in an MUA, ie. should we? Simon, could you try it? > >What you get is the image preview in google which consists of an image in >the top frame, and the page that it came from in the bottom frame, and in >the bottom frame was a link "click here for ......." so yes it definately >does work... I guess this is a 'framer' rather than a redirector. A url that points to a frameset that loads an external page specified in the query url in one of the frames. For all practical purposes it's similar to a redirector, but with complications. The http response code from a GET request using the url will not indidate that it's a redirector. For a user loading the page in a browser, the adress of the spamvertized website will not be visible in the location bar. For some spammers, this would probably be prefered over traditional redirectors. Patrik

2 1

Re: Simon's complex redirection
by jm＠jmason.org 28 Apr '04

28 Apr '04

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Theo Van Dinter writes: > On Tue, Apr 27, 2004 at 03:46:32PM -0700, Justin Mason wrote: > > > http://images.google.ca/imgres?imgurl=gmib.free.fr/viagra.jpg&imgrefurl… > > eek! > > > It's double-encoded. We can catch that easily. But first, my question -- > > does this *work* in an MUA, ie. should we? Simon, could you try it? > > I don't know about an MUA, but it doesn't work in any browser I've tried > so far. You end up at http://images.google.com/images: > > Information returned from request (not including data): > <Connection> = <Keep-Alive> > <Content-Type> = <text/html> > <Content-length> = <161> > <Date> = <Tue, 27 Apr 2004 22:59:47 GMT> > <Location> = <http://images.google.com/images> > <Server> = <GWS/2.1> > <Set-Cookie> = <PREF=ID=2342b69354b6a6f5:LD=en:TM=1083106787:LM=1083106787:S=4Q5Kb3zcm_6V0Tjm; expires=Sun, > 17-Jan-2038 19:14:07 GMT; path=/; domain=.google.ca> > <code> = <301> Try http://images.google.ca/imgres?imgurl=gmib.free.fr/viagra.jpg&imgrefurl=htt… (NOTE: with & replaced with just &) that works. (although it still uses google's image results page for the display.) But just because it ends up at the imgres page doesn't mean we don't have to worry about it. This URL: http://www.google.com/url?q=http://www.google.com/url?q=%68%74%74%70%3A%2F%… also works -- Google redirects it to the target site http://www.expage.com/manger32 successfully -- but SpamAssassin doesn't parse it. - --j. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Exmh CVS iD8DBQFAjurMQTcbUG5Y7woRAi2KAKDkGfoSzXWzzP73ggTr0FEZYlbEEgCg7861 JrRcI4laMJDWumLpHAbWaT4= =/Hlu -----END PGP SIGNATURE-----

1 0

RE: [SURBL-Discuss] Fwd: Re: Goofy domain names
by Chris Santerre 26 Apr '04

26 Apr '04

>-----Original Message----- >From: Mark [mailto:admin@asarian-host.net] >Sent: Saturday, April 24, 2004 12:11 AM >To: David B Funk; SURBL Discussion list >Cc: SpamAssassin Users >Subject: Re: [SURBL-Discuss] Fwd: Re: Goofy domain names > > >David B Funk wrote: > >> On Sat, 24 Apr 2004, Mark wrote: >> >>> Why use expand_regex.pl at all? Instead of taking "/\d00\dhosting/", >>> and expand it into 100 potential domain names, why not simply accept >>> a query like: >>> >>> 9001hosting.com.be.surbl.org >>> >>> And, internally, do the regex "/\d00\dhosting/" on it? (and all >>> BigEvil regexes, for that matter). And return 127.0.0.2 on a match. >>> >>> Ultimately, using expand_regex.pl, imho, is utterly >self-defeating -- >>> without stretching the imagination too much, even. If only I change >>> the above regex to: >>> >>> /\d+00\d+hosting/ >>> >>> We're already looking at trillions of permutations! >>> >>> It seems to me be.surbl.org should not do an internal database query >>> on single domain names, but instead do a BigEvil regex (like SA >>> does). At least, that is what I thought it would do; and how I had >>> hoped it would have been. >> >> Because the DNS system is designed to work with static lists of data, >> not regex patterns. >> >> Theoretically you could design a DNS server front-end on a regex >> engine that would take fixed DNS queries and do regex matches. >> However the DNS zone-transfer mechanism is designed to deal with >> fixed data-sets, not regexes. > >DNS zone-transfers would not be affected. It would still be >the same, static >data-set being pushed back and forth. In fact, the >be.surbl.org DNS server >would not contain a regular DNS database at all (!), as it merely does >regexes on incoming queries! If anything, you'd have to transer BigEvil >regex rules, really. > >> Another detail, is that the DNS system is heavily based upon the >> concept of cacheing. Currently there's no meaningful way to cache a >> regex, only the results of every lookup permutation. That would >> impose an unrealistic demand on the down-stream DNS servers (remember >> those trillions of permutations!), > >Not at the client-side. When I were to query >"9001hosting.com.be.surbl.org", >there would be no overhead whatsoever, cache-wise, for a front-end DNS >system that does a behind-the-scenes regex on the query. It would still >reply with a single response. It would, to the client, appear >no different >then if the DNS server had retrieved the domain name from database. > >The caching DNS server at be.surbl.org itself might grow big, >though, as it >potentially holds those trillions of those domain names. But >it would, as >you say, really only cache the result of each looked-up >permutation. And no >more. And, in this, it would be no different from a regular DNS server. > >> Another detail, you're adding to the CPU load on the "regex-ifed" DNS >> servers. Doing a regex match is more CPU intensive than a database >> lookup on a fixed data set. When you have the potential of thousands >> or millions of clients beating upon a handfull of DNS servers, the >> CPU load considerations become critical. > >This is true. And it may indeed be quite a problem. > >> Might as well stick with BigEvil.cf > >My point exactly; without using real regexes in be.surbl.org, dropping >BigEvil.cf, for now, is ill-advised, really. > >Cheers! > >- Mark Wow that was a lot to read for a Monday Morning :) Bigevil won't be dropped. But it will change. You will have lots of options in the near future. Lots of options ;) --Chris

1 0

problems when integrating SpamCopURI
by Arnold Nipper 26 Apr '04

26 Apr '04

Hi all, I just installed SpamCopURI (0.14) and restarted spamd. But now I get Apr 26 00:14:38 server spamd[6504]: Failed to compile URI SpamAssassin tests, skipping: (syntax error at /etc/mail/s pamassassin/spamcop_uri.cf, rule SPAMCOP_URI_RBL, line 1, near "eval:" syntax error at /usr/share/spamassassin/20_uri_tests. cf, rule E_MAILPROMO_URL, line 175, near "; }" ) Apr 26 00:14:39 server spamd[6504]: server started on port 783/tcp (running version 2.63) And further down a lot of lines like Apr 26 00:16:10 server spamd[16954]: Subroutine HTTP_ESCAPED_HOST_uri_test redefined at /usr/share/spamassassin/20_uri_tests .cf, rule HTTP_ESCAPED_HOST, line 5. Apr 26 00:16:10 server spamd[16954]: Subroutine BARGAIN_URL_uri_test redefined at /usr/share/spamassassin/20_uri_tests.cf, r ule BARGAIN_URL, line 8. Apr 26 00:16:10 server spamd[16954]: Subroutine URI_OFFERS_uri_test redefined at /usr/share/spamassassin/20_uri_tests.cf, ru le URI_OFFERS, line 8. ... So what's going wrong? Arnold

2 1

second and third level domains - again!
by John Fawcett 26 Apr '04

26 Apr '04

One of the things I noticed after upgrading to SpamCopURI 0.14 was that previously I had been identifying all mail containing ads.msn.com as spam and after the upgrade this was no longer happening. (BTW ads.msn.com was still listed in sc.surbl.org when I observed this behaviour. It has since been removed.) Version 0.14 includes the changes which were being discussed last week, so that if ads.msn.com is found in an email only msn.com is being checked against sc.surbl.org. So the choices available to the list maintainer are either: - list all of msn.com - list none of msn.com Since listing all of msn.com is likely to be too wide, this means msn.com will not get listed even if there are subdomains which are candidates for listing. I've used msn as an example, but the same logic applies to any of the big names like yahoo etc where the list maintainer may want to have more granularity in what is listed rather than list the whole registered domain. The solution could be to use a special return code which indicates "query again with more detail". (I remember someone bringing up something similar in the context of ccTLDs as well). So if ads.msn.com were to be listed in sc.surbl.org it would need two records: msn.com IN A 127.0.0.255 ads.msn.com IN A 127.0.0.2 The client (in this case SpamCopURI) would find a url ads.msn.com in the email but would query for msn.com as per the current logic. The return value of 127.0.0.255 then indicates to the client to query for one level lower, ie ads.msn.com. This same mechanism could be used for ccTLDs. sc.surbl.org could contain: co.uk IN A 127.0.0.255 co.nz IN A 127.0.0.255 So that if I get xxxxxxxx.co.uk in an email, the client queries for co.uk and it will be told to query with the lower level. The client queries for xxxxxxxx.co.uk The disadvantage of this solution is that it needs a changed logic on the client side, however the client does not need to know anything about which ccTLDs have two or three level domains. The extra query overhead is minimal, since on a busy server these additional records would almost always be in the DNS resolver cache. Even clients which don't implement this extra logic would however have to be careful to check the value of the A record rather than deduce the listing from the presence of an A record. Maybe we're still in time to make changes like this before there is too much software to change (currently I think there are 3). John

5 10

unique IPs in the blacklist
by Robert Brooks 26 Apr '04

26 Apr '04

how many unique IPs are there in the spamcop blacklist? has anyone considered creating a blacklist of the IP addresses of the hosts in the url blacklist? I'm guessing that the spammers are stuffing their sites on a relatively small number of webhosts. Regards, Rob -- Robert Brooks, Network Manager, Hyperlink Interactive Ltd <robb(a)hyperlink-interactive.co.uk> http://hyperlink-interactive.co.uk/ Tel: +44 (0)20 7240 8121 Fax: +44 (0)20 7240 8098 - Help Microsoft stamp out piracy. Give Linux to a friend today! -

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss