Discuss

discuss@lists.surbl.org

1876 discussions

Good news/bad news on .us ccTLDs
by Jeff Chan 01 May '04

01 May '04

1. Found a list of .us reserved domains at: http://www.nic.us/registrars/fcfs/dotus_reservedlist_v3.zip 2. The list has 50k domains at 1 MB of text uncompressed, so it will probably be somewhat impractical to use on the client side. 3. It only needs to be processed on the data side to make sure none of the .us ccTLDs gets into the SRUBLs, and that's now in place. 4. There are very many more reserved domains as described at: http://www.nic.us/faqs/index.html#list_reserved_names as: > Where can I see a list of reserved .US names? > See http://www.neustar.us/registrars/. In addition to the > specific names contained in this reserved name file, the > following categories of names are also reserved: > > * All numbers five digits and higher > * All numbers in the format five digits-four digits (zip codes) > * All telephone numbers including toll-free numbers > * Tagged domain namesall labels with hyphens in the third > and fourth character positions (e.g. "bq--kn2n4h4b") 5. It's very unlikely that any of these geographic or reserved domains will become a major source of spam hosting. I don't envision city or country registrars handing subdomains under reserved domains to random spammers. I had a heck of a time trying to get the late, lamented Jon Postel to let the City of Mountain View change theirs from mtview or mtnview to mountainview.... I know things are different now, but I still doubt reserved subdomains are going to be handed out like Mardi Gras candy or anything like that. :-) 6. Note that none of this prevents spammerdomain.us from getting onto the blocklists. That will still happen as it should. Jeff C.

1 0

Re: better TLD handling
by Daniel Quinlan 01 May '04

01 May '04

It's lib/Mail/SpamAssassin/Util/RegistrarBoundaries.pm -- Daniel Quinlan anti-spam (SpamAssassin), Linux, http://www.pathname.com/~quinlan/ and open source consulting

1 0

[SURBL-Discuss] Re: Don't want to give ham lists for creating whitelists
by Daniel Quinlan 01 May '04

01 May '04

For what it's worth, most of the URLs hitting SURBL in my ham are spam URLs anyway. On Saturday, 140 messages hit BE or WS SURBL out of 14971 total ham in my corpus. Retesting those 140 messages now, only 105 messages hit, so you've probably removed some domains since Friday. Hopefully, because they're not spamming domains. In those 105 messages, there were about 123 hits (40 when looking for unique domain-rule pairs) on 34 different domains as follows: count domain rule 80 xxxxx.xxx URIBL_WS_SURBL 3 xxxxxxxxxx.xxx URIBL_WS_SURBL 2 xxxxxxxxx.xxx URIBL_WS_SURBL 2 xxxxxxxxxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxx.xxx URIBL_BE_SURBL 1 xxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxx.xxx URIBL_BE_SURBL 1 xxxxxxxxxxxxxxx.xxxx URIBL_WS_SURBL 1 xxxxxxxxxxxxxxx.xxxx URIBL_BE_SURBL 1 xxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxx.xxx URIBL_BE_SURBL 1 xxxxxxxxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxxxxxxxx.xxx URIBL_BE_SURBL 1 xxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxx.xxx URIBL_WS_SURBL 1 xxxx.xxx URIBL_BE_SURBL 1 xxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxx.xxx URIBL_BE_SURBL 1 xxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxxxx.xxx URIBL_WS_SURBL 1 x-xxxxxx.xxx URIBL_BE_SURBL 1 xxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xxxxxxxxxxx.xxx URIBL_WS_SURBL 1 xx-xxxxxx-xxxxxxxxxx.xxx URIBL_WS_SURBL Most of them look like spam domains. -- Daniel Quinlan anti-spam (SpamAssassin), Linux, http://www.pathname.com/~quinlan/ and open source consulting

2 1

[SURBL-Discuss] Re: Don't want to give ham lists for creating whitelists
by Daniel Quinlan 01 May '04

01 May '04

Jeff Chan <jeffc(a)surbl.org> writes: > Your arguments make perfect sense in terms of SA development > and use, but hopefully my reasons for asking also make some > sense. Not really once you consider than most of the listed URLs in our ham are actually spam domains being discussed in the context of anti-spam efforts. Looking at URLs in the body is never going to have a perfect S/O ratio -- even non-developers discuss spam, forward funny spam, innocently link to interesting content on primarily spamvertized sites, etc. > I guess it's time to stop asking. :-( There are plenty of ways for you to get whitelist URLs without relying on our corpora. -- Daniel Quinlan anti-spam (SpamAssassin), Linux, http://www.pathname.com/~quinlan/ and open source consulting

2 1

[SURBL-Discuss] Don't want to give ham lists for creating whitelists
by Daniel Quinlan 01 May '04

01 May '04

(was: Want your ham lists for creating whitelists (Was: Re: [SURBL-Discuss] This ROCKS!)) Jeff Chan <jeffc(a)surbl.org> writes: > Is there any way we can get the message body URI domains (or even raw > messages) from some of the ham lists? This is a really really really really really really bad idea. > The intent is not to "cheat" the S/O scores, but to compile a good, > hand checked whitelist of legitimate message body domains. *I don't > really care where they come from*, but we have this nice, untapped > source of them just sitting there, and a glaring need for some > legitimate domains to keep off the blocklists.... I'm very concerned that if people who plan on submitting their corpus results into the SpamAssassin mass-check process for 3.0 will end up with no ham hits whereas your average user will have some. Cheating is not the word I would use, but it would completely completely throw off our GA process by making it look like SURBL cannot ever issue a false positive, screwing over non-developers. Even if you only get results from people who are never going to submitting corpus results, you're going to end up with a very heavy whitelist bias towards technical users, penalizing non-technical users who will have the rule scored too high compared to what it should be. > I'm results oriented. If there are some good hand-checked ham lists, > I see no practical reason why we should not use them to generate > message body domain whitelists. See above. It would be much better if you came up with other methods for generating a whitelist besides using our benchmark data which is comparatively tiny compared to the world of ham. One other note: just because a URL comes out of ham does *not* mean it should be listed in SURBL. I expect spam URLs to be present in some ham, discussion of spam mostly, so there should always be a small false positive rate (and the rules should be appropriately scored). Daniel -- Daniel Quinlan anti-spam (SpamAssassin), Linux, http://www.pathname.com/~quinlan/ and open source consulting

2 1

Re: [SURBL-Discuss] This ROCKS!
by Daniel Quinlan 01 May '04

01 May '04

Chris Santerre <csanterre(a)MerchantsOverseas.com> writes: > Hey there guys! This was the crazy idea I was discussing with Doc. I wished > for a realtime form of DB or flat file to be updated continuously on rule > hits. No grepping thru logs or anything. Simply when an email is sent thru > SA, whatever rules hit, increase a counter in a db or flat file for that > rule. Seperate db or flat file for ham and spam. This gives live stats on a > system. No grep'n going on. Just a counter per rule. That might give you a good hit rate, but it won't give you an accurate S/O number. > This is to be used on some advanced rule writing we want to work on. It also > alows an admin to see what might not be worth keeping around. Allowing them > to remove poor performers and increase system speed. Sort of like http://www.pathname.com/~corpus/DETAILS.new ? The corpora have to be sorted by humans to be accurate and runs need to be synchronized so everyone tests the same rules so runs only happen once a day, which is fast enough. We've been doing this for well over a year and it works great. If only we had more active developers working on rules... Daniel -- Daniel Quinlan anti-spam (SpamAssassin), Linux, http://www.pathname.com/~quinlan/ and open source consulting

2 1

This ROCKS!
by Doc Schneider 30 Apr '04

30 Apr '04

Just wanted to say how much this software and spam fighting tool really Rock and rolls them spams into neat little balls and tosses them into never never land. I am on my days off and hope to get some type of stats gathering ditty going. But not sure what stats folks want to see. I had some conversations with Chris "The Big Evil One" and he had some ideas. Am just not sure what direction to pursue. I did join the dev list for SA and sort of got talked into checking the docs for SA 3.0 which I'm slowly working on. Just checking them for proper API coverage in the different modules it is using. So my question to you folks is what sort of statistics would you like to see? Number of rules hit, even if a ham (I always get hungry when talking about ham and spam but I digress) or just total number and the actual rules that are being hit for all spam? And isn't there something already in SA that does these rules and hits? corporra(sic and too lazy to look). Anyway let me know what you all would like and I can whip something up. -- -Doc (The D - Ninja)

3 4

RE: [SURBL-Discuss] This ROCKS!
by Chris Santerre 30 Apr '04

30 Apr '04

>-----Original Message----- >From: jm(a)jmason.org [mailto:jm@jmason.org] >Sent: Friday, April 30, 2004 1:10 PM >To: SURBL Discussion list >Cc: SpamAssassin-dev(a)incubator.apache.org >Subject: Re: [SURBL-Discuss] This ROCKS! > > >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > > >Doc Schneider writes: >>I am on my days off and hope to get some type of stats >gathering ditty >>going. But not sure what stats folks want to see. I had some >>conversations with Chris "The Big Evil One" and he had some ideas. >>Am just not sure what direction to pursue. >>I did join the dev list for SA and sort of got talked into >checking the >>docs for SA 3.0 which I'm slowly working on. Just checking them for >>proper API coverage in the different modules it is using. >> >>So my question to you folks is what sort of statistics would >you like to >>see? Number of rules hit, even if a ham (I always get hungry when >>talking about ham and spam but I digress) or just total >number and the >>actual rules that are being hit for all spam? And isn't there >something >>already in SA that does these rules and hits? corporra(sic >and too lazy >>to look). > >Hi Doc -- cc'ing sa-dev, since it's really an SpamAssassin thing >rather than a SURBL thing ;) > >I'm not sure if you mean measuring rule accuracy in advance to >pick good >scores, or reporting stats after the fact for sysadmins. > >For the first one, read: > > http://wiki.apache.org/spamassassin/MassCheck > http://wiki.apache.org/spamassassin/HitFrequencies > >For the second: > >We recently added some additional stats output to spamd in SpamAssassin >3.0.0. This should improve the accessibility of info about rules >being used during scanning for tools to summarise. > >- --j. Hey there guys! This was the crazy idea I was discussing with Doc. I wished for a realtime form of DB or flat file to be updated continuously on rule hits. No grepping thru logs or anything. Simply when an email is sent thru SA, whatever rules hit, increase a counter in a db or flat file for that rule. Seperate db or flat file for ham and spam. This gives live stats on a system. No grep'n going on. Just a counter per rule. This is to be used on some advanced rule writing we want to work on. It also alows an admin to see what might not be worth keeping around. Allowing them to remove poor performers and increase system speed. HTH Chris Santerre System Admin and SARE Ninja http://www.rulesemporium.com 'It is not the strongest of the species that survives, not the most intelligent, but the one most responsive to change.' Charles Darwin Charles Darwin

1 0

query failed
by Grant Baxter 29 Apr '04

29 Apr '04

Just installed spamcop uri .14. I ran spamd in debug mode just to make sure everything is alright. For every email I've seen so far I see a bunch of "query failed yahoo.com.sc.surbl.org; query failed ying.com.sc.surbl.org; query failed usda.gov.sc.surbl.org; query failed yahoogroups.com.sc.surbl.org". However, I received one email that shows a hit on "SPAMCOP_URI_RBL " with a value of 3. Did I do something wrong when I set this up? Or is this normal behavior? TIA, grant

2 1

[SURBL-Discuss] RE: Add "$DATASET dnset @" directive to SURBL rbldnsd zone files?
by Dallas L. Engelken 28 Apr '04

28 Apr '04

> > > >> Can we please add a $DATASET definition to rbldnsd zone > >files for sc,ws, > >> and be? > >> If this gets added, rbldnsd operators would need to change their invocation of rbldnsd from zone:dnset:file to zone:combined:file To run a single zone... Because dnset does not expect to set $DATASET... Or if they want to combine lists into a single zone, use zone:combined:befile,scfile,wsfile how about making a duplicate of each set available with the $DATASET line added... So you have something like this, [root@rhn-1 surbl]# ll total 1040 -rw-r--r-- 1 root root 112368 Apr 28 10:22 be.surbl.org.dataset.rbldnsd -rw-r--r-- 1 root root 112268 Apr 27 06:13 be.surbl.org.rbldnsd -rw-r--r-- 1 root root 13957 Apr 28 10:22 mw.surbl.org.dataset.rbldnsd -rw-r--r-- 1 root root 13857 Apr 24 00:15 mw.surbl.org.rbldnsd -rw-r--r-- 1 root root 185420 Apr 28 10:22 sa.surbl.org.dataset.rbldnsd -rw-r--r-- 1 root root 185320 Apr 27 06:12 sa.surbl.org.rbldnsd -rw-r--r-- 1 root root 9416 Apr 28 10:22 sc.surbl.org.dataset.rbldnsd -rw-r--r-- 1 root root 9316 Apr 27 23:47 sc.surbl.org.rbldnsd -rw-r--r-- 1 root root 185420 Apr 28 10:22 ws.surbl.org.dataset.rbldnsd -rw-r--r-- 1 root root 185320 Apr 27 06:12 ws.surbl.org.rbldnsd That would allow you to combine zones -> uribl.yourdomain.com:combined:be.surbl.org.dataset.rbldnsd,sc.surbl.org. dataset.rbldnsd,ws.surbl.org.dataset.rbldnsd Or still run each individual zone by -> be.yourdomain.com:dnset:be.surbl.org.rbldnsd -> ws.yourdomain.com:dnset:ws.surbl.org.rbldnsd -> sc.yourdomain.com:dnset:sc.surbl.org.rbldnsd And not create any issues, besides having to maintain duplicate zone files. Dallas

1 0

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss