Discuss

discuss@lists.surbl.org

1862 discussions

Computer being repaired
by SURBL 16 Nov '04

16 Nov '04

Hi all, Please note that I may not be able to respond as quickly as I'd like for a day or so since I need to repair my main computer. This is only referring to my personal mail. SURBL stuff should run just fine, as usual. (Chris S., my PC locked up in JO. Now the video BIOS won't even come up.... :-( Power supply measures ok with no load, so maybe the motherboard is hosed. Might be the power supply though.) Jeff C.

2 1

RE: [SURBL-Discuss] Re: [SURBL-Announce] SURBL stats: list hit r a tes measured in DNS
by Chris Santerre 15 Nov '04

15 Nov '04

>-----Original Message----- >From: Jose Marcio Martins da Cruz [mailto:Jose-Marcio.Martins@ensmp.fr] >Sent: Monday, November 15, 2004 3:50 PM >To: Chris Santerre >Cc: SM; SURBL Discussion list (E-mail); Jeff Chan >Subject: Re: [SURBL-Discuss] Re: [SURBL-Announce] SURBL stats: list hit >r a tes measured in DNS > > >Chris Santerre wrote: > >>Thanks, but..... >> >>These stats are pretty useless: >> STATS BY URL BL >> >>================================================= >> >> * ab.surbl 139803 38226 >> * j-chkmail 207945 54310 >> * multi.surbl 175657 54486 >> * ob.surbl 990 260 >> * sc.surbl 220 180 >> * ws.surbl 354 246 >> >>Because you said it goes by alphabet and stops at first hit. >Mutli.surbl >>contains everything in SC and WS. So everything after mutli >is of no use. >> > >You're wrong. > >If some entries were found at WS and SC, this means that they >aren't at >MULTI. > >Why ? There may be a race condition : as multi is created from other >bls, it may be possible that when I rsync, SC and WS where already >updated but not MULTI. > >Quite interesting indeed... > >I rsync data every two hours. It could be interesting to change >periodicity and see what happens. > Well yeah, you could look at a rotten apple and see it as a chance to study penicillin as well ;) So yeah, that would be the only explanation as to why there are any hits at all for those servers. But as general hit sites for those servers, those stats aren't going to help. Heck, it's only good for your first one :) --Chris Wait a minute! Whats a guy named Jose doing with a .fr TLD? ;)

1 0

RE: [SURBL-Discuss] safe.surbl.org concerns
by Chris Santerre 15 Nov '04

15 Nov '04

>-----Original Message----- >From: Jeff Chan [mailto:jeffc@surbl.org] >Sent: Friday, November 12, 2004 9:36 PM >To: 'SURBL Discussion list' >Subject: Re: [SURBL-Discuss] safe.surbl.org concerns > > >On Friday, November 12, 2004, 6:16:48 PM, Rob McEwen wrote: >> RE: safe.surbl.org concerns > >> While I do see the reasoning behind the need for a "golden" >list that users >> can feel **particularly** secure about using in a MTA-like >"set it and >> forget it" type of blocking... I can help but wonder: > >> (1) Wasn't the original goal of ALL surbl.org to have a low >enough FP rate >> to be able to operate in "set it and forget it" MTA-like >blocking mode? (I >> guess the answer to this question is that Fortune-500 types >want a really, >> really powerful assurance that they can employ SURBL with >virtually zero >> FPs?) > >Yes, but it seems really difficult to achieve perfection in any >given list. > >This is a proposal for some experiments to see if we can derive a >"carrier grade" list, as Carl Friend called it off list, from >some of the existing data. Haven't picked any names yet. > >> (2) Even so, can I (we) be assured that having a "golden" >list like this >> will NOT diminish our standards with the regular list. > >Yes, we will not be dumping a bunch of grey entries into any of >the existing lists. > >> In other words, I >> would like to know that the FP rate of the regular list will >at least stay >> the same (and hopefully continue to improve). > >Yes, that's what should happen. The FPs per existing list should >remain stable or be further improved. That remains a goal. > >> I'd hate to see SURBL >> administrators get "slack" by virtue of thinking "well, if >they wanted >> **that** level of low FPs, they should have used the golden list." > >I agree. We should not let that happen. As far as I'm concerned our >biggest challenge is reducing FPs. We're already doing a good >job of catching spam. I agree with Jeff. The ONLY list that is getting grey domains added is the UC list. Which is NOT an official SURL.org list. And it isn't ready for prime time yet. Lowering the FP rate in WS is the first and foremost goal for many contributors. WS will be as close to zero FP as we can humanly get. --Chris

1 0

RE: [SURBL-Discuss] Re: [SURBL-Announce] SURBL stats: list hit ra tes measured in DNS
by Chris Santerre 15 Nov '04

15 Nov '04

Linky no worky?? --Chris >-----Original Message----- >From: Jose Marcio Martins da Cruz [mailto:martins@ensmp.fr] >Sent: Sunday, November 14, 2004 3:25 PM >To: discuss(a)lists.surbl.org >Cc: martins(a)ensmp.fr >Subject: [SURBL-Discuss] Re: [SURBL-Announce] SURBL stats: list hit >rates measured in DNS > > > >Jeff, > >You'll find some stats from our mail server at : > > http://j-chkmail.ensmp.fr/urlbl/urlbl.txt > >Some words about it : > >- When a list get a hit, it stops searching other lists. >- The first list checked is the our one. surbl lists are checked > just after our one, in alphabetical order. This means : > j-chkmail > ab.surbl.org > ... > >So, our stats doesn't show all lists, but if this interests you, >I can easily adapt our stats scripts to get hits for all lists. > >Either way, this shows that most spam detection using SURBL, at >our domain, comes from ab and multi. This conclusion is mitigated >by : >- many entries from our list (j-chkmail) may exist on ob/sc or ws. >- our list (j-chkmail) isn't being updated too frequently, because > as I said you before, I add entries only when they aren't yet > at surbl lists and surbl lists are being updated very fast... > >Best > >Jose-Marcio > > >> >> In order to explore some conjunctions of existing lists that >> might have fewer false positives, we've created some stats >> measuring the number of DNS hits against all blocklists that the >> individual lists get, along with some permutations of those lists. >> >> >> Jeff C. >> -- >> Jeff Chan >> mailto:jeffc@surbl.org >> http://www.surbl.org/ >> > >_______________________________________________ >Discuss mailing list >Discuss(a)lists.surbl.org >http://lists.surbl.org/mailman/listinfo/discuss >

1 0

RE: [SURBL-Discuss] Computer being repaired
by Chris Santerre 15 Nov '04

15 Nov '04

>-----Original Message----- >From: SURBL [mailto:surbl@horse.supranet.net] >Sent: Sunday, November 14, 2004 11:10 PM >To: discuss(a)lists.surbl.org >Subject: [SURBL-Discuss] Computer being repaired > > >Hi all, >Please note that I may not be able to respond as quickly as I'd like >for a day or so since I need to repair my main computer. This >is only referring to my personal mail. SURBL stuff should run >just fine, as usual. > >(Chris S., my PC locked up in JO. Now the video BIOS won't even >come up.... :-( Power supply measures ok with no load, so maybe >the motherboard is hosed. Might be the power supply though.) > Crud! Any overclocking? Did you throw in a older video card to check? This is damn important to fix quickly. No not because of emails.... The JO expansion comes out tomorrow!!! :-) Let me know if I can be any help. Make sure your hard drive is still good. You got lots of good data! --Chris (The Geforce 6800 OC freaking rocks!!!)

1 0

Re: [SURBL-Announce] SURBL stats: list hit rates measured in DNS
by Jose Marcio Martins da Cruz 14 Nov '04

14 Nov '04

Jeff, You'll find some stats from our mail server at : http://j-chkmail.ensmp.fr/urlbl/urlbl.txt Some words about it : - When a list get a hit, it stops searching other lists. - The first list checked is the our one. surbl lists are checked just after our one, in alphabetical order. This means : j-chkmail ab.surbl.org ... So, our stats doesn't show all lists, but if this interests you, I can easily adapt our stats scripts to get hits for all lists. Either way, this shows that most spam detection using SURBL, at our domain, comes from ab and multi. This conclusion is mitigated by : - many entries from our list (j-chkmail) may exist on ob/sc or ws. - our list (j-chkmail) isn't being updated too frequently, because as I said you before, I add entries only when they aren't yet at surbl lists and surbl lists are being updated very fast... Best Jose-Marcio > > In order to explore some conjunctions of existing lists that > might have fewer false positives, we've created some stats > measuring the number of DNS hits against all blocklists that the > individual lists get, along with some permutations of those lists. > > > Jeff C. > -- > Jeff Chan > mailto:jeffc@surbl.org > http://www.surbl.org/ >

1 0

Re: [SURBL-Discuss] RFC: consensus list?
by Frank Ellermann 14 Nov '04

14 Nov '04

Jeff Chan wrote: >> (35 + 35 + 21 + 7 + 1 = 99, is that correct ?) It is, 128 -1 -7 -( 6+5+4+3+2+1 ) = 99. Where -1 is "not no bit", -7 is "not only one bit", and the rest "not two bits". > Actually it may be necessary to create some new lists, > partly depending on whether the application can combine > individual lists by itself. Yes, bit 0 could replace only one "new list", not several. But as long as you can express the condition for the "new list" by a single boolean expression of the 7 multi bits, one bit is good enough. Excl. the special case "no bits", because that would result in 127.0.0.1. > And I just realized a major flaw in my suggestion to try > 127.0.0.84 as a single list 84 = 64+16+4 (you said JP+OB+WS, I don't check it). Let's say that you want any combo where these 3 bits are set. Then you have 3 fixed bits, and 4 free bits (at the moment only 3 free bits, but you may want to add the 7th bit later). 2 ** 4 = 16, you need 16 result codes to identify all combos with JP+OB+WS: 84,86, 92,94, 116,118, 124,126, (= 8 without the 7th bit, for the remaining 8 combos just add 128, from 212 to 254) > it will NOT have any JP + OB + WS + SC That's 86 = 84 + 2. > That kind of combination needs to be done by creating a > new list on the data side, or special processing in the > application. Or by adding bit 0 on your side for THE one "optimal rule": 85,87, 93,95, 117,119, 125,127 (= same 8, now bit 0 is set) > I will create some temporary test lists in the second or > third octet of multi to test on corpora instead. Yes, post a real complex boolean expression, it's fun stuff, SURBL would set a new standard (impressing ISIPP and OPM ;-) But don't touch the zeros in 127.0.0.? without warning, it would break my poor <http://purl.net/xyzzy/src/rxwhois.cmd> : | if SockGetHostByName( arg(1) || arg(2), 'P.' ) then do | if abbrev( P.ADDR, '127.0.0.' ) | abbrev( P.ADDR, '127.1.0.' ) | then say arg(3) '(' || substr( P.ADDR, 9 ) || '):' arg(2) | else say 'erroneous result' P.ADDR 'for' arg(1) || arg(2) | return 1 | end /* 127.1.0.1 .. 127.1.0.7 for .opm.blitzed.org(1+2+4) */ | return 0 Bye, Frank

2 5

RFC: consensus list?
by Jeff Chan 14 Nov '04

14 Nov '04

Pondering the question of how to make a "telco grade" SURBL that had as close to zero false positives as possible, but would still catche many spams, I remembered that many of the biggest spam domains seem to appear in several different SURBL lists. What does anyone think about creating a "consensus" list that a telco or ISP might use to block at the MTA level? For example a domain that appears on: ((SC or AB) and (JP or OB)) or PH might be a candidate for such a list. The main reason I don't include WS is that it's a hand built list and I don't have a feeling for the latencies from it. SC and AB are both mostly based on SpamCop user reports. JP and OB are both mostly based on spamtrap data. PH represents really destructive fraud and phishing and probably should be included unless the FP rates from it are significantly above zero. I realize this is a simplistic scheme and other ways of combining the list are possible, but what does anyone think those idea? Conceivably we could have other combinations. Another possibility might be records that appear in SC and AB and WS and JP and OB I think we can nearly guarantee that those are 100% spam. :-) (Would want to check those that are in WS separately from JP, which is currently included in WS.) What other ways to combine lists might produce near zero FPs yet still hit most spam? Shall we just try some of them and see how well they work? Comments? Jeff C. -- "If it appears in hams, then don't list it."

6 13

Re: [SURBL-Discuss] RFC: consensus list?
by Frank Ellermann 13 Nov '04

13 Nov '04

Jeff Chan wrote: > LOL! I didn't think of that! Just an idea how to use bit 0 of multi in a way indicating "interesting" combinations. Ryan identified "any 3 out of 6 multi bits" as relevant. That's probably too KISS for you, and if you'd add the not yet used 7th multi bit it would result in 7 over 3 (any 3 out of 7) combinations, 99 result codes if I got it right. (35 + 35 + 21 + 7 + 1 = 99, is that correct ?) Apparently SA already supports Ryan's idea, and then you don't need it. But if you have more complex rules to get an "optimal" proper subset of all multi result codes, then you could (ab)use bit 0 to mark it. Or in other words, any odd N in 127.0.0.N would be optimal. And 127.0.0.1 would be still protected (= never returned). > without explicitly creating new, combined lists. Yes, that's unnecessary. Unless you have more than one definition of "optimal". Bit 0 can only handle one "rule", i.e. any combination of AND / OR / NOT multi bits you like. Bye, Frank

2 1

Re: [SURBL-Discuss] RFC: consensus list?
by Frank Ellermann 13 Nov '04

13 Nov '04

Jeff Chan wrote: > Comments? See Ryan's answer, it's easy to do interesting stuff with the multi bits, multi was a good idea. If you want to simplify it set bit 1 if 3 or more other bits are set, examples: 127.0.0.6 => 127.0.0.6 (2 or less bits unchanged) 127.0.0.100 => 127.0.0.101 (64+32+4 => 64+32+4+1) 127.0.0.14 => 127.0.0.15 (8+4+2 => 8+4+2+1) Bye, Frank

2 1

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss