Discuss

discuss@lists.surbl.org

1 participants
1867 discussions

[SURBL-Discuss] RFC: Combined SURBL list details, phishing list ready
by Jeff Chan 15 May '04

15 May '04

It's time to return to the question of a combined SURBL list again mainly because David Hooton's anti-phishing list is now ready. The list is too small to be a separate list at a little over a hundred entries, but it will probably grow. So I'd like to make it part of a combined list. 1. We had discussed two strategies for a combined list A records before: I. Separate A records: spammer.com IN A 127.0.0.1 IN A 127.0.0.2 Where the two addresses indicate it being on two lists. II. Bitmasked address: spammer.com IN A 127.0.0.3 where .3 means it's on the two lists corresponding to the .1 and .2 bits, and similarly for other lists in other bit positions. In the first case resolution on spammer.com.combined-list-name-here.surbl.org would give two separate Addresses 127.0.0.1 and 127.0.0.2 and in the second case it would give one result 127.0.0.3. The querying program would need to act accordingly. For the bitmasked case some SA code would need to be written or reused. As mentioned earlier, various other RBLs combine lists using either of these two strategies. Here's an example cited earlier of the bitmask style: http://opm.blitzed.org/info > Using the DNSBL > In opm.blitzed.org, the A record has an IP address of 127.1.0.x > where x is a bitmask of the types of proxy that have been > reported to be running on the host. The values of the bitmask > are as follows: > > WinGate 1 > SOCKS 2 > HTTP CONNECT 4 > Router 8 > HTTP POST 16 So the code using a combined list could be made to detect specific results, i.e., the specific list which triggered a matching A record could be determined, and not just that it matched "all" or any from the original list. On the other hand, the fact that matches from any list occurred may be good enough for some users. Personally I prefer more a detailed explanation that would come from being able to distinguish the source list, but that's a question of the program implementation and not the combined list itself. The combined list itself would always encode the source list, whether the querying program knew or cared how to decode that or not. 2. Another question would be the name of the combined list. Since there would be three or more lists, someone had suggested a name of "all" before. That sounds good to me unless there are other suggestions. 3. I'm assuming TXT records are no longer really feasible in a combined list and that descriptive messages will need to be signalled by the list (127. address) matched. I suppose it would be possible to create custom TXT records for every entry, but a generic TXT (or perhaps none) might be more likely. Is a generic TXT better than none? Even in a BIND file, where it incurs some use of space? 4. TTLs: If an entry has matches on more than one list, should it get a unique TTL? If so, should such a custom TTL on the multiply-matching entry be the longest TTL or the shortest TTL? I lean towards the inheriting the shortest TTL from the matching source list, plus setting a default TTL for the combined zone file to be near the longest. Am I right in thinking that TTLs are largely irrelevant for rbldnsd, since it reloads zone info whenever the files change? In other words, does the rbldnsd cache clear for a given zone when the zone reloads, or do the cached entries with TTLs longer than the last reload interval remain in the cache? (I'm kind of hoping for the simpler, case of rbldnsd clearing whenever reloading.) 5. We will likely want to combine the ws and be lists into a single entry in a combined list, probably using the .1 bit for both of them, since both lists contain the enumerated (non-wildcarded) domains from SA regular expressions. Also, things are moving towards combining the non-wildcarded domains sa-blacklist and BigEvil/MidEvil, so this would somewhat short-circuit that process and future-proof things. Comments? Jeff C.

6 15

Domain not identified by SpamCopURI
by John Fawcett 15 May '04

15 May '04

I just got the following in a spam message which was not identified by spamcopuri even though the numeric domain it is in sc.surbl.org <a href=3D"http://69.63.161.232/?rid=3D1528"> The debug output shows that the uri is found by the SA modules but that SpamCopURI queries for the wrong hostname. It should be querying 232.161.63.69.sc.surbl.org > debug: uri tests: Done uriRE > debug: checking url: http://69.63.161.232/?rid%1528 > debug: querying for 63.161..sc.surbl.org > debug: Query failed for 63.161..sc.surbl.org It looks like there are two problem is in SpamCopURI (v 0.15) in the routine _spamcop_uri (1) # strip any non alpha characters off of the end # this is to fix a bug where url parsing in core SA # leaves parens and other junk on the URL that URI # parses to the host This causes the domain to become 69.63.161. (including a trailing dot) (2) The domain is stripped back to a two level domain even though it is/should be an ip address. It becomes 63.161. (including a trailing dot) This domain then does not match an ip address format so it is not reversed and the lookup is done for 63.161. added on to .sc.surbl.org, i.e.: 63.161..sc.surbl.org Eric, can you help? John

3 4

RE: [SURBL-Discuss] RFC: Combined SURBL list details, phishing list ready
by Chris Santerre 14 May '04

14 May '04

>-----Original Message----- >From: Jeff Chan [mailto:jeffc@surbl.org] >Sent: Thursday, May 13, 2004 5:07 PM >To: SURBL Discuss >Subject: Re: [SURBL-Discuss] RFC: Combined SURBL list details, phishing >list ready > > >On Thursday, May 13, 2004, 7:21:35 AM, Chris Santerre wrote: >>>From: Jeff Chan [mailto:jeffc@surbl.org] > >>>Actually I was getting tricky and proposing to collapse ws and be >>>into a single response within a combined list. This was mainly >>>to prevent needing to remove separate be entries later since it will >>>probably be merged into ws eventually. I was proposing short >>>circuiting that process in the combined list. > >> I would say consider BE to be WS as of now. Just work with >WS, because BE is >> definetly going to be pulled in. How we do that on the >backend won't matter >> to the clients. For all intensive purposes, I won't be >updating BE, I will >> be updating WS directly thru the magic of Paul. (He's just >swamped at the >> moment.) > >> So again, consider BE non exhistant for future upgrades. It >will save one >> lookup ;) > >Sounds like you're saying we should not fold be in with ws for a >combined list. Could we fold be in transparently into ws for the >combined list, then remove it later (all invisibly to the users >of the combined list)? Or are you guys already merging them >behind the scenes? Want to try to get all the domains.... :-) > >Jeff C. Yeah we will just transfer. Actually WS is so more up to date, the amount of unique hits keeps getting smaller. Partly because I used to be the only game in town, I felt the preasure to to keep updated. Now I don't thanks to you guys :) So I'm trying to work on a bunch of things and not updating as often. My ninja regex skills were getting rusty ;) --Chris

1 0

Whitelist discussion
by Chris Santerre 14 May '04

14 May '04

So what do we do with candidates for whitelists? Also, what do we do with things that should NOT be whitelisted, but NOT reported as spam? Examples are cacheing services. Which are often used by spammers for images, but legit as well. I think images.exactis.com is one. And of course we have all the akamai.net servers. I'm just kind of thinking out loud. *man I'm hungry right now* :-) Chris Santerre System Admin and SARE Ninja http://www.rulesemporium.com 'It is not the strongest of the species that survives, not the most intelligent, but the one most responsive to change.' Charles Darwin

5 5

RE: [SURBL-Discuss] RFC: Combined SURBL list details, phishing list ready
by Chris Santerre 13 May '04

13 May '04

>-----Original Message----- >From: Jeff Chan [mailto:jeffc@surbl.org] >Sent: Wednesday, May 12, 2004 7:27 PM >To: SURBL Discuss >Subject: Re: [SURBL-Discuss] RFC: Combined SURBL list details, phishing >list ready > > *snip* > >Actually I was getting tricky and proposing to collapse ws and be >into a single response within a combined list. This was mainly >to prevent needing to remove separate be entries later since it will >probably be merged into ws eventually. I was proposing short >circuiting that process in the combined list. > I would say consider BE to be WS as of now. Just work with WS, because BE is definetly going to be pulled in. How we do that on the backend won't matter to the clients. For all intensive purposes, I won't be updating BE, I will be updating WS directly thru the magic of Paul. (He's just swamped at the moment.) So again, consider BE non exhistant for future upgrades. It will save one lookup ;) --Chris

3 2

RE: [SURBL-Discuss] RFC: Combined SURBL list details, phishing l ist ready
by Chris Santerre 13 May '04

13 May '04

>-----Original Message----- >From: Simon Byrnand [mailto:simon@igrin.co.nz] >Sent: Wednesday, May 12, 2004 7:12 PM >To: Jeff Chan; SURBL Discussion list >Subject: Re: [SURBL-Discuss] RFC: Combined SURBL list details, phishing >list ready > > >At 11:02 13/05/2004, you wrote: > >>5. We will likely want to combine the ws and be lists into a >>single entry in a combined list, probably using the .1 bit for >>both of them, since both lists contain the enumerated >>(non-wildcarded) domains from SA regular expressions. Also, >>things are moving towards combining the non-wildcarded domains >>sa-blacklist and BigEvil/MidEvil, so this would somewhat >>short-circuit that process and future-proof things. > >Is it necessarily a good idea to combine lists like ws and be >into a single >entity when the sources of information are different ? (One comes from >Bill, one comes from Chris) What policies of inclusion and >removal do they >each have ? Well this is Paul's magic project. Bill, Paul, and I have pretty much the same policies. We check everything. When in doubt we throw it out. Don't want ANY FPs. Removal is pretty easy. But this process is going to be somewhat better. They will be checked by groups of people instead of 3 :-) > >Say that a legitimate domain were somehow blocked, how would >an end user >know if it was Bill's data or Chris's that actually had it >listed, to try >and get it removed ? Etc... Paul's magic again. Posted to a list. A group of trusted reviewers see it. Make a decision. Poof! > >So from a technical point of view, fine no problem, but I wonder a bit >about compatibility of listing policies etc.. Thats the beauty of what Paul is working on. I'm hoping he can find some time to elaborate more. --Chris

1 0

RE: [SURBL-Discuss]
by Scott Truman 13 May '04

13 May '04

> > On Wed, 12 May 2004 16:02:33 -0700, Jeff Chan wrote: > > > 2. Another question would be the name of the combined > list. Since > > > there would be three or more lists, someone had suggested a name > > > of "all" before. That sounds good to me unless there are other > > > suggestions. > > > > This could potentially lead to confusion if you > subsequently add another > > list not included in the "all" for some reason. That may > seem unlikely > > now, but who knows? How about "multi"? > > How about surbl.surbl.org? Being the primary rbl, this kinda > makes sense :) > > Also on the other points about how to respond - I suggest the > different IP > per list as being smartest the octet based response is too > non-specific. > This only leaves the question on how do we handle multiple listings :) > > Cheers!! > > Dave Hi all, I'll put my hand up for the bit masked approach: -it would require less traffic, yeah? If so, then that's gotta be good for everyone. Possibly less load on the DNS servers. -I'm not sure what Dave means by too 'non specific', both methods achieve the desired results, which is all we really care about. The smarts lie in the code, it doesn't really matter if it is humanly 'harder' to read. You write a rule...it works...you forget whether you entered '2' or 127.0.0.2 as the value for the BigEvil list match some 5 minutes later - I don't need to remember it for an exam, and I hardly ever do manual lookups of URI's myself - the coding is much more efficient :) -The SA programmers would spit out the new code required whilst blindfolded I would imagine...it sounds like they are just waiting to see what is required As for the name, something like 'multi' seems good to me. Keep up the good work as the existing lists are working very well. Cheers Scott

1 0

[SURBL-Discuss] Re: RFC: Combined SURBL list details, phishing list ready
by Daniel Quinlan 13 May '04

13 May '04

Jeff Chan <jeffc(a)surbl.org> writes: > 1. We had discussed two strategies for a combined list A records before: We still don't really care as long as you don't it right. :-) If you think you might end up supporting more than 32 return codes (seems like a very remote possibility), separate A records is much easier. Maybe that helps make up your mind. > So the code using a combined list could be made to detect > specific results, i.e., the specific list which triggered > a matching A record could be determined, and not just that it > matched "all" or any from the original list. On the other hand, > the fact that matches from any list occurred may be good enough > for some users. Personally I prefer more a detailed explanation A more detailed explanation is merely the job of the client. If a client can't handle multi-DNSBLs, then it's really behind the times. Given that these clients are going to need to parse URIs and that is non-trivial, I think you can just require multi-DNSBL support for newer lists at some point if you want. > 2. Another question would be the name of the combined list. Since > there would be three or more lists, someone had suggested a name > of "all" before. That sounds good to me unless there are other > suggestions. "query" and "bl" are two somewhat more common names for a single combined blacklist. > 3. I'm assuming TXT records are no longer really feasible in a > combined list and that descriptive messages will need to be > signalled by the list (127. address) matched. I suppose it would > be possible to create custom TXT records for every entry, but a > generic TXT (or perhaps none) might be more likely. Is a generic > TXT better than none? Even in a BIND file, where it incurs some > use of space? You could return a single URL that if loaded contains information about exactly which lists hit. You can also return multiple TXT records that can be matched as SpamAssassin rules (yes!), but unless you promise to keep the format very very stable, then we'll just use the A records. My suggestion: start with the A and worry about the rest later. > 4. TTLs: If an entry has matches on more than one list, should > it get a unique TTL? If so, should such a custom TTL on the > multiply-matching entry be the longest TTL or the shortest TTL? > I lean towards the inheriting the shortest TTL from the matching > source list, plus setting a default TTL for the combined zone > file to be near the longest. > [...] Sounds fine to me. > 5. We will likely want to combine the ws and be lists into a > single entry in a combined list, probably using the .1 bit for > both of them, since both lists contain the enumerated > (non-wildcarded) domains from SA regular expressions. Also, > things are moving towards combining the non-wildcarded domains > sa-blacklist and BigEvil/MidEvil, so this would somewhat > short-circuit that process and future-proof things. As long as they have similar S/O ratios, I'm okay with it. If they are maintained separately, it might shift the decision towards keeping them separate, but if they're getting merged, then that makes more sense. Daniel -- Daniel Quinlan anti-spam (SpamAssassin), Linux, http://www.pathname.com/~quinlan/ and open source consulting

1 0

RE: X posting due to importance
by Chris Santerre 12 May '04

12 May '04

>-----Original Message----- >From: Matt Linzbach [mailto:MLinzbach@Merchant-Gould.com] >Sent: Wednesday, May 12, 2004 2:57 PM >To: Spamassassin-Talk (E-mail) >Cc: SURBL Discussion list (E-mail) >Subject: RE: X posting due to importance > > >> -----Original Message----- >> From: Chris Santerre [mailto:csanterre@MerchantsOverseas.com] >> Sent: Wednesday, May 12, 2004 10:24 AM >> To: Spamassassin-Talk (E-mail) >> Cc: SURBL Discussion list (E-mail) >> Subject: X posting due to importance >> >> >> This was posted to Spam-L list. Very interesting. We should all watch >> closely. Consider how we would defend our projects if we had to. :-) >> >> A Northern California District Court judge issued a temporary >> restraining >> order to prevent SpamCop, an antispam operation, from >> interfering with >> messages sent by alleged junk e-mailer OptInRealBig.com. >> >> http://news.com.com/2100-1024_3-5210518.html?tag=nefd.top > > >/. has an active discussion on this today. > >http://yro.slashdot.org/yro/04/05/12/1226222.shtml?tid=111&tid= 123&tid=126&t id=95&tid=99 And just like that, it is over :-) http://biz.yahoo.com/prnews/040512/sfw083_1.html Perhaps I should setup a BigEvil Drinking fund? I mean Legal defense fund. Yeah thats what I meant! --Chris

1 0

RE: X posting due to importance
by Matt Linzbach 12 May '04

12 May '04

> -----Original Message----- > From: Chris Santerre [mailto:csanterre@MerchantsOverseas.com] > Sent: Wednesday, May 12, 2004 10:24 AM > To: Spamassassin-Talk (E-mail) > Cc: SURBL Discussion list (E-mail) > Subject: X posting due to importance > > > This was posted to Spam-L list. Very interesting. We should all watch > closely. Consider how we would defend our projects if we had to. :-) > > A Northern California District Court judge issued a temporary > restraining > order to prevent SpamCop, an antispam operation, from > interfering with > messages sent by alleged junk e-mailer OptInRealBig.com. > > http://news.com.com/2100-1024_3-5210518.html?tag=nefd.top /. has an active discussion on this today. http://yro.slashdot.org/yro/04/05/12/1226222.shtml?tid=111&tid=123&tid=126&t id=95&tid=99

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss