Discuss

discuss@lists.surbl.org

1862 discussions

RE: [SURBL-Discuss] BigEvil + MidEvil as SURBL
by Chris Santerre 21 Apr '04

21 Apr '04

> -----Original Message----- > From: Jeff Chan [mailto:jeffc@surbl.org] > Sent: Wednesday, April 21, 2004 7:47 AM > To: SURBL Discussion list; Chris Santerre > Subject: Re: [SURBL-Discuss] BigEvil + MidEvil as SURBL > > > On Wednesday, April 21, 2004, 4:35:41 AM, Raymond Dijkxhoorn wrote: > > Hi! > > >> BigEvil is a fairly slowly moving list. Paul Barbeau's MidEvil > >> is quicker moving and gets new domains usually before Chris can > >> get them into BE. In that sense ME is a feeder of changes into > >> BE. Since they are closely related, I merged them into a single > >> be.surbl.org. I hope Chris and Paul agree that's appropriate. > >> > >> What I'd like to know is what TTLs I should use on the BE data. > >> Probably it depends on how often ME is typically updated. So... > >> how often does ME get updated Paul? :-) > >> > >> Also I'd like feedback on the TXT message. I've got the > >> placeholder: > >> > >> "Blocked in BigEvil. See: http://www.rulesemporium.com/" > >> > >> but would like feedback on it. > > > Do we get a different value on looking up? For example: > > > 127.0.0.2 for BE and 127.0.0.3 for ME ? > > > We should start doing that also to get the combined list going. > > Currently we will have them lumped together (i.e. it's > all .2 without differentiation as to the source). As I > understand it that may be appropriate since ME is meant > to be essentially updates to BE. I think of them as the > same list, especially since Chris eventually merges the > ME (update) entries into BE. I kind of short circuit that > process by merging them for them before turning them into > be.surbl.org. Hopefully that's ok. > > Lists with greater differences such as ws and sc probably > should get different A or TXT records when we eventually > combine them. > > FWIW even if we offer a combined list, the individual > ones will probably still be available, like SBL, XBL & > SBL-XBL at spamhaus. > > Jeff C. > > P.S. Chris please sign up for the SURBL Discussion and > Announce lists if you can: http://lists.surbl.org/ > I already am ;) Yeah, usually I update BigEvil a lot more often. I'm dealing with a lot of projects now. Some are even work related ;) And then some are beta testing a new game :-) Paul and I are still working out how we can merge ME and BE together without a lot of work. But I have no problems at all combining the ME and BE together and letting Paul add just as much as me. He knows my basic criteria for checking the domains. A few things off the top of my head. Sorry if they have been discussed, I have a LOT of email to read :) 1) BigEvil wildcards. Not sure how you would handle these. Something like evil\d{2,4}spam\.com is a general wildcard. Some of those domains don't even exhist. Not sure how SURBL will handle that. 2) Where would I send updates? As single domains, or a txt list? How would I remove an FP? 3) What is the quickest way to check a domain against the other SURBL lists? Basically I see no reason to duplicate the listings. *gulp* and on a Windowze machine? (Don't ask!) 4) Has there been any talk with the sendmail people? It would be interesting to actually block at the MTA level based on an evil URL. I realise the inherent dangers in this ;) --Chris

2 2

[SURBL-Discuss] Re: Bill Stearns' sa-blacklist available as SURBL: ws.surbl.org
by Doc Schneider 21 Apr '04

21 Apr '04

[forwarded response from announce list] Chris Santerre wrote: > The ONLY complaint anyone ever had with Stearn's list was the overhead. With > it setup this way, there is no stopping him!! :) > > Great job by Bill and Jeff! > > --Chris (Yeah I know,....the list.) :-) [snip] >>Wow, I implemented this yesterday and, after about 18 hours, >>this rule was >>#4 in my list of most-used rules! Good job, guys! [snap] I'm getting most excellent spam catching here. For a while was only catching 70-80% but now am catchng closer to 95+% Still have the odd one slipping through. Which reminds me I need to grab a new bigevil 8*) And even though Bill Stearns list does take a bit f overhead I am still using it and a whole slug of other rules as well as wc.surbl.org. But the surbl is the one that is now doing the most catching. -Doc

4 6

[SURBL-Discuss] Re: Bill Stearns' sa-blacklist available as SURBL: ws.surbl.org
by Jeff Chan 21 Apr '04

21 Apr '04

On Tuesday, April 20, 2004, 6:10:30 PM, Bill Landry wrote: > ----- Original Message ----- > From: "Jeff Chan" <jeffc(a)surbl.org> >> So the quick answer is they'll probably not be combined. >> >> However we probably will offer a combined version of Bill's >> list and Chris' BigEvil list since they are more similar in >> character. > Jeff, why not one DNS query that supports multiple result codes: > test: somedomain.com.sc.surbl.org > results: > 127.0.0.2 = Spamcop > 127.0.0.3 = WS List > 127.0.0.4 = BigEvil List > 127.0.0.5 = etc... > Same thing multi-test RBLs like AHBL, Sorbs, Blars, FiveTen, NJABL and > others do. Yes, we may end up doing that in a combined list. Jeff C.

2 4

BigEvil + MidEvil as SURBL
by Jeff Chan 21 Apr '04

21 Apr '04

Chris Santerre forwarded me a script from Gary Funck called expand_regexp.pl which apparently is designed to expand SA regexps into domains. I used that on BigEvil and MidEvil to turn them into an SURBL, be.surbl.org, but I want to get a couple details sorted out before turning it live. BigEvil is a fairly slowly moving list. Paul Barbeau's MidEvil is quicker moving and gets new domains usually before Chris can get them into BE. In that sense ME is a feeder of changes into BE. Since they are closely related, I merged them into a single be.surbl.org. I hope Chris and Paul agree that's appropriate. What I'd like to know is what TTLs I should use on the BE data. Probably it depends on how often ME is typically updated. So... how often does ME get updated Paul? :-) Also I'd like feedback on the TXT message. I've got the placeholder: "Blocked in BigEvil. See: http://www.rulesemporium.com/" but would like feedback on it. TIA & Cheers! Jeff C. -- Jeff Chan mailto:jeffc@surbl.org-nospam http://www.surbl.org/

2 2

RE: [SURBL-Discuss] Combined SURBL A record format (Was: Re: BillStearns' sa-blacklistavailable as SURBL: ws.surbl.org)
by Scott Truman 21 Apr '04

21 Apr '04

> Hi Scott, > Multiple A records can result from a single query: > > > % nslookup www.yahoo.com > > > Name: www.yahoo.akadns.net > > Addresses: 216.109.117.207, 216.109.118.75, > 216.109.118.66, 216.109.118.77 > > 216.109.118.73, 216.109.117.205, 216.109.117.204, > 216.109.118.70 > > Aliases: www.yahoo.com > > And they should all come from the locally cached copy of the zone > file, so they're fast. > > A single bit masked A record would be smaller but require a little > more application CPU to decode. Multiple A records may be faster > but they're bulkier. I find the multiple A records more readable. > It's one of those classic tradeoffs. > > Jeff C. > So desu ka. I would have thought that a bit masked record was faster, as you already have all the data you need with the first call. From then on bitwise CPU operations would be umpteen times faster than performing slower DNS callouts, even if they are cached. I am assuming, however, that perhaps the Spamassassin code 'caches' the first A record lookup for bitwise operations...I am not too familiar with the eval-rbl workings. It may be that it performs subsequent DNS lookups in either case? The only way to speed that up would be to load the first lookup into a variable for use in later bitwise calculations... Anyway keep up the good work all, cheers Scott

3 3

Re: [SURBL-Discuss] Re: Bill Stearns' sa-blacklist available as SURBL: ws.surbl.org
by Daniel Quinlan 21 Apr '04

21 Apr '04

Jeff Chan <jeffc(a)surbl.org> writes: > Does anyone have any comments about either approach? Bill seems > to indicate there was a precedent in other "combining" RBLs, but > Scott's suggestion is also clever. You know, I did mention this (and I requested it at least once) to you several weeks ago... Anyhow, using a bitmask is done. OPM is probably the cleanest example. We (used to, OPM is now included in another blacklist so we dropped the rules) do OPM like this: # header __RCVD_IN_OPM eval:check_rbl('opm', 'opm.blitzed.org.') # describe __RCVD_IN_OPM Received via a relay in opm.blitzed.org # tflags __RCVD_IN_OPM net # # header RCVD_IN_OPM_WINGATE eval:check_rbl_sub('opm', '1') # describe RCVD_IN_OPM_WINGATE OPM: sender is open WinGate proxy # tflags RCVD_IN_OPM_WINGATE net # # header RCVD_IN_OPM_SOCKS eval:check_rbl_sub('opm', '2') # describe RCVD_IN_OPM_SOCKS OPM: sender is open SOCKS proxy # tflags RCVD_IN_OPM_SOCKS net # # header RCVD_IN_OPM_HTTP eval:check_rbl_sub('opm', '4') # describe RCVD_IN_OPM_HTTP OPM: sender is open HTTP CONNECT proxy # tflags RCVD_IN_OPM_HTTP net # # header RCVD_IN_OPM_ROUTER eval:check_rbl_sub('opm', '8') # describe RCVD_IN_OPM_ROUTER OPM: sender is open router proxy # tflags RCVD_IN_OPM_ROUTER net # # header RCVD_IN_OPM_HTTP_POST eval:check_rbl_sub('opm', '16') # describe RCVD_IN_OPM_HTTP_POST OPM: sender is open HTTP POST proxy # tflags RCVD_IN_OPM_HTTP_POST net The second argument in check_rbl_sub is the bitmask (in decimal, not hex). We'd need to make some modifications to our URIBL module to do the same for a bitmasked SURBL, but I'm sure we would. We'd be just as happy with multiple A record returns. NJABL is a good example of this: ------- start of cut text -------------- header __RCVD_IN_NJABL eval:check_rbl('njabl', 'dnsbl.njabl.org.') describe __RCVD_IN_NJABL Received via a relay in dnsbl.njabl.org tflags __RCVD_IN_NJABL net header RCVD_IN_NJABL_RELAY eval:check_rbl_sub('njabl', '127.0.0.2') describe RCVD_IN_NJABL_RELAY NJABL: sender is confirmed open relay tflags RCVD_IN_NJABL_RELAY net header RCVD_IN_NJABL_DIALUP eval:check_rbl('njabl-notfirsthop', 'dnsbl.njabl .org.', '127.0.0.3') describe RCVD_IN_NJABL_DIALUP NJABL: dialup sender did non-local SMTP tflags RCVD_IN_NJABL_DIALUP net header RCVD_IN_NJABL_SPAM eval:check_rbl_sub('njabl', '127.0.0.4') describe RCVD_IN_NJABL_SPAM NJABL: sender is confirmed spam source tflags RCVD_IN_NJABL_SPAM net header RCVD_IN_NJABL_MULTI eval:check_rbl_sub('njabl', '127.0.0.5') describe RCVD_IN_NJABL_MULTI NJABL: sent through multi-stage open relay tflags RCVD_IN_NJABL_MULTI net header RCVD_IN_NJABL_CGI eval:check_rbl_sub('njabl', '127.0.0.8') describe RCVD_IN_NJABL_CGI NJABL: sender is an open formmail tflags RCVD_IN_NJABL_CGI net header RCVD_IN_NJABL_PROXY eval:check_rbl_sub('njabl', '127.0.0.9') describe RCVD_IN_NJABL_PROXY NJABL: sender is an open proxy tflags RCVD_IN_NJABL_PROXY net ------- end ---------------------------- Daniel -- Daniel Quinlan anti-spam (SpamAssassin), Linux, http://www.pathname.com/~quinlan/ and open source consulting

3 2

RE: [SURBL-Discuss] Combined SURBL A record format (Was: Re: Bill Stearns' sa-blacklistavailable as SURBL: ws.surbl.org)
by Scott Truman 21 Apr '04

21 Apr '04

> Good to know. Sounds like it's mostly a question of style > then, though multiple A records would require no new coding > whereas bitmasks would. > > http://opm.blitzed.org/info > > > Using the DNSBL > > > > Anyone can query our DNSBL through normal DNS means. Just > > reverse the octets and do a name lookup. For example, to check > > if 127.0.0.2 is present in opm.blitzed.org, do a DNS lookup on > > 2.0.0.127.opm.blitzed.org. Each entry in the DNSBL has an A > > record and a TXT record associated with it, the TXT record > > contains a URL to the proxy information page specific to that > > IP address telling the user a little information about how to > > sort out the proxy. > > > > In opm.blitzed.org, the A record has an IP address of 127.1.0.x > > where x is a bitmask of the types of proxy that have been > > reported to be running on the host. The values of the bitmask > > are as follows: > > > > WinGate 1 > > SOCKS 2 > > HTTP CONNECT 4 > > Router 8 > > HTTP POST 16 > > The bitmask approach is more compact, but the multiple A record > approach is more human-readable and transparent IMO. I'm leaning > towards the latter, but am interested in any other comments. > > Jeff C. > This is all quite interesting. I'm happy with either method obviously. Excuse my ignorance with regards to DNS lookups, but do all the A records come down with the first DNS lookup, or are they fetched each time from the remote server? What are the traffic overheads with incorporating either type of method - I would have thought that the bit mask method would cause less traffic - also with smaller zone transfers? Cheers Scott

2 1

RFC: SURBL software implemetation guidelines
by Jeff Chan 21 Apr '04

21 Apr '04

I'm adding a very brief section to the Quick Start of the SURBL site: > Implementation guidelines > Here are some very brief instructions for folks writing > software to use SURBL lists: You code should: > > 1. Extract URIs from message bodies > 2. (Extraction of URIs from message bodies should ideally > include full resolution of redirections into the final target > domain name. This can be a non-trivial problem.) > > 3. Extract base (registrar) domains from those URIs > 4. Not do name resolution on the domains > 5. Look up the domain name in the SURBL by prepending it to > the name of the SURBL, e.g., domainundertest.com.sc.surbl.org > then doing Address record DNS resolution. A non-result > indicates lack of inclusion in the list. A result of 127.0.0.2 > represents inclusion. > 6. Handle numeric IPs in URIs similarly, but reverse the > octet ordering before comparison against the RBL. This is > standard practice for RBLs. For example, http://1.2.3.4/ is > checked as 4.3.2.1.sc.surbl.org > > SURBL lists unusually have both names and numbers in the same > list. For example, 2.0.0.127 and example.com and similar actual > spam domains and addresses are both in all SURBL lists. > Numbered addresses in SURBLs are to have occurred as numbers in spams. Can anyone think of any additions, corrections, or suggestions before I announce it more broadly? Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.jeffchan.com/

5 19

[SURBL-Discuss] Administrivia: announce replies go to discuss, discuss replies go to list
by Jeff Chan 21 Apr '04

21 Apr '04

I probably should have cross posted this originally, but replies on the discussion list now go back to the list via a Reply-To: address. This was influenced by Kelsey's suggestion. It may be more convenient for most replies, but note that if you want to reply to only the poster, you'll need to edit your recipients. Jeff C. __ From: Jeff Chan <jeffc(a)surbl.org> To: SURBL Announce <announce(a)lists.surbl.org> Date: Tuesday, April 20, 2004, 5:15:45 PM Subject: [SURBL-Announce] Administrivia: announce replies go to discuss, discuss replies go to list We've changed the list settings slightly to direct replies to this announcement list to go to the discussion list, and for replies on the discussion list to go to the list by default rather than to the poster. discuss(a)lists.surbl.org Hope that's ok with everyone. Please reply off list if it's not. :-) Jeff C.

1 0

[SURBL-Discuss] RE: Bill Stearns' sa-blacklist available as SURBL: ws.surbl.org
by Chris Santerre 21 Apr '04

21 Apr '04

[forwarded response from announce list] The ONLY complaint anyone ever had with Stearn's list was the overhead. With it setup this way, there is no stopping him!! :) Great job by Bill and Jeff! --Chris (Yeah I know,....the list.) :-) > -----Original Message----- > From: Don Newcomer [mailto:newcomer@dickinson.edu] > Sent: Tuesday, April 20, 2004 8:41 AM > To: Jeff Chan > Cc: SURBL Announce; SpamAssassin Users > Subject: Re: Bill Stearns' sa-blacklist available as SURBL: > ws.surbl.org > > > Wow, I implemented this yesterday and, after about 18 hours, > this rule was > #4 in my list of most-used rules! Good job, guys!

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss