-----Original Message----- From: Jeff Chan [mailto:jeffc@surbl.org] Sent: Wednesday, April 21, 2004 7:47 AM To: SURBL Discussion list; Chris Santerre Subject: Re: [SURBL-Discuss] BigEvil + MidEvil as SURBL
On Wednesday, April 21, 2004, 4:35:41 AM, Raymond Dijkxhoorn wrote:
Hi!
BigEvil is a fairly slowly moving list. Paul Barbeau's MidEvil is quicker moving and gets new domains usually before Chris can get them into BE. In that sense ME is a feeder of changes into BE. Since they are closely related, I merged them into a single be.surbl.org. I hope Chris and Paul agree that's appropriate.
What I'd like to know is what TTLs I should use on the BE data. Probably it depends on how often ME is typically updated. So... how often does ME get updated Paul? :-)
Also I'd like feedback on the TXT message. I've got the placeholder:
"Blocked in BigEvil. See: http://www.rulesemporium.com/"
but would like feedback on it.
Do we get a different value on looking up? For example:
127.0.0.2 for BE and 127.0.0.3 for ME ?
We should start doing that also to get the combined list going.
Currently we will have them lumped together (i.e. it's all .2 without differentiation as to the source). As I understand it that may be appropriate since ME is meant to be essentially updates to BE. I think of them as the same list, especially since Chris eventually merges the ME (update) entries into BE. I kind of short circuit that process by merging them for them before turning them into be.surbl.org. Hopefully that's ok.
Lists with greater differences such as ws and sc probably should get different A or TXT records when we eventually combine them.
FWIW even if we offer a combined list, the individual ones will probably still be available, like SBL, XBL & SBL-XBL at spamhaus.
Jeff C.
P.S. Chris please sign up for the SURBL Discussion and Announce lists if you can: http://lists.surbl.org/
I already am ;)
Yeah, usually I update BigEvil a lot more often. I'm dealing with a lot of projects now. Some are even work related ;) And then some are beta testing a new game :-) Paul and I are still working out how we can merge ME and BE together without a lot of work. But I have no problems at all combining the ME and BE together and letting Paul add just as much as me. He knows my basic criteria for checking the domains.
A few things off the top of my head. Sorry if they have been discussed, I have a LOT of email to read :)
1) BigEvil wildcards. Not sure how you would handle these. Something like evil\d{2,4}spam.com is a general wildcard. Some of those domains don't even exhist. Not sure how SURBL will handle that.
2) Where would I send updates? As single domains, or a txt list? How would I remove an FP?
3) What is the quickest way to check a domain against the other SURBL lists? Basically I see no reason to duplicate the listings. *gulp* and on a Windowze machine? (Don't ask!)
4) Has there been any talk with the sendmail people? It would be interesting to actually block at the MTA level based on an evil URL. I realise the inherent dangers in this ;)
--Chris
On Wednesday, April 21, 2004, 6:30:05 AM, Chris Santerre wrote:
Paul and I are still working out how we can merge ME and BE together without a lot of work. But I have no problems at all combining the ME and BE together and letting Paul add just as much as me. He knows my basic criteria for checking the domains.
Sounds good. Can you let me know what kind of TTL I should set?
Basically I'd like to set the lifetime of the zone info to something relevant towards how often you and Paul usually update the lists. Nothing too specific is needed, just a general idea. Like is it daily, twice a day, every other day on average, etc.
Also does this TXT record work for you guys:
"Blocked in BigEvil. See: http://www.rulesemporium.com/"
It was just a generic placeholder. I'd like comments/improvements on it.
- BigEvil wildcards. Not sure how you would handle these. Something like
evil\d{2,4}spam.com is a general wildcard. Some of those domains don't even exhist. Not sure how SURBL will handle that.
Yes, I should have mentioned that I'm simply discarding them. Unfortunately there's no easy way to deal with them. Domains without any patterns in them, which are a majority, come right through. The script is at:
http://spamcheck.freeapp.net/handle-bigevil
http://spamcheck.freeapp.net/clean-bigevil.sed
- Where would I send updates? As single domains, or a txt list? How would I
remove an FP?
As you can see from the script, we are web-grabbing copies of both .cf files every time the script is run, which is currently hourly. It's all automatic; all you guys need to do is have the current versions on your web sites.
- What is the quickest way to check a domain against the other SURBL lists?
Basically I see no reason to duplicate the listings. *gulp* and on a Windowze machine? (Don't ask!)
I wouldn't worry too much about that for now. For now we just want to get an accurate record of everything. We're working on ways to merge things next.
- Has there been any talk with the sendmail people? It would be interesting
to actually block at the MTA level based on an evil URL. I realise the inherent dangers in this ;)
Yes, there is talk about sendmail milters using SURBLs. I haven't heard of anyone doing one yet, but they're feasible. The limiting factor is the FP rate. FPs must be as close to zero as possible before people will dare to reject spams at the MTA level using SURBLs, other than perhaps for personal servers, etc.
Jeff C.
- Where would I send updates? As single domains, or a txt list? How would I
remove an FP?
In case it's not clear, FPs will come out of be.surbl.org automatically when they come out of bigevil.cf and midevil.cf.
If you need to manually whitelist a domain, just send a message to us at whitelist at surbl dot org and we'll do that ASAP.
Jeff C.
-
Chris Santerre -
Jeff Chan