Hi
I'm adding SURBL support to the urlbody plugin of spampal. Spampal is a freeware windows local proxy anti-spam application, with a userbase of almost 100.000. (http://www.spampal.org/)
I use only multi. At the moment only JP, but this will become a configuration option. Probably also with a "two out of all" option.
- I've added a local skiplist with about top half of the public "whitelist", no need to query those.
- Domains are only queried one time per e-mail (the local pc DNS service would probably cache those, but anyway added...).
- If the're moere than 20 domains to check, only the first (or last) 5 are done.
Any suggestions are welcome.
Alain
On Friday, February 11, 2005, 8:00:35 AM, test test wrote:
I'm adding SURBL support to the urlbody plugin of spampal. Spampal is a freeware windows local proxy anti-spam application, with a userbase of almost 100.000. (http://www.spampal.org/)
I use only multi. At the moment only JP, but this will become a configuration option. Probably also with a "two out of all" option .
- I've added a local skiplist with about top half of the public
"whitelist", no need to query those.
- Domains are only queried one time per e-mail (the local pc
DNS service would probably cache those, but anyway added...).
- If the're moere than 20 domains to check, only the first (or
last) 5 are done.
Any suggestions are welcome.
Alain
Sounds good!
The only thing I would request is time-based caching of list lookups to cut down on DNS lookups. Can you set up some kind of caching of names that persists say for 15 minutes and lives across messages?
Generally speaking it may be better to apply this kind of filtering at the server level since there are economies of scale, especially in terms of things like DNS lookups and caching. If we suddenly get 100k more DNS clients, that could tax the name servers somewhat. If those same 100k users were using 100 servers instead, the DNS loading would be quite a bit less. In that sense centralization is desirable.
Thanks for implementing a local whitelist. Agreed there's no need to check hopefully obvious whitehats.
Jeff C. -- "If it appears in hams, then don't list it."
On Saturday, February 12, 2005, 12:38:44 AM, Jeff Chan wrote:
On Friday, February 11, 2005, 8:00:35 AM, test test wrote:
I'm adding SURBL support to the urlbody plugin of spampal. Spampal is a freeware windows local proxy anti-spam application, with a userbase of almost 100.000. (http://www.spampal.org/)
I use only multi. At the moment only JP, but this will become a configuration option. Probably also with a "two out of all" option .
Please consider using the sc.surbl.org data (bit 2 in multi) also. It's very effective and small.
- I've added a local skiplist with about top half of the public
"whitelist", no need to query those.
When you say half, that may be more than optimal (should be about 5000 records). SpamAssassin is using the top 125, which worked out to about the 50%th percentile of all whitelist hits when we first set this up. (Now that result is skewed *because* SpamAssassin isn't checking those 125 any more, but their snapshot of the 125 is still probably useful.
I'd say anything between 100 and 1000 would probably be a good compromise between list size and coverage.
Jeff C. -- "If it appears in hams, then don't list it."
Hi Jeff
I'm adding SURBL support to the urlbody plugin of spampal. Spampal is a freeware windows local proxy anti-spam application, with a userbase of almost 100.000. (http://www.spampal.org/)
I use only multi. At the moment only JP, but this will become a configuration option. Probably also with a "two out of all" option .
Please consider using the sc.surbl.org data (bit 2 in multi) also. It's very effective and small.
I'm considering scoring on multi and give the user some freedom, see other thread.
- I've added a local skiplist with about top half of the public
"whitelist", no need to query those.
When you say half, that may be more than optimal (should be about 5000 records). SpamAssassin is using the top 125, which worked out to about the 50%th percentile of all whitelist hits when we first set this up. (Now that result is skewed *because* SpamAssassin isn't checking those 125 any more, but their snapshot of the 125 is still probably useful.
I'd say anything between 100 and 1000 would probably be a good compromise between list size and coverage.
The only disadvantage I see from a bigger local skiplist is some local CPU usage for every uri in a email. Most pc's have plenty of CPU power ;-) If this could become a problem, I can lower or optimise the local checking. Are there any other disadvantages?
Alain
Hi Jeff
On Friday, February 11, 2005, 8:00:35 AM, test test wrote:
I'm adding SURBL support to the urlbody plugin of spampal. Spampal is a freeware windows local proxy anti-spam application, with a userbase of almost 100.000. (http://www.spampal.org/)
I use only multi. At the moment only JP, but this will become a configuration option. Probably also with a "two out of all" option .
- I've added a local skiplist with about top half of the public
"whitelist", no need to query those.
- Domains are only queried one time per e-mail (the local pc
DNS service would probably cache those, but anyway added...).
- If the're moere than 20 domains to check, only the first (or
last) 5 are done.
Any suggestions are welcome.
Alain
Sounds good!
The only thing I would request is time-based caching of list lookups to cut down on DNS lookups. Can you set up some kind of caching of names that persists say for 15 minutes and lives across messages?
As far as I know this will be done by windows. Maybe not for 15 minutes, but for at least a couple of minutes.
Generally speaking it may be better to apply this kind of filtering at the server level since there are economies of scale, especially in terms of things like DNS lookups and caching. If we suddenly get 100k more DNS clients, that could tax the name servers somewhat. If those same 100k users were using 100 servers instead, the DNS loading would be quite a bit less. In that sense centralization is desirable.
Mmmm isn't the dns server from the ISP caching the dns requests? I would think it doesn't make a big difference (except when a server is rsync'ing). The difference could be that end users check their e-mail not when arriving on the MTA, but later.
Alain
-
Alain -
Jeff Chan -
test