Discuss July 2004

discuss@lists.surbl.org

65 participants
151 discussions

Lists anounced on SA-dev and users
by Jeff Chan 31 Oct '04

31 Oct '04

FWIW I've announced these lists on spamassassin-dev and spamassassin-users. Hopefully we will get some additional interested folks to join in. Jeff C. __ Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

2 1

[SURBL-Discuss] Please beta test ms.surbl.org - data from MailSecurity
by Jeff Chan 04 Aug '04

04 Aug '04

David Hooton has made available to us MailSecurity's spamvertised site data for public use. I have turned that data into a SURBL list for testing: ms.surbl.org This is the same company that has provided the phishing data currently in multi.surbl.org. Please do corpus checks, etc. but don't use it on high volume mail servers yet since it's only hosted on my mail server right now. Please let us know what spam detection rates you get and particularly about false positive rates. At this point we don't know if the data will stay a separate list, be added to multi only, get folded into ws, etc. The results you report will help us decide that. Jeff C.

9 23

TTL/turnaround times for SURBL
by Rob McEwen 04 Aug '04

04 Aug '04

RE: TTL/turnaround times for SURBL This discussion seems to have gotten drowned out by other recent discussions. I'd like to see where this stands at this point. In particular, Jeff noted that Outblaze updates their data very fast in response to fast analysis of their spam-trap data. But the OutBlaze feed at SURBL get updated every six hours? Doesn't that defeat the purpose. Would it be possible to speed up the ob.surbl.org refresh so that we can reap more benefits from their quick responsiveness? Also, it was mentioned that the sc.surbl.org data updates every ten minutes? Is there really substantial new or different data in this feed to justify this? (in other words, is there a system where very, very new data causes quick updates to sc.surbl.org) Finally, has any progress been made speeding up the refresh times for multi.surbl.org? Rob McEwen

3 3

RE: [SURBL-Discuss] Please beta test ms.surbl.org - data from Mai lSecurity
by Chris Santerre 02 Aug '04

02 Aug '04

>-----Original Message----- >From: Jeff Chan [mailto:jeffc@surbl.org] >Sent: Friday, July 30, 2004 3:32 AM >To: SURBL Discussion list >Subject: Re: [SURBL-Discuss] Please beta test ms.surbl.org - data from >MailSecurity > > >On Friday, July 30, 2004, 12:05:52 AM, Raymond Dijkxhoorn wrote: >>> This released list has been developed from actual spam >recieved since >>> the SURBL effort started. I don't doubt there are a lot of >double ups >>> with ws.surbl.org however. >>> >>> > As a seperate list i would so far say, uhm, not really >worth the efford, >>> > as a extra list of domains for WS, i think its worth it. >>> >>> Sure - kinda what I was originally intending anyway :) > >> Sounds like a plan! :) > >>> The list is developed from messages which slip through existing >>> SURBL's and other custom SA rulesets - addmittedly less and less are >>> getting through, but we do keep on top of it. It is >current, but may >>> also be a different sampling of messages to those you see. >>> >>> Expiry of listings is something we're still working on, but are a >>> little loathe to do because we've recently noticed a spate of >>> "recycled" domain names. > >> Ok, great. It would be a nice addition to WS i think... Anyone ? > >Sounds good to me. Anyone else? > >If so, Chris please consider pulling these into WS. > >Jeff C. 1) Send me the -different- domains that aren't already listed and I will blast them right into WS. I'll do it in such a way I can track FPs back to them. 2) How will we handle new additions? Submit them to who? 3) When will we have an SURBL contributors BBQ? Soon I hope, I'm hungry! --Chris

6 7

RFC: SURBL flyer
by Jeff Chan 02 Aug '04

02 Aug '04

I'm making a flyer to distribute at the CEAS spam conference next week and would appreciate any feedback on it please: http://www.surbl.org/flyer.html The audience should be mostly academics and some industry and Internet types who work on fighting spam. The conference appears to be mostly about technical and some legal theories for identifying and fighting spam. Here's the program: http://www.ceas.cc/acceptedpapers.htm Please send me your comments, improvements, suggestions, typos, errors, grammar, etc. Jeff C.

8 14

RE: {SPAM} Re: SURBL DoS possible?
by Chris Santerre 30 Jul '04

30 Jul '04

>-----Original Message----- >From: Mariano Absatz [mailto:el.baby@gmail.com] >Sent: Friday, July 30, 2004 8:32 AM >To: SpamAssassin users list; SURBL discussion list >Subject: Re: {SPAM} Re: SURBL DoS possible? > > >On Thu, 29 Jul 2004 17:27:00 -0400, Matt Kettler ><mkettler(a)evi-inc.com> wrote: >> At 04:48 PM 7/29/2004, Dougie Nisbet wrote: >> > > It picks a random sample of URLs. This was one of the >main concerns when we >> > > started talking about this feature. We're always one >step ahead of Mr. >> > > Spammy ;) >> > >> >I'm not so sure :( >> > >> >I've just received a spam which has several embedded URLs. >Each URL is based >> >on the recipient's name, in this case djn(a)highmoor.co.uk. >So there's djn.org, >> >djn.net, djn.com etc. At the the end there's the spam URL itself - >> >> A "random sample" is a bit of an over-simplified description of the >> behavior, but is at least partly true. >> >> From looking at the source code itself, no true DoS is >possible this way, >> although it may be possible for a spammer to reduce their >chances of being >> checked. >> >> Disclaimer: I'm not a Perl programer, so my interpretation >of the code's >> behavior may be incorrect. However, clearly the code has >limits to the >> number of queries they will generate per email. >> >> Mail::SpamCopURI (the plugin for 2.63) has a hard coded >limit the number of >> URI's it will check. Once it hits that limit, it seems to >ignore the rest >> of the URIs in the email. I'm unsure if the build-order is >sequential, >> alphabetical, or random, but I suspect sequential. (This is >as-of 0.18, I >> haven't checked 0.19). You can check this yourself in SpamCopURI.pm. >> >> In SA 3.0-pre3, there is a limit defined by the config option >> uridnsbl_max_domains. First, all the domains are extracted >from the URIs of >> the email and duplicates are removed. Then up to >uridnsbl_max_domains are >> randomly selected from the list. If there are fewer domains >than the limit, >> all of them should end up being selected. You can check this >behavior in >> lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm. The default for >this limit is >> 20, so your example email didn't approach that limit. >> >> So both do have limits to prevent DoS's by generating tons >of URI's. SA >> 3.0's URIDNSBL code appears to be more abuse resistant than >> Mail::SpamCopURI for 2.63 is. >> >> If any spammer tries to start evading the URI RBLs by >stuffing with tons of >> garbage URIs it should be easy to detect them by doing a >rule that counts >> URIs in the eval tests. Even in this case, the existing code >will prevent >> that stuffing from resulting in an absurd number of queries. >> >So, for what people say, the DoS part is taken care of... > >What seems to be possible is an evasive attack by adding more and more >non-relevant URIs to a message so as that the spammy ones have a >larger chance of NOT being chosen randomly by the plugin... > >Maybe when I grow up and have more time and knowledge :-) I will take >the 3.0 plugin and see if the 'randomness' can be somehow >heuristically helped... that is, ignore invisible ones, give more >chances to domains that appear more than once, etc... > >Regards. > I *think* I remember the devs saying 3.0 will ignore blank (invisible) URL links. BUt that might have been a discussion on SARE or the SURBL list. The threads start to blur together sometimes :) --Chris

1 0

Re: {SPAM} Re: SURBL DoS possible?
by Mariano Absatz 30 Jul '04

30 Jul '04

On Thu, 29 Jul 2004 17:27:00 -0400, Matt Kettler <mkettler(a)evi-inc.com> wrote: > At 04:48 PM 7/29/2004, Dougie Nisbet wrote: > > > It picks a random sample of URLs. This was one of the main concerns when we > > > started talking about this feature. We're always one step ahead of Mr. > > > Spammy ;) > > > >I'm not so sure :( > > > >I've just received a spam which has several embedded URLs. Each URL is based > >on the recipient's name, in this case djn(a)highmoor.co.uk. So there's djn.org, > >djn.net, djn.com etc. At the the end there's the spam URL itself - > > A "random sample" is a bit of an over-simplified description of the > behavior, but is at least partly true. > > From looking at the source code itself, no true DoS is possible this way, > although it may be possible for a spammer to reduce their chances of being > checked. > > Disclaimer: I'm not a Perl programer, so my interpretation of the code's > behavior may be incorrect. However, clearly the code has limits to the > number of queries they will generate per email. > > Mail::SpamCopURI (the plugin for 2.63) has a hard coded limit the number of > URI's it will check. Once it hits that limit, it seems to ignore the rest > of the URIs in the email. I'm unsure if the build-order is sequential, > alphabetical, or random, but I suspect sequential. (This is as-of 0.18, I > haven't checked 0.19). You can check this yourself in SpamCopURI.pm. > > In SA 3.0-pre3, there is a limit defined by the config option > uridnsbl_max_domains. First, all the domains are extracted from the URIs of > the email and duplicates are removed. Then up to uridnsbl_max_domains are > randomly selected from the list. If there are fewer domains than the limit, > all of them should end up being selected. You can check this behavior in > lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm. The default for this limit is > 20, so your example email didn't approach that limit. > > So both do have limits to prevent DoS's by generating tons of URI's. SA > 3.0's URIDNSBL code appears to be more abuse resistant than > Mail::SpamCopURI for 2.63 is. > > If any spammer tries to start evading the URI RBLs by stuffing with tons of > garbage URIs it should be easy to detect them by doing a rule that counts > URIs in the eval tests. Even in this case, the existing code will prevent > that stuffing from resulting in an absurd number of queries. > So, for what people say, the DoS part is taken care of... What seems to be possible is an evasive attack by adding more and more non-relevant URIs to a message so as that the spammy ones have a larger chance of NOT being chosen randomly by the plugin... Maybe when I grow up and have more time and knowledge :-) I will take the 3.0 plugin and see if the 'randomness' can be somehow heuristically helped... that is, ignore invisible ones, give more chances to domains that appear more than once, etc... Regards. -- Mariano Absatz - El Baby el (dot) baby (AT) gmail (dot) com el (punto) baby (ARROBA:@) gmail (punto) com

1 0

RE: SURBL DoS possible?
by Chris Santerre 30 Jul '04

30 Jul '04

>-----Original Message----- >From: Mariano Absatz [mailto:el.baby@gmail.com] >Sent: Thursday, July 29, 2004 9:41 AM >To: SURBL discussion list; SpamAssassin users list >Subject: SURBL DoS possible? > > >I was wondering... > >I didn't look at the source code for the SpamCopURI or the SA 3.0 >plugin but I guess it just looks for URI's within the messages and >issues a DNS query to the configured SURBLs for every different >canonicalized domain name... is it? > >What would happen if a spammer intentionally starts putting hundreds >of different invisible random URIs within the message trying to DoS >SURBL? > >Does the SA plugins check for this condition? Or have a limit as to >how many SURBL queries will it issue for a given message? > >TIA > It picks a random sample of URLs. This was one of the main concerns when we started talking about this feature. We're always one step ahead of Mr. Spammy ;) --Chris

2 1

Re: SURBL DoS possible?
by Mariano Absatz 30 Jul '04

30 Jul '04

On Thu, 29 Jul 2004 09:24:31 -0500, Bob Apthorpe <apthorpe+sa(a)cynistar.net> wrote: > On Thu, 29 Jul 2004 10:40:44 -0300 Mariano Absatz <el.baby(a)gmail.com> wrote: > > > I was wondering... > [...] > > What would happen if a spammer intentionally starts putting hundreds > > of different invisible random URIs within the message trying to DoS > > SURBL? > > One can compensate for this by testing only a few, visible URIs, or > skipping the RHSBL tests altogether and triggering the > "MAIL_HAS_CRAPLOAD_OF_INVISIBLE_URIS" rule. Or something like that. Right... but I don't want to get rid of SURBL... it is working very nicely, it finds a lot of spam and I have yet to find a FP myself (though others have seen them)... My question is more to the people that developed the SURBL plugins for SA (or those that have read and understood them), to know if there's something in the plugins to avoid a DoS attempt of this kind. Thanx for your reply, anyway. -- Mariano Absatz - El Baby el (dot) baby (AT) gmail (dot) com el (punto) baby (ARROBA:@) gmail (punto) com

3 2

Fwd: Re: {SPAM} Re: SURBL DoS possible?
by Jeff Chan 29 Jul '04

29 Jul '04

This is a forwarded message From: Matt Kettler To: spamassassin-users Date: Thursday, July 29, 2004, 2:27:00 PM Subject: {SPAM} Re: SURBL DoS possible? ===8<==============Original message text=============== At 04:48 PM 7/29/2004, Dougie Nisbet wrote: > > It picks a random sample of URLs. This was one of the main concerns when we > > started talking about this feature. We're always one step ahead of Mr. > > Spammy ;) > >I'm not so sure :( > >I've just received a spam which has several embedded URLs. Each URL is based >on the recipient's name, in this case djn(a)highmoor.co.uk. So there's djn.org, >djn.net, djn.com etc. At the the end there's the spam URL itself - A "random sample" is a bit of an over-simplified description of the behavior, but is at least partly true. From looking at the source code itself, no true DoS is possible this way, although it may be possible for a spammer to reduce their chances of being checked. Disclaimer: I'm not a Perl programer, so my interpretation of the code's behavior may be incorrect. However, clearly the code has limits to the number of queries they will generate per email. Mail::SpamCopURI (the plugin for 2.63) has a hard coded limit the number of URI's it will check. Once it hits that limit, it seems to ignore the rest of the URIs in the email. I'm unsure if the build-order is sequential, alphabetical, or random, but I suspect sequential. (This is as-of 0.18, I haven't checked 0.19). You can check this yourself in SpamCopURI.pm. In SA 3.0-pre3, there is a limit defined by the config option uridnsbl_max_domains. First, all the domains are extracted from the URIs of the email and duplicates are removed. Then up to uridnsbl_max_domains are randomly selected from the list. If there are fewer domains than the limit, all of them should end up being selected. You can check this behavior in lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm. The default for this limit is 20, so your example email didn't approach that limit. So both do have limits to prevent DoS's by generating tons of URI's. SA 3.0's URIDNSBL code appears to be more abuse resistant than Mail::SpamCopURI for 2.63 is. If any spammer tries to start evading the URI RBLs by stuffing with tons of garbage URIs it should be easy to detect them by doing a rule that counts URIs in the eval tests. Even in this case, the existing code will prevent that stuffing from resulting in an absurd number of queries. ===8<===========End of original message text===========

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss July 2004