Discuss

discuss@lists.surbl.org

1862 discussions

RE: [SURBL-Discuss] What data sources are in ws now? :-)
by Chris Santerre 02 Aug '04

02 Aug '04

>-----Original Message----- >From: Jeff Chan [mailto:jeffc@surbl.org] >Sent: Monday, August 02, 2004 6:00 AM >To: SURBL Discuss >Subject: [SURBL-Discuss] What data sources are in ws now? :-) > > >I'm updating the SRUBL Lists about about ws, and may have lost >track of all the data sources now going into it: > > http://www.surbl.org/lists.html#ws > >> ws.surbl.org - sa-blacklist and other domains > >> ws.surbl.org has entries from three SpamAssassin rulesets: Bill >> Stearns' sa-blacklist, BigEvil.cf from Chris Santerre and his >> SARE cohorts, and Paul Barbeau's MidEvil.cf. ws.surbl.org also >> has entries from additional spam URI domain lists such as >> MailSecurity's formely proprietary SURBL lists, data from Joe >> Wein's jwSpamSpy Windows POP mail spam-filtering agent, plus >> some other manual lists, mostly maintained by Chris. > >Chris, Bill and Raymond, am I forgetting any that we should be >crediting publically? > >Jeff C. Yeah that is good. I'm still getting new 6dos (ds) submissions. I hope to check those today. I would love to send someone a few 100 to check, if anyone volunteers to split this with me. I'm falling behind due to real work bogging me down. :( --Chris

2 1

RFC: SURBL flyer
by Jeff Chan 02 Aug '04

02 Aug '04

I'm making a flyer to distribute at the CEAS spam conference next week and would appreciate any feedback on it please: http://www.surbl.org/flyer.html The audience should be mostly academics and some industry and Internet types who work on fighting spam. The conference appears to be mostly about technical and some legal theories for identifying and fighting spam. Here's the program: http://www.ceas.cc/acceptedpapers.htm Please send me your comments, improvements, suggestions, typos, errors, grammar, etc. Jeff C.

8 14

Additional data sources into ws.surbl.org
by Jeff Chan 02 Aug '04

02 Aug '04

Thanks to Raymond's efforts a couple new data sources are going into ws.surbl.org: 1. MailSecurity's own SURBL lists of about 2200 entries found in spam. 2. Joe Wein's list from his spam filter for pop mailboxes Windows client application jwSpamSpy: http://www.joewein.de/sw/jwSpamSpy/ Before I announce this more widely I thought I would ask if anyone else is noticing improved spam detection and/or any changes in FPs. Adding these two seems to have boosted Raymond's spam detection results with ws.surbl.org above sbl-xbl.spamhaus.org: > the hitrate on my own setup went up on WS with around 6%. > > Here's my todays top 10: > > SpamAssassin tag hits: (top 100) > #1 115259 BAYES_99 > #2 89993 WS_URI_RBL > #3 88006 RCVD_IN_SBL+XBL > #4 85157 HTML_MESSAGE > #5 81296 RCVD_IN_BL_SPAMCOP_NET > #6 80990 OUTBLAZE_URI_RBL > #7 67765 RCVD_IN_SORBS > #8 65982 SPAMCOP_URI_RBL > #9 61878 ABUSEBUTLER_URI_RBL > #10 53390 RCVD_IN_DSBL We're particularly concerned about FPs, so please report any that you find. Posting them to this discussion list is fine IMO. Every FP is very important to eliminate and understand and correct the source of. Jeff C.

3 2

RE: {SPAM} Re: SURBL DoS possible?
by Chris Santerre 30 Jul '04

30 Jul '04

>-----Original Message----- >From: Mariano Absatz [mailto:el.baby@gmail.com] >Sent: Friday, July 30, 2004 8:32 AM >To: SpamAssassin users list; SURBL discussion list >Subject: Re: {SPAM} Re: SURBL DoS possible? > > >On Thu, 29 Jul 2004 17:27:00 -0400, Matt Kettler ><mkettler(a)evi-inc.com> wrote: >> At 04:48 PM 7/29/2004, Dougie Nisbet wrote: >> > > It picks a random sample of URLs. This was one of the >main concerns when we >> > > started talking about this feature. We're always one >step ahead of Mr. >> > > Spammy ;) >> > >> >I'm not so sure :( >> > >> >I've just received a spam which has several embedded URLs. >Each URL is based >> >on the recipient's name, in this case djn(a)highmoor.co.uk. >So there's djn.org, >> >djn.net, djn.com etc. At the the end there's the spam URL itself - >> >> A "random sample" is a bit of an over-simplified description of the >> behavior, but is at least partly true. >> >> From looking at the source code itself, no true DoS is >possible this way, >> although it may be possible for a spammer to reduce their >chances of being >> checked. >> >> Disclaimer: I'm not a Perl programer, so my interpretation >of the code's >> behavior may be incorrect. However, clearly the code has >limits to the >> number of queries they will generate per email. >> >> Mail::SpamCopURI (the plugin for 2.63) has a hard coded >limit the number of >> URI's it will check. Once it hits that limit, it seems to >ignore the rest >> of the URIs in the email. I'm unsure if the build-order is >sequential, >> alphabetical, or random, but I suspect sequential. (This is >as-of 0.18, I >> haven't checked 0.19). You can check this yourself in SpamCopURI.pm. >> >> In SA 3.0-pre3, there is a limit defined by the config option >> uridnsbl_max_domains. First, all the domains are extracted >from the URIs of >> the email and duplicates are removed. Then up to >uridnsbl_max_domains are >> randomly selected from the list. If there are fewer domains >than the limit, >> all of them should end up being selected. You can check this >behavior in >> lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm. The default for >this limit is >> 20, so your example email didn't approach that limit. >> >> So both do have limits to prevent DoS's by generating tons >of URI's. SA >> 3.0's URIDNSBL code appears to be more abuse resistant than >> Mail::SpamCopURI for 2.63 is. >> >> If any spammer tries to start evading the URI RBLs by >stuffing with tons of >> garbage URIs it should be easy to detect them by doing a >rule that counts >> URIs in the eval tests. Even in this case, the existing code >will prevent >> that stuffing from resulting in an absurd number of queries. >> >So, for what people say, the DoS part is taken care of... > >What seems to be possible is an evasive attack by adding more and more >non-relevant URIs to a message so as that the spammy ones have a >larger chance of NOT being chosen randomly by the plugin... > >Maybe when I grow up and have more time and knowledge :-) I will take >the 3.0 plugin and see if the 'randomness' can be somehow >heuristically helped... that is, ignore invisible ones, give more >chances to domains that appear more than once, etc... > >Regards. > I *think* I remember the devs saying 3.0 will ignore blank (invisible) URL links. BUt that might have been a discussion on SARE or the SURBL list. The threads start to blur together sometimes :) --Chris

1 0

Re: {SPAM} Re: SURBL DoS possible?
by Mariano Absatz 30 Jul '04

30 Jul '04

On Thu, 29 Jul 2004 17:27:00 -0400, Matt Kettler <mkettler(a)evi-inc.com> wrote: > At 04:48 PM 7/29/2004, Dougie Nisbet wrote: > > > It picks a random sample of URLs. This was one of the main concerns when we > > > started talking about this feature. We're always one step ahead of Mr. > > > Spammy ;) > > > >I'm not so sure :( > > > >I've just received a spam which has several embedded URLs. Each URL is based > >on the recipient's name, in this case djn(a)highmoor.co.uk. So there's djn.org, > >djn.net, djn.com etc. At the the end there's the spam URL itself - > > A "random sample" is a bit of an over-simplified description of the > behavior, but is at least partly true. > > From looking at the source code itself, no true DoS is possible this way, > although it may be possible for a spammer to reduce their chances of being > checked. > > Disclaimer: I'm not a Perl programer, so my interpretation of the code's > behavior may be incorrect. However, clearly the code has limits to the > number of queries they will generate per email. > > Mail::SpamCopURI (the plugin for 2.63) has a hard coded limit the number of > URI's it will check. Once it hits that limit, it seems to ignore the rest > of the URIs in the email. I'm unsure if the build-order is sequential, > alphabetical, or random, but I suspect sequential. (This is as-of 0.18, I > haven't checked 0.19). You can check this yourself in SpamCopURI.pm. > > In SA 3.0-pre3, there is a limit defined by the config option > uridnsbl_max_domains. First, all the domains are extracted from the URIs of > the email and duplicates are removed. Then up to uridnsbl_max_domains are > randomly selected from the list. If there are fewer domains than the limit, > all of them should end up being selected. You can check this behavior in > lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm. The default for this limit is > 20, so your example email didn't approach that limit. > > So both do have limits to prevent DoS's by generating tons of URI's. SA > 3.0's URIDNSBL code appears to be more abuse resistant than > Mail::SpamCopURI for 2.63 is. > > If any spammer tries to start evading the URI RBLs by stuffing with tons of > garbage URIs it should be easy to detect them by doing a rule that counts > URIs in the eval tests. Even in this case, the existing code will prevent > that stuffing from resulting in an absurd number of queries. > So, for what people say, the DoS part is taken care of... What seems to be possible is an evasive attack by adding more and more non-relevant URIs to a message so as that the spammy ones have a larger chance of NOT being chosen randomly by the plugin... Maybe when I grow up and have more time and knowledge :-) I will take the 3.0 plugin and see if the 'randomness' can be somehow heuristically helped... that is, ignore invisible ones, give more chances to domains that appear more than once, etc... Regards. -- Mariano Absatz - El Baby el (dot) baby (AT) gmail (dot) com el (punto) baby (ARROBA:@) gmail (punto) com

1 0

RE: SURBL DoS possible?
by Chris Santerre 30 Jul '04

30 Jul '04

>-----Original Message----- >From: Mariano Absatz [mailto:el.baby@gmail.com] >Sent: Thursday, July 29, 2004 9:41 AM >To: SURBL discussion list; SpamAssassin users list >Subject: SURBL DoS possible? > > >I was wondering... > >I didn't look at the source code for the SpamCopURI or the SA 3.0 >plugin but I guess it just looks for URI's within the messages and >issues a DNS query to the configured SURBLs for every different >canonicalized domain name... is it? > >What would happen if a spammer intentionally starts putting hundreds >of different invisible random URIs within the message trying to DoS >SURBL? > >Does the SA plugins check for this condition? Or have a limit as to >how many SURBL queries will it issue for a given message? > >TIA > It picks a random sample of URLs. This was one of the main concerns when we started talking about this feature. We're always one step ahead of Mr. Spammy ;) --Chris

2 1

Re: SURBL DoS possible?
by Mariano Absatz 30 Jul '04

30 Jul '04

On Thu, 29 Jul 2004 09:24:31 -0500, Bob Apthorpe <apthorpe+sa(a)cynistar.net> wrote: > On Thu, 29 Jul 2004 10:40:44 -0300 Mariano Absatz <el.baby(a)gmail.com> wrote: > > > I was wondering... > [...] > > What would happen if a spammer intentionally starts putting hundreds > > of different invisible random URIs within the message trying to DoS > > SURBL? > > One can compensate for this by testing only a few, visible URIs, or > skipping the RHSBL tests altogether and triggering the > "MAIL_HAS_CRAPLOAD_OF_INVISIBLE_URIS" rule. Or something like that. Right... but I don't want to get rid of SURBL... it is working very nicely, it finds a lot of spam and I have yet to find a FP myself (though others have seen them)... My question is more to the people that developed the SURBL plugins for SA (or those that have read and understood them), to know if there's something in the plugins to avoid a DoS attempt of this kind. Thanx for your reply, anyway. -- Mariano Absatz - El Baby el (dot) baby (AT) gmail (dot) com el (punto) baby (ARROBA:@) gmail (punto) com

3 2

Fwd: Re: {SPAM} Re: SURBL DoS possible?
by Jeff Chan 29 Jul '04

29 Jul '04

This is a forwarded message From: Matt Kettler To: spamassassin-users Date: Thursday, July 29, 2004, 2:27:00 PM Subject: {SPAM} Re: SURBL DoS possible? ===8<==============Original message text=============== At 04:48 PM 7/29/2004, Dougie Nisbet wrote: > > It picks a random sample of URLs. This was one of the main concerns when we > > started talking about this feature. We're always one step ahead of Mr. > > Spammy ;) > >I'm not so sure :( > >I've just received a spam which has several embedded URLs. Each URL is based >on the recipient's name, in this case djn(a)highmoor.co.uk. So there's djn.org, >djn.net, djn.com etc. At the the end there's the spam URL itself - A "random sample" is a bit of an over-simplified description of the behavior, but is at least partly true. From looking at the source code itself, no true DoS is possible this way, although it may be possible for a spammer to reduce their chances of being checked. Disclaimer: I'm not a Perl programer, so my interpretation of the code's behavior may be incorrect. However, clearly the code has limits to the number of queries they will generate per email. Mail::SpamCopURI (the plugin for 2.63) has a hard coded limit the number of URI's it will check. Once it hits that limit, it seems to ignore the rest of the URIs in the email. I'm unsure if the build-order is sequential, alphabetical, or random, but I suspect sequential. (This is as-of 0.18, I haven't checked 0.19). You can check this yourself in SpamCopURI.pm. In SA 3.0-pre3, there is a limit defined by the config option uridnsbl_max_domains. First, all the domains are extracted from the URIs of the email and duplicates are removed. Then up to uridnsbl_max_domains are randomly selected from the list. If there are fewer domains than the limit, all of them should end up being selected. You can check this behavior in lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm. The default for this limit is 20, so your example email didn't approach that limit. So both do have limits to prevent DoS's by generating tons of URI's. SA 3.0's URIDNSBL code appears to be more abuse resistant than Mail::SpamCopURI for 2.63 is. If any spammer tries to start evading the URI RBLs by stuffing with tons of garbage URIs it should be easy to detect them by doing a rule that counts URIs in the eval tests. Even in this case, the existing code will prevent that stuffing from resulting in an absurd number of queries. ===8<===========End of original message text===========

1 0

SURBL DoS possible?
by Mariano Absatz 29 Jul '04

29 Jul '04

I was wondering... I didn't look at the source code for the SpamCopURI or the SA 3.0 plugin but I guess it just looks for URI's within the messages and issues a DNS query to the configured SURBLs for every different canonicalized domain name... is it? What would happen if a spammer intentionally starts putting hundreds of different invisible random URIs within the message trying to DoS SURBL? Does the SA plugins check for this condition? Or have a limit as to how many SURBL queries will it issue for a given message? TIA -- Mariano Absatz - El Baby el (dot) baby (AT) gmail (dot) com el (punto) baby (ARROBA:@) gmail (punto) com

3 2

RE: [SURBL-Discuss] RFC: SURBL flyer
by Chris Santerre 29 Jul '04

29 Jul '04

>-----Original Message----- >From: Jeff Chan [mailto:jeffc@surbl.org] >Sent: Thursday, July 29, 2004 4:09 AM >To: SURBL Discuss >Subject: Re: [SURBL-Discuss] RFC: SURBL flyer > > >On Thursday, July 29, 2004, 12:55:10 AM, John Andersen wrote: >> On Tuesday 27 July 2004 12:58 am, Jeff Chan wrote: >>> On Tuesday, July 27, 2004, 1:36:53 AM, Daniel Kleinsinger wrote: >>> > http://mailscanner.info >>> > MailScanner doesn't include or depend on SA (without SA >one can still >>> > check for viruses and do old fashioned RBL lookups), but >the only way to >>> > have MailScanner check SURBLs is to utilize its ability >to call SA 2.63 >>> > + SpamCopURI or SA 3.0. >>> >>> OK Thanks for that. I've taken MailScanner off the flyer. >>> >>> http://www.surbl.org/flyer.html >>> >>> Does anyone have any other comments for me? >>> >>> Jeff C. > >> Yeah... SURBLs don't BLOCK anything. >> Blocking is up to the user. >> That point should be made. Surprisingly, lots of people get >> bent out of shape if you use the word BLOCK and EMAIL >> in the same sentence. > >True, but I never said SURBLs block anything. I said: >"SURBLs... are used to block messages." Technically >SURBLs don't block anything. Programs that use SURBLs >do the blocking.... > >Jeff C. > It needs an attention grabber. How about these titles: Sexy Underwear Really Black Lace (SURBL) She Used Really Big Lemons (SURBL) Some United Redheads Bring Love (SURBL) Simultaneous Ugly Rats Bite Leroy (SURBL) Sorry the medicine is kicking in........ --Chris

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss