>-----Original Message-----
>From: Jeff Chan [mailto:jeffc@surbl.org]
>Sent: Friday, July 30, 2004 3:32 AM
>To: SURBL Discussion list
>Subject: Re: [SURBL-Discuss] Please beta test ms.surbl.org - data from
>MailSecurity
>
>
>On Friday, July 30, 2004, 12:05:52 AM, Raymond Dijkxhoorn wrote:
>>> This released list has been developed from actual spam
>recieved since
>>> the SURBL effort started. I don't doubt there are a lot of
>double ups
>&…
[View More]gt;> with ws.surbl.org however.
>>>
>>> > As a seperate list i would so far say, uhm, not really
>worth the efford,
>>> > as a extra list of domains for WS, i think its worth it.
>>>
>>> Sure - kinda what I was originally intending anyway :)
>
>> Sounds like a plan! :)
>
>>> The list is developed from messages which slip through existing
>>> SURBL's and other custom SA rulesets - addmittedly less and less are
>>> getting through, but we do keep on top of it. It is
>current, but may
>>> also be a different sampling of messages to those you see.
>>>
>>> Expiry of listings is something we're still working on, but are a
>>> little loathe to do because we've recently noticed a spate of
>>> "recycled" domain names.
>
>> Ok, great. It would be a nice addition to WS i think... Anyone ?
>
>Sounds good to me. Anyone else?
>
>If so, Chris please consider pulling these into WS.
>
>Jeff C.
1) Send me the -different- domains that aren't already listed and I will
blast them right into WS. I'll do it in such a way I can track FPs back to
them.
2) How will we handle new additions? Submit them to who?
3) When will we have an SURBL contributors BBQ? Soon I hope, I'm hungry!
--Chris
[View Less]
I'm updating the SRUBL Lists about about ws, and may have lost
track of all the data sources now going into it:
http://www.surbl.org/lists.html#ws
> ws.surbl.org - sa-blacklist and other domains
> ws.surbl.org has entries from three SpamAssassin rulesets: Bill
> Stearns' sa-blacklist, BigEvil.cf from Chris Santerre and his
> SARE cohorts, and Paul Barbeau's MidEvil.cf. ws.surbl.org also
> has entries from additional spam URI domain lists such as
> MailSecurity's formely …
[View More]proprietary SURBL lists, data from Joe
> Wein's jwSpamSpy Windows POP mail spam-filtering agent, plus
> some other manual lists, mostly maintained by Chris.
Chris, Bill and Raymond, am I forgetting any that we should be
crediting publically?
Jeff C.
[View Less]
>-----Original Message-----
>From: Jeff Chan [mailto:jeffc@surbl.org]
>Sent: Monday, August 02, 2004 6:00 AM
>To: SURBL Discuss
>Subject: [SURBL-Discuss] What data sources are in ws now? :-)
>
>
>I'm updating the SRUBL Lists about about ws, and may have lost
>track of all the data sources now going into it:
>
> http://www.surbl.org/lists.html#ws
>
>> ws.surbl.org - sa-blacklist and other domains
>
>> ws.surbl.org has entries from three …
[View More]SpamAssassin rulesets: Bill
>> Stearns' sa-blacklist, BigEvil.cf from Chris Santerre and his
>> SARE cohorts, and Paul Barbeau's MidEvil.cf. ws.surbl.org also
>> has entries from additional spam URI domain lists such as
>> MailSecurity's formely proprietary SURBL lists, data from Joe
>> Wein's jwSpamSpy Windows POP mail spam-filtering agent, plus
>> some other manual lists, mostly maintained by Chris.
>
>Chris, Bill and Raymond, am I forgetting any that we should be
>crediting publically?
>
>Jeff C.
Yeah that is good. I'm still getting new 6dos (ds) submissions. I hope to
check those today. I would love to send someone a few 100 to check, if
anyone volunteers to split this with me. I'm falling behind due to real work
bogging me down. :(
--Chris
[View Less]
I'm making a flyer to distribute at the CEAS spam conference next
week and would appreciate any feedback on it please:
http://www.surbl.org/flyer.html
The audience should be mostly academics and some industry and
Internet types who work on fighting spam. The conference
appears to be mostly about technical and some legal theories
for identifying and fighting spam. Here's the program:
http://www.ceas.cc/acceptedpapers.htm
Please send me your comments, improvements, suggestions,
typos, …
[View More]errors, grammar, etc.
Jeff C.
[View Less]
Thanks to Raymond's efforts a couple new data sources are going
into ws.surbl.org:
1. MailSecurity's own SURBL lists of about 2200 entries
found in spam.
2. Joe Wein's list from his spam filter for pop mailboxes
Windows client application jwSpamSpy:
http://www.joewein.de/sw/jwSpamSpy/
Before I announce this more widely I thought I would ask if
anyone else is noticing improved spam detection and/or any
changes in FPs. Adding these two seems to have boosted
Raymond's spam detection results …
[View More]with ws.surbl.org above
sbl-xbl.spamhaus.org:
> the hitrate on my own setup went up on WS with around 6%.
>
> Here's my todays top 10:
>
> SpamAssassin tag hits: (top 100)
> #1 115259 BAYES_99
> #2 89993 WS_URI_RBL
> #3 88006 RCVD_IN_SBL+XBL
> #4 85157 HTML_MESSAGE
> #5 81296 RCVD_IN_BL_SPAMCOP_NET
> #6 80990 OUTBLAZE_URI_RBL
> #7 67765 RCVD_IN_SORBS
> #8 65982 SPAMCOP_URI_RBL
> #9 61878 ABUSEBUTLER_URI_RBL
> #10 53390 RCVD_IN_DSBL
We're particularly concerned about FPs, so please report any
that you find. Posting them to this discussion list is fine
IMO. Every FP is very important to eliminate and understand
and correct the source of.
Jeff C.
[View Less]
>-----Original Message-----
>From: Mariano Absatz [mailto:el.baby@gmail.com]
>Sent: Friday, July 30, 2004 8:32 AM
>To: SpamAssassin users list; SURBL discussion list
>Subject: Re: {SPAM} Re: SURBL DoS possible?
>
>
>On Thu, 29 Jul 2004 17:27:00 -0400, Matt Kettler
><mkettler(a)evi-inc.com> wrote:
>> At 04:48 PM 7/29/2004, Dougie Nisbet wrote:
>> > > It picks a random sample of URLs. This was one of the
>main concerns when we
>> > …
[View More]> started talking about this feature. We're always one
>step ahead of Mr.
>> > > Spammy ;)
>> >
>> >I'm not so sure :(
>> >
>> >I've just received a spam which has several embedded URLs.
>Each URL is based
>> >on the recipient's name, in this case djn(a)highmoor.co.uk.
>So there's djn.org,
>> >djn.net, djn.com etc. At the the end there's the spam URL itself -
>>
>> A "random sample" is a bit of an over-simplified description of the
>> behavior, but is at least partly true.
>>
>> From looking at the source code itself, no true DoS is
>possible this way,
>> although it may be possible for a spammer to reduce their
>chances of being
>> checked.
>>
>> Disclaimer: I'm not a Perl programer, so my interpretation
>of the code's
>> behavior may be incorrect. However, clearly the code has
>limits to the
>> number of queries they will generate per email.
>>
>> Mail::SpamCopURI (the plugin for 2.63) has a hard coded
>limit the number of
>> URI's it will check. Once it hits that limit, it seems to
>ignore the rest
>> of the URIs in the email. I'm unsure if the build-order is
>sequential,
>> alphabetical, or random, but I suspect sequential. (This is
>as-of 0.18, I
>> haven't checked 0.19). You can check this yourself in SpamCopURI.pm.
>>
>> In SA 3.0-pre3, there is a limit defined by the config option
>> uridnsbl_max_domains. First, all the domains are extracted
>from the URIs of
>> the email and duplicates are removed. Then up to
>uridnsbl_max_domains are
>> randomly selected from the list. If there are fewer domains
>than the limit,
>> all of them should end up being selected. You can check this
>behavior in
>> lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm. The default for
>this limit is
>> 20, so your example email didn't approach that limit.
>>
>> So both do have limits to prevent DoS's by generating tons
>of URI's. SA
>> 3.0's URIDNSBL code appears to be more abuse resistant than
>> Mail::SpamCopURI for 2.63 is.
>>
>> If any spammer tries to start evading the URI RBLs by
>stuffing with tons of
>> garbage URIs it should be easy to detect them by doing a
>rule that counts
>> URIs in the eval tests. Even in this case, the existing code
>will prevent
>> that stuffing from resulting in an absurd number of queries.
>>
>So, for what people say, the DoS part is taken care of...
>
>What seems to be possible is an evasive attack by adding more and more
>non-relevant URIs to a message so as that the spammy ones have a
>larger chance of NOT being chosen randomly by the plugin...
>
>Maybe when I grow up and have more time and knowledge :-) I will take
>the 3.0 plugin and see if the 'randomness' can be somehow
>heuristically helped... that is, ignore invisible ones, give more
>chances to domains that appear more than once, etc...
>
>Regards.
>
I *think* I remember the devs saying 3.0 will ignore blank (invisible) URL
links. BUt that might have been a discussion on SARE or the SURBL list. The
threads start to blur together sometimes :)
--Chris
[View Less]
On Thu, 29 Jul 2004 17:27:00 -0400, Matt Kettler <mkettler(a)evi-inc.com> wrote:
> At 04:48 PM 7/29/2004, Dougie Nisbet wrote:
> > > It picks a random sample of URLs. This was one of the main concerns when we
> > > started talking about this feature. We're always one step ahead of Mr.
> > > Spammy ;)
> >
> >I'm not so sure :(
> >
> >I've just received a spam which has several embedded URLs. Each URL is based
> >on the recipient's …
[View More]name, in this case djn(a)highmoor.co.uk. So there's djn.org,
> >djn.net, djn.com etc. At the the end there's the spam URL itself -
>
> A "random sample" is a bit of an over-simplified description of the
> behavior, but is at least partly true.
>
> From looking at the source code itself, no true DoS is possible this way,
> although it may be possible for a spammer to reduce their chances of being
> checked.
>
> Disclaimer: I'm not a Perl programer, so my interpretation of the code's
> behavior may be incorrect. However, clearly the code has limits to the
> number of queries they will generate per email.
>
> Mail::SpamCopURI (the plugin for 2.63) has a hard coded limit the number of
> URI's it will check. Once it hits that limit, it seems to ignore the rest
> of the URIs in the email. I'm unsure if the build-order is sequential,
> alphabetical, or random, but I suspect sequential. (This is as-of 0.18, I
> haven't checked 0.19). You can check this yourself in SpamCopURI.pm.
>
> In SA 3.0-pre3, there is a limit defined by the config option
> uridnsbl_max_domains. First, all the domains are extracted from the URIs of
> the email and duplicates are removed. Then up to uridnsbl_max_domains are
> randomly selected from the list. If there are fewer domains than the limit,
> all of them should end up being selected. You can check this behavior in
> lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm. The default for this limit is
> 20, so your example email didn't approach that limit.
>
> So both do have limits to prevent DoS's by generating tons of URI's. SA
> 3.0's URIDNSBL code appears to be more abuse resistant than
> Mail::SpamCopURI for 2.63 is.
>
> If any spammer tries to start evading the URI RBLs by stuffing with tons of
> garbage URIs it should be easy to detect them by doing a rule that counts
> URIs in the eval tests. Even in this case, the existing code will prevent
> that stuffing from resulting in an absurd number of queries.
>
So, for what people say, the DoS part is taken care of...
What seems to be possible is an evasive attack by adding more and more
non-relevant URIs to a message so as that the spammy ones have a
larger chance of NOT being chosen randomly by the plugin...
Maybe when I grow up and have more time and knowledge :-) I will take
the 3.0 plugin and see if the 'randomness' can be somehow
heuristically helped... that is, ignore invisible ones, give more
chances to domains that appear more than once, etc...
Regards.
--
Mariano Absatz - El Baby
el (dot) baby (AT) gmail (dot) com
el (punto) baby (ARROBA:@) gmail (punto) com
[View Less]
>-----Original Message-----
>From: Mariano Absatz [mailto:el.baby@gmail.com]
>Sent: Thursday, July 29, 2004 9:41 AM
>To: SURBL discussion list; SpamAssassin users list
>Subject: SURBL DoS possible?
>
>
>I was wondering...
>
>I didn't look at the source code for the SpamCopURI or the SA 3.0
>plugin but I guess it just looks for URI's within the messages and
>issues a DNS query to the configured SURBLs for every different
>canonicalized domain name... is it?
…
[View More]>
>What would happen if a spammer intentionally starts putting hundreds
>of different invisible random URIs within the message trying to DoS
>SURBL?
>
>Does the SA plugins check for this condition? Or have a limit as to
>how many SURBL queries will it issue for a given message?
>
>TIA
>
It picks a random sample of URLs. This was one of the main concerns when we
started talking about this feature. We're always one step ahead of Mr.
Spammy ;)
--Chris
[View Less]
On Thu, 29 Jul 2004 09:24:31 -0500, Bob Apthorpe
<apthorpe+sa(a)cynistar.net> wrote:
> On Thu, 29 Jul 2004 10:40:44 -0300 Mariano Absatz <el.baby(a)gmail.com> wrote:
>
> > I was wondering...
> [...]
> > What would happen if a spammer intentionally starts putting hundreds
> > of different invisible random URIs within the message trying to DoS
> > SURBL?
>
> One can compensate for this by testing only a few, visible URIs, or
> skipping the …
[View More]RHSBL tests altogether and triggering the
> "MAIL_HAS_CRAPLOAD_OF_INVISIBLE_URIS" rule. Or something like that.
Right... but I don't want to get rid of SURBL... it is working very
nicely, it finds a lot of spam and I have yet to find a FP myself
(though others have seen them)...
My question is more to the people that developed the SURBL plugins for
SA (or those that have read and understood them), to know if there's
something in the plugins to avoid a DoS attempt of this kind.
Thanx for your reply, anyway.
--
Mariano Absatz - El Baby
el (dot) baby (AT) gmail (dot) com
el (punto) baby (ARROBA:@) gmail (punto) com
[View Less]
This is a forwarded message
From: Matt Kettler
To: spamassassin-users
Date: Thursday, July 29, 2004, 2:27:00 PM
Subject: {SPAM} Re: SURBL DoS possible?
===8<==============Original message text===============
At 04:48 PM 7/29/2004, Dougie Nisbet wrote:
> > It picks a random sample of URLs. This was one of the main concerns when we
> > started talking about this feature. We're always one step ahead of Mr.
> > Spammy ;)
>
>I'm not so sure :(
>
>I've just received …
[View More]a spam which has several embedded URLs. Each URL is based
>on the recipient's name, in this case djn(a)highmoor.co.uk. So there's djn.org,
>djn.net, djn.com etc. At the the end there's the spam URL itself -
A "random sample" is a bit of an over-simplified description of the
behavior, but is at least partly true.
From looking at the source code itself, no true DoS is possible this way,
although it may be possible for a spammer to reduce their chances of being
checked.
Disclaimer: I'm not a Perl programer, so my interpretation of the code's
behavior may be incorrect. However, clearly the code has limits to the
number of queries they will generate per email.
Mail::SpamCopURI (the plugin for 2.63) has a hard coded limit the number of
URI's it will check. Once it hits that limit, it seems to ignore the rest
of the URIs in the email. I'm unsure if the build-order is sequential,
alphabetical, or random, but I suspect sequential. (This is as-of 0.18, I
haven't checked 0.19). You can check this yourself in SpamCopURI.pm.
In SA 3.0-pre3, there is a limit defined by the config option
uridnsbl_max_domains. First, all the domains are extracted from the URIs of
the email and duplicates are removed. Then up to uridnsbl_max_domains are
randomly selected from the list. If there are fewer domains than the limit,
all of them should end up being selected. You can check this behavior in
lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm. The default for this limit is
20, so your example email didn't approach that limit.
So both do have limits to prevent DoS's by generating tons of URI's. SA
3.0's URIDNSBL code appears to be more abuse resistant than
Mail::SpamCopURI for 2.63 is.
If any spammer tries to start evading the URI RBLs by stuffing with tons of
garbage URIs it should be easy to detect them by doing a rule that counts
URIs in the eval tests. Even in this case, the existing code will prevent
that stuffing from resulting in an absurd number of queries.
===8<===========End of original message text===========
[View Less]