[SURBL-Discuss] RFC: Drop multi.surbl.org rbldnsd TTL?

Jeff Chan jeffc at surbl.org
Sun Jul 25 22:41:38 CEST 2004


On Friday, July 23, 2004, 1:58:26 PM, Mike Atkinson wrote:
> On 7/22/2004 at 6:05 PM, Jeff Chan wrote:

>> Should we decrease the TTL on the rbldnsd version of
>> multi.surbl.org?

>> version of multi are the default for the entire file which is
>> currently 28800 (8 hours).

>> Obviously that's quite a bit longer than the 90 minutes now used
>> on ws or the 10 minutes for sc, so some multi/rbldnsd records

>> I could probably read the RFCs or source code, but does
>> anyone know if DNS implementations cache negative hits to
>> the default TTL.  In other words if a new record gets added
>> to a list, do caching name servers (that negatively cached it
>> before, i.e., got queried and said "I don't have that") get
>> the new record immediately or only after some TTL expires.

> I think that BIND will cache negative answers for 3 hours from an
> authoritative name server without regard to any other TTL settings in
> the zone.  It is possible to leave out the SOA record in the RBLDNSD zone
> and then BIND will see the remote name server as non-authoritative and
> will not cache the negative response.  (Excerpts from man pages below.)

> This means that if, for instance, my site is one of the earlier ones to
> get hit by a spam with a new URL that ends up in the SURBL; we are not
> going to see it as listed for up to 3 hours.  I'm experimenting with
> 'max-ncache-ttl 3600;' in my named.conf and may drop that lower.  3600
> doesn't seem to be a problem for the last several hours.

> As far as the TTL for entries; I'm not overly concerned about lingering
> entries in multi.surbl.org (which is what we are using.)  I think that
> 90-180 minutes would be a good range for that.

> ======
>>From 'man named.conf'

> max-ncache-ttl

> To reduce network traffic and increase performance the server stores
> negative answers. max-ncache-ttl is used to set a maximum retention time
> for these answers in the server is seconds. The default max-ncache-ttl
> is 10800 seconds (3 hours).

> ---

>>From 'man rbldnsd'

> It is recommended, but not mandatory to specify SOA record for every
> zone. If no SOA is given, negative replies will not be cacheable by
> caching nameservers.

Thanks for the feedback Mike.  We definitely want to keep
negative caching active in both BIND and rbldnsd since most
queries will probably give a negative result (i.e. not match a
SURBL). I'm pretty sure we wouldn't want to lose the performance
advantage of negative caching, even if it meant new (positive)
matches were delayed, since negative answers are probably a
majority.  However we need to examine in a little more detail
how negative caching is actually handled when a record that
didn't match before now matches (i.e. is no longer negative).

Three possibilities are:

1.  New (positive) entries in a given zone immediately override
previous negative cache hits (previously negative for the same
query).

2.  The negative cache TTL takes precedence.

3.  The positive caching TTL take precedence.

To test this, I watched a new domain get into to sc.surbl.org
and saw it served up between 10 and 20 minutes after the zone
file original modification time (after manually rsyncing the
change to a rbldnsd server).  sc has TTL and Refresh times of 10
minutes so this seems about right.  So does that mean the
negative cache time for a new positive match is constrained by
the TTL default *zone file TTL* and not the default negative
caching TTL?  If that were the case we probably do want to lower
the TTL on multi and probably the other constituent files like ob
and ab, in order to reduce the delay in new entries becoming
usable. 

Does anyone have more info on this question?

By the way looking at some of the refresh and ttl times of
other RBLs:

list.dsbl.org
        origin = a.list.ns.dsbl.org
        mail addr = admin.dsbl.org
        serial = 1080300617
        refresh = 600 (10M)
        retry   = 300 (5M)
        expire  = 86400 (1D)
        minimum ttl = 600 (10M)

sbl.spamhaus.org
        origin = need.to.know.only
        mail addr = hostmaster.spamhaus.org
        serial = 2004032608
        refresh = 3600 (1H)
        retry   = 900 (15M)
        expire  = 604800 (1W)
        minimum ttl = 300 (5M)

dnsbl.njabl.org
        origin = ns1.njabl.org
        mail addr = help.njabl.org
        serial = 1080298387
        refresh = 10800 (3H)
        retry   = 1800 (30M)
        expire  = 720000 (1w1d8h)
        minimum ttl = 900 (15M)

spam.dnsrbl.net
        origin = ns1.namesystems.net
        mail addr = dns\@namesystems.net
        serial = 2004031600
        refresh = 3600 (1H)
        retry   = 900 (15M)
        expire  = 864000 (1w3d)
        minimum ttl = 3600 (1H)

relays.ordb.org
        origin = a.ns.ordb.org
        mail addr = hostmaster.ordb.org
        serial = 1080300600
        refresh = 600 (10M)
        retry   = 300 (5M)
        expire  = 604800 (1W)
        minimum ttl = 1800 (30M)

Perhaps our TTLs should be shorter for all of our zones.  sc is
very short at 10 minutes ttl and refresh.  Perhaps the others
should be also.

Comments?

Jeff C.



More information about the Discuss mailing list