Good day, all, I'm running into a problem with the sa-blacklist content I host at http://www.stearns.org/sa-blacklist/ . The box (*) is hosted at pa.net, a colo facility once employing a good friend. A new guy was going over the bandwidth stats and noticed that that machine hogs the available bandwidth on its ethernet segment for a few minutes after each hour as people download the sa-blacklist. I asked him what kind of bandwidth he'd ideally like that system to use, and how much I'm using at the moment. I should be using around 10G/month. I'm currently using 1TB/month. Oops. I'm not in imminent danger of being kicked off their cable, but both they and I agree that I need to do something differently. I could put another physical box at another ISP with unlimited bandwidth, but I'm already paying around $1500/year to host the site, and am reluctant to double that. Because that box hosts 27 virtual machines, moving it is a project that would need a few months of lead time to arrange, and would be a nightmare in itself.
It would be great if someone already has enough bandwidth to host the content on a different cable, but I think people with a terabyte/month to spare may be rare. *smile* If you've got some bandwidth you could share, would you consider doing round-robin with me with the content? 10 sites spreading the load would have 100GB/month, or an average of about 300 kilobits/sec. 20 sites sould be half that each, and so on. I'd need to upload content via rsync over ssh. The actual content is published via web, rsync, and ftp, although I could easily set up www.sa-blacklist.stearns.org for the sites willing to share over http, rsync.sa-blacklist.stearns.org, for the sites willing to share over rsync, and ftp.sa-blacklist.stearns.org.
If you can spare some bandwidth, please respond. Let me know what you can spare in average kilobits/sec. That way, if only 10 people respond and one of them can provide 100 kilobits/sec, I'll know not to include that person in the mirror until I can get 30 people.
If you can take part, I'd be forever grateful. *sincere smile* Cheers, - Bill
* http://www.stearns.org/slartibartfast/uml-coop.current.html
--------------------------------------------------------------------------- (Referring to the 32 bit system that feeds out files for kernel.org) "We learned that the Linux load average rolls over at 1024. And we actually found this out empirically." -- Peter Anvin -------------------------------------------------------------------------- William Stearns (wstearns@pobox.com). Mason, Buildkernel, freedups, p0f, rsync-backup, ssh-keyinstall, dns-check, more at: http://www.stearns.org --------------------------------------------------------------------------
On Thursday, September 15, 2005, 2:28:54 PM, William Stearns wrote:
Good day, all, I'm running into a problem with the sa-blacklist content I host at http://www.stearns.org/sa-blacklist/ . The box (*) is hosted at pa.net, a colo facility once employing a good friend. A new guy was going over the bandwidth stats and noticed that that machine hogs the available bandwidth on its ethernet segment for a few minutes after each hour as people download the sa-blacklist.
Hi Bill, A couple suggestions:
1. Try to get people to grab the file at different times, not all at the same time of the hour.
2. Get people to use ws.surbl.org instead. It's mostly the same data, but more distributed and smaller in memory and file.
Jeff C. -- Don't harm innocent bystanders.