[RndTbl] slowing httpd access to cgi-bin scripts
Sean Walberg
sean at ertw.com
Fri Sep 17 16:26:42 CDT 2010
There used to be a third party module, something like mod_limit or
mod_bwlimit, that lets you limit connection rates.
That said, the developer in me says "cache". :)
Sean
On Fri, Sep 17, 2010 at 4:08 PM, Gilles Detillieux <
grdetil at scrc.umanitoba.ca> wrote:
> Every once in a while, some doofus points a web crawler at our web site
> and, ignoring the disallowed areas in our robots.txt file, starts
> crawling through some of our cgi-bin scripts at a rate of 4 to 8 hits a
> second. This is particularly annoying with some of the more processor
> and disk intensive CGI programs, such as man2html, which also happens to
> generate lots of links back to itself.
>
> Is there anything I can set up in Apache to throttle back and slow down
> remote hosts when they start hitting hard on cgi-bin? I don't want to
> do anything that would adversely affect legitimate users, nor make
> important things like the manual pages hard to find by removing any
> public links to them. But when a client starts making 10 or more GET
> requests on /cgi-bin in a 5 second period, it would be nice if I could
> get the server to progressively add longer and longer delays before
> servicing these requests, to keep the load down and prevent the server
> from thrashing.
>
> I'd appreciate any tips.
>
> Thanks,
> Gilles
>
> --
> Gilles R. Detillieux E-mail: <grdetil at scrc.umanitoba.ca>
> Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/
> Dept. Physiology, U. of Manitoba Winnipeg, MB R3E 0J9 (Canada)
> _______________________________________________
> Roundtable mailing list
> Roundtable at muug.mb.ca
> http://www.muug.mb.ca/mailman/listinfo/roundtable
>
--
Sean Walberg <sean at ertw.com> http://ertw.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.muug.mb.ca/pipermail/roundtable/attachments/20100917/85241a57/attachment.html
More information about the Roundtable
mailing list