[RndTbl] load throttling
Gilbert E. Detillieux
gedetil at cs.umanitoba.ca
Thu Jan 12 15:37:18 CST 2012
On 2012-01-12 15:28, Trevor Cordes wrote:
> (A real question this time!)
>
> I have a web (+ other tasks) server that sometimes does a lot of tasks in
> a short time that spike the load up to large values (like 30). The tasks
> are all long jobs of lots (tens of thousands) of little (10/s) jobs. I've
> written code to, between each little job, check the /proc/loadavg load (1
> and 5 min) and insert usleeps (.5+ secs) based on the loadavg (exponential
> relationship). My goal is to keep the load< 2.0 on a 2 core system.
>
> Anyhow, it seems to be working fine. The weird thing is, at 2.0 loadavg,
> looking at top, I see the CPUs mostly at idle. Not even very much wait
> going on. Why would the load be 2 when both CPUs are mostly at 5-10%
> including wait?
My guess is that your overall processing is disk-bound. Load average
includes not only processes in in the run queue, but those waiting for
disk I/O (normally, fairly short waits). The MUUG server sees similar
loads when a lot of people are hammering at the mirror file system (over
http, ftp and/or rsync), with relatively low CPU percentages.
> I basically want to make sure the system doesn't get overloaded and can
> respond to non-batch request (like a web hit) promptly. What loadavg
> should I target if not (1.0 * numcores)? I guess I can arbitrarily pick
> numbers and see what happens, but I like things like this to make sense,
> and to me 1.0 * numcores makes sense, unless I'm not understanding loadavg
> properly.
Yeah, I think you'll have to experiment, since it depends on how much of
your load is disk vs CPU. I would certainly make sure to set the target
threshold higher than 2, and probably lower than 10. Maybe start with
4, and if your CPU usage is still low, bring it up toward 8.
YMMV!
Gilbert
--
Gilbert E. Detillieux E-mail: <gedetil at muug.mb.ca>
Manitoba UNIX User Group Web: http://www.muug.mb.ca/
PO Box 130 St-Boniface Phone: (204)474-8161
Winnipeg MB CANADA R2H 3B4 Fax: (204)474-7609
More information about the Roundtable
mailing list