BTW if you're trying to manage jobs, something like beanstalkd might be more efficient than managing your own load, plus it's network aware. There's a Ruby gem called Stalker that makes it really easy to run a worker daemon that runs the commands you want (bindings in just about every language out there, too) Then it's just a matter of running the number of worker instances, say one per CPU.

Sean

On Thu, Jan 12, 2012 at 3:28 PM, Trevor Cordes <trevor@tecnopolis.ca> wrote:

(A real question this time!)

I have a web (+ other tasks) server that sometimes does a lot of tasks in
a short time that spike the load up to large values (like 30). The tasks
are all long jobs of lots (tens of thousands) of little (10/s) jobs. I've
written code to, between each little job, check the /proc/loadavg load (1
and 5 min) and insert usleeps (.5+ secs) based on the loadavg (exponential
relationship). My goal is to keep the load < 2.0 on a 2 core system.

Anyhow, it seems to be working fine. The weird thing is, at 2.0 loadavg,
looking at top, I see the CPUs mostly at idle. Not even very much wait
going on. Why would the load be 2 when both CPUs are mostly at 5-10%
including wait?

I basically want to make sure the system doesn't get overloaded and can
respond to non-batch request (like a web hit) promptly. What loadavg
should I target if not (1.0 * numcores)? I guess I can arbitrarily pick
numbers and see what happens, but I like things like this to make sense,
and to me 1.0 * numcores makes sense, unless I'm not understanding loadavg
properly.

Thanks!
_______________________________________________
Roundtable mailing list
Roundtable@muug.mb.ca
http://www.muug.mb.ca/mailman/listinfo/roundtable

--
Sean Walberg <sean@ertw.com> http://ertw.com/