Re: [SLUG] SMP-friendly scripting

From: Eben King (eben01@verizon.net)
Date: Thu May 24 2007 - 09:59:42 EDT


On Thu, 24 May 2007, Ian C. Blenke wrote:

> Eben King wrote:
>
>> I got a dual-core CPU (etc) a month or so ago. I need to rewrite some of
>> my scripts to take advantage of the second CPU. Is there a relatively easy
>> way of doing that? I've thought about:
>>
>> - "xargs -n 2"; what it'll do is run pairs at a time, and if one of the two
>> takes longer than the other, tough noogies; it won't run the next pair
>> until _both_ current processes exit. So it's better than serial
>> execution, but not optimal.
>
> xargs -P 2, will run 2 in parallel at the same time. If one ends, another is
> spawned in its place.

That's what I need. Thanks. And I'll make it "xargs -P $NUMCPUS" so it's
more general.

> There are many other job scheduling systems out there, particularly in the
> distributed/GRID computing space (Condor, etc etc). That is, if you're
> working on creating a complex batch scheduling platform.
>
> This question (and answer) seems to pop up 3 or 4 times a year anymore.

Check the FAQ for comp.unix.shell then?

-- 
-eben      QebWenE01R@vTerYizUonI.nOetP      royalty.mine.nu:81
A. A Top Poster                       \  http://www.fscked.co.uk/
B. Who's there?                        \  writing/
A. Knock-knock -- from bobward@xxx.com  \  top-posting-cuss.html
-----------------------------------------------------------------------
This list is provided as an unmoderated internet service by Networked
Knowledge Systems (NKS).  Views and opinions expressed in messages
posted are those of the author and do not necessarily reflect the
official policy or position of NKS or any of its employees.



This archive was generated by hypermail 2.1.3 : Fri Aug 01 2014 - 16:32:44 EDT