Re: [SLUG] High performance Linux clustering

From: Ian Blenke (icblenke@nks.net)
Date: Thu Feb 23 2006 - 13:01:12 EST


Levi Bard wrote:
> On 2/23/06, Richard Morgan <rmorgan@heavysystems.com> wrote:
>
>> Does anyone have any experience setting up a Linux cluster to be used for number
>> crunching, biomedical research, simulations, modeling?
>>
>
> Back in the day, I set up a cluster of 386s (gasp!) for
> performance-testing mosix(before it got forked into openmosix) vs pvm.
> For most tasks that we tested, it was more efficient to use mosix and
> spawn worker processes as needed than to pvmify the software and run
> it that way, even if you discount the overhead of pvmifying the
> software and setting up the pvm master/slave relationships.
>
> I have no idea if any of this is relevant ;-)
>

It all depends on the application and how you want to scale it.

The last time I used mosix it didn't support Distributed Shared Memory
and applications written with shared memory in mind wouldn't migrate.
This killed a simulation tool called "gamma" that the researchers wanted
to use. It was a few years ago though, and I believe they've added DSM
since then...

Today there are GRID management toolsets for managing large clusters of
computing resources. One of the better known ones is IBM's Globus
Toolkit, used by things like Oracle RAC.

If you're looking for a management infrastructure for Linux boxen that
is pre-canned, take a look at Linux Rocks clusters:

    http://rocks.npaci.edu

Another cluster management infrastructure that appeared at LISA was
Warewulf:

    http://www.warewulf-cluster.org

O'Reilly has a good book on this stuff "High Performance Linux Clustering":

    http://www.oreilly.com/catalog/highperlinuxc/

I've built a number of Linux clusters over the years. Everything from
MOSIX to PVM and MPI to RedHat Cluster Mananger and Xen with CLVM.
Lately I've been playing a lot with GRID computing and the Globus
Toolkit in particular.

Each cluster solved a specific problem. Sadly, there is no catch-all.

-- 
- Ian C. Blenke <icblenke@nks.net>


----------------------------------------------------------------------- This list is provided as an unmoderated internet service by Networked Knowledge Systems (NKS). Views and opinions expressed in messages posted are those of the author and do not necessarily reflect the official policy or position of NKS or any of its employees.



This archive was generated by hypermail 2.1.3 : Fri Aug 01 2014 - 18:46:17 EDT