BeoWulf Cluster Machines


I'm in the process of building a BeoWulf cluster, it has been a hard
task until recently due to the lack of any decent get it built type FAQ's

Luckily a few sparse FAQ's have started to appear, but they still seem
to lack some serious setup info and tend to gloss over a lot of things,
granted a BeoWulf cluster isn't something anyone can just setup, but
its in within reach to a lot of people.

I'm going to try and do a step by step build and log of my attempts to
build one using extreme linux and AMD K6-2 single cpu machines,
I hope to persuade AMD to let me have some K7's to add to it.

update: I have 12 nodes running now, however i`m finding that windows NT
with an rsh daemon and MPI work well, so I might just continue on with using
that instead.Its certainlly a damn sight easier to setup 8-)


Using PVM and Windows 2000/98/NT/95


    I`ve successfully managed to get win2000/98 and NT  to
work together using PVM and rsh. Its fairly easy to setup theres some bugs in
the win32 installer for PVM. I found its best to install it to say C:\PVM then follow
this for which platform you are using

Windows NT 4 / Windows 2000

Open the system environment settings control, click on system variables add the following


Of course set the first one to where you installed PVM. The second i created a TMP
directory just for PVM , the reason well if you crash PVM it doesn`t clean up properly,
this is more to do with windows than anything, so in order to get it to work again you have
to clear out the %PVM_TMP% directory.

Windows 95 / 98

Add these to your autoexec.bat


set PVM_TMP   = C:\TMP
set PVM_ARCH = WIN32


Presumably you`ve installed a rsh daemon already, theres one available on the PVM WIN32
page. Its a little tricky to install. Make sure you do a configure after you install.

say you`ve installed RSH to C:\rsh

configure -add charlie charlie 192.0.0.*

# then give it the password you log into windows with, (twice) don`t leave it blank it won`t work.
# the IP address ought to be changed to whatever yours is, and the usernames. i`m assuming you are
using the same username on all systems, you can map them around, seen the rsh docs.

Start the rsh daemom.

type rsh <machine_name> <command>

Something ought to happen, whatever command you used, I used

rsh frog c:\rsh\configure.exe -list

This ought to display the list of users you installed., if of course you used  C:\RSH as the
install path.


Running PVM


    Theres another bug in PVM win32, its batch file for running the daemon is incorrect,
the correct way to start it AFAIK is to go to %PVM_ROOT%\bin\ and type


this ought to start the daemon up, it takes a second or two.

Now say we have two machines pink and green. You`ve just started PVM on green

type at the (pvm>) prompt

add pink

    In a few moments, you ought to get a confirmation saying its been added. you can now spawn
executables on pink and green.

Now i`ve noticed after every spawn i have to re add the machine again, I don`t know if this
is the desired behaviour. I wish it wasn`t, I don`t  know if making a hosts list file solves this or

You ought to be able to run PVM apps now. If not check the rsh client, try rsh pink C:\rsh\configure.exe -list
on each machine you installed RSH on. if that works, check the above environment variables.


To be honest rsh on windows isn`t great, it crashes a fair amount, I couldn`t get Microsofts rsh daemon supplied
with the NT 4,0 resource kit to work at all, but i belive this was my fault in setting up the config .rhosts also when
you do the rshsetup.exe make sure the paths are right, the first time i added an extra slash, and didn`t see it, well
it doesn`t let you redo it, and I couldn`t find the entry in the registry. Also *DON`T* attempt to use the NT4 rsh
daemon on Win2000 it won`t work, it just crashes. I haven`t looked up the win2k resource kit to see if theres one
on there yet.

It all works reasonably well for being the hack that it is, you`ll get the odd crash that requires at minimum a shutdown
of the PVM software, usually a logout, or on occasion a reboot. Windows wasn`t really designed for this kind of work.


Using Extreme Linux

    As with most linux installs its a nightmare to setup, you need to make the boot floppies from the images
directory, then go through and setup as normal redhat. Note Redhat has stopped making Extreme Linux, I
don`t know why, perhaps its too small a marketplace for the amount of technical support it requires.


Getting PVM and RSH for WIN32

Markus Fischer he takes care of the WIN32 version of PVM and has an rsh daemon for WIN32

I just chatted to markus and he`s trying to get funding to work on this stuff, so if you can help, email him.


FAQ's already available

Beowulf Installation and Administration HOWTO (DRAFT)

Cluster QuickStart (DRAFT)

Building a BeoWulf System

Another FAQ


Hardware I Use


AMD K6-3D Now 450mhz/400mhz
1 Dual Pentium Pro/200mhz
Epox MVP-51-E
4gig Western digital IDE HD
3COM 16 Port LinkSwitch Switch.


3COM (3c905) Check if your card is supportedf

Ethernet Switch

    This can be a very expensive piece of hardware, usually
costing more than 1-6 nodes ( $340-$9,000). I suppose it depends on
your application, had a 16 port complex autosensing switch for
$349 upto $6,500 for a fore systems.

PureData 24 port switch

Acculan 16 port switch