I'm in the process of building a BeoWulf cluster, it has been a hard
task until recently due to the lack of any decent get it built type FAQ's
Luckily a few sparse FAQ's have started to appear, but they still seem
to lack some serious setup info and tend to gloss over a lot of things,
granted a BeoWulf cluster isn't something anyone can just setup, but
its in within reach to a lot of people.
I'm going to try and do a step by step build and log of my attempts to
build one using extreme linux and AMD K6-2 single cpu machines,
I hope to persuade AMD to let me have some K7's to add to it.
update: I have 12 nodes running now, however i`m finding that windows NT
with an rsh daemon and MPI work well, so I might just continue on with using
that instead.Its certainlly a damn sight easier to setup 8-)
I`ve successfully managed to get win2000/98 and NT to
work together using PVM and rsh. Its fairly easy to setup theres some bugs in
the win32 installer for PVM. I found its best to install it to say C:\PVM then follow
this for which platform you are using
Windows NT 4 / Windows 2000
Open the system environment settings control, click on system variables add the following
PVM_ROOT = C:\PVM
PVM_TMP = C:\TMP
PVM_ARCH = WIN32
Of course set the first one to where you installed PVM. The second i created a TMP
directory just for PVM , the reason well if you crash PVM it doesn`t clean up properly,
this is more to do with windows than anything, so in order to get it to work again you have
to clear out the %PVM_TMP% directory.
Windows 95 / 98
Add these to your autoexec.bat
set PVM_ROOT = C:\PVM
set PVM_TMP = C:\TMP
set PVM_ARCH = WIN32
Presumably you`ve installed a rsh daemon already, theres one available on the PVM WIN32
page. Its a little tricky to install. Make sure you do a configure after you install.
say you`ve installed RSH to C:\rsh
configure -add charlie charlie 192.0.0.*
# then give it the password you log into windows with, (twice) don`t leave it blank it
# the IP address ought to be changed to whatever yours is, and the usernames. i`m assuming you are
using the same username on all systems, you can map them around, seen the rsh docs.
Start the rsh daemom.
type rsh <machine_name> <command>
Something ought to happen, whatever command you used, I used
rsh frog c:\rsh\configure.exe -list
This ought to display the list of users you installed., if of course you used C:\RSH as the
Theres another bug in PVM win32, its batch file for running the
daemon is incorrect,
the correct way to start it AFAIK is to go to %PVM_ROOT%\bin\ and type
this ought to start the daemon up, it takes a second or two.
Now say we have two machines pink and green. You`ve just started PVM on green
type at the (pvm>) prompt
In a few moments, you ought to get a confirmation saying its been
added. you can now spawn
executables on pink and green.
Now i`ve noticed after every spawn i have to re add the machine again, I don`t know
is the desired behaviour. I wish it wasn`t, I don`t know if making a hosts list file solves this or
You ought to be able to run PVM apps now. If not check the rsh client, try rsh pink C:\rsh\configure.exe -list
on each machine you installed RSH on. if that works, check the above environment variables.
To be honest rsh on windows isn`t great, it crashes a fair amount, I couldn`t get
Microsofts rsh daemon supplied
with the NT 4,0 resource kit to work at all, but i belive this was my fault in setting up the config .rhosts also when
you do the rshsetup.exe make sure the paths are right, the first time i added an extra slash, and didn`t see it, well
it doesn`t let you redo it, and I couldn`t find the entry in the registry. Also *DON`T* attempt to use the NT4 rsh
daemon on Win2000 it won`t work, it just crashes. I haven`t looked up the win2k resource kit to see if theres one
on there yet.
It all works reasonably well for being the hack that it is, you`ll get the odd crash
that requires at minimum a shutdown
of the PVM software, usually a logout, or on occasion a reboot. Windows wasn`t really designed for this kind of work.
Using Extreme Linux
As with most linux installs its a nightmare to
setup, you need to make the boot floppies from the images
directory, then go through and setup as normal redhat. Note Redhat has stopped making Extreme Linux, I
don`t know why, perhaps its too small a marketplace for the amount of technical support it requires.
Getting PVM and RSH for WIN32
Markus Fischer he takes care of the WIN32 version of PVM and has an rsh daemon for WIN32
I just chatted to markus and he`s trying to get funding to work on this stuff, so if you can help, email him.
Beowulf Installation and Administration HOWTO (DRAFT)
Cluster QuickStart (DRAFT)
Building a BeoWulf System
AMD K6-3D Now 450mhz/400mhz
1 Dual Pentium Pro/200mhz
4gig Western digital IDE HD
3COM 16 Port LinkSwitch Switch.
3COM (3c905) Check if your card is supportedf
This can be a very expensive piece of hardware, usually
costing more than 1-6 nodes ( $340-$9,000). I suppose it depends on
your application, buy.com had a 16 port complex autosensing switch for
$349 upto $6,500 for a fore systems.
PureData 24 port switch
Acculan 16 port switch