Is it possible to cluster multiple Xservers in to one logical machine?

Question

I have (3) 2009 2.26 Xserves and I am wondering if it is possible to tie them together so it acts as one computer?

Answer 1

Yes and no.

It's certainly possible to create clusters out of disparate machines and use them in some co-ordinated fashion. But it's the realm of specialty software and specialty application programming.

Beowulf can take separate machines and have them act as one, large SMP machine with shared memory and all but programs need to be Beowulf compatible for this to work. There are lots of specifics to making Beowulf work well: you should use nearly identical machines and your machine interconnect fabric has to be really high speed and low latency (read: better than gigabit ethernet...like InfiniBand)

XGrid, an Apple-designed, job scheduling framework based on the BEEP framework is available for use and sits somewhere between Beowulf above and Condor/GE below. It's a process distribution framework along with a message passing interface for distributed process co-ordination. You have to run an XGrid head node and XGrid slaves in your pool and your code needs to be XGrid-aware and complied against the XGrid libraries for it to take advantage of an XGrid. From that article linked to above:

Unfortunately, Xgrid is not yet the silver bullet that can magically take an existing program such as iMovie, cut it in slices and run in on all the macs it can find in the neighborhood.

The above statement is true for every technology I'm talking about in this answer I'm afraid.

Condor (and Oracle Grid Engine) can provide somewhat easy access to pools of machines for batch-based workloads -- you queue up jobs and these programs will take care of scheduling their execution on machines that you have joined to the pool as execution nodes -- but they don't share CPU power across machines (at least not without MPI interfaces in the software using them) and they don't share memory. The interconnect requirements aren't nearly as heavy as in Beowulf clustering and you can get away with heterogenous pools of machines as the schedulers will handle matchmaking based on job requirements.

There are some off-the-shelf Mac apps that will let you do clustering in an application-specific way. For example: Logic Pro 8 and Logic 9 let you run "Logic Nodes" in your network and you can offload audio processing to these nodes. How much and what specifically you can offload is very dependent on your network and your machines but it does work. There are options in the Logic installer to create a 'Logic Node' when you install the software and run Logic in slave mode on the instance so it can be found by a master Logic session running on another machine.

This sort of one-off speciality parallelization is pretty unique in consumer and pro-sumer level applications and far from the norm though.

It's worth noting that Apple used to sell something called the Xserve Cluster Node. The name is somewhat misleading here. It was a hardware configuration for the Xserve, without a graphics card or an optical drive, intended to be used in cluster deployments. But the configuration didn't include any additional clustering capabilities. You still needed something like above mentioned software to create a cluster for your workloads.