asadharis

Setting up a SGE cluster on a single Amazon EC2 machine

The gist here provides a script to automate the process of installing Sun Grid Engine (SGE) on a single EC2 machine.

Motivation

SGE is a job scheduler for a computing cluster. This usually involves a cluster of multiple machines. However for many applications we don't need a massive computing cluster and a cluster of 8-30 nodes would be sufficient. In this tutorial we set-up SGE on a single amazon EC2 machine. The reasons for doing so are as follows:

Automation: Setting-up a cluster with SGE is fairly involved as it requires multiple machines communicating with each other and having some shared memory. A single machine with multiple cores is already a simple cluster where the memory is shared across cores.
Moderate Size: Amazon EC2 instances provide a variety of computing options with the number of cores ranging from 1 to 128.
Cost: The On-Demand price structure of AWS makes this a relatively cheap option. Further cost redu


	# A function to create the default colors of ggplot.
	#
	# n: Number of different colors to generate. E.g. if you have
	# 4 different colors then ggplot uses ggplot_cols(4).
	ggplot_cols <- function(n) {
	hues <- seq(15, 375, length = n + 1)
	hcl(h = hues, l = 65, c = 100)[1:n]
	}

	#!/bin/bash

	# This is a simple bash script which sets-up the things we need for running.

	# We begin with generating a key pair which will be used to communicate between
	# master and slove nodes.
	ssh-keygen -t dsa -f .ssh/id_dsa -N ''

	# Add generated key to list of authorized keys
	cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

	*.aux
	*.glo
	*.idx
	*.log
	*.toc
	*.ist
	*.acn
	*.acr
	*.alg
	*.bbl