Problem with these instructions? Please send a CFDDFC Support Query.

A Cluster of Instances

Once an instance is running, the user can launch further instances with the cluster subcommand.  A networked cluster is created with the original instance as the master node and additional instances as the slave nodes.  The user accesses the cluster via the master instance, via SSH login or remote desktop.  Data is written to the elastic block storage (EBS) of the master instance.  The storage is accessible to the slave instances across a virtual private network using a network file system (NFS).

Creating a Cluster of Instances

When creating a cluster, the user should first launch a master instance.  We recommend using the C5n series of instances which provide significantly higher network performance across all instance sizes.  Since the purpose of clustering is to access more cores than is available on a single instance, it makes sense to launch the largest instance in the series, so c5n.18xlarge is the natural choice.

The user needs to take care to provision sufficient storage on the master instance, since it stores all the data for the cluster.  The required storage depends on the type of simulation being performed and the configuration of data processing in OpenFOAM.  1 GB per core provides a reasonable starting point for determining the volume size.  EBS pricing is approximately $0.10 per GB-month (think: $1 per day for 300 GB).  Therefore, to launch the master instance with 300 GB of storage, the user would run the launch command with the following options:

cfddfc launch -instance c5n.18xlarge -volume 300

Once the master instance is running, a cluster of instances can be created using the cluster subcommand, specifying the number of slave instances.  The number of slave instances is specified by the -slaves option.  If the user wanted to created a cluster of 3 instances, i.e. of 3 × 36 = 108 cores, the user would add 2 slave instances to the master instance by the following command:

cfddfc cluster -slaves 2

In the above command the instance ID of the master instance can be specified , e.g. aB1cD2eF, as an argument with the -key option:

cfddfc cluster -slaves 2 -key aB1cD2eF

Running in Parallel across the Cluster

The cluster subcommand creates a file named machines containing the local IP addresses of the master and slave instances on the private network within AWS EC2.  The file is copied automatically into the $HOME/OpenFOAM directory.  The user can then run an OpenFOAM application, e.g. simpleFoam across all nodes on the cluster by launching the mpirun command with the machines file as an argument to the -hostfile option. 

For example, the following command runs simpleFoam over 108 cores on the 3 instances whose IP addresses are specified in the machines file, redirecting output to a file named log.simpleFoam:

mpirun -np 108 -hostfile $HOME/OpenFOAM/machines simpleFoam -parallel > log.simpleFoam &

Next Step → Terminate an Instance