CFDDFC on AWS: Data Storage

Data Storage for CFDDFC

A CFD simulation begins with configuration files describing geometry, models, control parameters, etc. The configuration typically uses only a small amount of data which can be transferred from the user’s local machine to a remote instance quickly and cheaply. While running, the simulation generates data for: the computational mesh; “fields” (e.g. velocity, pressure, temperature) at several steps in the solution; processed data for graphs, visualisation, monitoring, etc.; log data. Typical storage requirements often exceed 10 GB and can extend to multiples of 100 GB.

Reducing the Cost of Storage

People tend to store too much data. They do not give storage a thought until the disk they use is close to being full. It is particularly true for scientific computing on a dedicated “under the desk” PC, remote cluster or supercomputer. CFD computing in the cloud, however, is transitory: a user launches an instance and runs the CFD simulation. The user then terminates the instance once they have copied the data they need.

Managing that data is important to reduce the cost of cloud data storage and transfer. Users can avoid storing large volumes of data (that is invariably never accessed) by automating simulations which can regenerate data when required. They can configure simulations to output graphs, images and key values they really need and save that data, rather than the mesh and fields. The processed data is typically a fraction of the size of the field and mesh data, so can be transferred off the instance at reasonable cost before the instance is terminated.

Type and Size of Storage

Amazon EC2 instances use Amazon Elastic Block Store (EBS) to provide persistent storage of data.  The storage is configured while the user launches an instance with the Console at Step 4. Add Storage — which can be accessed from the Review Instance Launch window by clicking Edit Storage.  Once open and configured, Elastic Block Store information can be viewed in the user’s AWS console under Volumes.

When the storage volume is configured, the user selects the size and type of storage. Types of storage are described in the EBS product details, with the General Purpose (SSD) volumes sufficient for CFD applications. The required volume size depends on the size of CFD simulation, in particular the mesh size. A quick estimate for storage size, which users might find helpful, is:

• a minimum of 20GB;
• for larger parallel simulations, 5GB × number of physical cores.

Amazon Free Tier users qualify for up to 30 GB of EBS General Purpose (SSD) storage.

Next Step → Creating a Cluster