HPC Running and Converting Models

Introduction

This page contains information about converting an existing TUFLOW Classic or GPU (pre 2017 HPC release) model to a format that can be run using the TUFLOW HPC engine. This page provides a quick summary for experienced TUFLOW users to use as a reference point for updating their models. It is recommended that less experienced TUFLOW users refer to our TUFLOW Tutorial Modules for greater support and guidance on creating a HPC model.

To convert an existing TUFLOW Classic or GPU Model to run on HPC, an update to the TUFLOW Control File (TCF) is needed. Some features from TUFLOW Classic that are not currently supported in HPC, may prevent the HPC model running successfully. To find out more about unsupported features in HPC be sure to review the TUFLOW 2017-09 Release Notes or the HPC Features Wiki Page.

Converting TUFLOW Classic to HPC (TCF Updates)

To run an existing TUFLOW Classic simulation with the new HPC engine, the following lines of text need to be added to the TUFLOW Control File (TCF).
Solution Scheme == HPC !This command specifies that you want to run TUFLOW using the HPC solution scheme or engine.
The following command is also required to run the model using GPU hardware:
Hardware == GPU !CPU is default. The hardware command instructs TUFLOW HPC to run using GPU hardware. This is typically orders of magnitude faster than on CPU.
These two commands are all that's needed to run convert the TUFLOW Classic model to HPC and run using GPU hardware. There are however more commands provided below that give the modeller greater control over the Hardware that HPC uses.

Running HPC on Multiple CPU Cores

As mentioned in the HPC Introduction page, HPC can be parallelised to run across multiple CPU processors when run on CPU (i.e. not GPU). The following command allows the modeller to dictate the number of core processors to run TUFLOW HPC across.
CPU Threads == 8 !Default is 4. This instructs TUFLOW to search for and run the model across four different core processors.
If the number of processors or TUFLOW licences found by TUFLOW are less than the specified value, then TUFLOW will utilise the maximum number of core processors available within the licence and hardware limitations.
Alternatively, the number of CPU threads can be specified in the batch file / command line by using the -nt<number of threads> argument. If both control file and command line are used to specify number of threads, the command line option will prevail.

Running HPC on Multiple GPU Devices

Much like HPC can be run across multiple CPU processors, HPC can be run across multiple GPU cards. The following TCF command can be used to set the number of GPU devices and which devices to use.
GPU Device IDs == 0, 1 !This command instructs TUFLOW to run the model on GPU Device 0 and GPU Device 1.
If a machine only has a single GPU card, the GPU Device ID should be 0, this is a default. If a second GPU card was added, the Device ID would be 1 and so on. The GPU Device ID can be checked by reviewing the machines Device Manager.
Alternatively, the GPU devices can be specified in the batch file / command line by using the -pu<id> argument. Note that multiple arguments are required to run on multiple GPU cards, e.g. -pu0 -pu1 and so on.
If both control file and command line are used to specify devices, the command line option will prevail.

Converting TUFLOW GPU to HPC (TCF Updates)

When converting a TUFLOW GPU model across to HPC, first confirm that all features in the GPU model are available in HPC by referring to the TUFLOW 2017-09 Release Notes.
Delete the following command from the *.tcf file and insert the commands specified above, for converting a TUFLOW Classic model to HPC.
GPU Solver == ON

HPC Scenarios and Updating the Batch File

Modellers may want to change the hardware that HPC is run on throughout the course of a project. For example, if your company own more CPU than GPU licences it may be beneficial to run the model using CPU hardware during the initial model build phase so your colleagues have access to the higher speed GPU licences for production runs on other projects that are running in parallel. If this is the case, it may be easier to setup a Scenario Logic statement in the TUFLOW Control File (TCF), that allows the modeller to change the hardware being used with a simple switch in the batch file used to run the model.

To setup a scenario for varying hardware options, the following commands can be used in the TCF file: Solution Scheme == HPC
Hardware == <<~s1~>> !The scenario will either be "CPU" or "GPU"

This basic scenario logic can be configured further within the TCF as shown below:

If Scenario == CPU

CPU Threads == 8

Else If Scenario == GPU

GPU Device IDs == 0, 1

End If

If using the above Scenario Logic, the modeller must include a scenario call in the batch file.
-s1 <Hardware Type>
If you are unfamiliar with using Scenario Logic, please refer to Tutorial Module 05.

Other useful batch file switches include:
-nt <number_of_threads> : This switch is used to set the number of CPU threads used for CPU mode simulations.
-pu <GPU Device IDs> : This switch is used to select the number of GPU cards that TUFLOW uses.

An example of how this would be implemented into a simple batch file for CPU and GPU are shown below.

CPU
TUFLOW_iSP_w64.exe -s1 CPU -nt8 FMA_T2_~s1~_001.tcf
This example will run TUFLOW HPC on CPU using 8 CPU threads.

GPU
TUFLOW_iSP_w64.exe -s1 GPU -pu0 -pu1 FMA_T2_~s1~_001.tcf
This example will run TUFLOW HPC on GPU using 2 GPU cards.