View source for Configure CUDA device selection

Computers running TUFLOW may have multiple GPUs. These may be multiple NVIDIA GPUs with CUDA capabilities, used to accelerate simulation runs. Alternatively, they can be GPUs used for purposes such as rendering the interactive desktop or handling other computational tasks. A common occurrence on modern motherboards is the availability of an integrated GPU.

Generally, it is recommended to use a GPU that is not used for TUFLOW modelling as the primary GPU for rendering the desktop, if needed. If there is no additional GPU available, one of the NVIDIA GPUs can be used, in which case it is recommended to use the most capable card for running your models and the less capable one for rendering the desktop.

TUFLOW allows selection of a specific GPU for computation using command-line options such as -pu0 for the first GPU, -pu1 for the second, and so on. (See [[HPC Running and Converting Models]].)  

However, what TUFLOW considers the first or second GPU may not match the order shown in tools such as Windows Device Manager, Task Manager, or the output of <code>nvidia-smi</code> on the command line. Another common problem is that the needed GPUs are not actually in the expected order and may cause difficulty selecting GPUs in the preferred order.

To this end, an environment variable called <code>CUDA_VISIBLE_DEVICES</code> limits the devices that will be visible to CUDA-capable applications like TUFLOW, as well as specifying the order they will appear in. The remainder of this article outlines how to configure that setting. As an example, a Windows computer is used that has 2 NVIDIA GPUs, and an on-board AMD GPU. In Windows, all available GPUs can be listed using a PowerShell command like this:
<syntaxhighlight lang="powershell">
Get-CimInstance -Namespace root\cimv2 -ClassName Win32_VideoController | Select-Object DeviceID, Name
</syntaxhighlight>
(PowerShell commands can be run by opening PowerShell from the Windows Start Menu and pasting a command there)

The output for the example computer is as follows (note that virtual adapters, such as a Remote Desktop adapter, will also appear):
<pre>
DeviceID         Name
--------         ----
VideoController1 AMD Radeon(TM) Graphics
VideoController2 Microsoft Remote Display Adapter
VideoController3 NVIDIA GeForce RTX 4090
VideoController4 NVIDIA GeForce RTX 4090
</pre>
In this case, only 'VideoController3' and 'VideoController4' need to be visible to CUDA-enabled applications like TUFLOW. More details on those can be obtained by running the following command (from either PowerShell, Command Prompt, or a Linux shell):
<syntaxhighlight lang="batch">
nvidia-smi --query-gpu=name,uuid --format=csv,noheader,nounits
</syntaxhighlight>
And the output is as follows:
<pre>
NVIDIA GeForce RTX 4090, GPU-5060f556-4eb4-7155-4020-abadcb2fd735
NVIDIA GeForce RTX 4090, GPU-f3825978-37f8-b933-5327-583196d560cd
</pre>
The tool does not list the AMD card, but up to and including version 2025.1 of TUFLOW, that card may still interfere with the GPU selection order. Also, from this readout, it is not at all clear which card is which and the order here may not match the order you expect from tools like Task Manager ('GPU 0', 'GPU 1', etc.).

This issue can be resolved by setting the environment variable <code>CUDA_VISIBLE_DEVICES</code>. There are two possible formats. It can either have a value like <code>0,1</code> or a more explicit value like <code>GPU-5060f556-4eb4-7155-4020-abadcb2fd735,GPU-f3825978-37f8-b933-5327-583196d560cd</code> using the identifiers from the <code>nvidia-smi</code> output. 

The short format just affects the default order. If using <code>-pu0</code> with TUFLOW selects the GPU considered #1 and vice versa, setting <code>CUDA_VISIBLE_DEVICES</code> to <code>1,0</code> reverses the default order. However, this order may change as new hardware is installed, or existing hardware reinstalled. The recommendation is to use the explicit values in the long format.

The value of the environment variable can either be set at the start of scripts used to run models, like batch files, PowerShell scripts, or Linux shell scripts, or globally so that it automatically applies to all running applications.

In a batch file or from the Command Prompt use this (note there are no quotes around the values, replace the values with the identifiers for detected GPUs):
<syntaxhighlight lang="dos">
SET CUDA_VISIBLE_DEVICES=GPU-5060f556-4eb4-7155-4020-abadcb2fd735,GPU-f3825978-37f8-b933-5327-583196d560cd
</syntaxhighlight>
In a PowerShell script or from the PowerShell prompt use this (note the quotes around the values, replace the values with the identifiers for detected GPUs):
<syntaxhighlight lang="powershell">
$env:CUDA_VISIBLE_DEVICES = "GPU-5060f556-4eb4-7155-4020-abadcb2fd735,GPU-f3825978-37f8-b933-5327-583196d560cd"
</syntaxhighlight>

If a globally set value is preferred, it can either be set for a single user account by finding "Edit environment variables ''for your account''" in the Windows Start menu and entering the values without quotes, or it can be set for all users on the machine by finding "Edit the ''system'' environment variables" in the Windows Start menu and doing the same in the 'System Variables' section. Note that administrator rights are required (elevation) to be able to do the latter. 

'''Warning:''' setting the value globally affects all CUDA-capable applications, not just TUFLOW. Please ensure that no other applications need the CUDA capabilities of the GPUs that are left out or use a local value in scripts or batch files instead.