Run TUFLOW From a Batch-file

TUFLOW simulations can also be started from a batch file. Batch files are text files which contain a series of commands. Batch files allow a large degree of flexibility in starting TUFLOW simulations, advanced batch files can be used to:

• Run multiple simulations
• Set CPU priority for TUFLOW simulations
• Create a wait (e.g. start a simulation in 1 hour)
• Test simulation input
• Copy a TUFLOW model
• Post process results after simulation has finished.

Please add any suggestions (I know there are some batch file power users out there!) to the discussion page.

Batch file for Tutorial Model Module 1

To use a batch file, create a new text file in the TUFLOW\runs directory. The name is arbitrary but the file must have the extension ".bat". For example the file name could be _run_TUFLOW.bat.

Open this file in a text editor and add the following line.

"C:\TUFLOW\Releases\2011-09\w32\TUFLOW_iSP_w32.exe" "M01_5m_001.tcf"

This is the simplest command line for starting TUFLOW, the line above contains only two arguments, the first is the path to the TUFLOW executable and the second argument "M01_5m_001.tcf" is the simulation file name. Depending on the location of the executable the above command line will need to be modified. The quotations around the TUFLOW executable and the .tcf file name are only required if there are spaces in the file names. However, it is recommended to include these to avoid problems.

TIP:
A second line with the text "pause" can be added, this will prompt the user to press a key at the end of the batch file. This is useful if the DOS window disappears from the screen, before you can read the output, the pause command will keep the window open.

"C:\TUFLOW\Releases\2011-09\w32\TUFLOW_iSP_w32.exe" "M01_5m_001.tcf"
pause

Other Batch File Switches

These switches are available in all batch files, they are not specific to TUFLOW.

Starting process in new window

In order to run multiple simulations at the same time, each simulation needs be started in a new console window. To do this begin the batch file line with the "Start" command, a for the window is then added. For example, to start the simulation above in a new window titled "TUFLOW" the following would be used.

Start "TUFLOW" "C:\TUFLOW\Releases\2011-09\w32\TUFLOW_iSP_w32.exe"  "M01_5m_003.tcf"

Wait

If multiple simulations are to be run, it is often desirable to run these in series, i.e. the second simulation starts after the first has finished. a /wait switch can be added which makes the batch file wait until the process is finished before moving onto the next command.
An example of this is:

Start "TUFLOW" /wait "C:\TUFLOW\Releases\2011-09\w32\TUFLOW_iSP_w32.exe" M01_5m_003.tcf
Start "TUFLOW" /wait "C:\TUFLOW\Releases\2011-09\w32\TUFLOW_iSP_w32.exe" M01_2p5m_005.tcf

Without the wait command, both simulations would start at the same time.
Note: Unless there is a third simulation in the batch file the /wait command in the second line is not required, this prevents the batchfile from moving onto the third line.

Setting a priority

The priority of a process (such a TUFLOW simulation) can be specified within a batch file. This can be useful for a number of reasons:

• Lowering the priority of a simulation to allow you to continue working;
• Setting a high priority for critical runs; or
• Making sure you runs are high priority on a shared computing resource!

The following are valid priority switches (in increasing order of priority):

• /LOW
• /BELOWNORMAL
• /NORMAL
• /ABOVENORMAL
• /HIGH
• /REALTIME

An example of a low priority simulation is:

start "TUFLOW" /low "C:\TUFLOW\Releases\2011-09\w32\TUFLOW_iSP_w32.exe" "M01_5m_003.tcf"

TIP: The priority of a TUFLOW simulation can be changed at a later stage by using the Windows Task Manager, navigate to the Processes tab and right click on the TUFLOW process (e.g. TUFLOW_iSP_w64.exe) and select the set priority option.

TUFLOW switches in a batch file

These switches are specific to TUFLOW simulations.

Testing a simulation

The -t (test) switch is very useful for testing the data input without running the simulation. It is good practice to use this switch before carrying out the simulations, as this will tell you whether there are any data input problems. The -t switch runs TUFLOW to just before it starts the hydrodynamic computations. An example is:

start "TUFLOW" "C:\TUFLOW\Releases\2011-09\w32\TUFLOW_iSP_w32.exe" -t "M01_5m_003.tcf"

Batch mode in TUFLOW

The use of the –b (batch) switch which suppresses the need to press the return key at the end of a simulation. This ensures that one simulation proceeds on to the next without any need for user input. This is required for running multiple simulations in series (one after the other). An example is:

start "TUFLOW" "C:\TUFLOW\Releases\2011-09\w32\TUFLOW_iSP_w32.exe" -b "M01_5m_003.tcf"

Automatically Create Folders in a TUFLOW model

The use of the -acf (automatically create folders) switch prevents the dialog prompt from appearing when encountering non-existent folders (ie. results folders), and creates these folders automatically. If for any reason the folder can't be created, a dialog will appear. This feature was first introduced for Build 2012-05-AC and is not available for TUFLOW builds prior to this. For models simulated with Build 2013-12-AA or later, non-existent folders are now automatically created. If you would prefer to have the create folder query dialog to appear, you can specify the –qcf run time option (stands for query create folders).
An example is:

start "TUFLOW" "C:\TUFLOW\Releases\2012-05-AC\w32\TUFLOW_iSP_w32.exe" -acf "C:\TUFLOW\Tutorial_Wiki\TUFLOW\runs\M01_5m_003.tcf"

Suppress Queries in a TUFLOW model

The use of the -nq (no queries) switch prevents any queries from displaying when simulating a TUFLOW model. At this stage, the only query not displayed if this option is used is if you press Ctrl-C to terminate a simulation cleanly. If –nq is specified and Ctrl-C is pressed, the simulation terminates cleanly without a query dialog.. This feature was first introduced for Build 2012-05-AC and is not available for TUFLOW builds prior to this.
An example is:

start "TUFLOW" "C:\TUFLOW\Releases\2012-05-AC\w32\TUFLOW_iSP_w32.exe" -nq "C:\TUFLOW\Tutorial_Wiki\TUFLOW\runs\M01_5m_003.tcf"

Copy a TUFLOW model

TUFLOW can be run in copy mode. By default, this copies all TUFLOW inputs for the simulation into a subdirectory where the .tcf is located (generally TUFLOW\runs\). There are four copy options: -c, -ca, -cap, -cncf.
-c copies only the files read by TUFLOW, for example the .mif and .mid files are read by TUFLOW, but the MapInfo format files (.tab, .id, .dat and .map) are not read by TUFLOW and these files will not be copied.
-ca copies all files with the same name (regardless of extension), if the 2d_code_M01_003.MIF and 2d_code_M01_003.MID files are copied all of the MapInfo files (2d_code_M01_003.DAT, 2d_code_M01_003.ID, 2d_code_M01_003.MAP and 2d_code_M01_003.TAB), will all be copied.
-cap allows the user to specify an alternate path in which to copy all files with the same name. For example, -cap "C:\put_model_here" will write all model files including original layers into a folder C:\put_model_here.
-cncf copies the essential input files and excludes all check files.
An example that copies all files (-ca switch) is:

start "TUFLOW" "C:\TUFLOW\Releases\2011-09\w32\TUFLOW_iSP_w32.exe" -ca "C:\TUFLOW\Tutorial_Wiki\TUFLOW\runs\M01_5m_003.tcf"

An example that copies all files to an alternate path (-cap) is:

start "TUFLOW" "C:\TUFLOW\Releases\2011-09\w32\TUFLOW_iSP_w32.exe" -cap "C:\Copy_of_model" "C:\TUFLOW\Tutorial_Wiki\TUFLOW\runs\M01_5m_003.tcf"

Advanced Batch Files

This section is yet to be finalised.

Creating a delay

Windows 7

For a Windows 7 operating system a delay of a set duration can be created with the Timeout <seconds> command. For example:

Timeout 10

Will create a 10 second delay, a countdown timer is given. Press any key to continue.

Other Systems

It is possible to create a delay of specified length in a batch file. The best way do this is actually to create a second batch file! To do this create a text file and call it wait.bat. Within that file enter the following text:

@ping 127.0.0.1 -n 2 -w 1000 > nul
@ping 127.0.0.1 -n %1% -w 1000> nul

To create the pause we ping an address for a set time and disregard the output. Once the wait.bat has been created, it can be called from another batchfile using the following syntax:

call wait <time in seconds.

In the example below, the first simulation is started and then 60 seconds later (using the wait.bat file) the second simulation is started. Note there is no /wait flag specified, if there was the batchfile would wait for 60 seconds after the first simulation had finished before starting the second simulation.

Start "TUFLOW" "C:\TUFLOW\Releases\2011-09\w32\TUFLOW_iSP_w32.exe" M01_5m_003.tcf
call wait 60
Start "TUFLOW" /wait "C:\TUFLOW\Releases\2011-09\w32\TUFLOW_iSP_w32.exe" M01_2p5m_005.tcf

For this to work the wait.bat file needs to be in the same folder as the batchfile calling it. You can put the file in a specific location and add this path as an environment variable. If this is done "call wait can be added to any batch file.
To set the environment variable, you will need to have administrator access to the machine, and add a colon (;) and then the batchfile location (e.g. ";C:\batch_files\" , without the quotes) in the path system variables. The separator character is a colon (;) this is added to specify a new path. For more details on modifying the environment please see here: http://support.microsoft.com/kb/310519.

Variables

Batch files can be easily setup so that they are more generic and easily customised when moving from one project to another. For example, in the below a variable, TUFLOWEXE, is used to define the path to the TUFLOW exe to be used, and a variable RUN is used to incorporate options such as the /wait so that the simulations run in series (one after the other).

set TUFLOWEXE=C:\TUFLOW\Releases\2011-09\w64\TUFLOW_iSP_w64.exe
set RUN=start "TUFLOW" /wait "%TUFLOWEXE%" -b
%RUN% MR_H99_C25_Q100.tcf
%RUN% MR_H99_C25_Q050.tcf
%RUN% MR_H99_C25_Q020.tcf

The advantage of using variables is if the path to the TUFLOW exe changes, or to run a different version of TUFLOW, it is just a simple change in the .bat file. In the above, note the use of quotes around %TUFLOWEXE% in the definition for the RUN variable – quotes are needed around file pathnames whenever they contain a space.

Looping in a batch file

If you have a large number of events to run using the event files a for loop can be used in a batch file. In this example we have 5 event magnitudes (-e1 input) and 5 event durations (-e2 input). The following batch file can be used to run all 25 simulations one after the other:

@echo off

:: This sets the variables as local, so you can use another batch file with A and B variables
SetLocal

:: set up variables
set A=Q010 Q020 Q050 Q100 Q200
set B=10min 30min 60min 120min 270min

:: Loop Through
FOR %%a in (%A%) do (
    FOR %%b in (%B%) DO (
        start "TUFLOW" /wait C:\TUFLOW\releases\2011-09\w64\TUFLOW_iSP_w64.exe -e1 %%a -e2 %%b filename.tcf
    )
)
pause

In order to test the syntax, you can test the batch file by replacing the following line:
start "TUFLOW" /wait C:\TUFLOW\releases\2011-09\w64\TUFLOW_iSP_w64.exe -e1 %%a -e2 %%b filename.tcf
With the following line:
echo start "TUFLOW" /wait C:\TUFLOW\releases\2011-09\w64\TUFLOW_iSP_w64.exe -e1 %%a -e2 %%b filename.tcf
By using the echo in the command line, rather than starting the simulations the command line will be displayed in the DOS output. In the example above the DOS output is shown in the image below.

Alternatively this can be achieved using the following syntax, which is essentially the same but has the for loops compressed to a single line:

@echo off
setlocal
set A=Q010 Q020 Q050 Q100 Q200
set B=10min 30min 60min 120min 270min

FOR %%a in (%A%) do FOR %%b in (%B%) do start "TUFLOW" TUFLOW_iSP_w64.exe -e1 %%a -e2 %%b filename_~e1~_~e2~.tcf
pause

This can be extended to more variables for example:

@echo off
setlocal
set A=Q010 Q020 Q050 Q100 Q200
set B=10min 30min 60min 120min 270min
set C=Exg Dev Mit

FOR %%a in (%A%) do (
    FOR %%b in (%B%) do (
        FOR %%c in (%C%) do (
            start "TUFLOW" /wait TUFLOW_iSP_w64.exe -e1 %%a -e2 %%b -s1 %%c filename_~e1~_~e2~_~s1~.tcf
        )
    )
)
pause

Looping in a batch file up to a certain number of runs

The below example shows a method for controlling the number of concurrent simulations. See also this post by user Ben G T on the TUFLOW forums.

As a brief description of the below, the "do_while_loop_start" counts the number of processes in the task manager which have "TUFLOW" in the name. If this is greater than or equal to the variable "CPU_Cores" then then batch file waits for 60 seconds.

@ echo off
 
Set A=Q001 Q002 Q005 Q010 Q020 Q050 Q100
Set B=Exg Dev
set /a CPU_Cores=3
 
 
FOR %%a in (%A%) DO (               
    FOR %%b in (%B%) DO (
       call :do_while_loop_start
       Start "TUFLOW" "C:\TUFLOW\Releases\2012-05\w64\TUFLOW_iDP_w64.exe" -b -e1 %%a -s1 %%b M11_5m_~e1~_~s1~_001.tcf
       )
)

Pause
 
:do_while_loop_start
    set /a count=0
    for /f %%x in ('tasklist ^| find /c "TUFLOW"') do set count=%%x
    if %count% geq %CPU_Cores% (
        PING 1.1.1.1 -n 1 -w 60000 >NUL
        goto do_while_loop_start
)

Another way to achieve the same result using IF logic is shown below. This batch file will loop through the series of 14 simulations in groups of three.

@ echo off
setlocal enabledelayedexpansion
Set %exe%="C:\TUFLOW\Releases\2012-05\w64\TUFLOW_iDP_w64.exe"
 
Set A=Q001 Q002 Q005 Q010 Q020 Q050 Q100
Set B=Exg Dev
Set P = 3

rem A=Event
rem B=Scenario
rem P=Number of models to run in parallel 

set /A Counter=0
FOR %%a in (%A%) DO (               
    FOR %%b in (%B%) DO (
       set /A Counter+=1
       IF /i !Counter!==!P! (
           Start "TUFLOW" /wait %exe% -b -e1 %%a -s1 %%b M11_5m_~e1~_~s1~_001.tcf
           set /A Counter=0
	) else (
	   Start "TUFLOW"       %exe% -b -e1 %%a -s1 %%b M11_5m_~e1~_~s1~_001.tcf
	)
    )
)

Shutdown on Completion

Often we may have a number of TUFLOW simulations going when we leave work on a Friday afternoon, if these are not going to take all weekend to run, using some of the batch file logic above you can create a batch file that will shutdown the computer when no TUFLOW simulations are running. An example batch file for this purpose is below:

echo off

:do_while_loop_start
    set /a count=0
    for /f %%x in ('tasklist ^| find /c "TUFLOW"') do set count=%%x
    IF %count%==0 (
		shutdown.exe /s /t 30
	) ELSE (
        timeout 60
        goto do_while_loop_start
	)
pause

This batch file counts the number of processes with "TUFLOW" in the process name and if this is greater than 0, a delay of 60 seconds is created and the loop started again. If no "TUFLOW" simulations are running a shutdown command is issued.
To test the batch file it is recommended that you add an "echo" in front of the shutdown.exe /s /t 30 command to ensure it is working correctly (I use this on a Windows 7 machine, but have not tested it on either XP or Windows 8).
In order to restart the machine rather than power it off the /s can be replaced with /r. This can be useful when a heavily utilised modelling computer needs a restart!