View source for XPSWMM to TUFLOW-SWMM

= Introduction =
This Wiki page outlines recommended steps for the conversion of an XPSWMM model to TUFLOW. <br>

XPSWMM is a flood and urban stormwater drainage modeling software developed by Autodesk (previously Innovyze and XP Solutions). The XPSWMM solution uses EPA SWMM for its 1D calculations, dynamically linked to TUFLOW for its 2D calculations. The software functions within a custom build Graphical User Interface (GUI). Unknown to many XPSWMM modelers, during simulation, XPSWMM processes its inputs into TUFLOW files and also calls TUFLOW for the 2D calculations. As XPSWMM uses TUFLOW for its 2D engine, like-for-like results can be achieved using this software. The modeling workflow in TUFLOW differs from XPSWMM, as TUFLOW modeling is integrated with QGIS (Geographical Information System) GIS software instead of embedded within a software specific Graphical User Interface.

If you are building a TUFLOW SWMM model from scratch, not from XPSWMM, please refer to the <u>[[TUFLOW_SWMM_Tutorial_Introduction | TUFLOW SWMM Tutorials]]</u>. Tutorials are provided for the following topics:
* <u>[[TUFLOW SWMM Tutorial M01 | TUFLOW SWMM Module 1]]</u> - 1D SWMM Culverts
* <u>[[TUFLOW SWMM Tutorial M02 | TUFLOW SWMM Module 2]]</u> - 1D SWMM Pipe Network / 2D TUFLOW Direct Rainfall Hydrology
* <u>[[TUFLOW SWMM Tutorial M03 | TUFLOW SWMM Module 3]]</u> - 1D SWMM Pipe Network / 1D SWMM Urban Hydrology
* <u>[[TUFLOW SWMM Tutorial M04 | TUFLOW SWMM Module 4]]</u> - 1D SWMM Pipe Network / 1D SWMM Urban Hydrology: Executing multiple different event simulations from a single model control file.

== Dataset Download ==
The dataset used for this model conversion demonstration is available for download here: <u>[https://downloads.tuflow.com/SWMM/XPSWMM_XPX_to_TUFLOW_Conversion.zip XPSWMM XPX to TUFLOW Conversion Dataset]</u>. <br>
This dataset contains the XPSWMM model and the resulting TUFLOW model that is created. These can be found in the '''XPSWMM''' and '''Complete_Conversion''' folders respectively. <br>

'''If you are using this example conversion dataset, please rerun the XPSWMM model in the location where you save the dataset before beginning your own TUFLOW model conversion.''' Rerunning the model is necessary because XPSWMM will write TUFLOW files during its preprocessing, subsequently defining the correct file path information (for the location where you saved your files) in the newly written TUFLOW files. We also recommend creating your own TUFLOW model in a different folder from the provided TUFLOW dataset so you can easily compare your model against it.<br>

If you do not have access to a XPSWMM license and the XPSWMM Application, please refer to the <u>[[XPSWMM_to_TUFLOW-SWMM_Troubleshooting#Model_Conversion_Without_a_XPSWMM_License |XPSWMM to TUFLOW SWMM Troubleshooting]]</u> page. <br>

= XPSWMM to TUFLOW Model Conversion =
== Export Data From XPSWMM Model == 
'''Digital Terrain Model (DTM) Data'''<br>
XPSWMM reads its 2D DTM data in one of two ways:
* The DTM data can be directly specified in the '2D Model Settings', or 
* The DTM data can be internally processed by XPSWMM using its terrain tools and DTM builder. 
Depending on which method is applied to your XPSWMM model, the steps required to will vary. Review which method is used:
<ol>
<li> Open your existing model in XPSWMM.
<li> In the top dropdown menu options, go to Configuration > Job Control > 2D Model Settings. This will open a dialog. 
<li> Under '2D Hydraulics Job Control', select 'Surface & Sampling'. 
<li> Review the options in the 'Surface' section:
:* If 'Use DTM' is selected: Continue to the section below (<u>[[XPSWMM_to_TUFLOW-SWMM#Export_DTM_Data_from_XPSWMM |Export DTM Data from XPSWMM]]</u>) and complete the steps.
:* If 'Use Grid File for Topography' is selected and a 'Grid file' is specified, the steps outlined in the 'Export DTM Data From XPSWMM' section can be skipped. Go to the <u>[[XPSWMM_to_TUFLOW-SWMM#Export_XPX_Data_From_XPSWMM|Export XPX Data From XPSWMM]]</u> section.
'''Note:''' If you have a .tif elevation file, you can continue to the <u>[[XPSWMM_to_TUFLOW-SWMM#Export_XPX_Data_From_XPSWMM|Export XPX Data From XPSWMM]]</u> section. The conversion process will be identical however, once converted, you will have to create the '''TUFLOW\model\grid''' folder manually and copy the .tif file into it. 
<br><br>
{{Video|name=Animation_XPtoTUFLOW_Export_Data_01b.mp4|width=1350}}
<br>
</ol>

=== Export DTM Data From XPSWMM ===
If 'Use DTM' was selected, XPSWMM preprocesses its Digital Terrain Model (DTM) into a binary XPTIN elevation dataset for inclusion in the TUFLOW model. XPTIN is a binary format that can't readily be used in GIS software. For this reason, the following steps outline how to obtain a DTM dataset in a GIS friendly form.
<ol>
<li> In the XPSWMM Layers panel, under 'Topography', right click on '''DTM''' and select 'Export DTM Data'. 
:* Input TIN File: Select the relevant XPSWMM Input TIN file. 
:* Output File Format: 'ASCII Grid File Format'.
:* Cell Size Value: Choose a suitable DTM resolution. This resolution should be finer than the hydraulic model resolution. Typically, a DTM resolution is 1/5th to 1/10th of the hydraulic model 2D cell size is common (1 in the example dataset).
''Note: XPSWMM requires an integer cell size value for this export step. For example, Cell Size Values of 1, 2, 3, 4, 5 ... are suitable. The XPSWMM export will not function if you specify a decimal cell size value (e.g. 0.5, 2.5, etc.).''
  
<li> Click 'Export'.
<li> Save the file under an appropriate name (e.g. '''1D2D_Urban_Grid.asc''') to the folder where XPSWMM writes the .tcf during its simulation preprocessing. By default, this is the '''2D\Data''' folder. <br>
If this folder does not exist, either:
:* The XPSWMM model has not run, so the TUFLOW control files have not been created by XPSWMM. Run the XPSWMM model (go to Analyze > Solve... in top dropdown menu options).
:* Non-default output settings have been specified in XPSWMM. To determine the output location, in the top dropdown menu options, go to Configuration > Job Control > 2D Model Settings > Folder Options. Ensure the following 'Folder Locations' are selected.
::[[File: XPSWMM_to_TUFLOW_2DJobControl_FolderOptions_Dialog_01e.png]]<br><br>

<li> In the top dropdown menu options, navigate to  Configuration > Job Control > 2D Model Settings. This will open a dialog, under '2D Hydraulics Job Control', select 'Surface & Sampling'.
<li> Tick on 'Use grid file for topography' and select '...' to navigate to the Grid file saved in the '''2D\Data''' folder. This file will be read directly into TUFLOW. 
<li> Click 'OK' to save the settings. 
<li> Run the XPSWMM model to rewrite the TUFLOW files (including the new linkage to the DTM dataset) to the Data folder (go to Analyze > Solve... in top dropdown menu options).
<br><br>
{{Video|name=Animation_XPtoTUFLOW_Export_Data_02b.mp4|width=1350}}
<br>
</ol>

=== Export XPX Data From XPSWMM ===
A XPX file is a simple command line file that contains the information needed to create link and node objects. To export this data from the XPSWMM model:
<ol>
<li> In XPSWMM, in the top dropdown menu options, navigate to File > Import/Export Data > Export XPX Data. This will open a dialog. Leave all options as default and click 'Export'.
<li> When prompted, save the file under an appropriate name (e.g. '''1D2D_Urban_001.xpx''') to the folder where the XPSWMM model is located.
<br><br>
{{Video|name=Animation_XPtoTUFLOW_Export_Data_03b.mp4|width=1350}}
<br>

</ol>

== Convert XPSWMM Model to TUFLOW SWMM ==
Ensure you have QGIS and have the QGIS TUFLOW Plugin installed:
*Install QGIS 3.34 or later: <u>[https://www.qgis.org/download/ Latest 64-bit version of QGIS]</u>. 
*Install the QGIS TUFLOW Plugin by following the instructions, <u>[[TUFLOW_QGIS_Plugin | QGIS TUFLOW Plugin Installation]]</u>.

The following steps outline the process of converting a XPSWMM model to TUFLOW SWMM using the '<u>[[QGIS_SWMM_Convert_XPSWMM_Model_From_XPX|Convert - XPSWMM model from XPX]]</u>' processing tool.

<ol>
<li> Open QGIS and go to Processing > Toolbox from the top dropdown menu options to open the Processing Toolbox.
<li> Go to TUFLOW >> SWMM in the processing tool list and select '<u>[[QGIS_SWMM_Convert_XPSWMM_Model_From_XPX|Convert - XPSWMM model from XPX]]</u>'. This opens the dialog shown below.
:* XPSWMM Exported XPX File: Click '...' and navigate to the exported XPX data (e.g. '''1D2D_Urban_001.xpx'''). This should be located in the same folder as the XPSWMM model. 
:* TUFLOW TCF Filename: Click '...' and navigate to the XPSWMM .tcf. This should be located in the '''2D\Data''' folder (e.g. '''2D\Data >> 1D2D_Urban_001.tcf'''). 
:* SWMM File Prefix: Choose an appropriate prefix for the SWMM files (e.g. '''1D2D_Urban_swmm''').
:* Output Solution Scheme: 'HPC'
:* Output Hardware Specification: 'GPU' (if GPU is not available, select 'CPU')
:* Output Vector Format: 'GPKG'
:* Output Raster Format: 'GTIFF'
:* Output Profile: Any option can be used. 'ALL IN ONE' is used in this example for model design consistency with the <u>[[Tutorial_Introduction | TUFLOW SWMM Tutorials]]</u>.
:* Event name if no global storms: Choose an appropriate event name if applicable (100yr in the example dataset). 
:* BC width for created 1D/2D connections (HX/SX): This value should be approximately 2 times the width of the hydraulic model 2D cell size (10 in the example dataset).
:* BC offset distance for created 1D/2D connections (HX/SX): Distance from the channel endpoint to the midpoint of the BC line (2 in the example dataset).
:* Output Folder: Click '...' and navigate to an appropriate location to save your TUFLOW model. In this location, create a new folder called '''TUFLOW''' and select it.
:* Output CRS: Select an appropriate Coordinate Reference System (CRS) for the model. For the demonstration model, the CRS is 'EPSG:32760 - WGS 84 / UTM zone 60S'.
<li> Click 'Run'. Once the tool is finished, click 'Close'.
<br><br>
{{Video|name=Animation_XPtoTUFLOW_Convert_01c.mp4|width=1350}}
<br>
</ol>

Inspect the output of the '<u>[[QGIS_SWMM_Convert_XPSWMM_Model_From_XPX|Convert - XPSWMM model from XPX]]</u>' processing tool and load the TUFLOW model into QGIS:
<ol>
<li> In Windows File Explorer, inspect the files output by the processing tool. In particular:
* The '''TUFLOW\bc_dbase''' folder contains the BC Database which will contain any curves used in BC conditions, for easy extending to other events.
* The '''TUFLOW\model''' folder contains the TUFLOW SWMM Control File (TSCF) which holds all commands specific to SWMM, including links to the converted SWMM INP file. 
* The '''TUFLOW\model\swmm''' folder contains a GeoPackage database ('''*_convert_messages.gpkg'''). This GeoPackage contains locations and descriptions of any errors or warnings that occurred during the conversion process. 
: '''Note:''' Conversion messages are addressed in further detail here <u>[[XPSWMM_to_TUFLOW-SWMM_Troubleshooting#Common_Conversion_Issues |XPSWMM to TUFLOW SWMM Troubleshooting]]</u>. 
* The '''TUFLOW\runs''' folder contains the TCF. If event(s) are specified, the TCF filename with have event placeholder(s), i.e. '_~e1~_~e2~_' (e.g. '''1D2D_Urban_001_~e1~.tcf'''). Again, if event(s) are used, this folder also contains the TUFLOW Event File (TEF) with the event(s) specified. <br>
{{Video|name=Animation_XPtoTUFLOW_Convert_02b.mp4|width=800}}
<br>
<li> Open QGIS, and click on the ‘Load TUFLOW Layers from TCF’ symbol from the QGIS TUFLOW Plugin toolbar.
<br>
[[File: Tuflow_plugin_load_tcf_layers.png]]<br>
<br>
<li> Navigate to the location of the TUFLOW model and go to the '''TUFLOW\runs''' folder. Select the TCF.
<li> In the Load Layers window, select:
* Ordering Options: Alphabetical
* Grouping Options: Group by control file
* Raster Load Options: Load Normally 
<li> Click ‘Open’ and ‘OK’.
<li> Change the symbology of the DTM dataset:
* In the QGIS Layers panel, right click on the DTM Grid file and select 'Properties'.
* In the Symbology tab, under 'Band Rendering', set the 'Render type' to 'Hillshade' and the 'Z Factor' to 3.
<li> Inspect the output GIS layers.
<br><br>
{{Video|name=Animation_XPtoTUFLOW_Convert_03b.mp4|width=1350}}
<br>
</ol>

== Recommended Additional Conversion Steps == 
While the bulk of the XPSWMM to TUFLOW SWMM model conversion is automated by the '<u>[[QGIS_SWMM_Convert_XPSWMM_Model_From_XPX|Convert - XPSWMM model from XPX]]</u>' processing tool, some required or recommended conventions must be implemented/updated manually. Therefore, it is highly recommended to complete the additional conversion steps: <u>[[XPSWMM_to_TUFLOW-SWMM_Recommended_Additional_Conversion_Steps|Recommended Additional Conversion Steps]]</u>. <br> 
These steps can help improve model stability and they may also be helpful to address any messages that occurred when using the conversion tool. 
<br>

== TUFLOW Simulation Execution ==
Set up a simple batch file (.bat) to run TUFLOW. This approach calls the TUFLOW executable file (.exe) and runs the TCF file.
<ol>
<li>Create a new text file in the '''TUFLOW\runs''' folder and save as '''_run_HPC.bat'''. 
<li>Open the '''_run_HPC.bat''' in a text editor and include a file path to the TUFLOW executable and the TCF name: <br>
<font color="blue"><tt>set </font>exe<font color="red">=</font>"..\..\exe\2026.0.0\TUFLOW_iSP_w64.exe"<br>
<font color="blue"><tt>set </font>run<font color="red">=</font>start "TUFLOW" /wait <font color="orange">%exe%</font> -b<br>
<font color="orange">%run%</font> -e1 100yr 1D2D_Urban_001_~e1~.tcf</tt></font></tt> <br>
<br>
'''Note:''' A relative path is used for the executable and the TCF, full file path can also be used.
<li>Save the batch file and double click it in Windows File Explorer to run the simulation. This will open the TUFLOW Console Window and the simulation should be executed. 
</ol>

TUFLOW simulations can be executed via numerous ways. A comprehensive summary of the most commonly used approaches is documented in the <u>[[Running_TUFLOW | Running TUFLOW]]</u> Wiki page.

=== Troubleshooting ===
Did your TUFLOW SWMM model fail to run successfully? If so, here is a link to a troubleshooting guide: <u>[[TUFLOW_SWMM_Troubleshooting | TUFLOW SWMM Troubleshooting]]</u>.<br>

= TUFLOW SWMM Result Viewing =
Are you familiar with loading and viewing TUFLOW results in QGIS? If not, we strongly recommend self-registering and completing our free eLearning: 
<u>[https://www.tuflow.com/training/training-course-catalogue/tt001-e-introduction-to-qgis-for-tuflow-elearning/ Introduction to QGIS for TUFLOW]</u>. <br>

Our <u>[[TUFLOW_SWMM_Tutorial_Introduction | TUFLOW SWMM Tutorials]]</u> also demonstrate working with TUFLOW SWMM results.

= Recommended Further Reading =
For users who wish to get a better understanding of either 2D TUFLOW or 1D EPA SWMM, the following resources may be of use:<br>
'''TUFLOW''': <u>[https://www.tuflow.com www.tuflow.com]</u> 
*<u>[https://www.tuflow.com/downloads/#tuflow TUFLOW User Manual]</u> 
*<u>[https://docs.tuflow.com/classic-hpc/release/2023-03-AD/ TUFLOW 2023-03-AD release notes]</u> 
*<u>[[Tutorial_Introduction#Tutorial_Modules | TUFLOW Tutorial Models]]</u>

<br>
'''EPA SWMM''': <u>[https://www.epa.gov/water-research/storm-water-management-model-swmm www.epa.gov]</u> 
*<u>[https://downloads.tuflow.com/SWMM/SWMM5_Reference_Manual_Volume1_Hydrology_P100NYRA.pdf SWMM5 Reference Manual - Volume 1 (Hydrology)]</u> 
*<u>[https://downloads.tuflow.com/SWMM/SWMM5_Reference_Manual_Volume2_Hydaulics_P100S9AS.pdf SWMM5 Reference Manual - Volume 2 (Hydraulics) ]</u> 
*<u>[https://downloads.tuflow.com/SWMM/SWMM5_Reference_Manual_Volume2_Hydraulics_Addendum-20220210mas2wr.pdf  SWMM5 Reference Manual - Volume 2 (Hydraulics Addendum) ]</u> 
*<u>[https://downloads.tuflow.com/SWMM/SWMM5_Reference_Manual_Volume3_Water_Quality_P100P2NY.pdf  SWMM5 Reference Manual - Volume 3 (Water Quality) ]</u> 
*<u>[https://downloads.tuflow.com/SWMM/swmm-users-manual-version-5.2.pdf EPA SWMM5 User's Manual]</u>

=Contact=
For comments, requests and feedback contact <u>[mailto:support@tuflow.com support@tuflow.com]</u>.<br>
For further training opportunities see <u>[https://tuflow.com/training/training-course-catalogue/ TUFLOW Training Catalogue]</u> and/or contact <u>[mailto:training@tuflow.com training@tuflow.com]</u>. <br>
<br>

{{Tips Navigation
|uplink=[[Main_Page| Back to Wiki Main Page]]
}}