2024-04-08T23:35:49Z

RustyJonesTUFLOW: Created page with "{{TUFLOW_Message |tuflow_message=WARNING 2592 - Ignoring "Hardware ==" command due to -hw command line options |alt_msg=NA |type=WARNING |message_desc=When TUFLOW was laun..."

TUFLOW Viewer

2024-03-18T18:37:22Z

RustyJonesTUFLOW: /* TUFLOW-SWMM Results */

TUFLOW Viewer replaces Crayfish and TUPLOT as the TUFLOW result viewer for [[QGIS_Tips | QGIS]] (version 3.6 onwards). It uses the Mesh Data Abstraction Library (MDAL) available in QGIS to display and interact with TUFLOW map output results, and the TUFLOW results python library (the same library used by TUPLOT in earlier versions of QGIS) for viewing TUFLOW time series results. 
 
__TOC__

=Getting Started=
==QGIS Version==
''A few notes on the recommended QGIS version:'' 

It is recommended to use the latest version of QGIS. The reasons for this are:
* TUFLOW Viewer is developed using the latest version, and although backwards compatibility is maintained as best as possible, TUFLOW Viewer is tested more frequently on the latest QGIS version.
* The mesh data provider (MDAL) and the temporal controller (which now underpins the mesh datasets in QGIS) are both relatively new in comparison to other libraries (e.g. GDAL) and are therefore more regularly updated with new features, enhancements, and bug fixes by the QGIS developers. TUFLOW Viewer takes advantage of these updates. 

Although TUFLOW Viewer has been developed to be consistent across QGIS versions, it is not always possible. Some QGIS developments and behaviour changes have led to TUFLOW Viewer behaviour also changing either through necessity or to keep in-line with the QGIS development direction. The below is a current list of known behaviour changes due to QGIS version: 
<ol>
<li> '''Date-time format within TUFLOW Viewer''' - QGIS 3.14 introduced the '''temporal controller''' 
* Prior to QGIS 3.14, TUFLOW Viewer stored all results internally as relative time. The user could display as date-time and the reference time by changing '''Zero Date''' in '''Settings >> Options'''.
* QGIS 3.14, TUFLOW Viewer tried to mimic the behaviour of previous versions but the reference time could also be altered natively in the mesh layer '''Properties'''. Note on this version, this is the first QGIS release with the '''temporal controller''' and as a consequence some of the functionality matured and changed in subsequent versions. Users may experience strange behaviour when using TUFLOW Viewer with the QGIS 3.14 if using date-time format. It is recommended to upgrade to later versions of QGIS if date-time format is required.
* Post QGIS 3.14 TUFLOW Viewer stores all results internally as absolute time and the user must change the reference time in the native properties of the mesh layer to alter the dates being displayed. '''Zero Date''' now only changes how the relative time is displayed in TUFLOW Viewer and doesn't change the mesh layer reference time.
* Please follow the link below on how to use isodate (date-time) format in TUFLOW Viewer in QGIS 3.16+ (recommended minimum version if using date-time format): [[TUFLOW_Viewer_-_Isodate_(Date-Time)_Format | Working With Isodate (Date-Time) format]].
</ol> 

'''Other Known Issues''' 
* QGIS 3.24 uses Matplotlib v3.5.1 which contains the following known bug which may affect users when re-labelling datasets within the TUFLOW Viewer plot window: 
: [[TUFLOW_Viewer_Matplotlib_v3.5.1_Bug | TUFLOW Viewer - Matplotlib v3.5.1 bug]] 
* "ValueError: Failed to find font DejaVu Sans:style=normal:variant=normal:weight=normal:stretch=normal:size=10.0, and fallback to the default font was disabled"
: [[TUFLOW_Viewer_Matplotlib_Font_Error | TUFLOW Viewer - Matplotlib Font Error]] 

==Installation or Version Upgrade==
===Installation===
TUFLOW Viewer is a free tool that comes as part of the TUFLOW plugin in QGIS. For instructions on how to install the plugin, please follow these steps:[[TUFLOW_QGIS_Plugin#Installation_of_Plugin | Installation of Plugin]]. 
There are also instructions on installing plugins in the QGIS documentation - if you choose to follow the QGIS documentation, the plugin is called "TUFLOW" in the repository: [https://docs.qgis.org/3.16/en/docs/training_manual/qgis_plugins/fetching_plugins.html Link to QGIS Documentation - Installing and Managing Plugins]. 

===Plugin Upgrades===
It's recommended to upgrade the plugin whenever a new version is released. The upgrade process is typically done via the Plugin Manager ('''QGIS Drop Menu: Plugins >> Manage and Install Plugins'''). 
[[File: TUFLOW Plugin Update.JPG||600px]] 
If you encounter an error while upgrading the plugin please follow these steps: [[ TUFLOW_QGIS_Plugin#Error_While_Upgrading_Plugin | Error_While_Upgrading_Plugin ]].
==Development Version==
It's possible to test out the latest development version of the plugin and TUFLOW Viewer by following the instructions below: 
[[Installing_the_Latest_Development_Version_of_the_TUFLOW_Plugin | Installing the Latest Development Version of the TUFLOW Plugin]]
 

=Using TUFLOW Viewer=
==Opening the TUFLOW Viewer==
After installing the QGIS TUFLOW plugin, the TUFLOW Viewer tool can be opened by clicking the following icon in the plugin toolbar: 
[[File: TUFLOW_Viewer_In_Toolbar.PNG]]. 

==Loading Results==
TUFLOW simulation results can be loaded several ways: 
<ol>
<li> Select '''File >> Load Results''' from the TUFLOW Viewer drop down menu. 
[[File:TUFLOW Plugin Load 1.jpg||800px]] 
<li> Right Click in the '''Open Results''' panel and select '''Load Results'''. 
[[File:TUFLOW Plugin Load 2.jpg||800px]] 
</ol>
Using either of the above methods, the following result load options are available:
<ol>
<li> '''[[TUFLOW_Viewer_-_Load_Results | Load All Results]]''' --> This is done via a TCF, TLF, or FVC (TUFLOW FV) file and will load in all results (Map Outputs and ESTRY Time Series).
<li> '''[[TUFLOW_Viewer_-_Load_Results_-_Map_Outputs | Load Results - Map Outputs]]''' --> Select map output mesh results file ('''*.xmdf, *.dat, *.2dm, *.xmdf.sup, *.dat.sup, *.nc''' ''(supports netCDF format from TUFLOW FV output only)'').
<li> '''[[TUFLOW_Viewer_-_Load_Results_-_Time_Series | Load Results - Time Series]]''' --> Select ESTRY / SWMM time series output results ('''*.tpc *.gpkg''').
<li> '''[[TUFLOW_Viewer_-_Load_Results_-_Time_Series_FM | Load Results - Flood Modeller Time Series]]''' --> Load Flood Modeller results. This requires a '''*.gxy''' and result '''*.csv''' file to be exported from Flood Modeller.
<li> '''[[TUFLOW_Viewer_-_Load_Results_-_Particles | Load Results - Particle Tracking Module]]''' --> Select output from particle module ('''*.nc''') which will typically be suffixed with '''_ptm'''.
<li> '''[[TUFLOW_Viewer_-_Load_Results_-_NetCDF_Grid | Load Results - NetCDF Grid]]''' --> Load TUFLOW NC or HRNC map output results.
<li> '''[[TUFLOW_Viewer_-_Import_1D_Hydraulic_Tables | Import 1D Hydraulic Tables]]''' --> Select a '''_1d_ta_tables_check.csv''' check file.
<li> '''[[TUFLOW_Viewer_-_Import_2D_BC_Tables | Import 2D BC Tables]]''' --> Select a '''_2d_bc_tables_check.csv''' check file.
<li> '''[[TUFLOW_Viewer_-_Import_1D_ESTRY_Cross_-_Sections | Import 1D ESTRY Cross-Sections]]''' --> This will automatically happen if an appropriate TUFLOW input is opened in QGIS while TUFLOW Viewer is open (e.g. 1d_xs).
<li> '''[[TUFLOW_Viewer_-_Importing_a_User_Defined_Time_Series_To_Display_On_The_Plot | Importing a User Defined Time Series Dataset To Display In The Plot Window]]'''
<li> '''[[TUFLOW_Viewer_-_Loading_Results_While_TUFLOW_is_Running | Loading Results While TUFLOW is Running]]'''
</ol>

'''Troubleshooting''' 
If you receive an error similar to that shown below when attempting to load results from a TCF you will need to fix the encoding of your TUFLOW control files: 
<pre>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 626: invalid start byte
</pre>
This error is caused by incompatible encoding of the TUFLOW control files. You can resolve this issue by changing the encoding of all your TUFLOW control files to a single type (typically UTF-8). You can do this using Notepad++. Please see the following link for instructions on how to do this (refer last image): [[TUFLOW_Message_0060]]. 

== TUFLOW-SWMM Results ==
Results from a linked TUFLOW-SWMM model can be loaded via the standard menu options (available from the TUFLOW plugin version 3.10):
* '''[[TUFLOW_Viewer_-_Load_Results_-_Time_Series | Load Results - Time Series]]''' to load the GPKG time series format (_swmm_ts.gpkg) by itself. The GPKG time series format is a new format similar to the .tpc with some additional functionality. More information can be found in the [[#GPKG_Time_Series_Format | GPKG Time Series Format]] section.
* '''[[TUFLOW_Viewer_-_Load_Results_-_Map_Outputs | Load Results - Map Outputs]]''' to load in 2D map output results
* '''[[TUFLOW_Viewer_-_Load_Results | Load Results]]''' to load all available results from the model. This includes .xmdf, .tpc, and .gpkg results which could all be available for a given model run. Note that .tpc and .gpkg are both time series type results and will load separate GIS _PLOT_ layers.

== GPKG Time Series Format ==
The GPKG time series format is a new format similar to the .tpc with enhanced temporal functionality. Currently the GPKG time series format is only supported as an output from SWMM in TUFLOW-SWMM linked models and only supported in TUFLOW Plugin version 3.10+ and QGIS 3.16+. 
 
The GPKG format is different from the TPC format as it supports temporal styling in QGIS. As an example, the line width of the channels can be varied by both time and flow, with wider lines showing higher flow than thinner lines at a particular timestep. The format is fully compatible with the QGIS temporal controller and reacts dynamically as the temporal controller is updated. This results in a dynamic, and intuitive, method of showing the user the flood progression in the 1D system. Another benefit of being compatible with the core QGIS temporal capabilities is that the styling will also be dynamically updated if included in an animation export. TUFLOW Viewer's animation export tool has been updated to enable GPKG results to be exported with 2D results or even by itself. 
 
The layer's styling can be automatically set using the TUFLOW Plugin via the layer's right-click context menu (under the TUFLOW submenu) or will automatically by styled if the results are loaded via TUFLOW Viewer. Examples of this format in QGIS are linked below:
* [[Automatically_Styling_GPKG_Time_Series | Styling layers from the GPKG time series output]]
* [[TUFLOW_Viewer_-_Load_Results_-_GPKG_Time_Series | Loading results via TUFLOW Viewer]]
The GPKG time series format is an open format and the specification is detailed at the below link:
* [[GPKG_Time_Series_Format_Specification | GPKG Time Series Format Specification]]

== Data Selection, Display and Styling==
===Map Output===
Map Outputs are the time varying 2D result outputs from TUFLOW (or 3D outputs from TUFLOW FV). Data selection, display, styling and plotting instructions for Map Output results in TUFLOW Viewer are described below:
<ol>
<li> Load Map Output results either via:
<ol>
<li> '''[[TUFLOW_Viewer_-_Load_Results | Load All Results]]''', or
<li> '''[[TUFLOW_Viewer_-_Load_Results_-_Map_Outputs | Load Results - Map Outputs]]'''
</ol>
<li> '''[[TUFLOW_Viewer_-_Reload_Results | Reload Results]]''' --> Reload and update results
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Change_Result_Selection | Change Simulation Result Selection]]''' --> Changing between different simulation results is done by selecting the result name(s) in the 'Open Results' widget.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Changing_Result_Type | Change Result Type Selection]]''' --> Changing between different result types is done using the 'Result Type' widget.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Displaying_Maximum | Display Result Maximum]]''' --> If available, maximums can be toggled on/off for the different result types.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Displaying_Vectors | Display Vectors]]''' --> Vector results can be displayed in combination with any of the scalar result types (e.g. velocity vectors can be displayed with depth).
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Styling_Scalar_Types | Style Scalar Map Outputs]]''' --> Styling scalar map output results is similar to styling raster layers in QGIS.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Styling_Vector_Types | Style Vector Map Outputs]]''' --> Similar to the styling the scalar map outputs, there are a range of options for styling the vector layers.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Saving_Default_Styles | Save Default Styles]]''' --> Users can save default styles for result types so they are automatically applied each time results are imported using TUFLOW Viewer.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Displaying_The_Mesh | Display The Mesh]]''' --> The quickest way to toggle the mesh is to click the grid box in TUFLOW Viewer.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_3D_to_2D_Depth_Averaging_Method | 3D to 2D Depth Averaging]]''' --> For 3D map output results, the 3D to 2D depth averaging method can be changed in the layer properties.
<li> '''[[TUFLOW_Viewer_-_Toggling_Between_Output_Timesteps | Lock/Unlock Output Time Steps from Different Result Datasets]]''' --> Locking Plot Output Timesteps to the Map Output Interval.
<li> '''[[TUFLOW_Viewer_-_Isodate_(Date-Time)_Format | Working With Isodate (Date-Time) Format]]''' --> View results using absolute time format (dd/mm/yyyy hh:mm:ss) instead of relative time format (hh:mm:ss).
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Plotting_Time_Series | Map Output Plot - Plotting Time Series]]''' --> Plotting Map Output results.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Plotting_Cross-Sections_And_Longitudinal_Profiles | Map Output Plot - Plotting Cross-Sections and Longitudinal Profiles]]''' --> Plotting Map Output results.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Plotting_Flow | Map Output Plot - Plotting Flow]]''' --> Plotting Map Output results.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Curtain_Plot | Map Output Plot - 3D Curtain Plot]]''' --> Plotting Map Output results.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Vertical_Profile | Map Output Plot - 3D Vertical Profile]]''' --> Plotting Map Output results --> Plotting Map Output results.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_3D_to_2D_Depth_Averaged_Time_Series | Map Output Plot - Plotting 3D to 2D Depth Averaged Time Series]]''' --> Plotting Map Output results.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_3D_to_2D_Depth_Averaged_Cross-Sections | Map Output Plot - Plotting 3D to 2D Depth Averaged Cross-Sections]]''' --> Plotting Map Output results.
<li> '''[[TUFLOW_Viewer_-_Map_Outputs_-_Plotting_From_Vector_Layer | Map Output Plot - Plotting From Vector a Layer (e.g. shp file)]]''' --> Plotting Map Output results.

</ol>
 
[[file: MapOutputs HeaderImg.PNG|650px]] 
 

===Time Series Output===
Time series outputs are typically 1D result outputs or 2D time series results (plot outputs or reporting locations). Time series results consist of two elements, the result datasets and GIS layers that the user can interact with to customise the plot selection in TUFLOW Viewer.
<ol>
<li> Load time series output results either via:
<ol>
<li> '''[[TUFLOW_Viewer_-_Load_Results | Load All Results]]''', or
<li> '''[[TUFLOW_Viewer_-_Load_Results_-_Time_Series | Load Results - Time Series]]''' for TUFLOW models, or:
<li> '''[[TUFLOW_Viewer_-_Load_Results_-_Time_Series_FM | Load Results - Time Series Flood Modeller]]''' for TUFLOW linked Flood Modeller 1D models.
</ol>
<li> '''[[TUFLOW_Viewer_-_Time_Series_Outputs_GIS_Data | Time Series Output - GIS Data]]'''
<li> '''[[TUFLOW_Viewer_-_Time_Series_Outputs_-_Plotting_Time_Series | Time Series Output - Plotting Time Series]]'''
<li> '''[[TUFLOW_Viewer_-_Time_Series_Outputs_-_Plotting_Longitudinal_Profiles | Time Series Output - Plotting Longitudinal Profiles]]'''
<li> '''[[TUFLOW_Viewer_-_Showing_Selected_Elements_And_Selecting_Sub-Sets | Identifying Selected Elements and Selecting Sub-Sets]]'''
<li> '''[[TUFLOW_Viewer_-_Time_Series_Outputs_-_Plotting_1D_Cross-Section_Inputs | Plotting 1D Cross-Section Inputs (with / without results)]]'''
<li> '''[[TUFLOW_Viewer_-_Time_Series_Outputs_-_Plotting_1D_Hydraulic_Table_Check_Files | Plotting 1D Hydraulic Table Check Files]]'''
<li> '''[[TUFLOW_Viewer_-_Extracting_Median_And_Mean_Time_Series | Extracting Median and Mean Time Series - Australian Rainfall and Runoff]]'''
<li> '''[[TUFLOW_Viewer_-_Plotting_1D_Flow_Regime | Plotting 1D Flow Regime]]'''
<li> '''[[TUFLOW_Viewer_-_Import_2D_BC_Tables | Plotting 2D Boundary Condition Tables]]'''
<li> '''[[TUFLOW_Viewer_-_Toggling_Between_Output_Timesteps | Lock/Unlock Output Time Steps from Different Result Datasets]]'''
<li> '''[[TUFLOW_Viewer_-_Isodate_(Date-Time)_Format | Working With Isodate (Date-Time) Format]]'''
</ol> 
[[File: TimeSeries_HeaderImg.PNG|650px]] 
 

===Particle Tracking Output===
<ol>
<li> '''[[TUFLOW_Viewer_-_Particle_Outputs | Particle Tracking Outputs]]'''
<li>'''[[TUFLOW_Viewer_-_Isodate_(Date-Time)_Format | Working With Isodate (Date-Time) Format]]'''
<li> '''[[TUFLOW_Viewer_-_Toggling_Between_Output_Timesteps | Lock/Unlock Output Time Steps from Different Result Datasets]]'''
</ol>
[[File: Particles HeaderImg.PNG|650px]] 

==General Plot Display Options==
<ol>
<li> '''[[TUFLOW_Viewer_-_Summary_of_Plotting_Toolbar | Summary of Plotting Toolbar Options]]''' 
<li> '''[[TUFLOW_Viewer_-_Using_A_Secondary_Axis | Using a Secondary Axis]]'''
<li> '''[[TUFLOW_Viewer_-_Using_A_Date_Axis | Using a Date Axis]]'''
<li> '''[[TUFLOW_Viewer_-_Showing_The_Current_Time | Displaying the Current Time]]'''
<li> '''[[TUFLOW_Viewer_-_Customising_The_Plot_Legend | Customising The Legend]]'''
<li> '''[[TUFLOW_Viewer_-_Customising_The_Plotting_Styles | Customising The Plotting Styles]]'''
<li> '''[[TUFLOW_Viewer_-_Customising_The_Plot_Axes | Customising The Plot Axes]]'''
<li> '''[[TUFLOW_Viewer_-_Toggling_Plot_Grid_Lines | Plot Grid Line Display]]'''
<li> '''[[TUFLOW_Viewer_-_Importing_a_Custom_Colour_Ramp_For_The_Curtain_Plot | Importing a Custom Colour Ramp For The Curtain Plot]]'''
<li> '''[[TUFLOW_Viewer_-_Navigating_And_Querying_The_Plot | Navigating And Querying The Plot]]'''
<li> '''[[TUFLOW_Viewer_-_Auto_Update_Plot_From_Cursor_Location | Auto Update Plot From Cursor Location]]'''
<li> '''[[TUFLOW_Viewer_-_Viewing_The_Vertical_Mesh | 3D Mesh Vertical Layer Display]]'''
<li> '''[[TUFLOW_Viewer_-_Hiding_The_Plotting_Window | Hiding the Plot Window]]'''
<li> '''[[TUFLOW_Viewer_-_Customising_The_Plot_Background_Colour | Customising the Plot Background Colour]]'''
<li> '''[[TUFLOW_Viewer_-_Setting_The_Plot_Default_Font_Size | Setting the Default Font Size For the Plot]]'''
<li> '''[[TUFLOW_Viewer_-_Toggling_Between_Output_Timesteps | Lock/Unlock Output Time Steps from Different Result Datasets]]'''
<li> '''[[TUFLOW_Viewer_-_Changing_Icon_Size | Changing the Icon Size]]'''
<li> '''[[TUFLOW_Viewer_-_Summary_Of_Options | Settings Options]]'''
</ol>

[[File: Plotting_HeaderImg.PNG | 650px ]] 
 

==Exporting Data, Plots, Maps and Animations==
TUFLOW Viewer offers the ability to export data interegation points/lines, maps, plots and animations: 
<ol>
<li> '''[[TUFLOW_Viewer_-_Exporting_An_Animation | Exporting An Animation]]''' 
: [[File: Animation_cover.gif]] 
<li> '''[[TUFLOW_Viewer_-_Exporting_And_Copying_A_Plot | Exporting and Copying a Plot]]'''
<li> '''[[TUFLOW_Viewer_-_Exporting_The_Drawn_GIS_Plot_Features | Exporting The Drawn GIS Plot Points / Lines]]'''
<li> '''[[TUFLOW_Viewer_-_Batch_Exporting_Maps | Batch Exporting Maps]]'''
<li> '''[[TUFLOW_Viewer_-_Batch_Exporting_Plots | Batch Exporting Plots]]'''
</ol>
: [[File: Maps_Cover_Image.PNG | 550px]] 

= Python Error Troubleshooting =
Occasionally the TUFLOW plugin will throw an exception and this will produce a '''Python Error''' which is displayed either as a yellow banner at the top of the map window or a window may appear stating than an 'Error has occurred while executing Python code'. 
[[File: PythonError.PNG]] 
When this occurs it means that the TUFLOW plugin has encountered something unusual or a situation that it does not know how to handle (i.e. it has reached a line in the code that has failed to execute and as a consequence Python has bailed out). This means all the code below this point that was meant to execute has not. This can have knock-on consequences as variables may not exist or be set to incorrect values and signal handling (e.g. what happens when a menu item is clicked) may be broken. As such, a python error can lead to further python errors that would normally not have occurred. Because of this flow-on effect the first python error is usually the most important. 

If you encounter a python error please:
<ol>
<li> Email the '''Stack Trace''' to [mailto:support@tuflow.com support@tuflow.com] with a description of the steps that produced the python error (as best you can describe it). This is to help us identify bugs and fix the plugin so that it catches this exception in the future.
<li> If you find that you are now experiencing further python errors (probably caused by the initial error) you can try the following alternative in order of severity (least to worst):
<ol>
<li> On the TUFLOW Viewer menu bar '''File >> Reload TUFLOW Viewer''' - this will reload TUFLOW Viewer, resetting all variables and signals. You will be required to load in any time series results again and other settings may also be reset. Map output results will remain in the workspace and be reloaded into TUFLOW Viewer.
<li> Save the QGIS workspace (.qgz) and restart QGIS.
<li> Restart QGIS - you can save the workspace (.qgz), however you should first select on the TUFLOW Viewer menu bar '''File >> Close TUFLOW Viewer Completely''' - this will close the Viewer and also remove all settings associated with it from the workspace so that the problematic variable is not accidentally reloaded with the workspace.
<li> The last resort option is to restart QGIS and load and create a new workspace from scratch (do not load a saved workspace).
</ol>
</ol>
 
{{Tips Navigation
|uplink=[[TUFLOW_QGIS_Plugin#Usage| Back to TUFLOW QGIS Plugin Main Page]]
}}
 
 
 
{{Tips Navigation
|uplink=[[Main_Page| Back to Wiki Main Page]]
}}

TUFLOW SWMM Tutorial Introduction

2024-03-04T15:02:51Z

RustyJonesTUFLOW: Add links to TUFLOW-SWMM related documentation

On request from USA XPSWMM users, TUFLOW linkage with 1D EPA Storm Water Management Model (SWMM) has been added as a supported feature in TUFLOW 2023-03-AD release or newer. EPA SWMM is used for simulating urban watershed hydrologic and 1D hydraulic behavior. Initially developed in the early 1970's by the USA Environmental Protection Agency (EPA), the software has since undergone numerous updates and enhancements to become one of the most used 1D stormwater runoff and water quality analysis tools in North America. The primary uses of EPA SWMM include designing and evaluating stormwater systems and also planning and assessment of low-impact developments (also known as Water Sensitive Urban Design). For more details about EPA SWMM, please visit: [https://www.epa.gov/water-research/storm-water-management-model-swmm https://www.epa.gov/water-research/storm-water-management-model-swmm].

 
This EPA SWMM series of tutorials has been written in USA English for the predominantly USA SWMM user base.

= Introduction =
The following tutorial models are aimed at new and experienced users, stepping through the process of building and running TUFLOW SWMM together. The main objective aims to demonstrate the ability for TUFLOW to link to 1D EPA SWMM in a variety of different common use case scenarios.

'''Note, if you are not already familiar with TUFLOW, we recommend first completing Module 1 and 2 of the [[Tutorial_Introduction | "TUFLOW Only" Tutorials]] to establish an understanding of 2D modeling, before progressing to learning how to link 1D EPA SWMM to 2D TUFLOW.''' 
 TUFLOW has supported 1D/2D dynamic coupling since its initial 2D solver development in 1989. Traditionally, 1D (1 Dimensional) linking and associated modeling has been applied using TUFLOW's 1D solver, ESTRY. As demonstrated in:
:*[[Tutorial_M03 | Module 3 - 1D Culverts]]
:*[[Tutorial_M05 | Module 5 - Integrated Urban Pipe Network Drainage]]
:*[[Tutorial_M11 | Module 11 - 1D Open Channel]] 
EPA SWMM's inclusion in TUFLOW does not replace ESTRY, rather both 1D solvers complement each other. Links to the TUFLOW SWMM tutorials are included in the section below.

=Software Requirements=
TUFLOW uses QGIS as its Graphical User Interface (GUI), in combination with text editor and spreadsheet software, for its model creation and result viewing. The group of software creates an extremely workflow efficient and flexible modeling environment. Notably, QGIS does not experience the data load/visualization lag and display issues some other hydraulic modeling software GUI's are limited by when working with larger datasets. 

{| class="wikitable" width="75%"

! style="background-color:#005581; font-weight:bold; color:white;"| Requirement
! style="background-color:#005581; font-weight:bold; color:white;" | Brief Description
! style="background-color:#005581; font-weight:bold; color:white;" | Download
|-
| '''TUFLOW''' || TUFLOW is a computer program for simulating depth-averaged, one and two-dimensional free-surface flows such as occurs from floods and tides, with the 2D solution occurring over a regular grid or quadtree mesh of square elements. 
It is recommended to always use the latest release version of TUFLOW. 

This tutorial model does not require a TUFLOW license. 

This tutorial is configured to use a NVIDIA GPU card. If this is not available, your CPU can be used via the Hardware command. 
||The TUFLOW executable is provided within the [[TUFLOW_SWMM_Tutorial_Introduction#Tutorial_Data | Tutorial Download Dataset]].
|-
| '''QGIS''' QGIS TUFLOW plugin || QGIS is the Geographic Information System (GIS) Graphical User Interface (GUI) used to build models and view results. The QGIS TUFLOW plugin includes numerous tools to increase workflow efficiency.

This tutorial was developed using QGIS 3.34 and QGIS TUFLOW Plugin 3.9.0.110. Please use these or newer versions of QGIS and the plugin. Earlier releases will not include the TUFLOW SWMM automation tools demonstrated in the Tutorials.

||[https://qgis.org/en/site/forusers/download.html Latest 64-bit version of QGIS]. 

[[TUFLOW_QGIS_Plugin| QGIS TUFLOW Plugin Installation]].
|-
| '''NotePad++''' Syntax Highlighting || A text editor is required for creation of the TUFLOW control (script) files. This tutorial was developed with NotePad++. Ideally a text editor should be able to: 
*Color code the TUFLOW control files;
*Open other files from the active control file; and
*Launch a TUFLOW simulation. 
TUFLOW color coding can be enabled using syntax highlighting.
|| [https://notepad-plus-plus.org/downloads/ Latest 64-bit version of Notepad++]. 

[https://www.tuflow.com/Download/Miscellaneous/NPP_TUFLOW_Syntax_Highlighting.zip TUFLOW syntax highlighting for Notepad++]. 

For instructions on configuring Notepad++ for TUFLOW modeling, see [[NotepadPlusPlus_Tips |Notepad++ tips]].
|-
| '''Microsoft Excel''' || A spreadsheet software is required for working with tabular data and .csv files. This tutorial has been created using Excel. ||
|}

 

=Tutorial Data=
To build the tutorial model, download the dataset below. This includes a digital elevation model (DEM), aerial photography, background model data for the tutorial model and a working version of the model. GeoPackage database has been used as the GIS format for the model build tasks. For tips on using GeoPackage databases see [https://wiki.tuflow.com/GeoPackage_Tips GeoPackage Tips].
:*[https://downloads.tuflow.com/TUFLOW/Wiki_Tute_Models/TUFLOW_SWMM_Tutorial_Models_QGIS_GPKG.zip TUFLOW SWMM Tutorial Dataset - QGIS - GPKG]

=Tutorial Modules=
The download dataset contains the input files and working version of the tutorial model for reference. Results and check files are not included to keep the size of the download file manageable; they will be created when running the model simulations. The folder should be placed in a location with write permissions. 

The tutorial model is presented in several modules. All modules can be completed independently, however new users are encouraged to undertake the modules in sequence:

:*[[TUFLOW_SWMM_Tutorial_M01 | TUFLOW SWMM Module 1]] - 1D SWMM Culverts
:*[[TUFLOW_SWMM_Tutorial_M02 | TUFLOW SWMM Module 2]] - 1D SWMM Pipe Network / 2D TUFLOW Direct Rainfall Hydrology
:*[[TUFLOW_SWMM_Tutorial_M03 | TUFLOW SWMM Module 3]] - 1D SWMM Pipe Network / 1D SWMM Urban Hydrology
:*[[TUFLOW_SWMM_Tutorial_M04 | TUFLOW SWMM Module 4]] - 1D SWMM Pipe Network / 1D SWMM Urban Hydrology: Executing multiple different event simulations from a single model control file.
:*[[XPSWMM_to_TUFLOW-SWMM | XPSWMM to TUFLOW SWMM]] - How to convert an XPSWMM model to TUFLOW SWMM.

== TUFLOW-SWMM Documentation ==
Documentation can be found in the 2023-03-AD release notes for TUFLOW-SWMM and in the TUFLOW plugin documentation for the related QGIS tools.

* [https://docs.tuflow.com/classic-hpc/release/2023-03-AD/ TUFLOW 2023-03-AD Release notes - SWMM Section]
* [[TUFLOW QGIS Plugin#Usage|TUFLOW QGIS Plugin Documentation including SWMM tools]]

=Recommended Further Reading=
For users who wish to get a better understanding of either 1D EPA SWMM or 2D TUFLOW, the following resources may be of use:
 
'''EPA SWMM''': [https://www.epa.gov/water-research/storm-water-management-model-swmm www.epa.gov]
*[https://downloads.tuflow.com/SWMM/SWMM5_Reference_Manual_Volume1_Hydrology_P100NYRA.pdf SWMM5 Reference Manual - Volume 1 (Hydrology)]
*[https://downloads.tuflow.com/SWMM/SWMM5_Reference_Manual_Volume2_Hydaulics_P100S9AS.pdf SWMM5 Reference Manual - Volume 2 (Hydraulics) ]
*[https://downloads.tuflow.com/SWMM/SWMM5_Reference_Manual_Volume2_Hydraulics_Addendum-20220210mas2wr.pdf SWMM5 Reference Manual - Volume 2 (Hydraulics Addendum) ]
*[https://downloads.tuflow.com/SWMM/SWMM5_Reference_Manual_Volume3_Water_Quality_P100P2NY.pdf SWMM5 Reference Manual - Volume 3 (Water Quality) ]
*[https://downloads.tuflow.com/SWMM/SWMM5_User%27s_Manual.pdf EPA SWMM5 User's Manual]
 
'''TUFLOW''': [https://www.tuflow.com www.tuflow.com]
*[https://www.tuflow.com/downloads/#tuflow TUFLOW User Manual]
*[https://docs.tuflow.com/classic-hpc/release/2023-03-AD TUFLOW 2023-03-AD release notes]
*[[Tutorial_Introduction#Tutorial_Modules | TUFLOW Tutorial Models]]

{{Tips Navigation
|uplink=[[Main_Page| Back to Wiki Main Page]]
}}

TUFLOW Message 1455

2024-02-23T11:49:48Z

RustyJonesTUFLOW: Added information for TUFLOW ERROR 1455

{{TUFLOW_Message
|tuflow_message=ERROR 1455 - "Channel has the same starting and ending node which is invalid".
|alt_msg=NA 

|type=[[ERROR]]
|message_desc=It is invalid for a 1D channel to start and end at the same node. This error indicates that there is a problem with the network that needs to be addressed.
|suggestions=Connect the channels to the proper nodes or delete if accidentally created.

|uplink=[[1xxx_TUFLOW_Messages|1xxx Messages]]
}}

QGIS Installation with OSGeo4W

2023-11-01T15:28:38Z

RustyJonesTUFLOW: Added libraries required by the SWMM tools

=Installation Steps=
<ol>
<li> Download OSGeo4W setup installer
: [https://qgis.org/en/site/forusers/download.html https://qgis.org/en/site/forusers/download.html] 
: [[File: osgeo4w_download_link.PNG]]
<li> Run the downloaded EXE
<li> Select '''Advanced Install'''
: [[File: osgeo4w_installation_1.PNG]]
<li> Select '''Install from Internet'''
: [[File: osgeo4w_installation_2.PNG]]
<li> Select a directory and other installation options (Note some options may require admininstrator rights)
: [[File: osgeo4w_installation_3.PNG]]
<li> Click '''Next'''
: [[File: osgeo4w_installation_4.PNG]]
<li> Select '''Direct Connection''' (Note some firewall systems will require proxy settings)
: [[File: osgeo4w_installation_5.PNG]]
<li> Select a download site. It is recommended to use with one with an SSL certificate (i.e. starts with https://)
: [[File: osgeo4w_installation_6.PNG]]
<li> Use the search bar to search for "QGIS". Expand the '''Desktop''' dropdown and find "qgis: QGIS Desktop" under the '''package''' column. Left-click the "skip" text and the text will change to the latest version number. Once the verseion number is showing, this will be the version installed when you proceed. If you left-click again, the version number may change again to the previous version. If you keep left-clicking it will cycle back to "skip". (Note: you could also choose the "qgis-ltr: QGIS Desktop (long term release)" instead, and this would install the long term release version).
: [[File: osgeo4w_installation_7.PNG]]
<li> The installation also requires manually choosing several Python libraries. Use the search bar again and search for "scipy". Expand the '''Libs''' dropdown and find "python3-scipy". Left-click the "skip" text once to choose the latest scipy library version.
: [[File: osgeo4w_installation_8.PNG]]
<li> Repeat the above step for the following Python Libraries:
* fiona
* geopandas
* matplotlib
* netcdf4
* pandas (this may return 2 options - make sure to select "python3-pandas")
<li> Click '''Next'''
<li> Continue to work through the steps of the installation process - the installer will ask you to also install a number of dependencies (make sure to click yes to these) and prompt for license agreements
<li> Once installed, QGIS should be installed in the location specified in an earlier step
</ol>
 
 
=Upgrading Steps=
<ol>
<li> As part of the installation, OSGeo4W should install a program called "setup" which you can search in the Windows start bar. You can also run the downloaded OSGeo4W installer again (they are the same).
<li> Once running the setup, simply continue clicking '''Next'''. The setup should remember all your previous settings and automatically choose the latest version of the installed libraries at the '''Packages Selection''' stage of the installation
</ol>
 
==Upgrading Silently==
It's possible to upgrade silently from the Windows CMD prompt or from a batch file. 
 
===From CMD===
<ol>
<li> Search "osgeo4w" in the Windows start bar and choose '''OSGeo4W Shell'''
<li> Type the following command and press enter
<pre>setup -q -g</pre>
: <tt>"-q"</tt> : quiet installation flag
: <tt>"-g"</tt> : upgrade installed packages flag
</ol>
 
===From BAT===
<ol>
<li> In the OSGeo4W installation folder under the "bin" folder find the following batch file "o4w_env.bat" (e.g. "C:\OSGeo4W\bin\o4w_env.bat").
<li> In your batch file call the "o4w_env.bat" file
<li> On the next line, type "setup -q -g"
: <tt>"-q"</tt> : quiet installation flag
: <tt>"-g"</tt> : upgrade installed packages flag
<li> Your final batch file should look like the below with the path to "o4w_env.bat" customised to match your installation
<pre>
call C:\OSGeo4W\bin\o4w_env.bat
setup -q -g
</pre>
<li> Run the batch file
</ol>
 
 
{{Tips Navigation
|uplink=[[QGIS_Tips| Back to QGIS Tips page]]
}}