Difference between revisions of "1D Integrity Tool - Tutorial"

From Tuflow
Jump to navigation Jump to search
 
(22 intermediate revisions by 3 users not shown)
Line 11: Line 11:
 
The following exercises will take the user through the various tools using a pipe drainage network which contains a number of common errors which are found in these types of datasets.
 
The following exercises will take the user through the various tools using a pipe drainage network which contains a number of common errors which are found in these types of datasets.
  
=Loading the Data=
+
=Accessing the Tutorial Data=
  
 
<ol>
 
<ol>
<li> Firstly download the tutorial dataset from the following link and unzip the dataset.</li>
+
1. Firstly download the tutorial dataset from [https://tuflowdownload.blob.core.windows.net/$web/TUFLOW/Wiki_Tute_Models/Integrity_Tool_Tutorial.zip here] and unzip the dataset.<br>
<li> From the '''Integrity_Tool_Tutorial\Datafolder''' load the '''1d_nwk_MI01_Pipes_001_L.shp''' , '''1d_nwk_MI01_Pipes_001_L.shp''' and '''dem_DEV_EG07''' into QGIS. </li>  
+
2. From the '''Integrity_Tool_Tutorial\Data''' folder, load the '''1d_nwk_MI01_Pits_001_L.shp''' , '''1d_nwk_MI01_Pipes_001_L.shp''' and '''dem_DEV_EG07.asc''' into QGIS.<br>  
<li> Review the dataset and use the [[QGIS TUFLOW Styles | ‘Apply TUFLOW Styles to open layers’]] tool, [[File:Check files open.png||30px]], to theme the data.</li><br>
+
3. Review the dataset and use the [[QGIS TUFLOW Styles | ‘Apply TUFLOW Styles to open layers’]] tool, [[File:Check files open.png||30px]], to theme the data.<br>
 
[[File:Sub-surface drainage overview.png]]<br>
 
[[File:Sub-surface drainage overview.png]]<br>
 
+
</ol>
 
This is a small sub-surface drainage network model.  This example will showcase use of the tools to check the integrity of such a dataset, however the tool really comes into it’s own when assessing much larger datasets where checking things manually would be a time-consuming task.<br>
 
This is a small sub-surface drainage network model.  This example will showcase use of the tools to check the integrity of such a dataset, however the tool really comes into it’s own when assessing much larger datasets where checking things manually would be a time-consuming task.<br>
<li> From the TUFLOW plugin, open the 1D Integrity tool.</li><br>
+
<ol>
 +
4. From the TUFLOW plugin, open the 1D Integrity tool.<br>
  
 
[[File: integrity_tool_in_toolbar.PNG]].<br>
 
[[File: integrity_tool_in_toolbar.PNG]].<br>
 
+
</ol>
 
The resulting dialogue will pop up.  The numbered fields are described below.<br>
 
The resulting dialogue will pop up.  The numbered fields are described below.<br>
 
+
<ol>
 
[[File:integrity_tool_overview.PNG]]<br>
 
[[File:integrity_tool_overview.PNG]]<br>
 
<br>
 
<br>
*'''1. Input Network Lines''' - input location of 1d_nwk lines such as pipes, culverts, and open channels. Use dropdown box and '+' button to add layers as inputs to tool.<br>
+
*'''1. Input Network Lines''' - Input location of 1d_nwk lines such as pipes, culverts, and open channels. Use dropdown box and '+' button to add layers as inputs to tool.<br>
*'''2. Input Network Points''' - input location of 1d_nwk points such as pits and nodes. Use dropdown box and '+' button to add layers as inputs to tool.<br>
+
*'''2. Input Network Points''' - Input location of 1d_nwk points such as pits and nodes. Use dropdown box and '+' button to add layers as inputs to tool.<br>
*'''3. Input Tables''' - input location of 1d_ta lines such as 1d_xs, 1d_cs, and 1d_hw. Use dropdown box and '+' button to add layers as inputs to tool.<br>
+
*'''3. Input Tables''' - Input location of 1d_ta lines such as 1d_xs, 1d_cs, and 1d_hw. Use dropdown box and '+' button to add layers as inputs to tool.<br>
*'''4. Input DEM''' - input raster. Currently only 1 input is supported, so if you are using tiled rasters datasets, they would need to be mosaiced into a single raster prior to running the tool.<br>
+
*'''4. Input DEM''' - Input raster. Currently only 1 input is supported, so if you are using tiled rasters datasets, they would need to be mosaiced into a single raster prior to running the tool.<br>
*'''5. Tool Selection''' - the active tab will determine the active tool. Prior to plugin version 3.2 a radio button was used to select the active tool. See [[#Tool_Documentation|Tool Documentation]] for information on each tool.<br>
+
*'''5. Tool Selection''' - The active tab will determine the active tool. Prior to plugin version 3.2 a radio button was used to select the active tool. See <u>[[1D_Integrity_Tool#Tool_Documentation|Tool Documentation]]</u> for information on each tool.<br>
*'''6.  Status Bar''' - contains the progress bar and status of the integrity tool. Includes information such as the current active task being performed (if one of the tools is running), and the status of the previously run tool.<br>
+
*'''6.  Status Bar''' - Contains the progress bar and status of the integrity tool. Includes information such as the current active task being performed (if one of the tools is running), and the status of the previously run tool.<br>
*'''7.  Run''' - runs the currently active tool (active tool determined by the current active tool tab - prior to plugin version 3.2 a radio button was used to select the active tool).<br>
+
*'''7.  Run''' - Runs the currently active tool (active tool determined by the current active tool tab - prior to plugin version 3.2 a radio button was used to select the active tool).<br>
 
<br>
 
<br>
<li> Within the Input Network Lines tool, select the '''1d_nwk_MI01_Pipes_001_L.shp''' in the dropdown menu and click the ‘+’ icon to add the '''1d_nwk_MI01_Pipes_001_L.shp''' to the list of 1D network lines as below.</li><br>
+
 
 +
5. Within the Input Network Lines tool, select the '''1d_nwk_MI01_Pipes_001_L.shp''' in the dropdown menu and click the ‘+’ icon to add the '''1d_nwk_MI01_Pipes_001_L.shp''' to the list of 1D network lines as below.<br>
  
 
[[File:Input Network Lines.png]]<br>   
 
[[File:Input Network Lines.png]]<br>   
 
This means we will be undertaking the integrity tests on this layer.  We could also use the integrity tools on point layers and/or cross-section/Height/Width tables but for the purpose of this demonstration we will only use the network line layers.
 
 
</ol>
 
</ol>
  
=1D integrity Tool=
+
This will mean we will be undertaking the integrity tests on this layer.  We could also use the integrity tools on point layers and/or Cross-section/Height-Width tables, but for the purpose of this demonstration we will only use the network line layers.
==Empty Geometry Check==
 
  
The first tool we’ll look at is the Empty Geometry Check tool.  This tool loops through the inputs file and identifies any features that have an empty or NULL geometry.  Should any null geometries be encountered than there is an option to automatically delete those features.
+
=Using the 1D Integrity Tool=
  
 +
The following pages go through the various checks available within the 1D Integrity Tools.  For the purpose of running through the tutorial these pages, and the steps within them, should be undertaken in order.
 
<ol>
 
<ol>
<li> Navigate to the bottom of the 1D integrity tool and select the ‘Empty Geometry’ tabOnce selected click ‘Run’ to initiate the tool.</li>
+
<li> <u>[[Integrity_Tool_-_Empty_Geometry_Check|Empty Geometry Check]]</u> --> This tool loops through the inputs file and identifies any features that have an empty or NULL geometry.
 +
<li> <u>[[Integrity_Tool_-_Channel_ID|Channel ID Check]]</u> -->Checks the ID of each channel in the network and flag channels that have duplicate names or a NULL / empty ID.
 +
<li> <u>[[Integrity_Tool_-_Snapping_Check|Snapping Check]]</u> -->The snapping tool checks that all input 1D network lines (channels) and 1D network points are correctly snapped.
 +
<li> <u>[[Integrity_Tool_-_Pipe_Direction|Pipe Direction Check]]</u> -->The pipe direction integrity tool will check the pipe direction and will correct based on the selected options.
 +
<li> <u>[[Integrity_Tool_-_Flow_Trace_Tool|Flow Trace Tool]]</u> -->The flow trace tool creates traces upstream from the selected channel, selecting all upstream network objects in the QGIS map window (network lines and network points).
 +
<li> <u>[[Integrity_Tool_-_Continuity_Tool|Continuity Tool]]</u> -->The continuity 1D integrity tool will check all 1D inputs for downstream continuity based on the selected processing optionsThis is similar to the above Flow trace check but will undertake the assessment on the whole set of inputs rather than upstream of a selected link.
 +
</ol>
  
[[File:Empty Geometry.png]]<br>
+
=Summary=
 
+
The 1D integrity tool provides a set of checks and fixes to ensure that the 1D input data is suitable to be used within a TUFLOW hydraulic modelThe checks assess common errors within 1D datasets which will either cause TUFLOW errors, or influence resultsCommonly these common errors are a consequence of digitisation and given the nature of large urban drainage networks are difficult to spot visually.  The automated, 1D integrity tools provide a quick and easy approach to allow the modeller to be proactive in identifying problematic parts of a 1D system prior to loading into TUFLOW.
The tool should highlight two features which have null geometry, one with an empty ID and the other called SKSM_37. <br>
+
<Br>
 
+
<br>
[[File:Empty Geometry Found.png]]<br>
+
{{Tips Navigation
 
+
|uplink=[[1D_Integrity_Tool| Back to 1D Integrity Tool Main Page]]
<li> At this point we’ll click cancel, we’ll come back to delete the items from the layer. </li>
+
}}
<li> Open the attribute table for the 1d_nwk_MI01_Pipes_001_L.shp file.  Find the SKSM_37 feature, right click on it and select ‘Zoom to feature’.  You should get an error reporting that the ‘Feature does not have a geometry’Do the same for the field which has a ‘null’ ID and you will receive a similar error message.</li>
 
 
 
[[File:Null geometry error.png]]<br>
 
 
 
The presence of these null geometries would cause an error when running TUFLOW.  Rather than wait for TUFLOW to pre-process and generate the error, the 1D integrity tool allows the user to be more proactive in checking the geometry. 
 
 
 
<li> Go back to the 1D Integrity tool, ensure the ‘Empty Geometry’ tab is selected and again click ‘Run’.  This time, in the resulting dialogue, select the option to ‘Confirm Delete’ to delete the two items.  This will generate a new layer called '''1d_nwk_MI01_Pipes_001_L_EG1''' which does not contain the wo null geometries.</li>
 
 
 
A new dialogue will pop up as per the below.</li>
 
[[File:Replace Inputs.png]]<br>
 
<li> Click Yes to update the Input Network Lines to the new layer.</li>
 
[[File:Input Network Lines updated.png]]<br>
 
 
 
<li> Review the attributes table for the '''1d_nwk_MI01_Pipes_001_L_EG1''' layer to ensure deletion of the two features identified in step 4.</li>
 
 
 
''Note: Whenever the 1D Integrity makes changes to an input layer it will always output a new temporary layer with a unique suffix. It will never overwrite or edit the input layers directly. Each tool outputs a different suffix and serves as somewhat of a log to record which tools have been run and which order they were run in. The temporary layers only exist in memory and have not been written to the disk anywhere, this means that they will be lost if they are removed from the workspace or QGIS is closed.''
 
</ol>
 

Latest revision as of 11:27, 5 April 2024



Introduction

The 1D Integrity Tool is a suite of tools that help modellers find and fix potential errors in 1D networks prior to running TUFLOW. The tool was primarily designed for sub-surface pipe drainage networks, but can be used for all 1D channel types, however not all checks will be available. The tool is designed to be fast and provide automated fixes as well as visualisations to help the process. The 1D Integrity Tool is a free tool that comes as part of the TUFLOW plugin in QGIS.

The 1D Integrity tool is designed to accept multiple inputs and will treat all inputs holistically as a complete system. Each tool is designed to be run separately and progressively as part of the entire tool suite. It is assumed that most of the channel attribute data has been populated, most importantly that each channel has been given a valid type.

The following exercises will take the user through the various tools using a pipe drainage network which contains a number of common errors which are found in these types of datasets.

Accessing the Tutorial Data

    1. Firstly download the tutorial dataset from here and unzip the dataset.
    2. From the Integrity_Tool_Tutorial\Data folder, load the 1d_nwk_MI01_Pits_001_L.shp , 1d_nwk_MI01_Pipes_001_L.shp and dem_DEV_EG07.asc into QGIS.
    3. Review the dataset and use the ‘Apply TUFLOW Styles to open layers’ tool, Check files open.png, to theme the data.
    Sub-surface drainage overview.png

This is a small sub-surface drainage network model. This example will showcase use of the tools to check the integrity of such a dataset, however the tool really comes into it’s own when assessing much larger datasets where checking things manually would be a time-consuming task.

    4. From the TUFLOW plugin, open the 1D Integrity tool.
    Integrity tool in toolbar.PNG.

The resulting dialogue will pop up. The numbered fields are described below.

    Integrity tool overview.PNG

    • 1. Input Network Lines - Input location of 1d_nwk lines such as pipes, culverts, and open channels. Use dropdown box and '+' button to add layers as inputs to tool.
    • 2. Input Network Points - Input location of 1d_nwk points such as pits and nodes. Use dropdown box and '+' button to add layers as inputs to tool.
    • 3. Input Tables - Input location of 1d_ta lines such as 1d_xs, 1d_cs, and 1d_hw. Use dropdown box and '+' button to add layers as inputs to tool.
    • 4. Input DEM - Input raster. Currently only 1 input is supported, so if you are using tiled rasters datasets, they would need to be mosaiced into a single raster prior to running the tool.
    • 5. Tool Selection - The active tab will determine the active tool. Prior to plugin version 3.2 a radio button was used to select the active tool. See Tool Documentation for information on each tool.
    • 6. Status Bar - Contains the progress bar and status of the integrity tool. Includes information such as the current active task being performed (if one of the tools is running), and the status of the previously run tool.
    • 7. Run - Runs the currently active tool (active tool determined by the current active tool tab - prior to plugin version 3.2 a radio button was used to select the active tool).

    5. Within the Input Network Lines tool, select the 1d_nwk_MI01_Pipes_001_L.shp in the dropdown menu and click the ‘+’ icon to add the 1d_nwk_MI01_Pipes_001_L.shp to the list of 1D network lines as below.
    Input Network Lines.png

This will mean we will be undertaking the integrity tests on this layer. We could also use the integrity tools on point layers and/or Cross-section/Height-Width tables, but for the purpose of this demonstration we will only use the network line layers.

Using the 1D Integrity Tool

The following pages go through the various checks available within the 1D Integrity Tools. For the purpose of running through the tutorial these pages, and the steps within them, should be undertaken in order.

  1. Empty Geometry Check --> This tool loops through the inputs file and identifies any features that have an empty or NULL geometry.
  2. Channel ID Check -->Checks the ID of each channel in the network and flag channels that have duplicate names or a NULL / empty ID.
  3. Snapping Check -->The snapping tool checks that all input 1D network lines (channels) and 1D network points are correctly snapped.
  4. Pipe Direction Check -->The pipe direction integrity tool will check the pipe direction and will correct based on the selected options.
  5. Flow Trace Tool -->The flow trace tool creates traces upstream from the selected channel, selecting all upstream network objects in the QGIS map window (network lines and network points).
  6. Continuity Tool -->The continuity 1D integrity tool will check all 1D inputs for downstream continuity based on the selected processing options. This is similar to the above Flow trace check but will undertake the assessment on the whole set of inputs rather than upstream of a selected link.

Summary

The 1D integrity tool provides a set of checks and fixes to ensure that the 1D input data is suitable to be used within a TUFLOW hydraulic model. The checks assess common errors within 1D datasets which will either cause TUFLOW errors, or influence results. Commonly these common errors are a consequence of digitisation and given the nature of large urban drainage networks are difficult to spot visually. The automated, 1D integrity tools provide a quick and easy approach to allow the modeller to be proactive in identifying problematic parts of a 1D system prior to loading into TUFLOW.

Up
Go-up.png Back to 1D Integrity Tool Main Page