Difference between revisions of "Convert TUFLOW Model GIS Format"

From Tuflow
Jump to navigation Jump to search
 
(19 intermediate revisions by one other user not shown)
Line 8: Line 8:
 
[[File: convert_gis_format_in_panel.PNG]]<Br>
 
[[File: convert_gis_format_in_panel.PNG]]<Br>
  
'''Parameters'''<Br>
+
====Parameters====
''<B>TCF</b>''<Br>
+
=====TCF=====
 
Location of an existing TUFLOW model TCF file. This is the TCF of the model that will be converted.<br>
 
Location of an existing TUFLOW model TCF file. This is the TCF of the model that will be converted.<br>
  
''<b>Output Vector Format</b>''<br>
+
=====Output Vector Format=====
 
Output vector format of the converted model. Supported formats are ''GPKG, SHP, MIF''.<br>
 
Output vector format of the converted model. Supported formats are ''GPKG, SHP, MIF''.<br>
  
''<b>Output Raster Format</b>''<Br>
+
=====Output Raster Format=====
 
Output raster format of the converted model. Supported formats are ''GTIFF, GPKG, ASC, FLT''.<Br>
 
Output raster format of the converted model. Supported formats are ''GTIFF, GPKG, ASC, FLT''.<Br>
  
''<b>Output Profile</b>''<Br>
+
=====Output Profile=====
 
This parameter is only used for GPKG format outputs. GPKG is a database and can store multiple layers inside one file. This option determines how the output GPKG layers will be grouped into databases.<br>
 
This parameter is only used for GPKG format outputs. GPKG is a database and can store multiple layers inside one file. This option determines how the output GPKG layers will be grouped into databases.<br>
  
Line 37: Line 37:
 
Optional parameter specifying the output location of the converted model. By default the output location will a new folder with the same name as the input TCF located in the same folder as the input TCF.<Br>
 
Optional parameter specifying the output location of the converted model. By default the output location will a new folder with the same name as the input TCF located in the same folder as the input TCF.<Br>
  
''<b>Root Folder</b>''<br>
+
=====Root Folder=====
 
By default the tool will try and find the 'root' folder containing the model files (commonly this folder is named 'TUFLOW'). The tool will try and guess the root folder based on the location of the TCF and the directory structure. If it can't find the root folder, or it picks the wrong folder then this parameter can be used to override this location within the tool.<Br>
 
By default the tool will try and find the 'root' folder containing the model files (commonly this folder is named 'TUFLOW'). The tool will try and guess the root folder based on the location of the TCF and the directory structure. If it can't find the root folder, or it picks the wrong folder then this parameter can be used to override this location within the tool.<Br>
  
''<b>Scenarios</b>''<br>
+
=====Scenarios=====
 
''Available since v3.6''<Br>
 
''Available since v3.6''<Br>
The conversion can be restricted by specifying scenario and event names. Using this option will stop the conversion of layers that are within an 'IF Scenario/IF Event' logic blocks if the scenario name is not provided by the user. This option will also clean the control files and remove unused scenario names. Event names can also be used, however are only applied to logic blocks in the control files and does not filter event sources provided in the TUFLOW Event File (.tef).
+
The conversion can be restricted by specifying scenario and event names. Using this option will stop the conversion of layers that are within an 'IF Scenario/IF Event' logic blocks if the scenario name is not provided by the user. This option will also clean the control files and remove unused scenario names. Event names can also be used, however they are only applied to logic blocks in the control files and are not used to filter event sources provided in the TUFLOW Event File (.tef).
 +
 
 +
To use this option:
 +
<ol>
 +
<li> Ensure 'Restrict Conversion by Scenario' is checked on
 +
<li> List scenario and event names in the text edit. This should be done in the same manner as TUFLOW batch files e.g. <tt>-s <scenario-name> -e <event-name></tt>. The order does not matter and numbering scenarios is not required, although is accepted (e.g. -s1 -s2).
 +
</ol>
 +
 
 +
=====GPKG Name=====
 +
''Available since v3.7''<br>
 +
Sets the output GPKG database name. Will only be used when outputing to GPKG and using a grouped output profile. If using a 'grouped by control file' profile, then the control file extension will still be added to the name e.g. <output-name>_TCF.gpkg.<Br>
 +
 
 +
=====Write Empty Files=====
 +
''Available since v3.7''<br>
 +
If this option is checked on, the tool will create TUFLOW empty files as part of the conversion. The location of the empty directory will be guessed by the tool unless the user provides a relative path in the <b><tt>Write Empty Files Relative Directory</tt></b> input parameter. The path provided should be a relative path from the model root directory (e.g. commonly this is a folder called "TUFLOW", so a relative path might be <tt>model\gis\empty</tt>). The projection of the empty files will be based on the first <tt>"Projection == "</tt> command found in the TCF. If no <tt>"Projection == "</tt> is found, then empty files will not be created.<Br>
 +
 
 +
=====Output CRS=====
 +
''Available since v3.7''<br>
 +
Forces output files to be in given projection (no reprojection or warping is performed). Useful when converting a model with a combination of SHP and MIF files as these can generate slightly different projection definitions. Set blank to turn off.
 +
 
 +
=====Force TUFLOW Directory Structure=====
 +
''Available since v3.10''<br>
 +
Forces the output files to be placed in a default TUFLOW folder structure, ignoring the input folder structure. If this setting is on, the "Log Folder", "Output Folder", and "Write Check Files" commands will be updated to also reference standard TUFLOW locations.
  
 
===Checking the Output===
 
===Checking the Output===
Line 54: Line 76:
 
A known conversion issue could occur if the tool is converting both MIF and SHP files into a single format (e.g. GPKG) as the projection of these files may be interpreted slightly differently by GDAL (even though they are the same). Users may therefore be required to manually update the projection of the affected files, or add the TUFLOW command "<tt>GIS Projeciton Check == WARNING</tt>" to the TCF. '''Be careful''' to ensure they are in fact the same projection before using this command.<Br>
 
A known conversion issue could occur if the tool is converting both MIF and SHP files into a single format (e.g. GPKG) as the projection of these files may be interpreted slightly differently by GDAL (even though they are the same). Users may therefore be required to manually update the projection of the affected files, or add the TUFLOW command "<tt>GIS Projeciton Check == WARNING</tt>" to the TCF. '''Be careful''' to ensure they are in fact the same projection before using this command.<Br>
  
===Python and the Soure Code===
+
===Python and the Source Code===
 
It's possible to run the tool outside of QGIS in Python. The tool uses GDAL to read and write the GIS files and does not rely on any QGIS libraries. The source code and instructions on how to run the tool in Python can be found at the following location on the '''<u>[https://gitlab.com/tuflow-user-group/tuflow/model-conversions/convert_tuflow_model_gis_format TUFLOW Community GitLab page]</u>'''.
 
It's possible to run the tool outside of QGIS in Python. The tool uses GDAL to read and write the GIS files and does not rely on any QGIS libraries. The source code and instructions on how to run the tool in Python can be found at the following location on the '''<u>[https://gitlab.com/tuflow-user-group/tuflow/model-conversions/convert_tuflow_model_gis_format TUFLOW Community GitLab page]</u>'''.
 
<Br>
 
<Br>

Latest revision as of 20:41, 22 December 2023

This tool will convert a TUFLOW model's GIS format (vector and raster layers) into a user specified, TUFLOW Supported, format. It works very similarly to the package model functionality that exists in TUFLOW and will try and package files from all scenarios/events. The difference between the package model functionality and this tool is that this tool will perform additional format conversion steps and update relevant control files to reflect these conversions.

Available since TUFLOW Plugin v3.3

Running the Tool

This tool can be found in the QGIS processing toolbox panel under 'TUFLOW'. To open the Processing Toolbox Panel, navigate to 'View' in the menubar and select Panels >> Processing Toolbox Panel.
Processing toolbox panel toggle.PNG

Convert gis format in panel.PNG

Parameters

TCF

Location of an existing TUFLOW model TCF file. This is the TCF of the model that will be converted.

Output Vector Format

Output vector format of the converted model. Supported formats are GPKG, SHP, MIF.

Output Raster Format

Output raster format of the converted model. Supported formats are GTIFF, GPKG, ASC, FLT.

Output Profile

This parameter is only used for GPKG format outputs. GPKG is a database and can store multiple layers inside one file. This option determines how the output GPKG layers will be grouped into databases.

Options:

  • SEPARATE

This option will write each GPKG layer into its own database. Where the tool encounters commands that have multiple geometry types read in on the same line (using the vertical line delimiter "|"), the layers will be combined into a single database.

  • GROUP BY CONTROL FILE 1

This option will group layers into databases defined by the control file they were specified in. In this option, TRD and TEF files are not considered separate control files.

  • GROUP BY CONTROL FILE 2

Similar to "GROUP BY CONTROL FILE 1" except will treat TRD and TEF as separate control files.

  • ALL IN ONE

Groups all layers together into one database.

Output Folder
Optional parameter specifying the output location of the converted model. By default the output location will a new folder with the same name as the input TCF located in the same folder as the input TCF.

Root Folder

By default the tool will try and find the 'root' folder containing the model files (commonly this folder is named 'TUFLOW'). The tool will try and guess the root folder based on the location of the TCF and the directory structure. If it can't find the root folder, or it picks the wrong folder then this parameter can be used to override this location within the tool.

Scenarios

Available since v3.6
The conversion can be restricted by specifying scenario and event names. Using this option will stop the conversion of layers that are within an 'IF Scenario/IF Event' logic blocks if the scenario name is not provided by the user. This option will also clean the control files and remove unused scenario names. Event names can also be used, however they are only applied to logic blocks in the control files and are not used to filter event sources provided in the TUFLOW Event File (.tef).

To use this option:

  1. Ensure 'Restrict Conversion by Scenario' is checked on
  2. List scenario and event names in the text edit. This should be done in the same manner as TUFLOW batch files e.g. -s <scenario-name> -e <event-name>. The order does not matter and numbering scenarios is not required, although is accepted (e.g. -s1 -s2).
GPKG Name

Available since v3.7
Sets the output GPKG database name. Will only be used when outputing to GPKG and using a grouped output profile. If using a 'grouped by control file' profile, then the control file extension will still be added to the name e.g. <output-name>_TCF.gpkg.

Write Empty Files

Available since v3.7
If this option is checked on, the tool will create TUFLOW empty files as part of the conversion. The location of the empty directory will be guessed by the tool unless the user provides a relative path in the Write Empty Files Relative Directory input parameter. The path provided should be a relative path from the model root directory (e.g. commonly this is a folder called "TUFLOW", so a relative path might be model\gis\empty). The projection of the empty files will be based on the first "Projection == " command found in the TCF. If no "Projection == " is found, then empty files will not be created.

Output CRS

Available since v3.7
Forces output files to be in given projection (no reprojection or warping is performed). Useful when converting a model with a combination of SHP and MIF files as these can generate slightly different projection definitions. Set blank to turn off.

Force TUFLOW Directory Structure

Available since v3.10
Forces the output files to be placed in a default TUFLOW folder structure, ignoring the input folder structure. If this setting is on, the "Log Folder", "Output Folder", and "Write Check Files" commands will be updated to also reference standard TUFLOW locations.

Checking the Output

The first thing to check is the output log that is written in the dialog window. At the bottom, the log should specify whether any errors occured during conversion. If errors did occur, the tool should skip the problematic files/commands and continue with the conversion process. The tool will bail only if a more serious error occurs, for instance the tool was unable to determine the 'root' folder. Or alternatively if an unexpected error occurs then this can also cause the tool to finish early.

If errors did occur, then they should be reported in the log with more context. Users can copy the log (or save it) using the tool icons in the bottom right of the log window. Copying this text into a text editor (e.g. Notepad++) can be useful for searching for the keyword "ERROR" to track down these locations.

Checking the model
The tool will produce a "ready to run" model in TUFLOW most of the time. There may be situations where the user will be required to make small adjustments to the output to get the model to run. If you encounter a situation where this happens, feel free to contact support@tuflow.com and let us know what the issue was.

A known conversion issue could occur if the tool is converting both MIF and SHP files into a single format (e.g. GPKG) as the projection of these files may be interpreted slightly differently by GDAL (even though they are the same). Users may therefore be required to manually update the projection of the affected files, or add the TUFLOW command "GIS Projeciton Check == WARNING" to the TCF. Be careful to ensure they are in fact the same projection before using this command.

Python and the Source Code

It's possible to run the tool outside of QGIS in Python. The tool uses GDAL to read and write the GIS files and does not rely on any QGIS libraries. The source code and instructions on how to run the tool in Python can be found at the following location on the TUFLOW Community GitLab page.

Errors

Python version error
If you receive a Python version error, this means that you are running a version of Python older than the required version of Python 3.9. To fix this, you will need to update your Python interpreter version. If you are running this tool in QGIS, then you will be required to update your QGIS installation to the latest version, or the latest long term release version.

Other errors
If you receive any other errors, please send through a copy of the output log to support@tuflow.com.



Up
Go-up.png Back to TUFLOW QGIS Plugin Main Page