GeoPackage Tips: Difference between revisions

Content deleted Content added
 
(38 intermediate revisions by 3 users not shown)
Line 1:
= Introduction =
This page includes tips and tricks for working with the GeoPackageGPKG Vectorvector format (supported since the TUFLOW 2023-03-AA Release). GeoPackageGPKG is a widely supported, open GIS database format, built upon an SQLite database (data is stored as a single file with the extension .gpkg). <br><br>
This database format has several benefits, including:
*Multiple GIS layers can be stored in the single file database. For example, all GIS input layers can be stored in a single GeoPackageGPKG file.
*Supports spatial indexing, greatly reducing loading and viewing times, making it much faster to work within GIS packages.
*Faster to write from TUFLOW than the shapefile format.
For more information, seerefer to Section 4.4.3 of the <u>[https://wwwdocs.tuflow.com/downloads/tuflow-classichpcclassic-archivehpc/manual/latest/#2023 2023 ReleaseTUFLOW NotesManual]</u>.
 
= Database Organisation =
There are many ways to organise model inputs when using GeoPackagesGPKG. This allows modellers to decide how they structure their inputs files, for example some options include:
*a singleSingle database for an entire project,
*separateSeparate database for aneach existingcontrol orfile developed(e.g. casegeometry, orboundary control file)
*Separate database for an existing or developed case
*separateSeparate databases for each type of input (e.g. elevation, materials, culverts).
 
== Referencing a GeoPackage Database ==
To reference a GeoPackageGPKG Database:
<ol>
<li>To specifySpecify a GeoPackageGPKG Database for the entire model, addby adding the following command to the beginning of the .tcf. This database will then be the default for all inputs in all control files. There is no need to reference the database again in the other control files. <br>
<font color="blue"><tt>Spatial Database </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt> <path_to_database> </tt>
<li>To specifySpecify a different GeoPackageGPKG Database for subsequent inputs, useusing the following command. A new database can be specified multiple times throughout control files, TUFLOW will change which database it looks to based oninto the last timedatabase thereferenced above commandthe has been referencedfeature. ThisThe command can be used in the .tcf, .ecf, .tbc, .tgc, .qcf, .tef. For example:<br>
<font color="blue"><tt>Spatial Database </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>gis\2d_zsh.gpkg </tt> <font color="green"><tt> ! Sets GeoPackageGPKG Database</tt></font> <br>
<font color="black"><font color="blue"><tt>Read GIS Z Shape </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>2d_zsh_existing_L </tt></font> <font color="green"><tt> ! References the 2d_zsh_L layer from 2d_zsh.gpkg</tt></font> <br>
<font color="blue"><tt>Read GIS Z Shape </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>2d_zsh_development_L </tt></font> <font color="green"><tt> ! References the 2d_zsh_L layer from 2d_zsh.gpkg</tt></font> <br><br>
<font color="blue"><tt>Spatial Database </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>gis\2d_mat.gpkg </tt></font> <font color="green"><tt> ! Sets GeoPackageGPKG Database</tt></font> <br>
<font color="blue"><tt>Read GIS Mat </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>2d_mat_existing_R </tt></font> <font color="green"><tt> ! References the 2d_mat_R layer from 2d_mat.gpkg</tt></font> <br>
<font color="blue"><tt>Read GIS Mat </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>2d_mat_development_R </tt></font> <font color="green"><tt> ! References the 2d_mat_R layer from 2d_mat.gpkg</tt></font> <br>
 
</ol>Note: The commands are localised to their relevant control file with the exception of the .tcf which acts as a global command (local spatial database will take precedence). If no database is referenced in the other control files, the database referenced in the .tcf will be used.<br>
 
==Referencing a GeoPackage Layer==
 
There are many ways to reference a GeoPackageGPKG Layer. The following commands can be used in the .tcf, .ecf, .tbc, .tgc, .qcf and .tef. <br>
<ol>
<li>To specify a table in a .gpkgGPKG, use ">>":<br>
<font color="blue"><tt>Read GIS Z Shape </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>gis\2d_zsh.gpkg >> 2d_zsh_L </tt><br>
<li>To specify more than one table in a .gpkgGPKG in the same command line, use "&&|". ThisThe isdatabase similarhas to "|",be howeverreferenced therefor is no need to referenceall the database againtables. <br>
<font color="blue"><tt>Read GIS Z Shape </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>gis\2d_zsh.gpkg >> 2d_zsh_L | gis\2d_zsh.gpkg >> 2d_zsh_P </tt><br>
<li>To specify more than one table in a GPKG in the same command line, use "&&". This is similar to "|", however there is no need to reference the database again. <br>
<font color="blue"><tt>Read GIS Z Shape </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>gis\2d_zsh.gpkg >> 2d_zsh_L && 2d_zsh_P </tt><br>
<li>"&&" can be used in conjunction with "|":<br>
<font color="blue"><tt>Read GIS Z Shape </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>gis\2d_zsh_R.gpkg | gis\2d_zsh.gpkg >> 2d_zsh_L && 2s_zsh_P </tt><br>
<li>To specify a .gpkgGPKG file path only. It will assume the table name is the same as the .gpkgGPKG database name. The following are equivalent:<br>
<font color="blue"><tt>Read GIS Z Shape </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>gis\2d_zsh_L.gpkg </tt><br>
<font color="blue"><tt>Read GIS Z Shape </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>gis\2d_zsh_L.gpkg >> 2d_zsh_L </tt><br>
<li>To specify all the tables in a .gpkgGPKG, use the command "USE ALL" with ">>":<br>
<font color="blue"><tt>Read GIS Z Shape </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>gis\2d_zsh.gpkg >> USE ALL </tt><br>
<br>
</ol>
 
If the GeoPackageGPKG Database is specified to use for subsequent inputs, the commands will become:<br>
<font color="blue"><tt>Spatial Database </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>gis\2d_zsh.gpkg </tt> <font color="green"><tt> ! Sets GeoPackageGPKG Database</tt></font> <br>
<ol>
<li>To specify a table:<br>
Line 54 ⟶ 58:
<font color="blue"><tt>Read GIS Z Shape </tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>2d_zsh_L && 2d_zsh_P </tt><br>
</ol>
Note: If the .gpkgGPKG for the model is referenced in the .tcf (using the 'Spatial Database' command), it does not need to be referenced again in the other control files. However, if the model inputs are not contained in a single .gpkgGPKG, the separate databases must be referenced accordingly.
 
== Outputs ==
Result and check file vector outputs can be written into separate databases or grouped, with 'grouped' being the default. Grouped databases will group by output folder location, for example, the check file folder will contain one GeoPackageGPKG Database that contains all check file layers within it. Separate databases will group geometries together, for example, PLOT_P, PLOT_L, PLOT_R will be written to one database. The following command can be used to set the output to either separate or grouped: <br>
<font color="blue"><tt>Spatial Database Output</tt></font> <font color="red"><tt>== </tt></font> <font color="black"><tt>SEPARATE | {GROUPED} </tt></font> <br><br>
This is a huge benefit in terms of file organisation. The below shows athe TUFLOW 'Check' folder from a model using the GeoPackageGPKG (grouped) and a model using the GPKG separate. The same 'Check' folder for a model using Shapefile would have 44(!) items in it. <br><br>
*Grouped:
{{Video|name=GPKG Tips CheckFilesAnimation_Outputs_grouped.mp4|width=1000}}<br>
*Separate (grouped geometries):
{{Video|name=Animation_Outputs_separate.mp4|width=1000}}<br>
 
= QGIS Tips & Tricks =
==Convert TUFLOW Model GIS Format==
The <u>[[Convert_TUFLOW_Model_GIS_Format| Convert TUFLOW Model GIS Format]]</u> tool has been developed to convert fromconverts one GIS format to another for an entire model (input files and control files). For example, if you have a model currently using the SHP vector format and FLT raster format, you can automatically convertbe itconverted to GPKG and TIF. This tool works both ways (to or from GeoPackageGPKG) for any TUFLOW supported formatformats and will automatically beis available in the 'Processing Toolbox' if you haveonce the <u>[[TUFLOW_QGIS_Plugin | TUFLOW Plugin]]</u> is installed. <br><br>
 
==Configuring a New Project==
QGIS is used to configure TUFLOW projects with the TUFLOW plugin. To configure a new TUFLOW project with GeoPackageGPKG as the GIS format, use the TUFLOW plugin:
<ol>
<li>Open the 'Configure TUFLOW Project' tool by selecting Plugins > TUFLOW > Editing > Configure / Create TUFLOW Project.
<li>Click 'Select CRS' to select the required CRS for the project.
<li>Click 'Browse...' to select the folder location containing the TUFLOW folder. This creates the default TUFLOW model directory and sub-folders.
<li>Click 'Browse...' to select the location of the TUFLOW single precision executable file.
<li>Select 'GPKG' as the GIS Format.
<li>Select the required TUFLOW Engine and box options.
<li>Select the relevent remaining options (Note: Do not select "Tutorial Model" if you are building a project model)
<li>Click 'OK'. This will configure the TUFLOW project using GeoPackageGPKG.<br>
<br>
[[File:M01_GPKG_Configure_01.PNG]]<br>
Line 82 ⟶ 89:
</ol>
 
== Setting a BrowerBrowser Panel Favourite ==
For quick access, a project folder can be set as a favourite in the QGIS Browser Panel.
<ol>
Line 101 ⟶ 108:
<li>Move a layer from one GPKG to another.<br>
<br>
{{Video|name=browser_panel_move_layerbrowser_panel_move_layer_b.mp4|width=1000}}
<br>
<li>Move a shapefile into a GPKG. QGIS will automatically convertconverts it to a GPKG layer.<br>
<br>
{{Video|name=browser_panel_move_shpbrowser_panel_move_shp_a.mp4|width=1000}}
<br>
<li>Rename a layer in a GPKG.<br>
<br>
{{Video|name=Animation_rename_layerAnimation_rename_layer_a.mp4|width=1000}}
<br>
<li>Delete a layer in a GPKG.<br>
<br>
{{Video|name=Animation_delete_layer.mp4|width=1000}}
</ol>
<br>
 
== Create a New Layer ==
Use the TUFLOW plugin to create a new layer.
<ol>
<li>Click on the ‘Import Empty File’ symbol from the TUFLOW Plugin toolbar. <br>
<br>
[[File:import_empty_file.png]]<br>
<br>
<li>This opens the Dialog shown below. The TUFLOW plugin accesses the 'Empty Directory' created when the project was configured and provides a list of files to choose from. It appends the 'Empty Type' to the beginning of the 'RUN ID' and the 'Geometry Type' letter to the end. This becomes the name of the file.<br>
<br>
[[File:import_empty_dialogue.png]]<br>
<br>
<li>In the 'Spatial Database Options' section, there are the following options:
<ol>
<li>'''Convert to GPKG'''. This converts the imported format to a GPKG even if the empty file format is not a GPKG. The video below shows it being used for the 'Separate' option, however it can be toggled on for any of the export options. <br>
<br>
{{Video|name=Animation_import_layer_convert.mp4|width=1000}}
<br>
<li>'''Separate:''' Creates a separate GPKG for each new file.<br>
<br>
{{Video|name=Animation_import_layer_separate.mp4|width=1000}}
<br>
<li>'''Group geometry types:''' Creates a GPKG containing the new files for all selected geometry types.<br>
<br>
{{Video|name=Animation_import_layer_group_geom.mp4|width=1000}}
<br>
<li>'''All to one:''' Adds the new files to a new or existing GPKG.<br>
<br>
{{Video|name=Animation_import_layer_alltoone_preexisting.mp4|width=1000}}
<br>
<br>
{{Video|name=Animation_import_layer_alltoone_new.mp4|width=1000}}
<br>
</ol>
</ol>
For more information, see [[QGIS TUFLOW Import Empty|QGIS TUFLOW Import Empty]].
<br>
<br>
 
== Increment a Layer ==
This TUFLOW Plugin tool saves a copy of the active layer with a new filename. To use the tool:
<ol>
<li>Select the layer to increment in the Layers Panelpanel.
<li>Click on the 'Increment Selected Layer' symbol from the TUFLOW Plugin toolbar.<br>
<br>
Line 124 ⟶ 173:
There are three options for incrementing a layer:
<ol>
<li>Increment layer only (default): This option will incrementincrements the filename and the .gpkgGPKG database by 1. It will then closecloses the old layer and openopens the new one. For example, the tool increments '''M01_00''1''M01_001.gpkg''' to '''M01_00''2''M01_002.gpkg''' and '''2d_code_M01_00''1''_R2d_code_M01_001_R''' to '''2d_code_M01_00''2''_R2d_code_M01_002_R'''.
:*Note: In the video below, the output database is manually changed back to '''M01_00''1''M01_001.gpkg''',. howeverIf the ifdefault leftsettings aswere thenot default,changed the new layer would behave been added to a new gpkgGPKG, '''M01_00''2''M01_002.gpkg'''. <br>
<br>
{{Video|name=Animation_increment_layer_default.mp4|width=1000}}<br>
<li>Increment layer and preserve database: This option will incrementincrements the filename of the selected layer and the .gpkgGPKG database by 1. It will also copycopies all other layers from the .gpkgoriginal GPKG database into the newincremented .gpkgGPKG database. For example, the tool makes a copy of '''M01_00''1''M01_001.gpkg''' and increments it to '''M01_00''2''M01_002.gpkg'''. All layers are copied across with their original filename, except the selected layer, which is incremented to '''2d_code_M01_002_R'''.<br>
<br>
{{Video|name=Animation_increment_layer_preserve_db.mp4|width=1000}}<br>
<li>Save layer out into superseded folder: This option will incrementincrements the the filename of the selected layer by 1. It will then closecloses the old layer and openopens the new one. The old layer will begets moved into a new folder called 'ss'. For example, the tool increments '''2d_code_M01_00''1''_R2d_code_M01_001_R''' to '''2d_code_M01_00''2''_R2d_code_M01_002_R''' and moves '''2d_code_M01_00''1''_R2d_code_M01_001_R''' to the 'ss' folder. <br>
<br>
{{Video|name=Animation_increment_layer_ss.mp4|width=1000}}
<br>
</ol>
For more information, see the <u>[[QGIS_TUFLOW_Increment_Layer| Increment Layer Tool]].</u><br>
<br>
 
==Apply GeoPackage Layer Names ==
This tool will change the layer name (within the Layers Panelpanel) to be the same as the layer name within the GeoPackageGPKG database. To use the tool:
<ol>
<li>Drag and Drop the GeoPackageGPKG Database from Windows File Explorer to QGIS.
<li>Click on the 'Apply GPKG Layer Names' symbol from the TUFLOW Plugin toolbar.<br>
<br>
[[File: GPKG_layer_names_a.png]]<br>
<br>
{{Video|name=Animation_apply_GPKG_layer_namesAnimation_apply_GPKG_layer_names_a.mp4|width=1000}}<br>
</ol>
Note: If the layers are opened from within the QGIS Browser Panel, they will automatically be named the same as within the GeoPackageGPKG database.<br>
<br>
For more information, see the <u>[[QGIS_TUFLOW_Apply_GPKG_Layer_Names| Updating a GPKG Layer Name]]</u>
=Tutorials=
Free TUFLOW tutorials using the GeoPackage format are availble via: <u>[[Tutorial Introduction|TUFLOW HPC Tutorials]]</u>