GRID to GRID draft: Difference between revisions
Content deleted Content added
Anne.Kolega (talk | contribs) m →Introduction: The |
|||
| (27 intermediate revisions by one other user not shown) | |||
Line 1:
=Introduction=
The grid_to_grid.exe is a utility that can be used to perform a range of operations on gridded files. The input grids can be a combination of GeoTIFF (.tif), ESRI ASCII
For some options (such as processing maximums or differences) the input files must be of the same row/column dimensions.<br>
Line 31:
|"-src"||For use with -min,-max or various stat options to supress the source grid output. Will only output the minimum, maximum, mean, median, frac grid. The src_legend.csv is also supressed.
|-
|"-grc"||For use with the "-classify" option and
|}
Line 37:
==Convert==
'''"-conv"'''<br>
Converts
*Converts all
:<tt>grid_to_grid.exe -conv -tif *.flt</tt>
*Converts the file "DEM.asc" to
:<tt>grid_to_grid.exe -conv -flt DEM.asc</tt>
* Converts an ASC grid to a GeoTIFF with an LZW compression and horizontal predictor, see <u>[[#Creation_Options | Creation Options]]</u>.<br>
:<tt>grid_to_grid.exe -conv -tif DEM_M01.asc -co COMPRESS=LZW -co PREDICTOR=2</tt>
* Converts GeoTIFF to a Cloud Optimised GeoTIFF. Requires <tt>-out</tt> flag since the extension for COG is also
:<tt>grid_to_grid.exe -conv -cog DEM_M01.tif -out DEM_M01_COG.tif</tt>
==Maximum==
'''"-max"'''<br>
Determines the maximum value in all the input
*A numerical grid containing the maximum value.<br>
*A classified grid with the
*Creates a new grid containing the maximum of the 3 input water level grids.<br>
:<tt>grid_to_grid.exe -max Q100_30min_h.
*As
:<tt>grid_to_grid.exe -out Q100_Max_Levels.
*Wildcard character "*" is supported in filenames.<br>
:<tt>grid_to_grid.exe -out Q100_Max_Levels.
* Runs a maximum operation on all FLT grids in a given location and outputs to a GeoPackage raster.<br>
:<tt>grid_to_grid.exe -max -out "output_grids.gpkg >> maximum_h" *.flt</tt>
Line 65:
==Difference==
'''"-dif"'''<br>
Takes the difference between the first two
Two
*The first is the difference values between the
*A second grid with a “_wd” suffix is output to indicate which
Example:
:<tt>grid_to_grid.exe -out difference.asc -dif after_h.asc before_h.asc</tt>▼
*Runs a difference operation on GeoTIFF outputs.<br>
:<tt>grid_to_grid.exe -dif proposed.tif existing.tif</tt>
*As above with specified output name "difference.tif".<br>
▲:<tt>grid_to_grid.exe -out difference.
* Runs a difference operation on GPKG outputs. The layer name needed is assumed to be the same as the database name.<br>
:<tt>grid_to_grid.exe -dif proposed.gpkg existing.gpkg</tt>
* Runs a difference operation on GeoPackage outputs where the layer name is different than the database name, see <u>[[#GPKG_and_NetCDF | GPKG and NetCDF]]</u>.<br>
:<tt>grid_to_grid.exe -dif "proposed_grid_results.gpkg >> proposed_max_h" "existing_grid_results.gpkg >> existing_max_h"</tt>
Line 89:
*If two grids are specified, the first value is used for setting the cutoff depth and the values from the second grid are processed.<br>
*If three grids are specified, the third grid is used for setting the filename for the output grid. This is ignored if the "-out" option is used.<br>
Example:
*Creates a new depth grid only where the depth is greater than 0.1m.<br>
:<tt>grid_to_grid.exe -cd0.1 depth.
*Creates a new level grid only where the depth is greater than 0.1m.<br>
:<tt>grid_to_grid.exe -cd0.1 depth.
*As per the example above, but sets the output name as "filtered_levels.
:<tt>grid_to_grid.exe -cd0.1 -out filtered_levels.
==Resize==
'''"-resize<factor>"'''<br>
Resize the grid based on the factor above. The factor must be an integer (greater than 1). The output grid has the same origin (lower left) coordinates as the input grid. The new cell size is the factor times the old cell size.
===Average (default)===
This is the defualt processing method, which can also be specified with the '''-rm_avg''' (resize method - average) input flag. When processing for each tile in the new grid, the average of all non-null values in the input grid is taken
Example:
*Creates a new 10m DEM
:<tt>grid_to_grid.exe -resize5 -out DEM_10m.
===Minimum===
Alternative processing method for resize, specified with the '''-rm_min''' (resize method - minimum) flag. At each output grid the values
Example:
*Creates a new 10m DEM
:<tt>grid_to_grid.exe -resize5 -rm_min -out DEM_10m_min.
===Maximum===
Alternative processing method for resize, specified with the '''-rm_max''' (resize method - maximum) flag. At each output grid the values
Example:
*Creates a new 10m DEM
:<tt>grid_to_grid.exe -resize5 -rm_max -out DEM_10m_max.
==Transform==
Transforms the values using a simple y = mx + c approach.
Example:
*Multiplies all values by 0.3048.<br>
:<tt>grid_to_grid.exe -trans_m0.3048 DEM.
*Multiplies all values by 0.3048 and then adds 10.0.<br>
:<tt>grid_to_grid.exe -trans_m0.3048 -trans_c10 DEM.
==Decimal==
Converts
Example:
*Converts all
:<tt>grid_to_grid.exe -conv -asc -decimal5 *.
==Classify==
Classifies the grid based on the input classifications.
{| class="wikitable"
Line 146:
|}
Example:
*Outputs a classified grid, based on the cut off values and names in the "classifications.csv".<br>
:<tt>grid_to_grid.exe -classify classifications.csv results_VMax.
*If using vertical mapper the -grc option can be used to create
:<tt>grid_to_grid.exe -RGB -grc -classify depth_classify.csv results_dMax.
:{| class="wikitable"
Line 174:
==Extract Breaklines from DEM==
'''"-brkline <gis file in 2d_zsh format>"'''<br>
Extracts elevations from a DEM for breaklines in the 2d_zsh format. This GIS input should have four attributes, z, dz, width and shape_options, as described in the <u>[https://docs.tuflow.com/classic-hpc/manual/latest/ TUFLOW Manual]</u>. For the utility the '''dz''', '''shape_width''' and the '''shape_options''' are used. Valid shape_options are "Max" and "Min" or blank (average used). The utility
▲Extracts elevations from a DEM for breaklines in the 2d_zsh format. This GIS input should have four attributes, z, dz, width and shape_options, as described in the <u>[https://docs.tuflow.com/classic-hpc/manual/latest/ TUFLOW Manual]</u>. For the utility the '''dz''', '''shape_width''' and the '''shape_options''' are used. Valid shape_options are "Max" and "Min" or blank (average used). The utility will extract an elevation at each <u>vertex</u> along the line.<br>
Depending on the line width and DEM cell size, the following methods are used:<br>
* If the width is set to 0, the closest DEM value to the vertex is used. If a "shape_option" is specified, this is ignored.
* If the width is greater than 0, but less than 1.5 times the DEM cell size, the four DEM values surrounding the vertex are used. If max or min is specified in the "shape_option" the max or min of the four values is used. If max or min option is not specified (attribute is blank), the average of the four values is used.
* If the width is greater than 1.5 times the DEM cell size, at each vertex a buffer region (search radius) is created and all non null DEM values within the buffer object are processed. The diameter of the buffer region is equal to the width specified. If max or min option is not specified, the average of the values within the region is used. These buffer regions are outputted as a separate file, if '''"-check"''' switch is specified in the batch command.<br>
Line 183 ⟶ 182:
'''NOTE:''' For the output GIS file, the '''dz''' attribute is set to 0.0 regardless of the value set in the input field. <br>
Example:
*<tt>grid_to_grid.exe -brkline 2d_zsh_breaklines_L.shp DEM.
*<tt>grid_to_grid.exe -brkline 2d_zsh_breaklines_L.gpkg DEM.tif</tt>
==Fuzzy Map==
'''"-fuzzy <input grids>"'''<br>
A fuzzy map can be used when comparing a large number of simulations.<br>
For each input grid (or scenario) the grid is classed as either 1 if the results grid is wet or 0 if dry.
This can be useful for quantifying the sensitivity of the model to parameters.<br>
Example:
*Create a fuzzy map from all maximum depth grids in the directory.<br>
:<tt>grid_to_grid.exe -fuzzy *_d_Max.
==Histogram==
'''"-histogram bins.csv <input grid>"'''<br>
Calculates the number of cells and percentage of cells that fall in each range. Requires and input .csv file containing the bin values.<br>
Example:
*An example inputs bin file is:<br>
:{| class="wikitable"
Line 242 ⟶ 239:
|Above 5.0|| Above 5.0 || 973 || 0.06
|}
*Example
:<tt>grid_to_grid.exe –histogram histogram_bins.csv results_d_Max.
==Extract 1D Model Inputs==
'''"-egc control_file.egc"'''<br>
This allows 1D Network (1d_nwk) and/or 1D
A simple input file similar to the TUFLOW geometry control file is input into the utility. This is nominally given the extension .egc (Estry Geometry Control).<br>
<br>
Line 307 ⟶ 304:
|Read GIS XS == <1d_cut GIS file>|| Reads the cross-section cut lines, these are described above.
|-
|Read GIS FLC == <GIS file>|| Optional GIS layer of polygons that can be used to specify form loss coefficients to the 1d_nwk layer. A single GIS (float) attribute is required. A FLC value of greater than 0 will be split between all 1d_nwk channels within the region. A FLC value of less than 0, will apply the absolute value to all channels within the polygon. For example if an FLC value of -0.1 is specified, all channels that fall within the polygon will have a form_loss attribute of 0.1. If a channel falls within multiple polygons, the form loss values will be added.
|-
|Set XS dx == <distance>|| Sets the default distance across section to extract elevations and materials data. This can be specified on the cut lines as detailed above.
Line 332 ⟶ 328:
|FLC per Unit Length == <form loss per unit length> || The form loss per unit length is written to the "exit_loss" attribute of all 1d_nwk channels.
|}
Example
*<tt>grid_to_grid.exe -egc MR_1d_001.egc</tt>
Line 347 ⟶ 343:
Set XS Mat == 1 !default Material ID of 1
Read Grid Mat == grid\grid_5m_2d_mat_MR_001.
Set XS Z == 99
Read Grid Z == grid\dem_5m_regional.
Read Grid Z == grid\dem_1m_bathymetry.
Read Grid Z == grid\dem_1m_survey.
</pre>
Line 360 ⟶ 356:
The input GIS region file should have the same attributes as the 1d_tab file format produced by TUFLOW. The first attribute ("Source" ) should contain a valid .csv file name.<br>
The out nodal area table extends from the minimum to the maximum elevation in the DEM. A user defined minimum value can be specified using the "Skew" attribute of the GIS region.<br>
Example
*<tt>grid_to_grid.exe -na
==Calibration Points==
'''"-cp <cp_file>"'''<br>
For appending the value from a
'''"-config <config_file>"''' (optional)<br>
Points to a csv file containing accuracy thresholds and style control options, e.g. symbol, colour and size.<br>
Line 372 ⟶ 368:
'''"-cpmm<mm>"''' (optional)<br>
The second search radius used to search for maximum and minimum values in meters.<br>
Example
*<tt>grid_to_grid.exe -cp Flood_Marks.MIF -config diff_config.csv -cpsd100 -cpmm50 results_h_Max.
==Statistics==
<big>'''"-stat<type>"'''</big>
The available options are:
Line 396 ⟶ 392:
* A value grid: the resulting median value
* An SRC grid: a grid listing which source input grid the median result value came from (a corresponding .csv file is also written as a legend)<br>
Example
*<tt>grid_to_grid.exe -statMedian Q100_30min_TP1_h.
[[File: asc_to_asc_statsMedian_eg.PNG|1000px|]]<br>
<br>
Note, the median value for
<br>
[[File:A2A median.png|400px]]<br>
Line 410 ⟶ 406:
* A value grid: the resulting value
* An SRC grid: a grid listing which source input grid the result value came from or was the next above (a corresponding .csv file is also written as a legend)<br>
Example
*<tt>grid_to_grid.exe -statMean Q100_30min_*_h.
==="-statFrac"===
Line 417 ⟶ 413:
The utility outputs one grid:
* A Frac grid: the resulting
Example
*<tt>grid_to_grid.exe -statFrac Q100_30min_*_h.
==="-statMin"===
The -statMin flag outputs the minimum value at each grid cell from all the input grids. This flag is analogous with the [[#Minimum|-Min]] operation flag.
The utility outputs two grids:
* A value grid: the resulting value
* An SRC grid: a grid listing which source input grid the minimum result value came from (a corresponding .csv file is also written as a legend)<br>
Example
*<tt>grid_to_grid.exe -statMin Q100_30min_*_h.
==="-statMax"===
Line 436 ⟶ 432:
* A value grid: the resulting value
* An SRC grid: a grid listing which source input grid the maximum result value came from (a corresponding .csv file is also written as a legend)<br>
Example
*<tt>grid_to_grid.exe -statMax Q100_30min_*_h.
==="-statAll"===
Line 446 ⟶ 442:
* An SRC grid: for the median, mean, min and max calculations (calculation of source grids as detailed above)
* A .csv file that contains the legend for all SRC grids<br>
Example
*<tt>grid_to_grid.exe -statAll Q100_30min_*_h.
==="-statRank<integer id>"===
The -statRank<integer> flag outputs the
The utility outputs two grids:
* A value grid: the resulting value
* An SRC grid: a grid listing which source input grid the ranked result value came from (a corresponding .csv file is also written as a legend)<br>
Example
*<tt>grid_to_grid.exe -statRank5 Q100_30min_*_h.
==Flood Extent==
Returns a simple grid with value of 1 where the input grid has an active, non-zero value.<br>
Example
*<tt>grid_to_grid.exe -fe results_dMax.
==Remap==
'''"-wl <wl_file> -dem <dem_file>"'''<br>▼
▲'''"-wl <wl_file>"'''<br>
▲'''"-dem <dem_file>"'''<br>
The default interpolation method is TIN. But "-idw_npt<number of points>" flag can be used to change the interpolation method to IDW and set the number of points used for IDW interpolation (default is 12).<br>
Example:
*
:<tt>grid_to_grid.exe -remap -wl lowres_h.
*The utility can also
:<tt>grid_to_grid.exe -remap -wl lowres_h.
*The utility can also apply a buffer distance or add a freeboard. The below applies a buffer distance of 20m, and adds a freeboard of 0.2m to the original water level grid.<br>
:<tt>grid_to_grid.exe -remap -fb0.2 -bd20 -wl lowres_h.
Note that, for any output types other than depth, this utility does NOT interpolate the result from the coarser grid to the finer grid, but only extends/reduces the output extent to the dry/wet extent. More discussions on this function is documented here [[TUFLOW_Remapping | TUFLOW Remapping Tool]].<br>▼
▲Note
=GPKG and NetCDF=
Line 490 ⟶ 484:
=Creation Options=
The grid_to_grid.exe supports GDAL style creation options. Creation options can be passed in using the "-co" flag then the name of the creation option and the value:<Br>
<pre>-co COMPRESS=DEFLATE</pre>
Multiple creation options can be used (each will require its own "-co" flag).<Br>
<Br>
The creation options have adopted the same naming as GDAL (more information on the options can be found at <b><u>[https://gdal.org https://gdal.org]</u></b>), however, not all options in GDAL are supported in the grid_to_grid.exe utility. The following creation options are supported:<Br>
Line 511 ⟶ 505:
====NetCDF====
* COMPRESS=NONE/DEFLATE- output compression. Default - DEFLATE
* ZLEVEL=[0-9] - Compression level. Zero is no compression, 9 is
====ASC====
* DECIMAL_PRECISION=N - The number of decimal places in the output. This option is analogous to the <tt>"-decimal[N]"</tt> switch. Default - 3.
| |||