Tuflow

GRID to GRID draft: Difference between revisions

Browse history interactively
← Older edit
Content deleted Content added
VisualWikitext
No edit summary
 
(19 intermediate revisions by one other user not shown)
Line 1:
=Introduction=
The grid_to_grid.exe is a utility that can be used to perform a range of operations on gridded files. The input grids can be a combination of GeoTIFF (.tif), ESRI ASCII (.asc), binary float (.flt) geopackage (.gpkg), NetCDF (.nc) and Cloud Optimised GeoTIFFs (.cog) grids, all of which can be input and output from TUFLOW and any of the format can be set as output from the utility.<br>
For some options (such as processing maximums or differences) the input files must be of the same row/column dimensions.<br>
 
Line 38:
'''"-conv"'''<br>
Converts grid formats as specified by the output switch.<br>
ExamplesExample:<br>
*Converts all FLT grids to TIF format.<br>
:<tt>grid_to_grid.exe -conv -tif *.flt</tt>
Line 53:
*A numerical grid containing the maximum value.<br>
*A classified grid with the suffix _src which contains the source grid for the maximum value.<br>
ExamplesExample:<br>
*Creates a new grid containing the maximum of the 3 input water level grids.<br>
:<tt>grid_to_grid.exe -max Q100_30min_h.tif Q100_45min_h.tif Q100_60min_h.tif</tt><br>
Line 65:
==Difference==
'''"-dif"'''<br>
Takes the difference between the first two grids, the output is the first grid minus the second grid. If an optional third grid is specified, this is used as the output file, otherwise, grid_to_grid creates its own output filename using the names of the two input grids.<br>
Two grids are output:
*The first is the difference values between the grids. A difference value only occurs at grid cells that have a value in both grids. If the cell has a null value (i.e. the TUFLOW output was dry at that location) in either or both grids, a null value is output.
Line 72:
*Runs a difference operation on GeoTIFF outputs.<br>
:<tt>grid_to_grid.exe -dif proposed.tif existing.tif</tt>
*As above with specified output name "difference.tif".<br>
:<tt>grid_to_grid.exe -out difference.tif -dif proposed.tif existing.tif</tt>
* Runs a difference operation on GPKG outputs. The layer name needed is assumed to be the same as the database name.<br>
Line 89:
*If two grids are specified, the first value is used for setting the cutoff depth and the values from the second grid are processed.<br>
*If three grids are specified, the third grid is used for setting the filename for the output grid. This is ignored if the "-out" option is used.<br>
Example:
Examples:
*Creates a new depth grid only where the depth is greater than 0.1m.<br>
:<tt>grid_to_grid.exe -cd0.1 depth.tif</tt>
Line 99:
==Resize==
'''"-resize<factor>"'''<br>
Resize the grid based on the factor above. The factor must be an integer (greater than 1). The output grid has the same origin (lower left) coordinates as the input grid. The new cell size is the factor times the old cell size. For example if a -resize5 is used on a 2m resolution grid the output grid will be 10m. For processing the grid three options are available:
===Average (default)===
This is the defualt processing method, which can also be specified with the '''-rm_avg''' (resize method - average) input flag. When processing for each tile in the new grid, the average of all non-null values in the input grid is taken. For example with a resize factor of 5, the 25 values in the input grid are averaged.<br>
Example:
*Creates a new 10m DEM based on the the 2m DEM using an averaging approach.<br>
Line 107:
 
===Minimum===
Alternative processing method for resize, specified with the '''-rm_min''' (resize method - minimum) flag. At each output grid the values isare the minimum of the values in the input grid.<br>
Example:
*Creates a new 10m DEM based on the the 2m DEM using an minimum approach.<br>
Line 113:
 
===Maximum===
Alternative processing method for resize, specified with the '''-rm_max''' (resize method - maximum) flag. At each output grid the values isare the maximum of the values in the input grid.<br>
Example:
*Creates a new 10m DEM based on the the 2m DEM using an maximum approach.<br>
Line 119:
 
==Transform==
Transforms the values using a simple y = mx + c approach. theThe multiplier (m) is specified with flag '''-trans_m<value>'''. The add value (c) with flag '''-trans_c<value>'''. Only one valuesvalue is required to be specified.<br>
Example:
Examples:
*Multiplies all values by 0.3048.<br>
:<tt>grid_to_grid.exe -trans_m0.3048 DEM.tif</tt>
Line 133:
 
==Classify==
Classifies the grid based on the input classifications. The output file is a classified grid. The classification .csv file should have two columns, cutoff value and name (in that order). The first line in .csv treated as a header line and ignored. Any values over the greatest cutoff with be outputted to class "above".<br>
{| class="wikitable"
 
Line 146:
|}
 
Example:
Examples:
*Outputs a classified grid, based on the cut off values and names in the "classifications.csv".<br>
:<tt>grid_to_grid.exe -classify classifications.csv results_VMax.tif</tt>
*If using vertical mapper the -grc option can be used to create the tifflt file in classified grid format,. inIn this format, rather than storing a numerical value when interrogating an area, a label is returned. This format is not recognised by ArcMap or QGIS. If using the grc option it is also possible to specify the desired RGB (red, green, blue) values for the output grid in the .csv file used for the classification. The RGB values should be specified in the 3-5th columns of the .csv file.<br>
:<tt>grid_to_grid.exe -RGB -grc -classify depth_classify.csv results_dMax.tifflt</tt><br>
 
:{| class="wikitable"
Line 174:
==Extract Breaklines from DEM==
'''"-brkline <gis file in 2d_zsh format>"'''<br>
Extracts elevations from a DEM for breaklines in the 2d_zsh format. This GIS input should have four attributes, z, dz, width and shape_options, as described in the <u>[https://docs.tuflow.com/classic-hpc/manual/latest/ TUFLOW Manual]</u>. For the utility the '''dz''', '''shape_width''' and the '''shape_options''' are used. Valid shape_options are "Max" and "Min" or blank (average used). The utility will extractextracts an elevation at each <u>vertex</u> along the line.<br>▼
This feature was added to the 2013-06-AA version of the utility.<br>
▲Extracts elevations from a DEM for breaklines in the 2d_zsh format. This GIS input should have four attributes, z, dz, width and shape_options, as described in the <u>[https://docs.tuflow.com/classic-hpc/manual/latest/ TUFLOW Manual]</u>. For the utility the '''dz''', '''shape_width''' and the '''shape_options''' are used. Valid shape_options are "Max" and "Min" or blank (average used). The utility will extract an elevation at each <u>vertex</u> along the line.<br>
Depending on the line width and DEM cell size, the following methods are used:<br>
* If the width is set to 0, the closest DEM value to the vertex is used. If a "shape_option" is specified, this is ignored.
* If the width is greater than 0, but less than 1.5 times the DEM cell size, the four DEM values surrounding the vertex are used. If max or min is specified in the "shape_option" the max or min of the four values is used. If max or min option is not specified (attribute is blank), the average of the four values is used.
* If the width is greater than 1.5 times the DEM cell size, at each vertex a buffer region (search radius) is created and all non null DEM values within the buffer object are processed. The diameter of the buffer region is equal to the width specified. If max or min option is not specified, the average of the values within the region is used. These buffer regions are outputted as a separate file, if '''"-check"''' switch is specified in the batch command.<br>
Line 188 ⟶ 187:
==Fuzzy Map==
'''"-fuzzy <input grids>"'''<br>
This feature was added to the 2014-03-AB version of the utility.<br>
A fuzzy map can be used when comparing a large number of simulations.<br>
For each input grid (or scenario) the grid is classed as either 1 if the results grid is wet or 0 if dry. The total score for each grid cell is is calculated and then divided by the total number of input grids. A value of 1 indicates that the cell was wet in each simulation ,a value of 0 is dry in all simulations. Grids with a value of 0.5 are most sensitive.<br>
This can be useful for quantifying the sensitivity of the model to parameters.<br>
Example:
Line 198 ⟶ 196:
==Histogram==
'''"-histogram bins.csv <input grid>"'''<br>
This feature was added to the 2014-03-AB version of the utility.<br>
Calculates the number of cells and percentage of cells that fall in each range. Requires and input .csv file containing the bin values.<br>
Example:
Examples:
*An example inputs bin file is:<br>
:{| class="wikitable"
Line 247 ⟶ 244:
==Extract 1D Model Inputs==
'''"-egc control_file.egc"'''<br>
This allows 1D Network (1d_nwk) and/or 1D Crosscross-Sectionssections to be extracted from a series of grids.<br>
A simple input file similar to the TUFLOW geometry control file is input into the utility. This is nominally given the extension .egc (Estry Geometry Control).<br>
<br>
Line 307 ⟶ 304:
|Read GIS XS == <1d_cut GIS file>|| Reads the cross-section cut lines, these are described above.
|-
|Read GIS FLC == <GIS file>|| Optional GIS layer of polygons that can be used to specify form loss coefficients to the 1d_nwk layer. A single GIS (float) attribute is required. A FLC value of greater than 0 will be split between all 1d_nwk channels within the region. A FLC value of less than 0, will apply the absolute value to all channels within the polygon. For example if an FLC value of -0.1 is specified, all channels that fall within the polygon will have a form_loss attribute of 0.1. If a channel falls within multiple polygons, the form loss values will be added.
Reads the polygons off losses from GIS layer. cross-section cut lines, these are described above.
|-
|Set XS dx == <distance>|| Sets the default distance across section to extract elevations and materials data. This can be specified on the cut lines as detailed above.
Line 332 ⟶ 328:
|FLC per Unit Length == <form loss per unit length> || The form loss per unit length is written to the "exit_loss" attribute of all 1d_nwk channels.
|}
Example usage:
*<tt>grid_to_grid.exe -egc MR_1d_001.egc</tt>
 
Line 360 ⟶ 356:
The input GIS region file should have the same attributes as the 1d_tab file format produced by TUFLOW. The first attribute ("Source" ) should contain a valid .csv file name.<br>
The out nodal area table extends from the minimum to the maximum elevation in the DEM. A user defined minimum value can be specified using the "Skew" attribute of the GIS region.<br>
Example usage:
*<tt>grid_to_grid.exe -na 1d_na_storage_Rstorage_R.shp DEM_1m.tif</tt>
 
==Calibration Points==
Line 372 ⟶ 368:
'''"-cpmm<mm>"''' (optional)<br>
The second search radius used to search for maximum and minimum values in meters.<br>
Example usage:
*<tt>grid_to_grid.exe -cp Flood_Marks.MIF -config diff_config.csv -cpsd100 -cpmm50 results_h_Max.tif</tt><br>
 
==Statistics==
<big>'''"-stat<type>"'''</big>
As of the 2017-10-AA build, theThe grid_to_grid.exe utility allows the user to perform a number of statistical analyses on a group of input grids. This is particularly useful when extracting information from assessments that produce multiple result files for a single event, for example, processing grids from an ensemble assessment with numerous temporal pattern arrangements.
 
The available options are:
Line 396 ⟶ 392:
* A value grid: the resulting median value
* An SRC grid: a grid listing which source input grid the median result value came from (a corresponding .csv file is also written as a legend)<br>
Example usage:
*<tt>grid_to_grid.exe -statMedian Q100_30min_TP1_h.tif Q100_30min_TP2_h.tif Q100_30min_TP3_h.tif Q100_30min_TP4_h.tif Q100_30min_TP5_h... Q100_30min_TP10_h.tif </tt><br>
 
[[File: asc_to_asc_statsMedian_eg.PNG|1000px|]]<br>
<br>
Note, the median value for thatthe cellexample wasbelow dryis (a 'NULL') value, then the output wouldis bea dry cell. For example, 7 out of the 10 inputs are dry at athis particular locationcell. Once theythe outputs are ordered them (with NULL showing at the lowest spots), then the median (6th) value for thisthe cell is 'NULL' (dry) and the source would be 8, as shown below. Therefore, at this location you would see a dry cell in the median output, and '8' inis the source grid output.
<br>
[[File:A2A median.png|400px]]<br>
Line 410 ⟶ 406:
* A value grid: the resulting value
* An SRC grid: a grid listing which source input grid the result value came from or was the next above (a corresponding .csv file is also written as a legend)<br>
Example usage:
*<tt>grid_to_grid.exe -statMean Q100_30min_*_h.tif</tt><br>
 
Line 417 ⟶ 413:
 
The utility outputs one grid:
* A Frac grid: the resulting Fractionfraction value<br>
Example usage:
*<tt>grid_to_grid.exe -statFrac Q100_30min_*_h.tif</tt><br>
 
==="-statMin"===
The -statMin flag outputs the minimum value at each grid cell from all the input grids. This flag is analogous with the [[#Minimum|-Min]] operation flag.
 
The utility outputs two grids:
* A value grid: the resulting value
* An SRC grid: a grid listing which source input grid the minimum result value came from (a corresponding .csv file is also written as a legend)<br>
Example usage:
*<tt>grid_to_grid.exe -statMin Q100_30min_*_h.tif</tt><br>
 
Line 436 ⟶ 432:
* A value grid: the resulting value
* An SRC grid: a grid listing which source input grid the maximum result value came from (a corresponding .csv file is also written as a legend)<br>
Example usage:
*<tt>grid_to_grid.exe -statMax Q100_30min_*_h.tif</tt><br>
 
Line 446 ⟶ 442:
* An SRC grid: for the median, mean, min and max calculations (calculation of source grids as detailed above)
* A .csv file that contains the legend for all SRC grids<br>
Example usage:
*<tt>grid_to_grid.exe -statAll Q100_30min_*_h.tif</tt><br>
 
==="-statRank<integer id>"===
The -statRank<integer> flag outputs the ''n<integer>'' ranked value at each grid cell from all the input grids. All the input grids are considered, regardless of if the grid cell is dry in some of the input grids (dry areas are still ranked as low e.g. Rank1, Rank2 and so on). For example, if the number of input grids is 10, at each grid cell the values are ranked lowest to highest (from 1 to 10). -statRank8 will output results taken from the 8th ranked grid. In this example, -statRank1 is equivalent to ''Min'', -statRank10 is equivalent to ''Max'', and -statRank6 is equivalent to ''Median''. The Rank flag allows additional functionality that lets users extract results in between those that are pre-programmed.
 
The utility outputs two grids:
* A value grid: the resulting value
* An SRC grid: a grid listing which source input grid the ranked result value came from (a corresponding .csv file is also written as a legend)<br>
Example usage:
*<tt>grid_to_grid.exe -statRank5 Q100_30min_*_h.tif</tt><br>
 
==Flood Extent==
'''"-wl <wl_file>fe"'''<br>▼
Returns a simple grid with value of 1 where the input grid has an active, non-zero value.<br>
Example usage:
*<tt>grid_to_grid.exe -fe results_dMax.tif</tt><br>
 
==Remap==
'''"-wl <wl_file> -dem <dem_file>"'''<br>▼
RemapRemaps acoarser resolution water level grid to a higherfiner resolution DEM.<br>
▲'''"-wl <wl_file>"'''<br>
Sets the coarser resolution water level grid to remap from.<br>
▲'''"-dem <dem_file>"'''<br>
Sets the finer resolution DEM.<br>
The default interpolation method is TIN. But "-idw_npt<number of points>" flag can be used to change the interpolation method to IDW and set the number of points used for IDW interpolation (default is 12).<br>
Example:
Examples:
*TheRemaps utilitywater canlevel also remap additional map output grids (e.g. velocity, hazard and others)grid to the resolution of the DEM file, e.g.:<br>
:<tt>grid_to_grid.exe -remap -wl lowres_h.tif -dem DEM_highres.tif</tt>
*The utility can also applyremap adepth bufferoutput distancegrids or add freeboard. For example,to the below applies a buffer distanceresolution of 20m, and adds a freeboard of 0.2m to the original water levelDEM grid.file:<br>
:<tt>grid_to_grid.exe -remap -wl lowres_h.tif -dem DEM_highres.tif lowres_v.tif lowres_hazardlowres_d.tif</tt>
*The utility can also apply a buffer distance or add a freeboard. The below applies a buffer distance of 20m, and adds a freeboard of 0.2m to the original water level grid.<br>
:<tt>grid_to_grid.exe -remap -fb0.2 -bd20 -wl lowres_h.tif -dem DEM_highres.tif</tt>
 
Note that, for any output types other than depth, this utility does NOT interpolate the result from the coarser grid to the finer grid, but only extends/reduces the output extent to the dry/wet extent. More discussions on this function is documented here <u>[[TUFLOW_Remapping | TUFLOW Remapping Tool]]</u>.<br>
 
 
=GPKG and NetCDF=
Line 490 ⟶ 484:
 
=Creation Options=
The grid_to_grid.exe supports GDAL style creation options. Creation options can be passed in using the "-co" flag then the name of the creation option and the value:<Br>
<pre>-co COMPRESS=DEFLATE</pre>
Multiple creation options can be used (each will require its own "-co" flag).<Br>
<Br>
The creation options have adopted the same naming as GDAL (more information on the options can be found at <b><u>[https://gdal.org https://gdal.org]</u></b>), however, not all options in GDAL are supported in the grid_to_grid.exe utility. The following creation options are supported:<Br>
Line 511 ⟶ 505:
====NetCDF====
* COMPRESS=NONE/DEFLATE- output compression. Default - DEFLATE
* ZLEVEL=[0-9] - Compression level. Zero is no compression, 9 is hightesthighest compression. Only supported for DEFLATE compression method. Default - 9.
====ASC====
* DECIMAL_PRECISION=N - The number of decimal places in the output. This option is analogous to the <tt>"-decimal[N]"</tt> switch. Default - 3.
Retrieved from "https://wiki.tuflow.com/GRID_to_GRID_draft"