TUFLOW NetCDF Raster Output Format: Difference between revisions

Content deleted Content added
No edit summary
No edit summary
 
(5 intermediate revisions by 2 users not shown)
Line 1:
=Introduction=
This page contains information on the NetCDF file format used for gridded (raster) outputs from a TUFLOW simulation. To specify NetCDF output from TUFLOW, the 2016 version or later of TUFLOW must be used and "NC" must be included in the Map Output Data Formats, e.g.<br>
<font color="blue"><tt>Map Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> xmdf nc </tt><font color="green"><tt>! Output both xmdf and NetCDF formats</tt></font><br>
<br>
Additional NetCDF related commands are:<br>
There are a range of additional NetCDF related commands, to control the output further these are:<br>
<font color="blue"><tt>NetCDF Output Compression </tt></font> <font color="red"><tt>==</tt></font><tt> {OFF} | ON | compression level <0 – 9>
<font color="blue"><tt>NetCDF Output Compression </tt></font> <font color="red"><tt>==</tt></font><tt> OFF | {ON} | compression level <0 – 9>
</tt><br>
<font color="blue"><tt>NetCDF Output Start Date </tt></font> <font color="red"><tt>==</tt></font><tt> {2000-01-01 00:00} | OFF | <date in isodate format>
Line 10 ⟶ 11:
</tt><br>
<font color="blue"><tt>NetCDF Output Direction </tt></font> <font color="red"><tt>==</tt></font><tt> {ANGLE} | BEARING</tt><br>
<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> {GenericFEWS} | FEWSGeneric</tt><br>
<br>
Please refer to the TUFLOW manual for more details on these.
Please refer to the <u>[https://docs.tuflow.com/classic-hpc/manual/latest/ TUFLOW Manual]</u> for more details on these commands.
 
=File Format=
The output file has been created to work with GIS and the [http://www.delft-fews.nl/ Delft-FEWS software] and therefore should be compliant with the [http://cfconventions.org/ NetCDF CF 1.6 conventions]. It is noted that the [http://cfconventions.org/standard-names.html CF 1.6 standard names] do not include any allowance for number of the output names such as '''Hazard''' and '''Time of Peak''' but these are still output.<br>
There are currently two output formats from TUFLOW, this is controlled with the command:<br>
There are two slightly different formats (<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> {Generic} | FEWS</tt><br>), which differ only in the storing of the [[TUFLOW_NetCDF_Raster_Output_Format#Non_Temporal_Variables | Non Temporal Variables]].
The file formats are very similar, but for the FEWS format an additional time dimension and time variable "static_time" are used to store a time for the maximum and time of peak data so that it loads into the FEWS interface at an appropriate time.
=Generic=
This is the default output format and can also be specified by including "<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> Generic</tt>" in the .tcf file.
 
==Global Attributes==
=FEWS=
The NetCDF file has four global attributes:<br>
To output in this format users will need to specify "<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> FEWS</tt>" in the .tcf file.
* Title - The simulation name
* Source - The TUFLOW version which has been used
* References - The documentation of the NetCDF file, i.e. this page!
* Comment - A description of the output.
* NetCDF File Format Version - A unique value for file version, for the 2017 version this is 2.
* NetCDF Data Type - This is 1 for "NC" raster output and 2 for <u>[[TUFLOW_NetCDF_Cell_Centre_Output_Format | "NC" cell centered results]]</u>.
An example of the '''Global Attributes''' is:
<pre>
Global Attributes:
title = 'M01_5m_002_NC2'
source = 'TUFLOW Build: 2015-12-BB-Dev-iSP-w64'
references = 'TUFLOW NetCDF Raster Output Format (http://wiki.tuflow.com/index.php?title=TUFLOW_NetCDF_Raster_Output_Format)'
comment = 'Contains raster TUFLOW output, TUFLOW netcdf version: 1'
</pre>
 
==Dimensions==
The output NetCDF will have three or four dimensions depending on if the '''Generic''' or '''FEWS''' format has been specified. These are:
* x - the number of x coordinates (columns of data)
* y - the number of y coordinates (rows of data)
* time - the number of outputs. Note that this is an unlimited dimension meaning that the number of time changes each time a new output is written
* static_time (FEWS Format only) - This has a size of 1 and is used for storing a time for the static data (such as maximums)
An example of the '''Dimensions''' is:
<pre>
Dimensions:
x = 1946
y = 856
static_time = 1
time = 161 (UNLIMITED)
</pre>
==Variables==
All outputs for the simulation are written to a single NetCDF file. These are split into three categories:
* Common variables
* Temporal variables
* Non Temporal variables.
For example if the model writes depths our every 10 minutes and tracking maximums is enabled (default) then two depth datasets are written: '''depths''' and '''maximum_depths'''. These are described further below.
 
===Common Variables===
For each simulation the following variables are written:<br>
* x - contains the x coordinates of the output grid
* y - contains the y coordinates of the output grid
* time - contains the times for the temporal outputs
* static time (FEWS only) - contains a single time which is used by FEWS to display the non time varying datasets (such as maximums)
Each of these variables has only a single dimension.
<pre>
time
Size: 7x1
Dimensions: time
Datatype: double
Attributes:
standard_name = 'time'
long_name = 'time'
units = 'hours since 2000-01-01 00:00'
axis = 'T'
y
Size: 467x1
Dimensions: y
Datatype: double
Attributes:
standard_name = 'projection_y_coordinate'
long_name = 'y-coordinate in cartesian system'
units = 'm'
axis = 'Y'
x
Size: 422x1
Dimensions: x
Datatype: double
Attributes:
standard_name = 'projection_y_coordinate'
long_name = 'x-coordinate in cartesian system'
units = 'm'
axis = 'X'
static_time
Size: 1x1
Dimensions: static_time
Datatype: double
Attributes:
standard_name = 'static_time'
long_name = 'static_time'
units = 'hours since 2000-01-01 00:00'
</pre>
 
===Temporal Result Variables===
For each of the map outputs data types specified in the control file, the outputs are written into a variable that has the dimensions x, y and time.<br>
For vector variables, this is split into two components, for example for the velocity output this is split into "magnitude_of_velocity" and "direction_of_velocity". The notation for the direction can be controlled with the command:<br>
<font color="blue"><tt>NetCDF Output Direction </tt></font> <font color="red"><tt>==</tt></font><tt> {ANGLE} | BEARING</tt><br>
<br>
An example of the temporal variables for a simulation with the h, d and v outputs is below:
<pre>
water_level
Size: 1946x856x161
Dimensions: x,y,time
Datatype: single
Attributes:
_FillValue = -999
depth
Size: 1946x856x161
Dimensions: x,y,time
Datatype: single
Attributes:
_FillValue = -999
magnitude_of_velocity
Size: 1946x856x161
Dimensions: x,y,time
Datatype: single
Attributes:
_FillValue = -999
direction_of_velocity
Size: 1946x856x161
Dimensions: x,y,time
Datatype: single
Attributes:
_FillValue = -999
</pre>
 
===Non Temporal Variables===
There are currently two output formats from TUFLOW, this is controlled with the command:<br>
<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> {FEWS} | Generic</tt><br>
The file formats are very similar, but for the FEWS format (default) the non-temporal variables have a time associated with them to allow them to display correctly in FEWS.
For the generic output (which requires "<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> Generic</tt>"), the data is stored in an array that has the dimensions x and y:
<pre>
maximum_depth
Size: 1946x856
Dimensions: x,y
Datatype: single
Attributes:
_FillValue = -999
</pre>
<br>
For the FEWS output (which is the default or has "<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> FEWS</tt>" specified), the data is stored in an array that has the dimensions x, y and static_time:
<pre>
maximum_depth
Size: 1946x856x1
Dimensions: x,y,static_time
Datatype: single
Attributes:
_FillValue = -999
</pre>