Latest revision as of 10:21, 23 September 2024

Introduction

This page contains information on the NetCDF file format used for gridded (raster) outputs from a TUFLOW simulation. To specify NetCDF output from TUFLOW, the 2016 version or later must be used and "NC" must be included in the Map Output Data Formats, e.g.
Map Output Format == xmdf nc ! Output both xmdf and NetCDF formats

There are a range of additional NetCDF related commands, to control the output further these are:
NetCDF Output Compression ==OFF | {ON} | compression level <0 – 9>
NetCDF Output Start Date =={2000-01-01 00:00} | OFF | <date in isodate format>
NetCDF Output Time Unit ==DAY | {HOUR} | MINUTE
NetCDF Output Direction == {ANGLE} | BEARING
NetCDF Output Format == {FEWS} | Generic

Please refer to the TUFLOW Manual for more details on these commands.

File Format

The output file has been created to work with GIS and the Delft-FEWS software and therefore should be compliant with the NetCDF CF 1.6 conventions. It is noted that the CF 1.6 standard names do not include any allowance for number of the output names such as Hazard and Time of Peak but these are still output.
There are two slightly different formats (NetCDF Output Format == {Generic} | FEWS), which differ only in the storing of the Non Temporal Variables.

Global Attributes

The NetCDF file has four global attributes:

Title - The simulation name
Source - The TUFLOW version which has been used
References - The documentation of the NetCDF file, i.e. this page!
Comment - A description of the output.
NetCDF File Format Version - A unique value for file version, for the 2017 version this is 2.
NetCDF Data Type - This is 1 for "NC" raster output and 2 for "NC" cell centered results.

An example of the Global Attributes is:

Global Attributes:
           title      = 'M01_5m_002_NC2'
           source     = 'TUFLOW Build: 2015-12-BB-Dev-iSP-w64'
           references = 'TUFLOW NetCDF Raster Output Format (http://wiki.tuflow.com/index.php?title=TUFLOW_NetCDF_Raster_Output_Format)'
           comment    = 'Contains raster TUFLOW output, TUFLOW netcdf version: 1'

Dimensions

The output NetCDF will have three or four dimensions depending on if the Generic or FEWS format has been specified. These are:

x - the number of x coordinates (columns of data)
y - the number of y coordinates (rows of data)
time - the number of outputs. Note that this is an unlimited dimension meaning that the number of time changes each time a new output is written
static_time (FEWS Format only) - This has a size of 1 and is used for storing a time for the static data (such as maximums)

An example of the Dimensions is:

Dimensions:
           x           = 1946
           y           = 856
           static_time = 1
           time        = 161   (UNLIMITED)

Variables

All outputs for the simulation are written to a single NetCDF file. These are split into three categories:

Common variables
Temporal variables
Non Temporal variables.

For example if the model writes depths our every 10 minutes and tracking maximums is enabled (default) then two depth datasets are written: depths and maximum_depths. These are described further below.

Common Variables

For each simulation the following variables are written:

x - contains the x coordinates of the output grid
y - contains the y coordinates of the output grid
time - contains the times for the temporal outputs
static time (FEWS only) - contains a single time which is used by FEWS to display the non time varying datasets (such as maximums)

Each of these variables has only a single dimension.

    time 
           Size:       7x1
           Dimensions: time
           Datatype:   double
           Attributes:
                       standard_name = 'time'
                       long_name     = 'time'
                       units         = 'hours since 2000-01-01 00:00'
                       axis          = 'T'
    y    
           Size:       467x1
           Dimensions: y
           Datatype:   double
           Attributes:
                       standard_name = 'projection_y_coordinate'
                       long_name     = 'y-coordinate in cartesian system'
                       units         = 'm'
                       axis          = 'Y'
    x    
           Size:       422x1
           Dimensions: x
           Datatype:   double
           Attributes:
                       standard_name = 'projection_y_coordinate'
                       long_name     = 'x-coordinate in cartesian system'
                       units         = 'm'
                       axis          = 'X'
    static_time                  
           Size:       1x1
           Dimensions: static_time
           Datatype:   double
           Attributes:
                       standard_name = 'static_time'
                       long_name     = 'static_time'
                       units         = 'hours since 2000-01-01 00:00'

Temporal Result Variables

For each of the map outputs data types specified in the control file, the outputs are written into a variable that has the dimensions x, y and time.
For vector variables, this is split into two components, for example for the velocity output this is split into "magnitude_of_velocity" and "direction_of_velocity". The notation for the direction can be controlled with the command:
NetCDF Output Direction == {ANGLE} | BEARING

An example of the temporal variables for a simulation with the h, d and v outputs is below:

    water_level                  
           Size:       1946x856x161
           Dimensions: x,y,time
           Datatype:   single
           Attributes:
                       _FillValue = -999
    depth                        
           Size:       1946x856x161
           Dimensions: x,y,time
           Datatype:   single
           Attributes:
                       _FillValue = -999
    magnitude_of_velocity        
           Size:       1946x856x161
           Dimensions: x,y,time
           Datatype:   single
           Attributes:
                       _FillValue = -999
    direction_of_velocity        
           Size:       1946x856x161
           Dimensions: x,y,time
           Datatype:   single
           Attributes:
                       _FillValue = -999

Non Temporal Variables

There are currently two output formats from TUFLOW, this is controlled with the command:
NetCDF Output Format == {FEWS} | Generic
The file formats are very similar, but for the FEWS format (default) the non-temporal variables have a time associated with them to allow them to display correctly in FEWS. For the generic output (which requires "NetCDF Output Format == Generic"), the data is stored in an array that has the dimensions x and y:

    maximum_depth                
           Size:       1946x856
           Dimensions: x,y
           Datatype:   single
           Attributes:
                       _FillValue = -999

For the FEWS output (which is the default or has "NetCDF Output Format == FEWS" specified), the data is stored in an array that has the dimensions x, y and static_time:

    maximum_depth                
           Size:       1946x856x1
           Dimensions: x,y,static_time
           Datatype:   single
           Attributes:
                       _FillValue = -999

@@ Line 1: / Line 1: @@
 =Introduction=
-This page contains information on the NetCDF file format used for gridded (raster) outputs from a TUFLOW simulation.  To specify NetCDF output from TUFLOW the 2016 version or later of TUFLOW must be used and "NC" must be included in the Map Output Data Formats, e.g.<br>
+This page contains information on the NetCDF file format used for gridded (raster) outputs from a TUFLOW simulation.  To specify NetCDF output from TUFLOW, the 2016 version or later must be used and "NC" must be included in the Map Output Data Formats, e.g.<br>
 <font color="blue"><tt>Map Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> xmdf nc </tt><font color="green"><tt>! Output both xmdf and NetCDF formats</tt></font><br>
-Additional NetCDF related commands are:<br>
+<br>
-<font color="blue"><tt>NetCDF Output Compression </tt></font> <font color="red"><tt>==</tt></font><tt> {OFF} | ON | compression level <0 – 9>
+There are a range of additional NetCDF related commands, to control the output further these are:<br>
+<font color="blue"><tt>NetCDF Output Compression </tt></font> <font color="red"><tt>==</tt></font><tt> OFF | {ON} | compression level <0 – 9>
 </tt><br>
 <font color="blue"><tt>NetCDF Output Start Date </tt></font> <font color="red"><tt>==</tt></font><tt> {2000-01-01 00:00} | OFF | <date in isodate format>
@@ Line 10: / Line 11: @@
 </tt><br>
 <font color="blue"><tt>NetCDF Output Direction </tt></font> <font color="red"><tt>==</tt></font><tt> {ANGLE} | BEARING</tt><br>
-<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> {Generic} | FEWS</tt><br>
+<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> {FEWS} | Generic</tt><br>
-Please refer to the TUFLOW manual for more details on these.
+<br>
+Please refer to the <u>[https://docs.tuflow.com/classic-hpc/manual/latest/ TUFLOW Manual]</u> for more details on these commands.
 =File Format=
+The output file has been created to work with GIS and the [http://www.delft-fews.nl/ Delft-FEWS software] and therefore should be compliant with the [http://cfconventions.org/ NetCDF CF 1.6 conventions].  It is noted that the [http://cfconventions.org/standard-names.html CF 1.6 standard names] do not include any allowance for number of the output names such as '''Hazard''' and '''Time of Peak''' but these are still output.<br>
+There are two slightly different formats (<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> {Generic} | FEWS</tt>), which differ only in the storing of the [[TUFLOW_NetCDF_Raster_Output_Format#Non_Temporal_Variables | Non Temporal Variables]].
+==Global Attributes==
+The NetCDF file has four global attributes:<br>
+* Title - The simulation name
+* Source - The TUFLOW version which has been used
+* References - The documentation of the NetCDF file, i.e. this page!
+* Comment - A description of the output.
+* NetCDF File Format Version - A unique value for file version, for the 2017 version this is 2.
+* NetCDF Data Type - This is 1 for "NC" raster output and 2 for <u>[[TUFLOW_NetCDF_Cell_Centre_Output_Format | "NC" cell centered results]]</u>.
+An example of the '''Global Attributes''' is:
+<pre>
+Global Attributes:
+           title      = 'M01_5m_002_NC2'
+           source     = 'TUFLOW Build: 2015-12-BB-Dev-iSP-w64'
+           references = 'TUFLOW NetCDF Raster Output Format (http://wiki.tuflow.com/index.php?title=TUFLOW_NetCDF_Raster_Output_Format)'
+           comment    = 'Contains raster TUFLOW output, TUFLOW netcdf version: 1'
+</pre>
+==Dimensions==
+The output NetCDF will have three or four dimensions depending on if the '''Generic''' or '''FEWS''' format has been specified.  These are:
+* x - the number of x coordinates (columns of data)
+* y - the number of y coordinates (rows of data)
+* time - the number of outputs.  Note that this is an unlimited dimension meaning that the number of time changes each time a new output is written
+* static_time (FEWS Format only) - This has a size of 1 and is used for storing a time for the static data (such as maximums)
+An example of the '''Dimensions''' is:
+<pre>
+Dimensions:
+           x           = 1946
+           y           = 856
+           static_time = 1
+           time        = 161   (UNLIMITED)
+</pre>
+==Variables==
+All outputs for the simulation are written to a single NetCDF file.  These are split into three categories:
+* Common variables
+* Temporal variables
+* Non Temporal variables.
+For example if the model writes depths our every 10 minutes and tracking maximums is enabled (default) then two depth datasets are written: '''depths''' and '''maximum_depths'''.  These are described further below.
+===Common Variables===
+For each simulation the following variables are written:<br>
+* x - contains the x coordinates of the output grid
+* y - contains the y coordinates of the output grid
+* time - contains the times for the temporal outputs
+* static time (FEWS only) - contains a single time which is used by FEWS to display the non time varying datasets (such as maximums)
+Each of these variables has only a single dimension.
+<pre>
+    time
+           Size:       7x1
+           Dimensions: time
+           Datatype:   double
+           Attributes:
+                       standard_name = 'time'
+                       long_name     = 'time'
+                       units         = 'hours since 2000-01-01 00:00'
+                       axis          = 'T'
+    y
+           Size:       467x1
+           Dimensions: y
+           Datatype:   double
+           Attributes:
+                       standard_name = 'projection_y_coordinate'
+                       long_name     = 'y-coordinate in cartesian system'
+                       units         = 'm'
+                       axis          = 'Y'
+    x
+           Size:       422x1
+           Dimensions: x
+           Datatype:   double
+           Attributes:
+                       standard_name = 'projection_y_coordinate'
+                       long_name     = 'x-coordinate in cartesian system'
+                       units         = 'm'
+                       axis          = 'X'
+    static_time
+           Size:       1x1
+           Dimensions: static_time
+           Datatype:   double
+           Attributes:
+                       standard_name = 'static_time'
+                       long_name     = 'static_time'
+                       units         = 'hours since 2000-01-01 00:00'
+</pre>
+===Temporal Result Variables===
+For each of the map outputs data types specified in the control file, the outputs are written into a variable that has the dimensions x, y and time.<br>
+For vector variables, this is split into two components, for example for the velocity output this is split into "magnitude_of_velocity" and "direction_of_velocity".  The notation for the direction can be controlled with the command:<br>
+<font color="blue"><tt>NetCDF Output Direction </tt></font> <font color="red"><tt>==</tt></font><tt> {ANGLE} | BEARING</tt><br>
+<br>
+An example of the temporal variables for a simulation with the h, d and v outputs is below:
+<pre>
+    water_level
+           Size:       1946x856x161
+           Dimensions: x,y,time
+           Datatype:   single
+           Attributes:
+                       _FillValue = -999
+    depth
+           Size:       1946x856x161
+           Dimensions: x,y,time
+           Datatype:   single
+           Attributes:
+                       _FillValue = -999
+    magnitude_of_velocity
+           Size:       1946x856x161
+           Dimensions: x,y,time
+           Datatype:   single
+           Attributes:
+                       _FillValue = -999
+    direction_of_velocity
+           Size:       1946x856x161
+           Dimensions: x,y,time
+           Datatype:   single
+           Attributes:
+                       _FillValue = -999
+</pre>
+===Non Temporal Variables===
 There are currently two output formats from TUFLOW, this is controlled with the command:<br>
-<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> {Generic} | FEWS</tt><br>
+<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> {FEWS} | Generic</tt><br>
-The file formats are very similar, but for the FEWS format an additional time dimension and time variable "static_time" are used to store a time for the maximum and time of peak data so that it loads into the FEWS interface at an appropriate time.
+The file formats are very similar, but for the FEWS format (default) the non-temporal variables have a time associated with them to allow them to display correctly in FEWS.
-=Generic=
+For the generic output (which requires "<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> Generic</tt>"), the data is stored in an array that has the dimensions x and y:
-This is the default output format and can also be specified by including "<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> Generic</tt>" in the .tcf file.
+<pre>
+    maximum_depth
-=FEWS=
+           Size:       1946x856
-To output in this format users will need to specify "<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> FEWS</tt>" in the .tcf file.
+           Dimensions: x,y
+           Datatype:   single
+           Attributes:
+                       _FillValue = -999
+</pre>
+<br>
+For the FEWS output (which is the default or has "<font color="blue"><tt>NetCDF Output Format </tt></font> <font color="red"><tt>==</tt></font><tt> FEWS</tt>" specified), the data is stored in an array that has the dimensions x, y and static_time:
+<pre>
+    maximum_depth
+           Size:       1946x856x1
+           Dimensions: x,y,static_time
+           Datatype:   single
+           Attributes:
+                       _FillValue = -999
+</pre>

Difference between revisions of "TUFLOW NetCDF Raster Output Format"