QGIS ARR to TUFLOW

The ARR to TUFLOW utility has been developed to help users set up a TUFLOW model that uses Australian Rainfall and Runoff (ARR) input parameters and the Australian Bureau of Meteorology (BOM) rainfall by automating the collection and processing of the data. The inputs and outputs of the tool are explained in detail below. Note: this tool helps with data processing, however is not a substitute to reading ARR. Similarly this wiki page discusses the use of the tool and the options, however context should be derived from ARR directly. Please see the ARR Website for more information.

Getting Started

Installation

The ARR to TUFLOW tool is a free tool that comes as part of the TUFLOW plugin in QGIS. For instructions on how to install the plugin, please see: QGIS TUFLOW Plugin Installation

There are also instructions on installing plugins in the QGIS documentation - if you choose to follow the QGIS documentation, the plugin is called "TUFLOW" in the repository: Link to QGIS Documentation - Installing and Managing Plugins

QGIS Version

The ARR to TUFLOW tool is, for the most part (see note below), a Python tool that runs independently of QGIS. Therefore the tool should not be sensitive to the installed QGIS version, however occasionally QGIS will update the Python version that is packaged with the installation which means there is a small risk that the tool will break between QGIS versions. If the tool does break, please see The Tool Isn't Working for information (the solution is most likely to update the plugin rather than re-install an older version of QGIS).

Note: The tool does use QGIS for pre-processing tasks, such as area and centroid calculation as well as reprojection or cartesian to long-lat.

Opening The ARR to TUFLOW Tool

Once the TUFLOW plugin is installed, the ARR to TUFLOW tool can be opened by clicking the following icon in the plugin toolbar:

Quickstart

1. Load a catchment GIS file (e.g. shp file) into QGIS - this can either be a polygon or point layer
2. Open the ARR to TUFLOW tool
3. Select the GIS catchment layer in Input Catchment File dropdown box.
   
   
   
4. Select the attribute field to use as the catchment name in the Unique Catchment Identifier Field dropdown.
   
   
   
5. Select storm event combinations from the list of magnitudes and durations.
   
   
   
6. Choose an output location.
   
   
   
7. Click OK and wait for the tool to finish.
   

Output

Log File

A log file (<catchment_name>_log.txt) will be written to the output folder. The log file will contain all WARNING and ERROR messages as well as logged inputs and processing steps. It's recommended to check the log file after running the tool for any warning or error messages. Most warning messages will be grouped together near the bottom of the log file to make it easier to review them.

Data Output

The following outputs are located in the folder <output_folder>\data

1. Raw BOM and ARR Datahub outputs
   • ARR_Web_data_<catchment_name>.txt
   • BOM_raw_web_<catchment_name>.html
   • Areal_<TP_Zone>_Increments.csv (Areal temporal patterns)
2. Rainfall depths
   • BOM_Rainfall_Depths_<catchment_name>.csv - processed rainfall depths (after ARF factors)
3. ARF Factors
   • <catchment_name>_ARF_Factors.csv - calculated ARF factors
4. Storm Burst Rainfall Losses
   • <catchment_name>_Burst_Initial_Loss.csv - calculated Storm Burst Initial Rainfall Losses (Complete Storm Initial Loss - Preburst Rainfall Depth)

TUFLOW Output

The following outputs are generated by the utility for input into TUFLOW. Please see the example below on how to use the output files for a model.

1. TUFLOW Rainfall Inflows
   • The rainfall inflows are output in the following location <output_folder>\rf_inflows. This output folder is where all the direct rainfall inflows that would be used in the TUFLOW model are output. An inflow file is generated for each event magnitude - duration combination. All available temporal patterns are included in a single file (including climate change options).
2. bc_dbase.csv
   • TUFLOW boundary condition database. Setup to point to the RF folder with wildcards applied.
3. event_file.tef
   • TUFLOW Event File. Setup with wildcards applied. To keep the file names from becoming overly long and complicated, Climate Change scenarios are labelled as CC1, CC2.. etc. These correspond to the combinations of climate change scenarios requested by the user e.g. 2090_8.5 (climate change year: 2090, RCP: 8.5)
4. soil_infiltration.trd or rainfall_losses.trd - the name of the file will be dependent on the selected loss method, but they are essentially the same file
   • This file is a TUFLOW Read File (.trd) that contains variable initialisation for initial and continuing loss values
5. soils.tsoilf or materials.csv - which file is output will be dependent on the selected loss method
   • soils.tsoilf - rainfall loss file for the infiltration loss method
   • materials.csv - rainfall loss file for the rainfall excess method

Example: Using the outputs in a TUFLOW model

• Example use of the ARR to TUFLOW tool

Additional Options

Climate Change

Climate change options can be added by clicking on Edit under the climate change section. This opens up a table widget that the user can then add/remove climate change scenarios based on the guidance in ARR v4.2. Scenarios added in the table will be displayed in the climate change section (underneath the edit button). Note: each scenario name must be unique and generally the table widget will force unique names. The scenario names will be used throughout the outputs and the user is free to customise the scenario names as they want.

The table columns within the table widget are based on ARR guidelines v4.2 and the examples presented in Book 1, Chapter 6. Please note the following assumptions:

• Near-term horizon = 2030
• Medium-term horizon = 2050
• Long-term horizon = 2090
• Temperature Change - Setting this to -1 will adopt the temperature changes output in the climate change part of the ARR datahub download. If a value or zero or greater is set by the user, this will be adopted instead of the datahub values and it will be assumed that these are changes relative to the pre-industrial period. Note: the temperature change will be stopped from going negative within the calculation.

By adding climate change scenarios, the following outputs will be turned on:

• An extra boundary database file: bc_dbase_CC.csv
• Climate change inflows will be appended to the end of each event's inflow file
• One additional event option will also be added to the event file for each combination of selected climate change scenarios labelled based on the scenario name. It is intended that the climate change scenarios would be treated as a separate event (e1 = magnitude, e2 = duration, e3 = temporal pattern, e4 = climate change scenario) however the output files can easily be manipulated by the user to configure the model differently.
• The following CSV files will be written to the data output folder:
  • <scen_name>_rainfall_factors.csv
  • <scen_name>_rainfall_depths.csv
  • <scen_name>_losses.csv
    
    

Temporal Patterns

Temporal pattern options can be found by expanding the Temporal Patterns section in the dialog.

• Manually Specify Point Temporal Patterns (csv) - lets the user specify the point temporal patterns which is expected to be in the ARR datahub format
• Manually Specify Areal Temporal Patterns (csv) - lets the user specify the arealtemporal patterns which is expected to be in the ARR datahub format
• Optional Additional Temporal Patterns - Lets the user add additional temporal patterns in a couple of different ways:
  
  • The user can choose to add temporal patterns from other temporal pattern regions. This is only if the user wishes to included additional temporal patterns. The temporal patterns from the catchment location (taken at the catchment centroid) will always be included. The additional temporal patterns will be included in the inflow files and added to the event file.
  • The user can choose to add temporal patterns from other temporal pattern bins within the same region. For point temporal patterns, this would add temporal patterns from the 'frequent', 'intermediate', and 'rare' bins. For areal temporal patterns, this would add temporal patterns from other "Area" bins. The "correct" bin will always make up the first 10 temporal patterns (TP01 to TP10).
    • For Areal temporal patterns - The user has the additional option to choose how many additional areal temporal pattern sets to add. The default is 2 as this matches the number of additional point temporal patterns that would be added (but it doesn't necessarily have to be the same). The "Area" bins chosen will be the next closest in order.

Rainfall

LIMB Rainfall Data
Since v3.8.2.16 ( development version), LIMB data will be used for rainfall depths where available (SEQ specific). The user has the option of selecting the IFD curve from the available sets:

• Enveloped (Default)
• High Resolution
• BOM Resolution

The user also has the option of turning off LIMB data and using BOM 2016 rainfall data instead. If LIMB data is not available, the tool will default to BOM data.

If the user has selected AEP or durations outside of the provided LIMB data range, the tool will use BOM 2016 rainfall data. If the user has selected AEP or durations within the LIMB data range, but the value is not part of the provided IFD, the tool will interpolate the rainfall depth from the provided LIMB data.

Rainfall Losses

Since version v3.5 initial loss for durations greater than 72 hrs is assumed to be the same as the initial loss for 72 hrs. Previous to this version, these losses were output as zeros.

Probability Neutral Losses

Probability neutral losses can be toggled on / off in the dialog (default is on). If 'on', this option will use probability neutral losses for the design burst initial loss rather than using the storm initial loss and pre-burst depths. If probability neutral losses are not available in the catchment area, then the method will automatically revert to using the storm initial loss). Currently this option cannot be used in conjunction with the complete storm option.

NSW Continuing Loss

Continuing loss values for NSW will be automatically multiplied by 0.4. This will be reported in the log file as:

• "Catchment is in NSW, multiplying Datahub continuing loss by 0.4"

Pre-burst Percentile

The pre-burst percentile used to calculate design burst initial loss, or for pre-burst rainfall depth for the complete storm option, can be changed using the dropdown box in the dialog.

User Defined Losses

Users can use their own rainfall loss values. The initial loss value will be treated as a complete storm value and not just for the design burst. For most regions, the design burst loss will be calculated my removing the pre-burst depth from the input loss value. If probability neutral losses are available, the burst initial loss will be calculated using the following equation:

IL burst = User IL x IL ARR Prob Neutral / IL ARR Complete Storm

Event Independent Continuing Loss

The event independent continuing loss only affects how the TUFLOW output files are written. This option will remove the continuing loss variable and will instead input the value directly. Ths option was added since the continuing loss value is (currently) event independent and therefore simplifies the output and also makes editing this value easier.

Impervious Losses

The impervious loss value adds the impervious loss values to the materials.csv output. This option only affects the rainfall excess loss approach.

TUFLOW Loss Method

The loss method used by TUFLOW can be changed using the TUFLOW Loss Method dropdown box:

• Infiltration (soil file) - will output files for the infiltration approach in TUFLOW. This approach in TUFLOW applies the total rainfall to the 2D grid and ponded water in the 2D domain can then infiltrate based on the underlying soil type and impervious fraction. See TUFLOW Manual for more information.
• Rainfall Excess (material file) - will output files for the rainfall excess approach in TUFLOW. This approach removes the losses from the rainfall inflow prior to application so only excess rainfall is added to the model.

Initial Losses For Durations Less Than 1 Hour

This selects the method for determining intial loss values for events less than 1 hr in duration:

• Interpolate to zero - will linearly interpolate loss values (assuming a loss value of 0 mm for 0 min)
• Log-Interpolate to zero - will use a log-linear interpolation (duration will use a log scale) (assuming a loss value of 0 mm for 0 min)
• Static Value - uses a user defined value for all event magnitudes
• Use 60 min Losses - uses the 60 min loss values
• Rahmen et al 2002 - uses an equation determined by a study conducted by Rahmen et all in 2002
• Hill et al 1996:1998 - uses an equation determined by a study conducted by Hill et all 1996:1998. This requires the user to provide a mean annual rainfall value.
• Interpolate pre-burst - will linearly interpolate pre-burst depths (rather than losses) (assuming a depth of 0 mm for 0 min)
• Log-Interpolate pre-burst - will use a log-linear interpolation on the pre-burst depths (duration will use a log scale) (assuming a depth of 0 mm for 0 min)

An example of the differences in initial losses from a few of the methods:

Output Complete Storm

The user can choose to output the complete storm (preburst + design burst) rather than just the design burst by checking on the following option (please see ARR Book 2 Chapter 5 [Section 5.9.9] for more information):

The complete storm generated by the tool will be the preburst depth (from the Preburst percentile extracted from the datahub) plus the design burst. The user will be required to select a preburst temporal pattern from the options listed below. The options below have been included at the request of users and are not a direct reference to any specific methodology outline in ARR. If you would like to see another method included, please email support@tuflow.com.
Currently the ARR to TUFLOW tool supports the following preburst temporal pattern methods:

• Constant Rate - This will apply the preburst depth equally over a given time period. The time period can be specified as an absolute time (either as minutes or hours) or proprotional to the design storm e.g. a proportional value of 0.5 will apply a 30 min preburst to a 1 hr storm and a 15 min preburst to a 30 min storm.
• Temporal Pattern - This will apply the preburst depth using a temporal pattern obtained from the datahub for a given duration. The preburst duration can either be an absolute time or be proportional to the design storm. The user will need to select which temporal pattern to use (e.g. TP01, TP02,... TP10). Note: there is a minimum preburst duration of 10 min for this method.
  • If "design burst" is selected, this will cause the pre-burst temporal pattern to use the same temporal pattern number as the design burst (e.g. TP01 design burst will use TP01 pre-burst, TP02 design burst will use TP02 pre-burst, etc.)

The initial loss value output in this method will be the raw storm initial loss value from the datahub with no post processing. Note, the complete storm option and using probability neutral losses are mutually exclusive and aren't able to be used together.

Preburst temporal pattern and duration details are ignored if the "Use Complete Storm" checkbox is not selected.

Why Is the Tool Outputting Negative Loss Values?

The tool will output negative loss values when the preburst depth is greater than the storm initial loss. The tool will not floor loss values at zero since the value may be useful in helping the user determine the appropriate course of action. E.g. if IL = -1mm, this may be considered negligible and can be simply set to zero. An IL = -50mm may be considered significant enough that it simply can't be set to zero and alternate, or sensitivity runs, may need to be carried out, like using a Complete Storm.

Areal Reduction Factors

Areal reduction factor options can be changed in the ARF section:

• Calculate Area - will calculate the catchment area using the GIS feature in QGIS. For point objects, this will be set to zero and no ARF will be applied (i.e. a value of 1.0)
• User Area - will let the user define the area. This can either be done by selecting an attribute field that contains the catchment area, or by entering a value. All user inputs will be assumed to be in square km.
• Minimum ARF - this value sets a floor for the calculated ARF values
• ARF for events < 50% AEP - this will calculate ARF values using the same equations used for larger events

Additional Output Options

Additional output options include:

• Output Format - can choose to output in either csv or ts1 format. Ts1 files will be quicker to read in by TUFLOW, however the difference may not be substantial depending on how many are being read in. TUFLOW will also use .xf files by default which means that the raw data is only processed if there is a change, and subsequent runs will not suffer from any slow down from having to read csv files.
• Output Notation - users can toggle between '% AEP' and 'yr ARI' output notation.

Offline Mode

Offline can be used to pass in data that has already been pulled from the ARR datahub and BOM. This can be useful if the user would like to make use of new features in the tool, however would like to ensure they are getting consistent results with previous data that was pulled from the BOM and the ARR datahub. The files required to process offline are saved as part of the tool's process when pulling from the web. Currently offline mode doesn't support areal temporal patterns directly, however this can easily be added by using the manually specify areal temporal pattern option:

• ARR_Web_data_<catchment_name>.txt
• BOM_raw_web_<catchment_name>.html

The Tool Isn't Working

The first step in identifying the problem is to check the log file. The issue may be something simple like:

• one of the files the tool is trying to write to is locked by a process or user (e.g. one of the outputs is open in Excel)
• the tool can't access a network location
• there is no internet connection
• the ARR Datahub or BOM is down (see Can't Access the Websites section below)

A useful check if you're unsure if the issue is with the tool or with something else (like the Datahub), is to check if you can manually access and pull data from the Datahub for your catchment site.

The second step is to ensure you're using the latest version of the QGIS plugin. Occasionally updates to the ARR Datahub will break the tool and this may have already been caught and fixed in the latest update.

Issue With The Tool

If there's an issue with the tool, or you are unsure, please contact support@tuflow.com for help. Please include a description of the problem, the log file, and the GIS catchment layer with your email.

Can't access the websites

Sometimes the BOM or ARR Datahub websites are down or unable to be accessed by the tool (e.g. sometimes to manage traffic, the Datahub server will limit, or disallow scraping). The below are a set of instructions that can be used as a workaround should this occur:

• ARR Datahub website cannot be accessed by the tool

Up
Back to TUFLOW QGIS Plugin Main Page