TUFLOW Simulation Speed - Revision history

Chris Huxley at 01:19, 10 December 2025

2025-12-10T01:19:30Z

← Older revision		Revision as of 11:19, 10 December 2025
Line 36:		Line 36:
	<ol>		<ol>
	* Increase your 2D cell size. Thanks to SGS the cell size can typically be increased whilst still achieving a good cell size convergence.		* Increase your 2D cell size. Thanks to SGS the cell size can typically be increased whilst still achieving a good cell size convergence.
	* Review your SGS Sample Frequency in the TLF. Typically, a value of 11 or less is all that is necessary to set the SGS parameterisation to a level that achieves excellent accuracy. If the SGS Sample Frequency is excessively large, test reducing the default value by using either the <font color="blue"><tt>SGS Sample Frequency </tt></font> ~~<font color="red">~~<tt>~~==</tt></font><tt>~~ or <font color="blue"><tt>SGS Sample Sample Target Distance </tt></font~~> <font color="red"~~><tt>~~==</tt></font><tt>~~ commands.		* Review your SGS Sample Frequency in the TLF. Typically, a value of 11 or less is all that is necessary to set the SGS parameterisation to a level that achieves excellent accuracy. If the SGS Sample Frequency is excessively large, test reducing the default value by using either the <font color="blue"><tt>SGS Sample Frequency</tt></font> <tt>or <font color="blue"><tt>SGS Sample Sample Target Distance </tt></font><tt>commands.
	</ol>		</ol>
	* Inputs might be clipped if their extent is much larger than the code, e.g. grids, material layers and so on. For example if a 100GB DEM is being input to TUFLOW with the majority of the DEM being outside the TUFLOW model, TUFLOW allocates memory to read the entire DEM in for processing. Memory for reading inputs is only used temporarily and is released after the processing of the input, e.g. the DEM.		* Inputs might be clipped if their extent is much larger than the code, e.g. grids, material layers and so on. For example if a 100GB DEM is being input to TUFLOW with the majority of the DEM being outside the TUFLOW model, TUFLOW allocates memory to read the entire DEM in for processing. Memory for reading inputs is only used temporarily and is released after the processing of the input, e.g. the DEM.

Chris Huxley: /* Common Questions Answered (FAQ) */

2025-12-10T01:18:21Z

Common Questions Answered (FAQ)

← Older revision		Revision as of 11:18, 10 December 2025
Line 33:		Line 33:
	* Using Quadtree may decrease RAM requirement as with Quadtree mesh only active code polygon is processed instead of the whole model domain rectangle used for HPC and Classic. Also, judicious refinement and using larger cells in non-focus areas will decrease the number of active cells and therefore the RAM requirement.		* Using Quadtree may decrease RAM requirement as with Quadtree mesh only active code polygon is processed instead of the whole model domain rectangle used for HPC and Classic. Also, judicious refinement and using larger cells in non-focus areas will decrease the number of active cells and therefore the RAM requirement.
	* The larger cell size, the less RAM is required. <u>[https://www.youtube.com/watch?v=1jWyIXHR_lM Cell size sensitivity testing]</u> can be carried out to find out the largest cell size without compromising model accuracy.		* The larger cell size, the less RAM is required. <u>[https://www.youtube.com/watch?v=1jWyIXHR_lM Cell size sensitivity testing]</u> can be carried out to find out the largest cell size without compromising model accuracy.
	* Using SGS ~~does~~ ~~increase~~ RAM requirement compared to a non SGS model with the same cell size. ~~However,~~ ~~thanks~~ ~~to SGS~~ the ~~cell size can typically be increased whilst still achieving a good cell size convergence and therefore~~ RAM ~~requirement~~ ~~may~~ be ~~lower~~ ~~than~~ the ~~original non-SGS model.~~		* Using Sub Grid Sampling (SGS) increases the RAM requirement compared to a non SGS model with the same cell size. If SGS increases the RAM beyond what is desired, consider the following:
			<ol>
			* Increase your 2D cell size. Thanks to SGS the cell size can typically be increased whilst still achieving a good cell size convergence.
			* Review your SGS Sample Frequency in the TLF. Typically, a value of 11 or less is all that is necessary to set the SGS parameterisation to a level that achieves excellent accuracy. If the SGS Sample Frequency is excessively large, test reducing the default value by using either the <font color="blue"><tt>SGS Sample Frequency </tt></font> <font color="red"><tt>==</tt></font><tt> or <font color="blue"><tt>SGS Sample Sample Target Distance </tt></font> <font color="red"><tt>==</tt></font><tt> commands.
			</ol>
	* Inputs might be clipped if their extent is much larger than the code, e.g. grids, material layers and so on. For example if a 100GB DEM is being input to TUFLOW with the majority of the DEM being outside the TUFLOW model, TUFLOW allocates memory to read the entire DEM in for processing. Memory for reading inputs is only used temporarily and is released after the processing of the input, e.g. the DEM.		* Inputs might be clipped if their extent is much larger than the code, e.g. grids, material layers and so on. For example if a 100GB DEM is being input to TUFLOW with the majority of the DEM being outside the TUFLOW model, TUFLOW allocates memory to read the entire DEM in for processing. Memory for reading inputs is only used temporarily and is released after the processing of the input, e.g. the DEM.
	* Large DEM and material grids can use up a lot of memory during model initialisation as the entire grid is read for processing. Re-tiling the grid into smaller sections will use up less memory as each smaller grid will be processed at the time avoiding high memory peaks.		* Large DEM and material grids can use up a lot of memory during model initialisation as the entire grid is read for processing. Re-tiling the grid into smaller sections will use up less memory as each smaller grid will be processed at the time avoiding high memory peaks.

Pavlina Monhartova: /* Common Questions Answered (FAQ) */

2025-07-04T04:45:43Z

Common Questions Answered (FAQ)

← Older revision		Revision as of 14:45, 4 July 2025
Line 34:		Line 34:
	* The larger cell size, the less RAM is required. <u>[https://www.youtube.com/watch?v=1jWyIXHR_lM Cell size sensitivity testing]</u> can be carried out to find out the largest cell size without compromising model accuracy.		* The larger cell size, the less RAM is required. <u>[https://www.youtube.com/watch?v=1jWyIXHR_lM Cell size sensitivity testing]</u> can be carried out to find out the largest cell size without compromising model accuracy.
	* Using SGS does increase RAM requirement compared to a non SGS model with the same cell size. However, thanks to SGS the cell size can typically be increased whilst still achieving a good cell size convergence and therefore RAM requirement may be lower than the original non-SGS model.		* Using SGS does increase RAM requirement compared to a non SGS model with the same cell size. However, thanks to SGS the cell size can typically be increased whilst still achieving a good cell size convergence and therefore RAM requirement may be lower than the original non-SGS model.

	* Inputs might be clipped if their extent is much larger than the code, e.g. grids, material layers and so on. For example if a 100GB DEM is being input to TUFLOW with the majority of the DEM being outside the TUFLOW model, TUFLOW allocates memory to read the entire DEM in for processing. Memory for reading inputs is only used temporarily and is released after the processing of the input, e.g. the DEM.		* Inputs might be clipped if their extent is much larger than the code, e.g. grids, material layers and so on. For example if a 100GB DEM is being input to TUFLOW with the majority of the DEM being outside the TUFLOW model, TUFLOW allocates memory to read the entire DEM in for processing. Memory for reading inputs is only used temporarily and is released after the processing of the input, e.g. the DEM.
	* Large DEM and material grids can use up a lot of memory during model initialisation as the entire grid is read for processing. Re-tiling the grid into smaller sections will use up less memory as each smaller grid will be processed at the time avoiding high memory peaks.		* Large DEM and material grids can use up a lot of memory during model initialisation as the entire grid is read for processing. Re-tiling the grid into smaller sections will use up less memory as each smaller grid will be processed at the time avoiding high memory peaks.
	* Inspecting .tlf file can help recognising which model features require the most RAM and more RAM efficient methods might be considered. For example, using gridded rainfall is more RAM efficient than using large number of rainfall polygons. Other features such as soil infiltration, hazard outputs and output zones all require additional memory.		* Inspecting .tlf file can help recognising which model features require the most RAM and more RAM efficient methods might be considered. For example, using gridded rainfall is more RAM efficient than using large number of rainfall polygons. Other features such as soil infiltration, hazard outputs and output zones all require additional memory.
	* Grid (raster) outputs from TUFLOW (including DEM check files and Map Output Formats ASC, FLT, NC, TGO and WRR), require significant memory for storing interpolation factors. An interpolation is required as these outputs are all north-south aligned, while TUFLOW models can include rotation, water level lines and varying cell sizes. By default the grid output resolution is half the 2D cell size. Increasing the output cell size with the .tcf command <font color="blue"><tt>Grid Output Cell Size </tt></font> <font color="red"><tt>==</tt></font><tt> <grid_output_cell_size></tt> can reduce the amount of memory required for this. Refer to the <u>[https://docs.tuflow.com/classic-hpc/manual/latest/ TUFLOW Manual]</u> for more information on grid outputs.		* Grid (raster) outputs from TUFLOW (including DEM check files and Map Output Formats ASC, FLT, NC, TGO and WRR), require significant memory for storing interpolation factors. An interpolation is required as these outputs are all north-south aligned, while TUFLOW models can include rotation, water level lines and varying cell sizes. By default the grid output resolution is half the 2D cell size. Increasing the output cell size with the .tcf command <font color="blue"><tt>Grid Output Cell Size </tt></font> <font color="red"><tt>==</tt></font><tt> <grid_output_cell_size></tt> can reduce the amount of memory required for this. Refer to the <u>[https://docs.tuflow.com/classic-hpc/manual/latest/ TUFLOW Manual]</u> for more information on grid outputs.
			* When using Quadtree the .qcf command <font color="blue"><tt>Quadtree Mesh Processing Method </tt></font> <font color="red"><tt>==</tt></font><tt> Memory Efficient</tt> can reduce the amount of memory, although the model initialisation might take longer.

	Other processes can slow TUFLOW simulations down if they are competing for resources. A few potential causes:		Other processes can slow TUFLOW simulations down if they are competing for resources. A few potential causes:
	* Running another model at the same time (as the CPU RAM is shared between both simulations)		* Running another model at the same time (as the CPU RAM is shared between both simulations)

Emilie Nielsen at 00:23, 23 September 2024

2024-09-23T00:23:42Z

← Older revision		Revision as of 10:23, 23 September 2024
Line 38:		Line 38:
	* Large DEM and material grids can use up a lot of memory during model initialisation as the entire grid is read for processing. Re-tiling the grid into smaller sections will use up less memory as each smaller grid will be processed at the time avoiding high memory peaks.		* Large DEM and material grids can use up a lot of memory during model initialisation as the entire grid is read for processing. Re-tiling the grid into smaller sections will use up less memory as each smaller grid will be processed at the time avoiding high memory peaks.
	* Inspecting .tlf file can help recognising which model features require the most RAM and more RAM efficient methods might be considered. For example, using gridded rainfall is more RAM efficient than using large number of rainfall polygons. Other features such as soil infiltration, hazard outputs and output zones all require additional memory.		* Inspecting .tlf file can help recognising which model features require the most RAM and more RAM efficient methods might be considered. For example, using gridded rainfall is more RAM efficient than using large number of rainfall polygons. Other features such as soil infiltration, hazard outputs and output zones all require additional memory.
	* Grid (raster) outputs from TUFLOW (including DEM check files and Map Output Formats ASC, FLT, NC, TGO and WRR), require significant memory for storing interpolation factors. An interpolation is required as these outputs are all north-south aligned, while TUFLOW models can include rotation, water level lines and varying cell sizes. By default the grid output resolution is half the 2D cell size. Increasing the output cell size with the .tcf command <font color="blue"><tt>Grid Output Cell Size </tt></font> <font color="red"><tt>==</tt></font><tt> <grid_output_cell_size></tt> can reduce the amount of memory required for this. Refer to ~~Section~~ 9.6.~~4 of the 2018~~ TUFLOW ~~manual~~ for more information on grid outputs.		* Grid (raster) outputs from TUFLOW (including DEM check files and Map Output Formats ASC, FLT, NC, TGO and WRR), require significant memory for storing interpolation factors. An interpolation is required as these outputs are all north-south aligned, while TUFLOW models can include rotation, water level lines and varying cell sizes. By default the grid output resolution is half the 2D cell size. Increasing the output cell size with the .tcf command <font color="blue"><tt>Grid Output Cell Size </tt></font> <font color="red"><tt>==</tt></font><tt> <grid_output_cell_size></tt> can reduce the amount of memory required for this. Refer to the <u>[https://docs.tuflow.com/classic-hpc/manual/latest/ TUFLOW Manual]</u> for more information on grid outputs.
	Other processes can slow TUFLOW simulations down if they are competing for resources. A few potential causes:		Other processes can slow TUFLOW simulations down if they are competing for resources. A few potential causes:
	* Running another model at the same time (as the CPU RAM is shared between both simulations)		* Running another model at the same time (as the CPU RAM is shared between both simulations)

Pavlina Monhartova: /* Common Questions Answered (FAQ) */

2023-12-27T05:05:21Z

Common Questions Answered (FAQ)

← Older revision		Revision as of 15:05, 27 December 2023
Line 30:		Line 30:
	** For more information on hardware specification, see <u>[[Hardware_Selection_Advice \| Hardware Selection Advice]]</u>.<br>		** For more information on hardware specification, see <u>[[Hardware_Selection_Advice \| Hardware Selection Advice]]</u>.<br>
	Reducing RAM requirements:		Reducing RAM requirements:
	* Model domain should be as tight to the model code (active areas) as possible - search the .tlf for "Isolating redundant perimeter sections", this will tell you how much larger the domain is than it needs to be as a percentage.		* Model domain should be as tight to the model code (active areas) as possible - search the .tlf for "Isolating redundant perimeter sections", this will tell you how much larger the domain is than it needs to be as a percentage.
		⚫	* Using Quadtree may decrease RAM requirement as with Quadtree mesh only active code polygon is processed instead of the whole model domain rectangle used for HPC and Classic. Also, judicious refinement and using larger cells in non-focus areas will decrease the number of active cells and therefore the RAM requirement.
	* The larger cell size, the less RAM is required. <u>[https://www.youtube.com/watch?v=1jWyIXHR_lM Cell size sensitivity testing]</u> can be carried out to find out the largest cell size without compromising model accuracy.		* The larger cell size, the less RAM is required. <u>[https://www.youtube.com/watch?v=1jWyIXHR_lM Cell size sensitivity testing]</u> can be carried out to find out the largest cell size without compromising model accuracy.
	* Using SGS does increase RAM requirement compared to a non SGS model with the same cell size. However, thanks to SGS the cell size can typically be increased whilst still achieving a good cell size convergence and therefore RAM requirement may be lower than the original non-SGS model.		* Using SGS does increase RAM requirement compared to a non SGS model with the same cell size. However, thanks to SGS the cell size can typically be increased whilst still achieving a good cell size convergence and therefore RAM requirement may be lower than the original non-SGS model.

⚫	* Using Quadtree may ~~also~~ decrease RAM requirement as with Quadtree mesh only active code polygon is processed instead of the whole model domain rectangle used for HPC. Also, judicious refinement and using larger cells in non-focus areas will decrease the number of active cells and therefore the RAM requirement.
	* Inputs might be clipped if their extent is much larger than the code, e.g. grids, material layers and so on. For example if a 100GB DEM is being input to TUFLOW with the majority of the DEM being outside the TUFLOW model, TUFLOW allocates memory to read the entire DEM in for processing. Memory for reading inputs is only used temporarily and is released after the processing of the input, e.g. the DEM.		* Inputs might be clipped if their extent is much larger than the code, e.g. grids, material layers and so on. For example if a 100GB DEM is being input to TUFLOW with the majority of the DEM being outside the TUFLOW model, TUFLOW allocates memory to read the entire DEM in for processing. Memory for reading inputs is only used temporarily and is released after the processing of the input, e.g. the DEM.
	* Large DEM and material grids can use up a lot of memory during model initialisation as the entire grid is read for processing. Re-tiling the grid into smaller sections will use up less memory as each smaller grid will be processed at the time avoiding high memory peaks.		* Large DEM and material grids can use up a lot of memory during model initialisation as the entire grid is read for processing. Re-tiling the grid into smaller sections will use up less memory as each smaller grid will be processed at the time avoiding high memory peaks.

ElizaCollison at 12:18, 25 September 2023

2023-09-25T12:18:46Z

Pavlina Monhartova: /* How can I speed up my TUFLOW model, make it more storage efficient and reduce RAM requirements? */

2023-05-19T14:21:21Z

How can I speed up my TUFLOW model, make it more storage efficient and reduce RAM requirements?

← Older revision		Revision as of 00:21, 20 May 2023
Line 11:		Line 11:
	** Use output zones to write localised check files when specific areas of a model domain are of interest.		** Use output zones to write localised check files when specific areas of a model domain are of interest.
	* Simulation:		* Simulation:
	** Use single precision version of TUFLOW if double precision is not required. More information on single and double versions <u>[[~~TUFLOW_FAQ~~#What_is_the_difference_between_single_and_double_precision_and_when_should_I_use_them.3F \| here]]</u>.		** Use single precision version of TUFLOW if double precision is not required. More information on single and double versions <u>[[TUFLOW_General_Discussion#What_is_the_difference_between_single_and_double_precision_and_when_should_I_use_them.3F \| here]]</u>.
	** If using TUFLOW HPC, review the <u>[[HPC_Model_Review#dt_Map_Output \| minimum dt map output]]</u> to check for locations with low timestep and whether any improvements can be applied to speed up the simulation.		** If using TUFLOW HPC, review the <u>[[HPC_Model_Review#dt_Map_Output \| minimum dt map output]]</u> to check for locations with low timestep and whether any improvements can be applied to speed up the simulation.
	* Writing results:		* Writing results:

Pavlina Monhartova: /* How can I speed up my TUFLOW model, make it more storage efficient and reduce RAM requirements? */

2023-05-19T14:16:25Z

How can I speed up my TUFLOW model, make it more storage efficient and reduce RAM requirements?

← Older revision		Revision as of 00:16, 20 May 2023
Line 4:		Line 4:
	* Initialisation:		* Initialisation:
	** Use XF files to speed up the model initialisation (the default).		** Use XF files to speed up the model initialisation (the default).
	** Use FLT grids for grid inputs instead of ASC as they are faster to read. FLT files are ESRI binary (float) version of the ASC files, their size is about 1/5 of an ASC file. To convert ASC to FLT, use -conv switch with our <u>[[ASC_to_ASC#Convert \| ASC_to_ASC utility]]</u>.		** Use FLT grids for grid inputs instead of ASC as they are faster to read. FLT files are ESRI binary (float) version of the ASC files, their size is about 1/2 to 1/5 of an ASC file. To convert ASC to FLT, use -conv switch with our <u>[[ASC_to_ASC#Convert \| ASC_to_ASC utility]]</u>.
	* Writing check files:		* Writing check files:
	** Writing check files to a local solid state drive usually take less time than to a network drive.		** Writing check files to a local solid state drive usually take less time than to a network drive.

Chris Huxley at 03:49, 29 May 2022

2022-05-29T03:49:58Z

← Older revision		Revision as of 13:49, 29 May 2022
Line 44:		Line 44:
	* Antivirus scans		* Antivirus scans
	* Others activities (e.g. copying large files, writing large files).		* Others activities (e.g. copying large files, writing large files).

			<br>
			{{Tips Navigation
			\|uplink=[[ TUFLOW_Modelling_Guidance \| Back to TUFLOW Modelling Guidance]]
			}}

Chris Huxley at 03:49, 29 May 2022

2022-05-29T03:49:37Z

← Older revision		Revision as of 13:49, 29 May 2022
Line 1:		Line 1:
			= Common Questions Answered (FAQ)=
	==How can I speed up my TUFLOW model, make it more storage efficient and reduce RAM requirements?==		==How can I speed up my TUFLOW model, make it more storage efficient and reduce RAM requirements?==
	The following tips provide advice on how to make TUFLOW models initialise, run and write results faster. Most of the suggestions will also ensure the model output size is kept to minimum, minimising required disk space.		The following tips provide advice on how to make TUFLOW models initialise, run and write results faster. Most of the suggestions will also ensure the model output size is kept to minimum, minimising required disk space.

← Older revision		Revision as of 22:18, 25 September 2023
Line 4:		Line 4:
	* Initialisation:		* Initialisation:
	** Use XF files to speed up the model initialisation (the default).		** Use XF files to speed up the model initialisation (the default).
	** Use FLT grids for grid inputs instead of ASC as they are faster to read. FLT files are ESRI binary (float) version of the ASC files, their size is about 1/2 to 1/5 of an ASC file. To convert ASC to FLT, use -conv switch with our <u>[[ASC_to_ASC#Convert \| ASC_to_ASC utility]]</u>.		** Use TIF or FLT grids for grid inputs instead of ASC as they are faster to read. FLT files are ESRI binary (float) version of the ASC files, their size is about 1/2 to 1/5 of an ASC file. To convert ASC to FLT, use -conv switch with our <u>[[ASC_to_ASC#Convert \| ASC_to_ASC utility]]</u>.
	* Writing check files:		* Writing check files:
	** Writing check files to a local solid state drive usually take less time than to a network drive.		** Writing check files to a local solid state drive usually take less time than to a network drive.
Line 17:		Line 17:
	** Make sure there is enough free space as drives slow down when they are nearing to being full.		** Make sure there is enough free space as drives slow down when they are nearing to being full.
	** Use output zones to write localised results when specific areas of a model domain are of interest.		** Use output zones to write localised results when specific areas of a model domain are of interest.
	** Use FLT for grid results instead of ASC as they are more efficient in terms of write speed and file size.		** Use TIF or FLT for grid results instead of ASC as they are more efficient in terms of write speed and file size.
	** Specify XMDF for temporal output instead of DAT as they are more compressed and faster to write.		** Specify XMDF for temporal output instead of DAT as they are more compressed and faster to write.
	** Use appropriate <font color="blue"><tt>Map Output Interval</tt></font> for XMDF, e.g. around 100 output increments for the whole simulation unless an animation of high temporal resolution is to be created.		** Use appropriate <font color="blue"><tt>Map Output Interval</tt></font> for XMDF, e.g. around 100 output increments for the whole simulation unless an animation of high temporal resolution is to be created.
	** Output only maximum FLT grids with <font color="blue"><tt>FLT Map Output Interval </tt></font> <font color="red"><tt>== </tt></font><tt>0</tt> instead of every <font color="blue"><tt>Map Output Interval</tt></font>. Keep the temporal output only as XMDF.		** Output only maximum TIF or FLT grids with <font color="blue"><tt>FLT Map Output Interval </tt></font> <font color="red"><tt>== </tt></font><tt>0</tt> instead of every <font color="blue"><tt>Map Output Interval</tt></font>. Keep the temporal output only as XMDF.
	** Specify only <font color="blue"><tt>Map Output Data Types </tt></font> that are needed.		** Specify only <font color="blue"><tt>Map Output Data Types </tt></font> that are needed.
	** For further details, see <u>[https://www.youtube.com/watch?v=-CsKKjG7jpQ&ab_channel=TUFLOW Output Management Advice]</u> webinar.		** For further details, see <u>[https://www.youtube.com/watch?v=-CsKKjG7jpQ&ab_channel=TUFLOW Output Management Advice]</u> webinar.