TUFLOW 2D Cell Size Selection: Difference between revisions
Content deleted Content added
Chris Huxley (talk | contribs) |
Chris Huxley (talk | contribs) |
||
| (21 intermediate revisions by 2 users not shown) | |||
Line 1:
=Introduction=
This page of the TUFLOW Wiki discusses 2D cell size convergence. In well
[[File:Mesh_Converge_XS_20m.png|500px]][[File:Mesh_Converge_XS_10m.png|500px]][[File:Mesh_Converge_XS_05m.png|500px]]<br>
Line 29:
This Wiki page uses two test cases to discuss this topic. Quantitative results are presented demonstrating how and when resolution assumptions have a tangible impact on model results. TUFLOW HPC and it's GPU Module have been used for all simulations documented in the following sections. The computer used for the modelling has a NVIDIA GeForce GTX 1080 Ti GPU card.
=Webinar Link=
A webinar was recorded that summarises the written content below. It is hosted on <u>[https://www.youtube.com/watch?v=dZWpnKZBKJk&feature=youtu.be YouTube]</u> or available via our <u>[[TUFLOW_Modelling_Webinar | Webinar Page]]</u>.
= Test Case 1 - Rural Dam Break (United Kingdom)=
Line 38 ⟶ 41:
The model has a single inflow at the top of the catchment. The inflow hydrograph that has been used is shown below. Although the inflow only introduces water into the model for 100 minutes the model has been run for a simulation period of 30 hours. This was a requirement in the original EA model benchmark documentation. <br>
[[File:Mesh Converge Model Inflow 001.png|500px]]<br>
{| class="wikitable "
| [[File:Mesh_Converge_SMS_0.5hr.png|300px]] || [[File:Mesh_Converge_SMS_1.5hr.png|300px]] || [[File:Mesh_Converge_SMS_2.5hr.png|300px]] ||[[File:Mesh_Converge_SMS_4hr.png|300px]]
|-
| t = 0.5h || t = 1.5h || t = 2.5h || t = 4.0h
|}
The EA benchmark testing originally assumed a 50m cell resolution. For the purpose of this assessment a range of cell sizes have been used to determine the impact changing grid resolution has on the model results. The following grid resolutions were used:
Line 58 ⟶ 65:
| '''Cell Size''' || '''Location 4 Result'''|| '''Location 5 Result''' || '''Mesh Resolution Figure'''
|-
| '''10m (33ft)''' || [[File:Mesh_Converge_Exg_P4_010.png|500px]] || [[File:Mesh Converge Exg P5 010.png|500px]] ||[[File:Mesh Converge Grid 010.png|500px]]
|-
| '''20m (66ft)''' || [[File:Mesh Converge Exg P4 020.png|500px]] || [[File:Mesh Converge Exg P5 020.png|500px]] || [[File:Mesh_Converge_Grid_020.png|500px]]
|-
| '''50m (164ft)''' || [[File:Mesh Converge Exg P4 050.png|500px]] || [[File:Mesh Converge Exg P5 050.png|500px]] ||[[File:Mesh_Converge_Grid_050.png|500px]]
|-
| '''100m (328ft)''' || [[File:Mesh Converge Exg P4 100.png|500px]]|| [[File:Mesh Converge Exg P5 100.png|500px]] || [[File:Mesh Converge Grid 100.png|500px]]
|-
| '''150m (492ft)''' || [[File:Mesh Converge Exg P4 150.png|500px]] || [[File:Mesh Converge Exg P5 150.png|500px]] ||[[File:Mesh Converge Grid 150.png|500px]]
|-
| '''200m (656ft)''' || [[File:Mesh Converge Exg P4 200.png|500px]] || [[File:Mesh Converge Exg P5 200.png|500px]] ||[[File:Mesh Converge Grid 200.png|500px]]
|-
| '''250m (820ft)''' || [[File:Mesh Converge Exg P4 250.png|500px]] || [[File:Mesh Converge Exg P5 250.png|500px]] ||[[File:Mesh Converge Grid 250.png|500px]]
|}
== Test Case 1 - Discussion ==
The graphed results indicate convergence is observed for all cases with a cell resolution equal to or less than 100m (328ft). The grid figures presented to the right of the graphs suggest the 100m cell resolution case has approximately (4) four cells laterally across the valley that the dam break flow is contained within. Although the 400-600m (1,300ft - 2000ft) wide valley isn't a traditional creek or river channel, the scale of flow associated with the dam break means the valley is behaving like one. Minor topographic features within the valley are too small to have a significant impact on the flood behaviour for flow conditions of this large magnitude. This result trend is consistent with the Australian Rainfall and Runoff (ARR) cell size recommendation for the flow in a channel, "In order to adequately resolve flow in a channel it is desirable to provide at least 5 grid/mesh elements laterally across the channel".
Simulation time can influence model resolution selection. This is not the case in this situation. All models have run in under 5 minutes using TUFLOW HPC's GPU hardware module.
Line 80 ⟶ 87:
{| class="wikitable "
| '''Cell Size''' || '''Model Size <br> (Cell Count)'''|| '''Simulation Run Time <br> (seconds)''' || '''Judgement of Convergence <br> (Yes / No)''' || '''Required GPU Memory <br> (GB)'''
|-
| 10m (33ft) || 188,240 || 284 || Yes || 0.15
|-
| 20m (66ft) || 47,080 || 98 || Yes || 0.04
|-
| 50m (164ft) || 7,540 || 32 || Yes || 0.01
|-
| 100m (328ft) || 1,880 || 15 || Yes || <0.01
|-
| 150m (492ft) || 840 || 10 || No || <0.01
|-
| 200m (656ft) || 480 || 9 || No || <0.01
|-
| 250m (820ft) || 300 || 7 || No || <0.01
|}
Since simulation time is not a limiting factor in this case, selection of cell size only needs to consider the result accuracy. What is appropriate in terms of accuracy is influenced by the intended use of the model. For example, if lot scale assessment of inundation and structural damage risk is required, selection of the 10m (33ft) cell size may be appropriate to provide high resolution mapping of the velocity gradients. Alternatively, if the model is only intended to inform broad-scale inundation risk, the 50m (164ft) or 100m (328ft) cell size may be sufficient.
= Test Case 2 - Urbanised Catchment (USA) =
This test case uses a hypothetical urban model. The model encompasses the entire catchment and covers an area of approximately
[[File:Mesh Converge Direct Rainfall Topo.png|700px]][[File:Mesh Converge Direct Rainfall Manning n.png|700px]]<br>
A direct rainfall approach has been used, applying rainfall directly to every cell within the model. The event hyetograph is a hypothetical 24 hour extreme event. The simulation duration has also been set to 24 hours.<br>
Line 119 ⟶ 126:
{| class="wikitable "
| '''Cell Resolution, Model Size <br> and Simulation Run Time''' || '''Difference in Water Level Result (ft)''' <br> (Relative to 10ft Resolution Model)<br> (Agreement Scale: Green = Excellent, Yellow = Good, Orange = Average, Red = Poor) || '''Sample Peak (Maximum) Water Level Result'''
|-
| '''10ft (3.0m) cell size <br><br>5,821,533 total cell count<br><br>
|-
| '''12ft (3.7m) cell size<br><br>4,042,960 total cell count<br><br>
|-
| '''15ft (4.6m) cell size<br><br>2,587,629 total cell count<br><br>
|-
| '''20ft (6.1m) cell size<br><br>1,455,869 total cell count<br><br>
|-
| '''30ft (9.1m) cell size<br><br>647,003 total cell count<br><br>
|-
| '''50ft (15.2m) cell size<br><br>232,957 total cell count<br><br> 0.
|-
| '''75ft (22.9m) cell size<br><br>103,525 total cell count<br><br> 0.
|}
Line 140 ⟶ 147:
2m (6.6ft) to 5m (16.4ft). <br>
{| class="wikitable "
| '''Cell Size''' || '''Model Size <br> (Cell Count)'''|| '''Simulation Run Time <br> (hours)''' || '''Judgement of Convergence <br> (Yes / No)''' || '''Required GPU Memory <br> (GB)
|-
| 10ft (3.0m) || 5,821,533 ||
|-
| 12ft (3.7m) || 4,042,960 ||
|-
| 15ft (4.6m) || 2,587,629 ||
|-
| 20ft (6.1m) || 1,455,869 ||
|-
| 30ft (9.1m) || 647,003 ||
|-
| 50ft (15.2m) || 232,957 || 0.
|-
| 75ft (22.9m) || 103,525 || 0.
|}
Line 161 ⟶ 168:
* Broad-scale quick run simulation for urban flood forecasting purposes may adopt a coarser 20ft cell resolution or investigate the suitability of a 25ft cell resolution to achieve a quicker simulation turnaround time.<br>
= Common Questions Answered (FAQ)=
==How much will my runtime change when changing cell size?==
Solution time for a model is dependent on a number of factors: the solution scheme used, the size and duration of the model, the nature of the hydraulics within the model, the model features selected, and the hardware that it is executed on. However, once a model is developed at a relatively coarse resolution, there are some guidelines that can assist with predicting what the solve time will be as the resolution is refined. These can be useful for estimating whether the refined model will run overnight or need a few days.<br>
TUFLOW classic uses an implicit solution scheme that sweeps along columns and rows of cells. The solution scheme is one of the most efficient ever devised for solving the Shallow Water Equations (SWEs), but cannot be parallelised as the order of calculations are not independent. As a result the code is written to execute on a single core of a CPU. The time required to solve a model is almost exactly proportional to the number of wet cells in the model and the number of timesteps required. When the model cell size is halved, the number of cells increases by a factor of 4, and the timestep reduces by a factor of 2, so the model solve time increases by very close to a factor of 8.<br>
TUFLOW HPC uses an explicit solution scheme, for which the cell-based calculations are able to be performed in parallel. This can be done using multiple cores of the CPU, but it is more efficiently done using the many thousands of cores of modern GPUs. The model is cut into small tiles that are divided up amongst the streaming multi-controllers (SMs) of the GPU, with each SM distributing the cell calculations of its tile amongst its cores. However, there are still some calculations that must be done at the ‘top-level’ of the model, such as updating boundary flows and levels, extracting the plot output data, timestep control, and the process of configuring and launching the GPU parallel code. These tasks remain on the CPU and cannot be parallelised as the order of execution of these instructions is vitally important. So the overall model execution contains a mix of both parallel and serial work, with the total model solve time being the sum of both. As a result, the solve time for a model doesn’t follow a simple scaling rule. The cell-based component is still proportional to the number of cells times the number of timesteps, whereas the serial component of the work is generally only proportional to the number of timesteps. To be correct, this was also true of the single core solver for TUFLOW classic, but because the cell-based component was generally more than 99% of the work load, the scaling followed the simple rule of 8 times longer for half the cell size.<br>
In summary, for small models running with TUFLOW HPC, the solution time will increase by a factor somewhere in the range of 3 to 8 for a halving of the cell size. Once the model is sufficiently large that the cell-based calculations (running in parallel on the GPU) are taking most of the time, the time increase factor becomes close to 8. Exactly what is a “large” model will depend of the relative speed of the GPU and CPU. For that latest GPUS (as of 2021 this means the Nvidia A100 and RTX 30xx series) large typically means well over a million cells.<br>
However, there is one last and important caveat when refining cell size. The explicit algorithm used by TUFLOW HPC requires that the timestep is also subject to Peclet (diffusion) number control, not just Courant (velocity) number control. As Peclet number is inversely proportional to cell size squared, whereas Courant number is just inversely proportional to cell size, the model will transition from Courant number control to Peclet number control once the cell size becomes small enough. This means that the factor of 8 on time for a halving of cell size will eventually become a factor of 16. This was not a feature of TUFLOW Classic. In general, we do not recommend refining cell size well into the region where timestep control is completely dominated by the Peclet number (the Nd column in the HPC output). The flows in the primary channels are usually well converged by this point, and the 2D map outputs may not reveal a lot of additional detail in the velocity fields.<br>
<br>
<br>
{{Tips Navigation
|uplink=[[ TUFLOW_Modelling_Guidance | Back to TUFLOW Modelling Guidance]]
}}
| |||