View source for Flood Modeller-TUFLOW Benchmarking

<span style="color:#FF0000"><font size=4><b>THIS PAGE IS CURRENTLY UNDER CONSTRUCTION</b></font></span><br><br>
<Br>
=Flood Modeller-TUFLOW Benchmarking=
==Introduction==
The fixed grid TUFLOW software is available in 2 solvers, an implicit finite-difference solver which is now branded as TUFLOW Classic, and an explicit finite volume engine called TUFLOW HPC which stands for Heavily Parallelised Compute.  The solution scheme for TUFLOW HPC has highly independent calculations which can be parallelised and when run over multiple processors or on a GPU provides a significant reduction in run times.
The following wiki post highlights the potential speed up that could be achieved with utilisation of the TUFLOW HPC engine using GPU card technology:-

[https://wiki.tuflow.com/index.php?title=Hardware_Benchmarking_(2018-03-AA)|TUFLOW HPC and GPU Benchmarking]

The benchmark models in these tests are 2D only models or 1D-2D models which utilise TUFLOW 1D ESTRY engine.  The speed up is due to 2D calculations being distributed across multiple processors and undertaken in parallel.  The impact of the parallel processing is also seen in integrated 1D-2D models.  As well as its own 1D engine which allows the modelling and integration of pipe network models and river models together with a 2D domain, TUFLOW can also be integrated with a range of other 1-Dimensional software including Flood Modeller Pro amongst others.

==Flood Modeller-TUFLOW Benchmarking==

A common query is what is the speed up of running simulations utilising TUFLOW HPC and GPU technology when TUFLOW models are linked to external 1D schemes such as Flood Modeller Pro.  Therefore, some benchmarking of linked Flood Modeller Pro-TUFLOW models have been undertaken to demonstrate the show the impact of running the linked models with the TUFLOW HPC solver on a single processor, multiple cores as well as utilizing Nvidia GPU cards.  The outputs from these tests have been compared against the TUFLOW Classic engine which due the various inter-dependencies in the matrix solver cannot distribute calculations across multiple processors. 

The benchmark simulations have been undertaken internally and unfortunately due to model licencing issues cannot be supplied externally.  The models are real world example models typical of the kind that Flood Modeller Pro and TUFLOW are used on.

==Model 1==
The first simulated model was a 287 node Flood Modeller Pro network coupled with a TUFLOW model comprising of 297,746 cells with a 2m resolution.  The input boundaries corresponded to a 100 year return period. The model was run for 10 hours of hydrograph time.

The models were run on a number of different setups and run for 5 scenarios.  The 5 scenarios are as follows:

*TUFLOW Classic with a fixed timestep of 0.5 seconds
*TUFLOW HPC run on 2 Cores
*TUFLOW HPC run on 4 cores
*TUFLOW HPC run on 8 cores
*TUFLOW HPC run on a Nvidia GPU card.  The Nvidia GPU card varies depending on the machine being used.

The TUFLOW version used for the simulations was the 2018-03-AC release and Flood Modeller 4.4.  

The results of the benchmarking of model 1 are shown in Table 1.  The implicit TUFLOW Classic scheme is comparable to the TUFLOW HPC when it is run on a couple of cores.  This is due to the smaller timestep used in the explicit finite volume scheme which uses an adaptive timestep.  As the number of cores used increases, the linked Flood Modeller Pro-TUFLOW HPC simulation is quicker, with run times reduced by up to 42%.  A GPU card has a significant number of cores within it which can be utilized to undertake the 2D calculations.  Running the same model on a GPU card leads to up to an 83% reduction in run times.  This means that nearly 8 simulations can be run on a GPU card in the time that a single simulation could be run using TUFLOW Classic, highlighting the efficiencies of investing in hardware and also GPU modules.


'''Table 1: Runtimes for Flood Modeller-TUFLOW benchmarks'''
{| align="center" class="wikitable"
! style="background-color:#005581; font-weight:bold; color:white;" width=23% | Processor Name
! style="background-color:#005581; font-weight:bold; color:white;" width=15% | Graphic Card
! style="background-color:#005581; font-weight:bold; color:white;" width=5% | GPU RAM (GB)
! style="background-color:#005581; font-weight:bold; color:white;" width=10% | TUFLOW Classic Runtime (s)
! style="background-color:#005581; font-weight:bold; color:white;" width=10% | TUFLOW HPC Runtime (s) on 2 Cores
! style="background-color:#005581; font-weight:bold; color:white;" width=10% | TUFLOW HPC Runtime (s) on 4 Cores
! style="background-color:#005581; font-weight:bold; color:white;" width=10% | TUFLOW HPC Runtime (s) on 8 Cores
! style="background-color:#005581; font-weight:bold; color:white;" width=10% | TUFLOW HPC Runtime (s) on GPU Card
! style="background-color:#005581; font-weight:bold; color:white;" width=10% | % Speed up between TUFLOW Classic and TUFLOW HPC on a GPU
|-
|Intel(R) Core(TM) i7-7820x CPU @ 3.60GHz ||NVIDIA GeForce GTX 1080||8||6041||8071||4718||3502||1022||83%
|-
|Intel(R) Core(TM) i7-7820x CPU @ 3.6 Ghz ||Nvidia GeForce RTX 2080 ti||11||5944||8611||5666||4567||726||88%
|-
|Intel(R) Core(TM) i7-7700 HQ CPU @ 2.8 Ghz ||Nvidia GeForce GTX 1050||4||6622||10867||7857||N/A||3171||52%
|-
|}

==Model 2==

The 2nd Model contains 228 Flood Modeller Pro nodes, a 5m resolution TUFLOW grid with a total of 570,214 cells.  The model was run for 26 hours of model time with a 100 year boundary condition.  The TUFLOW Classic model was run with a 1.25 second timestep.  Table 2 shows the run times for model 2 when run with the same scenarios are model 1.  The table shows that model run times can be up to 82% quicker when run on a GPU compared to TUFLOW Classic.  Even when comparing the TUFLOW HPC GPU run time with the TUFLOW HPC run time when run utilising 8 cores, the simulation can be up to 74% quicker.


'''Table 2: Runtimes for Flood Modeller-TUFLOW benchmarks for Model 2'''
{| align="center" class="wikitable"
! style="background-color:#005581; font-weight:bold; color:white;" width=23% | Processor Name
! style="background-color:#005581; font-weight:bold; color:white;" width=15% | Graphic Card
! style="background-color:#005581; font-weight:bold; color:white;" width=5% | GPU RAM (GB)
! style="background-color:#005581; font-weight:bold; color:white;" width=10% | TUFLOW Classic Runtime (s)
! style="background-color:#005581; font-weight:bold; color:white;" width=10% | TUFLOW HPC Runtime (s) on 2 Cores
! style="background-color:#005581; font-weight:bold; color:white;" width=10% | TUFLOW HPC Runtime (s) on 4 Cores
! style="background-color:#005581; font-weight:bold; color:white;" width=10% | TUFLOW HPC Runtime (s) on 8 Cores
! style="background-color:#005581; font-weight:bold; color:white;" width=10% | TUFLOW HPC Runtime (s) on GPU Card
! style="background-color:#005581; font-weight:bold; color:white;" width=10% | % Speed up between TUFLOW Classic and TUFLOW HPC on a GPU
|-
|Intel(R) Core(TM) i7-7820x CPU @ 3.6 Ghz ||Nvidia GeForce RTX 2080 ti||11||5946||10229||6543||4722||1251||79%
|-
|Intel(R) Xeon(TM) E5-2670 v3 CPU @ 2.3 Ghz ||Nvidia GeForce GTX 980||4||19693||21242||14438||9516||3603||82%
|-
|Intel(R) Core(TM) i7-770HQ CPU @ 2.8 Ghz ||Nvidia GeForce GTX 1050||4||6840||20816||XX||N/A||3886||43%
|-
|}

==Discussion==

The results from the benchmarking of the hydraulic runs, clearly shows the impact of running models on TUFLOW HPC compared to TUFLOW Classic, and in particular the significance of running TUFLOW HPC on a GPU to significantly reduce model run times.  The results show that linked Flood Modeller-TUFLOW HPC run times are up to 8 times faster than TUFLOW Classic and up to 6 times faster than TUFLOW HPC when run on 8 CPU.  Reducing model run times allows more efficiency by reducing the time taken to wait for simulations when various options are tested during the model build phase, increase productivity when running through the final design runs over a suite of return periods and durations and open up opportunities in terms of uncertainty analysis through Monte-Carlo type approaches which require a large number of simulations to be conducted. 

The simulation times were when running the models using the latest release of Flood Modeller 4.4.  Flood Modeller 4.4 although supporting TUFLOW HPC and GPU cards does not support the concurrent simulation of Flood Modeller Pro, TUFLOW HPC and TUFLOW’s 1D scheme, ESTRY.  The ability to run all three simultaneously is scheduled to be available in the Flood Modeller 4.5 release which has been provided in a beta testing phase for this analysis.  The ability to run Flood modeller Pro, TUFLOW 2D and TUFLOW 1D components allows the running of a fully integrated drainage models comprising open channels, 2D floodplains as well as 1-Dimensional pipe networks and their interaction with the surface as represented by the 2D domain.  The above same models were run with Flood Modeller 4.5.  The model results are presented in table 3.

A further model was run which comprised of a Flood Modeller network together with a 2D domain and a X node ESTRY model.



Clearly demonstrates how TUFLOW HPC can leverage the technological improvements that are being made in terms of GPU cards and technology changes.