Kart Git And TUFLOW

Introduction

This page describes how to setup your TUFLOW model with Git and Kart (as this tends to be the tricker part). It doesn't necessarily discuss workflows once the repositories are setup.

Notes

• This wiki page was written with Kart v0.12.2 which is still considered a pre-release version. Kart is in rapid development and some of the constraints and/or methods may no longer be valid or have changed in subsequent versions.
• The examples on this page use git bash (on Windows) which is installed with Git. A lot of the methodology can also be done through Windows command prompt with the same syntax however on this version of Kart, for private remote repositories, Kart will not prompt for any authentication while using command prompt and will simply fail to push/pull. Git bash will prompt for authentication and is also easier to add and save ssh keys so is a lot better to use in these contexts. For consistency, it will be adopted for all examples over command prompt.
• The examples below have used SSH authentication with either Gitlab or Github. SSH authentication appears to be work better with Kart.
• It's a good idea to ensure you have hidden folders visible. Both Git and Kart use hidden folders to store information and it's nice being able to see them and access them if needed.

Software and Installation

The following software is required:

• Git git-scm.com/downloads - There is a portable version Git for Windows, therefore it doesn't necessarily require admin privileges to install. There are a lot of user installation options, you should be able to simply use the default options throughout.
• Kart kartproject.org/ - Kart will require admin privileges to install. Kart is packaged with it's own Git version, however this will not contain git bash so the above installation will be required.

Recommended software:

• QGIS with the Kart plugin (and the TUFLOW plugin for importing empty files)

Setting Up Git Bash

Git Bash is installed with Git and is similar to Windows command prompt but uses bash (linux version of command prompt). When using Git bash and Kart together you will need to use kart.exe instead of simply kart e.g.

kart.exe commit -m "commit message"

Using kart with Windows command prompt will accept simply kart and all the Kart documenation suggests just using kart, but just be aware that ".exe" will be required if using Git Bash.

Git Bash can be opened up from the start menu. The examples on this page all use the SSH authentication method as this approach appears to currently work better with Kart. You will need to set this up on the remote repository you are using (e.g. Github or Gitlab). The steps on how to set up an SSH key is linked below, the instructions are for Github, however the process is very similar for Gitlab.

• Check for existing SSH keys (if you have already set one up and need to find its name)
• Generate an SSH key
• Adding an SSH key

Each time you open a new instance of Git Bash, you will need to add the ssh key again in the bash window if you do not want to repeatedly have to enter your password (to be clear, the key does not require to be generated or linked with your Gitlab or Github account each time). The steps to add the ssh key are described below (taken from the above link) for ease of access.

eval "$(ssh-agent -s)"

and then (replacing id_ed25519 with the name of your key).

ssh-add ~/.ssh/id_ed25519

Setting up Control File Repository

Since most of the TUFLOW input files (exclduing GIS/GRID inputs and some boundary formats) are plain text files, it is possible to simply use Git to version these files. There are a number of ways to initialise this, the approach shown below is done by intialising the remote repository on Github or Gitlab first and then cloning the repository, then adding files.

1. Create Remote Repository

Create your remote repository (usually using a Blank template), setting the name and whether the repository should be public or private.

2. Clone Repository

In Git Bash, use cd to change directory to where your model is going to sit. Note: Git Bash requires forward slashes /

cd c:/projects

Use git clone to clone your newly created repository. Note: make sure to use the SSH clone link if required (will start with "git@" rather than "https"). Tip: pasting is done by Ctrl+Shift+Insert (not Ctrl+V).

git clone git@gitlab.com:ellis_symons/tutorial-model.git

Once created, you should change directory into the newly cloned project directory:

cd tutorial-model

3. Adding Files

Once the repository is cloned, you can then add files. These can be existing files or files you start creating as part of a new model build. As an example, one of the first files that can be added is a .gitignore file. The example below is a template that can be copied when needed. To create this file, simply create a new text document and rename it to ".gitignore" and save it inside the cloned project folder (next to the ".git" folder).

check*
results*
model/gis*
model/grid*
model/tin*
model/xf*
runs/log*
runs/_ TUFLOW Simulations.log
bc_dbase/xf*
*.qgz

Once created, the .gitignore file can be added via git bash, Git GUI, or any other software that has version control integration (e.g. Visual Studio Code). Using Git bash is shown below:

git add .gitignore

4. Committing and Pusing

Once files have been added they can be committed and then pushed to the remote repository.

git commit -m "<commit message>"
git push

Setting up Kart Repository

Kart can be used for version control the GIS vector inputs into the model. For Kart, the approach described below starts by initialising the local version first then linking it to a remote repository.

1. Initialise Kart Locally

Setup your TUFLOW folder structure if you have not done already. If this is a brand new project you can use the "Configure Project Tool" in the TUFLOW Plugin in QGIS (make sure to select GPKG as the GIS format). This tool may create a new folder called "TUFLOW" within the model folder structure, you can copy the contents out of this folder into the root project directory if you like. In Git Bash, change directory to the "model" folder, where the "gis" folder will sit.

cd model

Initialise the Kart repository. Note that Kart will create the folder if it does not exist and the name of the GPKG that it creates will match the name of the folder. In this example, the GPKG will be called "gis_database.gpkg".

kart.exe init gis_database

At this stage you may have a folder called "gis" where you have your existing files (empty files, projection etc) and a new folder called "gis_database" which is where the kart repo has been initialised. It's a good idea to consolidate all these into a single folder. In this example (assuming a new project) the empty folder and projection.gpkg is going to be copied into the "gis_database" folder. Once copied, the "gis" folder can be deleted and the "gis_database" folder can be renamed to "gis" to keep it consistent with TUFLOW folder naming convention. Once renamed, change directory in Git Bash to the kart repo

cd gis

2. Import Layers

GPKG layers can be added to the kart repo. If the existing layers are shp files, then they will need be imported via the Kart plugin in QGIS which converts them first before importing them. In this example the projection.gpkg will be imported using the command below:

kart.exe import projection.gpkg projection

Once the projection layer is added to kart, the original projection.gpkg can now be deleted. It is probably a good idea to keep your kart repo clean of other input files and only have the GPKG created with the kart repo in this location (it will be created when the first layer is imported). Keeping the empty folder in this location is probably ok and convenient.

3. Link to Remote Repository

You will need to delete the following file to be able to push your kart repo to a remote:
.kart/hooks/pre-push

Create a new remote repository using similar steps as the control file repository. Once created the local kart repo can be linked and pushed using the following commands:

kart.exe remote add origin git@gitlab.com:ellis_symons/tutorial-model-gis.git
kart.exe branch -M main
kart.exe push -u origin main

You must have something to push (at least one commit). Importing layers counts as a commit (you are not required to manually commit after importing).

Cloning an Existing Repository

Once your model repository is setup it is desirable to be able to clone the repository so that others can work on it. The steps below outline how this can be done, it is assumed that Git Bash has been setup and configured with an SSH key and that the user has been given access to push/pull to the remote:

1. Clone Control File Repository

In Git Bash, change directory to where the TUFLOW model will sit (Note: bash will require forward slashes in file paths /):

cd c:/projects

Use git clone to clone the repo (Note: make sure to use the SSH repository address):

git clone git@gitlab.com:ellis_symons/tutorial-model.git

2. Clone GIS File Repository

Once the main project has been cloned, change directory into the model folder (where the "gis" folder will sit)

cd tutorial-model/model

Clone the kart repository assigning it the desired GPKG name (in this case the gpkg will be called "gis_database.gpkg"):

kart.exe clone git@gitlab.com:ellis_symons/tutorial-model-gis.git gis_database

Rename the "gis_database" folder to "gis" to match TUFLOW folder naming convention.

Summary

The steps above will copy all the input text control files (and boundary files) as well as the gis layers. The grid inputs have been excluded thus far from being versioned, this is due to their size (usually very big) and the fact that they don't often change meaning they don't typically have to be versioned. So at this point in the process any grid files will have to be copied from where they are being stored into the correct directory location.

Workflow

Once setup, the specific workflow will be dependent on how many many modellers will be working on the project concurrently and the specific goals of the project (e.g. are there going to be a lot of mitigation scenarios etc). The use of branches may be a robust approach in complex situations with a lot of mitigation options, or even a separate repository and GPKG for mitigation options.

For the workflow, the easiest approach is to use something like Visual Studio Code for TUFLOW control file editing as this has a version control manager which will make the stage/commit/push/pull a lot easier and skips the need to be using the CLI. For kart, using the kart plugin in QGIS is the easiest method for skipping the need to use the CLI, however for pushing/pulling it may be necessary to use Git Bash to setup the SSH authentication. The TUFLOW plugin 'import empty' tool has an option to import directly into a kart repository, skipping the need to double handle the creation of TUFLOW GIS input files.