Continuous integration

Continuous integration (CI) is about commits being merged frequently, resulting in new features being released frequently. When proposing new changes to the code base a pull request is opened. When a new commit in that pull request, a series of tests will be done to make sure that this commit is error-free and robust in different environments. This process drive each new development through building, testing, quality checking.

graph LR
    A[New development]-->B[Continuous integration]
    B-->C[Merge]

This page contains an extensive explanation on how the Ribasim continuous integration works.

1 GitHub Actions

With GitHub Actions, GitHub provides their own continuous integration service. They include the following checks:

  • Julia Run Testmodels: This is to make sure all the test models can be run successfully
  • Julia Tests: Runs Julia unit tests on multiple platforms
  • Docs: Builds the documentation and executes the examples
  • Pre-commit: Checks Python code quality with linters and enforces formatting
  • Python codegen: Makes sure code generation is still up to date by checking that code generation wouldn’t change any files
  • Mypy: Runs mypy on our Python codebase to check types
  • Ribasim Python tests: Runs Ribasim Python tests on multiple platforms and multiple Python versions
  • QGIS Tests: Runs QGIS unit tests

2 TeamCity

Ribasim has another cloud-based CI service based on TeamCity. TeamCity interacts with GitHub and manages the release process.

graph LR
    A[TeamCity]-->|Monitoring|B[GitHub]
    A-->C(Release)

2.1 Conditions of using TeamCity

TeamCity only runs workflows with the following conditions:

  • When the workflow would take too long to run on GitHub Action
  • When the release depends on the artifacts of the workflow.
  • When other TeamCity projects depend on artifacts of Ribasim (e.g. iMOD coupler)

2.2 Release process

In the release, we include the generated testmodels, Ribasim CLI on Windows and Linux, Ribasim QGIS, and the source code.

We have the following pipeline to generate artifects for releasing:

  • Generate Testmodels: produces generated_testmodels artifact which is part of the release.
  • Make GitHub Release: uses artifacts and makes the release. TeamCity constantly monitors the GitHub repository. When a tag starts with v20 is added, it triggers the release process.
  • Build Ribasim: builds library and executable of Ribasim on Linux and Windows. The artifacts are tested in Test Ribasim Binaries and used by iMOD Coupler.
  • Test Ribasim Binaries: tests libribasim artifact and ribasim_cli artifact on Linux and Windows
Note

Make GitHub Release does not publish artifacts of “Test Ribasim Binaries”. It only publishes artifacts of “Build Ribasim” if the beforementioned tests pass.

graph LR
    A[Make GitHub Release]-->B(Release)
    F[Generate Testmodels]-->A
    G[Make QGIS plugin]-->A
    H[Build Ribasim]---D[Test Ribasim Binaries]
    D-->A