Many of us will have had the experience of bugs in our code, that is, mistakes that impact the intended function and functioning of our software. Such mistakes slow down development, impinge on collaboration, reduce the likelihood of our code being used by others, and in the scientific context can lead to serious mistakes in publications. Approaches to software verification are therefore useful to help reduce the occurrence of bugs and assess whether code implements its intended specification / model. One well-established lightweight technique for evaluating software correctness is testing, where additional code is written that provides a partial specification of program behaviour.
This workshop studies the foundations of software testing, including the use of tools to automate the deployment of tests. It specifically looks at the mechanics of, and best practices for, three kinds of tests: unit tests, integration tests, and property-based tests. The first two are more widely deployed already in science whereas the third technique (property-based testing) is still an underutilised, but powerful, tool.
Python is used for running examples and exercises, with the pytest framework,
but the concepts can be applied in almost
any programming language. We will point to resources for a few other languages
popular in science such as Fortran.
It assumes that the attendees have some programming skills, but are not necessarily Python experts. There is some emphasis on scientific computing, with the example being a simple 0D Energy Balance Model (EBM). It would therefore be beneficial if you have some experience in this field, though not strictly necessary.
This material was first delivered at the 2025 ICCS summer school, see the part 1 and part 2 videos.
- Learning Objectives
- Teaching material
- Preparation and prerequisites
- Installation and setup
- License information
- Contribution Guidelines and Support
- Understand a high-level overview of verification and validation in the context of computational science;
- Explain the purpose and limitations of testing, including why testing cannot guarantee the absence of bugs but still increases confidence in code.
- Write basic automated unit tests using pytest, including covering edge cases, invalid inputs, and considering the role of numerical representation;
- Apply testing best practices, including parameterisation, fixtures, negative tests, and Test-Driven Development (TDD).
- Understand integration and end-to-end tests, recognising common interface issues and trade-offs in testing strategies.
- Write property-based tests with Hypothesis to generate diverse inputs and define meaningful properties.
- Select and combine appropriate testing approaches to improve reliability, support refactoring, and diagnose errors in scientific software.
The slides are included in this repository.
The example folder provides a small 0-dimensional Energy Balance Model for a planet (with its main configuration being for Earth). See
example/README.md for instructions on its usage. It contains a test suite in the example/tests directory which is used for demonstration and is the source material for the exercises in exercises.md, which also provides setup instructions.
Coming soon
Session 1 - 1h
- 10 minute intro about correctness and testing
- 50 minutes explaining concepts about unit testing including
- Parameterised tests
- Fixtures
- Negative tests
- Approximation and floating point
- TDD
- Code coverage
Session 2 - 1h30
- 20 minute unit test exercises
- 40 minutes lecture
- Integration and end-to-end tests
- Property-based testing
- 30 minutes exercises (property-based test exercises)
- Basic programming knowledge
- At least beginner experience in Python, e.g., understanding of
- Basic mathematical operations
- Writing and running scripts/programs
- Writing and using functions
The examples require Python 3.11 and above, and will use pytest. See the installation and setup instructions below.
The workshop exercises involve writing Python code and running tests, so you will need to already have a text editor or IDE (Integrated Development Environment) setup and ready to use. The examples and exercises all use Python 3.
If you require assistance or further information with any of these please reach out to us before the session.
- You can obtain the code for the exercises by cloning this repository locally:
git clone https://github.com/Cambridge-ICCS/testing-workshop.git
- Set up a virtual environment and install the requirements, e.g.
cd example
python3 -m venv venv
source venv/bin/activate
python3 -m pip install -r requirements.txt
- You can then run the provided tests by running:
pytest tests
The examples and exercises will build from this setup.
The code materials in this project are licensed under the MIT License.
If you spot an issue with the materials please let us know by opening an issue here on GitHub clearly describing the problem.
If you are able to fix an issue that you spot, or an existing open issue please get in touch by commenting on the issue thread.
Contributions from the community are welcome. To contribute back to the repository please first fork it, make the necessary changes to fix the problem, and then open a pull request back to this repository clearly describing the changes you have made. We will then perform a review and merge once ready.
If you would like support using these materials, adapting them to your needs, or delivering them please get in touch either via GitHub or via ICCS.