A keynote to understand the Predictive Uncertainty in Flood Forecasting, by Prof. Ezio Todini
From deterministic to probabilistic thresholds: possibly a better way to take advantage of predictive uncertainty in flood emergency management
Prof. Ezio Todini is Honorary President of Italian Hydrological Society and Co-Founder of “GH2MF2 – Association for a Participated Global Hydrological Monitoring and Flood Forecasting System” Non-Profit intiative. He is currently retired but continues to work actively in Hydrology, making its extensive baggage available to the operational hydrological community
For quite a long time, traditional approaches to flood warning and flood emergency management have been based on the comparison of measured water stages with pre-determined ‘threshold levels’ (warning, alert, flooding levels, etc.). The measured water stage values were generally considered as ‘deterministic’given the relatively small measurement errors (1–2 cm). In recent decades, with the aim of anticipating both decisions and the consequent emergency actions, the original water stage measurements have been substituted by hydrological or hydraulic models’ forecasts. By doing so, the model forecasts were assumed to be of the same quality as the measurements and forecasting errors were totally disregarded. Fortunately, the recent introduction of the predictive uncertainty concept has created the prerequisite to change the original ‘deterministic threshold’ paradigm into a new ‘probabilistic threshold’ paradigm. The aim of this post in the Website of Non-Profit GH2GM2 association is to discuss the ‘probabilistic thresholds’ paradigm by showing how this may lead to a dynamic application of the ‘principle of precaution’ as a function of the degree of predictive uncertainty with consequent benefits both in terms of increased reliability and robustness of decisions.
Today, similarly to what was done for more than a century, in order to trigger their decisions, the majority of water authorities involved in flood emergency management prepare their plans on the basis of pre-determined water depths or thresholds ranging from the warning water level to the flooding level. Decisions, and consequent actions, are then taken as soon as a real time measure of the water stage overtops one of these thresholds. This is a way of anticipating events on the basis of water level measures in the cross-sections of interest or in upstream cross-sections, when flood forecasting models are not available,but can only be effective on very large rivers where the time lag between the overtopping of the warning and the flooding levels is sufficiently large to allow for the implementation of the planned flood relief strategies and interventions.
Given that all the water stage measures are affected by relatively small errors (1–2 cm), they can be, and have been, considered as ‘deterministic’; therefore, in the sequel this approach will be referred to as the ‘deterministic threshold paradigm’.
Unfortunately, the advent and the operational use of real time flood forecasting models has not changed this paradigm, which has been the cause of several unsatisfactory results. Today, instead of comparing the actual measurements, as previously done, flood managers started comparing the hydrologic or hydraulic models forecast to the different decision triggering water stage thresholds. This is obviously done with the intent of further anticipating decisions by taking advantage of the forecast lead-time. Unfortunately, by doing so, the forecasts are implicitly assumed to be ‘real’ and ‘deterministic’, which is not the case, given that the forecasts, by their nature are ‘virtual reality’ and are affected by prediction errors, the magnitude of which is far larger than that of the measurement errors.
More recently, the concept of ‘predictive uncertainty’ has radically changed the ‘deterministic threshold paradigm’. Predictive uncertainty is defined as the probability of occurrence of a future value of a predictand (such as water level, discharge or water volume) conditional on prior observations and knowledge as well as on all the information one can obtain on that specific future value, which is typically embodied in one or more hydrological/hydraulic model forecasts (Krzysztofowicz, 1999).
This inherent uncertain nature of forecasts, as opposed to the higher accuracy of measurements, requires the definition of a ‘probabilistic threshold paradigm’, defined in terms of the ‘probability of flooding’ taken at different probability levels (20%, 50%, etc.) instead of the definition of ‘deterministic’ threshold values. In other words, the decision triggering threshold will not be based on different water stages (warning level, alert level, flooding level), but rather on different probabilities of flooding. It is true that a more formal Bayesian approach would require the definition of a utility function expressing the decision maker propension at risk and the computation of its expected value. The probabilistic thresholds coincide with the simplistic assumption of a constant utility, which may be reasonable in the absence of additional information. The probabilistic thresholds, as opposed to the deterministic water level thresholds, can result in improved tools in the hands of decision-makers. As it will be shown in the sequel, using the probabilistic thresholds, the same predicted water level may not have the same meaning and the same effect on decisions owing to the different reliability of the prediction. In other words, the same forecast may or may not trigger the decision of issuing a warning or evacuating an area, conditionally to its assessed level of uncertainty. More uncertain forecasts need necessarily to be treated more cautiously than more reliable ones; uncertain lower water stage forecasts could then trigger a protective measure, whereas higher, albeit more accurate water stage forecasts would not.
Nonetheless, the pre-requisite to implement the new probabilistic threshold paradigm is an accurate and effective estimate of predictive uncertainty. It is the aim of this post to discuss the introduction of a new ‘probabilistic thresholds’ paradigm and how this is conditioned upon a reliable estimate of predictive uncertainty. The post also aims at showing how the probabilistic threshold paradigm may lead to a dynamic application of the ‘principle of precaution’ as a function of the degree of predictive uncertainty with consequent benefits both in terms of increased reliability and robustness of decisions.
The definition of predictive uncertainty
In water resources management, and more specifically in flood emergency management, decisions, which may generate dramatic social and economical consequences, must be taken on the basis of variables such as water stages, discharges, runoff volumes, etc. without perfect knowledge of the future evolution of the hydro-meteorological phenomena. This lack of knowledge or uncertainty on future occurrences is commonly called ‘predictive uncertainty’. The state of knowledge of a decision-maker may be assumed to be a mixture of ‘what he knows’, or better still ‘what he believes he knows’ (in the sense that he may be wrong), which is a ‘subjective state of mind’ and what he learns from observations (which include data and models), which can be considered as ‘objective’. Therefore, following Rougier (2007), a possible definition of predictive uncertainty is:
“Predictive uncertainty is the expression of a subjective assessment of the probability of occurrence of a future (real) event conditional upon all the knowledge available up to the present (the prior knowledge) and the information that can be acquired through a learning inferential process.”
From this definition, the need emerges for using hydrological model forecasts to reduce the predictive uncertainty, usually expressed in terms of a probability density (or probability distribution) function, ‘conditional’ upon the available observations and hydrological model forecasts, which are now seen as the available, although uncertain, extensions into the future of observations. In other words, hydrological model forecasts are a way to complement the prior belief of the decision-maker in order to reduce ‘his’ prior uncertainty within the frame of the decision- making process. This way of looking at hydrological model forecasts is the opposite of current operational practice where (explicitly or implicitly) models are assumed to provide deterministic (and therefore ‘certain’) forecasts such as future levels, flows, etc. Krzysztofowicz (1999), was the first to clarify, within the hydrological context, that the objective of forecasting is the assessment of the probability that future values of water stage, discharge, runoff volume, etc. will be smaller, greater or equal to given values (generally threshold values, such as for instance the elevation of the dykes), rather than the estimation of the uncertainty of the same quantities forecast by hydrological models.
Combining measurements and models to improve predictability
As previously stated, hydrological prediction must aim at the reduction of the decision-maker’s uncertainty on the future occurrence of quantities such as future water levels, discharges or water volumes, that will be called ‘predictands’ in the sequel. To do so, the decision-maker generally starts from his prior belief. For instance, he can use the climatological distribution of extreme discharge occurrence to describe his prior belief on the possibility of flooding, but in general, the relevant probability density function is very flat and is not sufficiently dense around some specific value to allow reliable decisions, such as issuing a flood alert. Therefore it is necessary to gather additional information, additional measurements or to generate future scenarios by means of one or more forecasting models.
There is no substantial difference between a measured or a modelled quantity apart from the type of errors affecting them and the fact that modelled quantities may be available at a future time. Measurements, although affected by measurement errors, can be reasonably accurate. But if these measurements are indirect measures of the predictand, they become ‘predictors’, which implies that they will also be affected by modelling errors, similarly to ‘model predictions’. Modelled quantities incorporate both measurement errors and model errors, that can be large if the model is not very accurate. Nonetheless, models become essential when dealing with ‘forecasting’; because measurements are not available at any future time, therefore one can only use modelled quantities ito increase insight into the future, and consequently reduce uncertainty.
The forecasting problem can be usually tackled on the basis of two different approaches, depending on its nature and on the decision problem to be solved. The first approach relates to cases where only the total probability (namely the integral of the predictive density) above or below a threshold is needed. This is the case for instance when one has to decide whether a landslide will or won’t occur on the basis of one or more sensors or models. The second approach relates to continuous processes, requiring the estimation of the entire predictive probability function: for instance when dealing with flood damages, which vary with the water level. In this case decisions tend to be taken on the basis of the expected damages, which can only be estimated if the full probability density of predicted future water levels is available.
The probabilistic threshold paradigm
Today, similarly to what was done for more than a century, in order to trigger their decisions, the majority of water authorities involved in flood emergency management prepare their plans on the basis of pre-determined water depths or thresholds ranging from the warning water level to the flooding level. Decisions, and consequent actions, are then taken as soon as a real time measure of the water stage overtops one of these thresholds. This approach, which is correct and sound in the absence of flood forecasting models, is a way of anticipating events on the basis of water level measures (in the cross sections of interest or in upstream cross sections), but can only be effective on very large rivers where the time lag between the overtopping of the warning and the flooding levels is sufficiently large to allow for the implementation of the planned flood relief strategies and interventions. Given that all the water stage measures are affected by relatively small errors, they can be, and have been, considered as deterministic.
Unfortunately, the advent and the operational use of real time flood forecasting models, has not changed this approach, which has been the cause of several unsatisfactory results. Today, the flood managers compare forecasts, and not the actual measurements, to the different threshold levels; this is obviously done in order to further anticipate decisions by taking advantage of the prediction time horizon. Unfortunately, by doing so the forecasts are implicitly assumed to be deterministic, which is not the case since they represent virtual reality and are affected by prediction errors, which magnitude is by far larger than that of the measurement errors. More recently, the concept of predictive uncertainty has changed this approach. This uncertain nature of forecasts, opposed to the higher accuracy of measurements, requires the definition of probabilistic thresholds, defined in terms of the probability of flooding taken at different probability levels, instead of the definition of deterministic threshold values.
Using the probabilistic thresholds, the same predicted water level may have different meaning owing to the reliability of prediction. In other words, the same forecast may or may not trigger the decision of issuing a warning or evacuating an area, conditionally to its assessed level of uncertainty. More uncertain forecasts need necessarily to be treated more cautiously than more reliable ones; in fact, uncertain lower water stage forecasts could then trigger a protective measure, whereas higher, albeit more accurate water stage forecasts, would not. As can be seen from the Fig. 1, for the same expected value (the horizontal dashed line) a better forecast (Model A), characterised by a narrower predictive density, will show a smaller probability of exceeding the flooding level when compared to a worse one (Model B). This property can be also looked at from an alternative perspective, as shown in Fig. 2 the same flooding probability corresponds to lower expected values as the spread of PU increases. This implies that if a probabilistic threshold is defined instead of a deterministic threshold level, when the PU is larger the decision maker must be more cautious and would be advised to issue an alert even when, looking at the expected value of the forecast, he would not think of issuing it, because he may regard it as being too low.