From deterministic to probabilistic thresholds: possibly a better way to take advantage of predictive uncertainty in flood emergency management
Prof. Ezio Todini is Honorary President of Italian Hydrological Society and Co-Founder of “GH2MF2 – Association for a Participated Global Hydrological Monitoring and Flood Forecasting System” Non-Profit intiative. He is currently retired but continues to work actively in Hydrology, making its extensive baggage available to the operational hydrological community
For quite a long time, traditional approaches to flood warning and flood emergency management have been based on the comparison of measured water stages with pre-determined ‘threshold levels’ (warning, alert, flooding levels, etc.). The measured water stage values were generally considered as ‘deterministic’given the relatively small measurement errors (1–2 cm). In recent decades, with the aim of anticipating both decisions and the consequent emergency actions, the original water stage measurements have been substituted by hydrological or hydraulic models’ forecasts. By doing so, the model forecasts were assumed to be of the same quality as the measurements and forecasting errors were totally disregarded. Fortunately, the recent introduction of the predictive uncertainty concept has created the prerequisite to change the original ‘deterministic threshold’ paradigm into a new ‘probabilistic threshold’ paradigm. The aim of this post in the Website of Non-Profit GH2GM2 association is to discuss the ‘probabilistic thresholds’ paradigm by showing how this may lead to a dynamic application of the ‘principle of precaution’ as a function of the degree of predictive uncertainty with consequent benefits both in terms of increased reliability and robustness of decisions.
Introduction
Today, similarly to what was done for more than a century, in order to trigger their decisions, the majority of water authorities involved in flood emergency management prepare their plans on the basis of pre-determined water depths or thresholds ranging from the warning water level to the flooding level. Decisions, and consequent actions, are then taken as soon as a real time measure of the water stage overtops one of these thresholds. This is a way of anticipating events on the basis of water level measures in the cross-sections of interest or in upstream cross-sections, when flood forecasting models are not available,but can only be effective on very large rivers where the time lag between the overtopping of the warning and the flooding levels is sufficiently large to allow for the implementation of the planned flood relief strategies and interventions.
Given that all the water stage measures are affected by relatively small errors (1–2 cm), they can be, and have been, considered as ‘deterministic’; therefore, in the sequel this approach will be referred to as the ‘deterministic threshold paradigm’.
Unfortunately, the advent and the operational use of real time flood forecasting models has not changed this paradigm, which has been the cause of several unsatisfactory results. Today, instead of comparing the actual measurements, as previously done, flood managers started comparing the hydrologic or hydraulic models forecast to the different decision triggering water stage thresholds. This is obviously done with the intent of further anticipating decisions by taking advantage of the forecast lead-time. Unfortunately, by doing so, the forecasts are implicitly assumed to be ‘real’ and ‘deterministic’, which is not the case, given that the forecasts, by their nature are ‘virtual reality’ and are affected by prediction errors, the magnitude of which is far larger than that of the measurement errors.
More recently, the concept of ‘predictive uncertainty’ has radically changed the ‘deterministic threshold paradigm’. Predictive uncertainty is defined as the probability of occurrence of a future value of a predictand (such as water level, discharge or water volume) conditional on prior observations and knowledge as well as on all the information one can obtain on that specific future value, which is typically embodied in one or more hydrological/hydraulic model forecasts (Krzysztofowicz, 1999).
This inherent uncertain nature of forecasts, as opposed to the higher accuracy of measurements, requires the definition of a ‘probabilistic threshold paradigm’, defined in terms of the ‘probability of flooding’ taken at different probability levels (20%, 50%, etc.) instead of the definition of ‘deterministic’ threshold values. In other words, the decision triggering threshold will not be based on different water stages (warning level, alert level, flooding level), but rather on different probabilities of flooding. It is true that a more formal Bayesian approach would require the definition of a utility function expressing the decision maker propension at risk and the computation of its expected value. The probabilistic thresholds coincide with the simplistic assumption of a constant utility, which may be reasonable in the absence of additional information. The probabilistic thresholds, as opposed to the deterministic water level thresholds, can result in improved tools in the hands of decision-makers. As it will be shown in the sequel, using the probabilistic thresholds, the same predicted water level may not have the same meaning and the same effect on decisions owing to the different reliability of the prediction. In other words, the same forecast may or may not trigger the decision of issuing a warning or evacuating an area, conditionally to its assessed level of uncertainty. More uncertain forecasts need necessarily to be treated more cautiously than more reliable ones; uncertain lower water stage forecasts could then trigger a protective measure, whereas higher, albeit more accurate water stage forecasts would not.
Nonetheless, the pre-requisite to implement the new probabilistic threshold paradigm is an accurate and effective estimate of predictive uncertainty. It is the aim of this post to discuss the introduction of a new ‘probabilistic thresholds’ paradigm and how this is conditioned upon a reliable estimate of predictive uncertainty. The post also aims at showing how the probabilistic threshold paradigm may lead to a dynamic application of the ‘principle of precaution’ as a function of the degree of predictive uncertainty with consequent benefits both in terms of increased reliability and robustness of decisions.
The definition of predictive uncertainty
In water resources management, and more specifically in flood emergency management, decisions, which may generate dramatic social and economical consequences, must be taken on the basis of variables such as water stages, discharges, runoff volumes, etc. without perfect knowledge of the future evolution of the hydro-meteorological phenomena. This lack of knowledge or uncertainty on future occurrences is commonly called ‘predictive uncertainty’. The state of knowledge of a decision-maker may be assumed to be a mixture of ‘what he knows’, or better still ‘what he believes he knows’ (in the sense that he may be wrong), which is a ‘subjective state of mind’ and what he learns from observations (which include data and models), which can be considered as ‘objective’. Therefore, following Rougier (2007), a possible definition of predictive uncertainty is:
“Predictive uncertainty is the expression of a subjective assessment of the probability of occurrence of a future (real) event conditional upon all the knowledge available up to the present (the prior knowledge) and the information that can be acquired through a learning inferential process.”
From this definition, the need emerges for using hydrological model forecasts to reduce the predictive uncertainty, usually expressed in terms of a probability density (or probability distribution) function, ‘conditional’ upon the available observations and hydrological model forecasts, which are now seen as the available, although uncertain, extensions into the future of observations. In other words, hydrological model forecasts are a way to complement the prior belief of the decision-maker in order to reduce ‘his’ prior uncertainty within the frame of the decision- making process. This way of looking at hydrological model forecasts is the opposite of current operational practice where (explicitly or implicitly) models are assumed to provide deterministic (and therefore ‘certain’) forecasts such as future levels, flows, etc. Krzysztofowicz (1999), was the first to clarify, within the hydrological context, that the objective of forecasting is the assessment of the probability that future values of water stage, discharge, runoff volume, etc. will be smaller, greater or equal to given values (generally threshold values, such as for instance the elevation of the dykes), rather than the estimation of the uncertainty of the same quantities forecast by hydrological models.
Combining measurements and models to improve predictability
As previously stated, hydrological prediction must aim at the reduction of the decision-maker’s uncertainty on the future occurrence of quantities such as future water levels, discharges or water volumes, that will be called ‘predictands’ in the sequel. To do so, the decision-maker generally starts from his prior belief. For instance, he can use the climatological distribution of extreme discharge occurrence to describe his prior belief on the possibility of flooding, but in general, the relevant probability density function is very flat and is not sufficiently dense around some specific value to allow reliable decisions, such as issuing a flood alert. Therefore it is necessary to gather additional information, additional measurements or to generate future scenarios by means of one or more forecasting models.
There is no substantial difference between a measured or a modelled quantity apart from the type of errors affecting them and the fact that modelled quantities may be available at a future time. Measurements, although affected by measurement errors, can be reasonably accurate. But if these measurements are indirect measures of the predictand, they become ‘predictors’, which implies that they will also be affected by modelling errors, similarly to ‘model predictions’. Modelled quantities incorporate both measurement errors and model errors, that can be large if the model is not very accurate. Nonetheless, models become essential when dealing with ‘forecasting’; because measurements are not available at any future time, therefore one can only use modelled quantities ito increase insight into the future, and consequently reduce uncertainty.
The forecasting problem can be usually tackled on the basis of two different approaches, depending on its nature and on the decision problem to be solved. The first approach relates to cases where only the total probability (namely the integral of the predictive density) above or below a threshold is needed. This is the case for instance when one has to decide whether a landslide will or won’t occur on the basis of one or more sensors or models. The second approach relates to continuous processes, requiring the estimation of the entire predictive probability function: for instance when dealing with flood damages, which vary with the water level. In this case decisions tend to be taken on the basis of the expected damages, which can only be estimated if the full probability density of predicted future water levels is available.
The probabilistic threshold paradigm
Today, similarly to what was done for more than a century, in order to trigger their decisions, the majority of water authorities involved in flood emergency management prepare their plans on the basis of pre-determined water depths or thresholds ranging from the warning water level to the flooding level. Decisions, and consequent actions, are then taken as soon as a real time measure of the water stage overtops one of these thresholds. This approach, which is correct and sound in the absence of flood forecasting models, is a way of anticipating events on the basis of water level measures (in the cross sections of interest or in upstream cross sections), but can only be effective on very large rivers where the time lag between the overtopping of the warning and the flooding levels is sufficiently large to allow for the implementation of the planned flood relief strategies and interventions. Given that all the water stage measures are affected by relatively small errors, they can be, and have been, considered as deterministic.
Unfortunately, the advent and the operational use of real time flood forecasting models, has not changed this approach, which has been the cause of several unsatisfactory results. Today, the flood managers compare forecasts, and not the actual measurements, to the different threshold levels; this is obviously done in order to further anticipate decisions by taking advantage of the prediction time horizon. Unfortunately, by doing so the forecasts are implicitly assumed to be deterministic, which is not the case since they represent virtual reality and are affected by prediction errors, which magnitude is by far larger than that of the measurement errors. More recently, the concept of predictive uncertainty has changed this approach. This uncertain nature of forecasts, opposed to the higher accuracy of measurements, requires the definition of probabilistic thresholds, defined in terms of the probability of flooding taken at different probability levels, instead of the definition of deterministic threshold values.
Using the probabilistic thresholds, the same predicted water level may have different meaning owing to the reliability of prediction. In other words, the same forecast may or may not trigger the decision of issuing a warning or evacuating an area, conditionally to its assessed level of uncertainty. More uncertain forecasts need necessarily to be treated more cautiously than more reliable ones; in fact, uncertain lower water stage forecasts could then trigger a protective measure, whereas higher, albeit more accurate water stage forecasts, would not. As can be seen from the Fig. 1, for the same expected value (the horizontal dashed line) a better forecast (Model A), characterised by a narrower predictive density, will show a smaller probability of exceeding the flooding level when compared to a worse one (Model B). This property can be also looked at from an alternative perspective, as shown in Fig. 2 the same flooding probability corresponds to lower expected values as the spread of PU increases. This implies that if a probabilistic threshold is defined instead of a deterministic threshold level, when the PU is larger the decision maker must be more cautious and would be advised to issue an alert even when, looking at the expected value of the forecast, he would not think of issuing it, because he may regard it as being too low.
Posted: July 30, 2019 by wp_9282510 Leave a Comment
A Pilot Hydrological Modelling in Transboundary KOSHI River Basin [Tibet(China), Nepal, India]
By Enrique Ortiz et Al.
Executive Secretary of the NGO GH2MF2 – Association for a Participated Global Hydrological Monitoring and Flood Forecasting System. Via Settimio Mobilio 16 Int.21 – 84127 (SA) Salerno – Italy. Fiscal Code: 95165880659
The development of GH2MF2 project was motivated from solidarity and from the need to dignify the most disadvantaged people living in the poorest countries in the world (Asia, Africa and South America), which are continually exposed to changes in the hydrologic cycle suffering events of large floods and/or long periods of droughts due to climate change. The key philosophy behind GH2MF2 is that anyone citizen in the world should deserve free access to hydrological information and forecasts, through a Web-GIS based platform in an interactive and bidirectional way. GH2MF2 is an open non-profit tool, designed to provide real time hydrological monitoring and forecasting at global scale, together with an assessment of predictive uncertainty, by combining hydrological to meteorological ensemble uncertainty, described as a function of the meteorological ensemble spread. To carry out the project, a Non-Profit association was established with legal headquarters in the Italian Republic in 2017, founded by three of the authors of this Conference Paper: ” GH2MF2 – Association for a Participated Global Hydrological Monitoring and Flood Forecasting System”. We present the development of a demonstrational pilot implementation of GH2MF2 in Transboundary Koshi River Basin (China, Nepal and India).
Koshi River (कोशी नदी in nepali or Kosi River in indian) is one of the biggest tributary of River Ganga originating from Tibet (China) and joins the Ganges in Bihar State (India) via Nepal. Total drainage area of the Koshi River is 88000 km2. Majority of the area comes from Tibet and Nepal (80%), and only 20% drainage area is in Indian Territory. The River Khosi originates at an altitude of over 7000 m above MSL in the Himalayas, 6 of the 14 eight-thousanders are in the basin (Mount Everest, Kangchenjunga, Lhotse, Makalu, Cho Oyu and Shishapangma).
In the north, the river is bound by the ridge separating it from the Tsangpo (Brahmaputra) River, while the River Ganga forms its southern boundary. The eastern and western boundaries are the ridge lines, separating it from the Mahananda and the Gandak/Burhi catchments respectively. The upper catchment of the river system lies in Nepal and Tibet. It enters the Indian Territory near Hanuman Nagar in Nepal. It joins the Ganga River near Kursela in Katihar district. In Nepal, this river is known as “Saptakoshi”. It is formed by the confluence of seven smaller streams, namely, the Sun Koshi, the Bhote Koshi, the Tama Koshi, the Dudh Koshi, the Barun Koshi, the Arun Koshi and the Tamor Koshi, meeting above Tribeni, about 10 km. upstream of Chatara. But for all practical purposes, the confluence at Tribeni in Nepal is considered to be formed by the three major tributaries out of the seven, the Arun Koshi from North, the Sun Koshi from West and the Tamor Koshi from East. Below the confluence at Tribeni, the Koshi flows in a narrow gorge for a length of about 10 km., till it debouches into plains, near Chatara in Nepal. Further down, the river runs in relatively flat plains of Nepal. The river flows through Nepal for 50 km below Chatara to Hanuman Nagar, before it enters the Indian Territory. Below Hanuman Nagar, the river Koshi runs about 100 km in a sandy tract and finds its way southward through several channels. After that, the river takes an eastward direction and has a single defined channel. The main channel joins the river Ganga near Kursela in Katihar district. In plains of Bihar, the river has two important right tributaries; these are the Bagmati and the Kamla Balan. The other tributaries worth mentioning on the right bank are the Trijuga and the Bhutahi Balan.
Socioeconomic conditions: Koshi River Basin has a population of 15.3 million (2009). The total annual GDP was about USD$ 10.4 billion in 2009 (i.e. less than USD$ 700 per capita/year). The region in India is mostly alluvial with subtropical climate and is very productive in agriculture. However, due to its large population (about 1000 people per km2), the average income in the region is below the national average of India (USD$ 1134 in 2009). The Nepalese in the region are about 6 million (1/4 of the country’s population). The population density is 200 km−2 varying from 32 (Solukhumbu) to 276 (Kavre) in the central part of the Koshi River. Forty percent of the residents are below Nepal’s poverty line (higher than the national average of 30%), but the GDP per capita in the region is near to Nepal’s national average (USD$ 427). The population in China’s territory is 94 thousand (with an average population density of 3.2 km−2 varying between 1.9 and 5.5 km−2). The average GDP per capita of Chinese in the region is USD$ 1970. Clearly, the region in the middle of the basin has the worst economic condition. The population density increases rapidly from the upstream to downstream.
The GH2MF2 System is integrated into a Decision Support System (DSS) platform, based on geographical data. The DSS is an innovative web application (for Pcs, Tablets, Mobile phones) not requiring installation (a web browser and an internet connection only) or updating (all upgrading being deployed on the remote server). The proposed platform is meant to be participative and interactive. To meet this objective, all information used in the system must be freely accessible to users, including the hydrological model, and allowed to set-up the models for the benefit of the community.
The novel aspect of the proposed project and its platform is that any user will be allowed to set-up, calibrate and validate hydrological models, by downloading from the platform the base maps (e.g. digital elevation model) and the hydro-meteorological forcing data. Validated models (according to provided standards) will be then uploaded on the platform to be part of the daily run of the system and to be available to other users, who can view and interrogate the platform for forecasts in the basins. The hydrological simulations will also be processed by the platform for their visualization and for diffusion of warnings when extreme conditions (e.g. floods and droughts) are reached. The platform comprises the following components:
To meet the requirements of availability, stability, interoperability and portability, a systematic architecture of three tiers, including the presentation, application and database and model, is considered. The Web GIS DSS will be built upon a platform of proven open source component including Geoserver, Open Layers, ExtJs, PostgreSql with PostGis extension, GDAL libraries. It implements Open Geospatial Consortium (OGC) standards, including Web Map Server (WMS), and Web Feature Service (WFS). The main available functions of the platform are:
The hydrological simulations are generated by distributed hydrological model TOPKAPI-X (TOPographic Kinematic APproximation and Integration – extended). TOPKAPI-X has already accumulated a long history in the literature as well in operational experiences all over the world. Thanks to its computational efficiency, global hydrological simulations can be performed with hourly time-step and a 1×1 km2 spatial resolution.
TOPKAPI-X is a fully distributed and continuous hydrologic model, with a simple and parsimonious parameterization. The model is based on the idea of combining the kinematic approach and the topography of the basin. Spatial distribution of catchment parameters, precipitation input and hydrologic response is achieved horizontally by an orthogonal grid network and vertically by soil layers at each grid pixel. Four ‘structurally similar’ non-linear reservoir differential equations characterize the TOPKAPI-X approach and are used to describe subsurface flow (superficial and deep layers), overland flow and channel flow. Moreover, the TOPKAPI-X model includes components representing the processes of the hydrologic cycle: infiltration, percolation, evapotranspiration and snowmelt, plus a lake/reservoir component, a parabolic routing component and a groundwater component. Being a physically based model, the values of the model parameters can be easily derived from digital elevation maps, soil type and land use maps in terms of topology, slope, soil permeability, soil depth and superficial roughness. A calibration based on observed streamflow data is then necessary for ‘fine tuning’ the model to reproduce the behavior of the catchment. Thanks to its physically based parameters, the TOPKAPI-X model can be successfully implemented also in un-gauged catchments where the model cannot be calibrated using measured data. In this case the model parameters can be derived from thematic maps. The reference base maps used for hydrological model in Koshi River Basin implementation are:
HydroSHEDS (Hydrological data and maps based on SHuttle Elevation Derivatives at multiple Scales) provides hydrographic information in a consistent and comprehensive format for regional and global-scale applications. HydroSHEDS offers a suite of geo-referenced data sets (vector and raster), including stream networks, watershed boundaries, drainage directions, and ancillary data layers such as flow accumulations, distances, and river topology information. HydroSHEDS is derived from elevation data of the Shuttle Radar Topography Mission (SRTM) at 3 arc-second resolution.
The parameters of hydraulic properties of soils have been obtained from the combination of three datasets at 1km2 resolution: The Harmonized World Soil Database Version 1.1. HWSD, SoilGrids 1Km and the HiHydroSoil database.
Land Use Maps Data: The Global Land Cover-SHARE (GLC-SHARE) is a new land cover database at the global level created by FAO, Land and Water Division in partnership and with contribution from various partners and institutions. It provides a set of eleven major thematic land cover layers resulting by a combination of “best available” high resolution national, regional and/or sub-national land cover databases. The database is produced with a resolution of 30 arc-second (~1×1 km2).
Last Updated: May 14, 2019 by wp_9282510 Leave a Comment
The importance of coupling meteorological and hydrological forecasts to anticipate Flash Floods. The case of Sant Llorenç des Cardassar (Illes Balears – Spain)
EGU2019-132 | Orals | HS4.1.3/NH1.32
The importance of coupling meteorological and hydrological forecasts to anticipate Flash Floods. The case of Sant Llorenç des Cardassar (Illes Balears – Spain)
Thu, 11 Apr, 16:15–16:30 Room 2.25
Geophysical Research Abstracts
Vol. 21, EGU2019-132, 2019
EGU General Assembly 2019
The following analysis tries to address if the combination of meteorological and hydrological models can give more information to the Civil Protection authorities to take measurements to palliate the impacts of flash flood damages of the town of Sant Llorenç des Cardassar on the island of Mallorca (Illes Balears-Spain) that suffered the afternoon of October 9, 2018 a flash flood due to a Convective Mesoscale System whereby the Torrent Ses Planes Basin (ephemeral creek) overflowed in the channeling of this town flooding part of its urban center with the tragic result of 13 killed and extensive material damages.
The basin is ungauged, and we proceeded to reconstruct the hydrometeorological event in a comprehensive manner with a feedback to calibrate/validate the response of the basin through a hydrological TOPKAPI-X model acronym of (TOPographic Kinematic APproximation and Integration-eXtended) physically-based distributed rainfall-runoff model deriving from the integration in space of the kinematic wave. The approach transforms the rainfall-runoff and runoff routing processes into four ‘structurally-similar’ non-linear reservoir differential equations describing different hydrological/hydraulic processes. Once having the hydrological model calibrated/validated and operative forcing it with the forecasts of the Meteorological model HARMONIE-AROME acronym of HARMONIE (HIRLAM ALADIN Research on Meso-scale Operational NWP In Europe), AROME (Application of Research to Operations at MEsoscale) which is a non-hydrostatic spectral model, the dynamical core is based on a two-time level semi-implicit Semi-Lagrangian discretization, in the reference cycle 40h1.1, lateral boundary conditions are routinely used from the ECMWF model. Sixty-five levels are used in the vertical with model top at 10 hPa and lowest level at 12 m. The horizontal resolution is 2.5 km, and time step is 75 s. The feedback process begins with the reconstruction of the spatial-temporal precipitation pattern, combining satellite products PERSIANN-CSS, GPM-IMERG, the product of the AEMET meteorological radar and the 4 closest raingauges to force the hydrological model TOPKAPI-X and obtain the response of the basin that in turn serves to force a bidimensional hydraulic model IBER, which is a 2D model for the simulation of free surface flow in rivers and can solve hydrodynamics, turbulence and sediment transport, to reproduce water depths, flooded area and its hydrodynamics in the surroundings of the urban center of Sant Llorenç contrasted in turn with recorded data post-event (observed/measured water levels, videography and satellite maps processed by Rapid Mapping COPERNICUS-EMS).
Finally, once the hydrological model was calibrated/validated and operational, we proceeded to force it with Runs (00Z, 06Z, 12Z) of the HARMONIE-AROME to analyze if the warning or alarm thresholds had been exceeded, in hydrological terms, with enough anticipation to warn to the citizens of an imminent situation of flash flood. The results reveal that coupling meteorological/ hydrological models at an operational level can give a better input for the decision-making process to mitigate extreme natural hazards. This coupling seams extremely necessary and useful to the authorities due to the high non-linearity of the rainfall-runoff processes. This modelist cascade could offer the possibility to issue different types of warnings from meteorological and hydrological or hydraulic point of view.
Posted: August 3, 2018 by wp_9282510 Leave a Comment
GH2MF2 has joined the CSDMS Project – Hydrology Focus Research Group
The Community Surface Dynamics Modeling System (CSDMS) deals with the Earth’s surface – the ever-changing, dynamic interface between lithosphere, hydrosphere, cryosphere, and atmosphere. We are a diverse community of experts promoting the modeling of earth surface processes by developing, supporting, and disseminating integrated software modules that predict the movement of fluids, and the flux (production, erosion, transport, and deposition) of sediment and solutes in landscapes and their sedimentary basins.
Last Updated: July 28, 2018 by wp_9282510 Leave a Comment
From catchment to global scale; towards hyperresolution modeling?
In this Post we want to share with the hydrological and meteorological community, the brilliant Keynote of Professor Eric F. Wood (Department of Civil and Environmental Engineering, Princeton University. Princeton NJ 08544 USA) that performed in the International Symposium on Distributed Hydrological Modelling. University of Bologna To mark the 70th birthday of Prof. Ezio Todini (5-7 June 2013).
This presentation was very important to us for the discussion of ideas and the subsequent foundation of the Non Profit Project: GH2MF2 – Association for a Participated Global Hydrological Monitoring and Flood Forecasting System. From here we want to thank Prof. Wood for his Keynote:
KEYNOTE Session IV: From catchment to global scale; towards hyperresolution modeling?
Continental-scale ‘hyper-resolution’ land surface modelling: Challenges and initial results.
By Eric F Wood (efwood@princeton.edu)
Abstract
Land surface models have their roots in numerical weather prediction and global circulation models. As a result, they emphasize simulating the land surface water and energy fluxes while oversimplifying the hydrology. This limits their usefulness for fine scale processes related to water management, floods, water quality, land-use effects, among others. Advances in data availability and computation have led the community to pose the grand challenge of creating global scale land surface models that can better capture high resolution hydrologic and landscape processes so as to improve their predictive ability at landscape scales (Wood et al. 2011).
In this presentation, we will discuss three challenges that need to be addressed in reaching this goal.
1. Computation: Future models must be highly parallelizable, allow for nested gridding, and take advantage of object-oriented programming. These would best be implemented with open source collaboration.
2. Data: To harness current and future high-resolution input data sets, data assimilation and ensemble frameworks should be used to account for uncertainties in the model parameters and input data.
3. Physics: Many underlying, local scale physical processes are still poorly understood and require improved parameterizations. This includes preferential flow paths, urban hydrology, landscape heterogeneity, and land-atmosphere interactions.
A convergence of broad research results is needed to bring the developments together into new parameterizations.Finally, we will illustrate the capabilities of current high-resolution hydrologic models by running the updated TOPLATS model (Pauwels and Wood, 2009; over the continental United States at a 15 arc sec (~500m) spatial resolution.The model is run with a suite of state of the art high resolution meteorological and land surface data sets. The results will be compared to coarser scale simulations using the VIC model to assess the potential for improved fine scale and up-scaled monitoring of the global hydrologic cycle. We will discuss the feasibility and steps necessary to apply this framework globally
References
Posted: July 21, 2018 by wp_9282510 Leave a Comment
Processing MOD16A2 Product (PET/ET) in the Ganges-Brahmaputra-Meghna River Basin for Hydrological Modeling in GH2MF2 Non-Profit Project
In the Non-Profit GH2MF2 association we have processed the MODIS product (Moderate Resolution Imaging Spectroradiometer) MOD16A2: MODIS / Terra Net Evapotranspiration 8-Day L4 Global 500 m SIN Grid V006 in the basin of the Ganges-Brahmaputra-Meghna rivers. The objective is to help the calibration and validation of the hydrological models in the project, initially in the Koshi river basin that will be extended to the other basins of the Ganges-Brahmaputra-Meghna System.
However, and following the founding principles of the association Non-Profit GH2MF2, we make available to the scientific community and government entities related to hydrology, water resources, agriculture of the states of Nepal, China, Bhutan, India, Bangladesh and Myanmar the processed and unified dataset in ESRI ASCII Grid format. Due to the size of the Dataset we make available for download the period between 01/01/2017 until 06/17/2018 in the following link. If you are interested in the complete Dataset please contact the secretariat of the Association GH2MF2 info@gh2mf2.org
Global climate change will affect precipitation and ET, and hence influence the renewable freshwater resources ET is the second largest component (after precipitation) of the terrestrial water cycle at the global scale, since ET returns more than 60% of precipitation on land back to the atmosphere (Korzoun et al., 1978, L’vovich and White, 1990 ) and thus conveys an important constraint on water availability at the land surface. In addition, ET is an important energy flux since land ET uses up more than half of the total solar energy absorbed by land surfaces (Trenberth et al., 2009). Accurate estimation of ET not only meets the growing competition for the limited water supplies and the need to reduce the cost of the irrigation projects, but also it is essential to projecting potential changes in the global hydrological cycle under different climate change scenarios (Teuling et al 2009).
MO
DIS instrument is operating on both the Terra and Aqua spacecraft. It has a viewing width of 2,330 km and views the entire surface of the Earth every one to two days. Its detectors measure 36 spectral bands and it acquires data at three spatial resolutions: 250-m, 500-m, and 1,000-m.
We have created a Dataset from the MOD16A2 data of potential evapotranspiration (PET) and real evapotraspiration (ET) at a daily time scale with spatial resolution of 0.0083º (about 1 Km at the equator) with geoDatum WGS84, 446 rows and 3133 columns with center of the first cell in the upper left corner: Longitude = 73.0041º, Latitude 32.9977º. For this, we have processed 9 tiles (download Excel sheet with the files) in the Sinusoidal grid tiling system used in many Land MODIS products from January 01, 2001 to June 17, 2018, the tiles are:
h23v05
h23v06
h24v05
h24v06
h25v05
h25v06
h26v05
h26v06
h27v06
MODIS filenames (i.e., the local granule ID) follow a naming convention which gives useful information regarding the specific product. For example, the filename MOD09A1.A2006001.h08v05.006.2015113045801.hdf indicates:
MOD09A1 – Product Short Name
.A2006001 – Julian Date of Acquisition (A-YYYYDDD)
.h08v05 – Tile Identifier (horizontalXXverticalYY)
.006 – Collection Version
.2015113045801 – Julian Date of Production (YYYYDDDHHMMSS)
.hdf – Data Format (HDF-EOS)
The MOD16A2 Version 6 Evapotranspiration / Latent Heat Flux product is an 8-day composite product produced at 500 meter pixel resolution. The algorithm used for the MOD16 data product is based on the logic of the Penman-Monteith equation, which includes inputs of daily meteorological reanalysis data along with MODIS remotely sensed data such as vegetation property dynamics, albedo, and land cover.
Provided in the MOD16A2 product are layers for composited Evapotranspiration (ET), Latent Heat Flux (LE), Potential ET (PET) and Potential LE (PLE) along with a quality control layer. Two low resolution browse images are also available for each MOD16A2 granule, (1) ET and (2) LE. The pixel values for the two Evapotranspiration layers (ET and PET) are the sum of the eight days within the composite period and the pixel values for the two Latent Heat layers (LE and PLE) are the average of the eight days within the composite period . Note that the last 8-day period of each year is a 5 or 6-day composite period, depending on the year.
Flowchart of the improved MOD16 ET algorithm. LAI: leaf area index; FPAR: Fraction of Photosynthetically Active Radiation.
Citation:
Running, S., Mu, Q., Zhao, M. (2017). MOD16A2 MODIS / Terra Net Evapotranspiration 8-Day L4 Global 500m SIN Grid V006 [Data set]. NASA EOSDIS Land Processes DAAC. doi: 10.5067 / MODIS / MOD16A2.006
Special thanks to Salvatore Falanga Bolognesi, PhD collaborator of the “GH2MF2 – Association for a Participated Global Hydrological Monitoring and Flood Forecasting System” for processing and post-processing this dataset.
Last Updated: July 14, 2018 by wp_9282510 Leave a Comment
A keynote to understand the Predictive Uncertainty in Flood Forecasting, by Prof. Ezio Todini
From deterministic to probabilistic thresholds: possibly a better way to take advantage of predictive uncertainty in flood emergency management
For quite a long time, traditional approaches to flood warning and flood emergency management have been based on the comparison of measured water stages with pre-determined ‘threshold levels’ (warning, alert, flooding levels, etc.). The measured water stage values were generally considered as ‘deterministic’given the relatively small measurement errors (1–2 cm). In recent decades, with the aim of anticipating both decisions and the consequent emergency actions, the original water stage measurements have been substituted by hydrological or hydraulic models’ forecasts. By doing so, the model forecasts were assumed to be of the same quality as the measurements and forecasting errors were totally disregarded. Fortunately, the recent introduction of the predictive uncertainty concept has created the prerequisite to change the original ‘deterministic threshold’ paradigm into a new ‘probabilistic threshold’ paradigm. The aim of this post in the Website of Non-Profit GH2GM2 association is to discuss the ‘probabilistic thresholds’ paradigm by showing how this may lead to a dynamic application of the ‘principle of precaution’ as a function of the degree of predictive uncertainty with consequent benefits both in terms of increased reliability and robustness of decisions.
Introduction
Today, similarly to what was done for more than a century, in order to trigger their decisions, the majority of water authorities involved in flood emergency management prepare their plans on the basis of pre-determined water depths or thresholds ranging from the warning water level to the flooding level. Decisions, and consequent actions, are then taken as soon as a real time measure of the water stage overtops one of these thresholds. This is a way of anticipating events on the basis of water level measures in the cross-sections of interest or in upstream cross-sections, when flood forecasting models are not available,but can only be effective on very large rivers where the time lag between the overtopping of the warning and the flooding levels is sufficiently large to allow for the implementation of the planned flood relief strategies and interventions.
Given that all the water stage measures are affected by relatively small errors (1–2 cm), they can be, and have been, considered as ‘deterministic’; therefore, in the sequel this approach will be referred to as the ‘deterministic threshold paradigm’.
Unfortunately, the advent and the operational use of real time flood forecasting models has not changed this paradigm, which has been the cause of several unsatisfactory results. Today, instead of comparing the actual measurements, as previously done, flood managers started comparing the hydrologic or hydraulic models forecast to the different decision triggering water stage thresholds. This is obviously done with the intent of further anticipating decisions by taking advantage of the forecast lead-time. Unfortunately, by doing so, the forecasts are implicitly assumed to be ‘real’ and ‘deterministic’, which is not the case, given that the forecasts, by their nature are ‘virtual reality’ and are affected by prediction errors, the magnitude of which is far larger than that of the measurement errors.
More recently, the concept of ‘predictive uncertainty’ has radically changed the ‘deterministic threshold paradigm’. Predictive uncertainty is defined as the probability of occurrence of a future value of a predictand (such as water level, discharge or water volume) conditional on prior observations and knowledge as well as on all the information one can obtain on that specific future value, which is typically embodied in one or more hydrological/hydraulic model forecasts (Krzysztofowicz, 1999).
This inherent uncertain nature of forecasts, as opposed to the higher accuracy of measurements, requires the definition of a ‘probabilistic threshold paradigm’, defined in terms of the ‘probability of flooding’ taken at different probability levels (20%, 50%, etc.) instead of the definition of ‘deterministic’ threshold values. In other words, the decision triggering threshold will not be based on different water stages (warning level, alert level, flooding level), but rather on different probabilities of flooding. It is true that a more formal Bayesian approach would require the definition of a utility function expressing the decision maker propension at risk and the computation of its expected value. The probabilistic thresholds coincide with the simplistic assumption of a constant utility, which may be reasonable in the absence of additional information. The probabilistic thresholds, as opposed to the deterministic water level thresholds, can result in improved tools in the hands of decision-makers. As it will be shown in the sequel, using the probabilistic thresholds, the same predicted water level may not have the same meaning and the same effect on decisions owing to the different reliability of the prediction. In other words, the same forecast may or may not trigger the decision of issuing a warning or evacuating an area, conditionally to its assessed level of uncertainty. More uncertain forecasts need necessarily to be treated more cautiously than more reliable ones; uncertain lower water stage forecasts could then trigger a protective measure, whereas higher, albeit more accurate water stage forecasts would not.
Nonetheless, the pre-requisite to implement the new probabilistic threshold paradigm is an accurate and effective estimate of predictive uncertainty. It is the aim of this post to discuss the introduction of a new ‘probabilistic thresholds’ paradigm and how this is conditioned upon a reliable estimate of predictive uncertainty. The post also aims at showing how the probabilistic threshold paradigm may lead to a dynamic application of the ‘principle of precaution’ as a function of the degree of predictive uncertainty with consequent benefits both in terms of increased reliability and robustness of decisions.
The definition of predictive uncertainty
In water resources management, and more specifically in flood emergency management, decisions, which may generate dramatic social and economical consequences, must be taken on the basis of variables such as water stages, discharges, runoff volumes, etc. without perfect knowledge of the future evolution of the hydro-meteorological phenomena. This lack of knowledge or uncertainty on future occurrences is commonly called ‘predictive uncertainty’. The state of knowledge of a decision-maker may be assumed to be a mixture of ‘what he knows’, or better still ‘what he believes he knows’ (in the sense that he may be wrong), which is a ‘subjective state of mind’ and what he learns from observations (which include data and models), which can be considered as ‘objective’. Therefore, following Rougier (2007), a possible definition of predictive uncertainty is:
“Predictive uncertainty is the expression of a subjective assessment of the probability of occurrence of a future (real) event conditional upon all the knowledge available up to the present (the prior knowledge) and the information that can be acquired through a learning inferential process.”
From this definition, the need emerges for using hydrological model forecasts to reduce the predictive uncertainty, usually expressed in terms of a probability density (or probability distribution) function, ‘conditional’ upon the available observations and hydrological model forecasts, which are now seen as the available, although uncertain, extensions into the future of observations. In other words, hydrological model forecasts are a way to complement the prior belief of the decision-maker in order to reduce ‘his’ prior uncertainty within the frame of the decision- making process. This way of looking at hydrological model forecasts is the opposite of current operational practice where (explicitly or implicitly) models are assumed to provide deterministic (and therefore ‘certain’) forecasts such as future levels, flows, etc. Krzysztofowicz (1999), was the first to clarify, within the hydrological context, that the objective of forecasting is the assessment of the probability that future values of water stage, discharge, runoff volume, etc. will be smaller, greater or equal to given values (generally threshold values, such as for instance the elevation of the dykes), rather than the estimation of the uncertainty of the same quantities forecast by hydrological models.
Combining measurements and models to improve predictability
As previously stated, hydrological prediction must aim at the reduction of the decision-maker’s uncertainty on the future occurrence of quantities such as future water levels, discharges or water volumes, that will be called ‘predictands’ in the sequel. To do so, the decision-maker generally starts from his prior belief. For instance, he can use the climatological distribution of extreme discharge occurrence to describe his prior belief on the possibility of flooding, but in general, the relevant probability density function is very flat and is not sufficiently dense around some specific value to allow reliable decisions, such as issuing a flood alert. Therefore it is necessary to gather additional information, additional measurements or to generate future scenarios by means of one or more forecasting models.
There is no substantial difference between a measured or a modelled quantity apart from the type of errors affecting them and the fact that modelled quantities may be available at a future time. Measurements, although affected by measurement errors, can be reasonably accurate. But if these measurements are indirect measures of the predictand, they become ‘predictors’, which implies that they will also be affected by modelling errors, similarly to ‘model predictions’. Modelled quantities incorporate both measurement errors and model errors, that can be large if the model is not very accurate. Nonetheless, models become essential when dealing with ‘forecasting’; because measurements are not available at any future time, therefore one can only use modelled quantities ito increase insight into the future, and consequently reduce uncertainty.
The forecasting problem can be usually tackled on the basis of two different approaches, depending on its nature and on the decision problem to be solved. The first approach relates to cases where only the total probability (namely the integral of the predictive density) above or below a threshold is needed. This is the case for instance when one has to decide whether a landslide will or won’t occur on the basis of one or more sensors or models. The second approach relates to continuous processes, requiring the estimation of the entire predictive probability function: for instance when dealing with flood damages, which vary with the water level. In this case decisions tend to be taken on the basis of the expected damages, which can only be estimated if the full probability density of predicted future water levels is available.
The probabilistic threshold paradigm
Today, similarly to what was done for more than a century, in order to trigger their decisions, the majority of water authorities involved in flood emergency management prepare their plans on the basis of pre-determined water depths or thresholds ranging from the warning water level to the flooding level. Decisions, and consequent actions, are then taken as soon as a real time measure of the water stage overtops one of these thresholds. This approach, which is correct and sound in the absence of flood forecasting models, is a way of anticipating events on the basis of water level measures (in the cross sections of interest or in upstream cross sections), but can only be effective on very large rivers where the time lag between the overtopping of the warning and the flooding levels is sufficiently large to allow for the implementation of the planned flood relief strategies and interventions. Given that all the water stage measures are affected by relatively small errors, they can be, and have been, considered as deterministic.
Unfortunately, the advent and the operational use of real time flood forecasting models, has not changed this approach, which has been the cause of several unsatisfactory results. Today, the flood managers compare forecasts, and not the actual measurements, to the different threshold levels; this is obviously done in order to further anticipate decisions by taking advantage of the prediction time horizon. Unfortunately, by doing so the forecasts are implicitly assumed to be deterministic, which is not the case since they represent virtual reality and are affected by prediction errors, which magnitude is by far larger than that of the measurement errors. More recently, the concept of predictive uncertainty has changed this approach. This uncertain nature of forecasts, opposed to the higher accuracy of measurements, requires the definition of probabilistic thresholds, defined in terms of the probability of flooding taken at different probability levels, instead of the definition of deterministic threshold values.
Using the probabilistic thresholds, the same predicted water level may have different meaning owing to the reliability of prediction. In other words, the same forecast may or may not trigger the decision of issuing a warning or evacuating an area, conditionally to its assessed level of uncertainty. More uncertain forecasts need necessarily to be treated more cautiously than more reliable ones; in fact, uncertain lower water stage forecasts could then trigger a protective measure, whereas higher, albeit more accurate water stage forecasts, would not. As can be seen from the Fig. 1, for the same expected value (the horizontal dashed line) a better forecast (Model A), characterised by a narrower predictive density, will show a smaller probability of exceeding the flooding level when compared to a worse one (Model B). This property can be also looked at from an alternative perspective, as shown in Fig. 2 the same flooding probability corresponds to lower expected values as the spread of PU increases. This implies that if a probabilistic threshold is defined instead of a deterministic threshold level, when the PU is larger the decision maker must be more cautious and would be advised to issue an alert even when, looking at the expected value of the forecast, he would not think of issuing it, because he may regard it as being too low.
Posted: July 13, 2018 by wp_9282510 Leave a Comment
Hydrological Modeling – Collaborate with us, for a better Earth in a moment of change, all ideas or proposals are welcome
There are many current hydrological models, almost as many as research groups of different research entities or university departments.
Hydrological models can be aggregated, semi-distributed, distributed conceptually or physically based, stochastic and black-box or data-driven (ANN’s), which is true, is that there is no universal model, an interesting reflection is the one made in 2002 and published in Advances in Water Resources 25 (8): 1313-1334: Advances in the use of observed spatial patterns of catchment hydrological response – doi: 10.1016 / S0309-1708 (02) 00060-X
In the Non-Profit GH2MF2 project we have initially started to use the physically distributed model based TOPKAPI-eXtended because it is free to use and open source software and part of the members of the association have contributed to its development.
Your hydrological model and your knowledge is important for the objectives of the Non-Profit Project GH2MF2, do you want to collaborate? get in touch with us at info@gh2mf2.org
You can model the basins in which we have started working or propose other basins in which you have a background and are useful to add in the open platform WEBGIS GH2MF2
The WEBGIS platform is able to incorporate other hydrological models, because it intends to be multi-model, and that is why we invite the International Hydrological Community to collaborate in this Non-Profit project by making available their models and collaborating in their incorporation to make a post- processed to estimate the Predictive Uncertainty based on the MCP (A Model Conditional Processor to Assess Predictive Uncertainty in Flood Forecasting) published in International Journal of River Basin Management (Todini, E. https://doi.org/10.1080/15715124.2008.9635342)
If you want to know more about the proposed methodology on which the Model Conditional Processor is based, you can download the paper published in HESS 2011 “Recent developments in predictive uncertainty assessment based on the model conditional processor approach” (Coccia. G and Todini, E. doi:10.5194/hess-15-3253-2011)
The following video shows the benefits of the methodology proposed in the project NON-Profit GH2MF2 in the excellent presentation by Coccia, G.: “MULTI-TEMPORAL UNCERTAINTY PROCESSORS FOR REAL TIME FLOOD FORECASTING“, with examples in the River Basins of the Po River( Italy) and Baron Folk River (USA) making use of three Hydrological Models widely used in the world and sanctioned by their high quality: TOPKAPI-eXtended, TETIS (GIHMA, UPV) and a Model of Neural Networks ANN:
Presented at International Symposium on Distributed Hydrological Modelling, University of Bologna To mark the 70th birthday of Prof. Ezio Todini, 5-7 June 2013.
CONVENERS
Prof. Patrick Enda O’Connell – University of Newcastle upon Tyne
Prof. Alberto Montanari – DICAM – University of Bologna
Prof. Enzo Farabegoli – BiGeA – University of Bologna
Posted: July 12, 2018 by wp_9282510 Leave a Comment
Using the fantastic Precipitation Dataset MSWEP V2.1 to force and calibrate the TOPKAPI-eXtended Hydrological Model within the Non-Profit GH2MF2 Project
In the Non-Profit GH2MF2 project the MSWEP V2.1 product dataset (Tile 243) is being used to force and calibrate the physically distributed hydrological model based on the Koshi River Basin together with the dataset derived from the CFSR Reanalysis of Temperature.
The observed flow data (Gauge Stations) for calibrate the Hydrological model on Koshi River Basin has been kindly provided by Department of Hydrology and Meteorology, Ministry of Energy, Water Resources and Irrigation – Government of Nepal
The Dataset MSWEP in NetCDF format of Tile 243 used in this project can be downloaded at the following link: MSWEP-KOSHI RIVER BASIN
By using MSWEP in any publication you agree to cite Beck et al. (2017). The dataset is being developed by Hylke Beck (Princeton University, Princeton, USA) in collaboration with Ming Pan, Eric Wood (Princeton University, Princeton, USA), Albert van Dijk (ANU, Canberra, Australia), Ad de Roo (JRC, Ispra, Italy), Vincenzo Levizzani (CNR-ISAC, Bologna, Italy), Jaap Schellekens (Deltares, Delft, The Netherlands), Diego Miralles (VU University Amsterdam, The Netherlands), and Brecht Martens (Ghent University, Ghent, Belgium). We gratefully acknowledge the precipitation dataset developers for producing and making available their datasets. The work was supported through IPA support for Hylke Beck from the U.S. Army Corps of Engineers’ International Center for Integrated Water Resources Management (ICIWaRM), under the auspices of UNESCO, to further develop a Latin America and Caribbean Drought Monitor.
Last Updated: July 11, 2018 by wp_9282510 Leave a Comment
SM2RAIN-CCI Derived Rainfall from the inversion of the satellite soil moisture
SM2RAIN-CCI: a new global long-term rainfall data set derived from ESA CCI soil moisture
https://www.earth-syst-sci-data.net/10/267/2018/essd-10-267-2018.pdf
Last Updated: July 10, 2018 by wp_9282510 Leave a Comment
Super Typhoon Maria – GPM (IMERG) Precipitation Product
Recent Posts
Recent Comments
Archives
Recent Tweets