Methodology for ex-post evaluation of measures and
instruments in flood risk management (
postEval
)
Summary of Contents: Measures and Instruments Conditions
Evaluation framework
Criteria, indicators and methods Selection of indicators
Case study results
Conclusions and recommendations
August 2007
Report Nr: T12-07-01
D
OCUMENT
I
NFORMATION
Title Methodology for ex-post evaluation of measures and instruments in flood risk management (postEval)
Lead Author Alfred Olfert
Contributors Jochen Schanze
Distribution Project Team
Document Reference
Olfert A and Schanze J (2007), Methodology for ex-post evaluation
of measures and instruments in flood risk management (postEval), Leibniz Institute for Ecological and Regional Development (IOER), FLOODsite Report T12-07-01, Dresden.
D
OCUMENT
H
ISTORY
Date Revision Prepared by Organisation Approved by Notes
07/01/07 1.0draft A. Olfert IOER 27/08/07 1.1draft A. Olfert IOER
D
ISCLAIMER
This report is a contribution to research generally and third parties should not rely on it in specific applications without first checking its suitability.
In addition to contributions from individual members of the FLOODsite project consortium, various sections of this work may rely on data supplied by or drawn from sources external to the project consortium. Members of the FLOODsite project consortium do not accept liability for loss or damage suffered by any third party as a result of errors or inaccuracies in such data. Members of the FLOODsite project consortium will only accept responsibility for the use of material contained in this report in specific projects if they have been engaged to advise upon a specific commission and given the opportunity to express a view on the reliability of the material concerned for the particular application.
S
UMMARY
Design, implementation and operation of measures and instruments for effective and sustainable flood risk reduction require knowledge from previous practice. Systematic ex-post evaluation of interventions into the flood risk system and is an important basis for learning in support of future development of strategies and strategic options in flood risk management. Lacking methodological approaches currently prevent from comprehensive ex-post evaluation. As an effect, these remain uncommon, irregular and unsystematic.
The report drafts a
“
Methodology for ex-post evaluation of pre-flood measures and instruments” (ex-post EFM) for the investigation of (side-)effects, effectiveness, efficiency, robustness and flexibility of physical measures and policy instruments. The methodology aims at providing a framework for the evaluation of measures and instruments after their implementation. The framework is laid out to be generically applicable with all measures and instruments at project level. By applying the methodology, information about existing measures and instruments shall be made available for the planning of future flood risk reduction.The Methodology addresses pre-flood and flood event measures and instruments at project level aimed at the reduction of flood risk respectively flood damage. Interventions in all elements of the Source-Pathway-Receptor-Consequences model are considers. Interventions of interest for ex-post evaluation are single measures and instruments or strongly connected combinations of those seen in the context of selected natural and societal conditions.
The Methodology mainly consists of criteria and methods for the evaluation of physical measures and policy instruments. These aim at exploring effects (incl. side-effects), effectiveness, cost effectiveness, robustness and flexibility of existing interventions in to the flood risk system under reverting to experiences from recent flood events. The overall performance of the interventions is investigated under consideration of hydrological, ecological, social and economic aspects. Corresponding to the multiple criteria approach of the methodology a wide range of methods is used including quantitative as well as qualitative approaches.
Natural and societal conditions are defined as part of the methodology and facilitate the case specific selection of criteria. The selection methodology enables a quick and systematic selection of appropriate criteria based on a partly formalised two step approach.
C
ONTENTS
Document Information ii Document History ii Disclaimer ii Summary iii Contents iv Table of contents A METHODOLOGY ... 1 1. Introduction ... 1 1.1 Background... 11.2 Research questions for ex-post evaluation... 2
1.3 Structure of the methodology ... 3
2. Measures and instruments... 6
2.1 Measures and instruments - interventions at project level... 6
2.2 Measures and instruments in the flood risk system ... 7
2.3 Measures and instruments – a classification... 10
2.3.1 Why a new classification... 10
2.3.2 Level 1: Physical measures and policy instruments... 11
2.3.3 Level 2: Functional character of measures and instruments ... 12
2.3.4 Level 3: Types of measures and instruments ... 14
2.4 Single measures and instruments and combinations of those... 16
3. Conditions for measures and instruments ... 18
3.1 Importance of conditions ... 18
3.2 Type of flood ... 19
3.3 Probability of flood... 20
3.4 Land use... 20
3.5 Type of water body... 20
3.6 Specific conditions... 21
4. Evaluation framework ... 22
4.1 Criteria – main viewpoints of evaluation... 22
4.2 Indicators – yardsticks of evaluation ... 23
4.3 Evaluation framework ... 24
5. Indicators, evaluation of criteria and evaluation methods ... 27
5.1 Evaluation of effects ... 27
5.1.1 Impacts of flooding and flood risk reduction ... 27
5.1.2 Categories of effects... 28
5.1.3 Indicators of hydrological and hydraulic effects... 29
5.1.4 Indicators of socio-cultural effects ... 29
5.1.5 Indicators of economic effects ... 30
5.1.6 Indicators of ecological effects ... 32
5.2 Evaluation of effectiveness... 40
5.2.1 Effectiveness - the degree of goal achievement ... 40
5.2.2 Objectives... 42
5.2.3 Determination of effectiveness... 42
5.3 Evaluation of efficiency – the benefit/cost ratio... 45
5.3.1 Economic and non–economic effects in cost-effectiveness ... 45
5.3.2 Benefits and costs in cost-effectiveness ... 45
5.3.3 Determination of cost-effectiveness... 47
5.4 Evaluation of robustness... 49
5.4.1 Robustness as reliability of intended performance... 49
5.4.2 Conditions of pressure... 50
5.4.3 Determination of robustness under conditions of pressure ... 50
5.5 Evaluation of flexibility... 53
5.5.1 Flexibility versus irreversibility ... 53
5.5.2 Determination of flexibility... 54
6. Selection of indicators ... 55
6.1 Introduction ... 55
6.2 Step 1 - Formalised reduction of the overall indicator set ... 55
6.3 Step 2 - Case specific selection of criteria... 57
B SUMMARY OF EVALUATION RESULTS ... 59
7. Case study summaries... 59
7.1 Risk reduction in private and commercial buildings during the April 2006 flood in Dresden... 59
7.2 Emergency storage at the Elbe river... 60
7.3 Risk reduction activities on the Odra river ... 61
7.4 Contingency planning in the Tisza river basin (Tisza A) ... 62
7.5 Hungarian-Ukrainian co-operation for flood and excess water defence along the upper Tisza river... 62
8. Overview of applied indicators and obtained results ... 64
C CONCLUSIONS AND RECOMMENDATIONS ... 69
9. Conclusions regarding the application of the methodology ... 69
9.1 Completeness and consistency of the indicator set... 69
9.2 Conclusions regarding the determination of criteria... 69
9.2.1 Determination of effects, effectiveness and cost-effectiveness... 69
9.2.2 Determination of robustness... 70
9.2.3 Determination of flexibility... 70
9.3 Conclusions regarding the framework of evaluation... 70
9.4 Challenges and constraints of ex-post evaluation in practice ... 71
9.5 Complimentarity of measures... 71
D REFERECES ... 73
E ANNEXES ... 81
ANNEX 1 APPENDICES TO THE METHODOLODY ... 82
Appendix 1 Indicators of hydrological/hydraulic effects ... 83
Appendix 2 Indicators of socio-cultural effects ... 93
Appendix 3 Indicators of economic effects... 105
Appendix 4 Inidcators of ecological effects ... 120
Appendix 5 List of identified measures and instruments ... 148
ANNEX 2 CASE STUDY REPORTS... 151 Report 1 Flood proving in the inundation areas of Dresden and Pirna (Germany) in the
April 2006 Elbe river flooding
Report 1 Emergency Storage at the Elbe River Report 3 Risk reduction activities on the Odra River
Report 4 Contingency Planning in the Tisza River Basin (Tisza A)
Tables
Table 1: Evaluation framework 26
Table 2: Indicators of hydrological and hydraulic effects 29
Table 3: Indicators of social effects 30
Table 4: Indicators of economic effects 32
Table 5: Indicators of effects related to soils and vegetation in source areas 33
Table 6: Indicators of limnological effects (WFD quality element level) 34
Table 7: Parameters of limnological effect-indicators and their relevance for the type of
water body (based on WFD, Annex V) 35
Table 8: Indicators of ecological effects in flood plains and at coastal shores 37
Table 9: Indicators of hydrological/hydraulic effectiveness 40
Table 10: Indicators of social effectiveness 41
Table 11: Indicators of economic effectiveness 41
Table 12: Examples of intervention specific indicators of effectiveness 42
Table 13: Overview of used criteria in the cases and schematic results 64
Figures
Figure 1: The concept of ex-post evaluation of measures and instruments 4
Figure 2: Realisation of programme goals through projects (simplified from Virtanen &
Uusikyla 2004) 6
Figure 3: Source-Pathway-Receptor-Consequence model (cf. DETR 2000) 8
Figure 4: Categories of measures and instruments 13
Figure 5: Types of measures and instruments 14
A METHODOLOGY
1. Introduction
1.1 Background
Ex-post evaluation of flood risk management activities is a valuable means of information for learning from existing practices. Existing measures and instruments for flood risk reduction show different degrees of success in sustaining their performance and in achieving intended and unintended effects. Feedback about these issues is required to support further development of planning and implementation practices. However, the ex-post evaluation so far remains irregular and unsystematic. Kumar et al. (2001, p. viii) observe that: “… there is little post-audit assessment of past mitigation efforts.” Indeed, little advance has been achieved 30 years after Gilbert White (1975, referred to in Burby et al. 1988, p. 9) stated that “there is a remarkable lack of knowledge about the … effectiveness of floodplain regulations”. Pottier (2003) claims the situation as described by White being totally true for France in 2003.
Where at least partial evaluation is implemented, performance of flood defences often is found to deliver the expected results, sometimes even unexpected benefits are reported (Thompson et al. 1991, MAFF 1996, 1997). However, systematic learning from what is being done should be a part of any intervention, whereby from the evaluation perspective it is less important, whether it delivers successes and failures from which to learn.
On the one hand, the intended performance of interventions can vary significantly. For example, considerable damage was caused during the August 2002 Elbe river flood in Germany after many flood defences failed incl. capacity exceedance of even large and major dams, overtopping and breach of levees, the burst of at least one retention basin etc. (cf. e.g. DKKV 2003, MZP 2003, Büchele et al. 2004, LFUG 2004). Danhelka and Kubát (2004, p. 42) compared the August 2002 flood wave propagation through Prague managed by the Vltava river cascade with the result, that “in case of non-existence of reservoirs the course of flood in Prague would be approximately similar to the one really observed.” Similar effects are reported from other structures, too (Tucci & Villanueva 1999). Thompson and Penning-Rowsell (1994) report of little evidence of vulnerability reduction after installation of flood defences in Bangladesh due to uneven distribution of benefits and increased losses in case of failure. László and Tóth (2001) have compiled 140 known dike failures over the past 55 years in Hungary, most of those due to overtopping. Robert et al. (2003) report of flood zone designation in Canada failing totally to influence floodplain encroachment, etc.
On the other hand, flood risk reduction activities can also have a range of unintended, unexpected or even undesired effects (cf. Green et al. 1994). For example, Smith (1990) shows a case of extreme flood acceleration as consequence of the installation of a flood defence dam. Takahasi (1976) summarises three main effects of structural flood defence measures at river Tone (Japan): significant increase in peak flood discharge, rapid change of river regime and an increase of damage potential in protected areas following improved defence standards. Especially the latter effect has been a major issue in research for decades. White et al. (1958) have presented a study showing massive encroachment in floodplains of 17 US-American cities. Parker (1995a) describes a similar trend for England and Wales observing that “progressively higher levels of flood defence are provided to protect against progressively increasing flood damage potential” (Parker 1995a, p. 341). What Parker calls “escalator effect”, the Australian Bureau of Transport and Regional Economics refers to as the “levee paradox” describing the “increase in potential damage resulting from floods greater than the design level” (BTRE 2002, p. xiii).
higher banks or massive realignment) that may simply transfer the flood problem downstream (cf. also IKSR 1997, Lammersen et al. 2002). Dahl and Fischer (2003) underline this issue by a few recent examples from the 2002 Elbe Flood in Germany. Green et al. (1994) observed this effects downdrift of coastal defence structures.
Montz and Gunfest (1986) claim, that despite massive expenses on structural flood control works flood losses continue to rise as effect of encroachment into floodplains. As a consequence, Thompson et al. (1991) conclude, that “urbanisation tends to increase the value of flood damages prevented” – subsequently improving the benefit/cost ratio (see above), while Montz and Gunfest (1986) in general call into question the effectiveness of defence measures that increase damage potential. Seen against the background of fast rising flood damages (Munich Re Group 2004), this issue has a considerable dimension.
Apart from the discussion of the projects’ performance in preventing damage, also reports of ecological and social side effects call for attention. These are particularly relevant for the performance balance of a measure or instrument (Green et al. 1994). Adverse ecological effects of flood defence have often been emphasised. Results range from negative impacts on water quality, hydrological and hydro-morphological processes or on landscape aesthetics to the complete loss of riverine marshes, forests and species (e.g. Wolters et al. 2001). The conflict of many structural solutions of risk reduction and river ecology has been ironically addressed by a German nature conservation organisation with its publication “The Flood Damage Mitigation Catastrophe” (Weber 2003). However, a significant interest of environmental implications of fluvial and coastal flood management arose only recently (Evans et al. 2004a, p. 155).
In the light of the European Water Framework Directive (CEC 2000) the question for measures combining attitudes of flood risk reduction and ecological improvement of waters is increasingly raised (Wolters et al. 2001, Birkland et al. 2003, Geilen et al. 2004, Morris et al. 2004). In some fields of flood risk management such as the river channel management, the direct implication of the WFD are already significant (cf. Evans et al. 2004a, p. 250).
While societal benefits of many flood risk reduction activities are beyond question, there still exists a row of issues that are worth considering as negative, some of which cannot be fully compensated by the benefiting society. On the one hand, this can relate to private properties. Impacts can range from limitations of property uses in specially designated areas (e.g. in case of construction ban) to a complete loss of properties where area is needed for defences up to the resettlement of complete villages from areas not at risk e.g. to allow for barrage construction. On the other hand, impacts caused by certain outcomes of risk reduction such as environmental degradation can indirectly cause loss of life resources in other places. The issue is well known from developing countries where communities still often depend on local resources. Prominent examples are declining inland fisheries in Bangladesh due to activities of the Bangladesh flood action plan (Sultana & Thompson 1997) or the loss of coastal fisheries in Egypt due to sediment trapped in the Assuan dam (McCully 2001).
1.2 Research questions for ex-post evaluation
The methodology provides a framework for systematic evaluation of measures and instruments for flood risk reduction (in the following synonymic with ‘interventions’). As subject of evaluation the methodology addresses measures and instruments at project level (see chapter 2.1) under consideration of conditions under which these are implemented and operated.
The methodology is dedicated to the following five research questions:
1) Which effects (intended/unintended; direct/indirect; short term/long term) referring to type and quality a measure or instrument cause in a certain flood event or over its life cycle through interaction with physical and socio-cultural conditions, into which it was introduced?
2) How effective has a measure or instrument been (to which extent did a measure or instrument achieve the underlying objectives)?
3) How cost-effective has a measure or instrument been (ratio of direct and indirect benefits and costs)?
4) How robust is a measure or instrument in the light of changing conditions? 5) How flexible is a measure or instrument in the light of changing conditions?
Addressees of this retrospective information are primarily planners, implementing bodies and decision makers of future risk reduction interventions. For their informed action requires detailed information which points directly towards strengths and weaknesses of certain measures and instruments under specific conditions. The methodology’s approach is to shed light on various single issues connected with the realisation and operation of measures and instruments. An aggregation of evaluated issues would lead to a loss of that concrete information and is therefore not attempted. Instead, each single aspect is emphasised, thus providing a ‘multi-spectral’ view of an intervention and its outcomes in all related fields.
1.3 Structure of the methodology
The focus of the methodology is the evaluation of the overall performance of measures and instruments. For this purpose, the methodology provides a framework for ex-post evaluation of such interventions. The framework is laid out to be generically applicable with all conceivable measures and instruments at project level in any element of the flood risk system (chapter 2.1). Figure 1 gives an overview of the general concept of evaluation into which the methodology is integrated. Here, the included elements provided by the methodology are connected to allow for an adequate evaluation of each case of evaluation.
METHODOLOGY EFFECT INDICATORS METHODS for data acquisition and effect analysis Effectiveness Efficiency Robustness Flexibility Ma Mb Mn Ia Ib In Case1 CONDITIONS T y pe of w a te r bod y T y pe of fl ood T y pe of land us e T y pe of int e rvention COMPARISON of c a se s ME ASUR E S a n d INST R U MEN T S METHODS for the evaluation of criteria Cross-case compa-rison CONCEPT OF EX-POST EVALUATION
Meas ur e/Instr u me nt 1 Effectiveness Efficiency Robustness Flexibility Ma Mb Mn Ia Ib In Case2 Meas ur e/Instr u m e nt 2 Effectiveness Efficiency Robustness Flexibility Ma Mb Mn Ia Ib In Casen M eas ur e/Instr u me nt n
Figure 1: The concept of ex-post evaluation of measures and instruments
As most important part of the operationalisation, the methodology provides methods for the acquisition of data necessary for the analysis of the indicators. Corresponding to the multiple indicator set, the methods also reflect different approaches to the generation of information including quantitative and qualitative approaches. A multitude of approaches is not only needed to enable data generation, but also to reflect needs of stakeholders and to enhance final applicability of derived information (cf. Stockmann 2004, p. 12). Provided methods always represent one possibility for analysis. In many cases different methods are applied in practice to describe the same issue. In this case only a general indication is given on the kind of method needed. The methodology does not attempt to present all potentially applicable methods. Where possible, the methodology seeks to include most accepted methods for each indicator.
This scope leads to the ex-post determination of intended and unintended effects of measures and instruments. In a further step, the methodology provides approaches for the evaluation of effects in relation to objectives and costs of the intervention. In relation with objectives, the effectiveness is calculated using evaluation results for intended effects. In relation with obtained benefits and the investment costs cost-effectiveness is calculated. Finally, the methodology also provides approaches for the discussion of robustness and flexibility of interventions with regard to contextual changes and other aspects.
2. Measures and instruments
2.1 Measures and instruments - interventions at project level
In the field of flood risk reduction, activities are realised at different levels. A basic differentiation exists between framework level and project level (cf. Mastop 2000). This differentiation is explicitly important for the evaluation, since also the effects of these activities can be regarded at different levels with consequences for the degree of detail, requiring other methodical approaches and evaluation designs.
The framework level is represented by strategies or programmes, as means of policies to provide the scope for subsequent action. Programmes can be seen as “frames of reference“ for action and are often the prime interest of evaluations (Mastop 2000). They usually integrate bundles of projects developed to contribute to the achievement of a programme’s goals. In flood risk management, programmes are represented by strategic plans for cooperation or action programs for certain rivers – both characterised by general goals for a wider area and aiming at the initiation of a certain course of action but not constituting such. Programmes need to be translated into single actions at project level before they lead to outcomes. As a result, their evaluation for outcomes in terms of risk reduction is only evaluable under consideration of activities they trigger and guide.
In contrast, project level activities are rather single, non-divisible interventions (cf. CEC 1997) characterised by their limitation in space and time and by the evolvement of clear effects (cf. Mastop 2000). Ossadnik (2003, p. 480f) defines a project as a well-defined effort, aiming at achieving specific goals under consideration of time limits and financial resp. cost oriented and capacity based restrictions. The project ends with the achievement of the defined goal, while premises are free for correction in the course of project development. Spatial limitations apply either to the physical extent of a scheme (dike, retention basin etc.) or to the area for which a specific information (e.g. flood zone regulation) is valid. Temporal limitation, here, can be understood in two ways. In a narrow sense, the temporal limitation is understood as the point of time where the measure or instrument of interest has fully materialised. In a broader sense, temporal limitation may be applied to the whole life time of a project, meaning that after its introduction a measure or instrument functions for a limited period time delivering the outcomes of interest for evaluation. During this life time, if necessary, the measure may receive further input in terms of operation and maintenance.
Essential for the differentiation is the understanding of the cause-effect relationship linking programmes and projects with their outcomes. Figure 2 explains the relationship of programmes and projects with respect to the generation of effects (outcomes). It founds on the understanding that each intervention into a system functions as an instance triggering specific processes which, in coincidence with individual conditions, lead to certain outcomes (cf. Pawson & Tilley 1997). The latter can be measured in terms of the number, type, scale, location and timing of development (cf. Vedung 1997 and others).
II
B A
Framework level goals providing the scope for action
Project level activities (measures and instruments)
Project level outcomes resulting the project’s materialisation
Framework level outcomes resulting from project level achievements
I
The main argument of the concept is that programme goals cannot be achieved directly. Instead, they are achieved through the implementation of single actions (e.g. measures and instruments) which induce certain changes. This understanding includes the assumption that the direct path from (A) (programme) to B (goal), does not exist. Instead, programme level goals are achieved by the realisation of projects, which translate the programme intentions into action (A→II). As a next step, these project level activities (measures and instruments) lead to project level results (I→II). Only after this, the cumulation of results from different single projects can contribute to the achievement of programme level goals (II→B).
This has a certain influence on the evaluation of measures and instruments for flood risk reduction. An evaluation of measures and instruments should aim at describing their outcomes following aspects individual to the interventions of interest. Given, that the actual implementation of goals of any level takes place at project level, also concrete „outcomes“ attributable to that interventions can only be achieved and evaluated at project level (cf. Klöti 1997, Virtanen & Uusikyla 2004). Therefore, the project level is the prime interest of the Methodology for ex-post evaluation of measures and instruments.
Thus, measures and instruments are understood as projects aiming at the reduction of flood risk or flood damage through planned impact on the source, pathway, receptor or consequence of a flood hazard (chapter 2.2). In detail, projects are defined by the following characteristics:
• Well defined by temporal and spatial limits • Limited and assignable costs
• Goals (at minimum those of flood risk reduction) • Measurable and assignable outcomes
In many cases, measures or instruments are implemented in compounds. As a result, also certain outcomes of those interventions can often not be allocated to single interventions. For example this can be the case with flood proofing measures at buildings. Often there is a combination of measures that finally all together lead to certain results. Therefore, in the evaluation, their interaction with the flood risk system needs to be seen as a compound. In order to give consideration to this, measures and instruments at project level can be:
1) Single measures or instruments at project level
2) Combinations of measures and/or instruments at project level (where single interventions are inseparably connected in terms of design or impact, chapter 2.4)
2.2 Measures and instruments in the flood risk system
As will be shown below, a large variety of measures and instruments exists to reduce flood risk respectively flood damage. Measures and instruments can be applied in different phases of flood risk management and aim at achieving various effects which influence flood risk (Penning-Rowsell & Peerbolte 1994, cf. e.g. Pottier 1998, LAWA 2000a, BTRE 2002, Hoojer et al. 2004). All these interventions develop their effects as a result of the interaction with the flood risk system. However, most measures and instruments are designed to impact a certain element of the latter by the mechanisms specific to each single type of intervention. For better understanding of the classification presented below (chapter 2.3), the relation of measures and instruments with the flood risk system shall be briefly described.
flood event with exposed vulnerable elements (= receptor, e.g. properties, population) certain losses (consequences) can occur. The latter can be understood as the actual ‘materialisation of risk’ (cf. a.o. HM Treasury 2004, p. 48).
Figure 3: Source-Pathway-Receptor-Consequence model (cf. DETR 2000)
The main advantage of this system lays in the simple description of the causal chain between source and consequences. It also summarises the basic elements of the flood risk system which can be approached in terms of flood risk and flood loss reduction. With its four elements, the model describes both components of risk: ‘hazard’ and ‘vulnerability’ with exposure seen as the linking circumstance, rather than a separate component (cf. FLOODsite Consortium 2005). Hazard is represented by sources describing the triggering event of a flood including conditions which affect the concentration of flood water (e.g. specifics of runoff) and pathways describing the conditions influencing the propagation of flood water along its route. Vulnerability is represented by the receptor with its attributes describing the actual degree of exposure and susceptibility of elements at risk to the flood and by the consequences which result from the interference of hazard and the receptor. Schanze (2004) proposes the description of flood risk as a function of the elements of the SPRC-model. This function is set up to describe the factors which make up these elements as well as to show the interfaces where flood risk can be influenced by human intervention. Within this function, flood risk reduction factors are highlighted, which can influence the conditions through intervention into the system (adapted from Schanze 2004):
flood risk = f ((p,m,t)source, (g,i,r)pathway, (v,s,e)receptor, (l,c)consequence )
with
p = probability (return period flood event) m = magnitude (amount of water)
t = retention (before concentration in rivers)
g = propagation (speed of downstream propagation) i = inundation (of area outside the discharge channel)
r = reduction/detention/protection (of flood water along the pathway)
v = value (of elements at risk exposed to flood hazard)
s = susceptibility (of elements at risk exposed to flood hazard)
e = exposure + resistance and resilience (of elements at risk)
l = losses (amount of incurred losses)
c = compensation (of incurred losses)
Thus, there are options for risk reduction at any point of the causal chain (SPRC) leading from the source to consequences. With regard to sources, this is basically the retention of water in the area of provenance by reducing its concentration through increasing the retaining capacity of the land surface including earth and vegetative cover. This option only applies to terrestrial surfaces, it can not be applied in cases where the source is the failure of a central technical device (retention dam) or if source of flooding is the sea. With regard to pathways, the main options lie in the - even temporal - reduction or delay of discharge, the detention of flood water in and along the discharge channel and the protection of parts of the flood plain against inundation. In case of coastal flooding, the options with regard to pathways are limited to certain forms of energy accommodation and mainly the protection of the hinterland against inundation.
2.3 Measures and instruments – a classification
2.3.1 Why a new classification
A classification in general donates order and lucidity to the large amount of existing measures and instruments. And usually, each classification reflects the perspective, from which interventions are seen. The past has seen manifold approaches dealing with the systematisation of approaches of what today we refer to as flood risk reduction. Many different classifications for flood risk reduction measures have been proposed in the past decades. The most often used classification of measures and instruments is based on the general distinction of ‘structural and non-structural measures’ (Smith & Tobin 1979, Penning-Rowsell & Peerbolte 1994, Marsalek et al. 2000, Kreibich et al. 2005). Furtherclassifications order by point of time including pre-flood, flood event and post-flood intervention, by character differentiating technical measures and regulatory, financial and communicative instruments (Hooijer et al. 2002) or by aim differentiating flood abatement, flood defence and flood impact mitigation (De Bruijn et al. 2003).
One of the most recent and most comprehensive of these was developed in the Foresight project differentiating “26 functional response groups” describing basic services, which they provide. Within these response groups “80 possible response measures, policies and interventions” are distinguished, which can contribute to the response groups (Evans et al. 2004b, pp. 25ff). However, none of these classifications provides a comprehensive and homologues classification of project level interventions describing their functioning with the flood risk system. However, such a classification is needed to describe the scope of interventions addressed by the methodology as well as for the operationalisation of the selection of evaluation criteria discussed in chapter 6.
Therefore, a new classification is proposed which tries to serve a number of requirements in order to be both consistent in itself and a basis for the methodology. This classification seeks to cope with following premises:
• The classification represents the full array of approaches to the reduction of flood risk before, during or after a flood event. In this scope, the balanced consideration of traditional as well as more recent approaches is a main challenge.
• The classification is based on the functional characteristics of interaction between intervention and the flood risk system.
• The classification considers that effects of interventions often go beyond the intended. Therefore, it avoids explicit and implicit reference to single effects in order to avoid prejudices regarding the full range of expectable impacts, which are issue of evaluation.
• The classification is applicable to different contexts within Europe and allows as far as possible an unambiguous attribution of measures and instruments to a category or sub-category by scientists and practitioners.
• The classification primarily addresses interventions at project level (clear temporal and spatial boundaries, +/- clearly assignable costs and outcomes).
The new classification is organised in three levels:
1. Differentiation of direct interventions (physical measures) and indirect interventions (policy instruments);
2. Description of the functional character of measures and instruments in their interaction with the flood risk system;
3. Description of types of measures and instruments differentiating the manner of intervention with the flood risk system within the functional groups.
non-structural measures is not followed, as the general understanding of these terms does not permit a consistent differentiation of direct and indirect a swell as traditional and recent approaches. Instead,, the terms ‘measures’ and ‘instruments’ mark the basic differentiation of interventions.
2.3.2 Level 1: Measures and instruments
At its first and most general level, the proposed classification is based on the differentiation of two general principles of intervention: direct interventions (physical measures) and indirect interventions (policy instruments).
Physical measures are direct interventions into the physical flood risk system, which generate physically tangible changes as their direct output. Examples are civil engineering works, flood proofing of buildings or cultivation techniques. The basic understanding of this group is widely consistent with the understanding shared by Marsalek (2000) and Petry (2002). Traditionally, physical measures primarily influence elements of the hydrological system (Amoros & Petts 1993, Pottier 1998) and are thus most prevalent in the field of hazard control addressing sources and pathways of flooding. However, physical measures are also applicable to the receptor (by evacuation of exposed population or flood proofing of buildings) and have certain importance for the mitigation of consequences (e.g. in course of post flood mitigation). Thus, physical measures can be applied at any point of the SPRC chain discussed above.
Measures are direct physical interventions into the physical flood risk system, which become manifest in the emerging or changing of technical and natural structures respectively of their patterns and properties as well as in the direct exposure of health and life to the flood hazard.
Complimentary to the widely accepted term measures, the term ‘instruments’ is applied to describe the indirect approaches of flood risk reduction. The term is not consistent with the often used term of non-structural measures. The term signals the principally different functioning of the concerned measures. Their essence is not the direct and tangible impact on the flood risk system, but the triggering of activities (incl. behavioural change) connected with flood risk. These interventions are thus rather tools that are used to achieve a certain goal that cannot be reasonably implemented directly. As synonym for ‘tool’ the term ‘instrument’ is widely used in European languages for policy action (policy instruments) based on economic options ('economic instruments', cf. OECD 1994, Tyagi & Saalmüller 2004), legal regulations (legal instruments, Pottier 1998 uses 'tools') and particularly in the field of spatial planning (Handmer 1996, ARL 1998, Morrison 2002, Hooijer et al. 2004). Also the EC communication (CEC 2004, p. 6) calls for “appropriate prevention instruments” to contribute to flood mitigation and the on proposed Floods Directive (CEC 2006) takes recourse to “financial instruments” as options of flood risk management.
Instruments are interventions, which influence the perception and behaviour of individuals and institutions, which in effect can also lead to the development of intended physical structures through inducing physical measures. Thus, the direct effect of policy instruments is the triggering of activities. The latter can be nearly obligatory (e.g. in case of binding land use regulations) or of a rather soft nature (in case of information instruments). Other instruments function by shaping vulnerability which does not necessarily materialise in physical changes (preparedness, risk perception etc.). While various types of vulnerability are being distinguished (Parker et al. 1997, Messner & Meyer 2006), most relevant for current flood risk management appear the social and partly the economic vulnerability.
The latter has a special importance for the evaluation of such instruments. In cases, where an instrument mainly influences the realisation of physical measures, the ex-post evaluation needs to take into account also its physical manifestation.
2.3.3 Level 2: Functional character of measures and instruments
Measures and instruments can be characterised by manifold aspects. However, many of them are do not necessarily allow an unambiguous allocation of interventions. For example, certain measures and instruments can be applied in different phases of flood risk management (e.g. pre-flood or flood-event) and by different stakeholders (public authorities, private). Other aspects, such as the permanence of intervention, offer rather general distinctions which are not sufficient for the set up of a classification. Therefore, the functional character of the intervention is used in order to provide an unambiguous classification. It describes at a general level the interaction of the intervention with either the physical or the socio-cultural flood risk system. The functional groups consider the fundamental functional differences between physical measures and policy instruments (Figure 4).
As functional groups for the description of interaction with the physical flood risk system three categories are defined:
- Adaptation of existing non-built physical conditions by special management techniques
- Control of physical conditions in the flood risk system by introduction and adaptation of built structures
- Retreat of uses by temporal evacuation or permanent relocation of elements at risk
The three functional groups represent the three principles of interaction with the physical flood risk system. The first physical option is the adaptation of existing natural conditions, mainly those of the source and along pathways - e.g. by periodical application or adaptation of specific treatment techniques. The second is the structural control by man made conditions through the introduction and adaptation of built structures mainly with regard to the source, pathway and receptor - e.g. by controlling processes of runoff or reducing exposure and susceptibility of receptors. Finally, a third option is the retreat of elements reducing the exposure of the receptor by temporary evacuation or permanent retreat of uses.
Policy instruments, in contrast, have no possibility to directly influence the physical flood risk system. Their functioning is based on the triggering of mechanisms in the socio-cultural flood risk system, by influencing the action of stakeholders which can lead to changes also in the physical flood risk system. However, the effect of policy instruments can also be restricted to social factors such as perception, awareness or preparedness, which need not find manifestation in physical changes.
Four functional groups are identified describing the interaction of policy instruments with the socio-cultural flood risk system:
- Regulation of (mainly spatial) development based on legal backing
- Stimulation of development and behaviour by positive and negative incentives
- Communication of stakeholders for the benefit of perception, awareness, preparedness and behaviour - Compensation of consequences through distribution of flood risk and losses
thus appealing to the individual interests and responsibility. Finally, accepting that often losses cannot be fully avoided compensation of losses induced by flooding is an option to mitigate occurred harm and to foster recovery by distributing the costs over the wider community.
Regulation, Stimulation and Information instruments are applicable with the source, along pathways and with receptors and can often also impact the development of physical conditions. Furthermore, the provision of information can influence social factors like preparedness and perception, which are important for the implementation of other interventions as well as for the acceptability of risk. The application of compensation instruments is restricted to the Consequence-part of the SPRC-chain.
Measures and Instruments for flood risk reduction (pre-flood and flood-event)
Stimulation
instruments Information instruments Regulation
instruments
PHYSICAL MEASURES POLICY INSTRUMENTS
Control measures Adaptation
measures measuresRetreat Compensation instruments
Figure 4: Categories of measures and instruments
PHYSICAL MEASURES Adaptation measures
Adaptation measures are direct physical interventions into natural structures, the effect of which emanates from the application of specialised treatment techniques on land surfaces as well as in and along river channels (SPRC).
Control measures
Control measures are direct physical interventions, the effect of which emanates from introduction, alteration or removal of built structures (SPRC).
Retreat measures
Retreat measures are direct physical interventions, the effect of which emanates from temporal evacuation or permanent retreat of elements at risk from the flooded or flood prone area (SPRC).
POLICY INSTRUMENTS Regulation instruments
Regulation instruments are policy interventions, the effect of which emanates from prescriptive regulations and the legally backed enforcement power of administrations, mainly restricting the type and quality of land uses. Regulations are mainly tools of sectoral policies (SPRC).
Stimulation instruments
Stimulation instruments are policy interventions, the effect of which emanates from financial incentives and disincentives offered to private or institutional stakeholders in response to certain behaviour or action (SPRC).
Communication instruments
Compensation instruments
Compensation instruments are policy interventions, the effect of which emanates from mechanisms distributing the risk respectively occurred losses over a wider community (SPRC).
2.3.4 Level 3: Types of measures and instruments
Types of measures and instruments further differentiate functional groups defined above (Figure 4). Types of measures and instruments are based on the manner in which the intervention interacts with the flood risk system. The driving question behind the categories is: “What is being done?” (Figure 5).
Measures and Instruments for flood risk reduction
Stimulation instruments Communication instruments Fina nc ia l inc ent iv es R isk a n d l o ss d ist ri b u ti o n Regulation instruments C o mm uni c a ti o n /D is s e m ina tio n W a rn in g /I n st ru ct io n W a te r man age m e nt Env iron m ent al p rot e c ti on Fina nc ia l dis in c e n ti v e s
PHYSICAL MEASURES POLICY INSTRUMENTS
S pat ia l p lan nin g Ri v e r c h a n n e l an d c o as ta l m ana g e m e nt F lo o d pr oof in g o f b u ild ings an d te c h n ic a l infr ast ructu re L and ma nag em en t Control measures Adaptation measures Retreat measures E v ac ua ti on of h u m an lif e E v ac uat ion of a s s e ts an d li fe s toc k Compensation instruments Dr aina ge a nd pum pin g s y s tem s Ch ann e l c onv ey an c e and c a p a c ity Floo d s torag e Floo d w a te r tr an s fer C oas ta l a lig nm ent C oas ta l e n e rgy a c c o m m od ati o n R e tr e a t of us es
Figure 5: Types of measures and instruments
In the following, types of measures and instruments are defined. Appendix 5 classifies all identified measures and instruments following the presented system.
MEASURES
Adaptation measures
Land management (SPRC)
Physical changes of patterns and properties of soil or vegetative cover by means of cultivation techniques and the choice of crop.
Examples Conservation tillage, Transformation of forests Channel and coastal management (SPRC)
Removal, displacement, introduction of physical features incl. bed sediments, vegetation etc. in rivers, along river banks, in estuaries and along coasts.
Example Dredging Control measures
Flood storage (SPRC)
Hydraulic engineering structures in river channels, valleys and floodplains retaining surface discharge.
Flood water transfer (SPRC)
Hydraulic engineering structures conveying parts of the discharge into other catchments. Example Flood water diversions
Channel capacity and conveyance (SPRC)
Hydraulic engineering structures in river channels, valleys, floodplains and estuaries influencing the shape and capacity of the river channel.
Examples Channelisation, Dikes, Channel restoration Coastal and estuarine defences (SPRC)
Physical barriers to coastal floods along the coastline. Examples Dike, Flood Wall, Managed or unmanaged retreat Coastal energy accommodation (SPRC)
Physical elements introduced along the coastline to control the energy of currents or waves. Examples Offshore barriers, Beach nourishment
Drainage and pumping systems (SPRC)
Drainage and pumping systems conveying waste, ground and flood water from vulnerable areas or pumping excess water into storage areas or into the river.
Examples Central pumping devices (in urban areas), Local ground water lowering Flood proofing of buildings and infrastructure (SPRC)
Additions, changes and adjustments at buildings and technical infrastructure following two main approaches:
Dry flood proofing by protection against the ingress of water
Wet flood proofing by provisions in structure and installations against damage by water Examples(dry) Shielding with mobile/temporary barriers, Elevated construction
Examples(wet) Waterproof construction material, elevation of installations Retreat measures
Evacuation of human life (SPRC)
Temporary or permanent relocation of human life or life stock from of flood-prone or flooded areas.
Examples Evacuation of especially vulnerable population Evacuation of assets (SPRC)
Temporary evacuation of assets (incl. life stock) from flood-prone or flooded areas. Examples Relocation of physical inventory at risk
Change of interior uses in lower levels of frequently flooded buildings Retreat of uses (SPRC)
Permanent relocation or give up of sensitive uses from flood-prone areas.
INSTRUMENTS
Regulation instruments
Water management designations (SPRC)
Land use regulations backed by the enforcement power of water management related legislature. Examples Flood zones
Prohibition to store harmful substances (e.g. chemicals) in floodplains Regulations for ‘Flood source areas’ (D, Saxony)
Spatial planning designations (SPRC)
Land use regulations backed by the enforcement power of spatial planning legislature. Example Priority areas for certain uses
Environmental protection designations (SPRC)
Land use relevant regulations backed by the enforcement power of environmental protection related legislature. Here, flood risk reduction is rather a side-effect of regulations addressing nature conservation.
Examples Landscape Protection Area, Nature reserve, Nature 2000 area Stimulation instruments
Financial incentives (SPRC)
Financial incentives encouraging activities which lead to the development of intended physical patterns/properties or behaviour.
Example Subsidies and/or allowances for flood-proof construction of buildings Financial disincentives (SPRC)
Financial disincentives discouraging activities which lead to the development of unwanted physical patterns/properties or behaviour.
Examples Levies/ imposts, Fines, Taxes Information instruments
Information/Dissemination (SPRC)
Transfer of knowledge to and/or between stakeholders of flood risk management and the public using various communication media.
Examples Information events, Brochures, Guidelines Warning/Instruction (SPRC)
Operational information of stakeholders and public about an upcoming/ongoing flood event incl. directives for action and behaviour.
Example Official flood warning Compensation instruments
Risk and loss distribution (SPRC)
Risk and loss distribution is realised by allocation of risk or the burden of losses on a larger community by public policy or private-sector initiative.
Examples Flood insurance payments, Public relief (compensation payments) after flood event
2.4 Single measures and instruments and combinations of those
of dike, flood walls and mobile defences can be applied to protect the flood plain. Flood proofing by shielding of single buildings may not provide sufficient risk reduction thus as further proofing measures may be taken which can additionally be complemented by evacuation measures. Or, regulation instruments may need support of financial stimulation in order to ensure implementation. In the end, the final outcome of risk reduction often cannot clearly be attributed to one single member of the implemented portfolio. Ex-post evaluation then needs to address a sensible combination of interventions.
In order to decide whether a single measure or a combination of several measures are the object of evaluation, three main questions need to be answered.
1. Are measurable outcomes generated by more than one intervention? If yes, can proportions of outcomes be certainly attributed to one intervention?
If the answer to both these question is no, than a combination of measures or instruments is the object of ex-post evaluation.
Example: If a protection line consists of different sections including a dike, flood walls and mobile defences, protecting together the same area, than the impact of neither of the measures can be evaluated without the other.
2. Does the intervention directly provide outcomes, or is it rather a trigger which requires subsequent action in order to achieve risk reduction?
This question mainly applies to instruments and implies that many instruments are triggers of action, which actually leads to risk reduction. Thus, if the answer is that risk reduction is actually a result of action triggered by the instrument, than the instrument’s impact on flood risk cannot be evaluated without taking account of the outcomes of the subsequent interventions.
Example: Instruments applied in the filed of risk reduction usually trigger subsequent action and by doing so contribute to risk reduction indirectly, being indispensable for the direct measures. As a result, an instrument “Contingent Plan” will remain worthless if it is not implemented by the foreseen contingent measures during a flood event. In order to describe the risk reduction effect of a contingency plan, the evaluation of the contingency action, which the plan guided cannot be omitted. 3. Does the intervention of interest provide its outcomes independently or does its performance rely on external input?
This question applies to most measures which are applied only during flood events and those which are implemented by non professionals (e.g. many flood proofing measures). Its aim is to explore whether all preconditions are fulfilled which are essential for the good performance of interventions. This is based on the assumption, that ex-post evaluation will provide valuable conclusions about measures only if these had the chance to be implemented in time and correctly. Thus, if the answer leads to the conclusion, that external input (e.g. trigger) is necessary, evaluation should consider the timeliness and quality of this input before conclusions can be made about the actually evaluated intervention(s).
3. Conditions for measures and instruments
3.1 Importance of conditions
Measures and instruments always act as part of a system. Their implementation, operation and performance is always influenced by pre-existing (and changing) external natural and socio-cultural conditions defined by the system in which they are introduced. Above this, measures and instruments have own, internal, conditions which are also decisive for their performance and which can be influenced by external factors. While internal conditions are descriptive for the state of a measure or instrument at a certain point of time, external condition describe the scope, under which the intervention functions. Internal and external conditions are usually seen as variables which remain more or less stabile over a certain period of time (Hutter 2005). However, also the dynamic of the system which an intervention is part of can be an important aspect for its performance.
Internal conditions influence the functioning of measures and instruments from within. This may include factors such as the structural quality of a measure, but also the change of its constitution during its life cycle. Another factor is the political (at a rather operational level, e.g. extent and duration of political support) or cultural (e.g. planning culture, obedience) background of instruments or the availability of resources for the continuous support of instruments. May and Burby (1996, referred to in Parker et al. 1997, p. 30) identified institutional ‘capacity’ and ‘commitment’ (cf. also Pottier 1998) of public authorities as two fundamental internal factors determining the effectiveness of policy implementation: Where the capacity and commitment is low, policy implementation tends to be weak.
External conditions represent the natural (physical, ecological, e.g. flood event, physiography, land uses), political, social, economical (e.g. growth or shrinking, real estate market development) factors with which interventions interfere. According to Hutter (2005) external conditions are understood as more or less static boundary conditions under which a measure or instrument is introduced and operated. However, especially the magnitude of flood events is a highly variable external factor against which ex-post evaluation is realised.
Conditions influence the appearance and the functioning of each intervention. Appearance describes the substance in terms of its actual shape (e.g. technical design of a dike), its information content (e.g. of a land use regulation), or its organisation and capacity (e.g. evacuation, communication). The functioning refers to the conversion of the substance of an intervention into outcomes. Therefore, the knowledge of conditions is important for constructing the hypotheses of the cause-effect relationship upon which evaluation is based and also to allow for the analysis and later comparison of results (chapter 2.1).
The methodology addresses impacts of measures and instruments on the systems with which they interact. With regard to intended effects this will be usually the flood risk system with its physical and socio-cultural components. However, with regard to unintended effects, often other corresponding systems may be affected, such as the ecosystem, or the general socio-cultural system. These also contribute to the flood risk system, but go beyond this.
As stated above, the primary interest of measures and instruments is usually restricted to intended effects in the field of flood risk reduction. Therefore, conditions describing the elements of flood risk concept including hazard and vulnerability are most relevant for the work with measures and instruments of flood risk reduction (cf. also De Bruijn 2005). Three factors are chosen to describe the hazard with which a measure or instrument interacts: The genesis and attributes of flood events represented by the flood type, their probability by the recurrence period. The prevailing land use describes the general conditions of the affected socio-cultural system in the affected area. Finally, also the type of water body is considered. This condition addresses specialised issues of water bodies and is particularly important for the selection of ecological criteria.
In total four main conditions are mandatory for definition of the case: • Type of flood event
• Probability of event • Type of land use • Type of water body
As mentioned above, further specific conditions should be considered by individual cases.
3.2 Type of flood
Flooding can evolve from different sources and occurs in different landscapes. Inland floods may be generated by heavy precipitation events, sudden snow melt, combinations of both or from technical failure of a retaining structure (river floods discussed e.g. by Merz 2002). Coastal floods arise from combinations of natural conditions at the sea involving high tide and landward wind storm. Through the conditions of genesis (source) and the physiography along the pathways different types of floods are characterised by specific dynamics. Fife types of floods are distinguished including the special case of urban flooding:
• Flash floods • Plain floods • Estuarine floods • Coastal floods • Urban floods
Flash floods are induced by heavy, often local, precipitation or sudden snow melt or combination of
both. It is characterised by fast onset (minutes to hours after a causative event), a heavy multiplication of discharge and high flow velocities and is often accompanied by large sediment and debris flow.
Plain floods are induced by large scale precipitation or snow melt. In contrast to flash floods, slow
rise floods or reverine floods are characterised by relatively slow onset (up to several days), a less severe increase of discharge, increased but still moderate flow velocities and moderate sediment and debris transport.
Estuarine floods are induced either by a costal flood whereby flood water is driven into the estuary or
Coastal floods or storm surges are induced by the combination of temporary sea level rise resulting
from extreme meteorological conditions and especially severe in coincidence with high-tide and is characterised by pronounced wave swell (Smith & Ward 1998, p. 148, Evans et al. 2004a, p. 264).
Urban floods or intra-urban floods are composed of storm water runoff induced by heavy
precipitation on urban surfaces, and includes surface flood waves, internal flooding of areas and properties through overloaded sewer systems as well as through backing up of culverted and other water courses (cf. Evans et al. 2004a, pp 13, 138). Urban floods are characterised by fast onset, high dynamics of discharge and often uncertain pathways of the flood water.
3.3 Probability of flood
The probability is a measure for the recurrence period of an event with certain magnitude. It delivers information about the severity of a flood event and is thus important for the valuation of the performance of measures or instruments. No classification is proposed for this condition.
3.4 Land
use
As emphasised above, the land uses affected by flooding or flood risk reduction can be decisive for the types and amount of losses. This general societal condition is described by the type of land use such as developed and undeveloped areas. The two categories describe the type of assets which are exposed to the hazard respectively to the risk reduction intervention. It is assumed that, compared to developed areas, in non-built up areas generally different types of effects are generated and potential for losses lower. Therefore, only these to two categories are distinguished:
• Developed (urban/industrial/rural buildings or urban land uses affected) • Undeveloped (no buildings, only rural land uses)
3.5 Type of water body
The type of water body is already partly considered by flood types. But, there are specific needs for the additional consideration of water bodies. Floods as well as flood risk reduction are often both closely related to certain water bodies. On the one hand, the ecological aspects of risk reduction requires due consideration of differences between water bodies. This allows the a priory exclusion of certain effects for certain types of intervention. On the other hand, certain interventions such as many flood proofing measures are only related to single buildings, but have no effect on the water body (this does not apply if these measures address exposure of hazardous substances). The inclusion of these relations enables the targeted consideration of potential side effects. For this reason, types of water bodies defined by the European Water Framework Directive are used for the specification of ecological factors touched:
• Rivers • Lakes
• Transitional waters • Coastal waters
Rivers are bodies of inland water flowing for the most part on the surface of the land but which may
flow underground for part of its course (Article 2(4)).
Transitional waters are bodies of surface water in the vicinity of river mouths which are partly saline
in character as a result of their proximity to coastal waters but which are substantially influenced by freshwater flows (Article 2(6)).
Coastal waters are waters on the landward side of a line, every point of which is at a distance of one
nautical mile on the seaward side from the nearest point of the baseline from which the breadth of territorial waters is measured, extending where appropriate up to the outer limit of transitional waters (Article 2(7)).
The relevance of an indicator for a water body is determined by the instruction given by the Water Framework Directive (Annex V). A table, allocating the limnological indicators to these types of water bodies is given in chapter 5.1.6.
3.6 Specific
conditions
The introduced conditions are considered to be the most important ones. However, these can be only a selection. Each case is specific and as such is defined by a multitude specific conditions which are important for understanding the effect spectrum as well as the intended performance of the intervention. A few of those shell be named here, the relevance of which should also be verified in all cases. These specific conditions include:
• Potential exposure of hazardous substances • Presence natural heritage sites
• Presence cultural heritage sites • Political support
• Quality of maintenance
4. Evaluation framework
4.1 Criteria – main viewpoints of evaluation
Within this methodology, two terms are distinguished when speaking of evaluation: criteria and indicators. On the one hand, criteria represent the basic viewpoints of evaluation. They mark the five basic aspects outcomes in general (effects), outcomes related to objectives (effectiveness), outcomes related to costs (cost-effectiveness), the reliability of intended outcomes (robustness) and the adaptability of interventions (flexibility). ON the other hand, indicators are defined mainly asspecific aspects of effects (chapter 4.2). In the following, the five categories of criteria are defined:
• Effects • Effectiveness • Cost-effectiveness • Robustness • Flexibility
Measures and instruments are characterised by different spectra of impact on the systems into which they intervene. These impacts are referred to as effects which are attributable to an intervention. Effects are understood as outputs and outcomes of measures and instruments. Basis for this definition is the result-chain-thinking presented in chapter 2.1. Two main types of outcomes are differentiated. Many effects correspond with intentions connected with an intervention (intended effects). However, effects can also occur outside the intended impact spectrum (unintended/side effects). Both types of effects are important to value the merits of an intervention as only the complete picture delivers sufficient information for future action:
The determination of effects is the basis for comprehensive evaluation of measures and instruments. They are essential for the determination of effectiveness, cost-effectiveness and robustness. Therefore, the methodology bases on effects as the basic unit of evaluation and uses these for the valuation in the other categories.
Effectiveness describes the extent to which the objectives of an intervention are achieved (OECD 2002, p. 20f, CEC 2003, p. 45). It is based on intended effects and is dependent on the availability of objectives which can either be formulated in course of project planning or which need to be interpreted from the context of the intervention. Being also the traditional focus of evaluation, effectiveness analysis constitutes a core element of post evaluation. However, the often found restriction of ex-post investigations to effectiveness only (Thompson et al. 1991, Subiras 1995) does not provide comprehensive evaluation.
Based on intended effects, the effectiveness of an intervention can be determined by relating those to certain objectives (chapter 5.2.3). Unintended effects might also become matter of effectiveness evaluation and could be valued against existing but not explicitly mentioned societal values. For example, for many ecological criteria related to water bodies, targets are defined and claimed by the Water Framework Directive entailing that flood risk reduction is expected to be achieved without negative impacts on the state of the water body respectively without endangering the later achievement of the required “good ecological status” or “good ecological potential” (WFD, Article 4).
