«Commissioned by: Swedish International Development Cooperation Agency Austrian Development Agency Belgian Federal Public Service Foreign Affairs, ...»
evaluation of coordination and
complementarity of european
assistance to local development
Swedish International Development Cooperation Agency
Austrian Development Agency
Belgian Federal Public Service Foreign Affairs, Foreign Trade and
Development Cooperation – Directorate General for Development Cooperation
French Ministry of Foreign Affairs
The Development Cooperation of the Netherlands
Evaluation of coordination and complementarity of European assistance to local development Paul Beaulieu and Robert N. LeBlanc Studies in European Development Co-operation Evaluation No 4 European Union November 2006 2
Table of Contents:
Annex 1: Detailed Methodology and Survey 4 Instruments Annex 2: Team Members Involved in this Mandate 21 Annex 3: Supplementary Information and Data on the 23 Execution of the Project Annex 4: Country Case Studies 41 1 Mozambique 42 2 South Africa 64 3 Indonesia 92 4 Nicaragua 126 3 Annex 1: Detailed methodology and survey instruments This annex describes the main elements of the methodology used in the mandate.
1. Local development and its players Stated briefly, the goal of this mandate was to characterize, evaluate and propose improvements for EU donors’ coordination and complementarity management in the domain of local development. Documentation clearly shows that there are many different and sometimes radically opposed views on what local development is and its role in a country’s overall efforts to eradicate poverty. A shared understanding of the domain of local development is therefore not only relevant but essential to ensure a common understanding of the results of the research. Assessing the pertinence of local development as a “development” strategy was not part of this mandate, nor was evaluating the performance of local development projects. Our interest was directed more at the specification of the “perimeter” of activities in that domain and, therefore, the methodology used required that we discuss with others the types of developmental activities that were part of our evaluation.
The segments of the domain of local development on which the evaluation focused
Key basic human needs (health and education)
These segments of activity were selected because they cover most of the ODA activities usually deployed in relation to the MDGs by the donor community as a whole for local development. They also represent the majority of investments in local service delivery worldwide.
We defined local administration as the governing bodies (and their related administrative units) dedicated to the provision of public services. These administrations could be the central government or an organizational unit created as part of a process of decentralized governance by a State. Local governance systems were defined as including the actions of local officials as well as those of civil society partners in a given local area or territory. This means that when talking about local governance and/or local capacity to manage development, we included dynamics and the potential inclusion of a range of NGO partners (including international or local non-governmental organizations and various forms of community groups) in these development efforts (e.g., the provision of all forms of service, from organization to delivery). This position was relevant for the establishment and description of local systems of actors and stakeholders to map their relations and transactions associated with coordination and complementarity management.
For the purpose of this mandate, local administration thus refers to any public administration entity that is part of a subdivision of a state. Each country has its own 4 term for the administrative territory covered (province, region, district, etc.) but in the end, “local administration” refers to the closest administrative bodies related to populations and territories. In this sense, recipient partners of ODA efforts in local development included official governing and administrative bodies as well as local stakeholders providing services to local populations.
Complicating the model described above is the fact that several aid agencies and international NGOs do not provide all of their aid to local agencies through national organizations, but rather provide direct assistance to local administrations and nongovernmental actors using various instruments and approaches. There are many reasons for this choice of development strategy but it is clear that it is partly due to the perceived weakness of and frustration with some national and local administrations. The evaluation sought to describe the mechanisms involved in this choice of strategy.
In summary and for the purposes of this mandate, we considered that local development is about the active participation and/or leadership of local people and organizations toward the achievement of sustainable and inclusive economic growth and social development for all in a given area or “locality.” It also includes the provision of local services to that same target group. This is in line with the overall conclusions of the international conference on local development held in Washington in 2004.
2. Typology of partnership contexts It cannot be assumed that the sociopolitical (or any other) context of ODA activities is homogeneous around the world. Every country is different and the ways in which donors interact or network to deal with the specific challenges they face are also different. It would therefore be a logical error to try to employ a general and uniform approach to assess the performance of donor countries and their partnership relations with recipient countries and regions. Seeking to meet the highest standards of quality in the methodological design of the present evaluation, it was proposed—and accepted—that the dynamics of the development of these partnerships vary based on the type of ODA context in which the partnership is developed.
Because this evaluation required that operational recommendations for the improvement of 2C practices be presented, it was logical that these recommendations be relevant and adapted to the main types of ODA contexts. “Context types” specific to this evaluation therefore needed to be defined.
In order to ensure methodological rigour in the selection of case regions and countries where the fieldwork for this evaluation would take place, a recognized research-quality evaluation model and approach were required. The Evaluation Team developed an approach to case studies based on the work of Robert K. YIN (see Case Study Research, Applied Social Research Methods Series, Sage, 2003) and Colin Robson (Real World Research, Blackwell Publishers, 2002). The data gathered in the evaluation were analysed in accordance with the protocols established by Mathew B. Miles and A. Michael Huberman (Qualitative Research Analysis, Sage, 1994).
2.1 The case research design The issue of ODA coordination and complementarity can be understood to be driven by two contextual factors: a) the level of complexity of the inter-donor network and b) the local capacity of the recipient country’s organizations and agencies to manage 5 development. These factors, illustrated below, are the two fundamental dimensions of any interface within a transactional context. At minimum, they represent the supply and demand dimensions of complexity and capacity. Perhaps more importantly, they define the need and motivation for creating and managing partnerships. This choice of axes was validated by key participants in all the countries and by ODA managers and experts around the world. An analysis of the results obtained also proved its validity.
As a tool for a preliminary assessment of each factor, a composite list of quantitative and qualitative elements needed to be identified. In order to classify a wide selection of recipient countries in relation to the ODA partnership context described above, the following list of elements was retained. Taken as a whole, they were instrumental in
determining the positioning (or “slot”) of a given country in the resulting matrix:
Complexity of the inter-donor network:
• Number of MSs working in the country
• Percentage of total MS ODA in a country
• Percentage of total MS ODA in a country spent on a specific sector in local development
• Percentage of MS and EU budget in a country compared to total ODA in the country
• Number of sectors (at local dev. level) covered in a country
• Percentage of and value of SWAPs and other PBAs in total MS and EU ODA
• Percentage of total assistance given by NGOs
• Number or importance of projects dealing with governance and/or democracy Capacity of local development organizations and agencies of the recipient
• Formal decentralized processes and decision making
• Low level of corruption
• Years of dealing with a number of MSs in same country
• Evidence of experience with SWAPs or other PBAs
• Recognition of being a good development partner
• Levels of need being addressed by ODA This means that the context of an ODA partnership could be characterized by an assessment of the interface point for the two classification factors, at least for the purposes of this evaluation.
Any recipient country can be positioned in one of the four “context types” with varying degrees of rigour based on the statistical objectivity of the research technique used (in this mandate, our individual estimates and a quasi-Delphi validation of the consensus of a group of experts in ODA management were used to provide the positioning). For the purposes of this evaluation, once countries were positioned in the classification matrix (four ODA context types), we needed to select as a potential country for our evaluation fieldwork those that were positioned in the mature stage of the partnership development curve for each quadrant. So, in each of the four parts of the classification matrix, countries positioned in the mature stage “zone” retained our attention as a priority for potential fieldwork. The reason for this choice of “zone” is that these countries represent the partnership configuration typical of a mature stage of the relationship in that quadrant.
6 Certainly, many other factors are involved in ODA partnership context characterization.
These would include, but are not limited to, the following: a) the politico-economic context; b) the scope of development needs addressed; and c) the history of the donorrecipient relationship. In our opinion, however, and based on preliminary research, the most prevalent effects are influenced by supply and demand capacities and the complexity of the system under management. That is why it was more useful to classify the potential ODA contexts in terms of the two previously identified axes.
Management literature in general and ODA management literature specifically are essentially silent on the application of contexts to ODA evaluations, but we consulted a large number of experts and renowned ODA managers on the international scene, and the approach used for the specification of the ODA context types was judged to be highly relevant. It was also said to have a high probability for being useful as a management tool. Certainly, further research and extensive case studies would be required if it is to be used as a predictive model.
With a qualitative scale for the assessment of each dimension going from a low level to
a maximum, an analysis enabled us to identify four types of partnerships:
I. Emergent partnership: A partnership where the definition of the working agenda and the relations process (specifically on 2C) are in the initial stages.
II. Donor-push partnership: A partnership characterized by a relatively strong presence and leadership on the part of the donor network.
III. Mature partnership: A partnership where 2C relations are well developed and where partnership processes and agendas are effective and efficient in the long term. They should also be sustainable.
IV. Recipient-pull partnership: A partnership in which a strong local capacity for ODA management and leadership is deployed to influence the 2C agenda and processes.
7 Based on our experience and a detailed literature search on institutional and organizational learning, we predicted that in each of the contexts, the maturity of ODA partnerships would evolve and develop according to a pattern similar to an “S”-type learning curve. In keeping with that notion, the evolution of the building process for ODA partnerships was believed to typically include four stages of maturity
• Minimal: Informal relationships exist but are not based on shared risks or decisions.
Institutions are networked for the sharing of status information. No shared agendas or development objectives.
• Non-systemic: More formal relationships exist but are consultative and decisions are not binding. Networks are informal and unstructured. Optimization and rationalization are not yet established as management objectives.
• Professionalized: Networks are formal and participants have common objectives.
Appropriate systems are mostly in place and resourced. Joint decision making is normal.
• Sustainable: A collective form of cooperation, based on common targets with individual contributions is the norm. The network is enlarged and open.
(See diagram entitled “Development Life Cycle of 2C Development and Effectiveness” below) These assumptions gave rise to our hypothesis that within a given context, all configurations of partnership develop following a relatively predictable trajectory and can be evaluated in terms of performance and maturity. In order to explain the patterns of maturity development for this mandate, research in the field of organizational design, and particularly in the management of complex networks and organizational development, was extensively used.