Synthesis Report – Ethiopia and Ghana Country Program Cluster Evaluation

List of Acronyms and Abbreviations

Executive Summary

Introduction

The Ethiopia and Ghana Country Program Cluster Evaluation was conducted between December 2013 and December 2014. It assessed the performance of the Department of Foreign Affairs, Trade and Development (DFATD) in providing development assistance to Ethiopia and Ghana during fiscal years 2008/09 - 2013/2014, with a particular focus on food security.

The objective of the evaluation was to assess the performance of the Ethiopia and Ghana Country Programs against the standard evaluation criteria of relevance, effectiveness, efficiency, and sustainability, as well as cross-cutting themes of gender equality, environmental sustainability and governance. The implementation of DFATD’s food security strategy in the two countries was also examined.

Approach and Methodology

The evaluation assessed the extent to which the Country Programs (including initiatives from the three DFATD Branches) attained planned results as specified in the goals and objectives and/or stated outcomes in the respective program and project logic models and performance measurement frameworks (PMFs).
Mixed methods were used, including document and literature review (245), key informant interviews (212), focus groups and case studies or site visits (9 districts in Ethiopia and 6 districts in Ghana). This approach allowed the team to gather and triangulate multiple lines of evidence. The evaluation sampled 40 projects: 20 in the Ethiopia Program ($344.32 million representing 54.3% of total disbursements, and 82.3% of those for food security); and 20 in the Ghana Program ($223.87 million representing 44.8% of total disbursements and 92.5% of those for food security).

Ethiopia - Findings and Conclusions

Relevance

Overall, the Ethiopia Program was relevant and well aligned to Government of Ethiopia (GoE) and DFATD priorities and commitments. The DFATD Country Program supported GoE flagship programs directly targeted at reducing hunger and extreme poverty, while others focused on improved agricultural productivity and practices.

Canada has met G8 commitments in Ethiopia through the L’Aquila Food Security Initiative, New Alliance for Food Security and Nutrition, Scaling-Up Nutrition, Food Assistance Convention and the Muskoka Initiative, and more recent projects have been consistent with DFATD’s Food Security Strategy.

Effectiveness

Approximately half of the funding ($176 million) during the evaluation period supported one of the GoE’s flagship programs, the Productive Safety Net Program (PSNP). Independent studies have confirmed a consistent and significant decrease in the food gap in the targeted PSNP districts and that approximately 2.5 million people were able to graduate out of PNSP since 2005. Canada made a significant contribution to these results.

Canada supported the GoE’s focus on improving nutrition that led to improved nutritional status in 100 food insecure districts, and increased school enrolment, with the provision of meals and take-home rations for girl students.

The Program also supported initiatives that contributed to increased productivity and incomes in sustainable agricultural development, including the rehabilitation of over one million hectares of degraded land, training of 58,000 farmers in new methods (43,000 focused on improved crop production, 12,000 on livestock and 3,000 on conservation), the creation of 1,500 farmer innovation groups and 263 primary cooperatives, and initiation of 125 irrigation schemes.

Funding intended to strengthen Ethiopia’s agricultural research capacity was provided, but outcomes were difficult to discern at the time of the evaluation.

The Ethiopia Program has been active and effective in policy dialogue through the Development Action Group (high level), donor coordination groups and project level working groups. Canada currently chairs the Rural Economic Development and Food Security Working Group and is active in the PSNP donor coordination committee and the Agricultural Growth Program (AGP) working group among others.

Sustainability

Donors have been committed to supporting the national flagship programs in Ethiopia that feature multi-donor pooled funding with strong national government leadership. The GoE, in turn, has committed to donor-supported projects that are consistent with its national planning priorities.

In addition to alignment with government priorities, sustainability of the initiatives supported by the Ethiopia Program has also been enhanced by extensive support for capacity building targeting not only service delivery agents, but also end users, including farmers, Development Officers of the Ministry of Agriculture (MoA), and local government officials. Success was, however, diminished to some extent by the high turnover rate of government staff (a common phenomenon in many developing countries).

Cross-cutting Themes

Canada is seen as a strong advocate for gender equality among donors, particularly evident in the coordination/working groups of donor-supported GoE programs. Projects sampled from the Ethiopia Program  showed proactive integration of gender equality in planning, however in some cases, the results achieved were mixed.

Environmental sustainability was well integrated into design and implementation in many of the sampled projects, including the integration of improved land and water management. A significant component of the large Productive Safety Net Program incorporated environmental sustainability – for example, rehabilitation of degraded lands, increased vegetative cover, embankments, reduced soil erosion and improved water management.

Governance was not an explicit aspect of the projects that were sampled in Ethiopia. There was little evidence that the Ethiopia Program had either assessed its projects against governance indicators or planned specific interventions to improve governance, with the exception of capacity building support. It should be recognised however that governance was not established as a cross-cutting theme until later in the evaluation period.

The two GoE flagship programs – PSNP and AGP – have extensive components that deal with improved program governance, including improved systems and structures for assessing community needs and responding to these needs, budgetary management, audit structures, and the extensive use of working groups at the national, regional and woreda levels.

Efficiency

The evaluation found the efficiency of the Ethiopia Program to be satisfactory. On average, it cost $2.65 million (including salary, program support unit (PSU) and Operating and Management funds) to disburse and manage $81 million annually. This was equivalent to 3.3% of the yearly disbursement of bilateral funds to the country. In should be kept in mind, however, that many of the bilateral projects are implemented by multilateral organizations, such as UNICEF, World Food Programme and the World Bank. These organizations also have administrative charges that are incorporated into project costs, which were not explicitly included in the calculation above.

Management Factors

The evaluation found satisfactory use of performance management frameworks, monitoring and reporting, and risk management in the Ethiopia Program. However, the program logic model and performance measurement frameworks for the Country Development Program Framework (CDPF) had not been updated since the current Country Program cycle began, although the Program has changed significantly.

There was a lack of consistency in reporting among DFATD-funded projects. Among the three branches, only certain bilateral projects had Management Summary Reports (MSRs) and they were not always completed consistently. While this is not specific to the Ethiopia Program, the current MSR format focuses to a great extent on cross cutting themes with limited attention paid to a review of progress towards expected outcomes in comparison with baselines or targets identified in the original or amended approval documents. This impedes the Programs’ ability to monitor and report effectively on progress.

Adherence to Paris declaration principles is reflected in the approach that DFATD has taken to complement the GoE’s strong sense of ownership over its policies, strategies and national flagship programs, and individual projects. Projects were well aligned with both national and local priorities and supportive of the use of local systems.

The evaluation team found evidence that coordination and coherence among the three DFATD Branches engaged in development in Ethiopia was informal but relatively good.

Conclusions

• The Ethiopia Program was relevant to and aligned with the GoE’s priorities and policies, as well as with international commitments and Canadian priorities.
• The Program was highly satisfactory in terms of results achieved. It supported key GoE flagship programs, one of which has been cited by the World Bank as a model for a social safety net program. The Program contributed to improved production practices, better soil and water conservation practices, sustainable land and water management, increased income for farmers, increased food security, decreased food gap, decreases in asset sales, improved nutrition and increased resilience. There is evidence that Ethiopia was not affected as much as other countries in the region during the 2011 Horn of Africa drought.
• The Ethiopia Program strengthened capacities at various levels of government (including MoA personnel) and communities. It has been effective in policy dialogue through donor working groups of the national flagship programs. An appropriate enabling environment, including community and government structures, has been put in place for the implementation of the two national flagship programs reviewed in this evaluation.
• Several projects have progressed or taken appropriate steps to ensure sustainability. Capacity building is intrinsic to almost all projects, and local systems were often used, especially for the national flagship programs (PSNP and AGP). One of the key challenges to sustainability has been to get the government to make budget provisions for operational costs. Government commitments to national Programs represented a positive step, although given Ethiopia’s economic status, several cycles of donor support may be required, especially for the multi-donor flagship programs.
• Good progress was made in gender equality, and environmental sustainability was strongly integrated into a number of projects. Governance was a more recent addition as a cross-cutting theme and, hence, several projects have not addressed it explicitly.
• In general, projects were efficient. They started on time and stayed within budget. However, the efficiency of projects was affected by delays in approval and start-up, and consequently shortened timeframes for implementation.
• At the level of individual projects, performance management frameworks exist and were monitored and reported on regularly. At the Program level, although the CDPF was prepared for five years, the logic model and PMF underwent several rounds of revisions and were not always aligned. Overall monitoring and reporting at the Program level was weak, and corrective action is recommended.
• The GoE had a strong sense of ownership of its policies and strategies and developed national flagship programs. The Ethiopia Program used local systems, not only in flagship programs, but also for other smaller and focused projects.
• Interaction among the three DFATD branches programming in Ethiopia increased over time. However, no formal coordination mechanism existed, even in a large and decentralized Program such as Ethiopia’s.
• There was a balanced approach to agricultural development through three main flagship programs, which included food security and nutrition, agricultural growth and development, research, and sustainable land and water management. By supporting flagship programs, Canada supported all aspects of the agriculture sector in addition to creating value at different levels of government and in the community.

Lessons Learned

• Working through large government flagship programs in a program-based approach works best when technical assistance is provided to support program implementation. In Ethiopia, having a strong local team of advisors in agriculture, food security, gender and capacity building was a crucial factor for successful program delivery and management, as well as capacity building.
• Programming through multi-donor pooled funds helped expand the reach of the Program to a larger number of beneficiaries. It also leveraged Canadian policy dialogue through donor coordination and donor working groups.
• The PSNP is a well-designed and strongly performing social safety net system that could be a model for other countries, as is the system of donor coordination that it employs. The PSNP operationalizes the Paris Declaration principles effectively — government ownership, donor harmonization and use of local systems.
• The experience in Ethiopia indicates that individual projects that are not fully supported by the GoE and which are not well integrated into the government budgeting and planning systems have a reduced chance of sustainability and are less likely to be scaled up.
• The flagship programs in Ethiopia underline the importance of community involvement and bottom-up planning of major national programs as a way of ensuring that implementation leads to results.

Ghana - Findings and Conclusions

Relevance

Canadian programming has been highly relevant and congruent with the development priorities of the Government of Ghana as described in their Growth and Poverty Reduction Strategy and the Ghana Shared Growth and Development Agenda, of which agriculture is a key driver. The Ghana Program’s initiatives were also designed to support the National Decentralization Action Plan/National Decentralization Policy Framework.

The Ghana Program contributed to budget support at three levels of government (national, regional, district), thereby allowing DFATD to gain an overall perspective of the strengths and weaknesses in key national policies, plans and systems down to the district level.

Canada’s focus on Ghana’s three northern regions has addressed the higher levels of poverty (63% in the north compared to 20% in the south), food insecurity and malnutrition that exist there.

Investments in food security were already a part of Canadian development programming in Ghana prior to the launch of the Food Security Strategy in 2009 and helped Canada to meet its global commitments (L’Aquila Food Security Initiative and the New Alliance) on food security and agricultural spending in developing countries. Canadian support for sustainable agriculture development and nutrition in Ghana has also been very relevant to the Millennium Development Goal (MDG) objective of reducing by half the proportion of people suffering from hunger by 2105.

Effectiveness

Budget support provided to the Ministry of Food and Agriculture (MoFA) enabled the implementation of several policies, strategies and plans, as well as four key Programs, leading to increases in staple food and livestock production. However, the increase in crop production was due to an increase in cultivation area rather than increases in productivity. The expected increase in the number of agriculture extension agents did not materialize nor did the strengthening of farmer-based organizations.

Post-harvest losses are a serious issue in Ghana. The Government of Ghana (GoG) did not implement a comprehensive post-harvest loss action plan as planned. Linkages to markets and the value chain, in addition to improving infrastructure, are key factors to reduce harvest losses and improve farmers’ revenues. In addition, coordination among government ministries and departments was inconsistent across various agriculture related initiatives. This has implications for the future as decision-making and budgets shift to the regions and district level with greater decentralization. All this highlights the extent of the work that remains in achieving better returns for farmers.

As a leading donor to the agriculture sector, Canada influenced the development of the Ghanaian Country Cooperation Framework for the New Alliance for Food Security and Nutrition and successfully engaged with the Ministry of Food and Agriculture and the Ministry of Health in the development of the National Nutrition Policy, thereby making this a multi-sectoral effort.

Some key outcomes for improved nutrition in the northern regions included a drop in rates of acute malnutrition by 50-58% from 2008 to 2011 compared to a 36% reduction at the national level. This is attributable to improved coverage and capacity in community management of acute malnutrition (CMAM). In 2014, there were 41 districts, representing 50% of districts in the focus regions, providing CMAM services and in many of these the cure rate is above 60% and the death rate is below 5%. All districts also have the capacity to monitor nutrition indicators using internationally recommended methodologies.

The World Food Programme’s (WFP) School feeding program led to nearly doubling enrolment from 2006, with attendance improving from 89% in 2007 to 96% in 2009. The take-home ration component made a significant contribution to the improvement of girls’ education, including in districts with the highest gender disparity in Ghana. Two of the project regions, Upper West and Upper East, recorded a Junior High School Gender parity of 1.0, compared to 0.92 for the national level.

It was too early at the time of the evaluation to see tangible results from the research activities conducted under the research and development pillar. Reference was made to the completion of 12 of 17 funded projects in the 2013 Report on Agriculture, and fact sheets to support the dissemination of knowledge were produced. However, no further evidence on the results of these activities was found.

Sustainability

Overall, the Ghana Program made satisfactory progress towards assuring the sustainability of results. Two key contributing factors were the use of government systems to channel funds, which contributed to greater government buy-in, and capacity building. Enhancing management and monitoring systems and the capacity of personnel at the MoFA and Ministry of Local Government and Rural Development (MLGRD) were a particular contribution.

The evaluation also confirmed that the GoG was allocating budgets, including transferring funds to the district level, for current DFATD-supported initiatives. The District Development Facility (DDF), a multi-donor initiative with direct budget support to the districts, also saw the GoG devolving a significant proportion of development partner funds to the districts. The GoG is gradually taking over initiatives implemented through multilateral agencies such as school feeding and malnutrition care and management in the northern regions, as well as monitoring nutrition indicators using internationally accepted methods.

Remaining challenges to sustainability include uncertainty as to the level of government commitment to certain initiatives in the absence of donor funding, the challenge of integrating new processes into an existing system, and a whether or not there is long-term commitment for research and development activities.

Cross-cutting Themes

Gender Equality: The integration of gender equality into the design of programs and projects was consistent across all DFATD-funded initiatives. That said, actual implementation of gender strategies was found to be uneven; relatively strong in some smaller initiatives implemented in Ghana, but weaker within larger initiatives led by the GoG.

There was a notable contribution to the integration of gender equality issues and gender mainstreaming strategies into the MoFA’s policy processes, although there remains scope to scale up implementation. Likewise the MLGRD will need support as it further decentralizes through initiatives such as the District Development Facility.

The implementation of the Gender in Agricultural Development Strategy has been slower than expected and some key stakeholders within the Ministry have not been able to play their role fully. For instance, the Women in Agriculture Development Directorate (WIAD), which is one of seven directorates in MoFA, has been working to improve nutritional outcomes. While this is a crucial mandate and Canada has provided sector support to the MoFA, WIAD’s program funding from MoFA has been only about $200,000 per year - an amount deemed insufficient to play a significant role on gender and food security issues across Ghana.

Environmental Sustainability:The Ghana Program made satisfactory progress towards environmental sustainability. All actors (governmental, international or non-governmental, partners and agencies) implementing programs supported by Canada established processes for integrating environmental sustainability.

DFATD engaged management at the MoFA to integrate environmental sustainability into its plans and projects, leading to improvements in its performance on environmental sustainability. With technical assistance from DFATD, MoFA developed and officially adopted the Agricultural Land Management Strategy and Action Plan (2009-2015) in 2009. However, implementation of the action plan was slow due to lack of committed MoFA resources to support planned activities.

In the northern region, the Assistance to Ghanaian Food Insecure project contributed to slowing land degradation and soil erosion in intervention communities, while the Climate Change Resilience in Northern Ghana project (CHANGE) encouraged climate-friendly food production. The Food Security and Environment Facility also contributed to increased food production and alternative livelihood activities in an environmentally sustainable fashion.

Governance: Canada made a notable contribution to the processes of decentralization in Ghana through budget support to MoFA and the establishment of a District-Development Fund, which was a scaled-up version of the Canadian supported project District-wide Assistance Project (DWAP) implemented in the northern regions of Ghana. The institutionalization of the functional organizational assessment tool provides an incentive to metropolitan, municipal and district assemblies (MMDAs) to improve performance by tying annual funding levels to scores on performance measurement exercises.

Canada also played a significant role in helping to track the budgetary allocations from the MoFA to the regions and provide comparative data on all the MMDAs in Ghana. Initiatives to assist targeted communities in developing community action plans have also contributed to strengthening governance at the district level. These initiatives also provide an enabling environment for future initiatives related to food security.

Efficiency

The cost to manage the Ghana bilateral Program, including salaries and other overhead expenditures was 2.6% of total yearly disbursements.

The trigger system established by donors to ensure that certain targets were met by the government prior to receiving planned financial disbursements provides an incentive for MoFA to be efficient, but the sheer volume of triggers and disbursement tranches raises questions about the transaction costs for DFATD and the Ministry. The mid-term assessment of the budget support to MoFA suggested that the large number of triggers (12 in 2011; 11 in 2013) may distract from daily activities of the Ministry and lead to sub-optimal use of scarce qualified staff. A more focused approach may be required, with a smaller and more focused number of triggers.

Management Factors

The Ghana Program had appropriate instruments to manage and monitor their initiatives, such as performance measurement frameworks and sector working groups and steering committees. Risk management was also satisfactory.

The preparation of country strategy documents, including logic models and performance measurement frameworks, was completed in 2009 but only finalised in 2012 following corporate instructions. Overall, the multiple revisions and inconsistencies presented a challenge in tracking and reporting on results. While the evaluation ascertained that corrective actions were being undertaken, these were not always documented. Hence, it was difficult to assess how monitoring information had been used for project and program management decision-making.

In terms of ownership, alignment and harmonization, key principles of the Paris Declaration, modalities such as budget support to the GoG and initiatives implemented by it helped contribute to ownership, as did multilateral agency initiatives that worked closely with the government and used the latter’s systems.

The Ghana Program showed leadership in promoting donor harmonization in the agriculture sector through the Agriculture Working Group. It took tangible steps to promote greater harmonization – for example, undertaking a scoping study on the readiness of MoFA and donors to undertake a sector-wide approach and organizing the inaugural visit of key donors to the northern regions of Ghana.

Conclusions

• The Ghana Program is highly relevant and well aligned to the priorities of Ghana, particularly efforts to achieve the MDG 1 target of halving hunger by 2015. Canada’s focus on the three northern regions was highly relevant to the needs of the population given the disproportionately high levels of poverty, lack of access to basic services and level of malnutrition compared to other regions. The additional focus on water and sanitation there was an appropriate complement in achieving improved nutritional outcomes.
• The Program was also aligned with Canada’s G8 commitments most relevant to the Ghana context, and engagement in policy dialogue made a significant contribution to enabling conditions for food security, public sector reform, and decentralization.
• The level of results achieved was satisfactory, working mainly through budget support, which accounted for two-thirds of the disbursements in Ghana. Canada contributed to the development of key strategies and plans for agriculture development and to their implementation, which led to improved crop and livestock production.
• Government capacity was bolstered in the areas of public sector reform, financial management and in agriculture. More specifically, the Program contributed to an improved food and agricultural policy, improved financial management, a new performance management framework, and the integration of environmental sustainability and gender equality into these documents and tools.
• Given the emphasis of the Ghana Program on sustainable agricultural development, the lack of a clear long-term Program strategy or initiatives to promote value chain development or the development of farmer-based organisations should be addressed.
• A notable Canadian contribution has been support to the processes of decentralization. Targeted interventions at the district and the community level have served as a model for decentralization for Ghana in the years to come. 
• There was satisfactory progress toward achieving sustainability. Institutional strengthening of MoFA and efforts to use local systems effectively and channel funds through the government were contributing factors. Ownership by the GoG was most evident in general budget support, sector budget support and the District Development Facility. Nutrition initiatives implemented through the WFP and UNICEF in the northern regions also used local systems. However, GoG operational budgets need to be secured for these programs to ensure long term sustainability.
• DFATD programming made a notable contribution to the integration of gender equality issues and gender mainstreaming strategies into policy and processes of MoFA, including M&E frameworks.
• Donor agencies, including DFATD, established processes for integrating environmental sustainability. There was ongoing support provided to Ghana’s Environmental Protection Agency to implement a National Action Plan on desertification. Some projects in the northern regions contributed on a small scale to slowing land degradation in the intervention communities and improving environmentally sustainable food production practices.
• While delays in the DFATD approval process put additional pressure on partners to achieve results in a shorter timeframe than planned, the efficiency of the Ghana Program was satisfactory overall, both in terms of cost to manage program activities and the level of results achieved. Still, the broad focus of DFATD budget support to MoFA with the number of triggers and disbursement tranches likely increased transactions costs for both partners, and may have unduly taxed human resources within the Ministry.
• Overall, the Program has exerted adequate efforts in managing for results and mitigating risks. Logic models and performance measurement frameworks existed and were generally used to monitor and report results on a regular basis.

Lessons Learned

• Initiatives implemented by the government, or in close concert with it, lead to increased ownership, capacity and sustainability.
• Implementing innovative pilot projects that demonstrate results at the community and district levels and have a good potential for scaling up can effectively complement program-based approaches. The District-wide Assistance Project is a good example: after demonstrating its effectiveness in the northern regions, it became the model for multi-donor support to the implementation of the District Development Facility nationwide by the MLGRD. 
• In the context of a decentralized governance system, it is important for different ministries to coordinate their activities at all levels to provide better outcomes for the population. The experience linking nutrition and agriculture activities at various levels of government in the northern regions was a good example of this. Another example was the support provided by MOFA’s agricultural extension agents working at the district and community levels to the Food Security and Environment initiatives. Other Canadian-funded initiatives, such as the Investment Fund for Food Security, could have benefited from this approach to achieve greater results for the targeted communities.    
• Supporting communities to develop local action plans, which include food security, gender and environmental sustainability, can lead to more appropriate applied research projects and more inclusive and responsive decision-making to meet community needs in the context of decentralized governance.
• Addressing such issues as productivity, post-harvest loss, the development of farmer-based organisations and strengthening extension services are essential to having a real impact on sustainable agricultural development.
• The use of a trigger system proved to be an effective mechanism to encourage governments to prioritise issues and for the Ghana Program to monitor progress alongside other donors. However, too many triggers and disbursement tranches can increase transaction costs; fewer well-designed triggers may be more effective.

Considerations for Food Security Programming, and Policy Framework

The findings and lessons from this evaluation of DFATD’s food security programming in Ghana and Ethiopia, which constitutes a significant proportion of its bilateral programming on this theme, suggest a number of considerations for the Department’s future programming and policy framework. These include:

• increasing support to smallholder farmers to move beyond crop and livestock production and improving their access to markets and to services that can add value to their products;
• creating or strengthening farmer-based organizations/cooperatives/common interest groups so as to build a critical mass of viable groups, to strengthen farmers’ bargaining power and ability to add value to their products;
• enhancing the linkages between the Food Security and Sustainable Economic Growth strategies to promote overall economic growth in agriculture, including support for the development of small (village/community level) and medium-size (district level) enterprises and facilitation of private sector investment in agri-businesses;
• mainstreaming nutrition in food security and agricultural development strategies, including a recognition of the multi-sectoral determinants of good nutrition status, particularly water and sanitation; and,
• planning and implementing applied research so that it can be affordably taken up by farmers to improve yields, food security, and income.

Recommendations – Overall

Recommendation 1: DFATD should review its performance management tools for country development programs to ensure that there is appropriate integration of project and program level reporting, as well as an adequate degree of consistency for investments from different delivery channels.

The practice of performance management of country level development results at DFATD has evolved in recent years. Country Strategies, Country Development Program Frameworks, and performance measurement frameworks have been used to articulate, track and report on sector and country-level outcomes. Project level reporting tools, such as the Investment Monitoring and Reporting Tool (IMRT), are also used. In IMRT, the Management Summary Report and Investment Performance Report serve, in part, to inform higher-level reporting. However, this and other country program evaluations undertaken recently have noted imperfect integration of these tools to higher levels (Program and Corporate), and inconsistencies in their application. Statements of immediate, intermediate, and ultimate outcomes across Country Program and project logic models are not always complete or consistent. Baselines, indicators and data sources are not always complete. Lower levels do not necessarily sum to higher levels in ways that are useful to managers. As well, for investments from multilateral and partnership channels that can be tracked to a specific country or region, there is a question of whether sufficient tools and reporting exist to allow an appreciation of the overall results of Canadian sponsored programming at the sector or country level. Efforts to address these questions deserve renewed emphasis.

Recommendations – Ethiopia and Ghana Country Programs

Recommendation 2: The Ethiopia and Ghana Programs should ensure that Performance Measurement Frameworks for their next generation CDPFs allow for effective monitoring of program level results.

Among others, attention should be paid to collecting sufficient baseline information and regularly tracking chosen indicators.

Recommendation 3: The Ethiopia and Ghana Programs should ensure that food security-related research initiatives are user-focused and serve to improve poverty reduction and economic growth outcomes.

Recommendation 4: DFATD should improve coordination of Canadian-funded research initiatives from different Branches throughout the planning cycle.

DFATD funds food security research projects through different modalities and Branches. In order for research results to be taken up by national counterparts and targeted beneficiaries, DFATD should ensure that planned projects have credible end-use strategies from the outset, which are aligned with the national poverty reduction and economic growth agendas in both countries. Increased coordination and communication between Branches, as calls for proposals are designed and as projects are implemented in the field, is also necessary for research results to be complementary and applied to best effect.

Recommendations – Ethiopia Country Program

Recommendation 5: The Ethiopia Program should continue to complement proven existing programming, like support for the Productive Safety Net Programme, with activities to increase the self-sufficiency of beneficiaries.

The provision of additional funding and technical assistance to initiatives that emphasize the safety net aspects of food security programming in the districts where food insecurity is most pronounced should move more households out of a chronic dependency on food assistance and provide them with increased income, thereby improving their self-sufficiency. While the focus of PSNP is to increase basic food security by providing cash or food in exchange for labour on public works, increasing the funding in these food insecure districts for value-added or alternative income generation activities would help to ensure that PSNP graduates do not revert to a dependency on PSNP assistance. This would also support the Government of Ethiopia’s efforts to shift funds to higher value economic growth activities.

Recommendation 6: In its future programming, DFATD-development should ensure that Canadian-funded projects implemented by Canadian implementing agencies are adequately aligned with Government of Ethiopia systems and processes in order to promote ownership and sustainability.

The Ethiopian Government has a policy of maintaining tight control over development interventions in the country by outside agencies. It has also been very good at designing its own programs and initiatives with a view to systematizing processes and practices to ensure sustainability. The corollary of this practice is that initiatives that are not well coordinated with the Ethiopian planning and budgeting systems may not be taken seriously by the government and have a reduced chance of being sustained or supported once external development partner funding ceases. While understanding that sustainability is usually included in project design, DFATD should nonetheless ensure that any Canadian projects are coordinated with the government to the extent possible.

Recommendations - Ghana Country Program

Recommendation 7: The Ghana Program should consider supporting innovative pilot projects , such as those in food security, that have a potential to be scaled up nationally.

The Ghana Program should leverage its experience in demonstrating effective programming at the district and community levels by increasing funding to initiatives that have the greatest potential to enhance the economic benefits to and the resilience of farmers in the face of climate change. The Ghana Program should continue to focus on such initiatives in the northern regions as they can help to reduce poverty in a targeted manner while also strengthening national systems from the bottom-up.

Recommendation 8: The Ghana Program should continue assisting the GoG to strengthen multi-sectoral linkages at the community and district levels.

DFATD should build upon what is has learned using multi-sector approaches to foster greater coordination among all stakeholders at local levels – for example, extension services, farmer-based organizations and the research establishment – to ensure that the priorities of communities are integrated in district plans, that the transfer of technologies to farmers is accelerated and that resources are leveraged more effectively to address the food security needs of the population in a more comprehensive fashion.

Recommendation 9: The Ghana Program should strengthen its approach to addressing gender issues at various levels of government, including assistance to the MoFA to strengthen and leverage existing gender resources at the national and district level, so that these more effectively support food security-related initiatives to the benefit of communities.

In future programming, the Ghana Program should build on work done to strengthen MoFA policies and strategies, and in particular, to strengthen WIAD gender resources to play a more effective role as policy advisors on food security issues and to become focal points for programming at the district and community levels.

Recommendation 10: If the Ghana Program continues to provide sector budget support, it should ensure that the trigger and disbursement system is appropriately calibrated to national capacities to efficiently initiate, monitor, and measure desired changes.

Emerging evaluation evidence, and feedback from national authorities, suggests that strong national ownership and capacity to implement well-focussed reforms is more effective than numerous donor-driven ones.

1.0 Introduction

The Ethiopia and Ghana Country Program Cluster Evaluation was conducted between December 2013 and December 2014. It assessed Department of Foreign Affairs, Trade and Development’s (DFATD)^{Footnote 1} performance in providing development assistance to Ethiopia and Ghana between 2008/9 and 2013/14, with particular focus on food security. This evaluation is intended to provide key lessons and recommendations to support evidence-based decision making on policy, expenditure management and program improvement.

The first two sections of the report cover the rationale, purpose, approach and methodology and DFATD’s context by country. Findings, conclusions and lessons learned for Ethiopia and Ghana are then presented in Sections 3 and 4, respectively. Section 5 presents considerations for food security programming and policy framework and Section 6 presents recommendations.

1.1 Rationale and Purpose

In accordance with the Federal Accountability Act (2006), and the Treasury Board Policy on Evaluation (2009), the purpose of the Ethiopia and Ghana Country Program Cluster Evaluation is accountability and program improvement. This evaluation is one of the first Country Program evaluations to be conducted as a cluster of two country programs with a focus on one thematic area. It samples projects from three former CIDA Branches as part of the Country Program – Geographic Programs Branch, Multilateral and Global Programs Branch and Partnership with Canadians Branch.

In addition to DFATD’s senior management and Program managers, the audience for this evaluation includes the Governments of Ethiopia and Ghana, partner civil society and multilateral organizations, and other development agencies working with DFATD in Ethiopia and Ghana. The evaluation is expected to enhance stakeholder understanding of achievements and lessons from DFATD’s development interventions in both countries.

1.2 Objectives and Scope

1.2.1 Objectives

The objectives of the evaluation were:

• To assess the overall performance of the Ethiopia and the Ghana Country Programs (fiscal year 2008/09 to fiscal year 2012/13) in achieving development results, with a particular focus on food security; and,
• To provide Foreign Affairs, Trade and Development Canada with relevant information for future international development programming in the two countries especially with respect to food security.

1.2.2 Scope

The scope of this evaluation, as envisaged in the Terms of Reference, included an assessment of:

• The overall performance of the Country Programs with respect to relevance and aid best practices (alignment, harmonization, ownership), effectiveness (including in the use of policy dialogue) and efficiency (based on a sample of projects with a thematic focus in food security);
• Cross-cutting themes (gender equity, environmental sustainability, and governance), adequacy of coordination, linkages and synergy across DFATD’s Programming, as well as how DFATD Programming complemented national food security investments in each country;
• DFATD’s ability to meet commitments with regard to the effectiveness of its international development assistance, including alignment with the Paris Declaration (2005) and the Accra Agenda for Action (2008). The evaluation also took into account the Millennium Development Goals (MDGs) and the main measures taken by DFATD to implement the action plan for increasing the effectiveness of aid.

2.0 Approach and Methodology

The assessment of performance of the Country Programs in Ethiopia and Ghana was carried out primarily by reviewing a representative sample of projects, as well as policy dialogue activities, in each country. The extent to which interventions attained the outcomes described in their logic models and performance measurement frameworks (PMF), as well as unanticipated positive and negative impacts, was assessed using the Evaluation Design Matrix which appears as Appendix A. sol

2.1 Methods

The evaluation used mixed-methods—document and literature review, key informant interviews, focus groups and case studies/site visits, and an analysis of program and national statistics, where available, to gather multiple lines of evidence. There were focus groups with beneficiaries at different levels, including women and youth. The team interviewed 212 key informants in Canada, Ethiopia and Ghana (Appendix G, Table 1) and reviewed more than 245 Program and project documents, in addition to relevant literature (Appendix F).

There were project site visits in Ethiopia and Ghana (Appendix H). During field missions to Ethiopia, there were visits to project sites in nine woredas (districts) in two regions (Tigray and Amhara); and in Ghana to sites in six districts spread across three regions (Upper West, Upper East and Northern).

Project Sample

A sample of 40 projects was selected for in-depth review (Appendices B and C). The breakdown was as follows: 20 projects in the Ethiopia Program ($344.32 million—54.3% of total disbursements and 82.3% of food security disbursements) and 20 projects in the Ghana Program, ($223.87 million—44.8% of total disbursements and 92.5% of food security disbursements).^{Footnote 2}

The criteria used to select the sample of projects included initiatives that: a) met DFATD’s food security priorities, as well as contributed to improved food security in each country; b) represented the various delivery channels (bilateral, multilateral and global partnerships with Canadians); c) reflected the main delivery modalities used by each Country Program; and d) were at least 50% of the total of the DFATD expenditures in each country’s for its aid program. In addition to these criteria, the sample included a mix of status (closed/operational), investment type (e.g. Program-based approach, projects), and delivery mode - core support, responsive funding and directive funding.^{Footnote 3}

Data Collection and Analysis

Data collection for this evaluation largely relied on: a comprehensive document review; semi-structured interviews with stakeholders at various levels; and discussions with project beneficiaries during site visits. Data collection began with an initial review of documents and interviews in Canada followed by field missions to Ethiopia (February 24 – March 7, 2014) and Ghana (March 10 – 21, 2014). The in-country evaluation team included a mix of Canadian and local consultants.

The data collected was tabulated and aggregated for analysis. A content analysis of documents and interview notes helped to draw out emerging themes and issues. The sampled projects were assessed on a DFATD rating scale (see Table 2.2 below) following the document review and ratings from key informants and aggregated at the Program level in combination with the other sources of data.

The evaluation team triangulated by comparing information collected through various lines of evidence in the data collection and analysis stages. This included a comparison of findings and observations made by Canadian and local evaluation team members following key informant interviews, focus groups and field visits in Canada and in Ethiopia with formal program/project reports and analyses that were available. A summary of ratings by project can be found in Appendix L and M.

Limitations and Challenges

As with many evaluations of similar scope and scale, there were challenges that had to be addressed, including:

• Shortcomings in project documentation, including:
  • A limited number of independent project evaluation reports;
  • Different performance measurement frameworks and reports for bilateral, multilateral and partnership projects;
  • Differences in the templates for field/monitoring reports; and,
  • Limited detail in project budgets, which limited the analysis of efficiency and economy.
• Inconsistencies between the Country Program logic models and performance measurement frameworks in both the Country Programs. For example, some of the specified logic models results in the Ethiopia Program were not reflected in the PMF, and hence no indicators were identified. In these instances, the evaluation team had to rely on progress reports and key informants’ interpretation of progress;
• The nature of donor-harmonized and government-led initiatives where investment is generally pooled in multi-donor accounts or channelled through government financial systems made it difficult to attribute achievement of results to DFATD directly. In these instances, since DFATD was often a significant contributor, our comments pertain to the overall evaluation results of the project/or program compared to what was envisioned in the DFATD approval documentation;
• This evaluation covers the respective Country Development Program Frameworks from 2008/09–2012/13. Programming was designed and initiated in 2008 but CDPFs were revised during the evaluation period. The Food Security Strategy was approved in 2009, thereby rendering it somewhat challenging to assess the extent to which programming was aligned to and consistent with the Strategy; and,
• Data constraints rendered a thorough assessment of the efficiency of each Program challenging.

2.2 DFATD Context

In 2007, the Government of Canada established its Aid Effectiveness Agenda, committed to making Canada’s international co-operation more efficient, focused and accountable. Since then, Canada has taken important steps to reform its aid program in accordance with this agenda and in line with international agreements and recognized best practices. For example, Canada untied all food aid in 2008 and all non-food aid in 2012.

Within the framework of this Agenda, the Government of Canada also established three priority themes to guide its international development work: increasing food security; securing the future of children and youth; and stimulating sustainable economic growth. These three priority themes – in addition to Canada’s global commitment to the Muskoka Initiative (2010) to improve maternal, newborn and child health – guide DFATD’s international assistance. Additionally, three cross-cutting themes: increasing environmental sustainability; advancing equality between women and men; and helping to strengthen governance are meant to be integrated into all of DFATD’s international development programs and policies.

3.0 Ethiopia Country Program

3.1 Country Context

Ethiopia is one of the world’s poorest countries despite recent development progress (Table 3.1). In 2012, Ethiopia had a Gross Domestic Product (GDP) of $43.62 billion, with an annual growth rate high of 8.5%.^{Footnote 4} However, while GDP growth has remained high, per capita income is among the lowest in the world (gross national income (GNI) per capita at $410 in 2012).^{Footnote 5} The second most populous country in Africa, Ethiopia’s population was estimated at 91.73 million in 2012. Of this, 29.6% lived below the national poverty line and 80% lived on less than US$2 per day^{Footnote 6}. Ethiopia is ranked 173 of 187 countries on the Human Development Index (HDI).^{Footnote 7} According to the latest reports, Ethiopia is on track to meet most of the MDGs by 2015 if current progress continues. The Government of Ethiopia and the United Nations are now accelerating efforts to meet the MDGs that are slightly off-track (Gender Equality and Women‚ Women's Empowerment, Maternal Health and Environmental Sustainability) through the Growth and Transformation Plan and the United Nations Development Assistance Framework.^{Footnote 8}

Ethiopians suffer from low life expectancy, high rates of infant mortality (52 deaths per 1000 live births), under-five child mortality (198 per 1000 live births) and maternal mortality (680 per 100,000 live births), with an HIV infection rate of 1.4%.^{Footnote 9} The main development challenge has been food security, with 80% of Ethiopia’s population engaged in low-productivity, rain-fed, subsistence agriculture that is highly vulnerable to short-term shocks caused by drought and economic crises. Other threats include the long-term impacts brought on by climate change, land degradation, and population density. Ethiopia’s economy is largely agriculture-based, yet the agricultural sector suffers from poor cultivation practices and frequent drought.^{Footnote 10}

Key issues pertaining to DFATD cross-cutting themes include the following:

• Ethiopia ranks 126^th of 155 countries in the Gender Development Index (GDI, 2013).^{Footnote 11} In 2010, 50.2% of the female population lived in rural Ethiopia and 73.5% of economically active women worked in the agricultural sector.^{Footnote 12} The National Action Plan on Gender Equality (2006–2010) provides the framework for mainstreaming gender issues.
• Ethiopia’s ecological system is highly fragile and vulnerable to climate change, leading to a greater frequency of drought years, unpredictable rainfall patterns and shorter rainy seasons. This is compounded by population pressure and stresses on natural resources, especially land. Soil erosion, land degradation, deforestation, overgrazing, and poor water management are all critical environmental issues undermining Ethiopia’s potential for achieving food security. To help deal with these issues, the Government of Ethiopia (GoE) established policy frameworks, including an Environmental Policy (1997) and a National Biodiversity Strategy and Action Plan (2005); however, there is a need for more clarity on operational/implementation frameworks to integrate climate change adaptation and mitigation measures into sector plans and investments. It is also necessary to strengthen water resource management and to ensure sustainable development.^{Footnote 13}
• Ethiopia’s public administration capacity and its public financial management system are generally sound compared to its HDI cohort countries, which have similar HDI characteristics. Historically, Ethiopia has had a reputation for not tolerating corruption.^{Footnote 14} However, Transparency International’s Corruption Perception Index score for Ethiopia in 2013 was 33, and Ethiopia was ranked 111 of 177 countries.^{Footnote 15} Some of the issues in public administration that affect development partner support for Ethiopia’s programming and service delivery, include high staff turnover, political appointments, unattractive salaries, unfilled posts, and lack of technical competence. Vacancies and staff turnover are also problematic in the districts, particularly the poorest areas^{Footnote 16}, where isolation makes assignments unattractive, and budgets are insufficient to fill staff complements.

Since Ethiopia’s 2005 post-election crisis, DFATD, along with other major donors, stopped using general budget support as a delivery modality. However, a few donors have since provided funding to the GoE directly, although not as direct budget support.

Government regulations have sometimes led to restricted civil society growth and made it difficult to implement foreign-funded development initiatives. Regulations were introduced in 2009^{Footnote 17} that restricted local civil society organisations (CSOs)/charities that received more than 10% of their funds from overseas for human rights initiatives.^{Footnote 18} Likewise, another set of new laws stipulating that only 30% of non-governmental organization (NGO) budgets could be used for administrative expenses was interpreted in 2011 to include most travel and training costs. This has proved challenging for many NGOs and has caused them to limit training, capacity building, and monitoring and evaluation activities, all of which are important to knowledge and capacity transfer and sustainability of interventions.^{Footnote 19} In 2010, the World Rule of Law Index report ranked Ethiopia 88 out of 99 countries.^{Footnote 20}

The GoE adopted a five-year Growth and Transformation Plan (GTP) 2010/11–2014/15, which places special emphasis on the role of agriculture as a major source of economic development in Ethiopia. The GoE’s budget allocation to benefit the poor is the highest in Africa.^{Footnote 21} In particular, the government is addressing food insecurity through its long-term strategy of Agricultural Development-led Industrialization (ADLI).^{Footnote 22}

Following the ADLI strategy and building on achievements from its Plan for Accelerated and Sustained Development to End Poverty (PASDEP), the GTP has set priorities that will intensify productivity by smallholders and support market-oriented agriculture. In 2010, the Ministry of Agriculture and Rural Development launched the Food Security Program with a focus on rural areas. Additionally, the GoE created the Agricultural Transformation Agency in 2010, which reports to an Agricultural Transformation Council chaired by the Prime Minister of Ethiopia and vice-chaired by the Minister of Agriculture. The Agricultural Transformation Agency has been tasked with supporting all key stakeholders (including the Ministry of Agriculture) to transform Ethiopia’s agriculture sector so the country can reach middle- income status by 2025. Canada supported the Food Security Program but not the Agricultural Transformation Agency.

Overall Development Portfolio

Canada ranked between the seventh and fourth largest donor to Ethiopia from 2009 – 2013. Other major donors included the World Bank’s International Development Assistance fund (IDA), the United States, the African Development Fund, the United Kingdom and Japan.

DFATD disbursed $633.72 million to the Ethiopia Program from 2008/09 to 2012/13, of which about 65.8% ($416.67 million) went to initiatives related to food security.

Figure 3.1: Disbursements in Ethiopia by DFATD Delivery Channels (2008/09 to 2012/13), millions

Note: This data relates to the period before amalgamation and, as such, uses the former CIDA Branch titles.

The main delivery channel was the Geographic Programs Branch, which accounted for 63.8% of disbursements. The other two delivery channels were Multilateral and Global Programs Branch, which accounted for 29.4% of disbursements and the Partnership with Canadians Branch for 6.8% of disbursements (Appendix D, Figure 1).

Disbursements were made either through Program-Based Approaches (PBAs) (49.1%) or through projects (50.9%). PBAs in Ethiopia included multi-donor pooled funds and other PBA support activities, including technical assistance (see Appendix D). In terms of Program delivery models, while 57% of the disbursements were responsive, 38.5% were core funding and only 4.4% were directive. Disbursements were made to a number of partners/implementing organizations^{Footnote 23}, 61% to CSOs, 37% to multilaterals and 2% to governments and others. However in terms of dollar value, 77.9% of DFATD investments went through multilaterals (57.2% through multi-donor trust funds/other PBAs, which is 44.6% of total disbursements), 19.4% through CSOs and the rest (2.7%) through governmental and other organizations.

DFATD’s Food Security Portfolio in Ethiopia

From 2008/09–2012/13, Canada disbursed $416.54 million in Ethiopia for initiatives related to food security. A majority (55.1%) of these disbursements were attributed to the Food Assistance and Nutrition pillar of the Food Security Strategy. The Sustainable Agricultural Development pillar accounted for 41.4% of the food security portfolio, while the Research and Development pillar accounted for 3.5% (Figure 3.2).

Figure 3.2: Food Security Disbursements in Ethiopia by Pillar (2008/09–2012/13), millions

Source: Compiled from DFATD database

Disbursements in Ethiopia by sub-sector and related to the Food Security Strategy, were as follows: development food assistance (29.1%), agronomic and post-harvest activities (26.1%), enabling activities and support services (15.4%), emergency food assistance (12.7%), nutrition (12.5%) and agricultural research for development (3.5%).

3.2 Findings - Development Results

Relevance

The criteria used to assess relevance were based on whether the Country Program was aligned with:     a) Ethiopian priorities as expressed in the Plan for Accelerated and Sustained Development to End Poverty 2005/06–2009/10 and the Growth and Transformation Plan (2010/11–2014/15); b) the MDGs; c) international commitments made by Canada (G8 L’Aquila Food Security Initiative, New Alliance for Food Security in Africa, Scaling-up Nutrition, the Food Assistance Convention); and d) the DFATD Food Security Strategy.

The GoE has placed agricultural development at the centre of its poverty reduction strategy. A central strategy of the recently completed PASDEP, market-based agricultural development focusing on Ethiopia’s 13 million smallholder farm households, was given high priority by the GoE.^{Footnote 24} This strategy has been further emphasized in the new five-year Growth and Transformation Plan 2010/11–2014/15, as well as Ethiopia’s Agricultural Sector Policy Investment Framework 2010–2020. The evaluation found that given that the DFATD Country Program directly supported Ethiopian government programming that supports the Growth and Transformation Plan either directly or indirectly, it was well aligned with the GoE’s poverty reduction strategies during the evaluation timeframe.

The need for increased investments that promote food security, such as watershed rehabilitation, also feature prominently in the Growth and Transformation Plan and in Ethiopia’s Agricultural Policy and Investment Framework 2010–2020, which was completed in September 2010 under the Comprehensive Africa Agriculture Development Program process. A total of 10.2 million hectares of degraded land is targeted for rehabilitation in order to improve agriculture production and reduce food insecurity. The direct relevance of the DFATD Ethiopia Country Program to these priorities was evident in Canada’s investment in the important GoE flagship programs: the Productive Safety Net Program (PSNP), the Agricultural Growth Program (AGP), and the more recent Sustainable Land Management Project. The PSNP is a cornerstone of Ethiopia’s National Food Security Strategy.

By supporting the GoE flagship programs described above, which are directly targeted at reducing hunger and extreme poverty, as well as many of the other projects supported by Canada that are also targeting improved agricultural productivity and practices, DFATD Programming is also directly relevant to MDG 1. Canada has also supported various global initiatives in Ethiopia through its G8 Commitments, namely the L’Aquila Food Security Initiative, the New Alliance for Food Security and Nutrition, Scaling-up Nutrition, the Food Assistance Convention and the Muskoka Initiative.

The evaluation also found that projects approved and additional funding provided by Canada for the Ethiopia Country Program have been consistent with the DFATD Food Security Strategy, as well as other DFATD priorities. For example, the additional $35 million provided to the PSNP was highly consistent with the Nairobi Strategy issued on September 9, 2011 at the Summit on the Horn of Africa Crisis.

The Growth and Transformation Plan has identified private sector led growth in agriculture as an important element of the Ethiopian Growth strategy. The GoE would like to focus on building up micro and small scale industry in order to advance its objective of agriculture led growth. The country is starting to experience some investment in agribusiness, a trend that is likely to increase in the future. The Canadian program could support this trend by linking Canadian and Ethiopian businesses through trade and investment. This would be consistent with the bilateral Program’s efforts to focus on sustainable economic growth.

Effectiveness

A sample of 20 projects was assessed, from which the largest proportion of disbursement was to two of the Government of Ethiopia’s flagship programs – the Productive Safety Net Program and the Agricultural Growth Program. The narrative of this report is organized around the three pillars of DFATD’s Food Security Strategy: food assistance and nutrition; sustainable agriculture and development; and research and development.

Food Assistance and Nutrition

The projects that formed the backbone of the Food Assistance and Nutrition pillar included the Productive Safety Net Program, the Safety Net Support Facility and support to the World Food Programme (WFP). There was also a major initiative to improve nutritional outcomes implemented by UNICEF.
Tackling food insecurity is one of the priority concerns of the Ethiopian Government. In 2003, the government worked with development partners to formulate a Food Security Program to address the needs of food insecure households with a predictable and timely resource transfer approach. The Productive Safety Net Program^{Footnote 25} became an operational part of the Food Security Program in 2005 and was the main instrument used to assist chronically food insecure people in the rural areas of the country.

The Ethiopia Program has been an important contributor to the PNSP through support to the World Food Program for the food component of the PSNP and through a contribution to the World Bank Trust fund for the cash component. In addition, DFATD funded technical assistance to support the implementation of the PSNP through the Safety Net Support Facility (SNSF).

SNSF’s mandate is to support the increased effectiveness of the Government of Ethiopia’s institutions to deliver the PSNP and Household Asset Building Programme (HABP). It was designed by the GoE and its development partners to provide structured and coherent capacity development support targeted primarily at PSNP implementing agencies at regional, zonal and woreda levels. SNSF is working to strengthen the institutional systems, processes, coordination mechanisms, human resources and organizational capacity of federal, regional, zonal and woreda institutions to deliver PSNP/HABP.

The Program contributed to the food aid component of the PSNP, providing emergency food assistance to an annual average of 3.7 million people in the years 2008-2010. In addition, the Program supported WFP’s Protracted Relief and Recovery Operation (PRRO) project, which helped 300,000 beneficiaries who were not able to participate in the public works component of the PSNP.

Improving nutrition is a general health priority in Ethiopia. Canada supported WFP’s school feeding program, which included meals and take-home rations for girls. Canada’s funding covered 54% of the girls reached through take-home rations. Between 2007 and 2009, absolute enrolment in schooling increased by 31% for girls and 24% for boys, in part due to the availability of the school feeding program.

The WFP school feeding program benefited 555,000 children with nutritionally fortified meals during the period 2007-2011.

As part of the Muskoka Initiative, through the Improved Food Security for Mothers and Children project, Canada made efforts to improve nutritional status in 100 food insecure districts in Ethiopia through training of health extension workers, along with deworming activities and vitamin A supplementation. The UNICEF project, which is still underway, is training health workers in nutrition in 100 DFATD supported woredas. In addition, 243,000 children are being supplied with vitamin supplements and deworming. The project intends to scale up to cover almost 1 million children in its second round.

Sustainable Agriculture Development

The Ethiopia Program allocated approximately 41% of its funding to activities targeting sustainable agricultural development. Overall, the Program made significant contributions to improvements in land and water management, agricultural productivity and increased production, and improved market structures in the country.

a) Improved land and water management. The Program supported initiatives that contributed to the rehabilitation of degraded land, the construction of embankments (terraces, bunds and check dams) to prevent soil erosion, water harvesting, organic soil management practices and small-scale irrigation projects. Results achieved in these initiatives include:

• 144,803 hectares (ha) of rehabilitated land, 66,000 embankment improvements, 34,408 ponds and 15,671 hand-dug wells in the 319 woredas eligible for PSNP support;
• 985,000 ha of cultivated land treated with conservation terraces and biological measures, 2,120 springs developed, 2,500 water ponds, 6,600 micro-ponds and 26,900 kilometres of gully check dams were constructed; and,
• Increased vegetation cover through the planting of 2.8 million trees and shrubs.
• Increased agricultural production. The Program supported projects that achieved increased production in one or more of the following commodities: rice, onion seed, egg, sesame, vegetables, banana, livestock and/or honey. The GoE Agricultural Growth Program, a GoE flagship, reported that more than 58,000 farmers had been exposed to new methods, with 43,000 focused on improved crop production, 12,000 on livestock and 3,000 on conservation. The AGP created 1,500 farmer innovation groups, supported 263 primary cooperatives and initiated 125 irrigation schemes. PSNP’s small-scale irrigation projects helped between 4% and 12% of households in various woredas to increase their livestock holdings; and Improving Productivity and Market Success (IPMS) supported introduction of 5,000 ha of annual rice production in Fogera.

b) Improved productivity. Better farming practices and improved irrigation have resulted in increased productivity. The AGP project and others have supported improved productivity in teff, chickpea and rice production in selected woredas. In one case, the introduction of a high yielding, export-quality variety of chickpea replaced local varieties and doubled yield per ha in five years.

c) Increased income. A number of projects supported improvements in the value chains of small farmers, resulting in higher incomes from livestock fattening and increased production of rice, banana, vegetables and onion seeds. In the case of onion seeds (certified seed production) and rice (branding for supermarkets), increased value added provided further increases in income. Examples include:

• An increase of 85% in income for rice farmers in two woredas of the Amhara region; and,
• Increased income from banana production and sales in Metema of $3 million per year (IPMS).

d) Development of market and market structures. The Program contributed to improved access to markets. For example, the AGP recently reported that 303 km of road and 32 market centers are under construction to address problems associated with limited market infrastructure.

Research and Development

A number of activities supported by the Program contributed to strengthening Ethiopia’s research capacity. Among the potentially useful research results were increased yields in chickpeas; some of the policy analysis and research implemented by IFPRI; and applied research in value chains supported by the IPMS project implemented by ILRI and projects, such as the Agricultural Growth Program and the Rural Capacity Building Project (RCBP), which were aimed at transferring research results through farmers’ research extension groups.

Although interviews and document review indicated some positive research results, there was limited evidence that these had been disseminated or applied in practice.

Policy Dialogue and Enabling Environment

The evaluation team noted that policy dialogue with the GoE is challenging given their conviction that a national government should be in control of its development policy and implementation. Influence requires evidence-based approaches and leveraging of official policy statements. Notwithstanding this context, Canada has been actively involved in policy dialogue through the high-level Development Action Group, donor coordination groups and project level working groups, although it is sometimes difficult to influence policy unless a GoE minister is involved.

Canada currently chairs the Rural Economic Development and Food Security Working Group and is active in the PSNP donor coordination committee and the AGP working group. There was little evidence found of the impact of Canada’s leadership in the Rural Economic Development Working Group.

The PSNP donor coordination committee was considered the best means of influencing GoE policy. For example, the PSNP donor coordination committee provided assistance in developing operational policies and procedures during various phases of the PSNP development. The PSNP donor coordination group, in which Canada participates, has contributed to influencing the GoE’s social protection and disaster risk management policies and strategies. The group has also highlighted the benefits of SNSF (a stand-alone technical assistance project funded by Canada) to the GoE. Evidence also indicated strong collaboration and constructive dialogue on policies among donors in the SNSF technical assistance project. The PNSP donor coordination group has engaged in discussions with the GoE on improving the nutritional aspect of the food basket given to PSNP beneficiaries. As noted during interviews, this is likely to lead to integrating a technical assistance component into the next phase of PSNP.

The Ethiopia Program contributed to the development of operational policies and procedures in sustainable agricultural development, food assistance, and nutrition and supported the Bureau of Water Management (Amhara) by aiding in drafting a regional water law and accompanying regulations.

Sustainability

Sustainability was assessed based on the following criteria: the probability of benefits continuing beyond the intervention/project timelines; evidence of institutionalization of capacity developed through the intervention; the need for continued donor support; and any evidence or scope for integration of an intervention’s budget into national, regional or district budgets.

The GoE has been highly committed to projects that are consistent with its national planning priorities, including its own flagship programs. Although they rely on external funding for programming, the salaries of GoE staff working in these programs are mostly paid for by internal resources.

Sustainability has been enhanced by the extensive support for capacity building integrated into most of the DFATD-supported projects, including capacity building for personnel working at all levels in the in MoA, including farmers, researchers, community members, and others. A strength of the PSNP is the governance structure developed with the assistance of donors such a Canada, including various committees created at the community, kebele, woreda and regional levels that are actively engaged in the targeting of beneficiaries and overseeing PSNP implementation. Evidence of institutionalized capacities was also found in other projects. The GoE’s efforts to replicate SNSF training modules at its own cost in non-SNSF woredas in Tigray region was indicative of government commitment and a step toward institutionalization of capacities and sustainability.

A positive factor enabling sustainability was the leveraging and scaling-up of approaches, methodologies and models of completed activities by the next generation of projects. Examples included:

• Managing Environmental Resources to Enable Transition to more Sustainable Livelihoods’ (MERET) people-centred and community participatory approach to natural resource management activities and sustainable land management, to which Canada was a major donor, was found to have influenced two national flagship programs: PSNP (covered in this evaluation) and Sustainable Land Management Program (SLMP; not included in this evaluation);
• IPMS’s approach to sustainable transformation of subsistence agriculture using innovative approaches was incorporated into AGP and Livestock and Irrigation Value Chains for Ethiopian Smallholders. This included the use of training modules from IPMS in AGP and other projects.
• Lessons and models from RCBP were integrated into AGP. Examples included capacity building of district agents, strengthening of farmer’s training centres and transfer of research to farmers through farmers’ research extension groups.

Many of the projects reviewed for this evaluation were implemented in partnership with local institutions or other community organizations. This is considered a positive contributing factor to sustainability.

Cross-cutting Themes

The evaluation assessed how DFATD’s three cross-cutting themes – gender equality, environmental sustainability and governance – were integrated into the Program. The evaluation looked at barriers and enabling conditions for integration if any, as well as what worked and what could be improved.

Gender Equality

It is a policy requirement that gender equality concepts are included in project approval documentation and subsequent work plans. Development staff and Ethiopia-Canada Coordination Office (ECCO) gender experts ensured that gender equality issues were reviewed on a regular basis in program monitoring and during Steering Committee meetings for the PSNP and the AGP, in particular.

Sustainable agriculture projects made good efforts to integrate gender into the project design, however, there was limited evidence of demonstrable results in some cases. Particularly strong efforts to integrate gender were made in projects, such as IPMS, MBSIL and RCBP. Although AGP had integrated gender equality aspects, reports indicated that it had not been able to achieve its expected results since its inception in 2011.

Environmental Sustainability

In general, while projects related to food assistance and nutrition were not considered to have significant environmental impacts owing to their focus on food assistance distribution and health and nutrition education, a significant component of the PSNP project incorporated an environmental sustainability component. A review of PSNP found that almost all (98%) sub-projects in the four major highland regions had undergone environmental screening; however, some additional work was required on follow-up mitigation measures. Overall, the review noted that PNSP’s public works activities had had a highly positive environmental impact in PSNP districts throughout the country.

Governance

Through the PSNP, Canada has helped the government to develop strategies for management and preparedness that support PSNP beneficiaries. There are well-established structures and committees at community, kebele, woreda and regional levels to target beneficiaries and manage PSNP implementation. PSNP has enabled committees at the community level to target and select beneficiaries for PSNP. However, governance had not been defined as a cross-cutting theme at the start of the evaluation period and was not integrated into project designs as a result.

Efficiency

The evaluation looked at costs required to manage delivery of bilateral projects, which was on average $2.65 million per year based on salaries of Canadian and Ethiopian staff and operational budgets. This was equivalent to 3.3% of the yearly disbursement of bilateral funds to the country. The costs ranged from $2.42 million to $2.77 million per year between 2009/10 and 2012/13 (see Table 3.3). While the overall disbursement from 2008/09–2012/13 for Ethiopia through the three DFATD delivery channels (bilateral, multilateral and partnerships) was $633.73 million, the share through the bilateral Program was $404.42 million. Bilateral disbursements averaged $81 million a year during the evaluation period.

It should be kept in mind, however, that many of the projects are implemented by multilateral organizations such as UNICEF, WFP and the World Bank. These organizations also have administrative charges that are incorporated into project costs, and generally are between 7-13%.

Most projects sampled were delivered on time and on budget. For those that were not, some of the challenges included:

• Delays in DFATD approvals. For example, interviews with donors cited a 3-year delay in approving the SNSF project, which caused great frustration among the other donors in the PSNP coordination group.
• A condensed timeframe to complete projects, which is not an efficient way to achieve development results, especially as it relates to environmental sustainability and climate change. The Beneshangul project, for example, took 2 years to get off the ground. The project cycle was then 3 years, which was considered to be too short a cycle to achieve the desired outcomes.

3.3 Management Factors

Management Factors – RBM, Monitoring and Risk Management

The Ethiopia Program prepared a Country Development Program Framework and country strategy documents for a 5-year Country Program in 2009, along with a logic model and PMF, both of which went through several revisions. The latest version of the logic model is dated March 2010; however, the PMF was not correspondingly updated for example to have the same phrasing of ultimate outcome. Overall, these revisions and inconsistencies presented a challenge in tracking and reporting of results for the Program and the evaluators.

There was a lack of consistency in reporting among DFATD-funded projects. Among the three branches, only bilateral projects completed MSRs, which are a good way of monitoring and reporting. Completion of MSRs for all projects would facilitate portfolio level annual reporting, periodic review and evaluations. At the level of individual projects, logic models and performance measurement frameworks exist and are generally used to monitor and report results on a regular basis. Most projects used some form of results tracking through RBM systems and tools. Projects generally pay attention to project-related risk and mitigation measures. Monitoring and tracking of results has enabled several projects to take corrective actions. In three projects, baselines were not completed or were incomplete. In one case, no targets were identified, although the baseline information was presented.

Paris Declaration Principles

For the Ethiopia Program, the three principles assessed were: a) ownership —developing countries set their own strategies and priorities; b) alignment—donor countries align behind developing country objectives and use local systems for Programming; and c) harmonization—the donors coordinate, simplify procedures and share information to avoid duplication.

Ownership

The GoE has a strong sense of ownership of its policies and strategies and its national flagship programs (PSNP, AGP, SLMP).^{Footnote 26} The GoE has also taken ownership of some of the donor-funded, non-flagship projects, such as SNSF, IPMS, RCBP, MERET, and Sustainable Water Harvesting and Institutional Strengthening in Amhara (SWHISA). However, the GoE has been much less engaged and less prone to take ownership of some of the DFATD-funded NGO and university research projects, even though some of the research projects were carried out by local universities. It was found that government involvement in NGO and research projects was, at times, intentionally limited. The result in these cases was a failure of the government to take ownership, thereby reducing the potential uptake of results and/sustainability.

Alignment and the Use of Local Systems

In the Ethiopia Program, 80% of the overall disbursement was through multilateral organisations; however, it must be noted that 57% of this was through multi-donor trust funds (e.g. for PSNP, AGP) and other PBA-type activities, such as larger government service delivery Programs supported by multilaterals (e.g. WFP’s School Feeding Program and UNICEF’s Improved Food Security for Mothers and Children). In both scenarios, local government systems were used to implement the projects, which included the use of human, technical and administrative resources at various levels of government.

While Canadian university research projects have collaborated with local universities, it was not evident that local systems were being used to scale-up or replicate research results beyond the immediate community. NGO projects, which were smaller and more geographically dispersed than other projects, did not tend to use local government systems; implementation was mostly carried out by the NGOs’ own staff or local staff outside of the public sector.

Donor Harmonization

DFATD staff were actively involved in donor coordination, including various working groups, and leadership roles especially in the agricultural sector (Appendix L).

Donor coordination was found to be particularly strong at the national level but also well-structured at other levels of government. The structure and mechanisms of the donor coordination committee for the GoE’s flagship program, PSNP, was a good model. It included joint planning and monitoring, sharing of information, a six-month rotation for the Chair and periodic joint sector reviews. This high level of coordination was not yet evident in the AGP (the other flagship).

Optimizing Coherence within the Government of Canada

The Ethiopia Program is decentralized with the Country Director based in Addis Ababa, Ethiopia. Recently, the Ambassador has been given overall responsibility for all aspects of the Aid program. As a way of enhancing coherence, interviewees indicated that:

• Multilateral and Partnership program documents were often circulated to Geographic Program Branch staff for comment;^{Footnote 27}
• One person in the Aid Section of the Embassy is responsible for coordinating with other branches in DFATD;
• The Aid Section maintains an overall inventory of all projects underway in all three program Branches; and,
• Recently, a dedicated country-based staff member was given responsibility for tracking the partnership and global Programs in Ethiopia. This has helped the Program team based in Addis Ababa gain a broader and more holistic picture of Canada’s investment in the country, especially while interacting with development partners, ministers and government officials.

Interviews indicated that interactions among the three DFATD branches have improved over time. However, it was noted that there was still a lack of institutional mechanism for discussion between branches on programming in any given country. Many of the interactions on programming were a result of personal initiative or existing relationships.

Between 2008/09 and 2012/13, the largest share of investment for the Ethiopia Program came from the bilateral program, followed by multilateral and then partnership. PBAs had the advantage of permitting broader based engagement with the GoE through donor working groups, but technical assistance in support of PBAs is considered to be important to ensuring success.

The issue of coordination with the DFATD Trade Section is an area that staff believes will have to be addressed as the Program moves increasingly into support for Sustainable Economic Growth. The amalgamated DFATD structure is seen as an opportunity for greater synergy between the development and trade programs in pursuit of sustainable economic growth programming.

3.4 Conclusions

• The Ethiopia Program was relevant to and aligned with the GoE’s priorities and policies, international commitments and Canadian priorities.
• The Program was highly satisfactory in terms of results achieved. It supported key GoE flagship programs, one of which has been cited by the World Bank as a model for a social safety net program. The Program contributed to improved and higher levels of production, improved production practices, better soil and water conservation practices, sustainable land and water management, increased income for farmers, increased food security, decreased food gap, decreases in asset sales, improved nutrition and increased resilience. There is evidence that Ethiopia was not affected as much as other countries in the region during the 2011 Horn of Africa drought.
• The Ethiopia Program strengthened capacities at various levels of government (including MoA personnel) and communities. The Program has been effective in policy dialogue through donor working groups of the national flagship programs. An appropriate enabling environment, including community and government structures, has been put in place for the implementation of the two national flagship programs reviewed in this evaluation.
• Several projects have progressed or taken appropriate steps to ensure sustainability and as a result the evaluation rated sustainability of the program as satisfactory. Capacity building is intrinsic to almost all projects in the Ethiopia Program. Local systems were used to implement several projects, especially the national flagship programs (PSNP and AGP). One of the key challenges to sustainability has been to get the government to set budgets for the operational side. Government commitments to national Programs represented a positive step toward project sustainability, although given Ethiopia’s economic status, several cycles of donor support may be required, especially for the multi-donor flagship programs.
• Good progress was made in terms of gender equality, and environmental sustainability was strongly integrated into a number of projects in the Ethiopia Program;
• Governance was a more recent addition as a cross-cutting theme and, hence, several projects have not addressed it explicitly. In general, projects were efficient. They started on time and stayed within budget. However, the efficiency of projects was affected by delays in approval and start-up, and consequently shortened timeframes for implementation.
• At the level of individual projects, performance management frameworks exist and were monitored and reported on regularly. At the Program level, although the CDPF was prepared for five years, the logic model and PMF underwent several rounds of revisions and were not always aligned. Overall monitoring and reporting at the Program level was weak, and corrective action is recommended.
• The GoE had a strong sense of ownership of its policies and strategies and developed national flagship programs. The GoE drove implementation of these programs with support from donors. The Ethiopia Program used local systems, not only in flagship programs, but also for other smaller and focused projects.
• Interaction among the three DFATD branches programming in Ethiopia increased over time. However, no formal coordination mechanism existed, even in a large and decentralized Program such as Ethiopia’s.
• There was a balanced approach to agricultural development through three main flagship programs, which included food security and nutrition, agricultural growth and development, research, and sustainable land and water management. By funding flagship programs, Canada supported all aspects of the agriculture sector in addition to creating value at different levels of government and in the community.

3.5 Lessons Learned

• i. Working through large government flagship programs in a program-based approach works best when technical assistance is provided to support program implementation. In Ethiopia, having a strong local team of advisors in agriculture, food security, gender and capacity building was a crucial factor for successful Program delivery and management, as well as capacity building.
• ii. Programming through multi-donor pooled funds helped expand the reach of the Program to a larger number of beneficiaries. It also leveraged Canadian policy dialogue through donor coordination and donor working groups.
• iii. The PSNP is a well-designed and strongly performing social safety net system that could be a model for other countries, as is the system of donor coordination that it employs. The PSNP operationalizes the Paris Declaration principles effectively — government ownership, donor harmonization and use of local systems.
• iv. The experience in Ethiopia indicates that individual projects that are not fully supported by the GoE and which are not well integrated into the government budgeting and planning systems have a reduced chance of sustainability and are less likely to be scaled up.
• v. The flagship programs in Ethiopia underline the importance of community involvement and bottom-up planning of major national programs as a way of ensuring that implementation leads to results.

4.0 Ghana Country Program

4.1 Country Context

Ghana achieved lower middle-income country (LMIC) status in November 2010.^{Footnote 28} In 2012, it had a GDP of $41.71 billion, an annual growth rate of 7.9%, and a gross national income per capita of $1,550.^{Footnote 29} While Ghana has attained LMIC status, deep pockets of poverty and inequality exist and major structural challenges remain. Of the total population of 25.37 million, 28.5% live below the national poverty line; this represents a decline from over 50% in the 1990s^{Footnote 30} (Table 4.1). The Global Hunger Index is 8.2 (much lower than Ethiopia). MDGs on child mortality, maternal mortality and access to sanitation remain off-track.^{Footnote 31}

Agriculture has been and remains central to Ghanaian economic growth and poverty reduction.^{Footnote 32} Even though its share in the economy has decreased, agriculture still accounted for nearly 40% of GDP in 2008, and employed approximately 55% of the economically active population.^{Footnote 33} Yet, local production met only 60% of Ghana’s food needs between 2000 and 2010. According to experts, substantial productivity-enhancing investments, including rural infrastructure, irrigation, marketing, extension, and agricultural research and development would be required to improve the situation of small farmers, most of who live in the impoverished northern region and practice subsistence agriculture.^{Footnote 34}

The medium-term national development policy framework, Ghana Shared Growth and Development Agenda (GSGDA) 2010–2013, cites agriculture as the key driver of this development agenda. In turn, the Food and Agriculture Sector Development Policy II (FASDEP II) and the 2011–2015 Medium-term Agriculture Sector Investment Plan (METASIP) serve as the backbone for the development of the agriculture sector and are key components of Ghana’s Shared Growth and Development Agenda.

In addition, the Government of Ghana – Development Partners Compact (2012–2022) was recently established with its key objective to improve the effective and strategic use of Official Development Assistance and other forms of development finance and co-operation in the medium to long term. The Government of Ghana (GoG) and its development partners have agreed to respect the objectives and guiding principles of the GSGDA (2010–2013) and Ghana’s draft Aid Policy and Strategy (2011–2015).

Gender Equality

Ghana ranks 118th of 155 countries in the Gender Development Index (2013).^{Footnote 35} According to the Food and Agriculture Organization (FAO), nearly half of Ghanaian women lived in rural areas and half of those that were economically active worked in the agricultural sector.^{Footnote 36} However, 30% of households headed by a woman are either severely or moderately food insecure, compared with 15% of their male counterparts. Women’s land rights tend to be secondary rights, based on customary law and secured primarily through marriage, which discourages women from growing cash crops, owning animals or accessing credit. Female-headed households are particularly disadvantaged in their ability to feed their children from their own cultivation.

The 1992 Ghanaian Constitution sets out the country’s commitment to gender equality and the empowerment of women, stipulating that women and men are equal and that neither shall be discriminated against (Article 17). The national structure for the advancement of women and gender equality has been in place since 1975, and is currently the responsibility of the Ministry of Women and Children Affairs. However, weak capacities for inter sectoral coordination and advocacy limit the ability of the Ministry of Women and Children Affairs to implement gender-related policies.

Environmental Sustainability

Primary forests in Ghana have been reduced by 90% in the last 50 years with 26% of the country’s forest cover being lost between 1990 and 2005.^{Footnote 37} This has caused severe land degradation, erosion and siltation, and reduced the fertility of already degraded soils. Ghana’s first Environment Policy (1995) identified the Environmental Protection Agency (EPA) as the lead agency to respond. The National Action Program to Combat Drought and Desertification has been in place since 2002 and Ghana is now developing a National Climate Change Policy Framework as part of the work plan of the cross-sectoral National Climate Change Committee, hosted by the Ministry of Environment Science and Technology and Innovation (MESTI).^{Footnote 38} However, the principal challenge confronting environmental management processes in Ghana is the ineffective enforcement of existing policies and legislation. According to the World Bank, “poor management of mining, fisheries, forestry and wildlife —along with water and air pollution— is costing Ghana 10% of its GDP each year.”^{Footnote 39}

Governance

Ghana is a vibrant democracy with active political parties, a history of peaceful political transition, as well as freedom of speech and association. Ghana ranked first in Africa in four of nine criteria set out by the World Rule of Law Index: limits on government powers; order and security; fundamental rights; and effectiveness of criminal justice system. The same index ranked Ghana 37th overall among 99 countries.^{Footnote 40} However, as in many countries, corruption is identified as a problem by local experts. Ghana scored 46 and ranked 63 out of 177 countries^{Footnote 41}, one of the best among African countries, on the Corruption Perceptions Index.

The Local Government Act of 1984, enacted in 1988, initiated the process of decentralization in Ghana. It has progressed at varying pace across sectors, and the coordination of the process by the central and district levels is still lacking.^{Footnote 42}Systems at both the national and local level are weak and often insufficiently accountable to citizens. The Ministry of Food and Agriculture (MoFA), one of the largest and most decentralized ministries, lacks the institutional capacity to implement policies at all levels and to coordinate with other ministries, such as the Ministry of Local Governance and Rural Development (MLGRD).

Development Programming in Ghana

Canada ranked between the sixth and third largest donor to Ghana from 2009 – 2013. Other major donors included the United Kingdom, the World Bank’s International Development Assistance fund (IDA), the United States, the African Development Fund, and Japan.

Canada participates in the Multi-Donor Budgetary Support (MDBS) program with 10 other donors and the Government of Ghana to improve the effectiveness, transparency and accountability of country systems, as well as to increase coordination and harmonization among donors. Expected results through the MDBS include: improving budget planning for poverty reduction; delivery of basic services (access to education and health care for girls and boys, and other basic services); strengthening technical and management expertise; and improving public financial management and oversight.^{Footnote 43}

The thematic focus of the Ghana Program as set out in the October 2009 Country Strategy was food security (agronomic and post-harvest activities, food assistance and nutrition, and research and development). It also included securing the future of children and youth (nutrition and access to safe water). Much of this strategy focused on supporting efforts by the GoG to implement its development agenda and policies, particularly in the agriculture sector.

A key strategy for food security was to provide sector budget support to the MoFA in order to improve institutional and financial management with the ultimate goal of improving the delivery of goods and services to the ultimate beneficiaries. Accordingly, almost two-thirds of disbursements were made through program-based approaches. In effect, the Program provided general budget support to the GoG at three levels - general budget support, sector budget support to the Ministry of Food and Agriculture and budget support to the district assemblies. The Program made these disbursements based on a set of agreed upon outputs (or triggers) identified in the Finance Ministry and MoFA’s annual plans. The remainder of the programming was implemented through multilateral agencies and NGOs.^{Footnote 44}

Overall Development Portfolio

DFATD disbursed $499.83 million in Ghana between 2008/09 and 2012/13. Approximately half of this funding was for food security ($241.07 million). Some 82% of these funds went to initiatives pertaining to the sustainable agricultural development pillar of the Food Security Strategy, including agronomic and post-harvest activities (52.1%), followed by enabling activities and support services (30.2%). Food assistance and nutrition received 14.5% of overall food security disbursements with 3.2 % devoted to research and development (Figure 4.1).^{Footnote 45}

Figure 4.1: Food Security Disbursements in Ghana (2008/09 to 2012/13), millions

Source: Compiled from DFATD database

The main delivery channel for the Ghana Country Program was the former Geographic Programs Branch with 85.1% of disbursements. The former Multilateral and Global Programs Branch accounted for a further 5.1% of disbursements, and Partnership with Canadians Branch 9.8%. (Appendix E, Figure 4).

Figure 4.2: Disbursements in Ghana by DFATD Delivery Channel (2008/09-2012/13), millions

Source: Compiled from DFATD database

The major disbursements were made to government bodies (70.4%) and multilateral organizations (22.4%).^{Footnote 46} The remaining 7.5% was through civil society and other organizations (Appendix E).

4.2 Findings – Development Results

Relevance

The evaluation found Canada’s development efforts in Ghana to be highly satisfactory in their responsiveness to the GoG’s strategies and plans, such as the Ghana Shared Growth and Development Agenda 2010–2013 that featured agriculture. Programming also aligned well with the GoG’s second Growth and Poverty Reduction Strategy (GPRS–II), from which the Food and Agriculture Sector Development Policy emanates. GPRS–II identifies Ghana’s priority needs as: accelerated growth through private sector development (especially agriculture); accelerated human resource development; and improved good governance and civic responsibility. Canadian development programming was also designed to support the National Decentralization Action Plan/National Decentralization Policy Framework.

At the time of this evaluation, Canada was the only donor in Ghana providing budget support at three levels: a) general budget support (MDBS), b) sector budget support to the Ministry of Agriculture, one of the first ministries in Ghana to decentralize, and c) budget support to the district assemblies (through two key initiatives, the District Development Facility (DDF) that built upon another initiative Canada helped pilot in the northern regions, the District-wide Assistance Project (DWAP).

The budget support provided to all three levels allowed the Ghana Program to garner an overall perspective of the strengths and weaknesses in key national policies, plans and systems down to the district level. It also allowed the Program to work closely with the GoG to identify key issues and support initiatives designed to fill identified gaps at all levels. A final link in the chain was to bring in communities through smaller scale interventions in agriculture, nutrition and research in the northern regions to improve household incomes and nutrition, as well as to include them in the district level decision-making process.

The mid-term assessment of the Ghana Program’s Sector Budget Support to MoFA between 2009 and 2013 indicated that the approach taken allowed for an inclusive and comprehensive process involving all stakeholders in the agriculture sector.^{Footnote 47}

The focus on the three northern regions has been also highly relevant to the needs of the population as was made evident by the establishment of the Savannah Accelerated Development Authority to address the large developmental differences between the southern and northern parts of Ghana. The three northern regions suffer from high levels of poverty (63% in the north compared to 20% in the south), as well as food insecurity and malnutrition.^{Footnote 48} According to WFP data from 2013, 36% of children under five years of age in the Upper East Region suffer from chronic malnutrition, and 34% of the people in the Upper West Region are food insecure.^{Footnote 49}

Effectiveness

Overall, the evaluation found that the level of achievement and/or progress toward results was satisfactory in sustainable agriculture development, although with scant progress in post-harvest. Several key results were achieved in nutrition, and research and development was also satisfactory.

Agronomic and Post-Harvest Activities

The Ghana Program worked to achieve the following intermediate outcome: “Improved nutritional practices and more productive and sustainable agriculture practices amongst smallholders, with a particular focus on women in Northern Ghana.” Data sources corroborated MoFA’s progress in the implementation of policies, strategies and plans. One key performance measure was the change in MoFA’s ability to achieve the annual triggers,^{Footnote 50} which increased from 83% in 2009 to 90% in 2013.

The Ghana Program’s technical assistance aided the development of the Medium-term Agriculture Sector Investment Plan as a support to the implementation of the Food and Agriculture Sector Development Policy. This support contributed to an improved food and agriculture policy (including the drafting of seed and fertilizer policies), improved financial management, a new performance measurement and monitoring framework to measure progress towards implementation, as well as the integration of key issues, such as environmental sustainability and gender equality.

Evidence showed that sector budget support by the Ghana Program enabled the implementation of MoFA’s four key programs, which, in turn, led to an increase of 11.9% in the number of farmers accessing fertilizers, improved seeds, mechanized services and agriculture technologies between 2009 and 2010 (gender disaggregated data was included). In 2013, MoFA reported that 326 different technologies (e.g. crop production, livestock, fisheries and other technologies related to agriculture) were disseminated to over 1.7 million farmers, representing a 13.3% increase from 2010. Of these, 4,062 received training in agro-processing technologies, 83% of whom were women.^{Footnote 51}

It is evident however from the 2012 annual trigger assessment by an external reviewer^{Footnote 52} that little progress has been made to ensure a greater level of coordination for agriculture-related activities, though the assessment noted some efforts in that direction through the Food Security and Environment Facility, implemented by the MLGRD sought the expertise of district MoFA staff. In addition, the 2011 assessment of Ghana food and agriculture policy observed that the implementation of a number of critical action plans including Sustainable Land Management, Good Agricultural Practices, Improving Agricultural Commodity Standards and Reducing Post Harvest Losses requires not only adequate funding levels but also improved coordination between National Directorates and also ownership at the district and regional levels.  

Another area where MoFA has yet to make sufficient progress is in increasing the number of farmer-based organizations (FBO) and bolstering their cohesiveness to improve access to services. Efforts to link to commodity networks and markets were generally also inadequate.^{Footnote 53} According to the GoG data, the number of functioning FBOs has decreased between 2010 and 2013, as have the number of FBOs accessing financial services and marketing information (Table 4.3). All this highlights the extent of the work that remains to achieve better returns for farmers.

Little to no progress had been made on the number of agriculture extension agents (AEA) during the evaluation period. In 2013, the AEA to farmer ratio in Ghana remained 1:1,500, far below the recommended ratio of 1:500.^{Footnote 54} There were only 2,068 agricultural extension workers in Ghana as of 2012, while an estimated 3,909 are required. Of the former, 13% are women but the ratio is even smaller in the northern regions. In 2011, the percentage of women AEAs was 6.1% for the Northern Region, 6.6% for the Upper East Region and 9.3% for the Upper West Region. More recent figures for the northern regions were not available at the time of the evaluation.

The District Development Facility was established in 2008 with $80 million in funding from Canada, as well as support from DANIDA, Germany’s KfW and the Agence Française de Développement. The bulk of DDF funding was direct budget support to all 170 districts across the country. The ultimate expected outcome of the DDF is shared and sustainable economic growth and an improved quality of life for Ghanaian women and men, girls and boys. Although 20% of Canada’s support to the DDF was allocated to agronomic and post-harvest activities, it is not clear from available documentation what specific results were achieved. Among the 888 basic service infrastructure projects that had reached varying stages of completion by late 2011, less than one percent were agricultural. More recent figures on infrastructure projects were not available at the time of the evaluation.

Nutrition

Canada invested $33.6 million in initiatives to promote improved nutrition^{Footnote 55} in Ghana. This included support for the efforts of multilateral organizations, such as the WFP and UNICEF. Multi-million dollar projects were awarded to these two organisations to strengthen the policy framework and implementation of programs to improve nutrition – particularly in the three northern regions.

UNICEF and WFP’s initiatives in the three northern regions were coordinated so as to redress malnutrition using a multi-sectoral approach. The WFP featured education, health, nutrition, and gender equality issues in its nutrition programming, while UNICEF engaged both the Ghanaian Health Service and the MoFA to bolster nutrition, agriculture, and women’s economic empowerment to combat malnutrition.

Support from the Ghana Program allowed UNICEF to train community-based volunteers and trainers in community management of acute malnutrition. In 2011, 4,912 children with severe acute malnutrition were admitted - a coverage rate of 53%. Of these, 3,510 children, or 71% were cured. Severe acute malnutrition rates fell by 50-58% between 2008 and 2011 in the northern regions, compared to a 36% reduction at the national level. The reduction was attributed to the greater involvement of trained volunteers that enabled broader coverage, combined with improved community case management of common childhood illnesses such as malaria, diarrhoea and pneumonia. The number of community management of acute malnutrition centres increased to 41 districts, representing 50% of districts in the focus regions. In many of these districts, the cure rate is above 60% and the death rate is below 5%.

Community volunteers treated 4,000 children afflicted by diarrhoea with zinc and oral-rehydration salts. In addition, zinc supplements will become part of the diarrhoea treatment protocol within the Child Health Policy of the Government of Ghana, and it is now included among the essential drugs under the Ghana National Health Insurance Scheme. This new GoG policy and practice will ensure the sustainability of this critical component.

Funding from the Ghana Program also allowed UNICEF to support the Ghanaian Health Service to implement improved nutritional surveys to complement food security monitoring and its own health information system. The three northern regions can now monitor nutrition indicators using internationally recommended methodologies. Building on theseachievements,UNICEF has recently focused on scaling-up nutrition interventions to disadvantaged regions, improving nutrition-related policy and coordination, and developing and improving a nutrition information and surveillance system.

TheWFP Country Program (2006-2010), which received Canadian funding, alsohelpedtoreduce the level of malnutrition among pregnant and lactating women and children under-five years of age. It provided mothers and children under the age of two with take-home rations of maize meal or corn soy blend, iodized salt and vegetable oil. Children aged two to five received on-site meals of corn-soybean blend, rice, iodized salt, vegetable oil and sugar.

WFP was also funded to work on protecting the nutritional status of at-risk groups in northern Ghana, including children under five, pregnant and lactating women and people living with HIV/AIDS. Follow-up surveys showed a decrease in the prevalence of acute malnutrition among children in the three northern regions from 14.7% in the baseline survey in 2010 to 6.7% in 2013. The WFPtargeted 133,000 primary school children in 595 schools in the three northern regions through its School Feeding Program. Although the WFP did not collect nutrition outcome data for this intervention, it did report achievements over the period of the program (2007-2010) in terms of increased enrolment, which nearly doubled from 2006, while attendance improved from 89% in 2007 to 96% in 2009. The take-home ration component made a significant contribution to the improvement of girls’ participation rate in education in some of the districts with the highest gender disparity in Ghana.

An evaluation of the WFP Program indicates that it has produced significant results in improving the gender parity index across two of the three project regions of Upper West and Upper East Regions. These two regions have recorded a Junior High School Gender parity of 1.0, compared to 0.92 for the national level.

Research and Development

Approximately $7 million was disbursed to research and development activities by the Ghana Program during the evaluation period. MoFA and the Council for Scientific and Industrial Co-operation (CSIR) signed a Memorandum of Understanding to fund 17 research projects. The priority areas were identified through the Research-Extension Linkage Committees. However, delays in the release of funds by MoFA affected implementation and impeded results. Reporting on specific research project results was very limited at the time of the evaluation.

Policy Dialogue

Policy dialogue in the Ghana Program was designed to create or enhance the enabling environment for food security (sustainable agriculture development, nutrition and research and development), as well as governance (public sector reform, decentralization, gender equality and environmental sustainability). The Program made a significant contribution to enabling conditions in all these areas through various mechanisms, including the trigger system for the release of funds for budget support, the working groups on agriculture, gender and nutrition, and the joint sector reviews. The tools all proved useful in increasing inclusive and comprehensive dialogue with all agriculture stakeholders.

Furthermore, stakeholders noted that the broad range of Canada’s development programming strengthened its position of influence in policy dialogue with the GoG and has enabled access to key ministries. Indeed, Canada`s unique position of influence in the agriculture sector was acknowledged by and, at times, leveraged by other donors. Canada’s leadership on gender issues was also widely acknowledged among donors and key national partners.

Discussions with stakeholders revealed that a mix of delivery modalities that combined program-based approaches with projects and other modalities, such as technical assistance, created an environment more conducive to policy dialogue. Smaller projects provided an opportunity to demonstrate positive results on a small scale, which could then be replicated on a larger scale by the government or other donors. A good example of this was when Canada led other donors in a joint review to northern Ghana to see the District–Wide Assistance Project underway. This visit and ensuing discussions led to the adoption of the District Development Facility model at the national level.

One key achievement due, in part, by the Ghana Program’s advocacy efforts was that the allocation of the discretionary budget of the GoG to the MoFA, District Agricultural Departments and the Ministry of Fisheries and Aquaculture remained at 2.12% of the national budget. A donor condition for budget support was that Ghana’s own contribution to the sector should remain constant, at a minimum. Canada also played a significant role in ensuring that MoFA transferred funds to its district departments. By helping to track the budgetary allocations from MoFA to the regions, the Ghana Program could ensure that a certain percentage of MoFA’s budget allocation was getting to the sub-national levels. This was aided by the preparation and publication of audits on the accounts of all district authorities by the Ghana Audit Service.

In the nutrition pillar, the evaluation found that Canada played a significant role alongside the Renewed Efforts against Child Hunger (REACH) initiative and the Scaling up Nutrition (SUN) movement in engaging the GoG to develop a national nutrition policy. Canada encouraged the inclusion of food security in the nutrition policy and provided technical assistance in this process. As a leading donor to the agriculture sector, Canada influenced the development of the Country Cooperation Framework for the New Alliance for Food Security and Nutrition and successfully engaged the Ministry of Food and Agriculture to implement the National Nutrition Policy, making this a multi sector effort.

Canada has also advocated closer relationships between scientists and farmers, with scientists becoming more demand driven and responsive to farmers’ needs. Discussions with stakeholders indicated that the Ghana Program had a positive influence on reviving the Research-Extension Linkage Committees (RELCs) and the responsiveness of scientists to farmers. However, stakeholders also noted that such interactions have been dependent on donor funding in the past, and that when donor funding ended, the activities of the RELCs declined - a risk that policy dialogue could help to address.

Initiatives supported by the Ghana Program also contributed to an enabling environment at the district level. For example, the Community-driven Initiatives for Food Security project (CIFS I & II) helped bring community-level priorities in agriculture and food security to the district level. It also strengthened decentralized government processes in planning and managing food security initiatives. The National Development Planning Commission recognized the importance of the CIFS approach by including the requirement that community action plans must be supportive of the district medium-term plan in its own guidelines.

Sustainability

Overall, Canada’s programming in Ghana has been satisfactory in terms of progress toward achieving sustainability. Although Ghana has been considered a lower middle-income country since 2001, development challenges specific to agricultural growth, food and nutrition security in the three northern regions remain and likely require continued support for long-term sustainability.

One key factor has been Canada`s willingness to use local systems and channel funds through government mechanisms, including budget support and through other government-managed initiatives. This approach contributed to greater GoG ownership. Capacity building is another important contributing factor towards sustaining the results achieved in Ghana and it has been a major component of programming. In addition to strengthening the capacity of government personnel and systems, it has also assisted farmers, including women, community members and health extensions workers. Capacity development carried out through the Tackling Malnutrition in Ghana project has meant that three northern regions are now able to monitor nutrition indicators using internationally accepted methods.

The evaluation identified the following challenges for sustainability:

• There was some indication that MoFA may have successfully met its triggers only when placed under a certain amount of pressure to do so. If so, this has potentially negative implications on the sustainability of the programs being implemented;
• The GoG has, at times, been unable to integrate project elements into the work of relevant departments for want of financial and human resources. Staff turnover in GoG ministries has been relatively high and poses a challenge for current and future projects and programming; and,
• The need for continued external support for large initiatives from international donor partners persists.

It was too early to assess the sustainability of the 17 CSIR research projects funded by MoFA as part of the Ghana Program’s sector budget support (SFASDEP). Although initiated in 2009, MoFA’s inability to release funds for research hindered progress from 2009 to 2011. As well, ensuring long-term funding from the GoG for research and development and the research-extension linkage committees remains an issue.

Cross-cutting Themes

The evaluation assessed how DFATD-development’s three cross-cutting themes — gender equality, environmental sustainability and governance — were integrated into development programming over the period.

Gender Equality

The following are some key results achieved in the initiatives that were supported by the Program.

• Budget support enabled increased allocations to gender -sensitive activities by 83.4% from 2009 to 2012;
• Strengthened monitoring capacities led MoFA to collect and include sex disaggregated data in its reports since 2010;
• The capacity of the Environmental Protection Agency to deliver gender sensitive environmental programming was strengthened through training and the production of gender sensitive training manuals on sustainable agricultural practices, which were distributed to agricultural extension agents;
• Gender Desk Officers were placed in District offices;
• Canada was a lead member of the Gender Equality Sector Working Group and the Agriculture Sector Working Group where they took the opportunity to raise gender equality issues with its partners; and,
• DFATD also contributed to ensuring that the Land Administration Reform process be responsive to gender issues in land rights.

There is scope to scale up gender mainstreaming in MoFA’s programs and activities, as well as in those of the MLGRD as it moves forward in decentralized government through initiatives such as the District Development Facility.

In terms of policy dialogue, DFATD has been a very active of member of the Gender Equality Sector Working Group and a leader in this area. Canada was co-chair of the Gender Equality Sector Working Group several times during the evaluation period, and as leader of the Agriculture Sector Working Group raised gender issues with its partners. Representatives from the GoG and the donor community acknowledged this leadership on gender equality in agriculture. The trigger mechanism was used in the sector budget support funding for the integration of gender analysis into the planning, budgeting, implementing and monitoring processes and the Ghana Program provided MoFA with technical assistance on gender through a local gender specialist.

As a result, DFATD programming in Ghana contributed most notably to the integration of gender equality issues and gender mainstreaming strategies into policy processes of MoFA. The Mid-Term Assessment (MTA) of the budget support to MoFA found, for instance, that the Gender in Agricultural Development Strategy was updated in 2009 and that an accountability framework was adopted with Canada’s support to assist in removing barriers that inhibit women from playing active roles in the agricultural value chain.^{Footnote 56} The MTA reported that the budget support allowed for increased budget allocated to gender sensitive activities by 83.4% from 2009 to 2012.

However, progress in implementation of the Gender in Agricultural Development Strategy has been slower than expected and some key stakeholders within the Ministry have not been able to play their role fully. For instance, the Women in Agriculture Development Directorate (WIAD), which is one of seven directorates in MoFA, has been undertaking key programming activities to improve nutrition. Despite the importance of this role given national priorities and strong commitments from Canada and sector support to MoFA, the WIAD Program funding from MoFA has been only about $200,000 per year.

The strengthening of monitoring capacities also led the MoFA to include sex-disaggregated data in its reports as of 2010, although stakeholders indicated a shortfall in capacity to analyze or use the data in decision-making and implementation. Similarly, the key decentralization initiative implemented by the Ministry of Local Government and Rural Development, to which Canada contributes, has fallen short of expectations in certain regards. An external review of the DDF concluded that, “good gender mainstreaming practices that go beyond the establishment of Gender Desk Officer positions are yet to be identified and up-scaled”, and that inadequate capacities along with limited financial resources and low accountability requirements combine to hinder the implementation of mainstreaming initiatives. The review pointed out that the national policy framework, the National Development Planning Commission planning guidelines and the functional organizational assessment tool gender-related performance measures and indicators provide the mandate and impetus to identify interventions to progressively attain gender equality. As power and resources shift downwards to local governments, gender mainstreaming efforts, including gender responsive budgeting, must be mainstreamed into the planning processes at those levels, as well.

Environmental Sustainability

The Ghana Program made satisfactory progress towards environmental sustainability. The evidence gathered showed that all actors (governmental, international or non-governmental, partners and agencies) implementing Canadian-supported programs established processes for integrating environmental sustainability. The Ghana Program made use of a locally engaged environment specialist to advise the MoFA and other partners on the integration of environmental considerations into project design. DFATD also engaged management at the MoFA to integrate environmental sustainability into its plans and projects, leading to improvements in its performance on environmental sustainability and targets in 2012 and 2013. MoFA was encouraged to locate the Environment, Land and Water Management Unit within the Crops Services Division and, although the desired change was not achieved, environmental issues were given more prominence at the Ministry as a result.

A trigger mechanism was used in the sector budget support funding to integrate environmental sustainability into planning, budgeting, implementation and monitoring processes. With technical assistance from the Ghana Program, MoFA developed and officially adopted the Agricultural Land Management Strategy and Action Plan (2009-2015) in 2009. However, its implementation was slowed due to lack of committed MoFA resources to support planned activities. MoFA has met the trigger since 2011.

Initiatives supported by the Ghana Program in the northern region contributed to a slowing of land degradation and soil erosion and encouraged climate-friendly food production in intervention communities. The CHANGE project made links with district assemblies by training district planning officers on community-based climate change adaptation. CHANGE has strengthened FBOs in the project villages by improving food security for smallholder farmers using climate-smart practices and techniques. Other projects contributed to increased food production and alternative livelihood activities in an environmentally sustainable fashion. Two or three of the pilot initiatives were identified as very promising for scale-up.

Governance

As noted earlier, Canada’s programming in Ghana included a focus on strengthening of governance at the national, district and local levels and on creating linkages between the different levels to ensure that budgets flow down to the district level and service delivery to communities improves.

The 2013 Annual Review of the Multi-Donor Budget Support concluded that progress was made in terms of the principles of implementation of the Ghana Shared Growth and Development Agenda, sound budgeting and public finance management systems and good governance. For example, in 2012 the GoG met 19 of 25 targets and six of 11 triggers.^{Footnote 57}

However, stakeholders indicated that two major issues slowed progress. The first was the election of a new government in 2012, which led to the appointment of new staff to the MDBS Secretariat, all of whom required a period of adjustment. The second was an unexpected 11.8% fiscal deficit in 2012, forecasted at 6.7% by the GoG as late as November, which triggered delays in disbursements from donors. In response, the GoG enacted measures to control of wages, mobilise revenue, manage debt, remove subsidies and tighten fiscal controls.

An independent review showed that the DDF and DWAP were budgetary support mechanisms that helped decentralized local governments in Ghana and provided good value for money.^{Footnote 58} The report noted that these initiatives could be considered a model of best practice in decentralized government support to programming. The Ghana Program, through DWAP and DDF, helped build basic infrastructure (community health facilities, classrooms, etc.). While much remains to be done in the decentralization process in Ghana, DWAP and DDF have contributed significantly to enhancing the effectiveness and accountability of government and civil society to address the needs and aspirations of the Ghanaian population (intermediate outcome). These initiatives also provide an enabling environment for future initiatives related to food security.

The DDF adopted the functional organizational assessment tool (FOAT) to assess a variety of performance indicators, which functions like a trigger system to release funds to the districts. FOAT provides an incentive to metropolitan, municipal and district assemblies to improve performance by tying DDF annual funding levels to scores on performance measurement indicators. Institutionalizing FOAT contributed to the decentralization process and facilitated the implementation of the National Decentralization Action Plan. Additionally, the annual FOAT has been a nationwide monitoring and evaluation activity that provided comparative data on the 216 MMDAs in Ghana.

The community action plans provided a means to incorporate the non-food security needs of communities into medium-term development plans. Eighty percent of communities expressed the belief that they would be able to develop a new community action plan. The communities increased their abilities to lobby district assemblies for remedial action to solve problems.

The contribution to the Land Administration Project (LAP 1) enabled civil society representatives to take a seat at the table at an early stage to advocate for increased awareness of land rights issues and monitoring of implementation of the reform. Canada also contributed by focusing on sensitizing the administrative commission and judiciary on gender, environmental, and social protection (access to livelihoods) issues associated with land rights to ensure that they would be fully built into the reform.

Efficiency

The Ghana Program’s efficiency was assessed based on three factors: 1) whether initiatives were on time and within budget, and 2) an assessment of a few sample projects as to whether they achieved an appropriate return. As well, the evaluation looked at the overall costs required to manage the Ghana Program.

The overall disbursement for Ghana during this period through the three DFATD delivery channels (bilateral, multilateral and partnership) was $499.83 million. The total share through the bilateral channel was $425 million with an average $85 million a year. Based on these figures, it cost DFATD $1 million (including salaries, program support unit (PSU) and operating and management funds (O&M)) to disburse and manage $38.48 million of programming in Ghana. This was equivalent to 2.6% of the yearly disbursement by the bilateral program, which appears reasonable.

Key issues with respect to efficiency include:

• The efficiency of the Ghana Program in the approval of programs and projects has not been optimal and led to a delayed start for some projects, which increased pressure on partners to deliver results within tight timeframes or prevented completing activities as planned (e.g. the research projects). A contributing factor was that while the Program started in 2009, country strategy documents, including a logic model and performance measurement framework went through several revisions and were only completed in December 2012.
• More than 50% of disbursements were made through budget support and/or through the GoG. Therefore, the capacities and efficiencies of governmental organizations had an impact on efficiency. Disbursements based on the trigger system provided an incentive for government partners to be more efficient. However, the Mid-Term Assessment of Canada’s budget support to MoFA questioned whether the number of triggers that MoFA needed to fulfill for disbursement of funds from DFATD was appropriate given the transaction costs for both parties. The MTA suggested that perhaps the large number of triggers (12 in 2011; 11 in 2013) may be a distraction from daily activities of the Ministry and lead to inefficient use of staff time. It suggested that a more focused approach for Canadian support to the MoFA may be required and recommended a smaller number of triggers based on best practices.
• A related issue raised in the MTA is the relatively high number of disbursement tranches made during the life of budget support to the MoFA. The Assessment showed that whereas the Ghana Program made 10 disbursements over the course of the budget support project, the average number of tranches is 3 for other donors.

4.3 Findings - Management Factors

Management Factors– RBM, Monitoring and Risk Management

The Ghana Program has used appropriate instruments to manage and monitor the initiatives it contributed to, including budget support to the government of Ghana. Risk management was also satisfactory. The trigger system proved a useful tool to monitor progress made by the GoG at different levels and mitigate risk, as disbursements were made upon the achievement of milestones. The Ghana Program also made use of logic models, performance measurement frameworks and annual progress reports to monitor progress towards results, although there was inconsistency in the quality and detail of reporting. Further, while project and sector team leaders often took corrective action, these steps were not always documented. Consequently, it was difficult to assess how monitoring information had been used for project and program management decision-making.

A management challenge faced by the Ghana Program was delay in finalising key strategic planning documents. For example, the CDPF for the Ghana Program 2010–2014 was only finalized in January 2010. Country strategy documents were also created for Ghana, including a logic model and PMF, both of which went through several revisions and were only finalised in December 2012. Whereas earlier versions of the logic model and the PMF were not aligned, the latest versions were. The latest PMF lacked a baseline for many indicators, which impeded tracking progress or knowing with certainty the level of results achieved. Overall, the multiple revisions and inconsistencies presented a challenge in tracking and reporting on results.

At the level of individual initiatives, logic models and PMFs existed and were generally used to monitor and report results on a regular basis. Considerable effort was deployed to ensure that partners’ capacity was strengthened to manage for results. Most initiatives used some form of results tracking through RBM systems and tools and took risk mitigation measures. A review of progress also took place during periodic meetings between development partners and the MoFA through monthly Agricultural Sector Works Groups and the METASIP Steering Committee, especially for Canada’s sector budget support. However, the use of result tools and the GoG’s performance measurement systems for sampled projects in Ghana was mixed. A few projects focused only on activity reporting and experienced difficulties in demonstrating the link between activities and higher-level results.

As mentioned above, budget support used triggers for the release of funds. However, most triggers were process oriented, which made the linkages between what was achieved in the Ministry and the results on the ground tenuous

Lastly, reports from multilateral agencies showed reporting of activities and outputs rather than results. This has made the collection of evidence on results a challenge.

Paris Declaration Principles

The three principles that were assessed were: a) ownership—developing countries set their own strategies and priorities; b) alignment—donor countries align behind developing country objectives and use the local systems for Programming; and c) harmonization—the donors coordinate, simplify procedures and share information to avoid duplication.

Overall ownership of the GoG was evident within budget support projects. The Program channelled 70% of the disbursement during 2008/09–2012/13 through government or government agencies and 22% through multilateral organizations. This form of support also helped to build capacities and confidence in the local systems at various levels of government.

Multilateral agencies worked closely with the government and implemented largely through its systems, thereby contributing to government ownership, evidenced by the increased in budget allocations for these initiatives. The nutrition programs implemented through WFP and UNICEF were done using local systems. There was no strong evidence of the use of local systems by Canadian NGO partners, as implementation was generally by their own staff or that of a local NGO. Community ownership of NGO projects varied and was specific to the nature of the project, the NGO and the area.

In view of improving dialogue with the GoG for future programs, the Ghana Program completed a scoping study on the readiness of the MoFA and donors to undertake a sector-wide approach, which would serve as the basis for a discussion and reflection on how they would move forward with an agricultural sector-wide approach. More recently, Canada took the lead in organizing the first ever “donors trip” to northern regions of Ghana. This is being emulated by USAID in 2014.

Optimizing Coherence within Government of Canada

The Ghana Program was decentralized with the Country Program Director based in Accra, Ghana.

Canada’s investment in Ghana has been made through three different branches — bilateral, multilateral and partnership. Between 2008/09 and 2012/13, 85% of the investment for the Ghana Program came from the bilateral program. Funding to Canadian partners accounted for 10% of overall investment. Partnership programming in Ghana is the highest of any country where DFATD-development is making investments. The proportion of multilateral funding represented only 5% of the Ghana Program. There was a general consensus among DFATD respondents the Ghana Program had the right mix of investment from the different branches.

Recently, a dedicated country-based staff member has been given the responsibility for tracking partnership and global programs in the country. This has helped the bilateral Program team based in the embassy in Ghana to have a broader and more holistic picture of Canada’s overall development investments in the country - especially useful when interacting with development partners, ministers and government officials. A recent meeting between all Canadian NGOs working in Ghana was found to be a positive step towards coordinating and leveraging Canadian programming in the country.

The new, amalgamated DFATD structure was viewed as an opportunity for trade and development to leverage each other’s strengths. It was also noted during discussions that, in the past, more interactions occurred between development and foreign affairs staff at the mission level than at the corporate level in Canada. The new DFATD structure thus provided a formal mechanism for greater policy and program coherence.

Scaling-up of effective partnership projects by the bilateral program was found to be a useful strategy (e.g. Canadian Hunger Foundation’s Expanding Climate Change Resilience in Northern Ghana project). It should be noted that both the bilateral and multilateral programs in Ghana funded multilateral organizations, like WFP. Similarly, Canadian partners were funded through bilateral and partnership channels.

4.4 Conclusions

• The Ghana Program is highly relevant and well aligned to the priorities of Ghana, particularly efforts to achieve the MDG 1 target of halving hunger by 2015. Canada’s focus on the three northern was highly relevant to the needs of the population given the disproportionately high levels of poverty, lack of access to basic services and level of malnutrition compared to other regions. The additional focus on water and sanitation there was an appropriate complement in achieving improved nutritional outcomes.
• The Program was also aligned with Canada’s G8 commitments most relevant to the Ghana context and engagement in policy dialogue made a significant contribution to enabling conditions for food security, public sector reform and decentralization.
• The level of results achieved was satisfactory, working mainly through budget support, which accounted for two-thirds of the disbursements in Ghana. Canada contributed to the development of key strategies and plans for agriculture development and to their implementation, which led to improved crop and livestock production.
• Government capacity was bolstered in the areas of public sector reform, financial management and in agriculture. More specifically, the Program contributed to an improved food and agricultural policy, improved financial management, a new performance management framework, and the integration of environmental sustainability and gender equality into these documents and tools.
• Given the emphasis of the Ghana Program on sustainable agricultural development, the lack of a clear long-term Program strategy or initiatives to promote value chain development or the development of farmer-based organisations should be addressed.
• A notable Canadian contribution has been its support to the processes of decentralization. Targeted interventions at the district and the community level have served as a model for decentralization for Ghana in the years to come. 
• There was satisfactory progress towards achieving sustainability. Institutional strengthening of MoFA and efforts to use local systems effectively and channel funds through the government were contributing factors. Ownership by the GoG was most evident with regard to general budget support, sector budget support and the District Development Facility. Nutrition initiatives implemented through the WFP and UNICEF in the northern regions also used local systems. However, GoG operational budgets need to be secured for these programs to ensure long term sustainability.
• DFATD programming made a notable contribution to the integration of gender equality issues and gender mainstreaming strategies into policy and processes of MoFA, including monitoring and evaluation frameworks.
•  Donor agencies, including DFATD, established processes for integrating environmental sustainability. There was ongoing support provided to Ghana’s Environmental Protection Agency to implement a National Action Plan on desertification. Some projects in the northern regions contributed on a small scale to slowing land degradation in the intervention communities and improving environmentally sustainable food production practices.
• While delays in the DFATD approval process put additional pressure on partners to achieve results in a shorter timeframe than planned, the efficiency of the Ghana Program was satisfactory overall, both in terms of cost to manage program activities and the level of results achieved. Still, the broad focus of DFATD budget support to MoFA with the number of triggers and disbursement tranches likely increased transactions costs for both partners, and may have unduly taxed human resources within the Ministry.
• Overall, the Program has exerted adequate efforts in managing for results and mitigating risks. Logic models and performance measurement frameworks existed and were generally used to monitor and report results on a regular basis.

4.5 Lessons Learned

• Initiatives implemented by the government or in close concert with it lead to increased ownership, capacity and sustainability.
• Implementing innovative pilot projects that demonstrate results at the community and district levels and that have a good potential for scaling up can effectively complement program-based approaches. The District-wide Assistance Project is a good example: after demonstrating its effectiveness in the northern regions, it became the model for multi-donor support to the implementation of the District Development Facility nationwide by the MLGRD.
• In the context of a decentralized governance system, it is important for different ministries to coordinate their activities at all levels to provide better outcomes for the population. The experience linking nutrition and agriculture activities at various levels of government in the northern regions was a good example of this. Another example was the support provided by MOFA’s agricultural extension agents working at the district and community levels to the Food Security and Environment initiatives were being undertaken. Other Canadian-funded initiatives, such as the Investment Fund for Food Security, could have benefited from this approach to achieve greater results for the targeted communities.
• Supporting communities to develop action plans leads to more inclusive and responsive decision-making and research in the context of decentralized governance.
• Addressing such issues as productivity, post-harvest loss, the development of farmer-based organisations and strengthening extension services are essential to have a real impact on sustainable agricultural development.
• The use of a trigger system proved to be an effective mechanism to encourage governments to prioritise issues and for the Ghana Program to monitor progress alongside other donors. However, too many triggers and disbursement tranches, which increase transaction costs, is not necessarily more effective than fewer well designed triggers.

5.0 Considerations for Food Security Programming, and Policy Framework

The findings and lessons from this evaluation of DFATD’s food security programming in Ghana and Ethiopia, which constitutes a significant proportion of its bilateral programming on this theme, suggest a number of considerations for the Department’s future programming and policy framework. These include:

• increasing support to smallholder farmers to move beyond crop and livestock production and improving their access to markets and to services that can add value to their products;
• creating or strengthening farmer-based organizations/cooperatives/common interest groups so as to build a critical mass of viable groups, to strengthen farmers’ bargaining power and ability to add value to their products;
• enhancing the linkages between the Food Security and Sustainable Economic Growth strategies to promote overall economic growth in agriculture, including support for the development of small (village/community level) and medium-size (district level) enterprises and facilitation of private sector investment in agri-businesses;
• mainstreaming nutrition in food security and agricultural development strategies, including a recognition of the multi-sectoral determinants of good nutrition status, particularly water and sanitation; and,
• planning and implementing applied research so that it can be affordably taken up by farmers to improve yields, food security, and income.

6.0 Recommendations

6.1 Overall

Recommendation 1: DFATD should review its performance management tools for country development programs to ensure that there is appropriate integration of project and program level reporting, as well as an adequate degree of consistency for investments from different delivery channels.
The practice of performance management of country level development results at DFATD has evolved in recent years. Country Strategies, Country Development Program Frameworks, and Performance Measurement Frameworks have been used to articulate, track and report on sector and country level outcomes. Project level reporting tools, such as the Investment Monitoring and Reporting Tool (IMRT) are also used. In IMRT, Management Summary Reports, and Investment Performance Reports, serve, in part, to inform higher level reporting. However, this and other country program evaluations undertaken recently have noted imperfect integration of these tools to higher levels (Program and Corporate), and inconsistencies in their application.  Statement of immediate, intermediate, and ultimate outcomes across Country Program and project logic models are not always complete or consistent. Baselines, indicators and data sources are not always complete. Lower levels do not necessarily sum to higher levels in ways that are useful to managers. As well, for investments from multilateral and partnership channels that can be tracked to a specific country or region, there is a question of whether sufficient tools and reporting exist to allow an appreciation of the overall results of Canadian sponsored programming at the sector or country level. Efforts to address these questions deserve renewed emphasis.

6.2 Ethiopia and Ghana Country Programs

Recommendation 2: The Ghana and Ethiopia Programs should ensure that Performance Measurement Frameworks for their next generation CDPFs allow for effective monitoring of program level results.
Among others, attention should be paid to collecting sufficient baseline information and regularly tracking chosen indicators.

Recommendation 3: The Ghana and Ethiopia Programs should ensure that food security-related research initiatives are user-focused and serve to improve poverty reduction and economic growth outcomes.

Recommendation 4: DFATD should improve coordination of Canadian-funded research initiatives from different Branches throughout the planning cycle.
DFATD funds food security research projects through different modalities and Branches. In order for research results to be taken up by national counterparts and targeted beneficiaries, DFATD should ensure that planned projects have credible end use strategies from the outset, which are aligned with the national poverty reduction and economic growth agendas in both countries. Increased coordination and communication between the Branches, as calls for proposals are designed and as projects are implemented in the field, is also necessary for research results to be complementary and applied to best effect.

6.3 Ethiopia Country Program

Recommendation 5:  The Ethiopia Program should continue to complement proven existing programming, like support for the Productive Safety Net Programme, with activities to increase the self-sufficiency of beneficiaries.
The provision of additional funding and technical assistance to initiatives that emphasize the safety net aspects of food security programming in the districts where food insecurity is most pronounced should move more households out of a chronic dependency on food assistance and provide them with increased income, thereby improving their self-sufficiency. While the focus of PSNP is to increase basic food security by providing cash or food in exchange for labour on public works, increasing the funding in these food insecure districts for value-added or alternative income generation activities would help to ensure that PSNP graduates do not revert to a dependency on PSNP assistance. This would also support the Government of Ethiopia’s efforts to shift funds to higher value economic growth activities.

Recommendation 6: In its future programming, DFATD-development should ensure that Canadian-funded projects implemented by Canadian implementing agencies are adequately aligned with Government of Ethiopia systems and processes in order to promote ownership and sustainability.
The Ethiopian Government has a policy of maintaining tight control over development interventions in the country by outside agencies. It has also been very good at designing its own programs and initiatives with a view to systematizing processes and practices to ensure sustainability. The corollary of this practice is that initiatives that are not well coordinated with the Ethiopian planning and budgeting systems may not be taken seriously by the government and have a reduced chance of being sustained or supported once external development partner funding ceases. While understanding that sustainability is usually included in project design, DFATD should nonetheless ensure that any Canadian projects are coordinated with the government to the extent possible.

6.4 Ghana Country Program

Recommendation 7: The Ghana Program should consider supporting innovative pilot projects, such as those in food security, that have a potential to be scaled up nationally.
The Ghana Program should leverage its experience in demonstrating effective programming at the district and community levels by increasing funding to initiatives that have the greatest potential to enhance the economic benefits to and the resilience of farmers in the face of climate change. The Ghana Program should continue to focus on such initiatives in the northern regions as they can help to reduce poverty in a targeted manner while also strengthening national systems from the bottom-up.

Recommendation 8: The Ghana Program should continue assisting the GoG to strengthen multi-sectoral linkages at the community and district levels.
DFATD should build upon what is has learned using multi-sector approaches to foster greater coordination among all stakeholders at local levels – for example, extension services, farmer-based organizations and the research establishment - to ensure that the priorities of communities are integrated in district plans, that the transfer of technologies to farmers is accelerated and that resources are leveraged more effectively to address the food security needs of the population in a more comprehensive fashion.

Recommendation 9: The Ghana Program should strengthen its approach to addressing gender issues at various levels of government, including assistance to the MoFA to strengthen and leverage existing gender resources at the national and district level, so that these more effectively support food security-related initiatives to the benefit of communities.  
In future programming, the Ghana Program should build on work done to strengthen MoFA policies and strategies, and in particular, to strengthen WIAD gender resources to play a more effective role as policy advisors on food security issues and to become focal points for programming at the district and community levels.  

Recommendation 10: If the Ghana Program continues to provide sector budget support, it should ensure that the trigger and disbursement system is appropriately calibrated to national capacities to efficiently initiate, monitor, and measure desired changes.
Emerging evaluation evidence and feedback from national authorities suggest that strong national ownership and capacity to implement well-focussed reforms is more effective than numerous donor-driven ones.

Appendix O – Management Response

This management response reflects the departmental priorities as expressed at the end of the evaluation period. Going forward, management responses will reflect new Departmental priorities as they evolve.