Evaluation of the Honduras Country Program, 2010-11 to 2016-17

Acronyms and abbreviations

ACCBP

Anti-Crime Capacity Building Program

APP

Agency Programming Process

ATIC

Criminal Investigation Technical Agency

CCIC

Canadian Council for International Co-operation

CDPF

Country Development Program Framework

CSOs

Civil Society Organizations

EFA

Education for All

FORTA

District Education Management Project

FS

Food Security

FTA

Free Trade Agreement

FY

Fiscal Year

GC

Government of Canada  

GDP

Gross Domestic Product

GoH

Government of Honduras

G16

Group of 16 (donor coordination group)

HoM

Head of Mission

HQ

Headquarters

IFM

International Security and Political Affairs Branch

JES

Justice Education Society of British Colombia

OECD

Organisation for Economic Cooperation and Development

OECD-DAC

Development Assistance Committee (of OECD)

KFM

Partnerships for Development Innovation Branch

LGBQT

Lesbian, Gay, Bisexual, Queer, Transgender

MFM

Global Issues and Development Branch (Multilateral)

MACCIH

Mission to Support the Fight against Corruption and Impunity in Honduras

MNCH

Maternal, Newborn and Child Health

MoE

Honduran Ministry of Education

MoH

Honduran Ministry of Health

MSR

Management Summary Report

NGM

Americas Branch

NGOs

Non-Governmental Organizations

ODA

Official Development Assistance

PASOS III

Sustainable Water and Sanitation Project

PBA

Program Based Approach

PRA

International Assistance Evaluation Division

PSU

Program Support Unit

PTL

Project Team Leader

RBM

Results Based Management

RFP

Request for Proposal

SEG

Sustainable Economic Growth

SSF

Sectoral Support Fund

Executive summary

The evaluation assessed the relevance, effectiveness, efficiency and sustainability of development assistance programming in Honduras from 2010/11 to 2016/17. Its purpose is to inform decision-making and to support policy and program improvements. This report presents the evaluation findings, conclusions and recommendations. Considerations for future international assistance programming are also identified.   

Overall, the evaluation found that Global Affairs Canada programming was relevant to the country’s needs, although it was slow in adapting to a changing Honduran and Canadian context. A challenging Honduran context, alongside several departmental changes limited the efficiency and effectiveness of programming. Despite these external and internal challenges, the program achieved significant results. 

Canadian international assistance added value by delivering high quality technical assistance, building capacity and supporting innovative approaches. Many projects developed models and approaches that are being replicated by the Honduran government and other donors. This is contributing to the durability and scaling up of results. Significant contributions to policy dialogue and donor coordination were made, particularly in the areas of gender equality.  

The integration of cross-cutting issues, and results achieved, varied by theme and sector. Governance considerations were at the forefront of programming given the complex political and social context of Honduras. There was strong integration of environmental sustainability considerations in food security and sustainable economic growth programming driven by increased awareness of the need to reduce and mitigate the impacts of climate change. While gender considerations were better integrated at the project-level than in program-level planning and reporting, projects faced significant challenges in achieving gender-related results in Honduras.   

While most projects delivered their expected results, limited coordination and synergy across projects and sectors, alongside broad geographic and sectoral dispersion, reduced the program’s overall impact. The Program’s strategic planning was limited by a lack of long-term planning mechanisms, frequent staff turnover, weak program-level RBM, changing corporate priorities and slow approval processes.

The following recommendations were derived from the evaluation findings and conclusions:

1. Review project management processes to identify inefficiencies, expedite the time required to complete project agreements and further streamline and standardize requirements.   
2. Improve longer-term strategic planning at the program level.
3. Continue to improve mechanisms to share knowledge and create synergies across projects and partners support by different branches.
4. Target gender equality and the empowerment of women and girls with a program-level strategy.  
5. Implement information management mechanisms to better document and transfer corporate memory.

Program background

Honduran context

Human and Social Development – poverty, inequality & climate change

Although a lower middle income country, Honduras underperforms in many areas compared to its Latin American peers. The level of absolute poverty has increased in recent years with 62.8% of the population falling below the national poverty line in 2015. Those living in rural areas, ethnic and indigenous populations as well as women are among the poorest in Honduras. Inequality remains a major concern with almost no change in income inequality measures since 2009 despite rising levels of GDP.

Food security and nutrition among the most vulnerable populations has worsened, with one in four children experiencing chronic malnutrition. This is partly as a result of severe and prolonged droughts in the eastern and southern regions (2014-2016), known as the Dry Corridor. Honduras tops the long-term global climate risk index, with annual losses due to extreme weather events estimated to be more than 3% of GDP.

Governance and Human Rights – corruption, violence and impunity

Honduras has experienced governance challenges for much of its modern history. Most recently, the military coup of 2009 sparked an internationally condemned political crisis. While the country has returned to a democratic and multi-party political system, it is coping with high levels of corruption, violence and impunity. This poses significant challenges for the protection of human rights, the rule of law and the delivery of justice.  

Corruption is a pervasive problem with organized criminal elements being found among police, government officials and politicians. A lack of transparency, alongside revelations of the misappropriation of funds from state institutions, have sparked massive public demonstrations in recent years. Security is also a critical problem with Honduras ranking as one of the most violent countries in the world. The persistence of gang presence associated with high levels of criminality, violence and narco-trafficking continues to impede political, economic and social progress and has contributed to the outmigration of youth.  

Women and girls face high rates of violence inside and outside of the home. Although domestic violence is punishable by law, the law is seldom enforced. Women’s organizations have raised concerns over the growing trend of ‘femicide’, the targeting and killing of women because they are women.  

Donor context

Canada-Honduras Cooperation

The Honduras Program is Canada’s main bilateral development program in Central America, anchored in a long-standing relationship between the two countries. In 2009, Honduras was identified as a country of focus for Canada’s bilateral development assistance, a commitment that was reconfirmed in 2014. Trade relations between the two countries were formalized with the entering into force of the Canada-Honduras Free Trade Agreement in 2014.

Global Affairs Canada programming in Honduras is managed through a decentralized structure. A Head of Mission (HoM) covering Honduras, Costa Rica and Nicaragua is based in Costa Rica with a Development Director for the Honduras Program based in the country’s capital, Tegucigalpa. Financial transactions are conducted through a centralized office in Mexico. Until December 2016, a Program Support Unit (PSU) provided administrative, logistical and technical assistance to the Program.

Official Development Assistance (ODA) to Honduras

In 2015, total ODA to Honduras surpassed US$600 million, representing 2.8% of gross national income (net ODA). While dependency on development assistance is low, it is much higher than the Latin American regional average of 0.2% net ODA. The top providers of ODA in 2015 included the Inter-American Development Bank, the United States, Japan, the World Bank, the European Union, Germany, the Climate Investment Funds, Switzerland and Canada.

Donor Coordination

The country’s main bilateral and multilateral partners harmonize their development efforts and investments through a donor coordination group, presently known as the Group of 16 (G16). The joint coordination group aims to align donor aid to the Honduras Poverty Reduction Strategy and enable the implementation of the multi-donor basket funds, such as the Educational For All initiative. Several sector specific donor coordination groups are also operational.

Global Affairs Canada programming

Program Disbursements

Between 2010/11 and 2015/16, bilateral aid disbursements from Global Affairs Canada to Honduras totalled $175 million, with average disbursements of $29 million per year. The majority of disbursements (83%) supported geographic programming through the Americas Branch (NGM).  Remaining disbursements were divided between programming supported by Partnerships for Development Innovation Branch (KFM) (14%), Global Issues and Development Branch (MFM) (2%) and International Security and Political Affairs (IFM) (1%).

Geographic programming included two program-based approaches (PBAs), representing 13% of NGM program disbursements during the evaluation period.  One PBA provided support to a multi-donor basket fund with the Honduran Ministry of Education and the other allocated funds to health sector budget support through the Honduran Ministry of Finance. The remainder of programming was delivered through projects implemented by a mix of multilateral organizations, Canadian NGOs and private sector firms, Honduran institutions, and civil society organizations (CSOs) supported through local funds. Unlike other Global Affairs Canada country programs in the Americas, the Honduras Program did not have gender-focused bilateral programming during the evaluation period.  

Expenditures by Thematic Priority

During the evaluation period, 73% of all disbursements were allocated to programming in food security and children and youth (with a focus on health and education), consistent with geographic program priorities outlined in the Honduras Country Development Programming Framework (CDPF) 2010-2015. Remaining disbursements were largely allocated to programming in sustainable economic growth (SEG) priorities (19%), with many SEG projects also contributing to food security objectives. Relatively small disbursements were made to programming in areas of advancing democracy and humanitarian support.

Evaluation scope and methodology

Evaluation scope

• covered the period from FY 2010-11 to FY 2016-17 
• assessment of results focused on NGM projects alongside a purposively selected sample of KFM projects (Honduras Program Logic Model can be found in Annex I)
• the evaluation sample included 27 projects in total, representing 74% ($130M) of total bilateral aid disbursements ($175M) over the evaluation period (details on sampling approach can be found in Annex II)  
• considered Global Affairs Canada’s position within the development and policy space in Honduras along with the relation of non-project activities (e.g., policy dialogue, donor coordination) to the effectiveness of the Honduras Program

Purpose of evaluation

• to provide an evidence-based neutral assessment of development assistance in Honduras to Canadians, Parliamentarians, Ministers, Central Agencies, Global Affairs Canada management, partners and beneficiaries
• to contribute to informed decision-making and to support policy and program improvements by helping to identify good practices and lessons

Evaluation questions

Relevance

1. To what extent is the Honduras Program addressing the needs of the target populations?
2. To what extent is the Honduras Program aligned with the country’s needs and national priorities?

Effectiveness and Efficiency

3. What factors influenced the effectiveness of the Honduras Program?
4. Are there opportunities to improve Program efficiency?
5. Results achieved - To what extent did the Honduras Program achieve intended outputs and immediate outcomes?
6. Results achieved - To what extent did the Honduras Program contribute to intended intermediate and ultimate outcomes?

Sustainability

7. What is the likelihood that the results/benefits will continue beyond project funding?  Is there evidence that benefits have been sustained in areas where funding has ended?

Cross-Cutting Themes

8. To what extent did the Honduras Program integrate gender equality, environmental sustainability and governance considerations in its development programming? Have expected results been achieved?

Methodology

The evaluation was conducted by the International Assistance Evaluation Division (PRA), with the support of an external consultant with regional and gender expertise. Two data collection missions to Honduras were conducted in early 2017.

The evaluation used a mixed methods design with an embedded case study approach. Seven explanatory case studies were conducted. Case studies were project-focused and purposively selected across thematic priorities, time periods, implementing partners, delivery channels and investment types. Details on data limitations can be found in Annex III.

• Key Stakeholder Interviews (n= 74)
• Semi-structured individual and small group interviews, comprised of:
• Global Affairs Canada management and staff (n = 19)
• implementing agencies (n = 29)
• local partners (n = 4)
• Government of Honduras (n = 9)
• international donor agencies (n = 6)
• CSOs/academia/private sector (n = 7)
• Project Review (n = 27  projects)
• Assessment of project implementation and results (15 NGM and 12 KFM projects).
• Systematic review of all available project documents including:
• approval and implementation documents
• monitoring, performance  and management summary reports
• evaluation reports
• Project Financial and Timeline Analysis
• Review of planned vs. actual financial disbursements.
• Review of planned vs. actual project approval, project start and project end dates.   
• Focus Groups with Project Beneficiaries (n=23)
• Case studies included 23 focus group discussions with project beneficiaries.  
• Focus groups included a total 264 participants (134F/ 130M)
• Site Visits (n = 8 projects)
• Case studies included visits to various implementation sites for 8 projects (5 NGM, 3 KFM).
• Document Review
• Review of:
• internal departmental policy/strategy documents
• external documents pertaining to the political, economic and social context of Honduras along with other donor  evaluations

Findings

Relevance findings

Projects were aligned with target population needs and were largely designed and implemented using participatory and consultative approaches.

Projects across all sectors were relevant to the needs of their target populations.  The objectives of health and education sector projects were well aligned with national targets to improve child and maternal health and improve equitable access to quality education and health services. Geographically, the concentration of food security and nutrition projects in the Dry Corridor region targeted the needs of more vulnerable populations with priority areas jointly identified with the Government of Honduras (GoH). 

In the food security (FS) and sustainable economic growth (SEG) sectors, several projects were recognized as being timely in responding to environmental challenges (e.g., prolonged drought, land degradation, coffee leaf rust disease) faced by small producers. However, with a shift in programming to SEG, some stakeholders noted concerns that programming may neglect ongoing and significant needs of smaller producers and poorer farmers including their challenges in accessing markets on fair terms.

For KFM supported projects, the innovative use of participatory approaches and techniques to design and tailor projects to the needs of poor and vulnerable populations, and the use of a consultative approach to respond to institutional needs in the justice sector, were noted as strengths by stakeholders.

While programming was highly relevant to the country’s needs, the department was initially slow to respond to an evolving Honduran context, regional priorities, and issues of importance to Canadians.

At a program level, bilateral aid to Honduras focused on priorities to increase food security and secure the future of children and youth through investments in health and education. Programming was well aligned with the priorities of the Government of Honduras (GoH), as articulated in key policy documents.

However, Canadian Parliamentarians and Canadian CSOs voiced concern over the responsiveness of Canadian international assistance to the deteriorating human rights situation in Honduras and to the needs of Honduras civil society.2 They further recommended that the Government of Canada (GC) review aid and support longer-term efforts for democratization and the protection of human rights. At a regional level, there was additional pressure for departmental programming in Honduras to better align with regional migration and security priorities. And with the coming into force of the Canada-Honduras Free Trade Agreement (FTA) in 2014, it became clear that Canadians were expecting economic partnerships with different actors (e.g., businesses, cooperatives). Pressures increased within and outside the department to realign programming to evolving needs and priorities.

Programming began to shift towards the end of the evaluation period (2015) to align with priorities in areas of advancing democracy and stimulating sustainable economic growth. Opportunities for continuity were identified alongside planning for new programming.  However, there remains the perception that the program was slow in adapting to the changing Honduran and Canadian context. This was partly attributed to corporate changes within the department and high turnover of program staff and management.      

Efficiency and effectiveness findings

A challenging Honduran context alongside several departmental changes limited the overall efficiency and effectiveness of programming.

Honduras is recognized among donors as being one of the most challenging environments for development programming. An unstable political context, weak public institutions, pervasive corruption and security concerns alongside a weak social fabric and environmental vulnerabilities add layers of complexity to program/project implementation. Working within this environment requires a solid understanding of the local context and often entails increased transaction costs for all stakeholders.

Within the department, several corporate-level changes created additional challenges. During the evaluation period, decentralization of the Honduras program (implemented from 2009 to 2013) led to confusion in roles and responsibilities between HQ, HoMs in Central America and the Honduras Development Director. The amalgamation of CIDA and DFAIT in 2013 added new types of actors and levels of accountability, and further complicated coordination between silos that already existed between development and security programming. At the same time, new administrative processes introduced with the launch of the Authorized Programming Process (APP) in 2013 combined with reduced delegated authorities in the field, contributed to longer and more complex approval processes.

At the country level, frequent turnover of Canadian staff and management alongside a weak corporate information management system limited strategic planning. Field staff did not have access to corporate databases (e.g., EDRMS), and data from the PSU was not integrated into the Honduras country office data system. Consequently, the program lost corporate memory.  Additionally, before language requirements were put in place in 2013, some Canadian staff began their postings with little Spanish language capacity. This limited their ability to conduct responsibilities effectively which, combined with high staff turnover, increased transaction costs for partners.

The efficiency of program operations was limited by the relatively large size and slow closure of the PSU. The PSU employed a large team of technical specialists accompanied by the logistical support of eight vehicles and six full-time drivers.  This level of staff and vehicles had been intended to support a larger program than was realized in Honduras.^{Footnote 1} Further, the slow process to close the PSU, which began in 2012 and ended in 2016, proved to be costly, time consuming and created uncertainty for PSU and other field staff.

The efficiency and effectiveness of delivery modalities was mixed with the implementation of PBAs proving challenging in the Honduran context.

Programming used various responsive, directive and program-based modalities in Honduras. The Program’s primary bilateral responsive mechanism was local funds, such as the sectoral support fund (SSF). While the use of these funds incurred high transaction costs relative to the small amount of dollars programmed, they provided strategic, flexible and timely funding. They offered a high return on investment in terms of visibility as well as offering a mechanism to test potential new institutional partners, facilitate work with CSOs and “get a foot in the door” for new subsectors. And while most KFM projects promoted innovative and grass-roots initiatives, stakeholders noted that Honduras-specific resources for projects implemented in multiple countries were not always sufficient or used optimally because of limited knowledge and presence of Canadian partners in the country.

Cooperation in the justice sector involved a mix of responsive KFM and IFM funding due to a lack of bilateral (NGM) calls for proposals in the advancing democracy sector. The need to navigate different department funding streams increased administrative and transaction costs for implementing agencies.  This was largely due to different requirements for proposal submission, financial management and reporting.

For directive bilateral funds, the use of Request for Proposal (RFP) processes proved lengthy and time-consuming, often taking 2-3 years to put projects in place. In the education and health sectors, the use of PBAs proved less effective in building institutional capacity than contribution agreements providing technical assistance. Among other challenges, both the GoH and Global Affairs Canada lacked the capacity to effectively manage and implement PBAs. The effectiveness of technical assistance components to help develop GoH capacity to manage funds was limited by late implementation in the project cycle. While the use of PBAs supports commitments to Aid Effectiveness Principles (e.g., Paris Declaration and the Accra Agenda for Action) and facilitates donor coordination, concerns remain over the effectiveness of channelling investments through PBAs in the Honduran context.^{Footnote 2}

There were many well-performing projects but the strategic impact of the program was limited. 

Strategic planning was limited by an uneven application of results-based management (RBM). Implementing agencies and project partners noted the department’s strengths in applying RBM at the project level, along with its effectiveness in improving project implementation and management. However, the same attention and application of RBM was not used at a program level. This was partly because the department did not initially have program-level RBM tools.  When a program logic model and accompanying performance measurement framework were developed, these tools were not used systematically or strategically to plan and report at a program level, particularly with respect to higher-level outcomes. Towards the end of the evaluation period there were increased efforts to develop meaningful indicators across sectors with a view to improving strategic impact.

Limited coordination across initiatives supported by different branches (NGM, KFM, MFM, IFM), alongside a wide geographic and sectoral dispersion of projects, hindered the Program’s ability to develop strategic approaches. Stakeholders within and outside the department pointed to the need for a more strategic program view alongside missed opportunities for programming synergies. While there were some notable examples of effective knowledge sharing at the regional level along with coordinated approaches within the food security sector, these were often driven by key individuals within implementing agencies. Partners noted increased efforts by the Honduras country office to enhance coordination within and across branches towards the end of the evaluation period.

The ability to plan strategically and over longer-term time periods was further limited by the absence of an approved five-year Country Development Program Framework (CDPF). The lack of direction from senior management combined with an absence of corporate mechanisms and processes to facilitate longer-term planning were noted as a significant challenges by some departmental stakeholders.

Some projects have developed strategies to better reach the poorest and most vulnerable but there is a need to better define and differentiate target groups.

In terms of program reach, implementing agencies and local partners identified difficulties in reaching the poorest and most vulnerable, including marginalized groups, youth, indigenous populations and those living in rural areas.  Partners and implementing agencies are building on learnings but more can be done. The need to better define and differentiate target groups and develop strategies to meet their needs was identified across several projects. For instance, a health services project targeting adolescents identified the need to reach more vulnerable youth (e.g., returned migrants, LGBQT, ethnic minorities) and is developing strategies to address their differentiated needs. And in some SEG projects, the need to better define the target group of “poor rural families” and explore strategies for increasing access to related income opportunities to those lacking sufficient productive resources was noted.

While effectiveness of donor coordination varied, Canada was perceived as a strong contributor. 

Global Affairs Canada participated in several donor coordination tables, including assuming the presidency of the G16 donor coordination group in 2016. While the overall effectiveness of the donor coordination forums varied by sector and time period, Canada’s participation was recognized as active and effective, with Canada often perceived as a leader. Donors reported that Canada was willing to share experiences and raise critical issues, including human rights and gender equality. Canada was also recognized for facilitating south-south cooperation, including advocating for the inclusion of Mexico in the G16.  

Canada’s reputation as a trusted partner enabled Global Affairs Canada to make a significant contribution to policy dialogue on gender issues.

Canada was further viewed by GoH stakeholders and Honduran partners as a neutral and honest broker capable of getting players at the table to work collaboratively. This enabled the department to make a significant contribution to policy dialogue on gender issues. Notably, Global Affairs Canada facilitated dialogue between women’s organizations and the Honduran government to review the proposed new Criminal Code.

Canadian international assistance added value by delivering high-quality technical assistance, building capacity and supporting innovative approaches.

The quality of technical assistance provided along with a commitment to longer-term capacity building, institutional strengthening and accompaniment was recognized across programming. Implementing agencies and Honduran partners reported that working with Global Affairs Canada increased their capacity to function in an open and transparent manner as well as improving their monitoring and evaluation efforts. Some relatively small investments had a significant strategic impact and leveraged support from other donors. The department was further recognized for supporting innovative projects and approaches that are now being replicated by other donors, partners and the GoH. 

Results achieved: Education

Projects contributed to improving the delivery and quality of primary education, but with mixed progress in meeting Education for All targets.

Between 2004 and 2017, the department provided funding of $20 million to Ministry of Education (MoE) through the the multi-donor pooled Education for All (EFA) fund.  There were improvements in primary coverage, completion rates and education quality. However, not all expected targets were met.  Household survey data pointed to a persistence of education gaps by socioeconomic level, and standardized test scores in math remained well below targets. Late entry to schools, gaps in coverage of preschool education, cumulative learning shortcomings in basic competencies and adverse socioeconomic conditions further limited the achievement of the EFA goal of ensuring all boys and girls successfully complete six years of basic education.^{Footnote 3}

Some progress was made in strengthening the capacity of the Ministry of Education (MoE).

Global Affairs Canada’s contribution to the EFA fund, alongside a contribution agreement for the district education management project (FORTA), contributed to some strengthening in the institutional capacity of the Honduran MoE. Despite numerous challenges in managing the EFA common fund, the experience is perceived to have enhanced the Ministry’s internal capacity to work more efficiently and transparently in managing resources.  At a operational level, cost-efficiencies were gained with the transfer in ownership of copyright on new textbooks to the GoH.   

Training, mentoring and coaching activities delivered through FORTA contributed to an increase in the confidence of trained staff in gender-sensitive pedagogical coaching, communication skills, use of educational data and information and in transformational leadership skills. The project also helped to build gender awareness and nascent capacity in the MoE by guiding the creation of a gender committee that provided leadership, promoted gender mainstreaming, and worked on the development of a gender policy.       

Support to the Education for All program contributed to:

• increasing the primary education graduation rate to 84% in 2015, up from 69% in 2000
• reversing the education gap affecting girls, with girls now outperforming boys in basic education (both in terms of access to education & results)

Results achieved: Health

Projects contributed to improved protection against preventable diseases and other specific health challenges

While a lack of recent data at the national level limits the ability to track key health indicators, localized project data combined with anecdotal evidence, points to reductions in the rates and severity of preventable diseases. Perceived reductions in diarrhoea and child malnutrition rates were attributed by project beneficiaries to increased access to quality water, increased and diversified food consumption and enhanced health information services. Access to health services improved as a result of increased awareness among parents of the importance of bringing their children to health centers, and receiving pre and post-natal care. In areas of adolescent sexual and reproductive health, innovative service models were introduced but the need to scale up interventions to achieve sustainable impacts was noted. 

Results in some cases were limited by the capacity of national health stakeholders

Projects contributed to strengthened capacities to administer and deliver health-related services at the municipal level.  The capacity of municipal water boards was built, the technical and organizational capacity of community health committees was increased and networks of empowered youth communicators/leaders on sexual and reproductive health were formed. However, the complex and incomplete decentralization of health services, limited capacity of the Ministry of Health to manage funds, alongside a weak articulation of national institutions and municipal governance structures limited the potential to achieve and sustain intended results.   

Most health projects had a relatively low level of gender integration although some progress was made in increasing women’s participation in health committees and raising gender awareness at the local level.  For instance in a project to strengthen community health, a survey demonstrated changes in attitudes and behaviours by project end, whereby men recognized the importance of supporting women during pregnancy and taking part in health care.  In terms of policy dialogue, contributions were made through projects to influence national education and health policies, including the passing of legislation mandating that 5% of funding transfers of the central government be invested in children and youth. 

Health projects contributed to:

• eliminating one of the main vectors of Chagas and reducing the overall incidence rate by 74%
• a 28% decrease in cases of newborn deaths in targeted communities
• a 34% decrease in HIV cases from 2009 to 2014 in targeted municipalities

Results achieved: Food security and sustainable economic growth

Projects led to increased sustainable agricultural production but faced mixed results in improving consumption of quality nutritious food.

Sixteen bilateral and partnership projects were reviewed. Most projects had both food security and sustainable economic growth objectives. Projects improved access to agricultural inputs (e.g., seeds, plants, financing) and introduced innovative sustainable agricultural techniques and climate-smart practices that were largely accepted and adopted by producers. In most cases this led to increased sustainable agricultural production alongside the realization of environmental benefits, including improved biodiversity, water conservation, reduced contamination and watershed protection.

Innovations, including the development of water capture systems and irrigation tunnels, responded to growing environmental and climate change challenges (e.g., sustained drought, desertification, floods). Support for participatory agricultural research at the local level led to effective results with the development of 26 new varieties of locally-adapted bean and maize seeds. Users of the improved seeds reported increased income and well-being due to improved yields, crop quality and prices. New approaches to building local capacity to manage micro-credit also led to changes in beneficiaries’ perspectives towards allocating a portion of revenues to saving and thinking of production in terms of investment. 

Projects aiming primarily to improve the consumption of quality nutritious food achieved good results. Notably, support to the World Food Programme contributed to decreasing the percentage of the population with borderline food consumption alongside increased dietary diversity. Some complementary targets were not met as insufficient funds and organizational capacity of smallholder farmer associations limited the delivery of fresh food rations (vegetables and eggs) to only 30% of planned beneficiaries. In projects with broader objectives, success in meeting improved nutrition outcomes was mixed. In some cases, the introduction of diversified household production (i.e., cultivation of fruits and vegetables) did not lead to improved diets.  Some beneficiaries expressed limited interest in diversifying diets with a preference to sell produce and allocate revenue to other household priorities.  Projects with strategies to introduce and build capacity in preparing culturally appropriate meals using new vegetables and fruits achieved greater results. 

Food security projects contributed to:

• decreasing the prevalence of female-headed households with borderline food consumption from 18.8% in 2014 to 2% in 2016
• reducing the prevalence of stunting among targeted children under 2 from 43% in 2014 to 34% in 2016
• increasing the productivity of staple crops (maize and beans) by almost 50% through the introduction of bio-fortified seeds

Results achieved: Sustainable economic growth

Economic opportunities were limited by challenges faced by producers in moving up value chains.

Value-chain projects supported producers in increasing the productivity and quality of their products. Significant productivity gains were made.  In the coffee and cacao sectors alone, the rehabilitation and planting of over 5.5 thousand hectares in agroforestry systems is directly attributed to Global Affairs Canada support. Projects successfully leveraged and built upon Canada’s past contributions to developing agroforestry systems and supporting genetic research in Honduras. 

Less progress was made in supporting producers to capture the full value of their products and negotiate access to high-value international markets. Particularly in the cacao sector, support for post-harvest value-added activities were just beginning to lead to increased incomes at the project’s end. In both sectors, producers had limited success in accessing international markets. Challenges include limited access to transport and infrastructure, insufficient collective levels of production, weak marketing and commercial capacity of cooperatives alongside unresolved land tenure issues. 

Interviewees across all stakeholder groups noted the need for longer project time frames to achieve intended results, particularly in the Honduran context. The need for strong producer associations and good governance in order to advance across value chains was identified. It was further noted that building and strengthening cooperative associations takes time, due to a failed history of cooperativism alongside a broader culture of mistrust. Traditional project timeframes of three to five years were perceived as insufficient to allow for the type of substantive and sustained capacity building (including specialized training for business development and commercialization) that would be needed to achieve longer-term results in value-chain projects. 

The project review found most projects (11 out of 16) integrated gender considerations effectively because they had policies, technical expertise and gender-sensitive indicators.  These projects helped to increase women’s leadership, technical capacity, agricultural production and income generation.  A few projects also achieved significant SEG results such as promoting women’s economic empowerment and entrepreneurship.  Notably, the cacao value-chain project played a key role in establishing and strengthening women-led associations of cacao producers and processors.   

SEG projects contributed to:

• supporting the planting and rehabilitation of more than 75% of total cacao surface in Honduras
• elevating the profile of the cacao sector at the national level both in terms of income generation and environmental benefits 
• increasing daily incomes of coffee producers from US$1.25 per day in 2012 to US$7.85 per day in 2016

Results achieved: Advancing democracy

Projects contributed to policy dialogue on human rights and strengthened the capacity of justice sector actors and civil society organizations.

Support to promote democracy and human rights in Honduras was delivered through a mix of partnership (KFM) and security (IFM) programming.  Learnings have been informing planning for bilateral investments (NGM) and the inter-sectoral nature of work in this area has been helping to break down barriers between Global Affairs Canada’s branches.

Projects contributed to policy dialogue and increased awareness of human rights issues, including violence against women and the rights of migrants and indigenous persons. Capacity building in support of advocacy activities of civil society organizations contributed to improvements in working conditions (e.g., health, safety and labour rights), government recognition of femicide and national legislation on domestic violence.  

Support to strengthen the capacity of justice sector actors led to a better understanding of the roles and responsibilities of different actors and increased the capacity and confidence of justice
system actors to carry out investigations.  The use of South-South cooperation was noted as being particularly effective for knowledge sharing. Canadian experts trained criminal justice specialists in Guatemala, who then trained their counterparts in El Salvador and Honduras.  Overall, interventions in the sector were strategic but of limited scope, with some stakeholders noting that public perceptions and confidence in the justice sector have not yet increased. 

Sustainability findings

Support for capacity-building and developing replicable project models is contributing to the durability and scaling up of results.

Building technical and leadership capacity of beneficiaries was perceived as a key factor for increasing the likelihood that benefits would be sustained after funding ends. Good practices included the use of train the trainer approaches and an emphasis on building leadership and governance capacity before developing technical skills. For KFM-supported projects, institutional strengthening of local partners contributed to greater organizational stability. This increased their success in securing ongoing funding from new donors.  Interviewees also provided several examples of how fostering new relationships and networks across different levels of government, academia and local and international NGOs/CSOs helps to sustain results. Additionally, the transfer of credit funds to local institutions ensured the sustainability of financial and technical support for individual farmers and cooperatives in some target regions.

Strengths in project design led to the development of several project models and methodologies that are being replicated by the GoH, other donors and international and local NGOs. This is leading to the scaling up of results both in terms of geographic reach and stakeholder groups.  Notable examples include water collection models implemented in the Dry Corridor that are now being replicated by the GoH in other regions, and models for integral adolescent-friendly sexual and reproductive health services that are being replicated and adapted by other donors to reach new stakeholder groups.    

In health and education sectors, the limited capacity of GoH institutions, alongside reduced donor funding presents challenges for sustaining results. 

As international donors, including Canada, shift priorities away from the health and education sectors there are growing concerns regarding the sustainability of results in these areas. This is largely due to the weak institutional capacity of GoH institutions, including ongoing challenges with the decentralization of resources and responsibilities alongside frequent rotation of GoH personnel. Across all sectors, departmental project leads and implementing agencies noted that insufficient upfront attention to sustainability and the late development of project exit plans has been limiting the potential for sustainability.

Cross-cutting issues

The level of integration of cross-cutting issues, and results achieved, varied by issue and sector. 

The integration of gender and governance was widely recognized by most stakeholders as an important means to attain broader development objectives in all sectors. The success of FS/SEG projects also depended on the effective integration of environmental considerations. In terms of reporting, a review of management summary reports (MSRs) across all projects identified inconsistencies in definitions, the type of data collected, and how data was reported for all three cross-cutting themes. This limited the ability to roll-up project information to a program level.  

Governance considerations were at the forefront of programming given the complex political and social context of Honduras.  FS/SEG projects focused on strengthening local governance structures and capabilities and building leadership and advocacy skills.  Participation and inclusion in producer associations and community decision-making bodies increased and the capacity to engage with government institutions improved.  This sometimes influenced policy decisions and/or led to increased resource allocations.  Efforts to improve public sector management and service delivery within the Ministries of Education and Health achieved limited results with capacity gaps remaining.  

There was strong integration of environmental sustainability considerations in FS/SEG programming driven by increased awareness of the need to reduce and mitigate the impacts of climate change. The level of integration of environmental sustainability considerations and evidence of results varied across health and education sector projects. Strong integration of environmental considerations in water and sanitation projects has been leading to sustained environmental benefits. Other health and education projects used targeted approaches to integrate environmental issues in educational material and media campaigns. However, no performance data was collected to monitor if these efforts are leading to improved environmental awareness. 

Gender considerations were better integrated at the project-level than in program-level planning and reporting. 

Projects faced significant challenges in achieving gender-related results in Honduras. Limiting factors included a patriarchal culture, particularly in rural areas; women’s limited access to land ownership, credit, productive inputs and technical training; insufficient support to foster women’s leadership and empowerment; and inadequate capacity, commitment and/or resources of government agencies to promote gender equality.

Most projects in FS/SEG and advancing democracy integrated gender considerations well and achieved notable related results. Several projects strengthened women’s participation in agricultural production and promoted economic empowerment. Projects in the education and health sectors were less effective in addressing gender issues.  For example the Education for All (EFA) design lacked gender analysis, and strategic plans did not identify gender-related actions or indicators.

At the project level, strong local gender expertise provided valuable support and helped to implement concrete actions. The PSU gender advisor reviewed the integration of gender issues in project proposals, provided technical support and training to project team leads (PTLs) and implementing agencies, and conducted regular field monitoring of projects. The advisor also led a gender network, comprised of project gender specialists, project directors and Global Affairs Canada staff. The network was active until 2012. Several project partners noted the valuable role played by the gender network in building capacity, raising awareness of gender issues and improving results monitoring and achievement.

At the program level, gender considerations were not well integrated into planning and reporting. A Gender Equality Strategy was developed, but never approved or implemented. Gender specialists at Global Affairs Canada HQ changed frequently (three specialists in less than two years), did not have resources to travel to Honduras or sufficient time to provide country-specific support. The program did not have an earmarked budget for gender-related activities. Field staff did not all receive gender training or apply what they had learned.

Conclusions

Development programming in Honduras was limited by external factors such as pervasive corruption, security concerns and weak institutional capacities of partners (public institutions, CSOs). Corporate changes within the department, including decentralization and amalgamation, created internal challenges that undermined the effectiveness and efficiency of the Honduras program. Despite these external and internal challenges, the program achieved significant results.  

Several factors contributed to the program’s relative success. The Program built on effective past programming (e.g. agroforestry); selected several capable Honduran and Canadian partners; and relied on strong technical specialists within the PSU. Global Affairs Canada program staff developed good working relationships with GoH stakeholders, other donors and partners. Canada was perceived as a flexible and trusted partner.

Projects delivered quality technical assistance and helped build the capacity of several Honduran partners. Many projects introduced innovative approaches or models that are being replicated by the Honduran government and/or other donors. Significant contributions to policy dialogue and donor coordination were made, particularly in areas of gender equality.  

The Honduras Program was relevant to the country’s needs, although it was slow to respond to changing Canadian priorities and an evolving development context.  While most projects delivered their expected results, limited coordination and synergy across projects and sectors, alongside broad geographic and sectoral dispersion, reduced the program’s overall impact. The Program’s strategic planning was limited by a lack of long-term planning mechanisms, inadequate program-level RBM, changing corporate priorities and slow approval processes.

Recommendations and management responses

Recommendation 1

Review project management processes to identify inefficiencies, expedite the time required to complete project agreements and further streamline and standardize requirements. 

Agreed: The Honduras Development Program with support from the International Assistance Operations Bureau (IAOB) will identify and address process inefficiencies that are within the program’s control. The IAOB will address programming process and harmonization issues of a corporate nature by incorporating solutions into related corporate action plans. The Grants and Contributions Management Bureau will examine delays encountered in the agreement preparation phase for initiatives developed in Honduras. Findings will inform action plans to improve the efficiency and effectiveness programming processes and management practices.     

Recommendation 2

Improve longer-term strategic planning at the program level. 

Agreed:  In 2015, the Honduras Program adjusted its longer-term strategic planning to focus on two thematic areas currently referred to as: growth that works for everyone and inclusive governance (including the promotion and protection of human rights) with focus on gender equality and the empowerment of women and girls.  A performance measurement framework that is aligned with the Feminist International Assistance Policy and includes reporting on program-level results will be developed.      

Recommendation 3

Continue to improve mechanisms to share knowledge and create synergies across projects and partners supported by different branches.  

Agreed:  The Honduras Development Program will build on existing efforts to share knowledge with Global Affairs Canada branches and external project partners and to improve coherence at the strategic and implementation levels. The Partnerships for Innovation Branch will continue to seek opportunities to increase knowledge sharing, improve coordination and harness synergies where possible. 

Recommendation 4

Target gender equality and the empowerment of women and girls with a program-level strategy.  

Agreed:  The Honduras Development Program will build on existing efforts to: develop a performance measurement framework that identifies gender equality and the empowerment of women and girls as the driver of its development efforts; dedicate funds to continue to actively engage in donor coordination and policy dialogue on gender equality; to continue to engage local gender equality specialists, work with HQ gender equality specialists and provide ongoing training as needed; and to reactivate the local gender equality specialist network. 

Recommendation 5

Implement information management mechanisms to better document and transfer corporate memory.   

Agreed:  The Honduras Development Program is implementing an improved information management system to facilitate knowledge transfer between the Honduras Country Office and HQ and to better transfer corporate memory within the Country Office.  

Considerations for future international assistance programming

Considerations for Geographic Branch Programming

1. Local funds can provide flexible, agile support to strategic pilot projects that could be upscaled, including targeted initiatives to promote the empowerment of women and girls. They can also help to engage with new institutional partners and facilitate work with CSOs.   
2. In-country gender networks can play a valuable role in building capacity, raising gender awareness and improving the monitoring and achievement of results.   
3. PBAs require a strong degree of local ownership, management capacity and systems.  Having a technical assistance component at the start of a PBA helps to build necessary institutional capacity.

Considerations for International Assistance Programming across Branches

1. Partners and implementing agencies are developing strategies to better define and meet the differentiated needs of poor and vulnerable groups. Lessons could be documented and shared.    
2. Developing exit strategies early in project implementation can build local ownership, increase potential sustainability and help partners leverage resources from other sources.   
3. Longer-term or phased projects may be necessary to achieve sustainable results, including in cross-cutting areas.
4. Leveraging successful past project experiences and facilitating south-south cooperation can increase program impact in a cost-effective way.
5. When designing and implementing value-chain projects, attention needs to be placed on providing specialized resources to develop marketing and commercial capacities and strategies to access high-value markets.   
6. Joint monitoring missions across projects supported by different departmental branches could be considered to foster knowledge sharing and improve the effectiveness of monitoring.  
7. Providing country offices with access to corporate information management systems (e.g., EDRMS and Infobank) could improve operational efficiency and enhance knowledge transfer between the field and headquarters.    

Appendices

Appendix 1 : Program logic model

Appendix 2: Methodology – Sampling approach

A purposive sampling approach was used to ensure representation of thematic sectors, delivery channels, and investment types.  Twenty-seven projects (15 NGM, 12 KFM) out of the overall portfolio of 94 projects were selected in consultation with representatives from both the NGM and KFM branches. The evaluation project sample represents 74% ($130 million) of total bilateral aid disbursements ($175 million) to Honduras that were made through the Global Affairs Canada NGM, KFM, MFM and IFM branches over the evaluation period.  

No MFM projects were selected, as projects funded over the period were focused on humanitarian efforts and represented only 2% of bilateral aid disbursements. While IFM projects (representing 1% of bilateral aid disbursements) were not specifically included in the evaluation sample, the relation of IFM’s Anti-Crime Capacity Building Program (ACCBP) projects to KFM project support to the justice sector in Honduras was considered.      

Appendix 3: Data limitations

Limitations

• Security considerations restricted the evaluators’ ability to purposively select and conduct site visits and reach beneficiaries in rural and marginalized areas.
• Limited availability and consistency of performance data, including:
• lack of current and reliable quantitative national data, especially in the health sector;
• limited program-level performance data, particularly for longer-term outcomes
• project data sometimes not disaggregated by age, gender, and other characteristics.
• Inconsistencies in collection of timeline data for project approvals, start dates and end dates.  This limited the rigour of analysis that could be conducted to assess efficiency.      

Mitigation Strategies

• site visits selected in consultation with security program officer
• meetings held at alternative locations to reach some beneficiary groups and other stakeholders
• conducted detailed project reviews, including assessments of project-level indicators and data 
• used alternative data sources from well-respected and credible sources
• differing time stamps on source documents were noted and a consistent approach taken 
• projects with data gaps removed and analysis was conducted on a sub-set of projects (n=23)

Appendix 4: Management response to the recommendations