Evaluation of International Assistance Programming in Colombia, 2011-12 to 2017-18 – Summary Report

Why is it important?

This evaluation was conducted to provide an evidence-based, neutral assessment of international assistance in Colombia to Canadians, Parliamentarians, Ministers, Central Agencies, Global Affairs Canada management, partners and beneficiaries.

It aims to contribute to informed decision-making, to support policy and program improvements and to advance departmental horizontal learning.

What the evaluation assessed

Global Affairs Canada’s international assistance programming in Colombia between 2011-12 and  2017-18, with a focus on projects supported by the bilateral development program alongside a purposive sample of projects supported through partnership programming (KFM), humanitarian programming (MHI), and peace and stabilization programming (START/PSOPs).

The evaluation questions that were asked

Responsiveness and Agility:

• To what extent has international assistance programming responded to evolving needs and opportunities in Colombia?

Results, Gender Equality and Sustainability:

• To what extent has international assistance programming contributed to expected development outcomes in areas of sustainable economic growth, children and youth and peacebuilding?
• How has programming contributed to gender equality results and the empowerment of women and girls?
• Is there evidence that results have been sustained in areas where support has ended? What is the likelihood that the results achieved will continue?

Program and Policy Coherence:

• To what extent has coherence across international assistance programming improved? Across other areas of departmental engagement?

Evaluation Scope

• Evaluation Period: 2011-12 to 2017-18.
• Total international assistance disbursements 2011-12 to 2017-18: $298 million, with average disbursements of nearly $43 million per year.   
• Data Methods:
  • key stakeholder interviews (n=96).
  • focus groups with project beneficiaries (n=23), including a total of 227 participants (130F/ 97M).
  • site visits (n=7 projects).
  • project review (n=39 projects, assessment of implementation and results).
  • project financial analysis.
  • document and literature review.
• Evaluation Completed: October 2018

What the evaluation found

Responsiveness and Agility

Global Affairs Canada programming was responsive to an evolving Colombian context. Early and long-term support was provided to key areas of the Colombia peace process and niche sectors were selected where few donors were active. As new needs and opportunities emerged, Canadian expertise in areas such as rural economic development and cooperatives was leveraged. Moreover, approaches to international assistance were aligned with good practices for donor engagement in middle-income countries and fragile states. This included increased efforts to engage in policy dialogue and advocacy, to leverage private sector resources and to test new modalities for financing. It was noted that a focus on state-building was central; therefore, projects included strategies to accompany the state in fragile and conflict-affected areas.

Results, Sustainability and Gender Equality

• Significant results were achieved at a project-level across children and youth, sustainable economic growth. Support for the implementation of the 2016 peace accord yielded early results in areas of transitional justice and humanitarian de-mining.
• A key factor for project success was the ability to bring together private and public sector to collaborate in new and innovative ways.
• Projects that were successful in demonstrating both financial and social viability achieved high levels of uptake by private sector actors, increasing the prospects for sustainability and impact. In certain areas, increased conflict and insecurity posed risks for the long-term sustainability of initiatives.
• Projects contributed to strengthening civil society organizations, such as agricultural cooperatives and women’s rights groups.
• A significant contribution to policy dialogue was made in areas of child protection, rural education, rural women’s economic development and gender equality. The department played a convening role in bringing stakeholders together and enabling implementing partners to participate in policy dialogue efforts.
• Support for gender integration increased over the evaluation period, leading to improved gender analysis and reporting.
• Some projects developed good practices and innovative models to promote gender equality and women’s rights. Notable results in the area of sexual and gender-based violence were achieved across all international assistance streams. However, significant barriers for gender equality remained.

Program and Policy Coherence

• Some strong examples of coherence across international assistance programming and areas of mutual interest for international assistance, trade, and diplomacy streams were identified.
• Collaboration was often driven by external events or personal initiative and supported by informal systems of communication and information exchange.
• Corporate planning and reporting tools did not effectively support cohesive international engagement.
• Challenges in communicating a coherent “Canada message” made it difficult for external stakeholders to obtain a clear picture of Canada’s overall engagement in Colombia.

Summary of Recommendations and Management Responses

Recommendation 1

Continue to provide guidance and regular training for staff, implementing agencies and partners on how to align projects with the Feminist International Assistance Policy. Foster exchange of good practices and innovative approaches to women’s empowerment and rights among Global Affairs Canada projects in Colombia.

Agreed: The Colombia Development Program will continue to regularly convene staff, implementing agencies and partners in Colombia to exchange information, updates and good practices around the implementation of the Feminist International Assistance Policy (FIAP), with an emphasis on development innovation. In addition, the Colombia Program will continue to seek opportunities for training related to FIAP implementation (particularly in a conflict-affected / middle-income environment) building on previously-offered training on Results-Based Financing; GBA+; and FIAP-related policy dialogue. Identified corporate training gaps will be brought to the attention of CFSI and the Colombia Program will work with CFSI, where appropriate, to pilot new FIAP-related courses. CFSI will work with the Colombia program to address identified corporate training gaps and pilot new FIAP-related courses and other learning tools where appropriate.

Recommendation 2

Develop an advocacy and policy dialogue strategy with clear objectives, associated tools and training. The strategy should be developed with partners to promote greater coherence and impact across advocacy and policy dialogue efforts.

Agreed: The Colombia Program will enhance its efforts to explicitly communicate and prioritize its advocacy and policy dialogue objectives through a Program-level strategy, in consultation with implementing partners in Colombia and other international engagement streams (notably trade and diplomacy).  While advocacy and policy dialogue objectives will need to be tailored to the context and intended audiences, broader messaging around FIAP and Global Affairs Canada priorities will be developed in consultation with PVP, NLD and others responsible for various Action Areas. In coordination with MFM and IFM branches, the International Assistance Policy Coordination division (PVP) is finalizing policy guidance on each of the Action Areas of the Feminist International Assistance Policy. Related guidance, tools and training are also under development and will be shared with programming branches as they are finalized. The policies and other tools will all support missions in their policy dialogue and advocacy efforts.

Recommendation 3

At a whole-of-Mission level, continue to strengthen formal systems for knowledge and information sharing across international assistance streams (development, humanitarian, peace and security) and international engagement streams (international assistance, trade, diplomacy). Identify mechanisms to incentivize collaboration.

Agreed: BGOTA will further strengthen its knowledge and information sharing across international assistance streams, and international engagement streams. This will include formalization of policy/programming coherence mechanisms as well as additional collaboration as identified through the integrated country framework (ICF) planning process.   Through its daily interaction with BGOTA, NLA will promote greater information sharing and exchanges across all three streams at mission and headquarters, and with various HQ divisions working on foreign policy, trade and development issues. Spaces for information sharing on activities, meetings and events will also be maintained and encouraged e.g. regular conference calls at the Deputy-Director / Director level with all three streams at mission and HQ. These should be systematized and go beyond personal initiative with a view to having each stream aware of important issues other streams are working on.

Recommendation 4

Develop an integrated whole-of-Mission communication strategy to ensure a coherent and consistent “Canada message”. Strategy should include communication products that provide an overview of (i) how the different international assistance streams (development, humanitarian, peace and security) are collectively addressing Global Affairs Canada’s priorities in Colombia; and (ii) Global Affairs Canada’s overall engagement (international assistance, trade, diplomacy) in Colombia.

Agreed: BGOTA will more explicitly communicate its Whole-of-Mission activities and priorities in Colombia through an integrated communications strategy. LCD commits to working with the Americas Branch and BGOTA on an integrated, Whole of Canada message. Strategy should focus on what Canada is doing to support/work with Colombia in a comprehensive way, to tell the full picture of Canada’s engagement, not the individual pieces. The communications effort should be supported with targeted communications and partner outreach in Canada and Colombia. Development of plan could be supported by a communications temporary duty assignment (TD) in BGOTA to work with mission.

Recommendation 5

Review process for roll-out of integrated country framework (ICF) to ensure timelines allow for internal and external consultation and proper sequencing with complementary tools. Assess how the ICF can be better used to foster the coherence of Canadian engagement.

Agreed: The Integrated Country Framework (ICF) planning process is in a pilot stage for Canada’s plans and programs in select countries.  The pilot was launched in Fiscal Year 2017-18 to test the effectiveness of an integrated planning approach.  While two countries were selected for whole of government planning, Colombia was selected to pilot an ICF at the level of Global Affairs Canada.   The effectiveness of the ICF is in the process of being reviewed with a “lesson learned” approach led by the geographic bureaus involved.  Questionnaires were disseminated to geographic DGs in late September and are currently being collected.  Results will be assessed from October-November with a view to providing consolidated recommendations aimed at improving the ICF approach for the FY19-20 planning cycle.  Once the assessments are completed and a determination is made as to the value of continuing this process, POL will turn over the lead on ICFs to the Geographic Coordination and Mission Support Bureau (NMD). BGOTA will continue to engage with POL and others involved in the ICF pilot process to share lessons, opportunities and challenges of the ICF process roll-out on the ground.

Considerations for Horizontal Learning

Programming in middle-income countries

1. Middle-income countries can provide opportunities to test innovative ways of delivering and financing development assistance, such as results-based financing and partnerships with private sector actors. When piloting new initiatives, it is important to capture and disseminate early learnings.
2. Programming in middle-income countries requires shifting more time and resources to policy dialogue activities. Staff members require support and training to build the necessary skills and competencies.

Programming in fragile and conflict-affected states

3. Programming in fragile and conflict-affected states requires iterative and adaptive approaches for project implementation. Systems for feedback and continual learning are important. The flexibility allowed through contracting mechanisms, results-based management frameworks, and local funds should be maximized. 
4. There is an implicit need for projects to address issues of trauma when working in fragile and conflict-affected regions. Good practices include the integration of psycho-social support components, with ear-marked budgets, in projects across all sectors. 
5. In areas of protracted crisis, humanitarian agencies can have a deep knowledge of the local dynamics of conflict and the trust of affected communities. Mechanisms to share information and collaborate on projects can help development partners better identify and meet the needs of targeted groups. 

Feminist International Assistance Policy

6. Strengthening the use of Canada’s National Action Plan on Women, Peace and Security in development programming to complement the Feminist International Assistance Policy, in Colombia and other conflict-affected states, could help to promote synergies and coherence across international assistance programming in areas such as sexual and gender-based violence (SGBV).
7. In culturally conservative contexts, it can sometimes be more effective to approach feminism indirectly by demonstrating the social and financial viability of initiatives. Actively engaging boys and men can help to promote gender equality.

Coherence

8. Coherence often depends upon individuals’ initiative and willingness to collaborate. There is a need to balance informal practices with formal mechanisms and incentives for strengthening coherence. Tools such as the integrated country framework, the integrated conflict analysis and gender-based plus analyses have the potential to foster coherence if all business lines are meaningfully engaged in their development.
9. A mix of corporate governance structures, distinct work cultures, and frequent staff mobility creates challenges for departmental coherence at a country level. The role of the Head of Mission (HoM) is key to fostering and incentivizing coherence across all lines of business.