Evaluation of the Canada Fund for Local Initiatives (CFLI), 2015-16 to 2020-21

Evaluation Report

Prepared by the Evaluation Division (PRA)
Global Affairs Canada
January 2023

Acronyms and abbreviations

ADM

Assistant Deputy Minister

CBS

Canada-based Staff

CFLI

Canada Fund for Local Initiatives

CSDPs

Common Services Delivery Points

CSOs

Civil Society Organizations

DRF

Departmental Results Framework

EGM

Europe, Arctic, Middle East and Maghreb Branch

FAS-SAP

Financial Management System

FIAP

Feminist International Assistance Policy

FPDS

Foreign Policy and Diplomacy Service

GAC

Global Affairs Canada

GBV

Gender-based Violence

Gs&Cs

Grants and Contributions

GoC

Government of Canada

HA

Humanitarian Assistance

HOM

Head of Mission

IA

International Assistance

INGOS

International Non-Governmental Organizations

IT

Information Technology

KFM

Partnerships for Development Innovation Branch

LES

Locally Engaged Staff

LM

Logic Model

MFM

Global Issues and Development

MHA

International Humanitarian Assistance (Americas and Asia) and Natural Disaster Response Division

NGM

Americas Branch

NMS

Mission Support Division

OGM

Asia Pacific Branch

PIP

Performance Information Profile

PM

Program Manager

PMB

Program Management Board

PRA

Evaluation Division

PRD

Evaluation and Results Bureau

SGD

Grants and Contributions Management Bureau

SGGS

Foreign Affairs and Trade Grants and Contributions Services

SGPP

Grants and Contributions Policy and Training

SGS

Grant and Contributions Program Services Division

SPP

Contracting & Material Management Policy

WGM

Sub-Saharan Africa Branch

WVL

Women’s Voice and Leadership

Executive summary

The evaluation of the Canada Fund for Local Initiatives (CFLI) covered the period from 2015-16 to 2020-21 and aimed to generate insights, findings, lessons and recommendations to inform decision making at mission and HQ on how best to deliver the CFLI effectively. It had three core objectives: to contribute to informed decision making related to systems and program improvements; to determine the key enabling factors of effective programming; and to provide a neutral assessment of the CFLI in a transparent, clear, and useful manner.

The evaluation found the CFLI to be an effective decentralized, flexible program. Its design made it responsive to both mission and local needs while supporting departmental priorities. However, the duality of the program (both foreign policy and international assistance) led to multiple interpretations and understandings of what the CFLI is and what success looks like. While allowing for flexibility, this hindered its ability to be leveraged as a strategic whole-of-mission tool. In addition, the CFLI’s approach to risk management did not align with its willingness to be a risk-tolerant program. This often resulted in missions’ reluctance to partner with newer, less experienced local organizations. 

The evaluation also found that the support provided by the CFLI team at HQ was crucial to mission staff executing their CFLI responsibilities. At the same time, multiple challenges hindered program delivery. Critically, human resources (dedicated CFLI staff) at both HQ and mission were considered key obstacles to optimally deliver a resource-intensive program like the CFLI. Additionally, CFLI teams at mission did not consistently have the required capacity (e.g. relevant knowledge, skills, access).  Moreover, CFLI data/financial management systems and processes were burdensome, inefficient and not sufficiently transparent and thus exposed the CFLI to significant institutional risks. Finally, the foundational elements of CFLI’s performance management framework significantly limited its ability to accurately define, measure, track and report on CFLI’s results. 

Despite these challenges, the evaluation found evidence of positive results. The CFLI helped increase partner capacity in a range of contexts; however, in general, the capacity of organizations remained a barrier to effectively delivering the CFLI. The CFLI also contributed to a wide range of positive results at the local level and delivered useful humanitarian and emergency response programming. The CFLI proved to be an effective tool for access to and engagement with local stakeholders, while promoting Canada’s brand, visibility and values abroad. Results were commensurate with the small size and short duration of CFLI projects.

Addressing the CFLI’s key barriers and building on its successes will empower GAC to continue delivering quality and effective localized programming going forward. 

Program background

The Canada Fund for Local Initiatives (CFLI) is a global program established by the Government of Canada to support organizations and projects in Official Development Assistance (ODA) eligible countries.

For nearly 50 years (see Annex I for the CFLI timeline), the CFLI has funded modest (under $100K), local, and short-term (1-2 year) projects through Canada’s International Assistance Envelope in over 150 countries, at 70 missions, with program decision making delegated to the relevant Canadian embassy or high commission. As the name suggests, the CFLI principally funds local civil society organizations (CSOs), providing opportunities for grassroots organizations to gain experience with a large bilateral donor and to foster Canada’s relationships with local populations, their civil societies, as well as decision makers and influencers. Since 2015, approximately 80% of CFLI projects were implemented by local CSOs. 

The CFLI has three key objectives. It is meant to expand Canada’s diplomacy by advancing Canadian values and interests through advocacy and partnerships with local organizations. Simultaneously, the CFLI contributes to the international assistance toolkit as it funds targeted projects that are consistent with Canada’s international assistance priorities. In addition, CFLI funds are available for humanitarian and emergency response programming to respond to catastrophes and natural disasters.

General oversight and policy direction of the CFLI is led by the CFLI Unit in the Mission Support Division (NMS) of the Americas Branch (NGM). At Canada’s embassies and high commissions abroad, the CFLI is managed by the Foreign Policy and Diplomacy Service (FPDS) section, with projects being selected by Canadian diplomats and approved by Heads of Missions (HOMs).

Since 2017, CFLI projects were required to align with thematic priorities approved annually by the ADM-level Program Management Board (PMB). Initially, these were advancing democracy and ensuring security and stability. In 2018-19, the priorities were updated to align with Canada’s Feminist International Assistance Policy (FIAP). Since 2020, CFLI priorities have been a combination of the FIAP action areas and a focus on advancing democracy, protecting and promoting human rights (added in December 2020), and ensuring security and stability. Missions were not required to program across all thematic priorities. The CFLI Guidelines encouraged a strategic focus on a limited number of priorities based on local development needs that aligned with and supported Canada’s strategic policy objectives.

Resources

CFLI disbursements by FIAP action area 2020-21 (Can$)

Budget

The CFLI annual budget remained stable over the period from 2015-16 to 2018-19. It started to increase in 2019-20 and reached $30M in 2021-22. Over the period under review, between 10% and 13% of the total annual budget covered operations costs (Vote 1), with the remainder dedicated to program costs (Vote 10). In 2020-21, as part of Canada’s response to the COVID-19 pandemic, COVID-19 Crisis Pool Funds were established. Drawing from these funds, the CFLI was increased by $14M to support projects addressing local COVID-19 related efforts. Additionally, $1M of regular CFLI funds were repurposed to target COVID-19 related projects. Lastly, in December 2021, the Minister of International Development’s Mandate Letter referenced an “increase in annual investment for the CFLI to enable staff at Canada’s embassies around the world to support the work of feminists, LGBTQ2 activists and human rights defenders”. No confirmation of this funding increase has occurred during the evaluation period.

Disbursements from 2015-16 to 2020-21 (Can$)

The CFLI’s total disbursement during the evaluation period was just over $114,600,000. Total disbursements by region were just over $77,500,000^{Footnote 1} distributed to the following branches: $20,883,348 for WGM (sub-Saharan Africa), $20,416,326 for OGM (Asia Pacific), $16,694,505 for NGM (Americas), and $16,634,276 for EGM (Europe, Arctic, Middle East and Maghreb).

Disbursements for CFLI Humanitarian and Emergency Response projects were $2,881,933. These funds provided rapid support to countries and communities in need.

Evaluation scope and methodology

Evaluation scope and objectives

Evaluation scope

The evaluation of the CFLI was conducted by the Global Affairs Canada Evaluation Division (PRA) in line with the departmental Five-Year Evaluation Plan and the Treasury Board Policy on Results. The evaluation covered the period from 2015-16 to 2020-21 and is the second evaluation of the CFLI program (see sidebar). Initial scoping identified the need to focus on two key areas of inquiry: 

• CFLI as a program. This included its overarching design features, policies, systems, processes, current uses, and focuses, as well as current capacities and dedicated resources. 
• CFLI missions/projects. This included its application, influence, and results within the program’s intended audiences, as well as processes, practices, capacities and resources at the mission level.

These areas of inquiry became the foundation of the evaluation’s data collection and analysis. 

Evaluation purpose and objectives

The evaluation aimed to generate insights, findings, lessons and recommendations to inform decision making at mission and HQ on how best to deliver the CFLI effectively. The evaluation relied on both formative and summative^{Footnote 2} evaluation approaches to assess the program and its results, while helping it adjust course as needed. 

The evaluation had three objectives:

• Determine key enabling factors of effective programming for CFLI priorities and better understand the degree to which localization contributes to the CFLI being more effective, efficient, and sustainable.
• Provide a neutral assessment of the CFLI in a transparent, clear, and useful manner by highlighting why missions use the program and how departmental resources have been, or can be, used to optimize results.
• Contribute to informed decision making, support systems and program improvements and advance departmental learning.

Within these objectives, the evaluation team aimed to identify innovative thinking, processes and approaches that have had a positive influence on the CFLI, as well as COVID-19 related challenges.

Evaluation questions

The evaluation focused on four main questions. The questions sought to address the issues of relevance, efficiency, and effectiveness as they relate to the CFLI program within the broader mandate of the department. Localization, a key component of the CFLI, was addressed as a specific question as well as a cross-cutting theme. See Annex II. 

Source: CFLI Kazakhstan (2020)

Methodology

The evaluation used a mixed-methods approach, drawing on both qualitative and quantitative data from a variety of data sources and data collection methods, as shown in the table below. The evaluation team used different types of triangulation to ensure the validity and reliability of findings, conclusions and recommendations: data triangulation, methodological triangulation and investigator triangulation.^{Footnote 3} Data analysis was systematic and iterative. Qualitative data were coded by themes and sub-themes as per the evaluation matrix; and quantitative data were analyzed using descriptive statistics.

Document review

The evaluation included a review of departmental documents and files, including but not limited to: annual reports, past evaluations and reviews, financial data and statements, strategies, research and consultative papers, CFLI’s internal websites, and GAC internal documents and correspondence. The document review provided critical insights on the role and activities of the CFLI, its internal policies and management, financial data, and strategic approach. 

Mission/HQ reports - quality review and results harvest

Twenty-three (of 69) mission-level end-of-program reports for 2020-21 were reviewed for their quality and to harvest results for 14 selected indicators. The purpose of this exercise was to assess the quality of mission-level reporting and to identify limitations. In addition, examples of results for selected indicators were collected. Additionally, 6 end of year HQ reports were reviewed for quality and consistency.

Theory of Change reconstruction

A CFLI Program theory of change (ToC) reconstruction was undertaken with the following objectives: 1) To develop an updated CFLI ToC to address flaws in the original logic model (LM) and performance information profile (PIP); 2) To inform the design of the CFLI evaluation.

The CFLI ToC reconstruction followed a participatory approach involving the CFLI team at HQ (NMS) as the primary participants and drivers of the process. See Annex III.

Measuring Advocacy, Access, and Influence Pilot (mission case studies)

In collaboration with a research firm, the evaluation team developed a framework and toolkit to identify, understand and measure the effectiveness of CFLI advocacy efforts. The framework and toolkit were piloted with eight CFLI mission case studies. A final report and mission specific reports were developed. See Annex IV. Evidence from the case studies is context-specific and cannot necessarily be generalized to the CFLI as a whole.

Environmental scan of donor practices (n=10)

An analysis of the mission-level local funding of 10 different donor countries was undertaken. The analysis identified successes and barriers to delivering small grant programs at the mission level, while understanding their value added.

Key stakeholder interviews (n=82)

Semi-structured interviews were conducted via telephone or teleconferencing platforms. Evaluators interviewed the following groups: 37 current working-level CFLI mission staff (e.g. FPDS, locally engaged staff, and contracted coordinators); 13 HOMs; 32 HQ-level staff from 7 branches.

Interviews provided data to better understand the barriers and enablers to managing the program at mission and HQ. 

Local organization stakeholder survey (n=688)

Understanding the CFLI from a local perspective was an important element of the evaluation. The evaluation team designed a survey with 6 main areas of focus (relevance, capacity building, application process, project delivery, project results, and humanitarian assistance). The survey targeted all organizations that had been selected as CFLI partners in 2020-21. The survey had a response rate of 78.46%. 

CFLI mission-level surveys (n=232)

Two online surveys were designed for GAC missions: (1) CFLI mission staff and coordinators; (2) HOMs. The surveys sought to understand CFLI’s contribution to its stated objectives and to identify opportunities for improvement in its effectiveness and efficiency. The response rates for the surveys were 55% and 25.4%, respectively.

Evaluation limitations and mitigation measures

Findings - Design

Design

Relevance and alignment to GAC priorities at HQ and mission

According to survey data:

• 93% of mission staff agreed that the CFLI supported broader GAC priorities.
• 92% of mission staff agreed that the CFLI supported established mission priorities.
• 88% of mission staff agreed that the CFLI supported broader FPDS initiatives.

The CFLI program was aligned with departmental priorities.

Evaluation evidence demonstrated that there was strong alignment between the CFLI program and departmental priorities. Within the context of the Departmental Results Framework (DRF), the CFLI aligned with Core Responsibility 1: International Advocacy and Diplomacy, and Core Responsibility 3: Development, Peace and Security Programming. 

The CFLI’s alignment to departmental priorities is a function of its broad thematic priorities and its positioning at the intersection of international assistance and foreign policy. The duality of the CFLI (a Foreign Policy and Diplomacy Service (FPDS) tool funded by the international assistance envelope) provided Canadian diplomats the ability and flexibility to address both diplomatic and international assistance priorities. The CFLI’s Terms and Conditions clearly state that CFLI projects “must be consistent with Canada’s priorities for international assistance. The projects must also afford opportunities to advance Canada’s values and interests and/or strengthen Canada’s bilateral relations with foreign countries and their civil societies.” Organizations surveyed (n=688) reported that their projects all fell under one or more of the FIAP action areas, with 40.3% reporting that their projects focused on gender equality and the empowerment of women and girls.

The CFLI’s decentralized and flexible design made it responsive to mission objectives and priorities, but were also relatively resource-intensive for missions.

The CFLI is unique among GAC’s Grants and Contributions (Gs&Cs) programs in that the vast majority of projects are approved at mission, without prior authorization by HQ (with the exception of crisis pool projects over $100K). Evidence demonstrated that this decentralized model, combined with the CFLI’s broad thematic priorities, provided missions with the flexibility to use the CFLI to respond to context-specific, mission level-priorities. Furthermore, evidence from HOM and staff interviews emphasized the CFLI’s responsiveness and flexibility as significant strengths of its design. The design allowed them to choose the most appropriate approach for their mission and context, either using the CFLI to support multiple priorities, types of actors and regions, or by focusing deeply on one issue. 

At the same time, the evaluation found that, the CFLI was relatively onerous to manage and deliver, in particular for missions but also for the HQ team, because of its unique design characteristics: the large numbers of small and short term projects, managed directly by missions, with many diverse and often non-experienced local partners. Because of this, it was reported that the CFLI often added an increased level of burden for FPDS staff.

Relevance and alignment to GAC priorities at mission

According to survey data:

• 78% of mission staff agreed that the CFLI was included in mission planning all the time (another 14% said some of the time).
• 73% of mission staff agreed that their mission had a standalone CFLI strategy all the time (another 14% said some of the time).

Following a detailed review of a sample of FPDS initiatives in Strategia for 2020-21 (n=106): *

• 41% of missions did not explain how or why the CFLI would be strategically used to complete their FPDS initiatives.
• 48% of missions concretely included the CFLI in mission/FPDS planning.

*Source: Strategia reporting platform

The CFLI was not consistently leveraged as a strategic whole-of-mission tool.

The CFLI was widely acknowledged by consulted stakeholders as a Foreign Policy and Diplomacy Service (FPDS) tool that offered opportunities for coherence and linkages between different streams at mission, in particular development, trade and political. Each mission with a CFLI program was strongly encouraged to develop a mission-level CFLI strategy. The CFLI Guidelines state that the mission-level [CFLI] strategy… “is a planning tool to identify the objectives for the mission’s CFLI program and plans for achieving them. The strategy should consider how CFLI programming can and should advance the broader mission plan, and in particular FPDS objectives and thematic priorities.”

However, the evidence showed that three key issues limited the ability of missions to use the CFLI strategically and as a whole-of-mission tool:  

• First, defining the strategic focus for the broader FPDS sections at missions was a major challenge. Interviews from HQ and missions detailed the competing strategic directions received by FPDS staff from CFLI HQ, HOMs, and geographic desks. This challenge faced by FPDS often resulted in a subsequent lack of a clear CFLI strategy or limited its integration within mission planning (see sidebar). Interviewees noted that improving the strategic focus of FPDS could help improve the alignment of the CFLI with mission-level priorities and thus maximize its impact.
• Second, the evaluation evidence uncovered multiple interpretations and understandings of what the objectives and success of the CFLI are, given its duality as a diplomacy and international assistance tool. In many instances, the ability of missions to identify projects that could contribute to both objectives varied, and in some situations (e.g. more hostile environments to Canadian values) these objectives were difficult to pursue simultaneously. Numerous interview respondents highlighted the tensions and challenges faced by having diplomatic staff implementing a development-focused program. These types of “cultural challenges” have led to an inconsistent use of the CFLI as a whole-of-mission tool despite the CFLI Guidelines encouraging HOMs and mission staff to do so. 
• Lastly, better use of the CFLI as a whole-of-mission tool often came down to mission staff’s (from all streams) visibility of the program, as well as their capacity to participate in and contribute to the program’s direction. Evidence from the ITAD pilots reinforced that strategic project selection varied greatly, with limitations linked to context and mission capacity. CFLI staff described positive integration of the CFLI as a whole-of-mission tool when the various streams (development, trade, consular) were able to communicate with, contribute to, and leverage the program in coordinated ways, creating synergies across various mission-level issues and priorities. 

Relevance and responsiveness to local needs and priorities

CFLI COVID-19 programming in 2020-21

The CFLI responded to local needs and priorities.

One of the CFLI’s most important attributes was its ability to deliver programming that was meaningful and focused on local contexts, issues, and needs. Over 92% of CFLI-supported organizations surveyed agreed that the program responded to local needs and priorities in a meaningful way, while close to 90% believed the program allowed sufficient flexibility for them to conduct programming that responded to their organization’s needs and priorities. In addition, respondents also cited that the CFLI assisted in building relationships and networks in the communities in which they work, thus helping them identify and address local needs and priorities. There was less agreement among CFLI-supported organizations on whether the duration and amount of funding provided by the CFLI was sufficient to address the needs of local communities; 70% of organizations agreed that the amount of CFLI funding provided addressed the immediate needs identified in their project proposals. 

The mission survey also highlighted how mission staff understood the CFLI as a tool focused on the needs and priorities of local populations; 76% of respondents indicated that the CFLI supported an area of work, or stakeholder group that would otherwise not receive funding from Canada or other major donors. HOMs, especially in small missions, expressed the significance of the CFLI in responding to local contexts and providing Canada with a mechanism to respond. According to a HOM, “without a development program—which many of our “competing” missions have—it [the CFLI] allows us to demonstrate our commitment to key priority areas, in particular with the NGO community and civil society.” 

The CFLI’s focus on supporting local organizations was a key design strength.

Evaluation evidence demonstrated that a key strength of the CFLI’s design was its focus on developing and supporting new partnerships with small, local organizations. Over 80% of CFLI organizations surveyed for the evaluation were local non-governmental organizations or community-based organizations that had never received any Government of Canada (GoC) funding before. These organizations offered the CFLI valuable knowledge, networks, and abilities in responding to local needs and issues. Our analysis stressed that CFLI partners saw the experience as overwhelmingly positive, often remarking on the transformative nature of the opportunity. In many instances, the CFLI was the most visible, accessible, and often only option for local organizations to access international funding.

Testimonies:

“Small contributions allow the CFLI to take risks, to finance innovative projects.” - CFLI HOM

“Our risk framework needs to be adjusted. We need to accept certain things and learn from them so we can benefit.” - CFLI HQ staff

“If you want to give new groups a chance there must be room for failure…failure is also good to learn how to grow.” - CFLI mission staff

By design, the CFLI was expected to take risks to support local partners. However, the risk framework and guidance in place did not optimally support risk-tolerant programming.

As per the CFLI Guidelines, missions were encouraged to take risks with new and untested local organizations, particularly those with innovative ideas that represented potential strategic partners. Evaluation evidence demonstrated that the CFLI was seen as an opportunity to allow greater risk tolerance for departmental programming. In fact, allowing for more risk was noted as important for the CFLI to continue its focus on funding small, local organizations. However, the CFLI’s risk framework and associated guidance in place did not optimally support risk-tolerant programming. For example, the Risk Assessment Tool detailed how to identify and document risks but did not describe the amount and type of risk that is acceptable or encouraged to fund new and untested organizations. Similar gaps in existing departmental guidance on risk appetite and risk management were also seen as shortcomings. It should be noted, however, that a new “Risk Appetite Framework” is currently in development that aims to address some of these gaps. With the current guidance, the level of risk tolerance varied from mission to mission, which affected the types of organizations that received funding. This often resulted in a reluctance by missions to partner with newer, less experienced local organizations to avoid risks. The evidence confirmed that the CFLI’s risk framework needs to be adjusted, with the adoption of processes and systems that are more conducive to risk taking and that allow for learning, innovation and improvement. 

The CFLI application process was deemed fair, transparent and accessible by successful organizations, but onerous for potential applicants and CFLI staff.

According to the CFLI Guidelines, each mission with a CFLI program was required to solicit applications in a public manner, such as using mission websites, CFLI calls for proposals web pages, and social media. Evidence from the organization survey indicated that in 2020-21, organizations that were successful in applying for CFLI funding saw the process as largely fair (87%), transparent (87%), and accessible (88%). However, evidence from mission staff interviews and mission case studies described the application process as onerous (language barrier, format, length, etc.) for many small organizations with little to no experience in formal bilateral donor proposal writing; 82% of mission staff surveyed believed that organizations couldn’t address the requirements needed to successfully be approved for CFLI funding. Elements of the application and selection process (e.g. length of proposals, number of applications) made it onerous for mission staff and CFLI coordinators and led to delays in selection and money transfers. These delays had a cascading effect on the ability to deliver within already tight timelines. To mitigate these barriers, many missions experimented with various approaches, such as “concept proposals” and “preliminary applications for funding”. Additionally, the CFLI team at HQ recently developed a Preliminary Application for Funding template, which aimed to lessen the burden of the application process for both applicants and mission staff. 

Findings – Delivery

Delivery

The CFLI team at HQ provided clear direction to execute the CFLI at mission and was responsive to the requests for assistance. Its support was considered a positive factor for CFLI program delivery.

Evidence from mission staff highlighted that the CFLI team at HQ was always responsive, knowledgeable, and flexible. The delineation between CFLI’s HQ and mission roles and responsibilities was clear and was said to work well for the delivery of the CFLI. The CFLI HQ team also provided direction and quick support, which was understood as crucial to mission staff in executing their CFLI responsibilities. Additionally, it was noted that there had been a strong improvement over the last five years in areas of training, template updating, and process streamlining. However, challenges occurred where, at times, HQ did not understand the realities of the local context, including the need for more flexibility in areas such as payment modalities (e.g. use of cash) and deadlines. 

Human resources capacity was a major barrier affecting program delivery at both mission and HQ.

Data from multiple lines of evidence (interviews and surveys with HOMs, HQ staff, and mission staff) showed that human resources (HR) was the top concern for all staff in the execution of the CFLI at both mission and HQ. According to consulted stakeholders, HR pressures were exacerbated both at mission and HQ by antiquated and resource-intensive program processes and systems, increased periodic administrative workload, and consistent staff shortages.

63% of CFLI mission staff surveyed reported that they could deliver the CFLI with current HR levels. HR was also the most quoted barrier by surveyed HOMs (33% selected HR as the top barrier, ahead of Partner capacity 22%, and workload requirements 22%). It should be noted that no accurate year-over-year data is available on CFLI staffing at missions (number of staff, percentage of resources spent on CFLI) and, because of this, no overarching resource gap analysis was conducted for the CFLI during the period under review.

At HQ, to supplement available staff (see sidebar) and deliver on current operations, the CFLI team heavily relied on students, casuals and junior officers, including for critical roles. This was understood by staff as a significant program risk as it created inefficiencies, led to ad hoc burden sharing, and more importantly limited the team’s ability to address more strategic and systemic issues to improve program delivery. Available administrative data (see sidebar) was not useful to identify HQ staff shortages for the period under review, but showed the potential for higher HR needs in case of a significant budget increase.

The evaluation pointed to a need to systemically monitor human resources needs for the CFLI at both HQ and mission; to assess the risks and tradeoffs related to delivering the program with current staffing levels; and to reflect on implications of the expected increase in investment levels on staffing needs.

Source: CFLI Ecuador (2019)

Source: CFLI Canberra (2019)

While an experienced and stable CFLI coordinator was a positive factor for program delivery, there are significant risks in keeping this position external.

CFLI coordinators were the face of the CFLI program. They undertook the day-to-day administration of a mission’s CFLI program and reported to the CFLI program manager (an FPDS staff member at mission). The position was the nexus point between civil society, local organizations and networks, and the Canadian mission. Evaluation respondents saw the CFLI coordinator as a major enabler of program success and execution. Due to the significant cost and commitment of hiring locally engaged staff, CFLI coordinators were largely external contracted consultants with short-term (between 1 to 5 years) contracts.  This insecure workforce created concern from GAC staff over the potential reputational risk to Canada associated with contractors being “the face of Canada”. In addition, mission staff interviews flagged reputational risks due to heavy coordinator turnover, including the systematic retention of network contacts, the heavy HR burden of re-contracting and training of consultants, and consistent branding/communications of the program. 

External contracted consultants are not GoC employees and as such have restricted access to mission tools, information technology (IT) services, office space and training. These restrictions, meant to avoid creating an employee-employer relationship in alignment with contracting guidelines, generated several inefficiencies for the CFLI. For example, receiving emails from a non-government address (e.g. Gmail) often created confusion with partners and cast doubt on the legitimacy of the CFLI program. To mitigate these concerns, some missions provided external coordinators with office space, a GAC email address, training, a government workstation and renewed contracts over multiple years. These situations posed a serious liability risk to the department, however, as they pushed the boundaries of a non-employer-employee relationship. All interview groups agreed that resolving the issue of the external coordinator is imperative to improve CFLI delivery.

Testimony:

“CFLI’s data management system is ‘one creaky bridge’.” - CFLI HQ staff

While training was generally well received by CFLI staff at mission, it didn’t fully meet current programing needs.

Although not an explicit requirement of the program, the CFLI team at HQ coordinated and delivered regional training and pre-deployment training sessions for both Canada-based staff (CBS) and locally engaged staff (LES) at mission. Before the pandemic, regional training was provided in-person to a maximum of two regions a year, limiting its reach. Since 2020, all training was delivered virtually, making it more accessible for CFLI staff. 

CFLI training played a critical part in staff integration into the CFLI business process and culture. When delivered in person, GAC mission staff saw the training as a positive contributing factor to their self-confidence in delivering the CFLI and networking/knowledge sharing with other missions. However, evaluation evidence highlighted that FPDS staff training on traditionally development-oriented tasks (Results-based management, contracting, project management, contribution agreement development, identifying and collecting intended results, etc.) was not sufficient to meet the programming needs of the CFLI. This resulted in varying levels of program understanding, capacity, and delivery. Additionally, training for contracted CFLI coordinators was limited to avoid creating an employee-employer relationship, but left a knowledge gap, and an over-dependence on informal learning for CFLI coordinators.

The lack of a systematic and transparent data management system exposed and continues to expose the CFLI to significant institutional risks.

Evaluation evidence (interview and document review) highlighted significant issues with the CFLI data management system and processes. Collecting, collating and storing CFLI data (e.g. financial, contract and partner/project-level data) was inefficient and burdensome for both mission and HQ staff. The CFLI used an antiquated data management system and was one of only two Grants & Contributions programs not fully integrated in the use of purchase orders (PO) in the departmental financial management system (FAS-SAP).

In addition, evidence confirmed that these issues exposed and continue to expose the CFLI and the department to institutional risks. For example, with the current systems, the CFLI was not well placed to fulfill its accountability requirements in response to audits, proactive disclosure, Access to Information and Privacy (ATIP) requests, parliamentary requests and the Treasury Board’s requirements for Gs&Cs programming. To address these issues, there was a near uniform call from HQ staff to integrate the CFLI program fully into FAS-SAP. However, interviews also identified several key bottlenecks. Most importantly, technical issues did not allow certain permissions in FAS-SAP for mission staff. This was a key challenge given that the CFLI was managed by the FPDS section at mission and not by HQ staff. 

According to survey data:

• 62% of mission staff agreed that program-level reporting requirements were lengthy.
• 48% of mission staff agreed that program-level reporting requirements were burdensome.
• 52% of mission staff agreed that program-level reporting was used to inform decision making at mission.

The utility of both the CFLI end-of-year (HQ) and end-of-program (mission) reports was limited. They functioned as corporate accountability exercises rather than strategically useful tools for program management.

Each CFLI mission was required to draft and submit an annual CFLI end-of-program report to HQ, which fed into various corporate reporting products and formed the basis of the CFLI end-of-year report. Evaluation evidence highlighted that the end-of-program reports were often burdensome and not useful as a management tool for missions. Additionally, the evaluation’s quality check highlighted their lack of clear purpose beyond informing HQ reporting products. Their format (length, content, strategic/analytical input) was inconsistent and made them unfit as stand-alone reports for management purposes (mission and HQ), or for operational (mission-level) learning. 

Moreover, end-of-year-reports were designed and populated based on the data requested by the CFLI Program Management Board (PMB). Since this can change year after year, these reports did not permit year-to-year comparisons. Additionally, end-of-year reports were not leveraged for other strategic purposes beyond informing the PMB. The extent to which they were useful for the PMB was not documented in the evidence. 

Challenges with the availability and quality of performance data undermined the CFLI’s ability to report on results.

There were several challenges with the CFLI’s ability to report on its results. First, issues with both defining and measuring CFLI’s expected results limited the availability of relevant CFLI performance data (see sidebar).

Additionally, evaluation evidence identified challenges with the CFLI data collection systems and processes at multiple levels (HQ, mission and project), which affected CFLI data quality. For example, interviews highlighted the lack of a standardized process to compile CFLI performance data from missions to feed into corporate reporting products. Without a central reporting system to track performance data, relevant CFLI data were pulled from multiple documents by multiple individuals and then manually rolled up into reporting products, creating a high potential for error. Compounding this issue, missions reported challenges in respecting reporting deadlines. Mission-level data were often submitted late to HQ, resulting in incomplete data sets at HQ. Interviewees also noted the lack of a systematic quality check process in place to ensure data quality.

At the mission level, the evaluation’s quality check of end-of-program reports identified several inconsistencies between reports, highlighting issues with managing and interpreting data. At the project level, CFLI partner capacity to report on results was deemed limited by mission staff, despite positive self-assessments from partner organizations.

Findings - results

International assistance - development

According to survey data:

• 74% of surveyed organizations received informal guidance and/or coaching from CFLI staff at mission.
• 21% of surveyed organizations received formal training and/or workshops from CFLI staff at mission.
• 13% of surveyed mission staff mentioned local capacity building as an intended use of the CFLI.

While the CFLI increased the capacity of multiple partner organizations to manage and deliver their projects, the capacity of organizations in general was considered a barrier to CFLI program delivery. There was no systematic approach across missions to build partner capacity.

“Increased capacity of local organizations...” is one of the expected outcomes of the CFLI (see original LM, Annex V), however, there was no systematic approach across missions to build partner capacity. Capacity building support was not mandated in CFLI guidance documents, and mission staff did not always see local capacity building as one of the intended uses of the CFLI. The frequency and quality of these activities ranged greatly from mission to mission, with some missions unable to provide capacity building support due to resources and capacity. See sidebar. 

The evaluation evidence emphasized that the capacity of organizations in general was considered a barrier to delivering the CFLI; 50% of interviewed local mission staff (LES) mentioned partner capacity as the top barrier to CFLI program delivery (tied with human resources) while 22% of surveyed HOMs considered it a key barrier. Frequently cited capacity issues included, but were not limited to, difficulties with integrating a gender-sensitive approach in written proposals and project design, financial management as well as monitoring and reporting on project progress and results. Capacity issues should not come as a surprise given that working with small, often inexperienced local partners means having to accept the risk that they might not have the capacity to deliver programming as expected. However, in alignment with GAC’s localization agenda, this barrier pointed to a need for more sustained efforts to strengthen local partner capacity to effectively deliver projects.

That said, evaluation evidence demonstrated that in multiple instances and in a range of contexts, the CFLI increased the capacity of partner organizations to manage and deliver their projects. According to HOMs and mission staff, the CFLI helped local organizations gain experience to become eligible for other international funding opportunities, thus contributing to the strengthening and growth of local civil society. Evidence from mission case studies highlighted that the CFLI had also built organizations’ capacity to expand their networks and develop contacts with new stakeholders. In some cases, CFLI funding resulted in increased confidence and capacity of organizations to advocate for shared values. As a result of working with the CFLI, 75% of organizations surveyed confirmed that their capacity to manage and deliver projects improved. According to these organizations, the main capacity gains were in the areas of financial management, project delivery, gender-sensitive programming, relationship building and networking, as well as reporting. 

Organization survey data highlighted the following key words in relation to areas of change experienced by project beneficiaries:

According to survey data:

• 74% of mission staff agreed that CFLI improved international assistance outcomes in the missions’ countries of accreditation.
• 77% of HOMs agreed that CFLI was effective in contributing to the achievement of Canada’s priorities for international development.

Local beneficiaries of CFLI-funded projects experienced a wide range of positive results, including increased participation in addressing development challenges in their local country context. These results were commensurate with the limited scale and reach of CFLI projects.

According to the organization survey, project beneficiaries and their local communities experienced a broad range of positive changes as a direct result of participating in CFLI-funded programming, as shown below:

Organizations further highlighted the results achieved by CFLI projects in areas such as growing micro and small businesses, education, advocacy for and protection of human rights, empowerment of women and girls, health and nutrition, food security, environmental sustainability, democracy and access to justice to name a few. Similar results were noted in multiple examples in CFLI’s mission-level end-of-program reports. Evidence highlighted that while the CFLI funded small projects, significant positive results were experienced in the communities that they served. CFLI’s contributions to Canada’s international assistance priorities were also highlighted by consulted mission staff and HOMs (see sidebar). 

Several examples of these local results are provided in Annex II. However, despite the positive evidence summarized above, it should be noted that the limited size, budget and timeframe of CFLI projects, along with its diverse operating contexts, differentiated the CFLI from more traditional development programming. CFLI projects results tended to be small in scale, scope and reach. Results also tended to be very localized and diversified. Moreover, the evidence pointed to challenges with respect to partner capacity to report on results and mission capacity to monitor project progress (e.g. conduct site visits). Post-project evaluations were not conducted for CFLI projects due to their limited size and duration. Thus, reported CFLI results cannot be generalized to the CFLI program as whole.

Humanitarian assistance

Survey Reponses on CFLI Humanitarian Assistance

The CFLI was a useful and appropriate mechanism for small-scale humanitarian and emergency response programming, though the CFLI project approval process hindered its timely delivery.

As per the CFLI Guidelines, the CFLI humanitarian reserve provided financial support up to $50K (per project) to respond to sudden onset natural disasters and humanitarian emergencies. It was intended to be used as an immediate response mechanism to complement other Canadian rapid response mechanisms. CFLI humanitarian assistance (HA) projects required the concurrence of the Natural Disaster Response Division (MHA) and the International Humanitarian Assistance Operations Division (MHI) before following the regular steps for a CFLI project approval. Concurrence was required by MHA and MHI to avoid duplication and to ensure coordination and coherence for HA programming. Evidence from HQ staff interviews confirmed that HA delivered through the CFLI was useful and complemented other Canadian HA funding mechanisms while supporting local organizations. The organization survey emphasized that the CFLI was both an appropriate mechanism to fund small-scale rapid humanitarian response and relevant to the needs of crisis-affected populations. 

Evidence from HOMs and mission staff was more nuanced. This should be interpreted in light of the fact that only a small proportion of mission staff and HOMs had experience with delivering HA programming through the CFLI (between 6 and 11 missions delivered HA programming per year). HOM interviews (from those with experience in CFLI HA programming) demonstrated strong praise for the CFLI as a useful tool to respond to emergencies. When it came to mission staff, interviewees with experience in CFLI HA programming reported strong results in providing targeted assistance. For example, the CFLI HA reserve was used to fund small projects that provided masks, gels, and food baskets to crisis-affected populations at the onset of the COVID-19 pandemic. Additional examples of HA results, expressed in the words of CFLI recipient organizations, are provided in Annex II. In contrast, both HOM and staff survey evidence was less positive. This may be due to the limited experience among respondents. Staff expressed mixed views on the effectiveness of the CFLI as an HA tool (see sidebar). Similarly, the HOM survey ranked CFLI HA effectiveness far lower than the other main CFLI components (e.g. development, advocacy and influence). 

In terms of barriers, mission staff in both interviews and surveys highlighted the issue of timeliness as a challenge to HA delivery. However, in interviews, mission staff stated that, although CFLI approval processes could be long and cumbersome, with planning and preparation, the CFLI could be made to work as a responsive and flexible tool to support immediate needs while waiting for other HA mechanisms to be put in place. 

Advocacy and influence

According to survey data:

• 92% of mission staff agreed that the CFLI increased their mission’s interactions with local stakeholders.
• 72% of organizations agreed that the CFLI increased their access to local decision makers.
• 89% of organizations agreed that the CFLI strengthened their relationships with local stakeholders.
• 80% of organizations agreed that the CFLI strengthened their relationship with Canada.

The CFLI was an effective tool for both missions and recipient organizations to open doors and engage with local stakeholders. As a result, the CFLI helped build networks and strengthen relationships.

Evaluation evidence demonstrated that the CFLI was an effective tool for opening doors and creating opportunities for engagement and interaction with a wide spectrum of local stakeholders, including local decision makers, for both recipient organizations and CFLI missions. It was also widely used by missions to promote Canada’s brand and visibility in partner countries.

For CFLI missions, the CFLI opened doors to both local decision makers and civil society. For example, mission staff noted that they made concerted efforts so that senior mission staff could mention CFLI projects in interactions with local contacts (during high-level visits, academic panel discussions, and/or meetings, etc.) and use these projects as leverage to foster engagement and to advocate for shared values and interests.^{Footnote 4} The mission case studies highlighted that in countries where Canada doesn’t have a physical presence on the ground, the CFLI was the only source of connections in, and insights into, the country. 

In terms of visibility, the evidence highlighted that the CFLI strengthened Canada’s reputation as a strong partner and supporter of local civil society among a variety of stakeholders, including other donors, local governments, civil society organizations (CSOs) and the communities they serve. However, the case studies noted that in those contexts, the CFLI did not necessarily expand Canada’s brand and visibility much beyond its engagement with local CSOs. It must also be noted that improved visibility was not always a desired outcome depending on the nature of the programming and the surrounding political and cultural context.

For recipient organizations, CFLI projects also facilitated direct access to local decision makers. Depending on the project, local decision makers were targeted as project beneficiaries or were consulted during project design and implementation. The mission case studies emphasized that in fact more attention was paid to creating entry points for recipient organizations than for missions to engage with other local stakeholders. 

The evidence further demonstrated that these opportunities for engagement, interaction and improving visibility had led to building and strengthening relationships for both Canada and the recipient organizations. From the mission perspective, CFLI helped strengthen Canada’s bilateral relations with foreign countries and their civil societies. CFLI’s recipient organizations emphasized that working with the CFLI helped strengthen their relationships with Canada as well as with other local stakeholders (such as local civil society and influencers). 

Advocacy, access and influence levels of change (mission case studies)

The CFLI increased local stakeholders’ awareness of shared values and interests, however, there is limited evidence demonstrating Canada’s influence through the CFLI and its impact in countries of accreditation.

Evaluation evidence suggested that local stakeholders improved their understanding and awareness of shared values and interests through CFLI programming, though this was generally limited to project beneficiaries and the local communities served by recipient organizations. There were minimal examples in the evidence demonstrating increased awareness at the level of local decision makers. For example, in the mission case studies, LGBTQ2 communities in Accra were reported to be more aware of their basic human rights. 

It is unrealistic to expect that the CFLI alone will contribute to results beyond increasing awareness, such as shifting wider attitudes and sustainable behaviour change, given the small size and short duration of each project. Moving along the advocacy, access and influence levels of change takes time (see action and impact levels in the sidebar). These results are also context-specific and depend on the numerous stakeholders involved. Furthermore, evidence at these higher result levels were not tracked or reported on by the CFLI. While there was an indicator for “increase the mission’s level of influence” in mission reports, there was a noted inconsistency in how this was measured and reported on, affecting the reliability of data. 

Although evidence at these levels of change was limited, the evaluation uncovered noteworthy insights from the perspectives of the CFLI program and partner organizations. According to 93% of mission staff, the CFLI meaningfully increased their mission’s level of influence in advancing shared values and interests in their countries of accreditation. An increase in influence is assumed to lead to increased willingness among local decision makers, change makers and the public to act in alignment with shared values and interests (see Annex III - Reconstructed Logic Model). Illustrations of behaviour and institutional change also surfaced in the evaluation evidence. One successful project on gender-based violence led to the creation of a law aimed to protect women and girls. In another project, recipient organizations highlighted the lasting impacts on attitudes to LGBTQ2 issues in wider Costa Rican society, including rural communities. 

Additionally, evidence showed that partner organizations were already aware of and aligned with shared values when applying for the CFLI as demonstrated by their project proposals. In fact, all selected projects had to be consistent with Canada’s priorities for international assistance and were expected to advance shared values and interests among local stakeholders. This may point to the fact that partner organizations are primarily a means rather than a target for missions’ advocacy efforts through the CFLI. 

Conclusions

Design

Several elements made the CFLI unique as a GAC G&Cs program: its duality as an FPDS-managed tool using international assistance funds; its decentralized governance structure; its focus on supporting directly local organizations; the very high number of funded projects; and the small scale and short duration of the projects.

These characteristics made the CFLI a useful tool for the department, and in particular for missions, to respond to local needs, pursue mission priorities with a high degree of flexibility and build relationships with local actors, all while aligning with and supporting wider departmental priorities. At the same time, they also made the CFLI a relatively resource intensive program to manage and deliver, in particular for missions. The duality of the program also led to multiple interpretations of what the purpose of the CFLI is and of what success looks like for projects that are short term and small by design. This hindered its ability to be consistently and optimally leveraged as a strategic whole-of-mission tool. In addition, the CFLI’s approach to risk management did not align with its willingness to be a risk-tolerant program when supporting local organizations. This resulted in a variance of risk appetites at the mission level and often a reluctance to partner with newer, less experienced local organizations.

There is a need to further ground the strategic direction of the program in a well-articulated foundational framework, while understanding what results can realistically be achieved by the CFLI, for each mission, in their own specific context. 

Results

The evaluation found evidence of positive results for all three CFLI program objectives. The CFLI helped increase partner capacity in a range of contexts, even though there was no systematic approach across missions to do so. However, the capacity of organizations remained a barrier to delivering the CFLI, thus pointing to a need for strengthened efforts by the CFLI in this direction.

The CFLI also contributed to a wide range of positive results at the local level and delivered useful humanitarian and emergency response programming. These results were difficult to generalize and limited in scope, scale and reach due to the short-term nature and small size of CFLI projects. Finally, the CFLI was an effective tool for access and engagement with local stakeholders and for promoting Canada’s brand, visibility and values. However, evidence demonstrating Canada’s influence and impact (through the CFLI) was limited.

Delivery

The evaluation highlighted several positive factors for program delivery. The support provided by the CFLI team at HQ was crucial to mission staff executing their CFLI responsibilities. Similarly, when appropriately trained and experienced, CFLI coordinators and mission staff were positive factors for program delivery. 

At the same time, there were multiple challenges that hindered program delivery. Human resources (dedicated CFLI staff) at both HQ and mission were widely reported as a main obstacle to deliver optimally the CFLI program. In addition, CFLI teams at mission did not consistently have the required capacity (e.g. relevant knowledge, skills, access) for effective program delivery. The fact that the majority of coordinators were external contractors also posed significant reputational and operational risks. The evaluation also pointed to the fact that CFLI data/financial management systems and processes (e.g. collecting, collating and storing CFLI financial, contractual and partner/project-level data) were burdensome, inefficient and not sufficiently transparent, thus posing a barrier to program delivery and exposing the CFLI to significant institutional risks. There were also major challenges with the foundational elements of CFLI’s performance management framework, limiting its ability to accurately define, measure, track and report on CFLI’s results.

There is a need to address these barriers to empower the CFLI to continue delivering quality and effective localized programming and to better articulate its successes, in particular in view of an expected increase in its funding. 

Recommendations and Considerations

Recommendations

Considering the 50th anniversary of the CFLI, the potential increase in annual investment of the CFLI program as per the Minister of International Development’s mandate letter, the Future of Diplomacy initiative launched by GAC in 2022 and the department’s commitment to the localization of international assistance, the evaluation has put forth the following recommendations:

1. Strategic positioning and leveraging of the CFLI

1. Reaffirm the CFLI purpose: The PMB should conduct a strategic review that clarifies and articulates the CFLI’s purpose, objectives and uses.
2. Update and operationalize guidance: NMS should update its guidance to reflect the outcomes of the PMB strategic review.
3. Strategically integrate and plan: CFLI missions should integrate the CFLI within mission-level strategic planning exercises.

2. Resources and capacity

1. Review CFLI HQ resources requirements: NGM, in collaboration with other GAC geographical branches, should identify human resource needs for the CFLI Unit at HQ (NMS) and explore solutions to ensure the effective delivery of the CFLI program.
2. Review CFLI mission resources requirements: GAC geographical branches, in collaboration with CFLI missions and CFLI team at HQ, should identify human resource needs for the CFLI teams at mission and explore solutions to ensure the effective delivery of the CFLI program. In addition, particular attention should be given to identifying appropriate staffing mechanisms for the coordinator role.
3. Strengthen CFLI mission capacity: GAC geographical branches, in collaboration with the CFLI team at HQ, should ensure that capacity (i.e. knowledge, skills, access and time) of CFLI team members and coordinators at mission is adequate to deliver the CFLI.

3. Data management processes and systems

• The Grants and Contributions Management Transformation Team (KFMT) in collaboration with NMS, CFLI missions, and relevant S-branch and A-branch bureaus should identify and implement solutions to overcome the challenges related to CFLI’s data/financial management systems and processes.

4. Performance management

• NMS should re-visit the CFLI’s performance management framework, systems and tools, to ensure they can accurately define, measure, track and report on CFLI results. Reporting templates at the project, mission and corporate levels should also be updated to improve their strategic use and utility.

5. Localization

1. Encourage a higher risk appetite: NMS, in collaboration with SRD and KFMT, should update the CFLI program’s current risk management approach and associated guidance to allow for a higher risk tolerance in alignment with SRD’s soon to be launched Risk Appetite Framework and GAC’s Grants and Contributions Transformation initiative.
2. Build local capacities: Bearing in mind capacity constraints, NMS, in consultation with GAC geographical branches and SGF, should develop guidance on how to intentionally support and strengthen the capacity of small, local organizations to serve as effective CFLI partners. NMS, in collaboration with CFLI missions, should identify innovative strategies to share and operationalize this guidance across different mission contexts.

Considerations

Considerations for improving GAC’s commitment to localization

• While localization is gaining momentum within GAC, recent evaluations (KFM, WVL, and CFLI) have highlighted that several key barriers still exist. 
• There is currently no definition, policy or guidance specific to localization within the department. Work on this issue is currently led by the International Assistance Policy Coordination (PVP) division. Additionally, GAC’s Gs&Cs programs are governed by several laws, regulations and guidelines that directly affect localization efforts, including barriers to funding local actors directly. Moreover, GAC’s current risk culture and lack of appropriate guidance on risk appetite can hinder partnerships with local actors. Finally, local organizations face barriers to receiving donor funding, including due to their own capacity to deliver programming.   
• To solidify GAC’s commitment to localization, the department should consider prioritizing and strengthening PVD/PVP’s current efforts to develop a concrete “localization of aid” policy and associated guidance. This policy would bring clarity on how to operationalize GAC’s commitments to localization in the FIAP and other international agreements while addressing the key barriers hindering these efforts. 

Considerations for designing and implementing localized programming

• The localization analysis framework (see Annex II for details) highlighted a set of key dimensions for designing and implementing localization programming.
• In support of improving GAC’s commitment to localization, the department should consider the following dimensions for programs with a strong localization of aid approach:

1. Location and organizational structure of implementing partners
2. Nature of funding provided to local organizations (i.e. direct and untied)
3. Relevance of project design to local needs and priorities, including the most vulnerable
4. Level of local ownership and leadership in project design
5. Level of local ownership and leadership in project management
6. Level of local ownership and leadership in project governance
7. Level of local ownership and leadership in project monitoring, evaluation and reporting
8. Relevance of project design to local needs and priorities, including the most vulnerable
9. Level of capacity-building support received by local partners
10. Relevance of capacity-building support to the organizational needs of local partners
11. Degree that operational and administrative requirements hinder localization programming

Considerations for measuring advocacy, access and influence in diplomacy

• Lessons from the CFLI evaluation highlighted several challenges and limitations with measuring the effectiveness of the department’s advocacy, access and influence efforts.
• These challenges are not new for the department. They have emerged in other recent initiatives, notably through the work of the FPDS ginger group and of the Planning and Reporting Team in NMS. 
• Given the importance for the department of accurately and systematically defining, measuring and reporting on Canada’s diplomatic efforts and results across the world, including in light of The Future of Diplomacy: Transforming Global Affairs Canadainitiative, GAC should consider further building on and strengthening these efforts. 
• In particular GAC should consider building on the Advocacy, Access and Influence measurement tool piloted for the CFLI evaluation to strengthen the department’s capacity in this area and develop frameworks and tools that are flexible and easily adaptable to the various contexts in which Canada’s diplomatic missions operate. 

Annexes

Annex I: CFLI key events and milestones

Annex II: The evaluation approach to localization

Source: CFLI HQ (2022)

Greater localization of international assistance (IA)—broadly understood as shifting decision making, resources, power, capacity and project/program management to local partners (including national and local governments, and/or national and local CSOs)—is gaining momentum. Notably, the impact of COVID-19 has demanded an increased reliance from donors and international CSOs on local actors for delivery of international assistance. 

Current global efforts by Canada and other donors to operationalize localization include: direct funding (e.g. budget support and providing direct or indirect funding to local civil society partners), greater local presence in decision making (e.g. participatory development), investing in partner capacity (e.g. technical assistance), and opening spaces for participation and decision making in aid governance. 

Canada has long been engaged in local ownership of IA, particularly through consultation, capacity building, untied aid and delivery of humanitarian assistance involving local stakeholders. The Feminist International Assistance Policy (FIAP) commits Canada to building local capacity and working with local and national actors, including women’s groups, in design, delivery, and monitoring of international assistance initiatives. To this end, Canada is building on traditional development programming implemented by local actors, with technical assistance and capacity building from Canadian or international organizations and often involving local governments to increase sustainability. Signature GAC programs such as the CFLI, but also Women’s Voice and Leadership (WVL), and the Equality Fund (EF) have a strong local focus and channel funding to local organizations. 

GAC is at an important juncture to assess how these efforts improve the effectiveness and efficiency of international assistance and therefore the outcomes for beneficiaries. There is a growing interest to ensure that its policies, capacities, process and systems are well aligned to respond to the call to deepen the culture of localization throughout Canada’s IA. 

The CFLI evaluation provided an opportunity to learn from an existing program dedicated to localized projects. Consequently, localization features prominently throughout the report.

The following slides on localization provide a summary of the CFLI’s localization analysis as well as highlights of key CFLI results directly from the voices of local stakeholders.

Annex II: CFLI Localization Analysis

The localization analysis framework was designed to assess the extent to which the CFLI and WVL programs aligned with a localization of international assistance (IA) approach. It was also used to identify key barriers and enablers for localization programming for both programs as well as for the department more generally. The framework is based on 10 localization key dimensions (see sidebar). These dimensions encompass key elements of localization, however, they are not exhaustive and are still likely to evolve through further research.

Informed by the localization analysis against the key dimensions, the following key enablers and barriers were identified for the CFLI program in relation to its alignment with a localization of aid approach.

Annex II: Voices from the field - examples of CFLI’s development results

The following examples illustrate the range of development results experienced by project beneficiaries as a result of participating in CFLI-funded projects in 2020-2021. They are expressed in the words of CFLI recipient organizations and were taken directly from the organization survey.  

Source: CFLI HQ (2022)

Testimonies:

“Young women have created ‘Rural Young Women Ward Circles,’ which are platforms for engagement, and the leaders have a permanent seat in the Ward Development Committee representing the interests of young women. 10 young mothers are now financially stable through the young women sewing project initiatives. The junior council is now dominated by young female leaders.”

“Stakeholders in CFLI-funded projects have been strengthened and made more aware to be vigilant in reporting human rights violations. The more violations are documented and publicized, the more we see a reduction in these violations or the more the authorities become aware of them to increase vigilance.”

“All the schools that we are working with have established great and deep relationship with the students and teachers…Also…students from different schools and different ethnical backgrounds managed to work together and even make very good friends between them and help each other. In this way showing to the society that we can work together.”

“Project beneficiaries were heavily involved in the development and implementation of the project, elevating them to leadership roles while also reaping the benefits of the project outcomes. They have increasingly engaged with their communities…bringing necessary information into the homes of girls and women that they likely wouldn’t be able to access elsewhere.” 

“We monitored the obstacles [related] to access to reproductive health care during the COVID-19 health emergency. This…allowed us to approach the Ministry of Public Health to demand that these services be recognized as essential and that they not be discontinued despite the health emergency. We also [sensitized] women about these rights and the need to continue demanding these services.”

“Putting acquired knowledge into practice. Exercise of leadership of women community leaders, integrated, participating more actively in their organizations, articulating initiatives with key local actors for community development; female leadership was strengthened and they were encouraged to run as candidates in recent elections…influencing the local public agenda.”

Source: CFLI HQ (2022)

Annex II: Voices from the field - humanitarian and emergency response results

The following examples illustrate the range of humanitarian and emergency response results experienced by project beneficiaries as a result of participating in CFLI-funded projects in 2020-2021. They are expressed in the words of CFLI recipient organizations and were taken directly from the organization survey.

Source: CFLI HQ (2022)

Testimonies:

“Our focus was to help other civil society organizations to adapt their services to the COVID-19 pandemic. After the training, they held step-down trainings and adapted their SOPs to limit the chance of exposure and be more aware of beneficiaries’ rights and dignity in general.”

“Adoption of protective measures against COVID-19, especially the use of masks and hand washing; self-repair of homes affected by hurricanes; consumption of safe water and food in the first moments of the emergency.”

“As this project addresses the humanitarian needs of returned migrants and migrants in transit, we were able to deliver hygiene and protection kits for COVID-19, as well as training for better care by the personnel receiving them at the reception points.”

“Support provided as COVID-19 assistance to orphanages, elders homes etc. made a big change in their ability [to] face COVID-19 during the height of the pandemic.” 

“The project allowed people from vulnerable groups…to have access to and learn more about the COVID-19 pandemic through the development of messages that allowed them to better understand this issue, messages that were also adapted and translated into the different languages existing in the country, which contributed to a change in behavior and voluntary access to vaccination against COVID-19.”

“Better awareness about the worsening pandemic; leaderships with better capacities to act in emergency situations, developing local solutions.”

Source: CFLI HQ (2022)

Annex III: Reconstructed CFLI logic model

Reconstructed CFLI logic model

Annex III: Reconstructed CFLI logic model definitions

Recipient organizations: as per CFLI Terms and Conditions, eligible recipients include:

• Local non-governmental, community and not-for-profit organizations
• Local academic institutions working on local projects
• International non-governmental organizations working on local development activities
• Intergovernmental, multilateral and regional institutions, organizations and agencies working on local development activities
• Municipal, regional or national government institutions or agencies of the recipient country working on local projects
• Canadian non-governmental and not-for-profit organizations that are working on local development activities

Key decision makers: local, regional, national government authorities.

Key change makers: Stakeholder groups that influence change in local communities. They can include recipient organizations as well as other local activists and influencers.

Local/other stakeholders: Can include members of local communities, vulnerable and local populations, CSOs, local governments and institutions.

Crisis-affected populations: Can include women, men, boys, girls, refugees, internally displaced persons, asylum seekers, sexual minorities, and LGBTQ2 people who suffer the effects of natural disasters and humanitarian emergencies.

Gender-sensitive projects: Incorporate a gender-based analysis plus (GBA+) and gender equality perspectives approach appropriate to projects of this size; however, the objective of the project may not be to change gender roles or inequalities. 

Humanitarian and Emergency Response Programming: The CFLI humanitarian reserve is to be utilized as an immediate response mechanism at the local level and is intended to provide very quick Canadian profile, to complement other rapid-response mechanisms. All humanitarian-related CFLI programming will support Canada’s broad humanitarian objectives—namely, saving lives, alleviating suffering, and maintaining human dignity—and should be in accordance with relevant international humanitarian guidelines such as the Sphere Project guidelines and the principles of the Good Humanitarian Donorship initiative. This can include: material relief assistance and services, emergency food aid, and protection and support services. (Source: CFLI Guidelines p.35)

Shared values and interests: Includes, but not limited to, respect for democratic development, human rights, gender equality, and peace and security. (Source: CFLI Guidelines p.1)

Standing: Refers to Canada’s bilateral relations with foreign countries and their civil societies as well as Canada’s reputation within countries where CFLI engages.

Annex IV: Summary of results of the pilot on measuring advocacy, access and influence

Background

The Evaluation Division (PRA) commissioned a UK consulting firm, ITAD, to develop a framework for measuring advocacy, access and influence within the CFLI. ITAD conducted 8 ‘deep dive’ studies with Canadian missions in Accra, Bangkok, Budapest, Canberra, Lima, Nairobi, San Jose and Tunis covering issues such as promotion of LGBTQI+ rights; and have brought these together into a set of findings and advice on how the CFLI can be better used to advance Canadian values and interests through advocacy, access and influence.

Findings

Context: Canada’s clear policy direction created the potential to promote change but country context (including Canada’s status there) very much shaped opportunity.

Selection: Recipient organisations aligned with Canadian values but project selection, in a largely reactive process, was not always strategic.

Access and engagement: CFLI funding mostly created entry points and access to decision-makers for CSOs, not the mission. A behind-the-scenes approach on sensitive issues empowered CSOs and capitalised on their understanding of local politics and trends. The CFLI allowed Canada to identify potentially influential change agents and deepen relations with them. Their insights where Canada has no in-country presence were particularly valued.

The outreach of recipient organisations largely determined how far the CFLI increased understanding and awareness of shared values, although there are some instances where authorities and allies increased awareness. CFLI funding resulted in increased confidence of CSOs and enhanced their ability to advocate for shared values. CSOs gained in experience and credibility which improved their chance of securing other funding.

Action: In most contexts, it was unrealistic to expect the CFLI to shift wider attitudes and behaviours, especially when the government was not empathetic. But in propitious contexts, the CFLI helped bring about change in awareness, willingness and behaviour. Small, short CFLI projects were not likely to catalyse systemic change, but there are examples across the case studies where it helped in implementing mandated changes.

Benefits: Context was key. Benefits for CSOs sometimes included access to decision makers or safe spaces to build consensus but in more hostile contexts for minorities did not go far beyond reducing the sense of isolation. The CFLI gave Canada credibility – with CSOs, activists, and the Canadian public. Evidence that insights were used to inform policy, diplomatic engagement and reporting was limited. The benefits claimed for Canadian diplomacy could have been better evidenced.

Whilst Canada was respected by international partners where it played a facilitating or convening role, there was limited evidence that the international community engaged specific change makers using knowledge and insights shared by Canada or CSOs.

Summary of suggestions

1. Differentiate by context

In countries where Canada is not present or only thinly resourced, its influence will only ever be marginal. But in some it can be significant. The CFLI should set up a light-touch global approach to monitoring and reporting, seeking more robust evidence only where resources and context suggest it is realistic to do so.

2. Purposeful approach to selection

Where Canada is already significant, the CFLI should take a more purposeful approach to selection: i) using analysis to inform Strategia objectives and stimulate proposals; and ii) tracking effect of supporting CSOs and empowering diplomacy in promoting change aligned with Canada’s values.

3. Sharing and using insights and learning

The CFLI should i) ensure that CFLI relationships and insights are identified in diplomatic reporting; ii) drawing on the tools of the ITAD work, pilot deep dives on wider Canadian diplomacy to learn what is working best in promoting objectives in contrasting contexts, through synergy of assets.

Annex IV: CFLI Advocacy, Access and Influence Framework

ITAD developed, tested and refined an advocacy, access and influence measurement framework. The purpose was to address some of the data gaps in the CFLI evaluation while also creating a framework that could be adopted, adapted, and leveraged by the program and the department more broadly. The framework is designed to be used as a collation device to assemble evidence of change along advocacy and influence pathway at country level. It recognizes that change pathways are not necessarily linear, nor is it expected that change will be seen across all dimensions. As such, the framework enables the identification of 2-3 key stories of change relating to specific country plan themes/objectives. ITAD then conducted deep dives at 8 Canadian missions abroad.

Annex IV: CFLI Advocacy, Access and Influence Framework (continued)

Annex V: Original CFLI logic model (as per the CFLI performance information profile)

Original CFLI logic model (as per the CFLI performance information profile)

* Humanitarian assistance can include: material relief assistance and services, emergency food aid, and protection and support services.

** Crisis-affected populations can include: women, men, boys, girls, refugees, internally displaced persons, asylum seekers, sexual minorities, and LGBTQ2 peoples who suffer the   effects of natural disasters and humanitarian emergencies.

*** Gender-sensitive projects incorporate a gender-based analysis and gender equality perspectives appropriate to projects of this size; however, the objective of the project may not be to change gender roles or inequalities.

**** Stakeholders can include members of local communities, vulnerable and local populations, civil society organizations, local governments and institutions.