Evaluation of the Partnerships for Development Innovation Branch, 2015-16 to 2019-20

Evaluation report

Prepared by the Evaluation Division (PRA)
Global Affairs Canada
November 2021

Acronyms and abbreviations

CSO

Civil society organization

FIT

Fund for Innovation and Transformation

GE

Gender equality

GoC

Government of Canada

KED

Engaging Canadians Bureau

KEG

Global Citizens and Engagement Division

KEGC

Engaging Canadians Section

KES

Business Intelligence and Process Division

KFM

Partnerships for Development Innovation Branch

KGA

Innovation and Impact Division

KGD

Inclusive Growth, Governance and Innovation Partnerships Bureau

KGS

Inclusive Growth and Governance Division

KSD

Canadian Partnerships for Health and Social Development Bureau

KSI

Education and Social Empowerment Division

KSN

Canadian Partnerships for Health and Nutrition Division

KWD

Partnership for Gender Equality

LCAS

Strategic Communications, Development

M&E

Monitoring and evaluation

ODA

Official development assistance

PFM

Strategic Policy Branch

SGBV

Sexual and gender-based violence

SMO

Small and medium-sized organizations

Executive summary

This evaluation examined Global Affairs Canada’s Partnerships for Development Innovation Branch (KFM) for the period 2015-16 to 2019-20. The objective was to determine the effectiveness of the Branch’s approach to working with Canadian civil society organizations (CSOs) in advancing policy and programming objectives and engaging Canadians in international assistance. The report presents the evaluation findings, conclusions, recommendations and considerations to support decision-making and policy and program improvements. 

The Feminist International Assistance Policy, the Civil Society Partnerships for International Assistance Policy and other ministerial commitments introduced during the evaluation period shifted the Branch’s policy and program priorities. The Branch adapted its organizational structure to respond to new priorities but had gaps in addressing policy work related to working with Canadian CSOs. Several human resource issues constrained the Branch’s responsiveness. These included lower levels of permanent program staff, limited specialist capacity and workload fluctuations brought about by the cyclical nature of calls for proposals. Overall, partner selection mechanisms were a good fit to meeting established priorities. However, procedures tied to project selection and the finalization of funding agreements were lengthy, cumbersome and impacted project implementation. The Branch was effective in supporting partners’ application processes, and it expanded CSOs’ opportunities to target gender equality in programming. Funded programming—mostly multi-country and multi-sectoral—presented the Branch and department with opportunities, but also created challenges tied to monitoring and evaluation beyond immediate outcomes at project level and at the program (KFM portfolio) level. 

The Branch supported CSO programming that engaged diverse groups of Canadians in international development but lacked a strategic framework to direct and measure public engagement. It did not consistently demonstrate its commitment to diversifying partnerships in selection mechanisms; however, the CfPs targeting small and medium-sized organizations were a notable success. Although the Branch leveraged multi-stakeholder groups and social media to share knowledge from CSO experiences, it could benefit from structuring knowledge-exchange activities more consistently. The Partnership lead role helped coordinate relationships with CSOs and within the department, though it was used mainly as an information source internally. Enhanced collaboration with missions improved departmental programming and the visibility of Canada’s international assistance. 

Box

Background

Branch mandate

Source: CFO-Stats (2021-05-27), Global Affairs Canada, 2015-16 to 2019-20.

Branch overview

The Partnerships for Development Innovation Branch (KFM) is the focal point for Global Affairs Canada’s relations with Canadian organizations. The Branch puts the Feminist International Assistance Policy into operation and provides direct support to Canadian civil society organizations (CSOs) that partner with local actors to alleviate poverty in the Global South and with organizations engaging Canadians on international assistance issues in Canada. The Branch leverages Canadian CSOs’ expertise, knowledge, networks and resources, and facilitates knowledge exchange across the department with civil society and other partners to advance innovative approaches to development. Between 2015-16 and 2019-20, the Branch spent $1.4B on 330 projects implemented by 183 Canadian CSOs through programs grouped thematically under its bureaus.

Branch bureaus

Canadian Partnerships for Health & Social Development (KSD) manages health and social development partnerships programming, which includes reproductive, maternal, newborn and child health and rights, nutrition, education and social empowerment, as well as strategic initiatives. It is the Branch’s largest bureau, with $725.1M spent between 2015-16 and 2019-20 to fund 108 Canadian CSOs to implement 165 projects.

Engaging Canadians (KED) engages Canadian partners and individuals through global citizens programming, public engagement and policy dialogue, and supplies operational support services. Key programming targets include volunteers, youth and indigenous youth, diaspora communities, and diverse geographic regions. Between 2015-16 and 2019-20, KED supported 53 organizations with $338.2M to fund 95 projects. 

Inclusive Growth, Governance and Innovation Partnerships (KGD) manages programming on sustainable economic growth, inclusive governance, innovation, democracy, human rights and the rule of law, food security, and environment and climate action. It provides the Branch with sector expertise and other technical support and leads on corporate planning/reporting. It also leads on development innovation across the department, including through a community of practice. From 2015-16 to 2019-20, KGD supported 76 projects of 56 CSOs for a total of $237.0M in disbursements.

Partnership for Gender Equality (KWD) was a temporary taskforce set up in 2019 to develop the Partnership for Gender Equality, consisting of a $300M contribution to the Equality Fund. The Equality Fund is a partnership to create a sustainable and predictable source of funding for women’s organizations and movements in developing countries. It is a consortium-based project to which GAC contributed $105.0M in 2019-20. KWD was dissolved in 2020 and a few retained staff integrated the KGD Bureau once the Fund moved to the implementation phase.

Policy and program alignment

Contributions to GoC’s commitments

Branch’s response to a changing context

There were significant shifts in policy and programming priorities, funding selection mechanisms and operational modalities during the evaluation period which had an impact on the Branch and its partners. 

The Feminist International Assistance Policy and the Civil Society Partnerships for International Assistance Policy, both adopted in 2017, introduced new thematic priorities and ways of working. The Branch expanded its portfolios in the areas of maternal, newborn and child health (MNCH), sexual and reproductive health and rights (SRHR), education and gender equality. It also modified the levels of its commitment to other priorities such as economic growth, food security, and governance. In response to the Government of Canada’s 2015 commitment to making Canada a leader in development innovation and effectiveness, a new Development Innovation Unit was set up in the Inclusive Growth, Governance and Innovation Partnerships bureau.

The 2 policies introduced new principles and guidance governing the department’s engagement with CSOs. Among other things, the Branch took on the roles of departmental CSO policy lead and coordinator of departmental relationships with CSOs. This included creating a “partnership lead” role focused on ensuring consistent, coordinated and department-wide engagement with CSO partners. Though Partnership leads are present in different branches, most leads are in KFM. In the evaluation period, CSOs received funding from across the department’s branches. Large CSOs entered partnerships with the geographic branches for programming aligned with country priorities, and through the Global Issues Branch, particularly for humanitarian assistance as many CSOs share a dual development-humanitarian mandate. 

In addition, the Branch and other GAC branches along with CSOs (many of them represented by Cooperation Canada), set up working groups to address challenges raised during the 2016 International Assistance Review on ways of working and operationalizing partnerships between the department and CSOs. The Branch led departmental collaboration on specific commitments made in the working groups.

All these changes shaped selection mechanisms and operations involving the Authorized Programming Process (2013), Global Affairs Canada’s framework for programming processes. Comparative calls for proposals became the Branch’s predominant funding mechanism, and 13 Calls for Proposals were issued during the evaluation period. The Branch also used unsolicited proposals, department-initiated selection mechanisms and requests for proposals.  A mapping of the Branch response to thematic priorities and commitments of the Government of Canada and the department is in Annex 1.

Branch ODA to Canadian CSOs

Source: CFO-Stats (2021-05-27), Global Affairs Canada, 2015-16 to 2019-20.

Multi-sector and multi-country programming

Multi-sectoral and multi-country programming are 2 Branch focus areas. The top-10 sector disbursements from 2015-16 to 2019-20 varied markedly across thematic areas and bureaus. Basic and general health programming accounted for the largest percentage (28%) of Branch official development assistance disbursements to Canadian civil society. This contributed to the Canadian Partnerships for Health & Social Development Bureau’s large share (52%) of KFM funding to Canadian CSOs. Over the evaluation period, 60% of projects were implemented in more than one country. Of the projects implemented in a single country, 15% were deployed in Canada through Engaging Canadians Bureau (KED) programming. All multi-partner projects (consortia or others) covered multiple countries where each partner CSO had programming experience.

In-Canada programming

KFM is Global Affairs Canada’s main source of programming in Canada: it accounts to 96% of the department’s official development assistance allocated to public engagement in Canada. In addition to programs specifically targeting public engagement in Canada, many of the Branch’s envelopes allowed for a portion (up to 10%) of project funding to be spent on such efforts. Eight regional councils and 51 other CSOs received funding for programming targeting public engagement in Canada. Of the $64M total allocated to public engagement efforts through Canadian CSOs over the period, 67% was disbursed through the Engaging Canadians Bureau, almost 33% through the Canadian Partnerships for Health & Social Development Bureau, and just $0.2M (<0%) through the Inclusive Growth, Governance and Innovation Partnerships Bureau.

Countries in which KFM supplied 100% of Canadian CSO funding

Over the evaluation period, KFM-funded projects comprised 100% of the funding invested by Global Affairs Canada in Canadian CSO programming in 30 countries. Among countries where KFM disbursements were the sole source of contributions to Canadian CSOs, one third were in sub-Saharan Africa—63% received less than $1M and 30% received between $1M and $2M, 7% receiving more than $2M. Cambodia received the greatest amount over the period ($10.0M), followed by Tajikistan ($5.0M). KFM programming in these countries showed that the Branch played a key role in expanding the reach of Canadian assistance beyond countries served by the department’s other programming streams.

Evaluation scope and methodology

Evaluation scope and objectives

Evaluation Scope

The evaluation covered the 5-year period from 2015-16 to 2019-20 and included all ODA disbursed by the Branch to Canadian CSOs ($1.4B)—which represented 91% of Branch funding in the period. The process evaluation looked at the organizational structure, delivery mechanisms, approaches and practices related to CSO partnering. The evaluation considered a few of the immediate impacts of COVID-19-related measures, introduced at the end of the coverage period.

Objective

The evaluation assessed how the Branch responded to evolving policy and program priorities and the extent to which its structures and mechanisms were fit for purpose.  Its objective was to determine the effectiveness of the Branch’s approach to supporting Canadian CSOs in advancing policy and programming objectives and engaging Canadians in international assistance. 

The evaluation covered Branch pillars dedicated to Canadian Public Engagement and Partnerships for Development Innovation, as per the Program’s Performance Information Profile. 

Approach

The International Assistance Evaluation Division conducted the evaluation. External consultants supported the evaluation by undertaking a survey of Branch-funded CSO partners and completing an analysis of partners’ social media use aimed at engaging Canadians in international development. The evaluation team engaged a KFM Evaluation Steering Committee and a reference group of directors for insight and advice during all phases of the evaluation. This approach aimed to encourage stakeholder engagement in the aim of maximizing the use of generated findings and the impact resulting from the evaluation. 

Related evaluations

The previous evaluation of the Branch focused on program results achieved between 2010-11 and 2013-14. Three of that evaluation's 5 recommendations were relevant to the current evaluation, including improvements in performance management, internal approval processes, and a more systematic approach to knowledge transfer by the Branch. 

The recently approved Five-Year Departmental Evaluation Plan 2021-22 to 2025-26 includes an evaluation of the Partnership for Gender Equality (Equality Fund) and an evaluation of the Branch on innovation results.

Evaluation questions

Methodology

The evaluation used a mixed-methods approach, where data was collected from a range of sources using 7 main methods and triangulated across multiple lines of evidence.

Document review

A review of internal Global Affairs Canada documents:

• policy, planning and strategy documents 
• briefing notes and memos
• evaluations, audits and reviews

A review of external civil society documents:

• annual reports
• working group publications
• Web-based publications

Interviews

Semi-structured individual interviews were conducted with a variety of internal stakeholders at Global Affairs Canada and external stakeholders in Canadian civil society. This included:

• 29 Branch staff and specialists
• 14 representatives from other Global Affairs Canada branches, including at missions
• 42 representatives of Canadian partner organizations.

Project reviews

A review of project documents including approval documents, management summary reports, annual reports and other relevant documentation from 65 Partnerships for Development Innovation projects (KFM).

Survey

An electronic survey done in collaboration with Circum Network Inc. was distributed to a sample of 52 Canadian CSO partners. The sample was selected purposefully to ensure a wide representation of funded CSOs. In total, 44 out of 52 organizations responded, for an overall response rate of 85%. This represented data from 27% of CSO partners funded by KFM during the evaluation period.

Data correlation analysis

A correlation analysis measured the relationship between selection mechanisms, characteristics of organizations applying, proposal assessment ratings, and application outcomes. The analysis included:

• 248 applications under 13 Calls for Proposals
• 38 department-initiated proposals
• 20 unsolicited proposals  

Departmental data management

Data pertaining to the Branch were obtained from the Branch, Chief Finance Office, Corporate Communications (LCA), extracted from Spectrum and other data sources to prepare statistical descriptive analyses related to: ​

• funding distribution (based on official development assistance) ​
• partnerships data​
• funding mechanisms​
• allocations by bureaus
• profiling of Branch program content in corporate communications​
• CSO data by sector, types, coverage. ​

Social media analysis

An analysis of social media was completed to assess the extent to which social media contributed to engaging Canadians.

The qualitative analysis sampled social media posts from Feb 2019 to Feb 2020, and through scoring, identified relevant policy-related content on LinkedIn, Twitter and Facebook platforms, and  the extent of use of social media for public engagement. 

The analysis had 2 components: 

•  the Department’s use and Branch’s use of social media to promote the work of the Branch and reach out to Canadians
•  the social media activity of 17 CSOs (9 councils and 8 non-governmental organizations)

Evaluation limitations and mitigation measures

Findings - Responsiveness

Branch response to priorities

The Branch positively adjusted its structure in response to the shifts in priorities, programming and the added responsibilities it was mandated to undertake.

The evaluation found that the Branch adapted responsively and with flexibility to emerging programmatic and operational priorities. The Branch’s structure was organized around thematic areas, which aligned well with Canada’s international assistance priorities under the Feminist International Assistance Policy action areas and ministerial commitments. As a result, the Branch allocated resources to quickly respond to new priorities, scaling up or reducing levels of resources and funding envelopes.

This also enabled the Branch to act swiftly, as it did in its response to the Equality Fund, the CSO Partnership Policy, and the full design and set-up of the Technical Assistance Partnership. One ramification to this approach was a delineation of portfolios by themes, which challenged the multi-sectoral approach and the review of multi-sectoral proposals. The change in priorities strained resources in some areas, with less funds available despite their enduring importance (e.g. no increase in programming under Inclusive Growth, Governance and Innovation Partnerships Bureau, unlike other areas of KFM programming). Many other divisions faced resource challenges as the Branch took on additional programming responsibilities. 

Branch programming to engage Canadians in international development met policy needs but lacked a guiding strategy.

In contrast with other areas of Branch programming, the Engaging Canadians Bureau included a set of non-thematic programs aligned to the goal of engaging Canadians in international development. These programs expanded over the evaluation period, with the Bureau combining communications functions, program management functions, service provision for bilateral programs, policy engagement, and operational and knowledge-management processes. While the Engaging Canadians divisions delivered on their respective mandates, they lacked an effective vision and strategy to coordinate the diversity of portfolios.  The Bureau also took on new assignments that did not fit in elsewhere in the Branch, such as youth leadership. As a result, the Bureau lacked coherence and struggled to align its assignments. Communications roles specific to engaging Canadians expanded, leading to some confusion and duplication between Strategic Communications, Development (LCAS) and Engaging Canadians Section (KEGC). However, initiatives between these units resulted in positive cooperation. 

Departmental groups and CSO partners valued the Branch’s contribution and approach.

The Branch was seen as convivial, with its leadership and relatively flat management valued both internally and externally. Nearly 3/4 of interviewed CSO partners spoke about positive relationships with Branch staff, acknowledging staff`s service-oriented approach, sense of partnership and dedicated problem-solving efforts.

CSO response to priorities

New policy and program priorities presented opportunities for most CSOs to address gender equality in a more intentional way and to broaden the scope of existing projects.

Canadian civil society organizations generally found the new policy priorities well adapted to the desired outcomes of their programs. Most interviewed CSOs (86%) reported that their work had been aligned  to some degree with the gender equality agenda prior to the adoption of the Feminist International Assistance Policy, but that the enhanced focus allowed them to improve internal capacity on gender equality (90%), to expand programs to target in a more intentional way for gender equality results in action areas (98%), and to strengthen their justification for this focus or to support stronger capacity of their local partners (89%).  The inclusion of Sexual and Reproductive Health Rights and Maternal, Newborn and Child Health programming was a welcomed improvement that presented opportunities for health-focused partners to be more comprehensive in their programs.

Transformative changes of the gender equality agenda required a long-term perspective.

Interviewed and surveyed CSOs acknowledged that new policy priorities required changes in the approach or work with local partners, and longer time for transformative changes to take place toward gender equality.  Forty percent of CSOs spoke about finding a delicate balance between the gender equality agenda and local partners’ cultures and priorities. Though the policy framework provided leverage for many CSOs, some (18%) said they met cultural and social resistance from local partners and needed more time or flexibility to sustain valuable relationships. Some CSOs stated that the policy framework provided leverage to work through proposals, programming and organizational changes with their local partners as gender equality objectives were stated as a donor's priority. Among survey respondents, 89% supported partners in improving programming along gender equality and women empowerment, and 76% on policies and on monitoring and evaluation. Despite challenges, most CSOs stated that local partners now integrated gender equality programming at varying degrees though it was too early to see sustainable change. Some partners noted that uptake of Canada’s new priorities was easier with partners, such as women's rights organizations, already aligned with the Feminist International Assistance Policy​.

The Branch used effective means to support partners in application processes.

CSO applicants used Global Affairs Canada’s application website tools, and 56% found them useful. The workshop designed by the Business Intelligence unit’s “Demystifying Partnerships with Global Affairs Canada” and the capacity-building component of the Spur Change Program were promising initiatives to improving smaller CSOs’ internal capacity. However, CSOs noted the need for more guidance, especially in the areas of financial or budgetary requirements (77%), performance indicators (59%) and approval timelines (50%).

Design and delivery

Fit of organizational structure

While the Branch’s structure did not include a policy function, KFM staff engaged substantively on policy issues related to working with Canadian civil society.

The evaluation period saw growing demand for policy work on emerging themes related to operations and programs. Policy engagement expectations flowed from the Branch’s mandate to function as the main conduit for the department’s engagement with Canadian CSOs.  The Branch served as operational lead on CSO policy design and implementation and was central to discussions on core/strategic funding mechanisms. Recently, the Branch led on COVID-19 adaptations and the legal framework governing charities. It also led on programmatic issues related to youth, climate financing, and diversity and inclusion, and was central to discussions on localization of aid. Though all branches were engaged on policy issues pertaining to their work areas, the vast range of CSO-related issues added substantial pressure on Branch’s teams. As a result, needs were met on an ad hoc basis, affecting staff’s ability to meet workload and expertise requirements. This placed particularly high demand on the Engaging Canadians Bureau. 

The Branch experienced major challenges related to monitoring, evaluation and learning: systems and practices were inadequate to measure and show results at the program level.

Monitoring, evaluation and learning were not sufficiently integrated into the Branch structure, systems and human resources practices to support data collection, analysis, synthesis, and reporting of evidence from KFM programs. Monitoring and evaluation expectations were not standardized and, in some cases (e.g., Engaging Canadians) program objectives and expected results were not clearly articulated, which led to inconsistent reporting and limitations with respect to aggregating information.

The Branch faced challenges related to defining and reporting on program results beyond immediate outcomes level, which resulted in poor data harvesting at the program level (KFM portfolio). Surveyed partners required support from the Branch on Results Based Management (38%) and, more significantly, on performance indicators (59%). The evaluation noted other monitoring and evaluation specific gaps in systems and practices which made monitoring and evaluation work labour-intensive and prone to errors, including: manual tracking of application data; manual intake and processing of partner reports to document evidence-based results; delays in the approval of project implementation plans; aggregation of results from multi-country projects. 

Because of these challenges, information was not well documented and aggregated making it difficult for the Branch to learn about the overall impact of its programming. Further, it was not used to assess a projects’ performance over multiple phases which could improve program design and selection processes. Given the department's expectations regarding the scaling up of innovation in international assistance, it is essential for learning purposes to have proper systems, resources and practices in managing program information.  

Fit of human resources

Source: Workforce analysis archives (2021-05-07), Global Affairs Canada, 2015-16 to 2019-20.

Note: Short-term included term, casual and student positions.

The Branch's ability to provide stable support to CSOs was affected by vacancies, staff turnover and a high proportion of less experienced staff. Recent measures to address staffing issues are a positive step.

The Branch experienced high overall staff vacancy over the evaluation period (16% average), especially in the programme administration (PM) occupational group (26% average in KFM, compared to 18% average across the department). Key program positions were staffed by less experienced and short-term staff. Short-term employees made up 31% of KFM, compared to 21% in other program branches and 13% across the department. The Branch had more staff under 29 years old (16% average, with a high of 26% in 2019-20) compared to the department’s average of 10%. Though younger, properly trained staff can bring new energy and ideas, the younger staffing configuration impacted on the quality and stability of the response to partners and the achievement of operational process standards. Almost 50% of CSOs stated that they had encountered major challenges with the turnover of program staff and partnership leads, discrepancies in support levels or the need to regularly rebuild relationships—all at the cost of efficiency and corporate memory. This also impacted the quality of internal management across the Branch and between the Branch and financial and contractual branches: 20% of Global Affairs Canada staff recognized staff turnover as problematic for relationships and work processes. Recent increases in number of PMs (2020-21) together with the end of the Human resources mobile positions system could potentially improve service delivery if accompanied by ongoing measures to build staff capacity, especially in RBM and fiduciary and contracting processes.

Workload fluctuations affected the capacity of staff to respond to demands.

Additional work flowing from the Feminist International Assistance Policy and other government commitments did not always come by with more staff or money for implementation (e.g. the design of a Climate financing framework). For some staff, significant work demands related to the partnership lead role added a high level of work above regular duties. Staff from almost all divisions reported high levels of workload and demands. The evaluation found that KFM had no strategy or plan to manage workload balance, including through the peaks and troughs of Call for Proposals selection processes. Staff workloads were disproportionally increased due to the unpredictable volume of applicants to calls and to the absence of surge capacity for assessment and programming processes—particularly exemplified by Small and Medium Organization (SMO) calls—which impacted service quality. The first 2 SMO calls in 2017 and 2019 had 196 and 164 proposals respectively, each funding 19 projects. Most projects (63%) were under the “Inclusive Growth,” “Improved Governance” and “Environment and Climate Action” units.  Proactive measures ahead of Call for Proposals that might generate excess demand could reduce such stress. For instance, one window of the SMO Initiative, the Fund for Innovation and Transformation (FIT), was a good example of outsourcing selection to relieve internal pressures.

The capacity to respond effectively and efficiently to assessments and reviews was limited by the scarcity of in-Branch specialists on gender equality, Results Based Management (RBM), Monitoring and Evaluation (M&E), and targeted expertise.  

The evaluation identified gender equality assessment capacity and knowledge of RBM to enhance operations as key areas of opportunity to improve efficiency in programming processes and the quality of program oversight, especially because of the Branch’s pivotal role in ensuring the implementation of the Feminist International Assistance Policy as it relates to partnerships with Canadian civil society (CSO). Core to this work was engaging with partners on gender-sensitive programming and acting as a knowledge broker on development aligned with the Policy: over the evaluation period, gender specific or integrated (GE-03 and GE-02) projects accounted for 56% of Branch funding to Canadian CSOs, comparable to overall departmental disbursements to Canadian CSO partners (54%). The Branch’s percentage increased to 75% in 2019-20 due to disbursements related to the Equality Fund. Surveyed CSO partners identified several areas where they requested support and guidance from the Branch, among them the gender equality programming toolkit (55%), theories of change and other RBM issues (38%) and the use of a feminist approach (27%). The pressures on few over-solicited specialists affected timeliness of response to proposals and reports and the capacity to meet internal guidance needs. The workload-to-specialists ratio at peak times could not meet the demand. The Branch thus had to call on specialists in other branches, specifically within the Global Issues and Development Branch, who had limited availability. Branch staff shared significant examples where lack of available specialized expertise for guidance, gender and environmental assessments, and approval of project-level RBM frameworks contributed to delays and bottlenecks. At operational level, the lack of in-Branch capacity to design and set up Request-for-Proposals limited its agility in response, as experienced with the Technical Assistance Partnership.

The Branch did not maximize its use of staff knowledge and expertise.

Project Team Leads, who were expected to build Global Affairs Canada’s knowledge and coordination for partner organizations, gained valuable knowledge and perspectives on programs and partners. Project Team Leads often initiated the dissemination of partner experiences and innovative practices applied in the CSO projects they oversaw within immediate teams but had limited availability to share partners experiences to enhance learning across the Branch or department.  While they at times engaged in more strategic context discussion, this was generally ad hoc. Their knowledge of partner performance and operating context was not sought to support proposal assessment as the calls assessment process was kept separate from ongoing relationships. Internal interviews highlighted that limited capacity to scale project and partner-based knowledge sharing activities had a limiting impact on professional development and motivation of Project Team Leads and the expansion of the Branch's learning agenda. 

Agility of selection mechanisms

* KFM’s average was based on a sample of 79 successful partners from 6 out of 13 calls for proposals, excluding the SMO call, round 1. The SMO call, round 1 averages were based on a sample of 21% of successful partners.

Source: Master List of Calls (2021-05-07), Calls Secretariat, Global Affairs Canada, 2015-16 to 2019-20.

The Branch used flexible funding selection mechanisms, which enabled targeted response to policy priorities and use of partner expertise, but lacked transparency in selection decisions.

CSOs surveyed noted that the diverse selection mechanisms used were a good fit to respond to policy priorities (77%) and encouraged the use of CSO expertise (73%). Branch interviewees shared this sentiment. Most Calls for Proposals were launched thematically in response to new government priorities, and CSOs appreciated them as open and fair. Calls for Proposals remained the dominant mechanism, increasing from 30% of projects in 2015-16 to 64% in 2019-20 (along the decline of pre-Authorized programming process), and accounting for 59% ($824M) of total period disbursements.

Most Calls for Proposals received a high number of proposals, resulting in some highly-rated proposals being declined due to available budgets. In a sample of 246 Calls for Proposals proposals (36% of all proposals), 73 unsuccessful proposals (48%) had a rating of 4/5 (77%) or 5/5 (23%), compared to 53% of successful proposals with such ratings. Success rates by bureau (excluding Engaging Canadians) ranged between 10% and 19%. While top proposals were chosen, valuable proposals were not selected, placing significant administrative burden for unsuccessful CSOs and on KFM. Another concern was the transparency of selection decisions, with 33% of those surveyed citing a lack of public information on scoring and 38% citing issues related to the justifications used when proposals were rejected. CSOs rated higher the Unsolicited and the Department-initiated mechanisms on their use of CSO expertise, flexibility and suitability for innovation. Finally, several internal and CSO interviewees noted that the transactional nature of relationships with CSOs set up through this mechanism do not favour long-term impactful programming as would long-term partnership options. Through an initiative led by the Grants and Contributions Branch, the Branch has discussed strategic partnerships models with select CSOs for a few years in which many staff and CSOs expressed support.

Procedures tied to selection and contribution agreement finalization were resource-heavy and lengthy, impacting on partner operations and project implementation.

The Task Force for Improving Effectiveness set up after the 2016 International Assistance Review operated alongside other working groups to improve ways of working between the department and civil society organizations. Despite work undertaken, the Branch faced important hurdles to deliver programs with adequate agility and responsiveness. This was particularly true for Calls where, as a sample showed, an average of 417 days elapsed between the close and sign-off on agreements. Many CSOs cited 18-month to over 2-year timelines between proposal submission and final approval. While some CSOs noted the improvement of a 2-step application, selection processes still rated as non-efficient (39%) or moderately efficient (41%). In interviews, 86% of CSOs attested to long delays, excessive demands and bottlenecks, especially in relation to fiduciary and contractual procedures with 3/4 of CSOs stating negative impacts (administrative, financial, human, programmatic) on operations, local partners and targeted communities.

Effectiveness

Engaging Canadians

Source: CFO-Stats (2021-05-27), Global Affairs Canada, 2015-16 to 2019-20.

Note: See Annex 3 for a summary of Branch engagement efforts with Canadians.

The Branch’s strategic commitment to Engaging Canadians led to targeted CSO programming that involved diverse groups of Canadians in international assistance efforts.

Branch efforts to engage Canadians in international assistance included targeted public engagement programming by the provincial and regional councils for international cooperation, programming implemented by volunteer and internship agencies, and targeted sets of activities in parallel to development projects by other CSOs.

Most CSOs surveyed received either stand-alone funding for public engagement projects (61% of respondents) and/or spent up to 10% of development project funding on public engagement (41% of respondents). Over 80% of surveyed CSOs working in public engagement considered themselves effective at reaching a broad diversity of Canadians. Working with provincial and regional councils in many cases, CSOs provided extensive public outreach coverage across the country and supported North-South linkages between international assistance and locally relevant themes, and between Southern and Canadian partners. CSO partners effectively used social media channels to reach a broad public audience: rates of social media engagement in a sampled period revealed ratings well above non-profit industry benchmarks (between 1.15% to 5.64% compared to the sector’s averages of 0.13% on Facebook and 0.26% on Twitter). 

The Branch could not assess the effectiveness of its public engagement efforts due to a lack of articulated expected results, standardized measures and time needed to bring about such results. 

A goal of Canada’s policy on Civil Society Partnerships is to engage Canadians as global citizens in international assistance. Despite an implementation plan outlining key action areas and activities, the policy has no framework identifying expected public engagement results. Half of partners surveyed were not familiar with the Branch’s strategic objectives, a situation confirmed through interviews. Measuring public engagement can also differ depending on whether it is done from a communications perspective (as it was before the CIDA-DFAIT merger), or a development perspective (the OECD’s “development awareness” code). CSOs therefore aligned their work along broad project-level goals. 

Hence, a notable challenge for both partners and the Branch was the measurement of public engagement results beyond immediate activities and outputs. This stemmed from a gap in having defined and consistent objectives and indicators for what were largely intangible public engagement processes. Of CSOs surveyed with monitoring and evaluation (M&E) plans for public engagement, only 1/3 had support from the Branch on M&E plans. Close to half of the Branch interviewees highlighted the Branch’s—and the Global Citizens Division’s—limited M&E systems and capacity. These challenges made it difficult to obtain evidence from the program on Canadians’ engagement. Many partners welcomed the department’s recent recognition of qualitative stories of engagement, which CSOs viewed as multi-dimensional illustrations of impact in complex programming areas.

Diversity of partnerships

Source: CFO-Stats (2021-05-27), Global Affairs Canada, 2015-16 to 2019-20.

Efforts to diversify partnerships yielded mixed results, though the SMO Call for Proposals successfully generated new CSO partnerships.

The Civil Society Partnerships for International Assistance Policy committed to supporting a broad range of CSO partners. While surveyed CSOs were evenly divided as to whether Branch selection mechanisms encouraged a diversity of partners, CSO and departmental interviewees highlighted that the Small and Medium-sized Organizations for Impact and Innovation Call for Proposals (SMO Call) encouraged partner diversity. At the start of the evaluation period, the Call for Proposals mechanism favored large CSOs. This prompted the Inter-Council Network (ICN) to document the value-added of smaller CSO partners and advocate for a targeted fund. The resulting SMO calls attracted new and smaller partners with niche programming and geographic reach, as well as underserved groups. In the first SMO call, 42% of successful applicants were new CSO partners. In a survey of CSOs conducted by ICN and Cooperation Canada, 48.7% of respondents felt the preliminary call needed more work compared with other application processes. New partners often needed more support to navigate corporate procedures, leading to increased project team lead workloads and more time to finalize agreements.

Over the period, CSO funding increased though the number of partners declined. The Branch disbursed by sub-class to a diverse set of CSOs, and respective proportions remained consistent over the period. An analysis of factors that correlate with success in selection mechanisms found that Call for Proposals, as intended, selected highly rated proposals (4 out of 5 on criteria). However, as the number of quality proposals exceeded funding envelopes, factors that strongly correlated with selection success were prior experience working with the department and the size of organizations. This suggests a reliance on large and experienced partners.

Partner coalitions and outsourced mechanisms enhanced the diversity of CSOs accessing Branch funding. Some targeted calls encouraged the funding of partners having complementary expertise. For example, CanWaCH regroups 100 researchers and health practitioners on women and children’s health. The FIT window of the SMO calls selected 31 proposals in 3 rounds, which included 13 partners—among them 5 private sector partners—who had not previously received department funding. CSOs’ engagement with KFM beyond funding agreements, also increased substantively in areas of policy shaping and implementation.

The Branch provided little support for innovative, multi-stakeholder partnerships.

The number of organizations in formal consortia arrangements under Call for Proposals declined over the period. Despite the promotion in policies of multi-stakeholder partnerships, CSOs stated that selection mechanisms and the Branch, offered limited support for the establishment and management of such partnerships. Funding caps applied to contribution agreements did not build incentives for coalitions, and timelines for proposals submission were short for coordination of proposal development between multiple organizations.

Promotion of innovation

The Development Innovation Unit achieved early success in promoting development innovation internally and externally.

Since its establishment in 2015, the Development Innovation Unit made important steps toward defining and promoting innovation. It built coherence and collaboration through its leadership of the multi-stakeholder community of practice on development innovation which brought together internal and external stakeholders. It developed learning events and products (Newsletter, Wiki pages, Development Innovation Toolkit) and participated in international forums, including piloting a policy marker for innovation in development projects in the OECD-DAC. Almost one-half of Branch interviewees highlighted the unit`s achievements in promoting development innovation across the department. 

At the same time, internal and external stakeholders noted challenges in advancing the innovation mandate, such as the Department’s intentionally broad definition of innovation and lack of standard measures to assess what constituted innovation. ​ Although impact and results measurement are a key activity in the Development Innovation unit’s theory of change, the unit faced challenges on these issues. Inconsistent practices for monitoring and evaluating innovation beyond the project level made it difficult to identify the impact of innovative programming. While 7 of 13 Calls for Proposals included innovation criteria in proposal templates, only 5 of them used a corresponding assessment grid rating for innovation.

CSO partners acknowledged the Branch’s efforts to encourage development innovation in CSO programming, but identified significant barriers to implementation.

The Branch has encouraged innovation in its guidelines, guidance and criteria more systematically since 2017 (see Annex 4). Most (77%) of the 44 CSOs surveyed indicated that they had implemented some innovation in international development since 2015. Of those, 85% were supported by the Partnership Branch to implement those innovations. Surveyed CSOs most commonly reported innovation in approaches (86%), partnerships (59%) and technologies (55%) in their programming. Some 71% of concerned CSOs believed that the innovation tested produced greater development impact. 

On the other hand, in follow-up interviews, 82% of CSO partners said that the Branch`s clear intent to support innovation faced barriers at the implementation level. One-half of CSOs identified administrative barriers (e.g. constraints of contractual terms and conditions, extensive project administration demands, delays for approvals that affected implementation). Furthermore, one third of CSOs said that selection mechanisms, particularly Calls for Proposals, hampered collaboration among CSOs, and that the Results Based Management processes set in predictability of outcomes were not favourable to innovation. A high number of departmental staff and CSOs said risk aversion in the department was a challenge to innovation. 

Knowledge sharing

Branch engagement in multi-stakeholder groups and networks, and efforts of individual staff, contributed to the exchange of lessons learned.

The Branch’s mandate includes leveraging expertise, knowledge, networks and resources of Canadian CSOs and facilitating knowledge exchange to enhance innovation. However, the evaluation revealed that KFM lacked a systematic and consistent approach to leveraging and sharing knowledge within the Branch and within the Department. A sizeable number of Branch interviewees (41%) said there was limited to no knowledge sharing. Examples of how CSOs’ expertise was leveraged mostly focused on individual projects and did not systematically feed into the Branch policy and programming development cycle. Much of the knowledge sharing depended on personal initiative by Branch staff or CSO partners and circulated in peer groups. Some knowledge sharing was concentrated in closed collaboration of some Bureaus' divisions with their niche partners (e.g., the Global Citizens Division and the councils for international cooperation) and others centred on well-resourced partnerships with a focused agenda to help the Branch steer exchanges of experience (e.g., the Canadian Partnerships for Health and Nutrition Division's engagement in the CanWaCH network). 

Interviewees highlighted positive examples of leveraging and sharing CSO knowledge, including the Community of Practice on Development Innovation and the ongoing work of Canadian Partnerships for Health and Social Development Bureau with the CanWaCH health coalition. Branch engagement with provincial and regional councils for international cooperation and Cooperation Canada yielded benefits for the Branch, partners and the sector according to a majority of interviewees. Internal (21%) and external (50%) stakeholders noted the value-added of Branch participation in networks to build on strengths, enhance knowledge sharing and promote Global Affairs Canada’s recognition of the value of CSOs' contribution to international assistance.  CSOs praised Global Affairs Canada`s and KFM’s engagement and consultation through the International Assistance Review.

The Branch achieved good visibility by pursuing online communication channels.

The Branch increasingly used social media and its wiki page to share information and knowledge, feeding GAC’s corporate social media accounts through informal processes. A personal Twitter account for the Assistant Deputy Minister was set up for KFM work, allowing for nimbler and more responsive communication. KFM’s Assistance Deputy Minister account had fewer users than the corporate accounts but achieved 3.3% engagement compared to 1.5% for the corporate, showing that a dedicated following connected directly to the Branch.

KFM used Facebook and Twitter to share its content and re-share content posted by CSOs (between September 2019 and February 2020, for example, 57% of KFM content posted on corporate Facebook accounts was original and 43% was from shared publications). At the corporate level, having a joint Communications Branch and KFM social media strategy could help build coherence in communications with partners. Internally, the resources and toolkits on the Branch’s wiki page were useful ways to share knowledge internally.

Coherence

CSO relationships

Source: CFO-Stats (2021-05-27), Global Affairs Canada, 2015-16 to 2019-20.

The Partnership lead role was useful in coordinating CSO relationships but lacked consistency in practices, which impacted its overall value.

As of March 2020, the Branch had 54 Partnership Leads managing between 1 and 9 CSO relationships across the department. The Partnership Lead role, assigned to select Project team leads, was created to harmonize and facilitate the department's relationships with CSOs. The vast majority of top CSO partners received funding from multiple department branches. While CSO interviews revealed some confusion between the roles of Partnership leads and Project team leaders, they also pointed out that the Partnership lead role was generally appreciated as a coordination mechanism (43% of interviewed partners). Many CSO and Global Affairs Canada interviewees stated that the person occupying the Partnership lead role and its network of contacts were key to successful relationships. However, both groups of stakeholders observed that practices and efforts were inconsistent from one lead to another.

Some GAC interviewees valued the role as a way to obtain information about a given partner but did not see that Partnership leads brought added value as a coordination mechanism. It was common for CSOs to reach out directly across the department in support of their projects without engaging leads. Despite noted challenges, internal and external stakeholders acknowledged that having 1 liaison person serving as a repository of information and a source of engagement history with Global Affairs Canada was an advantage in the context of fluctuating programs and high departmental turnover. ​

The Civil Society Policy Advisory Group was a model of innovative partnership, though coordinating across the department came with a complex accountability setting.

The Civil Society Policy Advisory Group (CPAG), directly reporting to the Assistant Deputy Minister, was set up in 2019 as a means to cooperate for the implementation of the new Policy for Civil Society Partnership. The creation of CPAG was seen as a novel way of approaching joint work between civil society and government, with its operational model of co-creation rather than negotiation on 2 distinct agendas, that is, the agendas of the department and of CSOs. The group comprised 12 members (8 from CSOs and 4 from across the department) who met twice a year, except in 2020, and agreed on 9 initial priorities in support of policy objectives.

While it was deemed a promising and effective way to partner on common interests and advance the CSO policy agenda, operations were interrupted due to priorities resulting from the COVID-19 pandemic, transitions in membership and a shift in priorities. At least 5 Global Affairs Canada branches shared responsibility for the implementation of follow-up on decisions and there was a gap in mechanisms to ensure timely deliverables and accountability across the Department. The Branch was under-resourced in supporting CPAG, which was made more acute by the slow-down brought about by COVID-19.

International programming

Source: CFO-Stats (2021-05-27), Global Affairs Canada, 2015-16 to 2019-20.

Stronger communications that enhanced collaboration as exemplified between the Branch and missions abroad could improve department-wide cooperation in Canada. ​

At departmental level in Canada, more formal communications of the Branch’s strategic objectives could enhance cooperation, engagement on relevant issues of interest, services and communications. Stronger collaboration between the Branch and missions exemplified the benefits of improved communications. Branch programming disbursed funding in 120 countries over the evaluation period. As of October 2019, this included 10 countries with no Canadian representation or funding from other branches. During the evaluation period, the Branch supported many partners working in its top disbursement countries, with the average reaching a high of 43 partners in Tanzania. The Branch actively worked to strengthen relationships with missions, and in many countries with sizeable Branch programs, this led to mutually beneficial results, improving coherence and complementarity in international assistance programming. However, not all missions in countries of relevance for the Branch displayed the same level of collaboration. Efforts to coordinate relationships between the Branch and missions, including appointing a KFM liaison in missions, were pursued by both sides over the evaluation period. The evaluation acknowledged efforts pursued through the whole-of-department approach to in-country programming, mandate letters that outlined specific liaison responsibilities for the Branch’s in-country programming and countries of accreditation, and enhanced collaboration spearheaded by KFM’s Business Intelligence Unit. Interviewed Branch and mission representatives reported improvements in communication effectiveness and efficiencies, and in linkages in country with CSO partners. Improved relationships also contributed to better leveraging of Canada`s roles in international assistance where thematic focus of KFM projects was distinct from bilateral programming, and improved coherence where program focus was aligned to bilateral programming. This was particularly true in decentralized missions, and surveyed CSO partners commented positively on the close relationships between the Branch and in-country missions. 

According to interviewed mission representatives, the Branch programming and partners' projects added value to Canada’s international assistance in country. The relationship was mutually-beneficial to partners and missions. For example, input from missions in proposal pre-assessment improved the quality of selection, as it brought in local and contextual knowledge that had vital impact on decisions. ​ Many CSOs were involved in coordination efforts with missions at a country level, sometimes formalized through coalitions. Mission staff and CSOs highly valued these efforts, supported by ongoing relations and continuous engagement on issues between CSOs and Global Affairs Canada. In some instances, relationships were built between CSOs and specific Global Affairs Canada staff at missions working on thematic dossiers, such as trade commissioners, leading to a concerted approach covering different angles on which issues needed to be tackled (on-the-ground work of CSO partners and bilateral engagement for missions). 

Good practices

Engaging CSOs in Government of Canada policy consultations

The International Assistance Review was launched in May 2016 to examine how Canada could best refocus its international assistance efforts on the poorest and most vulnerable people in support of the Sustainable Development Goals. Consultations with civil society actors were held in June and July 2016. The Inter-Council Network submitted a report with overarching recommendations, commending Global Affairs Canada’s efforts (spearheaded by the Calls for Proposals Secretariat, at the time a standalone unit reporting directly to the Engaging Canadians Bureau) to engage with communities across Canada through its consultations. CSOs expressed their appreciation for this engagement during interviews. More importantly, the consultations generated a rich CSO-department agenda to enhance partnerships and build stronger effective international assistance delivery following the consultation process.

CSO capacity-building efforts

The Spur Change program and Fund for Innovation and Transformation (FIT) are two capacity-building windows of the SMO for Impact and Innovation initiative. The Spur Change program is a 5-year initiative funded by the Branch to increase the effectiveness of Canadian SMOs to deliver sustainable results in support of the Feminist International Assistance Policy and the Sustainable Development Goals. The program is managed by the Alberta Council for Global Cooperation on behalf of the Inter-Council Network (ICN). Roundtable consultations were conducted in fall 2019 as a needs assessment practice to inform capacity-building workshops and training. In response to a lack of resources for gender-equality assessments in the context of testing innovations, the FIT Program, managed by the Manitoba Council for International Cooperation, was created to build capacity in this regard. FIT also collaborates with the Spur Change and national ICN programs to improve programming; partnerships also extend to the Innovation Unit at Global Affairs Canada for feedback. 

Increased collaboration with Cooperation Canada

While Cooperation Canada (formerly CCIC) has relationships across the department, the umbrella group mirrors the Branch's mandate regarding stakeholder and CSO engagement. Open dialogue and communication enabled the Branch and Cooperation Canada to jointly address shared issues of interest. For example, Cooperation Canada supported discussions on Canadian Revenue Agency Direction and control, the COVID-19 Dialogue and Solutions Group, the Partnership for Gender Equality advisory group, the External Advisory Council (EXAC) on Public Engagement, and the Task Force for Increasing Effectiveness. The Branch has actively engaged with other thematic working groups on key development issues (food security, climate change).  

The Branch and Cooperation Canada deployed a collaborative approach to developing the CSO policy and implementation plan, both co-chairing the Civil Society Policy Advisory Group. They also collaborated regularly on the convening of civil society organizations. Indeed, they have a long history of jointly holding annual dialogue events and conferences with civil society organizations, allowing the Branch to hear the voices of Cooperation Canada’s members and the issues that concern them the most. In return, Cooperation Canada feeds departmental information back to its membership, ensuring access to a unified message. 

Proactive outreach to share Branch priorities and processes

The Business Intelligence and Process Division, through its initiative called "Demystifying Partnerships with Global Affairs Canada" met with CSOs across the country in late 2019 and early 2020 to provide support and improve transparency regarding partnering with Global Affairs Canada. Branch staff gave a presentation on the Partnership Branch to regional and provincial councils and their members, giving recommendations on project design, and insights on selection processes, contribution agreements and project implementation. Many CSO members praised the events for answering frequently asked questions from both current and potential partners. 

Leveraging Branch programming for bilateral international assistance efforts in the field

Branch programming in select countries provided leveraging opportunities for bilateral international assistance efforts. Through the Branch, a large investment in MNCH programming was made in select countries having no bilateral health focus, such as Kenya and Ghana. The mission in Kenya noted that the investment led to a critical mass of programming with several partners leading to increased engagement with the high commission. For example, a conference organized by Canadian MNCH partners and attended by key health sector actors from the Government of Kenya, international agencies and representatives from the Canadian mission allowed partners to discuss results and lessons learned from MNCH programming. This helped raise Canada’s visibility in the health sector and provide the high commission in Kenya with access to an expanded network, both within and outside of government—benefiting Branch programming and bilateral programming.

Leadership on the COVID-19 response

Branch staff worked alongside CSOs through the COVID-19 Dialogue Group and COVID-19 Solutions Group to address challenges associated with international assistance programming amid the global pandemic. Notable changes were widely cited as improving the efficiency of selection processes, including greater flexibility for implementing partners in terms of cost sharing, direct costs, release of holdback, reporting and other aspects of contracting instruments, and the delegation of accountability and authority to program teams for select approval processes. Although these changes were made after the evaluation period, they were made immediately in response to the pandemic—illustrating the Branch’s solid capacity to adapt. 

Conclusions

The Branch’s organizational structure and funding mechanisms were largely responsive to changes in policy and programming priorities.

The Branch’s organizational structure was responsive to the priorities that emerged from the Feminist International Assistance Policy, the Civil Society Partnerships for International Assistance Policy and high-level ministerial commitments. While the Branch structure did not include a policy function, staff actively worked on policy issues work affecting Canadian CSOs. Partner selection mechanisms allowed the Branch to respond to changes in priorities, although processes related to selection and to the finalization of contribution agreements were lengthy and cumbersome, ultimately affecting project implementation. Branch programming was multi-country and multi-sectoral, unique traits that presented opportunities for the department. However, monitoring and evaluation efforts at the program level were inadequate for measuring and demonstrating results in aggregate. 

The Branch supported Canadian CSOs in responding to new priorities.

The Branch used effective means to support partners in application processes, and Branch and external stakeholders highlighted the “Demystifying Partnerships with Global Affairs Canada” initiative, a proactive outreach effort, as a good practice. The new policy and programming priorities presented opportunities for CSOs to target gender equality in a more intentional way though a number of partners needed additional support from the Branch in this area. The transformative change called for by the shifts in policy and programming priorities, specifically around gender equality programming, required time to take root, but responses from local partners were encouraging.

Branch-funded CSO programming engaged a diversity of Canadians in international assistance.

The Branch increasingly supported CSO programming that engaged diverse groups of Canadians in international assistance issues, as mandated by the Civil Society Partnerships for International Assistance Policy. Targeted elements of diversity included volunteers, youth including and indigenous youth, diaspora communities, and diverse geographic regions. However, the Branch lacked a strategic framework for public engagement around Canada’s international assistance priorities, and the absence of consistent objectives and indicators used across CSO programming limited the extent to which outcomes could be measured and used for program improvement.

Selection processes did not consistently foster diversity and innovation in Branch partnerships, although the SMO Calls for Proposal were deemed a positive example.

Selection processes did not consistently demonstrate that the Branch placed a priority on diversity in partnerships. When it came to diversity, calls for proposals and department-initiated processes were rated as more conducive than unsolicited proposals. A promising mean used by many stakeholders was the calls for proposals targeting small and medium-sized organizations for impact and innovation, which introduced new and smaller partners to Branch programming efforts. Working with these new partners required a more intensive effort on behalf of Branch staff, which strained the Branch’s ability to manage processes with existing resources, procedures and systems.

Branch engagement with multi-stakeholder groups and social media platforms contributed positively to leveraging CSO expertise and knowledge.

Multi-stakeholder consultation groups, communities of practice and networks, such as CanWaCH and Cooperation Canada, demonstrated capacity to leverage and share CSOs expertise and knowledge. However, leveraging these CSOs experiences within the Branch and Global Affairs Canada depended on personal initiatives taken by Branch staff or by CSO partners. To promote opportunities for scale-up and replication of initiatives that yield effective outcomes, CSO knowledge and experience need to be systematically integrated into Branch strategic programming development. This would be supported by improved monitoring, evaluation and learning capacity. 

The Partnership Lead role, the Civil Society Policy Advisory Group, and collaboration with missions facilitated coherence within Global Affairs Canada and with partners.

Internal coherence and the coordination of CSO relationships benefited from the leadership provided by Branch Partnership leads. However, there was confusion regarding their role and the responsibilities of Project team leaders, and the Partnership lead role would benefit from being better disseminated and standardized in practices across the department and with CSO partners.

The Civil Society Policy Advisory Group exemplified Global Affairs Canada and Canadian civil society’s coherent approach to international assistance efforts with the co-creation and implementation of the Civil Society Partnerships for International Assistance Policy. However, internal and external factors—such as the division of roles across the department, and the COVID-19 pandemic—resulted in accountability challenges and a slowdown of efforts. Branch international assistance efforts and CSO relationships benefited from enhanced collaboration with missions, which improved programming coherence and enhanced the visibility of Canadian international assistance.

Recommendations

1. Organizational structure:

The Branch should articulate and share broadly its unique role in, and value of, engagement with CSOs to strengthen public engagement on Canada’s international assistance priorities and develop a clear framework to guide this work. This should include a Theory of Change and performance measures for public engagement with Canadians.

2. Organizational functions:

The Branch should clarify its policy mandate, strengthen policy capacity as it relates to the engagement of CSOs, and assess ways to address policy needs in a systematic manner, rather than on an ad hoc basis.

3. Organizational functions:

The Branch should develop and implement strategic Monitoring, Evaluation and Learning plans to enhance its capacity to draw program-level results and support evidence-based learning, knowledge-sharing and decision-making.

4. Human resources:

To improve effectiveness and efficiency in operations, the Branch should strengthen internal capacity on intersectional gender equality assessment, results-based management, and grants and contributions management. This could combine training, mentoring or coaching and should include an accountability mechanism for tracking capacity enhancement efforts.

5. Selection mechanisms:

The Branch should improve selection mechanisms by:

1. Improving agility by considering managing selection through the use of outsourced mechanisms, where relevant
2. identifying options to solve the need for surge capacity to reduce excessive pressures on internal resources during the Call for Proposals assessment
3. adopting means to improve transparency on criteria leading to selection decisions as well as enhancing partners learning by providing feedback, where possible.

6. Funding agreement and implementation:

KFM and corporate operations should work together to shorten timelines associated with the Authorized Programming Process in order to improve the delivery of international assistance. 

Considerations

Development innovation

Internal and external stakeholders acknowledged Branch efforts to encourage development innovation in CSO programming. The Branch should continue to develop modalities and incentives to encourage the implementation of innovative international assistance programming through program criteria, the selection mechanisms, and assessment processes.

Social media strategy

The social media analysis highlighted an unclear division of roles and responsibilities between the Engaging Canadians Section (KEGC) and the Strategic Communications, Development Division (LCAS). No formal processes existed to determine what content is relevant on which account. Clarifying roles and responsibilities between KEGC and LCAS could contribute to a more effective public engagement strategy.

Strengthening communications and the Partnership lead approach

Internal and external stakeholders recognized the value of having a single person coordinate relationships with CSO partners funded through multiple departmental sources. The Partnership lead role could bring increased value for Global Affairs Canada and CSOs following the development of a training plan covering mandate, standards, and expectations; efforts to ensure greater stability of staff in the role and more communication about the role internally and externally. Overall, clear formal communication of KFM`s strategic objectives to other branches could also help enhance coordination in a range of areas. 

Corporate-wide support for efficiency improvements

Significant improvements in efficiency require the collaboration and effort of branches with corporate, process-oriented responsibilities. Considerations could be given to making some of the changes made in response to the COVID-19 crisis (recommended by the COVID-19 Dialogue Group and Solutions Group) more permanent. This includes giving implementing partners greater flexibility in terms of cost sharing, direct costs, holdback, reporting and other aspects of contracting instruments, and the delegation of accountability and authority to program teams. 

Implementation of the strategic partnerships model with organizations that strongly align with international assistance priorities and that have solid governance, robust accountability systems and low fiduciary risk being championed in the Department promises to be an important contribution to improved efficiency. Forging partnerships using streamlined procedures and simplified selection and reporting could reduce workloads and allow Branch staff to invest more time in areas where gaps are present. 

Annexes

Annex 1: Evolution of GoC commitments and KFM selection mechanisms

Annex 2: Objectives of the Civil Society Partnerships for International Assistance Policy

1. Empower women and girls, promote gender equality, and reach the poorest, most vulnerable and most marginalized as the most effective means to eradicate poverty
2. Facilitate a safe and enabling environment for civil society
3. Protect human life and dignity
4. Foster CSO leadership in innovation
5. Integrate the role of CSOs as independent actors into international assistance programming
6. Establish more predictable, equitable, flexible and transparent funding mechanisms
7. Foster multi-stakeholder approaches to international assistance
8. Engage Canadians as global citizens in international assistance
9. Promote sustainability, transparency, accountability, and results

Annex 3: Branch pillar of Canadian public engagement

The Branch's mandate includes engaging Canadians as global citizens by maximizing opportunities to raise Canadians’ awareness of and engagement in international assistance. The Global Citizens and Engagement Division's programs responded to the Branch’s strategic commitment to engage a diversity of Canadians in international assistance efforts. The efforts outlined below targeted specific elements of diversity including volunteers, youth including indigenous youth, diaspora communities, and diverse geographic regions.

Annex 4: Branch pillar of partnerships for development innovation

To deliver transformative actions for impact, Canada committed to advancing innovation, research, and results, encouraging greater experimentation and scaling-up of new solutions to development challenges, and ensuring women and girls are involved in, and benefit from, the innovation process. The Branch's mandate includes the promotion of development innovation by facilitating knowledge exchange across Global Affairs Canada, with civil society and other partners, to advance innovative ideas and approaches to development. 

The Branch's Development Innovation Unit acted as a centre of expertise on development innovation to drive organizational culture change for poverty reduction. The Unit served as a catalyst, connector and knowledge disseminator at Global Affairs Canada to foster innovation for international development policies, programming and partnerships. The Development Innovation Toolkit launched in 2019 provided Global Affairs Canada employees with a resource that defines development innovation, outlines the Whistler Principles, discusses policy dialogue and collaboration including recommendations for creating effective partnerships for development innovation, and explains how to integrate innovation into the project management cycle. It offered development innovation insights, case studies, and additional resources.

Annex 5: Social Media Analysis

Departmental use of social media to share CSO experience

Both the department and the Branch used social media platforms to share information specific to Canadian CSO experiences.

Approximately 25% of the original content produced and shared by the department’s corporate social media accounts in international assistance was related to Branch-funded partners and projects overall. Figures from all Twitter and Facebook accounts tabulated over a 6-month period show 304 original Tweets and Facebook posts (40%) to KFM, or posts that provided visibility to KFM partner organizations. These posts were displayed 1,954,988 times and generated 36,492 engagements (sharing, clicks, comments, likes, etc.).  The use of social media platforms was significantly more prominent leading up to the International Development Week. Incidentally, traffic on IDW-related webpages increased by 150% (7 400 to 18 000 views) and on social media, the impressions of dedicated hashtags almost tripled (280%). 

Apart from corporate social medial accounts, the Branch (KEGC) managed the Branch ADM Twitter account, which was created as a more flexible way to efficiently share information and provide visibility to KFM programing. The KFM ADM Twitter account was considerably more active, representing almost 2/3 of the total KFM content tweeted from department. Despite having 1,058 followers compared to 81,960 followers on the corporate accounts, the ADM account engagement rate was 3.3% compared to 1.5% for the corporate accounts, demonstrating that users were more actively engaged.

Gender equality was a central theme in Branch content on the corporate accounts, more so on Facebook (68%) than Twitter (51%). Content was aligned with the Feminist International Assistance Policy, with health, education and public engagement being the most common themes.  

CSO partners’ use of social media related to funded work

The social media posts of the 17 CSOs sampled reflected themes related to their areas of work, many of which were aligned or linked to the Feminist International Assistance Policy’s action areas.  In terms of reaching their target audience, the analysis found that all sampled CSO partners were well above industry benchmarks for non-profit organizations’ social media engagement rates of 0.13% on Facebook and 0.026% on Twitter, with averages of 1.15% to 5.64%. The engagement on social media correlated strongly with planned events or observance days (Women Delivers, IDW, etc). This demonstrated a clear alignment between subject matter and audience interest. 

The study also found that organizations with broader audiences had higher impressions (number of times content is displayed), but organizations with specific audiences obtained a higher engagement rate.  The “social score” of partner varied and the council that rated exceptionally high on social score, thematic alignment and engagement rate had a dedicated social media staff at 50% FTE.  The organizations with a defined social media strategy had better uptake and outreach outcomes, while organizations with smaller funding had fewer social media outcomes and less reach. The organizations with the highest number of interactions on social media were the ones with both French and English content. Those organizations and Francophone organizations also tended to have stronger gender equality integration, as did organizations focused on themes of the Feminist International Assistance Policy action area themes. Volunteer sending organizations benefited from high quality content from volunteers or interns, leveraging program and communications efficiently. 

During the evaluation period, social media was mostly used in complement to other outreach and communications means, although interviewed CSO representatives pointed to a shift in using social media as a main engagement driver (rather than an add-on) in the COVID-19 context.