Synthesis of Evaluations of Grants and Contributions Programming funded by the International Assistance Envelope, 2011-2016

February 2017

Table of Contents


The Development Evaluation Division would like to acknowledge the contributions of the many individuals whose knowledge, expertise and efforts were central to completing this review. The Evaluation Division of the Office of the Inspector General jointly undertook this review, and staff from Global Affairs Canada’s Development Policy, Development Research, Program Coherence and Effectiveness provided valuable contributions to the planning and review processes. Specialists in the areas of results-based management, cross-cutting themes and financial resource planning offered technical advice and important contextual information to situate the findings of the synthesis.

A special thank-you is extended to the members of Global Affairs Canada’s Programs Committee and Directors’ General Program Committee for sharing their views on the preliminary findings and identifying opportunities for further analysis.

We would like to acknowledge the work of the team of consultants from Universalia who carried out the assignment: Mariane Arsenault, Anne-Marie Dawson and Katrina Rojas. The project was supervised by Andres Velez-Guerra and managed by Eugenia Didenko who prepared the final report for distribution.

David Heath
Head of Development Evaluation

Acronyms and Abbreviations

African Development Bank
Authorized Programming Process
Canada Fund for Local Initiative
Canadian International Development Agency
Conference of the Parties
Civil Society Organization
Development Assistance Committee (of OECD)
Department of Foreign Affairs and International Trade
Department of Foreign Affairs, Trade and Development
Global Affairs Canada
Grants and Contributions
Inter-American Development Bank
International Assistance Envelope
Millennium Development Goals
Non-Governmental Organization
Organisation for Economic Co-operation and Development
Development Evaluation Division
Results-Based Management
United Nations Development Programme
United Nations Population Fund
United Nations Children’s Fund
World Food Programme
Office of the Inspector General


The following terms are used in this report.


Operational Definition

Aid Modality

A way of delivering official development assistance, for example through bilateral, multilateral or Canadian partnership channels.

Civil Society Organization (CSO)

Non-governmental, non-profit and voluntary driven organizations, as well as social movements, through which people organize themselves to pursue shared interests or values in public life.


A conditional transfer payment from the Government of Canada to a recipient in which there are specific terms and conditions that must be met by the recipient. Contributions, unlike grants, are subject to performance conditions that are specified in a contribution agreement.

Country Program Evaluation

An assessment of the relevance, effectiveness, efficiency, sustainability and cross-cutting themes of development programming in a specific country over a given period.  Country program evaluations contain findings, conclusions and formulate recommendations to improve development programming.

Delivery Mechanism

A contractual arrangement or mechanism used to enter into agreements with entities delivering aid programs.

Development Effectiveness Review

A review that focuses on assessing the development effectiveness of multilateral organizations using common criteria of relevance, effectiveness, efficiency, sustainability and cross-cutting themes, and draws on evidence provided in evaluation reports from multilateral organizations being assessed.

Fragile state

A state that faces particularly severe development challenges and is characterized by complex national and regional contexts, weak institutional capacity, poor governance, political instability, ongoing violence or a legacy of past conflict. Programming in fragile states involves humanitarian assistance and emergency services, and seeks to enhance long-term development by improving the effectiveness of public institutions, fostering stability and security, and supporting the delivery of basic services. Fragile states where Canada provided development assistance include Afghanistan, Haiti and South Sudan.

Good Practice

A successful approach highlighted in evaluation reports that can provide directions to future development policies and programming.


An unconditional transfer payment where the eligibility criteria applied before payment assure that the payment objectives will be met. An individual or organization that meets grant eligibility criteria can usually receive the payment without having to meet further conditions.

Inter-Program Cooperation

Inter-Program cooperation is understood as cooperation between the development, humanitarian aid, trade and diplomacy programs of Global Affairs Canada, between different development programs (e.g., multi-bi; bilateral- partnership) or between different foreign affairs programming areas.

Lesson Learned

A lesson or good practice from previous programs or projects that could be integrated into future programming.

Non-Traditional Partner

A partner that differs from the Government of Canada’s traditional partners, such as multilateral institutions, Civil Society Organizations, or private sector as executing agency.

Programming Mechanism

A mechanism identified by the Authorized Programming Process, which refers to programming types as selection mechanisms. There are two groups of selection mechanisms: open track with request for proposals and call for proposals; and targeted track which includes Department-initiated, unsolicited proposals, and institutional support.


A review for analyzing and synthesising qualitative information from various sources to address specific research questions.

Executive Summary

This report presents the results of a structured review of departmental evaluation reports of Grants and Contributions (Gs&Cs) programming funded by the International Assistance Envelope. This was a first exercise undertaken by Global Affairs Canada (GAC) aimed at providing a whole-of-Department perspective on its Gs&Cs programming by including evaluations completed by PCE (Development Evaluation, 29 reports) and ZIE (Office of the Inspector General, 11 evaluation reports).

The objective of the review was to identify lessons and recurring challenges to inform future departmental programming and foster horizontal learning. Findings and conclusions presented in this report need to be situated and interpreted within the review’s historical lens. Most of the Gs&Cs programming examined was implemented prior to amalgamation of the former Canadian International Development Agency (CIDA) and Department of Foreign Affairs and International Trade (DFAIT).  The programming responded to the international priorities of the past rather than present priorities. Also, the review crossed all programming areas funded by the International Assistance Envelope. As a result, nuances specific to each programming area were not captured due to a small number of evaluation reports addressing those.


The review revealed that GAC has generated a strong body of evaluation evidence to conclude on the relevance and effectiveness of Gs&Cs programming.  However, the evidence to conclude on the efficiency and sustainability of its programming and the advancement of the cross-cutting themes (gender equality, environmental sustainability and governance) was less robust. The review also identified various factors contributing to, or limiting, the achievement of the desired outcomes of departmental programming.

Canada’s comparative advantage was reported to be strong in the areas of policy dialogue and promotion of gender equality and results-based management. Additional evidence was available to highlight successes in establishing partnership with multiple partners, engaging civil society organizations and applying a mix of programming mechanisms.

Relevance: The review found the Department’s Gs&Cs programming was highly relevant and strongly aligned with Canada’s stated international priorities. The programming largely met the needs of its targeted beneficiaries in developing countries, as well as the needs and priorities of Canadians. The programs evaluated were considered relevant to the Department’s mandate to support poverty reduction efforts, but did not reflect the recent emphasis on the poorest and most vulnerable. The review suggested that the Department did not have a common definition of these target groups and that different programs defined and reached them in different ways. Of note, the degree of relevance was shown to relate to a number of factors, such as conducting appropriate needs assessment, engaging partners, building ownership and leveraging their respective strengths, especially at the design and implementation stages.

Effectiveness: The review concluded that Gs&Cs programming was generally effective, although gaps existed in the Department’s ability to demonstrate the achievement of longer-term outcomes. The body of evidence largely came from assessing program success at the output and immediate outcome levels. Programming effectiveness was supported by utilizing strong development knowledge, expertise and skills within the Department and by partners. GAC’s programming showed strong capacity for establishing partnerships, applying a mix of programming mechanism and aligning programming to the needs of beneficiaries.

While Canada was recognized as a strong advocate for the management and measurement of results, the Department showed weaknesses in applying the results-based management approach to its own programming. Limitations were noted in the Department’s ability to generate and aggregate program and project-level data, to develop quality performance measurement tools (logic models, Performance Measurement Frameworks), and to share its performance measurement expectations with implementing agencies. Difficulties in applying results-based management in development cooperation by donors, particularly aggregating data and attributing results to aid funded programming, have been well documented in the literature.

Efficiency: The review pointed to limited evaluation evidence available to conclude on the efficiency of Gs&Cs programming. Most of the evidence was qualitative in nature and recommendations on ways to improve efficiency were not frequently provided in evaluation reports. In part, this was due to limited data available at the program level, lack of comparable data for comparative purposes and also lack of common tools or guidance available to develop and monitor efficiency indicators at the program and project levels.

Sustainability: The review found mixed results overall. In addition the evaluation reports had notable constraints in their scope, which affected the ability to draw firm conclusions in this area (e.g., the timeframe being reviewed that did not allow for the materialization of longer-term results and longitudinal performance data was limited).

Cross-Cutting Themes: The review determined that GAC’s programming was partially effective in integrating cross-cutting themes. Challenges in advancing these areas included lack of understanding of the Department’s expectations for cross-cutting themes by implementing agencies, difficulties integrating the themes operationally and from the performance measurement perspective (e.g., logic models, appropriate performance indicators) and a diminished focus and resources within the Department. Some evidence suggested that combining mainstreaming with targeted interventions in cross-cutting themes might be more effective. This was notable in the area of governance, with some examples in gender equality.

Considerations for Future Programming

The review identified several opportunities for GAC with the aim to advance its Gs&Cs programming. These considerations included:

1. Background

This report presents the results of an analysis of corporate evaluation reports that examined grants and contributions (Gs&Cs) programming funded by Global Affairs Canada (GAC) and its predecessor departments through the International Assistance Envelope (IAE).

The main objectives of this synthesis were to:

  1. Provide an aggregated evidence base to inform the Government of Canada’s International Assistance Review;
  2. Identify lessons that can help foster innovation, effectiveness and coherence; and
  3. Contribute to horizontal learning across the Department.

The report is structured as follows:

2. Methodology

Forty evaluation reports were considered for this review. Of the 40 evaluation reports, 29 were managed by GAC’s Development Evaluation Division (PCE) and 11 were from the Evaluation Division of the Office of the Inspector General (ZIE).

The evaluations were completed between 2011 and 2016 and covered past programming efforts, including some undertaken as early as 2002. The retrospective nature of the review presents its major limitation. Another limitation was a small number of evaluation reports addressing some areas of the programming continuum funded by the International Assistance Envelope (e.g., development, humanitarian aid, security and stabilization). As a result, specifics of such programming may not have been well captured by the review’s criteria, such as, for example, nuances of assessing sustainability in humanitarian action or integration of cross-cutting themes in security programming.

The evaluations provided coverage at the country, regional and thematic level and also covered the work of multilateral organizations:

In addition to the evaluation reports, the review team examined relevant documents on the evolution of the Department, lessons learned and previous studies conducted by the Department. The review team also carried out interviews with departmental stakeholders during the planning and data analysis phases.

The methodology was based on a structured review of the findings of evaluation reports. To guide the review, a framework was developed based on: the questions outlined in the Terms of Reference; the OECD-DAC criteria Footnote 1 of relevance, effectiveness, efficiency, sustainability; and questions on GAC’s cross-cutting themes (gender equality, environmental sustainability, governance). The framework was fine-tuned to reflect the perspectives gathered during inception phase interviews. To facilitate the review of evaluation reports and coding of information, the evaluation questions were deconstructed into criteria and sub-criteria and put into a review instrument, completed for each report.

Each review element was assessed under three dimensions:

The review team employed several data analysis methods to inform the findings and conclusions of the review. These included qualitative descriptive analysis, quantitative analysis, comparative analysis, and cluster analysis based on type of evaluation (e.g., country program evaluation, thematic, foreign affairs or development effectiveness review).

3. Findings

3.1 Overview

This section presents the results of the review as they relate to the criteria on relevance, effectiveness, efficiency, sustainability and cross-cutting themes. Each section includes:

3.2 Relevance of Gs&Cs Programming

3.2.1 Coverage of Relevance Criteria

As shown in the chart below, the evaluation reports reviewed consistently addressed relevance Footnote 3 and many of the topics of interest for future programming. Security and stability objectives as well as programming supporting fragile states were covered in less than half of the reports; coverage on these was found mainly in country program evaluations and evaluations from foreign affairs. In addition, while there appeared to be an implicit focus on reaching the poor in many evaluations, the question of whether Gs&Cs programming reached the “poorest and most vulnerable” was not always addressed explicitly in evaluation reports. Most evaluation reports did not discuss whether the programming strategy was appropriate given the country’s development status or income levels, i.e. lower income or middle income country.

Figure 3.1 Number of Evaluations that Addressed Relevance Criteria

Number of Evaluations that Addressed Relevance Criteria

Figure 3.1 - Text version
Rating1. Overall programming relevance2. Focus on stability objectives3. Focus on security objectives 4. Focus on the poorest and most vulnerable5. Focus on fragile states
Not addressed126221626

3.2.2 Key Findings

This section presents the assessment of how well Gs&Cs programming performed on relevance criteria and Figure 3.2 on the next page shows the ratings.

Overall Relevance

The overall relevance of Gs&Cs programming was assessed as strong in 77% of evaluation reports, with the rest (23%) reporting acceptable ratings. All reports described a clear rationale for the programming or initiative. Gs&Cs programming was relevant over the period under review, largely due to meeting the needs of beneficiaries and aligning programming to national plans and priorities, including the Millennium Development Goals (MDGs). Alignment with Canada’s priorities was also strong. Footnote 4

Figure 3.2 Distribution of Ratings by Relevance Criteria

Distribution of Ratings by Relevance Criteria

Figure 3.2 - Text version
RatingGs&Cs programming judged to be relevant overall (n=39)Gs&Cs programming reaches poorest and most vulnerable (n=24)Gs&Cs programming supports fragile states (n=14)Gs&Cs programming addresses security objectives (n=18)Gs&Cs programming addresses stability objectives (n=14)
Not applicable4%11%29%

Seven of the eight development effectiveness reviews examined provided positive assessments of the relevance of programming of the multilateral organizations assessed. These reviews included specific criteria that assessed relevance to target groups, which included beneficiaries, and thus the reports provided some analysis of this dimension of relevance. Many country program evaluations reviewed did not identify the exact needs of targeted beneficiaries, but overall, evaluators deemed the development interventions to be well aligned with targeted country priorities. Fifteen of the 18 country program evaluations described the overall relevance of country programs as strong.

The definition of relevance used by the Development Evaluation Division (PCE) differed from the definition used to assess the relevance of foreign affairs programming. In the latter, the Department included an emphasis on how programming was responsive to the needs of Canadians.. In the thematic and foreign affairs evaluation reports, 10 out of 14 focused on the needs and priorities of Canadians.

GAC’s programming targeted countries that were low income, middle income, as well as fragile states. However, programming evaluated in most country program, foreign affairs and thematic evaluations reviewed were not designed based on a country’s income and development status, with the exception of interventions in fragile states.

Relevance to reaching the poorest and most vulnerable

The evidence of Gs&Cs programming reaching the poorest and most vulnerable was mostly found in country program evaluations.  Evaluation reports reviewed did not provide substantial insights on whether this happened consistently or whether this goal was being addressed appropriately; 16 of the evaluations reviewed made no mention of reaching beneficiaries considered to be vulnerable. Country program evaluations mentioned reaching the most vulnerable beneficiaries, but more often described reaching the poor, which was in line with Canada’s emphasis on poverty reduction. There was no consistent definition of how the poorest and most vulnerable were defined across the different types of departmental programming and, as a result, across the evaluation reports reviewed. Some reports described reaching “the poor and the marginalized”, while others described reaching “people underserved by the central government,” “underprivileged populations” or the “ultra-poor”. The development effectiveness reviews assessed whether programming met target group needs, not whether they reached the poorest and most vulnerable. Some of the reviews, such as for UNFPA and WFP, implied that the organization focused on the poorest and most vulnerable due to the nature of the mandate, particularly with regard to humanitarian, conflict and post-conflict settings.

However, for the 24 evaluation reports where programming addressed the poorest and most vulnerable, 63% had a strong rating for relevance and another 29% were acceptable.

Relevance to supporting fragile states

The relevance of programming to supporting fragile states was not often described in evaluation reports reviewed and was expected given that only a handful of countries were considered fragile states under the GAC definition. Relevance to fragile contexts was not explicitly discussed in foreign affairs or thematic evaluations as most of those reports did not carry out an assessment of relevance from the point of view of beneficiaries in the development context.

Of the 14 evaluation reports where programming supported fragile states, 57% had a strong rating for relevance and another 29% were acceptable.

Relevance to security and stability objectives

The review team also examined whether security and stability objectives Footnote 5 were addressed and found that 18 reports put some emphasis on security and 14 reports contained elements related to stability. Footnote 6 In most cases, these were not explicitly stated as objectives of the intervention, but could be implied from the nature of the program reviewed. Interestingly, both country program evaluation reports and foreign affairs evaluation reports contained information on stability and security. However, despite GAC’s current interest in security and stability objectives, neither programming, nor evaluations consistently addressed or described how they tackled these objectives. As a result, the Department does not have a conclusive body of evidence from evaluation reports on what works well to inform its future programming. Issues such as democratic transition, good governance, and protection of human rights were often associated with stability in evaluation reports (e.g., Canadian Fund for Local Initiatives evaluation).  Specifically, 11% of evaluations that addressed security objectives and 29% of evaluations that addressed stability objectives could not be rated.

With these caveats, 44% of the evaluations that addressed security objectives had a strong rating and 44% had an acceptable rating. Similarly, 29% of the evaluations that addressed stability objectives also had a strong rating and 43% had an acceptable rating.

3.2.3 Contributing Factors

This review highlighted that the relevance of Gs&Cs programming was highly dependent on conducting appropriate needs assessment and situation analyses, engaging partners, and leveraging their respective strengths. These were the main contributing factors identified:

3.2.4 Limiting Factors

Although most evaluation reports provided a positive assessment of the relevance of programming, a few noted factors that limited the relevance of some interventions:

3.3 Effectiveness of Gs&Cs Programming

3.3.1 Coverage of Effectiveness Criteria

As shown in Figure 3.3 on the next page, coverage was high for the criteria that assessed overall program effectiveness, effectiveness of programming mechanisms, effectiveness of partnerships with traditional partners and use of performance management tools. Coverage of other criteria was significantly lower. Effectiveness of partnerships with non-traditional partners received little attention (2 reports).

Figure 3.3 Number of Evaluations that Addressed Effectiveness Criteria

Number of Evaluations that Addressed Effectiveness Criteria

Figure 3.3 - Text version
Rating1. Overall programming effectiveness2. Focus on effective programming mechanisms3. Focus on effective partnerships4. Focus on leveraging expertise from CSO5. Focus on complementarity of programming on development side6. Focus on use of performance management tools7. Focus on Canada's comparative advantage8. Focus on innovations in development 
Addressed 3731352220382419
Not Addressed 395182021621

3.3.2 Key Findings

This section presents the assessment of how well Gs&Cs programming performed on effectiveness criteria and Figure 3.4 on the next page shows the ratings.

Overall Effectiveness

In terms of average ratings for each effectiveness-related question covered by the evaluation reports reviewed, most ratings were either strong or acceptable for overall effectiveness and for progress towards expected development outputs and outcomes. Only a few reports noted insufficient evidence to confirm program effectiveness. Most country program evaluations reported positive results at the project level in various sectors, although these tended to be focused on the achievement of outputs and immediate outcomes (i.e., short-term results).

Figure 3.4 Distribution of Ratings by Effectiveness Criteria

Distribution of Ratings by Effectiveness Criteria

Figure 3.4 - Text version
RatingGs&Cs programming judged to be effective overall (n=37)Gs&Cs programming effectively leveraged expertise from CSO (n=22)Gs&Cs programming has effective partnerships with partner and other Canadian governments, CSO and multilateral organizations (n=35)Gs&Cs programming mechanisms are effective (n=31)Gs&Cs programming is recognized for innovations in development (n=19)Gs&Cs programming demonstrates Canada's comparative advantage (n=24)There is evidence of complementarity of Gs&Cs programming on development side (n=20)Gs&Cs programming effectively uses performance management tools (n=38)
Not applicable3%5%11%0%11%13%10%0%

Effective Leveraging of CSO Expertise

Most reports that addressed partnerships with CSOs provided a positive assessment of the program’s relationship with CSOs and received a strong rating with respect to the effective leveraging of CSO expertise (68%). In the context of fragile states, such as Afghanistan, using CSOs enabled access to certain populations but a disproportionate reliance on these organizations meant that linkages to national policies, strategies and implementation were often not sufficiently taken into account.

Effectiveness of Partnerships

In the reports reviewed, there was substantial evidence of diverse and strategic partnerships that added value to Gs&Cs programming. Partnerships with traditional partners, such as other Canadian government departments, national or sub-national governments in partner countries, CSOs and multilateral organizations, received strong or acceptable ratings (57% and 29%, respectively). The evaluation reports reviewed noted the existence of strong multi-stakeholder partnerships, with the evaluation reports on Afghanistan, Pakistan and Countries of Modest Presence providing particularly strong examples. Some evaluation reports mentioned that partnerships at different levels could be strengthened through enhanced coordination and communication (e.g., notably the foreign affairs evaluations of the Global Partnership Program, UNFPA and UNDP development effectiveness reviews and the Honduras country program evaluation).

Effectiveness of partnerships with non-traditional partners received little coverage. The two reports that provided information on non-traditional partners did not explicitly refer to these partners as non-traditional. The evaluation of the Canadian Police Arrangement mentioned work with policy services from three levels of government as bringing a new dimension to the whole-of-government approach. The evaluation of the Global Peace and Security Fund (2011) described a partnership with African Centers of Excellence to develop Africa’s peacekeeping training capacity.

Effectiveness of Programming Mechanisms

When assessed, the effectiveness of programming mechanisms was rated as either strong (52%) or acceptable (42%). Programming mechanisms were not always assessed individually in the evaluation reports; however, reviewer observations indicated it was the mix of programming mechanisms used to implement various projects in each program portfolio that contributed to overall program effectiveness. In the context of country program evaluations, the complementarity and flexible aid delivery resulting from combined mechanisms were identified in various evaluation reports as a factor contributing to success.


Recognition of Canadian innovations in development was, on average, rated as strong (47%) or acceptable (37%). The majority of innovations identified in country program evaluations were in governance sector programming, followed by the agriculture and environment sectors. Some innovations were also reported in the gender equality, education and health sectors. Innovations covered a broad range of initiatives in each of the sectors identified. Of note, most of the innovations in the governance sector were reported from countries of modest presence that had lower middle income status.

Canada’s Comparative Advantage

Canada’s comparative advantage was examined in 60% of evaluations reports. Of those, 33% assessed this area as strong and another 46% as acceptable. Most examples provided in reviewed evaluations concerned areas of gender equality and policy dialogue. Despite the limitations associated with RBM noted below, several reports (e.g., Peru country program evaluation, UNFPA and UNICEF development effectiveness reports) indicated that Canada’s promotion of RBM represented a distinct advantage among other donors and partners.

Complementarity of Programming

Only 20% of evaluation reports assessed complementarity of Gs&Cs programming as strong. A number of country program evaluations highlighted weaknesses in inter-program complementarity and synergies (e.g., the Inter-American Program, Colombia, Caribbean, Honduras, Haiti, Bolivia, Senegal, Mozambique-Tanzania). This appeared to be related to weak information dissemination or knowledge sharing among programs, including sharing regional trends, sectoral opportunities, results, lessons and good practices from different initiatives. This information was not readily available to other programs and projects that could benefit from it.

In the development effectiveness reviews, among the recurring challenges for the Department with regard to institutional support through the multilateral channel, were developing a strategy for engagement with the multilateral organizations that was known throughout the Department; and creating linkages between Canada's work at the Executive Board and in other parts of the Department. The objectives for institutional engagement were not always followed up with specific funding, yet in some cases (e.g., UNFPA) there was an expectation that it would be. This suggests a need to strengthen program coherence for development programming among the Multilateral, Geographic Programs, and Partnerships for Development Innovation Branches. To this effect, the Peru country program evaluation report suggested developing a liaison mechanism. In fragile states such as Haiti and Afghanistan, reports noted challenges in the integration of humanitarian assistance and development programming.

Complementarity between development and foreign affairs programming was discussed in a small number of evaluation reports, but appears to be more evident since the amalgamation of CIDA and DFAIT. Consultations with the Department suggested that complementarity between initiatives focused on security and on development was not always feasible given the distinct objectives and contexts of each. However, some of the evaluations suggested that there was room for improvement. Specifically, the evaluation of the Global Partnership Program (2015) noted a lack of coordination among the Department’s security and development programming in several Canadian missions abroad. The evaluation of the Global Peace and Security Fund referred to committees established to ensure that activities between foreign affairs and development did not overlap. However, these committees did not meet regularly and did not transfer information back to their respective agencies to help ensure that overlap did not occur.

Finally, evaluations spoke favourably of efforts to seek complementarity with other development partners. Canada participated actively in multi-donor working groups and discussion fora in several countries. These donor coordination mechanisms were used to strengthen coordination and aid effectiveness and to avoid duplication of efforts among stakeholders.

Use of Results-based Management Tools

Various results-based management (RBM) tools were used in evaluated Gs&Cs programming to monitor and report on results and manage risks. Results-based management was somewhat inconsistent at the project level and was generally scant and in some cases non-existent at the program level. The ratings for results-based management are almost evenly split between acceptable (47%) and weak (45%). Footnote 7

Several country program evaluation reports noted weaknesses in program-level reporting on the results achieved, largely due to performance measurement limitations, such as lack of Performance Measurement Frameworks, targets, baseline data, or ineffective monitoring and evaluation, which made reporting on higher-level program results difficult. The following observations were based on an analysis of the evaluations of Bangladesh, Bolivia, Pakistan, Indonesia, Senegal, Caribbean Regional Program, and Countries of Modest Presence:

As with bilateral programming, assessment of the effectiveness of Gs&Cs programming through the multilateral channels was affected by limitations in results-oriented planning, monitoring and reporting in multilateral organizations. Development effectiveness reviews noted that the programming of multilateral organizations was often ambitious, with objectives that lacked causal linkages, had poorly framed indicators and lacked baseline information. The reviews, notably of the UNFPA, UNICEF and IaDB that contained a section on Canada’s Management Practices, also reported that institutional engagement strategies were rarely accompanied by a Performance Measurement Framework to monitor and report on the Department's strategic engagement with the multilateral organizations. The Department was aware of this limitation and was piloting a Performance Measurement Framework with one of its new institutional engagement strategies.

While the practice of RBM was reflected in development programming and evaluations, the same practice was not evident in the foreign affairs evaluations. This could be attributed to the nature of the foreign affairs programming, which was often reactive and addressed immediate concerns in fragile circumstances where humanitarian assistance or military interventions were necessary. Ex-ante RBM designs were not feasible under such circumstances, nor were ex-post measurements. It may be useful to develop a generic Performance Measurement Framework that could be applied to measure contributions in this area of GAC’s mandate.

Consultations with stakeholders in the Department confirmed that the application of RBM remains a challenge. A small team of RBM experts within the RBM Centre of Excellence, together with RBM advisors (known as Program Management Advisors) located within operational branches provide technical assistance to more than 1,000 international assistance projects implemented by the Department this year. These projects are implemented by more than 800 different partners who all need to understand GAC’s expectations in relation to RBM. A guide on how to integrate RBM in international assistance programming is currently being developed.

During the consultations for this review, other factors that affected the use of RBM emerged. These included: the lack of an enabling environment in the Department (e.g., insufficient time to apply RBM during the planning, design and approval stages and insufficient time or resources to manage for results, lack of experience in formulating outcomes); different levels of RBM maturity within former CIDA and DFAIT; the lack of planning and budgeting to collect performance data; and high staff turnover within the Department. Footnote 8

3.3.3 Contributing Factors

The evaluation reports reviewed pointed to the following factors influencing the success of Gs&Cs programming:

Limiting Factors

The following factors contributed to some of the unsatisfactory evaluation findings in the area of programming effectiveness:

3.4 Efficiency of Gs&Cs Programming

Coverage of Efficiency Criteria

The evaluation reports reviewed generally addressed efficiency criteria (Figure 3.5). Most (88%) drew conclusions on programming efficiency and 76% generated information, qualitative mostly, to measure efficiency.

Figure 3.5 Number of Evaluations that Addressed Efficiency Criteria

Number of Evaluations that Addressed Efficiency Criteria

Figure 3.5 - Text version
Rating1. Overall programming efficiency2. Focus on data to measure efficiency 
Addressed 3531
Not Addressed 59

3.4.2 Key Findings

Overall Efficiency

The number of evaluation reports that rated the overall efficiency of Gs&Cs programming as strong and as weak was split fairly evenly.  Specifically, 26% of the evaluations reported strong ratings for efficiency and 23% had a weak rating. Of concern, only 6% of the evaluation reports had a strong rating for the generation of information to measure efficiency.  These numbers suggested that the Department faced challenges in clearly demonstrating the efficiency of its Gs&Cs programming.

Figure 3.6 Distribution of Ratings by Efficiency Criteria

Distribution of Ratings by Efficiency Criteria

Figure 3.6 - Text version
RatingGs&Cs programming judged to be efficient overall (n=35)Gs&Cs programming generates data to measure efficiency (n=31)
Not applicable0%26%

One of the main observations of this review was that the Department did not have a common set of criteria or approaches to assess efficiency across programs, which made it difficult to draw comparisons across types of programming or to trace good practices. Footnote 9 While it might not be feasible to have common efficiency criteria across all types of programming, it may be helpful to identify standard criteria by groupings (e.g., specific standards for assessing the efficiency of country programs, for example). Country program evaluations and development effectiveness reviews could use similar determinants to assess efficiency related to timeliness, allocation of resources relative to results achieved, or whether and how monitoring information is used to inform program implementation.

In the case of country program evaluations and foreign affairs programming, a number of factors that limited efficiency were under the direct control of the Department. These factors concerned the need to maintain stable workforce and avoid high staff turnover within the Department and streamlining approval authority. Further, the review highlighted that development programming and foreign affairs programming continued to operate distinctly after amalgamation, which was described as a limitation to programming efficiency. The number of processes for program design, implementation, monitoring and reporting that existed for programs and projects was high and was not yet consolidated following the amalgamation of CIDA and DFAIT. Finally, Performance Measurement Frameworks did not typically capture information to assess and monitor efficiency. Systems to collect data on costs per beneficiary were not designed upfront and no guidance was available in the Department on how to address measuring efficiency.

Multilateral organizations assessed in development effectiveness reviews received mixed ratings on efficiency; fewer than half of those reviews (3 out of 8) provided satisfactory ratings on efficiency. Multilateral organizations faced some of the same challenges faced by the Department. Many multilateral organizations lacked consistent data on efficiency indicators, such as costs of outputs, or were affected by the lack of timeliness of program delivery (often due to cumbersome administrative procedures). The UNICEF report stated that the most frequently cited factor impeding the cost and resource efficiency of UNICEF-supported programs was the lack of appropriate cost data reported regularly and on time to allow for a reasonably accurate calculation of services costs. Footnote 10 The fragmentation of programming in many multilateral organizations also seemed to undermine efficiency. In terms of Canada's engagement with the multilateral organizations, Canada was usually described as being responsive to the organizations, but on some occasions, especially during the period of the amalgamation of the departments (2013-2014), there was some lack of clarity about who took decisions on funding, and therefore a lack of timely response on funding requests (e.g., UNFPA, IaDB). The development effectiveness reviews also pointed to the need to ensure more regular and systematic communication between multilateral and bilateral programs (e.g., UNFPA, UNICEF).

In terms of departmental administrative processes, the evaluation of the Office of Religious Freedom/ Religious Freedom Fund explicitly referred to the Authorized Programming Process (APP). Footnote 11 It found that APP selection mechanisms were equal to or faster than non-APP mechanisms for approval and that the level of effort was only slightly higher for program staff.

Information to Measure Efficiency

The review highlighted a significant lack of quality data available to assess this criterion. Specifically, 26% of evaluation reports did not present any information with respect to this criterion and 45% were assessed as weak. Where efficiency was addressed it was commonly informed by qualitative evidence, such as interviews with various stakeholders.

Twenty reports identified the lack of financial data as an impediment to providing a complete assessment of efficiency.  Although Gs&Cs programming generated some data to measure efficiency, data were often incomplete or insufficient to provide a full picture of how resources were spent. Assessing costs by unit value were not commonly planned for at the design stage and were not collected. Consulted stakeholders also noted that providing a cost per unit value was more feasible for some type of programming than others (e.g., a program providing inoculation than programming focused on humanitarian action or political change).

3.4.3 Contributing Factors

Reports commented on factors that contributed positively to efficiency of Gs&Cs programming:

Although previous studies and analyses conducted by the Department did not focus heavily on efficiency, flexibility was sometimes highlighted as one of the factors leading to improved efficiency of programming (e.g., CIDA Learns: Lessons from Evaluations 2012). This was most apparent when programming was decentralized as it offered additional opportunities to transfer responsibilities to partners in the field allowing greater flexibility in terms of adaptation to specific local circumstances.

3.4.4 Limiting Factors

Many reports reviewed commented on the challenges in collecting quantitative financial information required to assess efficiency. Other obstacles to efficiency included:

3.5 Sustainability of Gs&Cs Programming

3.5.1 Coverage of Sustainability Criteria

While coverage of program sustainability was high overall (78%), many reports assessed the area qualitatively, commenting on the prospects for sustainability and not on the level of sustainable results achieved. Less than one-half (43%) addressed good practices for sustainability and a small number of reports (10%) examined the impact of inter-program cooperation on sustainability (Figure 3.7).

Figure 3.7 Number of Evaluations that Addressed Sustainability Criteria

Number of Evaluations that Addressed Sustainability Criteria

Figure 3.7 - Text version
Rating1. Overall programming sustainability2. Focus on inter-program cooperation3. Focus on good practices for sustainability
Not Addressed93623

3.5.2 Key Findings

In terms of overall sustainability, the most common rating was acceptable (42%). However, there were significant variations in individual report ratings for sustainability and a significant proportion was assessed as weak (32%). Of note, given that the time passed after completion of many of the programs and initiatives was too limited to allow for the full assessment of sustainability of results, evaluations generally drew conclusions on the prospects for sustainability rather than concluding on the level of sustainability actually achieved.

Figure 3.8 Distribution of Ratings by Sustainability Criteria

Distribution of Ratings by Sustainability Criteria

Figure 3.8 - Text version
RatingGs&Cs programming demonstrates good practices for sustainability (n=17)Gs&Cs programming results judged to be sustainable overall (n=31)Gs&Cs inter-program cooperation contributes to sustainability (n=4)
Not applicable18%0%25%

In the 31 reports that addresses sustainability, approximately half of the programming evaluated showed good prospects for sustainability. The majority of the evaluated programs contributed to strengthening the enabling environment for development. Evaluations of country programs often illustrated ownership of results at the country level, which boded well for sustainability, but there was limited concrete evidence of sustained results in the reports reviewed.

While there was limited evidence of good practices for sustainability, a few elements that emerged from the review team’s analysis in addition to the contributing factors outlined below were: skills transfer and training sessions on best practices adapted to country situations in various sectors (e.g., agricultural production and environmental protection in Haiti and West Bank and Gaza); building strong ownership at the local level; and planning for sustainability early in project/program implementation, for example by ensuring early buy-in from local governments and key stakeholders.

3.5.3 Contributing Factors

Evidence from the reports indicates the following factors enabled the sustainability of results:

There was limited evidence of the contribution of inter-program cooperation to sustainability, but some noteworthy examples emerged from the Egypt and Nicaragua country programs evaluated in the context of the countries of modest presence:

3.5.4 Limiting Factors

The reports also pointed to the following factors limiting the sustainability of results:

3.6 Integration of Cross-Cutting Themes in Gs&Cs Programming

3.6.1 Coverage of Cross-Cutting Theme Criteria

GAC has three cross-cutting themes that must be addressed and integrated in all development programming (gender equality, environmental sustainability and governance). Figure 3.9 shows the number of evaluation reports that addressed each cross-cutting criterion.

Figure 3.9 Number of Evaluations that Addressed Cross-Cutting Themes Criteria

Number of Evaluations that Addressed Cross-Cutting Themes Criteria

Figure 3.9 - Text version
Rating1. Overall programming on gender2. Overall programming on environmental sustainability3. Overall programming on governance
Not Addressed111616

Gender Equality: The theme of gender equality was addressed in 73% of the evaluation reports reviewed. A few country program evaluations identified gender equality both as a cross-cutting theme and as a program priority. Development effectiveness reviews of multilateral organizations indicated poor coverage of gender equality in several multilateral organizations (e.g. UNICEF, IDB, and WFP), though some recently began implementing new gender policies. The UN-System Wide Action Plan on gender equality and empowerment of women was only introduced in 2012.

Environmental Sustainability: While 60% of reports reviewed addressed environmental sustainability, only half of the reports that addressed this theme also explicitly incorporated information on the impacts of climate change. Environmental sustainability and climate change were often not covered by the development effectiveness reviews.

Governance: This theme was addressed in 57% of reports reviewed. Eight reports addressed governance solely as a program sector with no cross-cutting integration as these evaluations likely covered initiatives that preceded the departmental policy requirement that governance be considered a cross-cutting dimension of programming. Coverage of governance was particularly strong in 8 of the 18 country program evaluations reviewed, where governance was identified as both a program priority sector, e.g., in education or health, and a cross-cutting theme.

3.62 Key Findings

Evaluations of Gs&Cs programming did not consistently address cross-cutting themes, but where addressed, the majority of programs were rated as acceptable or strong (Figure 3.10).

Figure 3.10 Distribution of Ratings by Cross-Cutting Theme Criteria

Distribution of Ratings by Cross-Cutting Theme Criteria

Figure 3.10 - Text version
RatingGs&Cs programming effectively addresses governance (n=24)Gs&Cs programming effectively addresses gender (n=29)Gs&Cs programming effectively addresses environmental sustainability (n=24)
Not applicable0%7%4%

Six country program reports Footnote 16indicated that the cross-cutting themes were not fully captured in programming. In part, this appeared to be linked to limitations of applying a results-based management framework to identifying, measuring and monitoring the results associated with cross-cutting themes. Reports highlighted difficulties experienced by implementing agencies on how to integrate the themes operationally in logic models and Performance Measurement Frameworks, as well as challenges in understanding the Department’s expectations. Additional corporate guidance and technical support for the integration of gender equality, environmental sustainability, and governance, at project and program levels may help address these issues.

Gender Equality

Ratings for gender equality were generally acceptable, with 28% of evaluation reports showing a strong rating and 48% showing acceptable. Of note, 17% were assessed as weak and another 7% could not be rated due to the lack of evidence presented in evaluation reports.

While Canada has been recognized for its leadership in promoting gender quality dialogue and policy, the review observed a diminishing focus in programming in this area over time. Several country program evaluations reviewed (e.g., Indonesia, Mali, Countries of Modest Presence) noted a diminishing focus on gender equality over evaluation periods, while others (e.g., Caribbean, Ukraine) mentioned the elimination of Gender Equality Funds and limited availability of gender equality specialists to provide support with the integration of this theme in programming. Footnote 17 It was not clear whether these factors were a consequence of less corporate investment in gender equality over time, decentralization of programs and technical support from headquarters to the field, or whether it was a reflection of the maturity of the country in terms of progress towards an enabling environment for gender equality.

Among the eight multilateral organizations assessed through the development effectiveness reviews, UNFPA and UNDP received a positive assessment in terms of their implementation of gender equality principles. The other organizations either had insufficient information on, or coverage of, gender equality in their own evaluations, with only UNICEF rated as weak on gender equality integration. The latter was noted in the report as “surprising given that addressing gender equality represents a “foundation strategy” for UNICEF programming.” Footnote 18

A previous meta-synthesis of GAC evaluations on selected policy themes Footnote 19 also highlighted the challenge of having sustained capacity to address gender equality in programming. It indicated a need to develop capacity for gender equality integration and analysis within the Department, to ensure that projects had professional gender equality specialists available and to budget for these resources. The report further stressed that gender equality was often identified as a policy dialogue priority and was not always assigned a commensurate level of funding for programming.

Environmental Sustainability

Although environmental sustainability was addressed in 24 evaluation reports, evidence of its integration in programming was not sufficiently robust to draw conclusions on effectiveness in most instances. Only 17% of evaluations reviewed received a strong rating in this area, while 29% received a weak rating.

The development effectiveness reviews provided little information on how multilateral organizations addressed environmental sustainability (including climate change) and the extent to which Canada advocated for emphasis in these areas was not clear. The one exception was the UNDP review which reported that UNDP was effective in supporting environmentally sustainable development in several areas, such as enhancing national energy policies, improving rural and urban water resource management; strengthened conservation programs and improved promotion of bio-diversity and improved natural resource management capacity.

In foreign affairs programming, the evaluation of the Global Partnership Program found that environmentally sound nuclear disposal methods were implemented by Canada and its partners. For instance, the successful dismantlement of nuclear submarines by Canada, in cooperation with the United States and Russia, meant that a serious proliferation threat was contained and the environmental threat to the shores of Canada and other Arctic neighbours was reduced.

The country program evaluations generally indicated that environmental sustainability was not a major programming theme and tended to be integrated primarily in programs that included agriculture or environment-focused interventions. In several reports, including those concerning the fragile states of Afghanistan, Haiti, West Bank and Gaza, while projects were in conformity with the requirements of the Canadian Environmental Assessment Act at the design phase, environmental issues were not strongly addressed during implementation and, in the case of Haiti, environmental guidelines were reportedly not followed in some major electrical power and road construction projects, such as the Les Cayes-Jérémie Road project. On the other hand, the reports on Senegal, Ethiopia, Ghana and Indonesia country programs noted good integration of environmental considerations with results achieved in effective natural resources management, and the evaluation of the Bangladesh Program indicated that environmental requirements were met in all infrastructure-related initiatives.


Evidence of the effectiveness of governance integration in Gs&Cs programming was strongest in country program evaluations; development effectiveness reviews and foreign affairs evaluation reports made little explicit reference to governance issues. Overall, 46% of evaluation reports received a strong rating for governance and 50% were acceptable. The overall strong ratings in this area were likely due to the fact that governance was often identified as both a program priority sector and a cross-cutting theme and was evaluated accordingly.

Ten of the 18 country program evaluations reviewed Footnote 20 reported significant contributions to strengthening governance at national and sub-national levels and across various program sectors. The Mali and Indonesia country programs were particularly lauded for their effective role in strengthening governance at the local level.

Development effectiveness reviews had little information on how multilateral organizations are addressing governance. Only the UNDP review provided an assessment of the organization’s performance in this area. The report explained that UNDP achieved effective results in promoting democratic governance in recipient countries by promoting increased transparency, strengthening parliamentary systems, improving judicial and policing systems and enhancing peace-building.

Among the 11 evaluation reports on foreign affairs programming, the evaluation of the Canada Fund for Local Initiatives (CFLI) reported that the CFLI was effective in promoting democracy and human rights and that approximately 40% of CFLI funding was dedicated to initiatives supporting democratic transition and advancing democracy. The evaluation of the Americas Strategy noted important progress in the area of democratic governance in the Americas which received one-third of GAC’s democratic governance funding at the time of the evaluation (2011). The evaluation outlined several specific results achieved, including in the areas of conflict prevention and peace building efforts (Colombia) and in favour of civil society, open media outlets, and research networks focused on democracy.

3.6.4 Contributing Factors

This review identified that the following success factors in integrating cross-cutting themes:

3.6.4 Limiting Factors

The review pointed to the following factors limiting the integration of cross-cutting themes:

4. Conclusions and Considerations for Future Programming

4.1 Conclusions

This section presents the conclusions of this review based on the evidence derived from the evaluation reports, as well as from other GAC synthesis reports and consultations with stakeholders in the Department. Overall, GAC generated a strong body of evaluation evidence to conclude on IAE-funded Gs&Cs programming relevance and effectiveness, while evidence to conclude on the efficiency and sustainability of its programming and the advancement of the three cross-cutting themes was less robust.

Conclusion 1: GAC’s Gs&Cs programming was highly relevant

The programming funded by the International Assistance Envelope was aligned with Canada’s priorities in international development. The programming largely met the needs of its targeted beneficiaries in developing countries, as well as the needs and priorities of Canadians.  Most of the programs evaluated were implemented some time ago, thus they did not necessarily reflect the Department’s increased emphasis on the poorest and most vulnerable and how programming reached these groups.

Conclusion 2:  GAC’s Gs&Cs programming was generally effective, though gaps existed in the Department’s ability to conclude on the achievement of intermediate and ultimate outcomes

Gs&Cs programming was well covered by evaluations reports, although the body of evidence generally came from program success at the output and immediate outcome levels. Programming showed a strong leveraging of CSO expertise, partnerships, use of effective programming mechanisms and innovative approaches. Its effectiveness was supported by the knowledge, expertise and skills within Department and by partners.

While Canada was recognized as a strong advocate for the management and measurement of results, the Department had weakness in applying RBM for monitoring and evaluation purposes for its own programming. Those weaknesses impeded the ability to generate comprehensive evidence on effectiveness. The review noted that both the Department and its implementing partners faced challenges and lacked mutually acceptable methodologies in using RBM tools to measure, report on and improve performance. This was due in part to a lack of human and financial resources to collect and assess data, delays in implementing a results framework and methodology that connected project level performance information to program and corporate levels, and the absence of a fully functional data warehouse that could facilitate the aggregation of results information. RBM tools and frameworks were used broadly at the program and project design stages, but were not subsequently used to manage for results and take corrective action. Program-level performance information was often scant, especially for intermediate and ultimate outcomes. Frequent changes in corporate outcomes and indicators were also an issue.

Conclusion 3:  Canada was recognized for its engagement with traditional partners, but lacked differentiated partnership strategies to engage new and emerging partners

According to interviews and the evaluations reviewed, the Department demonstrates capacity to develop strong partnerships with traditional partners such as donors, governments, multilateral organizations, civil society organizations and with the private sector (as implementing partners). The view did not provide any insight on the Department’s work with non-traditional partners, such as emerging donors and southern philanthropic foundations, and other relationships with the private sector.

Conclusion 4: There was limited evidence to demonstrate that programming was efficient

The review pointed to limited quantitative evidence available to analyze the efficiency of IAE-funded Gs&Cs programming and draw firm conclusions. Most of the available evidence in the evaluation reports was qualitative in nature and recommendations on how to improve efficiency were not frequently provided. Efficiency data were limited to the project level, with little or no explicit links between costs and program-level expected results. There was no guidance or tools available within the Department on how to develop and monitor efficiency indicators at program and project levels.

Qualitative data suggested that decentralization and devolution of approval authority tended to reduce operating costs and bring the programs closer to targeted beneficiaries. Participation of local experts and advisors in the design and implementation processes was also shown to contribute to efficiency in some cases.

Conclusion 5:  From a methodological perspective, there was limited evidence to demonstrate that Gs&Cs programming results were sustainable

While the overall results were mixed, with some programming receiving a strong rating and others a weak rating, the review revealed limited evidence to demonstrate sustainability of the Department’s programming. The concern about demonstrating the sustainability of programming results is not unique to GAC; most donor organizations face the same challenge.

Sustainability is commonly measured ex-post, i.e., once the program has been completed and sufficient time has elapsed for impact (or ultimate outcomes) to become evident. Further, the assessment of sustainability requires a consistent measurement and results tracking mechanism. From a methodological perspective, the evaluations reviewed had notable constraints in both aspects: the timeframe they covered and the insufficient monitoring results available (i.e., a lack of longitudinal performance data).

Conclusion 6:  The Department was partially effective in integrating cross-cutting themes

In those evaluations that addressed the cross-cutting themes of gender equality, governance and environmental sustainability, the majority of programs were judged to have acceptable performance in these areas. The evaluations pointed to a number of challenges in integrating these themes into programming. These included challenges concerning incorporating cross-cutting themes operationally in logic models and Performance Measurement Frameworks, as well understanding the Department’s expectations with regard to cross-cutting themes.

In the area of gender equality, the evaluations reviewed suggested that the Department has not been able to sustain an emphasis on gender equality during the lifecycle of its programming. Elimination of gender equality funds and limited availability and uneven technical competencies of specialists to provide support were cited as contributing factors.

Environmental sustainability was inconsistently addressed across programming, while climate change was rarely covered in evaluations. This theme was integrated primarily in country-level programming that included agriculture or environment-focused interventions. The extent to which Canada has advocated for emphasis in this area with multilateral organizations was not clear.

Some evaluations reviewed suggested that a twin-track approach to addressing cross-cutting themes, which includes targeted interventions and mainstreaming of cross-cutting themes, may be more effective. This was most notable in the area of governance.

4.2 Considerations for Future Programming

The ongoing relevance of the Department’s programming will depend on how it adapts to the current context that is being shaped by the Agenda for Sustainable Development 2030 and other global commitments from COP 21 and the World Humanitarian Summit.

The Government of Canada has committed to focus its efforts on the poorest and most vulnerable. According to the World Bank, 70% of those living in extreme poverty now live in middle income countries, although the extreme poverty rate in low income countries is also extremely high, at around 50%. In addition, most of the fragile states in the world are now middle income countries, especially where the fragility has not been derived from natural causes.

The following considerations are based on lessons from this review and are presented to help the Government of Canada move towards this objective.

Considerations for Programming

Consideration 1: GAC should capitalize and further expand its recognized international strengths in the areas of policy dialogue (i.e., engagement with stakeholders and governments to improve policy and programs), the promotion of results-based management and gender equality, as well as the use of innovative programming approaches (e.g. partnerships with public, private and community-based entities)

The review noted several areas in which the Department was recognized for among its partners and other stakeholders. This includes Canada’s emphasis on advancing gender equality, policy dialogue, results-based management and also innovations in the governance and agriculture sectors. Other that other hand, the review highlighted that this emphasis was not consistent over time and showed some level of diminishing in recent evaluation reports. Also, the Department’s application of the promoted principles in these areas required strengthening.

Consideration 2: GAC should establish its common definitions of beneficiaries (e.g., the poorest and most vulnerable) and objectives (e.g., stability and security)

The review found that definitions of the poorest and most vulnerable varied across different contexts and types of programming. Canada’s activities in diplomacy, development, trade and defense would all benefit from a clear articulation and a common understanding of the overall target groups and objectives, including stability and security objectives, which appeared to vary between foreign affairs and development programming.

Consideration 3: GAC should explore opportunities to engage new and emerging partners

The evaluation reports provided limited information on how Canada engaged new and emerging partners. Canada’s current traditional partners are well-suited for work in low income countries but may not be sufficient to achieve development results in middle income countries or in rapidly changing environments.

Consideration 4: GAC should continue advancing the application of results-based management across the Department (including enhancing the ability to roll-up project results at the program and corporate levels and harmonizing reporting requirements) and provide tools and guidance on how to measure and improve efficiency and sustainability

The synthesis identified a number of limitations with regard to the Department’s application of results-based management across different types of programming. One of the main limitations was that performance management and measurement systems at the program and project levels did not work synchronously, making it difficult for the Department to show how it achieved its expected results. Intermediate and ultimate outcome level data were often lacking.

Another area for consideration identified by the review was the need for continued development and promotion of RBM across the Department and its implementing agencies, which includes training for staff, development of guidance documents, and information technology platforms. Challenges were also reported in RBM resourcing, as there were only small teams working across branches to support the application of RBM and provide technical advice to programming areas. Sector specialists could also help in the development of RBM tools, such as logic model and performance indicators in their respective sectors of programming.

Consideration 5: GAC should develop targeted programming in cross-cutting themes, in additional to mainstreaming

The review found some degree of integration of the cross-cutting themes in the programs and projects evaluated, but difficulties with effective operational integration of these themes persisted. The cross-cutting themes did not receive consistent and focused attention in Gs&Cs programming. The current framework review may provide an opportunity for GAC to combine its mainstreaming approach with a more active approach that includes developing explicit programs and/or strategies, particularly in the areas of gender equality and climate change, in cooperation with global efforts in these areas.

In the area of gender equality, for example, the Department could continue to support projects in gender-based budgeting and advocacy initiatives, but might also consider adopting an underlying cause approach, which means developing the capacity to trace gender inequalities back to their underlying root causes, design programming that is transformational, and establish partnerships with women’s movements. This would also be in line with the global development agenda as espoused in the Sustainable Development Goals.

The focus on governance could be strengthened and shifted from addressing broader, national-level governance concerns of the developing countries (where there is often insufficient influence to effect a change) to improving governance at the local levels in the recipient countries.

Consideration 6: GAC should strengthen technical support and provide guidance in the areas of implementing and measuring cross-cutting themes

Some of the challenges faced by the Department in integrating cross-cutting themes derived from limited guidance and technical support available to support program and project management. The quality of the technical support and availability of resources was reported as uneven across branches and programs, in part due to decentralization of programming and elimination of technical support teams. Guidance and support in identifying appropriate performance measures for cross-cutting themes were noted in the review.

Considerations for Future Evaluations

In terms of the conduct of evaluations, this review found that a number of areas required strengthening, which was applicable to both departmental evaluation functions:

Date Modified: