Development Effectiveness Review of the United Nations Population Fund (UNFPA) - 2008 – 2014

Synthesis Report

Acknowledgements

Global Affairs Canada’s Development Evaluation Division wishes to thank all who have contributed to this review exercise for their valued input.

Our thanks go to the team from the consulting firm, Goss Gilroy Inc. (Hubert Paulmer, Bruce Goodman and Tasha Truant), who conducted the review.

The team is grateful for the engagement by UNFPA and its independent Evaluation Office for its helpfulness and useful, practical advice.

The Development Evaluation Division would also like to thank the United Nation’s team of Global Affairs Canada’s Global Issues and Development Branch at Headquarters and in New York for their useful advice and valuable support throughout the process. In particular, we would like to thank Barbara Shaw for her facilitation role in this process.

Tara Carney led this review on behalf of Global Affairs Canada and Deborah McWhinney was responsible for developing the chapter on Canada’s engagement with UNFPA.

David Heath
Head of Development Evaluation
Global Affairs Canada

List of Acronyms and Abbreviations

Executive Summary

Background

This report includes two components. First, it presents the results of a review of the development effectiveness of the programs supported by the United Nations Population Fund (UNFPA) between 2008 and 2014. Second, it includes an assessment of Canada’s engagement with the UNFPA.

UNFPA is the United Nation’s lead agency for “delivering a world where every pregnancy is wanted, every birth is safe, and every young person's potential is fulfilled.”^{Footnote 1} It is the world’s largest international source of funding for population and reproductive health programs.

Purpose of the Review

The purpose of the development effectiveness review component is to: a) provide Canada and the broader donor community with an independent, evidence-based assessment of UNFPA’s relevance and performance; and, b) to support Canada and the donor community’s relationship with UNFPA by identifying lessons for program and policy improvements. The purpose of the assessment of Canada’s engagement is to help Global Affairs Canada strengthen its relationship with UNFPA.

Approach and Methodology

The report used a methodology developed under the guidance of the Organization for Economic Cooperation (OECD) and Development’s Development Assistance Committee (DAC) Network on Development Evaluation. It involved a quality assurance review of published evaluation reports produced by UNFPA between 2008 and 2014, followed by a systematic and structured meta-synthesis of the findings from a sample of 62 UNFPA evaluations that met quality standards. The analysis was complemented by a review of UNFPA corporate documents. Canada’s engagement with the UNFPA was assessed through a document review as well as interviews with Global Affairs Canada and UNFPA staff and other Executive Board member states.

As the scope of this review included a period following the amalgamation of the former Canadian International Development Agency (CIDA) and the Department of Foreign Affairs and International Trade (DFAIT) into the Department of Foreign Affairs, Trade and Development^{Footnote 2} which was subsequently renamed Global Affairs Canada, reference is made both to CIDA in relation to decisions made prior to amalgamation and to Global Affairs Canada to describe ongoing or post-amalgamation decisions or situations.

Key Findings

Development Effectiveness of UNFPA

1. UNFPA programs were highly relevant to the needs of target groups and UNFPA has been successful in developing effective partnerships with partner governments and in aligning its programs with national development goals, plans and priorities.
2. UNFPA was effective in achieving the development objectives of its programs and in contributing to significant changes in national development policies and programs. UNFPA produced positive benefits for target group members.
3. UNFPA’s performance with respect to gender equality was highly effective. This is in line with the fact that gender equality is an integral part of UNFPA’s mandate. UNFPA programs not only contain specific gender components but also mainstream gender equality into priority program areas.
4. Environmental sustainability was not covered. There was almost no coverage of environmental sustainability in the evaluations reviewed and hence this evaluation criterion did not merit a presentation of the findings reported.
5. The sustainability of UNFPA supported programs was mixed. UNFPA programs were reported to have contributed to strengthening institutional and/or community capacity and the enabling environment for development. However, the evaluation results for the likely continuation of benefits after program/project completion were negative.
6. The results for efficiency of UNFPA programming were mixed.^{Footnote 3} Although there is evidence of appropriate systems and procedures in place for project implementation and follow-up in UNFPA programs, not tracking cost data to assess cost/resource efficiency combined with rigid and bureaucratic administrative and financial systems caused delays in implementation and hindered efficiency.
7. UNFPA showed mixed results on performance management; evaluation is systematic and steadily improving, but results-based management is weak. While findings on evaluation systems and processes as well as the use of evaluations to improve development effectiveness were positive, those for monitoring and result-based management systems were predominantly negative.

Canada’s Engagement with UNFPA

The review of Canada’s engagement with UNFPA focused on Canada’s engagement at an institutional level, at the Executive Board, as a member of the Steering Committee of H4+ and in certain bilateral contexts. This review found that:

1. The strategic objectives defined by CIDA’s Strategy for Engagement with UNFPA (2011) remained relevant to Global Affairs Canada’s development priorities; however, Global Affairs Canada’s support for UNFPA has focused on only certain areas within the Strategy.
2. Canada’s work at the UNFPA Executive Board consistently focused on management effectiveness and gender equality concerns over the past five years.
3. There was a lack of performance measurement framework with which to monitor and report on Global Affairs Canada’s strategic engagement with UNFPA.
4. Canada’s bilateral support for UNFPA’s efforts to improve maternal health, promote gender equality and support the needs of youth centred on support for maternal health, gender equality promotion and, more recently, emergency programming.
5. Canada was widely recognised for its effectiveness at the UNFPA Executive Board.
6. Canada showed its support for UNFPA in UN reform through its work on the Executive Board and as a key supporter of the H4+ Partnership.
7. Global Affairs Canada assessed UNFPA to be a strong advocate for gender equality, which was corroborated by the findings of the review of development effectiveness of UNFPA.
8. Canada’s interactions with UNFPA were generally characterised by high levels of responsiveness, timeliness and openness. However, on some occasions, UNFPA perceived a lack of clarity on the status of funding requests.
9. There was a mixed level of awareness of the Department’s Strategy for Engagement with UNFPA and an absence of systems to facilitate linkages between Canada’s work at the Executive Board with other parts of the Department.

Recommendations for Global Affairs Canada

The following recommendations were developed with the intention of helping Global Affairs Canada to strengthen its engagement with UNFPA.

1. Update Canada’s Strategy for Engagement with UNFPA to include: a) a theory of change; b) clearly articulated alignment between UNFPA’s strategic results framework and Global Affairs Canada’s priorities; c) articulation of specific, concrete change in organizational behavior/performance/practice that Global Affairs Canada would like to see from UNFPA; and, d) a performance measurement framework that captures all engagement with UNFPA across the Department.
2. Develop a communications strategy for Canada’s engagement with UNFPA.
3. Develop internal mechanisms for sharing information between multilateral and bilateral branches in order to improve Canada’s effectiveness at the Executive Board.

Global Affairs Canada Management Response

The Development Effectiveness Review (DER) of UNFPA, prepared under the supervision of Global Affairs Canada’s Evaluation Division, provides an overview of the development effectiveness of the entity, an assessment of Global Affairs Canada’s engagement with UNFPA (Canadian chapter), and recommendations intended to help Global Affairs Canada strengthen its relationship with UNFPA.

The DER correctly notes that UNFPA is an important development partner for delivering on Canadian development goals. For example, UNFPA is the administrative agent and a key implementer of the Health 4+ Initiative for Maternal, Newborn and Child Health and is a key partner for Canada in working on issues related to child, early and forced marriage (CEFM). In 2014, Canada ranked 11th amongst all donors contributing to core resources (long-term institutional support), and 6th when both core and non-core contributions were considered. 

Global Affairs Canada agrees with the conclusions related to the development effectiveness of UNFPA as outlined in the DER. These findings support the findings of the earlier MOPAN (2014), as well as those found in Global Affairs Canada’s 2013 Multilateral Review of UNFPA. UNFPA demonstrated high relevance to the needs of target group members, had strong alignment to national development priorities and established good partnerships with governments, development partners and non-governmental organizations. UNFPA’s programs also demonstrated that they achieved stated objectives, and were highly effective in addressing gender equality. Less positively, UNFPA had mixed results with respect to the sustainability of program benefits and results, efficiency and performance management. We agree with the DER’s findings that recent efforts to improve in these areas through the implementation of the new Strategic Plan 2014-2017 are promising and that these improvements should be evident over time.

Global Affairs Canada welcomes the findings related to Canada’s engagement with UNFPA.  As previously noted, we confirm the finding that UNFPA is a highly relevant partner for Global Affairs Canada. We also appreciate the recognition of Canada’s key contributions and effectiveness in its participation at the Executive Board.

Global Affairs Canada accepts the recommendations included in the Canadian chapter. The recommendations call on Global Affairs Canada to: 

1. Update the strategy for Canada’s engagement with UNFPA to include a) a theory of change; b) clearly articulated alignment between UNFPA’s strategic results framework and Global Affairs Canada’s priorities; and c) articulation of specific, concrete change in organizational behavior/performance/practice that Global Affairs Canada would like to see from UNFPA; and d) a performance measurement framework that captures all engagement with UNFPA across the Department;
2. Develop a communications strategy for Canada’s engagement with UNFPA;
3. Develop internal mechanisms for information sharing between multilateral and bilateral branches in order to improve Canada’s effectiveness at the Executive Board.

Global Affairs Canada will undertake specific actions as outlined in the detailed responses to the specific recommendations provided in the table below.
This management response reflects the departmental priorities as expressed at the end of the evaluation period. Going forward, management responses will reflect new Departmental priorities as they evolve.

1. Introduction

1.1 Background

This report includes two components. First, it presents the results of a review of the development effectiveness of the programs supported by the United Nations Population Fund (UNFPA). Second, it includes an assessment of Canada’s engagement with the UNFPA.

The development effectiveness review relied on the content of published evaluation reports produced by UNFPA between 2008 and 2014, supplemented with a review of UNFPA corporate documents. It used an approach and methodology developed under the guidance of the Organisation for Economic Cooperation and Development’s (OECD) Development Assistance Committee (DAC) Network on Development Evaluation. This method was pilot tested in 2010 and since then has been used to conduct reviews of the development effectiveness of several multilateral organizations.^{Footnote 4} Since its inception, the process of reviewing the development effectiveness of multilateral organisations has aimed to complement the work of the Multilateral Organisation Performance Assessment Network (MOPAN) which focused on organisational effectiveness. By focussing on development effectiveness using OECD-DAC selecting assessment criteria (Appendix A) and evaluation evidence, the reviews seek to avoid duplication with MOPAN process, which assesses organisational effectiveness in a broad range of areas using both survey data and evidence from a document review.^{Footnote 5}

Canada’s engagement with the UNFPA was assessed through a document review as well as interviews with Global Affairs Canada and UNFPA staff and other Executive Board member states.

1.2 Purpose

The objectives of the development effectiveness review are to: a) Provide Canada and the donor community with an independent, evidence-based assessment of UNFPA’s relevance and performance; and, b) Support Canada and the donor community’s relationship with UNFPA by identifying lessons for program and policy improvements.^{Footnote 6} The purpose of the assessment of Canada’s engagement is to help Global Affairs Canada strengthen its relationship with UNFPA.

1.3 UNFPA: A Global Organization Focused on Population and Sexual and Reproductive Health Rights

1.3.1 Background and Objectives

• Established in 1969, UNFPA is an influential advocate for sexual and reproductive health and rights at a global level. It is the world’s largest international source of funding for population and reproductive health programs and the lead United Nation’s agency for population issues, and sexual and reproductive health and rights (including family planning).
• UNFPA is responsible for monitoring the implementation of the Programme of Action of the International Conference on Population and Development (ICPD). Its mandate is based on this Programme of Action and has been influenced by the Millennium Development Goals (MDGs), particularly MDG 5. In 2010, the UN General Assembly extended the ICPD beyond 2014. The organisation’s strategic plan is driven by these commitments along with guidance from the General Assembly resolutions related to the Triennial and Quadrennial Comprehensive Policy Reviews, other pertinent resolutions of the General Assembly and Economic and Social Council, and Executive Board decisions.  

1.3.2 UNFPA’s Strategic Direction

UNFPA’s key programming areas, as defined by the Strategic Plan 2008-2013,^{Footnote 7} included:

• Expanding and improving maternal and newborn health;
• Increasing access to voluntary family planning;
• Making HIV and sexually transmitted infection services more accessible to pregnant women people living with HIV, young people and key populations;
• Advocating for gender equality and reproductive rights;
• Increasing young people’s access to sexual and reproductive health services and information;
• Linking population dynamics, policy-making and development plans; and
• Harnessing the power of data.^{Footnote 8}

Compared to the original Strategic Plan 2008-2011, which had three focus areas (“goals”), 13 development outcomes, and 26 indicators, the revised and extended 2008-2013 version was more focused with one goal, seven outcomes and 17 indicators. The UNFPA Strategic Plan 2014-2017 aims to address the unfinished ICPD agenda, with a particular concentration on sexual and reproductive health and reproductive rights. It reaffirms the strategic direction set out in the mid-term review of the 2008-2013 Strategic Plan (”Bull’s Eye”).^{Footnote 9}

1.3.3 UNFPA Programming and Operations

In 2013, UNFPA offices combined had a total of 2,471 regular staff.^{Footnote 10} It worked with 159 countries, territories and other areas through a network of 112 country offices, six regional and three sub-regional offices, in addition to six liaison offices. Its gross income through support from donor governments, partner organizations, foundations and individuals accounted for USD 977 million divided between core (48%) and non-core (52%) resources.^{Footnote 11} UNFPA’s program expenses for 2013 accounted for USD 762 million. A little more than half of the program expenses were directed towards maternal and newborn health (26%) and family planning (25%) (Figure 1). In terms of regional focus, 39% of the program expenses were directed to sub-Saharan Africa (Figure 2).^{Footnote 12}

Figure 1: UNFPA Program Expenditures by Focus Area - 2013

Figure 1: Text Alternative

• Maternal and Newborn Health - 26%
• Family Planning - 25%
• HIV and STI Prevention Services - 4%
• Gender Equality and Reproductive Rights - 10%
• Young People's SRH and Sexuality Education - 7%
• Population Dynamics - 7%
• Data Availability and Analysis - 10%
• Program Coordination and Assistance - 10%
• Other - 2%

Figure 2: Proportion of UNFPA Program Expenditures by Region - 2013

Figure 2: Text Alternative

• Eastern and Southern Africa - 21.70%
• Western and Central Africa - 16.70%
• Arab States - 10.10%
• Eastern Europe and Central Asia - 3.30%
• Asia Pacific - 17.50%
• Latin America and Caribbean - 8.60%
• Global - 22.20%

1.4 Evaluation and Results Management Systems at UNFPA

This section presents a summary of UNFPA’s evaluation function, policies and practices.

1.4.1 Evaluation Function and Policy at UNFPA

Evaluation Policy

UNFPA’s first evaluation policy was approved by its Executive Board in June 2009 and revised in 2013. Prior to its revision, there were concerns with regard to the independence of the evaluation function and the use of lessons learned from evaluations and results-based management.^{Footnote 13} The 2013 revision ensured that the evaluation function is aligned with the norms and standards of the United Nations Evaluation Group and with international best practices. It took into account the review of the UNFPA evaluation policy in 2012, by the United Nations Office of Internal Oversight Services, as well as the General Assembly resolution^{Footnote 14} on the quadrennial comprehensive policy review of operational activities for development of the United Nations system. The revised policy ensured that the roles and responsibilities related to evaluation tasks were clearly articulated, including quality-assurance and standardised criteria for evaluations.^{Footnote 15} It led to the creation of the independent Evaluation Office and the position of Director of the Evaluation Office. 

Structure and Reporting

The independent Evaluation Office reports administratively to the Executive Director and directly to the Executive Board on the evaluation function. It is the custodian of the evaluation function, covering corporate evaluations and program-level evaluations, as well as all evaluation-related core tasks.^{Footnote 16} The 2013 policy places a strong emphasis on strategic planning and quality assurance, on prioritizing evaluation, and on ensuring sufficient resources.

At UNFPA regional and country office levels, monitoring and evaluation officers and focal points provide support to decentralized evaluations, including by managing evaluations and carrying out quality assurance activities. About 38% of UNFPA Country Offices have dedicated monitoring and evaluation officers. Sixty two percent of UNFPA country offices have a professional-level post that acts as the focal point for monitoring and evaluation. The broad span of these roles has implications for the amount of time that can be devoted to different activities and on the balance of expertise and capacity required to meet programming, monitoring and evaluation needs. The recruitment of officers with evaluation expertise in the principal program areas of UNFPA is a challenge for many country offices. ^{Footnote 17}

Quality Assessment and Assurance

In 2011, the former Evaluation Branch (part of the Division of Oversight Services) established evaluation quality assurance (EQA) mechanisms to improve the quality and credibility of UNFPA country program evaluations through the assessment of all evaluation reports against specific quality criteria. In 2013, the Handbook on How to Design and Conduct a Country Programme Evaluation at UNFPA was revised to reflect the changes in the UNFPA evaluation policy. Quality assessments mechanisms were strengthened and included in the review and approval of all evaluation terms of reference and a process to pre-qualify evaluators.^{Footnote 18}

1.4.2 Reporting on Development Effectiveness

UNFPA reports to the Executive Board on its performance, using outcomes identified in its Strategic Plan, in the Annual Report of the Executive Director. The report has been limited in that it was unable to provide sufficientevidence of UNFPA’s contribution to outcomes achieved.^{Footnote 19} As explained in the Annual Report of the Executive Director, this was partly the result of weak measurement metrics and inconsistent monitoring. For example, in working within UN Development Assistance Framework and Delivering as One context due to the lack of properly defined outputs andclear links to outcomes (with respective indicators).

The 2014 MOPAN assessment rated UNFPA as adequate in providing evidence of progress towards organization-wide results; it rated UNFPA as inadequate in terms of providing evidence of progress towards stated country-level results. It also noted that there are concerns around the quantity and quality of data used to inform performance reports and evaluations.^{Footnote 20}

2. Methodology

This chapter briefly outlines the methods used to conduct this review. A more detailed description of methodology is available in Appendix C.

2.1 Preliminary Review

The process of assessing the development effectiveness of UNFPA began with a Preliminary Review of its published corporate reports, and a brief assessment of the coverage and quality of the evaluations it produces. Based on the determination that UNFPA had an independent evaluation function that produces an adequate body of reliable and credible evaluations,^{Footnote 21} the review team proceeded to conduct a systematic analysis and synthesis of the findings presented in a sample of UNFPA evaluation reports.

2.2 Evaluation Population, Sample and other Data Collection

A purposive sample of 62 evaluation reports were systematically reviewed and their findings used for the analysis in this review (Appendix B). The selection ensured that there was adequate coverage by year and by region and is illustrative of UNFPA programming (Appendix D). Further information about UNFPA’s effectiveness was gathered through a qualitative review of the global and thematic evaluations (Appendix E) completed during the 2008 to 2014 period. This was supplemented by a review of UNFPA corporate documents to contextualize findings.

2.3 Assessment Criteria

The methodology focuses on the following essential characteristics of effective multilateral programming, derived from DAC evaluation criteria:

• Programming activities and outputs are relevant to the needs of the target group;
• Programming contributes to the achievement of development objectives and expected development results at the national and local level in developing countries;
• The benefits experienced by target group members and the development (and humanitarian) results achieved are sustainable in the future;
• Programming is delivered in a cost efficient manner;
• Programming is inclusive by supporting gender equality and being environmentally sustainable (thereby not compromising the development prospects in the future); and,
• Programming enables effective development by allowing participating and supporting organizations to learn from experience and use tools such as evaluation and monitoring to improve effectiveness over time.

The methodology involves a systematic and structured review of the findings of UNFPA evaluations, as they relate to six main criteria and 19 sub-criterion that are considered essential elements of effective development and humanitarian programming (Appendix A).

2.4 Limitations

As with any meta-synthesis, there are methodological challenges to be taken into account when considering and using evaluation findings. For this review these include:^{Footnote 24}

• The evaluation sample was illustrative rather than statistically representative of UNFPA programs, although it did cover a substantial body of programming as it evolved over time;
• Where qualitative observations could be made about a given program focus area and/or country classification, these were reflected in the report. However, these are only illustrative;
• The 62 evaluation reports in the sample, while they passed the quality screening process, do vary in quality (See Appendix E); and,
• The results reported apply to programs active as recently as 2013 or as long ago as 2004. During this period, UNFPA’s policies, strategies and approaches to programming changed.

3. Findings on UNFPA’s Development Effectiveness

This chapter presents the results of the review as they relate to the six main development effectiveness criteria and their associated sub-criteria.

Table 1 summarizes the findings with respect to coverage and the proportion of evaluations reporting findings of “satisfactory” or “highly satisfactory” for each of the criteria.^{Footnote 25}

This chapter then presents the extent to which each sub-criterion was addressed in the evaluation reports (coverage) and the results. This includes quantitative findings and the qualitative analysis of contributing or hindering factors.

3.1 Relevance

3.1.1 Coverage

There was strong coverage of the three sub-criteria relating to relevance (Figure 3). Ninety percent of the evaluations covered sub-criterion 1.1 on programs suiting the needs of the target group. All but one of the evaluations addressed sub-criterion 1.2 on aligning with national development goals and 1.3 on effective partnerships.

Figure 3: Number of Evaluations Addressing Sub-criteria for Relevance

Figure 3: Text Alternative

Relevance

	Addressed	Not Addressed
1.1 Programs suited to the needs of the target group	56	6
1.2 Programs align with national development goals	61	1
1.3 Effective partnerships	61	1

3.1.2 Key Findings

UNFPA’s programming was highly relevant to the needs of the target groups – women, adolescents and youth, people living with HIV and other key vulnerable populations (Figure 4). Seventy seven percent (43/56) of the evaluations reported that UNFPA programs were suited to the needs of the target group (sub-criterion 1.1) and that UNFPA had developed satisfactory or better on effective partnerships (sub-criterion 1.3). All 61 evaluations reported that UNFPA programs were satisfactory or better in terms of aligning with national development goals (sub-criterion 1.2). None of the evaluations reported highly unsatisfactory findings for suitability to the needs of the target group and aligning with national development goals.

Figure 4: Findings for Relevance

Figure 4: Text Alternative

Findings for Relevance

	Highly Satisfactory (4)	Satisfactory (3)	Unsatisfactory (2)	Highly Unsatisfactory (1)
1.3 Effective partnerships (n=61)	21%	56%	21%	2%
1.2 Programs align with national development goals (n=61)	57%	43%	0%	0%
1.1 Programs suited to the needs of the target group (n=56)	14%	63%	23%	0%

With regard to UNFPA’s effectiveness in entering into partnerships, evaluations identified a wide range of partners involved in UNFPA’s programming. Partnerships, specifically with national governments, local institutions and community organizations, strengthened local technical capacity and involvement which enabled local ownership. Partnerships helped UNFPA to leverage resources and share costs for programming. However, limitations included the absence of a clear partnership strategy, the inability to leverage partner’s strengths and not including key regional and learning institutions in consultations.

All evaluations indicated that UNFPA programming was aligned with national priorities as articulated in national plans, national programs, policies, plans and development frameworks.

3.1.3 Contributing Factors

Evaluations identified the following as contributing to the relevance of UNFPA interventions:

• The use of needs assessment and situation analysis to help assess the needs of target groups and enable appropriate programming choices (14 evaluations);
• The ability of UNFPA to establish strategic partnerships with institutions that were responsible for policy, planning and/or implementation (13 evaluations) – this included government agencies, NGOs, Community Based Organizations, local leaders, academic, research and professional bodies, and media; and,
• Appropriate targeting of vulnerable populations/groups (11 evaluations).

Evaluations also highlighted the following hindering factors:

• Weak coordination due to lack of capacity, no formal coordination mechanisms and limited engagement (8 evaluations); and,
• An inability to leverage implementing partners’ strengths as a result of their lack of involvement in UNFPA project planning, or to gaps in communication, fragmented and/or a siloed approach to program planning (7 evaluations).

3.2 Achievement of Objectives

3.2.1 Coverage

There was strong coverage of sub-criteria relating to the achievement of objectives^{Footnote 26} in the evaluations  the achievement of stated objectives by programs and projects (sub-criterion 2.1); benefits for target group (sub-criterion 2.2); and changes in national policies and programs (sub-criterion 2.4) (Figure 5). However, for results reaching a substantial number of beneficiaries (sub-criterion 2.3), coverage was moderate with only 32 evaluations including relevant findings.

The relatively lower coverage of sub-criterion 2.3 was explained by the fact that 30 evaluation reports provided neither quantitative nor qualitative estimates of the number of persons reached by UNFPA supported programs. Some evaluations indicated that UNFPA program resources were thinly spread across a large number of sub-programs or geographic locations (especially district level interventions), which may have affected both reach and the evaluation’s ability to estimate the number of persons benefiting (see section 5.1).

Figure 5: Number of Evaluations Addressing Sub-criteria for Achievement of Objectives

Figure 5: Text Alternative

Objective Achievements

	Addressed	Not Addressed
2.1. Programs and projects achieve stated objectives	53	9
2.2 Positive benefits for target group members	57	5
2.3 Difference made for a substantial number of beneficiaries / contributed to national development goals	32	30
2.4 Significant changes in national development policies and programs	58	4

3.2.2 Key Findings

The findings related to UNFPA programs achieving their stated development objectives were positive (Figure 6). Of the evaluations that addressed the sub-criterion 2.1, 64% (34/53) indicated that UNFPA supported programs were satisfactory or better in achieving their stated objectives. This did not necessarily mean that UNFPA programs achieved two-thirds of their objectives across the programs. Rather, it implied that in 64% of the evaluations UNFPA programs were reported to achieve more than 50% of their objectives, including the most important ones. This is consistent with 2014 MOPAN findings in terms of results achievement.^{Footnote 27}

Strong positive results were seen for programs’ ability to provide benefits for the target beneficiaries (sub-criterion 2.2) with 86% of the evaluations (49/57) reporting satisfactory findings. However, overall findings were evenly split on the differences made for substantial numbers of beneficiaries (sub-criterion 2.3), with 50% of the evaluations (16/32) reporting satisfactory results.

Programs were rated satisfactory or better for their ability to contribute to significant changes in national policies and programs (sub-criterion 2.4) in 67% of the relevant evaluations (39/58). With the exception of sub-criterion 2.1 “programs achieve stated objectives”; the other sub-criterion did not report any highly unsatisfactory results.

The review did not assess the differences in the achievement of objectives across UNFPA’s three program components – reproductive health, population dynamics and gender.

Figure 6: Findings for Achievement of Objectives

Figure 6: Text Alternative

Findings for Objective Achievements

	Highly Satisfactory (4)	Satisfactory (3)	Unsatisfactory (2)	Highly Unsatisfactory (1)
2.4 Changes to national policies/programs (n=58)	7%	60%	33%	0%
2.3 Substantial numbers of beneficiaries (n=32)	0%	50%	50%	0%
2.2 Positive benefits for target group members (n=57)	0%	86%	14%	0%
2.1 Programs achieve stated objectives (n=53)	4%	60%	32%	4%

3.2.3 Contributing Factors

Factors that positively contributed to the achievement of development objectives included:

• UNFPA’s ability to identify key policy actors and work with them in the development of strategic policies and frameworks in family planning, gender based violence, sexual and reproductive health, HIV/AIDS, reproductive health and/or, gender equality. This included advocacy work on key issue of ICPD goals and work with government departments and agencies, public institutions, CSOs and other donors (about 50% of the evaluations);
• The positive and active role of UNFPA in building coalitions among UN agencies, partner government ministries and service providers (for example, steering committees and/or technical working groups) (about 50% of evaluations); and,
• UNFPA’s support to increasing the use of quality information to enable better decision making and programming (8 evaluations).

Factors that negatively affected the ability of UNFPA programs to achieve stated objectives included:

• High staff attrition rates – doctors, midwives, nurses, health care workers and youth workers (10 evaluations); and,
• Weakness in program design. For example, ambitious goals with no causal linkages, lack of a results orientation, and a fragmented approach with program support geographically scattered and isolated with an absence of needed follow-on projects (6 evaluations).

Global thematic evaluations echoed similar contributing factors for achievement of program objectives and results.^{Footnote 28}

3.3 Cross-Cutting Themes

3.3.1 Coverage

The levels of coverage of the two cross-cutting themes contrasted strongly (Figure 7). There was strong coverage for effectively addressing gender equality (sub-criterion 3.1) – with relevant findings addressed in 55 of 62 evaluations. This was in line with gender equality being an integral part of UNFPA mandate (three key areas of its mandate are reproductive health, gender equality and population and development strategies).^{Footnote 29}

The coverage on sub-criterion 3.2 related to changes being environmentally sustainable was addressed in three evaluations. Given the weak coverage, the review was not able to report on effectiveness of UNFPA supported program in this area.

Eighty-nine percent of evaluations addressed gender equality. In order to rate this sub-criterion, evaluations had to not only identify the extent to which the program or project in question had incorporated gender equality objectives but also whether they had been achieved. Gender equality was also considered to be addressed in an evaluation report if it assessed the program’s success in mainstreaming gender equality. If an evaluation simply reported the proportion of girls or women receiving benefits compared to boys or men, the rating used was “not addressed” as there was no reference to the program’s success in addressing equality. Compared to previous Development Effectiveness Reviews, UNFPA evaluations achieve a high level of coverage of gender equality.

Figure 7: Number of Evaluations Addressing Sub-criteria for Cross-Cutting Themes

Figure 7: Text Alternative

Cross-cutting Themes

	Addressed	Not Addressed
3.1  Activities effectively address gender  equality	55	7
3.2 Changes are environmentally sustainable	3	59

3.3.2 Key Findings

The evaluations reported positive findings for gender equality (sub-criterion 3.1). Seventy five percent (41/55) of the evaluations that addressed gender equality reported satisfactory or better findings and none reported highly unsatisfactory results (Figure 8). ^{Footnote 30}

Figure 8: Findings for Effectiveness in Supporting Gender Equality

Figure 8: Text Alternative

Findings for Gender Equality

	Highly Satisfactory (4)	Satisfactory (3)	Unsatisfactory (2)	Highly Unsatisfactory (1)
3.1  Effectively address gender  equality (n=55)	4%	71%	25%	0%

Twenty-nine percent of evaluations (16/55) that reported on effective UNFPA support to gender equality noted that it had been mainstreamed into population dynamics, reproductive health, gender and human rights. Evaluations also noted that UNFPA-supported programs had undertaken several specific, gender-focused activities to strengthen gender equality and women’s rights with evidence of results at output and, sometimes, outcome level. Often this involved the integration of gender equality and women’s rights into national policies, frameworks and laws. Evaluations also reported that UNFPA-supported programs have been effective in improving services responding to gender-based violence (GBV).

3.3.3 Contributing Factors

The combination of gender mainstreaming in key thematic program areas with specific, targeted support to gender equality initiatives in areas such as GBV and empowerment of girls and women was the most frequently cited factor contributing to positive evaluation findings for gender equality in UNFPA evaluation reports. Additional positive factors included:

• UNFPA programs contained a specific gender component and gender equality also remained a cross-cutting issue within other components (16 evaluations); and,
• The ability to target the right group of decision makers and/or put together coalition and technical work groups to address gender equality issues (13 evaluations).

The global thematic evaluations also pointed to similar positive factors for effectively addressing gender equality.^{Footnote 31}

Negative factors hindering achievements in gender equality included:

• Overly ambitious goals and vaguely defined results indicators that made measuring results in gender equality programming difficult (5 evaluations);^{Footnote 32}
• The absence or limited scope of a gender equality strategy in addition to limited budgets and severe fragmentation (5 evaluations); and,
• A lack of qualified staff in terms gender equality expertise (3 evaluations).

3.4 Sustainability

3.4.1 Coverage

The level of coverage for all three sub-criteria with respect to sustainability were rated strong (Figure 9). Sub-criterion 4.1 benefits continuing after program completion and sub-criterion 4.2 sustainability through institutional/community capacity were addressed in 57 and 60 evaluations respectively. Sub-criterion 4.3 on a strengthened enabling environment for development, only just qualified as strong for coverage as it was addressed in only 46 evaluations, the minimum number required to receive this rating.

Figure 9: Number of Evaluations Addressing Sub-criteria for Sustainability

Figure 9: Text Alternative

Sustainability

	Addressed	Not Addressed
4.1 Benefits continuing or likely to continue	57	5
4.2 Programs support sustainability through institutional / community capacity	60	2
4.3 Programs strengthen enabling environment for development	46	16

3.4.2 Key Findings

Sustainability appeared to pose a serious challenge for UNFPA’s development effectiveness. Only 35% of the evaluations (35/57) rated sub-criterion 4.1 on benefits continuing or likely to continue satisfactory or better. This is the fourth lowest ranking sub-criterion among the 19 measuring development effectiveness.

The evaluations reported more positively on institutional/community capacity (sub-criterion 4.2) and enabling environment for development (sub-criterion 4.3) – with 60% (36/60) and 61% (30/46) respectively reporting satisfactory or better (Figure 10).^{Footnote 33}

Figure 10: Findings on Sustainability

Figure 10: Text Alternative

Findings on Sustainability

	Highly Satisfactory (4)	Satisfactory (3)	Unsatisfactory (2)	Highly Unsatisfactory (1)
4.3 Enabling environment for development (n=46)	2%	59%	37%	2%
4.2 Institutional and community Capacity (n=60)	2%	58%	38%	2%
4.1 Benefits continue after program completion (n=57)	2%	33%	58%	7%

The more positive results for sub-criterion 4.2 and 4.3 than for sub-criterion 4.1 may reflect a number of factors relating to UNFPA programming at country level: an assumption on the part of UNFPA’s partners that its support would continue in the longer term and thus presented no particular need to address issues of ownership and sustainability; a strong focus by UNFPA on providing capacity development support to its implementing partners over time; and, UNFPA’s strengthened focus on policy engagement and advocacy which would contribute to positive results regarding the enabling environment. The relatively poor results for sub-criterion 4.1 were cited in relation to the inability of the supported implementing partner and/or national and local government to assume the costs of UNFPA-supported programs in the absence of continuing support. It also reflects the absence of exit strategies in UNFPA-supported interventions.

3.4.3 Contributing Factors

Factors that facilitated the sustainability of UNFPA programming included:

• Well-targeted and well-designed capacity building at various levels for individuals and organizations, including at community level (24 evaluations);
• Strong national government ownership and engagement in programming (20 evaluations); and,
• Working strategically at the national level on policy and the legal framework while bringing different levels of government and civil society together, building capacity at all levels, and providing advice and tools for government and civil society to advocate for and implement policy in UNFPA’s three strategic areas (18 evaluations).

Factors hindering sustainability included:

• Excessive dependence on donor/external support for funding of programs and the non-integration of program costs into national budgets (23 evaluations);
• Misaligned capacity development approaches led to missed opportunities for sustainability. For example, the absence of capacity building plans, not recognizing the needs and abilities of government partners, and not leveraging local expertise (10 evaluations); and,
• High attrition rate of trained health care staff leading to recurring costs of recruitment and training and an inability to institutionalize capacity (10 evaluations).

The global thematic evaluations note that a major threat to the sustainability was the lack of funding after UNFPA support ceases.

3.5 Efficiency

3.5.1 Coverage

Coverage for the three sub-criteria relating to efficiency ranged from moderate for sub-criterion 5.1, to strong for sub-criteria 5.2 and 5.3 (Figure 11).

Figure 11: Number of Evaluations Addressing Sub-criterion for Efficiency

Figure 11: Text Alternative

Efficiency

	Addressed	Not Addressed
5.1 Program activities are cost / resource efficient	44	18
5.2 Implementation and objectives achieved on time	52	10
5.3 Systems and procedures are efficient	59	3

Moderate coverage of sub-criterion 5.1 indicated that the evaluations made some judgement on cost/resource efficiency and/or reference to the unit cost of program inputs or outputs. It did not mean that UNFPA evaluations regularly assessed cost efficiency. While information on overall program cost was available from 61 evaluations, only 44 had an assessment of cost efficiency.

The strong level of coverage for sub-criterion 5.2 on implementation being achieved on time and sub-criterion 5.3 on efficient systems and procedures indicated that these have most likely been systematically included as requirements of the evaluation terms of reference for country program evaluations at UNFPA. This was supported by an analysis of a random sample of 17 terms of reference for UNFPA country program evaluations. Within this sample, “cost-efficiency” was a requirement in eight, resource efficiency in 11 and reporting on timeliness in six.

3.5.2 Key Findings

While coverage is generally strong, only one-third of the evaluations reported satisfactory or better for sub-criteria 5.1 and 5.2 – 30% (13/44) and 35% (18/52) respectively. The availability of data, in addition to time and budget, may have been factors for not having a detailed assessment on cost efficiency in the evaluations.

The findings for the sub-criterion 5.3 on the efficiency of systems and procedures were mixed with 51% satisfactory (30/59) (Figure 12). None of the evaluations reviewed reported highly satisfactory results on efficient systems and procedures (see also Section 3.6.3).

Figure 12: Findings on Efficiency

Figure 12: Text Alternative

Findings on Efficiency

	Highly Satisfactory (4)	Satisfactory (3)	Unsatisfactory (2)	Highly Unsatisfactory (1)
5.3 Systems for program implementation and follow up efficient (n=59)	0%	51%	42%	7%
5.2 Implementation and objectives achieved on time (n=52)	2%	33%	60%	6%
5.1 Program activities cost/resource efficient (n=44)	3%	28%	55%	14%

Overall, the rating received by the three efficiency sub-criteria are the 3rd, 4th and 6th lowest among ratings for the 19 sub-criteria of the development effectiveness review.

3.5.3 Contributing Factors

Different factors contribute to results regarding cost efficiency compared to timeliness and efficient systems and procedures; therefore they are discussed separately in this section.
Positive factors that promoted cost/resource efficiency of programs included:

• Partnerships ensured more “value for money” and/or reduced transaction costs, including joint work with other UN agencies, development partners and NGOs (9 evaluations); and,
• Ongoing assessment and good management of limited resources (8 evaluations).

Negative factors that contributed to reduced cost/resource efficiency included:

• The lack of appropriate data to allow reasonably accurate cost calculations (11 evaluations);^{Footnote 34} and,
• The lack of synergies was linked to a fragmentation of support and of programming (6 evaluations.

There was a close relationship between the reported timeliness of implementation and findings regarding the efficiency of UNFPA’s systems and procedures for program implementation. Positive factors included:

• Joint planning process, partnerships and coordination with counterparts were reported to have contributed positively to the timeliness of project completion (7 evaluations);^{Footnote 35} and,
• The ability of UNFPA to respond immediately to requests without being overly bureaucratic, (specifically in relation to the disbursement of funds), overall administrative efficiency and sound program procedures contributed to timely program implementation (8 evaluations).

Negative factors that hindered the timely implementation of UNFPA programs/projects included:

• Problems with a rigid financial system and delays in releasing funds to implementing partners (31 evaluations);
• Delays in implementation caused by problems in UNFPA’s overall process for administering country programs (21 evaluations). Examples included: long annual planning process (up to four months); delays in signing the annual work plan; and lack of clear work-flow protocols.
• Issues associated with the capacity of both UNFPA and its implementing partners, such as:
  • Limited access to technical specialists in the UNFPA team, mismatch of skills required, under-staffing, staff turnover and delays in recruitment (18 evaluations); and,
  • Capacity of implementing partners to comply with UNFPA norms, including under-spending/over-estimating expenses on project activities and limited logistics capacity of the implementing partner. (5 evaluations)

The global evaluations identified similar strengths with UNFPA’s internal procedures (administration, finance, and procurement) and hindering factors, such delays in implementation, staff capacity (quantity and quality), and fragmented and/or siloed approach.

3.6 Using Evaluation and Monitoring

3.6.1 Coverage

The coverage for all the four sub-criteria related to using evaluation and monitoring is strong – systems and processes for evaluation are effective; systems and processes for monitoring and reporting are effective; result-based management (RBM) systems are effective; and evaluations are used to improve effectiveness (Figure 13). Each of the sub-criteria was addressed by 50 or more evaluations.

Figure 13: Number of Evaluations Addressing Sub-criterion for Using Evaluation and Monitoring

Figure 13: Text Alternative

Evaluation and Monitoring

	Addressed	Not Addressed
6.1 Systems and process for evaluation are effective	50	12
6.2 Systems and processes for monitoring and reporting are effective	61	1
6.3 RBM systems are effective	54	8
6.4 Evaluation used to improve effectiveness	62	0

3.6.2 Key Findings

The findings for using evaluation and monitoring are mixed. On one hand, they reflect a systematic practice of evaluation at country and regional office level as well as a structured approach to the use of evaluation results to improve the design of programs. On the other hand, monitoring and reporting on program results and the use of RBM systems was assessed negatively in the evaluations reviewed.

The results for sub-criterion 6.1 on effective systems and processes for evaluation and sub-criterion 6.4 on the use of evaluations to improve effectiveness are positive (Figure 14). The findings indicated that about three-fourths of the evaluations were satisfactory or better for these two sub-criteria, 78% (39/50) and 73% (45/62) respectively. Results for sub-criteria 6.2 on effective systems and processes for monitoring and reporting on program results and 6.3 on effective RBM systems were predominantly negative – 13% and 22% of the evaluations (8/61 and 12/54 respectively), reported results of satisfactory or better. Of all 19 sub-criteria, these received the lowest proportion of satisfactory or highly satisfactory ratings.

Despite findings related to weak systems and processes for monitoring and reporting and ineffective results-based management systems, findings were positive for effective evaluation systems and process, as well as use of evaluation to improve effectiveness. This reflected the fact that evaluation as a process was well integrated into the country and regional program planning cycle. It indicated that evaluation teams were able to access other primary or secondary data on program results where regular monitoring information was absent or weak.

Nonetheless, the findings reported for sub-criterion 6.2 and 6.3 indicated that UNFPA has faced serious challenges in establishing and strengthening systems for country and regional program monitoring and RBM during the review period. The preliminary review carried out for this study identified a considerable ongoing effort to strengthen the definition of program results and monitoring systems at country and regional level in UNFPA. This included the appointment of monitoring and evaluation officers and focal points; a mid-term review of the 2008-2013 Strategic Plan; an integrated results framework (of the 2014-2017Strategic Plan); and results-based budgeting. These efforts were recent enough that any resulting improvement in results definition and monitoring systems may not be reflected in the findings of the evaluations reviewed.

Figure 14: Findings for Using Evaluation and Monitoring

Figure 14: Text Alternative

Findings on Evaluation and Monitoring

	Highly Satisfactory (4)	Satisfactory (3)	Unsatisfactory (2)	Highly Unsatisfactory (1)
6.4 Use of evaluation to improve effectiveness (n= 62)	65%	8%	3%	24%
6.3 RBM systems are effective (n=54)	7%	15%	57%	20%
6.2 Monitoring and reporting on program results effective (n=61)	3%	10%	75%	11%
6.1 Systems and process for evaluation are effective (n=50)	18%	60%	14%	8%

The results for sub-criterion 6.4 reflected the extent to which UNFPA was preparing management responses for its evaluations and whether these adequately responded to the recommendations, including an action plan and a clear responsibility for its implementation.

This was one of the highest results of the review with 64% of the evaluations reporting highly satisfactory findings. This reflected the fact that formal management responses to evaluations were either complete, included positive responses to recommendations or identified reasons for rejecting recommendations. The management responses also included a plan with assigned responsibilities for implementing the response along with a status report on progress. The review team found management responses for 41 of the 62 evaluations reviewed; 39 of which were judged to be of high quality.^{Footnote 36}

An apparent paradox in the findings for sub-criterion 6.4 was the rather high proportion evaluations (24% - 15/62) reporting results of highly unsatisfactory. This reflected the decision to rate any evaluation for which a management response could not be located in UNFPA’s electronic data base as such. For at least some of these evaluations, a management response may have been prepared but not included in the database at the time of the review.

The UNFPA’s strong support of the use of evaluation can be seen in the 2009 UNFPA evaluation policy which required program managers to prepare management responses to evaluation recommendations and to undertake necessary follow up.^{Footnote 37} Further, the current management response tracking system was established in 2010. In 2012-2013, as noted in Chapter 2, UNFPA invested in a considerable effort to re-organize, re-emphasize and improve evaluation use and learning. Additionally, the current evaluation policy calls for the Executive Director to report regularly to the Executive Board on the use and follow up of evaluations, including implementation of recommendations.^{Footnote 38}

3.6.3 Contributing Factors

Positive factors that contributed to evaluation systems and process being effective, despite weak monitoring and RBM systems, included:

• Country program evaluations (CPE) were conducted at the end of each program cycle (all 57 CPEs);
• Mid-term evaluations and/or strategic reviews were undertaken as part of the country program cycle (24 evaluations); and
• Evaluations were carried out for projects supported by UNFPA as part of the country program (10 evaluations).

Factors that hindered the effectiveness of evaluation systems and processes included:

• More than 50% the evaluations noted weakness in the monitoring and evaluation systems, lack of clearly defined indicators, absence of targets, and the lack of monitoring data that can be used in CPEs; and,
• Confusion of the terms evaluation and monitoring (5 evaluations).

Positive factors that contributed to effective systems and process for monitoring and RBM included:

• Coherent and well-designed results framework with clearly defined inputs, outputs and outcomes and linkages between the various levels of results contributed to stronger monitoring systems (6 evaluations);
• Quarterly meetings between UNFPA and implementing partners to address challenges and ensuring progress remained on track (4 evaluations); and,
• A complete results framework with baseline and targets for most indicators (3 evaluations).

Negative factors reported included a lack of a results culture or institutionalization of RBM (10 evaluations).^{Footnote 39}

All global evaluations underlined weak monitoring and evaluation systems and reporting primarily focusing on activities instead of results achieved. This weakness was also attributed to lack of capacity at staff and implementing partner levels and lack of “results culture” (indicators were either ambitious or not clear enough to measure).

4. Canada’s Engagement with UNFPA

4.1 Introduction

The purpose of this chapter is to assess Canada’s engagement with UNFPA between 2009 and 2014 according to the criteria of relevance, effectiveness and efficiency. The focus is on Canada’s engagement with UNFPA on an institutional level, at the Executive Board, as a member of the Steering Committee of H4+^{Footnote 40} and in certain bilateral contexts. It complements the focus on UNFPA’s development effectiveness above.

As the scope of this review included a period following the amalgamation of the former Canadian International Development Agency (CIDA) and the Department of Foreign Affairs and International Trade (DFAIT) into the Department of Foreign Affairs, Trade and Development (Global Affairs Canada),^{Footnote 41} reference is made both to CIDA and Global Affairs Canada in this chapter.

4.2 Methodology

This assessment relied on a review of documents and interviews with Global Affairs Canada and UNFPA staff and other Executive Board member states. Documents reviewed included CIDA’s Institutional Strategy for Engagement with UNFPA, CIDA’s Multilateral Effectiveness Strategy, the Review of the Effectiveness of CIDA’s Multilateral Delivery Channel as well as instructions and reports from Executive Board meetings.^{Footnote 42}

4.3 Canada’s Relationship with UNFPA

Canada’s engagement with UNFPA occurs through various channels. The primary channel is engagement as a member or observer^{Footnote 43} of the UNFPA Executive Board, which is facilitated by Global Affairs Canada’s United Nations Programme Division in the Global Issues and Development Branch and the Permanent Mission of Canada to the United Nations in New York (PRMNY). Both the United Nations Programme Division and PRMNY participate in the preparatory meetings for the Executive Board organised by UNFPA or the WEOG^{Footnote 44} and in Board meetings themselves. PRMNY facilitates direct interaction with UNFPA HQ on a variety of issues.

Global Affairs Canada contributes to UNFPA to further Canada’s development objectives through: i) long-term institutional support; ii) support for specific UNFPA programming as part of multilateral initiatives (most notably maternal, newborn and child health-related (MNCH) and international humanitarian assistance); and, iii) contributions by Global Affairs Canada bilateral programs to UNFPA projects in specific countries.

Canada has regular contact with UNFPA in the context of its global work to address MNCH. In particular, Canada is one of the main donors and members of the Steering Committee for the multi-agency partnership initiative on maternal and child health, H4+.

As of March 2014, Canada was eleventh on the list of donors of UNFPA’s long-term institutional support. It was sixth in terms of initiative-specific contributions. The listing of the top 20 donors to UNFPA is provided in Appendix K.

4.4 Relevance

This section addresses alignment between UNFPA’s mandate and Canada’s development priorities, including cross-cutting themes, during the period covered.

There was a strong degree of alignment between UNFPA’s mandate and comparative advantage, and Global Affairs Canada’s development priorities related to children and youth – namely, child survival, including maternal health, and ensuring safe and secure futures for children and youth. There was also relevance to Canada’s cross-cutting theme of advancing gender equality, which is a central component of UNFPA’s work. Specific areas of convergence include MNCH, civil registration and vital statistics.

The CIDA Strategy for Engagement with the UNFPA (Appendix H) was used as a framework against which to assess relevance in this chapter. Whereas a revised Strategy had not been prepared post-amalgamation, the three strategic objectives defining Canada’s relationship with UNFPA on the Global Affairs Canada public website remained relevant^{Footnote 45}. They are to:

1. Broaden UNFPA’s efforts to improve maternal health, promote gender equality and support the needs of youth;
2. Work with UNFPA to improve institutional performance, particularly to strengthen capacity at the country level;
3. Support UNFPA to continue its active role in UN Reform to ensure its programs are coherent, coordinated and focused.

Finding 1: The strategic objectives defined by CIDA’s Strategy for Engagement with UNFPA (2011) remained relevant to Canada’s development priorities; however, Canada’s support for UNFPA has focused on certain areas within the Strategy.

Evidence indicated that the strategic objectives defined in the CIDA Strategy for Engagement with UNFPA remained relevant to Global Affairs Canada’s work at the Executive Board, in multi-donor initiatives, bilaterally and in emergencies. 

Since the G8 Muskoka Initiative on Maternal, Newborn and Child Health in 2010, there was a significant increase in Global Affairs Canada’s funding for UNFPA’s activities promoting maternal health. This was in direct alignment with UNFPA’s mandate; however, the strategic objectives defined above have been perceived narrowly as funding has been focused on maternal health to the exclusion of other areas listed in the strategy, with the exception of humanitarian funding for UNFPA’s work in conflict or post-conflict settings like Iraq and South Sudan.

UNFPA used CIDA’s Strategy for Engagement as a “road map” in order to build their own strategic approach to Canada as a donor. Their experience has shown that, despite the inclusion of a range of programming areas in the strategy and their understanding of Canada’s focus on MNCH, UNFPA’s interest to undertake programming with Canadian support in the area of sexual and reproductive health or adolescents – even adolescent mothers – did not receive much traction. Given UNFPA’s strong focus in these areas of work, the limited engagement by Canada on these issues was a challenge to the relationship during the evaluation period.

Finding 2: Canada’s work at the UNFPA Executive Board consistently focused on management effectiveness and gender equality concerns over the past five years.

Evidence showed that Canada’s interventions at the Executive Board prioritised issues related to managing for effectiveness (results based management and oversight functions, such as audit and evaluation), gender equality concerns and UN reform and harmonisation since 2009. Canada played a strong lead role in work to influence UNFPA’s revised Evaluation Policy (2013), Strategic Plan 2014–2017 and the recent audit of Global and Regional Programmes.

This was in keeping with the finding from the Review of the Effectiveness of CIDA’s Multilateral Delivery Channel, which found that “CIDA presents a very consistent Canadian position, continuously advocating for a strong commitment to managing for effectiveness” and was corroborated by the Report of the Auditor General of Canada, ^{Footnote 46} which stated that they had “identified specific areas where Canada has had a positive influence…particularly in the areas of promoting gender equality, implementing results-based management, and strengthen evaluation functions.” 

Finding 3: There was a lack of performance measurement framework with which to monitor and report on Global Affairs Canada’s strategic engagement with UNFPA.

There has been an absence of a performance measurement framework defining Canada’s engagement with UNFPA. The predictability of Executive Board meetings and advance agendas allows for the definition of expected results over a 3 year period. Without this, it is challenging to assess the effectiveness of Canada’s work at the UNFPA Executive Board.

Procedures were followed for the approval of long-term institutional support and initiative-specific funding in Global Issues and Development Branch. This included due diligence and fiduciary risk assessments, initiative assessment at the branch level, as well as gender equality and environment assessments. There was also some reporting on results included in funding proposals and the Departmental Performance Report. What lacked, however, was the means by which to measure the progress towards achievement of stated objectives.

4.5 Effectiveness

This section includes a broad analysis of the work done by Canada in each of the strategic objectives identified in its Strategy for Engagement. It does not attempt to document all of the results achieved by either Canada or UNFPA in these areas.

Finding 4: Canada’s bilateral support for “UNFPA’s efforts to improve maternal health, promote gender equality and support the needs of youth”^{Footnote 47} centred on support for maternal health, gender equality promotion and, more recently, emergency programming.

There was recognition of Canada’s support for maternal health since 2010. Canada’s leadership role in the G8 Muskoka Initiative and the Commission on Accountability and Information were often cited in interviews. The H4+ Partnership, for which UNFPA housed the Secretariat, was seen as a way of achieving results in MNCH while also promoting coordination, coherence and efficiency among UN agencies.

Canada was a strong advocate for gender equality in its work with the UNFPA Executive Board. This perspective influenced Canada’s interventions on the UNFPA Strategic Plan, country program plans and other strategic documents over the past few years.

Though there is a reference in the Strategy to “work with UNFPA to improve education and promote health for youth [including] through…HIV/AIDS prevention, care and treatment”, there was a shift away from support for HIV/AIDS-related work towards health system strengthening as defined in the Muskoka Initiative.

There was little UNFPA programming specifically for youth supported by Canada over the past 5 years. Further, though UNFPA’s work in sexual and reproductive health was cited in grant agreements for long-term institutional support from 2009 to 2014, Canada provided little funding to UNFPA in this area. There was also limited evidence of Global Affairs Canada efforts to “encourage UNFPA to increase its efforts on reducing female genital mutilation and cutting” nor to support its joint programming with UNICEF on the issue.

Canada began to fund UNFPA’s work in emergencies and post-conflict settings through humanitarian programming but this is quite recent.

Finding 5: Canada was widely recognised for its effectiveness at the UNFPA Executive Board.^{Footnote 48}

Canada was seen by both UNFPA and other member states as very active and effective at the Executive Board. Canadian delegates to the Board brought strong technical knowledge, were well-prepared and made strategic interventions. Their leadership and negotiation skills on tough issues and pragmatism were cited numerous times. Canada was able to reconcile different points of view in a constructive and practical manner. They were also seen as a trusted partner in their work to strengthen UNFPA’s effectiveness, particularly at the country level.

A review of delegations at the UNFPA Executive Board meetings since 2006 indicated a decrease in the number of Canadian representatives attending meetings since 2010. However, there continued to be a high degree of effectiveness in Canada’s work at the Board.

Finding 6: Canada showed its support for UNFPA in UN reform^{Footnote 49} through its work on the Executive Board and as a key supporter of the H4+ Partnership.

Canada supported UNFPA in its role as co-Chair of the United Nations’ Development Group and their efforts to improve the harmonisation of business practices and funding of operational activities with other UN agencies as part of the Quadrennial Comprehensive Policy Review.

Canada was a strong supporter of the Joint Partnership of UN Agencies Working Together to Improve Women’s and Children’s Health, which began in 2010 as a joint effort by UNFPA, UNAIDS, UNICEF, UN Women, WHO, the World Bank, governments and civil society organisations in 36 countries with a high burden of reproductive, maternal, newborn and child mortality and morbidity. Canada was one of the key donors and actively supported UNFPA in its role as the Secretariat.

As there was no performance measure articulated for this strategic objective it is not possible to determine the potential change brought about by this support.

Finding 7: Global Affairs Canada assessed UNFPA to be a strong advocate for gender equality, which was corroborated by the Development Effectiveness Review of UNFPA.

UNFPA was considered by Global Affairs Canada to be a strong advocate for gender equality. This was corroborated by the review of development effectiveness findings (Chapter 3).

4.6 Efficiency

This review looked at efficiency with a focus on the practices and systems used to manage the relationship, such as strategic planning, monitoring and communication. This was mainly because Canada’s primary engagement with UNFPA is through the UNFPA Executive Board, which was a difficult relationship to assess using a cost-based, input-output approach.

Finding 8: Canada’s interactions with UNFPA were generally characterised by high levels of responsiveness, timeliness and openness. However, on some occasions, UNFPA perceived a lack of clarity on the status of funding requests.

Numerous examples were cited where Canadian delegates to the UNFPA Executive Board were highly responsive, transparent and efficient in their communications and interactions in Board-related fora. The interactions at the Board were seen to be strategic and clear with Canada’s position on all major issues being known to all (member states and UNFPA staff alike). There were also areas where Canada worked efficiently with UNFPA on specific initiatives, such as the H4+. This was in contrast to UNFPA’s interactions with Global Affairs Canada at headquarters, where UNFPA has felt that there has been a lack of communication regarding the status of funding requests during the evaluation period.^{Footnote 50}

Finding 9: There was a mixed level of awareness of the Department’s Strategy for Engagement with UNFPA and an absence of systems to facilitate linkages between Canada’s work at the Executive Board and in other parts of the Department.

Evidence indicated that many staff at Global Affairs Canada and UNFPA were unaware of the CIDA Strategy for Engagement with UNFPA. This was also the case in the Review of the Effectiveness of CIDA’s Multilateral Delivery Channel, which reported that “the staff of multilateral organisations interviewed…often reported that they were not aware of a specific CIDA plan or strategy relating to how their own organisations should evolve over time.”

There was also a lack of awareness or understanding on the part of bilateral programs of certain decisions taken at the Executive Board. Within Global Affairs Canada, the differences in focus (bilateral vs. multilateral) and location (field vs. headquarters vs. New York) made collaboration challenging.

There were some regular points of contact and consultation. For example, a standing agenda item at the UNFPA Executive Board meetings was the review and approval of new UNFPA Country Program Documents. When these related to one of Global Affairs Canada’s countries of focus, there was a particular effort made to solicit comments from Global Affairs Canada staff at headquarters who ensured that perspectives from the field were included. However, it was acknowledged that while information was shared with staff in the field, there was not always time to respond to it.

Regular consultations were undertaken within Global Affairs Canada on specific issues – particularly related to financial issues, evaluation, audit, gender equality and results-based management, all of which are standing items on Executive Board agendas. Additionally, a mailing list “Friends of UNFPA” (comprised of all known programs within Global Affairs Canada who have programming or regular interactions with UNFPA) was used by the United Nations division to share Executive Board reports and other relevant information.

While consultation and collaboration was occurring, there was an opportunity to improve regular or systematic communication between the multilateral and bilateral programs around the work at the Executive Board.

5. Conclusions

Conclusions on the development effectiveness of UNFPA

• UNFPA programs were highly relevant to the needs of target group members and UNFPA was successful in developing effective partnerships with partner government and NGOs. They were very successful in aligning their programs with national development goals, plans and priorities of the countries. The relevance of UNFPA programming was enhanced by assessment of the needs of target groups and situation analysis of conditions in country.

• UNFPA was effective in achieving the development objectives of its programs and in contributing to significant changes in national development policies and programs. UNFPA programs produced positive benefits for target group members. Their ability to identify and engage with key policy actors in the development of strategic policies/frameworks in, for example, family planning, gender-based violence, and sexual and reproductive health, facilitated the achievement of objectives. This was also supported by UNFPA’s role in building coalitions and advocating for the targeting of vulnerable populations. When program objectives were not achieved this often related to: weak project design; fragmentation of UNFPA support; and, weaknesses in the technical capacity of UNFPA staff sometimes associated with high rates of attrition and staff turnover.

• UNFPA’s performance with respect to gender equality was highly effective. A key contributing factor is the fact that UNFPA programs not only contain specific gender components but also mainstream gender equality into priority program areas, including in reproductive health and population dynamics. Results achieved in gender equality often took the form of integration of gender equality and women’s rights into national policies, frameworks and laws, as well as supporting efforts to respond to gender-based violence.

• UNFPA achieved mixed results for sustainability. While the majority of evaluations reported that UNPFA programs made a contribution to strengthening institutional/community capacity and the enabling environment for development the likely continuation of benefits after program/project completion was not strong. This poses a challenge for UNFPA’s development effectiveness. A key factor explaining these findings was the continued reliance by government and partners on external program funding, possibly reflecting either the inability or lack of interest of the government to financially support the program. The non- integration of program costs into national budgets compounded by a high rate of attrition among staff trained with UNFPA support contributed to poor sustainability benefits.

• The results for efficiency of UNFPA programming were mixed. The negative results for the cost-efficiency of programs and timeliness of program implementation was due, in part, to the absence of appropriate and timely cost data gathered by programs to allow reasonable efficiency calculations or monitoring. Cost/resource efficiency was also hindered by program fragmentation across too many sub-activities or geographic locations and failure to realize opportunities for synergy. The timeliness of implementation was affected by administrative processes or delays in the release of funds. However, the results indicated that systems and procedures for program implementation and follow-up were adequate.

• Evaluation was systematic and steadily improving, but results-based management was weak. The use of evaluations to improve program design was supported by the preparation of management responses to evaluation, which included action plans, assigned responsibility and included measures for reporting on progress. In contrast, weaknesses in the design of results frameworks, especially in the development of appropriate indicators and baseline information, often undermined UNFPA’s ability to effectively monitor and report on results. Reporting was focused more on compliance and activity completion rather than on progress in achieving results. UNFPA’s recent considerable effort to strengthen results definition and reporting at Country Office level should bring positive changes in these areas.

Conclusions on Canada’s Engagement with UNFPA

• Canada continued to be perceived as very effective by UNFPA and other members of the UNFPA Executive Board.  Its willingness to take a leadership role on challenging issues at the Executive Board led to high levels of influence.

• Canada’s focus on management effectiveness at the Executive Board was particularly relevant given the development effectiveness findings that UNFPA’s evaluation and results-based management are improving but require continued efforts.

• The CIDA Strategy for Engagement with UNFPA has not been revised since it was prepared in 2011.

• The lack of a performance measurement framework made assessment of the effectiveness of Canada’s engagement with UNFPA challenging. Systematic planning for and feedback loops from Executive Board meetings would strengthen Canada’s effectiveness and efficiency.

6. Recommendations for Global Affairs Canada’s Engagement with UNFPA

1. Update Canada’s Strategy for Engagement with UNFPA to include: a) a theory of change; b) clearly articulated alignment between UNFPA’s strategic results framework and Global Affairs Canada’s development priorities; c) articulation of specific, concrete change in organizational behavior/performance/practice that Global Affairs Canada would like to see from UNFPA; and, d) a performance measurement framework that captures all engagement with UNFPA across the Department.

   The use of ‘theories of change’ to clarify intended and expected results is a common practice. It is particularly important when the changes that are being sought are behavioural, such as those at the Executive Board. Mapping out how one intends to influence others to bring about an outcome is a critical tool for planning and accountability.

   The institutional strategy for engagement with UNFPA should make a clear distinction made between those results for which UNFPA will be primarily responsible and those to which the Department will contribute. A performance measurement framework should be broad enough to capture all elements of Canada’s engagement with UNFPA.

2. Develop a communications strategy for Canada’s engagement with UNFPA.

   Given that engagement with UNFPA takes many forms, it is important for there to be clear and consistent messaging around Global Affairs Canada’s interactions with UNFPA. This includes in its communications around what Canada’s objectives and priorities are and how Global Affairs Canada implements them.

3. Develop internal mechanisms for information sharing between multilateral and bilateral branches in order to improve Canada’s effectiveness at the Executive Board.

   Communication leading up to and following Executive Board meetings has taken place over the past few years, but, particularly in the context of the amalgamation of the former CIDA and DFAIT, there is a need for a broader and more systematic exchange of information between the United Nations Programme division and other parts of the Department that engage with UNFPA.