Caribbean Regional Program Evaluation

List of Abbreviations

Executive Summary

This document is a Synthesis Report of the Caribbean Regional Program Evaluation from 2006-2007 to 2010-2011 that presents the analysis and findings of an evaluation that included field visits to the Caribbean in October-November 2011. The evaluation supports accountability and offers knowledge to improve performance for the future.

The CIDA Caribbean Program

The Caribbean Program of Foreign Affairs, Trade and Development Canada (DFATD)^{Footnote 1} serves 14 countries: eleven island states and three continental countries.^{Footnote 2} The program is a CIDA region of focus ^{Footnote 3} delivering on the Government of Canada’s 2007 commitment to provide $600 million in additional development assistance to the Caribbean^{Footnote 4}.

The program is currently guided by the Caribbean Strategy approved by the Minister in October 2009, and the Caribbean Development Programming Framework 2010-2015. The program’s objective is to contribute to a more prosperous and integrated Caribbean Community able to generate sustainable economic growth, providing opportunity and security to its citizens. The program focuses on sustainable economic growth and on security (including justice and disaster management). Regional integration, as well as gender equality and environmental sustainability, are crosscutting themes.

Evaluation Methodology, Challenges, and Limitations

The evaluation assesses the Caribbean Program’s results and performance. Given the challenges of assessing investments in a geographically dispersed region, the evaluation focuses on the Geographic Programs Branch or “bilateral” channel. However, short case studies were also undertaken to provide insight into CIDA’s support of the region through the Multilateral and Global Programs Branch and the Partnerships with Canadians Branch.

The evaluation follows the terms of reference and approach developed by CIDA for country program evaluations. The initial document review and desk study were supplemented by interviews in Canada and a field visit to meet a range of stakeholders in five Caribbean countries.

The evaluation reviewed a sample of thirty projects covering the program’s key programming areas and delivery modalities, representing 80.6% of the program’s total disbursements of $194.7 million during the five-year period.

Eight criteria were considered: relevance, effectiveness, sustainability, crosscutting themes (gender equality and environmental sustainability), coherence, efficiency, management principles, and performance management. Scoring of projects was done both by numeric (quantitative) ratings and by nominal (qualitative) ratings. In each case, a five-step rating scale was used, ranging from Highly Unsatisfactory to Highly Satisfactory. The program-level assessment used only nominal ratings. Program-level ratings are distinct from project scores. The evaluation assessed the program across all eight criteria, using different performance indicators than those used for projects. Scores for criteria may therefore differ at the program and project-level. Where significant differences emerge, they are investigated and explained.

A number of challenges were encountered that created limitations on the analysis. The difficult and protracted process of collecting key documents due to the program’s geographic dispersion impaired the deskwork in advance of the field mission. In some cases, limited project-level results reporting and limited previous evaluations impeded the measurement of results from secondary sources. Changes in the policy and programming framework over the reference period complicated the assessment of projects against the program objectives at the time they were designed and implemented. Data availability and reliability for small states with limited statistical capacity were also challenges. Appropriate care is taken in drawing inferences at the program level based on the project sample.

Analysis of Findings ^{Footnote 5}

During the five-year period under review the program disbursed a total of $194.7 million: 43% in democratic governance (including security and rule of law); 22% in sustainable economic growth (including in economic renewal and private sector development); 18% in human capital formation; 14% for projects focused on environmental sustainability (including disaster preparedness and disaster risk management); and 0.5 % for gender equality focused projects.^{Footnote 6}

Relevance – Satisfactory. The Caribbean programming framework of 2007^{Footnote 7} reflected consultations and analysis on priorities in the region. There are common strands, but a narrower focus, with the Program’s Caribbean Strategy 2009 and with the Caribbean Development Programming Framework 2010-2015. The removal of explicit reference to regional integration and institutional strengthening outcomes in the program’s expected results affects the program’s ability to clearly state, monitor and report on expected results in line with the emphasis of CIDA’s Sustainable Economic Growth Strategy on integration into regional and global markets, and with regional priorities expressed by Caribbean leaders.

Effectiveness – Satisfactory. The program has had good results in strengthening capacity in economic and financial management, trade and development initiatives, and environment and disaster risk management. Contribution to gender equality has been strong. Several projects have had good results in reducing vulnerabilities.

Sustainability – Moderately satisfactory. One of the challenges has been to ensure that the recipient institutions take actions to sustain the results achieved. Institutional weaknesses are more acute with many small Caribbean states. Yet, there have been identifiable and valuable contributions in terms of institutional strengthening of key regional organizations. Where the program has worked with regional institutions and partnerships, sustainability was increased because the network remained even when there was staff turnover.

Gender equality (crosscutting theme) – Moderately satisfactory. The degree of success for integrating gender equality as a crosscutting theme tended to be uneven and depended on the level of importance placed on it by different program staff at different times.

Environmental sustainability (crosscutting theme) – Unsatisfactory. The quality of analysis was not sufficient for integrating environmental sustainability into program components. program level reporting is not generally available. The budget appears inadequate. Environmental considerations have been identified, but not adequately addressed.

Coherence – Moderately satisfactory. Policy dialogue with beneficiaries and other donors has been reasonably effective, but there is room for improvement. While the program’s resources have increased, senior officials responsible for aid coordination at both the national and regional levels are less aware of substantial Canadian interventions, due to the program’s shift to a more regional approach and emphasis during the evaluation period on channelling funds through institutions and organisations that act at the regional level. The details of Canada’s 2007 commitment for an additional $600 million are not fully visible or understood in the region.

Efficiency – Moderately satisfactory. In addition to changes in programming strategy, there was also the introduction of new reporting mechanisms in the Agency, and the process of decentralization of the program. These factors interacted in ways that could not be fully foreseen, and affected efficiency.

Management Principles – Moderately satisfactory. There are good examples of consistency with the principles of alignment, ownership, harmonisation, and/or aid coordination in projects that have a joint governance structure. However, the greater emphasis on working through multilateral institutions has created challenges for interaction with stakeholders at the national level and the degree of ownership and alignment.

Performance Management – Unsatisfactory. Challenges arise from several sources. The introduction of CIDA’s Investment Monitoring Reporting Tool could have been done with greater guidance and support for the program’s staff. Reporting on results for bilateral initiatives executed through international financial institutions can also be challenging. Many of these development partners have mechanisms for measuring and reporting results and some are willing to make further accommodations, if CIDA were to engage more actively on these issues.

Recommendations

The evaluation formulated several conclusions (see Chapter 4) and eight recommendations, seven addressed to the program and one at Branch level (see Annex 18 Management Response). It is recommended to:

1. Retain and strengthen capacity building in selected areas of governance and regional integration at regional and subregional levels with complementary national initiatives as a comparative advantage of the program.
2. Strengthen the program’s capacity and effectiveness in supporting environmental sustainability, in addition to current focus on disaster risk management, as a critical element in reducing the region’s high vulnerability and increasing resilience. Increased professional resources in the field would be helpful in identifying strategic opportunities and to strengthen environment as a crosscutting theme.
3. Seek to capitalize on Canada’s comparative advantage and leadership in gender equality in the region by rebuilding and strengthening the program’s gender equality network and role. Gender equality as a crosscutting theme should receive higher priority at the project and program levels.
4. Review expected outcomes and indicators to clearly communicate and track the program’s performance on regional integration.
5. Provide for increased flexibility for selected national and subregional projects which may contribute to the program’s core emphasis on regional programming.
6. Strengthen performance management through: (a) improved training of personnel on results management and reporting to improve the quality of Investment Management Reporting Tool reporting; (b) revised indicators in the program’s performance measurement framework in accordance with new Agency guidance to better measure results from priority sectors and crosscutting themes; and (c) strengthened evaluation focused on results at the project, sector and thematic levels.
7. Ensure there is a clear whole-of-Agency communications plan that reports on progress to date and outlines the remaining elements for the Government of Canada’s 2007 commitment of $600 million in additional development assistance to the Caribbean region.
8. Branch-level Recommendation: It is recommended that the Geographic Programs Branch clarify the approval requirements and provide guidance on the development and adjustment of country/regional strategies and Country Development Programming Frameworks, including the review and revision of logic models.

Introduction

The Caribbean is the only regional program that is part of the Agency’s 20 countries of focus. CIDA’s Caribbean Program serves eleven island states and three continental ones (that is all the full members of the Caribbean Community except Haiti, which is a separate CIDA Program and therefore not included in this evaluation).^{Footnote 8} All these countries share extreme environmental vulnerability. There are great economic disparities in the Caribbean region, which includes some of the poorest economies in the hemisphere. Violence is also a major concern. Caribbean regional integration has faced challenges on several fronts: political, trade disputes, and macro-economic policy coordination. There are many small jurisdictions and the institutional mandate for regional/subregional organizations is often limited. The most significant institutions are the Caribbean Community Secretariat (CARICOM) at the regional level and the Organisation for Eastern Caribbean States (OECS) at the subregional level.

In 2007, under the Caribbean Community Strategic Programming Framework 2007-2008 to 2017-2018, the objective of CIDA’s Caribbean Program was “to strategically address root causes of vulnerability and institutional weakness by supporting the region as a whole so that it can be more self-reliant in ten years”. There were three overall themes: regional democratic governance; regional economic renewal; and regional human capital formation. Gender equality and environmental sustainability were the crosscutting themes.

The program is currently guided by the Caribbean Strategy approved by the Minister in October 2009, and the Caribbean Development Programming Framework (CDPF) 2010-2015. The objective of the CDPF is “to contribute to a more prosperous and integrated Caribbean Community able to generate sustainable economic growth, providing opportunity and security to its female and male citizens”. The program is currently focusing on sustainable economic growth and on security (including justice and disaster management). Regional integration, as well as gender equality and environmental sustainability, are crosscutting themes.

The program also has the responsibility to deliver on the Government of Canada’s 2007 commitment to provide $600 million in additional development assistance to the Caribbean.^{Footnote 9}

During the five-year period under review (fiscal years 2006-2007 to 2010-2011) the program disbursed a total of $194.7 million, in the following main areas: 43% in democratic governance (including security and rule of law); 22% sustainable economic growth (including in economic renewal and private sector development); 18% in human capital formation; 14% for projects specifically focused on environmental sustainability (including disaster preparedness and disaster risk management); and 0.5 % for projects specifically focused on gender equality.

This document is a Synthesis Report of CIDA’s Caribbean Regional Program Evaluation. It is based on the findings of an evaluation that included field visits to the Caribbean in October-November 2011. Conclusions and recommendations presented in this report are therefore based on findings and evidence gathered in 2011, and may not reflect changes since that time.

The evaluation supports accountability and learning through three objectives:

• To take stock of the results achieved by the program over five fiscal years (2006-2007 to 2010-2011).
• To assess the program’s overall performance in achieving these results.
• To document and disseminate findings and lessons learned, and formulate recommendations to improve performance.

The evaluation assessed the Caribbean Program as a whole and a sample of 30 projects (see Annex 8) that represented more than 80% of total program disbursements during the five-year period under review (2006-2007 to 2010-2011). The evaluation used the eight criteria^{Footnote 10} listed below. The projects were given numeric and nominal scores based on a five-point scale ^{Footnote 11}, whereas the overall program was rated using only the five-point nominal scale.

The evaluation assessed the program on its own merit across all eight criteria using different performance indicators than those used for projects (see Annex 7). Scores for criteria may differ at the program and project-level (see Chapters 1.0 and 2.0, and Annexes 9 and 10).

What has been accomplished (Development Results).

• Relevance: Extent to which the program’s objectives reflect the Caribbean region’s needs and CIDA’s policies and strategies
• Effectiveness/results: Extent to which the program’s objectives have been achieved, or are being achieved, considering their relative importance
• Sustainability: Maintenance of the development program’s benefits after a major development cooperation activity is completed
• Crosscutting themes: Extent to which gender equality and the environment are integrated into the program framework

Why and how were the expected results achieved (Management Factors).

• Coherence: External coherence (complementarity and use of synergy with the activities of other donor countries). Internal coherence (Coordination among CIDA’s various delivery mechanisms) and among Canadian development stakeholders, including governmental, non-governmental and private sector stakeholders.
• Efficiency: Extent to which resources (funds, expertise, time, and so on) are translated into results, based on optimum allocation of resources.
• Management principles: Degree of adherence to the following principles of the Paris Declaration on Aid Effectiveness: local ownership, alignment, and harmonization.
• Performance management: Integration of results-based management and the principle of mutual accountability in program management, including mechanisms and tools developed and used.

The first two sections of this report offer analysis based on the evaluation criteria. Section three provides a summary of findings by programming areas and the last section proposes conclusions, lessons and recommendations.

1.0 Major Findings: Development Results

1.1 Relevance

Analysis at the Program Level

At the program level, the overall rating was satisfactory (see Annex 9). Changes in the programming frameworks for the Caribbean, during the period under review, complicated the assessment. The evaluation attempted to assess relevance, including continued relevance in terms of changing circumstances, against the CIDA strategies and regional priorities that were in place when the frameworks were designed.

During the 2006 to 2011 evaluation period, the program had three distinct programming frameworks. The Caribbean Community Strategic Programming Framework 2007-2018 turned out to have a relatively short life span. In 2009 CIDA’s 20 countries of focus in 2009 were announced along with changes in thematic approaches as part of CIDA’s Aid Effectiveness Agenda. A rethink of the Caribbean strategy through a series of iterations from 2008 to 2010 led to the Caribbean Program Strategy 1.0 approved by the Minister in May 2009, followed by the shorter updated Caribbean Strategy 2.0 approved in October 2009, as part of CIDA’s general submission of strategies for the 20 countries of focus. The longer and more detailed Caribbean Development Programming Framework 2010-2015 was approved by the President of CIDA in December 2009.

There is a degree of consistency in the frameworks as they evolved, and a number of common strands, although with differences in emphasis. These include, in varying degrees, governance or institutional development, economic renewal or sustainable economic development, human resource development or human capital formation, environment, gender equality, and security or rule of law. The program was broadly consistent with CIDA thematic priorities, such as sustainable economic growth, and the Millennium Development Goals.

The Caribbean Community Strategic Programming Framework 2007-2018 reflected consultations and analysis on priorities in the region. There are common strands, but a narrower focus, in the Caribbean Strategy 2009 and the Caribbean Development Programming Framework 2010-2015. For example, in December 2010, the intermediate outcome statement that referred to an "improved enabling environment to increase trade and integration” was changed to “increased trade and economic activities" with no reference to the enabling environment or to regional integration. The immediate outcome of "increased capacity of Caribbean regional institutions to support Member States in achieving regional integration" was replaced by increased assistance to vocational education. The output related to trade negotiations was also removed. The changes seem to contradict the program’s expressed focus on regional integration and institutional capacity building. As well, CIDA’s Sustainable Economic Growth Strategy emphasizes integration into regional and global markets.

The program has benefited from strong relationships and input from regional partners, in the areas of gender equality and governance. However, the program’s shift to regional programming, as distinct from programming in individual countries, poses challenges in identifying interlocutors who can implement projects across the region.

Analysis of the Project Sample

60% of the projects in the sample were rated as highly satisfactory in terms of relevance, 36.7% as satisfactory, and 3.3% as unsatisfactory (see Annex 11, Figure 1). The ratings reflect a good correlation for many of the projects with regional and country needs. Individual project scores ranged from 2.0 (unsatisfactory) to 4.5 (highly satisfactory) – see Annex 10; the average numeric rating was 4.1 or highly satisfactory (see Annex 9-B).

Of particular relevance were environmental sustainability projects that supported poverty reduction through disaster risk management and capacity building for the management of OECS countries. Vulnerability reduction is essential for development in the Caribbean islands and low-lying coastal countries as they are among the most susceptible in the world to natural disasters and face disproportionately high economic, social, and environmental consequences due to their small size and isolation.

Good performing projects were the Caribbean Regional Technical Assistance Centre, the Caribbean Regional Negotiating Machinery, the Basic Needs Trust Fund, the CARICOM Trade and Competitiveness Project, and the Eastern Caribbean Education Reform Project. These were projects with a strong regional or subregional focus, expected to benefit a number of countries, and well targeted in terms of regional needs and priorities. They were also responsive to earlier CIDA strategies and the new Sustainable Economic Growth Strategy, and in some cases, to the Millennium Development Goals (MDGs) and the Caribbean-specific MDG targets.

1.2 Effectiveness

Analysis at the Program Level

At the program level, the overall rating was satisfactory (see Annex 9). Strengthening democratic governance at national and regional levels was the program’s principal priority at the beginning of the period, and reflected priorities over the previous decade. It was in turn reflected in the project portfolio. The Caribbean Strategy approved by the Minister in 2009, although structured differently and more focused, continued to emphasize some of the same objectives. These included stronger public and regional institutions that can effectively manage public finance and debt, strengthen human resources, and advance the rule of law.

The program’s use of targeted country-specific programming in the earlier part of the evaluation period, in addition to regional projects, was an important factor that contributed to outcomes to strengthen national capacity for participating in regional activities, and to address issues of economic and public sector management, law and security, and the conditions for sustainable economic growth. Careful monitoring and flexibility in adapting to changing circumstances and opportunities also contributed to the effectiveness of the program’s support for institutional strengthening of regional/subregional institutions.

The program’s shift to regional programming, which started around mid-2008, limited the scope for complementary subregional and national initiatives, and has meant that some successful national and subregional initiatives were not pursued. As a result, some projects planned for a second phase were not renewed. At the same time, the shift to regional programming led to a portfolio shift, at least in the initial stages, toward projects implemented by international financial institutions, which could offer and undertake regional programs. The United Kingdom’s Department for International Development (DFID) is the other donor, along with Canada, that has moved the most in this direction in the Caribbean, but DFID still retains the flexibility to have national or subregional projects. Other major donors have complementary programs with both regional support and national development strategies.

In terms of capacity development, the program has achieved valuable results at regional and national levels, with a good balance between those two levels. The program’s role at the governmental, non-governmental and community levels has also been significant, although the extent to which community-level capacity development will continue under the present programming framework is not clear.

The program has had some good results in strengthening governance capacity in economic and financial management, trade and development initiatives, as well as environment and disaster risk management. The program’s contribution to gender equality has been strong. Environment-specific projects, the Basic Needs Trust Fund, and regional trade projects have had good results in reducing vulnerabilities.

In several areas, the program’s results and influence have been significant. This includes Program-Based Approach type interventions such as the Caribbean Regional Technical Assistance Centre and the Basic Needs Trust Fund.

Analysis of the Project Sample

30% of the projects in the sample were rated as highly satisfactory in terms of effectiveness, 40% as satisfactory, 16.7% as moderately satisfactory, 10% as unsatisfactory, and 3.3% as highly unsatisfactory (see Annex 11, Figure 2). Individual project scores ranged from 0.5 (highly unsatisfactory) to 4.5 (highly satisfactory) – see Annex 10; the average numeric rating was 3.4 or satisfactory (see Annex 9-B).

The dispersion of the ratings for effectiveness depends on a number of factors, including the clarity of the logic models, the precision and suitability of the performance indicators, and the processes in place in the implementing entities and within CIDA to monitor and measure progress in achieving results.

There were issues in the measurement of effectiveness both with development partners and within CIDA, due to weaknesses in the specification of performance indicators, which were in some cases relatively general or lacking precision, and due to inadequate results reporting. It made effectiveness more difficult to assess. In some cases, the evaluation was able to obtain, from other sources, evidence about results that was not captured in CIDA’s results reporting.

In the area of Democratic Governance, twelve (12) projects were examined. The factors that contributed to the achievement of results were strong relevance to the needs of the region, support for regional/subregional cooperation, a high degree of beneficiary ownership, and a mechanism for both accountability and responsiveness to changing priorities. The three successive phases of the Caribbean Regional Technical Assistance Centre, jointly funded by several donors including CIDA, and the Eastern Caribbean Economic Management Program, demonstrated good results in terms of building capacity in selected areas of economic and financial management. Through the adoption of innovative approaches, the Democracy and Governance Fund for Jamaica mobilized experienced partner organizations, raising public awareness and yielding positive response from the community. The project contributed to building capacity of civil society, training institutions, and government agencies at relatively low cost, while raising CIDA’s visibility. There were two projects related to the justice sector that were less effective due to problems with the Canadian executing agencies.

• The Judicial and Legal Reform Project started in 2000 but suffered a very slow start. The Canadian executing agency took two years to formulate the project implementation document and another year for another inception mission. Mismanagement by the Canadian Executing Agency continued through 2004, and by the time a midterm review was conducted, only $1 million of the allocated $4.8 million was left, with little to show for it. CIDA terminated the services of the Canadian executing agency and asked the Eastern Caribbean Supreme Court to take over management of project implementation.
• The Justice Undertakings for Social Transformation project was launched in 2009 when Prime Minister Harper and Prime Minister Golding of Jamaica signed the Memorandum of Understanding. The executing agency, Justice Canada, encountered difficulties relating to the inability, under Canada’s legal and policy framework , of departments that do not have an international mandate to enter into contracts in foreign countries. It was concluded that Justice Canada lacked the necessary authority to carry out aspects of the project. To avoid further delay and risk to Canada’s reputation and leadership in the justice sector, CIDA restructured the project into different components executed by various partners, including the United Nations Development Programme, local civil society organisations, and Justice Canada for the technical assistance portion.

In the area of Economic Growth, eight (8) projects were examined. Although there have been changes in the programming frameworks in this area over the period evaluated, there have been elements of continuity in programming objectives. The programming framework of 2007 focused on Regional Economic Renewal, which had two key expected results: (i) improving the regional enabling environment for trade and competitiveness and strengthening linkage to markets; and (ii) strengthening the adaptation of disadvantaged states and sectors to the region’s single market and economy. In the Caribbean Development Programming Framework 2010-2015, the focus was changed to Sustainable Economic Growth, with two key results: (i) more effective leadership and management of gender-sensitive and accountable public institutions; and (ii) improved enabling environment for business development, trade and integration. The following are examples of results in the economic growth area:

• The Caribbean Regional Negotiating Machinery project and the CARICOM Trade and Competitiveness Project are contributing to the liberalization of trade in goods, services and investment in a region where many of the jurisdictions have had a persistent pattern of protectionism that compounds the economic fragmentation of the economies.
• The Basic Needs Trust Fund, delivered by the Caribbean Development Bank, exhibited good targeting of a range of objectives, achieving results in improved social infrastructure, poverty reduction, reduction of vulnerabilities, environmental sustainability, good governance at the local level, and gender equality.
• The country-specific projects, Building Community Capacity Project in Guyana, Small Farmers' Livelihoods in Guyana, and Improving Jamaica's Agricultural Product, achieved development results with their target groups and communities, and a degree of sustainability through the development of infrastructure and community capacity, as well as building networks and visibility for CIDA.

In the area of Human Capital Formation, three (3) projects were examined. While recent projects have encountered difficulties related to personnel and working with partners, higher performing projects have had distinct characteristics of strong credibility and trusting relationships between CIDA, local communities, partner institutions and other stakeholders. Such relationships were nurtured over a long period of collaboration. As the program moves towards more regional partnerships and working through international institutions, there are likely to be considerable challenges as a result of more limited opportunities to interact directly with partners in different locations.

• Intended as a hub for harmonization, the Eastern Caribbean Education Reform Project (ECERP) was an attempt at a sector-wide approach^{Footnote 12} in education, around which the donors’ activities would orbit. With its subregional focus and objective to improve the quality of education and labour mobility, it had significant impact in the development of a unified education standard in the OECS. Products in curriculum and teaching guides were of high quality.^{Footnote 13} ECERP is also credited with having facilitated educational networks among member states. Currently, the OECS Education Reform Unit sustains and draws on the outputs and outcomes of the project, and has shown itself as a learning organization, taking initiatives to account for results and reflecting on implementation lessons.
• Two projects were rated as unsatisfactory due to start-up delays and the difficulties in launching regional projects. The CARICOM Education for Employment faced challenges in mobilizing Canadian executing agency personnel, while the Canada-Caribbean Leadership Program suffered from weaknesses in design. Both had difficulties in developing partnerships and a work program throughout the region, which reduced their effectiveness.

The evaluation examined Gender Equality as a specific programming area because the portfolio included gender-specific projects that were credited with establishing CIDA as the leader and champion for the promotion of equality between women and men. Other donors looked to CIDA for leadership in guiding their own efforts to promote gender equality. However, the termination of the Gender Fund, following consultations for planned programming resulted in a certain loss of credibility for the Agency and created a void that has yet to be filled. The Canada/Caribbean Gender Equality Fund supported local initiatives in Barbados/OECS, Guyana, and Jamaica, and sought to improve enabling environments for empowering women in economic, social, and political participation, and to reduce violence against women and girls. The projects provided tangible results that gave Canada high visibility, as evidenced by the following examples:

• The University of the West Indies uses work on gender-based violence funded by the Gender Fund in Jamaica, in ongoing work with the Jamaican constabulary in dealing with victims and perpetrators, and with their own academic board on dealing with gender based violence on campus.
• The Convention on the Elimination of Discrimination against Women is now enshrined in the Guyanese constitution through work done by former beneficiaries and partners of the Gender Fund in Guyana. Subsequently, a National Commission for Women and Gender Equality has been established in Guyana, chaired by the women leaders of the ruling and opposition parties. Initiatives to establish pilot Early Childhood Centres have been replicated in every village and resulted in the creation of more jobs for Amerindians.
• A series of model laws on women, children, and family, produced through support by the Gender Fund in Barbados/OECS has been adapted by some of the OECS countries in drafting new family laws.

The evaluation also examined Environmental Sustainability as a specific programming area because the portfolio included environmentally-focused projects. Only one project had an ecological basis, reflecting a change in focus from CIDA interventions in previous development programming. The stronger emphasis on disaster risk management reflected recognition of the Caribbean’s vulnerability to natural hazards, and the critical need for strengthening capacity to reduce disaster risk and to respond effectively to dislocating events. Environmentally-focused projects generally performed well in terms of increasing resilience to disasters. Here are some examples:

• The Caribbean Catastrophic Risk Insurance Fund is the world’s only regional fund utilizing “parametric” insurance, by which CARICOM governments are able to purchase catastrophe insurance coverage that provides immediate liquidity after a major disaster. Because of the speed at which claim payments are processed, the instrument is particularly useful to finance immediate post-disaster recovery, while governments seek funds from other sources for long-term reconstruction.
• The Disaster Preparedness Project has improved capacity to implement comprehensive disaster management in areas of mitigation and preparedness, and increase collaboration in mitigation policy and planning, hazard mapping, vulnerability assessment and safer building education.

1.3 Sustainability

Analysis at the Program Level

At the program level, the overall rating was moderately satisfactory (see Annex 9). A number of projects in the program’s portfolio have been targeted at institutional capacity, with the intention of strengthening institutional sustainability.

In a number of cases, there were challenges for sustainability as a result of institutional weaknesses at the regional or national level, or weaknesses in stakeholder engagement and ownership.

One of the challenges has been to ensure that the recipient institutions take actions, such as providing continued funding or implementing institutional change, to sustain the results achieved. Institutional weaknesses are more acute with small states in the Caribbean region.

Yet there have been identifiable and valuable contributions in terms of institutional strengthening of key regional organizations such as the CARICOM and OECS Secretariats, and the Office for Trade Negotiations. Where the program has worked with regional institutions and partnerships, like the Caribbean Technical Assistance Centre or the CARICOM Secretariat, which have developed effective working networks with the member states and officials at the senior and working levels, sustainability was increased because the network remained even when there was staff turnover.

The Caribbean Program has achieved positive results through supporting the passage of legislation and the reduction of trade barriers in the region. Changes achieved through international agreements and through the Caribbean Single Market and Economy, which the program has supported, are leading gradually to significant policy changes in trade and investment regimes with sustainable economic and development impacts.

Analysis of the Project Sample

13.3% of the projects in the sample were rated as highly satisfactory in terms of sustainability; 46.7% as satisfactory; 30% as moderately satisfactory; 6.7% as unsatisfactory; and 3.3% as highly unsatisfactory (see Annex 11, Figure 3). Individual project scores ranged from 0.5 (highly unsatisfactory) to 4.5 (highly satisfactory) – see Annex 10; the average numeric rating was 3.3 or satisfactory (see Annex 9-B).

In terms of Sustainability, higher ratings for projects tended to be where there was some combination of good targeting, strong ownership and beneficiary input, continuing support mechanisms, good monitoring and accountability, and in some cases jointly supported funding and monitoring.

Capacity building was a focus of many of the projects. Generally, within individual Caribbean states, there tend to be challenges with project sustainability due to capacity inadequacies, and sometimes insufficient engagement with stakeholders or national ownership of the project intervention.

A number of projects, such as the Caribbean Regional Technical Assistance Centre and the Eastern Caribbean Economic Management Program, achieved a measure of sustainability through capacity building with Caribbean member states. The challenge to sustainability with very small governments is limited absorptive capacity and staff turnover.

Support for regional organizations (e.g. the CARICOM, the Caribbean Regional Negotiating Machinery, and the Caribbean Development Bank) has built capacity both at the regional level and at the national level. One of the coping mechanisms for smaller states in managing complex issues with limited time and resources is to rely upon interaction with regional peers and the secretariats of regional organizations through intergovernmental processes to provide the institutional memory, experience sharing and knowledge management, which is difficult to foster in small governments. Both the regional organizations and the national governments are quite fragile in their capacities, but regular interaction tends to build resilience.

Several “legacy” projects that were undertaken under the earlier programming frameworks (e.g. Municipal Governance and Management in Guyana, Small Farmers’ Livelihoods in Guyana, Improving Jamaica’s Agricultural Product, and various projects under the Canada/Caribbean Gender Equality Fund) have achieved a measure of sustainability through interaction with, and capacity building for, community groups and non-governmental organizations. The community groups and non-governmental organizations tend to be fragile, but collectively they have a surprising degree of resilience.

1.4 Crosscutting Theme: Gender Equality

Analysis at the Program Level

At the program level, the rating for gender equality as a crosscutting theme, as distinct from projects specifically targeting gender equality, was moderately satisfactory (see Annex 9). The degree of success for integrating gender equality tended to be uneven and depended on how significant it was viewed by the responsible personnel. Beneficiary participation was adequate where there had been adequate analysis of gender issues during the project design.

Implementation of the programming framework of 2007^{Footnote 14} was expected to address the gender equality and environmental sustainability themes strategically as crosscutting imperatives. The evaluation found that integration of gender equality was successful only when specific performance measures and the necessary resources put in place and accounted for. Because of CIDA’s leadership in gender equality, for example through the program’s previous gender-specific funds, regional institutions have now paid greater attention to gender equality issues and strengthened their capacity in doing so. The Gender Advisory Committee set up at the OECS, and the Gender Advisor position supported at the Caribbean Development Bank, are attributable to CIDA’s work in gender equality.

Gender equality reporting appears to have weakened program-wide in the early implementation of the Caribbean Development Programming Framework for 2010-2015 with CIDA’s new reporting formats. Whether this is a lack of familiarity with the new forms requires further investigation.

Analysis of the Project Sample

30% of the projects in the sample were rated as highly satisfactory in terms of gender equality (as a crosscutting theme), 30% as satisfactory, 23.3% as moderately satisfactory, 3.3% as unsatisfactory, and 13.3% as highly unsatisfactory (see Annex 11, Figure 4). Individual project scores ranged from 0.1 (highly unsatisfactory) to 4.5 (highly satisfactory) – see Annex 10; the average numeric rating was 3.1 or satisfactory (see Annex 9-B).

The results demonstrated by the project sample indicate that even when there was a specific gender equality component within a project, successful integration of gender equality in the overall project depended on whether explicit efforts were made for consistent and systematic integration throughout the project cycle (i.e. design, planning, implementation, and results monitoring).

For example, the CARICOM Capacity Development Project included a specific gender equality component. The project trained 60 teacher educators from 14 countries across the region in techniques for delivering gender training to candidate teachers at early childhood, primary and secondary levels, using a training module on "Gender Issues in the Caribbean" commissioned by CIDA. Improved quality of gender programs was reported in teacher colleges and schools of the region. However, the midterm review of the project found that the project had not been able to succeed in the additional step of integrating gender training into the training programs of teacher training institutions on an ongoing basis through integration into Ministry-approved curricula. Nevertheless, some of the teacher educators who participated in the gender training reported that they continued to use the training module in their own teacher training classes.

A number of projects with more limited success represented how the majority of the sample performed. For example, in the Partnership for CARICOM Private Sector Development there was good effort up front, but the project did not have enough sustained support for those efforts to gather traction. CIDA’s limited influence on the project’s decision making also confined its advocacy leverage. In the Municipal Governance and Management Project, lack of analysis by all parties involved led to a simplistic expectation that increasing the number of women participants would automatically translate into empowered leadership among women. The project was implemented without analyzing barriers to political participation and proactive measures to ensure they were properly supported institutionally.

Another observation is how well projects in the environment sector scored on their gender equality efforts. The various project strategies recognized the interaction between gender equality and environment, such as the greater vulnerability suffered by women in disasters.

1.5 Crosscutting Theme: Environmental Sustainability

Analysis at the Program Level

At the program level, the rating for environmental sustainability as a crosscutting theme (i.e. as distinct from projects focused on the environment sector) was unsatisfactory (see Annex 9). The quality of analysis is not sufficient for integrating environmental considerations into program components. Reporting on environmental indicators, targets and results is not generally included. The budget allocation also appears inadequate. Issues in addressing environmental considerations have been identified, but not adequately addressed.

Environmental sustainability has been a consistent theme for CIDA’s programming in the Caribbean since 1993, although approaches and emphases have shifted. The programming frameworks ^{Footnote 15} all articulated intent to reduce vulnerability and strengthen resilience of Caribbean countries. Environmental sustainability was to be integrated into initiatives to strengthen the region’s capacity for efficient mitigation of and response to natural disasters. Opportunities were to be identified through public sector program support to incorporate environmental considerations into government policies and the administration of public resources.

The evaluation has found, however, that integration of environmental sustainability is not taking place, because an intersectoral platform has not been created to facilitate this process. Policy documents articulate the environmental sustainability mandate, but action has not been commensurate. Environmental sustainability is a high development priority for the Caribbean, but development initiatives are still compartmentalized.

Analysis of the Project Sample

3.8% of the projects in the sample were rated highly satisfactory in terms of environmental sustainability (as a crosscutting theme), 11.5% as satisfactory, 38.5% as moderately satisfactory, 26.9% as unsatisfactory, and 19.2% as highly unsatisfactory (see Annex 11, Figure 5). Individual project scores ranged from 0.2 (highly unsatisfactory) to 4.3 (highly satisfactory) – see Annex 10; the average numeric rating was 2.1 or moderately satisfactory (see Annex 9-B).

Projects in the environmental sustainability programming area scored well. The Caribbean Disaster Risk Management project scored highest, followed by the single environmental resource-based project, namely the Environmental Capacity Development Project. The disaster risk management projects (which now fall within the “security” programming area of the current programming framework for 2010-2015), have performed well and are expected to continue to achieve the intended outcomes. In the programming area of regional economic renewal, two projects are worth mentioning. The first project, Improving Jamaica’s Agricultural Product (IJAP), evolved from disaster recovery programming following Hurricane Ivan and was implemented in collaboration with the Inter-American Institute for Cooperation on Agriculture. Building disaster resilience in the agricultural sector was the focus, and the visibility for CIDA among stakeholders in the Government of Jamaica and the community was excellent – an example of the value added to CIDA’s communications by community-based interventions.

The second example is the Basic Needs Trust Fund (BNTF). This social infrastructure project administered by the Caribbean Development Bank scored well as a result of the environmental screening that occurs through the Bank and directly in some countries. The BNTF has been described as the “best example of integrating organizational development and capacity building with infrastructure”. The case of the extension of BNTF to Jamaica, as a part of the Caribbean Program’s contribution, is instructive in terms of cross-fertilization among programming areas, and mainstreaming environmental sustainability. Of interest is the spinoff in collaboration with the World Bank:

• Implemented through the Jamaica Social Investment Fund (JSIF), the Basic Needs Trust Fund, which addresses poverty reduction interventions, focused initially on the rural poor in Jamaica, but was expanded to include urban areas. The Environmental Framework guiding principles of the World Bank’s Inner City Basic Services Project evolved into an Environmental Management System for the JSIF. The Canada-Caribbean Cooperation Fund (part of the evaluation’s governance sample) financed a study tour to Canada where JSIF officers received guidance on developing and implementing an ISO 14000 certification program, as well as facilitation of a mentor from Ontario Power. JSIF is the only Government of Jamaica entity that is ISO certified, and efficiencies in all operations, including environmental sustainability, are being realized.

More generally, technical stakeholders have indicated that reporting on the environment as a crosscutting theme has proved difficult, as there are few opportunities for technical inputs.

2.0 Major Findings: Management Factors

2.1 Coherence

Analysis at the Program Level

At the program level, the rating for coherence was moderately satisfactory (see Annex 9). There were strong elements of overall program coherence, but also some emerging challenges. While the program’s resources have increased, senior officials responsible for aid coordination at both the national and regional levels are less aware of substantial Canadian interventions, due to the program’s shift to a more regional approach during the evaluation period by channelling funds through international financial institutions that could offer and undertake regional projects.^{Footnote 16}

Stakeholders in the region view Canada as having made a major commitment when the Prime Minister announced in 2007 that Canada would provide $600 million in funding over 10 years for the Caribbean through CIDA.^{Footnote 17},^{Footnote 18} However, the details from that commitment are not fully visible or understood in the region.

The Caribbean Program had just recently articulated its Caribbean Community Strategic Programming Framework 2007-2008 to 2017-2018, based on sectoral analysis, when CIDA announced its 20 countries of focus and launched a new programming cycle. Commenting on the emergence of CIDA’s new planning process for country/regional strategies and frameworks, the Auditor General (2009) voiced some concern about its rapid introduction, the confusion and frustration about it among staff, and the little time to undertake meaningful consultation or robust sectoral analysis.^{Footnote 19} She noted that the “absence of a well-defined and transparent planning process and formally approved and public plans impedes communications with donors and recipient governments, leaving them unclear about the Agency’s direction and long term commitment in individual countries and regions.”^{Footnote 20}

The Agency’s Management Response to the Auditor General report accepted the criticism of the Country Strategy process and stated that the direction identified in country strategies would be implemented by the responsible program through five-year Country Development Programming Frameworks, which would be based on revised Country Development Programming Framework guidelines that would be put in place by the end of October 2009.” ^{Footnote 21}

There were sequencing issues between the Agency-wide shifts in programming approaches for the 20 countries of focus and the articulation of strategies for the new thematic priorities. For example, the Sustainable Economic Growth Strategy was fully articulated only in October 2010, after the identification of the countries of focus in 2009, and in the case of the Caribbean Program, after the approval in 2009 of the Caribbean Strategy and the articulation of the Caribbean Development Programming Framework 2010-2015.

In December 2010, the program changed the Caribbean Development Programming Framework 2010-2015 and logic model, to remove key outcomes related to regional integration and institutional strengthening. These changes, in addition to removing a key programming area under CIDA’s new Sustainable Economic Growth Strategy, also weakened support for key areas where CIDA had acquired a significant comparative advantage and a leadership role in the region. The Country Development Programming Framework guidelines indicate that Country Development Programming Frameworks should be changed only with the approval of the Vice-President of Geographic Programs Branch, and in response to a major change in the country/region or a major change in Canadian foreign policy, or in response to learning such as from an evaluation.

Donor coordination is challenging due to the Caribbean’s geographic dispersion. That said, the program’s policy dialogue with beneficiaries and other donors has been reasonably effective in some instances through specific forums (e.g. the Caribbean Technical Assistance Centre and the Caribbean Development Bank) and on country poverty assessments. In other cases, there is room for improvement. There has been dialogue and sharing of effort with the United Kingdom’s Department for International Development and the Caribbean Development Band, but coordination has been weak with the European Union.

Analysis of the Project Sample

33.3% of the projects in the sample were rated as highly satisfactory in terms of coherence, 56.7% as satisfactory, 6.7% as moderately satisfactory, and 3.3% as highly unsatisfactory (see Annex 11, Figure 6). Individual project scores ranged from 1.0 (highly unsatisfactory) to 4.5 (highly satisfactory) – see Annex 10; the average numeric rating was 3.8 or satisfactory (see Annex 9-B).

Coherence was rated higher where the program had been working for an extended period with reinforcing interventions and for projects with regional institutions with a governance structure that ensured a measure of donor coordination and alignment with regional and national priorities. program elements, such as those related to economic management, supporting sustainable growth, and strengthening core regional institutions, represented a significant part of the portfolio and demonstrated internal coherence in that they related to and reinforced each other. Program Based Approach-type interventions demonstrated significant external coherence through their complementarity with the activities of other donor countries

The projects with the Caribbean Technical Assistance Centre, the CARICOM (i.e. the Capacity Development project and the Trade and Competitiveness project), the Caribbean Regional Negotiating Machinery, and the Caribbean Development Bank (i.e. Basic Needs Trust Fund) were rated as highly satisfactory on coherence. This reflected responsiveness to national and strategic priorities and a high degree of donor coordination through the respective joint governance structures of these organizations.

The Gender Equality projects were rated overall as highly satisfactory, reflecting good correlation with identified priority needs and a shared commitment among donors under CIDA’s leadership at that time. Environmental sustainability projects were rated overall as satisfactory, reflecting strong coherence with key priorities in the Caribbean on regional and subregional mechanisms. The legacy national projects in Guyana and Jamaica also received satisfactory scores, which reflect good coherence with national priorities and the programs of other donors.

2.2 Efficiency

Analysis at the Program Level

At the program level, the rating for efficiency was moderately satisfactory (see Annex 9). In addition to major unplanned changes in the programming strategy, there was also the introduction of new reporting mechanisms at the Agency level, and the process of decentralization of the program. These factors interacted in ways that could not be fully foreseen and affected efficiency.

If decentralization is judged on narrow grounds for cost-efficiency and overhead costs, then the answer is clear. It is usually more costly to deploy and support staff in the field than in headquarters. Normally the goal of decentralization is to enhance effectiveness by supporting closer interaction with counterparts and stakeholders and better achievement of results, but sequencing and logistical issues during the decentralization process, the lack of devolution of authorities to the field, challenges with information and communication technologies, and limited resources for a geographically dispersed region, have limited the potential benefits and added to complexities in program management.

To address some of these challenges, CIDA is implementing a corporate approach to decentralization based on program-driven decentralization experiences between 2003 and 2010. It includes lessons from the Caribbean Program’s decentralization experience that started in advance of fully worked out CIDA policies and arrangements with the Department of Foreign Affairs and International Trade, and proceeded without a detailed step-by-step plan to provide support facilities and infrastructure.

• The decentralized program team’s lack of access to information and communications tools, which are available to staff at headquarters, has created significant difficulties in document retention, knowledge management, and operational performance. Another challenge for a decentralized and geographically dispersed program, is that advice and training by specialists is more readily available and at lower cost at headquarters.
• Transport costs can be increased by decentralization, especially in a region such as the Caribbean that is spread over an archipelago. The introduction of video conferencing facilities in Barbados will help, but adequate travel resources are required in a region so geographically dispersed. Both efficiency and effectiveness issues are involved in the lack of sufficient resources for travel within the region. For example, in some projects limited funding for staff travel has reduced the quality of policy dialogue and performance management. At the program level, coordination of work and management of thematic teams across different field locations also presents challenges.
• Decentralization has raised issues with respect to the delegation of authorities^{Footnote 22}. Efforts have been made by CIDA to develop a new delegation of authority instrument that addresses decentralized roles. However, progress to date has been limited.
• Since 2009, the program’s Director General has been based in Barbados, which is the regional “hub”. A small team headed by a Program Coordinator remains at headquarters to provide support to the program. The other offices in the region are a Guyana office headed by a Head of Aid (with 4 locally engaged staff/professionals); a Jamaica office headed by a Head of Aid (with 4 locally engaged staff/professionals); and a Trinidad office (with 2 locally engaged staff/professionals). The pattern of offices and allocation of staff in the region warrants further review to reflect the shift to regional programming (i.e. away from country-specific projects) and to projects implemented by international financial institutions, as well as the graduation of Trinidad & Tobago, which will no longer be eligible for official development assistance.

Analysis of the Project Sample

23.3% of the projects in the sample were rated as highly satisfactory in terms of efficiency, 50% as satisfactory, 20% as moderately satisfactory, 3.3% as unsatisfactory, and 3.3 % as highly unsatisfactory (see Annex 11, Figure 7). Individual project scores ranged from 1.0 (highly unsatisfactory) to 4.5 (highly satisfactory) – see Annex 10; the average numeric rating was 3.5 or satisfactory (see Annex 9-B).

Timeliness of response appears, at least in some cases, to have been adversely affected by the shift to a greater emphasis on regional programming and a resulting dependence on a relatively narrower range of interlocutors for project design and execution. There is a lack of information on the cost-effectiveness of the projects with international financial institutions, which accounted for three fifths of major project commitments from mid-2008 until March 2011.

Projects where costs were shared among donors (e.g. the Caribbean Technical Assistance Centre projects, and other program-based approaches) tended to be rated relatively highly. That said, an assessment of efficiency should also take into account cost-effectiveness, in that smaller projects with very good development results are not necessarily less desirable than larger projects that may have lower administrative costs per investment dollar.

The projects that were rated most highly included some bilateral projects, in particular the Municipal Governance and Management Project in Guyana, and the three Gender Funds, which were operated and managed with relatively low costs to CIDA. The Gender Funds drew on management and technical support by locally or regionally engaged consultants, and developed a simple and clear set of administrative procedures for quick decision-making (one operation manual was shared among all Gender Funds).

The importance of leadership and human resource capacity was underscored in the Environmental Capacity Development Project, as the project floundered for a few years following personnel changes, but required no extensions beyond the original timelines. The Caribbean Catastrophic Risk Insurance Fund has responded promptly to needs triggered by the parametric guidelines in the respective countries, but no reports were available on management and monitoring of disbursements with respect to disaster risk reduction and environmental sustainability. The Caribbean Disaster Risk Management Project is reportedly performing well, but one area considered successful in terms of aid effectiveness (i.e. the Harmonized Implementation Program for the Comprehensive Disaster Management framework) is reportedly utilizing more time for meetings than is necessary for efficient output, and the process of approving subprojects is also long and unproductive in some instances.

Serious delays in the implementation of some projects, because of weaknesses in design or preparatory work, or other circumstances, significantly reduced their efficiency (e.g. Justice Undertakings for Social Transformation and Canada-Caribbean Leadership Program).

2.3 Management Principles

Analysis at the Program Level

At the program level, the rating for management principles was moderately satisfactory (see Annex 9). There are some good examples of consistency with the principles of alignment, ownership, and harmonisation (but not necessarily all in any one case). These include the gender equality and environment projects, and program-based approaches (PBAs) or PBA-type projects, such as the Caribbean Technical Assistance Centre, Basic Needs Trust Fund and Caribbean Regional Negotiating Machinery.

The shift to increased regional programming and, in terms of delivery modalities, a greater reliance on multilateral institutions, has created challenges for direct Canadian interaction with stakeholders at the national level, and the degree of ownership and alignment.

There are also instances of useful aid coordination, as in Canada’s role in the Public Financial Management working group and the Comprehensive Disaster Management donor group, as well as in PBA-type projects with a joint governance structure. However, there is room for strengthening aid coordination more generally, even though much of this is done at the country level, while the program’s current focus is on regional institutions and regional channels.

Care is needed in the context of changes in programming focus and delivery modalities to ensure that aid effectiveness principles are taken into account in project selection and design, as well as implementation, including in the continued use of PBA-type approaches where applicable as endorsed in the programming framework.

Analysis of the Project Sample

36.7% of the projects in the sample were rated as highly satisfactory in terms of management principles, 40% as satisfactory, 13.3% as moderately satisfactory, 6.7% as unsatisfactory, and 3.3 % as highly unsatisfactory (see Annex 11, Figure 8). Individual project scores ranged from 1.0 (highly unsatisfactory) to 4.5 (highly satisfactory) – see Annex 10; the average numeric rating was 3.6 or satisfactory (see Annex 9-B).

The highest ratings were for projects where there was a strong engagement with stakeholders, such as the gender equality and disaster risk management projects, as well as the Municipal Governance and Management Project in Guyana.

The environmental sustainability projects had some notable achievements. This included, in the Disaster Preparedness Project, increased collaboration among regional actors, improved partnerships between the Caribbean Disaster Emergency Management Agency and regional institutions, and harmonisation among donors on several aspects of project initiatives. Ownership of the Comprehensive Disaster Management methodology and application varied across the region, however, underscoring the need for allocation of time for effecting change. Stakeholders welcomed CIDA’s programming flexibility to meet changing needs in relation to desired outputs and outcomes. The Caribbean Disaster Risk Management Project also demonstrated good results in harmonisation, and capacity building. Of particular note is the Caribbean Disaster Risk Management Fund, which has gained considerable traction with civil society, and is providing good visibility for CIDA’s work in the region.

Highly satisfactory ratings also went to some projects undertaken jointly with donor and regional partners such as the Caribbean Technical Assistance Centre, the Caribbean Regional Negotiating Machinery, and the CARICOM Trade and Competitiveness Project. Other similar projects were rated as satisfactory: the Eastern Caribbean Economic Management Program, the Canada/Eastern Caribbean Debt Management Advisory Service, the Democracy and Governance Fund in Jamaica, the CARICOM Capacity Development Project, the Basic Needs Trust Fund, and the Eastern Caribbean Education Reform Project.

Several country-specific projects in Guyana (e.g. the Building Community Capacity Project, and the Small Farmers’ Livelihoods Program) and in Jamaica (the Improving Jamaica’s Agricultural Product project) were also rated as satisfactory as a result of achieving development results with target groups and communities, and developing a degree of sustainability through the development of infrastructure and community capacity, as well as building networks and visibility for CIDA.

2.4 Performance Management

Analysis at the Program Level

At the program level, the rating for performance management is unsatisfactory (see Annex 9). There are several challenges with respect to performance management in the program that arise from several sources.

One challenge is the introduction of the new results-based management tools – the Investment Monitoring Reporting Tool (IMRT) – to replace the Annual Project Performance Report (APPR). This occurred simultaneously with CIDA’s 2009 announcement to concentrate on 20 countries of focus and with the ensuing Agency-wide process (also in 2009) to develop country strategies and programming frameworks for all of these countries. There were evident limitations in the previous APPRs, since there was no requirement in the past to have baselines or to identify collection methods and data sources, which led to a lack of clearly defined indicators of results. The new IMRTs are conceptually a significant improvement, but the Caribbean Program’s staff has noted a lack of guidance and support. There is a recognition by the Results Based Management Team of the Geographic Programs Branch that there needs to be more active guidance and support to the programs, to assist them in measuring and reporting on project results and in supporting the development and refinement of logic models, the establishment of baselines, and the articulation of performance measurement frameworks at the sector and program levels.

Another challenge arises in measuring results achieved by projects implemented by international financial institutions that receive grants from CIDA. Although grants are perceived as “hands-off”, guidance is available from CIDA’s Business Process Roadmap. The major grant arrangements contain a requirement for development partners to report on progress on results achieved. In future, dialogue about baselines, logic models, and performance measurement frameworks could be undertaken more effectively at the project design stage. This should include clear expectations, guidelines, and examples on reporting. Many of these development partners, including international financial institutions, have mechanisms for measuring and reporting results and some are willing to make further accommodations, if CIDA were to engage more actively on these issues.

A potential benefit from decentralization is that CIDA managers should be able to interact more effectively with counterparts and with executing agencies to acquire information about results achieved, but there seem to be particular challenges with results reporting, and these challenges are most pronounced with international financial institutions.

The shifts in the programming framework and subsequent changes in the program logic model, have made it more difficult to measure results achieved at the level of the projects, the sectors, and the program. In the search for simplicity and certainty, the program’s revised logic model and performance measurement framework do not measure key results at the intermediate outcome level related to regional integration and economic management, and also do not measure effectively key results for crosscutting themes.

The program’s performance measurement framework has volume of exports as an “indicator” of regional integration, which as a broad macroeconomic indicator may not provide adequate information on the expected result which is contrary to current guidance on performance measurement frameworks. A guidance note has been prepared by the Geographic Programs Branch to guide the preparation and annual review of logic models and performance measurement frameworks at the program level, which emphasizes that projects are the primary sources of information for the expected results.^{Footnote 23} There is now an opportunity to review and strengthen the process of measuring and reporting results at the project level, and the process for synthesizing results at the program level with information from projects, rather than from country-wide/national information and statistics.

Similarly, the logic model and performance measurement framework at the program level do not fully capture results with respect to the crosscutting themes of gender equality and environmental sustainability. The previous corporate guidance limiting the number of indicators was cited as a constraint by the program. In July 2012, the corporate guidance on indicators was changed permitting more indicators.^{Footnote 24} This provides an opportunity to revise the performance measurement framework in line with current CIDA guidance.

The evaluation has reviewed the actions taken by the program to respond to recommendations of the Caribbean Regional Program Evaluation 1993-2003. These are summarized in Annex 17.

Analysis of the Project Sample

6.7% of the projects in the sample were rated as highly satisfactory in terms of performance management, 26.7% as satisfactory, 43.3% as moderately satisfactory, 20% as unsatisfactory, and 3.3 % as highly unsatisfactory (see Annex 11, Figure 9). Individual project scores ranged from 0 (highly unsatisfactory for the Justice Undertakings for Social Transformation project) to 4.5 (highly satisfactory) – see Annex 10; the average numeric rating was 2.8 or moderately satisfactory (see Annex 9-B).

Factors related to monitoring and measuring results raise a critical set of issues, both in terms of measuring development results for the purposes of accountability and learning, and because this area was consistently rated as relatively weak.

There are challenges both with development partners and within CIDA. Results-based management is not applied systematically in the program. There is a pattern of logic models that are not clear on intended outcomes, and the indicators selected for performance measurement frameworks are sometimes difficult to collect and could be more sharply focused. There are also more fundamental challenges, in that there were no requirements in the past to establish baselines or identify collection methods and data sources, which led to a lack of clarity on the indicators at the outcome levels.

Some of the older legacy projects tended to receive somewhat better ratings on performance management because the Annual Project Performance Reports were completed and were informative. The evaluation noted that new the Investment Monitoring Reporting Tool reporting formats are often incomplete and lack data necessary to measure results and to assess risks. In 23 In some cases, this reflects lack of baseline information or a lack of articulation of a performance measurement framework agreed with counterparts. This is partly the issue of obtaining reporting on results.

One source of challenge is that for projects involving grants there are difficulties in obtaining appropriate results-based reporting. The executing agencies receiving grants report on progress according to their own systems and are not required by the Agency to use CIDA’s results-based management tools or provide reporting in a framework suitable for inclusion in Investment Management and Reporting Tools. Therefore, the onus is left on CIDA staff to translate and document results for the Agency’s accountability and learning needs.

On the whole, the available evaluations have been serious and useful. Coverage has been reasonably comprehensive over time, but the lack of baselines impedes the measurement of results. Monitoring reports, operational reviews, and evaluations have tended to focus on implementation and management issues, but have not often generated significant information on results achieved at the different outcome levels, due to the lack of baselines and systematic collection of data for performance indicators. Similarly, there has been less consideration of how the projects interact with the activities of other development partners, and there is a lack of knowledge management and dissemination of learning from evaluations.

Joint monitoring and evaluation has been limited to a few joint initiatives, such as the Caribbean Technical Assistance Centre and the Basic Needs Trust Fund. Lack of coordination of monitoring and evaluation for some of the other Program Based Approach-type arrangements or for grants remains a concern.

3.0 Delivery Modalities and Channels of Cooperation

The evaluation focuses primarily on the bilateral channel. For the multilateral channel, the Caribbean Development Bank’s special relationship with CIDA is examined. For the partnership channel, strategic considerations are looked at, and selected projects reviewed as case studies.

3.1 Channels of Cooperation

Multilateral Channel

Canada plays an influential role in relation to the Caribbean in several key institutions: the World Bank, the International Monetary Fund and the Caribbean Development Bank, as well being involved in the Inter-American Development Bank.

Canada’s Executive Directors at the World Bank and the International Monetary Fund have represented Commonwealth Caribbean countries ^{Footnote 25} on the Board of Directors since their independence. Canada is also an important member of the Inter-American Development Bank, where it helps to ensure support for Caribbean priorities. Canada has recently announced a contribution of $13 million to the Inter-American Development Bank’s Regional Infrastructure Integration Fund, with a particular focus on the Caribbean.

Canada played a key role in the Caribbean Development Bank’s establishment. Canada and the United Kingdom are its largest non-regional shareholders, and the largest contributors to its concessional lending facility, the Special Development Fund (SDF). Canada provides a significant part of the Bank’s resources in funding for the SDF through periodic replenishments. Canada’s funding level for SDF 7 was $17.6 million per year. Negotiations are underway for SDF 8. Canada’s paid-in portion to the Bank’s recent General Capital Increase is US $20.7 million (in instalments of $3.45 million per year starting in 2011-2012). From 2006-2007 to 2010-2011, CIDA’s multilateral disbursements to the Caribbean Development Bank totalled $133.5 million.

Canada’s relationship with the Caribbean Development Bank and role in the SDF provide a vehicle for policy dialogue and influence. Canadian policy perspectives are a significant input into the Bank’s policy directions, including on poverty reduction, support for the Caribbean Millennium Development Goals targets, sustainable economic growth, regional cooperation and integration, capacity building, environmental sustainability, the climate change agenda, and the Bank’s aid effectiveness agenda. There is a strong complementary relationship between the Caribbean Development Bank policies and those of CIDA and the program. The Director General of CIDA’s Caribbean Program is Canada’s Director on the Board. However, policy responsibility for the Caribbean Development Bank remains with CIDA’s Multilateral and Global Programs Branch, in consultation with the Department of Finance and the Department of Foreign Affairs and International Trade.

Other programs from CIDA’s Multilateral and Global Programs Branch that assist the Caribbean include the International Humanitarian Assistance program, United Nations organizations, and Commonwealth programs.^{Footnote 26} The annual expenditure figures for CIDA’s multilateral aid (attributed to individual countries from Canada’s contributions to multilateral organizations), for the countries that also receive assistance from the Caribbean Program, vary from a low of $14.5 million to a high of $30.7 million, or an annual average of $19.8 million and a total of $99.1 million over the 5 years.

Partnership Channel

The Partnership with Canadians Branch supported 155 projects in the Caribbean from 2006-2007 to 2010-2011. The total disbursed was $23.6 million. The average amount per project was $152,500. Two CUSO justice projects in Jamaica were selected for a case study.

• The Access to Justice for Women and Children project contributed to making formal and alternative justice mechanisms and processes in eight countries more accessible and responsive to women and children from marginalized groups, in confronting problems of violence and other abuses of their fundamental rights. The project directly supported and linked existing local and national initiatives, and facilitated exchange of relevant knowledge across national and regional boundaries. The project contributed to the achievement of MDG-3 to promote gender equality and empower women.
• The Children in Court project, implemented with the Victim’s Support Unit (VSU) of Ministry of Justice, aimed to improve psychosocial support for child witnesses, victims and perpetrators who must appear in court. The project has deepened its partnership at the highest level of the justice system in Jamaica through the placement of a volunteer with the judiciary through the Court Management Services. The project has partnered with two non-governmental organizations (i.e. Jamaicans for Justice, and the Dispute Resolution Foundation), which are well positioned to aid the exploration of additional Caribbean partnerships and programming throughout the region. The project has achieved useful results. (i) A strategic plan was drafted to decentralize the services of the VSU in every parish along with policies and guidelines. (ii) Training was provided to social workers and volunteers to develop clinical care-counselling services, with a standardized guide for VSU. (iii) A process was initiated to establish a National Judicial Training Institute of Jamaica with a resource center that will become a centre of excellence for judicial education. (iv) A survey was conducted with social workers and a field visit done to develop a Children in Court module into an education training program.

The Partnerships with Canadians Branch (PWCB) programming is complementary to the larger projects supported by the Caribbean Program. Indeed, in the justice sector in Jamaica, the smaller PWCB projects have been more successful than the Justice Undertakings for Social Justice project that encountered difficulties due to legal obstacles relating the role of Justice Canada. An advantage of the PWCB projects is that civil society groups are mobilized in Canada and in partner countries, which can lead to ongoing collaboration. On the other hand, administrative costs can be higher with many small projects, and monitoring of performance is more difficult. Some development officers in the field indicated that the large number of small PWCB projects required monitoring.

3.2 Delivery Modalities

Bilateral Directive

In directive programming, CIDA is responsible for project identification and design, for ensuring that the project conforms to CIDA policies and processes, and for managing the contractors to achieve development results. The program’s directive modality accounted for $27 million or 14% of total program-level disbursements, which amounted to $194.6 million from 2006-07 to 2010-2011. In terms of overall scores, the directive projects were rated as satisfactory for most criteria (moderately satisfactory for gender equality and performance management), but were rated as unsatisfactory for environmental sustainability (see Annex 14, Figure 20).

Bilateral Responsive

In responsive programming, CIDA responds to projects proposed by counterparts, who are expected to take the initiative in project identification and design. Accountability for development results is shared, but CIDA is responsible for ensuring that the proposals are compatible with CIDA policies and processes. The program’s responsive modality accounted for $144.7 million or 74% of total program-level disbursements (of $194.6 million) from 2006-2007 to 2010-2011. In terms of overall scores, the responsive projects were rated as highly satisfactory for relevance, and most other criteria were rated satisfactory, except for environmental sustainability and performance management that rated as moderately satisfactory (see Annex 14, Figure 21).

Bilateral PBAs and PBA-type Projects

The evaluation reviewed the four projects classified by the Caribbean Program as Program Based Approaches (PBAs) and four others that fulfilled some but not all of the characteristics of a PBA, to determine if there were common characteristics or issues that might be useful to identify for future programming.

According to the Development Assistance Committee of the Organization for Economic Co-operation and Development, PBAs are a modality of development cooperation based on coordinated support for a locally owned development program or the program of a specific organization. A full PBA is expected to have four characteristics: (1) leadership by the recipient government(s) or organization; (2) a single comprehensive program and budget framework; (3) a formalized process for donor coordination and harmonization of procedures for reporting, budgeting, financial management and procurement; and (4) efforts to increase the use of local systems for program design and implementation, financial management, procurement and monitoring and evaluation.

Some of the executing agencies were regional institutions, such as the Caribbean Development Bank and the CARICOM Secretariat. Some were broader international institutions, for example the International Monetary Fund and the Pan-American Health Organization, although in these cases they worked through a regionally based agency, namely the Caribbean Regional Assistance Centre and the Caribbean Epidemiology Centre.

All the Program Based Approach-type of projects (PBAs)were relevant to the Caribbean Program’s programming priorities and CIDA’s overall priorities of poverty reduction and sustainable economic growth, as well as to identified priorities in the region. The PBAs were strong in terms of local ownership and leadership by countries in the region or by regional organizations. There were also mechanisms for donor coordination, and the systems and procedures of a regional (or regionally based) agency were used. In several cases, the projects contributed significantly to the development of skills and capacities of local institutions. Overall, they were consistent with aid effectiveness principles and the commitments in the Paris Declaration, and were a good fit with CIDA’s own priorities.

The PBAs were rated as highly satisfactory in terms of relevance, and all of the other criteria were rated satisfactory (see Annex 14, Figure 22). There are, however, issues with the effective measurement and reporting of results reflecting in part from the new CIDA reporting tools and the lack of baselines for the performance measurement frameworks.

4.0 Conclusions, Lessons, and Recommendations

4.1 Conclusions

Strengthening of governance at regional/subregional, national and local levels have made a valuable contribution to a range of development objectives. The factors that contributed to good results include, in several cases, support for key elements in regional/subregional cooperation.

Regional integration commitments in policy agreements and legislation by CARICOM member states are making an important contribution to sustainable economic growth. Continued support for regional/subregional integration is needed to ensure current commitments are implemented and to support further incremental integration steps at regional and subregional levels.

In response to the Auditor General’s 2009 Report, CIDA accepted that the country strategy process had some deficiencies and undertook to use the Country Development Programming Framework and the Country Development Programming Framework guidelines to rectify the situation and implement the direction identified in country strategies approved by the Minister. The limited oversight of changes in the Caribbean Program’s logic model (i.e. removal of outcomes on regional integration and institutional strengthening in 2010), indicates that the application of the Country Development Programming Framework guidelines to the revision of intermediate outcome statements in the logic model, is not clearly understood in the program. The process by which logic models are revised and approved in the Geographic Programs Branch could be improved through technical support from headquarters.

Investing in human resources and economic mobilization of marginal socio-economic groups remains a high priority in the region. Better performing projects in this area were developed over a long period of collaboration, and there would be benefits in continuing and building on such relationships, including at the community level.

In terms of effectiveness, projects which have supported policy and legislative change have achieved significant results.

Building networks among Caribbean institutions and the member states contributed to sustainability. Embedding policy developments in domestic legislation and international agreements lead to sustainable results.

Targeted gender equality projects were highly successful and strengthened Canada’s leadership and reputation as a development partner. However, integration of gender equality as a crosscutting theme was less successful resulting in a loss of momentum towards sustainable results and a void in leadership.

Under Caribbean conditions, environmental sustainability is a critical element in resilience, reducing vulnerability, and supporting poverty reduction and sustainable economic growth. The positive results of targeted environmental projects provide evidence for their continuation. However, issues in addressing environmental considerations have been identified, but not adequately addressed.

In terms of coherence, removing outcomes on regional integration, institutional strengthening, and international trade negotiations gives the impression of narrowing the program’s focus, compared to the previous programming framework, in key areas where CIDA had acquired a significant comparative advantage and a leadership role. This has also weakened the program’s explicit relevance with CIDA’s Sustainable Economic Growth Strategy.

Projects with joint donor financing and projects with strong partnerships and engagement with civil society ranked well on aid effectiveness principles.

Performance management has been a challenge, in part reflecting CIDA’s changing reporting requirements and lack of baseline data. At the program level, the logic model and performance measurement framework need to be reviewed in light of CIDA’s Sustainable Economic Growth Strategy and the crosscutting themes in CIDA’s Aid Effectiveness priorities.

CIDA’s bilateral and multilateral channels of cooperation play mutually supportive roles in terms of strengthening results, and in terms of complementing each other to jointly support Canadian policy objectives. Partnership projects provide a flexible and responsive process to engage with civil society and to build links with Canadian civil society.

In terms of delivery modalities, there has been good experience with program-based approaches, which displayed appropriate results-based management frameworks and reporting.

Ongoing flexibility to program at the national and subregional levels in addition to the regional level would permit better responsiveness to the region’s development needs.

Efficiency issues, particularly in relation to decentralization-related factors (e.g. lack of access to information and communications technology, long approval time lines, location of staff, limited delegation of authority in the field, and limited travel budgets), undermine program effectiveness.

The Government of Canada’s 2007 commitment to a $600 million increase in development assistance for the Caribbean remains an overhanging issue, particularly in relations with Caribbean governments and agencies, and may be best addressed on an Agency-wide basis.

4.2 Lessons Learned

Strengthening of key institutions at regional or subregional and national levels requires time and continuity, as well as careful monitoring and flexibility to adjust to changing circumstances and opportunities for improving results. Canada’s role in this area over time and established relationships provide a comparative advantage that can help in achieving results. [Governance]

Joint, collaborative efforts, such as Program Based Approaches, can help to strengthen beneficiary ownership, provide broader outcomes and impacts, and contribute to relevance, effectiveness and efficiency as well as to coherence, harmonization and alignment. Such efforts, however, need to be well designed and monitored, with mechanisms for beneficiary input, and flexibility in making adjustments to strengthen results. [Governance and Sustainable Economic Growth]

Shifts in programming approaches and in delivery modalities reduce the effectiveness of projects and may lead to reduced visibility and effectiveness of Canadian policy dialogue in the region.[Gender equality]

Environmental sustainability is a development necessity for the Caribbean, but development initiatives are still compartmentalized. CIDA’s Strategic Environmental Assessment (SEA) for 2006-2007 to 2010-2011 speaks to the need to move decisively beyond simply strengthening capacity, to achieving tangible progress in applying environmental sustainability solutions. Environmental and disaster management interventions have demonstrated consistency with this approach through community-based initiatives. Community-focused investments provide excellent visibility for Canada, with direct contact between project implementation and beneficiaries. Unlike larger projects which may tend to be somewhat disconnected from the beneficiaries, smaller interventions at the national and community levels have direct interaction and results appear to be discernible and meaningful to citizens. [Environmental Sustainability]

Results in terms of gender equality and environmental sustainability can benefit significantly from building and maintaining consistent relationships and partnerships over time, as well as maintaining leadership, once established. [Crosscutting themes]

Developing clear logic models and well-defined performance measurement frameworks with indicators for first and second level outcomes, which are agreed with project counterparts and relevant stakeholders, are subject to baseline analysis, and include clear risk management indicators, are key to more effective performance management. Efforts to report on results and to evaluate progress toward intended outcomes are handicapped if the logic models and performance measurement frameworks are not well designed at the outset. [Performance management]

4.3 Recommendations and Corporate Considerations

The evaluation formulated eight recommendations, seven addressed to the program and one at Branch level (see Annex 18 Management Response). It is recommended to:

1. Retain and strengthen capacity building in selected areas of governance and regional integration at regional and subregional levels with complementary national initiatives as a comparative advantage of the program.
2. Strengthen the program’s capacity and effectiveness in supporting environmental sustainability, in addition to current focus on disaster risk management, as a critical element in reducing the region’s high vulnerability and increasing resilience. Increased professional resources in the field would be helpful in identifying strategic opportunities and to strengthen environment as a crosscutting theme.
3. Seek to capitalize on Canada’s comparative advantage and leadership in gender equality in the region by rebuilding and strengthening the program’s gender equality network and role. Gender equality as a crosscutting theme should receive higher priority at the project and program levels.
4. Review expected outcomes and indicators to clearly communicate and track the program’s performance on regional integration.
5. Provide for increased flexibility for selected national and subregional projects which may contribute to the program’s core emphasis on regional programming.
6. Strengthen performance management through: (a) improved training of personnel on results management and reporting to improve the quality of Investment Management Reporting Tool reporting; (b) revised indicators in the program’s performance measurement framework in accordance with new Agency guidance to better measure results from priority sectors and crosscutting themes; and (c) strengthened evaluation focused on results at the project, sector and thematic levels.
7. Ensure there is a clear whole-of-Agency communications plan that reports on progress to date and outlines the remaining elements for the Government of Canada’s 2007 commitment of $600 million in additional development assistance to the Caribbean region.
8. Branch-level Recommendation: It is recommended that the Geographic Programs Branch clarify the approval requirements and provide guidance on the development and adjustment of country/regional strategies and Country Development Programming Frameworks, including the review and revision of logic models.

Annex 1: Evaluation Approach

Evaluation Purpose, Objectives, and Scope

CIDA is subject to the Federal Accountability Act and the 2009 Treasury Board Policy on Evaluation, which make it an obligation to evaluate all programs within a five-year period. In accordance with the Treasury Board and CIDA’s evaluation policies, the evaluation had the following learning and accountability objectives: (1) To take stock of the results achieved by the program over five fiscal years (2006-2007 to 2010-2011). (2) To assess the program’s overall performance in achieving these results. (3) To document and disseminate findings and lessons learned, and formulate recommendations to improve performance.

The evaluation covered 14 countries served by the Caribbean Program.^{Footnote 27} Given the challenges of assessing investments in a geographically dispersed region, the evaluation focused on GPB’s “bilateral” Caribbean Program, with short case studies to provide some insight into MGPB’s “multilateral” investments and PWCB’s “partnership” support for the region. The evaluation did not analyze MGPB or PWCB’s “core-funding”, which are to be covered under other evaluations.

Evaluation Methodology

The evaluation followed CIDA’s methodology and terms of reference for program evaluations. The evaluation was not expected to undertake detailed “project” evaluations, but to rely largely on secondary sources. The evaluation undertook a document review and a desk study. Documentary evidence was supplemented by interviews with CIDA staff and managers at headquarters, and by a field mission to meet a range of stakeholders in five countries (Barbados, Guyana, Jamaica, St-Lucia, and Antigua & Barbuda). Interview guides were prepared to capture evidence and perspectives at the project and program levels. Multiple sources of information and methodologies were used to generate insights and facilitate triangulation of information to validate evidence.

The evaluation methodology included assessment of eight evaluation criteria grouped into development results (relevance, efficiency, effectiveness, and crosscutting themes, with the latter disaggregated into gender equality and environmental sustainability) and management factors (coherence, efficiency, management principles, and performance management).

The criteria were assessed and scored for the project sample for each of the evaluation criteria by a numeric score from 0 to 5 with specific increments, which was then used to determine a nominal rating in one of five performance bands (Highly Satisfactory, Satisfactory, Moderately Satisfactory, Unsatisfactory, or Highly Unsatisfactory).

The program level was rated according to the same eight criteria, but only in terms of a nominal rating. It should be noted that the project and program-level assessments involve different sources of evidence and types of analysis, as well as slightly different performance indicators. This explains why, for any given criterion, project and program-level ratings may be different.

The analysis was conducted at three levels: projects, key programming areas, and program. To ensure consistency and comparability of ratings among members of the evaluation team, a modified Delphi process was followed where team members presented project ratings based on the common interview guides and project assessment sheets for joint review.

Reviewing all bilateral projects in the Caribbean from 2006-2007 to 2010-2011 would not have been cost-effective. A sample of projects was selected using a stratified sampling methodology. The sample had to represent at least 50% of the disbursements in each programming area , and include bilateral directive and responsive projects, and joint interventions such as PBAs. Only projects with budgets of less than $250,000 were not considered.

The data was provided by CIDA’s Chief Financial Officer Branch. The sample of 30 projects was selected out of a total of 67 projects. The sample covered 80.6% of the program’s overall disbursements from 2006-2007 to 2010-2011. The coverage by programming area was: Democratic Governance 80.9%; Sustainable Economic Growth 84.6%; Human Capital Formation 76.6%; Gender Equality 92.4%; and Environmental Sustainability 96.7%.

Evaluation Challenges and Limitations

Significant challenges were encountered with the Caribbean Program’s tracking of documents at the project and program level as key inputs for the desk study. Problems of access, collection, organization, and archiving of documents limited the evidence on performance of some the projects in the sample. The protracted process of obtaining necessary documents impaired the deskwork in advance of the field visit and created additional work for members of the evaluation team who worked on searching for and downloading archived key documents.

There were only nine (9) previous evaluations or operational reviews available at the project level. Some were of good quality, especially the multi-donor evaluations, such as those of the Basic Needs Trust Fund (BNTF) and the Caribbean Technical Assistance Centre (CARTAC). The previous CIDA Caribbean Program Evaluation (2004) was examined, along with the program’s follow-up to the recommendations. The evaluation sought to utilize past evaluations and studies undertaken by other actors in the international community. The United Kingdom’s Department for International Cooperation and European Union were conducting evaluation activities in parallel with the present evaluation, therefore their reports were not yet available.

There was a lack of comprehensive information on results. For most projects, baseline data were lacking. In some cases, the focus of the project had changed, without the logic model or performance measurement framework being adapted appropriately, or the indicators being reported were incomplete. Attribution of results, especially with PBA-type projects or grant agreements implemented by international organizations, posed some additional challenges.

One of the challenges was to delineate the evolving programming framework for the projects in the sample over the 5-year period. Standard evaluation methodology requires that projects be assessed against their purpose and objectives and the programming framework at the time they were designed and implemented.

Data availability is a challenge in the region. Due to limited capacity of small states, many economic, social and environmental indicators are not collected or are available after a long time lag, data are not collected systematically by international organizations for the Caribbean.

Although the sample is quite comprehensive in its coverage of the program’s disbursements, care must be taken in drawing inferences at the program level based on the project sample. The types of evidence available at the program level are largely qualitative rather than quantitative, and rely upon the review of program documents, past evaluations, studies prepared by development partners, and interviews with counterparts and stakeholders.

Annex 2: Caribbean Context

Regional Context

CARICOM countries are members of the Small Island Developing States, small islands and low-lying coastal countries that share similar sustainable development challenges, including small populations, limited economic diversification, high debt levels, heightened exposure to globalization, and susceptibility to natural disasters. The region faces development challenges rooted in its economic vulnerability, risk of social decline, and differences among political leaders that hamper regional integration efforts. Integration is the only viable option through which the region can achieve sustainable development. To the extent that the Caribbean has a shared vision, it is embodied in the treaties and agreements underpinning CARICOM, but agreement on a coherent regional development strategy remains a work in progress.

Canada has strong ties with the Caribbean developed over centuries of trade, investment and migration flows, and a “special relationship” based on shared historical, political, business, immigration, and personal linkages. Canada is an important development partner for the region, and has contributed significantly in many areas through the bilateral and multilateral channels. Canada played a major role in the region’s leading development institution, the Caribbean Development Bank, the establishment of which was in part a Canadian initiative. Canada has had a longstanding preferential trading relationship with the region and is currently negotiating a renewed trade agreement with the Caribbean.

The Government of Canada’s Strategy for the Americas (2007) recommended the Caribbean as a region of focus for bilateral assistance and engagement on trade and investment partnership. In 2007 the Prime Minister made a commitment that Canada would provide $600 million in additional development assistance to the Caribbean over a 10-year period.^{Footnote 28}

Development Context

Although average per capita income is higher than in Central America, the Caribbean includes some of the poorest economies in the hemisphere. Significant proportions of the population in all the countries live in poverty, extreme poverty, or are highly vulnerable to falling into poverty. A significant element is the high proportion of single parent, usually female-headed, households.

There are also great economic disparities, which have expanded in recent years, among the Caribbean countries. The poorest economies, such as Haiti, Belize and Guyana, are slow growth economies, experiencing brain drain through high levels of emigration of highly educated people, or people who leave to be educated abroad but do not return. The differences in the income and development levels are reflected in differences in public and private sector capacity.

The Bahamas, Barbados, and Trinidad & Tobago have graduated from the OECD list of countries classified as developing countries for purposes of collection of statistics for Official Development Assistance (ODA). However, Barbados is much closer to ODA eligibility than the other two, and faces more challenging prospects. Antigua is also classified as High Income, but it is deeply indebted despite a recent debt restructuring. The World Bank classifies countries according to 2010 Gross National Income (GNI) per capita. The groups are: low income (US $1,005 or less); lower middle income (US $1,006 - $3,975); upper middle income (US $3,976 - $12,275); and high income (US $12,276 or more).

Economic Context

The Caribbean has faced serious external challenges in the trade regime. Traditional preferences for market access to Canada, Europe and the United States were challenged under the General Agreement on Tariffs & Trade in the early 1990s, and World Trade Organization in the 2000s. Waivers obtained for unilateral preference arrangements came under pressure from Latin American partners, with high profile trade disputes on bananas and sugar. This led the region to negotiate an Economic Partnership Agreement (EPA) with the European Union.

Caribbean integration has faced political challenges. After the collapse of the West Indies Federation in 1962, a Caribbean Free Trade Association was organized to support continued economic linkages between the English-speaking countries of the Caribbean. This evolved into the Caribbean Community (CARICOM) in 1973. After years of preparatory work, in 2006 the CARICOM Single Market and Economy (CSME) was established by 12 member countries.^{Footnote 29}

The CARICOM Single Market and Economy calls for a fully market-oriented approach to the regional economy, deeper macroeconomic policy coordination, increased harmonization of functional regulatory areas, free movement of goods, services, investment, and labour, and eventually a currency union. This is an ambitious program; implementation has been slow and uneven. Yet the 2011 CARICOM Conference of Heads of Government communiqué stated that:

[...] Heads of Government affirmed the importance of the CARICOM Single Market and Economy (CSME) in advancing the development of the Region. They reiterated the importance of ensuring a fully effective and efficient Single Market which is critical to making the Region more competitive [...]. While [...] putting a pause on specific elements, such as the creation of a Single Currency, Heads of Government highlighted the critical importance of advancing implementation of those elements of the Single Economy which would create an environment more conducive to investment and job creation.^{Footnote 30}

The CARICOM Conference of Heads of Government (July 4-6, 2012) in St. Lucia reaffirmed the commitment to the CARICOM Single Market and Economy and endorsed an implementation program. The Conference of Heads of Government also attached priority to trade negotiations with Canada, and progress in implementation of the CARICOM Dominican Republic Free Trade Agreement and the European Union CARIFORUM Economic Partnership Agreement,

Although there are difficulties in achieving trade integration in the Caribbean, it is critical to improving competitiveness and enhancing growth prospects. Competitiveness is a major challenge. The region’s share of world exports has declined for 50 years. Caribbean economies are fragmented by geographic separation, and high communications and transportation costs.

Fragmentation means that the economies are dominated by micro, small and medium sized enterprises, which tend to lag in innovation and productivity growth, especially if the business climate is adverse to their development. The World Bank’s Doing Business Survey 2012, ranked the Caribbean countries (excluding Haiti) from 52 to 158 (out of 186 countries) In comparison, Chile ranked 39 and Peru 41.The challenges in the business climate relate to regulatory approvals, trading across borders, and the legal framework.

Governance Context

Strengthening of governance at regional, national, and local levels, including the rule of law, economic management, and consultative processes with civil society, are important challenges. Another challenge for the Caribbean is that there are many small jurisdictions and the institutional mandate for regional organizations is often limited. Regional institutions have a challenging set of tasks and political challenges to the negotiating process. Political fragmentation impedes the progress of regional and global integration, and compounds the economic effects of relatively high transport and communication costs.^{Footnote 31}

Progress has been made in integrating the Caribbean economies, through the implementation of commitments in the World Trade Organization, incremental steps towards the CARICOM Single Market and Economy, and the negotiation of the European Union-Caribbean Economic Partnership Agreement (EPA).^{Footnote 32} Among the fragmented jurisdictions of the Caribbean, many barriers to deeper integration remain, and local politics often reinforce protectionist pressures.

Gender Context

It is sometimes said that gender inequalities in the Caribbean appear to display male marginalization in terms of educational underachievement and school dropout rates. However, while the educational attainment of women exceeds that of men, women‘s employment prospects remain limited, with many confined to low status and low pay positions. While concerns about the plight of adolescent boys are valid, this detracts attention from the fact that both boys and girls (particularly those from poor households) are underperforming, and the impact this has on gender relations and labour market outcomes.^{Footnote 33}

Gender identities remain strong predictors of opportunity in the Caribbean, where women and men remain trapped in socially prescribed roles, fuelling expectations and behaviour patterns that negatively affect their health and life-choices. The inability of women and girls to negotiate safe sex, in a region where multiple sexual partners are common, is linked to higher incidences of HIV/AIDS. A key indicator, selected by CARICOM for MDG-3, is the incidence of violence against women. All Caribbean countries have a rate of violence against women higher than the global average: Jamaica, The Bahamas, and St. Vincent & the Grenadines are among the top ten.^{Footnote 34} A PAHO/WHO regional study found that 47% of adolescent girls’ first sexual experience was not consensual.^{Footnote 35} Women comprise 66% of victims of violent crime in Jamaica.^{Footnote 36}

Poverty increased with the economic downturn in 2008. Poverty rates range from a low of 14% (Barbados) to a high of 38% (St. Vincent & the Grenadines), and are greater in rural areas where more than half the population lives. A larger proportion is highly vulnerable to poverty, and often moves into and out of poverty and extreme poverty. Women head up to two-thirds of households across the region. Female-headed households are more likely to be poor, and have a larger family size and a greater number of dependents (children and elderly relatives). Because of the large proportion of female-headed households, another important dimension to poverty is the failure of fathers to meet financial obligations to their children.

Migration outside the region also affects household composition. Often one or sometimes both parents migrate to North America or Europe to earn higher incomes, which leads to significant remittances, but contributes to fragmentation of households and affects outcomes for children.

Environmental Context

Vulnerability and resilience are key issues that underpin sustainable development for the region. Extreme vulnerability is the bane of Caribbean countries. They are subject to the consequences of factors outside their control, with high exposure to natural hazards and global economic conditions. Their capacity to cope, or resilience, is compromised by size and limited resources, environmental/ecological degradation, limited diversification, small economies, generally weak institutional capacity, high costs of basic infrastructure, and social structures.

The potential impact of climate change is a threat for the Caribbean islands and coastal states, in that sea level rise has serious implications for low-lying coastal settlements and physical infrastructure. The region is greatly impacted by hurricanes, with the associated wind, rain and storm surge damage, floods, droughts, landslides, earthquakes and volcanoes. The increasing frequency and intensity of natural hazards over the past two decades have seriously affected physical infrastructure, as well as social, political and institutional systems. There has been extensive damage to infrastructure and property, destruction of livelihoods, human suffering and dislocation. Efforts to alleviate poverty and to meet the Caribbean targets for the Millennium Development Goals have been stymied by the frequent assault of natural hazards.

Environmental and disaster risk management are integrally linked, in that the vulnerability of the Caribbean to climate-triggered events is exacerbated by environmental degradation, inadequate environmental monitoring, management and enforcement of laws, and overall weaknesses in policy at the institutional level. Climate change is expected to exacerbate these inadequacies. The economy of a Caribbean state is also further threatened when it is dependent on environmental resources for tourism and agriculture.

Annex 3: Development Assistance of other Donors

Among multilateral donors, the European Union has the largest program, followed by the Caribbean Development Bank and Inter-American Development Bank. The World Bank, International Monetary Fund, and Organization of American States are present in the region, as are United Nations agencies particularly for social development and poverty reduction.

The data for bilateral donors is not directly comparable, but Table 11 shows official development assistance disbursements^{Footnote 37} recorded by the Organisation of Economic Co-operation and Development from 2006 to 2010. Among the three bilateral donors, Canada ranked third overall after the United Kingdom and the United States (in 2010 Canada ranked first among the three bilateral donors).

Others donors include Japan, Germany, Netherlands, Belgium, Norway, Switzerland, Spain, China, India, and Australia. Funding from emerging development partners (China, Taiwan, Brazil and Venezuela) is often for capital projects such as roads, stadiums, hospitals and airports. There are a smaller number of development partners in the Caribbean than for other regions, but the Caribbean receives one of the highest per capita ODA disbursements. Donors have national, subregional and/or regional programs to varying degrees.

The European Union strengthened its focus on the Caribbean in 2006 through a new European Union-Caribbean Partnership for Growth, Stability and Development.^{Footnote 39} The current regional focus includes: (i) support for regional integration and completion of the CARICOM Single Market and Economy; (ii) trade-related assistance under the European Union-CARIFORUM Economic Partnership Agreement (EPA) in 2008, including improved governance through public financial management and judicial reform; (iii) economic diversification through interconnection, energy and private sector development; (iv) natural disaster mitigation; (v) HIV/AIDS and other endemic health problems; (vi) drug-related crime; and (vii) human resource development.

The World Bank has country strategies focusing on national priorities related to economic growth and competitiveness, improved public administration, greater social inclusion through education and the provision of social safety nets, disaster risk management, and crime prevention and reduction. The World Bank does not have country strategies for The Bahamas, Barbados, or Trinidad, due to their level of development. The International Finance Corporation has a regional office in Trinidad that implemented the CIDA-funded ‘Partnership for CARICOM Private Sector Development’ project. The World Bank has a Regional Partnership Assistance Strategy for the Organization of Eastern Caribbean States (2010-2014) and a previous strategy for 2006-2009. CIDA projects are important elements in the World Bank’s intervention logic.

The Inter-American Development Bank’s Regional Strategy for Support to the Caribbean Community (2007-2010) focused on two areas: full intra-regional market liberalization, and regional cooperation to improve CARICOM’s social and economic infrastructure. The overall objective was to help Caribbean countries transform the regional integration process into an effective instrument of global integration, competitiveness and economic growth. The Inter-American Development Bank and its affiliate for private sector development, the Multilateral Investment Fund, also provide technical assistance and financing through national and subregional offices and programs. However, the small economies, which are members of the OECS, are not members of the Inter-American Development Bank.

The assistance of the United States Agency for International Development (USAID) for Latin America and the Caribbean is designed to promote security, strengthen democracy and support long-term economic growth. The largest program in the region was for Guyana, with significant assistance to Belize and the Eastern Caribbean. The Sustainable Development Strategy for the Caribbean Region (2005-2009) set out four strategic objectives: business environment improved for increased trade and competitiveness; more effective environmental management; increased effectiveness of key legal systems; and enhanced Caribbean response to the HIV/AIDS crisis. The crosscutting themes were human capacity/gender, governance, and public support for change. USAID now works primarily at the country level on security, HIV/AIDS, and private sector development. USAID’s new Eastern Caribbean Program (2011-2015) is to continue the fight against HIV/AIDS. It contains new initiatives to create employment opportunities for youth and to assist the island nations in adapting to the effects of climate change.

The United Kingdom’s Department for International Development (DFID) has a Regional Assistance Plan for the Caribbean (2007-2010) to address the challenge of persistent poverty and inequality, as well as vulnerabilities to HIV/AIDS, natural disasters, climate change, and economic shocks. DFID maintained bilateral programs in Jamaica and Guyana while moving towards a regional programming perspective, with regional institutions that advance Caribbean integration. DFID’s guiding themes have been: strengthening regional capacity to address vulnerabilities; supporting evidence-based, inclusive and poverty-focused policy processes and frameworks; and promoting safety, security and access to justice. Mainstreaming gender equality, disaster risk reduction and HIV/AIDS prevention were crosscutting themes. DFID continues to work regionally and nationally on economic management and public administration; trade, competitiveness and economic integration; HIV/AIDS; and violent crime.

The Caribbean Development Bank provides financing for economic infrastructure and rural development, including basic needs, education and training, social protection, private sector development, environmental sustainability and disaster management, economic management, and regional cooperation and integration through the provision of regional public goods. An overarching objective is poverty reduction. The Bank’s concessional Special Development Fund has a strong emphasis on poverty reduction and the Millennium Development Goals, social infrastructure, environmental sustainability and climate change, regional cooperation and integration, and good governance. Gender equality is a crosscutting theme.

Annex 4: Characteristics of CIDA’s Caribbean Program

The Caribbean Regional Program is designated as one of CIDA’s 20 countries of focus. Yet it is a region comprising 13 independent states and one territory. The program has undertaken a process of administrative decentralization over several years, which has posed a number of difficulties and challenges. Since 2009 the Director General has been based in Barbados, which is the regional “hub”. A small team headed by a Program Coordinator remains at headquarters to provide support to the program. The other offices in the region are:

• A Guyana office headed by a Head of Aid, with 4 LES/LEP^{Footnote 40} officers.
• A Jamaica office headed by a Head of Aid, with 4 LES/LEP officers.
• A Trinidad office, with 2 LES/LEP officers.

Expenditures by Delivery Mode, Investment Type, and Fund Type

The data source for the review of CIDA’s project portfolio was CIDA’s Chief Financial Officer Branch (CFOB). From 2006-2007 to 2010-2011, approximately as many investments were through the “responsive” delivery mode (34 projects) as the “directive” mode (32 projects). However, the “responsive” investments were larger and accounted for 74% of disbursements, as compared to 14% for “directive” (14%). Only one investment was coded under the “core” delivery mode” (a $25 million PBA that accounted for 12% of the program’s disbursements).

In terms of investment types, over two-thirds of disbursements ($381 million or 68%) was channelled through 49 projects. The program-based approaches accounted for more than one quarter of disbursements ($81 million or 28%).

The four program-based approaches were delivered through “pooled funding”. Each targets a different sector: democratic governance, basic education, emergency assistance, and improving health. Overall, the Caribbean Development Bank received 78% of the investments in PBAs (representing 16% of the program’s total budget and 28% of disbursements in 2006-2011).

In terms of fund types, the program disbursed slightly less through contributions (49%). than grants (51%) Grant disbursements totaled just over $100 million (40% through UN programs and Commonwealth partners, 60% through other international development institutions).

About three-fifths of total commitments since mid-2008 were for “multi-bi” projects, whereby bilateral funds are channelled through international financial institutions.^{Footnote 41}

Annex 5: Caribbean Program Strategies and Frameworks

The program is currently guided by a regional Caribbean Strategy 2.0 (2009) and the CDPF 2010-2015 (discussed below). Regional integration, gender equality, and environmental sustainability are crosscutting themes. From 2006-2007 to 2010-2011, the program was guided by several distinct programming documents, which have a degree of consistency and a number of common strands ^{Footnote 42}, but also differences in emphasis and an increasingly narrower focus.

1. 1993-2003 Caribbean Regional Development Policy Framework;
2. 2002 OECS Sub-Regional Development Program Plan;
3. 2006-2011 Guyana Country Development Programming Framework;
4. 2007-2018 Caribbean Community Strategic Programming Framework (CCSPF);
5. 2009 Caribbean Strategy (versions 1.0 and 2.0) approved by the Minister; and
6. 2010-2015 Caribbean Development Programming Framework (CDPF).

Previous Program Strategies and Frameworks

The Caribbean Regional Development Policy Framework 1993-2003, under which a number of projects in the portfolio were initiated, represented a significant departure for the program. Emphasis was given to regional-based cooperative programming, as distinct from programs for individual countries. A new Caribbean Division was created from three previous divisions. This was a response to an assessment of the region’s priority needs and expected changes in CIDA. However, a subregional program for the Eastern Caribbean was retained, as well as country programs for Jamaica and Guyana.

• The goal was: To strengthen the region’s capacity for greater socio-economic self-reliance on the basis of sustainable development and mutual interests.
• The objectives were: To (a) increase the region’s capacity to compete in the global economy within a framework that promotes the equitable distribution of economic gains; and (b) strengthen the region’s capacity in environmental management.
• There were three main programming themes: (1) Human Resource Development; (2) Institutional Development; and (3) Environmental Management.
• There were three crosscutting considerations: (1) Promotion of regional integration and cooperation; (2) Gender equity; and (3) Canadian/Caribbean alliances.

TheCaribbean Community Strategic Programming Framework (CCSPF) FY 2007-2008 to FY 2017-2018, reflected further analysis, with various studies and consultations in the region and with Canadian stakeholders. It was seen as a response to Canada’s Strategy for the Americas and the Prime Minister’s announcement in July 2007 that Canadian assistance to the region would be increased by $600 million over a 10-year period^{Footnote 43}. This was presented as a doubling of CIDA’s resources for the Caribbean regional program, with an envisaged winding down of the program over the subsequent five years.^{Footnote 44} The CCSPF 2007-2018 subsumed the OECS Sub-Regional Development Program Plan (2002) and the Guyana Country Development Programming Framework (2006-2011). The separate country programs for Jamaica and Guyana were to be closed in about four years, after completion of specific projects.

• The objective was: To strategically address root causes of vulnerability and institutional weakness by supporting the region as a whole so that it can be more self-reliant in 10 years.
• There were three themes: (1) Regional Democratic Governance (accountable public institutions, rule of law, and freedom and democracy). (2) Regional Economic Renewal (improving the enabling environment for trade and competitiveness and strengthening linkages to markets). (3) Regional Human Capital Formation (providing youth with appropriate skills for future labour markets and strengthening the capacity of the next generation of leaders).
• There were two crosscutting imperatives: (1) Gender Equality – also included as an objective under human capital formation, and (2) Environmental Sustainability – environment related projects were also included under the democratic governance theme.
• The rationale for supporting regional integration as a strategic approach was presented in terms of vulnerability (the impact of risks such as natural disasters and external threats on the social, economic, and environmental well-being of small states) and resilience (the ability to absorb such risks). The aim of CIDA’s program was to bolster Caribbean resilience. A principal means of improving resilience was through the economic integration agenda under the CARICOM Single Market and Economy and the OECS’ move to a subregional Economic Union.
• Advocacy and policy dialogue were to influence the development discourse and program implementation in the region, so that Canadian objectives and positions were well understood and could have an effect on policy. Canada was a major donor to the region and this carried responsibility to promote harmonization with other donors and provide leadership on selected issues for policy dialogue, such as gender equality. ^{Footnote 45} To increase aid effectiveness, every effort was to be made to work in collaboration with other donors in co-financing and program-based approaches or PBA-like initiatives.^{Footnote 46}

Current Program Strategy and Framework

A change of Ministers responsible for CIDA in 2007 and development policy considerations as part of CIDA’s Aid Effectiveness Agenda, led the Agency to focus on three new priority themes ^{Footnote 47} and on 20 countries of focus, among which the Caribbean Program. At the program-level, this led to the rethinking of the regional strategy and programming framework, through a series of iterations, in spite of the recently approved CCSPF 2007-2018.

The initial result was a Caribbean Programme Strategy 1.0, approved by the Minister in May 2009. This was followed by a shorter Caribbean Strategy 2.0 approved by the Minister in October 2009, as part of the Agency’s submission of strategies for all 20 countries of focus. There was then a longer and more detailed Caribbean Development Programming Framework (CDPF) 2010-2015 submitted for approval to CIDA’s President in December 2009.

A structure for implementation was subsequently developed through an iterative process in terms of a program-level logic model and performance measurement framework.

Although there were some differences between the three strategy documents, the broad thrust remained the same, including an effort to provide a sharper, more limited focus to the program.

The Caribbean Programme Strategy 1.0 referred to the commitment to increase Canadian assistance to the region by $600 million from 2007 to 2017, and set out two themes for CIDA’s engagement in the region, which provided a degree of continuity with the ongoing program.

• The goal was: A prosperous and more integrated Caribbean community to generate and sustain economic growth, providing opportunity and security for its citizens.
• The objective was: To promote an enabling and predictable environment for economic growth through increased private and public sector efficiency, enhanced security through the primacy of the rule of law and better management of natural disasters, and furthering regional harmonization and integration.
• The two themes were:
  1. Economic Growth – focused on effective and accountable public institutions, and connecting to markets and entrepreneurship, with several specific program areas, with elements of human capital formation through strengthening regional capacity to equip youth with skills for them to enter the workforce; and
  2. Security – including the rule of law and disaster preparedness.
• In program delivery CIDA would:
  1. design and implement a small number of large projects to advance regional cooperation, harmonization and integration;
  2. identify and work through regional institutions that had the potential to plan and deliver sustainable programming;
  3. strengthen local institutions and mechanisms to address important regional challenges; and
  4. harmonize and coordinate CIDA’s efforts with other development actors.
• Programming was to be done in ways that recognized that Caribbean integration was the indispensable foundation upon which to advance national and regional development. CIDA would work with regional institutions and governments to foster an enabling environment for economic growth through improved access to economic opportunities and a predictable environment for harmonized trade and investment contributing to regional integration. The program would seek to enhance security through combating crime, advancing judicial and legal reforms, and harmonizing legislation to further integration. Efforts would also be directed to build up capacity for regional level mitigation and response to natural disasters.

The Caribbean Strategy 2.0 retained the same objective and the two thematic approaches of Economic Growth and Security, with the same subcategories, and with six expected intermediate outcomes and a larger number of expected immediate outcomes.

• Gender equality and Environment were also key issues. A strategy was to be developed to identify programming approaches to integrate gender. Programming would strengthen capacity to better mitigate against the effects of climate change and natural disasters.
• Other elements of strategic direction included CIDA’s thematic priorities, an indication that programming would be done in ways that strengthened Caribbean integration as the indispensable foundation for advancing national and regional development, program elements that would be key to reducing poverty and inequality, and a commitment to policy dialogue and effective use of Canada’s strategic influence.
• The document also discussed the Aid Effectiveness context and set out, as development objectives to be responded to in a consultative manner, Human Development, Economic Growth and Sustainability, Stability and Security, and strengthening Governance, particularly at the regional level.
• The proposed budget for 2009-10 to 2013-2014 was $45 million the first year and $55 million per year for the next four years. At the same time, whole-of-Agency and whole-of-Government approaches including supporting the implementation of the Americas Strategy were emphasized.

The Country Development Programming Framework (CDPF) 2010-2015 was a more extensive document, to provide operational guidance to the program, with an analysis of the development context, including Economic Vulnerability, the Risk of Social Decline, Human Development, the Millennium Development Goals and Regional Integration. The thematic analysis addressed issues of Sustainable Economic Growth, including private sector development, trade and regional integration; Security, including justice and the rule of law and natural disasters; and the crosscutting themes of the Environment and Gender Equality, aid effectiveness, the institutional capacity of governments and regional institutions, and the capacity of civil society organizations.

• The overall strategy was to strengthen Caribbean integration as the foundation for national and regional growth, development and poverty reduction.
• The objective was restated as: To contribute to a more prosperous and integrated Caribbean Community able to generate sustainable economic growth, providing opportunity and security to its female and male citizens.
• The priority themes were:
  1. Sustainable Economic Growth, with sub-themes of more effective leadership and management of gender-sensitive and accountable public institutions and an improved enabling environment for business development, trade and integration, with the latter including increased capacity to achieve regional integration; and
  2. Security, including Rule of Law and Disaster Preparedness.
• The crosscutting themes were:
  1. Gender equality, to be integrated through establishing and strengthening leadership and management of gender-sensitive and accountable public institutions, increasing opportunities for women to participate in decision-making and contribute to more equal power relations between women and men, and enhancing participation of women in decision-making in security and disaster risk reduction; and
  2. Environment sustainability, to be integrated into initiatives to strengthen the region’s capacity for mitigation of, and response to natural disasters. Programs to support accountable public institutions would identify opportunities to incorporate environmental considerations in government policies and the administration of public resources. CIDA would also analyze potential pressures on natural resources related to increased production and trade in manufacturing, agriculture, forestry, fisheries, and improved transportation.
• The document sets out Planned Spending and Human Resources required from 2009-2010 to 2014-2015, with $41 million for the first year, 2009-2010. The paper also identified estimated spending through Program-Based Approaches (PBAs), indicating that two ongoing projects met the DAC conditions for PBAs and that the program would continue working through a number of harmonized or jointly funded PBA-type initiatives.
• Development of the program’s logic model and performance measurement framework was an iterative process. The approach was designed to integrate the officers in the different posts in the field (Barbados, Jamaica, Guyana, and Trinidad & Tobago) through their participation in separate sector teams for each of the intermediate outcomes in the program’s logic model and performance measurement framework.

Agency Level Policy Changes

At the Agency level, there were also significant changes concurrent with the new Caribbean strategy and completed subsequently. These included the further development of CIDA’s Aid Effectiveness Agenda, with an Aid Effectiveness Action Plan (2009-2012), enhanced emphasis on countries of focus, selection of three priority themes for CIDA programming, including a thematic approach to Sustainable Economic Growth Strategy announced in late October 2010.

The Sustainable Economic Growth Strategy approved by the Minister is particularly comprehensive and has three main pillars:

• Building Economic Foundations: strengthen public financial management, improve legal and regulatory environments, support to regional and global integration, and sustainable management of natural resources and the environment;
• Growing Businesses: support development of micro, small and medium sized enterprises, especially for women, support sustainable productivity enhancement to foster formal employment opportunities for the poor, and increase the availability of financial services to MSMEs; and
• Investing in People: increase access to demand-driven training to enhance prospects for formal employment, increase the availability of workplace learning opportunities, and support relevant results based learning initiatives.

Some of these developments were taken into account in the final stages of development of the Caribbean Strategy (2009), but some were further developed later. These thematic approaches have been reflected to varying degrees in CIDA’s programming in the Caribbean.

Annex 6: Terms of Reference Summary

Context

The Federal Accountability Act (2006) requires that all programs be evaluated every 5 years. The previous Caribbean Regional Program Evaluation (CRPE) was for 1993-2003. The current CRPE will assess a five-year period from fiscal year 2006-2007 to 2010-2011. In 2007 the Prime Minister announced $600 million in Canadian assistance to the Caribbean, which essentially requires that the program double its resources to the region over the next 10 years ^{Footnote 48}. The program is scheduled to be fully decentralized, including people, systems, infrastructure and authorities, in 2013.

Objectives of the Evaluation

The current CRPE will pursue the following objectives:

• To take stock of the results achieved by the program over the past five year period (fiscal years 2006-2007 to 2010-2011).
• To assess the program’s overall performance in achieving these results (based on criteria of relevance, effectiveness, sustainability, coherence, efficiency, management principles, crosscutting themes, and performance management).
• To assess the performance of the program’s channels of cooperation (mainly bilateral) and delivery models (directive, responsive, core programming, etc.).
• To document and disseminate findings and lessons learned, and formulate recommendations to improve the performance of the current or future program strategy.

The CRPE will review the programming strategies and frameworks in effect from 2006-2007 to 2010-2011, along with strategic policy documents approved in 2007 and 2009 and others issued since then. Projects that were terminating or well advanced during the 5-year period under review will be included in the sample to document results achieved.

Scope and Channels of Cooperation

Official Development Assistance (ODA) expenditures of CIDA’s Caribbean Program from fiscal years 2006-2007 to 2010-2011 were approximately C$194.7 million ^{Footnote 49} for 67 projects with budgets over $250,000. During that period the program’s largest sector was Democratic Governance with disbursements of C$84.2 M (43%), followed by Private Sector Development (C$41.8 M or 21%). Strengthening Basic Education and Emergency Assistance were in the same range (13-14%). Health accounted for 5% and Environment 2%. The projects coded “Other” are Canada Funds for Local Initiatives that will not be part of this CRPE.

The CRPE will analyse the development results and management factors related to the program and to projects delivered through various mechanisms, including bilateral directive projects and responsive projects implemented through multilateral institutions and non-governmental organizations, and program-based approaches (PBAs).

Evaluation Key Questions and Criteria

The CRPE will respond to two key questions and eight criteria agreed upon internationally. These elements will be assessed from the perspective of (i) the program, (ii) the main programming areas, and (iii) a sample of projects.

What has been achieved?

• Relevance: Is the program relevant in relation to the international agreed-upon goals, CIDA’s objectives, and regional and subregional development strategy and needs?
• Effectiveness of Results: To what extent did the program reach its objectives (overall and in the various programming areas)?
• Sustainability & Risks: Were the actions undertaken sustainable?
• Crosscutting themes: Have gender equality and environmental sustainability been taken into consideration adequately?

Why were the intended results achieved or not?

• Coherence: Internal coherence at program/project levels (sectors, channels, delivery models), coherence with Canada’s whole-of-government efforts (including program’s policy dialogue, leverage, leadership), and external complementarity with other members of the international community?
• Management Principles: How did the program apply the Paris Declaration principles of ownership, partnership, and harmonization?
• Efficiency: Are the human, technical, financial resources sufficient, appropriate, efficiently used? Time needed to approve, manage, monitor? Were delivery mechanisms appropriate, cost-effective? Is program’s level of decentralization adequate?
• Performance Management: This includes results-based management as well as monitoring and evaluation activities.

Evaluation Methodology

Various data gathering methods will be employed in the context of the current evaluation to ensure multiple lines of evidence are obtained:

• Document review: CIDA program and project documents, past program or project evaluations, information on regional/subregional integration in the Caribbean, documents/statistics from international institutions and other related reports/studies;
• Interviews: including CIDA, partner country officials, international community, implementing agencies, etc.;
• Project visits: selected projects in various programming areas and countries in the region;
• Roundtables: at different occasions during the evaluation process;
• Case studies: could involve certain projects, partners, or programming/implementation modalities.

Tools such as interview guides and grids will be prepared to ensure the comparability of the data between projects and sectors. A sample of projects will be reviewed based on a five-point scale ranging from highly satisfactory to highly unsatisfactory.

Sampling The CRPE cannot review 100% of the projects supported by the Canadian cooperation during the period from 2006-2007 to 2010-2011. Approximately 25 projects will be reviewed including those that have been subject to an evaluation in the past. The sampling methodology will be based on the following criteria:

• Representativeness: projects selected should represent a substantive proportion of the program’s total budget and sector or thematic value;
• Development approaches: projects and activities related to policy dialogue, institutional capacity development and/or community level interventions;
• Implementation mechanisms: directive and responsive bilateral projects, and PBAs;
• Implementing agencies: government and non-governmental organizations, multilateral institutions, Canadian Executing Agencies, etc.;
• Evaluability/Materiality: the projects are significant enough, and sources of information are available and accessible cost-effectively.

Annex 7: Program and Project Level Evaluation Indicators

1. Relevance

PROGRAM-Level Evaluation Indicators

Overall relevance of the Regional Program including policy dialogue in relation to:

• The partner region’s development context and needs
• CIDA’s policies and the Government of Canada’s interests and foreign policy objectives
• The appropriateness of the program’s delivery modalities relative to the partner region’s and country context

PROJECT-Level Evaluation Indicators

Relevance in relation to:

• Regional and Country needs: Regional and country development plans (PRSP), Millennium Development Goals (MDGs)
• CIDA policy: Poverty Reduction, Sustainable Development
• Canada’s priorities: CIDA Regional Strategy and Programming Framework, foreign policy objectives (DFAIT)

2. Effectiveness

PROGRAM-Level Evaluation Indicators

Overall extent to which the program in achieving (or is expected to achieve) its outcome-level results in relation to:

• Poverty reduction
• Strategic objectives for the regional programming framework and Canadian foreign policy
• Thematic or crosscutting objectives

PROJECT-Level Evaluation Indicators

Effectiveness in achieving results at the outcome level (and impact if available based on secondary data) at the project level n relation to:

• Poverty reduction
• Project/sector/institutional objectives
• Cost-effectiveness per result unit

3. Sustainability

PROGRAM-Level Evaluation Indicators

Overall program sustainability in relation to:

• Assessment of the partner region’s aid dependence (ODA/GNP, ODA/Budget)
• Regional and partner country context (political, social, economic, administrative, environmental)
• Predictability of program’s ODA resources (3-5 year forward information on planned aid to partner countries) per Accra Agenda

PROJECT-Level Evaluation Indicators

Sustainability at the project level in relation to:

• Time needed to attain the results (life of the project: 3-5 years)
• Institutional capacity to maintain the results (after the project)
• Financial capacity to sustain the results (after the project)

4. Crosscutting Themes (CCI)

PROGRAM-Level Evaluation Indicators

Treatment of each crosscutting theme (namely gender equality and environment, plus others if applicable) at the Program level in relation to:

• Policy dialogue (role of CIDA/Canada in context of donor coordination)
• Existence and implementation of a strategy for the CCI in the CDPF
• Existence and appropriateness of dedicated resources (budget and human resources allocated to CCI, etc., including dedicated Fund or monitor/specialist)

PROJECT-Level Evaluation Indicators

Treatment of each crosscutting theme (namely gender equality and environment, plus others if applicable) at the project level in relation to:

• Policy dialogue
• Quality of analysis (including existence and appropriateness of baseline – with sex-disaggregated data for gender equality)
• Existence, relevance, and implementation of a strategy for the CCI
• Existence of, and reporting on explicit CCI indicators, targets and results
• Dedicated resources (budget, human resources, etc.) allocated to CCI

5. Coherence

PROGRAM-Level Evaluation Indicators

Coherence at the program level in relation to:

• Internal coherence within and among:
  • Sectors of intervention
  • Cooperation channels (bilateral, multilateral, partnership)
  • Delivery models (directive, responsive, PBAs)
  • PBA delivery mechanisms (GBS, SBS, Pooled Funding, Other)
  • Development actors (government departments, non-governmental organizations, international organizations)
• External coherence (consistency with programs of other donors, donor coordination)
• Coherence with Canada’s whole-of-government efforts (including program’s policy dialogue, leverage, and leadership played by Canada)

PROJECT-Level Evaluation Indicators

Coherence at the project level in relation to:

• Internal coherence within CIDA (delivery models/mechanisms and channels)
• External coherence in the context of international efforts (donor coordination)
• Coherence in the context of Canadian whole-of-government efforts

6. Efficiency

PROGRAM-Level Evaluation Indicators

Cost-efficiency at program level in relation to:

• Program’s O&M/G&C compared to other comparable CIDA programs
• Benchmarking with other donors (staff, O&M costs, decision making, knowledge related activities)
• Strengthened field presence, including distribution and location and adequacy of staff and support services under decentralisation

PROJECT-Level Evaluation Indicators

Cost-efficiency at the project level in relation to transaction costs:

• Efficiency in use of human resources (e.g. staff turn-over)
• Efficiency in use of financial resources (e.g. ratio of O&M+PSU to G&Cs)
• Time needed to approve, manage, and monitor

7. Management Principles

PROGRAM-Level Evaluation Indicators

Overall performance at the program level in relation to Paris Declaration Principles of:

• Ownership (developing countries set their own strategies for poverty reduction, improve their institutions, tackle corruption)
• Alignment (donor countries align behind these objectives and use local systems)
• Harmonization (donors coordinate, simplify procedures, share info to avoid duplication)

PROJECT-Level Evaluation Indicators

Performance at the project level in relation to Paris Declaration principles of:

• Ownership (developing countries set their own strategies for poverty reduction, improve their institutions, tackle corruption)
• Alignment (donor countries align behind these objectives and use local systems)
• Harmonization (donor coordinate, simplify procedures, share info to avoid duplication)

8. Performance Management

PROGRAM-Level Evaluation Indicators

Performance management at program level in relation to:

• Results Management (per CIDA guidelines)
• Roll up & measure of development results
• Application of lessons learned & best practices
• Follow-up on recommendations of previous evaluations
• Risk management (per CIDA guidelines)
• CIDA monitoring & evaluation (M&E)
• M&E by CIDA’s Evaluation Directorate
• Joint M&E activities with other donors (or division of labour between donors)
• Mutual accountability mechanisms (or activities to build local M&E capacity)

PROJECT-Level Evaluation Indicators

Performance management at the project level in relation to:

• Results-Based Management (per CIDA guidelines)
• Risk management (per CIDA guidelines)
• CIDA monitoring & evaluation (M&E)
• Branch-led M&E activities at the project/sector/institutional level
• Joint M&E activities with other donors (or division of labour between donors)
• Mutual accountability mechanisms (or activities to build local M&E capacity)

Annex 8: Project Sample

Note on Annex 8: The data and financial information provided in the tables below were obtained from CIDA’s Chief Financial Officer Branch as of March 28, 2011.

Democratic Governance (including Rule of Law)

Regional Economic Renewal /Sustainable Economic Growth / Private Sector Development

Human Capital Formation

Environmental Sustainability

Gender Equality

Annex 9: Program-Level Assessments – Overall Program and Programming Areas

Note 1:Different performance indicators were used to assess the projects and the program (see Annex 7), leading to different ratings at the project-level and at the program-level. The Program Level Rating Scale employed only nominal ratings, as follows:

• Highly Satisfactory: Meets all assessment indicators for that criterion;
• Satisfactory: Meets the main indicators of that criterion;
• Moderately Satisfactory: Meets some of the indicators for that criterion;
• Unsatisfactory: Does not meet the main indicators for that criterion;
• Highly Unsatisfactory: Does not meet any of the indicators for that criterion;

Note 2: The “Programming Area” scores represent the average project scores for that area. Different performance indicators were used to assess the projects and the program (see Annex 7), leading to different ratings at the project-level and at the program-level. The evaluation used the following rating scale to assess the projects:

• Highly Satisfactory: 4.1 – 5.0;
• Satisfactory: 3.1 – 4.0;
• Moderately Satisfactory: 2.1 – 3.0;
• Unsatisfactory: 1.1 – 2.0;
• Highly Unsatisfactory: 0 – 1.0;
• n/r: not relevant

Annex 10: Project-Level Assessments – Numeric and Nominal Ratings

Note to the reader: The performance indicators used to assess the projects listed in the tables below differ from those used to assess performance at the program-level (see Annex 7). The evaluation used the following rating scale to assess the projects:

• Highly Satisfactory: 4.1 – 5.0;
• Satisfactory: 3.1 – 4.0;
• Moderately Satisfactory: 2.1 – 3.0;
• Unsatisfactory: 1.1 – 2.0;
• Highly Unsatisfactory: 0 – 1.0;

Annex 11: Project Ratings – by Evaluation Criteria

Annex 12: Project Ratings – by Programming Area

Annex 13: Effectiveness Ratings – by Programming Area

Annex 14: Project Ratings – by Delivery Modality and Overall

Annex 15: Selected Documents Consulted

Bell, Jeanette and Martineve Pajonas L’Heureux, Canada Caribbean Gender Equality Program II: Barbados and Eastern Caribbean. Annual Report 2005/2006.

Bytown Consulting and C.A.C. International, Evaluation of CIDA’s Implementation of its Policy on Gender Equality, April 2008.

Canadian International Development Agency Caribbean Program, Canada/Caribbean Gender Equity Fund: Strategy for Barbados and the Eastern Caribbean, November 1997.

Canadian International Development Agency, Canadian Regional Program Evaluation, Synthesis Report, January 2004.

Canadian International Development Agency, Caribbean Community Strategic Programming Framework FY 2007-08 to FY 2017-18, June 2007.

Canadian International Development Agency, Caribbean Program, Operational Manual: Caribbean Gender Equality Fund Program Phase II (252/1-021397), 2000.

Canadian International Development Agency, Caribbean Strategy 2.0, October 2009.

Canadian International Development Agency, CIDA's Framework for Assessing Gender Equality Results, Last updated: August 2011.

Canadian International Development Agency, CIDA’s Gender Equality Action Plan 2010-2013.

Canadian International Development Agency, Internal Audit, CIDA’s Management Practices for Program-Based Approaches, 2008.

Canadian International Development Agency, CIDA’s Policy on Gender Equality, 2010.

Canadian International Development Agency, Evaluation Directorate, Evaluation Background Profile, Caribbean Program (FY2006-2007- FY 2010-2011), July 2011.

Canadian International Development Agency, Gender Equality and Trade-Related Capacity Building: A Resource Tool for Practitioners, Undated.

Canadian International Development Agency, Guidelines on the Application of Results-based Management at the Program Level, 2012.

Canadian International Development Agency, Minister Oda unveils CIDA’s Sustainable Economic Growth Strategy, Press Release, 25, 2010.

Canadian International Development Agency, Results-Based Management at Country/Regional Program Level, 2012.

Canadian International Development Agency, Evaluation Directorate, Synthesis Report: CIDA’s Review of Program-Based Approaches, March 2010.

Canadian International Development Agency, Terms of Reference, Caribbean Regional Program Evaluation (2006-007 to 2010-2011), July 2011.

Caribbean Development Bank, Basic Needs Trust Funds (BNTF-5), 2010 Country Reports: Gender and Environment Issues, 2010.

Caribbean Regional Negotiating Machinery, EU-Caribbean Economic Partnership Agreement – briefing, 15 October 2008.

Clarke, Roberta and Jackie Sealy-Burke, Eliminating Gender-Based Violence: UNIFEM/ELAC Regional Assessment of Actions to End Violence against Women in the Caribbean, 2003.

Consulting and Audit Canada, Mid-Term Review of CARTAC (UNDP RLA/01/011), August 2003.

Court, Charles (Canadian High Commissioner to Guyana), Speaking Notes for the Launch Ceremony, Guyana Basic Education Teacher Training project (GBET) CPCE Distance Education Secondary Teachers Certificate Programme, July 29 2009.

Cyril Potter College of Education and CRC Sogema, End of Project Report - Guyana Basic Education Teacher Training Project (GBET), January 31, 2011.

Draper, Jan, Alexis Asselin, Alma O’Connell, and Vivian Marco, Guyana Basic Education teacher Training (GBET) Mission Report, August 2010.

Duperly-Pinks, Deborah, Summary of the Design of 3 of D&G partnership Approved projects, Undated.

Economic Commission for Latin America and the Caribbean -Caribbean Development and Cooperation Committee (ECLAC-CDCC), Report of the ECLAC-CDCC Gender Equality Programme Regional Conference on Gender-Based Violence and the Administration of Justice, 3-5 February 2003.

European Commission, Communication of the European Commission to the Council, the European Parliament and the Economic and Social Committee of 2 March 2006 entitled “EU-Caribbean partnership for growth, stability and development” [COM(2006) 86 final – Official Journal C 104, 3 May 2006].

Freeman, J., Evaluation of OECS Environmental Capacity Development (ENCAPD) Project, August 2004.

Guerrero, Pablo, Krista Lucenti and Sebastián Galarza, Trade logistics and Regional Integration in Latin America & the Caribbean, IDB working paper series 148, December 2009.

Halcon, Linda, Trisha Beuhring and Robert Blum. A Portrait of Adolescent Health in the Caribbean.World Health Organisation/Pan-American Health Organization. 2000.

Hay, K., Operational Review of the Eastern Caribbean Economic Management Program: Phase lll, March 2006.

Inter-American Development Bank, Regional Strategy for Support to the Caribbean Community (2007-2010), October 2006.

Jackson, Jason and Judith Wedderburn, Gender and the Economic Partnership Agreement: An Analysis of the Potential Gender Effects of the CARIFORUM-EU EPA, UNECLAC/ UNIFEM, 2009.

Kouame, Auguste and Maria Ivanova Reyes, The Caribbean Region Beyond the 2008–09 Global Financial Crisis, Paper presented at the Options for the Caribbean After the Global Financial Crisis Conference, Bridgetown, Barbados, January 27–28, 2011.

Mansfield, Kathy, Ashley Schofield and Mark Watson, Independent External Evaluation, CARTAC, Mid-Term Evaluation Report, February 2010.

MarcilLavalée, Management Review Final Report: Comments and Recommendations - Democracy & Governance Fund Management Practices For the period from June 1, 2007 to October 31, 2009.

Massaiah, Joycelyn, Looking to the Future: A Review of Canadian Caribbean Gender Equality Program Phase II (CCGEP II), 2006.

Mehra, Rekha and Geeta Rao Gupta, Gender Mainstreaming: Making It Happen, International Centre for Research on Women, February 2006.

Meir, Peter and Karen Celis, Sowing the Seeds of Its Own Failure: Implementing the Concept of Gender Mainstreaming, Social Politics, Oxford Journals: Soc Pol (2011)doi: 10.1093/sp/jxr020, October 2011.

Mitchell, Suzette, What lies at the heart of the failure of gender mainstreaming: The strategy or the implementation?, Development Bulletin, no. 64, pp. 8-10, 2004.

Nurse, Osborn and Eric Bobb, Second Mid-Term Review of CARTAC (UNDP RLA/01/011), September 2006.

OECS Education Reform Unit (OERU), Final Narrative and Financial Report to Period Ending March 31. 2008, Organization of Eastern Caribbean States Eastern Caribbean Education Reform Project (ECERP) Phase II, April 2008.

OECS Secretariat, Facilitating Partnership and Progress in Education: A Strategic Plan for the OECS Education Reform Unit, October 2004.

OERU, Report on OECS/OERU Evaluation Meeting of the Eastern Caribbean Education Reform Project (ECERP), Flamboyant Hotel, Grand Anse, Grenada, 14 September 2007.

Office of the Auditor General for Canada, Chapter 8 “Aid Effectiveness”, Report of the Auditor General of Canada, Fall 2009.

Ogilvie, K. and Associates, Canada-Caribbean Cooperation Fund, August 2011.

Organisation for Economic Co-operation and Development (OECD), Development Assistance Committee, Glossary of Key Terms in Evaluation and Results Based Management, 2002.

Organization of Eastern Caribbean States, Report on Family Law and Domestic Violence Reform Project, 2003-2008, September 15, 2008.

Portugal, Alberto and John Wilson, Lowering Trade Costs for Development in Africa, World Bank, Development Research Group, June 17, 2008.

Proulx, Francine, CUSO-VSO/ Victim’s Support Unit of the Ministry of Justice, CIDA-PWCB Monitoring Report in Jamaica, June 17-25, 2011.

Richards-Johnson, Gloria, Report of Final Evaluation, Organization of Eastern Caribbean States (OECS) Judicial and Legal Reform (JLR) Project, (Confidential and Restricted) 2008.

Ron Hughes Consulting, Eastern Caribbean Education Reform Project (ECERP) Performance Review, July 14, 2005.

Sam, Olato, Consultancy Report – Guyana Basic Education Teacher Training (GBET) Project, September 2010.

Stiles, M. and C. Barnett, Mid-Term Evaluation of the Institutional Strengthening Project of the Organization of Eastern Caribbean States Secretariat. July 2010.

Treasury Board, Policy of Evaluation, 2009.

United Nations Children’s Fund (UNICEF), Situational Analysis of Children and Women in Twelve Countries in the Caribbean, February 2007.

United Nations Development Programme (UNDP), Strategic Context Analysis for the Caribbean.

United States Agency for International Development (USAID), Gender Assessment USAID/Barbados and Eastern Caribbean, March 2010.

Universalia, Mid-Term Evaluation of the BNTF 5, Jan. 2008.

Wood, Bernard, Julia Betts, Florence Etta, Julian Gayfer, Dorte Kabell, Naomi Ngwira, Francisco Sagasti, and Malika Samaranayake, The Evaluation of the Paris Declaration, Phase II, Final Report, OECD, May 2011,

World Bank Independent Evaluation Group, The Development Potential of Regional Programs: An Evaluation of World Bank Support of Multi-country Programs, 2007.

World Bank and Organization of American States (OAS), Accelerating Trade Integration: Policy Options for Sustained Growth, Job Creation and Poverty Reduction, 2009

Annex 16: Persons Consulted

Persons Consulted by Organization

Formerly Canadian Hunger Foundation now (CHF)

Denise Bentinck, Project Director, CHF

Canadian International Development Agency (CIDA)

Cam Bowes, Head of Operations (until Oct 2011), CIDA Barbados

Beverley Clarke, Contracts and Finance, CIDA Barbados

Louise Clément, Director General, Caribbean Program, CIDA Barbados

June Emmanuel, Program Officer, CIDA Barbados

Lisanne Garceau-Bednar, First Secretary, CIDA Barbados

Michele Gibson, Senior Development Officer, CIDA Barbados

Guylaine Grenier, Second Secretary, CIDA Barbados

Zahir Meghji, Senior Program Analyst, CIDA Barbados/HQ

Mark Mostovac, Head of Operations (from Oct 2011), CIDA Barbados/HQ

Calvin Piggott, First Secretary, CIDA Barbados

Anne Marie Ready, First Secretary, CIDA Barbados

Phyllis Roett, Senior Development Officer, CIDA Barbados

Jeanne Tucker, Program Officer, CIDA Barbados

Curtis Wilson, Documentation and Archives, CIDA Barbados

Zheng Zhang, First Secretary, CIDA Barbados

Raymond Drouin, Head of Aid, CIDA Guyana

Anna Iles, Program Officer, CIDA Guyana

Nick Adams-Aston, Senior Development Officer, Commonwealth Caribbean Program, BMB, CIDA HQ

Lucie Bazinet, A/Team Leader, Gender Equality, Thematic and Sector Specialist Division, Planning, Results and Specialists, CIDA HQ

Peter Berkely, Senior Development Office, Operations, Commonwealth Caribbean Program, BMB, CIDA HQ

Shawn Hayes, Senior Gender Equality Specialist Thematic and Sector Specialist Division, GPB, CIDA HQ

Odette Langlais, Education Specialist, Thematic and Sector Specialist Division, Planning, Results and Spec. Directorate, GPB, CIDA HQ

Réal Lavergne, Manager, Aid Effectiveness and Country Programs Unit (AECPU), PWCB, CIDA HQ

Susan Learoyd, Program Policy Analyst, AID Effectiveness and Country Programs, PWCB, CIDA HQ

Dean Moser, Senior Development Officer, Inter-American Program, GPB, CIDA HQ

Julian Murray, Senior Director, Strategic Planning & Operations, (GPB), CIDA HQ Alisa Niakhai, Junior Analyst, CIDA HQ

Alisa Niakhai, Junior Analyst, CIDA HQ

Yvetta Pass, Results Based Management (RBM) Manager, Strategic Planning, Geographic Planning Results and Specialists Directorate GPRSD GPB, CIDA HQ

Linda Periard, Technical Officer, CIDA HQ

Diana Rivington, Director, Human Development and Gender Equality, Strategic Policy and Performance Branch (SPPB), CIDA HQ

Michael Von Schoenberg, Program Officer, Decentralization, Pan-Geographic Programs, GPB, CIDA HQ

Gwen Walmsley, Senior Program Manager, BMB, CIDA HQ

Gresford Bennett, Development Officer, CIDA Jamaica

Sekeywi Carruthers, Development Officer, CIDA Jamaica

Vivian Gray, Development Officer, CIDA Jamaica

Marie Legault, Head of Aid, CIDA Jamaica

Pascal Dehoux, New Environmental Advisor, CHC / CIDA

Moreno Padilla, Former environmental advisor, CHC / CIDA

Kala Seegopaul, Director, CIDA PSU (Guyana)

Former CIDA BMB Staff

Carole Kerfoote, Senior Development Officer, CIDA HQ, CHC Jamaica and Guyana

Louise Valle, Senior Governance Programme Manager, CIDA HQ, CHC Jamaica

CIDA Consultants (Past and Present)

Savitri Balbahadur, Ex. Field Manager, GBET Guyana, Cyril Potter College of Education

Jeanette Bell, Coordinator, Gender Fund Barbados/OECS, Independent

Diane Cummins, Coordinator, Canada Fund Barbados/OECS, Independent

Dr. Deborah Duperly-Pinks, Coordinator, Jamaica D&G Fund, Independent Consultant

Carole Houlihan, Gender Equality Advisor, Independent Consultant

Dr. Joycelin Massaiah, Evaluator of CCGEP II, Independent Consultant

Dana Peebles, Gender & Trade Consultant, Kartini International

Vanda Radzik, Coordinator, Gender Fund Guyana

Dorienne Rowan-Campbell, Ex. Chief, Women in Development Directorate, Commonwealth Secretariat

Susan Sproule, Ex. Project Director, Guyana GBET, ACCC, ex. CRC Sogema

Mark Styles, Evaluator/Consultant to BMB, Independent Consultant

Caribbean Centre for Development Administration (CARICAD)

Jennifer Astaphan, Executive Director, CARICAD

Rosemund Warrington, Programme Specialist Institutional Strengthening, CARICAD

Caribbean Community Office of Trade Negotiations

Trevor Boothe, Director of Technical Cooperation, CARICOM Office of Trade Negotiations (OTN)

Ambassador Gail Mathurin, Director General, CARICOM Office of Trade Negotiations (OTN)

Cherry Sampson, Program Officer, CARICOM Office of Trade Negotiations (OTN)

Caribbean Development Bank (CDB)

Norman Cameron, Division Chief, Project Services, CDB/BNTF

Yuri Chakalall, Disaster Risk Management Specialist, CDB/BNTF

Adrian Debique, Director (Ag.). Finance and Corporate Planning, CDB

Cheryl Dixon, Operations Officer, CDB/BNTF

Marlene Johnson, Gender Specialist, CDB/BNTF

Monica La Bennett, Deputy Director (Ag.), Corporate Planning, CDB

Yvonne Moses-Grant, Director (Ag.), Projects Department, CDB/BNTF

Denise Noel-Debique, Gender Specialist, CDB/BNTF

Valerie Pilgrim, Project Officer, CDB/BNTF

Karen Small, Project Officer, CDB/BNTF

Clairvair Squires, Portfolio Manager, CDB/BNTF

Dr. W. Warren Smith, President, CDB

Caribbean Examinations Council (CXC)

Didacus Jules, CEO former Permanent Secretary for Education Saint Lucia, CXC

Caribbean Export Development Agency (Caribbean Export)

Pamela Coke Hamilton, Executive Director, Caribbean Export Development Agency

Caribbean Policy Development Centre

Cecilia Babb, Deputy Coordinator, Caribbean Policy Development Centre

CARICOM Caribbean Single Market and Economy (CSME)

Myrna Bernard, Programme Officer, CSME

Ivor Carryl, Programme Director, CSME

CARICOM Secretariat (CCS)

Safya Ali, General Counsel (ag), Legal and Institutional Framework, CCS

Valerie Alleyne-Odle, Director (ag), Foreign Policy and Community Relations, CCS

Ambassador Lolita Applewhaite, Deputy Secretary-General, CCS

Garfield Barnwell, Director, Sustainable Development, CCS

Myrna Bernard, Officer-in-Charge, Human and Social Development, CCS

Lyndell Danzie-Black, Senior Project Officer, CCS

Dr. Keith Gordon, Advisor, Resource Mobilisation and Technical Assistance, CCS

Sandy Griffith, Senior Project Officer Resource Mobilisation and Technical Assistance, CCS

Clement Humes, Senior Project Officer Resource Mobilisation and Technical Assistance, CCS

Elson Jordan, Deputy Programme Manager, Programme Support, CCS

Jacqulyn Joseph, Executive Director, Strategic Planning, Monitoring & Evaluation , CCS

Patricia McPherson Education Specialist, Human Development Unit, CCS

Patricia McPherson (ag), Deputy Programme Manager, Education Human and Social Development, CCS

Alexis Murray, Project Officer Resource Mobilisation and Technical Assistance, CCS

Anya Thomas, Senior Project Officer Sustainable Development, CCS

CARTAC

Howard Edmonds, Financial Sector and Capital Markets Adviser, CARTAC

Michel Marion, Macro-Fiscal Adviser, CARTAC

Dr. Arnold McIntyre, Programme Coordinator, CARTAC

Mark Waddington, Tax Administration Adviser, CARTAC

CDEMA

Jeremy Collymore, Executive Director, CDEMA

Andria Grosvenor, Technical Manager, Preparedness & Country Support ,CDEMA

Cisne Pascal, Programme Coordinator, CDEMA

Saudia Rahat, Senior Programme Officer, CDEMA

Elizabeth Riley, Deputy Executive Director, CDEMA

CUSO

Tarik Perkins, Jamaica Programme Office Representative, CUSO VSO

Department for International Development (DFID) UK

Simone Ballister, Sustainable Development Manager, DFID

Richard Carter, Regional Social Development Adviser, DFID

Cherrianne Clarke, Growth Policy Adviser, DFID

Harry Hagan, Head, DFID Caribbean and UK Director of the Caribbean Development Bank, DFID

Department of Foreign Affairs and International Trade (DFAIT)

HE Ruth Archibald, Canadian High Commissioner, Barbados, Foreign Affairs and International Trade (DFAIT)

Eastern Caribbean Central Bank (ECCB)

Sir Dwight Venner, Governor, Eastern Caribbean Central Bank (ECCB)

Eastern Caribbean Supreme Court

Gregory Gerard, Ex. Project Coordinator, Judicial and Legal Reform (JLR) Project ,Eastern Caribbean Supreme Court

Francis Letang, Director of Projects, Eastern Caribbean Supreme Court

Environmental Sustainable Development Unit, St Lucia (ESDU)

Keith Nicholls, Head, ESDU

Peter Murray, Programme Officer, ESDU

European Union (EU)

Hubert Perr, Head of Operations, EU Delegation Barbados

GRPA

Beverly Chan, Former Project Manager for MGMP, GRPA

Guyana City Council

Robert Williams, Mayor, Guyana City Council

IICA

Dr. Audia Barnett, IICA Representative in Canada, IICA

Trevor W. Murray, Representative, IICA

Marcia Phillips-Dawkins, Project Coordinator, IICA

Ainsworth Riley, Agri-Business Specialist, IICA

Inter-American Development Bank (IDB)

Joel Branski, Representative Barbados, IDB

Jeremy Harris, Economist, IDB

Anneke Jessen, Representative in Belize, IDB

Tara Lisa Persaud, Knowledge management coordinator, IDB

Jose Jorge Saavedra, Executive Director, IDB

Jamaica Social Investment Fund (JSIF)

Milton Clark, Environmental Officer, JSIF

Scarlette Gillings, Managing Director, JSIF

Carl MacKenzie, Program Manager, JSIF

Loy Malcolm, Project Manager, Operations & Environment, JSIF

Ministry of Health and Environment, SVG

Edmund Jackson, Environmental Coordinator, Ministry of Health and Environment, SVG

Ministry of Local Government and Rural Development (MLGRD)

Nigel Dharamsall, Permanent Secretary, MLGRD

Michael Lewis, Manager, MLGRD

Organisation of Eastern Caribbean States (OECS)

Marcellus Albertin, Head, Education Reform Unit (OERU), Organisation of Eastern Caribbean States (OECS) Secretariat

Cletus Bertin, Senior Officer, Organisation of Eastern Caribbean States (OECS) Secretariat

Beverly Best, Director, Functional Cooperation Services, Organisation of Eastern Caribbean States (OECS) Secretariat

Randolph Cato, Director, Economic Affairs, Organisation of Eastern Caribbean States (OECS) Secretariat

Dr. Len Ishmael, Director General, Organisation of Eastern Caribbean States (OECS) Secretariat

Planning Institute of Jamaica (PIOJ)

Pauline Morrison, Manager, Bilateral Programme, Planning Institute of Jamaica (PIOJ)

Barbara Scott, Director, External Cooperation, Planning Institute of Jamaica (PIOJ)

Marsha Woolcock, Programme Officer, Planning Institute of Jamaica (PIOJ)

Private Sector Organisation of Jamaica (PSOJ)

Sandra Glasgow, CEO, PSOJ

Statistics Department of Barbados

Shawn Skinner, Statistics Department

Sustainable Development, Grenada

Dr. S Thomas, Director, Sustainable Development, Grenada

United Nations Development Program (UNDP)

Stein Hansen, Deputy Resident Representative, UNDP

Dr. Margaret Jones-Williams, Energy and Environment Coordinator, UNDP

Ian King, Project Manager, UNDP

Paula Mohamed, Program Manager, Governance, UNDP

Reynold Murray, Environmental and Energy, UNDP

United States Agency for International Development (USAID)

Mansfield Blackwood, Senior Technical Specialist, USAID

University of the West Indies (UWI)

Prof. Eudine Barriteau, Deputy Principal and former Head of IGDS, UWI Cave Hill

Joan Cuffie, Head, Institute of Gender and Development Studies, UWI Cave Hill

Dr. Leith Dunn, Director Centre for Gender and Development Studies, UWI Jamaica

Prof. Keith Nurse, Director of the Shridath Ramphal Centre for International Trade Law, Policy and Services, University of the West Indies (UWI) Cave Hill Campus (Barbados)

UN Women

Roberta Clarke, Caribbean Regional Program Director, UN Women Leah Odle-Benson, Programme Coordinator, UN Women

Other

Indra Chanderpaul, Ex. Minister of Human Services and Social Security, Government of Guyana

Karen de Souza, Coordinator, Red Thread

Evelyn Hamilton, Chief Planning Officer, Ministry of Education, Government of Guyana

Margaret Kertzious, Director, Help and Shelter

Magda Pollard, Member, Women and Gender Equality Constitutional Rights Commission, Government of Guyana

Annex 17: Response to Recommendations of Previous Program Evaluation 1993-2003

The evaluation has reviewed the actions taken by the program to respond to recommendations of the previous Caribbean Regional Program Evaluation 1993-2003. The recommendations dealing with program directions were accepted and reflected in the programming framework both during the interim period after 2004 and in the planning set out in 2007 (i.e. in the Caribbean Community Strategic Programming Framework 2007-08 to 2017-18). These were for a continuing or increased emphasis on regional programming; on trade, economic integration and competitiveness; on human resource development; on environmental management; and on equity and poverty as an overarching consideration.

The previous evaluation of 1993-2003 also recommended consideration of a more strategic approach to the program, including a link to the Millennium Development Goals; a strengthening of coherence in overall Canadian assistance to the Caribbean; a rationalization in support for regional organizations in order to strengthen results-based management; and exploration of possible ways to reduce the scope of the program with fewer, larger grants based on long-term partnerships.

The other recommendations dealt with development of a policy and criteria for cooperation with middle-income countries, identification of ways to reduce risk and vulnerability for the poor, and efforts to draw lessons from experience on regional programming.

The program endorsed several of these recommendations and these were reflected in the development of the programming framework.

The development of programming strategy since 2007 has become more complex, and the strategy was reformulated in 2009. A number of policy and program directions, however, have continued, and are reflected also in the most recent programming framework (i.e. CDPF 2010-2015). This includes in particular regional programming, trade and economic integration, and larger grants. Environmental management also continues to be reflected, although only as a crosscutting theme, in which the program has not been effective over the period covered by the present evaluation.

While the program has responded to many of the recommendations of the previous evaluation of 1993-2003, there is evidence that the challenges in recent years in responding to shifts in the programming framework, in shifting to work with the international financial institutions on a regional basis, in managing the transition costs of decentralization, and in adapting to utilising CIDA’s new management and reporting tools, may have led to less focus on coherence within CIDA and the whole-of-Government approach.

Annex 18: Management Response

The evaluation formulated eight recommendations, seven addressed to CIDA’s Caribbean Program and one addressed to the Geographic Programs Branch. The program’s and the Branch’s comments, commitments and actions are provided below:

Caribbean Program’s General Comments:

1. CIDA appreciates the strong validation provided by this evaluation of the results achieved by many projects over the period. Capacity building in areas of governance is a long-term endeavour, requiring sustained commitment and continuity, which the program has strived to provide. The evaluation offered a unique opportunity to take stock of progress towards outcomes and objectives, and confirmed that investments made provided value for money for Canada and for Caribbean citizens.
2. Because of the timing of the evaluation (2006-2007 to 2010-2011), few projects developed under the current program strategy (approved in 2009) were included in the sample. As a result, some observations on the program’s management and approach are based on dated information. Since 2010-2011, solid progress was made and some of the issues raised have been, or are in the process of being addressed. Chief among them is the perception that the program has shifted towards an exclusively regional approach. While initiatives under the 2009 Strategy are regional or subregional in approach, several projects are focused on a subset of countries and all projects include national and in some cases local or community level partnerships, activities and results. Examples of initiatives that are leading to significant national level results include:
   1. The Partnership for CARICOM Private Sector Development project is strengthening the Bank of St. Lucia’s small and medium enterprise (SME) strategy, increasing the profitability and sustainability of microfinance operations under the Small Business Development Finance Trust in Guyana, and helping establish productive public private partnerships (PPP) that generate employment and growth, such as the divestment of the Norman Manley International Airport in Jamaica;
   2. The Caribbean Regional Technical Assistance Centre (CARTAC) has played an instrumental role in the introduction of more efficient value-added taxes in 5 countries, the modernization of custom systems and procedures in 6 countries, and the identification and resolution of potential regressive impacts of tax policy on vulnerable groups. Additionally, Reform Action Plans were completed in 3 out of a targeted 15 states. Training in 7 countries has resulted in improved macro-fiscal management and analysis.
   3. Under the Canada-East Caribbean Debt Management Advisory Services Project, 5 out of 6 targeted countries are now conducting debt sustainability analyses annually and all 6 countries have been provided training on Debt Management Performance Assessments.
   4. As the first multi-country risk pool scheme, the Caribbean Catastrophe Risk Insurance Facility (CCRIF) has made timely payments totalling over $32 million to 7 countries affected by natural disasters.
3. The Caribbean Program’s regional approach has also proven to be cost effective in the delivery of programming in a region composed of a large number of small and dispersed states. In this regard, the program has sought, achieved, and continues to seek greater efficiencies in training, travel, hospitality and supplies budgets. The program full time equivalents (FTEs) decreased by 36%, from 28 in 2008-2009 to 18 in 2012-2013. Caribbean Program O&M budgets were reduced by 20%, from $212K in 2008-2009 to $175K in 2012-2013. During the same period, total aid disbursements, excluding Canada Fund for Local Initiatives (CFLI), rose from $40M in 2008-2009 to $47M in 2011-2012. As a result, over the period of 2008-2009 to 2011-12, disbursements per FTE increased by 80% and every dollar of aid disbursed required 0.41 of one cent, rather than 0.53 of one cent from the O&M budget. Those efficiencies will be enhanced by further reductions in line with the Deficit Reduction Action Plan. For example, the Program FTEs will be further reduced to 15 FTEs in summer 2013 for a total reduction of 46% in the number of FTEs between 2008-2009 and 2012-2013.

Recommendations 1

It is recommended that the program:

Retain and strengthen capacity building in selected areas of governance and regional integration at regional and subregional levels with complementary national initiatives as a comparative advantage of the program.

Commitments and Action

We partially agree with this recommendation. The program recognises the importance of capacity building and regional integration to support sustainable development in the Caribbean. Capacity building is a key element of all programming undertaken by the Caribbean Regional Program under the current strategy.

We disagree with the recommendation to include complementary national initiatives. The program’s regional approach represents one of its comparative advantages. Regional cooperation/integration remains the only viable alternative for a region handicapped by a lack of economies of scale and high levels of vulnerability in a globalizing world. While initiatives are regional in scope, they include partnerships, activities and results at the national level – and in some cases at the community level - to respond to the specific needs of the participating countries. At the same time, the regional approach deals with an important constraint to sustainable development in the Caribbean, that of promoting cooperation and integration among countries of the region, as well as sharing knowledge, experience and best practices. From a management perspective, the regional approach has also proven to be cost-effective.

For example, the 2011 approved Caribbean Local Economic Development Project is building the competencies and capacities of local governments and agencies to strengthen economic growth by supporting micro, small and medium sized enterprises in fifty districts in seven countries. The CARICOM Trade and Competitiveness Program includes as one its key components, support to the CARICOM Single Market and Economy Unit to help them put in place the necessary framework at a regional level and within countries of the region, for increased regional integration through movement of people, goods and services.

Respon-sibility Centre: CIDA Senior Director, Caribbean Program

Planned Comple-tion Date: Completed

Progress: Completed

Recommendations 2

Strengthen the program’s capacity and effectiveness in supporting environmental sustainability, in addition to current focus on disaster risk management, as a critical element in reducing the region’s high vulnerability and increasing resilience. Increased professional resources in the field would be helpful in identifying strategic opportunities and to strengthen environment as a crosscutting theme.

Commitments and Action

We partially agree with this recommendation. The program recognises that environmental sustainability and disaster risk management are critical elements of Sustainable Economic Growth in the Caribbean. This is particularly evident in private sector development initiatives that support, for example, the creation of small and medium enterprises. It is less evident in projects working in the area of public financial management – strengthening taxes, customs, budgeting. This overall rating was partially lowered by the poor ratings given to integrating environment into the public financial management programming.

The program also recognises the convergence between environmental issues and its substantial disaster risk reduction programming. We note that the analysis did not fully recognize the substantial increased resilience to environmental threats gained through this programming, partly attributable to the difficulty of reporting on crosscutting results within a regional program covering 14 countries.

We agree that an environmental specialist in the field would be helpful. Currently, the program benefits from the services of a HQ-based environmental specialist and will seek to augment this with Caribbean based environment specialist consulting services.

Respon-sibility Centre: CIDA Senior Director, Caribbean Program

Planned Comple-tion Date: FY2013-14

Recommendations 3

Seek to capitalize on Canada’s comparative advantage and leadership in gender equality in the region by rebuilding and strengthening the program’s gender equality network and role. Gender equality as a crosscutting theme should receive higher priority at the project and program levels.

Commitments and Action

We agree. Gender as a crosscutting issue is an important consideration for effective development. The program will recruit a locally-engaged gender specialist who will be tasked with gender analyses and recommendations on how to enhance gender equality within the program. In the meantime, the program will work with gender specialists at CIDA HQ.

Respon-sibility Centre: CIDA Senior Director, Caribbean Program

Planned Comple-tion Date: FY2013-14

Recommendations 4

Review expected outcomes and indicators to clearly communicate and track the program’s performance on regional integration.

Commitments and Action

We agree. Regional integration is explicitly woven into all immediate outcomes as regional integration is a key rationale for the program and is an integral part of every project.

The program will put a greater emphasis on articulating the regional integration aspects of project results in its corporate reporting exercises such as the Annual Regional Report.

Respon-sibility Centre: CIDA Senior Director, Caribbean Program

Planned Comple-tion Date: FY2013-14

Recommendations 5

Provide for increased flexibility for selected national and subregional projects which may contribute to the program’s core emphasis on regional programming.

Commitments and Action

We do not agree. See comment #1 above. The Caribbean Program already has the flexibility to support projects that are regional or subregional in design, but deliver activities and results at the national level and, when appropriate, at local and community levels. The Caribbean goal of regional integration and CIDA’s regional approach to programming are intrinsically linked.

Respon-sibility Centre: CIDA Senior Director, Caribbean Program

Planned Comple-tion Date: Completed

Progress: Completed

Recommendations 6

Strengthen performance management through: (a) improved training of personnel on results management and reporting to improve the quality of Investment Management Reporting Tool reporting; (b) revised indicators in the program’s performance measurement framework in accordance with new Agency guidance to better measure results from priority sectors and crosscutting themes; and (c) strengthened evaluation focused on results at the project, sector and thematic levels.

Commitments and Action

1. We agree. The program provided training to the entire program team on Results Management and Reporting in the fall 2011, which has borne fruit in the improved quality of the 2012 IMRTs. Staff will be encouraged to avail themselves of opportunities to participate in follow-up corporate training.
2. We agree. All program PMF indicators have been and continue to be adjusted on a regular basis in line with project level PMF indicators. The program PMF is an on-going working document that is adjusting its indicators as new initiatives come on board and as operational projects adjust their project level indicators. This is undertaken in a participatory fashion to ensure that the two critical elements of indicators, namely frequency and reliability are met. The program PMF captures expected results over time and most importantly the trend. The process of ongoing revision of the PMF is informed by Agency guidelines.
3. We agree. The program will develop an evaluation plan.

Respon-sibility Centre: CIDA Senior Director, Caribbean Program

Planned Comple-tion Date:

1. Completed
2. Completed
3. End of FY2012-13

Progress:

1. Completed
2. Completed

Recommendations 7

Ensure there is a clear whole-of-Agency communications plan that reports on progress to date and outlines the remaining elements for the Government of Canada’s 2007 commitment of $600 million in additional development assistance to the Caribbean region.

Commitments and Action

1. We agree. Canada’s $600 million commitment to the Caribbean involves a whole-of-Agency effort. The program developed twelve year projections for the $600M, including other Branches’ contributions, and CIDA is on track to deliver on the commitment over 12 years, as per Canada’s Economic Action Plan 2012, and produce sustainable results. Progress made against this commitment is publically available through the Open Data link on the front page of CIDA’s website.
2. CIDA has developed a communications strategy to further enhance the visibility of its development assistance in the Caribbean. Implementation began in May 2012 and will continue throughout FY2012-2013 and 2013-2014.

Respon-sibility Centre:

1. CIDA Senior Director, Caribbean Program
2. D.G. Communications

Planned Comple-tion Date:

1. Completed
2. Completed

Progress:

1. Completed
2. Completed

Recommendations 8

Branch-level Recommendation: It is recommended that the Geographic Programs Branch clarify the approval requirements and provide guidance on the development and adjustment of country/regional strategies and Country Development Programming Frameworks, including the review and revision of logic models.

Commitments and Action

We agree. The Planning, Operations and Specialists Directorate is developing a revamped programming framework. The framework clarifies the governance of all components, including review of logic models and performance measurement frameworks. The proposed framework will be accompanied by guidelines and templates to guide the work of programs in developing and updating program analysis and planning tools such as strategies, programming frameworks, logic models and performance measurement frameworks.

Respon-sibility Centre: DG, POPS

Planned Comple-tion Date: March 2014