Mali Country Program Evaluation 2006-2007 – 2010-2011 - Synthesis Report

May 2013

Acknowledgments

I would like to thank the members of Department of Foreign Affairs Trade and Development Canada (DFATD) field team: Louis Verret, the program director, and his colleagues at the embassy; Pierre Beauchemin, director of the Program Support Unit, and his colleagues, for devoting time to the team, providing information and effective logistical help, and so on. I also wish to thank Josée Fluet, deputy director of the program, her colleagues on the Mali team at Headquarters, and the DFATD managers and officers involved in the Mali program during the evaluation period, for their time and for the very useful information they contributed to the evaluation.

Econotec, the Evaluation team that prepared the original report would also like to thank the three Malian consultants – Dr. Lamissa Diakité, agricultural economist; Ms. Djourté Fatimata Dembélé, legal counsel and gender equality specialist; and Konimba Sidibé, economist – for their effective cooperation and work before, during and after the field mission.

The evaluation team also expresses its gratitude to the donors (technical and financial partners) who received them.

The team wishes to note the warm welcome and support shown by all of the Malian and Canadian program partners, whether public servants at the national or local level, representatives of civil society, or team members of Canadian support agencies or Canadian non-governmental organisations. The evaluators also particularly appreciated the way that recipients, who had benefited from Canadian projects, participated in field workshops and meetings.

Finally, I thank the evaluation team for its excellent work; James Melanson, Denis Marcheterre and Joanne Nowak; and consultant Yves Boulanger for assiduously reading drafts of the report, and for their feedback and wise advice.

Sincerely,
Caroline Leclerc
Head of Development Evaluation

Executive Summary

The CIDA Mali Program^{Footnote 1}

Between 2006 and 2011, the period under review in this evaluation, CIDA disbursed C$486.5 million to Mali as a “country of focus,” making Canada the fourth largest OECD donor.

For the duration of the evaluation period, Mali ranked 175^th out of 187 countries on the United Nations Development Program’s (UNDP) Human Development Index. According to UNDP statistics, more than half of Mali’s population (51.4 percent)^{Footnote 2} lived below the income poverty line^{Footnote 3} in 2006. According to a recent survey,^{Footnote 4} however, the incidence of income poverty totalled only 47.4 percent in 2006 and decreased to 43.6 percent in 2010. Mali’s 2007-2011 growth and poverty reduction strategy paper (GPRSP) aimed at strengthening good economic and political governance, and opening broader access to social services. The GPRSP places special attention on the decentralization of public administration.

Correspondingly, the Mali Country Program aimed to align with the GPRSP and decentralise its aid delivery mechanisms. Three overarching aims shaped programming: i) strengthening health services and the education system, especially for women; ii) improving food security at the grassroots by improving agricultural production and by supporting the growth of a savings bank and credit union network, grain market restructuring, financial services, community mobilization, and decentralization; and iii) restoring confidence in the Malian government’s capacity to promote peacebuilding and good governance. As well, three crosscutting themes were pursued: bridging the social and economic gaps between men and women; promoting environmental sustainability; and strengthening decentralized governance mechanisms, especially in northern Mali.

Evaluation Process

This evaluation provides both a global assessment of CIDA’s Mali Country Program, and an analysis of a twenty-four project sample, for the period 2006-2011. It does not take into account events and program changes subsequent to that period.

The evaluation supports accountability and learning, and has three objectives to:

• Take stock of the results achieved by the program over five fiscal years (2006-2007 to 2010-2011).
• Assess the program’s overall performance in achieving these results.
• Document and disseminate findings and lessons learned, and formulate recommendations to improve performance.

Summary of Findings

This evaluation resulted in a positive overall assessment of CIDA’s Mali Country Program between 2006 and 2011.

Relevant program — CIDA’s Mali Country Program strategy coincided with and reflected the aims of Mali’s 2007-2011 growth and poverty reduction strategy paper (GPRSP) and it envisioned investments very relevant to Mali’s needs. The program also reflected CIDA’s priorities, and for at least fifteen years, maintained a continuous focus on the same key, long-term themes: children and youth, food security, and sustainable economic growth.

Good practices in development assistance – The use of program-based approaches and budget support reflected emerging best practices in aid effectiveness; with good mechanisms for aid coordination and policy dialogue which were positively recognised by other donors. One challenge, though, is to ensure efficient collaboration among stakeholders in the field.

Results – CIDA made noteworthy contributions to education, health, governance and public management reform, food security, equality between women and men, and strengthening of civil society. Through policy dialogue and financial support, the program helped decentralize education and health programs to targeted regions.

Attention to crosscutting themes. – The program promoted equality between women and men, achieving tangible results. While some partners perceived the program as less proactive in promoting gender equality than it has been in the past, the Agency remains engaged in broader gender integration efforts through policy dialogue. There was increased attention to environmental sustainability in recently planned and launched initiatives, although for the balance of the period reviewed here, the program’s attention to this issue was less evident.

Efficiency – Since decentralization, the program has realized various gains in efficiency. The average amount of aid administered in Mali per staff person-year increased. Cost effectiveness is second only to Ethiopia in a sample comparison of similar CIDA decentralized programs in Africa. The comparison also indicated that the program has one of the lowest ratios of field staff to total disbursements. Overall, the high caliber of the program’s staff contributed to CIDA’s leadership in key areas.

Program sustainability remains a challenge ­ The progress noted above promoted sustainability by building the Government of Mali’s capacity. However, external factors may undermine some of the gains: central and decentralized institutions remain weak in terms of institutional and financial capacity; strong population growth threatens social and economic objectives; corruption persists despite progress; and, unrest has come to northern Mali.

Recommendations

The approach to performance management of the Mali program has been evolving, with a number of best practices in ownership, alignment, and harmonisation. There is an opportunity to further improve it by:

• Maintaining up-to-date risk assessment and mitigation strategies.
• More systematic monitoring and evaluation, informed by risk assessment.

While there has been leadership on gender issues, attention to the environment has been less marked over the evaluation period. The program should:

• Carefully consider an appropriate emphasis on gender equality, in light of its past leadership role.
• Consider increasing the emphasis on environmental sustainability given the challenges to sustainable poverty reduction that environmental threats represent.

Introduction

Prior to the change of regime in 2012 in Mali, progress had been made in strengthening democracy and respect for human rights. Some current concerns, however predated the political change, including a dysfunctional justice system, lack of government dialogue with civil society, and meagre resources to strengthen the full exercise of social, economic, and cultural rights. Mali ranks 175^th out of 187 countries on the United Nations Development Program’s (UNDP) Human Development Index. According to UNDP statistics, more than half of Mali’s population (51.4 percent)^{Footnote 5} lived below the income poverty line^{Footnote 6} in 2006. According to a recent survey,^{Footnote 7} however, the incidence of income poverty totalled only 47.4 percent in 2006 and decreased to 43.6 percent in 2010. Due to high birth rates and migratory movements, more government funding is transferred to social sectors, at the expense of investment in economic sectors that can strengthen growth.

The Government’s public-sector financial management improvement and modernization plan led to tangible improvements, attested to by the 2009 evaluation of the plan, International Monetary Fund reports, and the Public Expenditure and Financial Accountability report.^{Footnote 8} Its Institutional Development Program lays the foundations for a stronger human resource management system in addition to increased transparency in transactions between the Government and citizens. Rapid environmental degradation adversely affects Mali’s economy and food security, and the incomes of the poorest citizens.^{Footnote 9} Mali’s environmental situation is characterized by a tendency for agro-climatic zones to shift south. The Saharan zone is advancing toward the Sahel zone, which is pushing the Guinean zone farther south, reducing the amount of arable land.

In Mali, foreign aid represents a large proportion of GDP (about 13 percent), and about three quarters of government investments. In 2011, Canada was the fourth largest donor to Mali among member countries of the OECD’s Development Assistance Committee (DAC), representing 12 percent of the DAC total.^{Footnote 10}

Donors have established a Joint Assistance Strategy (JAS) to define ways and means to support Mali in achieving its growth and poverty-reduction objectives. The JAS is based on three pillars: the adoption of new ways of cooperating among donor countries; the implementation of a more consistent funding strategy for the growth and poverty reduction strategy paper (GPRSP); and higher quality policy dialogue with the Government of Mali. The last involves a joint monitoring framework, joint risk management, and respect for the principles of partnership and national ownership.

Mali’s 2007-2011 growth and poverty reduction strategy paper (GPRSP) aimed at strengthening good economic and political governance, and opening broader access to social services. The GPRSP places special emphasis on the decentralization of public administration.

The Mali Country Program aimed to align with the 2007-2011 GPRSP, and decentralise its aid delivery mechanisms. Three overarching aims shaped programming: i) strengthening health services and the education system, especially for women; ii) improving food security at the grassroots by improving agricultural production and by supporting the growth of a savings bank and credit union network, grain market restructuring, financial services, community mobilization, and decentralization; and iii) restoring confidence in the Malian government’s capacity to promote peacebuilding and good governance. As well, three crosscutting themes were pursued: bridging the social and economic gaps between men and women; promoting environmental sustainability; and strengthening decentralized governance mechanisms, especially in northern Mali.

Evaluation Methodology

This report synthesizes the findings and conclusions of the evaluation of CIDA’s Mali Country Program (2006-2011).^{Footnote 11} The evaluation includes a sample of 24 projects representing 41% of the program’s C$486.5 million in disbursements. Conclusions and recommendations are based on findings and evidence gathered in 2011, and do not reflect either the changes in the country’s political situation (in 2012), or the program’s undertakings since 2011. The evaluation supports accountability and learning, and has three objectives:

• To take stock of the results achieved by the program over five fiscal years (2006-2007 to 2010-2011).
• To assess the program’s overall performance in achieving these results.
• To document and disseminate findings and lessons learned, and formulate recommendations to improve performance.

The following criteria and sub-criteria were used. Projects were assigned numeric and nominal scores based on a five-point scale, whereas the overall program was rated using only the nominal scale.^{Footnote 12}

What has been accomplished? (Development Results)

• Relevance: Extent to which the development program’s objectives reflect Mali’s needs and CIDA’s policies and strategies.
• Effectiveness and results: Extent to which the development program’s objectives have been achieved, or are being achieved, considering their relative importance.
• Sustainability: Maintenance of the development program’s benefits after a major development cooperation activity is completed.
• Crosscutting themes: Extent to which gender equality, environment, and governance are integrated into the program framework.

Why and how the expected results were achieved? (Management Factors)

• Coherence: External coherence (complementarity and synergy with the activities of other donor countries). Internal coherence (coordination among CIDA’s various delivery mechanisms and among Canadian development stakeholders, including governmental, non-governmental and private-sector).
• Efficiency: Extent to which resources (funds, expertise, time) are translated into results, based on optimum allocation of resources.
• Management principles: Degree of adherence to the following principles of the 2005 Paris Declaration on Aid Effectiveness: local ownership, alignment, and harmonization.
• Performance management: Integration of results-based management and the principle of mutual accountability in program management, including mechanisms and tools developed and used.

Special attention was paid to:

• CIDA’s support in strengthening the Government of Mali’s decentralization process;
• the policy dialogue process;
• mechanisms for harmonization and coordination among technical and financial partners and the Government of Mali.

Although the sample is quite comprehensive in its coverage of the program’s disbursements, care must be taken in drawing inferences at the program level based on the project sample. The types of evidence available at the program level are largely qualitative rather than quantitative, and rely on the review of program documents, past evaluations, studies prepared by development partners, and interviews with counterparts and stakeholders. It should also be noted that since budget support operations were evaluated by a recent OECD evaluation, they were not directly included in the project sample. However, since much of the Mali Program’s project portfolio was conceived to complement its budget support investments, the latter’s impact on the program is taken into account to some extent.

The key lesson learned from this OECD evaluation was that budget support (sectoral or general) is effective when it finances and monitors the implementation of a well-established national policy, for which there is clear political commitment and implementation structures. It is much less effective when it aims to change a given policy without these key elements already in place. Since donors’ objectives in Mali have primarily been the former, budget support has significantly contributed to achieving goals in the national poverty and growth strategy, and sectoral strategies (see Annex 7 for more detail).

1.0 Major Findings: Development Results

1.1 Relevance - Highly Satisfactory

Analysis at the Program Level

Analysis of the program’s relevance focuses on alignment with i) Mali’s development objectives and priorities, and ii) CIDA’s policies and strategies.

The program is highly relevant. Its objectives and results reflect priority needs expressed in Mali’s strategic policy documents, which served as a framework for the country’s development efforts during the period. The objectives and results pursued also reflect CIDA’s priorities. This performance can largely be attributed to consistent programming. For at least 15 years, CIDA’s programming has addressed consistent themes and priorities.

When CIDA adopted the program-based approach to education and health, it fully endorsed the objectives and directions of Mali’s ten-year program and provided a large share of its aid in the form of sector budget support. The same is true of food security and rural development activities, which closely match the priority areas of intervention of Mali’s GPRSP. Support to the Office of the Auditor General of Mali is well aligned with the Government’s policy of strengthening control mechanisms to combat corruption and mismanagement of public spending.

Analysis of the Project Sample

Education, health, and food security projects achieve close to the maximum score for relevance. The education sector’s score is reduced by the “Right to Play” project (3.8), which has less to do with the program’s priorities. The health sector’s score is reduced by the “Community-Based Management of Malaria Control” project which does not have a Mali specific strategy (3.2). The average for governance and public management projects is slightly lower, owing to the somewhat satisfactory relevance of the Municipal Partnership Program (2.7).

1.2 Effectiveness – Satisfactory^{Footnote 13}

Analysis at the Program Level

Effectiveness analysis is based on two criteria: the program’s contribution to poverty reduction, and results achieved.

CIDA’s investments in education, health, and food security, are correlated with a decline in the UNDP poverty index for Mali. More detailed enquiry (through an evaluation approach such as “contribution analysis” which establishes the causal explanation for an outcome) would be required to verify the extent to which the CIDA program was responsible for this decline. However the following can be observed.

In education, the program:

• contributed to accelerating the decentralization process in the targeted regions of intervention.
• introduced an integrated strategy to ensure continuing education for teachers.
• increased the number of textbooks available in schools. This new demand for textbooks helped build the Malian textbook industry, increasing the number of publishers from three to thirty. Related trades (authors-designers, graphic artists, illustrators) have also developed. Moreover, the textbook repair pilot project helped the environment for micro-business to flourish.^{Footnote 14}
• supported the development of a Malian school-feeding program with a positive impact on the school attendance of children, especially girls.

In the health sector, the program achieved the following results:

• Decentralization: Support for the health system enabled regions with the lowest health indicators to come closer to national averages, empowering managers to implement activities for which they are responsible.^{Footnote 15}
• Paramedic training: The program strengthened a public institute, applying international standards and introducing a skills-based approach advocated by the West African Health Organization.
• Infectious disease control: The program facilitated positive results in one region, although these are now jeopardized by provisions in the Government’s new community health policy, which redefine the profile of community health stakeholders and the principle of free treatment.

Constraints encountered in project implementation prompted intelligent ancillary investments: (i) to help develop and implement a human resources policy, and to broaden the scope of support for training health professionals; (ii) to strengthen ongoing teacher training and to add the evaluation of learning; and (iii) to include capacity-building components in several initiatives.

In food security, the program contributed to increased production, more successful organization of grain inventory management and marketing, and increased capacity of operators in the shea butter and shallot/onion/garlic sectors. Results include:

• creation in the Ségou region of an apex organization of 163 rural organizations, encompassing 3,278 members (25% of the membership are women). This apex organization acts as a marketing and credit agency, serving its members, mostly small farmers engaged in rice and dry grain cultivation.
• irrigation in rural areas in the Timbuktu and Mopti regions.

At least two factors account for success: long-standing CIDA engagement in food security, and prioritization of productive activities involving women. However, major constraints (common to all cooperation projects in this sector) continue to affect the achievement of results:

• The decentralized financial system is weak and fragile. This limits access to medium-term credit for small farmers, and may thus constitute a major obstacle to achieving the results of the Agricultural Sector Supply Chains support project;
• Complex land tenure systems, liable to provoke local conflicts, which occasionally pose an obstacle to resource development (fertile soil and water);
• The agriculture/environment sector is institutionally complex. There is no single, comprehensive national planning document, which has limited the use of the program-based approaches to support sub-sector strategies. Objectives in microfinance have been partially realized in strengthening monitoring agencies. However, the objective of expanding access to microfinance, and its availability to rural communities, has not been achieved. The microfinance sector’s ineffective management threatens the growth of its healthy components, and its contribution to economic development and poverty reduction.

The microfinance institutions’ (MFIs) supervisory authority does not exercise effective control of the sector. It lacks the capacity to penalize or decertify failed or unsustainable financial institutions. Capacity strengthening initiatives have proved difficult to implement. A number of MFIs are limited in scale and absorptive capacity, financially weak, and resistant to merger.

Nevertheless, CIDA has built the management capacities of many decentralized financial institutions to enable them to comply with new regulations, and helped merge or restructure others. Many studies and diagnoses have been carried out with support from the Microfinance Sector Support Program. They have allowed a better understanding of the problems and issues that the sector faces.

Governance and public management activities have yielded adequate results, mainly in terms of outputs, but few immediate or intermediate outcomes have emerged to date, given that most activities are at the stage of restructuring and developing tools. Most ministries have, however, completed organizational audits, organizational frameworks and procedures manuals for contracting, codes of ethics, policy development, internal control strategies, reorganization of the human resources function, and computerization. There have been few outcomes to date in terms of services to users.

Support for the justice sector generally yielded modest results that have not yet proven sustainable. There is not enough support from ministerial authorities in charge of the ad hoc unit responsible for implementing the Ten-year Justice Development Program and little support to the justice sector from other donors.

Significant progress has been made in structuring and (to some extent) strengthening civil society organizations. However, civil society organisations remain weak in terms of their capacity to develop well-established positions and to take concerted action.

Analysis of the Project Sample

While most projects scored “satisfactory” in health and education, the teacher training support program (PAMOFE) earned the lowest score (3.3) among projects in the education sector, since the training models developed could not be generalized before the end of the project, and only five of the nine expected outputs were achieved. In the health sector, the Social and Health Development Plan (PDSS) earned the lowest score (3.0) because it could not deliver all of the expected results, owing to the lack of qualified health staff, especially in remote areas. In microfinance, Microfinance Sector Support Program (PASMIF) results are largely disappointing. The sector has yet to be reformed on a national scale.

Support for the Office of the Auditor General of Mali (BVGM) has yielded positive outcomes in a short time. The Office is well established, with administrative and financial procedures, and trained personnel. It produces financial and performance audit reports, exposing corrupt dealings and mismanagement, and referring cases for legal proceedings and recovery of funds. In addition, the current constitutional review will strengthen the BVGM’s legal status. A new law will define its mandate and authorities in relation to the Court of Auditors.

It has not yet been possible to implement the technical assistance components of the Institutional Development Program, or the public-sector financial management sector budget support. In addition, the Internal Revenue Mobilization Support Project’s implementation was delayed for three years. This correspondingly delayed Mali’s efforts to increase its internal tax revenue, a move deemed crucial by the Government and donor countries.

1.3 Sustainability – Somewhat Satisfactory

Analysis at the Program Level

Sustainability aims to assess the long-term viability of a program or project’s achieved results. Three conditions help determine the future sustainability of program and project results: i) the time allowed to achieve results; ii) risks identified in each sector that specifically affect sustainability; and iii) institutional, financial, and technical capacities to maintain results.

Time allowed to achieve results

In education, the sample identified various projects designed to bring about radical changes in administrative culture and operational procedures. Because this takes time, the support project to the Decentralized Management of Basic Education (AGDEF) and the teacher training support project (PAMOFE) spanned periods ranging from seven to ten years, with extensions planned for the future. Activities in the health sector also take the long-term perspective by targeting cultural and behavioural change, and providing training. Projects were quickly renewed, and expected results reviewed and aligned with emerging realities. In food security, CIDA’s operations subscribe to the principle that rural activities must be of sufficient length to support changes in traditional behaviour. In governance and public management, most activities are expected to continue into new phases to implement reforms and apply the tools developed. Public service reform is necessarily a long-term project that will take a number of years to complete.

Capacity to maintain results

In education and health, Mali’s system has weaknesses in managing human resources. These weaknesses limit the long-term effects of CIDA’s operations. There is a lack of accountability in the public service, reinforced by a systemic lack of supervision, coaching, and performance evaluation; arbitrary appointments; and allocation of staff that favours urban centres at the expense of remote areas. Other factors weaken Mali’s capacity to own the program’s results, including a very low level of capacity for the vast majority of stakeholders, considering the responsibilities they must assume.^{Footnote 16}

In the agricultural sector, as the planning framework and better budget procedures are implemented, CIDA’s programming is shifting from projects to greater use of program-based approaches. To expedite this, Canada provides ad hoc support to the authorities responsible for developing the planning framework. In microfinance, the ability to maintain results depends on the size and management capacities of microfinance institutions (MFI). These capacities are insufficient in many cases. Once project support ends, there is no guarantee that the Association professionnelle de la microfinance will have the capacity to provide services to MFI. Monitoring agencies and pooled-funding mechanism managers do not attach enough importance to analyzing the sustainability of support provided. In governance and public management, sustainability of results largely depends on whether the Government of Mali and its ministries own these results.

Program-wide, CIDA funding has the following characteristics that contribute to the sustainability of results:

• By aligning with Malian policies and programs, CIDA provides the Government with additional means to implement its own policies and programs.
• CIDA’s support aims to strengthen existing systems and procedures, instead of creating a parallel system to manage its own investments.
• In the program-based approach CIDA participates in all of a given sector’s consultation frameworks, promoting information sharing, and complementarity of effort. The role of Canadian organizations is to provide support and to transfer expertise, not to implement or execute. This helps to increase the Government of Mali’s ownership.
• The many approaches used in country programming ensure that aid is flexible and can adapt itself to changes and constraints in the country of intervention. Program-based approaches, founded on national action plans (with or without budget support) can be complemented by more targeted approaches that provide the technical assistance and direct funding needed for increasing ownership by the stakeholders concerned. The most conclusive results achieved in Mali’s health and education sectors stem from projects supporting national sectoral policies. Maintaining such a variety of approaches entails major challenges. Among other things, it assumes strong capacity to become involved and to provide value added in technical and policy dialogue. CIDA programming has largely demonstrated this capacity.

Analysis of the Project Sample

In some projects, there is a risk (beyond the program’s full control) that achievements may not be maintained. There is a need to analyze risks related to the program’s various components more systematically, to identify more effective mitigation measures.

In education and health, sustainability is the most challenging factor. Textbook support projects offer satisfactory guarantees of sustainability (see p.14 for detail).

In food security, the scores ranged widely. The Microfinance Sector Support project’s (PASMIF) poor score drags the average downward. In microfinance, there are varying prospects for the sustainability of project results, depending on the size and level of capacity of microfinance institutions.

In governance and public management, prospects for the sustainability of results are somewhat satisfactory. They were weaker for the Justice Development Project (PRODEJ), but could be improved in the new phase of support. Prospects for the sustainability of the Office of the Auditor General will be improved, since the adoption of a new Constitutional law, which brings additional precision to the Auditor General’s mandate. However, there are still some risks with regard to its mandate and autonomy.

1.4 Crosscutting Themes – Somewhat Satisfactory

The evaluation assessed the program’s attention to three themes: gender equality, environmental sustainability, and decentralized governance. Are they sufficiently well-developed in the program framework, and have adequate resources been allocated?

Gender Equality

Analysis at the Program Level

The program actively promoted equality between women and men throughout its projects and policy dialogue. From 2007 to 2010, Canada led the multidonor gender thematic group, whose work promoted greater consideration of gender in Mali’s development policies. However, Canada’s efforts in this sector have been impeded by the Government of Mali’s lack of resources to strengthen the exercise of social, economic and cultural rights. Notably, a new Personal Status and Family Code was developed to address persistent social, economic and legal discrimination towards women and girls. However, the amended Code that was eventually approved removed many of the progressive measures.

Despite these challenges, the Malian government has shown some progress. For instance, Canada’s support to the justice sector allowed the Ministry of Justice to develop its gender policy. The Ministry was also actively involved in developing the Personal Status and Family Code. Canada has also supported the Ministry for the Promotion of Women, Children, and the Family to develop the first National Gender Policy in Mali, in coordination with civil society organizations. The gender equality project has also encouraged NGOs to become involved in proposing and carrying out activities to promote women’s health and rights, and their participation in governance.

A second success factor was that CIDA has given priority to gender equality in all of its programming. The Agency has been recognized as a particularly active lead donor in this area. It led the Gender working group from 2007 to 2010, and was subsequently an active member. On the other hand, stakeholders noted Canada’s decrease in leadership on gender equality towards the end of the evaluation period. Thus, while CIDA remains engaged in policy dialogue, it no longer has gender-specific programming. Nevertheless, it continues to integrate gender as a crosscutting theme at the program level.

Environmental Sustainability

Analysis at the Program Level

Planning documents attach only limited importance to the environmental dimension. This gives the impression that project planning is limited to conducting statutory environmental analyses, and that program implementation merely involves compliance with the Canadian Environmental Assessment Act.^{Footnote 17} With a few exceptions for projects by Partnerships with Canadians Branch, the general approach seems to be to ensure that a project is environmentally neutral. Projects aside, the program carried out some advocacy activities. In 2007, it funded a national workshop to build the capacity of private and public actors in integrating environmental considerations in Mali’s projects and programs, including the Growth and Poverty Reduction Strategy Paper. In 2008, the program drafted an environmental action plan as a result of the workshop, and participated in a pan-African workshop. The 2007 evaluation had already identified poor representation of the environment in CIDA’s programming. Over the course of the evaluation period, the Agency generally played a limited role in ensuring greater integration of environmental considerations in Mali’s development policies and strategies. However, environmental considerations were largely integrated in initiatives launched (or still at the planning stage) towards the end of the evaluation period, particularly in the food security sector.

Governance

Analysis at the Program Level

Governance is the program’s third crosscutting theme. Its activities had a significant impact on relations among stakeholders in the education and health sectors. In several regions, stakeholders are little accustomed to cooperating, being traditional rivals for authority to manage regional and local services. The program’s forums addressed this historically conflictual dynamic by seeking and creating opportunities for dialogue, facilitating development of shared vision, and contributing significantly to improved local management of health and education services in Mali.

In education, new school boards were created. The recommendations of the national colloquium on improving working relations between government services and collectivities are yet to be implemented. School management committees still need strengthening. In health, efforts should continue to strengthen communication among the various mechanisms responsible for facilitating the health pyramid. Efforts should continue to build regional planning capacity.^{Footnote 18}

In other sectors, where the Government of Mali did not have explicit decentralization strategies, activities often supported decentralization, more incidentally than by design. This is the case in food security, and in governance, two sectors in which all projects have had a marked regional impact.

Analysis of the Project Sample for Crosscutting Themes

Consideration of crosscutting themes is one of the program’s weakest factors together with sustainability. In some projects, despite focused activities, women and girls are regarded more as passive recipients than as development stakeholders.^{Footnote 19} With regard to the environment, biomedical waste is not always safely managed and may pose a public health risk that requires immediate attention. Support to the Social and Health Development Plan (PDSS) earned the lowest environmental score in the sample, owing to non-compliance with national hygiene standards for public health buildings and facilities.

In food security, there was good integration of the gender dimension of projects in the agricultural sample (despite constraints associated with rural social and cultural traditions). On the other hand, there was lacklustre handling of environmental protection and sustainable management of natural resources. However, environmental considerations were integrated to a greater extent in initiatives launched, or still at the planning stage, towards the end of the evaluation period. In microfinance, the average for the sector as a whole was lowered by an unsatisfactory score (1.0) for the Microfinance Sector Support project (PASMIF which experienced many constraints that impeded the achievement of the expected results). In governance and public management, gender equality is highly satisfactory, except for CIDA’s support for the Office of the Auditor General.

2.0 Major Findings: Management Factors

2.1 Coherence – Satisfactory

Analysis at the Program Level

Coherence analysis focuses on two dimensions: (i) external coherence, which assesses the nature, quality, and scope of Canada’s coordination with technical and financial partners active in Mali; and (ii) internal coherence, which focuses on coordination among CIDA’s various delivery mechanisms and among Canadian development stakeholders, including government, non-governmental, and private sector.

External Coherence

The program achieved excellent synchronization with technical and financial partners by using mechanisms designed by donor countries and implemented in consultation with the Government of Mali. The program also contributed to their effective facilitation and operation. Another factor that greatly facilitates external coherence is CIDA’s field presence with a qualified and better-staffed technical team than other donor countries have.

The consultation process put in place by donors in the education sector creates the conditions for a satisfactory level of coherence between donor-country representatives and the Government of Mali. Donor country representatives liaise on a regular basis with Canadian support agencies and their specialists.

Support in establishing school boards in territorial communities should make it possible to improve the program’s external coherence in operational terms. This process is still in its early stages, and consolidating its gains should be a programming priority in the coming years.

The health sector is also organized to facilitate external coherence by enabling partners to participate in annual regional evaluation and planning exercises and the program participated in these activities when they arose.^{Footnote 20} This helps resolve communication challenges between government and donors on one side and project executing agencies on the other.

As leader of the microfinance thematic sub-group, Canada’s involvement mainly entails supporting Mali’s national microfinance development strategy, co-funded with Denmark, to ensure total coherence with Malian policy. However, 2007-2011 saw little participation in the thematic sub-group by other donors, because some wanted to maintain their own approach to microfinance development. The lack of coherence is a problem that hinders the success of international cooperation in this sector. Finally, in the food security sector, Canada has increasingly used delegated cooperation (i.e. where CIDA channels funding to a partner government via another donor who executes the joint PBA initiative) to improve harmonization with other donors.

Internal Coherence

Long-term engagement in several related sectors,^{Footnote 21} evolved from the program’s practice of consolidating project achievements, and of reflecting changing needs among national institutions. In education, for example, in the Kayes region, a decentralized school management pilot project allowed the development of a replicable model of school management for local communities. When the textbook repair pilot project proved convincing in the Kayes region, the program replicated it throughout Mali in a Canada-funded project. In health, a diagnostic study was conducted in several communes in the Ségou and Kayes regions, which helped build the capacities of health facilities in the Kayes region, giving priority to reproductive health. In food security, there are fewer such examples, since Canada’s interest in this sector is more recent. However, given the positive outcome of the PACCEM project, the model could be replicated in other regions.

There were synergies between initiatives in the education and health sectors. This was one of the objectives of the sector budget support capacity-building project for the health, education, and public-sector financial management sectors in general. In the health sector, there has been dialogue among managers of the Clinical and Community Education Development project (DÉCLIC) and Strengthening Decentralized Health Systems projects, which allowed more effective targeting of training. Synergy can also be identified between the Teacher Training Support project (PAMOFE) and the Decentralized Management of Basic Education Support project (AGDEF) in the education sector. Canadian support agencies combined their efforts to train and raise the awareness of decentralized managers concerning their mutual roles and responsibilities. Moreover, the program created networks throughout Mali to promote dialogue and experience-sharing among mechanisms in the various regions of intervention. The integration promoted by the networks contributes to the organization of local delivery of basic social services, and strengthens dialogue between civil society and government.

The 2007 evaluation^{Footnote 22} assessed the interaction among CIDA’s delivery mechanisms, and the coherence of Partnerships with Canadians Branch (PWCB) initiatives with the country development programming framework (CDPF). Since then, the Program Support Unit (PSU) has hired a full-time consultant to monitor projects funded by PWCB. Moreover, there are now regular meetings between the local teams of non-governmental organizations and representatives of the embassy and the PSU, coordinated and organized by the civil-society advisor. In addition, PWCB seeks to ensure that when Canadian organizations apply for funding of proposed initiatives for Mali, they do so in line with the CDPF. PWCB also published a report^{Footnote 23} that broadly outlines CIDA’s Mali programming, including a sectoral breakdown of its investments, the list of the principal organizations, and a brief description of the key projects. Based on the information provided in this report, there is a generally satisfactory degree of alignment between PWCB programming and the CDPF.

Analysis of the Project Sample

Scores are satisfactory for most of the sectors, although some PWCB projects (Right to Play for example) were not well known to the CIDA field team. Another project, (the Structured Response Program, PSI) appears not to be complementary with other similar Canadian funded projects.

In education, the high-scoring textbook support (4.3) established links with all the education projects and demonstrated strong coherence within the sector. The Children’s Millennium Program, on the other hand was implemented practically in a vacuum by Plan International, earning the low score of (1.6).

In governance, CIDA relied on the technical assistance components of the Institutional Development Program Support, and the Sector Budget Support Capacity Building Project (PRECABS), to create dynamic links among these activities, also extending to projects supporting the Office of the Auditor General (BVMG) and the justice system (PRODEJ). Since some technical assistance components could not be put in place during the period as planned, interaction among projects remained limited.

2.2 Efficiency – Satisfactory

Analysis at the Program Level

The notion of efficiency focuses on program performance as measured by i) decentralization and decision making, ii) human resource management, and iii) management costs in relation to investments.

The program decentralized its team in 2006, stationing the program Director in the field. There are two assistant managers (one at Headquarters and one in the field), and a program team between Headquarters and the field. A Program Support Unit, comprised of a team of experienced specialists, primarily Malian, adds further support in the field. The staffing arrangement performs especially well in aid coordination. The field team also provides an efficient and reliable source of knowledge and information about Mali to guide the choice and design of initiatives at headquarters.

Decentralization to the field has not been accompanied by delegation of decision-making authority. Procedures remain the same, regardless of the program manager’s physical location.^{Footnote 24}

CIDA’s approval times have affected the conduct of planned activities. There has been a lack of synchronization between the approval of budget or financial support, and the hiring of technical assistance meant to help implement this support. This has occurred in a few instances, such as in the multi-sectors budget support ($140 million) and the Institutional Development Program. In these cases, the team had to resort to short-term technical assistance projects to compensate for the time required to select and hire the ultimate executing agencies.

As shown in Table 6, from 2007 to 2011, there was an increase in the human resources that Geographic Programs Branch allocated to the Mali Program.

This increase was approved for two reasons: 1) the field team is becoming more and more involved in policy dialogue activities; and 2) the volume of disbursements increased from $65.1 million in 2004-2005 to $101.98 million in 2010-2011.^{Footnote 25}

The last column of Table 6 shows that the program’s cost-effectiveness, measured in bilateral aid dollars delivered per person-year has remained stable. It is notable that many of the employees at Headquarters have been working with the program for many years, which also contributes to efficiency.

Table 7 compares the Mali Program with other programs in African countries of focus where management is also decentralized. The table shows annual aid disbursements, annual Program Support Unit (PSU) costs, and the number of program team members, at Headquarters and in the field.

The figures are averages for two successive periods (2004-2005 to 2006-2007) and the current evaluation period (2007-2008 to 2010-2011). These data make it possible to calculate PSU costs as a percentage of disbursements, as well as the annual amount of aid managed per person-year, at Headquarters, in the field, and for the team as a whole.

Table 7 shows that:^{Footnote 26}

• Mali ranks third for the size of annual disbursements between 2007 to 2011, up from fourth in the previous period.
• Mali had the smallest average field team during the evaluation period.
• Since the previous period, Program Support Unit (PSU) costs have decreased considerably as a percentage of aid administered (from 2.24 percent to 1.52 percent). Mali’s PSU costs rank fourth.

Disbursements managed per person-year (Headquarters and field combined) are only slightly lower than those of the highest performing program, Ethiopia. The average amount of aid managed per person-year in Mali rose significantly from $3.6 million in 2004-2007 to $5.04 million in 2007-2011. The Mali Program has one of the lowest ratios of field personnel to annual disbursements among African countries with decentralized program management. CIDA has not only ensured an effective presence in the various aid effectiveness mechanisms, but has often played a lead role, especially in policy dialogue. The Agency has been able to mobilize expertise in key sectors of its program, especially health and education. CIDA has thus been able to ensure an effective transition from a traditional type of program to one where budget support plays a substantial role, and which requires the development of new skills.

Various technical and financial partners stated that CIDA continually had access to adequate resources to participate in thematic groups in education, health, and other sectors throughout the 2007-2011 period.

Table 7: Comparative Cost Analysis, 2004-2005 to 2010-2011

Analysis of the Project Sample

Ratings were generally satisfactory across the sectors. Opportunities to improve decentralisation, local ownership, and flexibility to respond to changing demands in school feeding were noted.

2.3 Management Principles - Highly Satisfactory

Analysis at the Program Level

The evaluation assesses the program’s adherence to three principles of the 2005 Paris Declaration on Aid Effectiveness: (1) local ownership; (2) alignment and compliance with national policies, strategies and plans, and integration with national systems; and (3) harmonization of mechanisms for coordination and cooperation among donor countries and a shared approach to policy dialogue.

Ownership

Between 2007 and 2011 CIDA shifted decisively to support Malian ownership of the country’s development and discretionary deployment of Canada’s assistance funding. The shift was particularly marked for the program’s financial management of education, health, and public-sector programming, where Malians deployed aid to support national programs. CIDA’s provision of general budget support since 2009 likewise supported Malians in implementing their own development policies for the country.

Alignment

In 2010, Canada was one of 11 donor countries to sign a new budget support framework arrangement aimed at integrating aid delivery with Mali’s budgeting and accounting mechanisms. Since 2006, CIDA’s Mali country program had steadily increased the percentage of its fundings disbursed through national systems, peaking at 88 percent in 2010-2011.^{Footnote 27}

Budget support in Mali proved a particularly effective and influential mechanism for supporting Mali’s national poverty reduction strategy, offering sizable and more predictable funding. Budget support by definition promotes alignment, harmonization, and reduces transaction costs. For Mali, budget support favoured the establishment of frameworks for policy dialogue, and its conditionalities have decisively influenced the implementation of decentralization and deconcentration. By its impact on economic growth, inflation control, and greater availability of education and health services, budget support helped achieve sustainable development results throughout Mali.

Harmonization

CIDA achieved good performance in harmonization by helping to design mechanisms for donor coordination and policy dialogue with the Government of Mali. CIDA also participated actively in these mechanisms, playing a leadership role from 2009 to 2011. Canada is regarded as a champion of harmonization in Mali.

In social sectors, policy dialogue occurred within the comprehensive frameworks established by the Education Development Program (PRODEC), the Education Sector Investment Program (PISE), the Social and Health Development Plan (PDSS), the Health and Social Development Program (PRODESS), and through specific policies such as the national textbook policy.

The Quality and Management thematic group provides a forum for dialogue in the education sector, and a dozen thematic sub-groups do likewise in the health sector. Canada is actively involved in at least three of these sub-groups. Canada will lead the health thematic group starting in 2012. Canada is also involved in the Global Partnership for Education, managed by the World Bank.

Analysis of the Project Sample

In health and education, most activities are implemented in coordination with other donors active in the sector, but there remain a few challenges. There is a need to strengthen the capacity of decentralized (or community) entities responsible for education, and further efforts in the Malaria Management project will be required to ensure complementarity between a strategy to support community health centres with the new community-based essential care policy, so that the project’s achievements are not lost.

Agriculture and rural development projects generally achieve satisfactory performance in terms of ownership, alignment, and harmonization. The Municipal Partnership Program, lowered the otherwise satisfactory performance of the governance sector because it performed poorly in all three management related criteria of ownership, alignment, and harmonization.

2.4 Performance Management - Satisfactory

Analysis at the Program Level

This criterion addresses results-based management, risk management, monitoring and evaluation activities, and mutual accountability mechanisms.

A new Country Strategy was developed in 2008^{Footnote 28} and revised in 2009.^{Footnote 29} Before this period, the program made use of a performance measurement framework developed in 2005 (revised in 2006), along with a logic model. The 2009 Country Strategy includes a table of expected outcomes, including examples of immediate outcomes and outputs for each of the three areas of focus. This version focused solely on the geographic program under the new CDPF for 2009-2010 to 2014-2015. A more complete version, dated May 2010, covers the same period.

Ideally an evaluation assesses program performance and future direction against a stated performance measurement framework (PMF). However, data in this case were available for only a few indicators, and there were questions about reliability. Mali’s statistical system is being improved with the support of several donors. As the quality of data improves, the program’s PMF should be revised based on available data, and thus become a more useful management tool.

There was not a complete, documented risk analysis for the evaluation period. The 2009 strategy provided a brief risk assessment for the 2009-2014 period.^{Footnote 30} Three development risks are described: i) the fragility of Mali’s political, social, and economic situation, leading to possible conflicts and failing governance, ii) weak institutional capacities, and iii) poor choices of programming terms and conditions, possibly resulting in missed opportunities. A summary description of program characteristics that mitigate these risks is also provided.

An “Integrated Risk Profile, Mali Program, 2010-2011”, prepared in August 2011, did not apply to the period under review, being intended instead for the 2010-2015 programming period. That profile favours risk-planning for the achievement of results, and is less focussed on long-term sustainability. It addresses corporate risk categories (operational, financial, developmental, and reputational risks) and the mitigation measures proposed appear sound. A risk tolerance analysis is well considered. Each risk is thoroughly analyzed in terms of possible causes, consequences, impact, likelihood, and appropriate responses.

Attention should be paid to the following issues in reviewing the risk profile for the next programming period:

• Risks that specifically affect the sustainability of initiatives, and mitigation measures, should be addressed.
• A more thorough inventory of risks should include non-fiduciary financial risks, climate change-related environmental risks that cannot be categorized as disasters, and institutional weakness.
• The risk profile should be regularly updated, perhaps yearly, when the program team conducts its annual review. This could result in a re-evaluation of likely impacts and likelihoods, and a rereading of the analysis of each risk based on changes in the environment.

Analysis of the Project Sample

There were varying levels of results management and monitoring across the project portfolio, rated as “somewhat satisfactory” in the education sector, to overall “satisfactory” in other sectors. In some cases, security conditions in the north impeded adequate monitoring. In others, RBM practices were adopted late in the cycle, or inadequate risk analysis was undertaken. By the same token, multidonor umbrella arrangements such as in the Health and Social Development Program (PRODESS) were sometimes quite effective. “Highly satisfactory” practices were also observed in the Development International Desjardins monitoring and evaluation system.

3.0 Conclusions, Lessons and Recommendations

3.1 Conclusions

This evaluation resulted in a positive overall assessment of CIDA’s Mali Country Program between 2006 and 2011.

Relevant program – CIDA’s Mali Country Program strategy coincided with and reflected the aims of Mali’s 2007-2011 growth and poverty reduction strategy paper (GPRSP), undertaking investments very relevant to Mali’s needs. The program also reflected CIDA’s priorities, and for at least fifteen years maintained a continuous focus on the same key, long-term themes: children and youth, food security, and sustainable economic growth.

Good practices in development assistance – The use of program-based approaches and budget support reflected emerging best practices in aid effectiveness. The CIDA Mali Program employed good mechanisms for aid coordination and policy dialogue, which were positively recognised by other donors. One challenge though is to ensure efficient collaboration among stakeholders in the field.

Results – CIDA made noteworthy contributions to education, health, governance and public management reform, equality between women and men, and strengthening of civil society. Through policy dialogue, the program helped decentralize education and health programs to targeted regions.

Uneven attention to crosscutting themes – The program promoted equality between women and men, achieving tangible results. There is, however, a perception among partners that CIDA is less proactive in promoting gender equality than it has been in the past. There was increased attention to environmental sustainability in recently planned and launched initiatives, although for the balance of the period reviewed here, the program’s attention to this issue was minimal.

Efficiency – Since CIDA decentralization, the program has realized various gains in efficiency. The average amount of aid administered in Mali per staff person-year increased. Cost effectiveness is second only to Ethiopia in a sample comparison of similar CIDA decentralized programs in Africa. The comparison also indicated that the program has one of the lowest ratios of field staff to total disbursements. Overall, high quality human resources contributed to CIDA’s leadership in key areas.

Program sustainability remains a challenge – The progress noted above promoted sustainably by building the Government of Mali’s capacity. However, external factors may undermine some of the gains: central and decentralized institutions remain weak in terms of institutional and financial capacity, strong population growth threatens economic objectives, corruption persists despite progress; and unrest has come to northern Mali.

3.2 Lessons Learned

Merits of small-scale post-project support – Post-project support strongly contributes to the maintenance of results achieved. Low-cost post-project support makes it possible to protect the results of an investment. This is especially true in high-risk environments.

Merits of maintaining a diversity of aid delivery approaches – A variety of approaches in country programming guarantees that aid will be flexible and adaptable to changes and constraints in the country of intervention.

Effectiveness of institutional partnerships in governance – The Office of the Auditor General of Mali support project demonstrated the strategic importance and effectiveness of institutional partnership. The Auditor General of Canada and the Auditor General of Mali have developed a partnership that has been an effective tool in building Mali’s capacities.

Effectiveness of sustaining engagement over the long-term – Supporting organizations over the long-term can play a significant role in professionalizing and building their capacity. Notably, two organizations that CIDA has supported for over a decade were recently recognized nationally and internationally for their growth, success, and productivity.

Need for additional resources to predict (and plan) for national crises – Few, if any, donors predicted the political change that took place in Mali in 2012. Though the CIDA Program identified a possible political takeover in its risk analysis, it underlined that additional resources and tools to more thoroughly analyse risk scenarios, and make relevant plans and adaptation measures, would be useful for the Mali Program as well as other CIDA Country Programs operating in countries in conflict.

In addition, making a transition to the program-based approach is a significant factor in mitigating the risk of non-sustainability. This means making CIDA initiatives an integral part of a coherent sectoral program, with a budgeting, monitoring, control, and reporting process.

3.3 Recommendations

The approach to performance management of the Mali program has been evolving, with a number of best practices in ownership, alignment, and harmonisation. There is an opportunity to further improve it by:

• Maintaining up-to-date risk assessment and mitigation strategies.
• More systematic monitoring and evaluation, informed by risk assessment.

While there has been leadership on gender issues, attention to environment has been less marked over the evaluation period. The program should:

• Carefully consider an appropriate emphasis on gender equality, in light of its past leadership role.
• Consider increasing the emphasis on environmental sustainability given the challenges to sustainable poverty reduction that environmental threats represent.

Annex 1: Evaluation Approach^{Footnote 31}

Evaluation Purpose, Objectives, and Scope

CIDA is subject to the Federal Accountability Act and the 2009 Treasury Board Policy on Evaluation, which make it an obligation to evaluate all programs within a five-year period. In accordance with the Treasury Board and CIDA’s evaluation policies, the evaluation had the following learning and accountability objectives: (1) To take stock of the results achieved by the program over five fiscal years (2006-2007 to 2010-2011). (2) To assess the program’s overall performance in achieving these results. (3) To document and disseminate findings and lessons learned, and formulate recommendations to improve performance.

The following objectives are pursued in evaluating Canada’s 2006-2007 to 2010-2011 Mali program:

• to report on the results achieved through activities by the Canadian International Development Agency (CIDA) during the evaluation period;
• to analyze the program’s overall performance, based on the following criteria: relevance, effectiveness, sustainability, crosscutting themes, coherence, efficiency, management principles, and performance management;
• to assess the performance of the various delivery channels (bilateral, multilateral and Partnership with Canadians Branch programs) and delivery mechanisms (program-based approaches, as well as directive and responsive projects);
• to document lessons learned from Canadian cooperation’s experience in Mali during the evaluation period, and to make recommendations to improve the performance of the country program’s current or future strategy.

Given the continuity of programming between the 2005-2010 and 2010-2015 periods, the country program evaluation focuses on the following three areas:

• children and youth (health and education);
• food security (agriculture, microfinance);
• governance and strengthening of the public sector (public-sector financial management, internal and external control institutions, and administration of justice).

General and sector budget support were considered on the basis of the Joint Evaluation of Budget Support Operations in Mali 2003-2009 (JEBSO), conducted in 2010-2011.^{Footnote 32} Special attention was paid to synergy, coherence, and complementarity between budget support and other programming components.

The evaluation incorporated the crosscutting themes of gender equality and the environment. Special attention was paid to Canada’s support in strengthening the Government of Mali’s decentralization process.

Using the findings of the Joint Evaluation of Budget Support Operations, the evaluation also examined the program’s overall performance. The evaluation analyzed the contributions of the various mechanisms; the degree of coherence and synergy among delivery channels, areas of intervention, projects, and Canadian stakeholders in Mali; and the strengths, weaknesses, and relative efficiency of the various delivery channels. Special attention was paid to the policy dialogue process, as well as mechanisms for harmonization and coordination among technical and financial partners, and between them and the Government of Mali.

Evaluation Methodology

The evaluation followed CIDA’s methodology and terms of reference for program evaluations. The evaluation was not expected to undertake detailed “project” evaluations, but to rely largely on secondary sources. The evaluation undertook a document review and a desk study. Documentary evidence was supplemented by interviews with CIDA staff and managers at headquarters, and a field mission to meet a range of stakeholders. Interview guides were prepared to capture evidence and perspectives at the project and program levels. Multiple sources of information and methodologies were used to generate insights and facilitate triangulation of information to validate evidence.

The program evaluation focused on three levels of analysis: program, programming area, and projects. The evaluation covered the three programming areas: (i) social development, (ii) economic development, and (iii) governance and strengthening of the public service. The evaluation also covered the crosscutting themes of gender equality and the environment. The work was done by a team of three Canadian specialists (two acting as co-directors of the evaluation team), and three Malian specialists.

To analyze the results obtained, the evaluation reviewed a representative sample of projects reflecting the various types of investment and delivery mechanisms in each area considered. The sample is made up of initiatives planned and implemented over a sufficiently long period, including all or some of the years during the evaluation period. The sample was based on the following key selection criteria:

• Sectoral and thematic initiatives: The selected activities account for a significant proportion of the value of the programming area. They are representative of the relative weight of the sector or theme within the programming area.
• Delivery mechanisms: Based on the relative weight of each mechanism within the program, activities are distributed among (i) directive and responsive bilateral projects; (ii) targeted multilateral activities; and (iii) targeted PWCB activities.
• Evaluability: The selected activities are significant enough in terms of budget allocation to project an accurate picture of the program. Information sources are available and accessible at a reasonable cost.

A sample, totalling more than 30 projects, was selected and reviewed for this evaluation. However, five successive textbook support projects, three justice development support projects, and two Office of the Auditor General of Mali support projects were consolidated into single projects and evaluated as a whole. Given the nature and sequence of these projects, they can be regarded as ongoing activities. The final sample comprises 24 projects, including 15 Geographic Programs Branch (GPB) projects, 6 Partnerships with Canadians Branch (PWCB) projects, and 2 Multilateral and Global Programs Branch (MGPB) projects.

The following criteria and sub-criteria were used:

What has been accomplished?

• Relevance: Extent to which the development program's objectives reflect Mali's needs and CIDA's policies and strategies
• Effectiveness/results: Extent to which the development program's objectives have been achieved, or are being achieved, considering their relative importance
• Sustainability: Maintenance of the development program's benefits after a major development cooperation activity is completed
• Crosscutting themes: Extent to which gender equality and the environment are integrated into the program framework

Why and how were the expected results achieved?

• Relevance: External coherence—complementarity and use of synergy with the activities of other technical and financial partners—and internal coherence—coordination among CIDA's various delivery mechanisms (Geographic Programs Branch, Partnerships with Canadians Branch, and Multilateral and Global Programs Branch programs) and among Canadian development stakeholders, including governmental, non-governmental and private-sector stakeholders.)
• Efficiency: Extent to which resources (funds, expertise, time, and so on) are translated into results, based on optimum allocation of resources
• Management principles: Degree of adherence to the following principles of the Paris Declaration: local ownership, alignment, and harmonization
• Performance management: Integration of results-based management and the principle of mutual accountability in program management, including mechanisms and tools developed and used.

The team of Canadian evaluators collected data in Canada, at CIDA Headquarters, and from Canadian stakeholders, Canadian support agencies, NGOs, and monitors involved in implementing the program. Supported by three Malian experts, the team collected the additional data it needed in Mali, during a three-week mission from September 24 to October 14, 2011.

Four data collection methods were used: (i) reviewing relevant literature; (ii) conducting semi-structured individual or focus group interviews with key program stakeholders, based on interview guides previously prepared and approved by CIDA; (iii) holding round-table discussions with representative of relevant groups (civil society, media, microfinance, technical and financial partners, Canadian and Malian NGOs) and with the program's field team; (iv) making a limited number of field visits to some of the projects in the sample.

The evaluation matrix and sample project fact sheets served as a basis for analyzing documentary information. Analyses were complemented by interviews, group discussions, and project field visits. Analyses focused on:

• descriptions of projects in the sample, their objectives, and results expected and achieved;
• information provided by documentary sources, and information gathered during meetings in Canada and in Mali, and during field visits, regarding evaluation issues and questions;
• the performance of projects in the sample, with qualitative and quantitative assessments based on evaluation criteria, types of investment, and delivery channels.

Standard tools were used to ensure the validity of the analysis, findings, and conclusions. Where information was collected from various sources, it was compared and triangulated wherever possible.

Evaluation Challenges and Limitations

The evaluation faced a number of constraints and limitations that affected the process of collecting and analyzing data.

• Evaluation period: The evaluation period (2006-2007 to 2010-2011) mainly covers the 2000-2010 Country Development Program Framework (CDPF), but also the beginning of the 2010-2015 CDPF, whose planning began in 2008. (In this regard, the first two draft strategies are dated 2008 and 2009). The evaluation considered both the directions of the former CDPF and those of the new CDPF, especially to reflect the continuity observed in the directions and implementation of these two programs.
• Budget support and activities based on the program- and project-based approaches: This evaluation used the Joint Evaluation of Budget Support Operations (JEBSO), finalized in 2011, as its source for evaluating budget support.^{Footnote 33} The Joint Evaluation used the same methodology as that of the OECD Development Assistance Committee, which is compatible with the methodology used here. The evaluators thus had no difficulty incorporating the Joint Evaluation's findings and recommendations into the present report. However, some differences in data were observed between this study and that of the other sources.
• Limitations due to the sample: The evaluation sampled a representative crossection of bilateral initiatives, but fewer for Multilateral and Global Programs and Partnerships with Canadians Branches. This is especially true since PWCB-funded projects are generally multisectoral, and results are generally not distributed by sector.
• Unit of analysis: programming area vs. sector: The program supports Malian health, education, microfinance, and public service reform policies. The evaluation must thus consider Canada's contribution to achieving the expected results of these policies. The CIDA program defines programming areas at an aggregate level that does not always correspond to the sectoral framework that Mali uses to outline objectives and results achieved.
• Canadian contribution and attribution: As noted in the terms of reference, the program evaluation focuses mainly on examining Canada's contribution. However, Canada accounts for only a proportion of contributions to program-based approaches and projects. The attribution of results thus can only be seen as Canada's contribution to achieving the results of program-based approaches as a whole, in a context of partnership with the Government of Mali and other technical and financial partners.
• Availability and quality of data: The Government of Mali's data are not always complete and consistent, since they are based on a national statistical framework that is still being developed. This was a constraint in presenting the results of initiatives using program-based approaches. It should also be noted that, when this report was prepared, some CIDA financial data were not available for the last year of the period.
• Although the sample is quite comprehensive in its coverage of the program's disbursements, care must be taken in drawing inferences at the program-level based on the project sample. The types of evidence available at the program-level are largely qualitative rather than quantitative, and rely upon the review of program documents, past evaluations, studies prepared by development partners, and interviews with counterparts and stakeholders.

Nevertheless, the evaluators deem that these constraints had only a marginal impact on the reliability of the analysis. They do not detract in any way from the validity of the principal findings, conclusions, and recommendations. In the table below, program and project level indicators are listed.

1. Relevance

Program-Level Indicators

Overall relevance of the program including policy dialogue in relation to:

• The partner country's development context and needs
• CIDA's policies and the Government of Canada's interests and foreign policy objectives
• The evolving nature of Canadian and CIDA policy environment

Project-Level Indicators

Relevance in relation to:

• Country needs: Country development plans (PRSP), Millennium Development Goals (MDGs)
• CIDA policy: Poverty Reduction, Sustainable Development
• Canada's priorities: CIDA Country Strategy (CDPF), foreign policy objectives (DFAIT)

2. Effectiveness

Program-Level Indicators

Overall extent to which the program in achieving (or is expected to achieve) its outcome-level results in relation to:

• Poverty reduction
• Strategic objectives for the country (CDPF) or Canadian foreign policy
• Sector/thematic or crosscutting objectives

Project-Level Indicators

Effectiveness in achieving results at the outcome level (and impact if available based on secondary data) in relation to:

• Poverty reduction
• Project/sector/institutional objectives
• Cost-effectiveness per result unit

3. Sustainability

Program-Level Indicators

Overall program sustainability in relation to:

• Assessment of the partner country's financial capability (ODA/GNP, ODA/Budget)
• Partner country's policy environment and reform context (political, social, economic, administrative, environmental)
• Predictability of program's ODA resources (3-5 year forward information on planned aid to partner countries) per Accra Agenda

Project-Level Indicators

• Time needed to attain the results (life of the project: 3-5 years)
• Institutional capacity to maintain the results (after the project)
• Financial capacity to sustain the results (after the project)
• Policy environment
• Ownership/commitment

4. Crosscutting Themes

Program-Level Indicators

Treatment of each crosscutting theme (gender equality, environmental sustainability, or other) at the program level in relation to:

• Place given to the CCI in the CDPF
• Existence of a strategy regarding the CCI
• Existence of monitor or dedicated specialist
• Existence of a Fund or dedicated resources
• Role of CIDA/Canada on this issue in the context of donor coordination

Project-Level Indicators

Treatment of each crosscutting theme (gender equality, environmental sustainability, etc.) in relation to:

• Policy dialogue
• Preliminary analysis (disaggregated data)
• Implementation of activities
• Results achieved: (i) Institutional strengthening; (ii) Improving quality of life
• Resources invested (budget)

5. Coherence

Program-Level Indicators

Coherence at the program level in relation to:

• Cooperation channels (bilateral, multilateral, partnership)
• Development actors (government departments, NGOs, international orgs)
• External coherence in terms of consistency with programs of other donors (donor coordination)
• Coherence with Canada's whole-of-government efforts (including program's policy dialogue, leverage, and leadership played by Canada)

Project-Level Indicators

Coherence in relation to:

• Internal coherence within CIDA (delivery models/mechanisms and channels)
• External coherence in the context of international efforts (donor coordination)
• Coherence in the context of Canadian whole-of-government efforts

6. Efficiency

Program-Level Indicators

Cost-efficiency at program level in relation to:

• The appropriateness of the program's delivery modalities relative to the partner country's context
• Delivery models (directive, responsive, PBAs)
• PBA delivery mechanisms Pooled Funding, Other)
• Program's O&M/G&C compared to other comparable CIDA Programs
• Benchmarking with other donors (staff, O&M costs, decision making, knowledge related activities)
• Strengthened field presence

Project-Level Indicators

Cost-efficiency in relation to transaction costs:

• Efficiency in use of human resources
• Efficiency in use of financial resources
• Time needed to approve, manage, and monitor

7. Management Principles

Program-Level Indicators

Overall performance at the program level in relation to Paris Declaration Principles of:

• Ownership (developing countries set their own strategies for poverty reduction, improve their institutions, tackle corruption)
• Alignment (donor countries align behind these objectives and use local systems)
• Harmonization (donors coordinate, simplify procedures, share info to avoid duplication)

Project-Level Indicators

Performance in relation to Paris Declaration principles of:

• Ownership (developing countries set their own strategies for poverty reduction, improve their institutions, tackle corruption)
• Alignment (donor countries align behind these objectives and use local systems)
• Harmonization (donor coordinate, simplify procedures, share info to avoid duplication)

8. Performance Management

Program-Level Indicators

Performance management at program level:

• Results Management (per CIDA guidelines)
• Roll up & measure development results
• Apply lessons learned & best practices
• Follow-up on recommendations of previous evaluations
• Risk management (per CIDA guidelines)
• CIDA monitoring & evaluation (M&E)
• Joint M&E activities with other donors (or division of labour between donors)
• Mutual accountability mechanisms (or activities to build local M&E capacity)

Project-Level Indicators

Performance management:

• Results-Based Management (per CIDA guidelines)
• Risk management (per CIDA guidelines)
• CIDA monitoring & evaluation (M&E)
• Branch-led M&E activities at the project/sector/institutional level
• Joint M&E activities with other donors (or division of labour between donors)
• Mutual accountability mechanisms (or activities to build local M&E capacity)
• Timely corrective action

Annex 2: Mali Context and CIDA Program Description

Mali had an average population growth rate of 3.6 percent between 1998 and 2009.^{Footnote 34} Today, Mali is home to 14.5 million people;^{Footnote 35} 68 percent are under 25^{Footnote 36} and 78 percent^{Footnote 37} live in rural areas. The District of Bamako is home to more than half of Mali's urban population (55 percent). In 2011, Mali ranked 175^th out of 187 countries on the UNDP's Human Development Index. According to UNDP statistics, more than half of Mali's population (51.4 percent)^{Footnote 38} lived below the monetary poverty line ^{Footnote 39} in 2006. According to a recent survey,^{Footnote 40} however, the incidence of monetary poverty totalled only 47.4 percent in 2006 and decreased to 43.6 percent in 2010. Nevertheless, regardless of the sources from which statistics are obtained, Mali remains one of Africa's poorest countries.

High population growth is explained, not only by a high birth rate, but also by migratory movements. This high population growth hinders Mali's development. The benefits of growth must be distributed among a larger population. In addition, more government funding must be transferred to social sectors, at the expense of investment in economic sectors that can strengthen growth.

From 2007 to 2011, Mali's gross domestic product grew by an average of 5.0 percent. This rate is higher than the 3.4 percent average for member countries of the West African Economic and Monetary Union (WAEMU). However, it is lower than Mali's 7 percent objective for the period.

Democratic institutions and reform- Mali became a democratic country in 1991. Since then, there have been four successive presidential and legislative elections. A fifth is to be held in 2012. In recent years, progress has been made in strengthening democracy and respect for human rights. For example, an opposition has emerged in the National Assembly. The institutional reform support committee was formed in 2008. A constitutional amendment has been proposed and will be put to a referendum early in 2012. An Office of the Auditor General has been created with Canada's support. Finally, a national gender policy has been adopted. However, some concerns remain: a dysfunctional justice system, little dialogue with civil society, and a lack of government resources to strengthen the full exercise of social, economic, and cultural rights. A Personal Status and Family Code was recently enacted, formalizing the legal inferiority of women and girls, following pressure from religious groups. This shows that there are still weaknesses in applying rights and implementing the basic mechanisms of democracy.

Public-sector financial management: The Government's public-sector financial management improvement and modernization plan has led to tangible improvements, attested by the 2009 evaluation of the plan, International Monetary Fund reports, and the Public Expenditure and Financial Accountability report.^{Footnote 41} The Institutional Development Program ? IDP-1 (2006-2009) and IDP-2 (2010-2013) ?streamlines the organizational structure and framework of the various ministries. The IDP lays the foundations for a human resource management system, including processes to encourage effectiveness and promotion on the basis of merit and skill, so that central and deconcentrated governments are staffed with qualified and motivated personnel. The IDP also helps to increase transparency in transactions within the Government, and between the Government and citizens.

A worrisome climatic trend – Mali's environmental situation is characterized by a tendency for agroclimatic zones to shift south. The Saharan zone is advancing toward the Sahel zone, which is pushing the Guinean zone farther south, reducing its area in proportion to Mali's territory. This has resulted in a general trend toward increasing drought and irregular rainfall, causing more than 15 floods that affected at least 10,000 people between 1980 and 2007. Cropland and produce were lost. Soil was eroded. Sand accumulated in waterways, ponds, and lakes.^{Footnote 42} Environmental degradation has a seriously adverse effect on Mali's economy, and on the income of the poorest: sedentary or nomadic herders, fishers, and farmers engaged in irrigated or waterlogged cultivation on the banks of ponds and waterways. Environmental degradation also affects Mali's food security.

Growth and poverty reduction strategy paper – The evaluation period corresponds roughly to the 2007-2011 outlook of the Malian growth and poverty reduction strategy paper (GPRSP). The GPRSP focused special attention on the recovery of productive sectors and the strengthening of public-sector reforms, based on the promotion of private-sector initiatives and administrative decentralization. Through public-sector reform, GPRSP II also aimed to strengthen good economic and political governance, to create and promote sustainable employment, to develop access to basic social services, and to combat HIV/AIDS.

2. Development Assistance

According to data concerning official development assistance flows, Mali receives very significant assistance each year. Canada is the fourth lead donor among member countries of the OECD's Development Assistance Committee (DAC). From 2005 to 2009, Canada disbursed US$298 million, representing 12 percent of the DAC total.^{Footnote 43}

In Mali, as in other West African countries, foreign aid represents a large proportion (about 13 percent) of the GDP. Foreign aid funds about three quarters of government investments.

A key policy document – The Joint Assistance Strategy (JAS) is the result of a change in the terms and conditions for assistance to Mali, to adapt them to the principles of the Paris Declaration of March 2005. The 2008-2011 JAS aims to define ways and means to support Mali in achieving its growth and poverty reduction objectives.

To achieve this objective, JAS is based on three pillars:

• Adoption of new ways of collaborating: (i) new forms of programming, more closely aligned with national strategies; (ii) more predictable assistance; (iii) closer alignment with Malian systems and procedures; (iv) strengthened coordination of activities; and (v) more rational complementarity and division of labour.
• Implementation of a more consistent funding strategy for the growth and poverty reduction strategy paper: (i) concentration of financial support from technical and financial partners in areas that are priorities for the Government of Mali; (ii) aid delivery mechanisms that are more in line with aid effectiveness principles; and (iii) greater attention to public-sector financial management reform.
• Higher quality of policy and technical dialogue with the Government of Mali, based on a joint monitoring framework, joint risk management, and respect for the principles of partnership and national ownership.

Moreover, donor countries have established consultation frameworks to coordinate their activities better, and to exchange both general and sectoral information. Table 2 represents the organization of this coordination mechanism, which is designed to be flexible. Groups and sub-groups can be modified to reflect changing priorities, institutional changes within the Government of Mali, and the interest and availability of donor countries. The coloured boxes in the table indicate Canada's membership in thematic groups and sub-groups.^{Footnote 44} Between 2008 and 2011, Canada was an active member of eight thematic groups, the gender and environment crosscutting groups, and four ad hoc groups. Canada also led the thematic group on justice and combating corruption, the microfinance sub-group and, until late 2010, the gender group. The table does not indicate Canada's significant responsibility as a member of a multi-donor aid reform group (2009-2011).

Table 2 Structure of thematic groups and sub-groups showing Canada's presence

1. Infrastructure development and strengthening of productive sectors

Priority areas of intervention

1. Rural development and food security
2. Infrastructure development
3. Environmental protection and natural resource management

Thematic groups and subgroups

1. Agrocultural and rural economy
   1. Livestock production and fisheries
   2. Niger Office
   3. Food Security
2. Infrastructure development
   1. Several groups and subgroups
3. Environment and climate change
   1. Environment
   2. Climate

2. Continuation and consolidation of structural reform

Priority areas of intervention

1. Consolidation of public service administrative reform Promotion of democratic governance and civil liberties
2. Financial sector development
3. Continuation of business environment reform

Thematic groups and subgroups

1. Decentralization and institutional development
   1. Administrative and public service reform
   2. Decentralization and land use planning
2. Macro-economic management
3. Public-sector financial management
4. Statistics
5. Justice
6. Democratic process and civil society
7. Private-sector development and microfinance, including:
   1. Microfinance
   2. Financial sectors
   3. Vocational training and employment
   4. Business environment
   5. Small business and small industry development
   6. Regional integration
   7. Tourism, cultural development

Strengthening of the social sector

Priority areas of intervention

1. Strengthening of and access to basic social services
2. Combating HIV/AIDS

Thematic groups and subgroups

1. Education
   1. Thematic committee—Quality
   2. Thematic committee--Management
2. Health
   1. Health (4 of 12 groups and subgroups)
3. Water and sanitation
4. HIV/AIDS

Crosscutting themes

Priority areas of intervention

1. Gender
2. Special Groups

Thematic groups and subgroups

1. Excision
2. Governance roll-up group
3. JAS team
4. Ad hoc population and development group
5. Ad hoc Northern Mali group

CIDA Program in Mali^{Footnote 45}

Approved by Canadian and Malian authorities in 2000, the 2000-2010 Mali programming framework aims to support Mali's poverty reduction objective through three priorities:

• Better access to basic services: Strengthening health services and the education system, especially for women
• Security and household income growth: Improving economic conditions at the grassroots level, including the savings bank and credit union network, grain market restructuring, financial services, community mobilization, and decentralization
• Peacebuilding, security, good governance, and the rule of law: Restoring the confidence of Malians in government, institutions, and the legal system, to promote peacebuilding and good governance

CIDA's cooperation program is also based on three crosscutting themes:

• bridging social and economic gaps between women and men;
• better environmental balance;
• strengthening decentralized mechanisms, especially in northern Mali.

In 2007, as recommended by the evaluation of the cooperation program, the bilateral program reviewed its sectoral policy directions and related initiatives. This strategy has primarily made new contributions in the following areas:

• Investment has increased, with bilateral aid alone rising from $77 million in 2008-2009 to $100 million in 2010-2011.
• Canadian aid has been more closely harmonized with that of other TFPs, and better aligned with the Government of Mali's policies, sectoral strategies, and action plans.
• More intense synergy has been sought among CIDA's various programming mechanisms, based on information sharing at all stages of activities (planning, results achieved, and lessons learned). Coordination has improved among Geographic Programs Branch, Partnerships with Canadians Branch, Multilateral and Global Programs Branch, and other federal ministries.
• Policy dialogue has been strengthened, involving public authorities and civil society.
• Canada is increasingly involved in supporting national programs through budget support mechanisms.

More recent data, published by the Chief Financial Officer,^{Footnote 46} gives $117.6 M and $102.0 M as CIDA's total disbursements to Mali in 2009-2010 and 2010-2011. These data are not disaggregated by delivery mechanism and are preliminary for 2010-2011. This statistical report ranks Mali as the fifth-largest recipient of Canadian ODA from 2006-2007 to 2010-2011, with disbursements totalling $486.5 M, behind Sudan ($509 M), Ethiopia ($651 M), Haiti ($865 M), and Afghanistan ($1,155 M).

Finally, the data indicate the sectoral distribution of disbursements, as shown below. It must be noted, however, that this is based on a sectoral distribution of CIDA funding to partner organizations for all of their activities, which may differ from the distribution for Mali.

Multilateral and Global Programs Branch disbursement profile

According to the Chief Financial Officer's data, two United Nations agencies, UNICEF and WFP, received $29.8M, or 96 percent of the $31.0 M that MGPB allocated to Mali from 2006-2007 to 2009-2010. Ten other international organizations shared $1.1 M. Funding for UNICEF ($16.1 M) involves the Integrated Health Systems Strengthening in Africa project; 21 percent of the $105 M budget is earmarked for Mali. Funding for the WFP ($12.7 M) basically involves funding for school funding, all of which is earmarked for Mali.

Annex 3: Evaluation Matrix

1. Contextual relevance and proper design of the CDPF and related programming

Criteria and Sub-criteria

1.1 Consideration of Mali's development priorities (PRSP, GPRSP)

Questions

• To what extent does the CDPF reflect the development priorities and policies of the Government of Mali, especially those indicated in the 2002-2006 PRSP, and particularly the 2007-2011 GPRSP?
• To what extent have adjustments to the CDPF reflected changes in Mali's environment?
• Did CIDA help to establish and implement the joint evaluation framework for monitoring GPRSP results, with specific objectives and indicators, to ensure concerted, aligned, and strategic programming choices with a view to poverty reduction?
• Has CIDA been actively involved in monitoring GPRSP results, and does it make decisions consistent with this exercise in its programming?
• Do joint monitoring mechanisms exist for MDG results, and how has CIDA made sure to consider them in developing its programming?

Indicators

• Mechanisms favouring leadership by the Government of Mali and national ownership, in a spirit of consultation, partnership, and mutual responsibility
• CDPF priorities in relation to PRSP/GPRSP strategic directions and priorities
• Framework and mechanisms for joint monitoring of GPRSP results
• Framework and mechanisms for joint monitoring of MDGs
• Adjustments made to Canadian programming, based on strengths and weaknesses in achieving GPRSP objectives

Data Sources

• PRSP, PRSP implementation monitoring/evaluation studies
• GPRSP, GPRSP annual monitoring reviews
• 2008-2011 JAS
• Reports/studies of MDG achievement
• CIDA programming strategies 2005-2010. 2008. 2009
• Investment profile of Canada's Mali program 2006-2011
• Opinions and advice of key stakeholders and policy and strategic experts

Data Collection Method

• Literature review
• Individual and semi-structured group interviews with:
  • CIDA (HQ and field)
  • PSU
  • TFPs
  • Government of Mali, including GPRSP unit and Aid Harmonization Secretariat

Criteria and Sub-criteria

1.2 Strategic and policy priorities of Canada and CIDA

Questions

• Is the CDPF in line with the priorities and policies of Canada and CIDA for Africa, and especially West Africa?
• Does the CDPF reflect CIDA's strategic priorities (MDGs, aid effectiveness principles, Paris Declaration, Accra)? Where appropriate, has the CDPF been adjusted to reflect policy changes at CIDA?
• Are Canadian programming's approaches, activities, and expected results in accordance with CIDA's comparative advantages in Mali (including expertise and lessons learned from past experience)?
• What role does the CDPF assign to equality between women and men? To what extent do approaches, results, and activities support the objectives of CIDA's Policy on Gender Equality (1999)?
• What specific role does the CDPF assign to the issues of the environment and decentralization?

Indicators

• Aspects of CDPF's coherence with Canada's action plans and relevant strategies
• CDPF's integration of the Millennium Development Goals, aid effectiveness principles, the Paris Declaration, and the Accra Agenda for Action
• Specific expertise and comparative advantages of Canadian programming in Mali
• Consideration of strengths, weaknesses, success factors, and lessons of CIDA's past programming in Mali
• Statements and activities planned in the CDPF to reflect gender equality, in line with CIDA's Policy on Gender Equality
• Effective role assigned to the environment and decentralization in the sample of projects, strategies, statements, and strategic environmental assessments

Data Sources

• Aid effectiveness action plan, strategies (food security, children and youth, sustainable economic growth)
• Other strategic documents, including the Paris Declaration, the Accra Agenda for Action, reviews/evaluations of Canadian programming in Mali, CDPF, CDPF implementation strategies
• Documentation relating to projects in the sample
• Opinions and advice of Canadian stakeholders in the program

Data Collection Method

• Literature review
• Individual semi-structured interviews with:
  • CIDA (HQ and field)
  • PSU staff

2.0. Development results in terms of poverty reduction

Criteria and Sub-criteria

2.1 Actual versus expected results of the program

Questions

• What results have been achieved in relation to expected results for the sample of projects selected, both for activities specific to CIDA and for activities delivered through PBAs or pooled funds (non-SBS or non-existing SBS or GBS)?
• What were the key success factors or difficulties for projects in achieving expected results?
• What results were achieved in relation to objectives and expected results in the CDPF's three programming areas?
• Governance and gender equality were treated as crosscutting themes and as strategic focuses. How were results defined and monitored? What results were expected and achieved?
• To what extent are results achieved proportional to CIDA's contribution?
• Did the choice of delivery mechanisms make it possible to achieve expected results in the following areas?
• Education and health: Bilateral (directive projects, PBA), Multilateral and Partnership
• SHIG: Bilateral (directive / responsive projects), Partnership
• Governance: Bilateral (directive), Multilateral, Partnership
• Do activities include strategies conducive to integrating the crosscutting issues^* of gender equality, the environment, and decentralization? Have they effectively generated tangible results in these areas?

^(*) Where treated as such

Indicators

• Selected projects: achieved versus expected results
• Increased access to basic education, health care, literacy, and vocational training, particularly for women
• Increase in textbook / pupil ratio
• Increased access of small operators (women and men) to savings / credit; strengthening of rural associations
• Replenishment of safety stocks and stabilization of grain market prices
• Increase in production, productivity, and rural income
• Public policy and reforms implemented in the areas of justice, expenditure control, and taxation
• Reduction of regional disparity, especially in northern Mali
• Stakeholders' assessment of CIDA's performance (by project, by sector)
• Increased participation of civil society in dialogue, decision making, and program implementation in key CDPF areas
• Improved governance of government mechanisms
• Project performance by channel, by type of investment
• Degree to which selected projects implemented gender equality, environmental, and decentralization strategies

Data Sources

• PADs, PIPs, CSA reports, project monitoring/evaluation reports
• Memorandums for the Minister, grant / contribution agreements
• Annual program reports (program performance framework)
• Program evaluations by other TFPs (World Bank, UNDP, European Union, bilateral organizations active in CDPF areas or areas complementing those of the CDPF)
• Opinions and advice of key experts and stakeholders on the quality and functionality of results achieved
• Field observation

Data Collection Method

• Literature review
• Semi-structured individual interviews with:
  • CIDA (HQ and field)
  • PSU
  • CEAs/CSAs
  • Relevant ministries of the Government of Mali
  • TFPs
  • Other key partners in projects
• Project site visits
• Semi-structured group interviews with Malian, Canadian, or international CSOs/NGOs

Criteria and Sub-criteria

2.2 Program's contribution to poverty reduction

Questions

• Did the program have sectoral strategies in its areas of intervention, making it possible to articulate activities to achieve a critical mass of results in reducing poverty?
• Was the programming strategy designed to maximize CIDA investments with a direct impact on the poor?
• Are the program's expected results in accordance with the objectives, results, and indicators identified in the GPRSP?
• What results did the program achieve with regard to basic education and textbooks, access to health care, and literacy and vocational training for the poor? What methods were used to measure these results?
• What results did the program achieve in security and household income growth? What method was used to measure these results?
• What results did the program achieve in governance and strengthening of the public sector? What method was used to measure these results?
• What results did the program achieve in equal access for women to the resources and benefits of development (at the project and sector levels)?
• What did the CDPF contribute to decentralization, and what results did it achieve?

Indicators

• Existence of integrated sectoral strategies to guide CIDA's direct activities
• Profiles of communities targeted by CIDA activities versus communities affected
• Development of poverty reduction indicators
• Development of gender-based indicators of progress in basic education (enrolment ratio, repeater rate, and completion rate, textbook/pupil ratio, and so on), literacy and vocational training
• Development of access to health care; development of relevant indicators (mortality, morbidity, immunization, and so on)
• Development of SHIG progress indicators (access to savings/credit, rate of reimbursement, operation of rural groups, increase in productivity/rural income)
• Development of governance indicators (capacity building of local communities, school management committees, community health associations, participation of women in local governance, decentralized public-sector financial management)
• Development of gender equality indicators (political, economic, social participation)

Data Sources

• Annual program reports, project documents, GPRSP, evaluations of GPRSP implementation, PIPs, project evaluations, studies and reports of other development stakeholders in Mali
• Opinions and advice of key strategic and programming experts and stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with:
  • CIDA (HQ and field)
  • PSU
  • Government of Mali (GPRSP unit)
  • TFPs belonging to relevant thematic groups
• Semi-structured group interviews with civil-society partners (CSOs/NGOs and other associations)

3.0 Sustainability of results (degree of dependency on development assistance)

Criteria and Sub-criteria

3.1 Time required to achieve sustainable results

Questions

• Was enough time allowed to execute projects and produce results, in order to ensure their sustainability?
• To what extent did CIDA properly identify, analyze, and manage risks associated with the sustainability of results achieved?

Indicators

• Timeline of results achieved (outputs, outcomes, and impacts) vis-à-vis expected results in selected projects
• Quality and degree of execution of viability/sustainability strategies for each selected project
• Risk analysis and management mechanisms, strategies, and processes for the CDPF (by project, by sector)

Data Sources

• Opinions and advice of key experts and stakeholders on the quality and functionality of results achieved
• PIPs and project reports
• Reports. sectoral analyses by CIDA, other TFPs, joint committees
• CDPF annual review, work plans

Data Collection Method

• Literature review
• Semi-structured individual interviews with:
  • CIDA
  • PSU
  • Government of Mali (centralized and decentralized partner mechanisms)
  • TFPs belonging to thematic groups
• Semi-structured group interviews with NGOs/CSOs and local organizations that are partners in projects
• Project site visits

Criteria and Sub-criteria

3.2 Consideration of partners' absorption and ownership capacities

Questions

• By area of intervention, how dependent is the Government of Mali on donor contributions?
• By sector, which partners (central or decentralized government mechanisms, civil society or the private sector) proved best able to ensure the sustainability of results?
• Were CIDA's requirements and conditions enough to ensure that Malian partners met their responsibilities?
• Did partners generally plan the necessary budgets and financial resources to meet recurring costs?
• Must some partners still rely on the maintenance of medium-term financial assistance?

Indicators

• % of GDP covered by donor contributions
• Partner involvement in and contributions to projects, enthusiasm and initiative with a view to sustainable results, leadership. contributions. motivation, and interest for each key programming area
• Provisions made for capacity building of partners involved in Canadian programming areas
• Conditions required by CIDA for partners' financial contribution to maintain results
• Presence / degree of adequacy of withdrawal strategies where applicable

Data Sources

• Opinions and advice of key experts and stakeholders on the quality and functionality of results achieved
• Diagnostic studies of governmental and civil-society partners
• Memorandums of understanding
• Institution building strategy (public, private, and civil society sectors)
• Field observations

Data Collection Method

• Literature review
• Semi-structured individual interviews with:
  • CIDA
  • PSU
  • Government of Mali (centralized and decentralized partner mechanisms)
  • TFPs belonging to thematic groups
• Semi-structured group interviews with NGOs/CSOs and local organizations that are partners in projects
• Project site visits

4.0 Integration of crosscutting themes (gender equality, governance, environment, decentralization)

Criteria and Sub-criteria

4.1 Integration of crosscutting themes in the Mali CDPF

Questions

• Did the CDPF integrate crosscutting themes strategically and operationally?
• Did the program have enough resources to ensure implementation and monitoring?

Indicators

• Existence of strategic policy frameworks for crosscutting themes
• Specialized gender equality, environmental, and governance resources to support teams at Headquarters and in the field

Data Sources

• Program performance reports
• Project documents (PADs, monitoring reports, and so on)
• Relevant thematic group reports
• Documentation on themes and sectors
• Thematic opinions and advice of key experts & stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with sectoral and thematic specialists from:
  • CIDA
  • PSU
  • TFPs
  • Government of Mali
  • CSA

Criteria and Sub-criteria

4.2 Process for planning, monitoring, and evaluating cross-cutting results for the sample of projects selected

Questions

• Did projects have strategies to integrate cross-cutting themes effectively, and do project reports adequately monitor them?
• What were the results, by sector and by mechanism, of strategies designed to reflect cross-cutting themes?
• To what extent did the Government of Mali build its capacity to reflect cross-cutting themes (gender equality, the environment. decentralization)? Was there capacity building of civil-society organizations in these areas?

Indicators

• Existence of cross-cutting strategies in PIPs and monitoring of results in project reports
• Conduct of studies to evaluate the results of activities related to cross-cutting themes
• Demonstrated expertise of ministries and CSOs in considering / integrating gender equality, environmental, and decentralization issues (where applicable)

Data Sources

• PIPs, project narrative, monitoring, and evaluation reports
• Any studies or reports on the treatment of cross-cutting themes
• Program performance reports
• Thematic opinions and advice of key experts & stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with:
  • CIDA
  • PSU
  • CSA
  • Government of Mali
  • CSOs

5.0 Coherence

Criteria and Sub-criteria

5.1 Program's internal coherence

Questions

• What practical steps did CIDA take to ensure complementarity and use of synergy among the CDPF's areas of intervention? What level of coherence and synergy was achieved in terms of results?
• Same questions about coherence and synergy in each of the CDPF's areas of intervention
• Same questions about coherence and synergy among the various program funding mechanisms, specifically:
• Among Bilateral (directive. responsive. PBA), Multilateral and Partnership, by and among sectors
• Between education and health SBS, and non-SBS activities in these sectors
• To what extent did non-SBS activities contribute to or enable implementation of SBS?
• Was CIDA able to maintain and enhance its programming in an iterative manner to capitalize on investments and to adapt to change (priorities of the Government of Mali, increased administrative expertise, and so on)?
• Does the distribution of activities and efforts, as a percentage of investments per area of intervention, reflect the distribution initially sought? Over time, did this distribution evolve in accordance with the changing environment and Mali's priorities?

Indicators

• Operational measures to favour complementarity and synergy of results among Canadian programming activities
• Examples of complementarity and synergy among sectors and projects funded by various mechanisms
• Examples of coherence (or incoherence) among CIDA-funded bilateral, multilateral, and partnership programs
• Examples and lessons learned from projects experiencing problems (duplication, contradictory approaches, isolation, and so on) or performing well (flow of communication, sharing of roles, partnership agreements, and so on)
• Effectiveness of mechanisms for interface and dialogue among the various Canadian stakeholders in Mali
• Diachronous coherence of Canadian programming, based on a strategy of expanding and capitalizing on the achievements, adjusting the priorities, and improving the management procedures (especially public-sector financial management) of the Government of Mali
• Allocation of 2006-2011 disbursements among the three areas of intervention (distributed by sector within each area if possible)

Data Sources

• Project documents
• Program performance reports
• CDPF 2006-2011 investment profile
• Opinions and advice of key program experts and stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with:
  • CSAs in the field
  • Key project partners (Government of Mali, civil society, and private sector)
  • CIDA and PSU
  • TFPs active in CIDA's areas of intervention (or related areas), including pooled fund co-donors
• Semi-structured group interviews with CSOs/NGOs & grass-roots organizations

Criteria and Sub-criteria

5.2 External coherence

Questions

• What steps were taken to ensure complementarity and synergy with the activities of other donors?
• Does the Government of Mali play the role it should play in aid coordination?
• Where donor countries that are signatories of the Paris Declaration have put consultation mechanisms in place, do they make it possible to achieve a satisfactory level of co-ordination?
• What role did Canada play in these various mechanisms during the evaluation period (leader, active partner, silent partner)?

Indicators

• Examples of measures aimed at complementarity and synergy among TFPs
• Nature of leadership provided by the Government of Mali in aid coordination
• Level of implementation of recommendations of sectoral / thematic work groups and aid coordination committees
• Nature and duration of Canada's roles in coordination mechanisms
• Evaluation of Canada's performance of its role by its various partners

Data Sources

• Program performance reports
• Working group reports
• Reports on and evaluations of the Paris Declaration, budget support, and sector policies
• Opinions and advice of key program experts and stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with:
  • CIDA (HQ and field)
  • PSU
  • Government of Mali (GPRSP unit and Aid Harmonization Secretariat)
  • TFPs from thematic groups
  • CSAs in the field
  • Key civil-society partners

Criteria and Sub-criteria

5.3 Coherence of programming with Canada's objectives in Mali

Questions

• Does the program include activities (policy dialogue, consultation frameworks, other activities) to support synergy among the various Canadian stakeholders in Mali?

Indicators

• Examples of activities involving consultation / coordination among Canadian stakeholders

Data Sources

• CIDA policies, action plans, and strategies versus CDPF
• Program performance reports
• Opinions and advice of Canadian program stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with the CIDA team (HQ and field)

Criteria and Sub-criteria

5.4 Policy dialogue in support of programming

Questions

• Were CIDA's policy dialogue guidelines strategic in relation to programming? Were these guidelines followed to improve the program's performance?
• Did policy dialogue provide enough guidance for Canadian programming, based on Mali's priorities, while effectively promoting Canada's priorities (rights, democracy, transparency, gender equality, the environment, participation of civil society, and so on)?
• Did CIDA's dialogue and influence succeed in creating local networks, forming closer partnerships, strengthening national/local institutions, and influencing the Government of Mali's policies in key sectors covered by the CDPF?
• Did CIDA's activities strengthen dialogue between civil society and the Government of Mali?

Indicators

• Examples of policy dialogue activities conducted to support the performance of Canadian programming
• Integration of Canadian priorities in Malian policies and programs
• Establishment and/or activation of formal mechanisms to involve civil society in decision-making

Data Sources

• Reports, internal memoranda, and correspondence on policy dialogue
• Working group reports
• Program performance reports
• Opinions and advice of key program experts and stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with:
  • CIDA/PSU
  • Government of Mali
  • TFPs in thematic groups
• Semi-structured group interviews with civil-society organizations

6.0 Management efficiency ( cost/effectiveness, decentralization, and capacity to monitor, evaluate, and learn lessons from programming)

Criteria and Sub-criteria

6.1 Management, decentralization, and optimization of the program's cost-effectiveness

Questions

• How was the cost-effectiveness of programming affected by the increase in the volume of investment and the diversification of delivery mechanisms?
• Does a decentralized program allow closer monitoring and more timely decision making, considering risks or delays in achieving results?
• Are Headquarters-field team interface mechanisms conducive to timely decision making, helping to meet deadlines and to achieve program results?
• Was programming for 2006-2011 accompanied by the production of a performance management framework and a risk management framework?
• For each dollar disbursed, how do transaction costs compare with the CDPFs of other countries of focus where management is decentralized?

Indicators

• Change in person-years allocated for program management versus amounts disbursed
• PSU costs as a percentage of disbursements, comparison with other countries
• Clarity and logic in the respective definition of roles and responsibilities at Headquarters and in the field
• Cost of improving donor coordination (% of time that CIDA employees spend on joint coordination)
• Impact of the shift to the PBA and budget support on changes in program management costs in relation to the volume of disbursements
• Existence of risk / performance management frameworks, or impact on management thoroughness if absent

Data Sources

• CDPF, program investment profile
• Financial data on operational vs. development expenditures
• Program management framework
• Risk management framework
• Opinions and advice of Canadian program stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with CIDA (HQ and field), PSU

Criteria and Sub-criteria

6.2 Efficiency of resource allocation and utilization

Questions

• To what extent does the program have the proper human and technical resources for its management, both at Headquarters and in the field?
• To what extent do projects have the proper financial resources to achieve expected results?

Indicators

• Number and qualifications of human / technical resources by sector, focus, theme, ensuring program management; possible comparison with other donors
• Turnover rate of CIDA and CEA/CSA project personnel
• Budget and disbursements of projects in the sample in relation to results achieved

Data Sources

• PIPs / monitoring and/or evaluation reports on selected projects
• Possible audit reports
• Opinions and advice of Canadian program stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with:
  • CIDA (HQ and field) / PSU
  • CEAs/CSAs
  • Donors active in the same programming areas

Criteria and Sub-criteria

6.3 Project cost-effectiveness ratio (management costs versus amounts actually invested in Mali for development purposes)

Questions

• To what extent is there a reasonable ratio between project management costs and net investment in Mali?
• To what extent have program support and budget support made it possible to increase the efficiency of activities, in terms of time and resources devoted to management?
• How do CIDA's various aid delivery mechanisms in Mali compare in terms of relative efficiency and cost-effectiveness?

Indicators

• Expenditure mechanisms of selected projects: CIDA and CEA/CSA management costs versus development spending directly affecting the poor or local institutions
• Number of planned versus approved projects, and approval time
• Comparison of management costs / results achieved (% of time CIDA and/or CEA/CSA personnel spend managing various projects, score for each project) by channel, type of investment, and delivery mechanism per sector
• Application of potential project leverage (replication of models or strategies, application of lessons and best practices, and so on)

Data Sources

• Documentation on selected projects: PIPS / evaluation and/or monitoring reports, audit reports
• Program narrative and financial reports
• Opinions and advice of program stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with:
  • CIDA (HQ and field)
  • PSU
  • CEAS/CSAs of projects in the sample
  • Some TFPs (possibly)

7.0 Application of aid effectiveness principles

Criteria and Sub-criteria

7.1 Ownership

Questions

• To what extent do the CDPF's activities meet clearly expressed needs and demands?
• To what extent have projects been designed to enable local partners to lead their execution and/or to promote maximum ownership and management by local partners?

Indicators

• Process of planning the CDPF and designing adjustments made to the CDPF over time
• Linkage between the CDPF and the PRSP / GPRSP
• Definition of mutual responsibilities in memorandums of understanding for activities
• Diagnosis and capacity-building activities with a view to local ownership
• Degree of autonomy allowed for Malian partners in project execution

Data Sources

• PRSP/GPRSP
• Project documents
• CIDA/Government of Mali framework agreements
• CIDA/Government of Mali agreements for specific projects in the sample
• Opinions and advice of key program experts and stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with:
  • CIDA (HQ and field)
  • CEAs/CSAs (HQ and field)
  • Government of Mali
  • Local partners
• Semi-structured group interviews with CSOs/NGOs and local associations

Criteria and Sub-criteria

7.2 Alignment

Questions

• To what extent do local institutions control the program's strategy, management, and execution?
• To what extent have Canadian programming and projects been designed to reflect Mali's policies and procedures?
• To what extent are national / local financial management and procurement institutions strengthened in Mali?
• To what extent is CIDA's capacity building support for the Government of Mali coordinated with that of other TFPs?
• Does CIDA use parallel mechanisms in its program?

Indicators

• Integration of the Government of Mali's policies and procedures in projects
• Examples of initiatives coordinated to build the capacities of the Government of Mali
• Diagnosis / activities to build Mali's financial management, economic planning, and procurement/contracting capacities
• Number of investments with parallel mechanisms

Data Sources

• Project documents – PADs, PIPs, evaluations, monitoring reports
• CIDA/Government of Mali framework agreements
• CIDA/Government of Mali agreements for specific projects in the sample
• Opinions and advice of key thematic experts and stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with:
  • CIDA (HQ and field)
  • PSU
  • Malian partners in programming areas

Criteria and Sub-criteria

7.3 Harmonization

Questions

• What percentage of CIDA's program is delivered through program / budget support?
• What consultation mechanisms have made it possible to coordinate Canadian programming with that of other TFPs, and what role has Canada played? In what programming areas (including cross-cutting themes)? With what tangible results?
• For each sector where consultation mechanisms have been used, what is CIDA's share of overall funding? In what areas has CIDA had a significant and recognized impact on TFPs and the Government of Mali?

Indicators

• Program support as a % of CDPF budget, GBS and SBS; same for disbursements during the period
• Consultation mechanisms in sectors of Canadian programming, and Canada's role in these mechanisms
• Consultation mechanism activity reports
• Documents produced and efforts made by CIDA to disseminate acquired knowledge, best practices, and lessons learned from Canadian programming in Mali to other TFPs and the Government of Mali
• Examples of Canada's influence or leadership (such as its role as a leader and/or active partner)

Data Sources

• Documents/studies on consultation activities (including 2008-2011 JAS)
• Reports/minutes of TFP thematic groups in CDPF sectors
• Project reports/evaluations
• Opinions and advice of key policy and strategic experts and stakeholders

Data Collection Method

• Literature review
• Semi-structured individual or group interviews with TFPs belonging to thematic groups
• Semi-structured individual interviews with:
  • CIDA representatives
  • Relevant ministries/ institutions of the Government of Mali

8.0 Management results, including monitoring and evaluation

Criteria and Sub-criteria

8.1 Management results

Questions

• To what extent has Canadian cooperation based its activities on acquired knowledge, and has Canadian cooperation sought to acquire new knowledge?
• Has there been adequate risk management of Canadian cooperation's activities in Mali?

Indicators

• Degree of consideration of lessons learned
• Relevance of mechanisms for conveying knowledge and of learning systems
• Existence of a risk management framework

Data Sources

• Planning documents, country and sector strategies
• Program management framework
• Risk management framework
• Program review
• Opinions of Canadian program stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with CIDA (HQ and field)

Criteria and Sub-criteria

8.2 Monitoring and evaluation mechanisms put in place to ensure result- and risk-based management and accountability

Questions

• The CDPF prescribed a program performance measurement framework. Was it produced and implemented? If so, did its indicators allow adequate monitoring of programming results?
• Considering CIDA's role in Mali's overall development, does performance measurement realistically make it possible to attribute results to Canadian investments?
• Were project and program monitoring, evaluation, and reporting mechanisms adequate and properly used for decision making?
• To what extent did CIDA participate in joint monitoring and evaluation initiatives, and observe the principle of harmonization in managing the performance of its Mali program?

Indicators

• Existence and implementation of a program performance measurement framework
• Realism of indicators in terms of attributing results to CIDA investments
• Examples of the impact of monitoring and evaluation studies on decision making, and direction of the program as a whole and in each sector
• Joint evaluation reports (CIDA, TFPs, Government of Mali)
• Number and scope of joint vs. unilateral missions

Data Sources

• 2007 program evaluation report
• Project monitoring and evaluation analyses
• Program annual reports
• Result monitoring strategy (project, program assistance, and program levels)
• Opinions of Canadian program stakeholders

Data Collection Method

• Literature review
• Semi-structured individual interviews with:
  • CIDA (HQ and field)
  • PSU

Annex 4: Terms of Reference Summary

Context

The Federal Accountability Act (2006) requires that all programs be evaluated every 5 years. The previous Mali was for 2000-2006. The current evaluation will assess a five-year period from fiscal year 2006-07 to 2010-11.

Objectives of the Evaluation

The current evaluation will pursue the following objectives:

• To take stock of the results achieved by the program over the past five year period (fiscal years 2006-2007 to 2010-2011).
• To assess the program's overall performance in achieving these results (based on criteria of relevance, effectiveness, sustainability, coherence, efficiency, management principles, cross-cutting themes, and performance management).
• To assess the performance of the program's channels of cooperation (mainly bilateral) and delivery models (directive, responsive, core programming, etc.).
• To document and disseminate findings and lessons learned, and formulate recommendations to improve the performance of the current or future program strategy.

The evaluation will review the programming strategies and frameworks in effect from 2006-2007 to 2010-2011, along with strategic policy documents approved in 2007 and 2009 and others issued since then. Projects that were terminating or well advanced during the 5-year period under review will be included in the sample to document results achieved.

Scope and Channels of Cooperation

The evaluation will analyse the development results and management factors related to the Program and to projects delivered through various mechanisms, including bilateral directive projects and responsive projects implemented through multilateral institutions and non-governmental organizations, and program-based approaches (PBAs).

Evaluation Key Questions and Criteria

The evaluation will respond to two key questions and eight criteria agreed upon internationally. These elements will be assessed from the perspective of (i) the Program, (ii) the main programming areas, and (iii) a sample of projects.

What has been achieved?

• Relevance: Is the Program relevant in relation to the international agreed-upon goals, CIDA's objectives, and regional and sub-regional development strategy and needs?
• Effectiveness of Results: To what extent did the Program reach its objectives (overall and in the various programming areas)?
• Sustainability & Risks: Were the actions undertaken sustainable?
• Cross-cutting themes: Have gender equality and environmental sustainability been taken into consideration adequately?

Why were the intended results achieved or not?

• Coherence: Internal coherence at Program/project levels (sectors, channels, delivery models), coherence with Canada's whole-of-government efforts (including program's policy dialogue, leverage, leadership), and external complementarity with other members of the international community?
• Management Principles: How did the program apply the Paris Declaration principles of ownership, partnership, and harmonization?
• Efficiency: Are the human, technical, financial resources sufficient, appropriate, efficiently used? Time needed to approve, manage, monitor? Were delivery mechanisms appropriate, cost-effective? Is the program's level of decentralization adequate?
• Performance Management: This includes results-based management as well as monitoring and evaluation activities.

Evaluation Methodology

Various data gathering methods will be employed in the context of the current evaluation to ensure multiple lines of evidence are obtained:

• Document review: CIDA Program and project documents, past program or project evaluations, information on regional/sub-regional integration in the Caribbean, documents/statistics from international institutions and other related reports/studies;
• Interviews: including CIDA, partner country officials, international community, implementing agencies, etc.;
• Project visits: of selected projects in various programming areas and countries in the region;
• Roundtables: at different occasions during the evaluation process;
• Case studies: could involve certain projects, partners, or programming/implementation modalities.

Tools such as interview guides and grids will be prepared to ensure the comparability of the data between projects and sectors. A sample of projects will be reviewed based on a five-point scale ranging from highly satisfactory to highly unsatisfactory.

Sampling The evaluation cannot review 100% of the projects supported by the Canadian cooperation during the period from 2006-2007 to 2010-2011. Approximately 25 projects will be reviewed including those that have been subject to an evaluation in the past. The sampling methodology will be based on the following criteria:

• Representativeness: projects selected should represent a substantive proportion of the Program's total budget and sector or thematic value;
• Development approaches: projects and activities related to policy dialogue, institutional capacity development and/or community level interventions;
• Implementation mechanisms: directive and responsive bilateral projects, and PBAs;
• Implementing agencies: government and non-governmental organizations, multilateral institutions, Canadian Executing Agencies, etc.;
• Evaluability/Materiality: the projects are significant enough, and sources of information are available and accessible cost-effectively.

Annex 5: Selected Documents Consulted

General Program Documentation

CIDA

CIDA. Administrative Report Relating to the Canada-Mali Cooperation Program Evaluation. December 2007

CIDA. Annual Country Program Performance Report 2010-11. No date

CIDA. Annual Country Report Card 2009-2010 – Mali Program. EDRMS No. 5080165 (no date)

CIDA. Canada-Mali Cooperation Program Evaluation (2000-2010). Executive Report. November 2007

CIDA. CIDA in Mali: Annual Report 2009. Draft of April 28, 2008

CIDA. Corporate Risk Profile. Mali Program. 2010-2011. Final draft. August 5, 2011

CIDA. Country Development Programming Framework 2010-2015. Final version. December 2009

CIDA. Country Development Programming Framework for Mali 2000-2010

CIDA. Country Strategy. June 12, 2008

CIDA. Draft Framework for the Mali Program Evaluation (CPR) for the Period from 2006-2007 to 2010-2011. April 2011

CIDA. Mali Country Development Programming Framework Implementation Strategy 2005-2010. March 2005

CIDA. Mali Country Programme Strategy 2009. Draft 28. July 27, 2009

Econotec inc. Cadre d'orientation stratégique pour le programme du Mali. Croissance du revenu des ménages [Mali program policy framework – household income growth]. March 2008

Larouche. Ghislaine. Opérationnalisation des volets de renforcement des capacités dans les appuis budgétaires sectoriels de l'ACDI au Mali [implementation of capacity-building components of CIDA's sector budget support to Mali]. Reflection paper. Final version. March 2007

Minister of International Cooperation and Minister of La Francophonie and Official Languages. Canadian International Development Agency. 2006-2007 Estimates. Part III: Report on Plans and Priorities. Hull. 2006

Minister of International Cooperation and Minister of La Francophonie and Official Languages. Canadian International Development Agency. 2007-2008 Estimates. Part III: Report on Plans and Priorities. Hull. 2007

Minister of International Cooperation. Canadian International Development Agency. 2008-2009 Estimates. Part III: Report on Plans and Priorities. Hull. 2008

Minister of International Cooperation. Canadian International Development Agency. Report on Plans and Priorities for the Period Ending March 31, 2010. Hull. 2009

Minister of International Cooperation. Canadian International Development Agency. Report on Plans and Priorities for the Period Ending March 31, 2011. Hull. 2010

Minister of International Cooperation. Canadian International Development Agency. Report on Plans and Priorities for the Period Ending March 31, 2012. Hull. 2011

Note on the seventh review of the Extended Credit Facility with the International Monetary Fund. October 2011

Government of Mali

Aid Harmonization Secretariat. Programme d'activités 2011 [2011 activity report]. March 2011.

Aid Harmonization Secretariat. Rapport d'activités 2010 [2010 activity report]. September 2011.

Amadou Toumani Touré. Projet pour le développement économique et social (PDES) du Mali [Mali's economic and social development project]. Bamako. March 2007

Ecorys Research and Consulting. Évaluation de la gestion des finances publiques au Mali selon la méthodologie PEFA. Rapport sur la performance de la gestion des finances publiques [evaluation of public-sector financial management in Mali using Public Expenditure and Financial Assessment methodology – public-sector financial management performance report]. Final draft, June 28, 2011

Ministry of Territorial Administration and Local Communities. Kayes Region. Governor's Office. Décision No 11-1044/GRK-CAB portant creation, organisation et fonctionnement du cadre de coordination et des Groupes thématiques [decision no. 11-1044/GRK-CAN regarding the creation, organization, and operation of the coordination framework and thematic groups]. Kayes. May 26, 2011

Ministry of the Economy and Finance. Aid Harmonization Secretariat. Évaluation nationale de la mise en œuvre de la Déclaration de Paris. Rapport final [national evaluation of the implementation of the Paris Declaration – final report]. January 2011

Ministry of the Economy and Finance. CSLP Technical Unit. Revue 2010 du CSCRP. Groupe « Développement humain durable et réduction des inégalités » [2010 review of the GPRSP – sustainable human development and inequality reduction group]. Bamako. June 7, 2011

Ministry of the Economy and Finance. Secretariat General. Validation de la matrice commune des déclencheurs et organisation de la RBC 2011 de l'ABG [validation of the common trigger matrix and organization of the 2001 joint budget review of general budget support]. Letter from the Minister of the Economy and Finance to the World Bank Resident Representative leading the macro-economic management and public-sector financial management group. May 24, 2011.

Plan national d'action sur l'efficacité de l'aide au développement [national development aid effectiveness action plan]. April 2008

Prime Minister's Office. General Government Secretariat. Décret No 01-115/PM-RM du 27 février 2001 portant création des organes d'orientation, de coordination et d'évaluation du Programme de développement sanitaire et social [decree no. 01-115/PM-RM of February 27, 2001, creating mechanisms to steer, coordinate, and evaluate the Health and Social Development Program]. Bamako. February 2001

Prime Minister's Office. General Government Secretariat. Décret No 08-095/P-RM du 21 février 2008 portant création des comités régionaux, locaux et communaux d'orientation, de coordination et de suivi des actions de développement [decree no. 08-095/P-RM of February 21, 2008, creating regional, local, and communal development activity steering, coordination, and monitoring committees]. Bamako. February 2001

Republic of Mali. Cadre Stratégique pour la Croissance et la Réduction de la Pauvreté. CSLP 2ème Génération [growth and poverty reduction strategy paper (GPRSP), 2^nd generation] 2007 – 2011. December 2006

Republic of Mali. Déclaration de politique générale du Premier ministre Cissé Mariam Kaidama Sidibé [general policy statement of Prime Minister Cissé Mariam Kaidama Sidibé]. June 24, 2011

Revue annuelle 2011 du CSCRP [2011 GPRSP annual review]

Revue du CSCRP 2007-2011 portant sur 2010. Rapport de synthèse. Cellule technique CSLP. [2007-2011 GPRSP review focusing on 2010 – summary report – technical unit] September 2011.

Technical and Financial Partners

Fiche de support. Troika meeting – MEF, October 3, 2011

Grégoire. Luc. Le défi des OMD au Mali [the challenge of meeting the Millennium Development Goals in Mali]. Conference – Mali: Challenges and Economic Opportunities. UNDP-Mali. Bamako. March 17-18 2011

International Monetary Fund. Mali: Ex Post Assessment (EPA) of Longer-Term Program Engagement – IMF Services Report; Statement of the IMF Administrator for Mali; Public Information Notice; IMF Report 11/153; May 26, 2011

International Monetary Fund. Mali: Letter of Intent. Memorandum of Economic and Financial Policies, and Technical Memorandum of Understanding, December 31, 2010

International Monetary Fund. Mali: Letter of Intent. Memorandum of Economic and Financial Policies, and Technical Memorandum of Understanding, May 26, 2011

International Monetary Fund. Mali: Letter of Intent. Memorandum of Economic and Financial Policies, and Technical Memorandum of Understanding, November 23, 2011

Joint Evaluation of Budget Support Operations in Mali 2003-2009, carried out on behalf of the Commission of the European Union, Belgium and Canada, July 2011

Mali – Joint Assistance Strategy (JAS) 2008-2011

Mali: Ex Post Assessment of Longer-Term Program Engagement - Staff Report. IMF. June 2011

Minutes of the 2^nd civil society/TFP meeting. April 16, 2008. Aoua Keita Centre

Minutes of the Minister of the Economy and Finance/TFP Troika meetings of January 26, March 22 and April 26, 2011

Note on the complementarity matrix. Overall positioning of TFPs and their financial volumes. September 2011

Report. TFP annual retreat of February 8, 2011

Technical pool – Note on the operation of thematic groups. March 2011

TFP joint evaluation of 2011 GPRSP annual review. July 2011

TFP joint evaluation of PRSP implementation in 2006. Final document. July 2007

Troika. Principles and missions. No date (but after May 2011)

Work program 2011 – Mali's technical and financial partners

SBS – Health / Education and Public-Sector Financial Management

Assefa. Abebech (CIDA). Décaissement de la contribution du Canada de 4 millions $ en appui aux programmes nationaux en éducation et en santé au Mali. [disbursement of Canada's $4 million contribution to support national education and health programs in Mali] Memorandum. July 2011

Goulet. Claude (CIDA). Décaissement d'une contribution de 13 millions $ en appui au programme national en éducation et aux finances publiques au Mali. [disbursement of Canada's $13 million contribution to support national education and public-sector financial management program in Mali] Memorandum. March 2011

Goulet. Claude (CIDA). Décaissement de la contribution du Canada de 4 millions $ en appui aux programmes nationaux en éducation et en santé au Mali. [disbursement of Canada's $4 million contribution to support national education and health programs in Mali] Memorandum. December 2010

Pétillon. Yves (CIDA). Décaissement d'une contribution du Canada de 17 millions $ en appui au programme national en santé et à la réforme des finances publiques au Mali. [disbursement of Canada's $17 million contribution to support the national health program and public-sector financial management reform in Mali] Memorandum. February 2010

Pétillon. Yves (CIDA). Décaissement de la contribution du Canada de 13 millions $ en appui aux programmes nationaux en éducation et en santé et à la réforme des finances publiques au Mali. [disbursement of Canada's $13 million contribution to support national education and health programs, and public-sector financial management reform, in Mali] Memorandum. April 2008

Pétillon. Yves (CIDA). Décaissement de la contribution du Canada de 14 millions $ en appui aux programmes nationaux en éducation et en santé au Mali. [disbursement of Canada's $14 million contribution to support national education and health programs in Mali] Memorandum. October 2008

Pétillon. Yves (CIDA). Décaissement de la contribution du Canada de 18 millions $ en appui aux programmes nationaux en éducation et en santé et à la réforme des finances publiques au Mali. [disbursement of Canada's $18 million contribution to support national education and health programs, and public-sector financial management reform, in Mali] Memorandum. March 2009

Pétillon. Yves (CIDA). Décaissement de la contribution du Canada de 14 millions $ en appui aux programmes nationaux en éducation et en santé au Mali. [disbursement of Canada's $14 million contribution to support national education and health programs in Mali] Memorandum. November 2009

Joint Reports

Royal Dutch Embassy. 1ère tranche fixe des Pays-Bas à travers l'ABS pour PRODESS II en 2008. [first fixed tranche from The Netherlands through SBS for PRODESS II in 2008] Bamako. March 2008

TFP Group - Education (AFD). Analyse conjointe des PTF sur les conditions de décaissement de la deuxième tranche de l'appui budgétaire en faveur du PISE II au titre de l'année 2008. [TFP joint analysis of disbursement conditions for the second tranche of budget support for PISE II in 2008] Bamako. October 2008

TFP Group - Education (AFD). Analyse conjointe des PTF sur les déclencheurs de la 1ère tranche ABS 2008 - PISE II. [TFP joint analysis of triggers for the first tranche of SBS for PISE II in 2008] Bamako. March 2008

TFP Group - Education. Analyse des PTF sur les conditions de décaissement pour le Canada de la tranche fixe de l'appui budgétaire en faveur du PISE au titre de l'année 2010. [TFP analysis of disbursement conditions for Canada – fixed tranche of budget support for PISE - 2010] Bamako. (no date)

TFP Group - Education. Analyse des PTF sur les conditions de décaissement pour le Canada de la tranche variable de l'appui budgétaire en faveur du PISE au titre de l'année 2010. [TFP analysis of disbursement conditions for Canada – variable tranche of budget support for PISE - 2010] Bamako. November 2010

TFP Group - Education. Analyse des PTF sur les conditions de décaissement de la tranche unique ou fixe de l'appui budgétaire en faveur du PISE au titre de l'année 2011. Commission de suivi du 8 mars 2011 : conclusions des PTF [TFP analysis of disbursement conditions for the single or fixed tranche of budget support for PISE – 2011 – monitoring committee meeting of March 8, 2011 – TFP conclusions]

TFP Group - Education. PISE II / ABS / Deuxième tranche 2009. Analyse sur les conditions de décaissement. Commission de suivi du 16 octobre 2009 : conclusions des PTF. Conclusion finale. 4 novembre 2009 [PISE II / SBS / second tranche 2009 – analysis of disbursement conditions – monitoring committee meeting of October 16, 2009 – TFP conclusions – final conclusion, November 4, 2009]

TFP Group - Education. PISE II / ABS / Première tranche 2009. Conclusion finale des PTF au 21 juin 2009 sur les conditions de décaissement [PISE II / SBS / first tranche 2009 – TFP final conclusion on disbursement conditions as at June 21, 2009]

TFP Group - Health. Analyse conjointe des partenaires techniques et financiers (PTF) sur les conditions de décaissement de la deuxième tranche (Canada) et de la Tranche unique (Pays-Bas) de l'appui budgétaire sectoriel de l'année 2010 en faveur du PRODESS II prolongé. [joint analysis by the technical and financial partners (TFPs) of disbursement conditions for Canada's 2^nd tranche and The Netherlands' single tranche of sector budget support for the extended PRODESS II - 2010] Bamako. December 2010

TFP Group - Health. Analyse conjointe des partenaires techniques et financiers (PTF) sur les conditions de décaissement de l'appui budgétaire sectoriel de l'année 2011 en faveur du PRODESS II prolongé. [joint analysis by the technical and financial partners (TFPs) of disbursement conditions for sector budget support for the extended PRODESS II - 2011] Bamako. March 2011

TFP Group - Health. Analyse conjointe des PTF sur les conditions de décaissement de la deuxième tranche de l'appui budgétaire en faveur du PRODESS II au titre de l'année 2008. [TFP joint analysis of disbursement conditions for the second tranche of budget support for PRODESS II - 2008] Bamako. November 2008

TFP Group - Health. Analyse conjointe des PTF sur les conditions de décaissement de la première tranche de l'appui budgétaire sectoriel de l'année 2009 en faveur du PRODESS II. [TFP joint analysis of disbursement conditions for 2009 sector budget support for PRODESS II] Bamako. March 2009

TFP Group - Health. Analyse des PTF sur les conditions de décaissement de la deuxième tranche de l'appui budgétaire en faveur du PRODESS II au titre de l'année 2009. [TFP analysis of disbursement conditions for the second tranche of budget support for PRODESS II - 2009] Bamako. November 2009

Education Sectoral Documents

CIDA

Education program architecture. October 11, 2011

Education sector analysis, 2008 and 2010

Education work plans, 2007 to 2011

Ky. Abdoulaye and Mamadou Diabaté. Cadre d'orientation stratégique Éducation. Version finale. [education policy framework – final version] Bamako. May 2009.

Larouche, Ghislaine and Dembélé, Moussa. Étude sur le bilan et les leçons apprises de l'approche d'accompagnement, secteurs santé et éducation. [study – assessment and lessons learned of the support approach in the health and education sectors] CIDA. March 2010.

Government of Mali

Aide mémoire. Revue annuelle conjointe du Programme d'Investissement du Secteur de l'Éducation - Phase III. Version finale. [memorandum – joint annual review of the Education Sector Investment Program – Phase III – final version] Bamako. June 2011

Arrangement spécifique entre le Gouvernement du Mali et les partenaires au développement relatif à l'appui budgétaire sectoriel en faveur du Programme d'investissement dans le secteur de l'éducation (PISE 2). [specific arrangement between the Government of Mali and development partners regarding sector budget support for the Education Sector Investment Program (PISE 2)] July 2006

Arrangement spécifique entre le Gouvernement du Mali et les partenaires au développement relatif à l'appui budgétaire sectoriel en faveur du Programme d'investissement dans le secteur de l'éducation (PISE 3). [specific arrangement between the Government of Mali and development partners regarding sector budget support for the Education Sector Investment Program (PISE 3)] No date

Government of Mali – Ministry of National Education

DNEB. Formation continue des maîtres. Plan stratégique 2010-2013. Plan opérationnel pour 2010. [national directorate of teacher training – ongoing teacher training – 2010-2013 strategic plan – 2010 operational plan] February 2010

Document de politique nationale de la scolarisation des filles. [national policy paper - education of girls] 2007

Education ministries. Cadre des dépenses à moyen terme 2010-2012. Secteur éducation. [2010-2012 medium-term expenditure framework – education sector] April 2010

Education ministries. Cadre logique du PISE 3 2010-2012. Document final. [2010-2012 PISE 3 logic framework – final document] March 2010

Education ministries. Cadre partenarial entre les ministères en charge de l'éducation et de la formation professionnelle et leurs partenaires techniques et financiers. [partnership framework between vocational training ministries and their technical and financial partners] Bamako. 2010

État des lieux du processus de décentralisation et de déconcentration dans le secteur éducatif au Mali. Notes 1 et 2. [situational analysis of the decentralization and deconcentration process in Mali's education sector – Notes 1 and 2] October 2009

Les grandes orientations de la phase 2 du PISE. Version finale. [key directions of PISE Phase 2 – final version] March 2005

Lettre de politique éducative du Mali 2010-2012. [Mali's 2010-2012 education policy letter] Bamako. 2010

Ministry of Education (MÉALN) and USAID/PHARE Program. Temps réel d'apprentissage. [real-time learning] October 2010

Plan d'action pour la scolarisation des filles 2007-2009. [action plan - education of girls] February 2007

Politique nationale de l'alimentation scolaire au Mali. 2^e version. [Mali's national school feeding policy – 2^nd version] May 2008

Politique nationale de l'alimentation scolaire.[national school feeding policy] September 2009

Politique nationale de santé à l'école. Document no 1. [national school health policy – document no. 1] Bamako. September 2011

Politique nationale de santé à l'école. Plan d'action. Document no 2. [national school health policy – action plan – document no. 2] Bamako. September 2011

Programme d'investissement pour le secteur de l'éducation. Descriptif de la troisième phase 2010-2012. Version définitive. [Education Sector Investment Program – description of Phase III 2010-2012 – final version] March 2010

Programme décennal de développement de l'éducation (PRODEC). Les grandes orientations de la politique éducative.[Ten-year education development program (PRODEC). Key educational policy directions] January 2000

Technical and Financial Partners

World Bank. The Malian Education System: Sector Analysis for an Improvement in the Quality and Effectiveness of the System. Washington. D.C. 2010

Project Documentation

CIDA. Project performance reports for almost all projects in the sample (detailed list provided if necessary)

A020892-001 PAMOFE

CAC International. Évaluation du Projet d'appui à l'élaboration et à la mise en œuvre de la formation continue des enseignants de l'enseignement fondamental-PAMOFE (ML/20892). Version finale. [evaluation of the project for the development and implementation of continuing education of basic education teachers – PAMOFE (ML/20892)] January 2010

CIDA. Project approval document. Support project to develop and implement continuing education for basic education teachers. Hull (2002)

CIDA. Rapport final du PAMOFE (incluant les résultats opérationnels du 1^er semestre 2010). [PAMOFE final report (including operational results of the 1^st quarter of 2010] July 2010

MÉALN-DNEB. Rapport – bilan sur les activités de FCM Année scolaire 2008- 2009. [evaluation report on FCM activities – 2008-2009 school year] February 2010

A032009-001 AGDEF

CAC International. Évaluation de fin du projet d'appui à la gestion décentralisée de l'enseignement fondamental (AGDEF) (ML/32009) et Analyse de la complémentarité entre l'AGDEF et le projet d'Appui à l'élaboration et à la mise en œuvre de la formation continue des enseignants de l'enseignement fondamental (PAMOFE) (ML/20892). Version finale. [end-of-project evaluation - project for decentralized management of basic education (AGDEF) (ML/32009) and analysis of complementarity between AGDEF and the project for the development and implementation of continuing education of basic education teachers (PAMOFE) (ML/20892) – final version] February 2010

CATEK Gouvernance et Développement. Étude sur le bilan et les perspectives de la décentralisation au Mali. Rapport final. [study – decentralization in Mali - evaluation and prospects] Bamako. March 2011

CIDA. AGDEF. Memorandum for the Minister. July 2004

CIDA. AGDEF. Rapport semestriel narratif (janvier à juin 2011). [mid-term narrative report (January to June 2011)] July 2011

CIDA. Project approval document. Support in implementing decentralized management of basic education ML/32009. Hull (no date)

CRC Sogema et CAD/DE. Projet AGDEF. Évaluation de l'expérience d'appui à la mise en place de Commissions éducatives dans la circonscription scolaire de Kayes. Mission de consultation. [AGDEF project – evaluation of the experience of supporting the establishment of school committees in the Kayes school district] Kayes. October 2010

MÉALN. Colloque national sur l'opérationnalisation des relations de travail entre les services déconcentrés de l'État et les collectivités territoriales. Rapport général. [national colloquium on the implementation of working relationships between deconcentrated government services and territorial communities – general report] Bamako. May 2011

M012885-001 School Feeding

Muller. Franck et al.. Mali : une évaluation du portefeuille des activités du PAM. [Mali: an evaluation of the WFP's portfolio of activities] WFP Office of Evaluation. September 2010

World Food Programme. Standard Project Reports 2008. 2009. 2010 - Mali. Country Programme - Mali (2008-2012)

S062900-PRG Plan International

Bernard. Anne. Children's Millennium Programme Evaluation. April 2009

Fomba. Cheick Oumar. Évaluation finale du Projet d'éducation pour les enfants/OMD « Amélioration de la qualité de l'éducation de Base » (MLI 0046). Rapport final. [final evaluation of the child education project / MDG – improve the quality of basic education (MLI 0046)] May 2010

Ky. Abdoulaye and coll.. Évaluation à mi- parcours du projet d'éducation pour les enfants /OMD « Amélioration de la Qualité de l'Éducation de Base » (MLI 0046). [mid-term evaluation of the child education project / MDG – improve the quality of basic education (MLI 0046)] Bamako. May 2008

Plan Mali. Promouvoir la Sécurité Économique des Communautés africaines à travers l'éducation et le développement des compétences (2010-2015). Document de projet. [promoting the economic security of African communities through education and capacity building (2010-2015) – project document] Bamako (no date)

S063739-PRG Learning Through Play

Lester. Stuart and Wendy Russell. Children's right to play. An examination of the importance of play in the lives of children worldwide. Working papers No 57 in Early Childhood Development. Bernard van Leer Foundation. The Netherlands. December 2010

Right to Play. Aperçu sur l'intervention de Right to Play Mali [overview of the activity Right to Play Mali]. Bamako. 2010

Right to Play. Rapport annuel d'activités 2010 [2010 annual activity report]. Bamako. May 2011

Textbooks

Econotec inc.. Étude d'impact. Projets d'appui au secteur du manuel scolaire au Mali. [impact study – textbook industry support projects in Mali] Final report. August 2009

Projet 34260 Composante 2. Renforcement des capacités. Extrait de la demande propositions : Annexe A. Description du projet. [project 34260 – component 2 – capacity building – excerpt from request for proposals – Annex A – Project Description] July 2011

Projet 34260 Composante 3. Développement des capacités en réparation des manuels scolaires. Plan de mise en œuvre. [project 34260 – component 3 – textbook repair capacity building – implementation plan] November 2010

Termes de référence : audit technique : manuels scolaires achetés et distribués sur ressources 2010 du PISE III. [terms of reference – technical audit – textbooks procured and distributed using PISE III 2010 resources] June 2011

Health

Sectoral Documents

CIDA

CIDA Mali – Health sector analyses, 2008 and 2010

CIDA Mali – Health work plans, 2007 to 2011

CIDA Mali. Cadre d'orientation stratégique pour le programme du Mali - Santé. Ébauche [Mali program policy framework – health - draft]. Bamako. March 2008

Health program architecture. October 11, 2011

Government of Mali

Konaté. Mamadou Kani. Bakary Kanté and Dr. Fatoumata Djènèpo. Politique de santé communautaire et viabilité économique et sociale des Centres de santé communautaires au Mali. Étude de cas en milieu urbain et rural. [community health policy and the economic and social sustainability of community health centres in Mali] Draft paper prepared for the RUIG/UNRISD project on globalization, inequality, and health. September 2003

Ministry of Health. Cadre de dépenses à moyen terme 2009-2011 dans la perspective des objectifs 2015. Version finale. [2009-2011 medium-term expenditure framework with a view to 2015 objectives] Bamako. April 2009

Ministry of Health. COMPACT. Accroître les efforts et les ressources pour le Santé en vue de l'atteinte des OMD. Version finale. [increasing health efforts and resources to achieve the MDGs – final version] Bamako. March 2009

Ministry of Health. Évaluation du Plan de développement sanitaire et social. Draft final. [evaluation of the health and social development plan – final draft] Bamako. June 2011

Ministry of Health. PRODESS 2 prolongé 2009-2011. Composante santé. [extended PRODESS 2, 2009-2011, health component] Bamako. March 2009

Ministry of Health. Programme de développement socio-sanitaire 2005-2009 (PRODESS 2). Composante santé. [2005-2009 social and health development program (PRODESS 2) – health component] Bamako. December 2004

Ministry of Health. Rapport de la mission conjointe de suivi du PRODESS II : du 25 au 29 octobre 2010 (mission sur le terrain : niveau central et régional); du 01-07 novembre 2010. [PRODESS II joint monitoring mission report – October 25-29, 2010 (field mission – central and regional level); November 1-7, 2010] Bamako. No date

Prime Minister's Office. Décret No 01-115/PM/RM du 27 février 2001 portant création des organes d'orientation, de coordination et d'évaluation du programme de développement sanitaire et social. [decree no. 01-115/PM/RM of February 27, 2001, creating health and social development program steering, coordination, and evaluation mechanisms] Bamako

PRODESS evaluation reports (2007-2011) and programming (2009-2011)

Regional Health Directorate. Regional Directorate of Social Development and the Solidarity Economy. Annuaire statistique 2010. Région de Kayes. [statistical yearbook, Kayes region] July 2012

Specific agreement between the Government of Mali and development partners on sectoral budget support to assist the health and social sectors to implement the extended PRODESS II 2010-2011. No date

Technical and Financial Partners

BTC-Mali and Ministry of Health. L'approche sectorielle dans le domaine de la santé au Mali. [The sector approach in Mali's health sector] December 2009

Project Documentation

CIDA. Project performance reports for almost all of the projects in the sample (detailed list provided if necessary)

A031027-001 Kayes Support

CIDA. Project Approval Document. Community-Level Support for Reproductive Health in the Kayes Region - ML 31027. Hull. (no date)

Lefèvre. Alain and Bouaré Mountaga. Évaluation du Projet d'appui à l'amélioration de la santé de la reproduction au niveau communautaire dans la région de Kayes. Rapport final. [evaluation of the Community-Level Support for Reproductive Health in the Kayes Region project] December 2008

USI and coll.. Renforcement du système de santé décentralisé au Mali. Plan de mise en œuvre. [Strengthening Decentralized Health Systems in Mali – implementation plan] November 2010

USI. Projet d'appui à l'amélioration de la santé de la reproduction au niveau communautaire dans la région de Kayes au Mali. Rapport d'avancement janvier-août 2010 et rapport final. [Community-Level Support for Reproductive Health in the Kayes Region project in Mali – January-August 2010 progress report and final report] (no date)

A031027-003 Paramedics

CATEK Gouvernance et Développement. Suivi des diplômés de l'INFSS. Version provisoire. [follow-up of INFSS graduates - draft] Bamako. May 2011

CIDA. Memorandum for the Minister. Paramedic Training Support in Mali - ML-31027-003. Hull. May 2005

CIDA. Project Approval Document. Paramedic Training Support in Mali - ML-31027-003. Hull. February 2005

Ministry of Health and INFSS. Appui à la formation des paramédicaux au Mali. Plan de mise en œuvre du projet. Version préliminaire. [Paramedic training support in Mali. Project implementation plan. Draft] November 2004

Ministry of Health and INFSS. Appui à la formation des paramédicaux au Mali. Rapport de fin de projet. Version finale. [Paramedic training support in Mali. End-of-project report. Final version] November 2010

Sicotte. Alfred and Oumar Traoré. Appui à la formation des paramédicaux au Mali. Évaluation à mi-parcours. Version finale. [Paramedic training support in Mali. Mid-term evaluation. Final version] February 2009

A032202-001 PDDSS Support

CIDA. Memorandum for the Minister. Appui à la mise en œuvre du PRODESS au Nord Mali [support in implementing PRODESS in northern Mali] - ML-32202. Hull. July 2004

CIDA. Project Approval Document. Appui à la mise en œuvre du PRODESS [support in implementing PRODESS] - ML-32202. Hull (no date)

Gao Region. Regional Health Directorate. Financement du Canada dans la région de novembre 2004 à septembre 2011. [Canadian funding of the region, November 2004 to September 2011] Document submitted to the CDPF evaluation team by the Gao regional director of health. October 2011

PSU-Mali. Suivi des activités des programmes de santé dans les trois régions du Nord-Mali - Kidal - Gao - Tombouctou. Rapport de mission (29 mars au 28 avril 2010). [monitoring of health program activities in the three regions of northern Mali – Kidal – Gao – Timbuktu – mission report (March 29-April 28, 2010)] Bamako. May 2010

M012726-001 PSI

CIDA. Memorandum to the Minister. Community-based management of malaria using artemisinin-based combination therapies / M-012726. Hull (no date)

PSI. CIDA Community Case Management Impact Grant (M-012726). Annual Report. November 1, 2008, to September 30, 2009. December 2009

PSI. CIDA Malaria Grant (M-012726). Annual Report. October 1, 2009 to September 30, 2010. December 2010

PSI. Improving home-based management of malaria using artemisinin-based combination therapy. A proposal submitted to CIDA by PSI on October 1, 2008

PSI. Mise en œuvre de la PECADOM dans la région de Ségou. [PECADOM implementation in the Ségou region] PowerPoint presentation of October 10, 2011

Food Security Sectoral Documents

CIDA

Note on food security. February 2011

Government of Mali

Agricultural development policy, 2011-2020. Draft. March 2011

Ministry of Agriculture. Livestock Production, and Fisheries. Food Security Commission. Secretariat of State, Niger Office. Report of the 10^th rural development sector project and program review. June 2010

Ministry of Agriculture. National Directorate of Agriculture. Plan for the 2010/11 crop year. February 2010

Ministry of Agriculture. Secretariat General. Secrétariat du Comité Préparatoire du Programme National d'Investissement dans le Secteur Agricole. Plan de passage à une approche sectorielle pour le développement agricole. [Secretariat of the planning committee of the national agricultural sector investment program. Plan for the transition to a sectoral approach to agricultural development] October 2008

Ministry of Agriculture. Secretariat General. Stratégie nationale de développement de la riziculture. [National rice growing development strategy] May 2009

Note on the progress of the PDA/PNISA process. No date

Technical and Financial Partners

2011 work plan of the agricultural and rural economy thematic group (GT EAR)

GT EAR terms of reference. January 2011

Minutes - GT EAR meeting – June 30, 2011

Speech by the lead TFP on food security to the national food security council. Bamako, March 25, 2011

Project Documentation

33359 Grain Market Restructuring (PRMC)

Food Security Commission. Final report on Canada's GMRP budget support project. No date

Monitoring mission report. February 2007

Project approval document (PAD). July 2006

RRP 2006/07

34673 Local Irrigation Support Project (PAIP)

Analysis of the organizational and technical capacity building needs of stakeholders in the local irrigation sub-sector. April 2009

GTZ. Approche du programme Mali-Nord pour le développement de l'irrigation de proximité. [approach to northern Mali's local irrigation development program] March 2009

Ministry of Agriculture. Document de projet du Programme d'aménagement prioritaire (PAP). [priority development program project document] March 2010

Ministry of Agriculture. Programme national d'irrigation de proximité (PNIP). [national local irrigation program] December 2010

National irrigation development strategy. 2007/08 rereading (to be completed)

PAD. Annexes. July 2009

32242 Support to agricultural supply chains in Mali (PAFA)

Ministry of Agriculture. Programme compétitivité et diversification agricole. Plan de compétitivité des filières du PCDA : un outil pour l'amélioration de la compétitivité des chaînes d'approvisionnement agricole. [competitiveness and agricultural diversification program – plan for the competitiveness of PCDA sectors – a tool to improve the competitiveness of agricultural supply chains] No date

Monitoring reports: No. 1, January 2010; No. 2, April 2010; No. 3, October 2010; No. 4, May 2011)

PAD. December 2007

Project implementation plan. November 2007

SNC-Lavalin. Annual report, April 2009 to March 2010

SNC-Lavalin. Annual project report. April 2010 to March 2011

SNC-Lavalin. Funding strategy. August 2011

SNC-Lavalin. Definitions and tools for implementing PAFA's results-based management (RBM) system. February 2011

SNC-Lavalin. Study on environmental issues in the shallot/onion/garlic sector

31895 Grain Marketing Support (PACCEM) Phase II

Canadian support agency (CSA) final report. October 2009

Étude et analyse prospective du système collectif de mise en marché de Faso Jigi/PACCEM [prospective study and analysis of the Faso Jigi/PACCEM collective marketing system]. Guy Debailleul. Nicole St-Martin. Boubacar Bah. April 2009

Monitor's feedback on the final report. November 2009

Faso Jigi. Annual report, 2010/11 crop year

S0663682 SOCODEVI – Cooperative Partnership Program in Mali

Annual status report from April 1, 2010, to March 31, 2011

Centre d'appui à l'entrepreneuriat collectif pour les jeunes [centre to support collective entrepreneurship for youth]. 2008-2011 activity report

S064494 Volunteer Sending Program – CECI-WUSC (Uniterra)

Annual narrative report, April 2010 to March 2011

CECI. Organization profile

S063931 USC – Overall Program 2007-2010

Annual Activities Plan 2010-2011. USC Mali

Goss Gilroy inc. Summative Evaluation of the Program. March 2010

Governance

Sectoral Documents

CIDA

CIDA. Canadian Partnership Branch (CPB). Mali. Civil society policy paper. October 2011.

Governance in Mali. 2011. Tables of reforms and stakeholders.

Justice sector analysis. January-December 2007

Justice sector analysis. Mid-year report. July-December 2005

Justice sector roadmap, October 2007

Mali policy framework – governance. Econotec inc.. May 2008

Project approval document (PAD). Support for the Mobilization of Internal Resources Project II (PAMORI II). Mali. September 2008.

Réflexion sur la logique/cohérence des interventions du Programme Mali dans le secteur de la Gouvernance. [reflection on the Mali program's logic/consistency in the governance sector] 2005

SUCO. Final report. MOPOD Phase 2. 2002-2008. November 2008.

Government of Mali

Ministry of Justice. La stratégie nationale de lutte contre la corruption et la délinquance économique et financière. [national strategy to combat corruption and economic/financial delinquency] October 2011

Ministry of Labour, the Public Service, and Government Reform. Plan national d'actions de mise en œuvre de recommandations issues des États généraux sur la corruption et la délinquance financière 2009-2013. [national action plan to implement recommendations arising from the summit on corruption and financial delinquency] February 2010

Technical and Financial Partners

2011 work program of the thematic group on justice and combating corruption (final version). January 2011

Minutes of the joint meeting of the Justice and Gender thematic groups regarding the Personal Status and Family Code

Thematic group on justice and combating corruption. Terms of reference. Updated April 30, 2010

Budget Support Documentation

CIDA. PRECABS Mali. Annual performance report for the period from April 2010 to March 2011. July 2011

Évaluation de la mise en œuvre du PAGAM/GFP sur la période 2006 – mi 2009 [evaluation of PAGAM/GFP implementation (2006 to mid-2009)]. Final report. November 2009. ADE. Brussels

Rapport d'évaluation de la mise en œuvre du PAGAM/GFP au titre de l'année 2010. [evaluation report on PAGAM/GFP implementation in 2010] Public sector financial management reform support unit (CARFIP). March 2011

Project Documentation

A031398001: Support for the Implementation of the Justice Development Program (PACMO-PRODEJ)

CIDA. Support for the implementation of PRODEJ. Project approval document (PAD) 2001

Diabaté, Amidou and Dupuis, Louise-Annie. Évaluation à mi-parcours du projet d'appui canadien à la mise en œuvre du PRODEJ (PACMO/PRODEJ). Rapport final. [mid-term evaluation of the Canadian project supporting the implementation of PRODEJ (PACMO/PRODEJ) – final report] February 2007.

Evaluation of the 2010 work program of the thematic group on justice and combating corruption

Génivar-University of Ottawa. Rapport final. Projet d'appui à la mise en œuvre du PRODEJ. [final report – PRODEJ implementation support project] December 2009. Consortium

Ménard, Ivan; Haag, Peter; and Nassar, Éric: Canada/Denmark/Mali joint planning mission in Mali's justice sector. October 2008

Ministry of Justice. Bilan 2000-2009. Programme décennal de développement de la Justice. [2000-2009 evaluation – Ten-year justice development program] December 2009.

A033910001: Knowledge Acquisition III (A033910/001-002)

Public-sector financial management reform support unit. Ministry of the Economy and Finance. Plan d'Action Gouvernemental pour l'Amélioration et la Modernisation de la Gestion des Finances Publiques : Phase II. [government action plan to improve and modernize public-sector financial management – Phase II] Final report. June 12, 2010

Support in developing Phase II of the government action plan to improve and modernize public-sector financial management (PAGAM/GFP). Lapointe-Martineau-Rouillard mission report. Bamako. March 14-27, 2010

A032514001: OAG Support Project (including Short-term Support A033940001)

2009-2013 action plan to implement recommendations arising from the summit on corruption and financial delinquency. Ministry of the Public Service and Government Reform. 2009.

Auditor General of Mali. 2007 annual report. Office of the Auditor General. February 2008

Auditor General of Mali. 2008 annual report. Office of the Auditor General. February 2009

Auditor General of Mali. 2009 annual report. Office of the Auditor General. February 2010

Auditor General of Mali. 2010 annual report. Office of the Auditor General. February 2011

Auditor General of Mali. Rapport Bilan avril 2004-mars 2011. [April 2004-March 2011 evaluation report] Office of the Auditor General. March 2011

Canada School of Public Service. Office of the Auditor General of Mali support project. 2010- 2011 annual report. July 2011

Cohen, Alain-Gérard and Bagayoko, Sinse, Consultants. Rapport d'élaboration d'une Stratégie nationale de contrôle interne. Phase 1 analyse de l'environnement. [report on developing a national internal control strategy – Phase I – environmental scan] Bamako. October 2010.

Gaudette. Yvan. Canada School of Public Service. Rapport sur une revue mi-parcours du projet d'appui au BVGM et sur un recensement de ses principaux besoins pour une prolongation éventuelle du projet de deux années (2012-2014). [report on a mid-term review of the OAGM support project and on a survey of its principal needs for a possible two-year extension of the project] June 2010.

General Control of Public Services. Prime Minister's Office. Stratégie Nationale du Contrôle Interne (SNCI). [national internal control strategy (SNCI)] July 2011

Ministry of Justice. Communication sur la Stratégie nationale de lutte contre la corruption et la délinquance financière. [communication on the national strategy to combat corruption and financial delinquency] October 2011

Modified West African Economic and Monetary Union (WAEMU) Treaty

Office of the Auditor General of Mali support project. Project approval document. September 2007

Prime Minister's Officer. Report on the implementation of recommendations arising from the 2009 annual report of the Office of the Auditor General. January 2011.

Sidibé. Konimba. Communication sur les piliers de la corruption au Mali. États généraux sur la corruption. [communication regarding the pillars of corruption in Mali – summit on corruption] November 2008.

A032220001 Institutional Development Program Support

Claude Bernier. Rapport de mission Valorisation et renforcement des ressources humaines. [mission report – valuing and strengthening human resources] Institutional Development Program. February 2011

IDP mid-term evaluation. Validation of preliminary findings. August 12, 2012

Institutional Development Commission. Annual report on the implementation of the IDP operational plan in 2010. June 2011.

Institutional Development Commission. Institutional Development Program. État de mise en œuvre en 2009 [implementation status in 2009]

Institutional Development Commission. Institutional Development Program. État de mise en œuvre en 2008 [implementation status in 2008]

Institutional Development Program Support: Annex A- Project presentation. Annex B – Management strategy. Logical framework analysis. CIDA. 2007

Martin, Louisette, gender equality consultant. Appui en égalité entre les femmes et les hommes dans le cadre du Plan opérationnel du Programme de développement institutionnel (PO/PDI) 2010-2013. [support for gender equality in the Institutional Development Program's operational plan] Mission and end-of-mission report. June 2011.

Ministry of Government Reform. Institutional Development Commission. Rapport synthétique du Bilan du Plan Opérationnel du PDI 2006 – 2009. [summary report on the evaluation of the 2006-2009 IDP operational plan] June 2011

A033735PRG: Municipal Partnership Program 2007-2010 – Federation of Canadian Municipalities (FCM)

FCM Municipal Partnership Program 2007-2010. Volume 2 – Annexes. June 2007 – July 2010. December 16. 2010

Results Based Management Group. Évaluation du Programme des partenariats municipaux (PPM) 2007-2010 de la Fédération canadienne des municipalités (FCM) et la comparaison des modalités de programmation. Rapport d'évaluation final. [evaluation of the 2007-2010 Municipal Partnership Program of the Federation of Canadian Municipalities (MPP) and comparison of program terms and conditions – final evaluation report] February 2010.

Equality Between Women and Men

Thematic Documents

CIDA

Analysis of the report of the Task Force on the Personal Status and Family Code. May 2008

Assessment of Canada's mandate as leader of the technical and financial partners' thematic group on gender (2007-2010). 2011

CIDA. CIDA's Policy on Gender Equality

CIDA. Gender equality sector analysis. 2009.

CIDA. Gender equality sector analysis. January-December 2007

Mali Program. Gender equality action plan 2010-2015. Final version. June 2010.

Mali Program. Integrating gender equality in programming. 2011 retreat

Government of Mali

Ministry for the Promotion of Women, Children, and the Family. Action plan - Mali's national gender policy. 2011-2013

Ministry for the Promotion of Women, Children, and the Family. Mali's national gender policy. 2011

Technical and Financial Partners

Dr. Édouard Talnan. Genre. croissance économique et croissance démographique au Mali. [Gender, economic growth, and population growth in Mali] UN Women. UNFPA. UNICEF. July 2011

Gender thematic group: 2008, 2009, and 2010 evaluations

TFP Gender and Development thematic group: 2011 work program

Project Documentation

A032415001: Gender Equality (PROJES)

Ali Anwer, economist, and Mamadou Diallo, GE specialist. Évaluation à mi-parcours Projet d'appui à l'égalité entre les femmes et les hommes (PROJES) [mid-term evaluation, gender equality support project (PROJES)] May 2010.

Plan de mise en œuvre octobre 2007 – octobre 2011. Projet d'appui à l'égalité entre les femmes et les hommes (PROJES). [October 2007 – October 2011 implementation plan – gender equality support project (PROJES)] CECI. July 2008.

Project approval document (PAD) 2005

Rapport annuel An 3 (avril 2010-mars 2011). [Year 3 (April 2010 – March 2011) annual report] PROJES. CECI. April 2011

Microfinance Sectoral Documents

CIDA

Microfinance sector analysis. January-December 2007

Government of Mali

Rapport annuel 2009 sur l'évolution du secteur de la microfinance au Mali. [2009 annual report on the development of the microfinance sector in Mali] CCS/FD. MEF. December 2010

Stratégie nationale et Plan d'actions pour le développement de la microfinance 2008-2012. [2008-2012 national microfinance development strategy and action plan] 2008 Ministry of the Economy, Industry, and Trade

Project Documentation

A032992001 Microfinance Sector Support (PASMIF)

Étude sur l'état des lieux des SFD en difficultés ou en voie de l'être. Rapport définitif. Cellule de coordination et de surveillance des SFD. MEF. [Situational analysis of failed or failing decentralized financial systems. Final report. Decentralized financial system coordination and monitoring unit. Ministry of the Economy and Finance] August 2010

Évaluation à mi-parcours du Plan d'action de la Stratégie nationale de la microfinance. Rapport provisoire. [Mid-term evaluation of the national microfinance strategy action plan. Interim report.] Luc Lefèvre. IRAM - Paris and Cheikh Amadou Diop. DCEG-Dakar. MIIC and MEF. November 2011.

Joint mid-term review. Program to help promote employment in Mali's private sector. PAPESPRIM. Program review memorandum. Government of Mali. Government of Denmark/Danida. November 19, 2010.

Mécanisme de financement commun, rapport technique et financier des OEs 2008-2009. [pooled funding mechanism – technical and financial report of executing agencies, 2008-2009] General Secretariat. Centre for the promotion and support of decentralized financial systems (CPA/SFD). Ministry of Industry, Investment, and Trade. February 2010.

Plan de gestion et priorisation des actions d'assainissement du secteur de la microfinance. Rapport provisoire. [Plan to manage and prioritize activities to reorganize the microfinance sector. Interim report.] CCS/FD. MEF. November 2011

Rapport 1^er semestre 2011 du MFC. [pooled funding mechanism report, 1^st quarter, 2011] CPA/SFD. Ministry of Industry, Investment, and Trade. August 2011.

Revue conjointe du Mécanisme de Financement Commun mis en place dans le cadre de la mise en œuvre de la SNDMF/PA 2008-2012. Aide Mémoire de revue du programme. [joint review of the pooled funding mechanism put in place to implement SNDMF/PA 2008-2012] General Secretariat. Ministry of Industry, Investment, and Trade. May 2010.

Triodos Facet. Rapport de mission. Mécanisme de financement commun – Mali. 4ième Mission : 5 au 16 septembre 2011. [Mission report. Pooled funding mechanism – Mali. Fourth mission: September 5-16, 2011] Danida

S063682PRG: Building the Future: Poverty Reduction - Développement international Desjardins (DID)

Développement international Desjardins (DID). Rapport de fin de programme du programme DID 2007-2011 Bâtir l´avenir : la réduction de la pauvreté par un meilleur accès aux ressources financières. [DID 2007-2011 end-of-program report – Building the future: poverty reduction through better access to financial resources] June 2011

Éconotec. Rapport Final d'évaluation à mi-parcours du programme DID 2007-2011 Bâtir l´avenir : la réduction de la pauvreté par un meilleur accès aux ressources financières. [Final report – Mid-term evaluation of the 2007-2011 DID program - Building the future: poverty reduction through better access to financial resources] November 2010

EnvironmentSectoral Documents

CIDA

Environmental action plan. Mali bilateral program. 2011/16. July 2011

Mission report. Support in developing the environmental action plan for CIDA's bilateral program in Mali. Christian Alix. February 16-26, 2010

Proposed environmental action plan (EAP) for the Mali bilateral program for 2009/10. Christian Alix. Undated document

Strategic environmental assessment report. Programming Framework for Mali. 2010/15. Version of November 2009

Government of Mali

Ministry of the Environment and Sanitation. Assises nationales de l'environnement. Cadre stratégique national de l'environnement. [Environmental conference. National environmental policy framework] Document validated by the National Forum of July 8, 9, and 10, 2010

Other Agencies

Overseas Development Institute. Budget Support, Aid Instruments and the Environment: The Country Context. Mali Country Case Study. Andrew Lawson and Seydou Bouaré. January 2008

UNEP. UNDP. SBA. Economic analysis of the environmental management in Mali : Costs and benefits. Final report. February 2009

Annex 6: Persons Consulted

CIDA

Rémy Beaulieu - Manager. Evaluation Division - CIDA Gatineau

Denis Marcheterre - Manager. Evaluation Division - CIDA Gatineau

Yves Boulanger - Consultant, Evaluation Division - CIDA Gatineau

Josée Fluet - Assistant Director. Mali Program - CIDA Gatineau

Louis Verret - Director - CIDA Bamako

Yves Pétillon - Former Program Director - CIDA

Darquis Gagné - Former Program Assistant Director - CIDA Gatineau

Marie-Claude Harvey - Analyst. Mali Program - CIDA Gatineau

Christian Alix - Environment Specialist - CIDA Gatineau

Maroun Abou Fayssal - Senior Development Officer - CIDA Gatineau

François Dupuis - Health Program Officer - CIDA Gatineau

Catherine Bernier - Education / Gender / CSO Program Officer - CIDA Gatineau

Françoise Nduwimana - Gender Equality Specialist - CIDA Gatineau

Simon Boivin - Program Manager, Agriculture - CIDA Gatineau

Patricia Miaro - Education Specialist. Bilateral - CIDA Gatineau

Liziane Boisvert - Program Consultant, Health/Education - CIDA Gatineau

Nadia Hamel - Health Initiatives Manager (AHSI) - CIDA Gatineau

Wassala Nimaga - Aid Effectiveness Specialist, PWCB - CIDA Gatineau

Réal Lavergne - Manager, Aid Effectiveness, PWCB - CIDA Gatineau

Catherine Lapeyrie - Project Advisor, PWCB - CIDA Gatineau

Renata Pistone - Former UNICEF / Health Officer, MGPB - CIDA Gatineau

Claude Goulet - Former Assistant Director, Mali - CIDA Gatineau

Micheline Gilbert - Former Analyst - CIDA Gatineau

Christiane Vekeman - Former Program Officer, Health/Education - CIDA Gatineau

Virginie Saint-Louis - Ambassador - Canadian Embassy

Abebech Assefa - Assistant Director and Children and Youth Coordinator - CIDA Bamako

Jan Jakobiec - Development Officer, Food Security - CIDA Bamako

Hamadoun Tolo - Development Officer, Children and Youth - CIDA Bamako

Alice Birnbaum - Development Officer, Children and Youth - CIDA Bamako

Mme Dagmo Nour - Development Officer, Children and Youth - CIDA Bamako

Cathy Tremblay - Development Officer, Governance - CIDA Bamako

François-Philippe Dubé - Development Officer - CIDA Bamako

Alex Carrasco - Development Officer, Food Security - CIDA Bamako

Diamilatou Sangaré - Agriculture & Rural Development Advisor - CIDA Bamako

Pierre Beauchemin - Director - PSU Bamako

Abdoulaye Ky - Education Advisor - PSU Bamako

Mahamadou Diarra - Governance Advisor - PSU Bamako

Nouhoum Sankaré - Economic Advisor - PSU Bamako

Cheick Sadibou Keita - Rural Development Advisor - PSU Bamako

Ouassa Sanogo - Health Advisor - PSU Bamako

Amadou Bengaly - Financial Management Advisor - PSU Bamako

Mariam Kane - Gender Equality Advisor - PSU Bamako

Mamadou Diarra - Rural Economy Advisor - PSU Bamako

Samba Tembely - Civil Society Advisor - PSU Bamako

Canadian support agencies and NGOs

Canada

Hugues Lavoie - Mali Education Project Manager - CRC-Sogema

Sylvie Charron - Mali Health Project Officer - USI - Université de Montréal

Maxime Prudhomme - Program Manager - SOCODEVI (telephone interview)

Philippe Fragnier - Program Manager - UNITERRA (telephone interview)

Normand Jacob - PACCEM Project Director - UPA-DI (telephone interview)

Sarah Dalle - Project Manager - USC Canada (telephone interview)

Pierre Robert - PACMO-PRODEJ Project Director - Génivar

Louise Camiré - PROJES Project Director - CECI

Jean-Philippe Lapointe - IDP Technical Consultant

Louise Deslauriers - Canadian PRECABS Project Director - CRC-Sogema

Édith Gingras - MPP Project Manager - Federation of Canadian Municipalities

Sébastien Hamel - MPP Project Manager - Federation of Canadian Municipalities

Bamako, Mali

Mireille Trudel - Project Director - DÉCLIC / CCISD

Henri Bérubé - PAFA Project Coordinator - SNC-Lavalin

Marcelle St-Pierre - National Field Coordinator - CCNB Campus de Dieppe

Moussa Coulibaly - Education Officer - Plan-Mali

Aguibou Coulibaly - Health Officer - Bamako

Ms. Rodio Diallo - Resident Representative - PSI

Mr. Coulibaly - Program Assistant - PSI

Ms. Sow - Executive Assistant - PSI

Boris Fidèle Degan - National Director - Right to Play

Amadou Cissé - Program Director - Right to Play

Mohammed Gadiaga - Administrative and Financial Representative - Right to Play

Arianne Cartwright - Communication Specialist - Right to Play

Assa Keita - Gender Advisor - Right to Play

Alassane Sanogo - Director and Coordinator - CECI Mali

Souleymane Traoré - Program manager - CECI-Mali

Abdramane Goïta - Douentza Program Director - USC Canada

Moussa Sissoko - Douentza Financial and Administrative Officer - USC Canada

Modibo Goïta - Manager, Technical Aspects. West Africa - USC Canada

Anne Maltais - Representative - SUCO

Michel Sansfaçon - Program Manager - SUCO

Mamdou Ba - Microfinance Advisor - SUCO

Karime Traoré - Development Advisor and MOPOD Officer - SUCO

Aoua Djiré - PROJES Project Coordinator - CECI

Oumou Diallo - PROJES Assistant Coordinator - CECI

Alain Beauregard - Operations Advisor - Développement international Desjardins

Mali - Kayes Region

Soumaïla Diakité - Health Coord. / Kayes & Northern Regions - USI - Université de Montréal

Modibo Diarra - Textbook Repair Project Coordinator - New Brunswick

Mali - Ségou Region

Dr. Bangaly Cissé - PAFA Branch manager - Antenne de Niono

Tiécoura Traore - PAFA Marketing Specialist - Antenne de Niono

Christophe Brismé - Partner Support Services - SOS Faim

Faliri Boly - Director - Sexagon

Souleymane Traoré - Coordinator - Alphalog

Oumkou Cissé - Facilitator - Alphalog

Women's groups - Shallot producers - Village of Sangarébougou

Technical and Financial Partners

Kanyankore Marcel Rudasingwa - Resident Representative - UNICEF

Tiécoura Sidibé - Child Survival Program Manager - UNICEF

Mr. Thiam - ÉAR Thematic Group Member - WFP

Aboubacar Guindo - School Feeding Officer - WFP

Issa Diakité - Sub-office Manager. Kayes - WFP

Sarmoye Cissé - Health Specialist - WHO

Moussa Keita - Health Specialist - WHO

Joris van Bommel - Representative - Netherlands

Dr. Traoré - Health Specialist - Netherlands

Cheick Diop - Macro-economics group member - World Bank

Torotoumou Konfe Traoré - Education Specialist - USAID

Salif Coulibaly - Health Specialist - USAID

Rebecca Black - Representative - USAID

Fatou Keita - Gender Equality Officer - UNESCO

Quality technical group - Education Sector - Multi TFPs and MÉALN

Luc Risch - Development Cooperation Advisor - Belgian embassy

Birgit Joussen - First Secretary - German embassy

Tiina Huvio - Agriculture Program Advisor - Royal Danish embassy

Geneviève Federspiel - Resident Director - Swiss cooperation

Luc Grégoire - Resident Representative - UNDP

Pierre Amadou Nébié - TFP Technical Pool Coordinator - UNDP

Yaya Diarra - Technical advisor - GIZ

Alain Féler - Resident Representative - IMF

Olivier Benon - Director - West AFRITAC. IMF

Désiré Ballo - Human Rights Program Manager - Swedish embassy

Charlotte Just - Advisor. Microfinance - Royal Danish embassy

Ann Wesling - Technical Advisor. Microfinance - Danida

Soetkin Meertens - Representative - UN Women

Rachelle Mian - Advisor - UN Women

Florence Duvieusart - Development Cooperation Advisor - Belgian embassy

Keita Sokomna Sissoko - Advisor - Swiss cooperation

Hamadoun Pergourou - Advisor - French embassy

Désiré Ballo - Advisor - Swedish embassy

Government of Mali and Public Institutions / In the Capital

Fidèle Diarra - Director, Americas - Ministry of Foreign Affairs and International Cooperation (MAECI)

Sékouba Diarra - Coordinator - PRSP Technical Unit

Mamadou Amadou Dembélé - Director - Aid Harmonization Secretariat (SHA)

Denis Dougnon - Ségal - Ministry of Education. Literacy, and National Languages (MÉALN)

Bonaventure Maïga - Technical Advisor - MÉALN

Mamadou Diabaté - National Director - DNEF / MÉALN

Bakary Samaké - Director - DEN / MÉALN

Moussa Diaby - National Director - Non-formal education / MÉALN

Alpha Aboubacrine - Director General - CPS / MÉALN

Abou Diarra - Chief - CAD /DE / MÉALN

Fili Fofana Traoré - Executive Assistant - CAD /DE / MÉALN

Hama Boubacar Traoré - Director General - DFM / MÉALN

Simbo Tounkara - Chief. Finance Division - DFM / MÉALN

Youba Ba - Former Director, DFM - MÉALN

Touré Zalia Maïga - Director - National School Feeding Centre (CNAS) / MÉALN

Moctar Haïdara - Assistant - CNAS / MÉALN

Mamadou Sissouma - National Educational Director - MÉALN

Fofana Bouraïma - Chief, Textbook Division - MÉALN

Dr. N'Golo Konaté - National Technical Advisor - Textbook repair

Ousmane Touré - Ségal - Ministry of Health (MS)

Ibrahima Coulibaly - Technical Advisor to the Minister - MS

Salif Samaké - Director - CPS / MS

Mamadou Sidibé - Director - DRH / MS

Alhousseïni Ag Mohamed - Director General - INFSS

Dicko Fatoumata Maïga - Assistant Director General - INFSS

INFSS team - INFSS

Sidi Sosso Diarra - Former Auditor General - OAG Mali

Mamadou Daouda Traoré - Acting AG - OAG Mali

Mr. Doubia - Assistant DAF - OAG Mali

Fousseyni Mariko - Director. CPS - Ministry of Agriculture

Daniel Siméon Kelema - National Agricultural Director - Ministry of Agriculture

Yaya Nouhoum Tamboura - Commissioner - Food Security Commission

Dicko Bassa Diane - Deputy Commissioner - Food Security Commission

Oumar Traoré - PRMC Technical Secretary - Food Security Commission

Bakary Diallo - Director - OPAM

Mamadou Gakou - Director General - Environmental and Sustainable Development Agency

Soumaïla Diakité - Director - National Directorate of Rural Engineering

Souleymane Traoré - Chief. Agricultural Development Division - National Directorate of Rural Engineering

Mamadou Diakité - Chief. Training Division - National Directorate of Rural Engineering

Anne Hofer Camara - Senior Analyst, BPAS - CPA. Min. of Industry, Invest­ment, and Trade (MIIC)

Badou Hasseye Traoré - Ségal - Ministry of Justice

Mr. Maiga - Director, Planning and Statistical Unit - Ministry of Justice

Brehima Mamadou Diallo - Chief, PRODEJ Coordination Unit - Ministry of Justice

Bayouni Dembélé - Assistant Director, CPS - Ministry of Justice

Maky Traoré - Chief, Programming & Monitoring Directorate - Ministry of Justice

Mr. Diallo - Chief, Legal Data Analysis - Ministry of Justice

Mr. Maiga - President, Correctional Tribunal, Commune III - Min. of Justice

Aroun Keita - Justice of the Peace, Court of Dioïla - Min. of Justice

Amadou Gadiaga - Comptroller General - Comptroller General of Public Services

Abdramane Touré - Ségal - MTFP

Sidi Traoré - Director - DNFPP. MTFP

Berthé Mariétou Makalou - Ségal - Ministry of Government Reform (MRE)

Mamadou Sissoko - Head, Institutional Reform Mission - Ministry of Government Reform (MRE)

Ousmane Oumarou Sidibé - Institutional Development Commissioner - CDI

Dionké Diarra - Project Director - PRECABS

Samba Diallo - Economist - PRECABS

Bounafou Touré - CARFIP Coordinator - Public-sector financial management reform support unit (CARFIP)

Marimpa Samoura - Inspector of the Treasury - CARFIP

Abdoulaye Traoré - CARFIP

Aoua Sylla - CARFIP

Boubacar Balahira - CARFIP

Boubacar Sow - Ségal - Ministry of Territorial Administration and Local Communities (MATCL)

Boubacar Diarra - Executive Assistant - MATCL

Allaye Diall - Coordinator, Electoral Process Support Unit - MATCL

Modibo Keita - Former Prime Minister - Government of Mali

Oualy Diawara - Chair, Finance Committee - National Assembly

Mody N'Diaye - Rapporteur General, Finance Committee - National Assembly

Diarra Kadiatou Samoura - Ségal - Min. for the Promotion of Women, Children, and the Family (MPFEF)

Bintou Nimaga - Gender Technical Advisor - Min. for the Promotion of Women, Children, and the Family (MPFEF)

Haidara Aissata - Acting Director, Promotion of Women - Min. for the Promotion of Women, Children, and the Family (MPFEF)

Diané Mariam Koné - National Centre for Documentation and Information about Women and Children - Min. for the Promotion of Women, Children, and the Family (MPFEF)

Samba Sissoko - Chief, Monitoring and Supervision Unit - CCS/SFD- MÉF

Karim Kamissoko - CCS/SFD- MÉF

Madani Traoré - Coordinator, Support and Promotion Centre - CPA/SFD. MIIC

Moctar Ag Abdoulaye - CPA/SFD. MIIC

Government of Mali and Public Institutions - Regions

Kayes Region

Moussa Ali Maïga - Advisor, Economic and Financial Affairs - Karité Project

Alhousseyni Amadou Maïga - Director and Former Treasurer Payer - DRB / Kayes

Sissouma Edouard - Regional Representative - ANICT / Kayes

Dr. Yattara - Director - DRS / Kayes

Abdoulaye Maïga - Director - DRDSS / Kayes

Kimba Camara - Former Director, DRDSS, Kayes - DRDSS / Kayes

Acting Chief Medical Officer - Referral Hospital - Kayes

Sylvain Keita - Planning Manager - DRS / Kayes

Amadou Koulibaly - Accountant - DRS / Kayes

DRS and DRDSS teams - Kayes

Dr. Omar Magassa - Chief Medical Officer - CSCOM / Kayes-Di

Dr. Ousmane Diawara - Area Medical Officer - CSCOM / Kayes-Di

Maiga Adam - Midwife - CSCOM / Kayes-Di

Kaoudo Tangara - Director - DRS / Gao

Bouréhima Pléa - Director - DRS / Timbuktu

Zakaria Traoré - Director - Teaching Academy (AE) / Kayes

Mr. Dramé - Assistant to the Director - AE / Kayes

Hamidou Diakité - Chief, Planning, Examinations. Competitions, and Statistics Division - AE / Kayes

Oumar Diakité - Chief, Basic Education - AE / Kayes

Oumar Sissoko - Director - CAP Rive-Gauche / Kayes

Daouda Konaté - General Advisor - CAP Rive-Gauche / Kayes

Fatou Konaté - SCOFI Advisor - CAP Rive-Gauche / Kayes

Assiatou Soumaré Traoré - Director - Khasso School 1 / Kayes

Mahamadi Diallo - Teacher & Learning Community (CA) Member - Khasso School 1 / Kayes

Yamadou Sissoko - Teacher & Learning Community (CA) Member - Khasso School 1 / Kayes

Koina Ag Ahmadou. - Director / Nioro. Former DAE / Kidal - AE / Nioro du Sahel

Diarra Donzeye - Chief - Promotion of Children and the Family Division - Regional Directorate for the Promotion of Women / Kayes

Dramane Sangaré - Chief - Promotion of Children and the Family Program, and Manager / Kayes Circle - Regional Directorate for the Promotion of Women / Kayes

Ségou Region

Mr. Diawara - Regional Director - Ministry of Agriculture

Dioïla Circle

Waly Silamaka Sissoko - Sub-prefect - Dïolïa Circle

Joukounda Kanouté - Secretary General - Circle Council

Ibrahim Koné - Superintendent - Circle Council

Mamadou Konaté - Mayor - Kaladougou

10 members, incl. 5 women - Municipal Council - Kaladougou

Civil Society in the Capital

Ibrahim Balla Camara - Executive Director - Professional Association of Microfinance Institutions (APIM)

David Dao - President APIM - Assistant Director General: Kafo Jiginew

Sadi Diallo - APIM - APIM

Keita Estelle - Representative - CAFO

Traoré Nana Sisako - Representative - Pivotal group

Isssaka Keïta - President of the Bar - Malian Bar Association

Mamadou Baba Diallo - Legal Counsel / Secretary General - Ordre des avocats [law society]

Abdoulaye Sangaré - Treasurer - Ordre des avocats [law society]

Mamadou Samaké - Board Member - Ordre des avocats [law society]

Boubacar S. Chéra - Board Member - Ordre des avocats [law society]

Sylvain M. Keïta - President - Ordre des huissiers [order of bailiffs]

Moumouti Camara - President - National Chamber of Auctioneers

Discussion Group on Civil Society in Mali

Ms. Traoré - GT/GED FOSC - Forum of Civil Society Organizations

Nana Aicha Cissé - GT/GED FOSC - Forum of Civil Society Organizations (FOSC)

Fatou Doumbai - Association of Malian Women Lawyers

Bintou Founé - WILDAF

Bakary Doumbia - President, FECONG - FECONG, FOSC Secretariat

Mohamar El Moctar MAIGA - DESC Platform - Working Group / Justice and Human Rights

Abdullahi aj Mohammed - Working Group / Justice and Human Rights

Hamalal Keita - National Governance, Gender, and Citizenship Network

Mahamadiba Taoré - DEMESO Legal Aid Clinic

Madami Kounaré - Working Group / Justice and Human Rights

Haïdara Diahara Naïga - CAFO

Ibrahim Allassane Maïga - Lawyer - LJDH

Ida Hamadou Dicko - APETI Network

Nana Sissoko - GP/PCF

Dembélé Fatimata Doumbia - Lawyer - Association of Malian Lawyers

Ibarhima Sangho - President - APEM Network

Amadou Diarisso - Assistant Secretary General - FENASCOM

Traoré Nana Sissako - GP/DCF

Ahmed Sekou Diallo - SECO-ONG

Amadou Tiéoulé Diarra - LJDH

Bakary Traoré - SOCODEVI

Anne Maltais - SUCO

Lamine Diaou - SUCO

Mobido Goîta - USC/Canada

Lyne Caron - SAHEL 21

Al-Assane Sanogo - CECI

Meeting with Media Representatives

Adam Thiam - Journalist

Chana Takiou - Journalist - Newspaper: 22 Septembre

Yoro Diallo - Journalist - Newspaper: Le Tambour

Alexy Kalambry - Journalist - Newspaper: Les Echos

Sékouba Samaké - Journalist - Newspaper: Infomatin

Civil Society - Regions

Kayes Region

Badji Kanté - 1st VP, Finance, Education. Social Affairs, Politics, and Religion - Circle Council / Kayes

Sadio Diallo - 2nd VP, Development Issues (Mayor's Office) - Circle Council / Kayes

Moctar Doucouré - Ségal - Circle Council / Kayes

Bah Oumou Fomba - Mayor's Advisor to the Health Committee, Education Advisor to the Circle - Circle Council / Kayes

Touré Fatimata Sissoko - Representative, Mayor's Education Committee, School Principal - Circle Council / Kayes

Aliou Dem Bah - Chair of the Board - ASACO / Kayes-Di

Alassane Sangaré - Treasurer General - ASACO / Kayes-Di

Banel Diawara - Social Affairs Officer - ASACO / Kayes-Di

Aissata Fofana - Organizer and Awareness Officer - ASACO / Kayes-Di

M'Pié Zan Togola - Secretary, Demands – Teachers' Union - FEN / Kayes

Idrissa Keita - Assistant Ségal – Teachers' Union - SNEC / Kayes

Ibrahima Wattara - Ségal - Teachers' Union - SNEB / Kayes

Mamadou Diong - Spokesperson - Malian Association for the Protection and Promotion of the Family / Kayes

Kélé Maciré Coulibaly - Chair, School Management Council (CGS) - Khasso School 1 / Kayes

Ségou Region

Nouhoum Lah - Engineer, Chargé de Mission - Niger Office

Boré Fatoumata Lamine Traoré - Engineer, Chargée de Mission - Niger Office

Tidiani Traoré - Engineer, Director, Water Management and Network Maintenance - Niger Office

Ilias Dogoroum Goro - Engineer, Director, Rural Support - Niger Office

Kassim Coulibaly - Chair of the Board - Faso Jigi

Boubacar Soumaré - Accountant and Financial Advisor - Faso Jigi

Dramane Coulibaly - Administrative Secretary - Faso Jigi

Ousmane Traoré - Chair, Monitoring Committee - Faso Jigi

Hassane Attara - First Secretary. Information/Training - Faso Jigi

Moussa Diarra - Commercial Secretary - Faso Jigi

Adza Traoré - Third Secretary, Marketing - Faso Jigi

Kalifa Dembélé - Executive Secretary - Association for the Promotion of the Rural and Urban Sectors (ASPRU)

Sitan Traoré - Monitoring Officer - ASPRU

Kondé Diahara Thomas - Program Manager - Karité [shea butter] Project

Moussa Maiga - Karité [shea butter] Project

Yakouréoun Diarra - Karité [shea butter] Project

Project Monitoring Officers

Marcel Ouellette - Textbook Project - (Telephone interview)

Roger de Winter - PACCEM Project

Gaston Grenier - PAFA Project - (Telephone interview)

Louise-Annie Dupuis - PASMIF Project

Gérard Gagnon - Public-Sector Financial Management Project

Government Participants and Partners at the Mission Debriefing Meeting of October 14, 2011

Fidèle Diarra - Ambassador/Director. Americas Division - MAECI

Mohamed Diarra - Chief – North America Department - MAECI

Mamadou Traoré - Ségal - MIIC

Yaya Tamboura - Food Security Commissioner - CSA

Kassim Sidibé - Program Manager - CSA

Ousmane Oumarou Sidibé - Institutional Development Commissioner - CDI

Madoni Traoré - Coordinator - CPA/SFD

Moctar Ag Abdoulaye - SFD Promotional Officer - CPA/SFD

Mamadou Daouda Traoré - Office of the Auditor General

Hariatou Diarré - Technical Advisor - AMM

Denis Dougnon - Ségal - MÉALN

Badou Hasseye Traoré - Ségal - Ministry of Justice

Yafong Berthé - Ségal - Ministry of the Environment and Sanitation

Berthé Mariétou Makalou - Ségal - Ministry of Government Reform

Amadou Soussoko - Advisor - MATCL

Samaké Mouna Touré - CM / MA

Louise Annie Dupuis - PASMIF Project

Other invited guests (who did not sign the minutes)

Ousman Touré - Ségal - Ministry of Health

Diarra Kadiatou Samoura - Ségal - MPFEF

Abdramane Touré - Ségal - MTFP

Moussa Léo Sidibé - Ségal - Ministry of Agriculture

Boubacar Sow - Ségal - MATC

Boubacar Ba - President, AMM, Mayor of BKo Commune V

Amadou Ousmane Touré - Auditor General

Annex 7: Executive Summary of the Joint Evaluation of Budget Support in Mali

Joint Evaluation of Budget Support Operations in Mali
2003-2009

Final Report
Executive Summary
September 2001

Joint evaluation carried out on behalf of the Commission of the European Union, Belgium and Canada as a pilot exercise of the OECD Development Assistance Committee. Logo for the Royaume de Belgique, service public fédéral, Affaires étrangères, commerce extérieur et Coopération au Développement Logo for Agence canadienne de développement international

Joint evaluation carried out on behalf of the Commission of the European Union, Belgium and Canada as a pilot exercise of the OECD Development Assistance Committee.

Logo for the Royaume de Belgique, service public fédéral, Affaires étrangères, commerce extérieur et Coopération au Développement.

Executive Summary

Objectives and Scope of the Evaluation

The purpose of this study has been twofold:

1. to undertake an evaluation of the 32 (out of 33) budget support operations which have been conducted in Mali over 2003 to 2009 by ten Development Partners (DPs)^{Footnote 47} and to provide recommendations on the management of future budget support operations in Mali and elsewhere; and
2. to test a new methodological approach to the evaluation of budget support operations in order to assess its suitability.

The overall objective of the evaluation was: “to assess to what extent the budget support operations in question have been successful in providing the partner government with the means necessary to implement its national and sectoral development strategies, in facilitating improvements in the efficiency and effectiveness of these strategies, and, as a consequence, in attaining sustainable outcomes and impacts on growth and development“.

The Context for Budget Support in Mali

Mali is a land-locked country in the Sahel, with an estimated population of 14.5 million in 2009, and a population growth rate of 3.6% per year. Despite achieving an average annual GDP growth rate of 5% over 1998 to 2008, Mali still ranks among the poorer countries in the world with a Human Development Index of 0.309 in 2010 (160^th out of 169 countries in the HDI).

In spite of its limited economic diversification, its vulnerability to changing climatic conditions and a relatively unfavourable business climate, Mali has over the last ten years maintained a stable macroeconomic environment. The process of democratization, underway since the early 1990s, has generated peace and stability, whilst also promoting important social and institutional changes. On the basis of the Cadre stratégique de lutte contre la pauvreté 2002-2006 (CSLP – the PRSP) and the Cadre stratégique pour la croissance et la réduction de la pauvreté 2007-2011 (CSCRP – the poverty reduction and growth strategy), Mali has put in place key development programmes, which have contributed to the country’s improved socio-economic situation. A common country strategy for development assistance (SCAP) has been in place since September 2007, jointly developed by the Government and its Development Partners.

Within this context, the Malian government has sought to create the necessary conditions for the increased use of budget support. The following architecture for General Budget Support (GBS – ABG) and Sector Budget Support (SBS – ABS) has been put in place:

• the Framework Agreement of March 29, 2006, between the Government and 13 DPs;
• specific frameworks for GBS and for three SBS arrangements;
• bilateral agreements with each DP providing budget support.

Budget support to Mali has risen from 12% of Official Development Assistance (ODA) in 1999 to 42% in 2009, and the number of DPs with GBS or SBS operations has grown from three to ten. Total budget support disbursed during the period of the evaluation has amounted to FCFA 634 billion, some Euro 966 million.

Disbursements of Budget Support, Mali 2003-2009

Source: MEF. Bamako

The evaluation has covered the following 32 operations in Mali over 2003 to 2009:

• 17 GBS operations;
• one SBS supporting Decentralisation and Public Sector Reform;
• 9 SBS operations in the Education sector;
• one SBS operation supporting public finance management (PFM) reform; and
• 4 SBS operations for Health & Social Development.

The Evaluation Methodology

The methodological frame of reference has been provided by the new budget support evaluation methodology developed under the auspices of the OECD-DAC. The intervention logic comprises five levels inherent to all budget support operations: Level 1: Budget Support inputs; Level 2: Direct Outputs of Budget Support ; Level 3: Induced Outputs (Government outputs facilitated directly or indirectly by budget support); Level 4: Outcomes; Level 5: Wider-level Impacts.

The evaluation is based on 21 evaluation questions and 60 judgement criteria, each applied through a range of indicators adapted to the specific context of Mali.

Being based primarily on secondary data sources, the scope of this evaluation has been limited by the availability, the consistency and the quality of the available data. Nevertheless, there is no reason to doubt the validity of the trends, which have been identified and analysed. Quantitative data for the social sectors has been analysed applying econometric techniques, and all data has been cross-checked using qualitative methods. Initial hypotheses were verified through focus group discussions and interviews with resource persons, undertaken during the main field mission in October-November 2010. Analysis of the Decentralisation SBS was further strengthened through a small-scale beneficiary survey.

Principal Conclusions

The Principal Conclusions (PCs) are organised according to the three aspects of the evaluation’s overall objective:

Budget Support as a mechanism for financing the national development strategy

PC1. Over the evaluation period and most especially since 2006, budget support has provided the Government of Mali with a level of additional budgetary resources sufficient to constitute a critical mass of financing, which permitted a major increase in the level of discretionary spending and a fast expansion in the share of ODA managed through the national budget process.

PC2. Budget Support is the most predictable aid modality in Mali, with annual disbursements between 2002 and 2008 averaging 94% of projected disbursements. Nevertheless, the timing of disbursements by quarter, especially for SBS variable tranches, has been subject to regular delays, with considerable costs in terms of unplanned domestic borrowing and the delayed implementation of development programmes.

PC3. Due to the harmonisation efforts of the Government and the DPs, Budget Support has been the aid modality with the highest level of harmonisation and alignment in Mali, and very probably the modality with the lowest level of transaction costs per FCFA disbursed. Nevertheless, the annual process of assessment of the annual disbursement conditions for GBS and SBS is not yet fully harmonized and still generates unnecessary transaction costs.

PC4. Budget Support funds have been utilised in a manner consistent with the priorities identified in national and sectoral strategies, specifically in order to strengthen macroeconomic stability and increase spending in the priority sectors.

Budget Support as a mechanism to promote efficiency and effectiveness in the implementation of national policies

PC5. Due to its “critical mass” and its flexibility as a financing instrument, as well as the existence of the annual national strategy review that it helped to institutionalise over 2003 to 2009, Budget Support has assisted in improving the overall efficiency of resource allocation and in meeting the critical strategic requirements of the priority sectors.

PC6. GBS and SBS have contributed significantly to the establishment of structures for policy dialogue, through the introduction of a harmonised annual review calendar and through the provision of technical assistance support. These structures have facilitated the implementation and monitoring of national development programmes, particularly the CSLP/CSCRP. education and health sector policies, and policies to strengthen public finance management, decentralisation and public sector reform.

PC7. The conditionality and dialogue related to the strengthening of public finance management (PFM) helped to stimulate the establishment of the integrated PFM reform programme (PAGAM/ GFP) and the acceleration of its implementation. Despite the resulting improvements in the quality of PFM, important weaknesses still remain.

PC8. GBS and SBS conditionalities have had a decisive influence on the speed with which the Government proceeded with deconcentration and decentralisation. However, the evaluation identified no other clear-cut examples of policy changes or of accelerated implementation of policies resulting from Budget Support conditionality. However, several examples were identified of policy inconsistencies and increased transaction costs generated by conditionality.

Budget Support as a mechanism to facilitate the achievement of sustainable outcomes and impacts on growth and development

PC9. Given the prudent macroeconomic policy pursued by the Authorities, the financial contribution of Budget Support has been important in helping Mali to achieve, over the evaluation period, an average real rate of economic growth of 5% per annum, the highest attained within the West African Economic and Monetary Union (UEMOA-WAEMU).

PC10. Nominal per capita GDP has increased by 63% over the last ten years and, according to the most recent data available from household surveys, the incidence of poverty has fallen from 61% in 2000 to 51% in 2005, while the Gini coefficient has reduced from 0.400 in 2000 to 0.388 in 2005). Economic growth and low inflation seem to have been the determining factors in this process, aided in turn by the financial flows from Budget Support.

PC11. Between 2002 and 2009, the gross enrolment rate for basic (primary) education rose from 64% to 81%. Over the same period, the repetition rate fell (from 20% to 14%) and the completion rate increased (from 40% to 56%). Without the additional funding of SBS and GBS, it would have been impossible to finance the essential sector outputs (school buildings, teachers and textbooks) correlated with these achievements.

PC12. Between 2002 and 2009, the majority of the outcome indicators in the health sector have shown significant improvement (for example, the use of ante-natal services has increased from 54% in 2002 to 90% in 2009). GBS and SBS funds contributed in a major way to the financing of expanded sector infrastructure and GBS (on its own) to the financing of additional health personnel, each of which is closely correlated with the improved outcomes achieved.

Lessons on Budget Support modalities

The experience with Budget Support in Mali demonstrates above all that budget support – sectoral or general – is effective when its primary objective is to finance and monitor the implementation of a given policy. It is much less effective when its primary objective is to change a given policy or to adapt its contents, through the means of conditionality.

In Mali, the objective has been primarily the former rather than the latter, which has allowed GBS and SBS to make major contributions to the achievement of the goals of the poverty and growth strategy (CSCRP) and of sectoral strategies. However, when the intended objective has been to achieve changes in Mali’s national policies (through conditionality) – as was sometimes the case with policies on decentralisation, public sector reform or PFM reform – this has generated inconsistencies in policy implementation and high transaction costs in policy and budget support monitoring.

Thus, budget support functions best as a means of supporting a well-established national policy, for which there is a clear and consistent political commitment and for which implementation structures are in place, or can easily be established. When these essential elements are not in place, it is difficulty, if not impossible, to create them through budget support.

This lesson is applicable both to SBS and GBS. Indeed, the experience of Mali suggests that SBS and GBS are in practice remarkably similar, and in particular that they are subject to the same conditions for success.

Principal Recommendations

The most important contributions made by GBS and SBS in Mali have been their financial transfers and their support to the creation of structures for the coordination, programming and monitoring of policies and reform programmes (through the introduction of a harmonised annual calendar of policy reviews and through the provision of technical assistance support to the CSLP (PRSP) unit and the equivalent sectoral structures). Policy dialogue had less effect and was at times a source of confusion and unnecessary transaction costs, due to the efforts made to change or adapt national policies through conditionality.

In the future, the Government of Mali and its Development Partners should strive to strengthen further the benefits of the financial transfers made through budget support and of the support to policy coordination structures, while correcting past errors in the design and practice of policy dialogue:

• To strengthen the impact of Budget Support financial transfers, the following steps are required:
  • Increase the predictability of disbursements (in aggregate and in terms of timing within year).
  • Develop a harmonised monitoring framework for GBS and SBS, with a single set of indicators, so as to avoid duplications and inconsistencies.
  • Simplify the definition and assessment of indicators (particularly for variable tranches).
  • Increase the flexibility in the utilisation of SBS funds, by eliminating all earmarking requirements (tying disbursements to results rather than to budget lines).
  • Give priority to making disbursements in the first quarter of the year.
• To strengthen the effectiveness of structures for policy coordination, programming and monitoring:
  • Harmonise both the timing and the substance (including monitoring indicators) of the sectoral reviews, the global review of the poverty and growth strategy and the annual budget review (for GBS), so that the whole review framework might operate as a unified whole, with substantially reduced transaction costs.
  • Within the annual calendar, programme studies to evaluate priority policies and programmes and make available to Government the necessary technical assistance: (i) to support the implementation of such studies, and (ii) to strengthen internal capacity for policy review and analysis.
  • Increase the public availability of the information generated by annual reviews so as to promote public debate on development strategies and their outcomes.
• In order to strengthen the quality of policy dialogue, a process of reflection and self-assessment of current practices is needed. At present, dialogue is designed as a mechanism for monitoring disbursement conditions. Clearly, Development Partners need to ensure that the eligibility conditions for budget support are respected and that progress is made in the implementation of the policies supported by budget support, but policy dialogue should go beyond the mere tracking of disbursement conditions. It should also allow for the open exchange of ideas, in the interest of resolving problems of mutual concern [such as problems in policy design or implementation]. In order to establish structures and practices which would promote this type of dialogue, we recommend the following actions:
  • Redefine the objectives of policy dialogue so as to emphasize its contribution to national processes of policy design and implementation, as well as to the formulation and monitoring of the national budget.
  • Base the definition of “policy triggers” and disbursement indicators strictly on the implementation targets agreed with the Government for priority national policies and programmes.
  • Introduce, into the harmonised annual review calendar, space for reflection and debate on the implementation of national policies and programmes.

In order to eliminate the main obstacles to the implementation of the national development strategy, the following actions are recommended for the Government:

• In order to strengthen implementation capacity within the priority sectors, it would be prudent to give priority (in terms of timing and in the allocation of technical personnel) to the preparation of the sectoral MTEFs for education, health, rural development, social development and transport.
• In order to avoid human resource bottlenecks, exceptional measures need to be devised in advance to ensure the timely recruitment of qualified staff for the roles and the geographical locations where there is a high probability of shortages.
• Treasury management problems represent an almost persistent hindrance to the efficient execution of the State Budget. Both short- and medium-term measures need to be put in place to address this problem:
  • In the short term, measures need to be introduced, within the priority sectors, to speed up the processes of opening of budget credit lines, of commitment of expenditures, and of approval and payment of invoices. This might imply some streamlining of the current procedures but, even without any amendment of procedures, it would certainly entail training interventions to increase the efficiency of the Directorate of Financial Administration (DAF) and of the credit/warrant holders within the sectors. In addition, it would be necessary to remove any remaining earmarking requirements for SBS operations.
  • In the medium term, a programme of reform measures to strengthen treasury management and the related accounting processes should be prepared as a key component of the second PFM reform action plan (PAGAM/GFP II). The analyses of treasury management and accounting systems presented in the 2008 and 2009 annual reports of the Bureau du Vérificateur Général (BVG) and in the IMF’s reports of 2009 make it clear that the problem is not simply one of a lack of computerization. As a result of having a rigorous and inflexible system of controls over budget execution, the Malian public sector has over the years found ways to pass around the formal system – by using exceptional procedures (designed for emergency situations) for non-exceptional cases, or by establishing project accounts or autonomous government agencies which work outside of the system. A “cleaning up” of this situation is needed, which would entail a sharp reduction in the number of commercial bank accounts, stricter controls over the use of exceptional procedures, and the establishment of clearer rules for the setting up of autonomous government agencies. The PAGAM-GFP II does not include such a reform programme and continues to focus on the computerisation of expenditure control, a process that should logically come after the “cleaning up” of the existing anomalies.

Finally, it seems clear that the principal constraint to economic growth relates to the low level of private-sector investment. A realistic programme of measures needs to be put in place to improve the investment climate, aimed at addressing the constraints identified by national and international investors themselves. A new programme of reforms is required, based upon a detailed diagnosis of the problems.

Annex 8: Management response Evaluation of CIDA’s 2007-2011 Mali Program

The Mali Program of the Geographic Programs Branch (GPB), the Partnership with Canadians Branch (PWCB) and the Multilateral and Global Programs Branch (MGPB) would like to thank the team of consultants and the Evaluation Directorate for highlighting in this evaluation the significant work that the Canadian International Development Agency (CIDA) has done in Mali and for recognizing the efforts made. In addition to our responses to the recommendations, the Mali Program management team would like to take this opportunity to draw your attention to certain elements that are additional and complementary to the evaluation. These include the following:

1. The effectiveness of CIDA’s programming in Mali, its continuity in terms of the sectors supported, and Canada’s leadership role in policy dialogue in Mali have contributed to reducing poverty in Mali.
2. The major leadership role the cooperation program has played in coordination between donors and the Government of Mali has reinforced Canada’s visibility in Mali.
3. The program has been a key aid effectiveness catalyst thanks to its decentralized management and the high quality of its human resources. However, it would benefit from a delegation of authority commensurate with the degree of its decentralization.
4. To ensure the sustainability of its interventions, the Mali Program will have to maintain continuity in programming sectors and to use a variety of complementary implementation mechanisms, including program-based approaches.
5. The Program helped improve gender equality in Mali.

Recommendation #1: Keep the risk analysis and risk response strategies up to date.

Management response: Agree.

CIDA introduced the risk management methodology in 2008 and the Mali program has used it since 2009. Tools (including Mali’s risk profile) are updated at least once a year, and twice a year if necessary. In the current context environment of uncertainty, every effort will be made to ensure the risk analyses and risk response strategies are always up to date.

Of note is that the risk profile produced in November 2011 indicated three potential sources of risk that materialized in 2012: insecurity in northern Mali, the impact of climate change on food security, and challenges associated with a pre-election year. However, the cumulative effect of the three simultaneous sources of risk was greater than anticipated.

In recent years, the Mali Program has been cited repeatedly by management for the quality of its performance management tools (logic model, performance measurement framework, risk profile).

Commitments related to Recommendation #1:

1. Update the Mali risk profile at least once a year, and more often if circumstances warrant.
2. Provide for mitigation measures relating to risks that could affect the sustainability of results achieved.
3. Take the sustainability of results into account in planning new initiatives.
4. Press partners and executing and support agencies to take all necessary steps to ensure the sustainability of the results of their projects.
5. Maintain a variety of delivery approaches, to diversify risk and increase the likelihood of sustainability of CIDA programming results in Mali.

Responsibility for implementing commitments: Director

Implementation schedule: Ongoing

Implementation status: Ongoing

Recommendation #2: Systematically carry out risk-based monitoring and evaluation.

Management response: Agree. CIDA will look for opportunities to do more.

The field team systematically carries out regular monitoring of GPB, PWCB and MGPB initiatives taking into consideration the program’s risk analysis. The following are two recent examples:

• The Maternal, newborn and child health (MNCH) roundtable initiated by the PWCB in 2012 prompted the Mali program to create an innovative MNCH thematic network to facilitate monitoring, knowledge sharing and validation of indicators used by the organizations concerned. The lessons that will be drawn from this initiative can help determine the potential for replication of this monitoring framework in other program sectors.
• Evaluations are also conducted regularly and efforts are being made to conduct a growing number of joint evaluations (with other donors). The field shares the results of such evaluations with Headquarters and stakeholders concerned.

Commitments related to Recommendation #2:

1. Continue to monitor the progress of projects taking into consideration risk analysis.
2. Continue to encourage the creation of synergies among initiatives, in particular through the organization of quarterly sector-based meetings of stakeholders in a given sector (e.g. MNCH).
3. Look for opportunities to do more.

Responsibility for implementing commitments: Director

Implementation schedule: Ongoing

Implementation status: Ongoing

Recommendation #3: Carefully weigh whether the current and planned emphasis on gender equality is consistent with the leadership role Canada has already played in Mali.

Management response: Agree.

Although many of our projects in Mali have a significant gender component, we do recognize that, given the Gender Equality Project was the only project aimed specifically at gender equality, the end of that project meant “Canada’s partial abandonment of its leadership on gender equality” and that this was “strongly criticized” by the stakeholders met.

Recognizing the importance of specific projects aimed at gender equality, in addition to those in which gender equality is mainstreamed, and the role that women will need to play in rebuilding and in resolving the current conflict in Mali, GPB is planning to propose new initiatives in this regard in the coming months.

Canada will also remain actively involved in policy dialogue on issues related to gender equality.

Commitments related to Recommendation #3:

1. Continue implementation of the Gender Equality Action Plan — MALI (2010-2015) and the related annual work plans.
2. Allocate sufficient budgetary resources to take concrete action to advance gender equality within the program as a whole (project specific and gender mainstreaming).
3. Continue our policy dialogue on issues related to gender equality, including the implementation of the National Gender Policy, the participation of women in decision-making, women's rights, and women's access to resources and their control.

Responsibility for implementing commitments: Director

Implementation schedule: Ongoing

Implementation status: Ongoing

Recommendation #4: Consider placing greater emphasis on environmental sustainability, given the challenges that environmental threats could pose to sustained poverty reduction.

Management response: Agree.

Most new activities being planned, including those in the area of food security, will include budgetary allocations to allow for very concrete action to be taken in relation to the environment, with a view to ensuring the sustainability of our actions in order to take better account of Mali’s changing environmental context. Major issues addressed by the program will relate primarily to water management, land management, protection of natural resources and training. For instance, some new projects, whose implementation began in 2010-2011, have an explicit and strengthened environmental component. One such project is the Health Workers Training (DÉCLIC) project, which has an environmental resource person and supports the development of training modules that consider environmental management of health centres.

Despite its obvious importance, however, and while many projects will continue to have environmental components, the environment will not be the focus of specific GPB projects, since this theme is not among the priorities of CIDA’s Mali country strategy. The program will continue to implement its 2011-15 environmental action plan and its annual work plans. Project planning will also continue to include an environmental analysis.

Commitments related to Recommendation #4:

1. Continue to implement the 2011-2015 environmental action plan and the related annual work plans.
2. Allocate sufficient budgetary resources to take concrete action to advance the environmental sustainability of our projects, particularly in the area of food security.

Responsibility for implementing commitments: Director

Implementation schedule: Ongoing

Implementation status: Ongoing