Evaluation of Canada’s Development and Stabilization and Reconstruction Taskforce (START) Programming in South Sudan

January 2017

Acknowledgements

The Development Evaluation Division (PCE) would like to thank all of those who contributed to this evaluation, including colleagues from the program in headquarters and Juba, who provided support and advice throughout the evaluation process and who hosted the field missions to facilitate the data collection.

The design and collection of data for this evaluation was carried out by an independent evaluation team comprised of Louise Mailloux of Goss Gilroy Inc., Finn Pedersen of Tana Consulting ApS., Cathy Huser and John Oloya. The data analysis and reporting for this evaluation were carried out by Isabelle Mercier, Janis Grychowski and Denis Marcheterre of the Development Evaluation Division. PCE provided oversight and management throughout the evaluation process. The PCE Team Leads during this evaluation were Anne Lavender and Isabelle Mercier.

We would like to thank the many stakeholders and partners who participated in interviews in Canada and abroad and who helped to build a better understanding of Canada’s impact within South Sudan.

David Heath
Head of Development Evaluation, Global Affairs Canada

List of Acronyms

Executive Summary

This report presents the evaluation findings, conclusions, considerations and recommendations for future programming of Global Affairs Canada’s (GAC) development, stabilization and reconstruction (START) programming in South Sudan during the 2009/10 to 2015/16 period. This evaluation was conducted from October 2015 to December 2016.

The operating environment in South Sudan for the Canadian Development and START programs between 2009 and 2016 was marked by constant change and the presence of violent civil conflict. Canadian staff members working from the capital city of Juba were faced with security issues and uncertainty that required quick thinking, the development of strong ties with partners, flexibility and constant problem-solving. This report provides insight into the results of the South Sudan Development and START programs within this challenging context of extreme instability and uncertain circumstances.

Rationale, Purpose and Objectives

This evaluation of GAC programming in South Sudan was undertaken to ensure accountability and assess the Development and START programs’ relevance, effectiveness and efficiency in achieving their respective results. The evaluation covers the period 2009/10 to 2015/16. The evaluation was designed to support program improvement and provide a series of best practices and considerations for future GAC engagement in conflict-affected and fragile states.

The specific objectives of this evaluation were to:

1. Assess the relevance and performance (effectiveness and efficiency including cross-cutting issues) of GAC’s Development and START programming undertaken in South Sudan.^{Footnote 1}
2. To provide GAC with relevant fragile state-specific information for future programming, including best practices.^{Footnote 2}

South Sudan Program Context

GAC has provided development and humanitarian assistance to the people of what is now known as South Sudan since 2006.^{Footnote 3} Canada first identified Sudan as a priority fragile state following the creation of the Comprehensive Peace Agreement in 2006 that ended decades of war between the Government of Sudan and the Sudanese People’s Liberation Army, based in southern Sudan. Canada reconfirmed its commitment to the development of present-day South Sudan in 2014 when it was confirmed as a country of focus for international development engagement.

Canada focused the majority of its development and reconstruction activities on four states throughout the country: Eastern Equatoria State, Jonglei State, Northern Bahr el Ghazal State and Unity State. GAC investment in South Sudan included relief activities and international humanitarian assistance, stabilization and reconstruction activities, and development programming.^{Footnote 4}  The main focus of stabilization and reconstruction programming was security sector reform, conflict resolution and peace-building, mine action, and rule of law, and accounted for $46.5 million from 2009 – 2013. Development programming focused primarily on securing the lives of children and youth, sustainable economic growth, food security and advancing democracy, and accounted for $282.7 million from 2009/10 to 2015/16. In addition, GAC investments in the health of mothers, newborns and children through the Muskoka Initiative for Maternal, Newborn and Child Health (MNCH) of 2010, amounted to $102 million from 2010/11 to 2015/16.

Two years following Independence in December 2013, tensions escalated into civil war, producing devastating insecurity and violence and displacing over two million people. As a result, Canadian operations have increased programming in the area of international humanitarian assistance. Currently, Canadian investments in South Sudan include only development programming and humanitarian assistance, with stabilization and reconstruction programming having ended in 2013.

Methodology

The evaluation examined a purposeful sampling of a combined total of 29 Development and START projects. Information was collected through a literature review, a review of GAC program and project documentation, a review of previous GAC country program evaluations in fragile states, and key informant interviews (including GAC employees in Canada and Juba, partners from the Government of South Sudan and the United Nations agencies operating in South Sudan, think tanks and non-governmental organization personnel, and donor partner officials). Focus group discussions were conducted with locally engaged international partners, government officials and, in some cases, local beneficiaries.

As a fragile state, South Sudan was a challenging environment in which to carry out a full evaluation. Logistical constraints in South Sudan negatively impacted the collection of current and reliable data, particularly primary quantitative data using stakeholder or household surveys. More weight was therefore given to interviews and the literature/document review.

Main Messages

Development and START programming were relevant to the needs of South Sudan and aligned with the priorities of the Government of Canada. The evaluation found that development programming in South Sudan in the areas of health, especially maternal, newborn and child health; youth; governance; and food security were relevant to the acute needs of South Sudan and its people. While the focus of programming changed over the course of the evaluation period, investments always targeted the fundamental needs of the country. START’s focus on peace-building, security and rule of law were appropriately targeted to areas of great need. All programming in South Sudan, both Development and START, aligned with Government of Canada and Departmental priorities.

Despite the extreme challenges of working in South Sudan, GAC achieved outcomes. However it remains difficult to ascertain the extent of sustainability of these outcomes, given the fragile and changing context. The evaluation found evidence that some outcomes, particularly in health (through the Health Pooled Fund and maternal, newborn and child health (MNCH) investments) and food security were achieved during the time period of study. The evaluation also found evidence that START was able to achieve some outcomes. START was also an effective mechanism for beginning to bridge stabilization programming with more long-term development programming.

Canadian programming was generally aligned with many good practices for engaging in fragile states. However, more could have been done. GACs efforts to promote no harm; ensure non-discriminatory programming; prioritize prevention; and avoid pockets of exclusion, among other practices were in line with good practices for engaging in fragile state. While both Development and START programming integrated good practices for engaging fragile states, each program could have integrated these practices in a more structured and formal way. Specifically, GAC could have done more to establish stronger linkages between security and development objectives. More could also have been done to articulate and specify more predictable commitments to development sectors.

Canada worked effectively with other donors in South Sudan. Canada’s active participation in policy dialogue and pooled funds provide a valuable platform for dialogue on key issues. Canada remained an important donor in South Sudan throughout the evaluation period, though opportunities to influence policy discussion and direction afforded through START programming were diminished after the program ended.

Administrative requirements may have hindered the optimal delivery of relevant and effective programs. The evaluation found that internal systems designed for more traditional contexts may not have been appropriately tailored to a fragile state context. Requirements in place to manage fiduciary risks may have hindered the program’s ability to be flexible and nimble in a rapidly changing environment.

Programming in South Sudan and other fragile states could have benefited from a long-term vision and commitment that integrated GAC programming streams. A lack of integrated and linked priorities and the absence of a common understanding of the root causes of conflict resulted in missed opportunities for the Department to be more effective, efficient and responsive. As an additionally important element of long-term state-building in South Sudan, the evaluation found that some initiatives in development and START programming were at times, too short-term in vision, too focused on filling gaps rather than growing local capacity, and not necessarily linked to building capacity in the longer term.

There were limited resources and time available for the effective integration of some program requirements, particularly cross-cutting themes, drivers of conflict and the principles for engagement in fragile states. Expectations in implementing cross-cutting priorities, such as the environment, gender and governance, were not always congruent with resources in the field. The Canadian embassy in Juba would have benefited from the ability to view conflict as a fundamental priority in its development programming in South Sudan, and from having rapid and consistent access to specialists to assist with the development of conflict-sensitive programming that engaged the other cross-cutting themes of gender, environment and governance.

 Recommendations

1. GAC should ensure that future South Sudan programming is based on an integrated, whole-of-Department approach. Specifically, the continuum of programming should be based on:
   • A long-term, common, and documented vision for the country;
   • A recognition of the need for responsive, flexible and nimble programming to adapt to rapidly changing contexts;
   • The effective integration of cross-cutting themes; and,
   • An enhanced strategic analysis that addresses the drivers of conflict and is based in the Principles for Good International Engagement in Fragile States.
   These elements could be implemented by an approved and integrated country program strategy, an integrated logic model and a policy dialogue strategy.
2. GAC should ensure that any additional or refocusing of requirements for the integration, analysis and synthesis of cross-cutting issues and conflict drivers within fragile states be adequately supported by specialists.
3. GAC should ensure that it undertakes efforts to limit the delays in project and planning approval through a review and streamlining of processes.

Management Response

1. Introduction

This report is an independent evaluation of Canada’s development, stabilization and reconstruction (START)^{Footnote 8} programs in South Sudan. It includes field-level data collected in South Sudan in March 2016. This report draws on the emerging international consensus on the planning, management and assessment of development cooperation efforts in fragile and conflict-affected states. The evaluation was planned to recognize any achievements of the Canadian Development and START programs that have nonetheless been realized, and to provide recommendations in which Canadian interventions in South Sudan and other fragile states can be of optimal use now and in the future. The evaluation was conducted in accordance with the Government of Canada Policy on Results.

The operating environment in South Sudan for the Canadian Developmentand START programs between FY 2009/10 and 2015/16 was demanding. The working environment for the Canadian programs and its implementing partners was marked by constant change and the resurgence of violent civil conflict.

This report will provide insight into the South Sudan country program’s relevance, effectiveness and efficiency throughout this difficult period. It is comprised of an introduction that outlines the evaluation’s objectives and scope, methodology and limitations, country and Canadian programming context; a findings section which outlines key discoveries related to Canada’s intervention in the country; a conclusions section that provides analysis of these findings; and recommendations and considerations for future programming in South Sudan and other fragile state contexts.

1.1 Evaluation Objectives and Scope

The evaluation covered the period from FY 2009/10 to 2015/16, in which GAC programming involved shared operations and extensive consultation within the former Canadian International Development Agency (xCIDA), and the Stabilization and Reconstruction Taskforce (START) of the former Department of Foreign Affairs and International Trade (xDFAIT). Until 2013, this also involved a whole-of-government approach through partnership and collaboration with the Department of National Defence (DND) and the Royal Canadian Mounted Police (RCMP). However, this evaluation focuses on the development and reconstruction programs of xCIDA and xDFAIT only, whose contributions and disbursements were significantly higher than those of other partners within the Government of Canada. Within xCIDA programming, the evaluation focussed primarily on the bilateral program, as the previous evaluation of humanitarian assistance provided information on Canada’s intervention in South Sudan in these contexts. The formal amalgamation of xCIDA with xDFAIT in 2013 drove the decision to conduct a single evaluation that included stabilization, reconstruction and development programming, and to understand how these components worked together on the ground.

The specific objectives of this evaluation were:

• To assess the relevance, effectiveness and efficiency of GAC’s programming undertaken in South Sudan; and
• To provide GAC with relevant information to guide future programming in South Sudan and other fragile states, including best practices.^{Footnote 9}

The evaluation questions were based on the criteria established by the Organisation for Economic Cooperation and Development/Development Assistance Committee (OECD/DAC) – relevance, effectiveness, efficiency, and sustainability. However, faced with the significant challenge of obtaining reliable and current data in South Sudan, the evaluation addressed only those key questions for which there was reliable and sufficient evidence. In particular, the evaluation did not explicitly draw conclusions on sustainability.

In light of current events in South Sudan, this evaluation made all attempts to ensure that the information found within this report would be useful and applicable to a new and changing context. As much as possible, the report sought to ensure that the most applicable lessons learned were highlighted to inform future use.

The evaluation questions are found below.

Table 1: Evaluation Questions

1.2 Methodology

The evaluation sampled 29 projects (i.e., 17 Development projects with a combined value of $131.7M and 12 START projects with a combined value of $31.7M). The collection and analysis of primary and secondary data included a combination of sources, and a content analysis of documents and interviewer notes. More specifically, the following lines of evidence were employed as part of this evaluation:

• A literature review;
• A review of program and project documents; GAC country, sector and program strategies; program logic models and performance measurement frameworks; annual reports; evaluation reports;
• A review of previous GAC country program evaluations in fragile states;
• Informant interviews with actors including GAC staff from Development and START programming and International Humanitarian Assistance, officials within the Government of South Sudan, United Nations agency officials, think tanks, non-governmental implementing partners, and donor partner officials (a total of 162 informant interviews);
• Nine focus group discussions with 87 direct beneficiaries and locally engaged implementing partner staff during site visits to Bor, Jonglei State, and Torit in Eastern Equatoria/Imatong State;
• Two site observations in Bor, Jonglei State and Torit, Eastern Equatoria/Imatong State; and
• A validation session of preliminary findings with stakeholders at GAC headquarters.

1.3 Limitations and Gaps

Logistical constraints in South Sudan negatively impacted the collection of current and reliable data.

On-going uncertainty throughout South Sudan during the planning and data collection phases presented significant challenges for this evaluation. As a result, primary data collection was limited in its scope. Specifically, it was not possible to gather primary quantitative data using stakeholder or household surveys. With the deteriorating security situation and the closure of the Canadian embassy in July 2016, it was not possible to conduct additional data collection missions. To mitigate these constraints, more weight was given to interviews and the document review based on the professional judgement of the evaluators on the reliability of data.

Triangulation of evidence was difficult to achieve as a result of the absence of current and reliable quantitative data in South Sudan.

The evaluation strived for full triangulation of evidence where possible. However, the exclusion of household surveys and secondary information sources resulted in a shortage of reliable and current statistical data and limited avenues for corroboration. In particular, the underlying data from even well-respected and credible sources (e.g. the evaluation could only shed light on areas in which there are gaps in information or areas that required further interest and study.) often relied on models and estimates for their more recent statistics.

More information is needed on program coherence and cooperation between relief, reconstruction and long-term development programs and delivery channels, civil society and policy dialogue.

The findings found within this exercise were determined by an analysis of a narrower than desired set of data. These findings represented the most comprehensive and defensible evidence of the collected and analyzed data from this evaluation exercise. Some evaluation questions identified in the original Terms for Reference for the evaluation were omitted due to lack of evidence. In particular, significant gaps in the information collected for this evaluation made it difficult to determine solid findings around the effectiveness of formalized channels of cooperation and delivery with respect to relief, reconstruction and long-term development delivery channels, relevance in programming for civil society, and policy dialogue.

2. Context

2.1 South Sudan

Following a nation-wide referendum as outlined in the 2005 Comprehensive Peace Agreement^{Footnote 10} that included votes cast in both southern Sudan and around the world, South Sudan became the world’s newest country on July 9, 2011. As a result of decades of violent conflict that took place throughout the geographic area of present-day South Sudan, and a legacy of marginalization from previous policies practiced by governments based in Khartoum, South Sudan’s new national institutions and avenues for social services and governance were either underdeveloped or non-existent.^{Footnote 11} Lacking the experience in national governance, the fledgling Government of the Republic of South Sudan has faced acute and increasingly difficult political, economic, social and human rights challenges.

Power was centralized in the capital city and structured to reinforce patrimonial and ethnic lines.

Political power has been centralized in Juba and state level institutions were underdeveloped and lacking in capacity, resources and financing. Central government has been structured along patrimonial and often ethnic lines^{Footnote 12} , with power further reinforced by an emphasis on militarization. Within the central and state administrations, the effectiveness of “progressive” civil servants has reportedly often been blocked within a rigid hierarchical structure and a culture of deference to authority. The net result was an absence of accountability.

Many indicators associated with development were low during the evaluation period.

South Sudan was in the bottom five developing countries for 11 Millennium Development Goal indicators for which data was available.^{Footnote 13} It scored lowest with a ranking of 195 on the UN Human Development Index.^{Footnote 14} The literacy rate was 27% (40% for men and only 16% for women) with slightly higher rates for youth; 44% of primary school age children were in schools. The maternal mortality rate was 2,054 per 100,000 live births and infant mortality was over 10%. The rate of full immunization was 17%. Approximately 38% of the population had to walk more than 30 minutes one way to collect drinking water, and nearly 80% of South Sudanese did not have access to any toilet facilities.^{Footnote 15}

Food insecurity has been a significant challenge for rural and urban populations.

The predominantly rural population has been dependent on subsistence agriculture, with 78% of households depending on small-scale crop farming or animal husbandry as their primary source of livelihood.^{Footnote 16} Despite agricultural potential, the rural population has been consistently affected by food insecurity, climate change, poor road infrastructure and lack of access to markets. In 2013, 38% of households were considered moderately and 11% severely food insecure.^{Footnote 17}

Oil dependency has made the South Sudan economy highly vulnerable and has hindered its growth since 2012.

South Sudan’s economy has been almost entirely dependent on oil revenue, with 98% of the Government’s income and 60% of its GDP dependent on the extraction and production of this natural resource.^{Footnote 18} As a result of a dispute between the Government of Sudan and the Government of South Sudan, an oil pipeline stoppage in 2012 almost completely halted the production of oil in Unity and Upper Nile States, causing devastating effects on the economy. Due to this disruption, the country is not in a position to pay its debt of hundreds of billions of dollars to creditors.

Violence against women and gender inequality have been significant issues and their prevalence has escalated as a result of the prolonged conflict.

During the evaluation period, gender inequality remained a critical issue for women due to practices of early marriage, requirements of heavy domestic and agricultural labour, and restricted access to education. In addition, violence within the domestic and public spheres has been a continuous presence and has increased with the influx of small arms and light weapons during the Sudanese civil war. Notably, sexual and gender-based violence has affected over half of the population’s women and girls aged 15-24.^{Footnote 19} There has been a high prevalence of early, forced child marriage, dowry exchange, and sometimes violent abduction and sexual slavery of women and children. Women and girls have been regularly denied access to education and economic opportunities, and frequently marginalized from decisions that impact them and their families.

Violence has returned to South Sudan, and has increased chronic and widespread insecurity.

The first years of South Sudan’s independence saw positive movement towards the establishment of government structures and systems, and the development of a foundation for a stronger economy through the extractive and agricultural sectors. However, a dispute within the highest levels of the Government of South Sudan in December 2013 brought to the fore long-standing unresolved issues and grievances regarding land, access rights for herders, access to oil and mineral resources, as well as growing tensions within the governing political party itself. This resulted in highly violent armed conflict, often directed at civilian populations, and caused a major humanitarian crisis in addition to existing chronic underdevelopment and insecurity. A tenuous agreement to end the conflict was signed in August 2015.

South Sudan currently ranks second in the Fund for Peace’s Fragile States Index. The consequences of large-scale violence within South Sudan have been devastating for civilians. Since the creation of the country, tens of thousands have been killed and more than 2 million have been displaced. Fighting continues mainly in the northern states of Unity, Jonglei and Upper Nile. The prognosis for sustainable peace and security remains low as a result of persistent political and economic issues with neighbouring Sudan, the South Sudanese fiscal crisis, a further unravelling of the existing weak and limited governance and security structures, the proliferation of small arms and light weapons and, finally, underdevelopment, poor access to essential services and food insecurity.^{Footnote 20} Corruption has been a growing obstacle to economic and social development, and has furthered the drivers of conflict in much of South Sudan’s most recent history.

2.2 Donor Engagement in South Sudan

South Sudan’s donor community was led by the “Troika” of key international political actors: the United States, the United Kingdom, and Norway. Within the group of on-the-ground donors, Canada ranked 4th in 2011 and then 6th in 2014 among the top ten donors to South Sudan.

During the period spanning 2005 – 2011, Canada, the Netherlands, Norway, Sweden, and the United Kingdom established the Joint Donor Office for South Sudan. Its stated mission was to promote policies that reinforced the goals found within the Millennium Development Goals, support the work of the Multi-Donor Trust Fund (MDTF), manage programs that could not be implemented under the MDTF, and promote harmonization of donor work in Sudan.

In addition to donors, the United Nations was represented by a total 20 agencies working in all corners of the country on all aspects of the relief, reconstruction and development needs of South Sudan.

2.3 GAC Programming in South Sudan

The evaluation covers a period of seven years of Canadian development programming, of which the first two years were managed as the Sudan Country Program and the remaining five years, as the South Sudan program. The evaluation covers three distinct periods: pre-independence (2009-2011); immediate post-independence of South Sudan (2011-2013); and the period of renewed civil war (2013-2016). Canada identified Sudan as a priority fragile state in 2006.

From 2008 to 2013, a whole-of-government approach to the country was initiated through a task force that included xDFAIT, xCIDA, DND, and the RCMP. This whole-of-government approach focused on three objectives: contain violence and enhance security; reduce vulnerability and save lives; and build longer-term stability and resilience. To support these objectives, Canadian stabilization and reconstruction programming focused on: humanitarian assistance and early recovery, policy dialogue and peace-building, and security (i.e., support to the United Nations peacekeeping missions). During this same period, development programming sought to build longer-term capacity in basic health service delivery and food security to improve resiliency and reduce vulnerability. While Canadian intervention and consultation between most of the task force member departments formally ended in 2013, consultation and information has continued to the present day on issues and programming in South Sudan.

Table 2 below summarizes the role of Government of Canada Departments within the Government of Canada’s Sudan Task Force.

Table 2: Government of Canada’s Sudan Task Force 2009 to 2013

Over the course of 2009 to 2013, Canada’s interventions were primarily based in four states throughout the area, including Eastern Equatoria State, Jonglei State, Northern Bahr el Ghazel, and Unity. As the crisis intensified during and immediately following the conflict in December 2013, Canada’s emergency humanitarian relief increased in the affected states of Jonglei, Unity, and Upper Nile, and development programming continued at existing level in the less conflict-affected states of Eastern Equatoria State and the tristate Bahr el Ghazal region.

Stabilization and reconstruction work focused programming in South Sudan on building peace and stability through security programming. These activities were located within the Department’s START division of xDFAIT and funded through the Global Peace and Security Fund from 2009 until 2013. They included supporting the increased effectiveness of the UN peacekeeping missions in Sudan, a productive peace process in Darfur, strong cooperation between the Government of South Sudan, civil society and communities in addressing community-level security concerns, increasing the safety of civilians and reducing loss of life amongst conflict-affected populations, and increasing the capacity for stakeholders in North and South Sudan to peacefully address the post-Referendum transition arrangements.

Two strategies guided development programming in South Sudan during the evaluation period: the 2009-2014 Country Development Program Framework for Sudan, which was approved before South Sudan became an independent country, and the Draft Interim Bilateral Development Strategy 2014-2016 for South Sudan.

The three thematic areas included children and youth, food security, and governance and advancing democracy. These were prioritized within both guiding documents and offered a consistent approach to development programming in the country. Within the children and youth theme, the early years focused on skills development and alternative livelihoods. In 2012, the focus then shifted to basic health services and maternal, newborn and child health (MNCH). Canada also provided funding for child protection activities that allowed for the rehabilitation of child soldiers and recruitment prevention. Food security was a Canadian priority sector during the entire evaluation period, with an emphasis on alternative livelihoods for youth (2009 - 2014), followed by subsistence agriculture focusing on women from 2014 onward. Planning in governance and advancing democracy was focused on advancements in the referendum and further electoral processes, as well as institutional development through initiatives of the Capacity-Building Trust Fund and the United Nations Development Programme.

The former Canadian International Development Agency (xCIDA) invested $452.3 million in South Sudan from FYs 2009/10 to 2015/16 (Table 3), with an additional $102 million from 2010/11 to 2015/16 in direct investments to MNCH programming. Humanitarian Assistance programming^{Footnote 21} accounted for 37% of the total amount of Canadian investment in the country. Bilateral programming focused largely on children and youth (31%), followed by food security (13%) and sustainable economic growth (10%). Advancing democracy accounted for less than 2% of xCIDA funding, with the focus being on the 2011 referendum process and improving state-level government financial management systems. One mine action project was also funded.

Table 3: Disbursements in Canadian Development Assistance by xCIDA (2009-2010 to 2014-2015)

The top ten partners to xCIDA by disbursement in South Sudan during this period were: World Food Programme (WFP), United Nations Population Fund (UNFPA), United Nations Children’s Fund (UNICEF), Food and Agriculture Organisation of the United Nations (FAO), World Health Organisation, United Nations High Commissioner for Refugees (UNCHR), International Committee of the Red Cross (ICRC), UK Department for International Development, International Organization for Migration, and World Vision Canada.

Between 2009/10 and 2012/13, START implemented 37 initiatives throughout the whole of Sudan (pre- and post-South Sudan’s independence) representing $46.5 million in Canadian investments. The majority of these initiatives were implemented in South Sudan (or southern Sudan prior to independence) to improve governance and address peace and security at the community level. This was done mainly through support for rule of law, including justice, police, prisons and customary law provisions (46% of total program disbursements); community security and small arms control, including mediation and conflict resolution (36% of total program disbursements); and reduction of danger from mine and explosive remnants of war (13% of total program disbursements).

Table 4: START Disbursements 2000-10 to 2012-2013

The top ten START partners in Sudan and South Sudan during this period based on size of disbursements were: the United Nations Development Programme (UNDP), Deutsche Gesellschaft fur Technische Zusammenarbeit (GTZ), United Nations Office on Drugs and Crime (UNODC), Private Agencies Collaborating Together (PACT), Forum of Federations, International Development Law Organisation (IDLO), Danish Demining Group (DDG), United Nations Habitat, Saferworld and Norwegian’s People’s Aid.^{Footnote 22}

3. Findings

The following section presents the findings of the evaluation of Canada’s Development, Stabilization and Reconstruction Program in South Sudan.

3.1 Relevance

Q1. To what extent have Canada’s policy and program interventions responded to the needs of the population in South Sudan?

Finding 1: Program strategies and field operations were relevant to the context in South Sudan and responded to the differing national and community level needs.

During the evaluation period, South Sudan was one of the poorest and most unstable countries in the world. Social and economic needs were acute in every sector of the economy. Canada’s decisions to support health services, address food insecurity, and assist in building the rule of law and community security were undertaken to address some of the underlying causes of instability and vulnerability, and to create opportunities for long-term peace and development for the people of South Sudan.

Development Programming

Development programming investments were made in areas where Canada could make a difference. Given co-funding practices in South Sudan and the establishment of the Joint Donor Office, there were no sectors that GAC addressed alone. The Department therefore supported projects that were relevant and in line with other donor engagement. GAC worked in regions where other donors were less present (e.g., Upper Nile State and Unity State) until the conflict made it very difficult to work in those regions. In addition, investments were supported by an analysis of country needs that was required in developing the Country Development Planning Framework and the South Sudan Country Strategy.

GAC chose to engage partners at different levels of government. Capacity-building was provided by both Development and START programs to state institutions, civil society and communities within health, agriculture, rule of law, community security and small arms control, as well as the reduction of hazards posed by landmines and unexploded ordnance.

Most projects were multi-sectoral and complimentary, combining various thematic areas like youth, women, economic development and food security. This was in line with the context of South Sudan, which demanded a holistic approach. For example, mine action supported the implementation of projects in agriculture through the clearance of farming space. Through armed violence reduction and peaceful resolution programming, enhanced community security contributed to better access to health services.

The most significant and wide-reaching Canadian development initiative in South Sudan was the Health Pooled Fund (HPF), a multi-donor initiative designed to deliver health care services, particularly to mothers and children, and build the capacity of government health services at the national, state and county levels in a harmonized way. The HPF was considered by South Sudan’s Ministry of Health and international development partners to be the cornerstone of the health system. The flagship initiative of Canada’s health sector supported through the HPF was the Strengthening Midwifery Services project, which sought to build a cadre of skilled and professionalized health care workers, with an emphasis on midwives, across the country in both urban and rural areas.

Food security was equally relevant. As discussed, over 40% of the population experienced food insecurity during the evaluation period. The prevalence of food insecurity among female-headed households was markedly higher. Canada’s program focused on projects designed to boost food production and support the market participation of smallholder farmers in alternative livelihoods (e.g. vegetable production and fisheries) through involving households in food and cash-for-work activities, constructing and rehabilitating roads, clearing and converting land for farming, and introducing more nutritious crops.

In addition to the directed programming in health geared towards maternal, newborn and child health (MNCH), Canadian investments in development programming on children and youth in South Sudan were directed to vulnerable youth in areas far removed from the capital city. Programming sought to reduce the vulnerability of young men and women resulting from food insecurity and the absence of sustainable livelihood activities in the area. Canadian investments were based on the needs of the participating communities and contributed to addressing educational, agricultural and livestock needs of young people. Canada further contributed to the pressing needs of youth affected by violence following the resurgence of conflict in 2013 through programming in education and skills development, and the rehabilitation and recruitment prevention of child soldiers.

START Programming

Through START, Canada supported peace-building, security and rule of law projects that worked at all levels of South Sudanese society, from national to community, and were seen by partners as being appropriate responses to the complex and challenging operational context. At the state level and in line with the OECD/DAC principles for fragile states, Canada contributed to state‑building by supporting programs that improved the capacity of the Government of South Sudan to provide basic security for its citizens. Building longer‑term stability and resilience was of primary importance according to the 2010 Global Peace and Security Fund Multi-Year Strategic Framework for Sudan. The same report provided examples in which state parties had seen an increased ability to address post-referenda arrangements, an increased capacity for  the Community Security and Small Arms Control Bureau through training and equipment, and an increase in the responsive capacity of the South Sudan police services. These programs were relevant to the macro needs of the new country, which lacked the capacity to undertake the necessary legal and security sector reforms for long-term movement to good governance and peace-building. International NGO representatives regularly cited the preferential focus on state-building in many donors’ approaches to South Sudan, in particular during the Comprehensive Peace Agreement and Referendum periods.

Overall Programming Approach

International implementing partners generally agreed that there was a need to move overall international donor programming priorities beyond the capital city of Juba. Many partners indicated that, for the most part, the donor community placed less importance on community-level work – a topic decidedly important but thought to be overlooked. However, while there was a significant focus on macro-level efforts in state-building endeavours by the international community in South Sudan, both Canadian Development and START-funded programs were well aligned with regard to connecting the state with the people and they reached multiple layers of government. Canada’s approach included reaching more than just the national Government of South Sudan, and further demonstrated the important links between the government and its people. Canada’s focus on building the capacity of the Community Security and Small Arms Control Bureau (CSSAC) within START programming was highlighted by implementing partners as being an important element in developing government capacity to resolve insecurity and simultaneously build community resilience to violence and insecurity. Canada’s Development efforts in delivering health services while simultaneously building the capacity of the Ministry of Health to provide access and oversee basic service provision were seen as particularly relevant, both inside and outside the capital.

Q2. To what extent was programming aligned with Canadian priorities?

Finding 2: Programming in South Sudan was aligned with Government of Canada priorities.

During the evaluation period, operations in South Sudan followed the priorities of the Government of Canada. Canada's continued engagement in South Sudan’s foreign policy priorities (i.e., freedom, human rights and the rule of law) were closely aligned with Canadian security interests. Programming in South Sudan was also aligned with the priority themes of the Canadian International Assistance Envelope: stimulating sustainable economic growth; increasing food security; securing the future of children and youth; advancing democracy; and ensuring security and stability. Based on an analysis of planning documents spanning the time period 2009–2016, funding decisions were determined to be aligned with the Country Development Program Framework 2009–2014, as well as the draft Interim Bilateral Development Strategy 2014–2016.

In addition, the START program’s objectives for Sudan/Southern Sudan included supporting the implementation of the North-South Comprehensive Peace Agreement and the implementation of the Referendum on Independence in 2011. Both were foreign policy goals of the Government of Canada in South Sudan.

The introduction of the Muskoka Initiative in 2010, which focused on maternal, newborn and child health (MNCH), was a key Canadian development priority in South Sudan. In line with Canadian cross-cutting thematic priorities, gendered language was found in project documentation for all sampled Development projects. During interviews, implementing partners stressed that they had clear gender strategies or policies guiding their projects. However, it should be noted that while the language was present in program and project documentation, this evaluation was not able to verify if this language had made a difference on the ground.

Finding 3: Programming aligned with many principles for good practices in fragile states, but more could have been done.

Programming in South Sudan during the evaluation period did not fall within the normal scope found in stable, developing countries. Therefore, this evaluation sought to understand additional considerations that affected the relevance of Canadian programming in South Sudan in light of its complex and fragile status, and Canada’s commitments to international principles and best practices. These considerations followed those outlined in the 2007 OECD Principles for Good International Engagement in Fragile States, and further the guidelines produced by the Deputy Minister’s Committee on Conflict and Fragility, which reinforced and endorsed the recommendations of the July 2009 report, Sustaining Canada’s Engagement in Acutely Fragile States and Conflict-Affected Situations.

While there is evidence that the Principles were taken into consideration in the design of Development and START programs, it is not clear from the information obtained in this evaluation that all the Principles were fully incorporated in a deliberate and documented fashion within the larger, overarching Development program. A detailed analysis is provided in Annex 2. GAC incorporated the Principles across programming. However, GAC could have done more in structurally linking security and development objectives as well as articulating and specifying more predictable commitments to development sectors.

3.2 Effectiveness

Q3. To what extent did Canada achieve its expected results in South Sudan?

Finding 4: Targeted development programming in the health and food security sectors contributed to the achievement of project-level outcomes. Programming in stabilization and reconstruction led to limited improvements in peace and security. However, these achievements have likely been eroded as a consequence of prolonged conflict.

GAC played a useful role in the provision of health, food security, peace and stabilization in South Sudan. One of the leading outcomes to which the Department contributed was the improvement of access to health services for women and children. As of 2015, GAC’s funding of the Health Pooled Fund contributed to the operationalization of 563 primary health care facilities and 15 hospitals in six out of ten states. Particularly evident was the effect of GAC programming on neonatal services, including the Canadian contribution to saving the lives of thousands through emergency obstetrics.

In the Development program, the evaluation found evidence that a significant number of sampled projects achieved outcomes. Bearing in mind the challenges of achieving, maintaining, and measuring outcomes at intermediate levels in fragile states, detailed information was not available on the outcomes of several projects. Data on the successful achievement of outputs suggests that immediate outcomes related to accessibility were attained in some cases, leading to improvement in access to basic services for children and women across South Sudan. Canadian investments in development and stabilization included the following improvements:

• Education – building of primary schools, training of teachers and distribution of class supplies such as textbooks;
• Health – provision of sanitation and safe drinking water;
• Child Protection – increased civil registration and vital statistical information such as birth notification, emergency life skills training on the risks and dangers of unexploded ordnance, increased monitoring and reporting of child rights violations, and psychological support for children previously associated with the fighting forces;
• Community Security – increased capacity of the Community Security and Small Arms Control Bureau to analyse the causes of local insecurity and to move the discussion of these issues to the political level; and
• Community peaceful conflict resolution – increased mitigation strategies to resolve inter‑community conflicts.

Achievement of immediate and intermediate outcomes in food security were attained but not sustained due to the resurgence of conflict in 2013. A few projects were suspended or relocated due to the conflict. Nevertheless, Canada’s investments in agricultural production and training contributed to improving overall food security. However, due to the prolonged internal conflict, most gains in food security have been lost.

Over 95% of the food security projects and youth-targeted projects for the time period 2009–2015 sampled within this report contributed to improved opportunities for community-based peace-building. An example was the Peace and Livelihood for Women project, which established community-based peace committees that worked alongside traditional groups to bring about peace. During the project’s lifetime, the number of beneficiaries increased by 560%. Women’s development groups have been regarded as key allies in implementing agriculture, governance, gender equality and peace-building, despite the growing insecurity and violence that has been a fixture within much of the country at a national level since 2013.

Canadian efforts to positively impact the state-building process through START’s support of the constitutional reform process helped improve skills and knowledge that peacefully addressed the planned post-referendum arrangements. The creation of the Community Security and Small Arms Control Bureau helped to coordinate community security and arms control, and provided working collaboration among stakeholders. There is some evidence from the data reviewed in this evaluation to suggest that, through training, equipment and infrastructure support, START investments achieved a limited increase in the responsive capacity of the South Sudan Police Force. However, these efforts had only a limited influence on the creation of a peaceful and safe environment.

Specific work on disarmament and the professionalization of the security sector was impacted by larger issues of citizen mistrust of national institutions and the inability to carry out the full disarmament of the population. A notable example of this limited and fragile influence on peace and security was the presence of results related to mine clearance, risk education, and national capacity-building. Mine action programming provided for the physical security of cleared and reusable land during much of the evaluation time period. However, with the renewal of conflict since December 2013, many gains and community-level impacts achieved through this programming have been lost to recontamination and there has been a general deterioration of security.

Finding 5: Canada was effective and had a positive influence on building and managing relationships with the Government of South Sudan and other donor partners that impacted the direction of policy and the approach in South Sudan.

Canada’s approach to policy dialogue in South Sudan reflected the existing business culture norms of the country and the need for flexibility and quick action. Where needed, Canada maintained a strong presence in formal donor partner initiatives. As such, Canada was an active member of and advisor for six pooled funds over the course of the evaluation period, including: the Multi-Donor Trust Fund; the Capacity‑Building Trust Fund; the Basic Services Fund; the Sudan Recovery Fund; the Common Humanitarian Fund; and the Health Pooled Fund. GAC also co-chaired the Election Assistance Group and was a member of the Joint Donors Stabilization Working Group. From 2007 to 2013, Canada was a member of the Joint Donor Team alongside the governments of the Netherlands, Norway, Sweden and the United Kingdom. The objectives of the Joint Donor Team were to provide analysis and policy advice for donors working in South Sudan; manage pooled funds, and act as a main liaison point with the Government of South Sudan.

Canada also used its long and privileged relationship, built through sustained efforts from consecutive Heads of Cooperation, with the Ministry of Health and the Health Minister to tangibly influence policy and direction. Canada was commended by international partners for raising awareness of human rights violations (including violations of women’s rights), and the need for more government accountability. The Canadian Ambassador in Juba was widely recognized as one of the most informed and engaged diplomats who were active during the period covered by this report, while Canadian development and the Head of Aid were held in high esteem within the Ministry of Health. The Development program’s continuous presence of staff dealing with the issue of health led to strong relationships that helped in the achievement of results in the sector, such as the prioritization of MNCH in the Government’s Health Strategy. At the time of the evaluation and through Canadian development programming, Canada was the co-chair of the Mid‑Level Health Cadres Scaling-Up Task Force and a joint partner in the Reproductive Health Forum and Health Policy Review. As co-chair of the Health Sector Working Group, Canada contributed to strengthening dialogue and cooperation with the Government of South Sudan, specifically as regards to the accountability of existing budgets and the adequate funding of health districts. Canada also led a complex mapping of international partner investments to assist the Ministry of Health in its budget planning.

Canadian stabilization programming was bolstered by its experience within the Joint Donor Team Stabilization Working Group alongside the United Kingdom and the United States. However, the sharing of information and possibilities for joint approaches to stabilization efforts for Canada was lost when START programming ended in 2013.

Finding 6: Development and START programs in South Sudan worked alongside each other and shared information with one another in Juba. However, there was no systematic or formal coordination mechanism that linked the two programs towards achieving common outcomes or addressing mutually prioritized drivers of conflict.

From its inception in 2006 until its completion in 2010, the Sudan Task Force provided a prime opportunity for information sharing and cooperation between Government of Canada departments working on South Sudan. The evaluation found that the Task Force was a useful mechanism for sharing intelligence on security for Canadian activities in the country. The Task Force met weekly and was led by the former Department of Foreign Affairs and International Trade (xDFAIT). These meetings allowed the members to share information on security issues and to stay aware of one another’s activities, but were not intended to coordinate programming between the member departments. Coordination was not done in a formal or structured way.

In South Sudan, the Development and START programs were co-located in the Juba office for much of the evaluation period. Co-location provided program staff with the opportunity to know what all of the programs were planning in real time, and allowed for the possibility of coordination in the field – whether geographic or thematically focused. The common perception from GAC interviewees in this evaluation was that, while information-sharing helped to avoid duplication, there remained the tendency to act independently on programming objectives. According to the interviewees for this evaluation, integration between the GAC programs was perceived to be constrained by corporate culture, with each delivery channel guarding its space. In addition, each program was responsible for the management of results towards separate and independent logic models that reflected different timeframes, and different approval channels.

Finding 7: Canadian capacity-building investments were the most effective and complementary when they addressed the reach of government and reflected a long-term vision.

Given the low existing capacity within civil society groups, public services and all levels of government, capacity-building was an overarching development, stabilization, and reconstruction issue in South Sudan. Few building blocks of the state existed prior to independence. This situation was exacerbated by an exceptionally low general level of education for a significant portion of the public service, and coupled with a lack of basic structures and procedures in government and local organizations. Partners in South Sudan interviewed for this evaluation believed that the most successful capacity-building strategies were those that combined formal training with practical skills development over a longer period of time.

Canadian START investment in capacity-building at the Community Security and Small Arms Control Bureau was an important way to assist the Government of South Sudan in reaching the community level security needs of the people. However this approach to capacity-building had limited results. Canadian support of local and state-level capacity-building improved the conditions for partnerships and information gathering on issues of security, but overall community or civil society capacity-building was generally seen by partners as lacking. The approach to capacity-building on stabilization projects at the community level was viewed as too focused on short-term training, such as workshops and seminars. Other investments in capacity-building at a national level with regard to mine action were less effective in building the capacity of the national mine clearance organization to coordinate, implement and monitor safe access to land.

As low levels of education and wide-sweeping inexperience in governance were long challenges for Canadian investments in capacity-building, the Development program also invested in general capacity replacement to assist government organizations in addressing immediate gaps. The building of government capacity in relation to core functions of the state came to a halt with the December 2013 crisis but continued, alongside capacity replacement, for basic services in health. This capacity replacement sometimes came at the expense of a longer term capacity-building vision. For example, development initiatives such as the Canadian contribution to the Rapid Capacity Replacement Initiative sought to address shortcomings in public management of the newly independent state at the national level. In support of the United Nations Development Programme, this was designed to provide immediate capacity replacement or gap-filling for essential public administration positions and to build the institutional capacity of South Sudan’s governance structures. The project replaced government workers with international technical advisors who did the job instead of focussing on mentoring and advancing the capacity of staff to undertake these positions themselves. The project only provided short-term technical assistance to a limited number of people.

Of the Development projects sampled for this evaluation, 2/17 were dedicated capacity-building projects, though health and agriculture projects all included a capacity-building component. Capacity-building activities for less than five years was too short to really affect change in farming practices and in the attitudes of producers to meet their food security needs. On the other hand, capacity-building investments in health had positive outcomes and profited from a sustained and long-term approach in the MNCH sector. Program effectiveness was demonstrated in the increased number of midwives able to work at the local level, and a greater ability of the Ministry of Health to address and engage communities. It was noted in the document review that 1 of the 17 projects demonstrated a significant misalignment with the realities on the ground in South Sudan in terms of available resources and modes of knowledge exchange, which made the projects ineffective in raising the capacity of local doctors. Interview participants cited the use of internet-based training modules as inappropriate given the context and the inaccessibility of high speed internet.

Q4. To what extent was programming in South Sudan consistent with best practices for fragile and conflict-affected states?

Finding 8: Conflict drivers were not always appropriately considered in development and START programming decisions.

The period from the independence of South Sudan in 2011 to the fall of 2013 (pre-conflict) was characterised by optimism among all international players. Interviews with international development and humanitarian partners in South Sudan identified a perceived gap during this period in the complex appreciation of the dynamics of power, conflict drivers, cultural sensitivities and practices within South Sudan by all international players.

Conflict was a fundamental and unavoidable issue in South Sudan as it impacted all aspects of development and stabilization. While development and START programming were clear in their links to relevant programming in the country, there lacked sufficiently detailed and complex conflict analysis which could have informed programming and more closely addressed the root causes of community, tribal or national violence. There was no indication from the sampled projects that conflict analysis had been undertaken in an ongoing and thorough way which would have integrated the knowledge and needs of local populations and contexts at the payam (county) and boma (village) levels.

In the Canadian context, the evaluation did not find adequate information from the sampled projects to confirm that conflict drivers were taken into consideration for development and START programming during the time period. There was documentation that showed that development planning on issues such as food security was consistent with addressing the insecurity associated with famine and therefore assisting the promotion of peace and stability. There was little evidence to demonstrate that Development and START projects documented the interconnected risks and impacts of their objectives on each other’s programs as well as on the beneficiaries for which these projects were designed.

At times, decisions for the Department as a whole led to changes in programming which did not always take into consideration commonly understood drivers of conflict in South Sudan. For example, when MNCH became the central focus of development engagement in 2010, resources were shifted from Children and Youth programming on child protection and empowerment to MNCH-focused projects. While the shift in priority was no less relevant to South Sudan, interviewees said that this decision meant abandoning a number of initiatives that directly addressed youth-related issues in a way that aligned much more closely with the challenges associated with South Sudan’s conflict drivers, and therefore the causes of fragility. In particular, the challenges associated with the need to address unemployed, marginalized and militarized youth as a potential drive for future conflict.

Despite strong evidence that capacity-building within the South Sudanese government was a crucial issue, the Departmental decision to change governance from a thematic priority to a cross‑cutting theme for all development programming made it more difficult to tackle governance issues in South Sudan from 2012 onward. According to several interviewees, proposals for governance-related projects received significant scrutiny after this point.

It should be noted that other donors’ governance initiatives were also rejected or did not come to fruition when the conflict erupted in December 2013. In part, this came as a joint donor decision to retain support only for the delivery of basic services until peace was firmly in place and all sides were in compliance with the terms of the peace and ceasefire agreement. According to interviewees, Canada was better positioned than most donors to work on governance as it was already engaged at multiple levels of government and supported activities implemented by a number of diverse partners.

START operations in South Sudan came to an end in March 2013 following the completion of the Sudan Task Force’s mandate within GAC. This move was met at the time with objections by key informants in South Sudan with knowledge of stabilization and reconstruction activities of the country. According to these interviewees, the termination of START left a gap, particularly as it was felt that it would have been especially relevant in the post-2013 context. The START program was recognized for having launched cutting edge initiatives in response to crisis and conflict situations. As a representative of a partner agency stated, “With START, Canada really got it right with the mix of programs and the level of innovations they supported.”

As a result of the termination of South Sudan START programming, Canada was no longer a member of the Stabilization Donor Coordination Group alongside the United Kingdom and the United States. In retrospect, this meant that program staff lacked important information and opportunities for joint approaches to stabilization efforts in the country at a time when they were most needed. The elimination of START from the country program portfolio removed key tools that had previously enabled Canada’s interventions to remain relevant and able to address the needs of highly vulnerable populations.

Q5. Have interventions in addressing cross-cutting themes of gender equality, environmental sustainability and governance through policy dialogue efforts and project investments had an impact, intended or unintended?

Finding 9: There have been gains regarding cross-cutting themes, particularly in gender.

Canada’s active presence within the Joint Donor Office through the Development program helped the Ministry of Gender, Child and Social Welfare to develop its capacity in taking on gender challenges throughout the country with the creation of the National Gender Policy, as well as to develop national capacities in gender-sensitive legislative drafting. According to the Ministry of Gender, Child and Social Welfare, the initiative contributed to an improved understanding of gender equality principles and women’s human rights.

According to the Ministry of Health, between 2011 and 2013, South Sudan saw an increase from 5 obstetricians to 105, and an increase from 5 gynecologists to 30 in different parts of the country. In addition, 250 registered midwives were deployed across the country. According to the World Bank modeled estimate, there was a decrease in maternal mortality rates from 823 per 100,000 live births in 2010 to 789 per 100,000 in 2015.^{Footnote 23} Child mortality under 5 years decreased from 96 per 1,000 in 2010 to 93 per 1,000 in 2015.^{Footnote 24} The World Health Organization estimated that the number of women receiving emergency obstetric care increased fourfold, from 851 in 2012 to 3,526 in 2014. While these numbers are estimates and may reflect many influences beyond GAC’s development programming, the perception of improvement was independently confirmed by interviews with program staff and several partner agencies.

Finding 10: The program had limited access to expertise for the implementation of the cross-cutting themes of environment, gender equality and governance.

Headquarter specialists confirmed that while the work of sector experts continued to evolve, the quality and quantity of support they were able to provide from headquarters to operations significantly decreased over the years. The number of specialists within the Department had also decreased and they were no longer able to provide proactive support and quality control at the project level. Consequently, field staff in the Department were increasingly responsible for managing cross-cutting themes through locally engaged specialists.

Although program staff had access to a gender advisor during the project assessment phase, this expertise was not available for project monitoring. Despite these challenges, the evaluation found that genuine efforts were made on gender equality. Overall, the Development program complied with the analysis required by Departmental policies on gender equality. The Development program prioritized gender in its development strategy and identified the vulnerability of women in its planning documents. There was emphasis on women’s and girls’ participation in both Development and START operations, particularly with all Development projects having undertaken a gender analysis.

Both START and Development projects incorporated disaggregated monitoring data on women and men in their reports. Additionally, the work on gender through MNCH-related projects was an important way for Canada to engage on gender equality through its development programming. Corporate checklists were properly completed, planning documents referenced relevant considerations, and the annual country reports described progress accomplished on identified gender-related goals. Yet, the evaluation noticed a diminished capacity for communicating the concept of gender equality as a cross-cutting theme within the South Sudan program.

In the area of environmental sustainability, staff had access to headquarter and regionally based specialists. Links between climate change and conflict were clearly acknowledged in the Program’s 2009–2014 Environmental Sustainability Analysis, which stated that environmental stress had created the conditions for conflict to emerge and be sustained by political, tribal or ethnic differences. It further stated that this was “a tragic example of the social breakdown that can result from ecological collapse. Long-term peace in the region [would] not be possible unless these underlying and closely linked environmental and livelihood issues are resolved." The analysis concluded that, “the ability to gather and act on knowledge that leads to successfully advocate and mediate for peace that is in turn built on sustainable and equitable development is vital.” However, the evaluation did not find evidence that this analysis was clearly or formally linked with program planning or decision-making. Only one of the food security projects within the Development program reviewed reported on environmental sustainability.

According to interviews with GAC employees and international partners, the South Sudan Development and START programs team had to balance the consideration of cross-cutting themes with other administrative elements associated with managing projects. There were divergent views about whether the right balance was achieved.

3.3 Efficiency

Q6. Were human and financial resources used appropriately for the outcomes achieved so far, in light of context, priorities, and potential alternatives?

Finding 11: Financial resources and fiduciary risks were well managed by the Development and START programs. However, the management of START and Development operational risks could have been improved by a more robust risk assessment process.

In its 2014 publication, Development Assistance and Approaches to Risk in Fragile and Conflict Affected States, the OECD recommended measures for donors to manage risk in fragile states and situations, based on longstanding good practices. An assessment of the South Sudan program practices against these risks shows that financial resources and fiduciary risks were well treated.

In line with OECD/DAC best practices for working in fragile states, Canada was strong in its participation within pooled initiatives such as the Multi-Donor Trust Fund and the Health Pooled Fund, and in its use of existing country systems in implementing programming and in particular health projects. It also demonstrated an active role in coordination and joint work with donors through initiatives such as the Stabilization Coordination Working Group and the Joint Donor Office.

Canadian grants to select United Nations agencies ensured a lower degree of risk in development investments. United Nations agencies provided programming that assumed safety and security protocols for implementing partners. They had wide reach and an established presence throughout the country, which meant a lower cost in time and resources for project start-up. They also had established relationships with key partners in the Government of South Sudan. While this model was more expensive overall, the value-added in terms of time, reach and effect meant that it was well-suited for the continuously changing environment in South Sudan.

While financial resources and fiduciary risks were well managed, this was less true for risks related to both stabilization and development operations. The analysis of contextual risks took the form of risk analyses. However, there was little evidence within the projects sampled for this evaluation of in-depth, continually updated, and documented context analysis. This has been suggested by the OECD/DAC as a best practice for reflecting emerging changes in the programming environment and the implications they have on operations to achieve results. The risk management tools included a risk registry, but little suggest nuanced insights into program options. Moreover, the analysis was often outdated. While START programming allowed for quick action as a result of its compressed timelines and short-term objectives, development programming lacked faster disbursement mechanisms that could ensure greater programming flexibility and effectiveness to meet the needs of the population.

Finding 12: The selection and diversity of partners was appropriate given the local operating context.

Operations costs were high in South Sudan during the evaluation period, with obstacles such as geography, lack of infrastructure, political sensitivities, technical capacity, and insecurity limiting the normal implementation of programming. Given these circumstances, START and Development projects were administered in a cost‑effective and appropriate way.

Although more costly than other channels to work with, UN agencies had a constant presence in the country which provided them with an in-depth understanding of local triggers of humanitarian situations. They were also able to quickly deploy in different regions as a result of their established operational reach throughout the country.

According to interviewees, Canadian decisions for partnership with international non‑governmental organizations were informed by pragmatic considerations on the ground. Specifically, international NGOs had local knowledge and experience working in South Sudan, operational sustainability outside of Canadian funding, and pre-established presences in the country. For these reasons, the South Sudan program rarely partnered with Canadian organizations during this time period because few had the capacity or sufficient background in the country to meet these considerations.

Finding 13: Canadian programming in South Sudan would have benefited from more timely decisions on portfolio investments.
 
Since Independence in July 2011, the Development program created a series of draft program-level planning documents to guide operations in the new country. However, none of the planning documents outlined below were ever formally approved.

• Draft Country Development Strategy for South Sudan (2013–2018)
• Draft Interim Bilateral Development Strategy for South Sudan (2014–2016)
• Draft South Sudan Investment Plans

This meant that the former CIDA 2009 Country Development Programing Framework and Country Strategy for Sudan remained the only formally approved planning document for the South Sudan country program. However, the draft documents cited above provided the de facto guiding materials for guidance on expenditure intentions. Some project proposals that aligned with these documents were not approved despite the staff time involved. At the same time, the introduction of a renewed focus on MNCH by the Government of Canada led to a shift away from youth-focussed and other programming.

By contrast, START projects were relatively quick and flexible in their approval process, which meant that initiatives could be started in manner that met the needs of the issue in question.

Regardless of the size of financial commitment or length of time, Development projects took up to two years to prepare and approve in spite of the fast-changing context in the field. While this may have been a normal timeframe that had been applied throughout most of xCIDA’s development programing in other countries, the length of time to prepare and approve projects inhibited quick movement in South Sudan as context changed. Interviewees indicated that there was no fast-tracking of project approval for South Sudan as a fragile state, and little delegation of authority to the Canadian Embassy that would have allowed for expeditious or meaningful action.

4. Conclusions

The conclusions below are based on an analysis of the findings above.

Canadian investments contributed to meeting the needs of South Sudan in an important way.

The Development and START programs in South Sudan were relevant to the needs of the country and they played a positive role as instruments of Canada’s foreign policy objectives. Given the complexity of the challenges facing South Sudan and the need for donor intervention at all levels and in all sectors, the Department’s focus, albeit changing, remained relevant throughout the period assessed in this evaluation.

There was evidence to support the claim that the Development and START programs were effective in achieving their outcomes, but that these outcomes were not sustained. The Department made a significant contribution to improving the quality of life for South Sudanese for a period of time through its project results and policy dialogue initiatives. Program success was bolstered by effective practices which included working with key donors and actors who already had a presence on the ground.

Canada worked effectively with other donors.

To be relevant and effective in the political realm of a context like South Sudan required a strategy that included a variety of instruments. START and development operations provided a reasonable collection of instruments to support policy dialogue. Pooled funds also provided a valuable platform for dialogue with the Government of South Sudan, and other international donors within the country.

In addition to providing an effective platform for influencing policy, working with other donors ensured that the Government of Canada was able to increase its reach, build on shared resources, and work towards its and donor partners’ strengths in implementing projects for which there were more likely to be positive results.

Administrative processes were sometimes at odds with achieving optimal effectiveness in Development and START programming.

In fragile states, peace, state-building and development operations are costly, time-consuming, and often inefficient. Programming in a fragile state is very different from planning and executing within stabile developing countries. In this context, the termination of START in South Sudan was seen by almost all interviewees as being in direct contradiction to the objective of staying engaged long enough to give success a chance. Opportunities created were then lost due to this interruption. In particular, the discontinuation of START activities in 2013 meant the withdrawal of an instrument that allowed the Program staff in Juba to link the political sphere to operations in the field. Operations would have benefitted from being more grounded with the principles and recognized best practices for working in fragile states, rather than being driven by internal systems designed for more traditional contexts.

In retrospect, GAC would have benefitted from more in-depth analyses of the determinants of conflict, peace-building and state-building for all fields of programming, particularly in the pre-2013 conflict period. Program planning documents could have explicitly linked each of these determinants to the choice of issues, projects, and the use of policy dialogue, and described how these links were expected to play out in terms of expected results. Along similar lines, there could have been a formalized internal dialogue and coordination effort within GAC (and within the Government of Canada) to systematically search for innovative ways to ensure linkages between START and Development investments. The association with cross-cutting themes would also have been pro-actively considered. This implies that the management of the GAC’s interventions in South Sudan would have gone beyond the sound administration of financial resources and fiduciary risks, to address the more complex risks related to the achievement of Departmental-level results.

GAC did not work towards an appropriate balance between the administration of funds and fiduciary risk and the management of complex stabilization and development operations in a fragile context. xCIDA and xDFAIT may have had proper planning, monitoring, management and reporting tools. However, the use of these tools may have hindered the timely approval of projects, and as such, was not always optimal.

Successes in the program were tempered by administrative challenges faced by the program. Various factors influenced the program's ability to optimize its efficiency and effectiveness, including: lengthy project preparation and approval processes that were, at times, inappropriately adapted to a fragile-state context; insufficient linkages between short-term START projects to development programming, with particular reference to capacity building; insufficient support for project implementation related to the effective integration of cross-cutting themes; and, the termination of START programming, which had enabled a whole-of-government approach and enhanced GAC's flexibility.

Where the program engaged in short-term projects, it would have been bolstered by an accompanying consistent, explicit and long-term vision and commitment.

The disruptive changes in programming priorities noted throughout the evaluation period and an inadequate linkage between corporate priorities and root causes of instability in South Sudan within xCIDA and xDFAIT, came at the expense of a more strategic and integrated long-term vision of Canada’s role in South Sudan, and proper linkages between the management of stabilization and development operations. Despite these challenges, there were some areas where programming results were sustained.

Long-term involvement from donor countries in South Sudan remained relevant in promoting progress toward stability and recovery for social justice purposes. Long-term involvement also remained relevant because South Sudan and other fragile states represented potential threats to international peace and security. While there were no easy and standard recipes for working in a fragile context, donor countries could have created opportunities to work outside the traditional frameworks which drew a distinction between: stabilization, peace-building, development and humanitarian approaches; formal and informal structures; center and the periphery; state and civil society; and macro- and micro-level approaches. Simple contexts do not exist among fragile states, only situations simplified by donor countries.

In this context, capacity-building was an important element of the international community’s engagement within the state-building process in South Sudan. Low levels in education and inexperience in governance at the national and state levels were challenges for capacity-building. In order to address immediate gaps that these entailed, the Development program invested in capacity replacement to assist the government organizations in their operations. It is important to note that some long-term capacity-building programming did take place within the health sector. However, generally, capacity-building was often too short-termed and focussed on workshops and short training sessions, particularly in START programming due to the rapid nature of the interventions. Notable lessons^{Footnote 25} have since emerged within GAC’s work on the nature of effective programming in fragile states, particularly regarding capacity-building. These lessons include:

• Long-term training is far more valuable than short-term training;
• Training is most effective when accompanied by capital investments (office space, equipment and systems);
• Training should follow or update the master curriculum; and,
• Training is most effective when it is directly linked to the specific skills that selected civil servants require.

These lessons mean that capacity building required planning across both stabilization and development streams. It can take a long time to achieve results - and this suggests a phased approach rather than the traditional short-term, quick-results-oriented project-based schemes. The advantages in taking a long-term approach across all programs have been highlighted in successes of development programming in the health sector through the HPF and midwifery project. By contrast, the START program did not have an articulated long-term vision for the program, nor did any of its capacity-building efforts result in long-term, local capacity gains. Focus on capacity replacement over capacity building had the effect of moving the Development program’s vision away from the long-term goal of institutional improvement, to the short-term results of the provision of needed basic services and functions. From a longer-term perspective, development programming in an environment with extreme low levels of capacity that necessitated capacity replacement in government services still required an eventual shift towards capacity building. Looking forward, failures in capacity building can pose a fundamental challenge to the long-term sustainability of results.

There was insufficient support for some program requirements, particularly the effective integration of cross-cutting themes, drivers of conflict, and all of the principles for engagement in fragile states.

There were difficulties in integrating the multitude of considerations that were intended to guide Headquarters’ programming decisions in Canada's development engagement in fragile states. These considerations included cross-cutting themes, the drivers of conflict, and the principles for engagement in fragile states. Overall, GAC had well-defined objectives, principles and guidelines for engaging in fragile states at the Headquarters level. While several of these were implemented at a high level, some principles (particularly related to linking programming across the Department) were not easily accomplished given Departmental administrative mechanisms and structures. An additional complication stemmed from the fact that the South Sudan Development program was one of GAC’s smallest Country of Focus programs. The expectation to update and integrate thinking about cross-cutting themes was incongruent with the resources available to the program.

A flexible approach to meeting the needs and challenges of people living in conflict-affected areas was required across GAC programming.

The end of START in 2013 removed one key tool that was effective in providing flexibility and a key link towards a holistic, whole-of-government approach to South Sudan. Showing promising results in South Sudan until its discontinuation in 2013, START provided a tool for the Government of Canada to make a necessary link between humanitarian peace-building efforts in South Sudan and development programming. This was viewed many interviewees as an innovative approach. The requisite conditions for flexibility and stability in programming in a fragile context were only partly attained within the South Sudan program over the period of the evaluation. If Canada’s intentions were to position the Principles of Engagement in Fragile States as the central framework for its operations in fragile states, then South Sudan programming should have been allowed to explore emerging and innovative opportunities and be allowed to take risks. Instead, staff had to focus mostly on administrative delivery.

5. Recommendations

1. GAC should ensure that future South Sudan programming is based on an integrated, whole-of-Department approach. Specifically, the continuum of programming should be based on:
   1. A long-term, common, and documented vision for the country;
   2. A recognition of the need for responsive, flexible and nimble programming to adapt to rapidly changing contexts;
   3. The effective integration of cross-cutting themes; and,
   4. An enhanced strategic analysis that addresses the drivers of conflict and is based in the Principles for Good International Engagement in Fragile States.
2. These elements could be implemented by an approved and integrated country program strategy, an integrated logic model and a policy dialogue strategy.
3. GAC should ensure that any additional or refocusing of requirements for the integration, analysis and synthesis of cross-cutting issues and conflict drivers within fragile states be adequately supported by specialists.
4. GAC should ensure that it undertakes efforts to limit the delays in project and planning approval through a review and streamlining of processes.

6. Considerations for Future Programming in Fragile States

There are opportunities to promote learning across programming and integrating best practices and principles for engaging in fragile states across the Department.

A number of recent evaluations have identified useful lessons for future programming in fragile states including in Afghanistan, West Bank and Gaza, and Colombia. The key is to integrate these best practices and principles into programming.

Published in 2015, the Evaluation of Canada’s Afghanistan Development Program (2004 – 2013), recommended: “develop a vision for Canada’s future engagement in Afghanistan” and “ensure clear strategic direction”. The same evaluation underlined “clear changes in strategy and focus” (state-building from 2004 to 2007, stabilization in Kandahar from 2008 to 2011, and humanitarian work, social sector and gender equality after 2011). The evaluation also noted that “needs assessments and conflict analyses used did not enable a complete understanding of the drivers of conflict and grievance, thus limiting overall development performance.” Finally, the same report mentioned “more attention should have been paid to the elaboration of a development approach in conflict zones as an intrinsic part of the whole-of-government approach.”^{Footnote 26}

Also published in 2015, the Evaluation of Canada’s Development and Humanitarian Assistance Programming in West Bank and Gaza (2008 – 2013), identified as one of the programming lessons that “strengthening institutions, particularly in a fragile state context, is a long-term process that requires commitment to long-term funding.”^{Footnote 27} The Colombia Country Program Evaluation (2006 – 2011), published in 2013, noted that in conflict settings “...it is important to design interventions with a clear understanding of the context, and to allow for adaptability as conditions change. The program has implicitly internalized this reality, but its analytic work could further benefit by including a range of scenarios that would help to better forecast risks and respond to changing situations in the field.”^{Footnote 28}

The Principles for Good International Engagement in Fragile States couldprovide a useful avenue for program coherence between relief, stabilization and development. An overt and intentional application of these principles within planning, design and implementation of START, development and humanitarian assistance programming would allow the South Sudan program to ensure that context is: taken as a starting point; identify and build on the obvious and subtle links between political, security and development objectives; provide flexible alignment with localized priorities in various contexts; and, ensure rapid response through a long-game lens. This would be done while reinforcing areas in which Canada has maintained strong adherence to non-discriminatory and inclusive programming that strives to achieve minimal negative impact on the people of South Sudan. With evidence of this necessity found in other fragile state contexts in which Canada has engaged, the South Sudan program could benefit widely from its systematic and overt incorporation at the country level.

The current weight of administrative requirements may hinder the optimal delivery of relevant and effective programs, particularly in fragile states.

The 2012 Development Assistance Committee (DAC) Peer Review of Canada’s Assistance Program underlined the need for Canada to allocate sufficient financial and human resources for delivering on commitments (OECD/DAC 2014, 7 Lessons from DAC Peer Reviews). In 2009 in her report on strengthening aid effectiveness, the Auditor General noted that the long-standing issue related to xCIDA’s lengthy and complex business processes remained unresolved. The same report added: “the large majority of CIDA staff interviewed and various members of the external development community said that administrative processes continue to burden the Agency”.

Similarly, the 2012 OECD/DAC Peer Review of Canada’s Assistance Program made the following recommendation: “Building on progress already made with its business modernisation initiative, CIDA should further simplify and modernise its development co-operation by 1) completing its decentralisation, giving field-based teams in partner countries enough advisory and managerial capacity and programme and financial authority to deliver more effective aid; 2) streamlining approval procedures further and making them more predictable; 3) clarifying, harmonising and simplifying reporting requirements.” One of the lessons the Canadian country evaluation of the West Bank & Gaza Program identified was: “When delivering a complex program in a fragile state situation, it is critical that program decision-making be as agile and timely and possible.”

GAC may have yet to redefine the “right balance” between the need to properly manage fiduciary risks and the responsibility to deliver timely stabilization, reconstruction and development operations in the field, which is another component of the accountability concept. With this “right balance”, GAC may have develop processes that allow programs to be more flexible, nimble, and more context-driven, and with nuanced analysis of power relations, causes of vulnerability, drivers of conflict and resilience indicators. Literature also suggests that addressing the root causes and drivers of the conflict will require a much better coordinated international approach, and much greater international leverage, which some such as John Prendergast of the Enough Project, describe as “cripplingly and puzzlingly insufficient,” given the tools at the disposal of the international community This may be missing from the design and implementation of programs in fragile states. Advancement in this area could contribute to enhancing the relevance of Canada’s investments in South Sudan. Engagement in South Sudan, as with all fragile states, needs recognition of the complexity of operating in such a context. GAC would benefit from ensuring that there are appropriate resources available to programs to meet expectations, which will certainly include an assessment of the fit between administrative requirements and program-related requirements. An adjustment of either of these may be necessary.