Evaluation of the Vietnam Country Program (2009/10 – 2016/17)

Final Report

International Assistance Evaluation (PRA)
Global Affairs Canada
April 2018

Acronyms

Executive Summary

The evaluation assessed the relevance, effectiveness, efficiency, and sustainability of development programming results from 2009/10 to 2016/17 to inform decision-making, and support policy and program improvements. It focused on the Vietnam development program, which represented the majority of investments in the country.

Focusing primarily on workforce development and capacity building within six key sectors (agriculture, small-medium sized enterprise (SME) development, skills and training, financial and legislative policy reform, environment policy reform and advancing democracy), was appropriate to Vietnam’s identified priorities. The Program worked closely with government partners and this proved to be a successful strategy to advancing results. Results were achieved in most sectors: interventions in agriculture and SME development demonstrated growth of value chains and improved business services; legislative and environment reform work enhanced the capacity of public institutions; and, Program interventions helped protect the space for non-government organizations in the country. Interventions in the financial reform sector fell short and failed to achieve results.

The Program had positive results with regards to sustainability. Several projects enhanced the Government of Vietnam’s capacity in strategic planning, governance, and law-making. With regards to the environmental sector in particular, long-term investments in industrial pollution management coupled with support to high-level planning for climate change meant that the Program achieved enduring results at both the provincial and national level.

The Program did not effectively target identified groups: women, ethnic minorities and the rural poor. More analytical work to understand the diversity of needs is required. In addition, the Program will need to engage with the Government of Vietnam to identify new sectors of intervention where it can have a more direct impact on target groups.

The Program faced challenges with regards to strategic planning and lacked an approved articulated program vision. The timely implementation of projects was impacted by the large scope of planned objectives, limited project cycle length (five years), and the administratively complex Vietnamese operating context. The absence of a common information management system between the Mission and headquarters made the efficient and reliable sharing and storing of information a challenge.

Notwithstanding Global Affairs Canada’s relatively modest investments in Vietnam, the Program’s experience and international reputation assisted in building relationships to enhance its presence.

Program Background

1) Vietnamese Context

Human and Social Development

Over the past few decades, Vietnam has transformed itself from one of the poorest countries to a lower middle-income country. This transformation resulted from key political and economic reforms such as the “Doi Moi” in 1986 and Vietnam’s accession to the World Trade Organization in 2007. Since the 1990s, the percentage of the population living below the national poverty line has decreased by over 50%, and was at 13.5% in 2014. The reduction in poverty was primarily caused by growth in manufacturing, with exports increasing on average by 21% annually between 2000 and 2010.

Within ten years, Vietnam increased its Human Development Index ranking from 120 in 1995 to 108 in 2005. This ranking fell back to 116 in 2015 due to Vietnam’s vulnerability to climate change, natural resource depletion, corruption, and deepening socioeconomic inequalities caused by rapid urbanization. Ethnic minorities, about 14.3% of the total population, have accounted for almost half of the chronic poor, and their access to services has continued to lag. Despite significant progress in the legal, political, social, and economic sectors, benefits have been inconsistent for women, particularly among the poor and those in rural areas.

The competitiveness of the Vietnamese economy has been based on the low costs of labour and natural resource extraction. The poor have relied heavily on subsistence agriculture and employment in the informal sector, had lower access to skills training, and were more vulnerable to natural disasters. The challenge has been to move toward more sustainable and inclusive economic growth based on increased competitiveness and access to economic opportunities for all segments of the population.

Centralized Governance Structures and Vietnamese Ownership

Vietnam has made progress in economic reform and poverty reduction while maintaining its  one-party state system. This means that both governance structures and society have been highly centralized, and institutional capacity has been limited for legislative drafting and policy development.

State leadership in the country’s assistance process and development projects was integral for the Government of Vietnam. Its Socio-Economic Development Strategy (SEDS) 2011-2020 outlined the need for institutional reform and environmental sustainability to reduce inequalities and increase economic stability. It acknowledged the need for a healthy environment, which has deteriorated due to forest loss, intensive agriculture, industrial pollution, and population growth. The country was also vulnerable to climate change, particularly low-lying areas where economic activity and the population were concentrated.

2) Donor Context

Official Development Assistance to Vietnam

In 2016, total official development assistance (ODA) to Vietnam was US$2.8 billion. Vietnam receives significant bilateral support from Japan, with lesser amounts from the International Development Association, Asian Development Bank Special Funds, EU institutions, the Global Fund, Korea, Australia, Germany and the United States. Total development assistance provided only 3% of the national budget and was surpassed by foreign direct investment, tourism, and remittances. On average, the economic growth sector accounts for just over half of all contributions, with other social service development representing 13% of total bilateral ODA.

Donor Coordination

The Government of Vietnam (GoV) coordinates ODA in line with both the Paris Declaration on Aid Effectiveness and the Hanoi Core Statement on Aid Effectiveness. Donors, including Canada, have aligned ODA priorities in the country to those outlined in the GoV’s strategies and plans. Key donor coordination mechanisms include the Partnership Group on Aid Effectiveness and the Vietnam Development Forum, in addition to sector-specific working groups which include GoV partners and donors.

Canada-Vietnam Cooperation

Canada has provided development assistance to Vietnam since 1954, with significant increases in aid contributions within the last two decades. From the 1990s until 2016, Canada has provided a total of $1.5 billion in development assistance to Vietnam. Canada has been a middle-rank bilateral donor to Vietnam, having maintained a strong relationship with the government and a good reputation as a donor. Canada has contributed significantly alongside other donors, mainly to the economic and social development sectors.

In 2009, Vietnam was identified as a country of focus for Canada’s bilateral development assistance, a commitment that was reaffirmed in 2014. On November 8, 2017, Canada and Vietnam released a Joint Statement on the establishment of a Comprehensive Partnership. The statement reaffirmed the strong bilateral relationship and outlined key areas of cooperation, including trade and investment, development cooperation, diplomacy, and science, technology, and innovation.

In line with the GoV’s prioritization of economic and social development, Global Affairs Canada implemented activities with outcomes in human resource development and infrastructure creation, and the improvement of economic efficiency and competitiveness. Sustainable Economic Growth provided a thematic framework for capacity-building and institutional development across all sectors of engagement for the Program.

3) Global Affairs Canada Programming

Vietnam Program

The Vietnam International Assistance Program consisted of the Asia Pacific Branch (OGM), the Partnerships for Development Innovation Branch (KFM) and the Global Issues and Development Branch (MFM) programming in the country. During the evaluation period, several projects were delivered through program-based approaches (PBAs) with the objective of building government capacity by providing funding directly to recipient government institutions. Other projects were channeled through contributions to pooled funds and projects implemented through a mix of multilateral organizations, Canadian non-governmental organizations, and private-sector firms.

The Program was partially-decentralized: the Director operated at Global Affairs Canada headquarters in Ottawa while the Head of Cooperation was located at the Mission in Hanoi. Both KFM and MFM projects were managed separately from headquarters. In 2016, a Field Support Services project replaced the Program Support Unit for administrative, logistical, and technical assistance.

International Assistance Disbursements

Between 2009/10 and 2016/17, Global Affairs Canada invested $208 million in Vietnam, with average disbursements of almost $30 million per year. The breakdown was as follows:

Text version

A stacked graph demonstrating disbursements of international assistance funds to each of the Global Affairs Canada program branches.

• OGM (Africa Branch) receives 80% of funding
• KFM (Partnerships for Development Innovation) receives 17% of funding
• MFM (Global Issues and Development) receives 1% of funding
• IFM/BFM/PFM (International Security and Political Affairs/ International Business Development/Strategic Policy) is combined and receives less than 1% of funding

Disbursements by Global Affairs Canada Priority

Between 2009/10 and 2016/17, Global Affairs Canada’s bilateral assistance was predominantly focused on three priorities: Sustainable Economic Growth was the largest (50%, $83 million), followed by Food Security (20%, $33 million), and Advancing Democracy (16%, $26 million). The majority of SEG programming was in public sector policy and management, small and medium-sized enterprise (SME) development, and environmental policy. Food Security programming included agricultural development and agricultural services. Advancing Democracy programming included work on legal and judicial development, democratic participation and civil society, and human rights. The balance of spending (14%) in other areas included: children and youth, humanitarian assistance, security and stability, health, education and social services, and disaster prevention and emergency preparedness.

Methodology

1) Evaluation Purpose and Scope

The purpose of the evaluation was to improve the achievement of Global Affairs Canada’s  international assistance programming results. Additionally, the purpose was to enhance understanding of development programming results in Vietnam that Global Affairs Canada seeks to achieve, does achieve, as well as the resources used to achieve them.

The Evaluation covered the period from 2009/10 to 2016/17 and focused on Global Affairs Canada’s whole-of-development programing in Vietnam, representing $208 million. The assessment of results focused on the Vietnam Development Program (OGM – 81% of disbursements) and, where applicable, the Partnership for Development Innovation Program (KFM) and the Global Issues and Development Program (MFM).

The Evaluation team developed a reconstructed Program logic model with the Program, that clustered programming into six sectoral case studies: agriculture, SME development, skills and training, financial and legislative policy reform, environmental policy reform, and advancing democracy.

2) Evaluation Questions

Relevance

1. To what extent has Global Affairs Canada’s Program responded to the needs and priorities of the target populations?
2. To what extent is the Vietnam Program aligned with the country’s needs and national priorities?

Effectiveness

3. To what extent did Global Affairs Canada’s Program achieve immediate and intermediate outcomes and contribute to achieving the ultimate outcome?
4. To what extent did the Global Affairs Program in Vietnam achieve results related to gender equality, environmental sustainability, and governance?
5. To what extent did policy dialogue contribute to the achievement of outcomes?

Sustainability

6. To what extent have, or will, the results and benefits of Global Affairs Canada’s Program continue beyond Global Affairs Canada’s development assistance?

Efficiency

7. Are there any opportunities to improve the operational efficiency of Global Affairs Canada programming?

3) Methodology

The evaluation was conducted by the International Assistance Evaluation Division, with the support of external consultants in Vietnam for translation, data collection, and logistical support. A data collection mission to Vietnam was conducted in July 2017.

The evaluation used a mixed methods design with an embedded case study approach. Six explanatory case studies were conducted. Case studies were project-focused and purposively selected across Global Affairs Canada thematic priorities, time periods, implementing partners, delivery channels, and investment types. Projects from KFM and MFM were integrated where there was alignment with Program results and where they were part of the ODA funding envelope.

Data collection methods included:

6 Case Studies (Selected Projects)

Case study projects purposefully selected and based on the sectors identified in the reconstructed logic model.
Included 13 projects with disbursements of $66.4 million (36%) which represented:

• OGM $61.6 million (39%) - 8/46 projects
• KFM $3.5 million (15%) - 4/36 projects
• MFM $1.3 million (94%) - 2/22 projects

Key Stakeholder Interviews (n = 134 stakeholders)

122 Semi-structured individual and small group interviews for case studies and Program level review, comprised of:

• Global Affairs Canada (n = 19)
• Canadian executing agencies (n = 35)
• local government (n = 66)
• beneficiaries (n = 2)
• project evaluators/monitors (n = 8)
• International donors (n = 4)

Document and Data Review (450+ documents)

Review of:

• internal policy and strategic documents
• external documents pertaining to Vietnamese priorities and context
• project level documents for case studies

Literature review conducted in partnership with the University of Ottawa was used to inform the design of the evaluation.

Site Visits (n=11)

Case studies for Agriculture, Small and Medium-Sized Enterprise development, Environmental Policy Reform, and Skills and Training sectors included site visits.

Site visits also allowed the evaluation team to informally speak with project beneficiaries.

Financial Analysis

Review of statistical information available on total disbursements, thematic areas targeted, gender coding, and changes over time.

Evaluation Findings

1) Relevance

The Program was aligned with the official development needs and priorities of Vietnam.

In accordance with the Paris Declaration, the GoV exercised a great degree of control and ownership over aid interventions in the country. The GoV outlined its development needs and priorities through key documents including the SEDPs (2006-2010, 2011-2015, 2016-2020) and the SEDS (2011-2020). Key areas of focus include: improved market efficiency and competitiveness, human resource development and infrastructure. In addition, the recent development of national laws on gender equality and the environment signaled Vietnam’s recognition of these areas as priorities.

Rapid economic growth has posed risks to the environment, the economy, and social well-being, and the Program chose to strengthen systems in order to respond to these challenges. The Program’s direct engagement in national policy reform, as well as in geographic regions that were of greater risk to environmental and economic shocks, reflected strong alignment with the expressed needs of Vietnam.

Capacity building, primarily within government agencies, was an integral part of programming and helped shape Vietnam’s plans, priorities, and strategies across levels of government and thematic sectors.

The Program conducted sector-specific analyses and understood the need to pay attention to certain higher-risk populations. However, in practice, the needs of target groups, such as women and ethnic minorities, were not well reflected in projects.

Informed by the Vietnamese socio-economic context and vulnerabilities identified in the SEDPs, country strategies and accompanying logic models identified women, ethnic minorities, and the rural poor as target groups. Strategic planning documents identified the need for greater access to and inclusion in education, services, economic opportunities, and infrastructure.

The Program targeted different levels of stakeholders including rural populations. However, the Program’s overall explicit focus was on institutional capacity building at the national and provincial levels, and it therefore targeted interventions directly towards government institutions. The Program made an effort to focus some projects on provincial government stakeholders who represented rural areas and addressed the needs of those populations, but efforts to target groups of women and ethnic minorities were indirect and results were not immediately demonstrated.

Women

Projects in the agricultural, SME development, legislative policy reform, and skills and training sectors included comprehensive analyses of barriers to access for target populations. Efforts were undertaken in the legislative reform sector to train partners on gender mainstreaming in data collection within the lawmaking process in accordance with a draft strategy for gender equality.

However, specific activities to target populations were absent or not aligned to the identified needs of women in these sectors.

Ethnic Minorities

Vietnam has valued unity over diversity, meaning cultural attitudes towards ethic minorities favour integration. This focus on integration, along with weak mechanisms for obtaining ethnic minority perspectives meant that projects were not necessarily informed about the most appropriate ways to address the needs of ethnic minorities. This resulted in gaps between what the Program hoped to achieve and implementation.

For example, the Soc Trang SME Development Project operated in a region where the Khmer minority represented 30% of the population, but the products that were developed were not translated into the Khmer language. Skills and training programming operated in regions and offered programs where ethnic minorities were present, but relied on passive pull tactics to attract students from these communities.

Rural Poor

While several projects were operating in geographic regions with a primarily poor and rural population, efforts to adapt programming to correspond to the specific needs of this target group was underdeveloped. Project activities at the provincial level brought Canadian engagement closer to rural target groups, however most of the emphasis in programming was placed on institutions at the national level.

2) Effectiveness

Agriculture

Agriculture programming contributed to increased market efficiency and competitiveness; supported human resource development; and improved infrastructure.

Agriculture was one of the primary drivers of Vietnam’s transition from a low-income country to a middle-income country. Programming was designed to help increase the competitiveness and quality of agricultural production to support food security and sustainable economic growth.

Funding in the agriculture sector through a PBA with the provincial government helped identify and implement priority actions within Ha Tinh’s 2011-2015 Agriculture and Rural Development Plan based on market analyses and consultations with key stakeholders. The new planning approach encouraged interdepartmental coordination and the funding model was selected to support government ownership over programming and results.

Agricultural development supported the creation and maintenance planning of small-scale infrastructure projects that supported agricultural production work. These results included the building of roads, bridges, and irrigation systems in Ha Tinh province. Infrastructure planning included consultation with local stakeholders to assess and mitigate environmental and economic risks.

Agriculture programming also focused on building the capacity of provincial institutions to promote fair and sustainable production. Through study tours, technical assistance, and training, the Ha Tinh Department of Agricultural and Rural Development increased  its capacity to plan, manage, implement, and monitor agricultural programs province-wide.

At a higher level, agriculture programming contributed to increased access to economic opportunities by some target groups. There is evidence to indicate the growth of three value chains in participating communes in Ha Tinh province.

SME Development

Programming in the small and medium-sized development sector contributed to increased market efficiency and competitiveness, human resource development, and infrastructure improvement.

The Program’s overall objective of stimulating sustainable economic growth meant that supporting SME development was a primary area of focus during the evaluation period. The Program recognized that poverty in Vietnam was primarily rural and linked to small-scale agriculture, making the diversification of incomes, especially for women, central to continued poverty reduction. Further, in line with the GoV’s desire to increase market competitiveness, programming focused on higher productivity and private-sector development, particularly of SMEs in the agricultural sector.

The Program supported projects operating in rural areas to increase the capacity of institutions to facilitate SME development. Through a PBA with the provincial government, the Soc Trang SME Development project supported the implementation of the Five-Year SME Development Plan for 2011-2015, in which the main objectives were creating a favorable business environment, improving local infrastructure, and enhancing public governance to support SME development. The project also assisted in the development of the 2016-2020 plan which was approved by the Provincial People’s Committee in December 2016.

The Soc Trang Department of Public Investment (DPI) improved its capacity to support SMEs in the province, by issuing business registration certificates and completing small-scale infrastructure projects designed to facilitate the transportation of goods. The success of these components fostered a greater sense of trust between local SMEs and the DPI, creating a more favorable environment for business in the province.

At a higher level, there is evidence that SMEs owned by men and women in Soc Trang benefitted from improved business support services from the province and were able to increase business competitiveness as a result of the project. However, language and access barriers continued to limit the ability of the province to provide access to credit and offer a full range of services to ethnic minorities.

Skills and Training

Programming in the Skills and Training sector contributed to results in increased market efficiency and competitiveness, human resource development, and helped improve infrastructure.

Vietnam’s SEDS 2011-2020 emphasized that a skilled workforce was essential to future competitiveness and growth. In order to improve the quality of the workforce, the GoV emphasized addressing weak links between employers and labour market demands, outdated gender-stereotyped curricula, poor teacher quality, and lack of inclusiveness for ethnic minorities.

With this in mind, the Program aimed to increase access by women, youth, and ethnic minorities to demand-driven skills training and knowledge needed for formal labour market participation, including employment in small and medium enterprises, and to increase the availability of meaningful workplace learning opportunities across a variety of sectors.

During the evaluation period, the Vietnam Skills for Employment Project (VSEP) was still in its early stages and it was too early to fully determine its achievement of planned immediate outcomes. However, there was capacity development within selected community colleges to provide programs that are responsive to local industry partners. Through study tours, college staff were able visit Canada to gain a deeper understanding of how the Canadian college system works.

The project also funded outreach to determine how to better align curricula with local business needs, and studies to assess the barriers to access education for targeted groups. At the same time, early efforts also demonstrated that there had been a passive approach to reaching women and ethnic minorities for college recruitment.

Due to the early phase of the project and delays in project implementation for the VSEP project, it is too early to assess whether programming in this sector contributed at a higher level to increased competitiveness and access to sustainable economic opportunities for women and ethnic minorities.

Financial and Legislative Policy Reform

During the evaluation scope, the GoV was primarily interested in developing clear and more consistently implemented national laws, while also conforming to international standards in the supervision of the financial sector. Key Program objectives responded to this need by focusing on building capacity for financial sector regulation and standardization of the legislative process. Canada provided practical knowledge and tools including results-based management approaches, collaborative models, and evidence-based planning.

Legislative reform work through the National Legislative Development Project (NLD) helped improve the policymaking capacity of select ministries. Further, the Ministry of Justice and Ministry of Agriculture and Rural Development, among others, improved their capacity to draft legislation and to strategically plan for the development and revision of national legislation.

There was also evidence of increased capacity within State Audit Vietnam (SAV) to perform quality audits of public spending, and an increased understanding of international performance audit standards.

At the same time, there were substantial difficulties in implementing project components related to public finance in financial policy reform. This required the intervention of the Head of Mission when the Banking Regulation and Supervision Support (BRASS) project stalled. This meant, at the time of the evaluation, there were few demonstrable results in the implementation of systemic change within the State Bank of Vietnam.

At a higher level, there is evidence that programming has contributed to increased effectiveness, transparency, and accountability of targeted institutions. Canadian technical expertise through NLD provided groundwork for a fundamental shift in how legislation is drafted, particularly surrounding the separation of legislative drafting and policy analysis through the Law on the Promulgation of Legal Normative Documents. The translation and publication of select audits contributed to increased transparency and accountability of national public institutions.

Environmental Policy Reform

Environmental policy reform programming helped contribute to human resource development priority through capacity building of government employees.

Informed by environmental analysis and strategic planning, the Program focused on technical and financial assistance for the development of strategies, plans, and management of environmental budgets, industrial pollution, enforcement of environmental regulations at both provincial and national levels, and increased awareness of environmental issues across the government and the private sector.

Projects helped improve the ability of government employees to develop and implement effective environmental policies. Interviewees all agreed that Canada’s contributions helped shape and advance the national environmental action plan, despite its relatively modest investment ($4 million) in the Support Program to Respond to Climate Change (SP-RCC).

At the national level, the Ministry of Natural Resources and the Environment improved its capacity to draft regulations, policy, and guidelines related to climate change. At the provincial level, Departments of Agricultural and Rural Development increased its capacity to manage agricultural adaptation components of the national regulatory and policy framework linked to the National Target Program for Response to Climate Change. The Vietnam Provincial Environmental Governance (VPEG) project ensured provincial environmental protection agencies were better-equipped to collect data on industrial pollution in selected provinces.

Results were also achieved at a higher level, in terms of increased effectiveness, transparency, and accountability of environmental institutions. VPEG supported an improved regulatory framework for the management of industrial pollution developed through the Law on Environmental Protection (2014). Participation in the project also encouraged the use of collaborative approaches in policy design and implementation, a model which is now known as the “Canadian Way” in Vietnam.

Canada also supported the allocation of responsibilities by government institutions for climate change actions planned and monitored using Results Based Management.

Advancing Democracy

Advancing democracy programming supported human resource development by building government capacity to engage with the public.

The Program focused its work in this sector on institutional capacity building. With the support of NLD, the GoV codified the need for public consultation within the law-making process as a component of the Law on the Promulgation of Legal Normative Documents in 2015. This was a high-level recognition of the importance of meaningful engagement with stakeholders in the legislative process.

The Ministry of Justice, among others, increased its capacity to undertake and use evidence-based research for policy analysis through NLD. The Ministry of Justice also engaged with stakeholders through consultations in the policy-making and law-making processes. As this project is still ongoing, the evaluation was unable to assess if it was successful in increasing the influence of civil society.

Nevertheless, through policy dialogue, Canada successfully advocated against tighter controls on international organizations operating in Vietnam. In 2010, Canadian organizations faced barriers over their geographic and thematic focus and local partnerships from “Decree 93”, a law that regulates the implementation of international non-governmental organization-funded initiatives. Canada was among the leaders in the donor community working to promote civil society space in Vietnam. The Program was active in representing Canadian partners in discussions with the GoV that included donors, and local and international non-governmental organizations.

This policy dialogue work was complementary to Program outcomes related to bolstering the voice of civil society. Notwithstanding, the evaluation was unable to determine the impacts of this work on achieving results in this area.

Cross-Cutting Themes – Gender Equality

Identifying and communicating gender equality as an issue in Vietnam was an ongoing challenge, as stakeholders across all sectors of engagement did not perceive it as an issue.

Many stakeholders cited the presence of the Gender Equality Law, the Women’s Union, Gender Equality Working Groups, and the significant representation of women in the workforce as reasons why gender equality was not seen as an important societal concern in Vietnam.

The socio-cultural dimensions of gender equality issues in Vietnam made it challenging to incorporate these issues in economic programming. Other issues identified by the GoV, such as compulsory retirement age, gender-based violence, and female infanticide, were not addressed by the Program. Gender equality, as a cross-cutting theme, was not always integrated in meaningful ways across programming.

Gender equality and its importance for Global Affairs Canada was not well understood or accepted by some projects. The need for reporting and action on gender equality was seen as a burdensome administrative step by some. Access to and use of gender experts, both at headquarters and the Program Support Unit/Field Support Services, was limited for Program and project staff.

Gender Integration in Projects 2009/10 - 2016/17

Text version

A bar chart demonstrates the percentage of projects that have integrated gender equality into their programming.

• None is displayed in dark green with 22% of projects
• Limited is displayed in green with 36% of projects
• Integrated is displayed in light green with 42% of projects
• Specific is not displayed as there is 0% of projects represented in this category

The Gender Equality Program Strategy was last approved in 2009. Interviewees confirmed that gender was not an objective of the Program, and that this was consistent with the direction they were receiving from headquarters.

Gender was integrated, at some level, in the majority of projects. There were no gender-specific projects or programming across the Program as a whole.

Of the 48 projects active during the evaluation period, none were specifically focused on gender. Thirty-one projects, representing 78% of disbursements, included results for women or girls at the intermediate or immediate levels. However, while some projects planned for these gender equality results, the evaluation was unable to find significant achievements for women in any projects.

Some efforts to integrate gender equality appeared superficial and could not be expected to achieve significant gains for women and girls. In some cases, plans to integrate women were far removed from the project objectives.

Cross-cutting Themes – Environmental Sustainability

The Program integrated and implemented environmental sustainability into sectors where projects impacted or were impacted by environmental degradation and climate change.

The national and provincial governments identified environmental degradation and climate change as key challenges and priorities in their economic strategies. Consequently, the Program included them as both targeted and cross-cutting interventions.

Overall support to policy work on environmental degradation and climate change across 9 national ministries and 12 sectors added to the breadth of environmental sustainability planning and implementation to address environmental impacts. The Program integrated disaster risk management and cleaner production within projects supporting the agricultural and industrial sectors at the provincial level. Small-scale infrastructure projects were implemented to address the negative environmental impacts of rapid economic growth. The Program implemented specific projects in industrial pollution management and cleaner production practices, that increased the capacity of 8 provincial environmental protection agencies to plan and monitor for pollution.

In many cases, results in environmental sustainability helped spark progress in work on climate change and environmental degradation in key sectors for the GoV.

Initially, government partners did not have the information needed to inform policy. There was a disconnect between high-level work done at the national level and the on-the-ground realities faced by local institutions. VPEG helped bridge the gap between national and provincial work by supporting the integration of provinces into the national process of regulation and enforcement.

In 2011, Canada’s support to environmental protection evolved to include policy direction on climate change. The SP-RCC was a strong policy dialogue mechanism that cut across priority sectors. Canada played a strong part in the implementation of the policy matrix within the agriculture sector, and helped to address the challenges of turning climate change plans into policy, and environmental financing. Canada’s two-year modest contribution to this pooled fund provided grants to capacity-building and technical assistance that the GoV was not receiving through the loans structures adopted by other donors.

Cross-Cutting Themes – Governance

As a result of the needs and context of the country, the Program focused on high-level governance work within institutional capacity-building and accountability and did not focus on participation and inclusion.

By virtue of the Program’s design, traditional governance parameters were present across programming sectors in the areas of public sector management, accountability, and service delivery. Several projects included technical assistance related to capacity-building and improved public governance, and PBAs focused primarily on governance-related results.

It is important to note that Global Affairs Canada governance principles were not binding, and guidance from headquarters was given to identify the most contextually-relevant principles for application within programming, with the understanding that not all principles were required. Evaluation interviews and discussions indicated that all governance principles were not well understood in some projects.

Consequently, results related to participation and inclusion of the voice and interests of civil society in projects were weak. Representation of the interests of civil society, with the exception of women’s unions, was not included in the design, implementation, or management of projects. Culturally, a strong and vocal civil society were neither supported nor encouraged. Projects that included a civil society engagement component, such as NLD, did not achieve results in this area during the scope of this evaluation.

Governance Programming

Text version

A pie chart indicates governance programming in Vietnam. The largest piece of the pie is indicated in green for Institutional Capacity-building and Accountability. The remaining two pieces of the pie are of equal amounts and represent Governance Sector not indicated, and Enabling Environment for Civil Society.

Institutional Capacity Building and Accountability DAC Sectors

Text version

A pie chart indicates Institutional Capacity Building and Accountability DAC Sectors. The largest piece of the pie is dark green and demonstrates Public Sector Management. It is followed by the second largest pie of the pie which is light blue and indicates Service Delivery. The smallest piece of the pie is bright blue and indicates Rule of Law.

3) Sustainability

Projects helped the Government of Vietnam develop and implement their plans in the sectors of environmental policy and legislative reform, agriculture and rural development, and SME development.

VPEG supported adjustment of the Law on Environmental Protection in 2005 and 2013, and was instrumental in ensuring the allocation of 1% of state budgets to environmental protection. The implementation of SP-RCC helped develop policies related to climate change in Vietnam as well as supporting the development of an institutional climate change framework and its implementation at the national and provincial levels.

NLD was a key partner in supporting the drafting of the Law on the Promulgation of Legal Normative Documents in 2015 which had widespread implications for future lawmaking in Vietnam. The Program supported the implementation of Ha Tinh’s Agriculture and Rural Development Plan (2011-2015) and helped inform its subsequent plan for 2016-2020. The Program also supported the development of Soc Trang’s SME Development Plans for 2012-2016 and 2016-2020.

The legacy of capacity building activities and training was evident in multiple sectors.

Many VPEG-trained staff continued to work in national and provincial environmental protection agencies, creating an informal community of practice. Several graduates assumed leadership roles and continue to implement VPEG practices such as Results Based Management. As well, provincial environmental monitoring centres remained in operation, though at different levels of capacity, four years following the end of the project. KFM programming with SAV provided hands-on knowledge of audit practice. Several graduates continued to work as auditors in SAV and some took on leadership roles within the institution.

NLD produced two guidebooks for legal drafting and policymaking, and built the capacity of colleagues to be able to train others once the project is over. A Ministry of Justice Gender Equality Working Group was established to share and build knowledge on the implementation of the Gender Equality Law and policy analysis related to gender equality.

There is evidence that communities were involved in the creation and maintenance of small-scale infrastructure. Infrastructure, equipment, and practices put in place by the projects were still in use.

Small-scale infrastructure built in the agricultural and SME development sectors continued to be used and maintained throughout the project cycle. Annual maintenance plans were developed in conjunction with communal and district-level authorities, and included environmental assessments. The fact that these were produced in consultation with local stakeholders and the provincial government helped ensure a high degree of ownership over these infrastructure works.

Project results were successfully facilitated by relationships with individuals and within networks.

Relationships were important in the Vietnamese context and assisted in obtaining the buy-in of partners. Establishing and working through networks was an important and enduring feature of some projects. For example, VPEG resulted in the creation of informal networks and working groups that continued to exist years following the end of the project. In general, there was a correlation between the breadth and depth of the relationship and results achieved.

In some cases, project theories of change required contextual changes beyond project scope to achieve sustained results.

Project activities did not always realistically match planned outcomes. Results were often necessary steps towards reaching high level results, but could not be expected to achieve them without significant societal change supported by a sustained and long-term engagement.

For example, VPEG supported enhanced data collection on industrial pollution. However, the regulatory framework linking industrial pollution data and the enforcement of environmental protection laws were largely absent in the country, and Vietnam continues to face critical environmental management issues. Additionally, the ‘fee for service’ financing model to offset a lack of government funding in industrial pollution management, potentially threatened the independence of the monitoring process.

In the case of KFM’s audit projects, there was a need to improve the government's ability to respond adequately to the audits being generated as a result of capacity building activities within the auditing community in Vietnam.

4) Efficiency

Working directly through and within Government of Vietnam structures was appropriate, but complex administrative processes resulted in operational delays and required no-cost extensions in four out of six sectors.                               .              

Both PBA and project-type interventions demonstrated that the GoV’s buy-in, approval processes, and coordination functions were key determinants of the successful implementation of projects. There were several requirements for complex governance structures such as establishing Project Steering Committees and Project Management Units, both in the case of PBAs and for projects implemented through a Canadian agency. Day-to-day operations required multiple levels of GoV approvals, meaning delays occurred frequently.                  ..

Both Global Affairs Canada and the GoV required ministerial approval for any changes to project designs and outcomes. Lengthy approval processes and unclear differentiation of roles and responsibilities in some project steering committees contributed to project delays. In some cases, there were difficulties defining project management responsibilities and a lack of clarity on which budgets should cover project-related expenses when multiple government institutions were involved. Administrative processes related to hiring and procurement of equipment also led to delays. Staffing technical specialist positions at the project level delayed implementation in some cases, particularly as a result of the limited pool of local specialists in the areas of environment and gender.

Key concepts that are central to Global Affairs Canada, such as Results Based Management and Governance, were difficult to translate across cultures and negatively impacted the timeliness of project implementation.

There were challenges around some Global Affairs Canada  corporate language and how it was understood within the Vietnamese context and by Canada’s partners in the field, in particular surrounding Results Based Management programming and Governance. In VSEP, the democratic and participatory approach to education required additional negotiations to find consensus in its application among local partners and the executing agency.

Investment plans were not aligned with evolving program management tools, resulting in sub-optimal Program planning and reporting.

Multiple changes to the strategic priorities and vision of Global Affairs Canada over the course of the evaluation period challenged the Program to operate strategically. Core guiding documents such as the country development programming framework, the gender equality strategy, and the policy dialogue action plan were either outdated or in draft form.

While the Program sought to consolidate corporate planning and reporting tools with updated corporate priorities, such as the Program’s logic model and performance measurement framework, final approvals were not provided. Multiple iterations of these documents meant that they were not used as they were intended, nor were they used to guide programming decisions reflected in investment plans.   

The impact of Program decentralization was mixed.

The Program became a standalone program following the closure of the Cambodia program in 2013. It became a decentralized program shortly after. In comparison to other decentralized programs, the Program had higher operating costs as a percentage of total budget, although these costs are in line with other programs in the region. Most Program staff (71%) were based in Canada’s Mission in Vietnam through a mix of Locally-Engaged Staff and Canada-based staff, with a small compliment (29%) at Canadian headquarters.

Staff in Vietnam and Canada reported increased levels of interaction and communication contributing to improved capacity to make informed decisions following decentralization. Conversely, Canada-based field staff were rotational as a result of corporate mobility cycles, and were engaged for shorter durations than Locally Engaged Staff who remained at the Mission for many years.

Until 2016, the Program relied on a Program Support Unit to assist with logistics and technical assistants engaged to support programming. In the same year, this mechanism was replaced with a Field Support Services project. Despite a relatively seamless transition between the two mechanisms, there was a time where services were not fully available and logistical planning had to be shifted to the Program. In addition, during the evaluation period, work between Canada and Vietnam was operationally challenging due to the time difference and distance required for field visits. The Program and project implementers had to dedicate significant amounts of time to managing communications, logistics and schedules.

Challenges with information and knowledge management in Canada and Vietnam contributed to operational inefficiencies.

The Program did not have access to an information system common to both Headquarters and the Mission, resulting in lost or misplaced program documents and difficulties in version control. The Mission staff were unable to access the information management system at headquarters, which necessitated the creation of a duplicate shadow information system. This was evidenced during the evaluation through difficulties in tracing key program documents for review.

Knowledge management and corporate memory relied on Locally Engaged Staff who were present for longer periods and able to fill knowledge gaps. Challenges in both information management and knowledge management opened the Program to risks of redundancy, sub-optimal planning, reporting, and accountability, and had an impact on corporate memory. Key documents were not always appropriately stored or archived.

Conclusions

The Program was responsive to GoV needs and to Global Affairs Canada’s priorities, and programmed in areas where there were shared interests and opportunities. The Program maintained its strong partnership with GoV and worked within GoV structures to further mutual objectives, demonstrating a strong understanding of the operational context in Vietnam. This alignment of priorities reinforced the Program’s ability to sustain results even after projects ended.

The Program achieved results in the agricultural, environmental, and legislative reform sectors as a result of its engagement at multiple levels of government, long-term presence, and positive policy dialogue with key interlocutors. In sectors where these success factors were not present, projects were slower to implement and had poorer results. A focus on strengthening government institutions contributed to systemic changes needed for inclusive growth and governance, environmental action, and food security in the country.

Capacity-building through the provision of technical expertise was a core component of many projects. In most cases, Canadians were regarded as contributing positively and brought technical expertise that was valued and appreciated by GoV partners. In the environmental sector, Canada’s reputation and expertise granted the Program a seat at the negotiations table and an ability to influence decision-making beyond the size of its financial contributions.

While women were identified as a target population for the Program, gender considerations were only marginally integrated into projects, there was a lack of gender-specific programming, and reporting was generally at an activity level. While there was an acknowledgement of the unique needs of women, analyses did not appear to guide programming decisions. The Program also had limited success in targeting ethnic minorities or the rural poor. This underlines the importance of identifying where the Program achieved results and using these to strategically determine sectors of engagement.

In terms of program management, a common and reliable information management system between the mission and headquarters would have allowed for greater efficiency. The Program benefitted from the presence of locally engaged staff who helped bridge cultural and linguistic barriers and were a strong source of corporate memory. Canadian staff was essential to interpreting and reinforcing Canadian approaches and in promoting and integrating Global Affairs Canada’s priorities.

The Program has demonstrated an in-depth understanding of how best to operate within the complex Vietnamese context. The Program’s strengths in relationship-building and respect for partner ownership of development assistance has led to sustainability of results. Overall, the Program’s approach to long-term work and presence in Vietnam has provided platforms for further discussion on Canada’s international assistance priorities and other areas of mutual investment interest.

Recommendations and Management Responses

Recommendation 1: Consider sun-setting projects in sectors where the achievement of results has been limited and find different ways of engaging in those sectors where there is mutual interest.

Management Response 1: Agreed. The Program will have regular policy and program dialogue with the GoV to deliver more development results in sectors of mutual interest including rural development and climate change. The Program will develop a programming paper to identify strategic orientations and action areas, consistent with the Feminist International Assistance Policy and the SEDP 2016-2020 and based on lessons learned. The paper will identify ways of engaging for optimal impact, including private sector involvement climate action.

Recommendation 2: Use analyses of the challenges that have made it difficult to advance gender issues to identify new programming opportunities. Where this is not possible, use policy dialogue to lay the groundwork for future programming.

Management Response 2: Agreed. The Program will leverage the resources of the FSS and Development Partners to refine analyses and to inform policy dialogue with the GoV on gender equality and women and girls. The Program will launch a review through the FSS of current Global Affairs Canada programming in Vietnam and identify opportunities to optimize the achievement of gender equality results in operational projects. The Program will discuss, in Hanoi, with the senior leaders of the GoV on opportunities to advance gender equality and results for women and girls in development cooperation.

Recommendation 3: Consider the length of time needed for societal change and lengthy approval processes when designing project timelines.

Management Response 3: Agreed. The Program will engage with senior leaders of the GoV to explore effective ways to address operational challenges. Besides building in appropriate time in project timelines to allow for the approval processes of both governments, the Program will seek to program in new ways that do not require lengthy approval and management processes -- to maximize development effectiveness. The Program will discuss with senior leaders of the GoV, operational issues, approval processes for projects and Memoranda of Understanding for each project, and mechanisms to deliver Official Development Assistance in the most effective way.

Recommendation 4: Articulate a coherent strategy in core documents, which are approved by senior management to serve planning and results management.

Management Response 4: Agreed. The Program will realign its strategy with the Feminist International Assistance Policy and take into account the feminist approach, the Action Area Policies that are under preparation, and additional policy guidance on innovation. The Feminist International Assistance Policy Implementation Plan includes initiatives to renew the strategic planning process to improve integration and coherence. This renewal process involves staged piloting of new strategic planning tools. The Program will apply these renewed strategic planning processes when called upon as part of this staged process.

Recommendation 5: Global Affairs Canada should pursue a solution for the management and sharing documents between Missions and headquarters.

Management Response 5: Agreed. Information Management (IM) Services will continue the development of an international, departmental IM system. A GCdocs technical pilot will run from 2018-04-01 to 2018-09-01.Information Services IM Support will use the IM Committees and the Information Management Improvement Plan to survey employees at missions and Canada to identify specific  challenges, and will create ‘workarounds’ in unique situations to reduce workload and meet requirements for good IM. IM teams will develop a communication plan to explain the effective use of collaboration tools.

Appendix 1 – Management Response to the Recommendations

The Vietnam Development Program would like to thank the Development Evaluation Division (PRA) of Global Affairs Canada, responsible for the Vietnam Country Program Evaluation 2009-16, for allowing the Program to participate in the evaluation process. We are satisfied with the findings of the evaluation. We are pleased with the recognition of the high degree of alignment with the official development needs and priorities of Vietnam given the importance of country ownership for development results. We understand the need to focus even more on women, ethnic minorities and rural poor, in future planning in Vietnam.

Appendix 2 – Limitations and Mitigation Strategies

Evaluations face constraints that may have implications for the validity and reliability of the evaluation’s findings, conclusions, and recommendations. The limitations encountered along with the impact experienced and the mitigation strategies employed to ensure decision makers have confidence in the evaluation findings and recommendations are detailed below.

Appendix 3 – Program Evaluation Logic Model