Summary of the report of the formative evaluation of the Volunteer Cooperation Program (2015–2020)

This evaluation, done midway through the five-year program, helps assess the overall progress toward achieving results. It provides the department and the volunteer cooperation agencies (VCAs) with the information they need to adjust the program, its implementation and plan future programs.

Objectives

The specific objectives of the evaluation are to:

• evaluate the effectiveness, relevance and sustainability of results;
• evaluate the efficiency of the Volunteer Cooperation Program (VCP); and
• provide findings, conclusions, recommendations and lessons to inform the implementation of the current program and future program design.

Development context

Major progress has been made in documenting the roles played by volunteers in sustainable development, and, since 2011, in integrating volunteerism into key global development processes. This includes the designation of United Nations Volunteers as the focal point within the United Nations, development of a plan of action to integrate volunteering in peace and development, and inclusion of volunteering in the Sustainable Development Goals. The work of umbrella groupings of international VCAs, such as the International Forum on Volunteering in Development and the department’s Civil Society Partnership Policy, is also important.

Intervention

VCP 2015–2020 is the current five-year-cycle support to volunteer-sending projects implemented by Canadian VCAs in collaboration with their developing country partners (DCPs). The program started on April 20, 2015, and is scheduled to end on March 31, 2020.

The department provides financial support to 12 volunteer-sending projects being implemented by 15 Canadian VCAs (three consortiums) with some 700 DCPs in 42 countries. The total value of the VCP is $460 million of which VCA contributions are approximately $160 million, mostly the value of volunteer time. The department is contributing up to $300 million to VCAs to allow them to recruit skilled, qualified Canadian volunteers and place them within DCPs to build their capacity to achieve sustainable development results; and also to conduct public engagement activities in Canada. The VCP also supports the placement of southern volunteers to Canada, lateral placements of southern volunteers and e-volunteers.

The themes of the VCA projects are varied and include:

• economic/social empowerment
• capacity building
• institutional strengthening
• protection of human rights
• technical skills

They also include three crosscutting themes:

• environmental sustainability
• gender equality
• governance

Intervention logic

At the ultimate outcome level, the VCP 2015–2020 aims to improve the economic and social well-being of poor and marginalized communities in developing countries. The desired intermediate outcomes are:

• increased capacity of DCPs (civil society or private-sector organizations, as well as local, municipal, regional and national authorities in developing countries) to deliver sustainable development results, in response to local needs, by making use of the skills and expertise of qualified Canadian volunteers; and
• enhanced participation of Canadians in Canada’s sustainable development efforts (so as to foster a better understanding of development issues).

The ultimate and intermediate outcomes were included in the VCP call for proposals and are thus the same for all projects funded under the program. The immediate outcomes and other elements of the logic models were designed by the individual project proponents and are unique to each project.

Stakeholders

The key Canadian and developing country stakeholders, in addition to the department as the donor organization, are the VCAs, which act as executing agencies or implementing organizations; Canadian volunteers; DCPs; and Canadian men and women in general.

Approach and methodology

The team used an approach to maximize the practical value of the evaluation to stakeholders. Quantitative and qualitative data was collected from January to July 2018, using mixed methods, which included document review, four-country studies using key informant interviews (KIIs) and focus group discussions (FGDs); desk-based KIIs and FGDs; and two electronic surveys using SurveyMonkey.

A sampling methodology used helped identify an optimal mix of countries for field studies to ensure exposure to all VCA operations and areas of programming and an array of program stakeholders for the desk-based reviews.

Ten-day field studies were undertaken in Ghana, Honduras, Senegal and Peru between April and June 2018. Contact was made with the department personnel, representatives of each in-country VCA, in-field volunteers, managers/directors of DCPs, beneficiaries and community leaders. Two e-surveys were conducted: one with 3,469 volunteers and the other with 759 DCPs. The volunteer survey had a 33.1% response rate, while the DCP survey had a 51.1% response rate.

The team used the qualitative software platform Dedoose to help capture KIIs and FGDs and document insights, which enabled the team to sort by question and by respondent type. For the e-surveys, the online data platform SurveyMonkey automatically generates tables and graphs; additional statistical manipulations were undertaken to disaggregate responses by gender, VCA and volunteer type.

There were no material limitations to the conduct of the evaluation.

Key findings*

Improvements among beneficiaries:

82%—DCPs reporting “considerable” (46%) or “modest” (36%) beneficial change among people/communities identified as beneficiaries

59%— In-field volunteers reporting “considerable” (23%) or “modest” (36%) beneficial change among people/communities identified as beneficiaries

81%—DCPs attributing these beneficial changes to VCA supports (including volunteers) specifically and to a “major” (41%) or “moderate” (40%) extent

Improvements in partner capacity/performance:

86%—DCPs reporting “considerable” (50%) or “modest” (36%) improvement in the way their organizations function (e.g. in relation to program/service delivery, stakeholder engagement, gender equality (GE) policies and practices, governance)

82%—DCPs reporting specific improvements in GE policies and practices to a “major” (48%) or “moderate” (34%) extent; the area of improvement with the highest rating

62%— In-field volunteers reporting “considerable” (22%) or “modest” (40%) improvement in the way their organizations function 

69%— In-field volunteers reporting specific improvements in GE policies and practices to a “major” (30%) or “moderate” (39%) extent; the area of improvement with the second-highest rating

88%—DCPs reporting that volunteer contributions have contributed to perceived organizational improvements to a “major” (43%) or “moderate” (45%) extent

Extent of continuing involvement of returned volunteers in international development activities:

84%—North-south volunteers reporting that they are more involved in public awareness raising as a result of being a volunteer in a program like VCP (combining “minor,” “moderate” and “major” extent)

On sustainability:

84%—DCPs trace the attention they pay to sustainability to their partnership with their VCA to a “major” (40%) or “moderate” (44%) extent

59%—DCPs reporting it “likely” that they could continue the sustainability gains already made (25% report the likelihood at fifty-fifty); volunteer perceptions are similar

Partner and volunteer perceptions on use of innovative practices:

86%—DCPs saying that their VCA partnership was assisting them to innovate with new ideas and approaches to a “major” (39%) or “moderate” (47%) extent

76%— In-field and returned volunteers saying that the partnership was innovating with new ideas and approaches to a “major” (32%) or “moderate” (44%) extent

Conclusions*

The VCP, long considered a flagship of Canada’s international cooperation effort, has adjusted well to its strategic orientation and grown in size and complexity. While opportunities for improvement are evident, achievements have been considerable and widely appreciated by stakeholders.

Effectiveness

The extent of the progress reasonably matches program expectations for the end of the third year. Overall, the program story is rich though hampered somewhat by inconsistencies in reporting. At the beneficiary level, in particular, the story is still largely anecdotal.

Efficiency

To date, the VCP is keeping costs to a reasonable level. A broad range of cost-saving measures and strategies has forced some choices that have at different times helped (e.g. fostered synergy) or hindered (e.g. stretched volunteer resources too thin) the delivery of the VCP.

Relevance

Overall, VCA activities do align well with local development priorities and DCP needs. As the VCP engages with a broader range of actors and adopts programming approaches that embrace whole sectors or systems (not just individual partners), there is pressure on VCAs to be nimble in the way that they gauge relevance, to be less classically linear and more complex and adaptive in their management approach. In the end, the process of discerning (continuing) relevance comes down to having probing yet not overbearing assessment and planning tools along with the presence of skilled, personable field office personnel to use them.

Sustainability

There are examples across the programming sectors of projects and/or organizational processes either showing longevity or strong potential for such. Understandably, though, it is not a completely clean picture. Discontinuities between volunteers coming and going or between volunteers and local personnel do hamper sustainability. At times, DCPs lose sight of the idea that the VCP is to be a catalyst rather than simply a supplier of technical assistance to address gaps within the organization.

Coordination

The higher the level of coordination, the greater the array of opportunities for exploring complementarities across the suite of Canadian programming. However, coordination tasks are also seen to take a significant level of effort, especially regarding programming for major Canadian public engagement events. Differences in sizes of VCAs mean they need a varying amount of time for coordination activities and to be flexible.

Engaging Canadians

VCAs are aware that the scope and depth of their work in developing countries is highly dependent on the extent to which they capture imaginations and support at home. Returning volunteers, empowered by their experience, long or short term, can do a lot to ignite further interest and to free up additional human and financial resources for development. While diaspora communities have been targeted as a source of volunteers from Canada, indigenous peoples as a group have not, thus far.

Innovation

There is an abundance of innovative practice across the scope of the VCP. It manifests in programming approaches and in the tools and techniques used to deliver capacity building and Canadian engagement activities.

Gender equality

Technical support provided by volunteers, especially gender advisers, is essential to the shift in GE awareness and capacity building across the DCPs and beneficiary groups. In many cases, the emphasis on GE has resulted in no tolerance to the absence or non-participation of women and girls in program activities. Furthermore, men have been making progress as champions of GE in their own settings.

Environmental sustainability

The department has appropriately integrated environmental sustainability considerations at multiple levels, from screening proposals, to contracting, to the inclusion of management and reporting requirements. All projects/VCAs are compliant with environmental sustainability requirements of the VCP, though a few require adjustments to their trajectory to ensure yet more favourable outcomes in this respect.

Governance

Overall, the governance crosscutting theme is well integrated and is contributing to more sustainable results for poverty reduction in developing countries. Transparency and accountability are actively promoted by the department and the VCAs, as is GE and social inclusiveness. The VCP is rich in terms of human resources (i.e. VCA and DCP volunteers and staff); however, bilateral projects have additional financial and infrastructure resources which, if brought together, could lead to greater results by 2020.

Key recommendations*

1. That VCAs continue to integrate sustainability planning into their cycles of performance assessment, planning and project design with partners, paying more attention to potential alliances, to networking opportunities, to prospects for developing revenue streams, to proving business cases and to the formulation of exit strategies and system-level learning.
2. That VCAs enhance their engagement with other donor initiatives (Canadian or otherwise), capturing potential for complementarities and added value. Specifically, VCAs should seek opportunities where volunteers can add sectoral expertise and/or advice on technical matters, and/or where bilateral or other-funded projects can lend attributes to VCA-supported activities. This may involve reaching out to the embassies, along with other development partners, for information and assistance, as well as tapping into the expertise of the department at home.
3. That VCAs refine knowledge transfer among themselves, particularly in those countries without functional coordination mechanisms. This should occur around programmatic issues to build upon the successes of VCP volunteer interventions and to learn from the not-so-successful ones; operational matters, such as volunteer management, sustainability planning, innovation, remuneration, health, emergency preparedness, safety and security; and sharing and building capacity across the crosscutting themes through coordinated training.
4. That the department continue to support VCAs by earmarking support for joint public engagement events on behalf of the VCP and by encouraging coordination of the same.
5. That VCAs assess their existing activities in engaging Canadians to see if they are sufficiently robust to access and motivate target publics in provinces and territories across Canada and to engage return volunteers as supporters of program activities (e.g. in carrying out public awareness activities and/or in providing advice to volunteers starting on their mandates).
6. That VCAs, supported by the department, collectively develop an evidence-informed messaging campaign aimed at helping Canadians discern good volunteer practises in the service of international development and inviting Canadians to consider supporting VCA activities.
7. That the department and the VCAs put in place measures to increase recruitment of indigenous peoples (First Nations, Inuit and Métis) as volunteers, matching skills and experience to partner settings including, but not limited to, those that are specific to indigenous populations. In the same vein, it would be advisable to increase reciprocal south-north placements for indigenous peoples to come to Canada.
8. That VCAs identify more explicitly the aspects of their work programs that constitute innovative practice and attempt to isolate the tracking of those innovative practices for the specific purpose of learning, sharing and adaptation.
9. That VCAs collaborate more in the design and implementation of pre-posting and in-country, GE and diversity training including gender-based analysis plus (GBA+), which is available online.
10. That VCAs be proactive in showcasing, both in Canada and within the target countries, examples of volunteer initiatives directly related to raising environmental awareness and eliminating or mitigating environmental harms, including actions related to climate change and adaptation.

Key lessons*

• Encourage coordination, connectivity and networking. Participation and inclusion allow the program to take full advantage of learning processes and collective brainpower to deliver the best service with the wisest use of resources.
• Explore the potential for increased engagement of specific communities of learning, including indigenous peoples, rural Canadian farmers and the diaspora, to expand the potential for reciprocal relationships and contribute to international development practices.

*The findings, conclusions, recommendations and lessons presented above are those of the consultant and do not necessarily reflect the views of the department or the Government of Canada. Global Affairs Canada does not guarantee the accuracy of the information provided in this report.

Management response to recommendations

The department response:

1. The department recognizes the importance of integrating sustainable development planning into all project cycles in collaboration with partners.
2. The department supports engagement with other donor initiatives only if this engagement achieves the expected results of the volunteer cooperation project. Such engagement with other initiatives should avoid duplication of human and financial resources and should only be considered if there is real complementarity.
3. The department recognizes that the added value of establishing a coordination mechanism between VCAs by the end of the program will have to be determined on a case-by-case basis.
4. The department recognizes the importance of supporting joint public engagement events on behalf of the VCP.
5. The department recognizes the importance of evaluating Canadian public engagement activities to see if they are sufficiently developed to reach target audiences across the country and to engage return volunteers in program activities.
6. The department recognizes the importance of helping Canadians identify good volunteer practices in international development and inviting them to support the activities of VCAs. The department also recognizes that the VCAs are already doing this work.
7. The department will continue to encourage VCAs, where appropriate, to increase the recruitment of indigenous peoples as volunteers and reciprocal south-north placements and will collect data on this subject from the next annual reports and the final project report of the VCAs.
8. The department recognizes the importance of identifying more explicitly the aspects of the VCP that constitute an innovative practice.
9. The department recognizes the importance of greater collaboration in the design and implementation of training before and during assignments in countries in support of diversity and inclusion.
10. The department recognizes the importance of being proactive in presenting examples of volunteer initiatives directly related to environmental awareness and the elimination or mitigation of environmental harm.

Response from VCAs:

The VCAs took note of the consultant’s conclusions and recommendations and shared them within their organizations for consideration in the current iteration of the program. 

Language: The final report is available only in English. If you would like a copy of this report, please contact info@international.gc.ca.