Evaluation of the North American Platform Program (NAPP) Partnership Summary Report
Why is it important?
Following consultations with senior management in the Americas Branch (NGM), the NAPP was identified for evaluation in FY2016-2017. It was included in the Global Affairs Canada (GAC) Five Year Evaluation Plan, which was approved by the Deputy Head and submitted to the Treasury Board Secretariat. The evaluation was led by the Diplomacy, Trade and Corporate Evaluation Division (PRE) of the Evaluation and Results Bureau (PRD). The NAPP previously underwent an evaluation in FY2011-2012.
The NAPP originated from the Enhanced Representation Initiative (ERI), established in 2003 for a five-year term. The NAPP has been renewed two times (2013-2016, and 2016-2019). There are six Government of Canada departments and agency Partners (including GAC) who make up the NAPP partnership: Two Partners support advocacy programming: Agriculture and Agri-Food Canada (AAFC) and Atlantic Canada Opportunities Agency (ACOA), and three Partners support International Business Development (IBD) programming: Canada Economic Development for Quebec Regions (CED-Q), Export Development Canada (EDC), and Innovation, Science and Economic Development Canada (ISED).
What did the evaluation assess?
- The NAPP’s capacity to promote international business development, advance Canadian interests, and foster a coordinated and integrated approach in the U.S. and Mexico.
- The NAPP’s governance and partnership structure
- The use of performance information to inform improvements.
What were the evaluation questions asked?
- To what extent has the NAPP contributed to:
- Fostering a coordinated and integrated approach,
- Advancing Canadian interests through advocacy efforts, and
- Promoting International Business Development?
- Is the Temporary Duty and Training Program achieving its objectives?
- To what extent are the recent changes to the NAPP driving program improvement?
- How well is the current governance structure supporting the achievement of results?
- How does the NAPP track its performance? To what degree is performance information enabling decision-making?
- Data Sources:
- Key informant interviews (n=49)
- TRIO 2 data mining
- Strategia data mining
- File and document review
- Evaluation Scope: 2013/14 – 2015/16
- Total funding for 3 years: $12.6 million
- Evaluation Completed: June 2017
What did the evaluation find?
Performance-related findings
- NAPP is fostering a coordinated and integrated approach among the NAPP Partners through coordination and collaboration, leading to better quality initiatives and services for clients.
- There are opportunities to increase collaboration within the NAPP through partnerships with additional organizations such as Natural Resources Canada and Environment and Climate Change Canada.
- NAPP is contributing to the advancement of Canadian interests in North America by reaching and engaging target audiences, raising awareness and support, and through the achievement of policy changes.
- NAPP advocacy priorities are not well aligned with the priorities for Canada in Mexico creating challenges for the advancement of Canadian interests in Mexico.
- NAPP is promoting International Business Development by providing a significant source of program funding to TCS officers leading to Opportunities Pursued and Economic Outcomes Facilitated.
- The Temporary Duty Program is deeply valued by all stakeholder groups. It fosters understanding and collaboration, enhances knowledge, skills, and networks, and increases the TCS’ and FPDS’ capacity to deliver projects in North America.
- The LES Professional Development and Training programs are achieving its objectives and is valued by missions and NAPP Partners.
- The NAPP is a well-designed and well-managed program with effective governance structures that support achievement of results through sound project planning and implementation.
- Performance measurement systems exist for two of the NAPP’s objectives but not for the objective of fostering an integrated approach, leaving gaps in knowledge of the totality of results achieved.
- Significant efforts have been made in recent years to develop clear and measurable indicators for advocacy activities, however they are not being used by NNB to inform or support decisions on which initiatives to fund and challenges remain in capturing the actual results of advocacy initiatives in Strategia.
Recommendations and Action Plans
| Our Recommendations | Program Response and Action Plan |
|---|---|
| Recommendation 1: The NAPP should develop a concrete strategy for seeking additional Partners to join the NAPP. | The NAPP Coordination Unit regularly engages in outreach with potential Partners aimed at securing additional membership. The most recent concerted effort, took place in the summer of 2016, and involved all levels of Global Affairs Canada’s management team (e.g. Director General of the North America Advocacy and Commercial Programs Bureau; Assistant Deputy Minister for the Americas; and Associate Deputy Minister of Foreign Affairs) coming together to solicit the interest of their counterparts in non-Partner departments/agencies. As a direct result, it is anticipated that Environment and Climate Change Canada will join the NAPP in FY17/18. Management acknowledges that while the efforts during the evaluation period were concrete and focussed they were not based on a formal strategy and appreciating the value of a formalized plan, management commits to the development and launch of a concrete strategy for seeking additional Partners to join the NAPP, by March of 2018. |
| Recommendation 2: The NAPP should clearly define which Canadian advocacy priorities in Mexico it supports. | Management has reviewed NAPP strategic priorities multiple times since the incorporation of Mexico into the North America geographic (from Latin America). The decision to add a fourth NAPP advocacy strategic priority of Public Diplomacy (i.e. Culture, Values and Branding) in FY16/17 was, in part, intended to increase flexibility for the Embassy of Canada in Mexico (MXICO) and to accommodate an increased percentage of their traditional advocacy programming. That being said, the NAPP is neither designed nor intended to support all Canadian priorities in the U.S. & Mexico. Rather, it is designed to focus on the core advocacy objective mutually shared by all NAPP partner departments/agencies, of targeted advocacy in support/defence of maintained or expanded market access for Canada in North America. There is no intention to further diffuse NAPP Advocacy priorities away from that primary objective, by expanding it to cover Governance and Democracy, Capacity-Building, Development, etc. Other programs and funding sources within GAC already exist to address those Canadian priorities in Mexico. Management commits to formally clarifying and documenting how NAPP advocacy strategic priorities support/align to the aforementioned core Canadian advocacy priority in Mexico. |
| Recommendation 3: The Program should develop a performance measurement approach for the Program’s objective of fostering a coordinated and integrated approach, as well as for measuring the contributory effect of Partner’s use of retained funding towards the effectiveness of the NAPP. | Management commits to the development and implementation of a performance measurement framework for the Program’s objective of fostering a coordinated and integrated approach, as well as for measuring the contributory effect of Partner’s use of retained funding towards the effectiveness of the NAPP, by March of 2019. |
| Recommendation 4: The NAPP, in conjunction with other divisions that oversee advocacy activities, should continue developing and refining advocacy indicators in order to drive program decision-making. | Management, in its continuous effort to improve the performance measurement frameworks relating to programs under its purview, commits to the ongoing development and refinement of advocacy-related indicators, in collaboration with the relevant departmental stakeholders. |
