Evaluation of the Global Commerce Support Program - Final Report

Foreign Affairs, International Trade and Development Canada
Office of the Inspector General
Evaluation Division

February 2015

Abbreviations, Acronyms and Symbols

Acknowledgements

The evaluation team would like to extend its appreciation to the many individuals who volunteered to share their experiences with and thoughts on the Global Commerce Support Program (GCSP). These individuals included, but are not limited to: representatives from the International Business Development and Innovation Bureau (BFM), Department of Foreign Affairs, International Trade and Development (DFATD) and its sub-divisions; representatives from other federal government departments (OGDs); and, representatives from various municipalities, industry and trade associations, commercial enterprises, and Canadian academic and research institutes corresponding to the targets groups served by the GCSP’s three sub-programs. The evaluation team would further like to acknowledge the contributions of the Evaluation Advisory Committee (EAC) which guided the evaluation throughout all its phases.

Executive Summary

Introduction

The evaluation of the Global Commerce Support Program (GCSP) was conducted by the Evaluation Division (ZIE), Office of Audit, Evaluation and Inspection (ZID), Department of Foreign Affairs, Trade and Development (DFATD), in compliance with a Treasury Board (TB) requirement to report to the same on the relevance and performance of the said program. The target audience for the evaluation is DFATD’s senior management, the Assistant Deputy Minister (ADM) of the International Business Development, Investment and Innovation Branch (BFM), the Director General (DG) of the Global Business Opportunities Bureau (BBD), the DG of the Invest in Canada Bureau (BID), the divisions responsible for administering and monitoring the performance of the GCSP, and the general public.

Background

In Fiscal Year (FY) 2007-08, DFATD, then the Department of Foreign Affairs and International Trade (DFAIT), launched an initiative to amalgamate three pre-existing programs – the Community Investment Support Program (CISP), the Program for Export Market Development – Associations (PEMD-A), and the Going Global Science and Technology Program (Going Global) – under the umbrella of the GCSP. These programs following amalgamation became: Invest Canada – Community Initiatives (ICCI); Global Opportunities for Associations (GOA); and Going Global – Innovation for Researchers (GGI). This initiative was undertaken to better align programming with the Government of Canada’s (GoC) Global Commerce Strategy (GCS) and to improve efficiency and effectiveness of program delivery. Administration of the new program was to be enhanced through the development and implementation of common project processes with the support of a Common Administrative Unit (CAU).

The overall objective of the GCSP is to build stronger and more competitive Canadian capacity to effectively compete in the global economy by providing targeted support to: Canadian municipalities to promote inward Foreign Direct Investment (FDI); to national trade or industry associations undertaking international business development (IBD) activities; and to Canadian researchers in the academic and commercial private sectors in their efforts to forge collaborative science and technology (S&T) and research and development (R&D) projects with their foreign counterparts. How the three sub-programs of the GCSP achieve these objectives is briefly described below:

• ICCI: This sub-program provides financial assistance to Canadian municipalities to: undertake research; formulate FDI strategic plans; develop promotion and marketing tools, including websites; conduct training in FDI; and to participate in direct meetings with prospective foreign investors.
  
  
• GOA: This sub-program provides financial assistance to national trade or industry associations to: undertake market research; develop promotional and marketing tools; and participate in trade shows, networking functions, meetings and market visits.
  
  
• GGI: This sub-program provides financial assistance to Canadian researchers in both the academic and commercial private sectors to defray the costs of forging R&D partnerships with their foreign counterparts. More specifically, GGI contributes to the costs of travel, accommodation and legal services involved in the formation of these partnerships.

The GCSP is supported by a budget of around Canadian (C) $7 million  per year, of which $6 million is allocated to Grants and Contribution (G&C) funding (Vote 10) with the balance used to cover the costs of salaries and operations and maintenance (O&M). Notional annual allocations of Vote 10 resources to the three sub-programs of the GCSP are as follows: $2.9 million (ICCI), $2.2 million (GOA), and $0.7 million (GGI).

Key Findings

Relevance

Evaluation findings suggest that Canadian communities, Small and Medium Size Enterprises (SMEs), and researchers all require some degree of government assistance in the development of their capacity to attract FDI, participate in IBD, and advance R&D objectives through partnerships with foreign counterparts. Addressing the needs of these different populations has been identified by both the Government of Canada (GoC) and DFATD as priorities and each of the three sub-programs of GCSP are, in their own ways, designed to address these needs. Evaluation evidence further suggests that the sub-programs of the GCSP do in fact address these needs in a targeted manner, thereby contributing to the realization of the objectives of the GoC and DFATD in the domains of FDI promotion, IBD, and international R&D.

Each of the sub-programs of the GCSP fills to varying degrees a funding gap. With regard to ICCI, evaluation evidence suggests that though institutional capacity development at the community level may seem an odd objective for a federal department with an international mandate, the suite of activities supported by ICCI is unique, thereby filling a funding gap in the area of FDI promotion. GOA, to the extent that it focuses on national industry and trade associations, is also unique and by extension fills a funding gap in the area of IBD. GGI also fills a funding gap, but the gap it fills is extremely narrow and fragile, meaning that minor modifications to the eligibility criteria of other federal programs which support R&D Partnership Development Activities (PDAs) could render this sub-program redundant.

Achievement of Expected Outcomes

All three sub-programs of the GCSP have succeeded in achieving their expected results and have, in their own ways, contributed to Canadian economic growth and prosperity. ICCI investments in communities across the country have increased their capacity to retain and attract FDI, which has resulted in new FDI estimated in the tens of millions of dollars and thousands of new jobs created. GOA has increased Canadian SME participation in IBD, which in turn has translated into estimated tens of millions of dollars in new sales and contracts, thereby adding value to the Canadian economy. Though the monetary value of GGI’s contribution to the Canadian economy is difficult to determine, it too has had considerable success in achieving its objectives, which is to forge new and sustainable S&T and R&D partnerships which may yield sizable returns on investment in the future. As such, all three sub-programs of the GCSP generate value in excess of costs.

Efficiency and Economy

Program delivery has been supported by a highly developed and, for the most part, efficient administrative infrastructure. The current governance structure composed of the DG Steering Committee, the Director level committee, and the Deputy Directors’ Working Group, provides appropriate strategic direction and operational oversight to the program, evidenced by their timely responses to ongoing strategic and operational challenges, including those raised in past evaluation and audit reports. Further, roles, responsibilities and accountabilities are clearly defined, documented and, for the most part, accepted by all involved in the administration of the program.

Supporting the work of these governance bodies is the CAU which, in its capacity as a program secretariat, schedules meetings and prepares the necessary documentation to support decision-making while assiduously maintaining records of those decisions. In addition to the foregoing, the CAU is charged with drafting all key GCSP level program documents, tracking and recording performance data, ensuring all contribution agreements comply with the program’s Terms and Conditions (T&Cs) and for monitoring and updating directors and senior management on program finances, all tasks which it has performed effectively. More specifically, the CAU has played a vital role in furnishing senior management with timely and accurate information on the status of program finances, including the risk of lapses, which informs decisions on de-commitments and the reallocation of resources between the sub-programs. This has contributed to an overall diminution in the quantum of lapses to around 10% per annum, thus achieving one of the key objectives of amalgamation.

Most of these lapsed funds may be attributed to the GOA sub-program, which continues to register high year end lapses in excess of 28% despite the adoption of a range of measures to reduce lapses. The reasons for these large lapses are varied but the most common reasons cited were the timing of the launch dates for proposals that have varied from year to year, delays in the release of adjudication results that have reportedly frustrated sub-program delivery, and inflated cost projections from project proponents that contributed to significant variances between planned verses actual expenditures. Addressing these issues will be essential if GOA is to maintain its current notional allocation.

Over the evaluation reference period (2010-11 to 2012-13) program delivery for ICCI and GOA was facilitated by an on-line administrative system supported by the Virtual Trade Commissioner (VTC), a web-based electronic platform that allowed registered clients to submit applications for funding on-line. However, the platform was adapted from another program and built on an older Information technology (IT) architecture that was unstable, resulting in in user challenges for both program administrators and program beneficiaries alike. The VTC was de-commissioned in 2013 and efforts are ongoing to replace it with a more user friendly system. The evaluation noted some gaps and discrepancies in financial and program data housed in the program’s varied data repositories. Though these gaps and discrepancies were relatively minor, they could be addressed with a more harmonized approach to data recording and more frequent updates on the part of the sub-programs. On balance, the Information Technology (IT) infrastructure in place and management of information was deemed adequate for program administration and accountability purposes.

Human resources dedicated to the program are adequate, but for the CAU which has recently lost a financial officer and has assumed responsibilities beyond what was originally envisaged for it, thus putting stress on the Unit. Further, although GOA training is formal and scheduled, other sub-program training is more informal and ad hoc. However, guidance materials and counsel provided by senior management, the CAU and the Centre of Expertise for Grants and Contributions (SMFC) appears to be sufficient to support effective program delivery. Although sector officers charged with administering the GOA conveyed discomfort with the provision of advice on such matters as eligible expenses and in reviewing expense claims, they do have the benefit of GOA’s Policy team who seek the counsel of the CAU, IT, SMFC and JUS, as required.

In the area of performance monitoring and results reporting, program management has demonstrated exceptional due diligence. This is evidenced by its development of a performance measurement framework and reporting templates for beneficiaries, the systematic tracking and recording of this performance information, the administration of two beneficiary questionnaires to assess each sub-program’s contributions to GCSP’s expected outcomes, and the consolidation of this information in annual performance reports. This said, the current performance measurement framework is in need of refinement to better align activities with objectives and higher order expected outcomes. Further, indicators currently in use to track expected outputs and outcomes are too numerous, not complementary and, in many instances, of questionable relevance, all of which have undermined the reliability and utility of this performance information that has been collected and assembled at considerable expense.

In sum, the three sub-programs of the GCSP all produce results which contribute to Canadian economic growth and prosperity that can be calculated in the millions of dollars, far in excess of program costs. Though it is near impossible to determine whether amalgamation has resulted in cost savings of the magnitude originally expected, it would be hard to envisage that the program could be managed with significantly fewer resources than are currently allocated. What can be said is that amalgamation has contributed to increased efficiencies and economies in program delivery by allowing resources to be reallocated between the sub-programs in accordance with need. As such, the GCSP is, on balance, good value for money. This said there are potential areas to explore to strengthen the relevance and efficiency in program delivery which are taken up in the following section.  

Recommendations

Based on the foregoing analysis, the following recommendations are presented for consideration.

• Recommendation 1: That DFATD continue to address factors cited as probable contributors to GOA’s lapses, including factors relating to the sub-program’s planning cycle.
  
  
• Recommendation 2: That DFATD continue efforts to upgrade and refine its IT infrastructure to support both program delivery and administration.
  
  
• Recommendation 3: That DFATD review the HR needs of the CAU with the view to making such adjustments as may be needed to support and maintain the quality of services provided.
  
  
• Recommendation 4: That DFATD revise its current performance measurement framework and related indicators with the view to simplifying the same and improving the relevance and utility of these instruments to support decision making.

1. Introduction

The Evaluation Division (ZIE) at the Department of Foreign Affairs, Trade and Development (DFATD)^{Footnote 1} is housed within the Office of the Inspector General (ZID) and is mandated by the Treasury Board (TB) through its 2009 Policy on Evaluation to conduct evaluations of the relevance and performance of DFATD’s direct program spending, including Grant & Contribution programs. ZIE reports to the Departmental Evaluation Committee (DEC) which is chaired by three Deputy Ministers (DMs). The Evaluation of the Global Commerce Support Program (GCSP) was identified in the ZIE Evaluation Plan. The target audience for the evaluation is DFATDs senior management, the Assistant Deputy Minister (ADM) of the International Business Development, Investment and Innovation Branch (BFM), the Director General (DG) of the Global Business Opportunities Bureau (BBD), the DG of the Invest in Canada Bureau (BID), the divisions responsible for administering and monitoring the performance of each of the GCSP’s three pillars/sub-programs, and the general public.

1.1 Background

Advantage Canada,^{Footnote 2} the Government of Canada’s (GoC) 2006 Economic Plan to eliminate Canada’s net debt and reduce taxes, highlights the GoC’s unique and important role in fostering economic conditions that help Canadian businesses succeed and outlines key initiatives that seek to strengthen Canada’s competitive advantage in the global economy. To this end, the GoC approved the Global Commerce Strategy (GCS),^{Footnote 3} Canada’s international commercial policy framework in support of Advantage Canada.

The objective of the GCS was to enhance productivity and position Canada as a world leader in the highly competitive global economy through strengthening Canada’s investment environment and enhancing support for Canadian firms seeking to participate in International Business Development (IBD). The GCS identified mechanisms for using existing resources more effectively and identified new initiatives which, together, provided a seamless suite of services and programs for businesses. As a strategic framework for enhancing Canada’s competitive advantage, the GCS consisted of three (3) complementary and mutually supportive objectives, including:

• Making Canada a partner of choice for international business through aggressively pursuing two-way investment, particularly investment that connects priority Canadian sectors to foreign markets through global value chains;
• Negotiating improved access to global markets, capital, technologies and talent through  reducing trade and investment barriers and pursuing opportunity-generating agreements that cover trade in goods and services, air transport, investment, and science and technology (S&T) with Canada’s partners around the globe;
• Connecting Canadian business with expanding global opportunities through addressing competition from other governments which may threaten the ability of Canadian firms to take advantage of new business opportunities.

In designing and implementing the GCS, DFATD identified three (3) existing contribution programs as mechanisms that assist in making Canada a partner of choice for international business through the provision of financial assistance to Canadian businesses and organizations: the Community Investment Support Program (CISP), the Going Global Science and Technology Program (GG), and the Program for Export Market Development – Associations (PEMD-A).

1.1.1 Pre-existing Contribution Programs

The aforementioned programs and their mandates are discussed in greater detail below.

Community Investment Support Program (CISP)

CISP was based on the premise that many municipalities and regions in the country are in need of increased capacity building to attract, retain and expand foreign investment. This program was designed to support Canadian community efforts to achieve the ultimate outcome of creating jobs through attracting, retaining and expanding Foreign Direct Investment (FDI). CISP was also a key instrument at the domestic level to promote prosperity and employment, supporting projects through partnership with other levels of government and the private sector. Through CISP, communities were provided with assistance to aid in the development and delivery of localized investment attraction strategies, a critical element of a community’s ability to be innovative.

Going Global Science and Technology Program (Going Global)

The GG was designed to promote and enhance Canada’s international S&T efforts through the provision of financial support to Canadian researchers to facilitate the pursuit of collaborative research and development (R&D) opportunities with researchers in other markets. The GG sought to strengthen the ability of Canadian researchers to succeed in the global marketplace through enhancing the quality of their research activities and creating opportunities for collaboration, innovation and commercialization.

Program for Export Market Development – Associations (PEMD-A)

PEMD-A was initially designed as part of a larger suite of export development programs delivered by DFATD. The program sought to increase Canadian prosperity and competitiveness in the international marketplace through disbursing funds to trade and industry associations seeking to undertake export promotion activities on behalf of their members. Specifically, the program’s objective was to increase export sales of Canadian goods and services by sharing the costs of activities that associations normally could not or would not undertake alone, thereby reducing potential risks involved in IBD.

1.1.2 Amalgamation to the Global Commerce Support Program (GCSP)

A policy review was conducted to assess the alignment of the program objectives of the three aforementioned contribution programs with those of the GCS. Consequently, a strategic realignment of the program objectives as well as a reorganization of the administrative and management structure were deemed necessary to enhance the efficiency and effectiveness of the Department’s complement of IBD-promotion programs. Reductions in the funds allocated to PEMD-A and CISP following successive economic reviews, along with the inability to transfer additional funds to GG during its funding period, led the Department to question the value of maintaining separate administrative and management structures for these funding programs. The amalgamation also provided an opportunity to streamline administrative processes and standardize customer service practices in accordance with the recommendations of the 2006 “Report of the Independent Blue Ribbon Panel on Grant and Contribution Programs,” which sought to make programs more accessible to recipients and simplify accountability and reporting requirements.^{Footnote 4}

In FY 2007-08, DFATD launched an initiative to amalgamate CISP, GG and PEMD-A under a single management structure to address common challenges inherent in the management of G&C programs. Under GCSP, CISP became Invest Canada – Community Initiatives (ICCI), GG became Going Global – Innovation (GGI) and PEMD-A became Global Opportunities for Associations (GOA).^{Footnote 5} Essentially, the GCSP was created to improve alignment with the GCS, to improve the management of the three contribution programs to achieve cost-savings over the long-term and to improve consistency with the Treasury Board Secretariat (TBS) recommendation for horizontal coordination of program administration.

1.2 Program Objectives

The GCSP is a contribution program under the TBS’ Policy on Transfer Payments designed to enhance Canada’s productivity and to make Canada a partner of choice for international business. The GCSP’s overall objective is to build a stronger and more competitive Canadian capacity to compete in the global economy. The program is intended to build Canadian prosperity through increased Canadian participation in international commerce, focusing on three complementary pillars of investment, innovation and industry.^{Footnote 6} More specifically, GCSP objectives are expected to be achieved through the retention, attraction and expansion of FDI (ICCI), the enhancement of S&T innovation and commercialization opportunities (GGI), and increased involvement of Canadian businesses in IBD (GOA).

1.3 Program Beneficiaries

The GCSP’s three pillars (hereafter referred to as sub-programs) target different groups of eligible recipients in Canada, as illustrated in Table 1 (Eligible Recipients of GCSP Funds) below.

Table 1: Eligible Recipients of GCSP Funds

• ICCI
  • Canadian municipalities, communities, and non-profit organizations
  • locally-based organizations acting as a third party or intermediary (e.g. regional economic development organizations) on behalf of a number of eligible communities
• GOA
  • Canadian national trade or industry associations (or a regional association with a national perspective)
  • National horizontal/multi-sectoral organizations undertaking IBD activities for the benefit of member and non-member firms
• GGI
  • Canadian researchers from private companies, universities and non-government research centres

Government departments, agencies, crown corporations and other government organizations at the federal or provincial level are specifically excluded as eligible recipients. To allow greater flexibility and maximize the results achieved through funded projects, the GCSP allows for the further distribution of contributions to a third party. The conditions under which this further distribution can occur are outlined in the program’s Terms and Conditions (T&Cs). The T&Cs also set out the roles and responsibilities of recipients, including responsibilities related to maintaining appropriate financial systems, maintaining appropriate financial and administrative records, providing the department with access to their premises and records and providing the department with activity and financial reports.

1.4 Program Activities and Expected Results

Program Activities

Through its three sub-programs the GCSP supports projects involving a number of specific activities which are explained in greater detail in the Table 2 (GCSP Activity Types by Sub-Program) below.

Table 2: GCSP Activity Types by Sub-Program

• FDI Training
  • Provision of investment training to municipal employees or employees of eligible locally-based organizations.
  • Sub-Programs:
    • ICCI
• Strategic Planning and Analysis
  • Development and delivery of strategies for FDI attraction and retention by Canadian communities.
  • Sub-Programs:
    • ICCI
• Research
  • Conduct of SWOT (strengths, weaknesses, opportunities, threats) analyses, market research, investment profiles, and situational assessments by Canadian communities and national trade and industry associations.
  • Sub-Programs:
    • ICCI
    • GOA
• Development of Tools and Material
  • Development of tools and materials needed by communities and industry associations to conduct marketing and promotion.
  • Outputs include: advertisements, web sites, e-commerce tools, e-newsletters, databases and promotional displays.
  • Sub-Programs:
    • ICCI
    • GOA
• Networking and Promotion
  • Activities that allow communities, industry associations, and researchers to meet with investors and researchers in foreign markets, develop partnerships, pursue opportunities for collaboration and gather intelligence.
  • Sub-Programs:
    • ICCI
    • GOA
    • GGI

Expected Results

The outputs of the GCSP’s projects and activities are expected to lead to a set of immediate and intermediate outcomes and long-term results that are aligned with the strategic priorities of the Department. Immediate outcomes are expected to be achieved either immediately or within a few months of the completion of a project, whereas intermediate outcomes may be achieved many months after the completion of a project.

The GCSP’s immediate, intermediate and ultimate outcomes are identified below^{Footnote 7}:

Immediate Outcomes

• Communities have improved capacity to attract FDI
• Industry has improved capacity to conduct IBD
• Potential partners are more aware of Canadian investment opportunities
• Increased knowledge in communities of investment attraction opportunities
• Increased knowledge among Canadian researchers of S&T opportunities with key players in foreign countries
• Increased knowledge of international business opportunities among Canadian communities, businesses, and trade/industry associations

Intermediate Outcomes

• Receipt of foreign interest and investment projects
• Expansion of industry’s participation in IBD
• Initiation of S&T collaborative partnerships

Ultimate Outcomes

• FDI to Canada is retained and expanded
• Canadian share of foreign markets is increased
• Canadian commercialization opportunities are increased

1.5 Program Resources

Asdescribed in Table 3 (Budget Allocations for GCSP – Vote 1 and Vote 10), approximately $7 million has been allocated to the GCSP annually from 2010-11 to 2012-13. Of this, approximately $6 million per year has been allocated to G&C funding (Vote 10) for contributions to recipients while the remaining allocations were used to cover costs for salaries and Operations and Maintenance (O&M).

Vote 1 operating costs were not monitored from 2008-09 to 2010-11. However, the CAU began to track salaries and O&M in response to a recommendation in the 2011 Early Implementation Audit of the Global Commerce Support Program. Table 3 above shows that salary costs for 8.6 Full Time Equivalent (FTEs)^{Footnote 8} and O&M costs as a percentage of total program costs were 11.1% in 2010-11, 14.4% in 2011-12 and declined to approximately 13% in 2012-13 with the closure of Regional Offices (ROs).

Table 4 (Budget Information for GCSP) below describes G&Cs budget information for the GCSP from 2010-11 to 2012-13. A total of over $18 million was notionally allocated for GCSP program activities over the three reference years, of which 10% was lapsed on average. The program over-committed funds by an average of 21.4% per year, for a total of almost $22 million over the three FYs. Of these funding commitments, an average of 25.8% was de-committed per year for the GCSP overall.

Authority for reallocation of GCSP funds was delegated to the ADM for the International Business Development and Innovation Branch (BFM) and allows the ADM to respond to the evolving market environment and priorities of the GoC and to mitigate the risk of lapsing funds. This delegated authority permits flexibility within the amalgamated GCSP to reallocate a part of the annual budget allocations from one sub-program to another to meet recipient demands, as long as annual allocations for each component remain within specified ranges. Minimum and maximum allowable allocations for ICCI may range from $2.4 million to $3.6 million, $0.5 million to $1.5 million for GGI, and $1.7 million to $2.7 million for GOA.

Table 5 (Financial Information by Sub-Program) below describes the breakdown of funding allocations, commitments, expenditures and de-commitments by GCSP program component. Overall, the highest proportion of GCSP funds was allocated to ICCI programming at a total of approximately $9 million in notional funding allocations over 2010-11 to 2012-13, compared to almost $7 million for GOA and over $2 million for GGI. ICCI has over-spent an average of almost 3% of its annual budget each year over this reference period, lapsing funds only in 2012-13. GGI, however, lapsed funds by an average of about 5% per year. GOA has consistently lapsed funds every year at a rate of 28.4% per year on average over last three FYs.

GGI overcommitted the highest percentage beyond its notional allocated funds (an average 42.1% per year) while GOA overcommitted the lowest (13.5% on average per year) from 2010-11 to 2012-13. These two GCSP components were unable to spend these funds fully and each de-committed an average of approximately one-third of its programming funds across the three reference years. ICCI, however, was able to spend a higher proportion of its committed funds, de-committing only an average of 15.2% of its funding commitments annually. Overall, ICCI appears to fare better than GOA and GGI in meeting its funding commitments.

1.6 Governance

Overall responsibility and accountability for the GCSP rests with the Minister for International Trade. The ADM of the International Business Development and Innovation Branch (BFM) is accountable to the Deputy Minister of International Trade (DMT) for the overall operations and management of the GCSP in accordance with GoC policies and priorities and departmental mandates. Each sub-program of the GCSP is implemented by bureaus and divisions within BFM, as outlined in Table 6 (Implementing Divisions of the GCSP), below.

Table 6: Implementing Divisions of the GCSP

• ICCI
  • Invest in Canada Bureau (BID)
    • Investor Services Division (BIS)
• GGI
  • Invest in Canada Bureau (BID)
    • Innovation, Science and Technology Division (BBT)
• GOA
  • Global Business Opportunities Bureau (BBD)
    • Multi-Sectors Practices Division (BBI)
    • CleanTech, Infrastructure and Life Sciences Practices Division (BBM)
    • Aerospace, Automotive, Defense and ICT Industries Division (BBR)
    • International Education (BBY)

The divisions noted in the table above are responsible for program coordination and deliverables in accordance with annual priorities as well as the GCSP T&Cs. Program leads in each of these divisions are responsible for overseeing the application process for their respective program pillar, adjudicating proposals and monitoring the implementation of approved projects.

The advisory function is performed by the Centre of Expertise for Grants and Contributions (SMFC), which ensures that contribution agreements comply with the GoC’s 2008 Policy on Transfer Payments updated in 2012 and TBS guidelines, and by the Office of the Inspector General (ZID), who is responsible for audits and evaluations. SMFC is accountable to the Departmental Chief Financial Officer (CFO) while the Office of the Inspector General reports directly to the Deputy Minister of Foreign Affairs (USS). These accountabilities provide the GCSP with arms-length, objective advice and recommendations, as necessary.

Program delivery is overseen by a Director General Steering Committee, Directors Meetings and a Deputy Director Working Group as well as supported by the Grants and Contributions Administrative Unit (CAU) within the Trade Portfolio Strategy and Coordination Bureau (GPMC). These oversight and administrative mechanisms are described in greater detail in the sections below.

GCSP Steering Committee

The GCSP Steering Committee is responsible for ensuring that each sub-program of the GCSP is implemented in accordance with the priorities of the department and the respective bureau strategies. They are also accountable for tracking program performance and for ensuring that the GCSP is implemented in accordance with its T&Cs.

Membership on the GCSP Steering Committee comprise the Directors General of BID and BBD, who co-chair the committee, the Directors for each GCSP pillar (BIS, BBT and BBI), the Deputy Directors for each pillar, the GCSP Finance Manager from the CAU and the GCSP Project Manager who acts as the secretariat for the Committee.^{Footnote 9} Terms of Reference (TOR) have been developed that outline the roles and responsibilities for committee members.

As the GCSP’s mechanism for decision-making, the Steering Committee meets quarterly to discuss ongoing operational issues pertaining to the management of the GCSP as well as potential modifications to the program’s policies, procedures and processes. Based on its discussions, the GCSP Steering Committee formulates recommendations for consideration by BFM. The GCSP Steering Committee then meets with BFM to decide on annual priorities, performance targets and funding allocations for each sub-program.

Director Level Committee

Director Level meetings are convened monthly to share information on the status of GCSP and discuss operational and/or financial issues, to strategize on program operations and raise concerns and to discuss other pertinent topics of interest to be raised at the Steering Committee. This Directors Meeting is chaired by the GCSP Project/Financial Manager and membership consists of the Directors from each GCSP component, their respective Deputy Directors and other members of the GCSP team, as required.

Deputy Directors Working Group

The Deputy Directors responsible for each sub-program of the GCSP meet on an ad hoc but regular basis to discuss issues pertaining to the management of the GCSP, including administration of the application and approval process, financial management, challenges pertaining to individual projects and the achievement of results. This information is compiled and vetted by the Directors of BBI, BBI, BIS on a monthly basis for submission to the GCSP Steering Committee.

Grants and Contributions Common Administrative Unit (CAU)

The Branch’s Grants and Contributions Common Administrative Unit (CAU) is responsible for coordinating common functions and providing administrative support to all three pillars of the GCSP. Some of the support activities performed by the CAU include:

• overall financial administration, control and monitoring;
• drafting of key GCSP program documents, including the Performance Measurement Strategy;
• drafting of reports to Parliament, including as the Departmental Performance Report (DPR) and the Report on Plans and Priorities (RPP);
• preparation of legal agreements in compliance with the Program’s T&Cs; and
• coordination of recipient audits; and
• tracking of program performance and production of the GCSP annual report.

The CAU is supported by DFATD’s Centre of Expertise for Grants and Contributions (SMFC), which is mandated to provide support to G&C programs across the department in order to ensure compliance with the 2018 Policy on Transfer Payments updated in 2012.

1.7 Profiles of the Sub-programs of the GCSP

The sub-programs of the GCSP are distinguished by different program objectives, target recipients, eligibility criteria, application processes and project selection criteria. The characteristics of ICCI, GOA and GGI are each described in the following sections. Profiles of projects supported by each of the sub-programs are described in Appendix 4.

1.7.1 Invest Canada – Community Initiatives (ICCI)

The Invest Canada – Community Initiatives (ICCI) of the GCSP promotes the attraction, retention and expansion of FDI, which drives productivity growth and stimulates the creation of high-wage jobs. The ICCI’s programming strategies, priorities, and policy objectives are designed to promote and attract FDI in Canada through the implementation of proactive investment attraction strategies in priority sectors and key markets as well as the provision of aftercare services to key investors.

ICCI funds are allocated by calendar year. These funds support Canadian municipalities to facilitate the implementation of local initiatives that seek to build or maintain their investment readiness and improve their capacity to respond to foreign investment inquiries. Matching funds of up to 50% of eligible expenses are provided to successful applicants and contributions can range from $3,000 to $300,000. Total federal support cannot exceed 50% for those applicants with other federal sources of funding.

ICCI funds a range of projects depending on their position on the FDI continuum. Communities with little FDI capacity at the far end of this continuum frequently first apply to develop an FDI strategy through market and sector assessments. This could include developing an FDI Strategy and Action Plan, an inventory of current FDI and aftercare program or identifying best practices within their community. Communities further along this continuum that have a formal plan for FDI are then likely to apply to ICCI to help build a skilled workforce to contribute to their FDI attraction, expansion and retention efforts. Knowledge of existing FDI and FDI sectors in the region may then be deepened through, for example, the development of community profiles, sector fact sheets and investment tracking systems. Promotional material such as investment portals and online investment guides could then be developed to highlight the community’s capabilities to potential foreign investors. The final step in the FDI continuum for ICCI funding would be identifying the key investors and attracting them to the community through, for example, developing a lead generation program or preparing meetings with potential foreign investors.

Because communities vary in terms of their position along this FDI continuum, ICCI’s projects are likewise varied. ICCI applicants are encouraged to consider the size of their community or organization and the FDI projects already undertaken when considering the FDI activities they are submitting for funding. Recipients may apply to fund eligible FDI activities in subsequent years as their capacity improves. Examples of eligible expenses include, but are not limited to: research, studies, economic data collection; strategic FDI planning; website development; translation of documents and websites; design and layout of promotional materials or advertising; investment training (including travel costs to training events); and meeting room rentals (in Canada) for functions with potential investors. A regional approach to FDI is encouraged through joint initiatives between neighbouring communities and municipalities.

Application Process and Evaluation Criteria

Funding recipients are selected through an annual competitive application process in which preference is given to projects that target DFATD’s priority sectors and/or key foreign markets. ICCI’s planning cycle begins in mid-September with an invitation to would-be applicants to prepare their proposals and submit the same through the Virtual Trade Commissioner (VTC) web-based portal by a deadline usually pegged between the 20th and 25th of October. The submissions are reviewed by program staff (BIS) to determine compliance with mandatory requirements. Submissions which satisfy these mandatory requirements are subject to further scrutiny which may involve, where appropriate, consultations with other stakeholders, including the BIS Account Executive, OGDs and posts, both in-Canada (ROs) and abroad, resulting in a final list of candidates which are put before the ICCI Adjudication Committee, whose decision is final. This entire process starts in the last week of October and runs through to the end of the calendar year, with applicants being notified of the Committee’s decision in the first week of January the following year. Progress reports are received, beginning in April, and reviewed throughout the summer following which, based on the likelihood of completion, a decision is made to commit or de-commit funds, as the case may be. All projects for the year are expected to be completed by December 31 of the applicable year.

Applications are evaluated based on the quality of following criteria:

• Community’s FDI strategy and links to DFATD’s FDI strategy;
• Proposed project components, including activities, related FDI priority sectors and targeted key measures, and cost breakdown for all eligible expenses;
• Performance measures to capture anticipated short-term and intermediate outcomes resulting from the project;
• Capacity of community to complete the project, including project governance and sustainability of the project in the future;
• Work plan, including key dates and milestones towards project results and outputs; and
• Partnership commitments, including dollar value of partner contributions. 

Applicants are required to register with the VTC on the GoC’s Canadian Trade Commissioner Service (STC) website in order to access the ICCI online system and application form.

1.7.2 Global Opportunities for Associations (GOA)

GOA provides non-repayable contributions to national trade and industry associations (or regional trade associations with a national focus) to undertake new or expanded IBD promotion activities for the benefit of member and non-member firms.  This program helps reduce the risk associated with promoting IBD through sharing the cost of activities that these associations normally would not or could not undertake alone. Whereas the focus of its predecessor, PEMD-A, was on export market development and export sales, GOA incorporates a broader definition of IBD that allows for integrative trade activities beyond export promotion, including global value chain activities.

Three types of activities are funded through GOA: direct contacts, such as trade shows or networking events; marketing or promotional tools and materials such as website development or print materials; and research reports such as market intelligence. For GOA, applicants are encouraged to group project activities with similar objectives or shared results into packages,^{Footnote 10} for a maximum of five packages per application. A “package” represents an IBD project within an application and “activities” are the related and complementary activities that are required to complete the “package” or project. Applicants may propose up to a maximum of five activities per package.^{Footnote 11}

Funded activities are expected to lead to business relationships and marketing potential for Canadian industries. In general, eligible expenses include all those incurred expenditures that clearly relate to the proposed IBD activities, whether incurred in Canada or abroad, that are not deemed ineligible. Ineligible expenses include but are not limited to: associations’ ongoing or core activities, capital costs such as for office space and equipment, entertainment and hospitality costs including bar tabs, event sponsorship and membership fees and training costs.

Allowable annual contributions range from $20,000 to $150,000 per association, with a maximum of $450,000 payable to any organization over a three year period. Funds are disbursed on a cost-sharing basis where eligible expenses are shared up to a maximum of 50% between GOA and the approved applicant. The stacking limit is 75% and associations that have secured other sources of government funding for a package are required to declare these sources by expense item.

Eligibility Criteria

To be eligible for GOA, an association must be:

• A national association (or regional association with national perspective) whose objective is to promote sector-specific IBD for its members and industry at large;
• Incorporated and active for at least two years;
• Able to generate sufficient private sector revenues to fund the minimum 50% portion of project costs;
• Able to fund proposed activities in full, prior to re-imbursement;
• Not directly selling products that are sold by its association members; and
• Registered as a partner-client (industry association) with the VTC.

Ineligible applicants include: Chambers of Commerce and Trade Councils representing foreign geographic areas; associations with funding from other federal contribution programs with similar objectives to GOA and that support sector-specific IBD efforts; associations funded fully through municipal, provincial and other federal government resources, except where the association’s contribution for proposed activities comes from a different, private source such as membership dues. (See Section 1.3 for list of Program Beneficiaries.)

Application Process and Evaluation Criteria

For GOA, the planning cycle begins in late November or early December with an invitation to would-be applicants to prepare their proposals and submit the same through the VTC web-based portal by mid-January of the following year. During this phase (6 to 7 weeks) applicants can refine their proposals based on feedback received from the sector officer assigned to the file and relevant posts. Following the closing date for applications in mid-January, sector officers assess each package in the proposal against pre-established evaluation criteria. Scores are reviewed by peer groups to ensure consistency in the application of the aforementioned evaluation criterion, following which the proposals are submitted to a Steering Committee which is charged with making recommendations for approval (conditional or unconditional) to the inter-departmental adjudication Board, which convenes in and around mid-February, and which makes a final determination on proposals. Applicants are notified of the results in mid-March.

Applications are evaluated against on the following criteria:

• Provision of national perspective and level of benefit to respective Canadian industry sector;
• Eligibility of proposed activities and expenditures and degree of alignment between project’s proposed strategy to GOA objectives;
• Alignment to GCSP objectives;
• Capacity of association to complete proposed activities within timelines;
• Previous use and management of GOA/PEMD-A funds;
• Potential for project activities to contribute to IBD sectoral growth over the next three years;
• Level of funding requested relative to performance measurement;
• Support from posts abroad; and
• Completeness of application.

1.7.3 Going Global – Innovation for Researchers (GGI)

GGI was the innovation, S&T component of the GCSP that was designed to promote and enhance Canada’s international innovation efforts by supporting international R&D collaborative opportunities through the development of targeted relationships with key foreign partners. Through GGI’s partnership development activities (PDAs), Canadian researchers were expected to be better positioned to proceed with formal discussions that facilitate collaboration on R&D initiatives and better equipped to succeed in the global marketplace. In an effort to increase the likelihood of downstream commercialization, the GGI placed more emphasis on private sector participation and on projects that were more likely to lead to initiatives with strong market potential.

GGI contributed up to 75% of eligible project costs. The value of allowable GGI funding ranged from $5,000 to $75,000, with the maximum level of assistance from all federal, provincial and municipal government funds (the stacking limit) set at 75% of project costs. Eligible expenses included, but were not limited to: travel and related costs such as airfare, local transportation, accommodations or meals and incidentals; other related costs such as visa applications or immunization for project-related travel; and other non-research expenses such as meeting costs or legal expenses (pro-rated) to develop binding agreements for R&D partnerships. Ineligible expenses included, but were not limited to: actual R&D activities such as technology, equipment and product development or stipends for student researchers; commercial and business activities such as marketing (including market research or expansion activities; and other ineligible expenses such as overhead and salary costs or website development.

Eligibility Criteria

To be eligible, GGI recipients were required to be Canadian researchers from:

• Private Canadian small and medium sized enterprises;
• Universities in Canada; or
• Canadian non-government research centres.

Applicants were encouraged to form research consortiums and submit one application together. Preference was given to those applications submitted by a group of researchers from multiple eligible organizations.

Application Process and Evaluation Criteria

Unlike the other sub-programs of the GCSP, GGI receives applications throughout the FY, though applicants are advised to submit their proposals at least 8 weeks in advance of the anticipated start date to allow for processing. Also unlike the other sub-programs of the GCSP, GGI had no web-based portal to support the application process. Application forms were downloaded from the sub-program website, filled in manually, and then submitted to the program via mail, email or fax. Upon receipt, applications were subject to a pre-verification and verification check by the program administrator to determine if the application satisfied mandatory requirements, a process which on average took around two weeks. If the application following the aforementioned review satisfies all mandatory requirements, it is forwarded to a review committee composed of the officer at the RO from which the application originated, the relevant sector officer, geographic and innovation officers responsible for the country/countries the applicant intends to visit and an officer from the Innovation Science and Technology Division (BBT) at HQ for assessment – a process which likewise takes approximately two weeks. Proposals recommended for approval by the Committee, either in whole or part, are then submitted to the Director BBT for final determination. Applicants are notified within 6 weeks of submitting their application whether they have been successful and contribution agreements issued to the same within seven to eight weeks following receipt of the application.

Applications are evaluated against the following criteria:

• Potential to establish significant collaborative R&D initiatives with foreign partners;
• Demonstration that identified foreign partners are among the best in their field;
• Degree to which project objectives are aligned with DFATD S&T priorities;
• Potential for economic benefits to Canada
• Potential to engage other Canadian participants in the project;
• Coverage of international relationships and field/sector;
• Effectiveness of proposed approach to meet project objectives (e.g. work plan is well-defined, proposed activities are specific, logistics for meetings/functions are identified and pre-arranged); and
• Demonstration that sources of funding for subsequent international collaborative R&D research projects are being investigated.

2. Evaluation Goals & Objectives

In accordance with the TBS Policy on Evaluation, the evaluation assessed the relevance and performance of the current systems and practices in place to achieve expected outcomes of the GCSP. Through a systematic evidence-based data collection process, the evaluation reviewed the activities undertaken and results achieved, assess and made recommendations regarding their relevance and performance, and identified lessons learned and opportunities for future consideration. In addition, because the GCSP is an amalgamation of its forerunners – the CISP, Going Global and PEMD-A – the evaluation determined whether or not the amalgamation resulted in expected increases in efficiency and reductions in administrative costs, as well as better alignment with the GCS.

The objectives of the evaluation were to identify:

• the extent to which the GCSP addresses an ongoing need;
• the extent to which the GCSP is aligned with GoC priorities as well as the mandate and objectives of DFATD;
• the extent to which the GCSP has contributed to the achievement of its declared outcomes;
• whether the current systems of governance and resource allocation are optimal or whether there are more economical ways to achieve the same results;
• the extent to which the findings and recommendations from previous audits and evaluations have been addressed; and
• lessons learned and best practices to guide future programming.

3. Key Considerations

3.1 Global Market Action Plan and Related Strategies

As remarked in Section 1.1, the GCSP was created to improve the alignment of the three pre-existing G&C programs with Canada’s then-GCS. While the overall objective of Canada’s GCS was to build a stronger and more competitive Canadian capacity to compete in the global economy, the means by which this objective is to be achieved underwent some minor modifications in articulation. More specifically, the goals of the GCS were to:

• Boost Canadian commercial engagement in global value chains;
• Secure competitive terms of access to global markets and networks of Canadian businesses;
• Increase foreign direct investment in Canada and Canadian direct investment around the world; and
• Forge stronger linkages between Canada’s science and technology community and global innovation networks.^{Footnote 12}

Building on the success of the 2007 GCS, the GoC announced its Global Markets Action Plan (GMAP) in 2013. Under the GMAP the GoC declares its intent to concentrate efforts on the 66 markets that hold the greatest promise for Canadian business with focus on core objectives within those markets. The GMAP further identifies 22 priority sectors where existing capabilities and expertise provide Canada with a strong competitive advantage vis-à-vis other countries, or where there are dynamic opportunities to grow and develop Canadian business. Under the GMAP, comprehensive strategies are to be developed that will keep Canada at the forefront of key economic sectors and strengthen Canada’s trade, investment and partnerships in priority markets

The GMAP declares that all GoC diplomatic assets are to be harnessed to support the pursuit of commercial success by Canadian companies and investors, placing a heavy emphasis on improving emerging-market access to small and medium-sized enterprises (SMEs). To reach this goal, the GoC further declares its intent to pursue new trade agreements, foreign-investment protection agreements, taxation agreements, air transportation agreements and science and technology agreements. A core mandate of Canadian diplomats and other officials will be to “open doors, generate leads and resolve problems” for SMEs and other Canadian businesses.

Informing Canada’s GMAP is the key concept of global value chains (GVC), which is both an outcome of and driving force behind global economic integration.^{Footnote 13} In a global economic environment distinguished by falling barriers to trade and investment across borders, plummeting costs of processing and transmitting information, along with declining costs of transportation, firms are better able to outsource different aspects of the production process, including product design, acquisition of raw materials, production, marketing, distribution, and support to the final consumers, to wherever the necessary skills and materials are available at least cost. This has led to an increasing “fragmentation” of the production process whereby inputs from one country are exported to another for production and assembly and in turn distributed to yet another country for distribution and final consumption.^{Footnote 14}

Firms thus engage in global commerce today not only by selling finished products, but also by investing in production facilities abroad and by forming new partnerships with suppliers, producers, distributors and innovators around the world. This “unbundling” of stages and tasks in the production process has opened up opportunities for firms old and new seeking to expand operations globally while at the same time deepening specialization as firms seek to capitalize on whatever comparative advantage they may possess vis-à-vis any one or more stages or tasks within the production process and in so doing find their niche or link within the GVC.

Although the rise in GVC presents opportunities for Canadian SMEs, SMEs face a number of challenges in capitalizing on these opportunities, including finding market information, the right contacts, securing access to capital (e.g. venture capital), as well as R&D resources.^{Footnote 15} In this regard, government can play a role in helping firms overcome these challenges through targeted assistance, the aims of which find expression in the GCS.

Within the generality of the foregoing, engagement strategies have been developed for several of the four pillars of the GCS. With respect to GVC, for example, the Department has identified three core objectives and related activities. These include:

• Enhancing departmental understanding of GVC through the mapping out of GVC of select multinationals and through the conduct of GVC research;
• Increasing industry’s awareness and understanding of GVC through the facilitation of FDI abroad, helping to identify GVC opportunities for Canadian firms, and developing educational tools to assist the same; and
• Strengthening the department’s network abroad through the creation of a cadre of personnel dedicated to GVC both in Canada and abroad, encouraging missions abroad to work collaboratively on GVC initiatives, and in delivering an integrated value proposition linked to sector strategies.^{Footnote 16}

With regards to FDI promotion, the Department’s strategy identifies four broad objectives and related activities. These include:

• Improving Canada’s business/investment climate by benchmarking the same vis-à-vis competitors, identifying impediments to FDI, raising awareness of issues impacting FDI, and engaging stakeholders to advocate for policy changes;
• Raising Canada’s visibility as a competitive investment location through the development and implementation of a Private Sector Champion program, development and implementation of “Invest in Canada” Visibility Campaign, producing sector specific marketing tools, and leveraging investment signature events;
• Implementing proactive attraction strategies in priority sectors and key markets through the promotion of “Invest in Canada” services, identifying priority sectors and markets for proactive FDI promotion, identifying company leads in the same, and producing industry/sector investment cases; and
• Collaborating with partners to provide aftercare services to key investors by raising the profile of aftercare services with partners, supporting aftercare capacity building and by developing training courses on the same for partners and DFATD officers, and supporting the domestic network in maintaining relationships with investor clients through regional offices and missions abroad.^{Footnote 17}

In the domain of Science, Technology and Innovation (STI), the Department has been guided in its approach by the government-wide strategy, Mobilizing Science and Technology to Canada’s Advantage (2009), which highlights the importance of strengthening science, technology and innovation in ensuring long-term economic prosperity for Canadians, as well as the need to increase private sector investment in R&D and the commercialization of new products and services in international markets.^{Footnote 18} Within the generality of the foregoing, the Department’s specific goals are to:

• Increase the competitive advantage of Canadian firms through international innovation partnerships;
• Enable global R&D partnerships and access to technology and technology related expertise abroad;
• Increase access to global supply chains through business and technology partnerships; and
• Attract technology-based investments to Canada.^{Footnote 19}

These goals are to be achieved through the support provided by the Trade Commissioner Service (TCS) network, with the accent on managing bilateral STI cooperation agreements, and through the administration of pre-existing STI funding programs, namely the International Science and Technology Partnership Program (ISTP), and GGI within the GCSP.

In light of the foregoing, the evaluators placed particular attention on ascertaining how and to what extent GCSP and its sub-programs support, either in whole or in part, the strategic objectives of the Department as expressed through the GCS and the strategies referenced above.

3.2 Economic Context

When assessing the performance of a program designed to attract and retain FDI, increase the participation of Canadian businesses in foreign markets, and promote innovation through cooperative ventures in R&D, it is important to understand something of the economic context in which the program has been implemented. (See Appendix 3: Economic Context for details.) Over the last three years (the evaluation reference period), Canada has enjoyed sustained economic growth, registering a 6.1% increase in Gross Domestic Product (GDP) in 2010, followed by a 5.8% increase in 2011, and 3.4% in 2012.^{Footnote 20}

Although Canada’s share of global FDI stock-in remained static between 2010 and 2012, (roughly 3%), Canadian FDI stock-in increased by 7% during the same period, with FDI stock-in originating from Europe and Asia and the Pacific regions registering the largest increases. Canadian FDI stock-out also experienced significant growth, registering an 11.6% increase in the same between 2010 and 2012, with Canadian FDI stock-out to the US registering the largest increase, followed by Latin America and the Asia and Pacific regions.

Between 2010 and 2012 Canadian merchandise exports grew by 13.9%, with the export of goods to the Asia and Pacific regions the Middle East registering the largest growth. Export in services also experienced growth between 2010 and 2011, though at more modest levels (5.1%). Merchandise imports grew by 12.9% between 2010 and 2012, with growth in the import of services between 2010 and 2011 registering a 5.2% increase. But for modest declines in the import of services from Latin America and Africa, the overall geographic distribution of service imports during the reference period remained stable.

In the domain of R&D, Canada is a middling performer among Organization of Economic Cooperation and Development (OECD) countries, ranking 16th in R&D expenditures. Approximately 51% of expenditures on R&D originate from the commercial private sector, followed by expenditures originating from institutions of higher learning (academia) at 38% and government (federal and provincial) at 10%. Notwithstanding Canada’s modest performance among OECD countries in R&D expenditures overall, expenditures originating from institutions of higher learning is almost double that of OECD average.

Evidence of GVC participation is incomplete, largely confined to data on the use of imported intermediate goods and services in the production of exports, which finds expression in the vertical specialization index (VSI). Although there is evidence that Canadian participation in the GVC has increased over the last decade, Canada’s level of participation is well below that of other OECD countries. Based on data generated by the VSI, Canada’s level of participation has declined in recent years.^{Footnote 21} The evaluation explored the interface between Canadian economic trends and program relevance and performance.

3.3 Going Global – Innovation for Researchers Program

A decision was taken not to commit funds for GGI for 2012-13. Notwithstanding this development, the evaluation revisited the original assumptions underlying the creation of GGI and assessed its performance during the evaluation reference period with the view to identifying lessons learned and best practices.

3.4 Performance Measurement and the Attribution of Results

GCSP primarily funds initiatives which promote the creation of relationships that may require several years to culminate in a tangible investment thus making it difficult to provide a fulsome account of the program’s performance. Further, direct attribution of the results achieved is difficult because the funds provided through the GCSP may account for only a small proportion of the total efforts by recipients and other partners to build awareness of opportunities, foster research and development partnerships and attract investment leads. Notwithstanding these challenges, a Performance Measurement Strategy (PMS), complete with performance indicators, has been developed and in effect for the duration of the evaluation reference period. The evaluation relied heavily on the performance data collected through existing systems and, in doing so, assessed the effectiveness of these systems for performance reporting purposes.

3.5 Previous Assessments

3.5.1 Risk Assessments

The risk associated with the GCSP is considered low and is unlikely to jeopardize the program’s ability to meet its overall objectives. Previous evaluations of GG, CISP and PEMD-A conducted in 2007-08 indicated that all three programs were operating successfully and lessons learned from the three programs were then incorporated into the planning and design of the GCSP. Furthermore, the GCSP disburses relatively small amounts of money to Canadian recipients vetted through a rigorous application process.

The GCSP’s 2009 Accountability, Risk and Audit Framework (ARAF) noted, however, that the amalgamation of the CISP, GG and PEMD-A into one program with three pillars could pose administrative and implementation risks given that the amalgamation occurred at the same time as other major changes in the department and the GoC (e.g. Transformation, New Business Model, new Policy on Transfer Payments, changes in Departmental processes, Canada’s Economic Action Plan 2012, etc.). The evaluation therefore examined the extent to which the program has successfully managed these risks.

3.5.2 Previous Evaluation Findings and Recommendations

In 2007-08, ZIE conducted a formative evaluation of PEMD-A and summative evaluations of both CISP and GG. These evaluations found that all three programs continued to be relevant and have generated results for Canadians and the economy. However, deficiencies in performance reporting for all three programs made it difficult to come to firm conclusions on the extent to which they have achieved their long-term objectives. With regard to the evaluation issue of efficiency and cost-effectiveness, the evaluations recommended that: (1) the three existing programs be amalgamated to reduce administrative costs and address funding shortfalls and (2) the performance monitoring framework be strengthened to enable more comprehensive reporting on the results achieved. This evaluation examined whether amalgamation has generated the expected costs savings and the effectiveness of current performance measurement frameworks and systems.

3.5.3 Previous Audit Findings and Recommendations

Internal Audits

An internal audit of the GCSP conducted in 2011 found that the program was well-managed, but identified several areas for improvement. For example, the audit noted that:

• There was no practice in place to track and report on the cost of delivering the GCSP (i.e., expenditures other than transfer payments);
• The program faced challenges with respect to lapsed funds;
• Key financial monitoring tools were not integrated with the program’s database or with DFATD’s financial management system;
• Tools used to collect performance data were underdeveloped; and
• The DG Steering Committee had not been convening frequently enough to provide effective oversight over the program.

This evaluation assessed the extent to which each these areas of concern have been addressed.

Recipient Audits

Recipient audits conducted by DFATD’s Recipient Audit Division (ZIEA) involving projects approved under CISP, GG, and PEMD-A did not result in significant recoveries of funds by DFATD. Although no notable issues were identified concerning the management of public funds, the recipient audits for PEMD-A remarked that the Program’s T&Cs did not address the fact that recipients could further distribute contributions to third parties. This oversight was subsequently addressed in the current GCSP T&Cs, which require recipients to ensure that accountability and management frameworks are in place to monitor and evaluate the activities of third parties and that the selection of third party recipients is conducted in a manner that is clear and transparent.

With few exceptions, recipient audit reports conducted for projects approved under the GGI, GOA and ICCI sub-programs have concluded that project costs have been incurred and claimed in accordance with the GCSP’s T&Cs and that due diligence has been exercised with regard to the expenditure of public funds.

4. Evaluation Approach & Methodology

A mixed-method approach was used to increase the accuracy and clarity of findings, using multiple data collection methods to ensure the evaluation captured the full performance story of the program. Evaluation data was analyzed and triangulated with other sources of evidence to improve the quality and strength of the findings. Preliminary findings were then validated with stakeholders most knowledgeable about the GCSP and its sub-programs.

4.1 Evaluation Scope

Although the GCSP was formally constituted in December 2008, complete amalgamation of the former three contribution programs (CISP, GG, and PEMD-A) under a single management structure was not fully in effect until mid-2009. Furthermore, though an ARAF had been developed for the GCSP in the spring of 2009, performance data was neither consistently nor systematically collected against the performance metrics set out in the ARAF until 2010. In light of the foregoing, the evaluation’s assessment of expected outcomes for each component was confined to the reference period from fiscal years 2010-11 to 2012-13. This provided three years of performance data, which was more than adequate to address evaluation issues and questions. The scope for the assessment of relevance and demonstration of efficiency and economy, however, was from the beginning of amalgamation in 2009 to the end of data collection in February 2014. (See Appendix 1 for complete list of Evaluation Findings)

4.2 Methodological Approach

4.2.1 Evaluation Matrix

Following the approval of the evaluation TOR, consultations with program management were undertaken with the view to refining and, where necessary, clarifying the questions under each evaluation issue, as well as to identify appropriate indicators, data sources, and data collection methods. The results of these deliberations find expression in the Evaluation Matrix, which serves as the principal guide to the development of data collection tools and structure for the reporting of evaluation findings. (See Appendix 2 for the Evaluation Matrix.)

4.2.2 Methodology

The following sections describe these evaluation methods in more detail.

Key Informant Interviews

Qualitative data were collected through interviews conducted with key informants knowledgeable about the GCSP and its components, including program stakeholders and selected contribution recipients. These interviews were conducted in-person or by telephone and guided by interview questions that were shared in advance of the meeting. All data collected through interviews remained confidential and were aggregated in the final evaluation report to ensure participants were not identifiable.

These key informants included the following:

• DFATD ADM, BFM;
• DFATD DG, BID;
• DFATD DG, BBD;
• Directors and program officers from BFM, BID, BBD, BIS, BBT, BBI, BBR and BBM;
• Representatives from ROs in Canada and missions abroad (Brazil, China, India, Israel);
• Representatives from BFM’s CAU;
• Other Government Departments (OGDs);^{Footnote 22}  and
• Identified contribution recipients.

The names of funding recipients for interview were recommended by GCSP program officers and this list was supplemented using a purposive selection method.

Document and File Review

Statistics on Canada’s economic performance during the evaluation reference period have been assembled with the view to providing context to the relevance and performance of the GCSP and its pillars. The evaluators further undertook a review of the current literature on FDI promotion, integrative trade and the role of GVC, as well as the role of R&D in relation to the foregoing, with the view to understanding how the components of the GCSP, in design, support these economic objectives and are consistent with best practices.

In addition to the above, the evaluators have examined program documents including, but not limited to:

• TB Submissions;
• Audits and evaluations of the GCSP and its predecessors, CISP, GGSTP and PEMD-A;
• ARAF and sub-program level performance metricise;
• Management, planning and governance documents;
• Meeting minutes;
• Documents supporting the ICCI, GGI and GOA application and approval processes;
• Contribution agreements;
• Communications plans and tools;
• Financial documentation detailing FY budgets, funds allocation and actual expenditures by major program component (e.g. ICCI, GGI, GOA);
• recipient level progress/performance reports (e.g. output by activity, outcome and annual reports); and
• Correspondences.

This review was undertaken not only to enhance the evaluators’ understanding of the program and its modus operandi, but also to help identify some of the issues and challenges faced by the program.

Secondary Analyses of Existing Performance Data

As previously remarked, GCSP has had in place an ARAF since the spring of 2009 and has been collecting performance data against the performance metrics contained therein since 2010. Two administrative systems were created to capture program performance, one for PEMD-A, that was adopted by GOA with the amalgamation into GCSP, and one for ICCI that was created around 2008-09. Discussions with program managers suggest that plans were initiated to develop a similar system for GGI but it was not possible to secure sufficient funds. In addition, at the time the evaluation was conducted the existing administrative systems for GOA and ICCI required updating.  Nevertheless, these tools capture performance information for GOA and ICCI, including information on the application process, project information, funding details and management reports.

Additionally, GCSP administered two questionnaires to 2009-10 and 2010-11 program recipients in 2012 and 2013, respectively. These questionnaires were intended to gauge the impact funded activities on the same. The overall response rate to the questionnaires was around 50%, thus providing a high degree of confidence (95%) in the accuracy of the representations provided.

Data generated from these questionnaires, plus performance data recorded in the program’s performance monitoring systems, provided the evaluators with a rich reservoir of both quantitative and qualitative performance information. The survey results and related documentation was provided to the evaluators as well as direct access to the ICCI and GOA Administrative Systems. Accordingly, the evaluators relied primarily on the information contained in these data repositories to report on GCSP performance with respect to the achievement of expected results.

4.2.3 Limitations to the Methodology

The methodologies advanced herein engender several limitations and risks. These limitations and risks along with the proposed strategies to mitigate the same are briefly described below:

Program Administrative Systems

While ICCI and GOA have Intranet sites populated with financial and program data, GGI financial and program data is not consolidated in any one data repository, thereby limiting the ability of the evaluators to access key program and performance data. In response to this limitation and risk the evaluators worked in cooperation with program personnel to assemble key program and performance data in a usable format.

Recipient Informant Selection

Program personnel were requested to identify a representative sample of funding recipients relating to their respective sub-programs for interviews. While the evaluators have no reason to believe that the list of recipients provided is anything other than representative, there remains a risk that the sample may be perceived as biased. To address this risk, the evaluators proposed to supplement the list of recipients provided by program personnel with a number of additional recipients randomly selected.

Results Data

A review of the performance data for the three sub-programs revealed that there is some measure of inconsistency in the way certain performance indicators are interpreted and recorded both within and between the sub-programs, resulting in some cases in double-counting. Further, output and outcome data relating to the same indicator differ depending on the source and context. To address this limitation and mitigate the risk of misrepresenting and misinterpreting the results data, the evaluators worked with program personnel to reconcile identified inconsistencies. Additionally, results data gleaned from program systems were triangulated with results data obtained from other lines of evidence.

5. Evaluation Findings

5.1 Relevance: ICCI

5.1.1 ICCI: Continued Need for the Sub-Program

Finding 01: Communities across Canada are in need of support to strengthen their capacity to attract and retain FDI.

Although FDI carries with it some risk of displacement for domestic producers, the literature suggests that there is a general consensus among economists that the overall benefits of FDI far outweigh any potential risks associated therewith. Common economic benefits of FDI include its contribution to: 1) capital formation; 2) employment generation; 3) increased tax revenues; and 4) increased productivity.

Beyond the generality of the foregoing, FDI can bring increased access to new production and managerial techniques otherwise unavailability to domestic producers. It can facilitate increased access to foreign markets through established distribution networks (such as GVCs), thereby contributing to the achievement of greater economies of scale. FDI can further serve to stimulate ancillary economic activity by creating markets for local suppliers of goods and services (backward linkages) and by acting as suppliers of intermediate goods and services to local businesses (forward linkages). Finally, FDI can contribute to greater competition, which in turn stimulates innovation, efficiencies in production, and ultimately lower prices for products and services, which benefits consumers.

In light of the above, countries today compete vigorously among themselves to attract FDI. Factors influencing FDI are varied and complex, including but not limited to: a nation’s natural resource endowment; access to markets; access to skilled and mobile labor; the state of a nation’s economic infrastructure (e.g. transportation, communications, etc.); the existence of a legal and judicial system that operates in conformity with international norms, particularly with respect to the enforcement of property rights and the law of contract; the existence of a robust banking system; and a nation’s economic policies (e.g. fiscal, monetary and regulatory), all of which have a profound impact on the cost of doing business in a particular country and thus on its attractiveness as an FDI destination.

Canada, on the whole, scores very well in reference to the foregoing factors.^{Footnote 23} As briefly remarked in Section 3.2 of this report, and addressed in greater detail in Appendix 1: Economic Context, Canada’s recent performance in attracting FDI has been positive, registering a 7% increase in FDI inflows during the evaluation reference period. This said Canada’s share of global FDI has remained relatively static for over a decade which has led some economists to remark that Canada needs to do more to attract new FDI.^{Footnote 24} Beyond the use of macro-economic instruments to increase Canada’s attractiveness as an FDI destination, a subject outside the scope of this evaluation, economists have frequently referenced the importance of investment promotion as an effective means of encouraging inward FDI, particularly at the local level, a function which is commonly performed by investment promotion agencies.

Investment promotion agencies typically perform four functions: 1) policy advocacy, which entails conducting research and developing policy proposals designed to improve the investment environment; 2) marketing, which involves activities such as advertising and public relations events designed to create a positive image of the investment site; 3) investment generation, which entails targeting specific sectors or companies to invest; and 4) investor services, which encompasses a range of services that help prospective investor set up a business, including expediting approval processes.^{Footnote 25} Today, investment promotion agencies perform most, if not all, of the functions cited above, and can be found at virtually all levels of government (national, provincial, and municipal) in Canada. The capacities of these entities to perform the aforementioned functions, however, are as variable as the constituencies they represent.^{Footnote 26}

Finding 02: ICCI is responsive to the needs of communities across Canada in both design and operation.

ICCI is specifically designed to address the institutional capacity deficits referenced in the preceding Finding. It does so through the provision of financial assistance to municipalities to: 1) undertake research (e.g. SWOT analysis) with the view to understanding the community’s strengths and weaknesses and to develop investment profiles; 2) develop targeted investment strategies with the view to identifying and reaching prospective investors; 3) design instruments to market and promote the community; and, 4) train personnel to undertake the foregoing and to provide investor services.

Notwithstanding the foregoing, ICCI does not cover all the expenses related to the aforementioned activities, and for some sub-program beneficiaries this has presented a challenge. For example, although the procurement of training material is deemed an eligible expense under the program, the costs associated with procuring the services of a training facilitator are not an eligible expense. Nor, reportedly, are the costs associated with the use of audio-visual equipment in the delivery of training covered by the sub-program. ICCI covers the costs of the design of promotional documents, but does not cover the costs of producing these documents.

In considering the materiality of the aforementioned complaints, it is important to remember that beneficiaries are expected to contribute a portion of their own resources to funded activities. ICCI beneficiaries, on the whole, understood the rationale for cost sharing and did not challenge the merits of this approach. The grounds for their complaint appears to rest on the view that costs associated with implementing certain activities cannot be easily disaggregated and that the practice of covering some costs and not others adds an element of complexity to the accounting process, thereby increasing the administrative burden. In short, but for the issues referenced above, beneficiaries on the whole found the sub-program design responsive to their needs.      

Finding 03: To the extent that ICCI remains focused on building capacity within communities with the view to making them “investment ready,” ICCI continues to fill a funding gap in the area of FDI promotion.

Although communities across Canada do have access to a variety of federal and provincial funding sources to support local economic development, including some aspects of inward FDI promotion, none precisely match the suite of activities supported by ICCI. For example, although provincial economic development organizations make funds available to communities, they do so primarily for infrastructure projects which are relevant but incidental to FDI promotion. While federal regional development  agencies such as Western Economic Diversification (WED) and the Atlantic Canada Opportunities Agency (ACOA) have historically provided funds for FDI promotion, they did so in support of  activities at the far end of the FDI continuum, namely facilitating meetings between community representatives and businesses with potential foreign investors. Federal regional development agencies are reported to be no longer providing this kind of support. In short, ICCI fills a funding gap to the extent that it remains focused on building capacity within communities with the view to making them “investment ready.”

5.1.2 ICCI: Alignment with Government Priorities

Finding 04: ICCI in design and operation remains fully consistent with and supportive of GoC and Departmental priorities.  

The GoC in its Global Markets Action Plan: the Blueprint for Creating Jobs and Opportunities for Canadians through Trade (2013) affirms the government’s commitment to “address barriers to trade and investment through targeted advocacy, troubleshooting and dispute settlement, especially in established markets and priority emerging markets.”^{Footnote 27} Additionally, FDI promotion figures among the four core objectives of DFATD’s GCS, wherein the Department declares its commitment to “making Canada a magnet for global investment.”^{Footnote 28} As stated in DFATD’s FDI Strategy, the Department aims to achieve this higher-end objective by: 1) raising awareness of the issues impacting on FDI with the view to encouraging stakeholders to advocate for policy changes; 2) raising Canada’s visibility as a competitive investment location through the development and implementation of sector specific marketing tools; 3) implementing proactive attraction strategies in priority sectors and markets; and, collaborating with partners to provide aftercare services to key investors through capacity building initiatives.

As stated in the preceding finding, ICCI in design aligns well with all these objectives. For example, funds provided through the sub-program to conduct research helps identify policy issues at the municipal level which need to be addressed to improve investment attractiveness. Funds provided through the sub-program to develop promotional tools to market a community as an attractive FDI destination serves to raise Canada’s visibility as a competitive investment destination. Funds provided through the sub-program to communities to develop FDI strategies helps channel FDI originating from priority markets into priority sectors. Finally, funds provided through the sub-program to support training helps in the provision of aftercare services.

In terms of the alignment of ICCI programming with declared priority sectors, over 91% of activities supported through the program align with these sectors.^{Footnote 29} As shown in Table C (Distribution of ICCI Programming by Priority Sectors) in Appendix 4, almost 20% of FDI promotion activities relate to the Clean Technologies, followed by 19% for ICT, 15% for Advanced Manufacturing, 12.5% for Life Sciences and 12% for Business and Financial Services, with the balance distributed between Agrifood (food processing) and Plastics and Chemicals. A high degree of alignment of activities with priority sectors is assured by the fact that ICCI applicants are required to demonstrate in their applications that proposed activities relate to these priority sectors.

With regard to declared FDI priority markets, over 95% of ICCI beneficiaries target their promotional activities to these markets. In descending order of importance around 65% of ICCI identify Europe as their preferred market for FDI, with 31% identifying  the Asia and the Pacific region as their preferred market, followed by   around 20% for the Americas as their preferred market for FDI (see Table D: Number of Activities by Identified Markets in Appendix 4).

In addition to the foregoing, the GoC in its Global Markets Action Plan: the Blueprint for Creating Jobs and Opportunities for Canadians through Trade (2013) states that the GoC affirms its commitment to work with “the Federation of Canadian Municipalities to ensure that communities of all sizes across Canada benefit from, contribute to and work alongside the Government of Canada towards shared objectives.”^{Footnote 30} ICCI similarly endeavors, through an annual notional allocation of sub-program resources to the provinces and territories based on predetermined formula, to maintain an element of equity in its programming approach. While, as provided in Table B (Distribution of Committed Funds for ICCI across Provinces and Territories) in Appendix 4 of this report, communities in Ontario and Quebec account for close to an average of 60% of sub-program disbursements, communities large and small from across the country are eligible for ICCI funding.

While the attention given to making sub-program funds available to all provinces and to smaller communities is clearly aligned with GoC policy, some stakeholders interviewed in support of this evaluation questioned the merits of this approach, suggesting that program funds would be more likely to generate results, and thus better spent, if targeted to communities and geographic areas where significant FDI attraction capacity already exits. Such an approach, if adopted, would almost certainly result in sub-program funds being concentrated in Canada’s major metropolitan cities where, in fact, the funds are less likely to be needed. Additionally, were the sub-program to focus on communities with pre-existing capacity, it would in effect eschew its capacity development focus.  As such, the sub-program’s current focus on capacity building within communities remains appropriate, all things being the same.

Notwithstanding the foregoing, it is acknowledged that support to smaller communities does engender a higher risk of failure, but one which is in part mitigated by the calibrated approach to funding which takes into account an applicant’s place on the FDI continuum, which in turn helps ensure that applicants are able to deliver on realistic and achievable objectives. Further, the sub-program makes efforts to encourage communities within a geographic area to cooperate and adopt a regional approach to their FDI attraction strategies, one which fully capitalizes on the collective assets of all the communities represented, as a means of enhancing the likelihood of success.

5.1.3 ICCI: Federal Roles and Responsibilities

Finding 05: To the extent that ICCI continues to fill a funding gap in the area of FDI promotion, the sub-program and the activities it supports remain an appropriate role for the federal government.

As remarked in the preceding findings, ICCI does fill a funding gap in the domain of FDI promotion and is aligned with GoC and Departmental priorities, which would suggest that FDI promotion is an appropriate role for the federal government. However, it remains somewhat counter-intuitive that DFATD provides funds for local, community-level FDI capacity development activities while provinces and municipalities direct resources towards international FDI attraction activities. Given DFATD’s international mandate, and unique competencies in the domain of FDI, one would expect the roles to be in reverse, where provinces and municipalities would focus on local capacity development and DFATD, through its network abroad, would focus on attracting and facilitating FDI to those communities.

The fact remains, however, that no other funding source supports the suite of activities currently supported by ICCI. Furthermore, direct federal government funding to communities helps raise the profile of FDI as an instrument for local economic development while simultaneously providing an incentive to communities to develop the capacity to exploit the opportunities relating to the same. Direct federal government engagement with communities also grants the former an opportunity to influence and shape the FDI promotion activities of communities in a manner consistent with and supportive of GoC and Departmental policies and priorities. Finally, federal government engagement with communities allows it to perform an important coordinating role, reducing the likelihood of duplication of effort and the risk of incoherent messaging. In short, to the extent that ICCI addresses the aforementioned issues, it remains an appropriate responsibility of the federal government.

5.2 Relevance: GOA

5.2.1 GOA: Continued Need for the Sub-Program

Finding 06: Canadian SMEs are in need of support to effectively take advantage of global commercial opportunities and GOA, both in design and operation, is largely responsive to those needs.

Small and medium sized enterprises (SMEs) are those business establishments with 1 to 499 paid employees and annual sales ranging from less than $10 million for small businesses and between $10 million and $50 million for medium-sized businesses.^{Footnote 31} In 2012, SMEs represented close to 99.8% of all business establishments with employees,^{Footnote 32} accounting for 39% of Canada’s GDP and 89.9% of all private-sector jobs.^{Footnote 33} In terms of Canada’s job creation, SMEs accounted for more than 90% of all private jobs created from 2002 to 2012.^{Footnote 34}

SMEs are essential to improving Canada’s economic trade performance.^{Footnote 35} In 2011, however, only 10.4% of SMEs exported goods and services^{Footnote 36} yet accounted for about 40.1% (or approximately $150 billion) of the $374 billion total value of exports from all Canadian businesses.^{Footnote 37} These numbers suggest that there is considerable room for growth in exports among Canadian SMEs that could yield significant economic benefits.

Recent research suggests that exporting boosts profits and sales of SMEs, regardless of company size, average wages, number of export destinations and leverage.^{Footnote 38} According to the OECD, prospective high-growth SMEs may realize their potential by integrating into global markets that provide a range of business opportunities in larger and new niche markets, increased potential for upgrading technological capacity and help to gain efficiencies by exploiting economies of scale.^{Footnote 39} Research also suggests that integration into global markets increases exposure to diverse products and services through formal relationships with foreign partners and increased competitive pressure from foreign companies may also spur productivity and innovation for SMEs.^{Footnote 40} Formal arrangements with foreign partners, such as specialized suppliers to multinational enterprises, or in informal global networks can also deliver a number of benefits to SMEs, including access to financial resources, pooled research efforts, product development and wider distribution channels.^{Footnote 41} There are, therefore, tangible benefits for Canadian SMEs to enter into global markets, pointing to a need for the federal government to facilitate this integration.

Canadian SMEs are in a good position to enter the global market with the emergence of GVCs as the prevailing model for the production of goods and services,^{Footnote 42} particularly for those SMEs that already engage in interprovincial trade.^{Footnote 43} As discussed in Section 3.1, the fragmentation and international dispersion of the production process inherent to GVCs provides an opportunity for smaller firms to specialize and capitalize on their unique skills and products, and grow their businesses. Canadian businesses may export its products to a foreign partner to input into production and assembly abroad (forward participation) or import foreign parts for input into Canadian export products (backward participation). In 2009, forward participation was 14% and backward participation was 20%.^{Footnote 44}

According to the OECD (2013), entrée into a GVC is advantageous for smaller firms that can exploit their flexibility and speed in new niches for the supply of novel products and services, without having to build an entire value chain for their products. Research suggests that these benefits may be maximized by integrating into fast growth markets like China, Brazil and India, where top performing businesses almost double their annual sales and the weakest performers exit these markets in less than a year^{Footnote 45} and where the likelihood of survival for Canada’s SMEs depends on their capacity for innovative technological advantage, their understanding of and strategic entry into selected foreign markets, their international reputation as good global partners and their ability to develop arrangements or build relationships with potential foreign partners.^{Footnote 46}

Compared to larger firms, Canadian SMEs often lack the internal resources and international networks for IBD and are more reliant on services and programs offered by government agencies, trade associations and other external partners for financing, market intelligence, identification of potential foreign partners, navigation through regulatory impediments, and overcoming all of the other challenges involved in expanding beyond domestic markets.^{Footnote 47} The business environment of each target foreign market is unique and complex with its own regulatory, administrative, policy and cultural environments within which Canadian SMEs must maneuver successfully. SMEs have the same needs as larger firms for research and preparation prior to global markets engagement yet have fewer resources to meet the challenges.^{Footnote 48}

A review of business literature and survey results suggests that the most critical impediments for SMEs in entering global markets include (in ranked order of importance):

• Identifying good market opportunities;
• Finding reliable business partners;
• Managing supply chain relationships and the demands of complying with quality control and systems requirements;
• Getting access to financing;
• Handling authentication and security;
• Protecting intellectual assets and business information; and
• Overcoming trade and regulatory barriers.

Likewise, results from the 2012-13 Management Issues Survey by the Canadian Manufacturers and Exporters (CME) indicate that 43% of respondents report lack of expertise or knowledge of new markets is a significant constraint to expanding global business. This is followed by difficulty in finding partners, distributors, sales agents (38%), difficulties identifying market opportunities (35%) and lack of internal expertise (31%).^{Footnote 49}

There is, therefore, an ongoing need for funding programs like GOA to assist SMEs given their challenges in entering GVCs, particularly in fast-growth markets and for smaller Canadian SMEs that have fewer resources and capacity to enter into the global market. GOA’s funded activities help to equip SMEs to engage in foreign markets. By funding associations to facilitate their members’ participation in events like trade shows, GOA can help foster direct contacts with foreign enterprises to solidify Canadian SMEs’ international partnerships in priority foreign markets. Supporting the development of marketing tools like websites and research reports can extend associations’ reach and strengthen their market intelligence for the benefit of their SME members, particularly for smaller associations. GOA programming is needed to address some of the challenges and contribute to SME engagement in foreign markets.

While responsive to the needs of Canadian SMEs and the associations that represent them, some recipients and program officers report that GOA’s design also creates some challenges. For example, applications are evaluated according to the potential for project activities to contribute to IBD sectoral growth. This suggests that repeated activities are not eligible for funding unless applicants can demonstrate how the proposed activities provide incremental progression towards larger objectives. This is difficult to demonstrate for some associations that apply every year for support to the one major trade event the industry holds annually for key players in the field. These annual events provide opportunities that may not be available elsewhere for Canadian SMEs to make contacts with potential foreign partners. This suggests that the notion of “incrementality” must be more clearly understood and defined in the application evaluation criteria to accommodate the context of industry trade events.

Funding recipients also note that limits on eligible expenses are sometimes constraining. Funding caps, for instance, limit the number of SME representatives that associations can support for attendance at annual trade shows, thereby forcing associations to exclude some SME members with the most need. Interviews with GOA staff suggest that raising the limits could also help to reduce GOA’s funding lapses since higher dollar values could be released for activities. The evaluation evidence demonstrates that there is a continued need for GOA, but some of its evaluation criteria and eligibility expenses could be refined to improve its relevance for Canadian businesses.

Finding 07: GOA is unique among other IBD programs in its support for national industry associations, thereby filling a funding gap.

GOA is unique compared to other programs in its funding of Canadian associations rather than businesses directly. The Export Market Access (EMA) program, which is a global expansion initiative of the Ontario Chamber of Commerce that is supported by both federal and provincial governments, is a case in point. Like GOA, EMA supports the expansion of Canadian businesses into foreign markets through exports but focuses specifically on Ontario businesses and funds SMEs directly rather than their industry associations. Eligible expenses include activities related to direct contacts, market tools development and market research, like for GOA, but also include foreign bidding projects to assist professional services businesses (e.g. construction, consulting and engineering) compete globally for projects abroad.

Advantages to supporting industry associations include the ability for GOA to support a larger number of SMEs through associations’ broad membership base, develop the capacity of Canadian associations that are often small and under-resourced, and act as a leveraging tool to affect change within industry through their associations. Additionally, because GOA funds national associations or regional associations with a national perspective rather than businesses directly, there are few issues with duplication or overlap with other funding programs. In short, GOA fills a funding gap.

5.2.2 GOA: Alignment with Government Priorities

Finding 08: GOA’s current focus on increasing exports remains consistent with and supportive of GoC and Departmental priorities as well the priorities of industry associations.

The GCSP was designed as part of the International Commerce Program Activity within DFATD’s Program Activity Architecture (PAA) and generates results to support the department’s strategic outcome, International Services for Canadians, and the GoC’s targeted outcome for A Prosperous Canada through Global Commerce. The 2009 Global Commerce Strategy (GCS) and the 2013 Global Markets Action Plan (GMAP) identify foreign markets with the best potential for advancing broad Canadian commercial interests and enhancing specific opportunities for Canadian businesses. As described in Section 3.1, the three GCSP components address one of the GCS’ three core objectives: Making Canada a partner of choice for international business.

GOA more specifically is aligned with the GCS and GMAP by promoting linkages between Canadian businesses with international business partners in priority markets and sectors. All funded activities do not necessarily have to be in priority markets or sectors since GOA is driven by the needs of Canadian associations, but application adjudication scores are weighted to give preference to projects in priority sectors in priority countries. As described in Table E (Expenditures and Number of Packages for GOA by Year and Region) in Appendix 4, almost 90% of GOA funds from 2010-11 to 2013-14 on average were expended on activities directed to Canada’s priority markets in North America, Europe and Asia Pacific. These markets are also Canada’s largest trading partners, though the Asia Pacific region now supersedes Europe as a destination for Canada exports (see Appendix 3: Economic Context). Further, as described in Table F (Number of Associations and Amount Claimed by Sector per Year) in Appendix 4, approximately 60% sub-program resources were allocated to industry associations identified as sectoral priorities.^{Footnote 50}

GOA is also aligned with governmental priorities by assisting SMEs access GVCs to advance Canada’s commercial interests. The GVC model provides opportunities for SMEs to grow their businesses by exporting goods for production abroad or importing foreign parts for input into domestic products. Evaluation evidence indicates that industry associations have tended to use GOA to increase their footprint abroad primarily through exports, even though the eligibility criteria can accommodate both exports and imports. GOA was designed to incorporate a broader definition of IBD to allow integrative trade activities beyond export but this has largely not been adopted by industry associations in practice.

5.2.3 GOA: Federal Roles and Responsibilities

Finding 09: Given GOA’s current focus on strengthening the IBD capacity of national industry associations, DFATD remains the logical home for this sub-program.

Evidence from international agencies and other countries demonstrate that there is a federal role and responsibility for supporting the integration of SMEs into the global economy. The OECD (2004) asserts that governments have a role to play in lowering the barriers faced by entrepreneurs who wish to globalize their activities. In addition to ensuring that regulatory, administrative and policy environments do not inhibit SME’s access to global markets, governments are believed to have a role in ensuring that programs facilitate access to international business networks by building global networks of intermediary organizations and coordinating national support programs.^{Footnote 51, Footnote 52} Canada’s own concerns with SMEs in the export market have been long-standing. The first report by the Parliamentary Standing Committee on Foreign Affairs and International Trade (1996) considers the appropriate governmental role in promoting SME IBD, noting that ongoing knowledge gaps in market information and intelligence have been a deterrent for SMEs in integrating into the foreign market and that federal initiatives has a responsibility to fill those gaps left by the private sector.

A more concerted approach is thus required by governments to help SMEs address challenges in identifying and assessing the commercial value and associated risks to pursuing specific opportunities in foreign markets. This approach “should include third parties such as trade associations and other organizations providing services in this area”^{Footnote 53} and should enhance capacity within SMEs to overcome key obstacles, including their lack of expertise or knowledge of new markets and identifying market opportunities and qualified business partners. This approach has been adopted by GOA in its focus on industry associations. Given DFATD’s mandates and objectives for international trade and as the home of the TCS that provides logistical support, initial contacts and general market and country information to Canadian businesses in foreign markets, DFATD has the appropriate role and responsibility for GOA delivery.

5.3 Relevance: GGI

5.3.1 GGI: Continued Need for the Sub-Program

Finding 10: Canadian academic and industrial researchers continue to require assistance in the formation of international partnerships to advance their S&T and R&D objectives.

Labor productivity, the measure of how efficiently goods and services are produced, is the single most important determinant of business competitiveness and, by extension, one of the most important determinants of the per capita income of a nation. Although Canada enjoys a relatively high level of per capita income, largely on account of the wealth generated from its rich natural resources, labor productivity in the country is, compared to other OECD countries, low. Among the 16 countries monitored by the OECD, Canada ranked 13 in labor productivity in 2012, a performance which has prompted many economists to declare that Canada must do more to improve productivity if living standards are to be preserved and enhanced.^{Footnote 54}

Productivity growth is primarily the function of innovation which is defined by the OECD as “the implementation of a new or significantly improved product (good or service), or process…or organizational method.”^{Footnote 55} Innovation is in turn significantly influenced by R&D spending. Although such spending is not a direct measure of innovation, it is an important indicator of a business’s commitment to develop new or improved products, processes and services, which is the ultimate expression of innovation. Studies have shown, for example, that a sustained increase in R&D spending of 0.1 % can translate, all things being equal, into a 1.2% increase in per capita income, thus demonstrating the correlation between R&D spending, increased productivity and income growth.^{Footnote 56}

As briefly referenced in Section 3.2 of this report, and elaborated on in greater detail in Appendix 3: Economic Context, Canada has been a middling performer in R&D spending, ranking 16 in terms of spending as a percentage of GDP out of 33 countries monitored by the OECD. Canadian business spending on R&D is even worse, placing Canada 15th out of 16 peer nations.^{Footnote 57} The reasons for Canada’s weak performance in R&D are varied and complex, but reference has frequently been made to Canada’s continued reliance on the extractive sector as the backbone of the country’s economy, which presents fewer opportunities for product innovation, and the fact that Canada’s small domestic market does not create a strong incentive for businesses to invest in innovation.^{Footnote 58}

Whatever the reason, economists and policy makers generally agree that Canada’s future prosperity depends on renewed and sustained efforts to develop and leverage its pool of talent, and encourage investment in leading edge technologies, a view that found expression in the GoC’s 2007 S&T Strategy wherein it states that “Canada can and must do more to turn our ideas into innovations that provide solutions to environmental, health, and other important social challenges, and to improve our economic competitiveness.”^{Footnote 59}

That same year, the GoC commissioned the Canadian Council of Academies (CCA) to undertake a study of the innovation performance of Canadian business. In 2009, the CCA released its report entitled, Innovation and Business Strategy: Why Canada Falls Short, which recommended that “Canada must encourage both new venture financing and commercialization of innovation; also, more must be done to transform Canada’s strong university research into commercially viable products.”^{Footnote 60}

The challenges in achieving these higher-end objectives are not insignificant. Scientific and technical R&D is costly and fraught with risk, particularly, as alluded to earlier, in Canada where its small domestic market acts as a disincentive to businesses to invest limited resources in R&D. These costs and risks, however, can be reduced through partnerships: partnerships between academic institutions, partnerships between academic institutions and private businesses, partnerships between businesses, and, of course, partnerships between both academic institutions, private businesses and government funded research institutions. It is for reason of the benefits of sharing costs and risks in R&D, including among them the fact that collaborative arrangements between researchers often serves as a catalyst for additional financial support, that governments the world over have instituted policies and programs specifically designed to encourage partnerships of the kind referenced above.^{Footnote 61}

While partnerships constitute an effective means of overcoming some of the impediments to R&D, forging such partnerships entail their own set of costs and risks, particularly when these partnerships involve parties in different jurisdictions. R&D, like all aspects of the design, production and distribution process, is today increasingly global, where collaborative arrangements are sought with prospective partners wherever the requisite resources and expertise reside. Like Canadian SMEs seeking to find their place within GVCs, Canadian researchers seeking to forge collaborative arrangements with partners abroad face a number of questions and cost challenges. Key questions researchers face are:

• Where do the desired resources and expertise reside?
• With whom should one partner?
• What are the encumbrances, if any, to partnership within the jurisdiction selected?

Direct cost challenges associated with establishing relationships with R&D collaborators include:

• the costs associated with finding answers to the aforementioned questions;
• the costs associated with meeting prospective partners and exploring partnership options; and
• the costs of developing the legal documentations to solidify the R&D partnership.

As remarked in Section 1.7.3 of this report, GGI was designed specifically to help Canadian academic and private sector researchers answer many of the questions referenced above, as well as defray the direct costs of negotiating and concluding a partnership arrangement, including procuring the services of legal counsel in support thereof. Although GGI expects applicants to have identified their prospective partners in advance of an overseas mission and by extension the selected market in order to discourage “fishing expeditions”, funds provided to defray travel costs assist Canadian researchers confirm, through direct contact with prospective partners and other relevant stakeholders, whether both the partners identified and the environment within which they operate are appropriate and conducive to achieving the beneficiary’s S&T and industrial R&D objectives. GGI beneficiaries interviewed in support of this evaluation confirmed that the sub-program significantly reduces the costs and risks associated with exploring collaborative S&T and R&D opportunities with overseas partners, thereby helping the same to establish appropriate and durable R&D partnerships.

While GGI beneficiaries value the funds made available to them through the sub-program, GGI beneficiaries, like the beneficiaries of the other sub-programs under the GCSP, also conveyed some frustration with the limitations on what the sub-program covered. For example, some beneficiaries, while acknowledging the rationale for imposing a minimum threshold for the application of funds, also remarked that this threshold was too high, thereby excluding some PDAs of utility. Others cited limitations on what constitutes an eligible expense, referencing the prohibition on expenses incurred in the procurement of travel documents, as an issue, while still others complained of restrictions on the how often a beneficiary could apply under the sub-program (the “incrementality” issue) as a problem, given that partnership development occurs over an extended period of time, involving many steps. These complaints, however, were deemed of marginal importance, and offset by the significant benefits of the sub-program.

It is also worth noting that TCS personnel posted at Canadian missions abroad are also tasked with supporting Canada’s international S&T strategy and are indeed required to report on activities in support of the strategy against Key Performance Indicators (KPI). In this regard GGI presented one of the few funding instruments available to the TCS community to realize this aspect of their mandate. TCS was fully integrated into the planning and delivery of GGI, contributing to the assessment of project proposals, assisting in the execution of those PDAs approved and in providing post-PDA implementation follow-up. GGI beneficiaries interviewed for this evaluation remarked on the value of the support offered by TCS at mission. TCS personnel interviewed for the evaluation in turn remarked on the importance of GGI as an instrument to support the formation of S&T and R&D collaborations.

Finding 11: Though GGI does fill a funding gap, the niche it occupies is extremely narrow and fragile.

Canadian researchers have access to a wide range of funding sources that provide financial support for R&D. These include, among others, tax incentive programs both at the federal (e.g. the Science Research and Experimental Development (SR&ED) Tax Credit Program) and the provincial (e.g. Ontario Research and Development Expenditure Program) levels, as well as a host of other programs, again both at the federal (e.g. the Industrial Research Assistance Program [IRAP]) administered by the National Research Council (NRC),^{Footnote 62} and provincial (e.g. New Brunswick Innovation Foundation) levels, which directly fund R&D initiatives and R&D collaborative arrangements.

These sources of direct and indirect funding, however, vary considerably in their objectives and modes of assistance. Tax deduction and credit programs, for example, are primarily focused on encouraging domestic investment in R&D. Some direct funding programs are targeted exclusively at the academic community while others focus only on the commercial private sector. Still others offer funding for R&D that is sector specific, like health, energy, ICT or environmental sciences. While most programs are designed to support ongoing R&D initiatives, there are a few which also support partnership development, but here again the objectives and modes of assistance vary. Some partnership development programs are primarily focused on supporting domestic partnerships, while others do have an international focus, like NRC’s IRAP. Though IRAP does support partnership development with foreign researchers, it does so only with businesses within the commercial private sector.     

What distinguishes GGI from these other programs, apart from its focus on nurturing international partnerships, is that: it serves both the academic and commercial private sector research communities; it is not bound to any specific sector, and; compared to other PDA programs, is more permissive of beneficiaries of different sizes. Within the general parameters of the sub-program’s priorities, sub-program beneficiaries are more or less free to determine for themselves the area of R&D to pursue and the partners with whom they wish to collaborate. This measure of flexibility built into the sub-program, which is more accommodating to the specific interests of Canada’s S&T and industrial R&D community, was cited by GGI beneficiaries interviewed in support of this evaluation as one of the sub-program’s most attractive features. Sub-program beneficiaries further remarked that because PDAs are largely self-directed under GGI, they are generally more targeted and thus more likely to produce results.

Notwithstanding the foregoing, there is another program administered by DFATD which supports PDAs – DFATD’s International Science and Technology Partnerships Program (ISTPP). Although the ISTPP was mandated to fund specific R&D projects in four identified priority markets (Brazil, China, India, and Israel), its funding authority permitted the use of program resources for PDAs. An evaluation of ISTPP was conducted in 2010 which remarked that a higher than expected amount of program funds were used to support PDAs.^{Footnote 63} This observation, combined with little evidence of these PDAs leading to R&D projects, largely on account of the fact that the vast majority of PDA funds went to support beneficiary attendance in networking events, such as conferences and seminars, prompted a commitment on the part of DFATD (then DFAIT) to subject PDA proposals to more rigorous assessment and oversight. Since 2010, the number of PDAs supported by ISTPP is reported to have dropped significantly, though the program retains the authority to fund PDAs within the program’s suite of priority countries.

There is, therefore, overlap between GGI and ISTPP, but only in those priority countries in which ISTPP is mandated to operate, namely: Brazil, China, India, and Israel. GGI engagement with these countries accounts for over a third of funded PDAs over the evaluation reference period. Further, only private businesses under ISTPP are eligible for PDA funding while both private business and academic institutions under GGI are eligible. Additionally, while monies spent in support of legal counsel relating to the negotiation of a partnership agreement is an eligible expense for GGI, the same is not eligible for ISTPP. Still, there is a potential for duplication because these two DFATD programs can be accessed to support similar activities and with the same country partners.

What is evident from the foregoing is that, although GGI does arguably fill a funding gap, that gap is extremely narrow and fragile. More specifically, minor modifications to the eligibility criteria of some other programs that also support PDAs would likely render GGI redundant. This being the case, some stakeholders interviewed in support of this evaluation questioned whether the direct funding model (funding individual project proponents) is the most appropriate vehicle to reach Canada’s S&T and R&D community and to support their S&T and R&D collaboration objectives.

5.3.2 GGI: Alignment with Government and Departmental Priorities

Finding 12: GGI in design and operation is well aligned with GoC and Departmental priorities.

The GoC is currently in the process of updating its Science, Technology and Innovations Strategy. While the role international partnership development will play within this new strategy remains a matter of speculation at this juncture, it would be fair to presume that it will occupy a prominent place in the same given that the GoC’s Global Market Action Plan: The Blueprint for Creating Jobs and Opportunities for Canadians through Trade (2013) expressly reaffirms the government’s continued commitment to supporting participation by innovative Canadian companies in international business and research partnership initiatives.

As remarked in Section 3.1 of this report, forging stronger linkages between Canada’s science and technology community and global innovation networks remains one of the four core objectives of the GoC’s GCS. The accent in these R&D policy statements is clearly on supporting R&D partnership arrangements with a commercial focus between businesses. Although the commercialization of technology borne from R&D constitutes GGI’s ultimate outcome, that R&D project proposals are assessed in reference to their commercial potential, and that, as shown in Table G (Funded Projects by Recipient Type – GGI) in Appendix 4 of this report, an average of over 64% of sub-program beneficiaries are indeed commercial enterprises, the fact that over a third of sub-program beneficiaries have been academic and not-for-profit organizations might suggest a misalignment of the use of sub-program resources with declared GoC and Departmental priorities.

Such an interpretation, however, in the view of the evaluators, misapprehends the important relationship between academic research and commercial research. Academic research institutes are often the incubators of new technologies whose commercial potentials are explored and developed by researchers in the private sector. This is why large private companies often fund academic research and enter into collaborative relationships with the same. Further, academic beneficiaries often bring, through their connections with the commercial private sector, representatives of the same on their missions overseas, thereby possibly understating the sub-program’s actual support to the Canadian commercial private sector.

In terms of alignment with declared sector priorities, roughly 87% of GGI projects over the evaluation reference period (see Table I: Number of Approved Projects by Sector in Appendix 4) relate to declared Departmental sector priorities, namely: environmental science and technology, health and life sciences, natural resources and energy, and ICT. Among these sectors, partnership development activities relating to the health and life sciences sector constituted the largest single sector supported by GGI, followed by environmental science and technology, and natural resources and energy. Again, strong alignment between PDAs supported by GGI and sector priorities is more or less assured given that applications are assessed in reference to these sector priorities.

Evidence also demonstrates a relatively strong alignment between the country partners identified by GGI beneficiaries and Departmental priority markets. On average, at least 77% of the countries identified by GGI beneficiaries for partnership (see Table J: Distribution of Target Regions for Cooperation in Appendix 4) are aligned with the first and second tier countries identified by DFATD as priority markets. This percentage is likely higher given that GGI has supported PDAs in a number of countries in Europe, Latin America and Asia which, though among the top priorities of the Department, were not listed among the top 20 countries of the sub-program. In sum, GGI is consistent with and supportive of GoC and Departmental priorities given its mandate to support international R&D collaborative relationships.

5.3.3 GGI: Federal Roles and Responsibilities

Finding 13: Although support for the formation of international S&T and R&D collaborations is an appropriate role for the federal government and DFATD is in need of a funding instrument to advance its S&T and R&D collaborative objectives, questions remain whether GGI in its current configuration is best abled and positioned to achieve these objectives.

Based on the preceding finding, GGI is in design and operation well aligned with GoC and Departmental priorities, which would suggest that the sub-program is consistent with federal roles and responsibilities. Further evidence that PDAs in support of international S&T and R&D collaboration is an appropriate role for DFATD is the interface and mutual dependence that exists between the sub-program and the TCS community. TCS have relied heavily on GGI as a funding instrument to strengthen bilateral cooperation in S&T and industrial R&D and the sub-program itself has benefited from the knowledge and expertise of TCS in assessing project proposals, assisting in PDA implementation, and in providing post-PDA implementation follow-up.

As remarked earlier, however, the niche which GGI fills among the panoply of federal and provincial funding sources availably to support PDAs is narrow and fragile. The question, therefore, is less whether support for PDAs is an appropriate role for the federal government than whether DFATD is best positioned to administer the sub-program.

5.4 Performance: Achievement of Expected Results

5.4.1 ICCI: Achievement of Expected Results

Finding 14: The centralization of ICCI’s application review process to HQ following the closure of several of DFATD’s ROs resulted in a decline in the quality of applications. This had an initial negative impact on the number of funded ICCI projects, but the situation has since stabilized.

Table 7 (Applications and Project Approvals for ICCI) provides an overview of the number of applications and approvals over the reference period. The information reveals that the number of communities seeking ICCI funding increased by 7.4% during the reference period, averaging around 113 applications per year of which around 70% received full funding, 17% partial funding and 13% no funding. Table 7 further shows that despite an increase in the number of applications received in 2011 there was a fairly dramatic decline in the proportion of applications receiving full funding and a corresponding increase in the proportion of applications receiving partial and no funding. 2012 witnessed an increase in number of applications received as well as an increase in the number securing full funding. The reason for the decline in the number of applications that received partial funding in 2012 (zero) remains unclear.

Prior to 2011 the Department’s network of ROs performed a significant role in the administration of the sub-program, assisting communities to prepare their applications, providing input into the approval process, and monitoring project implementation. Following the closure of some of the department’s ROs in 2011, which saw ROs retained in only 5 of the country’s major cities, along with the centralization of client service at HQ, the sub-program reported seeing a significant deterioration in the quality of applications, thereby resulting in the aforementioned decline in applications receiving full funding and corresponding spike in applications receiving partial or no funding. 2012 however witnessed an increase the number of applications receiving full funding, suggesting that the initial disruption brought about by the Departmental reorganization has stabilized.

Finding 15: Despite registering an overall decline in the volume of outputs and outcomes during the evaluation reference period, ICCI continues to generate positive results.

While BIS maintains a registry of all activities undertaken by beneficiaries in receipt of ICCI funds, it does not consistently record the outputs of these activities or their specific contributions to project outcomes. For example, BIS maintains a record of the number of persons trained in FDI as a result of funding, but it does not record corresponding outputs for research and strategic plan development or the development of marketing and promotional tools. Outputs for these activities are treated as being synonymous with the execution of the activity itself, which finds numeric expression in either the number of communities conducting these activities or the number of projects involved in the same. These numeric values are summarized in Table 8 (ICCI Outputs) below.

As much as can be gleaned from the data in Table 8 is that the volume of activities cum outputs have, but for personnel trained and organizations reached though promotional materials, decreased during the evaluation reference period. This observation is consistent with identified trends described in Finding 13 where Departmental reorganization adversely impacted on the number of approved projects. The observed increase in the number of personnel trained and organizations reached during the evaluation reference period can, at least in part, be explained by the reported increase in the number of projects funded involving training and the development of marketing tools (see Table A: Number of Activities by Activity Type in Appendix 4).

Funded activities all, to varying degrees, contribute to the realization of a set of immediate outcomes, taken from the end-of-project reports that ICCI recipients are required to submit upon completion of a project, which are summarized below in Table 9 (ICCI Immediate Outcomes).

Table 9 shows that the sub-program assisted beneficiaries in identifying over 216,000 potential investors abroad during the evaluation reference period which in turn generated around 12,466 leads and 5,411 prospective investors. Consistent with the observations made in reference to Table 8 (ICCI Outputs), Table 9 reveals a decline in the number of reported leads and prospects. Again, this trend can in part be attributed to the challenges the sub-program confronted following Departmental re-organization. This said, the number of foreign companies reported to have expressed an interest in investing in communities supported by ICCI at the conclusion of a project was 14, up from 10 reported in 2010.

As remarked earlier, GCSP administered two successive questionnaires to sub-program beneficiaries, the first targeted at beneficiaries in FY 2009-10 and the second targeted to beneficiaries in FY 2010-11, with the view to ascertaining the longer term impact of funded activities on the same. The response rates for these two questionnaires as they pertain to ICCI beneficiaries were 41% for 2009-10 and 39% for 2010-11. Table 10 (Questionnaire Results – Improved Capacity) below summarize the responses of ICCI beneficiaries to questions posed with respect to investment readiness.

Results from the GCSP questionnaires demonstrate that there have been improvements in capacity among the majority of ICCI beneficiaries as a result of the sub-program. As presented in Table 10, at least 93% of both 2009-10 and 2010-11 beneficiaries report that their capacity to serve investors had improved to either a great or moderate extent ^{Footnote 64} as a result of having participated in the sub-program. Similarly, roughly 92% of respondents declared that funded activities had improved their understanding of how to attract investors, with at least 90% of the same declaring that participation in the program had improved their readiness to respond to investment inquiries in a timely manner. Further, at least 86% of respondents declared that funded activities had enhanced partnerships within and across communities, a practice which is encouraged by the sub-program, and at least 87% reported increased knowledge of investment attraction opportunities.

Respondents frequently cited the value of funds made available through ICCI to conduct research on community assets (strengths) as well as liabilities (weaknesses), which is a critical component in the development of any effective promotion strategy which aims to market a community as an attractive FDI destination. Funds made available through the sub-program to conduct research on prospective FDI partners abroad was also cited as highly valued, not only for the purpose of lead identification, but also to calibrate marketing approaches and tools to reach the same. Funds made available through the sub-program to provide training in FDI was additionally cited by respondents as an important enabler, allowing communities to better serve prospective investors.

In terms of outcomes generated, Table 11 (Questionnaire Results – ICCI Outcomes) below shows that funding recipients report positive impacts on their community’s FDI as a result of ICCI. 74% of 2009 beneficiaries and 65% of 2010 beneficiaries reported that funded activities resulted, either directly or indirectly, in the generation of FDI leads. Participation in ICCI also contributed to the identification of prospective investors for their communities among 84% beneficiaries in 2009 and 73% in 2010. These investors were “serious,” meaning they had advanced information requests, site visits or actual investment proposals.

Further, funded activities also improved FDI capacity among Canadian communities. 58% of 2009 beneficiaries declared that ICCI projects had contributed to increased ability of their community to expand FDI with 45% declaring that participation in the sub-program had helped their capacity for FDI retention.  Although data are not available for FDI expansion, more than half of 2010 recipients report increased capacity for FDI retention as a result of their participation in ICCI. 76% and 62% of projects funded in 2009 and 2010, respectively, resulted in the creation of new FDI approximately three years after project completion and, among these projects, the majority in 2009 resulted in job creation and/or retention for Canadians. Approximately one-fourth of beneficiaries reported the creation or retention of between 1 to 25 jobs in their communities as a result of ICCI funded activities, with between 26 to 100 jobs created or retained among 41% of 2009 beneficiaries and more than 100 jobs created or retained among 17%. These results illustrate the ways in which ICCI-funded projects have contributed to Canadian communities’ ongoing improved capacity for FDI and their positive impacts on employment.

Notable examples of projects funded by ICCI cited by questionnaire respondents which contributed to both increased FDI in their communities and job creation included: five investments in the Greater Toronto Area (GTA) valued at $800 million which was reported to have created nearly 850 new jobs in the area; two investments in the Niagara region of Ontario valued at $25 million which was reported to have created around 300 jobs; a $10 million investment, also in southern Ontario, which was reported to have created 280 new jobs for that community, and a $130 million investment in Quebec, with an undisclosed number of jobs retained and or created.

In sum, despite overall declines in the volume of outputs and outcomes generated by ICCI during the evaluation reference period, the sub-program has produced impressive results. Clearly beneficiaries highly value the sub-program, evidenced by the positive responses to questions relating to institutional capacity and the positive responses to questions relating to the number of potential investors reached and in expressing an interest in investing in their communities. While the actual number of investments undertaken which could be attributed to ICCI may in the low double digits, it is important to bear in mind that even one investment can be valued in the tens of millions of dollars, as demonstrated by the examples cited in the preceding paragraph.     

5.4.2 GOA: Achievement of Expected Results

Finding 16: While the majority of GOA projects were completed, incomplete activities within each package can account for the sub-program’s funding lapses.    

GOA project proposals are organized around activities and packages where each applicant may apply for up to five activities per package and up to five packages per application. As shown in Table 5 (Financial Information by Sub-Program) in Section 1.5, GOA de-committed on average 36.9% of its programming during the evaluation reference period and lapsed, on average, 28.9% of its annual funding allocation. This is in spite of a minimum completion rate of 90% for approved project packages in 2011-12 and 2012-13, as illustrated in Table 12 (Profile of GOA Applications) below. While the completion rate for approved GOA projects was over 93% on average across all years, individual activities within each package did not proceed as planned with 44.4% of proposed activities reportedly not completed, which appear to account for most of the lapsed GOA funds.

Finding 17: Results measuring the achievement of longer-term outcomes indicate that GOA is contributing to increased IBD capacity among Canadian SMEs.

GOA outputs focus on the reach of funded activities and deliverables across GOA’s three types of activities: Promotional and Marketing Tools and Materials, Direct Contacts and Research. As described in Section 1.7.2, funding for Promotional and Marketing Tools and Materials was directed to the creation of materials such as brochures, pamphlets and/or CD-ROMs and DVDs and the development of new or improvements to existing websites and/or web tools. Direct Contacts refer to funded activities that facilitate face-to-face interactions, such as missions to Canada by potential foreign partners or to foreign markets by Canadian companies, participation at exhibitions, targeted meetings or one-on-one matchmaking events and seminars or information sessions conducted by the association. Research activities refer to the development of market studies or intelligence by the association.

The three types of activities that are eligible for funding are presented in Table 13 (Approved and Completed Activities by Type) below according to the number of approved and completed activities, and their respective completion rates for 2011-12 and 2012-13.^{Footnote 65} These distributions demonstrate that, for both fiscal years, the majority of completed GOA activities (at least 80%) involved direct contacts such as trade shows, networking functions, meetings and market visits and that a higher percentage of these direct contact activities were completed successfully compared to the other two activity types. The percent completion rates for funded activities that facilitate face-to-face contact with potential IBD partners (68.7% in 2011-12 and 52.3% in 2012-13) were higher compared to rates for the development of promotional tools or research (approximately 50% completion for both activities in 2011-12 and approximately 10-15% completion in 2012-13).

Project results for GOA demonstrate that funding for these three activity types contributed to positive outputs for Canada. Program documents indicate that more than 64,000 entities (both individual users and association member companies) were reached in total over the two reference years as a result of these materials and tools. These tools include the creation of new or improvements to existing web tools or websites as well as the creation of promotional or marketing materials such as brochures, pamphlets, CD-ROMs and/or DVDs. Similarly for funded research activities, GOA recipients report that more than 74,000 industry personnel (likely association member companies) were reached through the market studies or market intelligence reports that were produced. These results suggest that GOA contributed to avenues for increased awareness among Canadian SMEs and their foreign partners on IBD opportunities. However, discussions with GOA program administrators suggest that the usefulness of these products and/or their distribution is changing with advances in technology and the ways in which associations and their foreign partners conduct business. Associations are less likely to produce hard copies of promotional or marketing tools or reports, relying more on relevant information or electronic versions of market reports posted on association websites to increase awareness and promote IBD. This may create greater challenges in tracking the impact of GOA funded activities for stakeholders without monitoring the number of users to websites or the number of downloads of reports.

Overall, direct contact events and their associated activities (including the distribution of materials at these events and the creation of market intelligence or after-trip reports) results in a total of more than 96,000 connections with target entities.^{Footnote 66} In addition, these direct contact activities may have greater potential for developing new or strengthening existing relationships with foreign partners compared to the other two activity types. End-of-project results indicate that direct contact activities facilitated interactions between Canadians and potential foreign partners, with approximately three foreign participants reached for every Canadian participant at direct contact events (approximately 22,000 Canadian participants interacted with 75,000 foreign participants for both fiscal years). Table 14 (Number of Canadian and Foreign Participants at Direct Contact Events) below illustrates the number of Canadian and foreign participants that interacted at these direct contact events.

As shown in Table 14, exhibitions, networking events and targeted meetings account for the highest number of funded direct contact events compared to the other event types. Exhibiting events refer to recipient associations’ participation at annual industry events such as trade shows. Interviews with some funding recipients demonstrate that GOA contributes to increases in Canada’s profile and improvements to Canada’s branding at these trade shows. Some associations use GOA funding to develop a Canadian pavilion at these events, inviting member companies to set up their own exhibitions at their own expense under this Canadian banner.

According to funding recipients, potential foreign partners are able to identify Canadian SMEs in one area of the trade show, under a common Canadian banner, thereby strengthening Canada’s presence among industry stakeholders. Some of the GOA funding may also be used to create meeting space for Canadian SMEs to meet one-on-one with potential foreign partners to establish or solidify formal agreements for foreign investments to Canada, meeting space that Canadian SMEs would have otherwise had to rent from trade show organizers. Networking events may include hall rental fees for receptions as part of a trade show or site visit, sometimes facilitated by trade commissioners at mission, where Canadian SMEs may meet and greet interested foreign industry officers in an informal setting. Targeted meetings are more formal gatherings between association officers, representatives from their member companies and identified foreign ministries or companies. Some recipient associations report that these targeted meetings are likely to produce greater interest among potential foreign investors because the discussion is purposeful towards strengthening ties with Canadian industry, the larger contingent travelling from Canada for the meeting signals a serious interest from the Canadian SMEs and the Canadian mission is typically involved in facilitating these meetings.

In terms of contact with potential foreign partners, larger events (e.g. exhibitions/trade shows, networking events) appear to broaden the network of Canadian SMEs with Canadian participants meeting with a higher number of foreign participants. These direct contact events, however, vary in purpose and in their likelihood to result in expanded IBD for Canadian SMEs. As well, these categories for direct contact events are not mutually exclusive. For example, incoming or outgoing missions may be captured under site visits and the difference between one-on-one meetings and targeted meetings is not apparent. As a result, recipients may be confused in how to report their project outcomes and GOA’s project results data may not be accurate in capturing its performance story. GOA program officers are aware of these discrepancies and report concern that the data do not best reflect project outcomes. They note that understaffing creates challenges in addressing these issues sufficiently.

Higher level program outcomes for GOA are presented in Table 15 (Immediate Outcomes for GOA). For both reference years, Direct Contacts generated the highest number of leads and resulted in the highest number of Canadian companies that expanded their IBD efforts compared to other GOA activities. Promotional or Marketing Tools and Materials resulted in the highest number of potential foreign partners identified in 2012-13 while Direct Contacts resulted in the same for 2012-13. Research activities for GOA resulted in the fewest immediate outcomes compared to other GOA activities for both FY.

GOA recipient questionnaire results are available for 2009-10 and 2010-11. Because performance information is not available for 2009-10 or 2010-11 recipients, the questionnaire report on the achievement of longer-term outcomes for different cohorts of GOA recipients and a direct link between GOA’s performance data and the GCSP questionnaire results cannot be established. General observations, however, may be made on progress towards longer-term outcomes for GOA based on questionnaire results.

The questionnaire results for GOA indicate that funded activities have contributed to improved knowledge of IBD opportunities and capacity to conduct IBD for Canadian participants (or companies) and for industry associations in general. As described in Table 16 (Questionnaire Results – Improved Capacity), at least 95% of respondents report that the knowledge of and capacity for international business among Canadian companies improved to a moderate or great extent across both fiscal years. Questionnaire respondents note that GOA funding contributed to educating Canadian SMEs on the potential of foreign markets, introducing Canadian companies to new foreign contacts and, in some cases, facilitating expansion to foreign markets.

Similarly, the majority of questionnaire respondents (at least 80%) report improvements in the association’s knowledge of and capacity for international business as a result of GOA funded activities. Associations’ participation at international events or in outgoing missions helped to refine their international market development targets, for example, and strengthen their industry’s profile abroad. However, almost 15% of 2009-10 questionnaire respondents report that GOA projects did not contribute at all to improvements in their association’s ability to conduct international business. Questionnaire data are inconclusive on the context behind these results for the association but evidence suggests that GOA funded projects have contributed to improved capacity within their respective industries. More than 95% of both 2009-10 and 2010-11 questionnaire respondents report that their funded projects contributed to their industry’s ability to market products or services in foreign markets and over 90% report that more Canadian companies within their industry are involved in international business as a result of GOA projects.^{Footnote 67} 82% of survey respondents also report that funded projects directly or indirectly resulted in foreign sales or contracts,^{Footnote 68} identifying sales in the German, Italian, Chinese and US foreign markets, for example, which were facilitated by GOA-funded projects.

Questionnaire results also reveal some industry associations’ success stories for their SME members funded through GOA. Canadian businesses in the Arts and Culture industry report that GOA contributed to improved marketability of their products abroad, resulting in 97 finalized and 229 initiated business deals with foreign partners for an estimated potential of $2.7 million. Sales resulting from GOA-funded delegations abroad were anticipated to be greater than $121 million for defense and securities businesses, with one SME alone making $12 million in immediate sales as a direct outcome of its participation. In the automotive sector, Mexico will contract over $16 million with Canadian SMEs as a direct result of the association’s participation in trade shows. In the renewable energy sector, missions to China resulted in a partnership between an organization in BC and Beijing to develop the largest biofuel pellet plant in Canada for the export of materials to Europe and Asia. In the education sector, a GOA-funded mission to India resulted in the signing of several institutional MOUs between Indian and Canadian universities to support student exchange, research cooperation and joint academic programming. These examples illustrate some of the impacts of GOA on Canadian businesses in priority markets.

In sum, despite ongoing challenges to disburse its allocated funds, GOA has managed to produce positive results. Performance data shows that the sub-program has significantly increased Canadian SME awareness of business opportunities abroad, increased their level of participation in IBD, and helped the same reach tens of thousands of prospective business partners. That over 80% of respondents to the GOA questionnaire reported that participation in the sub-program resulted in foreign sales and contracts valued in the hundreds of millions of dollars provides evidence of the benefits of the sub-program.

5.4.3 GGI: Achievement of Expected Results

Finding 18: GGI registers a high rate of non-completions, accounting for around a third of approved projects. Although this rate is reported to be unavoidable and normal given uncertainties relating to the securing of matching funds and changes in project plans, increased rigor in the up-front assessment of proposals could have reduced these rates.

As remarked in Section 1.7.3 of this report, and illustrated in Table 17 (Status of Adjudication Process for GGI Applications) below, the number of GGI applicants has remained relatively constant over the evaluation reference period at around 170 applications on average per year, with the percentage of completed projects averaging around 64% of approved projects across the three reference years. The proportion of GGI approved projects that were cancelled was around 35% on average, which resulted in a significant de-committing of funds (see Table 5: Financial Information by Sub-Program).

The high rates of non-completions under GGI are reported to be due to challenges applicants face in securing matching funds or to unanticipated changes in plans, which automatically results in a cancellation. These changes can be as minor as a modification to the dates for the proposed mission overseas or something as dramatic as a withdrawal of interest on the part of one or all of the prospective partners. Such changes are deemed by program administrators to be unavoidable and therefore a normal occurrence. However, the frequency and persistence of high rates of cancellations and de-commitments may have been alleviated through a greater level of rigor in the up-front assessment of proposals.

Finding 19: Cases of commercialization of R&D initiatives borne out of GGI supported collaborations are few in number. This said, the sub-program has succeeded to a significant degree in achieving its immediate objective, which has been to support the formation of international S&T and industrial R&D partnerships.

GGI provides “early-stage financing” to Canadian private sector and academic researchers to assist them in forging and strengthening international partnership arrangements with their foreign counterparts for the purpose of establishing future R&D collaborative initiatives. As such, GGI outputs and immediate outcomes find expression in the number of agreements generated by funded activities, which GGI beneficiaries are required to document in their end-of-project reports. Table 18 (Number of Agreements Forged by Type) below summarizes the number of agreements by type reported in these end-of-project reports.

Data provided in Table 18 shows that over the evaluation reference period there was an 8.7% increase in reported agreements forged between sub-program beneficiaries and foreign partners. In total, around 503 agreements of one form or another were secured over the evaluation reference period, roughly translating into average of two agreements per applicant. The most common agreements forged immediately after the conclusion of a project were NDAs, followed by MOUs, LOIs, student training/exchange, and JVAs.

As with the other sub-programs of the GCSP, GGI recipients were asked to respond to two on-line questionnaires to gauge the longer term impacts of funded activities. Again, the target populations for these two questionnaires were recipients of GGI funding in FY 2009-10 and 2010-11, with the corresponding response rates being 46% for the former and 50% for the latter. Table 19 (Questionnaire Results – Improved Capacity) below summarizes the responses to questions posed relating to the impact of funded activities on recipient capacity.

The results presented in Table 19 shows that GGI funded activities had positive impacts for beneficiaries for most recipients. Knowledge of international collaborative R&D opportunities increased among 90% and almost 80% of 2009-10 and 2010-11 beneficiaries, respectively.^{Footnote 69} Likewise, at least 94% of both 2009-10 and 2010-11 recipients reported that their international R&D business or commercial objectives were advanced through GGI. Respondents remarked that participation in the sub-program had contributed to a better understanding of the key persons and organizations active in a particular industry or area of research, which served to both increase their understanding of R&D opportunities with foreign partners and their ability to conduct collaborative R&D with the same.

A notable exception to this otherwise positive picture is percentage of respondents (13% in 2009-10 and 24% in 2010-11) who reported that participation in GGI had no impact on their ability to conduct international R&D. Evidence is not clear why this was observed but some comments from GGI recipients suggest that increased capability to conduct international R&D is a higher order outcome that may be observed after the partnerships fostered through the GGI become solidified. Among 2009-10 beneficiaries that reported a R&D project did not result from the GGI-funded activity, almost 16% reported that the relationship needed to be further developed before forming an innovation partnership. When asked to identify the primary cause of an unsuccessful R&D partnership, roughly a quarter of respondents identified the lack of funding as the primary cause.

Further to the generality of the foregoing, former beneficiaries of GGI were asked in the questionnaire to identify among a list of sub-program outputs or outcomes which could be directly attributed to the sub-program two years following project completion. The results for the two questionnaires are summarized in Table 20 (Questionnaire Results – Outcomes) below. As respondents could identify more than one agreement arising out of a single project, the percentages presented in Table 20 do not add up to 100 percent.

Data provided in Table 20 shows that, despite a decline in the number of collaborative R&D projects attributed to the sub-program in 2010-11, more than half of the questionnaire respondents on average for the two years claimed that participation in the sub-program resulted in ongoing collaborative projects. This said, over 90% of respondents for both years claimed that their participation in the sub-program had resulted in the forging of an agreement of one form or another. Again, in descending order, the most common forms of agreement were MOUs, knowledge exchange/technology transfers, exchange of resources, NDAs, joint publications, R&D Partnership agreements, and LOIs.

In addition to the foregoing, 46% of respondents in 2009-10 and 53% in 2010-11 declared having developed one or more technologies and/or products with their foreign partners since participating in the sub-program. Examples cited include, but are not limited to, the development of: a new blood test to diagnose rheumatoid arthritis; a welding simulator for training purposes; an alternative form of pasteurization using electric fields; Integrated Test Laboratory (ITL) equipment used to analyze stresses in steel; a new Liquid-Crystal Display (LCD) panel technology for interactive boards to be used in training; a new methane purification system; and, the development of un-manned drone to assist fire fighters. Further, a number of respondents declared that participation in the sub-program, and the resulting collaborations that ensue, allowed them to secure research grants valued in the tens of millions of dollars. Examples include a project to develop new carbon capture technology which received a $25 million grant from the European Union (EU), a project to develop new drugs using cell and molecular materials which received a $1.5 million grant and a project to develop biochemical reactors (new proteins) for therapeutic purposes which received $400,000 in grants.

In sum, though evidence of commercialization of R&D initiatives borne out of GGI supported collaborations is few in number and largely anecdotal, the sub-program has succeeded to a significant degree in achieving its immediate objective, which has been to support the formation of international S&T and industrial R&D partnerships. Beyond the actual formation of partnerships perhaps the most meaningful indicator of the strength of these collaborations and their prospects for success is the extent to which partnership has assisted the parties involved to secure financial support for their projects from other sources. GGI did not systematically track and record this immediate outcome, thereby contributing to an incomplete performance story. This said, anecdotal evidence gleaned from reported “success stories” demonstrates that GGI early investments in partnership development have been leveraged to secure additional funding for R&D estimated in the tens of millions of dollars. 

5.5 Performance: Demonstration of Efficiency and Economy

5.5.1 Governance

Finding 20: Mechanisms and processes for decision-making have been implemented and Steering Committee meetings have been regularized to provide timely strategic guidance to the GCSP. Briefings and consultations were also conducted with senior management as required to facilitate informed decision-making.

A governance structure was developed in 2009-10 and updated in 2013-14. In this governance structure, the DG Steering Committee, the GCSP Director Level meetings and the Deputy Directors Working Group serve as mechanisms for information sharing and discussion of issues and concerns. Any issues requiring decision are to be forwarded from the GCSP sub-programs for discussion at Directors Meetings, up to the DG Steering Committee and finally to the ADM, as required. Issues that arise between quarterly Steering Committee meetings requiring immediate attention or decision are to be brought forth to responsible DGs via email by the GCSP Project/Financial Manager.

However, a review of the CAU’s documentation on DG Steering Committee meetings indicated that Steering Committee meetings were not convened quarterly in 2009-10 as planned and met only once in 2010-11. Records of Decision (ROD) or Meeting Minutes were not consistently prepared following these meetings. Therefore, it is difficult to document and track the strategic decisions that were made for the GCSP during these early years. The Office of the Chief Audit Executive at DFATD made similar observations in its Early Implementation Audit of the Global Commerce Support Program (2011), recommending that the program “reinvigorate [its] governance structure to strengthen senior management oversight and tracking of key priorities, initiatives, and operating expenditures.”

As a result, formal Steering Committee meetings were re-implemented in May 2011 to convene quarterly as prescribed in the GCSP’s ARAF. Participation at these Steering Committee meetings consisted of the two GCSP DGs, their responsible Directors and the respective Deputy Directors, which permitted a more fulsome discussion on program delivery and administration to help inform decision-making. A deck was prepared for each meeting by CAU with input from the GCSP sub-programs that provided a meeting agenda and related materials for discussion. Financial updates for the quarter were always presented by the GCSP Financial Manager and funding allocations or re-allocations across the sub-programs were discussed for decision at each meeting, including risk mitigation strategies for identified funding lapses.

Discussion at Steering Committee meetings also typically included GCSP performance tracking and reporting (e.g. GCSP Annual Performance Report), operational concerns (e.g. HR challenges, IT requirements), corporate issues (e.g. impact of Strategic Operational Review) and other issues requiring review or decision. Records of Decision (ROD) were drafted for each meeting and approved at subsequent sessions, including action items clearly outlining specific tasks for follow-up and areas of ongoing business that required resolution. However, the GCSP’s governance structure notes that quarterly reports would be forwarded to the ADM for information or for decision. There is no supporting evidence that these quarterly reports were produced but program documents demonstrate that memos to the ADM were prepared when approvals were required for key issues.

Consultations and briefings were also provided to senior management at key points during program delivery. For example, the ADM of International Business Development and Innovation was briefed and updated as the sub-programs of the GCSP were being harmonized and when changes were required as the program was being rolled out. As well, there is documented evidence that briefings were provided to new BID and BBD DGs to expand their knowledge of the GSCP for improved strategic decision-making in their new roles.

Finding 21: There are mechanisms in place for sufficient coordination and oversight of the GCSP overall. The CAU is important in its coordination function as it facilitates ongoing communication, information exchange and in moving GCSP operations forward.

According to the ARAF, Directors Meetings were to be convened monthly to address operational concerns. Program documents indicate that these meetings were launched but they were inconsistent during the early years of the GCSP. Similar to the Steering Committee meetings, there are six documented meetings in 2009-10 and records exist for only one meeting for all of 2010-11, and decisions made at the director level and actioned items were not always documented. The regularity of meetings and tracking of decisions also improved following the program audit that was completed in 2011. Directors Meetings were held every month in 2011-12 except over the summer and in December, and convened roughly every three months over 2012-13 and 2013-14. Each of these meetings was attended by the Directors and Deputy Directors for each of the GCSP sub-programs and Records of Decision (RODs) were created following each session. There are no records for meetings during the first quarter of 2014-15 from April to July, however.

Discussion at Directors Meetings focused primarily on operational issues including, for example,

• Updates on funding commitments and mitigation strategies for potential lapses;
• Program administration (e.g. Calls for applications);
• Required performance tools (e.g. Information Technology (IT) system for components and related client confidentiality issues);
• Reporting to senior management (e.g. Office of the Minister [OMINT] reports);
• Corporate reporting (e.g. GCSP Annual Performance Report, RPP, DPR); and
• HR capacity.

These meetings provided an opportunity for each responsible division to monitor ongoing program spending and adjust funding allocations across the GCSP to manage the potential lapsing of funds throughout the fiscal year. Meetings also provided an opportunity for the coordination of required tasks for ongoing operations (e.g. GOA program management assigned to analyze alignment of GOA to calendar year) and improved communications up to senior management (e.g. CAU was to revise the Memo to the ADM regarding budget allocations). Evidence suggests that these Directors Meetings have improved the coordination of the GCSP across its sub-programs and oversight for the program as a whole, with relevant items being brought forth for decision to the DG-level at the Steering Committee meetings and to the ADM as required.

The benefit of regular Directors meetings is undoubtedly that the three sub-programs of the GCSP can address concerns in a timely fashion and maintain accountability for assigned operational tasks until they are completed. As well, monthly meetings can help to strengthen the integration and level of engagement of each sub-program into the GCSP as a whole rather than working in silos. This said it is debatable whether the Directors Meeting needs to be convened each month. For example, monthly financial updates presented at Directors meetings from September to November 2012 indicate only a 0.58% change in GCSP funding commitments across these three months. Minutes from Directors Meetings also suggest that most operational business require more than one months’ time to reach resolution and may have been sufficiently addressed without face-to-face contact.

Input into the GCSP Annual Report, for example, was identified as an action item across multiple months but assigned tasks likely could have been addressed secretarially without ongoing discussion every month. Further, meetings were convened every three months in 2013-14 without extensive detriment and discussions on key issues were addressed at quarterly Steering Committee meetings. Directors Meetings held every second or third month, therefore, may be sufficient to address operational concerns as long as lines of communication remain open across responsible directors and their deputies for informal discussion between meetings and as long as there is diligent follow-up on action items from previous meetings to ensure assigned tasks are fully addressed.

The work of the CAU and the GCSP Financial/Project manager has been paramount for the GCSP given the high administration and coordination required across the three sub-programs. Program documents demonstrate that the CAU, particularly the Financial Manager, has been vital in fostering and maintaining financial oversight over the GCSP through financial monitoring and providing updates to directors and senior management on potential risks for lapsing funds for discussion and decision. The CAU is also responsible for operational management of the GCSP. There is evidence that a GCSP Program Manager coordinated operational issues for the program during early implementation, as described in the governance structure. This responsibility, however, was moved to the Financial Manager so that the CAU now acts as the GCSP secretariat in addition to its financial management responsibilities, convening and coordinating both the Director-level and Steering Committee meetings as well as documenting decisions and tasking through RODs and Minutes. This shift in the coordination function is logical since financial and operational management are closely linked but adds to the work pressures for CAU staff. Evidence from RODs demonstrates that the resource issues for CAU were presented to the Steering Committee but a definitive solution was not apparent.

Program documents indicate that a Deputy Director (DD) Working Group was established and held its first meeting on October 17, 2011. Meetings are called as needs are identified at Director Meetings. RODs from the Directors Meetings indicate that the DD Working Group was tasked with specific actions requiring input from all components, such as the development of GCSP tools (e.g. for tracking of overall GCSP operating and salary budgets), creation of privacy statements on use of personal information collected through GCSP and designing of standard surveys to capture longer-term GCSP outcomes. There are no records of decisions or minutes from these DD working group meetings but there is some evidence of the deputy directors reporting at the Directors Meetings on progress on action items. This seems sufficient since there is a mechanism for follow-up on DD Working Group activities and additional administrative burden on GCSP teams should be avoided.

Finding 22: The roles and responsibilities are delineated in the governance structure for the program and communicated appropriately to all levels. Accountabilities for these roles and responsibilities are maintained appropriately through the delegation of authorities to ICCI, GOA and GGI managers and through consultations with COE as required on issues pertaining to G&Cs authorities. Due diligence was also observed through regular recipient audits of funded projects that were reviewed by senior management.

The roles and responsibilities for the GCSP have been clearly outlined in the ARAF and further elaborated through the program’s approved governance structure. This governance structure presents the authority and accountabilities for the program within the departmental context. As noted in Section 1.6 of this report, responsibility for program development, implementation and administration at an operational level has been delegated from the DM of International Trade (DMT) to the ADM of the International Business Development and Innovation Branch (BFM). Further responsibility and accountability was delegated to the DGs of Investment and Innovation and Global Business Opportunities and resources to administer the program were allocated to divisions under the respective DG Bureaus. The responsibility and accountability for each of the GCSP sub-programs rests with the Directors of respective divisions. Specifically, the ICCI is administered by the Investor Services Division, the GGI by the Science, Technology and Innovation Division and the GOA by the Multi-Sector Practices Division.

The ADM of BFM is responsible for overall management, priority setting and fund allocation for the GCSP, in consultation with the DGs of the responsible Bureaus. The Bureau DGs are responsible for the implementation of their respective program components in accordance with the GCSP’s T&Cs and ARAF, setting annual program targets and specific funding allocations for each component in addition to the monitoring and assessments of project results to ensure consistent alignment with Bureau mandates. Divisions are responsible for program implementation including coordination and delivery, risk management, assessment of project progress and achieved results as well as policy control and coherence with program policy objectives. The CAU provides support to each sub-program and is responsible for the overall financial management, administration, control and monitoring of the GCSP in addition to managing corporate reporting requirements. These roles and responsibilities are described in the TOR for the DG Steering Committee and clearly communicated in briefing documents to senior management. There is also demonstrated evidence that meetings were coordinated with incoming DGs and ADMs to provide background, context and updates on the GCSP and its performance to date.

Each sub-program of the GCSP has its own application evaluation process and adjudication structures to assess submitted project proposals. Although formalized TORs for committee members of each component were not evident, interviewees report that they are aware of their roles and responsibilities because they have been involved in the respective components for many years. Overall, these adjudication processes are well-defined and well-documented, with some noted areas for potential improvements. For ICCI, external adjudication board members noted the importance of having broad regional representation on the evaluation committee to provide authoritative input on the local context when evaluating submissions. Some board members reported that the completed ICCI application forms contained sufficient details on communities’ capacities for informed decision making while others noted that formal metrics on applicants’ past performance would have enhanced the process.

For GOA, interlocutors noted that the adjudication process is appropriate with the right groups involved to provide a range of perspectives on proposal submissions. Some improvements could be gained through involvement from OGDs such as Industry Canada for input on small businesses or NRC to speak to multi-sectors, although it was acknowledged these views could be provided through internal resources at DFATD. Some interlocutors also noted that the ADM of BFM now chairs GOA’s adjudication committee and questioned the need for high level senior management to be involved in operational sub-program activities. As regard to GGI, the adjudication of project proposals was largely undertaken by personnel within DFATD, though the counsel of subject matter experts external to the Department was reportedly sought as required. Some interlocutors interviewed in support of the evaluation remarked that the adjudication of project proposals under GGI, given their technical nature, could have been enhanced with the more regularized input from subject matter experts.

Other officers also have had roles in facilitating the implementation of the GCSP sub-programs. For example, Trade Commissioners in the ROs across Canada implemented the ICCI until the program was repatriated to HQ in 2011. Interviews with Trade Commissioners who were responsible for the ICCI suggest that there is a role for the Trade Commissioners in providing advice and guidance in the field to Canadian communities that wish to grow globally. Regional development agencies such as ACOA and WD were identified as possible sources of assistance but their involvement in the ICCI beyond the adjudication process has not been clearly defined. With the shift in responsibility for ICCI program delivery to HQ, the sub-program has diminished capacity to leverage existing relationships with Canadian communities in the field to understand and balance the risks for potential projects against long-term benefits for larger returns on investments for Canada. Similar remarks were made with respect to the role of TCS personnel at ROs in the administration of GGI.

Sector officers act as program administrators for GOA. When applications are received, the industry associations are identified and applications are distributed to the appropriate Directors of sector practices. Sector Directors then assign the management of these applications to available sector officers within their divisions. The associations that are assigned to sector officers are not always aligned with their sector of focus such that, for example, officers responsible for the automotive industry may be assigned to manage a GOA application from an association in ICT. This misalignment is reportedly unavoidable during some years due to resource constraints across the sector practice divisions. In these situations, the strong links of sector officers to their industry associations and their understanding of associations’ needs are not maximized for the benefit of the GOA program. As well, the GOA file is only one of many responsibilities among sector officers. The required management of competing priorities becomes particularly challenging during the application and adjudication process when GOA demands 80-90% of sector officers’ time, depending on the number and needs of the applicant association. In such circumstances, sector officers reported feeling torn between their responsibilities to their respective Sector Directors, to whom they are directly accountable, and their responsibilities to BBI (GOA), to whom they are not directly accountable.

As described in Section 1.7.2, the sector officers’ role is to contribute their expertise on industry sectors as they liaise with associations to help strengthen GOA applications and manage funded projects. However, much of this work involves liaising between applicants and the GOA program administrators on administrative details. A proportion of this interaction is related to applicants’ problems with the GOA online administrative system, which most agree is not the best use of GOA resources. Sector officers also report that they sometimes receive challenging questions from applicants on G&Cs or GOA itself that are beyond their expertise or accountabilities. GOA provides excellent training opportunities and resource materials but these do not address all the complex concerns that applicants direct to sector officers and officers are not comfortable with responding to questions that may put themselves and the Department at risk. As a result, many sector officers contend that their responsibilities should focus on providing advice and recommendations to industry associations and that duties relating to the administration of GOA ought to be transferred to the CAU, where the expertise is thought to reside. This said were the administration of the program to be centralized, the sector officers could find themselves at risk of losing the connection and influence they have with the associations by virtue of the funds currently at their disposal.

Beyond the responsibilities and accountabilities of sector officers, the GCSP has been practicing due diligence to ensure that accountabilities are observed in program delivery. Program documents suggest that there was a lack of clarity around delegated signing authorities in 2010-11 after the three sub-programs were amalgamated. With harmonization under the GCSP, neither the submission nor the approved T&Cs refer specifically to “delegated signing authorities,” and neither document stipulates the continuance of or changes to prior delegated authorities for each component. A review of program documents indicated that the CAU liaised with DFATD’s Corporate Accounting Bureau and the G&Cs Centre of Expertise (COE) for advice and to obtain delegated authorities for the DGs, Directors, Deputy Directors for each component under GCSP and STC in the ROs serving ICCI. Interview data suggests that responsibilities for ICCI shifted to program officers at HQ with the closure of ROs but there seemed to be little impact on sub-program delivery since authorities were already delegated to the ICCI Director and Deputy Director.

Program documents also demonstrate that recipient audits are conducted annually on GCSP projects as required. The CAU is the program lead on recipient audit activity to facilitate, monitor and report results to GCSP senior management. Each year, the CAU with the help of sub-programs, coordinates the completion of a Recipient Audit Risk Assessment Profile (RARAP) by the ICCI, GOA and GGI Deputy Directors to recommend funded projects to be audited. The RARAP assists the Recipient Audit Division to develop an annual recipient audit plan that identifies those GCSP projects to be audited, including cost-estimates and timelines for completion. The CAU maintains an ongoing recipient audit tracking sheet that monitors progress on recipient audits for each program component as well as providing a summary of audit results and recommendations upon completion. Responses from recipients on audit recommendations are also recorded as are auditors’ responses to these comments, as applicable. Audit results, recommendations and follow-up actions are presented to senior management at Steering Committee meetings. In this way, the GCSP can maintain accountability in accordance with the T&Cs of the Contribution Agreement.

5.5.2 Program Planning and Delivery

Finding 23: Program planning and administration procedures are well documented, however implementation has been hampered by IT infrastructure challenges and exogenous factors which have delayed sub-program launch and the release of application review results, particularly with respect to GOA. This has adversely impacted on performance.

All three sub-programs of the GCSP operate under different planning processes and cycles which must be addressed separately.

ICCI

As remarked in Section 1.7.1 of this report, ICCI’s 14 to 15 week application and review cycle begins in mid-September and concludes following the review of proposals with an announcement of the results around the first week of January the next year. In this process, ROs used to be the first and last point of contact for municipalities in sub-program delivery until administration thereof was centralized at HQ. With regards to the ROs, reference has already been made to the important role they performed in assisting communities prepare their proposals as well as providing valued intelligence on the same in support of the application review and adjudication process, contributions which were more or less lost with centralization. The diminished role performed by the ROs in the application process relating to ICCI along with the adoption of the VTC web-based portal in 2011 was reported to have resulted in a significant reduction in both the number and quality of applications, which ultimately found expression in a sharp increase in the number of rejections and partially approved projects. This said, centralization is reported to have contributed to greater consistency in the application of assessment criteria. As regard to the adoption of the on-line administrative system, this not only challenged applicants but also program administrators who, by default, found themselves increasingly performing an IT support function. Additionally, ICCI beneficiaries interviewed in support of this evaluation, while referencing the issues cited above, conveyed a high degree of satisfaction with current application process, which gives them ample time to plan and execute their activities.

GOA

The current planning cycle for GOA (described in Section 1.7.2 of this report) has presented significant challenges to both program administrators and beneficiaries.  Compared to the other sub-programs of the GCSP, the application, review, and approval process for GOA is longer at around 16 to 17 weeks. This includes a 6 to 7 week period where applicants may discuss draft proposals with assigned sector officers. Because this period straddles the holiday season (December and January) in practice consultations with applicants often occur close to the deadline in January. In addition, proposed project activities relate to a plurality of sectors and involve, for the most part, participation in networking events abroad that require engagement from a wider range of stakeholders.

Notwithstanding the foregoing, the required time for proposal review is reportedly less of a concern compared to the time required to provide notices of approval to successful applicants. As remarked earlier, GOA is significantly event driven and therefore time sensitive. From the point of view of the associations, the scheduled release date of mid-March for application review results already presents challenges for beneficiaries as many international trade shows occur in the spring. However, notification of the application review results has occurred as late as June and July in practice, forcing beneficiaries to determine whether or not to proceed with an activity without assured GOA funds, a risk that many beneficiaries are unwilling to incur. Further, since fewer industry activities occur during the summer months, GOA programming is for all practical purposes confined to the third and fourth quarter of the FY. This has contributed to, on average from 2010-11 to 2012-13,  a 56% completion rate for funded activities (see Table 12), a 37% de-commitment rate for committed funds, and year-end lapses  in excess of 28% annually (see Table 5). While the update of the department’s GCS delayed decision-making on GOA proposals in 2012-13, the principal reason for the delay in releasing results was reportedly due to the requirement to secure Ministerial approval before announcing results.

To address some of these issues and reduce GOA’s ongoing lapsing challenge, some consideration has been given to the idea of re-adopting the “open call for proposal” approach used by GOA’s distant predecessor, Program for Export Market Development (PEMD). Under PEMD, DFATD, then-DFAIT, received applications throughout the year which allowed the department to better align program funding with identified activities. The advantage of this approach was that the turn-around time for the processing of proposals was shorter, thereby accelerating the rate of disbursements. Notwithstanding the advantage of this “first come first serve” model, an evaluation of the PEMD conducted in 2005 observed that high volume disbursements were being achieved at the expense of support for quality projects, which ultimately translated into lower than expected results. The evaluation recommended the adoption of a competitive review process based on a single application window, a recommendation that was later affirmed in an evaluation of PEMD’s successor, PEMD-A, and the model currently in use for GOA.^{Footnote 70}

In considering the merits of reverting to an open call for proposal model, it is important to keep in mind that PEMD was a very a different program than GOA. PEMD provided direct funding to companies whereas the PEMD-A and GOA provide funding to industry associations which are fewer in number. Further, the “sole-source negotiated” approach that an open call for proposal entails can only be administered by personnel internal to the Department, thereby increasing the administrative burden on program administrators and attenuating the disciplinary effect of the competitive bid process. In addition, the value and rigor of the current adjudication process attained from the participation of a wide range of Departmental and extra-Departmental stakeholders would be lost. For the foregoing reasons, there is a general consensus among stakeholders that the open call for proposal model would likely create more problems than it would solve.

There has also been some discussion on the merits of GOA adopting a calendar year planning cycle akin to ICCI. The response from the sub-program has been largely negative, citing that GOA’s administrative systems are not currently configured to support a calendar year and that this proposed change would place the sub-program out of step with the Departmental planning cycle. Given that ICCI’s administrative system was adopted from GOA with some modifications, the alleged technical impediments to GOA operating on a calendar year basis may be overstated. Decisions have yet to be made on moving GOA to a calendar year but another option could be to move the launch date to one month earlier in the fiscal year. This would also shift the funding announcement earlier by one month if the application process proceeds as planned, thereby alleviating some of the pressures for beneficiaries in meeting project objectives and better aligning GOA programming with association objectives.

GGI

Unlike the other sub-programs of the GCSP, GGI receives applications throughout the FY. GGI, as noted in Finding 17, experienced high rates of non-completions/cancellations (around 35%) with corresponding high rates of de-commitments. However the reasons were reported to have had little to do with the application review and approval process. Indeed, GGI beneficiaries interviewed in support of this evaluation, while remarking that the sub-program could have benefited from an electronic platform from which to submit applications, conveyed on balance a high degree of satisfaction with the guidance and support provided by program administrators and the timeliness of decision-making. The high rates of non-completions are rather due to difficulties in securing matching funds and unanticipated changes in plans, which automatically results in a cancellation. GGI beneficiaries interviewed in support of this evaluation confirmed that securing matching funds and organizing meetings with prospective partners overseas is challenging and subject to a wide variety of risks which can and do impact on plans. 

5.5.3 Information Technology Infrastructure and Management

Finding 24: Apart from challenges relating to ICCI’s and GOA’s online administration systems, IT infrastructure supporting GCSP and management thereof is sound.

Program delivery for ICCI and GOA has been facilitated by online administration systems, supported by VTC, which permits clients to submit their applications for funding on-line. ICCI’s administrative system was developed based on GOA’s system, which was created first using Departmental resources. GGI for its part contemplated setting up a web-based data repository akin to ICCI and GOA, but this idea was abandoned following suspension of its funding authority. As remarked earlier in this report, the system was built on an old IT architecture which has become unstable over time. Both ICCI and GOA beneficiaries referenced challenges working with this system, among which was the inability to edit entries into the application forms. Program administrators reported having difficulty accessing the accounts of clients to provide real-time service during the submission process and concerns were also raised regarding the security of the system. Further, the system was reportedly not configured to handle second-round applications. Though this was not identified as an issue for GOA, it was reported to have presented a challenge to both ICCI clients and program administrators. Senior management is testing alternative systems to replace these platforms.             

With respect to financial budgeting and reporting, the CAU tracks planned and actual expenditures for each of the sub-programs which is shared with sub-program management. The CAU reconciles the information contained in these budgeting tools with the Department’s financial management system (IMS) on a monthly basis, thus providing a assurance that the information remains synchronized. Access to these spreadsheets by sub-program management allows the same to monitor their budget status against planned commitments and make adjustments where necessary. Since reconciliations between these budgeting tools and IMS are conducted manually, thereby potentially compromising data quality and reliability, actual discrepancies observed were in fact minor, likely due to time lags between reconciliations. On the whole, program management conveyed a high degree of confidence in and satisfaction with the systems in place to support budgeting and reporting.

As remarked earlier in this report, the CAU is charged with assembling all documentary material used by the GCSP’s three governance bodies (the GCSP Steering Committee, the Director Level Committee, and the Deputy Director Level Working Group) and for recording decisions made by the same. The CAU is further charged with preparing all legal agreements (CAs), tracking and recording finances, and monitoring and recording program performance. As such, the CAU is the common data repository for the GSCP and its three sub-programs. This said two of the sub-programs of the GCSP (ICCI and GOA) maintain their own web-based data repositories which include information on applications received and approved disaggregated by beneficiary, activity, region, sector and market, as may be applicable. Performance data is collected and assembled by Applications Solutions (SIA) of the Department. In absence of an online administrative system, GGI relies entirely on the CAU to manage its programming and performance data.

The evaluation observed that the program information contained in the administrative systems of ICCI and GOA were not fully aligned with the program information retained in the program’s I-Drive. These discrepancies included financial information (funds committed, funds disbursed, de-committed) as well as program information (applications received and applications approved). These discrepancies, as noted in the Early Implementation Audit of the Global Commerce Support Program (2011), were however minor and again likely due to lapses in updating the information contained in the data repositories managed by BIS and BBI. The evaluation also noted inconsistencies in the recording of program information within databases. Here again, while these discrepancies are likely a result of the information not being entered into the data repositories in a timely fashion, a more standardized approach to data recording would be beneficial. But for the issues referenced above, the program administrative systems appear to be reasonably well integrated with Departmental financial management systems and senior management is only provided financial information that has been reconciled against IMS.

With the exception of the online administrative systems, program management conveyed a high degree of satisfaction with the timeliness and quality of program information at hand to support decision-making and, by extension, satisfaction with the systems in place to render that information available to them. This said, sector practice officers responsible for GOA noted that limited access to the sub-program’s online administrative system encumbered their ability to serve their client associations in a timely and effective manner.

5.5.4 Financial Management

Finding 25: But for evidence of divagations in practice with respect to the processing of expense claims, the program has developed robust systems and procedures to support sound comptrollership of financial resources.

The creation of the CAU to serve all three sub-programs was intended to strengthen comptrollership over program resources and achieve efficiencies, particularly with respect to the management of financial resources. Charged, among other things, with the overall administration of the financial resources of the GCSP, the CAU performs an instrumental role in the management, control and monitoring of the transfer payments budget for both the GCSP sub-programs and the GCSP as a whole. As noted in Finding 23, the CAU conducts both periodic and annual reconciliations of the financial information contained in the program’s budgeting tools and the Department’s IMS, Though some discrepancies between these databases have been observed, they have not been of such a magnitude to in any significant way undermine confidence in data integrity, which might place decision-making at risk. The CAU, in its capacity as GCSP secretariat, further provides the same to responsible DGs and Directors at scheduled committee meetings with updates on the status of program finances, including the risk of lapses, which informs decisions on de-commitments and reallocations between the sub-programs throughout the year. In short, the services provided by the CAU to program management, and the underlying systems it relies on, support sound financial comptrollership.   

In addition to tracking the use of Vote 10 resources, the CAU is also charged with tracking Vote 1 resources (O&M). As remarked earlier in this report, one of the principal reasons for amalgamation of CISP, PEMD-A, and GG into the GCSP and for the creation of the CAU itself was to eliminate redundancies and reduce overhead costs. This said, the Early Implementation Audit of the Global Commerce Support Program (2011) noted that there was, at that time, “no reporting mechanism in place to allow for the identification, reporting and monitoring of salary and operating expenditures (Vote 1) for the GCSP” and recommended that program management institute such a reporting mechanism. Program management responded to this recommendation and the CAU now systematically tracks salary and operating expenditures, which is a best practice. 

Notwithstanding the foregoing, each of the sub-programs employ different standards with respect to the kind of supporting documents demanded to support payment claims. For example, neither ICCI nor GOA currently, as a matter of course, require beneficiaries to furnish invoices as evidence of expenses actually incurred, but rather requests proof of the activity having been performed (e.g. the production of a report). The adoption of this approach was thought to be less onerous on sub-program beneficiaries and administrators charged with reviewing claims but, as remarked in the Early Implementation Audit of the Global Commerce Support Program (2011), this approach constitutes divagation from established practice and may expose the program to abuse. This risk, however, is in part mitigated by the requirement for beneficiaries to furnish original receipts on request and by the implementation of periodic recipient audits.

Finding 26: Though the program has succeeded in reducing overall lapses, GOA continues to lapse on average 28% of its allocated funds annually.

As remarked at the outset of this report, the amalgamation of CISP, PEMD-A, and GG into the GCSP was seen as one means by which to address their lapses of 20% per annum noted in previous evaluations and audits. Amalgamation would do so by providing the necessary flexibility to reallocate funding between the sub-programs depending on recipient needs during the course of the year. Additional measures adopted by senior management to address the lapsing challenge included securing authority to commit up to 125% of the annual program allocation and by instituting a financial risk mitigation strategy, complete with a financial monitoring schedule for each sub-program, to serve as an early warning system to allow for reallocations when warranted.

Amalgamation and the adoption of the other measures referenced above have indeed reduced these lapses. As noted in Table 5 (Financial Information for GCSP) in this report, overall lapses for the GCSP have averaged around 10% per annum during the evaluation reference period, half that of the quantum of lapses reported prior to amalgamation. GGI and GOA program expenditures account for these lapses, with GGI averaging 5.1% in lapsed funds per year and GOA averaging 28.4%, as described in Table 5. ICCI over-spent its budget on average during the evaluation reference period. As noted in Findings 17 and 22, common difficulties in securing matching funds and changes in planned activities, which resulted in immediate project cancellations, are thought to be largely responsible for GGI lapses. Challenges in securing matching funds and common changes in plans are also cited as reasons for GOA lapses. However, the magnitude of GOA’s lapses and their consistency over time despite measures adopted to minimize the same like over-committing funds and securing the authority to re-profile, suggest challenges are unique to this sub-program.

The reasons for these large lapses experienced by GOA are varied. For example, the Early Implementation Audit of the Global Commerce Support Program (2011) found that, based on a review of a sample of projects funded by GOA, many beneficiaries overestimated project costs by roughly 33% and speculated that this practice has likely contributed to the sub-program’s lapsing difficulties. To the extent that this practice can be generalized, it further raised questions about the degree of rigor applied to the review of the cost projections in project proposals, thereby leading to a recommendation for “increasing the degree of challenge provided to applicants’ estimated project/activity costs in the program’s application and adjudication processes.” Still again, current funding caps on GOA activities or packages have been cited as artificially imposing limits on spending, which has contributed to funding lapses. Program managers interviewed in support of this evaluation, however, cited variable launch dates for GOA and delays in the release of funding decisions owing to the need to secure Ministerial approval as the most significant reasons for lapses. Assuming these issues are addressed and significant lapses persist, it would be logical to reduce the notional allocation to GOA to better align its budget with its capacity to spend.

5.5.5 Human Resource Complement and Management

Finding 27: The current staff complement dedicated to program delivery is adequate. However, there is a need to augment existing capacity in the CAU.

Program delivery, as remarked in Section 1.5 of this report, is supported by 8.6 FTEs, calculated as a percentage of the time personnel in each of the program’s administrative units (CAU, BIS (ICCI), BBI (GOA), and BBT (GGI)) dedicate annually to the program. This quantum was reported to have declined to 7.6 FTEs following the closure of some ROs. Among personnel dedicated to program delivery, three are wholly dedicated (100%) to the program, with the balance apportioning a fraction of their annual time (ranging from 10% to 35%) to the same. Depending on the sub-program and related planning cycle, time dedicated to program administration can increase to as much as 90%. This is often the case following receipt of the first round of project proposals when demands on the time of program administrators are at their most intense.

Views on the adequacy of this HR complement vary, depending on the administrative unit in question. The CAU, for example, reports that it is barely able to keep up with demand for its services given its current staff complement. This is in part due to the CAU having assumed program management duties relating to its capacity as a secretariat to the program, which it was not originally envisaged to perform. Further, while the CAU has benefited from the services of a certified and dedicated financial officer, that resource has recently been lost thereby obliging the current Program Manager to perform the duties of both program and financial manager, which is not sustainable. As for ICCI (BIS), officers reported that though there are periods during the course of the year when demands for their service were intense, compounded by the problems already noted regarding the online administrative system, these periods are generally predictable, and thus easier to plan and prepare for to meet those challenges. Although GGI (BBT) operated under an open proposal (first come, first serve) model, former officers involved in program delivery reported the process to be manageable for the most part. The program administrators expressing the most concern regarding the current process were those associated with GOA (BBI).

As remarked earlier in this report, the call for proposal, review and approval process for GOA is considerably more complex and lengthy than the processes relating to the other two sub-programs of the GCSP and, by extension, more demanding on the time of those involved in its administration. The challenges program administrators (sector officers) have faced in stewarding project proponents through the application and review process, including the problems related to the on-line administrative system, were reported to be compounded by the call for proposal launch date which resulted in peak periods during the GOA application process overlapping with peak periods relating to the other duties of program administrators, such as preparing for the annual renewal of the Commercial Economic Plan [CEP]. Notwithstanding the foregoing, the staff complement, but for the CAU, supporting the GCSP appears to be adequate.

Finding 28: Though the provision of training to program administrators, with the exception of GOA, is informal and ad hoc, guidance materials and counsel provided by senior management, the CAU and SMFC appear adequate to support effective program delivery.

Training provided to program administrators varies in type, subject matter, and frequency between the three sub-programs of the GCSP. Neither BIS (ICCI), BBD (GOA), or BID (GGI) have established training requirements for their program administrators beyond training specific to the administration of their respective sub-programs. Program administrators, for example, are encouraged to take training in FDI, IBD, S&T, and in G&Cs, but are not expressly required to do so. What is expected of program administrators is that they all be thoroughly familiar with the policies and procedures of the sub-programs they administer. To this end, GCSP management has developed Standard Operating Procedures (SOPs) for each of the sub-programs detailing the sub-program’s objectives, priorities, key dates or milestones in the planning cycle, the application process, assessment criteria and templates, eligible and non-eligible expenses, and claim interpretation and process. These SOPs are supplemented with guidelines relating to the foregoing. Additionally, program administrators can avail themselves of the advice of senior management, staff in the CAU and SMFC when required.

Training on the use of the aforementioned instruments for ICCI and GGI is largely informal, meaning counsel is provided by senior management to program administrators when advice is sought. Training in the use of the instruments for GOA is more formal, consisting of specific training modules relating to sub-program launch and application development, application evaluation, and claims processing, which is provided to all new program administrators and offered as a “refresher” for more experienced program administrators as well as their managers.. BBD’s more formal approach to training is reportedly due to the complexity of the application, review and implementation processes relating to GOA and the fact that GOA program administrators are largely rotational and work on the file part-time. Program administrators interviewed in support of the evaluation conveyed, for the most part, satisfaction with the SOPs, written guidance, and the counsel provided by program management, the CAU and SMFC in support of the exercise of their duties.

This said GOA program administrators, as remarked in  Finding 21 of this report, conveyed disquiet over their responsibility to provide clients with advice on eligible expenses – a responsibility which they are charged with and accountable for, but for which they claim to have no accredited expertise. Similar disquiet was expressed relating to the review of expense claims, again on the grounds that this is an area for which they have no accredited expertise. For the foregoing reasons, many GOA program administrators favored the transfer of these administrative responsibilities to the CAU, where the requisite expertise is thought to reside, which would free them to provide the kind of service (strategic advice) more aligned with their formal areas of competence. However, given that it is the funds provided through GOA which binds the relationship between program administrators and the associations, centralization of administrative responsibilities could risk undermining the leverage program administrators have with the associations, thus creating more problems than it would solve.

5.5.6 Communications and Marketing

Finding 29: Despite the closure of some ROs, which formally performed an important role in marketing the program, the program as a whole has retained its visibility among its client base.

All three sub-programs pre-date the formation of the GCSP itself and thus have long histories with their respective clients. This said, the visibility of some of the sub-programs is reported to have diminished. A case in point is ICCI which, as noted earlier, experienced a significant decline in the number of applications received over the evaluation reference period. Again, this decline has been attributed to the diminished standing of the ROs which once played an important role in marketing the sub-program. Indeed, several beneficiaries interviewed in support of this evaluation remarked that they came to know of ICCI through the ROs. Today, the sub-program’s principal marketing tool is its website. In addition, sub-program management is undertaking outreach activities across Canada to recruit additional applicants and to explain the sub-program’s objectives, and runs a number of webinars during the application period to assist potential applicants. This said ICCI is not a sub-program in search of clients as the number of applications received over the evaluation reference period actually increased. Further, proportional to the financial resources at its disposal, ICCI performs the best among the three sub-programs of the GCSP in disbursing funds. Be that as it may, Ontario and Quebec remain the primary beneficiaries of ICCI funding, accounting collectively for over half of all program disbursements (see Table 9: Distribution of Committed Funds for ICCI across Provinces and Territories) which has prompted some stakeholders to remark that more needs to be done encourage communities in other provinces and territories to participate in the sub-program.

As for GOA, the number of applications received has in fact increased over the evaluation reference period (see Table 12: Profile of GOA Applications) suggesting that the sub-program is well known within its client base, namely the national industry and trade associations who, as stated earlier, are best placed to market GOA to their members. As such, GOA is not confronted with a marketing challenge. With respect to GGI, here too the ROs, judging from the feedback received from beneficiaries interviewed in support of this evaluation, appear to have played an important role in making the sub-program known to prospective applicants. However, the diminished role of ROs in sub-program delivery does not appear to have had any impact on the volume of applications as the number of applications for GGI funding actually increased over the evaluation reference period (see Table 16: Adjudication Status of GGI Applications).

5.5.7 Performance Monitoring and Results Reporting

Finding 30: The Performance Measurement Framework developed for the GCSP and the instruments developed to support data collection are sufficient.  However, further refinements are necessary to better align objectives with the activities the program actually supports and to better align these activities with higher order outcomes.

Following amalgamation of CISP, PEMD-A, and GG under the umbrella of the GCSP, program administration developed an ARAF which, among other things, defines the program’s objectives, expected results, and the linkages between the activities supported by the program and those expected results. The document further includes a performance measurement strategy that defines the program’s outputs and outcomes (immediate, intermediate and ultimate), performance indicators to measure the achievement of the same, and the data sources and methods of collection to be used in support of the foregoing. Finally, the ARAF incudes an evaluation framework, constructed in loose conformity with TB Policy on Evaluation, which outlines the key evaluation issues and questions relating thereto the indicators to be used as evidence in support of the foregoing, and the corresponding data sources and methods of collection. This ARAF has served as the basis and guide for the design of all performance measurements instruments adopted by the program for performance and results reporting purposes. 

Regarding the aforementioned instruments, the program has invested considerable time and energy in developing templates (e.g. for end-of-project reports), guidelines, questionnaires and electronic systems to compile, analyze and report on performance and results. For example, each sub-program systematically tracks and records, albeit in different ways: trends in applications (received, approved, rejected, cancelled); activities supported; and the distribution of program resources by geography (provinces and territories), sector and foreign market. With regard to program results, information contained in the end-of-project reports serve as the primary source of output and immediate outcome data. These short-term results data have been supplemented with information obtained from two questionnaires targeted to the clients of all three sub-programs, designed to solicit feedback on the longer term impacts of program participation two years following project completion. This information provides input into the GSCP Annual Performance Report.

The foregoing clearly demonstrates the importance the program has placed on performance monitoring and results reporting. This said there remain areas for refinement. For example, while the program’s ARAF integrates the disparate, though complementary, elements of the GCSP into a coherent performance measurement framework, it does so arguably at the expense of fidelity to the actual operations of some of the sub-programs comprising the GCSP. The accent placed on capacity building among the three sub-programs is a case in point. While ICCI in its objectives and operations is very clearly focused on capacity building, and GGI to a certain extent given that partnerships grant Canadian scientists access to know-how and technology they may not otherwise have, capacity building under GOA is, for all practical purposes, a by-product of and incidental to the actual activities supported by this sub-program. To be sure, funds made available to associations through GOA influences their behavior and allows them to participate in international networking events which they might not otherwise be able or willing to do on their own, and that participation in these events generates market intelligence of benefit to the industries they represent. However, this is not programming of a kind that enhances the capacity of these associations to serve their members in a sustainable manner, which is what capacity development is all about.

When examining the activities supported by the three sub-programs of the GCSP and their respective higher order outcomes, the linkages are, in some instances, tenuous. Again, GOA is a case in point. As referenced in Finding 07, while GOA does contribute to increased participation of Canadian businesses in international trade, which can lead to enhanced competitiveness by increasing opportunities to benefit from economies of scale, GOA in design and operation does not fully support GVC participation. For it to be so, it would have to support Canadian businesses source foreign suppliers of intermediate inputs, which the sub-program can but reportedly does not do. As for GGI, though the ultimate outcome of this sub-program is R&D commercialization, GGI does not directly fund R&D as such, but rather the formation of partnerships upon which such R&D can be advanced in the direction of commercialization. In other words, there aretoo many intervening factors at play between partnership formation and eventual commercialization for the latter to be directly attributed to the former.  

Finding 31: Management has demonstrated exceptional diligence in the collection and recording of performance data. However, performance indicators are too numerous, not always complementary and of questionable usefulness, thereby contributing to a distorted performance story for the GCSP.

The indicators used to measure performance for the GCSP again vary in type and number between the sub-programs. ICCI, for example, uses no less than thirty-one indicators to track and report on outputs. These indicators constitute, for the most part, sub-activities relating the six core activities supported by ICCI. For example, community profiles, labor profiles, sector profiles, industry profiles and feasibility studies, all relate in one form or another to research. Some indicators are specific to anticipated outcomes (e.g. number and dollar value of new and retained FDI, number of jobs created and retained, after care service), while others are specific to actuals (e.g. number of hits on a new or improved website, number of inquiries, number of persons trained, number of Canadians participating in an event, number of foreign persons participating in an event). From a performance reporting point of view, many of the indicators currently used to measure outcomes, such as the number of persons trained and the number of Canadians participating in an event, which in fact ICCI does not in practice support, would appear better suited as outputs, while other indicators of outcomes, currently recorded under the category of anticipated (e.g. targets, leads and prospects), would be better suited as actuals. Still there are other indicators (e.g. jobs retained and created, number and value of new and retained FDI), which beneficiaries have struggled to interpret and have found almost impossible to respond to in a meaningful way within the immediate aftermath of a project. Finally, there are indicators (e.g. after care service, inquiries) which are rarely if ever populated with data, all of which suggests that there is a need to review the merits of the suite of indicators currently in use.

The utility of this kind of information for performance monitoring purposes is vitiated by several factors: first, performance indicators are not outputs per say, but activities; second, indicators count the number of projects or communities undertaking these activities, not the actual number of activities executed; third, because activities are inter-related, there is a high risk of double or triple counting; and finally, recorded results are anticipated, not actual, outputs. Interviews with program personnel conducted in support of this evaluation confirmed that confidence in the accuracy of this information is suspect and its utility uncertain, thus calling into question the rationale for collecting the data in the first place. Understanding trends in the demand for certain activities could be of potential interest to the sub-program, but this could be achieved by simply recording the resources disbursed to the six core activities supported by ICCI.

As for GOA, three overarching performance indicators are used to track and report on outputs for each of the sub-program’s three core activities.^{Footnote 71}  The three overarching output indicators measure the number of leads generated, the number of identified potential foreign partners and the number of Canadian companies that expanded their IBD efforts. These higher level indicators seem to be the most useful and meaningful to GOA, as long as respondents have a shared understanding of the terms (e.g. definition of “lead” is clear). The reach of each of the sub-activities under the three core-activities is also captured through indicators measuring, for example, the number of foreign and Canadian participants at networking events and the number of industry personnel reached. Because of the numerous GOA sub-activities, there are 35 performance indicators that measure outputs. These measures are problematic for a number of reasons.

Given that GOA’s objectives are to improve IBD among Canadian SME’s, it is challenging for the sub-program to obtain accurate and meaningful performance data since funding recipients are industry associations. Interviews with beneficiaries and program managers demonstrate that some associations survey their members to explore the impacts of funded activities on Canadian businesses. Other associations are less systematic when completing their reports on achieved project outcomes, resulting in compromised validity in the performance data. This could be alleviated with the development and distribution of a standard questionnaire template for associations to distribute to members that participated in the funded activity, which would standardize and increase the accuracy of the information collected. The accuracy of the data could also be improved with greater alignment between the unit of analysis of the performance measures and Canadian businesses, which are the ultimate beneficiaries of GOA. For example, performance indicators capture the number of Canadian participants or industry personnel involved in funded activities, which implies individuals from Canadian SMEs rather than SMEs themselves. In addition, the usefulness of the performance data could be increased with improved specificity where, for example, the number of user sessions of existing and new websites and web tools distinguish between Canadian and foreign users. As well, indicators that ask for the number of industry personnel reached through brochures or pamphlets could be capturing the number of documents distributed rather than individuals or businesses impacted by the funded activity, as intended.

In light of the foregoing, program management as well as administrators questioned the value of some of the program and performance data currently collected to support decision-making. Specifically, the utility of information relating to the distribution of resources by client type, location, activity, sector and market was limited for trend analysis purposes given that all three sub-programs of the GCSP are largely responsive in design and operation. Similar remarks were made in reference to results data collected. Be this as it may, the collection and analysis of this information is important for accountability purposes.

Finding 32: End-of-project reports are useful for collecting project results immediately following project completion but less so for capturing higher order expected outcomes. The administration of recipient questionnaires to assess the GCSP’s longer-term impacts, therefore, is a best practice. The development of a GCSP Annual Report is also a best practice.

GCSP’s use of end-of-project reports effectively enables each sub-program to track achieved project results against anticipated results proposed at the outset. For example, ICCI tracks and records anticipated and actual project outcomes using 21 different performance indicators. Notwithstanding the challenges in collecting useful and accurate performance information, ICCI’s end-of-project reporting template is quite good. Barring the section of the template which reiterates the anticipated outcomes documented in the application form, actual results are divided into two types: a list of qualitative indicators which invites beneficiaries to comment on the extent to which program participation enhanced the community’s knowledge of FDI opportunities, contributed to improved capacity to attract, retain and expand FDI, and increased potential partner/investor awareness of FDI opportunities in Canada; and, a list of quantitative indicators, which invites beneficiaries to record the number of foreign companies identified as having the potential for investing abroad (targets), the number of foreign companies identified as having expressed a desire and capacity to invest abroad (leads), and the number of foreign companies that have expressed an interest in investing in Canada (prospects). These indicators of immediate outcomes are reasonable and meaningful.

Likewise for GOA, its results reporting template tracks and records achieved project outcomes to facilitate comparisons against proposed anticipated results. As described in Finding 30, project results are reported for 13 sub-activities^{Footnote 72} with the three overarching indicators that measure higher order outcomes providing the most usefulness to the sub-program. Outcome indicators relating to Direct Contacts which constitutes the highest proportion of activities (see Table 13: Approved and Completed Activities by Type) supported by GOA, for example, are readily quantifiable and meaningful to the extent that counts of leads generated or the number of Canadian companies that expanded IBD are possible. The sub-activities relating to Direct Contacts, while providing a measure of reach, capture program outputs, and indeed are, with regards to interactions between Canadian and foreign participants, tracked as outputs. Here too, however, there are concerns because the sub-activities tracked are not mutually exclusive, which increases the risk of double counting thereby resulting in inflated figures for reported results. The same observation applies to the production and dissemination of promotion and marketing material, which are often used in conjunction with Direct Contact Event activities. With regard to Research/Studies, but for the indicator relating to industry personnel reached, which is likewise tracked as an output, the indicators used are meaningful and appropriate for immediate outcomes.

With respect to GGI, its outputs are synonymous with its immediate outcomes which find expression in the number of agreements generated as a result of beneficiary participation in PDAs. Over the reference period, GGI has consistently tracked these outcomes in reference to four types of agreements: NDAs, MOUs, JVAs, and LOIs, which collectively account for roughly 40% of the agreements recorded. The remaining 60% of agreements (“other”) are not defined or tracked in any systematic way, but rather constitute a mix of activities/outcomes (e.g. joint publications, knowledge transfer, workshop attendance, student exchange, etc.) which are defined by the beneficiaries themselves and thus vary from year to year. This said there are agreements falling into this category of “other” (e.g. IP agreements, licensing agreements, master research agreements, R&D support agreements, etc.,) as well as other outcomes (e.g. suppliers contracts, secured financing) which are meaningful immediate outcome indicators. That some of these potential indicators of immediate outcomes, in particular finances secured from other sources, are not systematically defined or tracked does the sub-program a disservice.

Higher order outcomes for all three sub-programs cannot be captured in an end-of-project report, as these outcomes are achieved months and sometimes years after the completion of a given activity. As such, the only way to capture this kind of information is through a recipient questionnaire. As remarked earlier in this report, the program has administered two questionnaires targeted to the beneficiaries of all three sub-programs, starting with those in receipt of program support in 2009-10 followed by 2010-11. The results of these questionnaires provide the best evidence available of the program’s overall contribution to the achievement of its declared objectives, thereby constituting a best practice in the domain of results reporting.

All of the performance information collected through the aforementioned instruments is assembled and analyzed in the GCSP Annual Report. In addition to this performance information, organized and presented in accordance with the program’s Logic Model, the GCSP Annual Report also includes an overview of the program’s administrative performance (applications received, approved, and completed), financial status, and of actions taken by program management in response to emerging challenges and recommendations advanced in past evaluation and audit reports. Though the drafting of the GCSP Annual Report is labor intensive and therefore costly, its production is nonetheless important, offering an incentive to management to reflect upon program performance in a systematic way while simultaneously demonstrating adherence to and respect for the principles of transparency and accountability. As such, the program’s commitment to generate annual reports is again a best practice.

5.5.8 Risk Management

Finding 33: Program management has demonstrated due diligence in the identification of potential risks to the program and in developing appropriate mitigation strategies.

In compliance with Treasury Board Policy, program management prepared a Risk-Based Audit Framework (RBAF) for the GCSP. That document concludes that the GCSP is and remains a low risk program for reason that: 1) the GCSP is not a high profile program; 2) funded projects have a relatively low dollar value and involve reputable Canadian recipients; and 3) shared costs between the program and recipients (25 to 50%) reduces exposure to financial losses. Evaluation evidence supports this conclusion. Most of the risks identified and associated mitigation strategies proposed in the RBAF speak to challenges relating to the process of program amalgamation itself and the potential impact of on-going Departmental changes (e.g. budget constraints, internal re-organization, new Policy on Transfer Payments, changes in Departmental processes, etc.).

Over the evaluation reference period, program management has responded effectively and in a timely fashion to the immediate challenges brought about by amalgamation, redefining and formally documenting the roles and responsibilities of stakeholders in accordance with the program’s new governance structure, setting up the CAU in a timely fashion, recalibrating existing administrative systems to serve the GCSP as a whole and its sub-programs, updating policies, procedures, and templates and developing related guidelines, and in providing staff with appropriate training for program delivery.

Program management has also responded well to exogenous challenges brought about by Departmental changes. These include the closure of some ROs, which initially had a disruptive effect on program performance, and in adjusting, at least in part, to the processes required to secure ministerial project approvals. Similarly in the area of financial management, though lapses are an ongoing concern, program management has taken measures to develop appropriate procedures to mitigate the risk of lapses. In short, the program management has demonstrated due diligence in the identification of risks to the GCSP and in developing appropriate responses thereto.

5.5.9 Value for Money

Finding 34: Though the monetary value of program outcomes is difficult to determine, thereby precluding a cost-benefit calculation, available performance data does suggest that all three of the sub-programs of the GCSP generate benefits in excess of costs.

The concept of value for money assesses the extent to which a program has obtained the maximum benefit from the outputs and outcomes it has produced with the resources available to it. Program resources (costs) are readily quantifiable but the monetary value of program outputs and outcomes (benefits) is not, though some inferences can be made from the results data available. With respect to program costs, amalgamation was expected to generate significant savings, estimated at $400,000 in the administration and delivery of the three sub-programs.  In the Early Implementation Audit of the Global Commerce Support Program, the Office of the Chief Audit Executive at DFATD was not able to determine whether these savings were achieved as no clear benchmark exists for a before-and-after amalgamation comparison.  Likewise, this evaluation is inconclusive on this point because steps taken to track salary and operating expenses were only instituted by management following the release of the internal audit report in 2011.

What can be said based on available information is that salaries and operational expenditures have averaged around 12.8% of the total annual budget for the evaluation reference period (see Table 2: Budget Allocations for GCSP Vote 1 and Vote 10). Whether or not this is an acceptable ratio for a program such as the GCSP, it is unlikely that the three sub-programs of the GCSP could be effectively managed with fewer than its current dedicated FTEs complement and that savings could be achieved from cuts in overhead (e.g. IT Infrastructure) without compromising the integrity and performance of program administration. As noted in this evaluation, GCSP’s program administration is distinguished as a best practice. Greater efficiencies could be achieved, such as streamlining the application, review, approval, and reporting processes, but these efficiencies would be on the margins and would not likely translate into significant cost savings.

Program costs, including administration, are, however, only one aspect of value for money: these costs must be measured against the value of the outputs and outcomes generated by the program. While determining the monetary value of the outputs and outcomes produced by the GCSP is highly speculative, the project results data available provide some indication of program benefits. As described in Section 5.4 on the achievement of expected outcomes, hundreds of jobs were created for communities and FDI valued at tens of millions of dollars resulted from projects funded through ICCI. Likewise, millions of dollars in sales to foreign partners were attributed to the support provided by GOA. These project results demonstrate that these sub-programs have generated benefits far in excess of their costs. As for GGI, assessing the monetary value of its outcomes is nearly impossible given that its outcomes are R&D partnerships. However, if one looks at the role this sub-program has played in leveraging financial contributions from other sources in support of R&D, then GGI has likewise generated monetary benefits far in excess of its cost. As such, the GCSP, and its sub-programs, is good value for money.

6. Conclusions of the Evaluation

Evaluation findings suggest that Canadian communities, SMEs, and researchers all require some degree of government assistance in the development of their capacity to attract FDI, participate in IBD, and advance R&D objectives through partnerships with foreign counterparts. Addressing the needs of these different populations has been identified by both the GoC and DFATD as priorities and each of the three sub-programs of GCSP are, in their own ways, designed to address those needs. Evaluation evidence further suggests that the sub-programs of the GCSP do in fact address those needs in a targeted manner, thereby contributing to the realization of the objectives of the GoC and DFATD in the domains of FDI promotion, IBD, and international R&D.

Program relevance, however, is not simply confirmed with evidence of a program being responsive to an identified need or being aligned with government priorities. For a program to be materially relevant it must fill a funding gap or, in other words, offer support of a kind not provided by any other assistance provider. With regard to ICCI, evaluation evidence suggests that though institutional capacity development at the community level may seem an odd occupation for a federal department with an international mandate, the suite of activities supported by ICCI is unique, thereby filling a funding gap in the area of FDI promotion. GOA likewise, to the extent that it focuses on national industry and trade associations, is also unique and by extension fills a funding gap in the area of IBD.

GGI also fills a funding gap, but, as stated earlier in this evaluation report, the gap it fills is extremely narrow and fragile, meaning that minor modifications to the eligibility criteria of other federal programs which support R&D PDAs could render this sub-program redundant. Working through an external umbrella organization, rather than providing funding directly to applicants, could strengthen the relevance of GGI by consolidating resources in support of PDAs, thereby reducing the risk of overlap with other funding sources, and possibly increasing the efficiency and effectiveness of sub-program delivery. However, these benefits remain hypothetical at this juncture, thus requiring further study to determine the viability and the merits of pursuing this approach.

Despite the aforementioned relevance issues all three sub-programs of the GCSP have succeeded in achieving their expected results and have, in their own ways, contributed to Canadian economic growth and prosperity. ICCI investments in communities across the country have increased their capacity to retain and attract FDI, which has resulted in new FDI calculated in the tens of millions of dollars and thousands of new employment opportunities. GOA, for its part, has likewise increased Canadian SME participation in IBD, which in turn has translated into tens of millions of dollars in new sales and contracts for the same, thereby adding value to the Canadian economy. Though the monetary value of GGI’s contribution to the Canadian economy is difficult to determine, it too has had considerable success in achieving its objectives, which is to forge new and sustainable S&T and R&D partnerships which may yield in future sizable returns on investment. As such, all three sub-programs of the GCSP generate value in excess of costs.

These favorable observations with respect to results are to a significant degree made possible by a highly developed and, for the most part, efficient administrative infrastructure put in place to support program delivery. The current governance structure, composed of the DG Steering Committee, the Director level committee and the Deputy Directors’ Working Group, provides appropriate strategic direction and operational oversight to the program, evidenced by their timely responses to ongoing strategic and operational challenges, including those discussed in past evaluation and audit reports. While some issues have been raised with respect to representation on the program’s adjudicative bodies, these too appear to function well. Further, roles, responsibilities and accountabilities are clearly defined, documented and, for the most part, accepted by all involved in the administration of the program. The exception is the sector officers responsible for administering GOA because they feel they are charged with and thus accountable for the provision of advice to clients without reportedly having the requisite G&Cs policy or GOA program expertise.

Supporting the work of the aforementioned governance bodies is the CAU which, beyond its general function as GCSP program secretariat, has played a vital role in furnishing senior management with timely and accurate information on the status of program finances, including the risk of lapses, which informs decisions on de-commitments and the reallocation of resources between the sub-programs. This has contributed to an overall diminution in the quantum of lapses to around 10% per annum, thus achieving one of the key objectives of amalgamation. This said, and despite the adoption of a range of measures to reduce lapses, GOA continues to register relatively high year end lapses in the order of around 28%. The reasons for these large lapses are varied but the most common reasons cited were the timing of the launch dates for proposals (which have varied from year to year), delays in the release of adjudication results (which have reportedly frustrated sub-program delivery), and inflated cost projections from project proponents (which have contributed to significant variances between planned verses actual expenditures). Addressing these issues will be essential if GOA is to maintain its current notional allocation.

Over the evaluation reference period, program delivery for ICCI and GOA was facilitated by an on-line administrative system supported by a VTC-based electronic platform that allowed registered clients to submit applications for funding on-line. The system however is based on archaic IT architecture which has challenged program administrators and beneficiaries alike.  Efforts are ongoing to replace these administrative systems with a more user friendly system. Although the evaluation noted gaps and discrepancies in program data housed in the program’s varied data repositories, such gaps and discrepancies observed could be addressed with a more harmonized approach to data recording and more frequent updates. On balance, the IT infrastructure in place and management of information was deemed adequate for program administration and accountability purposes.

Human resources dedicated to the program are adequate, but for the CAU which has assumed responsibilities beyond the GCSP. Though sector officers charged with administering GOA expressed concern about being torn between the competing priorities of their regular duties and their administrative responsibilities to GOA, these issues could in part be mitigated by adjustments to the sub-program’s planning cycle. Further, though training is informal and ad hoc, with the exception of GOA training, guidance materials and counsel provided by senior management, the CAU and SMFC appears to be sufficient to support effective program delivery. As remarked earlier, though sector officers charged with administering the GOA conveyed discomfort with the provision of advice on such matters as eligible expenses and in reviewing expense claims, they do have the benefit of the counsel of the CAU and SMFC in the exercise of these duties.

In the area of performance monitoring and results reporting, senior management has demonstrated exceptional due diligence as evidenced by its development of a performance measurement framework and reporting templates for beneficiaries in accordance with the former, the tracking and recording of this information in a systematic fashion, the administration of two beneficiary surveys to assess program contributions to outcomes, and the consolidation of all this information in annual performance reports. This said the current performance measurement framework is in need of refinement to better align activities with objectives and higher order outcomes. Further, indicators currently in use to track outputs and outcomes are too numerous, not complementary and of questionable relevance in many instances, all of which have undermined the reliability and utility of this performance information which has been collected and assembled at considerable expense.

In sum, the three sub-programs of the GCSP all produce results which contribute to Canadian economic growth and prosperity which can be calculated in the millions of dollars, far in excess of program costs. Though it is near impossible to determine whether amalgamation has resulted in cost savings of the magnitude originally expected, it would be hard for the program to be managed with fewer resources. What can be said is that amalgamation has contributed to efficiencies and economies in program delivery by allowing resources to be reallocated between the sub-programs in accordance with need. As such, the GCSP is, on balance, good value for money.

7. Recommendations

Based on the foregoing analysis, the following recommendations are presented for consideration.

Recommendation 1: That DFATD continue address those factors cited as probable contributors to GOA’s lapses, including factors relating to the sub-program’s planning cycle.

Despite efforts to address GOA’s lapsing challenges, this sub-program continues to register large and persistent year-end lapses in excess of 28%. Numerous causes for these lapses have been proffered by program management and Departmental audits, however the most common causes cited by stakeholders relate to the sub-program’s current planning cycle and delays in the release of adjudication results. These issues will need to be addressed if GOA is to retain its current allocation level.

Recommendation 2: That DFATD continue efforts to upgrade and refine its IT infrastructure to support both program delivery and administration thereof.

While current IT capacity is adequate to support program delivery, the evaluation observed continued challenges with the same that impede the program’s ability to evolve these systems and which may, if not addressed, compromise the efficiency of program management and effectiveness of program delivery. Further, the evaluation observed some lapses in the updating of information in its systems, thereby undermining the accuracy and reliability of information used to support reporting and decision making. As such, continued efforts are required to upgrade the systems in place and update the same with accurate and timely program data.

Recommendation 3: That DFATD review the HR needs of the CAU with the view to making such adjustments as may be needed to support and maintain the quality of services provided.

Program delivery is supported by a matrix management approach whereby Departmental personnel are assigned responsibilities specific to the GCSP in addition to their regular responsibilities. While this approach engenders certain challenges, the evaluation found on the whole that HR dedicated to program delivery is adequate. This said the CAU has found itself overburdened, in part due to recent loss of a financial officer, and due to the fact that the CAU has assumed administrative responsibilities over and above what was originally planned for it. Accordingly, the complement of staff dedicated to the CAU needs to be reviewed to ensure that the high quality of service it provides is preserved.

Recommendation 4: That DFATD revise its current performance measurement framework and related indicators with the view to simplifying the same and improving the relevance and utility of these instruments to support decision making.

Program management has demonstrated exceptional due diligence in the monitoring and reporting on performance data. This said, the current performance measurement framework is in need of refinement to better align activities with objectives and higher order outcomes. Further, indicators currently in use to track outputs and outcomes are too numerous, not complementary and of questionable relevance in many instances, all of which have undermined the reliability and utility of this performance information which has been collected and assembled at considerable expense. Accordingly, there is a need to review and revise the program’s performance measurement framework and related performance indicators to enhance the utility of this information for decision making.

8. Management Response and Action Plan

Recommendation 1

That DFATD continue to address those factors cited as probable contributors to GOA’s lapses, including factors relating to the sub-program’s planning cycle.

Despite efforts to address GOA’s lapsing challenges, this sub-program continues to register large and persistent year-end lapses in excess of 28%. Numerous causes for these lapses have been proffered by program management and Departmental audits, however the most common causes cited by stakeholders relate to the sub-program’s current planning cycle and delays in the release of adjudication results. These issues will need to be addressed if GOA is to retain its current allocation level.

Associated Findings: 15, 22, 24 and 25

Management Response & Action Plan:

Management of the GOA sub-program acknowledges that the nature of this component’s risk sharing design, combined with international business development projects often tied to industry specific events abroad, creates a significant potential for project cancellations and resultant lapses. To mitigate lapses, the GOA sub-program, with support of CAU, will continue to: challenge high estimate line item proposals; work with recipient associations to re-profile any unused funds from cancelled and partially claimed projects by way of a modification; maintain routine program year monitoring efforts to identify early on, projects that will not be fully realized so that funds can be re-profiled under GCSP; and use past performance as one of the key criteria in the adjudication of applications.

To eliminate or reduce delays pertaining to launch and release of results, GOA sub-program has advanced start dates of many elements and stages of the program to ensure additional lead time is built into various stages. GOA will continue to adjust and advance commencement dates in order to get approvals necessary to meet deadlines. Moreover, GOA will consider the recommendation of advancing the annual program application cycle by up to one month for the 2016-17 funding round to allow for earlier notification of results so associations can better plan for projects that take place early in the fiscal year.

The GOA sub-program will continue to approve proposals in a competitive manner with the aim of approving those that can be most realistically undertaken. As noted, one of the key criteria in the application evaluation process will continue to be applicants’ past performance which includes: planning and follow through on approved projects, as well as historical management of allocated funding. To further reduce lapses, GCSP and GOA will review and recommend adjustments to the sub-component baseline budget and overcommitment rates to better reflect GOA’s historical expenditure averages.

Responsibility Centre:

• GCSP/GOA - BBI

Time Frame: On-going

Recommendation 2

That DFATD continue efforts to upgrade and refine its IT infrastructure to support both program delivery and administration thereof.

While current IT capacity is adequate to support program delivery, the evaluation observed continued challenges with the same that impede the program’s ability to evolve these systems and which may, if not addressed, compromise the efficiency of program management and effectiveness of program delivery. Further, the evaluation observed some lapses in the updating of information in its systems, thereby undermining the accuracy and reliability of information used to support reporting and decision making. As such, continued efforts are required to upgrade the systems in place and update the same with accurate and timely program data.

Associated Findings: 22 and 23

Management Response & Action Plan:

Since the evaluation took place the amalgamation of the new Department led to a number of changes, including the phased-in transition of former DFAIT Grants and Contributions programs to an integrated portal/online system based on the former CIDA systems. One of the features of the upcoming system is that it is directly linked to the departmental financial system, thereby eventually reducing the number of steps in processing files and increasing accuracy and reliability of information in the systems. ICCI is piloting the transition to the new IT systems for the 2015-16 program year, for which interim solutions have been developed. Systems will be improved and required reporting modules will be added in the coming year. It is anticipated that the other GCSP Grants and Contributions programs will over time transition to these new systems.

Responsibility Centre:

• GCSP/ICCI – BIS
• GCSP/GOA – BBI
• GCSP/CAU – BPP

Time Frame: 2015-16 Application launched September 2014 (on-going)

Recommendation 3

That DFATD review the HR needs of the CAU with the view to making such adjustments as may be needed to support and maintain the quality of services provided.

Program delivery is supported by a matrix management approach whereby Departmental personnel are assigned responsibilities specific to the GCSP in addition to their regular responsibilities. While this approach engenders certain challenges, the evaluation found on the whole that HR dedicated to program delivery is adequate. This said the CAU has found itself overburdened, in part due to recent loss of a financial officer, and due to the fact that the CAU has assumed administrative responsibilities over and above what was originally planned for it. Accordingly, the complement of staff dedicated to the CAU needs to be reviewed to ensure that the high quality of service it provides is preserved.

Associated Findings: 26

Management Response & Action Plan:

Management is aware of the tenuous situation of the CAU, and currently trying to staff indeterminate employees. The CAU may be required to service new programs, and management will be looking at the proper complement of officers.

Responsibility Centre:

• BPP (formerly GPMC)

Time Frame: On-going

Recommendation 4

That DFATD revise its current performance measurement framework and related indicators with the view to simplifying the same and improving its relevance and utility of these instruments to support decision making.

Program management has demonstrated exceptional due diligence in the monitoring and reporting on performance data. This said, the current performance measurement framework is in need of refinement to better align activities with objectives and higher order outcomes. Further, indicators currently in use to track outputs and outcomes are too numerous, not complementary and of questionable relevance in many instances, all of which have undermined the reliability and utility of this performance information which has been collected and assembled at considerable expense. Accordingly, there is a need to review and revise the program’s performance measurement framework and related performance indicators to enhance the utility of this information for decision making.

Associated Findings: 29, 30 and 31

Management Response & Action Plan:

Management in its continuous effort to improve monitoring and reporting on performance will undertake review of its current performance measurement framework in an effort to better align activities with objectives and higher order outcomes. Further, management will undertake review of the indicators currently used to track outputs and outcomes with a view to confirming relevancy and reducing collection of information where not considered complementary for decision making purposes.

Responsibility Centre:

• GCSP/ICCI – BIS
• GCSP/GOA – BBI
• GCSP/CAU – BPP

Time Frame: February 2015