Evaluation of the Geographic Coordination and Mission Support Program

Final Report
Prepared by the Diplomacy, Trade and Corporate Evaluation Division (PRE) in the Evaluation and Results Bureau (PRD)
Global Affairs Canada

January 2020

List of Acronyms

ADM

Assistant Deputy Minister

CBSA

Canada Border Services Agency

CEP

Commercial Economic Plan

CFLI

Canada Fund for Local Initiatives

COSMOS

Consular Management and Operations System

CSF

Client Service Fund

DG

Director General

DPD

International Assistance Operations

DPI

International Assistance Programming Process and Coordination

ELC

Strategic Planning and Coherence Unit for Europe, Arctic, Middle East and Maghreb Branch

EGM

Europe, Arctic, Middle East and Maghreb Branch

FAS

Finance and Administration System (SAP)

FINSTAT

Monthly Financial Status Report

FPDS

Foreign Policy and Diplomacy Service

FTE

Full Time Equivalent

GoC

Government of Canada

Geo

Geographic Branch

HOM

Head of Mission

HQ

Headquarters

IA

International Assistance

ICF

Integrated Country Framework

IFM

International Security and Political Affairs Branch

ITSF

Integrated Trade Strategy Fund

IRCC

Immigration, Refugees and Citizenship Canada

LCM

Public Affairs Branch

MCF

Mission Cultural Fund

MSR+

Management Summary Report +

NGM

Americas Branch

NNB

North America Advocacy Unit

NDS

Strategic Planning, Operations and Policy Division for Americas Branch

NMD

Geographic Coordination and Mission Support Bureau

NMO

Geographic Coordination Unit

NMS

Mission Support and Advocacy Unit

NMV

Immigration, Refugees and Citizenship Canada / Canada Border Service Agency Coordination Unit

NMZ

Blueprint 2020 Unit

OAZ

Corporate Planning, Policy and Pan-Asian Regional Programming Division for Asia Pacific Branch

OGM

Asia Pacific Branch

OGDs

Other Government Departments

PIF

Post Initiative Fund

PIPs

Performance Information Profiles

PRE

Diplomacy, Trade and Corporate Evaluation Division

PRIME

Physical Resources Information Mission Environment

SAP

Finance and Administration System (FAS)

SIMS

Security Information Management System

TRIO 2

The Client Relation Management System for Trade Commissioner Service

WFO

Operations, Planning and Strategic Coordination Division for Sub-Saharan Africa Branch

WGM

Sub-Saharan Africa Branch

Executive Summary

Background

As part of Global Affairs Canada’s Five-Year Departmental Evaluation Plan, the Diplomacy, Trade and Corporate Evaluation Division (PRE) conducted an evaluation of the Geographic Coordination and Mission Support Program. The Program is delivered by the Geographic Coordination and Mission Support Bureau (NMD) within the Americas Branch. The Bureau has the complex mandate of coordinating geographic branches and other stakeholders to ensure coherence within the Geo network and to support planning, reporting, traditional and digital advocacy, and operations of missions.

The purpose of the evaluation was to assess: 1) NMD’s overall mandate, roles and responsibilities; 2) NMD’s role in providing support to missions; 3) The use of tools and data for planning and reporting purposes (with an emphasis on Strategia); and 4) Linkages with other departmental divisions with mission support functions.

Findings

Overall, the evaluation found that the individual units within NMD are meeting their mandates, and the activities these units conduct serve an ongoing need. As a whole, there is a strong need to clarify the overarching mandate, roles and responsibilities of the bureau. Furthermore, the planning and reporting tools that NMD administers are important and necessary for the department. Specific issues were raised with the multiplicity and lack of interoperability of departmental planning and reporting tools.

The evaluation found that while NMD plays an important coordination and support function, it is often confused with a “catch-all” bureau consisting of several units with diverse roles and responsibilities. In addition, the overall role of the Bureau does not appear to be well-known or understood within the Department.

The evaluation found little duplication of functions with other departmental divisions. The mission support services that NMD provides on advocacy and digital engagement are distinct within the Department and serve missions well, with only a few noted areas for improvement. While NMD’s geographic coordination unit role is unique within the department, its mandate could be better communicated.

Finally, the evaluation found that the department’s multiple planning and reporting tools have been a cause of frustration for users, and may undermine the effective use of Strategia as an integrated planning, monitoring and reporting tool. While significant work has been done to improve and simplify Strategia, system use and functionality issues remain. The incomplete reporting by users as well as the inconsistent headquarters engagement in the planning and reporting process affect both the quality of Strategia data and its subsequent use for decision-making. Strategia has also not had a permanent budget allocation since its inception, this could slow down future improvements to the system.

Recommendations

The following recommendations were derived from the evaluation findings, considerations and conclusions:

1. It is recommended that NMD, in collaboration with the geographic branches, develop and share across the department a Bureau Plan that clearly defines its distinct roles and responsibilities, and resourcing requirements.
2. It is recommended that NMD, in collaboration with other relevant departmental stakeholders, develop an approach to strengthen the Department’s planning, monitoring and reporting tools.

Program Background

Text version

The timeline of key events from 2013 to 2018 for the Geographic Coordination and Mission Support Bureau (NMD).

2013 – Amalgamation of the department
2014 – Creation of NMD, with units NMS, NMO, and NMV
2015 – NMZ is added to NMD
2017 – Mission Cultural Fund is created.
All International Assistance files (and 4 FTEs) move from NMO to International Assistance Operations (DPD)
2018 – Canada Fund for Local Initiatives transfers to NMS from I-Branch.

Text version

This organizational chart shows that the Geographic Coordination and Mission Support Bureau is comprised of four divisions. Each box in the chart includes the name of the unit, an associated icon, and a description of the mandate.

The Geographic Coordination and Mission Support Bureau (represented by an icon of two people with a hand underneath in a circle) is responsible for fostering relationships with missions, branches and other stakeholders; develop, streamline, and manage corporate systems, tools, processes, services and structures; promote knowledge, networks, and results-based approaches to support performance and reduce workload; and encourage cohesion between and within missions and headquarters (HQ).

The Mission Support Division (NMS) includes two units. The first unit (represented by an icon of a computer) is responsible for providing planning and reporting services (e.g. Strategia and MyInternational) to missions. The second unit (represented by an icon of a maple leaf on a cell phone) is responsible for developing advocacy campaigns, promote culture, and manage three funding programs (Post Initiative Fund (PIF), Mission Cultural Fund (MCF), and Canada Fund for Local Initiatives (CFLI)) for missions.

The Geographic Coordination Division (NMO) (represented by an icon of a globe) is responsible for facilitating discussions and improve information sharing and collaboration between geographic branches to strengthen overall coordination.

The IRCC/CBSA Coordination Division (NMV) (represented by an icon of three people connected by a circle) is responsible for coordinating Global Affairs’ relationship with Immigration, Refugee and Citizenship Canada (IRCC) and the Canada Border Services Agency (CBSA).

The Blueprint2020 Division (NMZ) (represented by an icon of a lightbulb) is responsible for leading the Public Service Renewal efforts at Global Affairs Canada, with a mandate to create a modern and open public service.

*See Appendix A - E for a detailed overview of human resources information for each unit.

Program Resources

Financial Resources

NMD’s total annual budget varies year to year, as its structure and purpose is to support missions and fund programs whose needs and requests vary. Planned expenditures for 2014-15 to 2017-18 averaged $3,032,496 per year, whereas expenditures for 2018-19 increased to $5,115,000 due to the addition of the CFLI funding program to NMD’s portfolio.

Human Resources

NMD has a total of 33 full-time equivalents (FTEs), including the Director General (DG).

Funds

NMD currently administers three funding programs, the Post Initiative Fund (PIF) with a budget of $1.5 million (with an additional reserve of $150,000 for additional funding requests); the Mission Cultural Fund (MCF) with a budget of $1.75 million; and the Canada Fund for Local Initiatives (CFLI) with a budget of $14.7 million.

*See Appendix A - E for a detailed overview of human resources information for each unit.

Evaluation Rationale

As part of Global Affairs Canada’s Five-Year Departmental Evaluation Plan, the Diplomacy, Trade and Corporate Evaluation Division (PRE) conducted an evaluation of the Geographic Coordination and Mission Support Bureau (NMD) in 2018-19, listed as a Program under the new Program Inventory. There has been no previous evaluation undertaken of Geographic Coordination and Mission Support.

It is important to note that while NMD is a “Program” under Global Affairs’ Program Inventory, in reality it is a set of distinct units with specific activities. The evaluation took an organizational assessment approach and assessed the services provided by NMD to missions and geographic branches. The evaluation scope takes into consideration the Performance Measurement and Evaluation Committee’s interest in understanding NMD’s role: “what does it do, and what should it do?”.

Evaluation Scope

The evaluation covered the period from 2013-14 to 2017-18, and the scope of the evaluation included the following:

• NMD’s overall mandate, roles and responsibilities;
• NMD’s role in providing support to missions for planning and reporting (including Strategia), and advocacy and digital diplomacy services;
• Use of tools and data for planning and reporting purpose (with an emphasis on Strategia), linkages to decision-making and the departmental data strategy;
• Linkages with other departmental divisions with mission support functions.

It is important to note that CFLI was not in the scope of this evaluation as it joined the Bureau in July 2018, which was not covered under the evaluation period. The evaluation also did not look into the performance of the MCF and the PIF.

Evaluation Questions

1. What are the main roles and responsibilities of NMD?
2. Is there duplication and/or complementarity of responsibilities between NMD and other divisions or branches? Are there any gaps?
3. Are NMD’s mission support activities meeting the needs of missions abroad? Are there any gaps?
4. Is NMD in a position to effectively support missions (e.g. tools, information sharing, documentation/reports, training, human resources, etc.)?
5. What data is being generated by NMD-supported tools? How is this data meeting reporting needs and contributing to timely and effective decision-making at missions and at HQ?

Evaluation Methodology

The evaluation used a mixed-methods approach where data was collected from a range of sources to ensure multiple lines of evidence when analyzing data and formulating findings. While examples are used for illustrative purposes throughout the report, each finding was triangulated using evidence from a mix of quantitative and qualitative data, and the recommendations provided are based on these findings. The methods/data sources listed below were used to answer the evaluations questions.

Key Stakeholder Interviews (n=28)

Semi-structured individual interviews were conducted in-person and via telephone with key stakeholders including DGs, Directors, Deputy Directors, and Heads of Unit, with a focus on the geographic branches and NMD. The questions were based on background knowledge and roles and responsibilities related to the evaluation areas of focus.

Survey (n=674)

An electronic survey was administered to 1,926 Canada Based Staff - including Heads of Mission and Program Managers – as well as Locally Engaged Staff in all missions, with a raw response rate of 35%. This solicited feedback and input on the views of mission staff on Strategia (contribution and duplication); advocacy, digital engagement, or cultural diplomacy tools; the management of funding programs (MCF, PIF, CFLI); and Blueprint 2020.

Focus Group Discussions (n=6)

Focus group discussions were conducted with HQ staff to acquire feedback on NMD’s role in planning and reporting. The focus was on obtaining a better understanding of what data is being generated by Strategia, and how this data is meeting the reporting needs and contributing to timely and effective decision-making at missions and HQ.

Document and Literature Review

A document review was conducted to gather information related to the evaluation questions. Sources included: annual reports and plans; Performance Information Profiles; results of NMD-conducted surveys (e.g. Results of 2018 Global Head of Missions Meeting Survey); NMD-generated reports; and tools and databases (e.g. Strategia, MyInternational). A review of academic literature was also conducted to develop a better understanding of the concept of coordination in terms of policy development and decision-making.

Limitations and Mitigation Measures

Confidentiality and bias: The evaluation of NMD required almost the entire staff of the Bureau to be interviewed. With some units only containing a few employees, this could lead to some comments being easily attributed to individuals. In addition, current staff might present bias or be reluctant to criticize the program.

• Mitigation: Interview participants were encouraged to be candid with their responses, and were assured that their input would remain anonymous. Interview information was triangulated with data collected through document reviews and survey data to reduce a potential for bias and to strengthen findings.

Availability and consistency of data: The evaluation team attempted to conduct case studies based on a number of mission reports in Strategia from 2014-15 to 2018-19, comparing data quality across the streams within a mission and across missions. Inconsistent and incomplete mission reports in Strategia revealed that this methodology could not be used to fairly assess data quality.

• Mitigation: Data quality was assessed through a triangulation of sources including an overview of various mission reports in Strategia, evidence of the use of Strategia data for decision-making, as well as interview and survey responses on perceptions of data quality.

Not a typical evaluation: Evaluations often measure a program’s success against stated outcomes. This evaluation was challenging because the Bureau consists of a set of units with different functions and the evaluation team found the outcomes of the Bureau broad and difficult to measure and compare.

• Mitigation: The evaluation team undertook a different approach for this evaluation, similar to an organizational assessment. The evaluation assessed the relevance of the Bureau as a whole, and assessed the effectiveness of the activities of each unit separately.

Findings

Finding 1: NMD plays an important coordination and support function with diverse roles and responsibilities.

NMD was identified through various interviews within the department as a “catch-all” bureau housing units that have no other logical placement. However, it was also widely recognized that in a large organization, it is often necessary to have such a bureau that can house outlying units and functions. In this case, NMD has taken on the challenge of creating a common vision and logic model that strives to bring together the various units that fall under its purview.

The Bureau’s vision is to “champion and coordinate missions and geographic branches by providing them with innovative solutions and enabling effective collaboration between partners to advance Government of Canada priorities”.  The vision itself - just as its logic model - is complex, attempting to incorporate all of its units with distinct mandates under one umbrella. Despite its efforts to create a cohesive Bureau, NMD’s individual units do not all interact or collaborate routinely.

NMS

All units within the Mission Support Division (NMS) provide direct mission support. The recent addition of the CFLI to NMS has been well-received, grouping all mission funds (PIF, MCF and CFLI) all under one division. The division is large in comparison to the other units within NMD, and some respondents believe it may warrant its own bureau. There are synergies within NMS where the planning and reporting unit works with the advocacy and digital engagement unit to conduct joint trainings. In addition, initiatives for the PIF and MCF are planned and reported in Strategia creating more linkages within NMS.

NMO and NMV

Staff indicated that all units within NMD leverage their proximity to the Geographic Coordination unit (NMO) to influence high-level meeting agendas. Apart from this, there is little interaction between the other units and NMO. The Bureau itself is looking at restructuring this unit, with potentially merging it with the IRCC/CBSA Coordination unit (NMV). Complementarities exist within these two units with similarities in their administrative roles in organizing meetings. However, the files are quite different, with NMV’s focus on coordinating the whole department on immigration-related files, and NMO looking at a variety of issues affecting geographic branches only.

NMZ

The Blueprint 2020 unit (NMZ) is widely recognized as an outlier within NMD. It does not directly provide geographic coordination or mission support. It has collaborated with NMS recently to publish MyInternational reports on a public website, however NMZ could collaborate with any unit outside NMD on similar activities.

“It is important to look at the diversity of files and see if the files are rightly placed within the group. CFLI and other funds are well-grouped together. But what about the rest?” - Geo DG Interview

Finding 2: While there are a number of benefits for NMD to be situated within a geographic branch, its role is not well-known or understood within the department.

The placement of NMD in the Americas Branch does not impede the delivery of its functions. Some stated there might be a potential for less access to the Assistant Deputy Minister (ADM) of the Americas Branch given the multiple responsibilities and large portfolio of the ADM.  However, in theory the DG of NMD reports to all Geographic ADMs. Most agreed that the Bureau should remain under a Geo to continue fostering closer relations, and were ambivalent which Geo it is ultimately located.

Some NMD staff raised concerns of not being as connected to other Geos, which affected their understanding of ongoing regional issues and changing priorities. An invitation to other Geo Town Halls and retreats was suggested as an option to better understand all geographic branch issues, provide an opportunity to further promote their work, and develop stronger connections.

Various stakeholders within the department revealed that NMD is recognized by its units and individual employees. Interviewees and survey respondents suggested NMD should better communicate their roles and responsibilities, including the services they provide. The Bureau does have a GCpedia webpage which outlines its vision, mission, and the responsibilities of individual units. However, it is unclear how often the webpage is updated as it includes outdated information and disabled links.

The Mission Support Division (NMS) is well-known for its work on Strategia as well as its advocacy services and funds. Individuals were able to name specific employees within NMS, as they were directly associated with the services they provide. This is due to the fact that they have been working at Global Affairs in the same or closely related roles for many years.

“I was not really aware of [NMD’s] services. Somehow you must market yourselves more loudly.”  - Survey respondent (mission staff)

“I am unaware of NMD [or its] support. More should be communicated about NMD and their role…”- Survey respondent (mission staff)

The coordination work of NMO and NMV were also recognized by interviewees with a basic awareness of what their activities were. NMO was best known for its Secretariat role in coordinating Geo DG and Geo ADM meetings, however its other activities were not known or understood. Some Geos were aware of NMV’s activities as they worked closely together on certain files, while others wanted more information about NMV’s work and how it could help their regions, for example on particular visa matters. Moreover, some respondents suggested that the department create similar groups to coordinate with other government departments.

The survey revealed that only one third of mission staff were aware of Global Affairs-led Blueprint 2020 initiatives, with some stating they did not know what Blueprint 2020 was and how it related to their work. Among those aware, only 1 in 5 participated in Blueprint 2020 initiatives led by NMZ.

* See Appendix F for an adapted logic model of NMD.

Finding 3: NMD’s geographic coordination unit role is unique within the department and its mandate could be better communicated. 

Part of NMD’s mandate is to strengthen the overall coordination between geographic branches. It does this through coordinating input (e.g. identifying issues that are unique to Geo branches; conducting research and analysis on these issues; and assigning tasks to Geos); representing Geo views at inter- and intra-departmental fora; and facilitating dialogue through organizing meetings.

Key stakeholders revealed that overall, the meetings that NMD organizes are useful and productive forums for Geos to get together. The biweekly Geo DG meetings had especially high praise, with DGs stating that these meetings were important, helped with coordination, had useful content, and were effective in bringing the Geos together.

“Geo DG meetings are important. [They are] a good and useful forum.” - Geo DG Interview

However, the attendance record tracked by NMD for Geo DG meetings showed that the average Geo DG attendance was relatively low. This may be due to the fact that Geo DGs found it difficult to attend biweekly meetings because of irregularly scheduled meeting days and times. Meanwhile, Geo DGs have been sending their Geo DG Advisors to the meetings they cannot attend. This makes the Geo DG Advisor meetings that NMD also organizes seem repetitive.

NMD, in its overall geographic coordination role, faces various challenges. More specifically, some of the obstacles include:

• Lack of effective two-way communication between NMD and the Geos. NMD does not articulate its roles and responsibilities well, meaning their activities are not well-known or understood among the Geos. This in turn leads to the Geos not expressing their coordination needs to NMD, as they are unaware of the services offered.
• Geographic branches are region-specific and therefore may prioritize different issues. Geos lobby for issues that their branch prioritizes, and do not have an incentive to coordinate on issues they are not invested in. Further, some Geos claim they do not have a coordination need – e.g. NGD does not manage missions, and NND is already connected to missions, so they do not use (or need) NMD services as much. Finally, other Geos question the need for horizontal coordination, stating that branch-wide coordination is sufficient. They state that this is especially the case as each Geo has their own internal coordination unit.

Each Geo contains its own internal coordination unit that deals with Branch-specific issues and corporate tasks. There are also department-wide coordination units such as the International Assistance Programming Process and Coordination unit (DPI), which manages coordination for the international assistance stream across the whole department.

In theory, there is no major duplication between NMD and other coordination units. This is because the roles and responsibilities of the individual coordination units within other branches apply only to their specific region, whereas NMD focuses on coordination between Geos, meaning it has a much larger scope. However, in practice, interviewees from certain coordination units stated that there is slight overlap. Despite this, most Geos did not find duplication between their internal coordination unit activities and the activities of NMD.

There is a continued need for coordination between Geos, providing a purpose for NMD’s Secretariat function. However, given the myriad of coordination units the likelihood of overlap and/or duplication is high.

There is a slight overlap between the coordination activities of DPI and NMD. In 2017, all international assistance (IA) files were moved from NMD’s purview to DPI’s. In theory there is no duplication because this separation is clear – DPI handles IA, and NMD handles all other coordination issues. However, there is potential for confusion over responsibilities, because NMD coordinates Geos on all topics, which should technically include IA. Since the division of responsibilities is fairly new, there still seems to be occasional overlap in the determination of responsibilities and division of files.

International Branch Coordination Units:

• ELC is the coordination unit for the Europe, Arctic, Middle East and Maghreb Branch (EGM). There is no duplication between ELC and NMD. Being responsible for the whole Geo world (NMD) is different from being responsible for a single branch’s needs. 
• NDS is the coordination unit for the Americas Branch (NGM). There is no duplication between NDS and NMD. NDS plays a coordination role for the Americas branch only, and a dedicated unit is required for its specific needs.
• WFO is the coordination unit for the Africa Branch (WGM). There is a slight duplication between WFO and NMD. WFO itself engages with other Geos on any of the large files where it deems a joint perspective is needed. It has a horizontal coordination focal point in its branch, making NMD’s activities duplicative at times.
• OAZ is the coordination unit for the Asia Pacific Branch (OGM). There is no duplication between OAZ and NMD. Similar to ELC and NDS, OAZ focuses solely on internal coordination within its branch, which is different than the horizontal coordination NMD provides.

Finding 4: NMD’s mission support services on advocacy and digital engagement are distinct from other similar units within the department, and overall serve missions well, with a few noted areas for improvement.

A key aspect of NMD’s mandate is to provide services surrounding advocacy and digital engagement for missions. From a services feedback questionnaire conducted by NMD, it appeared as though NMD was meeting the needs of missions. 78% of the respondents stated that information they received assisted in delivering their initiative/work. Despite the fact that there are other units within the department that deal with advocacy and digital engagement, NMD’s services were deemed distinct from the activities of other units. The two main services the advocacy unit within NMD provides and is known for can be split into two categories: (1) trainings (NMD conducts in-person and online training sessions for advocacy, culture and mission planning and reporting); and (2) various advocacy services.

NMD-Conducted Trainings

• In-person regional trainings were well-received by missions based on post-training surveys conducted by NMD. The feedback was overall extremely positive.
• NMD also has various online trainings and toolkits, which were found to be helpful for users. 87% of survey respondents found online training sessions useful, and 86% found the unit’s GCpedia page useful. However, human interaction was preferred over technology in terms of training.
• 94% of survey respondents stated that in-person regional effective advocacy training was moderately, very, or extremely useful.

Challenges with Trainings

• The lack of engagement in French was said to be a barrier to the use of services, and bilingual training/support was identified as a weakness.
• Survey participants identified a need for more regional and pre-posting trainings, and the lack of frequency of these trainings was highlighted as a significant weakness.
• “The lack of capacity to engage in French has been a significant barrier.” - Survey Respondent

Advocacy Services

• The most used tools and services were identified as: thematic toolkits; image bank and graphics for social media; MCF guidelines; PIF guidelines; and advocacy campaign reports.
• GCpedia pages on advocacy, digital engagement and the cultural diplomacy hub were deemed moderately to extremely useful by survey respondents.
• Users were generally satisfied with the services provided by advocacy strategists.
• 89% of survey respondents were satisfied with the service provided by advocacy strategists, particularly response times.

Challenges with Advocacy Services

• Survey and interview respondents commented about needing more updated and localized content. This was in order to allow missions to build these tools into their plans more effectively.
• There was also a request for more resources that could help contribute to a higher volume of services, dispersed across a wider range of missions.
• “Missions cannot be successful at carrying out advocacy when HQ doesn’t itself know what our objectives and strategies are in country X.” – Survey Respondent

The advocacy, digital engagement, and culture services NMD provides to missions are distinct from other similar units within Global Affairs. There are two other units within the department that provide advocacy and digital diplomacy services: LCM (Public Affairs) and NNB (North America Advocacy).

LCM has a mandate of managing “Global Affairs communications, (...) and has overall accountability for all communications activities undertaken by the department in Canada and internationally.” The branch does this through: communications (delivery of ministerial messages); public affairs (public enquiries, website management, etc.); public diplomacy (engendering a favourable image of the state); and digital diplomacy (use of information and communications technologies in the conduct of diplomacy).

In theory, when looking at the mandates and roles of the branches, there is overlap between LCM and NMD. In practice however, according to a review conducted for the department in 2017, NMD has taken over the work that LCM cannot conduct, such as social media advocacy. This is because LCM does not have the financial or human resources to go beyond their role of ministerial and media communications. Therefore, there is no unnecessary duplication. The division of roles does not appear to be problematic, and in fact there is widespread support for the dispersed model. It allows the LCM to focus on ministerial communications, while missions are “well-served” by NMD.

NNB manages the Government’s advocacy efforts in the United States and Mexico. NNB coordinates the development of advocacy strategies and tools across the North American network; manages the planning, reporting and performance measurement of key advocacy programs; and analyzes the results of advocacy activities, among other activities.

While there are definite linkages between NMD and NNB, there is ultimately no duplication of activities between the two units because the NNB advocacy is solely focused on the U.S. and Mexico. According to interview respondents NMD and NNB have worked closely together on some initiatives, with NNB making use of NMD toolkits and resources. However, more synergies could be leveraged between NMD and NNB, especially if resources were shared between the two units. With missions requesting more focus on trade, development and consular work, NMD could leverage trade expertise from NNB to create more targeted campaigns outside of the U.S. Additional synergies with other branches could also assist with creating local or specialized advocacy content for missions.

Finding 5: In the absence of a Chief Foreign Affairs Officer or a unit responsible for Foreign Policy and Diplomacy Service, NMD has taken on this role.

In integrating the Foreign Affairs, Trade and Development programs, the intent was to ensure that decision makers were provided with information, analysis and advice in order to establish integrated and well-informed decisions that advance Canada’s interests and values internationally. In this optic, a series of champions for each stream were named to ensure horizontal integration in the newly amalgamated department and ultimately, to complement each other.

With changes in senior management, the position for the Chief Foreign Affairs Officer has remained vacant and there has not been a dedicated secretariat for Foreign Policy and Diplomacy Service (FPDS).  Although not all attributable to this shortage, the lack of a centre of expertise for FPDS has had an impact on the departments’ capacity to report on related activities. Overall, political reporting is considered weak throughout the department, with performance indicators mainly linked to advocacy.

Given NMD’ expertise in advocacy, the division houses the role of Head of FPDS for planning and reporting and has taken on the responsibility of collecting and reporting on FPDS indicators as well as developing a client management relation tool similar to that of the Commercial stream. However, interviewees recognized that the location of the Head of FPDS planning and reporting within the same unit as the developers of Strategia could be considered a conflict of interest, as all the other Heads of planning and reporting for the other streams are located in their respective

sectors of expertise. This has contributed to a perception that Strategia is primarily an FPDS tool rather than a corporate planning and reporting tool.

While NMD contributes to the department’s horizontal coherence through its advocacy and planning and reporting tools, there remains a need for an FPDS community based on robust performance measures that deepens amalgamation and promotes co-operation and collaboration.

Finding 6: Over time, significant work has been done to improve and simplify Strategia. Despite this, system use and function challenges remain.

Strategia’s predecessor is the Commercial Economic Plan (CEP), which was initially developed for the commercial stream as an accompanying tool to the Client Relationship Management system used by the Trade Commissioner Service (TRIO 2) to report on commitments. The tool was then adapted to include all streams and launched in November 2013 as Strategia.

There have been significant improvements made to Strategia since its initial launch. Overall improvements including software updates and alignment of timelines with corporate processes.  Interviewees as well as survey respondents stated that Strategia is fairly easy to navigate and has become more user-friendly over the years, especially in comparison to other tools such as the Finance and Administration System (SAP/FAS). Strategia’s overall layout has remained consistent throughout the various iterations, helping mission staff familiarize themselves with the tool’s interface, leading to increased use over time.

There have also been stream-specific improvements, such as the introduction of cost drivers in the program budget sections, allowing missions the opportunity to give short “sales pitches” on why more funding is needed. In addition, recent improvements to the Common Services tab have meant that outcomes are pre-populated with an option to customize for commitments, which users have found helpful.

As Strategia evolves, it has started to incorporate existing corporate planning and reporting processes. For example, the department piloted Integrated Country Frameworks (ICFs) for selected countries in 2017 to improve the integrated planning process. However, the pilot was discontinued to avoid duplication because it was found that much of the information requested was redundant with mission Strategia plans.

According to interviewees and focus group participants, overall the department is pleased with NMD and its efforts. It was well-recognized that NMD has made continual improvements to Strategia, and has been responsive to errors and assisting with making changes. It was also acknowledged that NMD is under-resourced, yet despite this, the planning and reporting team remains responsive, driven and capable of undertaking the complex task of developing and rolling out Strategia.

Strategia Timeline

• 2014 – Strategia 1.0: The first version of Strategia included only mission planning; had consolidated key mission information (context, commitments, reporting); and had standardized budget and planning processes.
• 2015 – Strategia 2.0: The second version of Strategia included an additional tab for Security and Emergency Management, and Other Government Departments.
• 2016 – Strategia 3.0: Strategia 3.0 included an additional tab for Multilateral, an additional tab for Development/IA, aligned its timeline with the corporate planning cycle, included a country strategy field, and integrated HQ by including branch and bureau plans.
• 2019+ - Strategia 4.0: Currently, Performance Information Profiles (PIPs) are being integrated into Strategia. Strategia 4.0 is still in development, but the priorities for the new version are to: allow for multi-year initiatives; reinforce monitoring capacity; improve both government and mission coordination; enhance coordination between HQ and missions; reduce duplication; support corporate performance measurement; and ensure coherence with other databases.

Despite its many improvements, Strategia faces a number of challenges in meeting the distinct needs of all streams at Global Affairs. Interviews, focus group discussions and survey results identified a number of challenges missions face when using Strategia. Some challenges are technical in nature, while others are more conceptual and would require significant thought and investment in redesigning the platform.

Finding 7: The department’s multiple planning and reporting tools undermine the effective use of Strategia as an integrated planning, monitoring and reporting tool.

There are multiple corporate planning and reporting tools used by both HQ and mission staff (see Appendix G for a detailed overview of corporate reporting tools). Overall, the evaluation found little duplication between the tools, as each tool has a distinct purpose. However, where duplication exists, it has been a cause of frustration for users.

Duplication most often occurs when Strategia is used for reporting. For example, interviewees explained that financial reporting already occurs in other systems such as SAP/FAS and FINSTAT. To report on the allocation of resources in Strategia as well, becomes duplicative. Survey respondents agreed, with 29% finding “a lot or a great deal” of duplication between Strategia and the financial and administrative systems. Duplication was also highlighted for reporting on results for commercial initiatives, as key performance indicators are initially entered in TRIO 2, and then re-entered into Strategia at year-end. While the international assistance stream is integrated into Strategia, reporting on project-level results takes place in the Management Summary Report+ (MSR+), an offline spreadsheet. The stream is currently exploring developing a separate tool for reporting on projects, which could lead to potential for duplication with Strategia.

Interview and survey respondents indicated a need for more interoperability and dialogue between the systems in order to avoid duplication and maximize efficiency.

“All streams complain about the multitude of systems, and the fact that they are distinct. The systems and tools should all be better connected.” - Geo DG Interview

Strategia is intended to be a planning, monitoring and reporting tool, however it appears that most streams use the tool for planning purposes largely related to resources requests. For example, according to HQ staff, Trade Commissioners at mission primarily use Strategia to plan initiatives to apply for funding through the Integrated Trade Strategy Fund (ITSF) and the Client Service Fund (CSF).

A survey sent to mission staff revealed that 85% of respondents use Strategia for priority-setting, 81% for initiative planning, while 77% of respondents use Strategia primarily for requesting funds and financial resource planning. Based on program documents, missions in general respect planning deadlines, with 95% of missions submitting plans on time for the 2018-19 cycle. Survey results, interviews and a review of selected mission Strategia reports revealed that Strategia is not used widely as a monitoring tool, with mission staff using it primarily at the beginning and end of the annual reporting cycle. While 74% of survey respondents indicated reporting as a main use of Strategia at mission, program data revealed that the average level of activity in Strategia during the reporting period of 2018-19 was low.

Percentage of survey respondents found Strategia:

• 80% - Moderately to very useful for planning.
• 41% - Moderately to very useful for monitoring.
• 56% - Moderately to very useful for reporting.

Strategia is designed as an integrated planning, monitoring and reporting tool that, when “used properly, [should] help bring coherence to planning across the department”. However, in practice integrated planning at mission does not always occur.

The survey revealed that Heads of Mission (HOMs) consult the most frequently with the other streams when planning priorities. This is understandable as the HOM tab in Strategia was created to support HOMs in identifying the key mission priorities they will be supporting in the upcoming year, thus requiring consultations with all areas. Survey evidence showed that Common Services also consults with the other streams, especially Security and Emergency Management, before developing plans in Strategia. This is likely due to the fact that Common Services is responsible for the management of mission operations, including procurement, real property, and human resources.

While all streams at mission follow the same planning and reporting timelines, the International Assistance (IA) stream observes a slightly different timeline, as their planning and reporting is dependent on project-level data that is collected from external partners. With IA on a staggered timeline, this may affect the ability of the mission, as a whole, to conduct integrated planning.

Overall, there is little coordination across the streams when developing mission plans. Branches believe that missions mostly use Strategia for reporting independently. However, if it was used more strategically by all streams at mission for integrated planning, more cohesion could be found.

“Strategia is much better than it used to be to pull the streams together. Each section reports differently and integration only happens at the HOM level. In principle there is integration, but in practice there could be more integration.” - Geo DG Interview

*See Appendix G for a detailed overview of corporate reporting tools.

Finding 8: The quality of Strategia data is hampered by inconsistent and incomplete reporting.

Reporting in Strategia by missions on results of initiatives and overall outcomes lacks detail or is incomplete. Respondents explain this may be due to a number of factors.

Factors affecting data quality

There are some instances where missions use Strategia effectively, such as planning for resources. Some interview respondents believed that the value of Strategia input depends on the stream, with some streams utilizing the tool well based on reporting culture stemming from use of other tools such as TRIO 2 and SAP/FAS. There is a general agreement at headquarters that there is better quality of input and planning now, compared to earlier versions of the tool, and increased willingness to use the tool. Strategia is helping to enable missions to understand their role in corporate planning. However, a change is needed in the department to not only better understand the importance of data for the use of decision-making, but to also effectively use the tools available to capture results.

Case Studies – A failed attempt

The evaluation team attempted to review the data quality of a number of missions’ Strategia reports from 2014-15 to 2018-19. However, evaluators found that reporting was highly variable across streams and missions. Metrics for data quality included:

• Completeness – whether all the required/relevant fields were completed.
• Timeliness – whether the data was entered by the required deadline.
• Consistency – whether the data entered was consistent throughout the stream’s tab and in line with desired outcomes.
• Coherence – whether the data was complementary and in line with the desired outcomes/commitments/priorities of the mission/branch/ department.

This methodology was abandoned after evaluators found that upgrades to Strategia made it difficult to compare data as tool features, requirements for reporting, and reporting cycles changed over the years. In general, consistency of reporting was poor in the early years of the tool, with many sections left incomplete including the HQ feedback sections. Consequently, the evaluation team decided that a comparative analysis of selected mission reports would not offer an accurate assessment of the tool itself. However, such an analysis would be beneficial to assess the performance of missions in the planning and reporting in Strategia.

Data availability, consistency, and quality are all issues facing the department as a whole. A review of the findings from the past fifty evaluations conducted since 2014 by both the Diplomacy, Trade, and Corporate Evaluation Division and the International Assistance Evaluation Division at Global Affairs revealed that 28% of evaluations had findings related to poor data quality and the overall availability and accuracy of data. In addition, 32% of these reports cited either absent or poor quality performance measurement frameworks or systems. Strategia was referenced in several evaluation findings and conclusions, highlighting the lack of interoperability between reporting systems, user-error during data input, and incomplete Strategia reports.

In 2018, as a response to the data challenges faced by the department, Global Affairs developed a Data Strategy to maximize the value of data, strengthen capacity to use data, and improve performance through better utilization and management of data.

As part of the Data Strategy, branches have developed draft data plans. Strategia is included in various branch plans, mainly as a source for FPDS indicators. Branches have highlighted a number of challenges when having to rely on Strategia data for results reporting:

• In some cases, Strategia only reflects funded initiatives at missions;
• There is a gap in FPDS reporting of unfunded activities, such as engagements with interlocutors; and
• FPDS activities undertaken at HQ are not captured in Strategia.

Of the fourteen branch data plans reviewed, only nine referenced Strategia, with some branches showing interest in making Strategia the department-wide planning and reporting tool. Recommendations for Strategia within branch data plans included: better communication among Strategia and other tools via Business Intelligence; additional outreach and education for users on Strategia’s purpose, function, and use in decision-making; and investing in system upgrades.

Branch and bureau commitments have recently been integrated into Strategia. The purpose of the phased integration is to improve data quality in the department with increased measurement of HQ activities, and to better communicate priorities as they relate to the department’s priorities. The process of developing branch plans and Performance Information Profiles was already being completed offline and stored in Infobank. Through digitizing this process, Strategia improves the accessibility of these plans and allows for a monitoring and reporting component to be added in the future. However, the effective use of Strategia is highly dependent on users. There must be a buy-in at all levels in order for the integration of HQ plans into Strategia to be successful.

“The quality of data in Strategia is inconsistent across streams within missions, at HQ and from mission to mission.” - Interviewee

Finding 9: Inconsistent HQ engagement in the planning and reporting process may impact the use of Strategia data for decision-making.

There is alignment between Strategia and the department’s corporate planning and reporting cycles. NMD has recently integrated HQ branch and bureau planning commitments into Strategia in order to improve reporting coherence. However, the integration of branch and bureau plans into Strategia is too recent to be able to measure the effect on mission plans. Some of the HQ staff commended the integration of HQ plans into Strategia as an effective solution to improve transparency and the ability to link all commitments to results reporting. Conversely, others were against the idea of integration with HQ, claiming that Strategia is duplicative with, and not linked to, the other reporting tools in the department.

HQ reporting is not always completed on time to meaningfully inform mission plans. Discussions between HQ and missions should be taking place at the beginning of the planning process. The ICF pilot highlighted the importance of missions conferring with HQ before the annual planning cycle to establish priorities, and discussing objectives at management meetings to determine how each stream will contribute to the overall mission goals. With the pilot ending, it is important that these consultations continue under the Strategia planning process. However, respondents indicated that these consultations are most often taking place during the HQ review period, if at all.

The Strategia planning cycle includes a four-week HQ review period to allow a number of people, including planning and reporting heads from the various streams and the Geos, to review and provide comments on mission draft plans. HQ reviewers generally look for completeness, red flags, and to see if the plan makes sense overall. However, a comprehensive review of commitments and performance measures is rarely completed. Several interview respondents admitted that HQ seldom challenges mission reporting, and that HQ reviewers feel uncomfortable questioning missions on their plans and priorities. Consequently, only 48% of survey respondents at mission found HQ feedback only partly useful.

Respondents did, however, state that HQ feedback has improved over time. Previously, feedback was often technical in nature, such as identifying spelling errors. NMD has developed presentations and training for HQ reviewers to help improve feedback quality, while other streams are also making an effort to revise their training material and guidance for missions, to better reflect the quality of reporting HQ seeks.

Discussions with planning and reporting heads at HQ revealed a number of barriers to providing quality feedback on mission plans. Barriers included: time and capacity constraints of having to review several mission plans with a large number of initiatives; cumbersome review processes including various people; and no accountability by missions if plans were not updated based on HQ feedback.

Although Strategia reports often lack detail, data is still used for multiple different purposes across streams at HQ. Geo DGs review Strategia mission plans, especially prior to site visits, as they provide a broad overview of the country, including mission operations and challenges on the ground. Program documents revealed that NMD produces a number of reports on request using Strategia data. Recent requests for Strategia reports included data on Canada’s youth engagement in Europe; data on climate change initiatives; and data on any LGBTQ2 projects funded by the MCF and PIF. Across streams, the data within Strategia is broadly used for priority setting; developing briefing material for senior management; and identifying pressures within missions, including reviewing budgets and staffing to recalibrate resources as needed. However, there is general agreement that data usage at HQ is not where it should be, and that improvements could be made.

“Data in Strategia is still oftentimes fluff. Data usage is not where it should be, and we should get better.” - Geo DG Interview

Using Strategia Data at HQ Best Practices

The Commercial stream uses Strategia data to create their own customized reports for specific initiatives within sectors at missions. Recent examples include their Initiative Competitive Funding Report on the International Marrakech Air Show within the Aerospace sector at the ALGER Mission, and their report on Market Development within the Arts and Cultural Industries sector at the ABDBI Mission.

Consular uses Strategia data on an ongoing basis. The narrative within Strategia enables Consular staff at HQ to understand specific challenges and issues that affect how missions operate, which also helps with resource assessments. The narrative also helps support quantitative data, and further assists HQ in discovering innovative ideas that can be applied from one mission to the next.

Program Budget uses Strategia data throughout the year. The organizational charts from the HOM tab are useful, and data can be extracted and is often used for briefings.

Barriers to Using Strategia Data:

• Data in Strategia is rich, but not packaged well. NMD support is needed to help extract reports from Strategia, as branches or divisions at HQ often lack the capacity or access to extract reports.
• Qualitative data can be difficult to roll-up, especially if commitments are not linked to the Departmental Results Framework.
• Results data is often pulled from other systems, such as TRIO 2 or MSR+.
• There is inconsistent data across missions, making a mission-to-mission comparison not possible.
• There is a significant time delay between when information is entered and when it may be needed for decision-making.
• Information from the annual plans in Strategia is often outdated, as changes, such as staffing, occur throughout the year. Some branches request quarterly reports from mission directly rather than rely on Strategia data.
• Some streams have good working relationships with mission staff, facilitating direct requests for data, bypassing Strategia.

Finding 10: A lack of secured funding for Strategia could hamper future development of the tool, preventing improvements to the functionality of the system over time.

As Strategia incorporates branch and bureau plans in addition to missions planning and reporting, it is transforming into a corporate tool with a significantly wider scope. If the trend continues, NMD – a Bureau dedicated to mission support – may not be best positioned to continue managing the development and implementation of Strategia. The Strategia Business Model Working Group includes members from a number of branches and divisions that provide advice and feedback on the needs of their respective streams. However, it is unclear if this group plays a role in the governance of the tool, and if members have any decision-making authority.

Since its inception, Strategia has not had a permanent budget allocation to maintain the tool, requiring submissions to the Resource Management Committee on an annual basis in order to secure funding. Funding amounts allocated were often lower than the amounts requested. For example, in 2013-14 the amount requested was $400,000, but the amount allocated was only $16, 500.

A recent submission by the Bureau to the Resource Management Committee outlined a number of impacts of a non-functioning Strategia. Issues with functionality of Strategia could compromise the department’s capacity to provide data, and therefore undermine the ability to make evidence-based decisions and ensure the alignment of activities and resources with the government’s priorities. As it stands, the resources currently allocated towards Strategia do not enable its development as Global Affairs’ primary planning and reporting tool.

Finding 11: MyInternational is a narrative reporting tool that is widely used by missions yet not regularly accessed by intended recipients.

MyInternational, which launched in 2013, was originally created as a reporting tool to replace reporting by email. It hosts a variety of reports, from political reports to best practices, to monthly updates. The tool includes a search function where reports can be searched and filtered by theme and region. In 2018-19, MyInternational published 1,680 reports, under the top themes of governance, human rights and development; peace and security; and democratic institutions, participation and processes. However, since the tool was created, access has been low. A survey conducted by NMD in 2017 found that 40% of respondents did not access MyInternational reports at all, providing little incentive for mission staff to use the tool for narrative reporting. Interviews conducted with Geo DGs reinforced this, with respondents stating that they continued to read and receive narrative mission reports by email rather than accessing MyInternational. Of the few Geo DGs who did use the tool, they stated that they used it out of interest, or to see what was happening in missions abroad, but not to replace email reports. The main reasons why MyInternational was not accessed regularly, were the following:

Duplication

Both interviewees and MyInternational survey respondents stated that most people still used email to circulate their reports, because they could choose their own targeted distribution list. This was identified as the main issue with the lack of use of MyInternational.

“I receive so many emailed reports every day – I don’t see the need to refer to MyInternational, ever.”- MyInternational Survey Respondent

Usability

Respondents to the MyInternational 2017 Survey found the tool difficult to use, with 56.9% stating they encountered some kind of difficulty, such as finding the site; publishing reports; uploading photos; and searching for reports.

“MyInternational needs to be more user-friendly. (…) The current web design is very difficult to navigate.” - MyInternational Survey Respondent

Access

Access to MyInternational is limited to Global Affairs staff. If the author of a report would like to share it with external partners, MyInternational cannot be used. To mitigate this, NMD has launched the Open Content Initiative, wherein certain reports are published on a public website.

“Inability for OGDs and provincial partners to access the site directly is a critical weakness.”
- MyInternational Survey Respondent

Security

MyInternational only allows unclassified information to be posted. Consequently, if reports contain classified information they cannot be posted. This can be particularly problematic for reports which contain policy advice, reinforcing the issue of duplication.

“There needs to be a classified version as well as an open version.”- MyInternational Survey Respondent

Conclusions

Mandate, Roles and Responsibilities

Overall, the individual units within NMD are meeting their mandate, and the activities these units conduct are relevant. However, there is a strong need to clarify the exact mandate, roles and responsibilities of NMD as a whole, as well as each of its individual units. Throughout the department, there seemed to be a lack of understanding or consensus on what NMD’s role is and what specific activities they conduct (or should be conducting). This calls for clearer definitions of NMD and its units, as well as a clearer outline of their expected activities. Once the Bureau has a clear and precise direction, that direction must be communicated across the department. This will ensure that Geo Branches understand which services they can receive, and subsequently will allow them to articulate their needs.

Planning, Monitoring and Reporting Tools

The planning, monitoring and reporting tools that NMD administers are important and necessary for the department. However, there were many issues that were raised with the specific planning and reporting tools, especially Strategia. Currently, Strategia is not being used consistently or for its intended purpose. Going forward, the department would benefit from clarifying the purpose of Strategia across all streams in the department. There should be more consistent and complete reporting; increased HQ engagement; and a decrease in possible areas of duplication. This is especially true considering the increased emphasis on coherence within the department. If different streams are using different tools altogether, this inhibits stream to stream coherence and communication.

Key Takeaway: Clarification Needed

Key Departmental Considerations

The department would benefit from clarifying its vision for planning, monitoring and reporting at missions and Headquarters across all streams.

To optimize departmental performance, foster a data ecosystem for evidence-based decision-making and strengthen departmental coherence, further cohesiveness is needed between the departments multiple planning, monitoring and reporting tools. Strategia is successful as a mission planning tool. With further training and time, Strategia may find greater success as an integrated mission planning tool. However, its effective use as a monitoring and reporting tool requires better integration with other tools to avoid duplication, and potentially a redesign to accommodate the reporting needs of all streams such as International Assistance.  In addition, if Strategia is determined to be Global Affairs’ primary planning and reporting tool, and is key in reporting the department’s results to Parliamentarians, then the department should consider resourcing it accordingly.

There are risks to be acknowledged when considering the future integration plans for Strategia.

As Strategia continues to develop and expand its functions, consideration should be paid to whether the processes it incorporates are fully and adequately integrated into the platform. When considering the integration of other existing corporate planning and reporting processes such as ICFs, there are certain risks to be considered. There may be a danger of amalgamating all processes into Strategia if it is only being used as a platform to upload plans and reports that were developed quickly in siloes without deliberate consultations. There may also be a risk of lost analyses, such as environmental, conflict or bilateral relations analyses not completed during the Strategia planning process as it is not explicitly required, or because the platform does not support classified or sensitive information. In addition, it is unclear if thematic specialists at HQ are systematically reviewing mission Strategia plans and providing feedback on country analysis of various themes.

There is a need to further ameliorate the department's performance measures.

A change is needed in the department to not only better understand the importance of data for the use of decision-making, but to also effectively use the tools available to capture results. There is a need for robust performance measures and other tools that can deepen amalgamation and promote horizontal coherence, co-operation and collaboration while supporting better-informed corporate decision-making.

An assessment of data quality could be beneficial.

Having complete, accurate and timely data is key in the decision-making process. In turns, better decisions lead to more efficient and effective achievement of outcomes and/or results. Thus, an assessment of data quality could be beneficial in the future to assess the performance of missions in the process of planning and reporting in Strategia.

Recommendations

Recommendation 1

It is recommended that NMD, in collaboration with the geographic branches, develop and share across the department a Bureau Plan that clearly defines its distinct roles and responsibilities, and resourcing requirements.

This may include:

• A review of the vision, mandate and outcomes of the bureau and each unit to ensure all areas are working together towards the same goals and synergies are maximized;
• A clear definition of the roles and responsibilities of each unit to ensure they have the adequate human and financial resources to perform activities, and are working towards the bureau’s outcomes; and
• A concrete definition of the roles, expected activities, and human resources capacity of the Geographic Coordination unit developed through consultations with geographic branches to ensure their role meets the needs of client branches.
• Address knowledge transfer and capacity building for potential new employees that may be providing mission support services such as advocacy, digital engagement, and planning and reporting.

Recommendation 2

It is recommended that NMD, in collaboration with other relevant departmental stakeholders, develop an approach to strengthen the Department’s planning, monitoring and reporting tools.

This may include:

• An outline of its purpose as wells as, the series of proposed large scale upgrades and/or changes to be made with estimated timelines, within the suite of departmental planning and reporting tools;
• A consultation proposal that details the stakeholders to be engaged in the development of the next phases of the tool;
• A proposal to establish the governance for the tool as it continues to develop, with consideration of developing an intra-departmental governance group;
• An outline of the financial and human resource needs for the development of the tool; and
• A communication and outreach proposal for the whole department that clearly lays out the purpose, function, and intended use of this tool for decision-making.

Management Response and Action Plan

Recommendation 1

It is recommended that NMD, in collaboration with the geographic branches, develop and share across the department a Bureau Plan that clearly defines its distinct roles and responsibilities, and resourcing requirements.

Recommendation 2

It is recommended that NMD, in collaboration with other relevant departmental stakeholders, develop an approach to strengthen the Department’s planning, monitoring and reporting tools.

Appendices

Appendix A: Profile of Mission Support (NMS) – Planning and Reporting

The mandate of the Planning and Reporting Unit of NMS is to enhance departmental planning and reporting to support better decision-making through overseeing the development and maintenance of Strategia and MyInternational. The Unit consists of 5 FTEs (FS-03; EC-06; 2x EC-04; AS-04).

Tool Development and Enhancement

• NMS oversees the ongoing development of Strategia.
  - Strategia 3.0 includes all streams at Global Affairs.
• It initiates cross-stream dialogues through a Strategia working group to address needs.
• It is implementing a Client Relationship Management tool for FPDS.

Building Capacity

• NMS conducts in-person and online trainings on Strategia use and results-based management.
• It provides overall support and guidance for selected streams (HOM, FPDS, Multilateral, and Partners).

Sharing Results

• NMS develops reports upon the request of other Divisions using Strategia data on various themes, e.g. Canada’s Youth Engagement in Europe, FemParl Initiatives, etc.
• It provides FPDS results for departmental results (e.g. % of advocacy campaigns which met their objectives, or number of influencers reached through Canadian-hosted events).

NMS Stats

• 1006 Mission Plans (2018-19)
• 2945 Strategia Users (2018-19)
• 19 Strategia-specific trainings delivered (2018-19 planning cycle)
• 1633 MyInternational Reports published (2018)

Appendix B: Profile of Mission Support (NMS) – Advocacy and Culture

The mandate of the Advocacy and Culture Unit is to work with missions and geographic functional divisions to develop integrated advocacy and digital engagement campaigns to promote selected foreign policy issues in targeted countries and regions. They also support missions in promoting Canadian culture as a diplomacy tool. The Unit consists of 18 FTEs (2x EC-06, 3x EC-05, IS-06, IS-04, 2x IS-03, AS-01, AS-03, PM-05, PM-06, CS-02, EX-01. CFLI: FS-03, FS-02, PM-03.

Supporting Initiatives

• NMS helps missions apply, plan and report on initiatives under three funds:
  • Post Initiative Fund (PIF), for advocacy initiatives;
  • Mission Cultural Fund (MCF), for cultural initiatives;
  • Canada Fund for Local Initiatives (CFLI), for small-scale aid projects. CFLI joined NMS in July 2018.

Building Capacity

• NMS delivers regional and pre-posting training.
• It maintains thematic toolkits for missions (e.g. democracy, human rights, peace and security, etc.).
• It manages advocacy campaigns (e.g. Faces for Equality, FemParl, SheCANLead, etc.) and international days (e.g. Women’s Day).

Sharing Results

• NMS develops regional snapshots using Strategia data.
  - e.g. developed the Asia-Pacific Advocacy Snapshot for 2017/2018.
• It conducts analysis of digital campaigns and shares thematic and campaign reports.
• It captures mission advocacy initiatives via online stories in Eyes Abroad.

NMS Stats

• 701 PIF Initiatives reported on/applied in 2018-2019.
• 6 in-person regional effective advocacy training provided since 2016.
• $5,377,422 of MCF Funding for 2017-2018.
• 668 MCF initiatives reported on/applied in 2018-2019.

Appendix C: Profile of Geographic Coordination Unit (NMO)

NMO’s mandate is to strengthen coordination among the department’s four geographic branches and the missions abroad in support of greater coherence, integration and efficiency. The Unit consists of 2 FTEs (PM-05, AS-03).

Coordinating Input and Representing Geos

• NMO coordinates Geo input for corporate exercises and briefing material
• It identifies issues that are unique to the Geo branches (e.g. Parental Leave at Missions, Training of Official Languages, HR Issues Re: Staffing) in order to raise those issues at meetings and assign taskings to Geo branches. 
• It acts as a single window by gathering and channelling information among Geos and throughout the department.
• It represents Geo views at inter-and intra-departmental fora, such as for the “Duty of Care Task Force” or the now-defunct “Departmental Security Working Group”.

Facilitating Dialogue

• NMO acts in a Secretariat function to facilitate continued dialogue between the Geos through NMO Coordinated meetings.
• It maintains Geo distribution lists.

NMO Coordinated Meetings

• Bi-weekly Geo DG meetings
• Bi-weekly Geo DG Advisor meetings
• 2018 Global Heads of Mission meeting (with over 150 HOMs)
• Weekly Geo ADM meetings

Appendix D: Profile of IRCC/CBSA Coordination Unit (NMV)

NMV’s mandate is to coordinate Global Affairs’ relationship with Immigration, Refugee and Citizenship Canada (IRCC) and Canada Border Services Agency (CBSA), by addressing mutual issues and priorities and informing decision-making and policy development. The unit also supports and advises the geographic ADM serving the corporate role as Global Affairs' Senior Liaison for relations with IRCC and CBSA. The Unit consists of 2 FTEs (FS-03, EC-04).

Coordinating Input

• NMV facilitates policy input, departmental coordination, and briefings on high-level/ broad-reaching issues.
• It facilitates approval authority on IRCC/CBSA documents affecting multiple branches for Deputy Minister or Ministerial approval.
• It advises Global Affairs on best practices and ensures that they are in contact with the right interlocutors to discuss and resolve issues.

Sharing Information and Guidance

• NMV shares information on the IRCC/CBSA Coordination wiki about labour mobility, visas, citizenship, etc.
• It develops and shares guidance on MODUS.
• It consults internally to identify upcoming events and visits to Canada involving the department and shares details with IRCC and CBSA to proactively avoid potential visa problems.

Facilitating Dialogue

• NMV organizes and participates in inter-departmental meetings:
  • Secretariat for quarterly ADM-level GAC-IRCC Joint Management Committee,
  • Secretariat for ADM-level GAC-CBSA-IRCC Removals Working Group
  • Secretariat for Global Affairs representatives to interdepartmental committees on systemic admissibility issues under the Immigration Act.

Areas of Guidance (6+)

• Visa Policy
• Admissibility
• Immigration Programs
• Labour Mobility
• Removals
• Biometrics
• Visas/eTAs
• In-Canada events

Appendix E: Profile of Blueprint 2020 Unit (NMZ)

NMZ’s mandate is to implement the Blueprint 2020 initiative at Global Affairs Canada. It is a government-wide modernization project led by the Clerk of the Privy Council, with a goal of creating a modern, open and networked public service. NMZ works closely with PCO and other government departments to realize the vision of the Clerk at Global Affairs. The Unit consists of 3 FTEs (FS-03, EC-04, AS-02).

Sharing Global Affairs Innovation

NMZ reports on Global Affairs’ activities in innovation through:

• Stories@gac, a GCpedia page with Blueprint 2020 Success Stories;
• The department’s Evergreen Highlights, a GCpedia page outlining the Blueprint 2020 initiatives;
• Facebook Page @gacpb2020;
• Progress reports to the Clerk.

Promoting Collaboration

• NMZ established the Collaboration Centre.
• It hosts innovation clinics to support employees in implementing ideas for new policies or programs, through group discussions and coaching sessions (currently hosting the 5th clinic).
• It promotes Global Affairs’ Innolab, a website to explore innovative ideas.

Facilitating Dialogue

• NMZ created Global Affairs Gateway, a GCpedia page which connects communities within Global Affairs and across government, and promotes events.
• It used technology (Nureva Span Digital Sticky Notes) to engage employees to contribute to the High Performance Organization Dialogue.

Collaboration Centre est. 2017 at 125 Sussex

• 7437+ participants hosted since it was established
• Modern workspace with writable walls, mobile furniture and equipment
• State-of-the-art technology (Li-Fi access and Nureva Span Wall and Cart)
• 567+ events held since it was established

Appendix F: NMD Logic Model

*Adapted from NMD 2018-2019 PIP

Appendix G: Overview of Corporate Planning and Reporting Tools

Strategia 3.0: Planning, Monitoring and Reporting Tool

• Primary users: Mission Staff and HQ Staff
• Purpose: Enables the alignment of departmental goals & funding allocation. Officers at missions are required to use Strategia to submit their planning commitments to HQ on an annual basis during the reporting cycle; HQ officers then roll up the data into the departmental plan.

Financial and Administrative Tools: SAP/FAS, FINSTAT

• Primary users: All staff dealing with Budget matters
• Purpose: SAP/FAS is used for the management of financial operations and grants.
• FINSTAT is the internal financial report used to monitor the department’s financial situation and ensure effective overall management.
• Potential areas of duplication with Strategia: There seems to be the most duplication between Strategia and Financial and Administration Tools. Specifically, Strategia includes reporting on funding allocated, which is duplicative with reporting in financial systems. A slight duplication with FINSTAT was also identified, as both systems are used for monitoring.

TRIO 2.0: Client Relation Management System for the Trade Commissioner Service

• Primary users: Trade Commissioners at Missions, HQ and Regional Networks 
• Purpose: TRIO 2 is used by the Trade Commissioner Service to manage its relationships and activities with Canadian client organizations and key local contacts. It is the Trade Commissioner’s Service’s primary means of capturing activities which are then used for performance measurement purposes.
• Potential areas of duplication with Strategia: There is some duplication between Strategia and TRIO 2, with regards to data-entry for key performance indicators. They are initially entered into TRIO 2 and then re-entered into Strategia at year-end.

International Assistance Tools: SAP, MRT, MSR+

• Primary users: International Development Officers at Missions and HQ
• Purpose: The Monitoring and Reporting Tool (MRT) is a module in SAP used for reporting project level reporting data. It is not currently in use due to technical deficiencies. MSR+ is an excel spreadsheet used to report on the Feminist International Assistance Policy indicators.
• Potential areas of duplication with Strategia: While there is currently little to no duplication with Strategia and International Assistance Tools, there could be eventual duplication if the stream is not integrated with Strategia, along with using an entirely different tool than the other streams for reporting.

MyInternational: Narrative reporting Tool

• Primary users: Mission and HQ Staff
• Purpose: MyInternational is used for narrative reporting on political-economic, trade, and development reports, and it was originally created as an archiving system for departmental reports.
• Potential areas of duplication with Strategia: There does not appear to be duplication between Strategia and MyInternational. Currently, initiatives in Strategia can be directly linked to MyInternational reports to provide a narrative detail. However, interview respondents do not believe this feature is used often.

COSMOS: Consular Management and Operations System

• Primary users: Consular Officers at missions; Officers at HQ
• Purpose: COSMOS is a business information system that is used for consular task management, workflow, and client relationship management. It has four primary functions:
  1. Case Management;
  2. Passport Processing;
  3. Resource Allocations;
  4. Crisis Management.
• Potential areas of duplication with Strategia: There is very little duplication between COSMOS and Strategia. The little duplication comes from the fact that information needs to be copied/pasted from COSMOS into Strategia at year-end, duplicating their work.