Ethical and methodological framework for Open source data monitoring and analysis^{Footnote 1}

Rapid Response Mechanism Canada
Centre for International Digital Policy

Global Affairs Canada – June 2019

Table of Contents

• Purpose
• Background and Mandate
• Challenges and Obstacles
• Thresholds and Protocols
• Methodology and Tools
• Human Rights Approach
  • Privacy
  • Freedom of Expression
  • Gender Equality
• Principles and Ethical Considerations
  • Transparency and Accountability
  • Internal Oversight and Partnerships

Purpose

Contemporary international relations theory and practice must consider the rapidly evolving digital information ecosystem. This ecosystem, including social media platforms and their content ranking algorithms, is creating new opportunities for economic growth and connectivity, while also presenting a range of new challenges for foreign policy. The threat from malign foreign actors, who seek to leverage aspects of this ecosystem for nefarious activities that are detrimental to our democratic systems of governance, is among the most pressing issues requiring attention by democracies such as Canada. Governments are developing new capacities to better understand this threat, including by harnessing open source data monitoring and analytical tools and approaches.

The purpose of this framework is to outline ethical and methodological principles and guidelines for open source data monitoring and analysis undertaken by Rapid Response Mechanism Canada (RRM Canada). These open source data activities support the G7 Rapid Response Mechanism (RRM) – an initiative aimed at defending G7 democracies from foreign threats as well as Government of Canada efforts aimed at safeguarding its own democratic institutions and processes, including its general elections. The need for such a framework stems from the following three needs, among others:

1. Anchoring these relatively new activities in an existing policy, legal and regulatory context.
2. Ensuring that the activities are politically neutral, and respect and reinforce human rights and freedoms.
3. Providing transparency and accountability to Canadians and the G7.

Background and Mandate

Supported by a team of policy and data analysts, RRM Canada is housed at Global Affairs Canada in the Centre for International Digital Policy. The RRM was announced at the G7 Charlevoix Summit in 2018 and re-affirmed at the G7 ministerial meeting in Dinard, France, in 2019. Led by Canada on an ongoing basis, its mandate is “to strengthen coordination to identify and respond to diverse and evolving threats” to G7 democracies.^{Footnote 2}  More specifically, the foreign threats that G7 members committed to confront seek to undermine democratic institutions and processes through “coercive, corrupt, covert or malicious means.”^{Footnote 3}

While the threat landscape covered by the RRM is broad, disinformation in the digital context figures prominently. ^{Footnote 4} In the lead up to the RRM’s announcement, Foreign and Security Ministers recognised the threat posed by “acts or measures by foreign actors with the malicious intent of undermining trust in the independent media, manipulating public discourse, and violating privacy,” by including these activities in the Toronto Commitment, among other key illustrative examples.

Additionally, RRM Canada supports the Government of Canada efforts to safeguard the 2019 general elections. As part of the Security and Intelligence Threats to Elections (SITE) Task Force, RRM Canada works with Canada’s Security and Intelligence organisations “to prevent covert, clandestine, or criminal activities from influencing or interfering with the electoral process in Canada.” The other members of the Task Force are: the Communications Security Establishment (CSE), Canadian Security Intelligence Services (CSIS), and the Royal Canadian Mounted Police (RCMP).

The aim of RRM Canada’s open source data activities is to support the RRM mandate to defend democracies, and help safeguard Canada’s 2019 general elections, by better understanding foreign threats in the digital context, shining light on them, and recommending effective response options.

Challenges and Obstacles

Among the central challenges in addressing foreign interference in the digital ecosystems of G7 democracies is determining the foreign nature of the online activities being undertaken. This challenge reflects the limitations of employing open and publicly available information^{Footnote 5} from social media platforms for analytical purposes. This is because foreign states and state proxies exploit the anonymity offered by digital platforms, “weaponise” elements of the digital information ecosystem, and continuously adapt to strategies aimed at stopping them. While anonymity can be integral to facilitating sensitive discussions online (where those discussions are discouraged or are otherwise dangerous to individuals engaging in them), it is also the same mechanism exploited by foreign actors to conduct coercive, corrupt, covert or malicious activities.

Perhaps the most problematic aspect of distinguishing foreign interference from organic domestic debate is that foreign actors target domestic audiences with content that may resonate with the audiences’ pre-existing opinions and worldviews. This targeting, often undertaken clandestinely, involves a foreign actor creating content that has been designed to sow discord or exploit existing societal differences in the domestic population. When this foreign content is received by domestic audiences, it then can be amplified further either wittingly or unwittingly. This sequence intertwines foreign and domestic narratives in ways that are difficult to untangle. Foreign actors may also coerce or induce Canadians to promote a given narrative, but these overtures are often hidden and difficult to substantiate.

The challenge of separating foreign interference from domestic engagement raises the potential of inadvertently affecting the enjoyment of human rights and freedoms of Canadians, in particular, freedom of expression and privacy rights. To avoid this situation, open source data activities conducted by RRM Canada are subject to clear thresholds and protocols for monitoring, analysis and information sharing.

RRM Canada thresholds and protocols ensure that its open source data monitoring and analytical activities fall under the RRM mandate, safeguard and reinforce human rights and freedoms of Canadians, are politically neutral, comply with relevant legal and regulatory provisions, and meet high ethical standards.

Thresholds and Protocols

RRM Canada has developed thresholds for what are considered coercive, corrupt, covert or malicious activities as well as protocols for open source data monitoring and information sharing activities. To be the subject of RRM Canada monitoring activities, an account or network of coordinated accounts must display a number of characteristics outlined in the RRM Canada methodology. The methodology sets high thresholds established in cooperation with leading experts; computational social scientists; and security, intelligence, and law enforcement organisations. Additional factors for open source data activities include concurrence with secondary sources and significant impact on public discourse.

These thresholds inform RRM Canada’s monitoring activities, including its approach to analysing suspicious accounts and networks associated with foreign interference. These thresholds are not impacted by the accuracy, perceived acceptability, or the political/ideological tenor of content that any given account or network disseminates. In the event that a foreign connection cannot be established within a reasonable period, monitoring activities cease and no data is retained. However, in cases when suspicious activities may potentially meet criminal or national security thresholds, insight is shared with security, intelligence and law enforcement organisations. When activities may potentially contravene the Canada Elections Act, the Commissioner of Canada Elections will be notified by RRM Canada. These organisations independently determine whether or not an investigation is required pursuant their respective mandates and legislative frameworks.

Furthermore, in January 2019 the Government of Canada announced a number of new measures to protect the 2019 general election, including the Critical Elections Incident Public Protocol. This Protocol lays out a clear and impartial process by which Canadians may be notified of a threat to the integrity of the elections that occur within the writ period.

Methodology and Tools

RRM Canada examines trends, anomalies, and emerging narratives in online conversations across the digital information ecosystem pertaining to potentially divisive issues and public political actors that could be exploited by malign foreign actors, regardless of their political or ideological tenor or affiliation. By observing baseline structures of what are considered normal conversations surrounding issues, as they evolve over time, it is possible to identify abnormalities that may indicate a concerted foreign information operation. RRM Canada examines multiple indicators of coordinated foreign interference campaigns, some of which are indicative of foreign coercive, corrupt, covert or malicious behaviour.

Caution is exercised in divulging detailed indicators pertaining to threshold-setting in order to prevent malign foreign actors from developing counter strategies.  Nevertheless, indicators for covert behaviour can include artificial or inauthentic amplification of narratives, for example. Narratives can be amplified by employing different tactics including bots, botnets, and trolls. RRM Canada uses indicators to identify bot and troll activity, as well as those to identify the foreign nature of suspicious activities. All determinations are made based on a confidence scale and identified using estimative language.

To monitor and analyse potential foreign interference, RRM Canada uses tools that are publicly available. The combination of these tools is not released in order to prevent malign foreign actors from developing strategies in response. To complement their use, RRM Canada also experiments with open source data modeling, natural language processing, social network analysis, machine learning and algorithms all of which process only openly available public data. Any automated outcomes these technologies render can be meaningfully explained. RRM Canada has assessed these tools using the Government of Canada’s Algorithmic Impact Assessment (v.0.7), a questionnaire designed to help assess and mitigate the risks associated with deploying an automated decision system; an impact level of two^{Footnote 6}  was obtained for the tools used by RRM Canada.^{Footnote 7} Level two decisions will often lead to impacts that are likely reversible and short-term.

The methodologies and tools employed by RRM Canada are consistent with those adopted and employed by various actors in the private and public sectors as well as non-government organisations and advocacy groups.  This is in line with a growing awareness of the digital information ecosystem as an important political, social, economic, and cultural space.

Human Rights Approach

Leading to the Charlevoix announcement of the RRM, all G7 Foreign and Security Ministers endorsed a strategic approach to responding to foreign threats that is consistent with universal human rights and fundamental freedoms. Canada is committed to respecting its international commitments and obligations including being a party to the International Covenant on Civil and Political Rights. In Canada, the Charter of Rights and Freedoms protects a number of rights and freedoms, including those most evidently impacted by foreign interference in digital contexts, namely: privacy rights, freedom of expression and the right to equality.

Privacy

The subject of RRM Canada open source data monitoring and analysis is limited to publicly available data. RRM Canada monitors, analyses and shares information in a manner that is consistent with Canada’s privacy laws, the Access to Information Act and the Ministerial Direction for Avoiding Complicity in Mistreatment by Foreign Entities. The information sharing practices of Global Affairs Canada to which RRM Canada adheres are subject to review by multiple actors, including: the Privacy Commissioner, the Information Commissioner of Canada, the Office of the Auditor General and the National Security and Intelligence Committee of Parliamentarians. All RRM Canada analysts are required to complete the Access to Information and Privacy Fundamentals course, in order to strengthen the understanding of what is considered personal information and how best to protect it.

Moreover, RRM Canada takes care to limit unintended harms and therefore, is additionally guided by firm ethical and principled considerations to facilitate responsible practices for handling personal data, even if it is publicly available. The focus of RRM Canada’s open source data monitoring and analysis is trends, tactics and strategies undertaken by malign foreign actors. The questions RRM Canada seeks to answer include:  How do foreign states and their proxies exploit online discussions? What tactics do they employ for coercive, corrupt, covert or malicious activities? How do they leverage tactics such as artificial or inauthentic amplification to manipulate online discussions and what type of coordination strategies do they employ? How do these tactics and strategies evolve over time?

Freedom of Expression

RRM Canada seeks to identify foreign activities with a coercive, corrupt, covert or malicious dimension, which attempt to sway public opinion to undermine Canadian democracy.  To mitigate risks related to the difficulty of separating foreign and domestic activities and unwittingly impinging upon the freedom of expression of Canadians, RRM Canada:

• Focuses on the structure and context of conversations, as opposed to the content, to understand what indicators may signal foreign interference.
• Relies on established open source data monitoring protocols that set out thresholds for foreign activity and guide information sharing with Canadian security, intelligence and law enforcement organisations as well as the Commissioner of Canada Elections.
• Excludes personally identifiable information from public reports. In certain cases such as national security considerations shares such information with responsible security organizations.
• Does not undertake active measures or engage in any way with content creators or those sharing content.

Gender Equality

RRM Canada adopts a Gender-Based Analysis Plus (GBA+) approach as it undertakes open source data monitoring and analytical activities. Malign actors target, exploit, and sometimes co-opt women and marginalized groups and issues in their activities to undermine social cohesion. Understanding how these processes occur and how they differentially impact these groups is crucial to both countering foreign interference and protecting human rights.

The methodological approach is also informed by academics and civil society organisations who are experts on gender and intersectional identity issues. Several of these interlocutors are conducting research that directly supports the RRM Canada mandate. Finally, all RRM Canada analysts are required to take the Gender-Based Analysis Plus (GBA+) online course and integrate the approach systematically into their work.

Principles and Ethical Considerations

RRM Canada has incorporated principles and ethical considerations beyond the existing legal and policy considerations to enhance its approach to open source data monitoring and analysis that is effective in protecting Canadians, while limiting undue and unintended harms.

Transparency and Accountability

RRM Canada is committed to working in a manner that prioritizes transparency and openness.  Our commitment is reflected in the following actions:

• Treatment of open and publicly available data only.
• Focus on tactics, strategies and trends.
• Use of publically available tools, explainable algorithms and other technologies.
• Ethical, human rights respecting and politically neutral approach to monitoring and analysis.
• Established thresholds and information sharing protocols with Government of Canada organisations and G7 partners.
• Systematic engagement with a wide network of experts, academics and civil society actors.

Internal Oversight and Partnerships

RRM Canada has developed an internal review mechanism to ensure analytical accuracy and robustness. Its multi-disciplinary team of social scientists, data experts and policy analysts allows for the agility and capacity to address evolving threats, while incorporating broader perspectives into its open source data activities. RRM Canada frequently engages in challenge functions or a peer-review process within the Government of Canada. This allows final conclusions to be determined through a rigorous process whereby results are challenged by fellow analysts and possibilities of cognitive and other bias are reduced. Collaboration beyond the Government of Canada is essential for developing innovative open source data monitoring and analytical capacity. RRM Canada relies on a growing community of experts which includes representatives from other governments, academia, civil society and non-governmental organisations.