Results-Based Management: A how to guide

Call for proposal: Indo-Pacific Regional Connectivity Envelope

Table of contents

1. An Introduction to Results-Based Management
   • 1.1. Results-Based Management (RBM) / Managing for Results (MfR)
     • 1.1.1. What is Results-Based Management?
     • 1.1.2. Why use the Results-Based Management approach?
   • 1.2. Results-Based Management and the Theory of Change
   • 1.3. The results chain
   • 1.4. What are the expectations?
     • 1.4.1. What are the RBM expectations of the organization?
     • 1.4.2. What is the role of Global Affairs Canada in RBM for this initiative?
2. Results-Based Management tools and methodologies
   • 2.1. What is an output?
     • 2.1.1. How to formulate an expected output statement
   • 2.2. The logic model
   • 2.3. Indicators
     • 2.3.1. Developing indicators for your project
   • 2.4. The Performance Measurement Framework
3. Regional Connectivity Envelope (RCE) RBM examples
   • 3.1. Examples of logic model with RCE outcomes and optional - Project personalized outcomes
     • 3.1.1. Logic model for Climate Initiative Conference in India
     • 3.1.2. Logic model for Cybersecurity training in Japan
4. Annexes

1. An Introduction to Results-Based Management

1.1 Results-Based Management / Managing for Results

1.1.1. What is Results-Based Management?

The aim of Results-Based Management (RBM) / Managing for Results (MfR) is to optimize and improve the achievement of results.

Managing for Results (MfR), or Results-based Management (RBM), is a lifecycle approach to adaptive management that focuses on achieving results (outcomes): from initiation, to design and planning, to implementation (results-based monitoring including performance measurement, adapting/adjusting and reporting), to closure (final evaluations and reports, and integrating lessons learned into future programming). It is a way of working and thinking strategically – a mindset – to manage programs/portfolios, projects and other activities more effectively and efficiently to achieve expected outcomes.

According to the Global Affairs Canada Results-based Management Policy Statement 2008:

RBM is a life-cycle approach to adaptive management that integrates strategy, people, resources, processes, and measurements to improve decision-making, transparency, and accountability. The approach focuses on achieving outcomes, by implementing performance measurement, learning, and adapting, as well as reporting performance. RBM means:

• defining realistic expected results based on appropriate analyses;
• clearly identifying program beneficiaries and designing programs to meet their needs;
• monitoring progress towards results and resources with the use of appropriate indicators;
• identifying and managing risks while bearing in mind the expected results and necessary resources;
• increasing knowledge by learning lessons and integrating them into decisions; and
• reporting on the results achieved and resources involved.

In other words, Results-Based Management is not just a set of tools or instructions. It is a way of thinking strategically about projects and other initiatives that help you manage more effectively. By managing better, you can improve the achievement of results, that is, the positive changes you set out to achieve or contribute to with your programs/portfolios or projects.

1.1.2. Why use the Results-Based Management approach?

Historically, government departments—and implementing organizations—focused their attention on inputs (what they spent), activities (what they did) and outputs (what they produced). While information about inputs, activities and outputs is important, it did not tell implementers whether or not they were making progress in addressing the issues they had identified. Losing sight of the results their programs were aiming to achieve limited the effectiveness of their programming.

A new management approach was needed to raise the standards of performance and define success in terms of actual results achieved. Results-Based Management/Managing for Results was introduced to meet this need.

The focus on activities at the expense of results is what management scholar Peter Drucker, in 1954, referred to as the “activity trap”. Instead, Results-Based Management requires that you look beyond activities and outputs to focus on actual results (outcomes): the changes to which your programming is contributing and/or contributed. By establishing clearly defined realistic expected outcomes, assessing risk, collecting information to assess progress on the outcomes on a regular basis during implementation, and adapting/making timely adjustments, practitioners can manage their projects and programs/portfolios better in order to optimize and improve the achievement of results.

This focus on measuring at the outcome level during implementation was one of the fundamental changes introduced by Results-Based Management / Managing for Results. While traditional approaches to management may have identified objectives or expected outcomes during planning, once implementation began monitoring focused on inputs, activities and outputs. With the advent of Results-Based Management, the focus remains on outcomes, not only during design and planning, but also during implementation.

In sum, Results-Based Management is about effectiveness; it aims to maximize the achievement of ultimate outcomes, in other words the improvements to cultural ties between Canada and the Indo-Pacific. The nature of ultimate outcomes may vary depending on the type of programming. For example, in the case of the Regional Connectivity Envelope the ultimate outcomes must contribute to developments related to security, trade, people to people ties, climate, and Canadian engagement in the region.

The following example of a student’s journey through the education system provides a simple illustration of how Results-Based Management concepts are being applied in everyday lives all over the world, and why this approach is useful.

In order to be useful and enable you to manage your education and take corrective action, the information you get via regular feedback and reports focuses on your progress toward an actual change in your skills, abilities or performance, rather than on what was done or taught in class. A report that stated you attended math classes or that the school provided you with English and Science classes would not give you useful information. A report that provided an assessment of your progress toward the end-of-year learning outcomes, based on an analysis of the actual data from indicators (your marks, scores, etc.), on the other hand, provides you much more useful information for making decisions about your education, and thus helps you to manage your education better.

1.2 Results-Based Management and the Theory of Change

The theory of change is a fundamental part of managing for results. The Treasury Board of Canada Secretariat describes it as follows:

Every program [and project] is based on a "theory of change" – a set of assumptions, risks and external factors that describes how and why the program [or project] is intended to work. This theory connects the program's [or project’s] activities with its [expected ultimate outcome]. It is inherent in the program [or project] design and is often based on knowledge and experience of the program [or project design team], research, evaluations, best practices and lessons learned.

Theory of change reinvigorates the analytic roots of Results-Based Management, emphasizing the need to understand the conditions that influence the project and the motivations and contributions of various actors. When Results-Based Management is properly applied, project design is based on a thorough analysis of the issue and the context in which it exists, which informs an evidence-based solution to the issue: the theory of change.

A theory of change explains how an initiative is expected to produce its results. The theory typically starts out with a sequence of events and results (outputs, immediate outcomes, intermediate outcomes and ultimate outcomes) that are expected to occur owing to the initiative. This is commonly referred to as the “program logic” or “logic model.” However, the theory of change goes further by outlining the mechanisms of change, as well as the assumptions, risks and context that support or hinder the theory from being manifested as observed outcomes.

A programs/portfolios and project’s theory of change will be revisited regularly during implementation, as the program/portfolio and project and the context in which they are being delivered evolve. This is in keeping with the Results-Based Management principle of continuous adjustment: monitoring progress, comparing expected outcomes to actual outcomes, learning and adapting/making adjustments as required.

The importance of assumptions

Assumptions are the conscious and unconscious beliefs we each have about how the world works. From the perspective of the design team, assumptions constitute beliefs (validated or otherwise) about existing conditions that may affect the achievement of outcomes and about why each level will lead to the next. In the context of the theory of change and logic model, assumptions are the necessary conditions that must exist if the relationships in the theory of change are to behave as expected. Accordingly, care should be taken to make explicit the important assumptions upon which the internal logic of the theory of change is based.

Assumptions can be difficult to identify, as they are often taken for granted or are linked to deeply held convictions. Participatory exercises with a wide variety of local and non-local stakeholders are a good way of uncovering assumptions. This is because assumptions tend to vary among stakeholders and will become apparent when there are differing views on whether or not a project will lead to the desired change.

The importance of identifying risks

Global Affairs Canada defines risk as the effect of uncertainty on expected results (outcomes). Developing a theory of change will also help identify any risks that would affect the achievement of outcomes.

Note: Once risks are identified, suitable response strategies should be developed and managed throughout the life of the project.

1.3 The results chain

Developing a theory of change combines a reflective process and analysis with the systematic mapping of the logical sequence from inputs to outcomes in a project. The results chain provides the conceptual framework for articulating this logical sequence. Global Affairs Canada defines a results chain as follows (see Box 3 below).

The results chain addresses practitioners’ need for a concept that allows them to break complex change down into manageable building blocks or steps that lead to one another, making it easier to sequence and identify changes during both analysis and planning. These steps also become the points at which practitioners will measure whether or not the expected change is actually occurring throughout project implementation.

In sum, when practitioners approach a specific problem, their respective results chain will provide a structure to their project design, telling them what types of building blocks they should be identifying as they work on their theory of change.

Figure 1 - Global Affairs Canada Results Chain

Text version

Levels for Global Affairs Canada Results Chain

• Inputs
• Activities
• Outputs
• Immediate outcomes (Development Result)
  • (short-term)
• Intermediate outcomes(Development Result)
  • (medium-term)
• Ultimate outcome (Development Result)
  • (long-term)

Global Affairs Canada’s results chain

Global Affairs Canada’s results chain is divided into six levels. Each of these represents a distinct step in the logic of a project. The top three levels—ultimate, intermediate and immediate outcomes—constitute the actual changes expected to take place. The bottom three levels—inputs, activities and outputs—address the means to arrive at these changes.

Within the results chain, each level of outcomes is very distinct, with clear definitions of the type of change that is expected at that level. These definitions, along with the definitions for inputs, activities and outputs, are defined below. For more information, please refer to Tip Sheet 2.1: Results Chains and Definitions

Ultimate outcome – Change in state, condition or well-being of beneficiaries

The ultimate outcome represents the “why” of a project and should describe the changes in state, condition or well-being that a project’s ultimate beneficiaries should experience. These should not be confused with changes in surrounding circumstances, such as increased economic growth […]. An ultimate outcome should instead reflect changes in the lives of women, men, girls and boys in the partner country.

In this initiative the ultimate outcome is:

Enhanced realization of joint Canadian and Indo-Pacific stakeholder interests in key developments related to security, trade, people to people ties, climate, and Canadian engagement in the Indo-Pacific Region.

An ultimate outcome usually occurs after the end of the project, but should, when feasible, still be measured during the life of the project as changes may occur earlier. Once the project is over, the achievement of the ultimate outcome can be assessed through an ex-post evaluation.

Intermediate outcomes – Change in behaviour, practice or performance

Intermediate outcomes articulate the changes in behaviour, practice or performance that intermediaries and/or beneficiaries should experience by the end of a project. Intermediate outcomes usually stem from the application of the capacity built among intermediaries or beneficiaries at the immediate outcome level.

In this initiative the intermediate outcomes are:

• Knowledge sharing: Increased meaningful engagement of Canadian organizations with their Indo-Pacific counterparts in conferences, workshops, round table discussions, and panels, in and/or on the Indo-Pacific region.
• Training: Increased meaningful engagement of Indo-Pacific CSOs in Canada and Canadian CSOs in the Indo-Pacific region.
• Research: Increased collaboration between Canadian academics and researchers with their Indo-Pacific colleagues.
• Indo-Pacific cultural events: Strengthened cultural interconnectedness⁸ between Canadians and their Indo-Pacific counterparts.
• Indigenous events and reconciliation: Increased interconnectedness of Indigenous Peoples within the Indo-Pacific region and Canada.

Immediate outcomes – Change in capacities

* Changes in access can fall at either the immediate or the intermediate outcome level, depending on the context of the project and its theory of change.

Immediate outcomes articulate the changes in capacity that intermediaries and/or beneficiaries should experience during the life of a project.

Immediate outcomes represent the first level of change that intermediaries or beneficiaries experience once implementers start delivering the outputs of a project.

In this initiative the immediate outcomes are:

• Knowledge sharing: Increased access for Canadian organizations to participate in events which promote knowledge sharing with their counterparts in the Indo-Pacific region.
• Knowledge sharing: Increased capacity of Canadian organizations to facilitate knowledge sharing events with their counterparts in the Indo-Pacific region.
• Training: Increased capacity of Canadian and Indo-Pacific CSOs to effectively operate within the Indo-Pacific region and Canada.
• Training: Increased ability for Canadian and Indo-Pacific CSOs to facilitate training sessions with/for their counterparts in the Indo-Pacific region and Canada.
• Research: Increased capacity of non-GoC researchers to be able to conduct research in/on the Indo-Pacific region.
• Research: Increased capacity for Canadian researchers to facilitate events with/for their counterparts in the Indo-Pacific region and Canada.
• Indo-Pacific cultural events: Increased access for CSOs, to participate in events which promote partnerships with their counterparts in the Indo-Pacific region and Canada.
• Indo-Pacific cultural events: Increased capacity of Canadian CSOs to facilitate cultural events with their counterparts in the Indo-Pacific region and Canada.
• Indigenous events and reconciliation: Increased capacity for Indigenous Peoples to hold events within the Indo-Pacific region and Canada.
• Indigenous events and reconciliation: Increased access for Indigenous Peoples to participate in cultural events in the Indo-Pacific region and Canada.

Outputs – Products and services

In Global Affairs Canada’s results chain, outputs are the direct products or services stemming from the activities of an implementer.

Example:

Output: Academic conference on Canada in the Indo-Pacific Region attended.

Activities

In Global Affairs Canada-funded projects, activities are the direct actions taken or work performed by project implementers. Activities unpack an output into the set of tasks required to complete it. There can be more than one activity per output. For instance:

Activities:

• Attend an academic conference
• Consult with a regional civil society organization
• Deliver training to male and female staff in regional health centres
• Promote Canadian trade opportunities
• Engage with regional Indigenous Peoples

Inputs

1.4 What are the expectations?

1.4.1 What are the RBM expectations of the organization?

In the context of Results-Based Management (RBM), the partner plays a critical role in ensuring the success of the initiative. Let’s break down their responsibilities:

• Elaborate outputs, and activities:
  • The partner identifies the specific outputs (products or services) that the initiative aims to deliver. These outputs are directly linked to the project’s activities.
  • They define the activities required to achieve these outputs. These activities encompass the tasks, processes, and interventions necessary for implementation.
• Create indicators and targets:
  • The partner develops performance indicators to measure progress. These indicators are quantifiable metrics that assess the achievement of immediate and intermediate outcomes.
  • Targets are set for each indicator, representing the desired level of performance. These targets guide the initiative’s efforts and provide a benchmark for success.
• Collect data:
  • The partner establishes data collection mechanisms. This involves designing surveys, conducting interviews, or using other methods to gather relevant information.
  • Regular data collection ensures ongoing monitoring of progress. It allows the partner to track whether activities are leading to the expected outputs and outcomes.
• Report on results achieved:
  • Based on collected data, the partner prepares periodic reports. These reports highlight progress, challenges, and any adjustments made to the initiative.
  • Reporting is essential for accountability, transparency, and learning. It informs stakeholders about the impact of the project and guides decision-making.
• Optional RBM practice
  • Develop a logic model for the project that aligns with the RCE Logic Model. This logic model should consist of an ultimate outcome, with supporting intermediate and immediate outcomes. Each of these outcomes would be monitored and reported against via partner developed indicators.

In sum, the partner’s role in RBM encompasses planning, monitoring, and reporting, all aimed at achieving the desired outcomes effectively.

1.4.2 What is the role of Global Affairs Canada in RBM for this initiative?

In the context of the Regional Connectivity Envelope, Global Affairs Canada (GAC) has a role to play in RBM:

• Determine the ultimate, intermediate, and immediate outcomes
  • Global Affairs Canada has developed the Regional Connectivity Envelope’s ultimate, intermediate and immediate outcomes across the 5 project categories. These outcomes will be the same across each project through the RCE.
• Confirming alignment between the partners’ objectives, and the Regional Connectivity Envelope’s objectives
  • While reviewing partner proposals, GAC will make sure that the objectives of the project will align appropriately with those laid out in the Regional Connectivity Envelope's Logic Model.
• Consolidate data on results
  • Through the reporting prepared by partner organizations, GAC will be able to consolidate the achievements towards the stated outcomes of the program. As well as share the successes of partners.
  • GAC will create consolidated indicators to allow for the roll up of partner data across multiple projects
• Oversee project delivery through results reporting
  • Global Affairs Canada Project Officers will be monitoring the reporting of projects on their progress towards the results. If a project is underperforming, GAC and the partner can collaboratively update project elements to make sure progress towards the desired outcomes.

2. Results-Based Management tools and methodologies

Global Affairs Canada has adopted a set of methodologies and tools to make managing for results easier for staff, implementers and other stakeholders, you can find them here Part Two: Results-Based Management Methodologies and Tools. For the RCE, the ultimate outcome (stated above) is established by the program as well as the intermediate and immediate outcomes related to the project category. Applicants will only be required to draft outputs, activities and the indicators they will use to measure the immediate and intermediate outcome in accordance with the project category selected.

2.1 What is an output?

Outputs are direct products or services stemming from the activities of an organization, policy, program or project.

In the context of a project funded by Global Affairs Canada, outputs are the products and services stemming from the project activities undertaken by an implementer with the project funds. If there is more than one implementer, responsibility, whether individual or shared, should be clearly established.

Outputs are not results.

In the context of a project funded by Global Affairs Canada, outputs are not the products delivered or services rendered by the intermediaries or beneficiaries of the project.

Outputs are not the products or services of other actors in the country or sector.

2.1.1 How to formulate an expected output statement

It clearly indicates what the implementer will deliver

An output statement describes a product or service to be provided by an implementer to a specific population, group or organization (in other words, project intermediaries or beneficiaries). Output statements should be specific and detailed enough so that it is clear what product or service the implementer will provide, yet they should not attempt to cover every activity required to deliver the output.

It follows a syntax different from that of outcome statements

Since outputs are not results, an output statement is different from an outcome statement. An output statement refers to what an implementer produces or provides, as opposed to an outcome statement which describes the changes intermediaries or beneficiaries experience. It should therefore not begin by describing a change and its direction, and should avoid words such as “increased” or “improved.”

Syntax of an output statement

Phrased in the past tense.

Includes information on:

• what was delivered or rendered;
• in what subject;
• and to or for whom.

Remember! Outputs are not results

It should be objective

Outputs should be objective and contain no subjective terms. If words are added to further qualify the product or service the output describes, the words should have a standard and commonly understood definition. The definition can be included as a footnote in the logic model.

It represents a completed package of activities

In the logic model, an output statement is a package of completed work. In the outputs and activities matrix, each output is broken down into its component activities. Further breakdown below the activity level to sub-activities is possible. However, sub-activities should appear only in the project work breakdown structure and not in the outputs and activities matrix. Consequently, it is important to differentiate between the output itself, activities and sub-activities.

Example of outputs:

• Cultural event held in the region X
• Round table discussions held on climate issues with organizations from region Y
• Research report published on X theme
• Security workshop provided to Indo-Pacific counterparts

For more information, please refer to Tip sheet 2.2: Syntax structure of outcome, output and activity statements

2.2 The logic model

Like a roadmap or a blueprint, a logic model is a visual depiction of the main elements of a theory of change for a specific project or program, reflecting the series of changes that are critical to achieving project success. It depicts the logical connections between the planned outputs and the expected outcomes (immediate, intermediate and ultimate) that the project aims to achieve or contribute to. Global Affairs Canada’s logic model starts at the ultimate outcome level and now ends at the output level.

The logic model forms a pyramid shape with multiple complementary pathways branching off below one ultimate outcome level. Each pathway addresses a different aspect or element of the issue targeted by the project. Achievement of the ultimate outcome depends on the achievement of all outcomes along each pathway. Arrows between the levels represent assumptions (explained in the theory of change narrative) about why the outputs or outcomes from one level should lead or contribute to the changes at the next level, and about existing conditions, including risks, which may affect the achievement of the outcomes.

Remember! The logic model is a key Results-Based Management design and management tool—not a form to fill out and then file away

The logic model is used as both a planning and design tool during the development of a project, and a management tool during project implementation.

The purpose of the logic model is to:

• visually reflects the main elements of the project’s theory of change
• help the project team, stakeholders and managers develop and visualize the project’s theory of change, and validate that it is sound and that the expected outcomes are realistic and relevant;
• ensure that the project performance measurement framework and monitoring and evaluation strategy are clearly linked to the theory of change and expected outcomes
• help the project team manage the project for results;
• serve as a key reference point for monitors and evaluators; and
• facilitate communication about the project to staff and other stakeholders.

Ultimate Outcome

• 1000: Enhanced realization of joint Canadian and Indo-Pacific stakeholder interests in key developments related to security, trade, people to people ties, climate, and Canadian engagement in the Indo-Pacific Region

Intermediate and Immediate Outcomes

Knowledge Sharing

• 1100: Increased meaningful engagement of Canadian organizations with their Indo-Pacific counterparts in conferences, workshops, round table discussions, and panels, in and/or on the Indo-Pacific region
• 1110: Increased access for Canadian organizations to participate in events which promote knowledge sharing with their counterparts in the Indo-Pacific region
• 1120: Increased capacity of Canadian organizations to facilitate knowledge sharing events with their counterparts in the Indo-Pacific region

Training

• 1200: Increased meaningful engagement of Indo-Pacific CSOs in Canada and Canadian CSOs in the Indo-Pacific region
• 1210: Increased capacity of Canadian and Indo-Pacific CSOs to effectively operate within the Indo-Pacific region and Canada
• 1220: Increased ability for Canadian and Indo-Pacific CSOs to facilitate training sessions with/for their counterparts in the Indo-Pacific region and Canada

Research

• 1300: Increased collaboration between Canadian academics and researchers with their Indo-Pacific colleagues
• 1310: Increased capacity of non-GoC researchers to be able to conduct research in/on the Indo-Pacific region
• 1320: Increased capacity for Canadian researchers to facilitate events with/for their counterparts in the Indo-Pacific region and Canada

Indo-Pacific cultural events

• 1400: Strengthened cultural interconnectedness between Canadians and their Indo-Pacific counterparts
• 1410: Increased access for CSOs, to participate in events which promote partnerships with their counterparts in the Indo-Pacific region and Canada
• 1420: Increased capacity of Canadian CSOs to facilitate cultural events with their counterparts in the Indo-Pacific region and Canada

Indigenous events and reconciliation

• 1500: Increased interconnectedness of Indigenous Peoples within the Indo-Pacific region and Canada.
• 1510: Increased capacity for Indigenous Peoples to hold events within the Indo-Pacific region and Canada
• 1520: Increased access for Indigenous Peoples to participate in cultural events in the Indo-Pacific region and Canada

The work of others

Note that the logic model captures only the relationships between the outputs delivered by the project and outcomes to which they contribute. In many cases, logic model outcomes are also dependent on the work of other actors, for example, other donors or local organizations. The work of others is not usually captured in the logic model, but it should be captured as “assumptions” in the theory of change narrative.

Standard template – logic model

Global Affairs Canada has a standard template for a logic model.

Logic model structure

In a Global Affairs Canada logic model, an ultimate outcome (change in state, conditions or well-being of beneficiaries) should be supported by two or three intermediate outcomes (changes in performance, behaviour or practice) that are expected to occur in order for it to be achieved. This is because there are usually multiple changes in performance, behaviour or practice among various actors that need to occur to make the change at the ultimate outcome level possible.

Each intermediate outcome should be supported by two or three immediate outcomes (changes in capacity: skills, ability, knowledge, etc.). This is because there are usually multiple needs in terms of capacity that need to be addressed in order for a change in performance, behaviour or practice (the intermediate outcome) to occur.

Each immediate outcome should be supported by two or three outputs (direct products or services stemming from the project activities). This is because it will often take more than one product or service to bring about a change in capacity.

2.3 Indicators

2.3.1 Developing indicators for your project

Indicators are the core component of the performance measurement framework.

See section 2.4 Performance Measurement Framework for more information.

It is important that the stakeholders agree beforehand on the indicators that will be used to measure the performance of the project.

Quantitative indicators

Quantitative indicators are used to measure quantities or amounts.

 Example of quantitative indicators

• Number of regional civil society organizations engaged
• Number of cultural events attended
• Number of academics who attended a research conference

Qualitative indicators

Qualitative indicators capture experiential information, such as the quality of something, or beneficiaries’ perception of their situation. They can help measure the presence or absence of specific conditions, or an individual or group’s perception of how a service compares with established standards.

Qualitative indicators can capture contextual information about situations, events and practices.

Note: There has been much debate regarding the value of quantitative data and that of qualitative information and whether quantitative measures (or indicators) are better than qualitative ones. This debate is now almost settled in the evaluation field with the growing usage of mixed methods. Practitioners have abandoned the idea that these sources of information are irreconcilable: both types of information are necessary. In fact, all quantitative measures are based on qualitative judgments and all qualitative measures can be coded and analyzed quantitatively.

To adequately assess the achievement of results, an officer/manager needs both quantitative and qualitative measures. For example, ‎it is not enough to know how many women are participating in an activity. The quality of their participation and experience is also important to capture to have a full picture.

Because it is difficult to organize qualitative data for comparison or analysis, qualitative indicators should be quantified wherever possible. This can be done by using a scale, for example, “level of confidence (1-4 scale) of farmers (f/m) in the security of roads leading to local market”.

 Example of a qualitative indicator with scale:

• Scale (1 to 5) how likely surveyed Indo-Pacific civil society organizations are to engage with their Canadian counter-parts.
• Scale (1 to 5) willingness to participate in future Canadian and Indo-Pacific research events.

 Example of other qualitative indicators:

• %/total of participates who thought that an event effectively served a good purpose
• %/total of participates who feel that they can better engage in the Indo-Pacific region

Remember! Proper disaggregation of data is vitally important to the usefulness of the data collected.

Structure of a performance indicator

Performance indicators are composed of three elements: a unit of measure, a unit of analysis and a context.

The unit of measure is the first element of the indicator: number, percentage, level, ratio, etc. It is important to include in the unit of measure the notion of proportionality, by ensuring that it contains both a numerator and a denominator. This is often expressed by stating the unit of measure as number out of total (#/total) or percentage out of total (%/total).

The unit of analysis is who or what will be observed: individuals, institutions, social artifacts or social groups. The type of unit of analysis will determine whether the data will need to be disaggregated by sex, age, ethnicity, rural/urban setting, socio-economic status, ownership or any other category relevant to the project or program. This disaggregation is vitally important to the usefulness of the data collected. For example, it is impossible to measure changes in women’s access to basic services if the data collected during project monitoring does not disaggregate by sex. Similarly, a project that aims to improve the health of a specific marginalized ethnic group through rehabilitating and staffing remote regional health centres would need those centres to collect patient information in a way that allows disaggregation by ethnicity.

Table 1 - Unit of analysis by type

The context is the set of circumstances that specify the particular aspect of the output or outcome that the indicator is intended to measure. For example, if the expected outcome is "Improved access to government-funded primary schools for girls and boys of province X in country Y", and it has been determined that one way to measure progress is to see how many children live within a certain distance from a publically-funded school, then the context could be “living within a one-kilometre walk of a provincially-funded primary school.”

Table 2 - Illustration of the structure of a performance indicator

Leading, lagging and coincident indicators

We generally use indicators to measure progress on outcomes in the logic model. Sometimes, however, you may also want to measure the assumptions articulated in the theory of change narrative represented by the arrows in your logic model. In this case you can use “leading” indicators to measure things preceding the change or “lagging” indicators to measure things that follow the change. Data on these indicators can validate these assumptions. As explained above, at each level in the logic model, we are making assumptions. Leading and lagging indicators allow us to track those assumptions by measuring a little lower or a little higher than the actual outcome itself, without actually measuring the next level in the logic model.

Ideally, indicators would always measure things that directly coincide with the changes described in the expected outcomes of your logic model. In some cases it may be difficult or impossible to find such “coincident” indicators. In these cases, you can also use "leading" or "lagging" indicators.

The concept of leading, lagging, and coincident indicators is borrowed from the business cycle in economics. The following example of a traffic light is helpful to further explain the concept.

Types of changes measured by indicators

Each indicator can be classified according to what level it measures in the logic model: outputs, immediate outcomes, intermediate outcomes and ultimate outcomes.

• Output indicators: Measure the products and services stemming from project activities and delivered by the implementer to the intermediaries or beneficiaries. For example, number of workshops delivered, or the number of conferences attended.
•  Immediate outcome indicators: Measure the changes in capacity, such as skills, knowledge, abilities, or awareness, and sometimes access, of the intermediaries or beneficiaries as a consequence of the outputs. For example, number of participant of the climate awareness event capable to stating 3 ways climate impact the indo-pacific region, or a scale (from 1-10) on the level of confidence an organization has to use newly learned skills.
• Intermediate outcome indicators: Measure the changes in behaviour, practice or performance of intermediaries or beneficiaries as a consequence of the immediate outcomes. For example, # of collaborations or partnership formed as a result of the event, or %/total of attendees who have made policy changes to their CSO based on lessons from the workshops.
• Ultimate outcome indicators: Measure the sustainable changes in the lives of beneficiaries as a consequence of the intermediate outcomes. For example, impacts of partnerships formed during the event.

Fore more information, please refer to Tip sheet 3.1: Selection of performance indicators by level of outcome

Criteria of a strong performance indicator

1. Validity: Does it measure what it is intended to measure?
   • Will this performance indicator really measure the outcome and output? Remember that different indicators are meant to measure the different levels of the logic model.
2. Reliability: Will it be consistent over time?
   • Does this performance indicator enable you to measure the outcome and output over time?
   • Will it consistently produce the same data if it is applied repeatedly to the same situation over time?
   • If a different researcher collects the data, will it be consistent?
3. Sensitivity: Will it measure changes as they happen?
   • When the change described by the outcome statement starts to happen, will the indicator be sensitive enough to pick up on that change?
   • Will the performance indicator measure both improvements and deterioration in the situation (in other words is it neutral)?
4. Simplicity: How easy will it be to collect the data?
   • Are the sources of information easily accessible?
   • Are the equipment and/or expertise needed to track the performance indicator readily available?
   • Does this performance indicator enable a relatively easy analysis of the result?
   • Is it clear and direct enough to be understood by all stakeholders?
5. Usefulness: Will the information collected be useful for decision-making?
   • Does the information provided by the performance indicator meet the needs of its audience?
   • Does knowing this information help you or other stakeholders to do things better or more effectively?
   • Is the performance indicator expressed in a way that will resonate with the intended audience?
   • Does the performance indicator make it easy to communicate the status of the result?
   • Will this performance indicator provide the information in a timeframe that allows it to be useful?
6. Affordability: Do you have the resources to collect data?
   • Can you afford to collect data on this performance indicator?
   • Is the potential cost worth the information you will get?

2.4 The Performance Measurement Framework

It is important to house all of your project's indicators in an easy to manage document which shows the indicator's alignment to their respective outcomes. At GAC we use a Performance Measurement Framework (PMF); here are some details on how we create and use the PMF, please consider this when creating your reporting and monitoring tool.

At Global Affairs Canada, the performance measurement framework is the Results-Based Management tool used to systematically plan the collection of relevant indicator data over the lifetime of the program/portfolio and project, in order to assess and demonstrate progress made in achieving expected results. The performance measurement framework is the “skeleton” of the monitoring plan: it documents the major elements of the monitoring system in order to ensure regular collection of actual data on the indicators identified in the performance measurement framework. The performance measurement framework contains all of the indicators used to measure progress on or toward the achievement of the program/portfolio’s and project’s expected outcomes and outputs. In addition, it specifies who is responsible for collecting data on the indicator, from what source, at what frequency and with what method. It also includes the baseline data and target for each indicator.

As with the logic model, the performance measurement framework should be developed and/or assessed in a participatory fashion with the inclusion of local partners, intermediaries (duty bearers / responsibility holders), beneficiaries (rights holders) and other stakeholders, and relevant Global Affairs Canada staff.

Using the Performance Measurement Framework for management

The performance measurement framework facilitates the “management for results” during program/portfolio and project implementation. It provides a plan for the collection of data during implementation. The actual data collected on indicators identified in the performance measurement framework, and the program/portfolio’s and project team’s analysis of this data, allows the team to assess progress, and detect issues that may interfere with the achievement of expected outcomes early enough to take corrective action/make adjustments. An operationalized performance measurement framework is thus necessary for evidence-based program/portfolio and project management decision-making. Of course, this can only be done if there is a basis for comparison. For this reason, it is always necessary to capture baseline data and it is always necessary to set targets in the performance measurement framework. Remember: without knowing where you started and where you want to go, it is impossible to properly assess progress.

In sum, the performance measurement framework will help you:

• plan for the systematic collection of relevant data over the lifetime of the program/portfolio and project;
• document the major elements of the monitoring system; and
• ensure regular collection of actual data for every indicator in the performance measurement framework.

The data collected on the performance measurement framework indicators will help you:

• measure and assess progress on or toward the expected outcomes;
• demonstrate to all stakeholders the progress made in achieving outcomes; and
• make evidence-based decisions to keep the program/portfolio and project on track to achieve the expected outcomes.

Standard template – Performance Measurement Framework

Global Affairs Canada has a standard template for a performance measurement framework.

Content of the Performance Measurement Framework

The performance measurement framework is divided into eight columns: expected results, indicators, baseline data, targets, data sources, data collection methods, frequency, and responsibility. To complete a performance measurement framework, you will need to fill in each of the columns accurately.

For more information, please refer to Tip sheet 3.2: Outcomes, indicators, baseline, targets and actual data

3. Regional Connectivity Envelope (RCE) RBM examples

Please refer to these examples to create your own RBM for your specific project:

3.1 Examples of logic model with RCE outcomes and optional - Project personalized outcomes

3.1.1 Logic model for Climate Initiative Conference in India 

 Rainfalls Canada, in partnership with the Indian organization One Climate, is spearheading a week-long conference on climate initiatives to be held in New Delhi. Rainfalls Canada will act as the primary facilitator for this event, with One Climate offering crucial support in organizing and executing the project.

This conference is designed to enhance knowledge sharing, aiming to foster deeper and more meaningful engagement between Canadian organizations and their Indo-Pacific counterparts. It seeks to achieve key intermediate outcomes by facilitating increased interaction through conferences, workshops, roundtable discussions, and panel discussions focused on the Indo-Pacific region. Additionally, the project is expected to result in immediate outcomes by providing greater access for Canadian organizations to engage in events that promote knowledge exchange with Indo-Pacific counterparts.

The Climate Initiative Conference aligns with strategic objective #4 of Canada’s Indo-Pacific Strategy, which emphasizes building a sustainable and green future. This conference will serve as a platform for both organizations to exchange valuable experiences and explore collaborative efforts aimed at reversing biodiversity loss, fostering sustainable economic growth, and developing communities that are resilient to climate change. Through this initiative, both organizations are committed to driving forward impactful climate solutions and strengthening international cooperation in the pursuit of a greener future.

Ultimate outcomes

Regional Connectivity Envelope ultimate outcome 1000:

• Enhanced realization of joint Canadian and Indo-Pacific stakeholder interests in key developments related to security, trade, people to people ties, climate, and Canadian engagement in the Indo-Pacific region

Rainfalls Canada ultimate outcome 1000:

• Enhanced realization of joint Canadian and Indo-Pacific stakeholder interests on reversing climate change in Canada and the Indo-Pacific region.

Intermediate outcomes

Regional Connectivity Envelope intermediate outcome 1100:

• Knowledge Sharing: Increased meaningful engagement of Canadian organizations with their Indo-Pacific counterparts in conferences, workshops, round table discussions, and panels, in and/or on the Indo-Pacific region.

Rainfalls Canada intermediate outcome 1100:

• Increased cooperation and between Canadian and Indo-Pacific stakeholders on climate issues within Canada and the Indo-Pacific region.

Immediate outcomes

Regional Connectivity Envelope immediate outcome 1110:

• Increased access for Canadian organizations to participate in events which promote knowledge sharing with their counterparts in the Indo-Pacific region.

Rainfalls Canada immediate outcome 1110:

• Increased access for Canadian and Indo-Pacific organizations to participate in events which promote knowledge sharing on climate issues in Canada and the Indo-Pacific region.

 Outputs

• Canadian and Indian organizations engaged in discussions on climate related issues
• Knowledge sharing discussions facilitated
• Conference proceedings and materials made available to participants and the public
• Reports and publications summarizing the outcomes and recommendations from the conference published
• Feedback gathered from event participants
• Venue, accommodation and transport for the event booked
• Translation services booked
• Invitation for key speakers and participants sent
• Social media posts published and promotion materials distributed
• Discussion held with Indo-Pacific regional partner for event planning and logistics
• Identified and coordinated with key speakers from both regions
• Project plan developed with collaboration with Indo-Pacific regional organization

 Activities

1. Planning and coordination: 
   • Develop a detailed project plan and timeline
   • Identify and invite key speakers and participants from both regions (government, think tanks, academics institutions)
   • Coordinate with regional organizations in India for logistics and support
2. Marketing and outreach: 
   • Design and distribute promotional materials
   • Engage in social media marketing campaigns
   • Send invitations and manage registrations
3. Event execution: 
   • Book and set up the venue
   • Buy technical equipment
   • Facilitate conference sessions, workshops, and panel discussions
   • Manage logistics (transportation, accommodation, catering)
4. Follow-up and evaluation: 
   • Gather feedback from participants
   • Evaluate the success of the event in terms of attendance, engagement, and outcomes
   • Prepare a final report and disseminate findings

Indicators

• Ultimate outcome 
  • Frequency and quality of follow-up engagements between Canadian and Indo-Pacific stakeholders post-conference
  • Long-term impact of the knowledge shared and partnerships formed during the conference
• Intermediate outcome 
  • Scale (from 1-10) on the effectiveness of the event(s) to connect and create lasting partnerships between Canadian and Indo-Pacific stakeholders to address climate issues
  • %/total of participates who thought that an event effectively served a good purpose
  • Number of new partnerships or collaborations formed as a result of the conference
• Immediate outcome 
  • Number of Canadian organizations that participated in the conference
  • Number of Indian organizations that participated in the conference
  • Number of other Indo-Pacific organizations that participated in the conference
  • %/total of participates who feel that they can better engage in the Indo-Pacific region

3.1.2 Logic Model for Cybersecurity training in Japan

CyberCan, in partnership with Guardians of Japan’s Web, is launching a project to develop and deliver a series of comprehensive cybersecurity training programs. This will be tailored for Japanese civil society organization (CSO) representatives. CyberCan will take the lead as the primary facilitator, ensuring the effective execution and impact of the training initiative.

This project is categorized under training initiatives and is set to significantly impact key outcomes. It will contribute meaningfully to the intermediate outcome: enhanced engagement between Indo-Pacific CSOs and Canadian organizations. As well as strengthening interactions between Canadian CSOs and their Indo-Pacific counterparts. The immediate outcome consists of increasing the capacity of both Canadian and Indo-Pacific CSOs to operate effectively within their respective regions and enhance their ability to facilitate training sessions for their counterparts.

The cybersecurity training initiative in Japan aligns directly with Strategic Objective 1 of Canada’s Indo-Pacific Strategy, which emphasizes the promotion of peace, resilience, and security in the region. By addressing this objective, the project aims to strengthen the resilience of Indo-Pacific nations against security challenges. The training will build critical cybersecurity capabilities within Japanese CSOs, enabling them to better protect against and respond to cyber-crimes, threats, and attacks. The project will also foster collaboration between Canadian and Japanese cybersecurity organizations, facilitating knowledge exchange and joint efforts to address cyber threats. By improving cybersecurity defenses, the project contributes to a more stable and secure digital environment. This is essential for maintaining peace and stability in both the Indo-Pacific region and Canada.

Ultimate outcomes

Regional Connectivity Envelope ultimate outcome 1000:

• Enhanced realization of joint Canadian and Indo-Pacific stakeholder interests in key developments related to security, trade, people-to-people ties, climate, and Canadian engagement in the Indo-Pacific Region.

Cybersecurity training in Japan ultimate outcome 1000:

• Enhanced cybersecurity of Canadian and Japanese civil society organizations within Canada and Japan.

Intermediate outcomes 

Regional Connectivity Envelope intermediate outcome 1200:

• Increased meaningful engagement of Indo-Pacific CSOs in Canada and Canadian CSOs in the Indo-Pacific region.

Cybersecurity training in Japan intermediate outcomes 1100 and 1200:

• Increased engagement between Canadian and Japanese CSOs on cybersecurity matters in Canada and Japan.
• Increased effectiveness of cybersecurity protection measures by Canadian and Japanese CSOs in Canada and Japan.

Immediate Outcomes 

Regional Connectivity Envelope immediate outcomes 1210 and 1220:

• Increased capacity of Canadian and Indo-Pacific CSOs to effectively operate within the Indo-Pacific region and Canada.
• Increased ability for Canadian and Indo-Pacific CSOs to facilitate training sessions with/for their counterparts in the Indo-Pacific region and Canada.

Cybersecurity Training in Japan immediate outcomes 1110 and 1120:

• Increased access for Canadian and Japanese CSOs to participate in cybersecurity knowledge sharing events.
• Increased capacity for Canadian and Japanese CSOs to engage one another on cybersecurity matters.

Cybersecurity Training in Japan immediate outcomes 1210 and 1220:

• Increased knowledge of Canadian and Japanese CSOs on cybersecurity preparedness measures.
• Increased capacity of Canadian and Japanese CSOs to better protect themselves from cybersecurity threats.

Outputs 

• Need assessment: 
  • Conducted needs assessment with # Japanese CSO on cybersecurity training needs
• Training Materials: 
  • Develop a training program based on needs
  • Developed and distributed # comprehensive training manuals and # online resources.
• Logistics and Coordination 
  • Booked training venue, accommodation and travel
  • Sent invitations to participants and trainers
  • Arranged IT equipment
• Training Delivered: 
  • Conducted # cybersecurity workshops with participation from # Japanese CSOs.
• Capacity Building: 
  • Trained # Japanese CSO representatives as trainers.
• Networking: 
  • Hosted # workshops and # networking event with # participants.
• Post-Training 
  • Conducted post-training assessments

Activities 

1. Needs Assessment:
   • Conduct a needs assessment to identify specific cybersecurity training needs and priorities of Japanese CSOs.
2. Curriculum Development: 
   • Develop a tailored cybersecurity training curriculum based on the needs assessment and Japanese cybersecurity methods.
   • Create and distribute training materials, guides, and online resources for ongoing learning and reference.
3. Logistics and Coordination: 
   • Arrange logistical details including venue selection, accommodation, travel, training materials, and scheduling.
4. Training Program Delivery: 
   • Deliver a series of training workshops in Japan focusing on:
   • Introduction to core cybersecurity principles and practices.
   • Detailed sessions on advanced cybersecurity methods, tools, and incident response.
   • Hands-on training with real-world scenarios and cybersecurity tools.
   • Include sessions on regional cybersecurity issues and the specific cybersecurity challenges faced by Indo-Pacific CSOs.
5. Training of Trainers: 
   • Conduct a specialized module to train selected Japanese CSO representatives on how to facilitate cybersecurity training within their own organizations and networks.
6. Workshops and Seminars: 
   • Host workshops and seminars with Japanese CSOs to share best practices, case studies, and insights gained from the training.
7. Networking Events: 
   • Organize networking events to foster collaboration between Canadian and Japanese CSOs and other Indo-Pacific stakeholders.
8. Post-Training Assessment: 
   • Conduct assessments to evaluate the effectiveness of the training, including participant feedback and performance evaluations.

Indicators

• Ultimate outcome:
  • Number of joint initiatives: Number of new joint initiatives or collaborative projects between Canadian and Japanese CSOs.
  • Quality of partnerships: Assessments of the depth and quality of partnerships formed (through surveys or interviews with partners).
  • # of cyber threats against Canadian and Japanese CSOs thwarted.
• Intermediate outcome:
  • Number of attendees at networking events: Total number of participants at networking and collaborative events.
  • Application of skills: Number of participants who apply the cybersecurity practices learned in their work or in new projects (measured through surveys).
  • Improved cybersecurity practices: Number of improved cybersecurity practices or policies adopted by participating CSOs (measured through surveys).
  • Number of training sessions facilitated: Count of training sessions or workshops facilitated by the trained Japanese CSOs.
  • Feedback on facilitation: Quality of feedback received from participants in training sessions facilitated by the newly trained trainers (through evaluation forms).
  • Frequency of interaction: Number of interactions, meetings, or collaborative activities between Canadian and Japanese CSOs after the training.
• Immediate outcome:
  • Number of workshops conducted: Count of cybersecurity workshops and training sessions held.
  • Number of participants: Total number of participants attending the training, broken down by organization and role.
  • Number of training manuals created: Count of training manuals or guides developed.
  • Number of online resources: Number of online resources made available.
  • Number of trainers trained: Number of Japanese CSO representatives trained as trainers.
  • Quality of training materials: Participant ratings on the usefulness and clarity of training materials (through surveys using 1-5 scale).
  • Number of networking events: Count of seminars, and networking events organized and held.
  • Self-reported knowledge improvement: Percentage of participants reporting increased knowledge and skills in cybersecurity post-training (measured via pre- and post-training surveys).

Annexes

Tip Sheets

We designed the following tip sheets and checklists to help users navigate sections of the RBM guide

• Tip Sheet 2.1: Results chains and definitions
• Tip sheet 2.2: Syntax structure of outcome, output and activity statements
• Tip sheet 3.1: Selection of performance indicators by level of outcome
• Tip sheet 3.2: Outcomes, indicators, baseline, targets and actual data

Tools and templates

The following tools make it easier to manage for results throughout the entire life cycle of an investment or project for Global Affairs Canada staff, partners and executing agencies. Global Affairs Canada has standard templates for these tools:

• Logic model template and outputs and activities matrix template (PDF, 91 KB)
  • Help: Opening PDFs on your computer
• Performance Measurement Framework template (PDF, 1 MB)
  • Help: Opening PDFs on your computer