Results-Based Management checklist 2.1 – How to assess and/or review a performance measurement framework

This checklist is a companion to the Results-Based Management for International Assistance Programming at Global Affairs Canada: A How-to Guide.It expands on the discussion on performance measurement frameworks in the How-to Guide and provides step-by-step instructions to help you review and assess performance measurement frameworks (PMFs).

Assessment criteria: performance measurement framework structure

• Does the performance measurement framework use the Global Affairs Canada template?
• If not^{Footnote 1}, does the organization have its own Results-based Management templates and methodology?
  • Does the multilateral or global organization use its own templates and Results-based Management methodology correctly?
• Is the overall performance-measurement plan, as reflected in the performance measurement framework, realistic given time and resources available?

Assessment criteria: by performance measurement framework (or equivalent) component

Expected outcomes

• Are the expected outcomes and outputs listed in the first column exactly the same as those listed in the logic model?

Performance indicators

• Have indicators been identified for each outcome and output?
• Do the indicators measure each of the expected outcomes and outputs for which they have been selected?
  • Do the indicators at the ultimate level measure the change in state, condition or wellbeing of the ultimate beneficiaries described in the ultimate outcome statement?
  • Do the indicators at the intermediate level measure the changes in behavior, practices or performance of the intermediaries or beneficiaries described in the intermediate outcome statement?
  • Do the indicators at the immediate level measure the changes in capacities, such as skills, knowledge, abilities, etc., of intermediaries or beneficiaries, described in the immediate outcome statement?
  • Do the output-level indicators measure the existence or quality of the products or services for which the implementer is accountable or the process by which those products and services were produced?
• Will the implementer or local partner be able to collect data on the qualitative and quantitative indicators identified in the performance measurement framework?
• Are there 2-3 indicators for each outcome and 1-2 for each output?
• Is there at least one qualitative and one quantitative indicator per outcome?
• Do the indicators meet the standard criteria of validity, reliability, sensitivity, simplicity, usefulness and affordability?
• Are the indicators neutral?
• Are indicators that deal with people disaggregated by sex and other necessary social and/or demographic categories (age, ethnicity, etc.) wherever possible?
• Does each indicator include clear units of measure and of analysis?
• Are the units of measure and of analysis consistent with those used in the indicators’ baseline data and targets?
• Do the indicators for each outcome address all dimensions of the expected outcome for which they have been identified (gender equality, environment, human rights, targeted regions/areas etc.)?
• Do the indicators capture proportionality (i.e. does the unit of measure have a numerator and a denominator, such as # out of total or % out of total)?

Baseline data

• Are there baseline data for each indicator? Do these data identify the units of measure and of analysis?
• Is each baseline-data value accompanied by a date that will serve as a point of reference?
• Do the units of measure and of analysis match those listed in the indicators and targets?
• Do the baseline data include proportionality (i.e. does the unit of measure have a numerator and a denominator)?
• Do the baseline data describe the same dimensions (age, sex, targeted regions/areas, etc.) as presented in the indicators and their targets?
• Are the baseline data for indicators that deal with people disaggregated by sex and other necessary social and/or demographic categories (age, ethnicity, etc.) wherever possible?
• If baseline data have not yet been identified, does the performance measurement framework indicate when these data will be collected?

Targets

• Are the targets realistic and achievable given the scope, funds and timeframe of the project?
• Do the targets specify an achievement date?
• Do the units of measure and of analysis match those units in the indicators and baseline data?
• Does the target include proportionality (i.e. does the unit of measure have a numerator and a denominator)?
• Do the targets describe the same dimensions (age, sex, targeted regions/areas, etc.) as presented in the indicators and their baseline data?
• Are the targets for indicators that deal with people disaggregated by sex and other necessary social and/or demographic categories (age, ethnicity, etc.) wherever possible?
• If targets have not yet been identified, does the performance measurement framework indicate when they will be established?

Data sources

• Do the data sources identify the individuals, groups, organizations, or publications from which data on performance indicators will be obtained?
• Are the data sources easily accessible to allow the project officer to receive timely performance information?
• Are the data sources diversified, credible and reliable?
• Are existing data sources used where possible?

Data collection methods

• Do the data collection methods identify how the information will be collected (e.g. analysis of records or documents, literature review, survey, interview, focus group, comparative study, direct observation, etc.)?
• Are the data collection methods identified appropriate?
• Are existing data collection tools used whenever possible?

Frequency

• Will the collection frequency identified for each outcome and output contribute the performance information required for effective Management for Results by both Global Affairs Canada and the implementer of the initiative during implementation and for Global Affairs Canada’s reporting?

Responsibility

• Does the performance measurement framework identify the best group (e.g. beneficiaries, local professionals, partner organizations, etc.) to collect, manage and validate and assess the performance information?
• Will the group identified have the capacity to collect, manage, validate and assess the data in a timely fashion?

Assessment criteria: performance measurement framework narrative (or equivalent)

• Does the performance measurement framework narrative (or equivalent) outline how the performance measurement framework was developed (i.e. who was consulted, how indicators were selected, alignment with local systems, baseline study, how targets were selected, etc.)?
• Does the performance measurement framework narrative (or equivalent) outline weaknesses in the local monitoring and evaluation systems to be used, and describe how this will be addressed?
• Does the performance measurement framework narrative (or equivalent) identify who will be responsible for development and testing of data collection tools and processes?