Internal audit of Grants and Contributions in Partnerships with Canadians Branch

Final Report
December 2012

Acronyms and Initialisms

Executive Summary

In accordance with its approved Risk-Based Audit Plan for 2011–2014, the Office of the Chief Audit Executive (OCAE) at the Canadian International Development Agency (CIDA) conducted an internal audit of grants and contributions in Partnerships with Canadians Branch (PWCB). The audit objective was to provide reasonable assurance that effective measures are in place to ensure that the use and management of grant and contribution funding in PWCB is compliant with both Treasury Board's Policy on Transfer Payments and Directive on Transfer Payments as well as with applicable CIDA policies.

On October 1, 2008, the updated Policy on Transfer Payments and its supporting directives took effect. The policy and directives call for a risk-based approach in the administration of transfer payments by all departments; simplification of program requirements; and level of monitoring, auditing, and reporting that is proportionate to the risk associated with the recipient.

PWCB supports development projects and programs that are designed and implemented by Canadian organizations in cooperation with their counterpart organizations in developing countries. Projects are selected for funding primarily via a call for proposals business process and are funded on a cost-shared basis.

CIDA's corporate suite of policies, guidelines, and procedures to support the application and adoption of risk management provides a comprehensive framework for the Agency. This is intended to provide a consistent and systematic approach that helps programming branches, including PWCB, execute their responsibilities for financial stewardship and risk management. However, for projects that were launched prior to the introduction of the Risk-Based Administration Matrix (RBAM), we found there is rarely a discernible link between the level of risk associated with the recipient and the administrative requirements imposed upon that recipient. We expect this issue will be addressed as the RBAM is implemented for all of CIDA's funded projects.

PWCB's Project Officers stated the training and tools they have been provided are sufficient and have adequately prepared them to discharge their responsibilities for assessing and managing risk.

A suite of corporate guidance documents exists that set out roles, responsibilities, and accountabilities relating to the administration and management of grant and contribution funding. Although high-level roles and responsibilities are consistently understood, there is limited guidance or detailed procedures and tools to help Project Officers execute their day-to-day monitoring responsibilities, such as processing advance payment requests and advance justifications, analyzing performance and financial reports, and planning and conducting site visits. In our review of a sample of project files, we noted several issues that may have been mitigated by the existence of such documented processes and procedures.

Performance results and achievements are monitored and used in some decision making, such as what types of calls for proposals to undertake, as well as in the design and redesign of programming.

Although CIDA has developed a suite of tools and training that provides for a comprehensive framework for performance measuring and monitoring, this framework does not necessarily address the nature and operational realities of PWCB's business. As a result, PWCB has experienced challenges in rolling up performance results for higher-level reporting, and in aligning and reporting the results of its various projects with the branch's various sectors and with branch and Agency objectives. Furthermore, there is currently substantial manual input and tracking of results by PWCB as a result of the limitations of the Investment Monitoring and Reporting Tool.

PWCB is not formally coordinating its efforts with other implicated branches in program design, delivery, monitoring, and reporting to ensure harmonized programming. Furthermore, the Fiduciary Risk Evaluation Tool (FRET) is currently the only formalized means through which information on recipients is shared across branches. However, FRET is not an all-encompassing tool for capturing intelligence on and experience gained in dealing with recipients. CIDA is making efforts to streamline the administration of grant and contribution programming, and several new initiatives that would allow for improved information and knowledge sharing are currently underway as part of the Agency's Business Modernization Initiative (BMI).

Audit Conclusion

Effective measures are in place to ensure the use and management of grant and contribution funding in PWCB is compliant with both the Policy on Transfer Payments and Directive on Transfer Payments and with applicable CIDA policies. However, there are opportunities to strengthen and improve the operation of the branch's measures, specifically around oversight and performance measurement.

Statement of Conformance

In my professional judgment as the Chief Audit Executive, this audit was conducted in conformance with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing and with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program. Sufficient and appropriate audit procedures were conducted, and evidence gathered, to support the accuracy of the findings and conclusion in this report, and to provide an audit level of assurance. The findings and conclusion are based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed upon with management, and are only applicable to the entity examined and for the scope and time period covered by the audit.

Chief Audit Executive

1.0 Background

An internal audit of grants and contributions in PWCB was included in the 2011–2014 Risk-Based Audit Plan recommended by the Audit Committee, and was approved by the President on March 25, 2011.

On October 1, 2008, the updated Policy on Transfer Payments and its supporting directives took effect. The policy was updated in response to specific recommendations made in the Report of the Independent Blue Ribbon Panel on Grant and Contribution Programs. The report identified a fundamental need for change in the way that the Government of Canada understands, designs, manages, and accounts for grant and contribution programs. The report also highlighted that the simplification of administrative requirements for recipients and program administration could strengthen accountability in the management of grant and contribution programs. To this end, the updated Policy on Transfer Payments and its supporting directives call for a risk-based approach in the administration of transfer payments by all departments; simplification of program requirements; and level of monitoring, auditing, and reporting that is proportionate to the risk associated with the recipient.

PWCB supports development projects and programs that are designed and implemented by Canadian organizations in cooperation with their counterpart organizations in developing countries. Partners' projects and programs are funded on a cost-shared basis, and are supported by the:

• Global Citizens Program, which engages and mobilizes Canadians as global citizens to participate in international development initiatives; and
• Partners for Development Program, which leverages the development expertise and initiative of Canadians by funding the most meritorious proposals put forward by Canadian organizations to deliver development results on the ground and contribute to poverty reduction.

Projects are selected for funding primarily via a call for proposals business process. This business process was introduced in 2011–2012.

Half of PWCB's grant and contribution funding is allocated to 20 countries of focus, while the remaining 50 percent supports initiatives in other countries eligible for official development assistance. Furthermore, 80 percent of these partnership investments align to CIDA's three priority themes: sustainable economic growth, children and youth, and food security.

2.0 Audit Objective, Scope, Approach, and Criteria

2.1 Objective

The objective of this audit was to provide reasonable assurance that effective measures are in place to ensure the use and management of grant and contribution funding in PWCB is compliant with both Treasury Board's Policy on Transfer Payments and Directive on Transfer Payments, as well as with applicable CIDA policies.

2.2 Scope

The audit focused on PWCB's compliance with the requirements of the Policy on Transfer Payments and Directive on Transfer Payments that directly address the Report of the Independent Blue Ribbon Panel on Grant and Contribution Programs. These requirements centre on:

• risk-based program management and administration; and
• internal coordination among CIDA's various program branches.

The audit examined the use and management of grant and contribution funding by PWCB over the last three years (2009–2010, 2010–2011, and 2011–2012), as well as its compliance with select requirements of the Policy on Transfer Payments and Directive on Transfer Payments, which took effect October 1, 2008. Compliance with requirements 6.5.8 and 6.5.9 of the Policy on Transfer Payments, as well as the call for proposals business process, were excluded from the audit's scope, as an internal audit of the calls for proposals process is scheduled for 2013–2014.

2.3 Approach and Methodology

The audit of grants and contributions in PWCB was conducted in accordance with Treasury Board policy, and directives and standards on internal audit, and conforms to the International Standards for the Professional Practice of Internal Auditing of the Institute of Internal Auditors. The evidence gathered was sufficient to provide senior management with proof of the conclusion derived from the internal audit.

The audit methodology included, but was not limited to:

• reviewing relevant Treasury Board and CIDA policies, guidelines, and directives;
• Identifying key internal controls and assessing their effectiveness;
• analyzing financial and non-financial information;
• file reviews;
• conducting interviews;
• gathering information and conducting analyses; and
• validating key findings

Projects funded through a grant were excluded from the scope of our file review because our testing was focused on requirements relating to cash management, reporting, audit and evaluation, and monitoring, and in most cases these requirements do not apply to grants.

2.4 Audit Criteria

The audit criteria are the benchmarks used to assess the adequacy of the measures in place to ensure that the use and management of grant and contribution funding within PWCB complies with Treasury Board's Policy on Transfer Payments and Directive on Transfer Payments, and with applicable CIDA policies. The criteria were developed after conducting a risk assessment, and are based on the requirements of the Policy on Transfer Payments and Directive on Transfer Payments, and CIDA policies. The audit was to determine:

1. Risk assessment is diligently performed and integrated into the:
   • design and redesign of PWCB programming (taking into consideration program risk, value of the funding, and recipient risk profiles); and
   • monitoring and oversight activities, which strike a balance between control and flexibility.
2. Program and project achievements are monitored to ensure that grants and contributions programming remains relevant in meeting objectives.
3. Roles, responsibilities and accountabilities for the management of transfer payment programs and projects are clearly defined and understood by the branch.
4. PWCB coordinates its efforts with other implicated branches in program design, delivery, monitoring, and reporting to ensure harmonized programming and streamlined administration.

3.0 Main Audit Findings and Recommendations

3.1 Risk Management and Oversight Framework

Risk Management Framework

CIDA's corporate suite of policies, guidelines and procedures to support the application and adoption of risk management provide a comprehensive framework for the Agency. They provide a consistent and systematic approach that helps programming branches, including PWCB, execute their responsibilities for financial stewardship and risk management. The recently launched FRET allows the Agency to uniformly assess the fiduciary risk associated with each individual recipient, while RBAM provides structured, consistent, risk-based guidance on the nature and frequency of monitoring activities to be conducted with regard to a given recipient.

PWCB Project Officers stated that the training, tools, and guidance relating to risk management, particularly FRET, are sufficient and have adequately prepared them to discharge their responsibilities for risk management. The Project Officers interviewed consistently understood FRET to be the primary tool for assessing the level of risk at the project and recipient level.

For projects launched prior to the introduction of RBAM, we found that there was rarely a discernible link between the level of risk associated with recipients and the administrative requirements imposed on recipients. We also observed there was rarely any documentation explaining why certain administrative requirements were or were not imposed. PWCB personnel asserted that until the most recent revision of CIDA's Terms and Conditions for International Development Assistance, the branch was required to provide quarterly payments to most funding recipients. Going forward, the issues identified above should be remedied when RBAM is adopted and implemented across all of CIDA's funded projects, as we have already seen a marked improvement resulting from the pilot phase of RBAM. During our audit testing, we observed where RBAM was used, and administrative requirements can be directly linked and are proportionate to the assessed level of risk associated with a recipient. However, we noted that PWCB has no measures in place to ensure that administrative requirements are enforced.

Recommendation 1

PWCB should minimize the risk that administrative requirements defined in the project agreement are not complied with.

Oversight Framework

Before any project is recommended to the Minister for funding, a Financial Management Advisor must review the project's proposed budget. Once the Minister has approved the project, the Financial Management Advisor is responsible for assessing and approving the proposed project budget against program/project eligibility criteria and guidelines, as well as against applicable CIDA policies and directives. Over the life of a project, recipients will then report to PWCB, at a frequency prescribed in the funding agreement, on expenditures incurred at the budgetary line item level.

Although both CIDA's Business Process Roadmap and Manager's Guide to Contribution Agreements provide a framework to guide PWCB personnel in managing and administering contribution agreements, this guidance is generally kept to a high level. It provides guiding principles as opposed to offering Project Officers detailed procedures and tools to help them execute their day-to-day monitoring responsibilities, such as processing advance payment requests and advance justifications, analyzing the performance and financial reports submitted by recipients, and planning and conducting site visits. Roles and responsibilities at this level are communicated primarily through on-the-job coaching and training, as well as through some formal training. This has resulted in inconsistent practices among Project Officers in conducting monitoring activities.

From our review of project files, we noted several issues that may have been mitigated by the existence of documented processes and procedures to guide PWCB's Project Officers in executing their day-to-day monitoring responsibilities. We found that Project Officers did not always follow up on or challenge recipients' reports even though recipients did not always adhere to the reporting requirements detailed in their respective funding agreements. Recipients did not always submit reports, nor did they always submit reports on time and in accordance with the content requirements detailed in the funding agreements. Furthermore, recipients were not providing detailed explanations of significant variances between planned and actual expenditures, despite an explicit requirement in their funding agreements to do so. There was often no evidence of follow-up by Project Officers who approved the release of scheduled payments to these recipients.

Recommendation 2

PWCB should strengthen the internal control process to minimize the risk that Project Officers will not execute their day-to-day monitoring responsibilities, such as processing advance payment requests and justifications, analyzing the performance and financial reports submitted by recipients, and planning and conducting missions.

3.2 Performance Measurement and Monitoring

CIDA's results-based management approach focuses on achieving outcomes, implementing performance measurement, learning and adapting, and reporting performance. Tools such as logic models, performance measurement frameworks, and risk registers help the Agency to look beyond activities and outputs, and to focus on expected results. At PWCB's programming level, a branch-wide logic model based on business lines (some of which are aligned with thematic priorities) has been developed. PWCB is currently attempting to develop thematic performance measurement frameworks in order to guide thematic projects. Currently, PWCB does not report against thematic performance measurement strategies; however, this is about to change as CIDA is working to define common indicators at the Agency level and to report against thematic performance measurement strategies.

At the project level, primary responsibility for measuring performance resides with PWCB's funding recipients. Funding recipients, under the terms and conditions of their respective funding agreements, are required to submit a logic model, performance measurement framework, and annual work plan or project implementation plan for their projects. Recipients are also responsible for collecting, validating, and using data to report to PWCB on project performance. PWCB's Project Officers will then monitor progress against achievement of outcomes and performance indicators through their review of the narrative progress reports submitted by recipients. We found that recipients are generally submitting logic models, performance measurement frameworks, and work plans in accordance with the terms and conditions of their respective funding agreements. We also observed that the performance information included in narrative progress reports can generally be linked to the expected results and achievements established by the project performance measurement framework and logic model. PWCB personnel noted, however, that the performance information the branch collects from recipients is not always useful, but that the branch is in the process of developing guidelines to assist recipients in preparing meaningful narrative reports. These guidelines have been shared across the Agency and are now being incorporated into the implementation of CIDA's Agency Programming Process (APP). Furthermore, the guidelines have been tested and rolled out to recipients in draft form in advance of the APP implementation in order to improve the quality of results reporting.

Although CIDA has developed a suite of tools and training that provides for a comprehensive framework for performance measurement and monitoring, this framework does not necessarily address the nature of PWCB's business and the branch's operational realities. Interviewees noted that the projects funded by PWCB often touch different countries and different thematic priorities. Since one project may be implemented in multiple countries and because in each country the programming (and thus the results) may differ (due to the generally responsive nature of PWCB's programming), PWCB has experienced challenges in aligning and reporting the results of its various projects to the branch's different sectors and with branch and Agency objectives, as well as in rolling up performance results for higher level reporting.

The Investment Monitoring and Reporting Tool (IMRT) was designed by the Agency to facilitate the reporting on results, risks, and crosscutting sectors. However, PWCB personnel have expressed concerns with the IMRT design, which has resulted in substantial manual input and tracking of project risks and performance information in disparate systems. These issues will be discussed in the OCAE's report on its internal audit of results-based management.

Recommendation 3

To assure effective performance measurement and monitoring, PWCB should finalize and communicate guidelines it is developing to assist recipients in preparing narrative reports.

3.3 Roles and Responsibilities

The Agency has a suite of corporate guidance, including the Business Process RoadMap, Policy on Fiduciary Risk, Results-Based Management Policy Statement, and Manager's Guide to Contribution Agreements, that documents roles, responsibilities, and accountabilities relating to the administration and management of grant and contribution funding. These high-level roles, responsibilities, and accountabilities are clearly defined and effectively communicated, as interviewees had a consistent understanding of the roles, responsibilities, and accountabilities for the administration and management of grant and contribution funding. However, as previously noted, this guidance does not include detailed procedures and tools to help Project Officers execute their day-to-day responsibilities for monitoring.

3.4 Internal Coordination

PWCB is generally responsible for maintaining CIDA's institutional relationships with Canadian-based partner organizations. PWCB consults with the Agency's other programming branches when it has to prepare briefings, backgrounders, Treasury Board submissions, and other informational documents on partner organizations, or when it has to update the organizational profile of partner organizations. However, multiple interviewees asserted that the grant and contribution programming of CIDA's programming branches (Geographic Programs Branch, Multilateral and Global Programs Branch, and PWCB) is generally very distinct. As such, it is a challenge for PWCB to coordinate the design and development of grant and contribution programming with other branches. This point was further reinforced by the fact that PWCB's programming is primarily responsive and thus is not “designed” in the traditional sense. Rather, it is the prospective partner organizations who design projects to align to CIDA's thematic priorities. It was noted that the responsive nature of PWCB's programming allows partners to add value by identifying needs that would not have been otherwise identified by the Agency.

Within the Agency, there are no defined roles and responsibilities for coordination, nor is there a defined process. The only defined process for coordination among CIDA's programming channels relates to PWCB's call for proposals business process in which there is a defined requirement that meritorious project proposals be provided to field staff for review and comment.

We found that individual directorates within PWCB coordinate their monitoring activities (field missions, for example) with one another. However, the extent of these directorates' formal coordination of monitoring activities with CIDA's other branches is generally limited to coordinating with Geographic Programs Branch, specifically with the field, for on-site field visits/missions.

FRET is currently the only formalized means through which information on recipients and prospective recipients is shared across branches (although use of the Web-based Project Browser is gradually being adopted across the Agency). However, FRET is not an all-encompassing tool for capturing intelligence on and experience gained in dealing with a particular recipient. We also observed that there are no formal mechanisms in place within the Agency through which to share lessons learned in designing and administering grant and contribution programming.

PWCB is not formally coordinating its efforts with other program branches in program design, delivery, monitoring, and reporting to ensure harmonized programming. However, the Agency is making efforts to streamline the administration of grant and contribution programming, and several new initiatives that would allow for information and knowledge sharing, such as CIDA-wide adoption of the Partners@Dévelopment tool, are currently underway as part of CIDA's BMI.

To maximize the knowledge and information sharing of the Agency's development efforts in a country, BMI recently introduced a Mandate Letter for a Head of Aid. Signed by the President, the letter clearly defines the roles and responsibilities for a Head of Aid, who is mandated to assume various roles and responsibilities as the Agency's senior representative of all CIDA programming channels for development cooperation in a mission. Among other things, these roles and responsibilities are in the areas of official and representational duties, policy and program management, bilateral programming, reporting, communication and consultation, programming support, and human resources management. The plan is to implement the Mandate Letter in all CIDA decentralized countries.

Recommendation 4

To support the implementation of the Mandate Letter, the Vice-President of PWCB should further develop guidance for monitoring its grants and contributions programs in coordination with other program branches.

Appendix A: List of Recommendations and Management Action Plan