Audit of Grants and Contributions - Oversight and Monitoring

Context

Transfer payments are one of the Government of Canada’s key instruments in meeting its broad policy objectives and priorities and are a major commitment of the federal government’s resources to advance Canadian interests.

The Treasury Board Policy on Transfer Payment and its Directive govern the grants and contributions programs. The policy and directive set out clear roles and responsibilities for Ministers, Deputy Heads and departmental managers in the design, delivery and management. Grants and contribution agreements that provide official development assistance are also subject to the Official Development Assistance Accountability Act. There are three different types of transfer payments (see text box).

In 2013, the former Department of Foreign Affairs and International Trade, and the Canadian International Development Agency were amalgamated. At that time, senior management in the newly created department, Global Affairs Canada, saw an opportunity to adopt a streamlined approach to international engagement by strengthening the alignment between foreign affairs, trade and development transfer payment objectives.

Grants and contributions represents 67 percent ($4.6B/$6.9B) of the total departmental expenditure. Of the total expenditures, $3.6 billion was used to deliver international development and humanitarian assistance, approximately $914 million was for foreign affairs programs, and $38 million was for trade programs. More details about this expenditure by business line, branch and transfer payment category are presented in Appendix B.

In 2017, the Office of the Chief Audit Executive conducted an audit examining the harmonization process between the three business lines for grants and contributions. Some of the findings from that audit continue to exist today such as those related to governance, risk management and monitoring. In particular, harmonization has been challenging because of the systemic differences between the program delivery frameworks. As well, coordination is complex as the grants and contribution programs are delivered across 12 branches.

Findings and Recommendations

This section sets out the key findings. It is divided into five areas: governance and oversight; investment planning; compliance with the Financial Administration Act; program and project monitoring; and, program reporting. Recommendations are provided, where appropriate. Details related to the audit approach are included in Appendix A.

1. Governance and Oversight

The government is committed to ensuring that transfer payments are managed in a manner that respects sound stewardship and the highest level of integrity, transparency, and accountability. An effective governance framework should be designed to support these overall objectives.

1.1 Corporate Governance

The department has a decentralized governance framework for grants and contributions. Two key committees – the Policy and Programs Committee and the Director General Program Committee provide guidance and direction amongst other areas of responsibility.

The Policy and Programs Committee is chaired by an Assistant Deputy Minister (ADM), and  provides a challenge function, leadership, strategic guidance, advice and decisions on programming issues (including programming integration and excellence), and the implementation of the delivery and results agenda for the department. The Committee may also escalate topics and make recommendations to the Deputy Minister’s Executive Committee.

The Director General Program Committee is a forum for information sharing as well as a venue to discuss issues related to programming, delivery and results across the department. Officials seek input from the Committee on program policies or initiatives under development.

While both committees typically recommend next steps and action items, there was no evidence of a formalized tracking process for records of decision. There was also no clear guidance regarding which projects and programs are expected to be presented at each committee. Stakeholders interviewed noted that project and program managers are responsible for self-identifying the projects and programs to be presented to the committees, if at all. As well, the Terms of Reference and governance procedures for both the committees do not outline a process for escalation of higher-risk topics from program level to the committees nor standardized grants and contributions performance reporting and risk reporting. Such reporting could provide additional context to senior management, and allow committees to proactively identify and mitigate risks.

1.2 Program Governance

At the program level, committees are in place to varying degrees to act as a challenge and guidance function for proposed projects. These committees are led by either a Director General, Director or Deputy Director, and serve respective programming areas, such as:

• The Geographic Branch Investment Review Committee (BIRC) for projects proposed within the geographic branches’ development programs;
• The Project Accountability Team for endorsing projects within the Peace and Stabilization Operations Program (PSOPs); and
• The CanExport Steering Committee, a trade governance and oversight committee.

Similar to the corporate-level committees, there was limited documentation on monitoring of committee recommendations and feedback at the program and project levels. Interviews with stakeholders highlighted that collaboration and communication among committees at this level is limited, particularly between business lines. While these committees may encourage horizontal information sharing, committee members interviewed noted a lack of a centralized system or repository to support relevant, timely oversight and issue resolution. This increases the risk that grants and contributions are being managed and delivered inconsistently across the department.

Overall, a governance and oversight structure exists however, the department would benefit from enhancing the governance to have a senior level corporate committee focused on grants and contributions especially given the horizontal transformation initiatives underway including the IT development of a new grants and contributions management system (see Recommendation 1).

1.3 Roles and Responsibilities

The key stakeholders within the department who support the delivery of grants and contributions are the Corporate Planning, Finance and Information Technology Branch, mainly through its Grants and Contribution Management Bureau, the International Assistance Operations Bureau, the Strategic Policy Branch and Program Branches.

Grants and Contribution Management Bureau

The Grants and Contribution Management Bureau is the functional authority responsible for the management control framework including governance, fiduciary risk management and financial internal controls including securing delegated transfer payment and financial signing authorities from Treasury Board Secretariat. The Bureau also provides strategic advice and guidance related to the Policy on Transfer Payment and houses the Office of Innovative Finance, which supports programs in the due diligence, assessment, structuring and negotiation of the financial terms of blended finance investments/repayable contributions.

International Assistance Operations Bureau

The Bureau was created in 2017 to provide centralized guidance and coordination for all Branches responsible for international assistance programs. The Bureau is leading the implementation of the Feminist International Assistance Policy (FIAP) and reports directly to the Deputy Minister for International Development (DME). It provides guidance, advice, tools and training to program branches on how to operationalize international assistance including streamlining, simplifying and harmonizing programming approaches.

Strategic Policy Branch

This Branch designs, delivers and coordinates strategic policy advice on current and emerging foreign policy, economic diplomacy and international assistance matters, in addition to other responsibilities. The branch delivers grants and contributions in relation to research and knowledge, and for Canadian contributions to certain international organizations.

Program Branches

Twelve program branches administer grants and contributions. Program branches are accountable for all projects funded with transfer payment funds. Each branch is thematically aligned with certain key departmental and Government of Canada priorities. Program branches oversee grants and contribution planning, management and administration at the operational level, and interact directly with in-field project officers as a first point of contact.

Consistent with the findings of the 2017 Audit of the Harmonization of the Grants and Contributions Program Administration, differences in the delivery frameworks and approaches continue to exist among the three business lines. As a result, each business line has a varying degree of understanding of the different roles and responsibilities between the centralized functions and the project and/or program levels. For staff that work on International Assistance transfer payments a placemat has been developed which clearly defines the roles and responsibilities for Project Officers, Financial Analysts and Contract Officers. A similar tool would benefit those staff that work on foreign affairs and trade transfer payments.

While the department is in the process of finalizing and implementing the harmonization vision that was identified in 2017, it has been three years and there is a risk that inconsistencies and inefficiencies continue. There is a need to enhance the operating effectiveness of the governance at the corporate level to bring focussed attention to the transfer payment program and continue to work to have more consistent roles and responsibilities across business lines.

Recommendation 1

The Assistant Deputy Minister of Corporate Planning, Finance and Information Technology and the Director General of the International Assistance Operations Bureau, in partnership with the relevant Assistant Deputy Ministers, should strengthen the operating effectiveness of the governance framework, including roles, responsibilities and accountabilities.

2. Investment Planning

The common objective of the program investment planning process across the three business lines is to establish and communicate, through approved investment allocations, key priorities and programs for the upcoming year.

The Corporate Planning, Finance and Information Technology Branch and the International Assistance Operations Bureau lead the annual investment planning process for the development portfolio. For the last three fiscal years, approval was provided through a set of program summary sheets at the beginning of each fiscal year, including factors such as the available reference level budget, alignment with Feminist International Assistance Policy, key commitments and targets, notional country and program budgets, and country/program strategies.

The foreign affairs and trade processes are distinct from program to program, and investments are mostly approved and managed through Treasury Board submissions.  While these two business lines have an investment planning process that allocates program funding, documentation of the annual processes could be strengthened.

Stakeholders in all three business lines noted pressure stemming from the timing of annual investment processes, given the significant outflow of funding that occurs in a short period. As summarized in the figures below, over 45 percent of all transactions from 2017-18 to 2018-19 occurred in the fourth quarter of the fiscal year (January – March). The increased administrative burden resulting from this may impact a Project Officer’s ability to effectively perform Section 34 of the Financial Administration Act (certification of work performed) and monitoring, particularly given the extent of review and due diligence required to ensure that projects have met objectives and financial targets. Stakeholders interviewed noted that the significant outflow of funds in the fourth quarter might be attributed to various structural issues (e.g. delayed expenditure and investment plan approvals) or that some recipients operate on a calendar year.

Overall, investment planning processes are in place for all three business lines. The risk related to in-year and year-end pressures will be the subject of a future audit related to resource allocation.

3. Compliance with the Financial Administration Act

The audit team performed walkthroughs, project file reviews and transaction testing to assess the completion of both Section 32 (commitment of funds) and Section 34 (certification of work performed) of the Financial Administration Act (FAA). The audit team sampled 64 transactions across all three business lines based on funding type (grants, assessed contributions and non-assessed contributions).

Walkthroughs with Project Officers indicated an inconsistent understanding of when a Section 32 approval should take place in the process. It was also noted that the Section 34 due diligence supporting the process varied depending on the project.

From the transaction testing, while 67 percent of the files were complete, one or more of the following issues were identified in 33 percent of the files sampled:

• Section 32 and 34 signed and/or dated approval not on file;
• Section 34 evidence of delegation of authority not on file; and
• Section 32 requirements not identified by project team, and/or approvals not provided.

Oversight and consistent application of Section 32 and 34 financial controls exist to support budgeting, planning, accurate commitments and payments. These controls also support the Program Officer due diligence function to pay for only eligible program activities that have been performed.

It was noted during interviews and debriefings on the audit findings, that Program Officers might require additional or routine training to strengthen their capacity to properly perform Section 32 and Section 34 approvals, as well as to assess progress and to determine value for money components.

Recommendation 2

The Assistant Deputy Minister of Corporate Planning, Finance and Information Technology in partnership with the Assistant Deputy Ministers and the Director General of the Canadian Foreign Service Institute should develop and implement a strategy to improve compliance related to Section 32 and Section 34 of the Financial Administration Act.

4. Program and Project Risk Management

Monitoring requirements are largely driven by project risk profiles, determined at the project planning stage through the risk assessment tools and tolerance levels. While there were risk management processes in place for projects in each business line, there are currently no formal departmental-wide risk appetite statement(s) or risk tools or guidance to support a cohesive and consistent risk management approach for transfer payments. As a result, the risk appetite at the program level is informally established. The department would benefit from having a consistent aggregation of risk information to provide insight across the branches as to high-risk programs and risk trends.

Within the Development stream, the Fiduciary Risk Evaluation Tool (FRET) is completed at the project initiation stage for projects valued at $251K or higher. For projects under this amount, the Project Officer and Program lead are responsible for exercising appropriate due diligence. The risk tool, which requires Project Officers to assess key project and partner-related risk factors, produces an overall risk rating that drives administrative and monitoring requirements. Although the risk assessments are expected to be updated at least annually, Program officials stated that this is not a hard requirement and updates are not tracked.

The Foreign Affairs stream also has risk assessment tools that are used at the project initiation stage and define the frequency and design of monitoring activities. However, the approach taken depends on the bureau and program. For example, Peace and Stabilization Operations Program has designed its own risk management framework, risk management guide and risk tolerance strategy, which identifies administrative requirements depending on the overall risk level. Project Officers use the Security and Stabilization Project Management Tool (SSPMT) to evaluate the risks and capture other assessments, in addition to the results of the fiduciary risk assessment (specifically, the financial viability risk factor).

The Trade stream uses a trade-developed assessment template completed at the outset of a project. Their risk assessment tools do not specifically address financial viability or fiduciary risk as a risk assessment factor. These areas are reviewed through past performance and recipient capacity. Monitoring, performance and reporting requirements are based, to varying degrees, on the upfront project risk level identified.

The table below summarizes the file testing results of completed risk assessments for a sample of non-assessed contribution projects (due to their inherently higher risk), including whether resulting monitoring and reporting requirements were sufficiently and appropriately documented.

Overall, the approach to project risk management and subsequent monitoring is inconsistent across business lines and is not driven by a departmental-level risk management approach. Project risk management is largely focussed on upfront due diligence, as opposed to risk management throughout the project lifecycle. As such, changes in project risk during the delivery of a project may not be identified and appropriately managed and monitored in a timely manner to support project success.  There is an opportunity to amalgamate best practices from the three business lines to identify a departmental approach (see Recommendation 3). 

5. Program and Project Monitoring

Clearly documented and well supported monitoring requirements set expectations and capture the approach by which a project will be implemented in alignment with its risk profile. Multiple factors are considered when assessing the appropriateness of risk-based monitoring, including the type of transfer payment. Expectations will differ depending on the project and type of transfer payment, with non-assessed contributions typically receiving more scrutiny than assessed contributions and grants.

Other factors include whether program monitoring and reporting is centralized (Headquarters) or decentralized (locally or at Missions) and the level of monitoring is determined based on the diligence and experience of the Project Officer. As such, there is a risk that monitoring may not be commensurate with the project’s risk profile, nor appropriately modified should the risk level change.

During the risk assessment process, monitoring requirements were not consistently identified and documented. From the sample of projects tested several instances were noted where development project agreements required site visits as part of project monitoring however, the Project Officers contacted were unable to provide documentation of these site visits to demonstrate that monitoring had occurred. Overall, limited documentation was available to demonstrate a Project Officer’s review of recipient narrative and/or financial reports. In addition, stakeholders raised concerns that programming staff at headquarters and in the field may have limited capacity or lack the financial knowledge to perform monitoring activities, such as assessing project progress and value for money.  

It should be noted, however, that a number of activities are in place at the department level to support project and program monitoring, after payments to recipients have been made and during project implementation. The Financial Compliance Unit coordinates financial monitoring and auditing of recipients on the department’s behalf, including financial capacity-building activities (before project implementation) and compliance/recipient audits of financial instruments (post-implementation).

In 2019, the Corporate Planning, Finance and IT branch also piloted a 4-year International Assistance Field Monitoring Mission plan. The pilot is intended to reinforce management oversight and to strengthen the internal control framework through mission visits. It was expected that these visits would eventually build a performance baseline and set expectations on different types of international assistance programming.

In summary, while monitoring is supported by some department-led initiatives, the department would benefit from having a standardized process for project-level monitoring activities including how it should be documented and maintained. Considering that the approach to project risk management is also inconsistent across business lines, and is largely focused on upfront due diligence, there are risks that project-related information is not up to date and available throughout project lifecycle. This may affect decision-making and management responsiveness, as well as whether monitoring activities appropriately reflect risks. While risk management and monitoring should be flexible to the specific needs of each business line, establishing clear requirements would assist the department to meet its objectives.

Recommendation 3

The Assistant Deputy Minister of Corporate Planning, Finance and Information Technology and the Director General of the International Assistance Operations Bureau in partnership with the Assistant Deputy Ministers should develop a departmental risk assessment framework for grants and contributions, which should also set the expectations for risk-based monitoring.

6. Training

Monitoring and project implementation relies on a Project Officer’s due diligence and their level of experience. As mentioned, Project Officers may require more financial knowledge to appropriately perform Section 34 approvals and monitor implementation activities (such as assessing project progress and the value for money component of programs). Currently there is departmental-level training specific to only the development stream. Some programs within foreign affairs and trade have developed their own training, however this is not consistent and Project Officers indicated that their training needs should be more fully considered. Moreover, Program officials may not have the appropriate expertise to develop and lead training.

As well, it was noted that there was limited data collection, aggregation and analysis, of training information as attendance and course evaluation results are not tracked. Monitoring training data could increase the department’s opportunities to improve the relevancy of its courses and would allow the department to be able to confirm staff participation in an effort to strengthen Project Officers ability to discharge their responsibilities.

The grants and contribution function would benefit from having a comprehensive, systematic training plan or roadmap that is tracked to support developing grants and contribution expertise (see Recommendation 2).

7. Program and Project Reporting

Project reporting is performed by recipients and reviewed by Project Officers to assess progress. It is also used to inform the program reporting. Project reports are aggregated against key indicators to capture an overview across programs and business lines. The availability and aggregation of information reported by projects provides insight into the projects and supports decision-making when making payments, rolling up progress and for performance information purposes. As such, project-reporting requirements should be clearly defined in signed agreements, and program-reporting requirements should be in place for each branch and designed to align with departmental needs.

7.1 Project Reporting

The key business processes and Standard Operating Procedures of each business line note that reporting requirements must be considered when drafting project agreements. Project reporting requirements are generally defined in signed agreements and in alignment with respective program mandates, Standard Operating Procedures and Terms and Conditions. However, the procedures regarding how project information should be structured and where documentation should be maintained is not clear and is inconsistently managed. In many cases, only Project Officers who worked on a file were familiar with the project’s file structure. Moreover, Project Officers and Program Managers interviewed noted challenges when working with Project Officers in the field who do not have access to corporate information management systems, such as the Enterprise Document and Records Management System (EDRMS). In 20 percent of the projects sampled, project documentation (such as project planning approval documentation and recipient reports) could not be located by Project Officers. Limited availability of information to support project reporting impacts insights into project performance and results.

Recommendation 4

The Assistant Deputy Ministers, in partnership with the Assistant Deputy Minister of Corporate Planning, Finance and Information Technology and the Director General of the International Assistance Operations Bureau should develop a project file documentation strategy to support project file management and information retrieval.

7.2 Program Reporting

Stakeholders across all three business lines noted that the department would benefit from having program-level reporting against standardized key performance indicators. Branches use the Performance Information Profiles to report on the performance of their respective programs; however, there is limited clarity regarding how project information is consolidated into the corresponding program file and further rolled up for departmental reporting.

In addition, program reporting varies depending on the business line.

• The Annual Reporting exercise for International Assistance involves the completion of a Management Summary Report for approval. The report includes the Project Officer’s assessment of performance and it rolls up to branch management for review in some cases. However, officials interviewed noted that it is not clear whether reports are communicated upward within the branch or bureau and how issues are addressed and escalated.
• Foreign affairs program-level reporting requirements depend on the bureau and program. For example, Peace and Stabilization Operations Program completes Management Summary Reports, while the Weapons Threat Reduction Program does not have a similar reporting mechanism; this program only reports via the annual Departmental Results Report.
• Trade program-level reporting for the CanExport program delivers an annual internal report to their Assistant Deputy Minister, describing the profile and performance of all project pillars within the program. This report is drafted by the Coordination and Administration Unit and provides an in-depth view of program operations, activities and costs, risk analysis and monitoring, and overall performance.

Overall, while project delivery reporting requirements are aligned with program mandates, Standard Operating Procedures and Terms and Conditions, supporting documentation should be better maintained in the various systems used. This would support reporting activities at the project level to be more consistent across and within business lines (see Recommendation 4).  

As well, the grants and contribution function would benefit from defining expectations for aggregate, program-level reporting either from programs to corporate bureaus, or from corporate bureaus upward, which would support the department-wide tracking of grants and contribution performance. Tracking departmental-wide performance could also be enhanced if reporting at the program level better consolidated project information as well as funding information to show how funds are being used to meet departmental objectives.

Recommendation 5

The Assistant Deputy Ministers, in partnership with the Director General of the International Assistance Operations Bureau should develop program and project level reporting requirements against standardized key performance indicators.

Conclusion

The department has a grants and contributions framework for oversight and monitoring, however the framework is made up of three different program delivery frameworks each with a different sets of processes and practices. There is an opportunity to optimize this transfer payment function to provide better line of sight on the entire portfolio. Opportunities exist in the areas of horizontal governance, risk management practices, financial controls, monitoring and performance measurement. 

Appendix A: About the Audit

Objective

The objective of the audit was to assess whether appropriate grants and contributions oversight and program monitoring are in place and operating effectively to support the achievement of departmental objectives. Specifically, the audit focused on the following key areas:

• Governance and oversight;
• Investment planning;
• Compliance with the Financial Administration Act;
• Program and project monitoring; and
• Program reporting.

Scope

The audit examined Grants and Contributions management and operational practices and controls at headquarters and at the program/project level, including both centralized and decentralized programs. The audit assessed the department’s Gs&Cs for FY 2017-2018 and FY 2018-2019 up to and including February 2020.

While the audit included a review of key Gs&Cs planning and administration tools in place, these tools were reviewed for the purpose of assessing compliance for each project sampled during the course of the audit rather than the effectiveness of the tools themselves.

While Vote 10 of the Appropriations Act gives the department authority to spend public funds on Gs&Cs, including contracts issued through International Assistance Requests for Proposal and other financial instruments (e.g. administrative arrangements and letters of agreement), Vote 10 Contracts and other financial instruments were not included in the scope of this audit given their low value. Repayable contributions, which are conditional or unconditional contributions that are fully or partially repayable, was also excluded from the scope of this audit given that an Audit of Repayable Contributions was completed in September 2018.

Criteria

The criteria were developed following the completion of the detailed risk assessment and considered the audit criteria related to the Management Accountability Framework developed by the Office of Comptroller General of the Treasury Board Secretariat. The audit criteria were discussed and agreed upon with the clients. The detailed criteria are presented as follows.

Audit criteria are reasonable and attainable expectations against which compliance, the adequacy of controls and overall performance are assessed. These audit criteria are based on acts and regulations, policy, guidelines, generally recognized industry norms, results of previous audits or other criteria developed in consultation with Program management. The following criteria were assessed during this audit and form the basis for developing audit observations and recommendations.

* Rating definition (from The Institute of Internal Auditors – IIA):

• Met – There is sufficient evidence to support audit expectations defined in the criteria. For example, appropriate controls are found (and are commensurate with the risk) and they are operating effectively.
• Partially met – There is some evidence to support the audit expectations defined in the criteria but improvements are required. For example, some controls are in place and some are operating effectively – recommendations are made.
• Not met – There is insufficient evidence to support audit expectations defined in the criteria. For example, controls are not in place or are not operating effectively (risk exposure is high) – recommendations are made, and other action are taken as required.

Approach and Methodology

The audit was conducted in conformity with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing and with the Treasury Board Policy and Directive on Internal Audit. These standards require that the audit be planned and performed in such a way as to obtain reasonable assurance that the audit objective is achieved.

The audit methodology included, but was not limited to, the following:

• Identification, review and analysis of related documentation (e.g. applicable policies, directives and regulations);
• Reviewing and analyzing Gs&Cs financial and non-financial information;
• Identification and analysis of funding authorities;
• Process walkthroughs and reviews;
• Interviews with relevant stakeholders and process owners as applicable;
• Assessment of related risks and internal controls; and

Other tests and document review as deemed necessary

Appendix B: Expenditure Details

The following table summarizes the transfer payment expenditures by business line from 2017 - 2019:

Source: CFO-Stats, June 26, 2019

The following table summarizes transfer payment expenditures by Branch from 2017 - 2019:

Source: CFO-Stats, June 26, 2019

The following table summarizes the breakdown between transfer payment categories for Gs&Cs expenditures from 2017 – 2019:

Source: CFO-Stats, June 26, 2019

Appendix C: Recommendations and Management Response and Action Plan