Audit of Cabinet and Treasury Board Submission Costing Practices

Audit report
Office of the Chief Audit Executive
June 2022

Table of Contents

• Acronyms and Symbols
• Executive Summary
• Background
• Findings and Recommendations
  • 1. Oversight
    • 1.1 Roles and Responsibilities
    • 1.2 Training
  • 2. Process and Controls of Costing Exercises
    • 2.1 Costing Tools and Guidance
    • 2.2 Retention of Key Documentation
    • 2.3 Lessons Learned
• Conclusion
• Appendix A: About the Audit
• Appendix B: Summary of Findings
• Appendix C: Recommendations and Management Response and Action Plan

Acronyms and Symbols

CFO

Chief Financial Officer

FMA

Financial Management Advisor

GAC

Global Affairs Canada

MC

Memorandum to Cabinet

TB

Treasury Board

Executive Summary

In accordance with Global Affairs Canada’s (the department) approved 2021-2022 Risk-based Audit Plan, the Office of the Chief Audit Executive conducted an audit of Cabinet and Treasury Board Submission Costing Practices.

Background

All departments are required to complete a detailed cost estimate as part of the Memorandum to Cabinet and Treasury Board submission process. Expectations related to costing exercises are documented in Treasury Board Secretariat’s Guide to Cost Estimating (Guide), which describes a costing estimate as a prediction about the future cost of an investment based on the data and information available at a given point in time. Departments and agencies use this guidance to provide accurate, relevant and timely costing information to support decision-making. The Chief Financial Officer is required to exercise due diligence before formally attesting to the accuracy and reasonability of the cost estimate. Therefore, a sound costing process to produce reliable cost estimates is important to avoid internal financial pressures or lapses.

Objective and Scope

The objective of the audit was to determine whether departmental controls are in place to support reliable costing information as part of Cabinet and Treasury Board submission process.

A sample of submissions were selected for the period from April 1, 2019 to March 31, 2021 to review the costing practices in place at the time of submissions. For additional details regarding the audit approach, see Appendix A.

Areas of enquiry

For a summary of findings, see Appendix B.

Conclusion

Departmental controls are in place to support reliable costing information as part of Cabinet and Treasury Board submissions; however, there are opportunities for improvement related to roles and responsibilities, training, documentation retention, and lessons learned practices.

Recommendations

1. The Assistant Deputy Minister of the Corporate Planning, Finance and Information Technology should formally document and communicate the roles and responsibilities of various stakeholders (Costing Unit, Programs, and FMAs) involved in the costing process.
2. The Assistant Deputy Minister of the Corporate Planning, Finance and Information Technology should implement mandatory costing training for FMAs, which includes an annual refresher course.
3. The Assistant Deputy Minister of the Corporate Planning, Finance and Information Technology should provide guidance on the retention of key documentation to support the costing process. These guidelines should include the minimum documentation to retain, period of retention, a central repository location, and the identification of the unit responsible to capture and maintain documentation.
4. The Assistant Deputy Minister of the Corporate Planning, Finance and Information Technology should formally document and carry out lessons learned activity on the costing process.

Statement of Conformance

The audit was conducted in conformance with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing and with the Treasury Board Policy and Directive on Internal Audit, as supported by the results of the quality assurance and improvement program.

Background

All departments are required to complete a detailed cost estimate as part of the Memorandum to Cabinet (MC) and Treasury Board (TB) submission process to allow ministers to make informed decisions. Expectations related to costing exercises are documented in Treasury Board Secretariat’s Guide to Cost Estimating (Guide), which describes a costing estimate as a prediction about the future cost of an investment based on the data and information available at a given point in time. The Guide helps users to improve the understanding of uncertainties, risks, and sensitivities associated with the estimate that could impact the cost, including validation of data and evidence used for costing. Credible cost estimates help to persuade decision-makers that the initiative can be successfully implemented.

Departmental cost estimates need to be supported by clear costing assumptions that are translated into cost estimates through sound costing practices and methodology. A strong process supports the Chief Financial Officer’s (CFO) due diligence and assurance over the reasonability and reliability of departmental financial information, which is ultimately covered by the CFO attestation letter that accompanies the MC/TB submission.

As elaborated in Treasury Board Secretariat’s Guideline on Chief Financial Officer Attestation for Cabinet Submission, the following six fundamental assertions have been identified to characterize elements of a CFO’s attestation to support decision-making.

Six fundamental assertions

Roles in the Costing Process

The costing exercise within the MC/TB Submission process in the department is a shared responsibility among the following key participants. The roles and responsibilities of each player are described below.

Findings and Recommendations

This section sets out the key findings, divided into two key areas of the costing exercise: Oversight, and Process and Controls. Findings related to tools and training are included in each section as applicable.

1. Oversight

The Treasury Board’s Policy on Financial Management requires effective oversight of the financial management activity so that reliable financial information is produced and used to support decision-making. As indicated in the policy, a critical oversight component is the establishment of mechanisms so that individuals responsible for financial management are aware of their roles and responsibilities and receive the necessary training to carry them out. In addition, the CFO is expected to perform a challenge function on financial information provided in key departmental documents to provide reasonable assurance of its reliability. For MC/TB submissions, this due diligence function is demonstrated by way of a CFO Attestation.

1.1 Roles and Responsibilities

Overall, the governance and oversight structure in place for the costing process is in line with policy expectations. In particular, the inclusion of the Costing Unit in the submission costing process has enhanced oversight. Since 2018-19, the Costing Unit’s role has continuously evolved from a focus on maintaining costing tools and processes to actively exercising a review function. This evolution was apparent during the audit team’s testing of a sample of files from 2019 to 2021. Recently, the Costing Unit has provided more feedback to Financial Management Advisors (FMA) and Programs to support a consistent approach to costing analysis.

As well, a challenge function is performed with the assistance of the TB Submission and Attestation Team and the Costing Unit. While a Costing Framework has been developed that articulates each stakeholder’s role and responsibilities, FMAs and Programs do not always have a clear understanding of the role of the Costing Unit and their level of involvement in costing activities. Based on interviews and an examination of sample files, the extent of participation and challenge function performed by the Programs and FMAs varied greatly. The relatively new and evolving role of the Costing Unit was noted as a contributing factor, as well as the high turnover of FMAs in three of six files sampled.

The circulation of a costing process flow map, accompanied by the defined roles and responsibilities, to all stakeholders would add clarity and efficiency to the costing process.

Recommendation 1

The Assistant Deputy Minister of the Corporate Planning, Finance and Information Technology should formally document and communicate the roles and responsibilities of various stakeholders (Costing Unit, Programs, and FMAs) involved in the costing process.

1.2 Training

The TB Policy on Financial Management expects training, guidance, and support to be provided to those involved in costing activities to effectively carry out their roles and responsibilities.

The Costing Unit offers on-demand training to FMAs and Programs focusing on the costing process and updates to costing tools. For each significant update to the costing template, the Costing Unit offers a facilitation session with the FMAs to inform of changes to the template or process and at the same time to obtain feedback on whether their needs are being met. FMAs can also contact the Costing Unit for specific queries anytime during the costing process. All of these support mechanisms are optional for FMAs. However, FMAs indicated that they seldom accessed these resources due to the tight turnaround time during the costing process.

According to interviews with FMAs, much of the required costing expertise is acquired through on the job experience, rather than through the available training. Normally, FMA teams comprise both junior and experienced FMAs who are involved in the preparation of the costing estimates, discussions with the Program, and challenging/reviewing of a submission’s costing. Those FMAs, who did not have sufficient background knowledge of their client’s program or experience in costing, indicated that they could not perform an effective risk-based challenge function of the assumptions underlying the costing estimates. The level of involvement by experienced FMAs is based on availability, competing priorities, and the experience of junior FMAs. It was noted that in five out of six files reviewed, evidence of review by junior FMAs and experienced FMAs was not retained (e.g., no senior FMA sign off on the costing). In addition, as some FMAs may not regularly participate in costing estimates, the latest methodology and tools may not be used in the costing exercise.

Making departmental costing training mandatory and reoccurring for FMAs would help to strengthen their costing expertise, allow them to better challenge assumptions, and improve the costing estimate’s alignment to program needs. This training would also help to onboard junior FMAs, improve their understanding of their roles and responsibilities in the costing process, and enhance their program knowledge and technical skills.

Recommendation 2

The Assistant Deputy Minister of the Corporate Planning, Finance and Information Technology should implement mandatory costing training for FMAs, which includes an annual refresher course.

2. Process and Controls of Costing Exercises

As stated in Treasury Board Secretariat’s Guideline on Chief Financial Officer Attestation for Cabinet Submission, the CFO is required to conduct a robust due diligence review to help ensure the reliability of a proposal's financial information used for decision-making. A costing exercise supported by clearly documented assumptions and based on appropriate costing methodologies, is the basis for assertion 2 that significant risks having a bearing on the financial requirements, the sensitivity of the financial requirements to changes in key assumptions, and the related risk-mitigation strategies have been disclosed. In addition, assertion 6 indicates that key financial controls are in place to support the implementation and ongoing operation of the proposal.

A general overview of the department’s costing process for submissions is below.

Submission Costing Process

2.1 Costing Tools and Guidance

It was expected that the department would have in place sufficient guidance and tools to support the effective drafting of MC/TB submissions and their associated costing in compliance with TB costing-related financial policies. It was found that the department provides costing guidance through intranet sites that describe the submission process and the key steps for developing the costing component. In addition, stakeholders are referred to various resources, such as Treasury Board Secretariat’s Guide to Costing Estimating and Guidance for Drafters of Treasury Board Submissions. A link is also provided to the departmental Costing Template, which is described below.

Appropriate costing methodologies require reliable data sources, reasonable costing assumptions, and the application of lessons learned. The Costing Unit developed a macro-based excel costing tool to develop cost estimates of submissions. Instructions are provided within the template to help users (Programs, FMAs, and Costing Unit) compute comprehensive cost estimates using consistent methodology. The department and other government departments have set standards for salary, bilingual bonuses, employee benefit plan costs, and internal services costs. These cost elements are automatically extracted into the costing workbook through salary tables and macros, which pre-populate the related tabs based on assumptions about the number of employees required for the project, their job category, and their level. The Costing Unit reviews these costs internally with key stakeholders for completeness and accuracy based on the most up-to-date information.

The available departmental guidance and tools described above contribute to the consistency and reliability of submission cost estimates.

2.2 Retention of Key Documentation

Under Treasury Board Secretariat’s Guideline on Chief Financial Officer Attestation for Cabinet Submissions, a management record must be appropriately maintained to document the due diligence review supporting the CFO’s analysis, conclusions, and attestation letter. The documentation should be sufficiently detailed to enable a third party to understand the extent of the due diligence and grasp how the CFO’s conclusions and observations in the attestation letter are supported. The level of documentation required is also defined in Treasury Board Secretariat’s Guide to Cost Estimating as part of the creation and adjustment of the costing estimate.

There is currently no formal departmental process or guidance for maintaining and retaining key costing-related documentation to substantiate the depth and breadth of work and due diligence review performed to support the CFO attestation. Some documentation was retained to support costing estimates, but was inconsistent among the submissions sampled. Documentation that was available was often insufficient to fully justify key assumptions and decisions supporting the costing exercise, and it lacked the rigour of version control.

Aside from the costing template, which was retained centrally, documentation to support key decisions were dispersed among stakeholder groups. Consequently, for file testing purposes, it was difficult to obtain documentation to demonstrate review and challenge performed by stakeholders. In addition, limited or no documentation was available in certain sample files where the original FMA was no longer in that position.

While a revised costing template was released in August 2021 with improved requirements to provide greater detail for certain costing elements, additional requirements are needed to provide documentation to substantiate the assumptions, review, challenge, and approvals.

Retaining documentation in a centralized location to support significant costing assumptions, key decisions, and various levels of approvals, is important to demonstrate due diligence throughout the costing exercise. This will provide greater assurance to the CFO when carrying out the attestation function.

Recommendation 3

The Assistant Deputy Minister of the Corporate Planning, Finance and Information Technology should provide guidance on the retention of key documentation to support the costing process. These guidelines should include the minimum documentation to retain, period of retention, a central repository location, and the identification of the unit responsible to capture and maintain documentation.

2.3 Lessons Learned

A formalized and consistent lessons learned process allows for continuous sharing and improvement of processes and methodology. Currently, informal feedback is mainly gathered by the Costing Unit through regular stakeholder consultations and frequent queries received throughout costing exercises. These comments and the nature of queries are taken into consideration when updating guidance and tools.

Where possible, assessing the accuracy of a costing estimate after the submission is tabled would be a valuable exercise in identifying potential process gaps or highlighting a good practice. This type of variance assessment is performed to a varying degree within the department. For example, capital project cost estimates are reviewed against expenditures as a part of standard project management. Similarly, if a new organizational unit is created as a result of funds received from a TB submission, then that unit’s actual expenditures can be compared to the initial cost estimate. In other cases, a TB submission may simply increase a program’s existing overall operating budget, in which case a variance analysis against one specific funding decision may not be possible or applicable. The Costing Unit indicated that conducting a variance analysis between initial cost estimates in a submission and actual expenditures is not usually feasible for a number of reasons, including; expenditures spanning multiple years and/or various divisions; approved funding that is less than the cost estimate; and budgets linked to annual reference levels rather than individual submissions.

While the Costing Unit undertakes some lessons learned activities, as described above, there is no reference or description of a lessons learned activity in the department’s submission process map and costing framework. Documenting lessons learned after each costing exercise would help to improve the process, tools, guidance, and cost estimates especially to support continuity of knowledge and processes when turnover of program staff and FMAs are high.

Recommendation 4

The Assistant Deputy Minister of the Corporate Planning, Finance and Information Technology should formally document and carry out lessons learned activity on the costing process.

Conclusion

Overall, departmental controls are in place to support reliable costing information as part of Cabinet and Treasury Board submissions process, and support the CFO attestation. The costing exercise has improved in recent years due to the creation of the Costing Unit within the Financial Resource Planning division. Various elements have been implemented to support the costing exercise for stakeholder groups, such as oversight mechanisms and tools and guidance. There are opportunities to improve the effectiveness of certain controls and practices in the areas of roles and responsibilities, training, retention of documentation, and lessons learned practices.

Appendix A: About the Audit

Objective

The objective of the audit was to determine whether departmental controls are in place to support reliable costing information as part of Cabinet and Treasury Board submissions.

Scope

The scope of the audit included current practices in place on the costing reflected in MC/TB submissions. For testing purposes, the scope period covered April 1, 2019, to March 31, 2021, although any developments during the audit, including the update to the costing tools, were taken into account.

This audit did not include costing exercises or systems outside of the MC/TB submission process. Further, the audit did not conclude on the accuracy of the costing estimates in relation to actual spending, but assessed reliability of the cost estimation practices.

Criteria

Criteria were developed following the completion of the detailed risk assessment. This audit focused on key areas of enquiry pertaining to the costing process, namely:

• Criterion 1 – Oversight:The department has effective oversight structures and mechanisms in place to support and manage the costing process for TB/MC submissions.
  • An effective oversight structure is in place to support costing activities for TB/Cabinet submissions and to ensure compliance with Treasury Board Secretariat’s Guide on Costing Estimates.
  • Roles, responsibilities, and accountabilities within the costing process are clearly defined, documented, and understood.
  • Stakeholders involved in the cost estimation process consistently participate to ensure costing information is complete and accurate.
• Criterion 2 – Processes and Controls:The department has established processes to guide costing activities related to assumptions and cost estimates.
  • Assumptions supporting costing exercises are formalized aligned with program needs and are established uniformly across programs.
  • Internal services costs are allocated appropriately to the project.
  • There is consistent evidence of review and approval of completion of each phase of the cost estimation process by those responsible.
  • A risk-based challenge approach is in place to ensure the costing reflects embedded assumptions, which are reasonable, supported by reliable data sources, and documented.
• Criterion 3 – Tools and Training:The department has tools and training in place to support staff in performing costing activities effectively.
  • Costing tools are available, meet the needs of users and enable them to effectively perform costing activities.
  • Training, guidance, and support are provided for costing activities to ensure that those involved are able to effectively perform their roles and responsibilities.
  • A process is undertaken regularly to review and continuously enhance the tools to improve the adequacy and reliability of costing activities.

Approach and Methodology

The audit methodology included, but was not limited to, the following:

• Policy/framework analysis;
• Data analysis to ensure consistency between samples;
• Process review and mapping;
• Review and testing of selected MC/TB Submissions; and
• Interviews with Departmental officials

The audit team reviewed and reconciled both the TB Submission Team and Costing Unit lists of completed submissions in both the 2019-20 and 2020-21 fiscal years. Based on sampling methodology, six TB submissions were selected out of 23 completed submissions. The TB/MC submissions were selected to include a representative sample of varying costing complexity, such as one-liner costing, the involvement of multiple branches, and that led by the Costing Unit.

Appendix B: Summary of Findings

Appendix C: Recommendations and Management Response and Action Plan