Audit of the International Science & Technology Partnerships Program

December 2013

Executive Summary

The Treasury Board approval of the International Science and Technology Partnership Program (ISTPP) included requirements for both an internal audit and a recipient audit to be carried out in 2012-2013. As a result of growing concerns regarding the management of this program, the Deputy Minister (DMT) and the Chief Financial Officer proactively requested that this audit work be accelerated. In accordance with this requirement, DFAIT requested that Deloitte conduct audits of the ISTPP covering both the internal audit and the recipient audit requirements. The purpose of this report is to present the internal audit observations and recommendations. The results of the recipient audit were issued through a separate report to DFAIT management.

Why is this important?

The International Science and Technology Partnerships Program (ISTPP) was renewed by the Government of Canada in Budget 2010. The five-year, $18.5 million program is expected to promote international collaborative research and development and to increase the international competitiveness and prosperity of Canada by building stronger science and technology relationships with Israel, India, China and Brazil. The ISTPP is designed to stimulate bilateral science and technology networking and matchmaking activities to further new partnerships and research projects and accelerate the commercialization of research and development between Canada and partner countries.

DFAIT has selected International Science and Technology Partnerships Canada (ISTPCanada) as the delivery organization for the India, China and Brazil components of the ISTPP and the Canada-Israel Industrial Research & Development Foundation (CIIRDF) as the delivery organization for the Israel component of theISTPP.The primary activity of the arms-length delivery organizations is to deliver program activities under the respective bilateral agreements, including: providing funding and other support to partnering activities that foster alliances between Canadian small and medium-sized enterprises, Canadian research organizations (industry, university and government) and their counterparts in the selected countries; and, supporting Canadian participation in collaborative research projects.

The two delivery organizations, ISTPCanada and CIIDRF, share an administrative structure (e.g. office space and administrative services) and the CEO is the same individual for both organizations. For the purposes of this report, these two organizations are referred to as the ‘delivery organization’.

What did we examine?

The objective of this internal audit was to address senior departmental management concerns and to provide assurance that program management is effectively and efficiently managing the program to demonstrate sound stewardship of public funds and accountability for results.

The internal audit planning phase utilized a risk-based approach to identify areas of higher audit priority in order to focus the audit team’s efforts in the audit examination phase. Based on the planning phase risk analysis, the higher priority risk areas which were examined included the following:

• Governance, accountability, roles and responsibilities;
• Project life-cycle management;
• Program oversight; and,
• Performance measurement

What did we find?

Throughout the audit fieldwork, the audit team observed several examples of how controls are properly designed and applied effectively. This resulted in several positive findings, examples of which are listed below:

• Program management has provided guidance and support to the delivery organizations (i.e. ISTPCanada and CIIDRF) in an effort to help the delivery organizations evolve their financial reporting practices;
• Program management has repeatedly reminded the delivery organizations of the requirement to operate as an independent arm’s length organization;
• With the introduction of the Common Administration Unit and the arrival of the ISTPP Program Officer, oversight practices conducted prior to issuing payments have improved; and,
• Program management has met with the delivery organizations to explain the contribution agreement requirements in May 2012 and has proactively conducted a preliminary compliance review in October 2012.

The audit team confirmed the issues noted by senior management and identified areas where management practices and processes could be improved, which are summarized below:

Improvements are required to certain program management practices

Areas for improvement were noted with respect to Steering Committee governance, partnership development activity approvals, financial oversight and reporting, performance measurement, and audits of ultimate recipients:

• In 2010, the department committed to redesign the ISTPP Steering Committee to clarify and enhance accountability for oversight; however, the ISTPP Steering Committee has not effectively fulfilled its oversight responsibilities. There is a need to review the role, purpose, mandate and required membership for the Steering Committee as well as the frequency of Steering Committee meetings;
• Further clarification is required with respect to the approval process for partnership development activities and whether expenses can be incurred by the delivery organization prior to the partnership development activity being officially approved;
• While ISTPP financial oversight practices and reporting requirements have improved, further enhancements are required to the delivery organization reporting requirements, as requested by ISTPP program management, in order to provide adequate information to allow for effective oversight;
• The performance measurement strategy for ISTPP has not been formally approved. In addition, there is an opportunity for ISTPP program management to develop an approach to ensure performance information is collected after the current contribution agreement with the delivery organization expires; and,
• Lastly, ISTPP program management needs to ensure audits of ultimate recipients are conducted by the delivery organization.

Program management exceeded their financial authorities

There is a need to improve documentation of approval decisions and to discontinue the ISTPP program management practice of providing verbal approvals. Improvements are also required to ensure that ISTPP program management obtains appropriate approvals by delegated authorities for key decisions.

Some stakeholders are uncertain whether the current delivery model is appropriate for this type of program and/or sustainable going forward and there are ongoing areas of concern with the delivery organization's operating practices that need to be resolved.

Areas for improvement were noted with respect to further clarifying the relationship between the delivery organizations and DFAIT, confirming required delivery organization overhead costs, and timely resolution of ongoing areas of concern with respect to certain delivery organization operating practices:

• There is a need to confirm and clarify whether it is essential that the delivery organization operate as an agent of the government;
• Further analysis is necessary to determine the appropriate levels of overhead funding for the delivery organizations to deliver the ISTPP; and,
• Further efforts are necessary to address a number of the delivery organization’s current operating practices which continue to concern the department.

3.0 Conclusion

The audit identified a number of required improvements to the ISTPP’s program management practices, and a need to resolve a number of concerns regarding the current delivery model. These findings represent areas of significance, which need to be addressed in a timely manner in order to ensure an effective governance and control environment is in place, and to allow for the successful delivery of this program on a go-forward basis.

1.0 Background

The Treasury Board approval of the International Science and Technology Partnership Program (ISTPP) included requirements for both an internal audit and a recipient audit to be carried out in 2012-2013. In accordance with this requirement, DFAIT requested that Deloitte conduct audits of the ISTPP covering both the internal audit and the recipient audit requirements. The purpose of this report is to present the internal audit observations and recommendations. The results of the recipient audit were issued through a separate report to DFAIT management.
The internal audit planning phase utilized a risk-based approach to identify areas of higher audit priority in order to focus the audit team’s efforts in the audit examination phase. Based on the planning phase risk analysis, the higher priority risk areas which were examined included the following:

• Governance, Accountability, Roles and Responsibilities;
• Project Life-cycle Management;
• Program Oversight; and,
• Performance Measurement

International Science and Technology Partnerships Program

The International Science and Technology Partnerships Program (ISTPP) was renewed by the Government of Canada in Budget 2010. The five-year, $18.5 million program is expected to promote international collaborative research and development and to increase the international competitiveness and prosperity of Canada by building stronger science and technology relationships with Israel, India, China and Brazil. The ISTPP is designed to stimulate bilateral science and technology networking and matchmaking activities to further new partnerships and research projects and accelerate the commercialization of research and development between Canada and partner countries.

Program Delivery

DFAIT has selected International Science and Technology Partnerships Canada (ISTPCanada) as the delivery organization for the India, China and Brazil components of theISTPP. ISTPCanada is a not-for-profit arms-length organization incorporated in Canada which will support up to 50% of the costs of approved joint research projects proposed by companies and other private sector research and development organizations. In addition, ISTPCanada will conduct partnership development activities to support upcoming calls for proposals.

The Canada-Israel Industrial Research & Development Foundation (CIIRDF) is the delivery organization for the Israel component of theISTPP.CIIRDF is an arms-length bilateral organization incorporated in Canada to foster and support bilateral industrial research projects between Canadian and Israeli companies and supports up to 50% of the costs of approved joint research projects.

The primary activity of the arms-length delivery organizations is to deliver program activities under the respective bilateral agreements, including:

• Providing funding and other support to partnering activities that foster alliances between Canadian small and medium-sized enterprises, Canadian research organizations (industry, university and government) and their counterparts in the selected countries; and
• Supporting Canadian participation in collaborative research projects.

Calls for proposals for collaborative R&D with identified countries are announced periodically, conditional on the funding processes of the partner country. ISTPCanada andCIIRDFare responsible for issuing the calls for proposals in Canada.

The delivery organizations have considerable independence on their choice of projects and activities to fund, and operate; however, they are responsible for ensuring that all projects contribute to the strategic objectives of theISTPPprogram and the priorities of the Canadian government.

The delivery organizations are funded by the department through contribution agreements.

The two delivery organizations, ISTPCanada and CIIDRF, share an administrative structure (e.g. office space and administrative services) and the CEO is the same individual for both organizations. For the purposes of this report, these two organizations are referred to as the ‘delivery organization’.

Program Governance

The overall responsibility and accountability for the ISTPP rests with the Minister of International Trade. Responsibility for program development, implementation and administration at the operational level has been delegated to the Global Business Opportunities – Innovation, Science and Technology Division of DFAIT. A Steering Committee internal to the department provides oversight for the program delivery organizations, and advises the Minister on changes and additions to program priorities. An Advisory Committee, made up of senior-level government representatives from science based departments involved in supporting international research and development efforts, supports coordination with other funding programs across federal departments and agencies. Strategic direction on bilateral S&T priorities is provided by the Joint Cooperation Committees set-up under the science and technology cooperation agreements Canada signed with Brazil, China and India. For Israel, in place of a Joint Committee, the CIIDRF Board is responsible for establishing strategic direction.

Overview of Funding

Approximately $18.5 million has been allocated to the ISTPP for these contribution agreements (Israel, India, China and Brazil) from April 1, 2011 to March 31st, 2015. The allocations are depicted in the table below:

2.0 Observations and Recommendations

2.1 Improvements are required to certain program management practices

Areas for improvement were noted with respect to the following program management practices: Steering Committee governance, partnership development activity approvals, financial oversight and reporting, performance measurement, and audits of ultimate recipients.

The ISTPP Steering Committee is not effective in fulfilling its oversight responsibilities.

Governance and management oversight is critical to ensuring senior management involvement in identifying key initiatives and risks, aligning operational plans with priorities, resolving issues, and monitoring progress against objectives. The audit expected to find an appropriate governance structure in place and operating for the ISTPP.

When ISTPP was renewed in 2010, a key commitment made by the department within its renewal documents was to redesign the ISTPP governance structure to clarify and enhance accountability for oversight of this program on the part of DFAIT. The ISTPP Steering Committee was reconstituted to include only DFAIT officials with direct responsibility for the management of the ISTPP; previously, the Steering Committee was an interdepartmental body which the department co-chaired with Industry Canada. As per the program renewal documents, the Steering Committee, chaired by the Director General, Global Business Opportunities, includes representatives from the Chief Financial Officer’s Branch, as well as representatives from the four relevant Geographic Bureaux, to provide a link between the ISTPP Steering Committee, Canadian diplomatic missions in partner countries and the bilateral Science and Technology (S&T) Joint Committees. Steering Committee observers include representatives from the Centre of Expertise (COE), Evaluation Division, Legal Services, and the Office of the Inspector General/Chief Audit Executive (CAE).

As part of the updated governance structure, the Steering Committee is expected to review and approve the delivery organization’s Work and Performance Measurement Plan. The Work and Performance Measurement Plan is used to determine the funding requirements before the initial transfer of funds under the contribution agreement and on an annual basis thereafter. The Steering Committee is expected to use the Work and Performance Measurement Plan to monitor and track progress against stated goals and objectives and approve any major modifications. Furthermore, all requests for advances, including for operating costs and for direct support for partnership development activities are to be assessed by the Steering Committee against the approved Work and Performance Measurement Plan. The Steering Committee has met four times since its inception in February 2011.
The audit found several areas for improvement with respect to the ISTPP Steering Committee:

• The composition of the Steering Committee was questioned by most members of the Steering Committee (i.e. COE, Legal, CAU, Financial Resource Planning, CAE, program management, Evaluation) as they expressed concern with regards to the Committee members’ limited knowledge of the ISTPP, and hence, their ability to make informed decisions. In addition, participants who are Committee observers and also have DFAIT corporate oversight responsibilities noted that their independence could be impaired by participating on the Steering Committee. As a result, there has been delay in finalizing and approving the Terms of Reference and establishing regular agenda topics. At the time of this audit, the Steering Committee had met twice per year and was not functioning as a decision-making body, rather it was used mostly used for information sharing and discussion purposes;
• Given the importance of the updated governance structure and commitment made by the department at program renewal, interviewees felt the Steering Committee should meet more regularly. A Working Group has recently been established to perform some of the more operational oversight activities which were originally intended for the Steering Committee;
• The Terms of Reference for the Steering Committee were finalized in December 2012 but have yet to be approved by the Steering Committee. The Terms of Reference currently incorrectly describe the Steering Committee’s responsibilities as including a ‘review’ of the Work and Performance Measurement Plan rather than ‘approval’; and,
• Lastly, the Work and Performance Measurement Plan has not been submitted for Steering Committee approval.

Recommendations:

1. Clarify the purpose, role, and composition of Steering Committee to ensure appropriate oversight of program delivery.
2. Confirm the appropriate frequency of Steering Committee meetings to provide adequate oversight.

The approval process for partnership development activities is not clearly defined.

The audit expected that delegated authorities of the ISTPP with respect to approvals of partnership development activities would be understood and reflected in program management actions. The audit also expected program management oversight roles would be clear and decisions taken by program management would be documented and made within approved authorities.

As noted earlier, the Steering Committee is expected to review and approve the delivery organization’s Work and Performance Measurement Plan. All requests for advances, including direct support for partnership development activities are also subject to Steering Committee assessment against the approved Work and Performance Measurement Plan.

The audit found two instances where the approval process related to partnership development activities was not clear:

• In May 2012, the ISTPP Director General signed off on (i.e. approved) the partnership development activities, as opposed to receiving Steering Committee approval, because Steering Committee members reportedly did not feel they had the necessary expertise to make program decisions; and,
• The delivery organization is allowed to charge personnel costs to implement partnership development activities which have been approved by DFAIT. In April 2012, however, ISTPP program management approved that the delivery organization could claim expenses related to staff time up to 5% of the total forecasted partnership development activity budget, in advance of DFAIT approval of the partnership development activity.

Recommendations:

3. Ensure the ISTPP Steering Committee approves a Work and Performance Measurement Plan from the delivery organization and all requests for advances, including for operating costs and for direct support for partnership development activities are subject to Steering Committee assessment against the approved Work and Performance Measurement Plan.
4. Clarify whether expenses can be incurred prior to having an approved partnership development activity in order to identify appropriate topics for, and the scope of, the partnership development activity.

Program financial reporting requirements should be enhanced to provide additional information to allow for more effective oversight.

The audit expected that ISTPP program management would be making adequate use of oversight tools (e.g. business plan, progress reports) and would be adequately reviewing and challenging information provided in oversight tools, prior to approving payments.

Historically, from 2005-10, interviewees noted that advance payments were provided from the ISTPP to the delivery organization in an annual lump sum amount with little verification or analysis from program management before issuing the payments. As of 2010, the ISTPP initiated a requirement for quarterly progress and financial reporting which shows all funds received, disbursements made, funds committed, forecasts of revenues and expenditures as well as the key risks in these forecasts. The reports allow the ISTPP to monitor disbursements and identify potential issues in a timely manner. The ISTPP Program Officer receives quarterly progress reports and compares them against the contribution agreement, against previous progress reports (planned vs. actual), and reviews them for accuracy and logic. In an effort to assist the delivery organization in completing the progress reports, the ISTPP Program Officer has also developed instructions on how to submit the required information, which have been shared with the delivery organization.

While ISTPP program management financial oversight of the delivery organization has improved substantially, areas for further improvement were noted:

• Timesheet-based reporting: although clearly identified as a delivery organization requirement in the contribution agreements since 2010, there has been delayed implementation of timesheets by the delivery organization. Timesheets were only first implemented in July 2011; however, they have not been required by ISTPP program management as evidence for progress payments. As such, the ISTPP is not able to validate that personnel costs are claimed against approved projects; and,
• Overall reconciliation of delivery organization costs: the audit found that insufficient information is being requested to allow for a clear overall reconciliation of how the delivery organization’s costs are being split across the various funding arrangements to ensure appropriate allocation of expenses to the different funders.

Recommendation:

5. Implement enhancements to oversight practices including requiring the delivery organization to submit timesheets to substantiate claims, and ensuring that the financial information received from the recipient includes an overall reconciliation of the delivery organization’s costs to the total expense claims being submitted to various funders.

Performance measurement strategy needs to be finalized and implemented in a timely manner.

The audit expected that an effective performance measurement strategy would have been established, performance measures would have been reviewed on a periodic basis and updated as required, data sources would have been identified, and responsibility for monitoring and updating performance measures would have been clear and communicated. The audit also expected that management performance reports (e.g. management reports, project progress reports, annual reports, activity reports, etc.) would be used by program management to assess whether key activities are performed and whether these activities are linked to program objectives.

An ISTPP Performance Measurement Strategy was developed in February 2011 and the approved June 2011 ISTPP Terms and Conditions included performance measures, expected results, performance indicators and economic benefits. In the fall of 2011, however, ISTPP decided to initiate an update to the performance measurement indicators to better align to departmental priorities. Revised performance indicators were finalized in February 2012 and included in the revised ISTPP Terms and Conditions which were submitted for approval in August 2012. The purpose of updating the performance measurement indicators was to better align the ISTPP performance measures to departmental priorities. At the time of the audit, however, the revised ISTPP Terms and Conditions and performance measurement indicators had not yet been approved.

A revised draft Performance Measurement Strategy (which includes additional indicators beyond those found in the ISTPP Terms and Conditions) was also developed and approved by the Steering Committee in December 2012. The delivery organization reports that it is on track to implement its formal data collection system based on the revised performance indicators in 2013-14. ISTPP anticipates it will be 2014-15 before initial performance indicator data is made available.

In addition, the audit noted that all the R&D projects funded by the delivery organization are scheduled for completion several years after the expiration of the contribution agreement. The results of these projects are to be sent in to the ISTPP and would provide important performance information to enable assessment of the program’s performance. There is currently no strategy in place to ensure that this performance data is indeed gathered after the contribution agreements expire.

Recommendations:

6. Obtain proper approval on the revised performance indicators in the revised program terms and conditions.
7. Conduct diligent oversight to ensure the delivery organization implements data collection system and implements the Performance Measurement Plan.
8. Define a strategy to ensure performance data is collected after the contribution agreements expire to ensure important performance information is gathered to enable assessment of the program’s performance.

Audits of ultimate recipients are not being planned or conducted.

Ultimate recipients are companies, universities/colleges, and other private sector R&D institutions that conduct joint R&D projects and receive funding via the delivery organization. The audit expected that ISTPP would ensure that the delivery organization conducts audits of ultimate recipients in order to gather information on project results and outcomes for ISTPP performance measurement purposes and to ensure funds were used for their intended purposes.

As per the contribution agreement signed by the delivery organization, the delivery organization agreed to conduct audits of these ultimate recipients. The delivery organization can use a portion of the transfer payment funding provided by DFAIT to offset its administrative /overhead costs in order to complete these audits of ultimate recipients. ISTPP has provided templates to the delivery organization to assist in the development of a risk-based audit plan for these ultimate recipient audits.

The audit found, however, that audits of ultimate recipients are not being planned or conducted by the delivery organization. Although the delivery organization’s Annual Report 2011-12 and revised 2013 work plan note that the delivery organization intends to conduct audits of ultimate recipients, the delivery organization reported to program management in October 2012 that they do not feel there are sufficient funds provided by the ISTPP to conduct such audit activities of ultimate recipients. The delivery organization has reported to DFAIT that is forecasting an operational deficit situation and if no additional funds became available by April 1, 2013, the delivery organization would need to implement reductions in payroll, staff and operating expenses. There is concern within the department that audits of ultimate recipients may not be performed, given the delivery organization’s current financial challenges.

Recommendation:

9. Obtain proper approval on the revised performance indicators in the revised program terms and conditions.

2.2 Program management exceeded their delegated authority.

The audit expected that delegated authorities would be understood and reflected in program management actions. The audit also expected program management oversight roles to be clear and decisions taken by management would be documented and made within their approved authorities. The audit also expected that, as required, the perspective of subject matter experts would be considered by program management when making program decisions.

Under the initial ISTPP Terms and Conditions and authorities (2006), DFAIT entered into three contribution agreements with the delivery organization for activities taking place in India, China and Brazil. Contribution agreements were approved for the following amounts: India $6.75M; China $5.25M; Brazil $1.4M. In June 2011, the ISTPP Terms and Conditions and authorities were renewed from 2011-12 to 2014-15 for an additional total of $18.5M in transfer payment funding, and the maximum amount payable per contribution agreement under the 2011 Terms and Conditions was limited to a maximum of $5M.

The audit found that in November 2011, ISTPP program management was informed by the delivery organization that two of the projects funded under the 2007 India contribution agreement would be cancelled because the partner country, India, did not want to participate and approximately $1.1M could lapse. The delivery organization indicated to program management that five projects for China and one for Brazil had been recommended during the previous call for proposals; however, they were not approved due to lack funds. The delivery organization requested a transfer of the India funds to China and Brazil. The audit team was informed that ISTPP program management gave verbal approval to the delivery organization that the available funds from the India contribution agreement could be transferred to the China and Brazil contribution agreements.

In this context, it is important to note that once a department enters into a contribution agreement, the department is bound by the clauses of that contribution agreement, regardless of what happens to the program’s overall terms and conditions and authorities; however, when amendments are made to a contribution agreement, amendments must be made in accordance with the terms and conditions and authorities in effect at the time of the amendment. As such, in November 2011, when the verbal approval was provided by program management to transfer funds from the India contribution agreement to the China and Brazil agreements, the June 2011 program terms and conditions would have applied. This transfer required Ministerial approval to amend the June 2011 ISTPP authorities and to remove the $5M maximum amount payable limit. ISTPP program management inappropriately gave verbal approval prior to obtaining Ministerial approval.

Between November 2011 and March 2012, ISTPP program management sought advice on what actions were required to formalize their verbal approval and to amend the contribution agreements. In March 2012, amendments to the China and India contribution agreements were developed and sent to the delivery organization for signature; however, these amendments have yet to be signed by program management.

In August 2012, ISTPP program management sent a memo to the Minister, requesting approval for revisions to ISTPP Terms and Conditions. The memo, however, did not clearly indicate that verbal approval had already been provided in advance of requesting Ministerial approval. In addition, revisions to ISTPP Terms and Conditions have yet to be approved.

As of the January 31, 2013, approximately $5.5M had been spent under the China agreement with the remaining balance planned to be disbursed by March 2013. This amount is approximately $250,000 in excess of the original contribution agreement ($5.25M) without a signed amendment and also exceeds the $5M maximum amount payable under the 2011 ISTPP Terms and Conditions and authorities that are currently in effect.

Recommendations:

10. Obtain proper approval on required revisions to ISTPP Terms and Conditions in order to authorize funding transfers.
11. Sign contribution agreement amendments to formalize verbal approvals.
12. Discontinue the practice of making verbal approvals, and document supporting rationale, including those consulted, for key approval decisions.

2.3 Some stakeholders are uncertain whether the current delivery model is appropriate for this type of program and/or sustainable going forward; and there are ongoing areas of concern with the delivery organization's operating practices that need to be resolved

Areas for improvement were noted with respect to further clarifying the relationship between the delivery organizations and DFAIT, confirming required delivery organization overhead costs, and timely resolution of ongoing areas of concern with respect to certain delivery organization operating practices:

Maintaining a non-agent relationship remains a challenge.

The audit team expected to find that a non-agent relationship existed between the department and the delivery organization this relationship was understood and respected by both parties.

The ISTPP has been granted Treasury Board approval and authority to enter into contribution agreements with the delivery organization. As per approved ISTPP authorities, the delivery organization is to be an arms-length, Canadian non-government organization with appropriate expertise and experience who is commissioned to conduct activities in support of the bilateral science and technology relationships with partner countries and is responsible for administering the program within those selected bilateral science and technology relationships. In addition, the delivery organization is to have considerable independence on its choice of projects and is expected to operate with minimal guidance from the department. The delivery organization is the initial recipient of contributions under the ISTPP, who in turn, contributes funding to projects conducted by companies, universities/colleges, and other private sector R&D institutions.

The ISTPP contribution agreement clearly states that the delivery organization is not an agent of the government (Clause 27).

The audit found that the Board of Directors for the ISTPCanada delivery organization (Board) has stated their understanding is that from the first introduction of ISTPP, as a new program, a ‘delivery agency’ would be required to deliver the program and to validate the delivery model. In the Board’s view, the delivery organization is “in fact and in intent a delivery agent of the Government of Canada”. The Board has noted that the contribution agreement used by ISTPP is not suited to reflect the intent and purpose of the relationship and suggests that a new agreement structure is required, one more in alignment with the delivery organization’s mandate.

The audit also found differing views exist in the department on whether it is essential that the delivery organization operate as an agent of the government. Lastly, the audit team was informed that one partner country has indicated its preference for working directly with government-led programs rather than an arms-length body as they are more comfortable working with high-level government officials in the selection of activities for collaborative research. This may indicate that a non-agent relationship may not be viable delivery option for this particular country.

Recommendations:

13. [REDACTED]
14. [REDACTED]

A “bottom-up” analysis has not been conducted by program management to determine the required overhead/operating costs to deliver the program.

The audit team expected to find an effective and optimal delivery model would be in place. The audit team also expected to find that program management had considered program design options and conducted an analysis to determine the required overhead/operating costs to deliver ISTPP.

As per ISTPP Terms and Conditions, DFAIT agreed to set aside 15% of the available $18.5M Vote 10 funding ($2.78M) for the delivery organization‘s overhead costs and to provide this funding based on the delivery organization’s estimated cash flow requirements, along with an assessment of how past overhead funding was used. The delivery organization is required to limit its requests for reimbursement for overhead to no more than 15% of the total funding provided.

As per the contribution agreement, administrative and overhead costs include the following types of expenditures/costs: project personnel (i.e. actual annual salary and fringe benefits), non-recipient personnel and outside consultants engaged solely for the purposes of the project, reasonable travel and meal expenses, all other actual necessary reasonable and justifiable out-of-pocket travel related expenses etc.

The delivery organization first reported to program management in August 2012 that it was forecasting an operational deficit situation, where operating costs /expenses would be in excess of the 15% of total funding allowed under the contribution agreement. The delivery organization’s January 2013 revised work plan assumed that if no additional funds became available by April 1, 2013, the delivery organization would need to implement reductions in payroll, staff and operating expenses.

The delivery organization has reported that the 15% ceiling is insufficient to deliver the program; however, the audit noted that no formal analysis has been conducted by program management to confirm the actual required overhead funding to deliver the ISTPP at its current overall funding levels.

Recommendation:

15. Conduct a ‘bottom-up” analysis to determine the required overhead/operating costs to deliver the ISTPP and determine whether the current delivery model is optimal.

Ongoing disagreements between DFAIT and the recipient are causing frustration in the working relationship and there are unresolved concerns regarding some of the delivery organization’s operating practices

The audit expected to find that an effective working relationship would be in place and the delivery model selected by program management would be working as intended. The audit also expected to find both parties to the relationship, DFAIT and the delivery organization, had practices in place to achieve a common understanding in order to resolve issues.

Previous recipient audits on the delivery organization have identified several areas for improvement with regards to the delivery organization’s financial management, record keeping and internal oversight mechanisms to ensure expenses are properly approved and reported. These recipient audits also recommended that DFAIT more closely monitor the operating expenses of the delivery organization. In response to these audits, the ISTPP and the delivery organization both agreed to work together and take action to make these improvements.

The audit found examples where ongoing disagreements and concerns between DFAIT and the delivery organization are causing frustration in the working relationship, making it difficult to move forward:

• The contribution agreement in place designates the ISTPP Program Officer and the delivery organization VP, Operations as the main points of contact. The audit found that, historically, both the department and the delivery organization have not always followed this communication protocol. The channels of communication used between the department and the delivery organization have varied, depending on the issue and the information being requested. This has caused inconsistency and confusion in the information communicated;
• While progress has been made in resolving some issues, some non-compliance issues still remain and continue to occur (e.g. undocumented contracting practices, unsupported travel claims, etc.). Similar issues were once again noted in the recipient audit conducted at the same time as this internal audit.
• A number of the delivery organization’s current operating practices continue to be concerns for the department. Some of these practices include:
  • The President and Chief Executive Officer for the ISTPC delivery organization and CIIDRF delivery organization is the same individual and this person is employed as a contractor as opposed to a full-time employee;
  • The salary and performance pay provided to the President and CEO and senior management is considered justifiable by the delivery organization but subject to further scrutiny by the department, especially when the delivery organization is stating that it is struggling to stay out of a deficit position. These pay levels may be inadequately benchmarked to current local market compensation; and,
  • In the event that the delivery organization implements planned reductions in payroll, staff and operating expenses, there is also concern within the department regarding the recipient’s ability to deliver on its program activities with a reduced operating structure or staff.

Given the above, there is increased concern regarding the capacity of the delivery organization which has resulted in increased oversight by program management. This oversight is perceived by the delivery organization as overly burdensome.

Recommendation:

16. DFAIT should develop an action plan with clear departmental accountabilities and timelines to resolve all outstanding issues with the delivery organization so all stakeholders can move forward with program delivery.

3. Conclusion

The audit identified a number of required improvements to the ISTPP’s program management practices, and a need to resolve a number of concerns regarding the current delivery model. These findings represent areas of significance, which need to be addressed in a timely manner in order to ensure an effective governance and control environment is in place, and to allow for the successful delivery of this program on a go-forward basis.

Appendix A: About the Audit

Objective

The objective of the internal audit was to provide assurance that program management is effectively and efficiently managing the program to demonstrate sound stewardship of public funds and accountability for results.

Criteria

The selected criteria for the examination phase have been derived from elements of the Office of the Comptroller General’s draft document "Core Management Controls". The criteria are mapped to the Management Accountability Framework (MAF) and were selected to enable the audit team to respond to the audit’s objectives.

Scope

The internal audit was conducted from February 2013 to April 2013.

The internal audit planning phase utilized a risk-based approach to identify areas of higher audit priority in order to focus the audit team’s efforts in the audit examination phase. Based on the planning phase risk analysis, the higher priority risk areas which were examined included the following:

• Governance, accountability, roles and responsibilities;
• Project life-cycle management;
• Program oversight; and,
• Performance measurement

Methodology

The internal audit was conducted in accordance with the International Standards for the Profession of Internal Auditing and the Treasury Board Standards for Internal Audit.

The methodology consisted of:

• Interviews with DFAIT management and staff;
• Document review (policies, procedures, program procedures, and relevant audit and evaluation reports);
• Analysis of financial and non-financial information;
• File review; and,
• Testing of in-scope controls (e.g. progress payment review and approval)

Appendix B: Management Action Plan