Audit of the Colombia Development Program

Office of the Chief Audit Executive

March 2015

Executive Summary

In accordance with its approved Risk-Based Audit Plan for 2014-15, the Office of the Chief Audit Executive at the Department of Foreign Affairs, Trade and Development (DFATD) conducted an internal audit of the Colombia Development Program.

Why is this Important?

The Colombia Development Program (the Program) is decentralized and currently operating out of the nation’s capital of Bogota. Alongside Canada’s foreign policy interests in Colombia which include high involvement in the extractive sector and the Free-Trade Agreement signed in 2008, Colombia is a country of focus for DFATD and has become a growing priority for Canada in terms of development. While Colombia has had a long-standing democracy and is a middle-income country, income inequality and extreme poverty continue to persist alongside a long standing internal armed conflict resulting in one of the highest rates of internally displaced people in the world.

What did we examine?

The objective of the audit was to provide assurance that the management of the Colombia Development Program is efficient and effective to support achievement of program objectives. Therefore the audit asked:

• Are roles and responsibilities for the management and oversight of the Colombia Program clearly defined and communicated to support effective and timely decision making?
• Is the planning and management of projects efficient and effective?
• Are resources optimally planned and managed to support the Colombia Program’s objectives in a cost-effective manner?
• Is the Colombia Program effectively managing its reputational risk?

The audit scope included an examination of:

• Processes and practices in place;
• Projects that are operational or closed;
• Financial information;
• Information related to human resources; and
• Human resources and financial information of Peru and Honduras programs for benchmarking purposes.

What did we find?

Overall, the Program is efficient and effective in the management and achievement of program objectives. It has clearly defined roles and responsibilities, which are well understood by all staff. The planning and management of projects is efficient and effective, and human resources are managed in a cost-effective manner. Furthermore, the Program has demonstrated initiatives in working towards mitigating its reputational risk involving the extractive sector.

However, the Program could improve the management of its documentation and improve its long term sustainability through the development of a human resources analysis.

Roles and responsibilities for management and oversight are clearly defined and communicated to support effective and timely decision making.

Frequent and clear communication by Program management and staff has created an environment of collaboration and cohesiveness that inform clearly defined and well understood roles and responsibilities. This, in turn, supports effective and timely decision-making.

The planning and management of projects are efficient and effective.

Overall, the Colombia Development Program was found to adhere to Departmental policies and procedures in terms of project planning and monitoring. There was a clear process in place that ensures projects are effectively managed across their life cycle. However, opportunities for improvements were identified regarding documentation retention.

Resources are optimally planned and managed to support the Colombia Program’s objectives in a cost-effective manner; however, the Program could benefit from a business needs analysis.

The Department is going through a period of transition that provides an opportunity at developing a more cost-effective environment. In comparison to benchmarked programs, the Colombia program currently utilizes its resources in a cost-effective manner. However, questions regarding the sustainability of operations should be explored through a human resources analysis.

Due diligence has been conducted by the Program in its efforts to mitigate reputational risk.

Reputation and stakeholder confidence was one of the high risks listed on the 2013-14 Colombia Program risk register. The Colombia Development Program has been working to improve the mitigation of its reputational risk, derived from the use of private partnerships with the extractive sector, through diligent practices and participation in the development of the Embassy Strategic Communication Plan.

Recommendations

1. The Director of Cooperation of the Colombia Development Program should ensure that project file documentation follows the appropriate information management life-cycle, including collection, organization and disposition.
2. The Assistant Deputy Minister Americas should conduct a human resources analysis to confirm resource requirements in support of future Program sustainability.

Statement of Conformance

In my professional judgment as the Chief Audit Executive, this audit was conducted in conformance with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing and with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program. Sufficient and appropriate audit procedures were conducted, and evidence gathered, to support the accuracy of the findings and conclusion in this report, and to provide an audit level of assurance. The findings and conclusion are based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed upon with management and are only applicable to the entity examined and for the scope and time period covered by the audit.

Jean Goulet
Chief Audit Executive

1.0 Background

The Audit of the Colombia Development Program (the Program) was part of the 2014-2015 Risk Based Audit Plan recommended by the Audit Committee and approved by the Deputy Minister of Foreign Affairs.

While Colombia is a long-standing democracy with a dynamic and growing market economy, income inequality persists with 16% of the population living on less than US $1.25 per day. Colombia continues to suffer from an internal armed conflict that has lasted fifty years and has forced the displacement of approximately 4.9 million of its population, one of the highest rates of internally displaced persons in the world. Violations of human rights as well as infractions against international humanitarian law continue to be of concern.

The Colombia Development Program is under the responsibility of the Americas Programming Bureau in the Americas Branch. Colombia is a country of focus for international assistance and a foreign policy priority. Canada has interests in Colombia’s extractive sector, representing 70% of the foreign extractive companies in the country.

The Program’s focus is well-aligned with the development priorities of the current Government of Colombia’s International Cooperation Strategy. According to DFATD’s draft Colombia Country Development Strategy 2014-2019, Canada’s bilateral development programming objective is to contribute to more inclusive social and economic development. To achieve this objective, the Program was provided with a programming budget of $23 million in 2014-2015, an increase of 22% from the previous years.  The Program will continue to focus on the two thematic priorities: children and youth, and sustainable economic growth, with gender equality, environmental sustainability, and governance as cross-cutting themes. Human rights programming has been complementary to both sectors of focus.

The Program is decentralized, with a staff of eleven in the field (one Program Director, four Canada-based staff and six locally-engaged staff) and one at headquarters. In fiscal year 2013-2014, approximately 16 projects were operational. At the time of the audit, the program was operating with no Program Support Unit (PSU); however, the Program was expected to put in place a new model for in-country delivery of program support services called the Field Support Services (FSS) Model.

2.0 Observations and Recommendations

2.1 Oversight

To effectively manage staff, clear roles and responsibilities must be established for team members and there must be adequate oversight. Doing so facilitates an atmosphere of collaboration and cooperation, reduces duplication of efforts and allows for effective and timely decision-making. The audit found that the Program has clearly defined and communicated roles and responsibilities and appropriate oversight.

The Program Committee, chaired by the Assistant Deputy Minister of Americas, is a component of the governance structure for all Country Programs, including the Colombia Program. It plays an oversight role responsible for providing guidance and ensuring coherence with the Department’s programming. Its role is to ensure consistency and rigour in program design, delivery and performance across all the programs of the department, with particular reference to DFATD’s grants and contributions programs which are funded from the International Assistance Envelope.

As depicted in Figure 1, the Director of Cooperation reports upwardly to the Colombia Head of Mission (HoM/Ambassador) on a day-to-day basis. This includes the HoM’s involvement and consultation on all development activities at program strategic and project levels. In addition, the Director of Cooperation receives functional direction from, and reports on aid and Operation & Maintenance (O&M) budgets to the Director General of the Americas Programming Bureau (NDD) who in turn reports to the Assistant Deputy Minister. 

At the mission level, two management meetings involving the Program occur weekly to provide guidance to management. At these meetings, main initiatives and issues relating to the Program are presented and discussed. These meetings are the:

• Committee on Mission Management - Chaired by the HoM and attended by the managers of the programs in the mission, this committee reviews and discusses a broad range of mission management issues;
• Decentralized Executive Committee for Americas Programming Bureau - Chaired by Director General of the Americas Program and attended by the Directors of Strategic Operations, the directors of the decentralised countries, and corporate services advisors, this committee within the Americas Program (NDD) provides support to the directors in the decentralised missions and provides briefings on HQ affairs.

Across the Program, roles and responsibilities are clearly defined and documented including current project descriptions for all staff and a specific Mandate Letter for the Director of Cooperation.  The Mandate Letter, which is prepared for each decentralized Director of Cooperation includes but is not limited to governance framework, official and representational duties, and policy and program management. The Mandate Letter also acknowledges the duality of the reporting responsibilities of the Director of Cooperation as seen in Figure 1. Interviews with the Director of Cooperation found that that roles and responsibilities are clearly understood.

Two good Program management practices were noted. First, the Latin America & Caribbean Bureau has developed a Pooled Support Service which allocates a Headquarters (HQ) coordinator to each decentralized country program. This allows continual liaison with HQ and additionally more effective communication with external stakeholders located in Canada. Second, the Program developed an informal colleague back-up system. This ensures that staff members have an understanding of their colleague’s roles and responsibilities, and ongoing projects, therefore allowing for knowledge sharing and business continuity in case of absence.

2.2 Project planning and management

Efficient and effective planning and management are essential to meet the objectives of each project within the Program and to ensure the Program remains aligned with Departmental priorities. Based on the examination of a sample of nine projects, the audit team found that processes are in place to:

1. Effectively assess and review planned projects;
2. Enable the timely and appropriate disbursement of funds to recipients; and
3. Monitor whether projects are achieving expected results.

To assess the effectiveness of the project planning, nine projects funded through bilateral programming, representing 71% of the total budget of all projects within the audit scope, were examined. This sample also provided representative coverage of both thematic sectors and funding mechanism, as illustrated in Appendix B.

2.2.1 Processes are in place to effectively assess and review planned projects.

Based on the nine projects reviewed, the audit team found that project objectives were aligned with Program objectives, and to the Department and Government of Canada’s priorities. Furthermore, the planning process of projects was well understood and followed by staff as demonstrated through the review of sampled project files in which most of the programming process requirements were completed.

Though the processes in place were well understood and applied, the audit identified some areas of improvements related to project documentation.

Due to the decentralized nature of the Program, it is important that project documentation is complete, well managed, and easily retrievable. During the course of the audit, it was found that key control project planning documentation, such as CFO Attestations/Initiative Assessments (44% of the project sample) and evidence of Financial Management Advisor review (22% of all project samples) was missing. Furthermore, duplicates of documentation and a poor filing structure impede access to accurate key project documentation.

Recommendation

1. The Director of Cooperation of the Colombia Development Program should ensure that project file documentation follows the appropriate information management life-cycle, including collection, organization and disposition.

2.2.2 Processes are in place to enable the timely and appropriate disbursement of funds to recipients.

The disbursement of public funds requires a high standard of integrity, accountability and transparency. This requires the establishment of responsible account verification processes that ensure sound stewardship of financial resources. Account verification provides assurance that the work has been performed, that the services were rendered or the goods were supplied, the terms and conditions of the contract or agreement/arrangement have been met, the transaction is accurate and executed in a timely manner, and that they are in compliance with applicable authorities, acts, regulations, policies and/or directives.

The audit found that the processes in place allow for the timely and appropriate disbursement of funds. The review of project files show that all agreements had a set disbursement schedule with a defined basis of payment and that no initial disbursements were made before the signing of the agreement/arrangement. Additionally, all disbursements were made based on a review of reports and on financial justifications. Complete documented evidence of advance request forms submitted by partners and account verification prior to disbursements was also available. The financial project files included all the appropriate financial documentation per disbursement and were up to date.

An in-depth examination of a representative sample of expenditures found that all were aligned with the terms and conditions of the agreements, were incurred after the contribution was signed, were eligible, corresponded with planned budget, and had appropriate records.

2.2.3 Processes are in place to monitor whether projects are achieving expected results and corrective actions are taken as necessary.

The Results-Based Management (RBM) Policy Statement refers to a life-cycle approach that integrates strategy, people, resources, processes, and measurements to improve decision making, transparency, and accountability. RBM focuses on achieving outcomes, implementing performance measurement, learning, and adapting, as well as reporting performance. Based on the file review, all agreements/arrangements in the sample contained RBM required information. Processes were also in place to monitor whether projects were achieving expected results and corrective actions were taken when necessary.

Project monitoring in the Program consists of practices such as field visits to project sites; reviewing financial and progress reports; and maintaining continual, close and open communication with partners and stakeholders. During interviews, Program staff informed the audit team that the decentralized environment of the program strengthened its monitoring practices due to in-field proximity.

The file review found that all reporting requirements were fulfilled. Completed performance progress reports were reviewed, and evidence showed that follow-up and feedback were done when necessary.

2.3 Resource planning and management

Optimizing the workforce to enable high performance and effective use of human resources is a good management practice and a requirement of the Treasury Board Secretariat’s Policy on Management, Resources and Results Structure as well as the Management Accountability Framework.

Over the past two years, the Department has gone through a period of transition and change which has had a significant impact on programs. The Deficit Reduction Action Plan (DRAP), decentralization, and amalgamation have imposed challenges on programs resources to become more cost-effective while continuing to achieve its overall objectives.

To assess the cost-effectiveness of the Program, a benchmarking analysis was conducted. Two comparable Country Programs were selected based on similar budget, programming and geographic location. The level of disbursement was the main indicator selected to measure effectiveness of the program. The audit did not take into account the specific country context, level of policy dialogue, coordination with other donors, and amalgamation at the embassy. The central difference between the benchmarked countries and the Colombia program is the existence of a Program Support Unit (PSU). A PSU supports the selected benchmarked programs by providing expert advisory, administrative and logistical support. Of note, the Department is currently converting PSUs into a comparable Field Support Service (FSS) model.

2.3.1 Human Resources are planned and managed to support the program’s needs and sustainability of operations.

According to DFATD guidance, all branches and all human resources managers should carry out human resources planning at least once a year. They should also regularly monitor identified human resources (HR) needs and proposed HR strategies.  The Program currently does not have a formal HR analysis. The two tools used to plan HR are the salary forecasting tool and the program’s organizational chart.

A current and robust analysis would benefit the Program in its ability to identify: the level of effort required, and appropriate mix of specialists; retention and succession plans; training needs; and will be a valuable tool to ensure continuity of operations in the event that the Program experiences sudden changes in staff.

Because of the changing environment of the Department, and the complex nature of development programming, the Program remains at risk of being negatively impacted by new corporate demands, staff turnover and aid budget fluctuations. With the anticipated growth of the Program’s budget it is critical to ensure that the Program is well positioned to meet the projected workload.

The audit examination indicated that increasing workloads within the Program are a result of: an increase in the aid budget; an increase in the number of contribution agreements over grant arrangements; an increase in political interest in the country involving frequent high-level visits; and a decrease in corporate support services due to DRAP.

Recommendation

1. 2. The Assistant Deputy Minister Americas should conduct a human resources analysis to confirm resource requirements in support of future Program sustainability.

2.3.2 Resources are utilized in a cost-effective manner that supports the Program’s needs and sustainability of operations.

In comparing the Colombia Program to selected benchmarked Country Programs, disbursements were relatively consistent. Travel, training, operations and maintenance (O&M) costs, and grants to contributions ratios (as a percentage of total administrative expenses) were consistent amongst the three programs.

An analysis was conducted to compare the funds spent on administration costs (salaries, O&M, and PSU expenditures) against the disbursed programming aid budget. The analysis measured the output realized based on funds spent on staff and the administration costs of staff in managing their projects. Based on the benchmarking analysis, the Colombia program currently utilizes its resources in a cost-effective manner. While project spending has been augmented and staffing levels have remained the same, program objectives continue to be achieved.

Although the benchmarked findings indicate that the program is cost-effective, there is a risk that increasing workloads may jeopardize the sustainability of program effectiveness and the ability to meet objectives.

2.4 Management of reputational risk

Canadian firms are the largest extractive sector (oil and gas, mining) foreign investors in Colombia. There is an expectation that Canadian expertise could help Colombia better manage its natural resources. In terms of development, there is the additional expectation that Canada should assist in diversifying rural economies, and also support other development initiatives in extractive zones.

The Program’s 2015-2019 risk register rated “reputation and stakeholder confidence” as a high and increasing risk. This risk implies that DFATD’s reputation could be negatively affected by association to projects with the extractive sector.  To mitigate this risk, the Program described the following activities:

• Put in place a communications strategy to communicate DFATD’s approach to working with the private sector, notably the extractive industry;
• Implement an outreach plan to explain and promote better understanding of DFATD’s goals, including progress and results of projects co-financed by the private sector;
• Perform due diligence analysis when partnering with private sector to support economic opportunities for vulnerable populations;
• Strengthen efforts with regards to information sharing, consultation and engagement with Government of Colombia and partners to more carefully manage expectations;
• Establish permanent contact with other sections within the Mission (e.g. Trade and Political) to exchange information and manage specific events related with the extractive sector;
• Advance the Mission Corporate Social Responsibility (CSR) Roadmap. CSRs are voluntary activities undertaken by companies, working in the extractive sector, to operate in a more economically, socially and environmentally sustainable manner.

2.4.1 The Colombia Program communicates its priorities and objectives to external stakeholders.

In recent years, the Program has grown significantly in size, scope and visibility. Increased visibility requires additional effort to ensure that Canadian development priorities are in line with the needs of local communities.

To promote Canadian visibility and in an effort to mitigate reputational risk, an Embassy Strategic Communication Plan was developed. This plan was a coordinated effort between DFATD HQ and the Embassy of Canada in Colombia under the direction of the HoM and was approved in January 2015. The plan aims to increase awareness of Canada’s multifaceted engagement in Colombia over the next three years and covers most of the communication engagements described in the Program’s 2015-2019 risk register.

The Program has also developed a statement of work for a communication advisor consultant. This consultant will work to increase the visibility and understanding of Canada’s bilateral development assistance program in Colombia and will produce communication deliverables in coordination with the DFATD Communications Advisor based at headquarters, as well as other local advisors and project partners.

2.4.2 Due diligence was performed to mitigate reputational risk.

To ensure that DFATD is co-financing projects with extractive companies that are aligned with Canadians standards of integrity, ethics and values, the Program conducts due diligence to mitigate its reputational risk. Due diligence was performed in each project sampled where the Program was partnering with an entity from the private sector working in extractives. The following activities were conducted with prospective bidders prior to entering into arrangements:

• Background and history checks;
• Assessments of financial results;
• Consultation with Colombia’s Attorney General's Office to assess the existence of any unlawful activity;
• Company searches through public domain, state web pages and official web pages of institutions and organizations investigated; and
• Assessment of other relevant and available information.

Once the due diligence report was completed, a technical assessment for compliance with the Department’s cross-cutting themes and with the project terms of reference was conducted. These results were presented to the development team for review and selection. The selected projects were then presented to the Local Advisory Steering Committee for advice and recommendations to the Head of Aid for final approval.

In addition to this process, the audit noted that the Program performed other activities to mitigate its reputational risk. These activities included the Program’s participation in the development of the mission’s CSR Roadmap. The CSR Roadmap refers to key development, trade, and political actions to be taken within the year to ensure the advocacy and promotion of Canada's CSR policies in alignment with Government of Colombia priorities. A survey was also conducted by the mission regarding CSR activities of Canadians companies in the extractive sector in Colombia.  The survey feedback provided useful information to define issues for further engagement with government, companies, and communities which will assist the Program better manage reputational risk.

The due diligence procedures reviewed during the audit corroborates and effectively assists the Program in managing its reputational risk.

3.0 Conclusion

The audit concludes that the Colombia Development Program is efficient and effective in the management and achievement of program objectives. The Program has clearly defined roles and responsibilities. The planning and management of projects is efficient and effective, and human resources are managed in a cost-effective manner. Furthermore, the Program has demonstrated initiatives in working towards mitigating its reputational risk involving the extractive sector.

However, the Program could improve the management of its documentation and improve its long term sustainability through the development of a human resources analysis.

Appendix A: About the Audit

Objective

The objective of the audit was to provide assurance that the management of the Colombia Development Program is efficient and effective to support achievement of program objectives.

Scope

The scope of the audit included:

• Processes and practices in place;
• Projects that are operational or closed;
• Financial information for 2011-12 to 2014-15;
• Information related to human resources; and
• Human resources, financial and other relevant program information for Peru and Honduras programs for benchmarking purposes.

Criteria

The criteria were developed following the completion of the detailed risk assessment and considered the Audit Criteria related to the Management Accountability Framework developed by the Office of the Comptroller General of the Treasury Board Secretariat. These criteria are also known as Core Management Controls. The audit criteria were discussed and agreed upon with the auditees. The detailed criteria are presented as follows.

Criterion: 1.0 Roles and responsibilities for the management and oversight of the Colombia development program are clearly defined and communicated to support effective and timely decision-making

Criterion: 2.0  The planning and management of projects are efficient and effective

Sub-criterion: 2.1  Processes are in place to effectively assess and review planned projects

Sub-criterion: 2.2  Processes are in place to enable the timely and appropriate disbursement of funds to recipients

Sub-criterion: 2.3 Processes are in place to monitor whether projects are achieving expected results and corrective actions are taken, as necessary

Criterion: 3.0  Resources are optimally planned and managed to support the Colombia program’s objectives in a cost-effective manner

Sub-criterion: 3.1  Human resources are planned and managed to support the program’s needs and sustainability of operations

Sub-criterion: 3.2  Resources are utilized in a cost-effective manner that supports the program’s needs and sustainability of operations

Criterion: 4.0  The Colombia program effectively manages its reputational risk

Sub-criterion: 4.1 The Colombia program effectively manages its reputational risk

Sub-criterion: 4.2 Due diligence is performed to mitigate reputational risk

Approach and Methodology

In order to conclude on the above criteria, and based on identified and assessed key risks and internal controls associated with the related business processes, the audit methodology included, but was not limited to the following:

• Conducting walkthroughs of project planning and implementation processes;
• Identifying and reviewing relevant policies, guidelines, and legislation;
• Analyzing financial and non-financial information;
• Reviewing relevant program documentation and a sample of project files;
• Benchmarking with comparable country programs;
• Conducting interviews with the Colombia program staff, management, as well as with key personnel in corporate units and in comparable country programs; and
• Conducting field visit to interview program staff, donors and partners and to make on-site project visits.

Appendix B: Lists of Projects Sampled

Appendix C:  Management Action Plan

Audit Recommendation: 1. The Director of Cooperation of the Colombia Development Program should ensure that project file documentation follows the appropriate information management life-cycle, including collection, organization and disposition.

Management Action Plan:

The Colombia Program will develop an Information Management Plan, in compliance with DFATD recordkeeping directives and the TBS GC Directive on Recordkeeping and with the support of the Information Management and Technology Bureau, that includes the following elements:

1. Staff training in Information Management (IM) processes and responsibility (documented business activities; Information resources generated through the activities; retention and disposition)
2. Review of the IM integrity of all Colombia Program documents (including project, analysis, finance and administration)
3. IM management commitments to be included in all Colombia Program staff Performance Management Plans (PMPs) for 2015-2016

Responsible: Director, Colombia Development Program (NDO)

Expected Completion Date: December 1, 2015

Audit Recommendation: 2. The Assistant Deputy Minister Americas should conduct a human resources analysis to confirm resource requirements in support of future Program sustainability.

Management Action Plan:

1. Americas (NGM) will request that a human resources analysis be completed for the Americas Programming Bureau (NDD) to confirm resource requirements for development programs in the Americas.   Implementation of the resulting recommended actions will depend on the results of corporate processes such as the management review of decentralisation and the NGM Optimisation of Missions Resources Review being conducted in the context of amalgamation.  Therefore, implementation of the outcome of the human resources analysis cannot be considered part of the management response to the Colombia Program Audit. 
2. In response to identified resource requirements for the Colombia development program, two (2) additional resources were identified for creation as part of the 2015-2016  mission planning exercise (actual creation of positions is dependent on capacity to offset these additional costs).  Furthermore, implementation of a Field Support Services (FSS) initiative in Colombia is planned for 2015-16, which will contribute to future program sustainability.

Responsible:

1. ADM, NGM
2. DG, NDD

Expected Completion Date:

1. End of June 2015
2. March 2016