Audit of the Maternal, Newborn and Child Health Commitments

Global Affairs Canada
Office of the Chief Audit Executive

December 2016

Executive Summary

The Audit of the Maternal, Newborn and Child Health commitments was included in the Global Affairs Canada 2015-2018 Risk-Based Audit Plan. The plan was recommended by the Departmental Audit Committee and approved by the Deputy Minister on March 30^th, 2015. 

Since 2010, Canada has taken leadership in a global action to end the preventable deaths of mothers, newborns and children. Further to the 2014 announcement of the renewed commitment of $3.5 billion to Maternal, Newborn and Child Health (MNCH), there has been a shift in Global Affairs Canada’s (the Department) priorities. The mandate letter for the Minister of International Development asked for a refocus of Canada’s development assistance on reproductive rights and health care for women that is driven by evidence and outcomes. Global Affairs Canada launched a public review and consultations to renew Canada’s international assistance policy and funding framework. The aim of the International Assistance Review (IAR) is to rethink Canada’s international assistance policies and programs to better respond to the challenges and opportunities of the new global context. Results from the IAR process will help to refocus Canada’s international assistance and determine Canada’s approach internationally to supporting the 2030 Agenda on Sustainable Development.

Why is this audit important?

Maternal, newborn and child health is a key Canadian development priority. In June 2010, the G8 Summit launched the Muskoka Initiative on MNCH, with an aim to accelerate progress on improving women’s and children’s health in developing countries. Under the Muskoka Initiative, Canada committed $2.85 billion over a five-year period, from 2010 to 2015. In 2014, Canada pledged an additional $3.5 billion at the International Summit in Toronto to help shape the future of child and maternal health from 2015 to 2020.

Global Affairs Canada advances Canada’s values and interests internationally, delivers international programs, and administers Canada’s international assistance program to reduce poverty in the developing world and provide humanitarian assistance. The Ministers of Foreign Affairs and International Development co-manage, with the Minister of Finance, the resources of the International Assistance Envelope.  This envelope is a dedicated pool of resources used by the Government of Canada to fund the majority of the federal international assistance.

The Department has five thematic priorities for international development; one includes securing the future of children and youth. Global Affairs Canada supports MNCH through the Children and Youth sub-program of the International Development program.

MNCH represents significant multilateral and bilateral investments. For the last five years, the Department has spent over $600 million^{Footnote 1} annually on MNCH. In the 2014-15 and 2015-16 fiscal years, the scope of this audit, the Department disbursed $1.23 billion^{Footnote 2} in MNCH programming. For these two years, it represented respectively 20%^{Footnote 3} and 19%^{Footnote 4} of the Department’s international development expenditures. MNCH is funded through transfer payments (i.e. grants and contributions) from the International Assistance Envelope (IAE) and is reported as Official Development Assistance (ODA).

What did we examine?

The audit examined the operating procedures, processes, and controls in place for the management and delivery of MNCH programming under the two commitments for both Headquarters administered and decentralized projects. This was accomplished by assessing projects funded through multilateral or bilateral investments, or co-investments in partners’ commitments; and managed within the Department’s existing programming. The audit examined the governance structure and management accountabilities, controls, guidance and tools, expert capacity, as well as performance measurement and reporting.

The audit covered both MNCH commitments for the years 2014-2015 and 2015-2016.

What did we find?

Based on interviews with key departmental personnel at HQ and Missions, documentation review and project file examination, the audit found that:

• A governance structure and different levels of committees are established.
• There is limited documentation to demonstrate oversight at the operational level over the MNCH commitments.
• Roles and responsibilities, including reporting for MNCH commitments, are not well understood among key MNCH stakeholders within the Department.
• For the 2015-2020 MNCH commitment, the process and decisions made on funding allocation were not well communicated to program-level staff.
• MNCH financial performance is monitored and reported on, financial management is sound, controls are consistently applied, and funds were spent in accordance with the terms and conditions of the grants and contributions agreements and arrangements.
• Project management practices and controls are established. Monitoring and reporting practices at the project level are generally strong. However, there were some gaps in documenting evidence of due diligence and in retaining project documentation.
• There is limited MNCH-specific training available offered to both program officers and health specialists to build their capacity. In the absence of MNCH-specific training, branches have organized informal training or information sharing sessions related to MNCH.
• The Department developed a draft Performance Management Strategy for the 2015-2020 commitment, however, at the time of the audit, it had yet to be finalized and approved.
• At the departmental level, no performance results for the MNCH commitment were reported in 2014-2015.

Recommendations:

Based on the findings listed above, the audit team made the following recommendations:

1. The Assistant Deputy Minister, Global Issues and Development should:
   1. clarify accountability and responsibilities for department-wide MNCH commitments, especially for performance reporting; and
   2. ensure that coordination and communication with all departmental MNCH stakeholders takes place on a regular basis.
2. The Assistant Deputy Minister, Global Issues and Development, in collaboration with KFM and Geographic branches, should communicate the process and decisions with regard to MNCH funding allocations to program staff for effective planning of their resources.
3. The Assistant Deputy Minister, Global Issues and Development, in collaboration with KFM and Geographic branches, should ensure that evidence of due diligence with regard to MNCH projects is documented and that key documentation is retained.
4. The Assistant Deputy Minister, Global Issues and Development should develop tools and guidance for MNCH program officers to assist them in MNCH programming; and in collaboration with the Chief Financial Officer, provide necessary training on MNCH coding to ensure completeness and consistency of data reporting.
5. The Assistant Deputy Minister, Global Issues and Development, in collaboration with KFM and Geographic branches, should finalize and approve a performance management strategy that includes a results framework and performance indicators for the current MNCH commitment to measure results for reporting. As well, the Assistant Deputy Minister, Global Issues and Development, in collaboration with KFM and Geographic branches, should implement a more efficient reporting mechanism that allows for a consolidation of MNCH performance results across the Department.

Statement of Conformance

In my professional judgment as Chief Audit Executive, this audit was conducted in conformance with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing and with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program. Sufficient and appropriate audit procedures were conducted, and evidence gathered, to support the accuracy of the findings and conclusion in this report, and to provide an audit level of assurance. The findings and conclusion are based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed upon with management and are only applicable to the entity examined and for the scope and time period covered by the audit.

________________________________
Chief Audit Executive

1. Background

The Audit of the Maternal, Newborn and Child Health commitments was included in the Global Affairs Canada 2015-2018 Risk-Based Audit Plan. The plan was recommended by the Departmental Audit Committee and approved by the Deputy Minister on March 30th, 2015.

Maternal, Newborn and Child Health (MNCH) is a key Canadian development priority. Since 2010, Canada has taken leadership in a global action to end the preventable deaths of mothers, newborns and children. In June of that year, the G8 Summit launched the Muskoka Initiative on MNCH, with an aim to accelerate progress on improving women’s and children’s health in developing countries. Under the Muskoka Initiative, Canada committed $2.85 billion^{Footnote 5} for a five-year period, from 2010 to 2015. In 2014, Canada pledged an additional $3.5 billion^{Footnote 6} for the 2015 to 2020 period at the International Summit in Toronto to shape the future of children and maternal health.

Canada’s Forward Strategy on MNCH, developed in 2014, introduced the government’s renewed commitment toward MNCH for 2015 to 2020. The goal of this commitment is to end preventable deaths of mothers, newborns and children under the age of five in select developing countries. Canada’s investments are tailored to specific country needs, and prioritize those countries and issues where concrete results can be attained for vulnerable women and children. Canada’s contribution for both MNCH commitments concentrate on three paths: strengthening health systems, reducing the burden of diseases, and improving nutrition. The renewed commitment for 2015-2020 also includes a focus on improving accountability for results to better track progress and ensure investments are making a difference. Since 2015, the renewed MNCH commitment also incorporates an increased focus on sexual and reproductive health rights.

Global Affairs Canada is the primary Department responsible for the delivery of the Government of Canada’s renewed MNCH commitment. Of the $3.5 billion for 2015-2020, approximately $3.39 billion^{Footnote 7} is managed by Global Affairs Canada. The remaining $110 million^{Footnote 8} is managed by Finance Canada.

Global Affairs Canada has focused on five thematic priorities for the delivery of its international assistance. One of the thematic priorities is to secure the future of children and youth. The Department supports MNCH through the Children and Youth sub-program of the International Development program. The overall goal of the Children and Youth sub-program is to contribute to the well-being and empowerment of children and youth in targeted developing countries. The objectives of this sub-program encompass the goals of the MNCH commitments. They are to:

• encourage efforts to improve child and maternal health by ensuring children have the best possible start to life, beginning with healthy mothers;
• improve the equitable access of children and youth to quality education and learning opportunities, with a special emphasis on girls; and
• help ensure the rights and protection of children and youth, particularly girls, from violence, exploitation, and abuse.

MNCH projects are delivered through multilateral or bilateral investments, or co-investments in partners’ commitments, and within the Department’s existing programming. While projects are implemented by executing agencies and partners, including governments, they are managed by program officers located either in Headquarters or directly in the field by mission-based teams. About 78%^{Footnote 9} of MNCH projects are administered at Headquarters.

The MNCH Commitments’ Management and Administrative Structure

Within the Department, the Social Development and the Health and Nutrition bureaus, under Global Issues and Development (MFM), are responsible for the Children and Youth program. MFM is responsible for the overall policy and strategic direction of the MNCH commitments. The Health and Nutrition bureau (MND) is the focal point for communication and coordination with all six programming branches that manage and deliver MNCH programming. These branches are:

1. Global Issues and Development (MFM);
2. Partnerships for Development Innovation (KFM);
3. Sub-Saharan Africa  (WGM);
4. Asia-Pacific (OGM);
5. Americas (NGM); and,
6. Europe, Middle East and Maghreb (EGM).

These branches deliver international development assistance via transfer payments, through different funding channels. In addition to its policy role, MFM is also responsible for multilateral programming. It provides international aid through the support of multilateral organizations. KFM supports development projects that are designed and implemented by Canadian organizations, in cooperation with their counterpart organizations in developing countries. The geographic programs (EGM, OGM, NGM, and WGM) are responsible for bilateral development assistance programming. This bilateral assistance supports recipient country and regional plans and works with Canadian, multilateral, international and local partners, including NGOs, local governments and the private sector. All six programming branches contribute to delivering MNCH projects.

MNCH projects are delivered within the above mentioned branches’ existing programming. To plan, implement and administer the programs, the six branches develop and implement projects that adhere to the programs’ thematic focus and priorities, as set out in their strategies and to departmental policies, directives and guidelines.

Audit of the MNCH Commitments

The audit examined the governance, performance management, processes and controls in place for the management and delivery of the MNCH commitments, for projects managed at both Headquarters and at Missions. The audit team also took into consideration the Department’s MNCH formative evaluation which covered the first four years of the Muskoka Initiative, from 2010-11 to 2013-14. The formative evaluation focused on the Muskoka Initiative’s relevance, effectiveness, efficiency, cross-cutting issues, sustainability, and performance management. Its overall conclusion was that the Department’s MNCH Initiative has effectively contributed to the UN Global Strategy for Women’s and Children’s Health and is on course to deliver upon the Muskoka commitment of the Government of Canada. In contrast and to complement the evaluation, this internal audit focuses on the adequacy and effectiveness of the management control framework over the MNCH commitments. It includes the operating procedures, processes, and controls in place for the management and delivery of the MNCH commitments. The audit covered the 2014-15 and 2015-16 fiscal years. 

In the 2014-15 and 2015-16 fiscal years, the Department disbursed $1.23 billion^{Footnote 10} for the MNCH commitments. During these two years, there were 354 MNCH projects ongoing across the Department. The following charts illustrate the distribution of MNCH disbursements as well as the number of projects delivered per branch, for these two fiscal years.

The following table represents the distribution of grants and contributions for MNCH projects for the period under audit.  

Source: MNCH data is provided by Corporate Planning, Finance and IT (CFO Branch) as of September 2016.

*Contribution: A contribution is a conditional transfer payment subject to performance conditions specified in a funding agreement. It is used to contribute to approve proposals from eligible recipients. A contribution is to be accounted for and is subject to audit.

**Grant: A grant is a transfer payment subject to pre-established eligibility and other entitlement criteria. A grant is not subject to being accounted for by a recipient, nor normally subject to audit by the Department. The recipient may be required to report on results achieved.

2. Observations and Recommendations

Audit results are derived from examination of documentation, interviews conducted and file review. The audit team reviewed key documentation including relevant policies and guidance, strategic and planning documents, performance reports, and committee terms of references and minutes. The team conducted 36 group and individual interviews with employees involved in MNCH programming across the Department at Headquarters (HQ), and in Missions, in person and via videoconference. It also interviewed two recipient organizations. The file review was comprised of 26 project files and covered 42 transactions. The audit team also took into consideration the Department’s MNCH formative evaluation which covered the first four years of the Muskoka Initiative, from 2010-11 to 2013-14.

Based on this work, observations and recommendations were developed under the themes of: governance and accountability; delivery and administration; guidance, tools and capacity; and performance management and reporting.

2.1 Governance and Accountabilities

The audit team examined whether a governance structure is established and provides oversight and strategic direction for the MNCH commitments. We also examined whether accountabilities, authorities, and responsibilities are clearly defined, documented and communicated to key Department stakeholders.

Governance Structure and Oversight

The 2015 mandate letter to the Minister of International Development and La Francophonie committed to a government that is accountable to Canadians, that tracks the results and reports on the progress toward fulfilling the Department’s commitments. The audit team expected to find an established governance structure providing oversight over the MNCH commitments to ensure effective coordination and to support decision-making. To do so, the audit team examined documentation and interviewed key personnel for both the Muskoka Initiative and the renewed commitment. The review of documentation indicated that programming for MNCH is delivered through existing departmental program structures; MFM is the policy, thematic coordination and communication lead for the MNCH commitments. Within MFM, the Health and Nutrition bureau (MND) is the focal point for MNCH strategic policy, advice and analysis on the status of the commitments.

The audit team found that, for both the Muskoka Initiative and the renewed MNCH commitment, a governance structure and different levels of committees were in place to ensure oversight. With the Executive Board at the strategic and decision-making level, the following key committees oversee the implementation of the MNCH commitments:

Oversight Level

• Programs Committee – The mandate of this committee is to ensure consistency and rigor in program design, delivery and performance across all the programs of the Department, with particular reference to grants and contributions programs funded by the International Assistance Envelope. The Programs Committee is chaired by the Assistant Deputy Minister (ADM) of MFM and its members cover a variety of programs. The committee provides leadership, guidance and advice on programming issues for the Department; monitors the Department’s implementation of international programming commitments from a strategic perspective; and advances coherence between programs and policy, including thematic areas of focus.
• Resource Management Committee – The mandate of this committee is to provide strategic direction and oversight on the alignment of financial, non-financial and human resources with departmental priorities. Asia-Pacific (OGM) chairs the committee. The Resource Management Committee oversees the alignment of investment planning against departmental priorities and the international financial assistance commitments.
• Assistant Deputy Minister MNCH Committee - An ADM Committee, which consists of all ADMs with MNCH responsibilities, meets on an ad hoc basis. It is chaired by MFM. Discussions and issues raised in the MNCH Working Group and DG Ad-hoc committee are brought up to the ADM committee for decision-making.

Operational Level

• Director General Programs Committee (DGPC) – The mandate of this committee is to facilitate coherent and effective programming practices, and consider issues related to programming coherence and effectiveness. The committee provides advice to the Evaluation Committee, Resource Management Committee and Programs Committee. It is responsible for overseeing the whole-of-department implementation of international programming key deliverables and deadlines, and providing updates on key operational and coherence practices. This forum serves in sharing information regarding the MNCH commitments.
• Director General MNCH Committee – An ad hoc DG MNCH Committee, which consists of DGs with MNCH responsibilities, meets on an ad hoc basis to address key issues related to the MNCH commitments.

Working Level

• MNCH Working Group – From October 2014 to July 2015, an informal MNCH Working Group led by Geographic Coordination and Mission Support division (NMD) provided communication and coordination across branches on MNCH programming. In January 2016, an MNCH departmental Working Group was formed as a formal consultative forum intended to improve coordination amongst relevant branches working on Canada’s $3.5 billion MNCH commitment (2015-2020) and to ensure coherent advice and effective implementation of activities and reporting related to MNCH. The working group is chaired by Maternal, Newborn and Child Health and Nutrition division (MNC), joined by up to two members of each programming branch with MNCH responsibilities (KFM, MFM, OGM, NGM, and WGM).

The audit noted that different levels of committees were established. However, the number of Programs Committee meetings where MNCH and its performance was an item for discussion was limited. The records of decisions for the Programs Committee showed that the last documented discussion on MNCH was dated January 2014. Furthermore, while there was a discussion regarding the performance data for international development, which links to MNCH, no records of follow-up actions or decisions were found.

Similarly, at the operational level, the audit team found some evidence of meeting documentation from August 2015 to September 2016. For the same period, while both the DGPC and DG committee were used as information sharing forums, records of decisions showed that updates on the MNCH were only provided at four DGPC meetings. Other documentation reviewed showed limited management control over the status and progress of the MNCH commitments. Considering the significance of MNCH, a documented decision-making is important to demonstrate the existence of oversight, and to ensure evidence of the provision of strategic direction.

At the working level, interviews with program officers and management indicated that the informal MNCH Working Group worked well and included all key MNCH stakeholders in the Department. It provided coordination and communicated key information. As mentioned, the new MNCH departmental Working Group replaced the previous informal MNCH working group in January 2016, but, meetings have been held infrequently. Program officers and management interviewed had mixed views regarding the effectiveness of the communication of decisions: some interviewees indicated that information-sharing worked well while many others felt that information sharing could be improved by providing timely documentation. The audit team was advised that the cause of the infrequent meetings is attributable to a high turnover rate of staff within MND.

Accountability, Roles and Responsibilities

Clearly established roles and responsibilities are essential for ensuring that effective program implementation, monitoring and reporting requirements are met. In order to support the effective delivery of the MNCH programming and projects, the audit team expected to find defined accountabilities for the MNCH commitments, that are in line with roles and responsibilities, and that are shared with employees.

The Treasury Board (TB) ‘Instructions to Departments for Developing a Management, Resources, and Results Structure (MRRS)’ indicates that the Responsibility Position has overall responsibility for overseeing the program and for collecting and assessing its actual versus its expected results. In July 2016, the new TB Policy on Results replaced the TB Policy on MRRS. The essence of the policy remains unchanged as compared to previous versions. According to the TB Policy on Results, the Department is required to designate an official who is responsible for establishing, implementing and maintaining program performance information and ensure its data collection.

The departmental Program Alignment Architecture (PAA) identifies the Director General of Social Development (MGD) and Director General of Health and Nutrition (MND), within MFM, as the Responsibility Positions for MNCH. In this role, MFM is expected to collect and assess MNCH performance information against expected results. This responsibility for overall reporting was also confirmed by the Corporate Planning, Performance and Risk Management bureau (SRD).  However, the audit team was advised by MNCH programming branches that they each are accountable for reporting on MNCH results at the project level. At the departmental level, there is a lack of clarity with regard to accountability for reporting on the MNCH commitments. Thus, there is a risk that the Department cannot demonstrate the progress and results of the renewed MNCH commitment and report on the achievement of departmental development priorities.

In conclusion, while governance structure and different levels of committees were established, improvements are required in clarifying and communicating roles and responsibilities to ensure cohesion and coordination of the renewed MNCH commitment. As well, the responsibility for reporting on MNCH performance needs to be clarified and communicated to all departmental MNCH stakeholders.

Recommendation #1:

The Assistant Deputy Minister, Global Issues and Development should:

1. clarify accountability and responsibilities for department-wide MNCH commitments, especially for performance reporting; and
2. ensure that coordination and communication with all departmental MNCH stakeholders takes place on a regular basis.

2.2 Delivery of the MNCH Commitments

The audit team examined whether MNCH programming is designed to support the delivery of the commitments and whether MNCH projects were delivered according to applicable authorities and policies. The audit team also examined if controls in place ensure that the management of grants and contributions for the MNCH commitments are in line with applicable policies and legislation.

Project Planning and Approval Process

Since 2013, the planning and approval process for all development projects of the Department has followed the Authorized Programming Process (APP), the Department’s standardized approach to programming aid initiatives, from the planning phase to implementation.

MNCH projects are delivered through multilateral or bilateral investments or co-investments in partners’ initiatives, and within the Branches’ existing programming. Projects are managed either at Headquarters or at Missions. Projects decentralized to Missions are managed by the program officers located in decentralized country programs and supported by their HQ counterparts.

A review of documentation and interviews with key personnel indicated that for the Muskoka Initiative, there were clear funding allocations for each of the ten country programs; however the project selection and resource allocation under the renewed commitment were done through the Department’s investment planning process. Specifically, the six programming branches each completed an exercise of program and project planning in line with agreed notional allocations. Within the branches’ authorities and with respect to their own funding channels, development programs submitted investment plans to senior management for their respective division. For example: geographic country programs planned bilateral investments in line with their country strategies; KFM launched a call for proposals to work with Canadian organizations; and MFM renewed and signed agreements with international organizations.

The audit team found evidence of a fund allocation approach for both MNCH commitments. Funding allocation was discussed in ADM meetings and presented to the Minister of International Development for approval. For both MNCH commitments, each programming branch was allocated a portion of the MNCH funds and given a specific disbursement target for the five-year period. For the 2010-2015 commitment, MNCH program staff interviewed indicated that they were aware of the branch targets and that there was a process for funding allocation. However, for the 2015-2020 renewed commitment, interviews conducted with MNCH program staff indicated that they were not aware of branch allocations or how decisions were made with regard to funding. Better communication of the funding allocation process and resulting decisions would allow branches to plan their resources more effectively.

Recommendation #2:

The Assistant Deputy Minister, Global Issues and Development, in collaboration with KFM and Geographic branches, should communicate the process and decisions with regard to MNCH funding allocations to program staff for effective planning of their resources.

Project Management and Controls Pertaining to Applicable Policies and the Authorized Programming Process

The audit team assessed the project management practices and controls in place for the delivery of the MNCH commitments by examining a sample of 26 MNCH projects managed from both Headquarters and missions, and by the six programming branches involved with MNCH. The APP is the standardized process used by the Department and it has embedded controls within each phase of the project lifecycle. The audit team examined this process in assessing decisions made and documentation in place for the project planning, due diligence, agreement structure, terms of payment, eligibility and accuracy of expenditures, as well as monitoring requirements and practices.

Overall, the file review confirmed that project management practices and controls were generally both established and implemented according to applicable policies and the APP.  The audit team came to this conclusion based on the following observations of the project files examined: a standard template was used to prepare the agreement in 100% of the projects; 100% of the files included a payment schedule, and eligible costs were defined in 94% of the cases. As well, project activities and expected results were described in the agreement respectively for 92% and 88% of the projects examined.

The auditors’ analysis of the financial transactions concluded that financial management was sound and that controls were consistently applied to ensure that the funds were spent in accordance with the terms and conditions of the agreement and with relevant policies. Furthermore, the testing of the eligibility and accuracy of expenditures revealed that all were eligible costs, and in line with the approved budget. All expenditures were incurred once the agreement was signed and during the period claimed. Also, all expenditures reviewed for professional services and travel covered completed activities and for which a contract was properly granted. Expenditures examined represented accurate amounts and were accurately reported in the financial system (SAP). Finally, all travel expenses analyzed in our sample were in compliance with the Department’s travel policy and 100% of the sampled expenses were in compliance with the terms and conditions of the signed agreement.

For project risk assessments, the file review demonstrated that the programs were in compliance with the requirements. For example, the audit tests confirmed that forms for consultation with gender equality and environment specialists were found on file for 96% of the projects, and a form for the fiduciary risk assessment tool (FRET) was found for 95% of the files examined.

The audit team’s review of project files demonstrated that improvements are needed to better document decisions. While all 26 projects had a grant or contribution agreement that was signed by the appropriate authority, 36% of these projects’ approved initiative assessments were not found on file. The initiative assessment gives explicit assurance that financial and contracting officers exercised due diligence in financial and contracting matters. Similarly, in 19% of the sampled projects, the audit team did not find evidence on file that a contracting officer reviewed the approval memo. In addition, for the projects at the implementation stage, 20% of files reviewed were missing reporting documentation. This matters because almost all of these gaps refer to contribution agreements, which are conditional transfer payments. The Department’s Due Diligence Reference Guide for Program Managers states that program Officers and Managers at HQ and in the field are all bound by the elements of the due diligence process. They work together to ensure that all of the elements of due diligence are addressed throughout the life cycle of any project or program.

Based on the APP requirements, each project must be entered in the Grants and Contributions System (GCS) and have a work plan created. The work plan includes a list of pre-determined steps and tasks. Maintaining the work plan and updating the planned activities and actual dates is required. Program officers can refer to the work plan to update the program manager, director or team of the project progress. Keeping the work plan current also facilitates resource forecasting for all team members. The file review indicated that all projects assessed in our sample were entered into the GCS. However, none of the work plans were updated with actual dates of completion. Therefore, according to the work plan in the system, it is not clear whether the project milestones were achieved.

To conclude, the audit team found that project management practices and controls were generally established and implemented according to applicable policies and the APP; and that financial management was sound. However, the file review showed weaknesses in retention of key documents. Improvement in retaining project documentation is needed to avoid inefficient work or duplication of effort that may occur due to the time associated with searching for documents or the effort spent on preparing the documentation again, when needed. Furthermore, the documentation is evidence of sufficient work completed to meet the requirements of the project. Given the departmental context of employee mobility and rotation, where projects may be managed by a number of program officers throughout their lifecycle, complete documentation will ensure efficiency and evidence of due diligence and retention of corporate memory.

Recommendation #3:

The Assistant Deputy Minister, Global Issues and Development, in collaboration with KFM and Geographic branches, should ensure that due diligence with regard to MNCH projects is exercised and that key documentation is retained.

2.3 Guidance, Tools and Capacity Supporting the Delivery of MNCH

The audit team assessed whether tools and guidance were developed to support the delivery of the MNCH commitments in a consistent manner. We also examined whether MNCH capacity was appropriately designed to support the delivery of the MNCH commitments.

Support for the Delivery of MNCH Programs and Projects

Program officers manage development projects, including MNCH. They rely on the expertise of thematic specialists, including gender equality and health specialists at HQ and through locally engaged staff based at Missions and consultants that work at Project Support Units (Field Support Services), to assist them in the assessment of proposals and review of project performance reports. Further to the field-based specialist support, KFM and other programming branches rely on program officers and locally engaged staff to review proposals, monitor projects and bring partners together to ensure effective knowledge management. Decentralized geographic programs manage programming directly from the field. Field officers have equal need for support in tools and training as their HQ counterparts, but will have different access to training and resources than HQ colleagues.

At HQ, MNCH programming staff has limited support from health specialists; there are four within the Department that provide support to all development programs, including MNCH. Having the support from a specialist is one way to obtain expert capacity to deliver the MNCH programming. Another is to acquire knowledge through guidance, tools and training.

1. Guidance and Tools

MNCH is a Global Affairs Canada’s international development commitment and as such, follows standardized procedures. Development projects are managed according to the Department’s APP. Guidance and tools have been developed in support of the APP. Review of documentation identified that the policies, programming process, guidance, tools and templates are available online. As well, Terms and Conditions for International Development Assistance, a set of parameters under which transfer payments may be made for a given transfer payment program, are also available online.

The audit team found that there is limited guidance developed specifically for the MNCH commitments. For example, guides were prepared on how to capture key performance indicators and on the reporting of results but these tools date back to 2013 and are not up-to-date with the renewed commitment (2015-2020). At the time of the audit, progress was being made to renew some of the guidance to bring it in line with the new commitment to reflect the new Minister’s priorities. For example, definitions on sexual and reproductive health rights were prepared to address the new Department priorities. As well, guiding frameworks for maternal and child health were drafted. However, this guidance has yet to be finalized and shared within the Department. Therefore, without up-to-date and consistent guidance, program officers reported that they do not have all the tools they need to develop programming for the MNCH commitments.

In addition to the corporate investment guides for APP and pre-APP projects, a coding guide was developed by the Department to help program officers capture all reproductive, maternal, newborn and child health (RMNCH) spending by development partners and to track RMNCH resource flows. Although this guide was put in place, interviews with program officers and managers indicated a disparity in the level of awareness of guidance and tools related to MNCH coding. Coding provides information about how the investments are categorized based on elements such as MNCH sectors, business delivery models, investment attributes, and geographic scope. Coding investments provides the Department with accurate data for reporting and decision-making purposes.

Adequate tools and guidance would enhance knowledge and skills required of program officers to effectively manage the MNCH projects.

2. Training

The Department offers a number of training opportunities to program officers and other staff members through the Canadian Foreign Service Institute (CFSI) and the Canada School of Public Service (CSPS). Departmental staff has access to a variety of courses, related to project management such as risk management, result-based management, and gender equality in programming. Training notices are posted on the Department’s Intranet in the Training and Learning section and they are communicated via Broadcast messages to all employees on a regular basis.

The audit team’s review of documentation and online training indicated that the CSPS and the CFSI course catalogues do not include courses on health programming nor on MNCH-specific subjects. In the absence of MNCH-specific training, several branches have organized informal training or information sharing sessions related to MNCH. For example, Partnerships for Innovation (KFM) organized an orientation session in preparation for the assessment regarding the Call for Proposals. Sub-Saharan Africa (WGM) and Asia-Pacific (OGM) have also hosted a few information sessions regarding human rights, gender equality and ongoing initiatives. MND assisted with organizing an internal and external learning event related to the findings of the formative evaluation of the MNCH.

Some program officers and health specialists expressed a need and an interest for more MNCH training, such as training on MNCH coding, and refreshers to keep current in the areas of health and nutrition. This is particularly important in the context of staff mobility and rotation, where program officers, who may not necessarily possess a background in MNCH programming, change positions on a regular basis and need to become familiar with the subject matters they are assigned to work on. Without MNCH-specific training to support the delivery of the MNCH commitment, program officers may not have the support they need to discharge their responsibilities related to MNCH projects.

Recommendation #4:

The Assistant Deputy Minister, Global Issues and Development should:

1. develop tools and guidance for MNCH program officers to assist them in MNCH programming; and
2. in collaboration with the Chief Financial Officer, provide necessary training on MNCH coding to ensure completeness and consistency in data reporting. 

2.4 Performance Measurement and Reporting

The audit team examined performance reporting activities to determine whether they support informed decision-making with regard to the achievement of the MNCH commitments. We also examined whether a performance measurement strategy was in place for the commitments and if performance information was sufficient and appropriately documented to track progress against the expected results.

MNCH Financial Performance Monitoring

To ensure that a grant and contribution program is managed and delivered consistently, controls for performance monitoring and reporting must be in place to measure progress towards planned results. The audit examined whether the Department’s financial information was collected to track the progress of MNCH.

The audit team found that the Department has mechanisms and tools in place that help manage and closely monitor the MNCH disbursements. A review of documents showed that reports are produced on a monthly and quarterly basis to keep track of and to report on the financial results of the MNCH commitment. The auditors also reviewed two annual reports that present a summary of international assistance activities: the Report to Parliament on the Government of Canada’s Official Development Assistance and the Statistical Report on International Assistance. These two reports consist mostly of financial performance information.

MNCH Project Performance Monitoring

According to the APP, the project team leader should monitor actual performance against planned results on an ongoing basis to ensure that the project is on track. As well, at the outset of each project, it is important to establish performance objectives, targets, and indicators. This is done through the development of two tools: the logic model and the Performance Measurement Framework (PMF).

The file review of select projects showed that logic models and PMFs were developed for 100% of the projects examined. As well, evidence that performance reports and audit reports were reviewed by project team leaders was found on file for almost all projects, respectively 86% and 89%. Similarly, our file review showed that program officers have analyzed and documented all instances where significant variance related to progress was observed and where unplanned events occurred. Finally, a management summary report, summarizing the project performance, was completed for all implemented projects. Overall, the file review indicated that project monitoring and reporting practices are strong.

MNCH Performance Management Strategy and Reporting

According to the TB Policy on Transfer Payments, the Department is required to establish a Performance Measurement Strategy (PM Strategy) for a transfer payment program, such as MNCH, at the time of program design, and ensure that it is maintained and updated throughout the life cycle of the program. A PM Strategy assists in: ensuring that adequate and relevant information supports the program monitoring for results; making informed decisions and taking appropriate timely action; and providing effective departmental reporting. To report on the 2010-2015 Muskoka Initiative, MFM developed a Performance Management Strategy (PMS), including a Performance Measurement Framework (PMF^{Footnote 12}) and logic model. However, at the time of the audit, the PMS for $3.5 billion for the 2015-2020 period had yet to be finalized and approved. In the absence of a formal PMS, the Department’s achievements may not be fully accounted for and reporting information available may be limited for the Department to report on results achieved on MNCH commitments.

Furthermore, interviews with program officers and managers indicated that in the absence of an up-to-date and approved PMF, they use different MNCH performance indicators. Some program officers use indicators from the 2010-2015 PMF to support the development of their projects and while others use the draft version of the 2015-2020 PMF. This inconsistent use of indicators could result in difficulties in rolling up performance information. It may also impact the ability of senior management to make fully informed decisions across all projects, programs and elements of the MNCH commitments and to manage them in a timely and comprehensive manner. 

The audit team’s review of documentation found that examples of project-level results were collected and reported to senior management. However, at the departmental level, no performance results for the MNCH commitment were reported in 2014-2015. For the Children and Youth sub-program, which includes MNCH, instead of reporting results, the Department reported on the number of programs and projects contributing to the performance results.

With regard to internal performance reporting, the audit team found limited documentation with the exception of one progress report prepared in 2013 linked to the first MNCH commitment. The audit noted that some branches took the initiative of compiling results and preparing progress reports for their own purposes and needs. This process involves a manual roll-up of data and a significant commitment of resources. These challenges affect the Department’s efficiency in data collection and reporting. The absence of a mechanism to aggregate the data impacts the Department’s ability to report on MNCH results in a comprehensive and strategic manner, linked to the priorities of the Department. Collecting sufficient performance information would contribute to the new priority of accountability for results, which is one focus of the 2015-2020 MNCH commitment.

Recommendation #5:

The Assistant Deputy Minister, Global Issues and Development, in collaboration with KFM and Geographic branches, should finalize and approve a performance management strategy that includes a results framework and performance indicators for the current MNCH commitment to measure results for reporting. As well, the Assistant Deputy Minister, Global Issues and Development, in collaboration with KFM and Geographic branches, should implement a more efficient reporting mechanism that allows for a consolidation of MNCH performance results across the Department.

Conclusion

Despite the MNCH commitments’ complexity and magnitude, the audit team noted that a governance structure and different levels of committees were established, and that MNCH financial performance was monitored and reported. At the project level, management practices and controls were established and implemented; project monitoring and reporting practices were strong; financial management was sound and controls were consistently applied; and funds were spent in accordance with the terms and conditions of the agreements.

Opportunities for improvement were noted in the areas of: operational-level oversight; responsibility for reporting; communication of funding allocation; document retention; capacity building; and performance management.

Appendix A: About the Audit

Objective

The objective of this audit was to assess the adequacy and effectiveness of the management control framework over the MNCH commitments and its delivery, including projects managed at Headquarters and those decentralized in Missions abroad.

Scope

The audit examined the operating procedures, processes, and controls in place for the management and delivery of the MNCH program for both Headquarters administered and decentralized projects. The audit covered both MNCH commitments: 2010-2015 (MNCH 1.0) and 2015-2020 (MNCH 2.0). More specifically, the audit covered the last year of MNCH 1.0 and the first year of MNCH 2.0.

The audit did not examine the MNCH projects delivered by other Canadian government departments.

Criteria

The following criteria were developed based on the controls expected to be in place to manage high risk areas, and are designed to conclude on the audit objective within the defined audit scope.

Approach and Methodology

The Audit of the MNCH Commitments was in accordance with Treasury Board Policy on Internal Audit, Directives, and the Internal Audit Standards for the Government of Canada and conforms to the International Standards for the Professional Practice of Internal Auditing of the Institute of Internal Auditors.

In order to conclude on the above criteria and report against the audit objective, the following methods were used:

• Identify and review relevant policies, directives, and guidance;
• Review and analyze documentation including financial and non-financial information;
• Perform walk-throughs of key controls and processes;
• Interview key MNCH personnel at HQ and in selected Missions (including video and tele conferences); and
• Analyze management practices from selected Missions and review both centralized and decentralized MNCH projects.

Select Projects

The audit tested key controls and processes for selected projects that, for the period of April 2014 and March 2016, were either operational or terminating for MNCH 1.0, and approved for MNCH 2.0, (see table below). For the last year of MNCH 1.0, projects selected were drawn from the new funding of $1.1 billion for the Muskoka commitment and excluded projects from the existing baseline funding. The reason for this is that the new funding focused on the programming objectives agreed to in the 2010 G8 Summit: strengthening health systems; improving nutrition for mothers, newborns and children; and reducing the disease burden. Also, to ensure that aid was focused and could deliver maximum results, the Muskoka funding was primarily allocated to ten countries of focus with high rates of maternal and child mortality. In addition, Global Affairs Canada was responsible for programming all of the projects under the new funding, whereas for the baseline funding, other government departments were also responsible for programming. The 26 projects selected for file review are listed in Appendix B.

Appendix B: Projects Selected for File Review

Appendix C: Management Action Plan

The MNCH Formative Evaluation (2015) underscored that the successful implementation of the Muskoka Initiative was due to the decentralized decision-making structure for the MNCH commitment. The structure placed accountability for MNCH programming with responsible geographic and multilateral branches. This Management Action Plan builds on the decentralized governance structure, with MFM providing overall policy leadership for the MNCH commitment, and a stronger horizontal leadership, coordination and communication role. The goal is to ensure efforts across the Department are aligned and mutually reinforcing. Direct responsibility for MNCH programming will continue to be decentralized across the department through existing lines of accountability to the DME and Minister. This is to ensure that ADM accountabilities are not duplicated. As such, ADMs with specific accountabilities for MNCH related to their overall responsibilities are partners in the implementation of the Management Action Plan.

Appendix D: Acronyms