Archived - Audit of Investment Planning Report

Table of Contents

Executive Summary
1.0 Introduction
1.1 Background
1.1.1 The Investment Plan
1.1.2 Key Agency Organizational Players
1.1.3 The Investment Planning Process
1.1.4 Investment Planning Process Improvements
1.2 Objective
1.3 Scope
1.4 Methodology and Approach
1.5 Statement of Assurance
2.0 Findings and Recommendations
2.1 Governance Structure and Related Governance Mechanisms
2.1.1 Acquired Services
2.1.2 Clarification of Investments that should be managed through the ePMF
2.1.3 Improved IMIT Branch Interface for IT Enabled Business Initiatives
2.2 Effectiveness of the Investment Plan
2.2.1 Performance Measurement System
Appendix A: Audit Criteria
Appendix B: Investment Governance Board Responsibilities
Appendix C: Management Response and Action Plan

Executive Summary

The evolution of the Canadian Food Inspection Agency's (the Agency) investment planning process and related Investment Plan has been influenced by two major factors - one internal and one external.

Internally, in December 2010 the Agency senior management approved the CFIA Project Management Governance and Policy Framework. To implement this policy the Agency developed the Enterprise Project Management Framework (ePMF) to provide common guidelines for project management.

Externally, the Treasury Board (TB) introduced two new policies. The first, Investment Planning - Assets and Acquired Services was approved by TB in June 2007 and became effective April 1, 2012. The second, the Management of Projects also took effect April 1, 2012.

Bringing together the Agency's new policy with those of the TB and implementing an investment planning process across the Agency represents an extensive challenge. Currently, the fundamentals of a sound management control framework for investment planning are in place. The Agency recognizes that numerous issues must be addressed to further improve the efficiency and effectiveness of the process. During the course of the audit we found that some improvements had been completed.

As stated above, the investment planning process and related governance structure have been developed in response to major changes introduced by the Agency itself and by the TB; these on-going changes affect all branches and organizational levels within the Agency. Given the magnitude of the change, the investment planning process should be viewed as evolutionary and maturing. The Agency is currently adopting process changes, and at this point in time it recognizes that further improvements are required to make the investment planning process more efficient and effective.

The objective of the audit was to provide assurance that a management control framework is in place to effectively support integrated investment planning in a manner that is compliant with applicable Government of Canada requirements. The Agency's Audit Committee approved the audit as part of the 2014/15 to 2016/17 Risk-Based Audit Plan.

Conclusion

The audit identified that the CFIA has a management control framework in place to effectively support investment planning in a manner that is compliant with applicable Government of Canada requirements. A governance structure, methodologies to ensure proposals are aligned with strategic priorities and risk, and monitoring and reporting processes are in place. Nonetheless, the audit also identified opportunities for improvement in the following respects:

• The need for the Investment Governance Board (IGB) to develop direction for the Agency in order to effectively implement the acquired services section of the TB Investment Planning - Assets and Acquired Services policy;
• The need to develop a methodology to better define what initiatives should be managed through the ePMF;
• The need to improve the interface between branches sponsoring IT enabled business initiatives and the Information Management and Technology Branch (IMITB); and
• The development of a performance measurement system suited to assessing the efficiency of the investment planning process and the effectiveness of the Investment Plan as a whole.

Action in these areas will promote an enhanced level of integration in the investment plan and for the investment planning process across the Agency.

1.0 Introduction

1.1 Background

1.1.1 The Investment Plan

The TB policy on Investment Planning - Assets and Acquired Services requires the Agency to submit an investment plan to the TB at least every three years. The plan must clearly set out Agency priorities and strategies for the upcoming five-year period, and outline a three-year investment function that meets the needs of the Agency within available resources. Investment planning is described as the function of allocating and reallocating resources to new and existing assets and acquired services that are essential to program delivery. Acquired services are defined as services obtained through formal arrangements, such as contracts, memoranda of understanding, and letters of agreement, to support internal or external clients or stakeholders in achieving specific outcomes.

The Agency submitted its first plan to the TB in March 2012. The plan covered the five-year period from April 1, 2012 to March 31, 2017. The next formal Agency Investment Plan is due in early 2015 and will cover the five-year period April 1, 2015 to March 31, 2020. On its own accord, the Agency updates its Investment Plan annually and provides these updates to the TB.

The Agency's first plan included investments in four asset categories - Information Management and Information Technology, Real Property, Fleet (vehicles) and Laboratory and Scientific Equipment. In accordance with new related TB policy, the Agency included a fifth investment category, Acquired Services. Exhibit 1 below summarizes where the Agency allocated its resources in its first Investment Plan:

1.1.2 Key Agency Organizational Players

While most of the Agency's branches play some part in the investment planning process, there are four branches whose directorates and divisions plan and "own" the majority of the investments included in the plan. In the case of the Corporate Management Branch (CMB), in addition to planning for and "owning" a significant portion of the investments, it is also responsible for the corporate level investment planning management and control processes. The other three branches which plan for and "own" significant portions of the Plan are the Information Management and Information Technology Branch (IMITB), the Science Branch and the Operations Branch.

The CMB is responsible for managing the Agency's budget, including the allocation of capital and operating resources to individual investment proposals approved by the Investment Governance Board (IGB). The VP, CMB leads the Agency-wide annual investment planning process and presents the annual Investment Plan for approval to the Agency's Senior Management Committee chaired by the President. Within the CMB, the Investment Planning & Enterprise Project Management Office (IP&ePMO) acts as the authority and center of excellence for project management. While its primary role is to foster and to promote the adoption of project management principles into the strategic and operational activities of the CFIA, it also plays a key role in the investment planning process. It is also responsible for producing the monthly Executive Project Dashboard for investments that the IGB has deemed as projects.

The CMB's Resource Management and Financial Services directorates maintain overall control of the Agency's capital and operating funds. They provide the initial allocation of funding to branches with individual IGB approved investments. At mid-year, these directorates in conjunction with the branches provide the IGB with the financial information necessary to reallocate resources amongst individual investments where the rate of spending is less than or more than forecast. Further, the CMB provides Financial Management Advisors (FMAs) to branches. The FMAs are responsible for providing support to branches in financial matters.

The CMB's Real Property and Environmental Protection directorate is responsible for the provision and management of multiple types of facilities (owned and leased) which support the Agency's Science, Operations and other branch staffs located across Canada.

The IMITB invests and manages the Agency's information technology infrastructure which centrally supports the Agency's operational and administrative functions. The IMITB also plays a role in providing support to other branches that have investment proposals requiring IT enablement. These proposals are referred to as "IT enabled business initiatives". Further, the IMITB provides personnel to serve branches in the role of Client Relationship Managers (CRMs). CRMs are expected to play a technical advisory role in support of branch planned IT enabled business initiatives.

The Operations Branch employs the majority of Agency staff. These employees primarily serve the Agency's diverse inspection function. The majority of Operations investments are IT-related, some of which seek to modernize the tools which inspectors use in the field, while others modernize the technological interface between the Agency and its clients.

The Science Branch conducts research and testing in support of the Agency's mandate in a number of laboratories across the country. The branch's annual investment proposals are related to science instruments and laboratory equipment that either require replacement or provide new or improved capabilities.

1.1.3 The Investment Planning Process

The investment planning process is complex with many sub-processes and numerous participants at both the branch and corporate levels. However, it is essentially a decision-making process that is designed to make investment choices that reflect the Government of Canada's priorities and the Agency's own Long Range Strategic Plan and Corporate Risk Profile. The management control framework is embedded in the investment planning process, which is currently under revision.

During the period of the audit, we found that the Agency was in the midst of making substantive changes to the investment planning process. The revised investment planning process requires each branch to submit a VP approved Investment Proposal Form (IPF) to support each of its "ideas". The IPF describes how each idea will contribute to Agency Strategic Outcomes, the primary reason for undertaking the potential project, a description of the business need/problem, the project proposal description, the proposal outcomes/benefits, who is affected, the dependencies, and the key dates.

The IP&ePMO then convenes a Working Group consisting of branch representatives who work together in completing a scoring template which provides the methodology for prioritizing ideas.
For those ideas with the highest priority ranking, the sponsoring branch must submit a VP approved Project Proposal Form (PPF) to the IGB at Q3 (November/December) which provides the Business Line priority, a Risk Rating analysis, and a cost estimate.

At Q4 (January/March), the IGB formally approves the final investments to be included in the Investment Plan as either initiatives or projects. During Q4, the IGB also allocates the budgets for these approved investments and determines the required governance oversight.

The level of monitoring and reporting that an investment is subject to is dependent on whether or not it is considered a project. Project managers must submit monthly project reports and individual project dashboards to the IP&ePMO. The IP&ePMO consolidates these project reports on an Executive Dashboard for review by members of IGB. All other approved initiatives are expected to be monitored by the branch concerned. In addition, a consolidated report on the Investment Plan is prepared by CMB-Resource Management and presented at mid-year and again at year end. CMB has recognized that financial monitoring of the Investment Plan could be improved, and has made significant advances in developing the tools to do so.

1.1.4 Investment Planning Process Improvements

The audit covered the period from April 1, 2013 to September 30, 2014. During this period, we found there had been a number of improvements identified to improve the efficiency and effectiveness of the investment planning process. Some of these improvements have been completed.

Completed improvements include the following:

1. Revised the Terms of Reference for the Investment Governance Board to ensure: all Treasury Board policy provisions are addressed by committee, decision-making authorities are clearly articulated, roles and responsibilities are clearly spelled out, and the appropriate changes to committee membership and frequency have been included as directed;
2. Senior Management Committee approved that that the IGB now has full authority over the Agency's Project Development Fund (used to fund pre-planning of investment proposals);
3. The development of a tool to monitor, control and track financial decisions that impact the Investment Plan; and
4. IGB formal overall approval for the timeline revisions to the Annual Investment Planning Cycle.

1.2 Objective

To provide assurance that a management control framework is in place to effectively support integrated investment planning in a manner that is compliant with applicable Government of Canada requirements.

1.3 Scope

The audit included significant investment planning activities from April 1, 2013 to September 30, 2014. The audit did not perform detailed testing of processes related to Fleet acquisitions. This area is of lower materiality, and Planning Phase review indicated management controls are in place to support sound decision making.

To further clarify the scope of the audit, it must be understood that not all investments become projects. Agency senior management makes the decision as to which investments are to be managed as projects. The audit focused on investment planning and not project management. Therefore, the scope of the audit does not include project planning, project execution, project close-out or post launch review.

The audit commenced in April 2014 and the field work was completed in October 2014.

1.4 Methodology and Approach

Lines of enquiry and audit criteria (see Appendix A) were developed to serve as standards against which our assessment could be made, and form a basis for the conduct of the audit. The criteria were developed from the Treasury Board's Policy on Investment Planning - Assets and Acquired Services and related directives, and the Office of the Comptroller General's Audit Criteria related to the Management Accountability Framework: A Tool for Internal Auditors.

The audit criteria were organized under three lines of enquiry (LOE):

LOE 1: A governance structure and related governance mechanisms are in place to support investment planning.

LOE 2: Formal processes have been designed and are in place to support the production / annual refresh of the Agency's Investment Plan.

LOE 3: Investment Plan monitoring activities and financial & non-financial reporting provide senior management with sufficient information to reallocate funds and to measure the effectiveness of the Investment Plan.

In order to examine the processes in place for different classes of investments, a sample of sixteen (16) investments was selected on a judgmental basis. Of the sixteen, four projects are subject to full ePMO oversight, eight are projects which did not meet the criteria for full oversight and four "unapproved" proposals were selected to ensure consistency of decision making.

1.5 Statement of Assurance

The audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of CFIA's Internal Audit quality assurance and improvement program. Sufficient and appropriate auditing procedures were performed and evidence gathered in accordance with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing to provide a high level of assurance over the findings and conclusion in this report. The findings and conclusion expressed in this report are based on conditions as they existed at the time of the audit, and apply only to the entity examined.

2.0 Findings and Recommendations

2.1 Governance Structure and Related Governance Mechanisms

Governance bodies associated with the investment planning process exist at both the branch and corporate levels.

At the corporate level, the Senior Management Committee, chaired by the President and composed of branch Vice-Presidents, provides final approval of the Investment Plan. The Investment Governance Board, the chief decision-making authority on planned investments, is chaired by the VP, CMB and membership includes branch Vice-Presidents. This Committee's chief responsibilities can be found in Appendix B.

2.1.1 Acquired Services

We expected that the Agency would have developed a comprehensive rationale to support its interpretation of acquired services. Under the TB Policy on Investment Planning - Assets & Acquired Services, the President is responsible for effective investment planning including the consideration of alternative and innovative options for meeting asset and service requirements through the use of internal and external delivery models and a range of instruments.

We found that, despite the fact that acquired services is the largest single amount on the CFIA 2012/13-2016/17 Investment Plan at $161.5 million over 5 years, the Agency has yet to develop a comprehensive rationale to support its interpretation of acquired services. We found that the amount included in the Plan was developed from a review of past expenditures as opposed to a conceptual and rational methodological approach.

Without clear direction derived from a comprehensive and rational approach, the inputs to the acquired services section of the Investment Plan may be inconsistent or incomplete. Further, there is a risk that the Agency will be found not to have fulfilled the accountabilities established under the TB Policy on Investment Planning - Assets & Acquired Services.

Recommendation (1): The VP, CMB should develop the internal direction necessary to effectively implement the acquired services section of the TB Policy.

2.1.2 Clarification of Investments that should be managed through the ePMF

We expected that the Agency would have a mechanism through which investment proposals could be "filtered" to ensure that investments that were of low risk would not necessarily be managed as projects through the ePMO.

The Agency invests in four types of assets - Fleet, Science and Laboratory Equipment, Real Property (RP) and Information Management and Information Technology (IMIT). These types of assets are capital in nature and as such are included on the Investment Plan. For instance, Fleet assets (vehicles) are generally acquired on a replacement basis through the Agency's procurement system. Similarly, a high percentage of scientific and laboratory equipment acquisitions represent the replacement of existing assets which have exceeded their economic useful lives. While these types of acquisitions require branch and corporate investment planning and management, they do not require ePMF project management.

Within certain limits, we believe the Agency should also consider some RP infrastructure replacements as not requiring project management through the ePMF. The August 31, 2014 Executive Project Dashboard includes the following 10 RP Projects:

The first two modernization projects clearly meet the definition of a project as a series of activities that has a beginning and an end and that will produce defined outputs and realize specific outcomes in support of a public policy objective, within a clear schedule and resource plan.

The remaining projects are all described as either replacements (7/8) or remediation (1/8). Unless a replacement adds significant new functionality to the asset being replaced, these types of investments, although capital in nature, do not meet the definition of a project. Nothing new in terms of outputs or benefits is being created. Further, these replacements represent regularly recurring operational maintenance activities that should not be classified as projects on the IP, nor subjected to the rigours of the ePMF.

Recommendation (2): The VP, CMB should determine whether the costs, benefits and risks of managing Real Property replacement projects through the ePMF are justified.

2.1.3 Improved IMIT Branch Interface for IT Enabled Business Initiatives

According to the ePMF, branches developing investment proposals are to engage the enabling branches as early as possible in the investment planning process. With the technological complexity involved in the development of IT enabled business initiatives, we expected that the IMITB (the enabling branch) would have prepared a directive to assist branches in preparing for the initial engagement session. We also expected that the roles and responsibilities of IMITB would be formally documented and include the basics of their responsibilities in relation to IT enabled business initiatives.

We did not find any such IMITB formal direction nor any documented roles and responsibilities.

We found that the IMITB had begun a Work Intake Review Process (WIRP) to identify a process to manage the initiation, intake, and estimation of new work items before they are given formal acknowledgement via established governance processes. The WIRP as drafted serves to partially clarify the interface areas in which their roles and responsibilities need to be developed and refined. The WIRP also proposes the steps that IMITB might undertake in its review of any IT enabled business initiative which the IGB has decided warrant further development.

Further development of the WIRP would improve the transparency and likelihood that the most important of branch sponsored IT enabled initiatives are successfully concluded. Using the WIRP to develop the appropriate roles and responsibilities and process solution warrants consideration.

Delays caused as a result of a lack of clarity of process or roles and responsibilities can give rise to the risk that investment ideas important to the Agency are not moving forward efficiently.

Recommendation (3): The VP, IMITB in collaboration with the VP, CMB should ensure that the ePMF intake and IMIT intake processes are integrated such that the responsibilities of branches that sponsor IT enabled business initiatives are clearly articulated.

2.2 Effectiveness of the Investment Plan

2.2.1 Performance Measurement System

We expected, as is required by the TB Policy on Investment Planning - Assets and Acquired Services, that the Agency would have in place a performance measurement system that would provide measurements as to the efficiency and effectiveness of the investment planning process and the effectiveness of the overall Investment Plan.

We found that although the Agency has some performance measurement information from mid-year and year-end reviews as well as from ePMF project close-out reports, it has yet to develop the measures necessary to comply with the TB policy.

The TB policy requires the Presidentto effectively manage the investment planning function including developing an investment plan, exercising oversight in the implementation of investment decisions, and ensuring appropriate ongoing measurement of investment performance. Further, the TB Guide to Investment Planning and Acquired Services requires the Agency to provide evidence to the Treasury Board of Canada Secretariat (TBS) that the investment planning policy's specific monitoring and reporting requirements are being met, which includes measuring and documenting the performance of investment planning and investments.

Without a performance measurement system in place, the Agency is not able to assess either the efficiency and effectiveness of the investment planning process or the overall effectiveness of its Investment Plan. The IP&ePMO staffs recognize the need to develop and implement a performance measurement system.

Recommendation (4): The VP, CMB should ensure that a performance measurement system is developed and designed to provide the performance measures necessary to assess the efficiency and effectiveness of the investment planning process and the annual overall performance of the Agency's Investment Plan.

Appendix A: Audit Criteria

Line of Enquiry 1: A governance structure and related governance mechanisms are in place to support investment planning.

Criteria 1: Governance and oversight of the investment planning process and the investment plan are in place and effective at both the branch and corporate levels.

Line of Enquiry 2: Formal processes have been designed and are in place to support the production / annual refresh of the Agency's Investment Plan.

Criteria 2: Guidance provided to prepare investment proposals and time allowed is sufficient for branches / business areas to understand and act upon requirements.
Criteria 3: Investment proposals are aligned to Agency and Government of Canada strategic priorities and demonstrably respond to Agency risks.
Criteria 4: Approved investments are adequately supported through resource allocation and financing.
Criteria 5: Process is in place to ensure investment proposals reflect the Agency's highest priorities.

Line of Enquiry 3: Investment Plan monitoring activities and financial & non-financial reporting provide senior management with sufficient information to reallocate funds and to measure the effectiveness of the Investment Plan.

Criteria 6: Investment Plan monitoring activities and financial & non-financial reporting provide senior management with sufficient information to reallocate funds.
Criteria 7: Reallocations are appropriately justified & done in a timely manner in order to avoid lapsing funds.
Criteria 8: Investment Plan reporting provides senior management with sufficient information to measure the effectiveness of the Investment Plan.
Criteria 9: Formal, periodic reports are reviewed and appropriately approved.

Appendix B: Investment Governance Board Responsibilities

The Investment Governance Board is chaired by the VP, CMB and membership includes branch Vice-Presidents. This Committee's chief responsibilities include (but are not limited to):

• Acting as a decision-making authority on planned investments in assets, acquired services and projects;
• Prioritizing investments and determining which can be undertaken and which cannot, taking into consideration affordability, sustainability and capacity of the complete life-cycle of each investment, at the planning stage;
• Ensuring investments are aligned with the strategic objectives and priorities within the Agency and Government of Canada;
• Overseeing the monitoring and reporting on investments, including ongoing measurement of investment performance and planning regime;
• Providing strategic advice to the SMC on effective investment planning and allocation/re-allocation for inclusion in the five-year Investment Plan:
  • Project investments (Information Management Information Technology (IMIT), Real Property and business projects including those that are asset enabled and recommended by the asset level governance);
  • Asset Acquisition investment plans (IMIT, Fleet, Scientific and Laboratory Equipment); and
  • Acquired Services investment strategies in Real Property, IMIT, Fleet and Scientific Equipment, and business initiatives;
• Providing ongoing advice to TBS on the Agency's Investment Plan (address any significant changes to content, performance, resource allocation, and/or requirement for the TB approvals).

Appendix C: Management Response and Action Plan