Evaluation of the CFIA's Food Safety Program Modernization - Part 1
Final Report

4.0 Findings

The evaluation identified a number of project-specific observations, as well as observations that were similar across projects.

This section uses consolidated findings tables to present findings from the evaluation (Table 6 and Table 7). The intent is to allow readers to quickly identify the similarities and differences among the projects, and to facilitate general statements about the FSMI as a whole. A summary of observations precedes each table.

Observations included in the consolidated findings tables were compiled from information collected through all of the evaluation's data collection activities. A list of the documentation used to support the development of the tables is included in Appendix A.

4.1.1 Need and Objective: There is a continued need for FSP and a demonstrated need for the FSMI.

4.1.2 Project Design: The design of individual FSMI projects is aligned with overall project objectives.

4.1.3 Priorities, roles, and responsibilities: The FSMI supports government-wide and CFIA priorities. It will enhance how the CFIA carries out its activities.

4.2.1 Implementation and Outputs: Despite some delays, project activities are producing their respective outputs, but there is a lack of performance measurement to track the initiative's effects on the FSP.

4.2.1.1 Dependencies: Project delays are largely a reflection of associated FSMI dependencies.

4.2.1.2 Challenges: Communication and stakeholder buy-in are common challenges across all FSMI projects.

4.2.2 Economy: Financial data is in line with project delays, but it is projected most funding will be spent within the five-year timeframe ending in fiscal year 2015-16. Remaining funding has been extended to fiscal year 2017-18.

4.2.2.2 Efficiency: There is minimal evidence to support the efficiency of FSMI projects, in part due to delays; however the fact that all projects were either implemented or are scheduled to be completed without significant overages provides reasonable evidence of efficiency.

4.2.3 Outcomes: Generally, most FSMI initiatives are at too early a stage to report on outcomes. However, there is no evidence of a plan to track to track the initiative's effects on Agency programming.

4.1 Relevance

4.1.1 Need and objectives

Overall, the evaluation found there is a continuing need for the FSP, as it supports the CFIA's strategic outcome and plays a key role under food safety legislation. The need for FSMI supports the need for modernizing FSP. As well, the needs and the objectives of each of the FSMI projects are well aligned. However, the information available about the FSMI projects suggests a lack of or limited emphasis on linking the FSMI objectives with the broader needs for the FSP.

The activities of the FSP strive to ensure Canadians have access to safe food (including imported food), and that the food we export is safe. The continued need for FSP is supported by evidence of the need for a stronger focus on prevention, a stronger focus on compliance, increased efficiency, and improved consistency for FSP. These needs point to broad outcomes, such as modernizing in order to strengthen the FSP, which the FSMI is striving to achieve.

This is supported by the Government of Canada's commitment to strengthen food safety in Canada by allocating funds directly for the purpose of modernizing the food safety system through inspection and science (Government of Canada, 2011). The CFIA's Long Term Strategic Plan and Corporate Risk Profile further support the need for the FSP, and place significant importance on modernizing the program.

While the FSP plays an important role in ensuring safe food in Canada, a number of factors continue to impact the effectiveness of its activities. These include:

  • changes in the global supply chain and the volume of trade
  • changes in consumer behaviour and expectations
  • advances in science and technology
  • changes to international regulations
  • the changes in the structure of the industry (CFIA, 2012b; CFIA, 2013j)

The CFIA's need to address these risks supports the relevance of the FSMI, which focusses on inspection effectiveness, scientific capabilities, IM/IT infrastructure, and transparency and leveraging relationships with consumers and industry. Under this initiative, the CFIA is modernizing the FSP through, a stronger focus on prevention and compliance, a citizen-centered service culture, optimizing performance, building capacity, and modernizing tools (CFIA, 2013j).

In many instances it is possible to understand the needs targeted by FSMI projects as the intended individual project outcomes. These projects share similar collective outcomes that support the overarching needs for FSP. For example, the need for a consistent approach to conducting inspections is being addressed by the design and implementation of a single inspection approach and a consistent approach to recruitment and training across all food commodities. Together, these activities address the need for consistency in the approach of the FSP. The relationship between the need for FSP and the need for FSMI, as observed during the evaluation process, underpins the relationship between the implementation of the FSMI projects and their eventual influence on the FSP and further Agency programming.

Table 6 demonstrates that the needs and objectives of each of the FSMI projects are closely aligned. For example, in the case of the Improved Food Inspection Model (IFIM) project, the need for consistency of inspection through a systems-based approach aligns well with the project's objective of shifting to an audit-based inspection ap proach that is based on the implementation of hazard control plans.

4.1.2 Project Design

The evaluation found individual FSMI projects were designed to meet overall objectives and were based on existing evidence, theories and experience.

In some cases, as with ESDP and IFIM, the Agency has looked to other jurisdictions, such as the United States, the Netherlands, Australia, and New Zealand for evidence of project similarity and success. These reviews examined not only the applicability of other IM/IT solutions and approaches, but also the success of alternative approaches to inspection in other jurisdictions. In the case of IFIM, proven industry practices have also been incorporated into project design. This includes moving toward using established health and safety plan approaches, as well as International Organization for Standardization (ISO) standards for control procedures.

Others, like the FSIN project, have leveraged concepts from similar projects that have been implemented domestically. The FSIN project is a clear example of how theory provides a strong justification for the project. Specifically, the project's approach rests on the idea that building a laboratory network will allow better information sharing, will lead to better and more readily-available food safety data. This will support a more preventative approach to food safety. In certain cases, such as with the Modernizing Equipment and Laboratories (MEL) project, activities represent an extension of the ongoing work of the Agency, allowing the project to leverage this existing experience. It is clear that a strong theoretical justification for the work remains.

Table 6 indicates that individual FSMI projects have all leveraged existing evidence, theories, or experience bases in their design.

4.1.3 Priorities, roles, and responsibilities

The evaluation found FSMI contributes to a variety of federal government and Agency priorities such as ensuring a healthy Canadian population and the Government of Canada's Blueprint 2020 vision. Evidence from the evaluation also suggests the FSMI could improve the FSP's ability to support the Agency's strategic outcome. The mandate for providing safe food will not change; however, under FSMI the way inspectors conduct their business will be different.

Overall, FSMI projects align with federal priorities. For instance, all projects support the federal priority of a healthy Canadian population. Many FSMI projects also support priorities outlined in the Government's Blueprint 2020 vision, which calls for a modern and service-oriented federal government. For example, the investments in IM/IT and ESDP represent a significant step toward better use of technology and improving service delivery. Individually and collectively, FSMI projects directly support the Agency's ability to meet its strategic outcome.

The FSMI projects will not substantially change the Agency's roles and responsibilities with respect to food safety; however, some initiatives may result in subtle changes in the way the CFIA interacts with industry, and scientific and international partners. For example, inspection modernization is intended to provide greater clarification around the role of industry vs. the role of the CFIA with respect to food safety. Industry has the principal role in food safety, while the CFIA is responsible for overseeing industry compliance with regulations.

Table 6 presents observations regarding the need, federal government and Agency priorities, project objectives and project design that support the FSP and its modernization through FSMI. This information supports the analysis presented in Section 4.1.

Table 6: Relevance
Description Improved Food Inspection Model (IFIM) Electronic Services Delivery Platform (ESDP) Recruitment and Training Information Management and Information Technology (IM/IT) Developing a Laboratory Network Strategy (CFSIN) Table Note 7 Modernizing Equipment and Laboratories (MEL) Enhanced Laboratory Response Capacity (ELRC)

Need - This describes how each of the individual FSMI projects is intended to strengthen the FSP.

Analysis: Section 4.1.1

  • A stronger focus on prevention.
  • A stronger focus on compliance.
  • A systems- based approach to inspection.
  • Ensuring inspection consistency through a single inspection approach across commodities.
  • Reducing overlap, duplication, and financial burden for industry.
  • Maintaining international obligations and more readily adapting to emerging global and scientific trends.
  • Enhancing service delivery, optimizing performance, and increasing transparency.
  • Moving away from a paper-based record keeping system.
  • Improving inspection delivery service and information exchange with stakeholders.
  • Moving away from a system of independent, commodity-specific inspector training.
  • Optimizing diverse talent supported by modern electronic tools for frontline inspectors.
  • Consistent and standardized training and competencies for inspectorate across Areas and commodities.
  • Targeted recruitment and retention.
  • Timely and continued training in order to keep up with operational changes and an evolving industry.
  • Optimizing program performance.
  • Providing electronic tools.
  • Access to CFIA's secure network for Industry and trading partners.
  • Access to the CFIA network from remote areas for CFIA staff.
  • Increasing speed of connectivity.
  • Increasing the volume of accessible information .
  • Increasing the analytical capabilities of the Agency.
  • Enabling electronic services for industry/ trading partners.
  • A stronger focus on prevention and responsiveness.
  • Enhancing overall capacity and information sharing.
  • Maintaining and enhancing scientific capabilities.
  • Leveraging advances in technology and science.
  • Managing the risks of contaminants and chemical effects in food.
  • Supporting an integrated and multi- jurisdictional food safety surveillance network .
  • Leveraging advances in technology and science.
  • Managing risks associated with new foods.
  • Maintaining and enhancing scientific capabilities.
  • Improving quality management controls within laboratories.
  • Maintaining and replacing equipment near or beyond its life expectancy.
  • Improving food safety through additional science capacity.
  • Managing risks associated with new foods.
  • Maintaining and enhancing scientific capabilities.
  • Leveraging advances in technology and science.
  • Managing an increased volume of global trade.
  • Implementing mandatory scientific and technical training across all food commodities.

Objectives – This row presents the objectives of individual FSMI projects.

Analysis: Section 4.1.1

  • Develop a consistent (single food) inspection approach across all food commodities.
  • Reduce the need for multiple inspections at a single facility.
  • Use an audit-based inspection approach based on hazard control plans.
  • Ensure clear and consistent inspection requirements for industry.
  • Develop a web-based electronic portal for industry to access Agency services such as registration.
  • Ongoing collection and tracking of inspection data and laboratory samples.
  • Enable managers to track inspection activity and to target inspection resource allocation.
  • Make inspector worksheets and daily tasks directly accessible to inspectors in the field.
  • Leverage existing training material to develop an inspector training curriculum aligned with the Agency's new inspection approach.
  • Build a mechanism to support the new consistent training curriculum.
  • Provide consistent training to CFIA inspectors (new and existing) and supervisors that reflects new competencies required by the new inspection approach.
  • Foster a new operational culture.
  • Improve data management through data consolidation.
  • Provide inspectors with tools to support their work and connectivity for remote areas.
  • Develop a plan for a network that would connect/ centralize information from food safety laboratories across Canada.
  • Improve facilities and equipment at Scarborough and St. Hyacinthe laboratories.
  • Purchase additional laboratory equipment to improve response time and laboratory capability.
  • Increase the number of highly-skilled scientists working at CFIA food safety laboratories.
  • Collaborate with academia and universities to enhance knowledge of new technologies and scientific methods.
  • Develop new testing methods.

Design foundations – This row provides information on how individual FSMI projects were designed and developed.

Analysis: Section 4.1.2

  • Established Hazard Analysis and Critical Control Point (HACCP) and International Organization of Standardization (ISO) standards.
  • The successful use of internationally recognized standards for inspection from other jurisdictions.
  • Commercially available off-the-shelf software products with associated with industry standard business processes.
  • Existing CFIA meat processing inspection curriculum.
  • International Food Protection Training Institute in the United States curriculum.
  • Commercially available off-the-shelf software packages with industry standard business processes.
  • The Canadian Animal Health Surveillance Network, and the Canadian Public Health Network as a basis for network planning.
  • Pan- Canadian food safety community and concepts, processes and mechanisms to support a country wide network.
  • Experience in ongoing equipment purchases for the laboratory equipment purchases.
  • Formal assessment of alternative renovation approaches for the two laboratory renovations.
  • Enhancements of ongoing science modernization activities at the Agency.

Alignment with Federal priorities – This row demonstrates how individual FSMI projects link with both federal and CFIA priorities.

Analysis: Section 4.1.3

  • A Canadian population that is healthy.
  • Ensuring that Canadians have access to safe food.
  • Blueprint 2020's goal of a whole government approach to service delivery.
  • A Canadian population that is healthy.
  • Ensuring that Canadians have access to safe food.
  • Blueprint 2020's goal of an open and networked environment that engages stakeholders.
  • Blueprint 2020's goal of a modern workplace that leverages technology.
  • A Canadian population that is healthy.
  • Ensuring that Canadians have access to safe food.
  • Blueprint 2020's goal of a capable, confident, and high- performing workforce.
  • A Canadian population that is healthy.
  • Ensuring that Canadians have access to safe food.
  • Blueprint 2020's goal of a modern workplace that leverages technology.
  • A Canadian population that is healthy.
  • Ensuring that Canadians have access to safe food.
  • Blueprint 2020's goal of a whole government approach to service delivery, an open and networked environment that engages stakeholders.
  • A Canadian population that is healthy.
  • Ensuring that Canadians have access to safe food.
  • Blueprint 2020's goal of a modern workplace that has smarter new technologies.
  • A Canadian population that is healthy.
  • Ensuring that Canadians have access to safe food.
  • Blueprint 2020's goal of a modern workplace that leverages technology.
  • Blueprint 2020's goal of a capable, confident, and high- performing workforce.

Roles and responsibilities and strategic outcome – This row links individual FSMI projects to the Agency's strategic outcome, and describes how projects will impact stakeholder relations.

Analysis: Section 4.1.3

  • Improved ability to meet the strategic outcome.
  • Improved clarity between the Agency and industry with regards to inspection requirements.
  • Improved ability to meet the strategic outcome.
  • Improved working relationship with existing industry stakeholders and international partners.
  • Improved ability to meet the strategic outcome.
  • Improved ability to meet the strategic outcome.
  • Improved ability to meet the strategic outcome.
  • Improved working relationship with existing laboratory and scientific partners.
  • Improved ability to meet the strategic outcome.
  • Improved ability to meet the strategic outcome.

Table Notes

Table Note 7

Developing a laboratory network strategy project is now known as the Canadian Food Safety Information Network

Return to table note 7  referrer

4.2 Performance

4.2.1 Implementation and outputs

The evaluation found that despite some delays, all project activities are progressing along their intended plans and are producing outputs. The alignment between individual project designs and their expected outcomes suggest that projects will eventually meet their outcomes. From the analysis the evaluation infers that delayed projects will be successfully and fully implemented as planned. However, the evaluation did not find evidence of performance measures to track how individual projects will impact the FSP once they are fully implemented.

There were a number of activities planned under each of the FSMI projects. For example, in the case of the IFIM project, planned activities included:

  • identifying common food safety objectives, strategies, and processes;
  • developing a draft inspection model; and
  • eventually developing and implementing a new inspection model.

Progress was made in all FSMI projects, and measurable outputs were produced. For example:

  • the CFSIN planning stage was completed
  • the MEL project completed its planned equipment purchases and is scheduled to complete its laboratory renovation work by March 2016
  • recruitment and training activities were implemented well within the five-year FSMI project timeline, with the delivery of coreFootnote 8 and refresher training to inspectors activities such as the PREP program and refresher training having largely met their delivery targets.

Certain aspects of other projects have yet to be completed. For example, the design and implementation of ESDP has been delayed by about two years. As well, IFIM - now being expanded to include commodities beyond the FSP - was only in the early stages of implementationFootnote 9 at the time of the evaluation. The standardized inspection approach had been validated in 125 facilities across the fish, feed, dairy, and greenhouse sectors, and the Agency was in the process of implementing it nationally for those commodities. Subsequent sectors are expected to follow a similar pattern, until all commodity areas have been transitioned over to the common inspection approach.

4.2.1.1 Dependencies

The evaluation found project delays to be in part a reflection of the number of dependencies associated with the FSMI projects.

Delays in the implementation of the IFIM [which includes its IT component Food Inspection Modernization System (FIMS)] and the ESDP are good examples of projects being interdependent.

The FIMS and the ESDP are closely related, and progress in one is often dependent on the other. This dependency led to the decision to merge these two projects under the FSMI. However, both also had associated dependencies outside the scope of the FSMI. For example, effective implementation of the IFIM is supported by the regulatory change underway at the Agency.

Other dependencies, such as buy-in and participation with respect to enhancing science capacity, had fewer consequences, as projects are on track to be completed within the original five-year timeframe. For example, the MEL project was dependent on approval by Health Canada for extra space, the capital equipment procurement process, and PWGSC project management.

4.2.1.2 Challenges

The evaluation found communication and stakeholder buy-in to be common challenges for the FSMI projects. This poses difficulties around fostering a culture of change and ensuring an integrated and measurable approach to support the momentum of long-term project and program goals.

Interviewees were asked to speak to the challenges faced by each of the FSMI projects. Many of the challenges identified in the consolidated findings table are common across other ongoing change activities within the CFIA. For example, common challenges include:

  • obtaining staff, industry, and international stakeholder buy-in for changes in the Agency operations
  • maintaining the momentum of long-term projects, particularly in the face of staff turnover among those responsible for their implementation
  • understanding and accommodating the dependencies between change activities when attempting to improve programming at the CFIA

These challenges, along with project delays and observations made by interviewees during the evaluation, point to the need for:

  • ongoing communication with stakeholders prior to, during, and after change activities;
  • fostering a culture of change at the Agency; and
  • undertaking a more integrated approach to individual change-related projects.

Building momentum appears to be an important aspect of successful change activities, and the need to do so appears even more acute given implementation delays in some of the FSMI projects noted above.

Table 7 presents observations regarding the implementation, production of outputs, achievement of outcomes, and efficiency of FSMI projects. This information supports the analysis presented in Section 4.2.

Table 7: Performance
Description Improved Food Inspection Model (IFIM) Electronic Service Delivery Platform (ESDP) Recruitment & Training Information Management & Information Technology (IM/IT) Developing a Laboratory Network Strategy (CFSIN) Table Note 10 Modernizing Equipment & Laboratories (MEL) Enhanced Laboratory Response Capacity (ELRC)

Planned activities - This row described the planned activities for individual FSMI projects. These activities link directly to the needs and objectives of FSMI.

Analysis: Section 4.2.1

  • Identify common food safety objectives, strategies, and processes.
  • Develop a strategy and engage internal and external stakeholders and subject matter experts.
  • Complete draft improved inspection delivery model.
  • Explore IM/IT options and solutions to support the inspection business process.
  • Develop preliminary project plan.
  • Seek expenditure authority and begin execution of IM/IT component.
  • Develop implementation strategy and implement the improved inspection delivery model.
  • Completion of supporting IM/IT solutions.
  • Perform a project risk and complexity assessment.
  • Validate business requirements, including IM/IT infrastructure.
  • Develop preliminary project plan.
  • Develop detailed project management plan and seek expenditure authority.
  • Develop ESDP system.
  • Inform industry of new system.
  • Train staff on ESDP.
  • Adapt existing training materials into a six-week training program.
  • Build mechanisms for implementing core and refresher training.
  • Update six-week core training program (PREP) to reflect the competencies required by the improved inspection delivery model and all new and existing inspectors will receive training.
  • Core training delivered to new inspection staff: ongoing.
  • Refresher training to existing staff: ongoing.
  • Access e-learning specialists and contractors and ensure capacity to support demand for online training and learning.
  • Initial planning and implementation of technology foundation pieces for increased connectivity; more modern tools; clarification of required support for new inspection model, and business foundations.
  • Negotiate and acquire additional data storage and backup.
  • Improve data and information management capabilities.
  • Pilot technology that can be implemented immediately.
  • Deploy new tools and devices.
  • Implement plan for ever- greening of technology and business foundations, given continual advances in technology.
  • Assemble small team and work in collaboration with partners to establish support for the creation and operation of a network.
  • Explore with experts the concepts, processes, and mechanisms available to conduct a lab systems analysis.
  • Engage IM/IT.
  • Analyse laboratory systems with partners: examine and profile potential Canadian laboratories as contributors for the Laboratory Network across food safety authorities.
  • Develop a strategy by combining common vision and value proposition.
  • Develop a plan for a food safety information network initiative.
  • Consultations to undertake a functional program assessment and options analysis.
  • Formal selection process for identifying the laboratory re-design and purchase decisions.
  • Purchase new microbiology equipment.
  • Complete construction.
  • Hire new scientists to respond to demands for food safety testing.
  • Develop academic and university partnerships to enhance training and knowledge of new technologies and scientific methods.
  • Develop research reports on novel rapid testing methods.

Measurable outputs produced - This row provides evidence of outputs achieved by individual FSMI projects.

Analysis: Section 4.2.1

  • Completed new inspection model - single food program.
  • Wave 1 implementation of the IFIM underway with limited mobile tools support.
  • System planning complete with development and implementation forecasted for 2017 - 2018 completion.
  • Core and refresher training implemented.
  • Implementation of supervisory schools.
  • Created IM/IT business function.
  • Enhanced data storage, backup, and information management capabilities.
  • Put in place remote hardware tools to support IFIM.
  • Completed plan for FSIN.
  • Laboratory renovations set for completion in March 2016.
  • Completed planned equipment purchases.
  • Completed planned hiring.
  • Testing development ongoing.

Dependencies - This row provides evidence of what each project's success is dependent on.

Analysis: Section 4.2.1.1

  • Development and implementation of electronic workbooks on handheld devices and remote connectivity for inspectorate on-site access.
  • ESDP capacity and capability to house inspection data and target inspection need.
  • Implementation of new food regulations to support the IFIM.
  • Development and implementation of the IFIM.
  • Development of core IM/IT functionality and business processes.
  • The roll out (waves) of the new inspection model.
  • Development of an enhanced IM/IT business function.
  • Buy-in and participation by provincial and territorial partners.
  • Participation from Co-location partner in shared departmental facilities.
  • Ongoing technology change and improvements in scientific approaches and testing procedures.

Challenges to project success - This row describes individual project challenges.

Analysis: Section 4.2.1.2

  • Significant project management requirements.
  • Reliance on other FSMI project completion and compatibility..
  • IM/IT privacy and security standards.
  • Acquiring buy-in from all inspectors and Agency staff.
  • Implementation of new regulations to better support the requirements of the new inspection model - e.g., audit-based recordkeeping by industry.
  • Maintaining the necessary momentum to fully implement the model.
  • Implementation is a long-term activity with success affected by attrition among key staff present during the IFIM development.
  • Expectation that ESDP could be developed independently of without underlying processes - e.g., IFIM.
  • Compliance with modern standards of privacy, business continuity.
  • Need for a continual working relationship with other governments regarding acceptance of ESDP system as information transfer system for international trade.
  • Integration of ESDP with other business lines.
  • Scope creep Table Note 11 during project implementation.
  • Business processes to appropriately design elements of ESDP.
  • Integration of work with other projects such as IFIM.
  • Confusion between the ESDP "project" and ESDP "platform".
  • Adopting Agency-wide culture change to support transformation.
  • Developing a new hiring process without additional funding for hiring.
  • Releasing inspectors from duties to obtain training - all levels.
  • Integrating previous work with new systems - LMS with ESDP.
  • Lacking an enhanced IM/IT business function at the start of the project.
  • Getting buy-in from all of the necessary stakeholders.
  • Using non-custodial space for laboratories and its impact on renovation planning.
  • Continual modernization activity.

Measureable project outcomes - This row provides evidence of FSMI outcomes.

Analysis: Section 4.2.3

  • None measured to date.
  • None measured to date.
  • Measured skill consistency among new and existing inspectors.
  • None measured to date.
  • None measured to date.
  • None measured to date.
  • None measured to date.

Forecasted project outcomes - This row outlines the expected outcomes of individual FSMI projects based on their respective needs, objectives and outputs.

Analysis: Section 4.2.3

  • Consistent and efficient approach to inspection for all commodities.
  • Improved information exchange between CFIA and stakeholders.
  • Support for IFIM implementation (all waves).
  • Consistent skills among new and existing CFIA inspectors.
  • Consistent skills among CFIA inspector supervisors.
  • Tools in place to support consistent program delivery at the CFIA.
  • Improved CFIA data collection and management systems.
  • More efficient collection, collaboration, and use of scientific information across Canadian laboratories.
  • More efficient laboratory installations and equipment.
  • More efficient testing procedures.
  • Greater laboratory capacity.

Measured influence on FSP - This row outlines the lack of performance measurement for FSMI benefits on FSP.

Analysis: Section 4.2.3

  • None measured to date.
  • None measured to date.
  • None measured to date.
  • None measured to date.
  • None measured to date.
  • None measured to date.
  • None measured to date.

Expected Influence on FSP - This row outlines the forecasted benefits of FSMI projects on the FSP.

Analysis: Section 4.2.3

  • Consistent and efficient approach to inspection across all commodities - single food inspection system.
  • Strengthened food safety system.
  • Consistent and efficient approach to inspection across all commodities.
  • Integrated and improved industry access to CFIA services.
  • Facilitated pre-clearance decisions and improved market access.
  • Strengthened food safety system.
  • Consistent and efficient approach to inspection across all commodities - single food inspection system.
  • Strengthened food safety system.
  • Improved Agency-wide data collection and management.
  • Consistent CFIA program delivery.
  • Strengthened food safety system.
  • Improved detection and response.
  • Integrated and improved information sharing network.
  • Strengthened food safety system.
  • Improved detection and response.
  • Strengthened food safety system.
  • Improved detection and response.
  • Strengthened food safety system.

Need beyond the FSMI - This row provides evidence of on-going and long-term FSMI project goals.

Analysis: Section 4.2.3

  • Expansion beyond the food program.
  • Expansion beyond the food program.
  • Integration of other functionality - Agency-wide.
  • Continued recruitment and training development.
  • Continued culture development.
  • Integration of CFIA training and recruitment needs with national educational programming.
  • Ongoing IM/IT development.
  • Expansion beyond the food program.
  • Expansion beyond the food program.
  • Expansion beyond the food program.

Table Notes

Table Note 10

Developing a laboratory network strategy project is now known as the Canadian Food Safety Information Network

Return to table note 10  referrer

Table Note 11

Scope creep is defined as "the tendency of a project to include more tasks than originally specified which may lead to higher project costs and/or possible missed deadlines"

Return to table note 11  referrer

4.2.2 Efficiency and economy

4.2.2.1 Economy

Financial data collected during the evaluation supports the evidence of delays in initial project implementation, as expenditure figures are below end of year/adjusted budgets during the early fiscal years of the FSMI.

The evaluation also found funding for most FSMI projects is anticipated to be spent within the five-year project timeframe ending fiscal year 2015-16; remaining funding has been extended until fiscal year 2017-18.

Table 8 (below) provides information on budgets and expenditures, by project, for the four initially planned years of the FSMI and forecasts subsequent years. The figures include:

  • Initial funding allocation - the initial amount of funding allocated to a project.
  • Actual budget - total funding allocated to each project, which takes into account the transfers of responsibilities and funding (reductions) to Shared Services Canada (SSC), Treasury Board approved funding profile changes (re-profiles), and the allocation (addition) of prior year funding lapses carried forward through an established Department of Finance and TB mechanism.
  • Expenditures - total actual resources utilized by the projects.
  • Less authorities carried forward to the next fiscal year - includes budget reallocations as a result of CFIA decisions to carry forward funding from a previous year's variance into the current year, through an established Department of Finance and TB mechanism.
  • Variance - unspent resources for a project in a given fiscal year. The difference between the amount planned and the amount actually spent for a given project of program in a given year. Calculated by subtracting expenditures for a given year from the actual budget in the same year. (Actual Budget less Expenditures)

The reinvestment of specific lapsed (the amount the variance captures) initiative funding back into the initiative is an Agency decision, not a TB requirement. These funds are pooled with other Agency variances and carried forward into subsequent years based on annual CFIA priorities and senior management decision making.

It is important to note FSMI allocations provided by TB were not considered fenced funding. The Agency independently decided to fence all funds associated with the FSMI, in order to track expenditures over time and ensure those funds remained within the initiative. Decisions to reallocate funding are made through the CFIA's governance structure. Since the Agency elected to track all FSMI funding, the overall level of funding for the initiative remained largely unchanged, despite changes to individual project allocations. The ESDP project, however, was tracked slightly differently, as it fit the federal government's requirements for tracking under the ePMFFootnote 12 structure.

Table 8 demonstrates funding allocations for the FSMI. This information supports the analysis presented in Section 4.2.2.

Table 8: FSMI Funding Allocations - Overal Snapshot

I. Fiscal Years 2011-2012 to 2014-2015
II. Project III. Initial Funding Allocation IV. Actual Budget V. Expenditures VI. Variance (Variance is calculated by subtracting expenditures for a given year from the actual budget in the same year.)
IFIM $13,653,537 $18,099,031 $18,509,330 ($410,299)
(2.3%)
ESDP (FIMS) $34,946,463 $16,969,065 $10,431,681 $6,537,384
38.5%
Recruitment & Training $13,000,000 $13,202,666 $6,233,591 $6,969,075
52.8%
IM/IT $12,700,000 $10,444,438 $8,409,507 $2,034,931
19.5%
FSIN $2,500,000 $2,907,200 $2,235,669 $671,531
23.1%
MEL $6,500,000 $7,112,000 $3,378,427 $3,733,573
52.5%
ELRC $4,100,000 $4,100,000 $3,921,551 $178,449
4.4%
Initial 4 Year Sub-total $87,400,00 $72,834,400 $53,119,756 $19,714,644
27.1%
Less Authorities Carried Forward to next Fiscal year ($16,647,499)
Actual 4 Year Sub-total $56,186,901 $53,119,756 $3,067,145
5.5%
I. Fiscal Years 2015-2016
II. Project III. Initial Funding Allocation IV. Actual Budget V. Expenditures VI. Variance (Variance is calculated by subtracting expenditures for a given year from the actual budget in the same year.)
IFIM $2,558,000 $2,558,000
ESDP (FIMS) $8,542,000 $9,385,815
Recruitment & Training $4,900,000 $4,900,000
IM/IT $4,100,000 $3,954,541
FSIN $0 $0
MEL $5,400,000 $8,643,400
ELRC $5,400,000 $1,300,000
2015-16 Total $26,800,000 $30,741,756 TBD TBD
I. Fiscal Years 2016-17 to 2017-18
I. Fiscal Years II. Project III. Initial Funding Allocation IV. Actual Budget V. Expenditures VI. Variance (Variance is calculated by subtracting expenditures for a given year from the actual budget in the same year.)
Funding transferred to SSC responsibility $0 $3,187,409
Reprofiled
to
2016-2017
ESDP + FIMS $0 $21,377,743
Reprofiled
to
2016-2017
IM/IT $0 ($37,172)
Reprofiled
to
2017-2018
ESDP + FIMS $0 $2,743,363
Reprofiled
to
2017-2018
IM/IT $0 ($29,016)
I. Fiscal Years II. Project III. Initial Funding Allocation IV. Actual Budget V. Expenditures VI. Variance (Variance is calculated by subtracting expenditures for a given year from the actual budget in the same year.)
2011-12 to 2017-18 Total Initiative $114,200,000 $114,170,984 TBD TBD

Source: Corporate Management Branch, CFIA

Table 8 shows most FSMI projects are expected to spend their adjusted funding during the five-year period; however, some notable exceptions exist.

Recruitment and Training, for example, had a total spending variance of 52.8 per cent during fiscal years 2011-12 through 2014-15. This is a result of an early end to the planned refresher programming, along with an interruption to the planned recruitment and training activities due to delays in the implementation of IFIM. That project is currently implementing Wave 1 of a multi-wave roll out, and training needs to be aligned with the final operational model. At the moment, it does not appear that much of the variance in funding for Recruitment and Training will be reallocated to future years of the project; however, it will remain within the FSMI.

Another project with significant variances (38.5 per cent) over the initial four years was ESDP (FIMS). The merger of ESDP and FIMS in 2013 may have contributed to this. Delays in ESDP implementation have also resulted in a significant portion of its budget being re-profiled into later fiscal years through 2017-18. These re-profiled amounts are currently forecasted to be spent within that timeframe.

The IM/IT variance of 19.5 per cent is forecasted to be spent in the fifth year of the FSMI. In addition, reallocations from the IM/IT project took place as certain IM/IT functions were consolidated under Shared Services Canada (SSC), as a change in government structured responsibilities occurred during the FSMI period.

The development of a laboratory network strategy - although initially planned to be completed in three years - had an overall variance of 23.1 per cent for the four-year period. The funds have been reinvested in the project for fiscal year 2015-16.

Finally, there was about $3 million in variances (52.5 per cent) associated with the modernizing laboratories and equipment project during the initial four years of FSMI. However, much of this funding is forecasted to be reallocated to the project for the 2015-16 fiscal year.

In summary, changes in the funding allocations for the FSMI, from 2011-12 to 2014-15, came through either re-profiling or carry forward of funding based on senior management decisions. While the overall variance shown in Table 8, Column VI, indicates 27.1 per cent of FSMI funds were not spent, it does not take into account this funding that was authorized to be carried forward to the next fiscal year. When you consider the authorized carried forward amount of $16,647,499 (Column IV), the overall variance is reduced to 5.5 per cent. If the funds forecasted fiscal year 2015-16 are spent as planned, close to 95 per cent of the budgeted FSMI funds will have been spent.

Considering re-profiled amounts that were pushed out to fiscal years 2016-17 and 2017-18, along funding transferred to SSC, it is possible to forecast the $114.2 million initially allocated for the five-year FSMI project will be spent.

4.2.2.2 Efficiency

The evaluation found it difficult to establish efficiency for FSMI project expenditures. However, the fact that most projects were either implemented or are scheduled to be completed as planned without significant overages provides reasonable evidence of efficiency.

It is difficult to establish efficiency of project expenditures under FSMI due to lack of a similar, fully-costed and implemented alternative to each of the projects. Even projects that examined very close alternatives at their early stages - such the laboratory modernization and equipment renovation projects - involved estimates for planning purposes, rather than implemented alternatives. Other projects, like the IM/IT and ESDP projects, looked at established best practices, similar past work and alternative strategies prior to implementation.

4.2.3 Outcomes

The evaluation found that while measures were in place to track the immediate outcomes of FSMI projects data, outcomes have largely not been collected as many of the projects are at too early a stage to expect outcomes to be realized.

The evaluation was also unable to find evidence of a clear plan to track the initiative's effects on Agency programming in the future indicating a lack of performance measurement in place for measuring the outcomes of FSMI on the FSP and the overall Agency.

When examining the FSMI logic model, immediate outcomes relate more directly to individual projects. However, the ultimate intent of all FSMI projects is to enhance the FSP and ultimately to strengthen the food safety system.

With the exception of the Recruitment and Training project, the evaluation was unable to find evidence of measured outcomes for the projects examined. In the case of recruitment and training, concerted efforts were taken to demonstrate core and refresher training among inspectors, along with the acquisition of skills, resulted in successful inspection work. Evidence also suggests that more such tracking is planned for the future.

The lack of a performance measurement strategy is particularly problematic, given many of the Agency's current FSP performance measures relate to stakeholder compliance with program requirements. Without additional information, these will be insufficient to assess the impact of the FSMI on the FSP and broader Agency programming.

For example, compliance measures used to track program requirements under the previous inspection model are different from compliance measures tracking program requirements under the new inspection model. Since these two forms of measuring compliance are different, it will not be possible to compare them in order to determine which form of compliance measurement is more effective or efficient. The reality is, compliance measurement is being modernized to align with the new audit-based inspection approach that is risk-based and designed to enhance the Agency's inspection system.

The Agency's plan to use a risk-based approach to determine inspection activities will further complicate the use of compliance measures as a tool for understanding improvements in food-related risks to the Canadian public. Under this model, appropriate targeting of resources to high-risk producers and industries will likely be associated with identifying higher levels of non-compliance. This does not necessarily reflect a systematic increase in food safety risk.

Both of these complications suggest more direct measures of food safety risks and other aspects of CFIA program performance will be necessary to determine FSMI effectiveness and efficiency.

The need for better performance measurement becomes even more acute when examining the need for each FSMI project outside the context of the initiative itself. As Table 7 notes, there are clear plans to build on all of the FSMI projects to support change beyond the FSP. In all likelihood, this will require not only concerted effort on the part of the Agency, but substantial additional funding. Without the means to establish the effectiveness of past change initiatives such as the FSMI, accessing future funding could become difficult.

Date modified: