Integrated Agency Inspection Model – Consultation Draft (December 4, 2013)
10.0 System performance and continuous improvement

This page has been archived

Information identified as archived is provided for reference, research or record-keeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Introduction

As part of inspection modernization, the CFIA is also modernizing its approach to performance measurement. This new approach recognizes not only the scope and complexity of the CFIA's role but also that there are many other participants who play a role in maintaining human, animal and plant health and protection of the environment. All these players influence the ultimate outcome of protecting Canadians and their environment from preventable health risks.

The CFIA is introducing a more holistic way of monitoring and evaluating the overall effectiveness of the inspection system. The new approach will determine whether the system, as a whole, is achieving the desired outcomes and will align with the CFIA's modernized approach to outcome-based regulations.

The CFIA will enhance the measures being used to demonstrate how well its inspection activities, business processes, and services contribute to the desired outcomes. The CFIA will also look at how it engages and influences the various stakeholders in the system and how the participants work together to prevent, mitigate and manage the risks.

Establishing key desired outcomes for the system as a whole and measuring performance against them will allow the CFIA, regulated parties, and other participants to use the results to adapt, improve, and become more effective at managing risks to human, animal and plant health and the environment.

The objectives of the performance measurement system are to assess:

  • the effectiveness of the inspection system as part of the broader system;
  • whether the regulatory program is risk-oriented and proactive;
  • whether the program is delivered as designed, and in a risk-based, consistent, efficient, relevant, and professionally conducted manner;
  • whether all participants in the system, including other governing and regulatory bodies, regulated parties, industry representatives and associations, and Canadians are engaging, sharing information, acting responsibly, constructively supporting and remaining confident in the system

to identify:

  • emerging risks; and
  • areas for system performance improvement.

The CFIA may also use the results of the performance assessment for:

  • corporate reporting
  • benchmarking over time
  • international comparative analysis
  • continuous improvement of program design, training, and inspection delivery
  • adjusting work plans.

10.1 Systems Performance Framework

To build its approach to measuring system performance, the CFIA developed a Systems Results Model (Figure 5, below) that outlines the relationships between CFIA resources and activities, the key participants who directly influence the system, and the desired outcomes.

The Results Model depicts three levels of assessment required to ensure that system performance objectives are met. Based on the information obtained at each level, a comprehensive analysis would be conducted and system improvements would be scheduled and implemented as required, with higher-priority improvements taking precedence.

Monitoring and evaluating performance is a CFIA responsibility. To be useful, the number of measures that are tracked and reported by CFIA needs to be limited, highly relevant and exception-based where possible. The CFIA will develop the key performance indicators and targets to check whether policies, procedures and practices that are critical for human, animal and plant health and protection of the environment are successful in achieving the desired results. The expectation is that the development of performance information would be iterative and would mature over time.

The three levels of assessment are as follows:

Level 1: CFIA activities and actions are risk-based, consistent, efficient, relevant and professionally conducted

a) Appropriateness of program design

Objective: To assess that the program is designed in a risk-based and proactive manner to meet regulatory objectives. For example:

  • Is there a risk-based and proactive regulatory framework for the program?
  • Are policies clear, complete and current?
  • Does the program support innovation?
  • Does the program include appropriate tools and supporting processes, including effective work planning and training?
  • Are the outcomes clear and achievable for inspectors?
  • Does the program define the external stakeholders and their roles, responsibilities and expected influence on the system?
  • Have the outcomes of the program been achieved?

b) Consistency and quality of service delivery

Objective: To measure the consistency and quality of the delivery of the inspection program. For example:

  • Are work plans delivered?
  • Are investment, efficiency and service standards met?
  • Are inspection, sampling and testing steps followed?
  • Are inspection forms complete and accurate?
  • Are decisions consistent?
  • Were change-initiative plans delivered and expectations met?

Level 2: Regulatory system stakeholders engage, comply and act in support of the system outcomes

Objective: To assess the extent to which key stakeholders engage, cooperate, coordinate and accept and act on their responsibility to prevent and mitigate risks. For example:

  • Do CFIA processes appropriately engage all key stakeholders?
  • Do key stakeholders understand key CFIA messages and requisite duties, obligations and needed actions?
  • Are the outcomes clear and achievable for all key stakeholders and do stakeholders understand their role in achieving outcomes?
  • What is the compliance profile of regulated parties and their status relative to trends in the industry?
  • Has the CFIA been successful in facilitating access of Canadian regulated commodities to international markets?
  • Is the system accepted internationally?
  • Have key participants who play a role in the system and influence the ultimate outcome of protecting consumers, animals, plants and the environment from preventable health and safety risks achieved the outcomes expected of them?

Level 3: Achievement of strategic outcomes is transparent

Objective: To assess the extent to which the CFIA has achieved its mission in relation to human, animal and plant health and protection of the environment. For example:

  • How effective was the regulatory system in meeting the CFIA's strategic outcomes?
  • Were risks to the system managed? Were emerging risks identified?
  • What is the risk versus compliance profile and what are the trends by program, industry and overall?
  • Is the system accepted internationally and are risks of trade barriers managed?
Figure 5 – The Regulatory Systems Results Model. Description follows.
Figure 5 – The Regulatory Systems Results Model

Figure 5 illustrates three levels of assessment that the CFIA would use to measure the performance of CFIA's sphere of indirect influence.

At the bottom, the first level of system performance (Level 1) identifies activities over which the CFIA has operational spherical control.

On top of level 1, the second level (Level 2) identifies stakeholders over which the CFIA has direct influence with the intermediate outcomes or actions and immediate outcomes or awareness, including

  • governing, regulatory bodies and support
  • industry representatives and associations, and
  • Individuals, Canadians and Consumer associations

The figure has arrows pointing to and from each stakeholder group in Level 2, demonstrating the complex relationships they all share. The figure also has arrows pointing up and down from Levels 1 and 2, demonstrating the many CFIA inputs that affect the actions and awareness of stakeholders, as well as the effect of the decisions by these same stakeholders on the CFIA's activities.

On top of level 2, the third level (Level 3) identifies a macro, strategic outcome, namely a safe and accessible food supply and plant and animal resource base where risks to the supply are minimized. The CFIA has indirect influence over this outcome.

The figure has arrows pointing from each of the stakeholder groups in Level 2 up to Level 3, demonstrating the roles and impact that all stakeholders have on a safe food supply.

10.2 Performance Assessment

Developing an innovative and modern approach to assessing system performance against desired outcomes is central to the CFIA's overall modernization agenda. It will be key to promoting continuous learning and improvement across the regulatory system while also supporting effective accountability.

A variety of sources of data will need to be incorporated into the assessment process to reflect the complex system, the numerous players involved and the wide scope of CFIA responsibilities. Assessments would be focussed both internally and externally and be either continuous or periodic. Assessment results would be used for

  • learning what works, in what conditions and for whom
  • changing activities to solve or avoid mistakes or problems
  • correcting the underlying causes behind problematic action
  • reporting on the CFIA's accountabilities, and
  • demonstrating transparency regarding our commitment to system performance.

As part of embedding a continuous improvement in its philosophy, the CFIA is improving and strengthening Canada's inspection system by establishing inspection system verifiers to oversee the performance of Canada's entire inspection system. Among other activities, the inspection system verifiers will conduct in-depth reviews of plans in the regulated party's operations to:

  • confirm if standards are consistently and thoroughly followed and enforced;
  • confirm if the activities are consistently achieving expected results;
  • identify any universal challenges and/or concerns; and
  • identify opportunities to increase transparency.

Examples of current performance assessment tools available:

  1. Field Observation – assess internal performance to identify gaps and issues in the effectiveness and quality of service delivery and in national consistency, the level of understanding of responsibilities, and the appropriate identification of non-compliance.
  2. Quality Management – assesses internal performance by tracking compliance with internal protocols, service standards and expectations.
  3. Audit – as per the Treasury Board Policy on Internal Audit, conduct periodic internal audits regarding the governance, risk management and controls in place for effective management practices and accountability.
  4. Evaluation – as per the Treasury Board Policy on Evaluation, conduct internal evaluations regarding the relevance and performance of programs, initiatives and policies.
  5. Surveys – assess internal and external performance by seeking input from CFIA staff, regulated parties and other system stakeholders on the function and quality of the system, the information and tools provided and suggestions for improvement.
Date modified: