Nutrition Labelling Compliance Test
Appendix 2 - Statistical Framework

Background

Nutrition values on labels and in advertising need to be accurate and measurable for compliance purposes. Consequently, the food industry seeks a high probability of passing a nutrition labelling compliance test while the consumer requires a high probability that the label value accurately reflects the nutrient content of the product. Balancing these objectives is key to the development and acceptability of a compliance test. To achieve this balance, one needs to consider, on the one hand, the likelihood that a lot meeting the label declaration will be deemed unacceptable and, on the other, the likelihood that a lot not meeting the label declaration will be judged satisfactory. In statistical terminology, this translates to a Type I error, or producer's risk, which is the probability that a lot of acceptable declared values is erroneously rejected, and a Type II error, or consumer's risk, which is the probability that a lot of unacceptable declared values is erroneously accepted as satisfactory. A statistically valid and defensible sampling plan then needs to consider the relative magnitude and ensuing consequences of these two competing errors. Moreover, not only does the mean nutrient content of the food need to be evaluated by the sampling scheme but equally important is its capacity to include and measure the associated variability.

A. Sampling Plan

To meet all these requirements, a number of assumptions, some based on past experience and observation, need to be made to enable the computation of appropriate Type I and Type II errors corresponding to a specific sampling plan. The sampling scheme adopted by the CFIA for assessing the accuracy of nutrition labelling involves the selection of twelve individual random units from a lot, arranged in three groups of four units, with each group properly composited to provide three composite sub-samples for lab analysis. The accompanying tables provide the risk/error information for this sampling regime under the following assumptions.

Assumptions

  1. All samples are randomly selected from a lot to ensure representativeness of the lot and its characteristics and to allow statistical inference from the lab results.
  2. Let S represent the variability among the units within a lot
    A represent the variability among the mean nutrient values of various lots
    R represent the measurement variability within a laboratory, namely, the repeatability variance
    B represent the measurement variability between laboratories
    b represent the number of lots (or batches) sampled
    c represent the number of composite sub-samples analyzed per lot
    d represent the number of individual units from the lot comprising a composite sub-sample.
    Then the total variance of a measurement for d units in each of c analyses from b lots is given as
    total variance of a measurement for d units in each of c analyses from b lots
  3. In all of the estimates, the nutrients are assumed to have an underlying normal distribution with mean µ and variance letter sigma - variance (or S). This implies that the underlying coefficient of variation (CV) is less than 50%.
  4. The lot-to-lot and lab-to-lab variability (A and B) are assumed independent and their combined relative standard deviation square root of A plus Bis historically accepted as 3%. Nonetheless, we have also investigated this model assumption for 7% in the accompanying tables.
    Note: In the following tables, the results are given for three different values of the measurement variation within a laboratory, namely the repeatability relative standard deviation (repeatability relative standard deviation), expressed as a coefficient of variation, specifically, 3%, 7% and 15%. It should be realized that, as this variability increases, the only way of reducing errors requires that one analyze individual rather than composite samples from the lot.

Tables 1 - 3 show how the test method variability and the variability of the nutrient distribution in a food impact on the producer's risk and the consumer's risk for different average lot quantities (true mean as % of label). Three tables, one for each of Class I, Class II (minimum limit) and Class II (maximum limit) are presented. Within each table are four individual tables, two for producer's risk and two for consumer's risk, and each of these is shown for 3% CV and 7% CV lab-to -lab and lot-to- lot variability respectively. Then, for each individual table, the producer's risk or consumer's risk is shown in relation to the variability of the analytical method within a lab (RSDr%) and variability of the nutrient distribution within a lot (CV%).

Using the relevant table, the producer's risk or chance that a lot of a given mean nutrient content would be erroneously rejected (found out of compliance) and consumer's risk or chance that the lot would be erroneously accepted (found compliant) can be estimated if the nutrient variability (CV%) and method variability lab (RSDr%) are known. For example, in Table 1, Class I, added vitamins and mineral nutrients, the producer's risk of a lot being erroneously rejected would be 5.9%, where the true average nutrient content of the lot is 110% of the label, a test RSDr% of 7% and a lot nutrient variability CV of 10%. The consumer's risk or chance that a lot of average nutrient content of 90% of label would be erroneously accepted would be 2.8% for the same test variability and within lot nutrient distribution.

In Table 3, Class II maximum limit (Calories, fat, saturated fat, trans fat, cholesterol, sugars and sodium), for a test variability of 7% and a CV of nutrient distribution of 20%, if the true lot average is 100% of label, the chance of erroneously rejecting a lot would be 0.5% (producer's risk); and under the same conditions of test variability and nutrient variability, a true lot average of 140% of the label value, there would be a 3.1% chance of accepting the products erroneously (consumer's risk).

Table 1
Producer's Risk and Consumer's Risk for Class I

Added vitamins and mineral nutrients

The sample is comprised of 12 consumer units taken at random from a lot arranged in three composites of four units.

The mean nutrient content of the sample is not less than the declared label value (adjusted for rounding)

Producer's Risk (%)
Rejection Level < 100% of Label
Between Lot & Between Lab Variability CV = 3%
True Mean
(% of label)
Method Variability
RSDr Table Note 8
(%)
Variability of Nutrients Within Lot (CV Table Note 9 of distribution as % of True Mean)
10% 20% 30% 40% 50%
110 3 2.2 8.8 16.5 22.5 27.0
7 5.9 11.8 18.2 23.5 27.6
15 17.2 20.1 23.5 26.9 29.7
120 3 0.0 0.7 3.7 8.3 13.1
7 0.2 1.5 4.8 9.3 13.8
5 4.1 6.2 9.3 12.9 16.5
130 3 0.0 0.0 0.7 2.8 6.0
7 0.0 0.1 1.1 3.3 6.6
15 0.8 1.7 3.4 5.9 8.9
140 3 0.0 0.0 0.1 0.9 2.7
7 0.0 0.0 0.2 1.2 3.1
15 0.10 0.40 1.2 2.6 4.7

Table Notes

Table Note 8

RSDr: repeatability relative standard deviation

Return to table note 8 referrer

Table Note 9

CV: coefficient of variation

Return to table note 9 referrer

Producer's Risk (%)
Rejection Level <100% of Label
Between Lot & Between Lab Variability CV = 7%
True Mean
(% of label)
Method Variability
RSDr (%)
Variability of Nutrients Within Lot (CV of distribution as % of True Mean)
10% 20% 30% 40% 50%
110 3 12.1 16.3 21.0 25.2 28.7
7 14.5 18.0 22.1 25.9 29.1
15 21.5 23.4 26.0 28.5 30.9
120 3 1.6 3.6 7.0 11.0 15.1
7 2.6 4.7 8.0 11.9 15.7
15 7.4 9.2 11.9 14.9 18.0
130 3 0.1 0.6 2.0 4.5 7.6
7 0.4 1.0 2.6 5.1 8.2
15 2.2 3.3 5.1 7.5 10.3
140 3 0.0 0.1 0.6 1.8 3.8
7 0.0 0.2 0.8 2.1 4.2
15 0.7 1.1 2.1 3.7 5.9
Consumer's Risk (%)
Acceptance Level >=100% of Label
Between Lot & Between Lab Variability CV = 3%
True Mean
(% of label)
Method Variability
RSDr (%)
Variability of Nutrients Within Lot (CV Table Note 10 of distribution as % of True Mean)
10% 20% 30% 40% 50%
95 3 12.2 21.7 28.6 33.1 36.1
7 18.2 24.6 30.0 33.8 36.5
15 29.2 31.4 33.8 36.1 37.9
90 3 0.7 4.9 11.7 17.8 22.7
7 2.8 7.3 13.4 18.9 23.4
15 12.4 15.3 18.9 22.6 25.8
80 3 0.0 0.0 0.4 1.9 4.6
7 0.0 0.1 0.6 2.4 5.1
15 0.5 1.1 2.4 4.5 7.2
70 3 0.0 0.0 0.0 0.0 0.2
7 0.0 0.0 0.0 0.0 0.3
15 0.0 0.0 0.0 0.2 0.6

Table Notes

Table Note 10

CV: coefficient of variation

Return to table note 10 referrer

Consumer's Risk(%)
Acceptance Level >=100% of Label
Between Lot & Between Lab Variability CV = 7%
True Mean
(% of label)
Method Variability
RSDr (%)
Variability of Nutrients Within Lot (CV of distribution as % of True Mean)
10% 20% 30% 40% 50%
95 3 24.9 28.4 32.0 35.0 37.2
7 27.0 29.8 32.8 35.4 37.5
15 32.4 33.7 35.5 37.1 38.6
90 3 7.6 11.5 16.2 20.7 24.6
7 9.8 13.2 17.4 21.5 25.1
15 16.7 18.8 21.5 24.4 27.1
80 3 0.1 0.3 1.3 3.3 6.1
7 0.2 0.6 1.7 3.8 6.5
15 1.5 2.3 3.8 6.0 8.5
70 3 0.0 0.0 0.0 0.1 0.4
7 0.0 0.0 0.0 0.1 0.5
15 0.0 0.0 0.1 0.4 0.9

Table 2
Producer's Risk and Consumer's Risk for Class II minimum limit

Protein, carbohydrate, fibre, vitamins, mineral nutrients

The sample is comprised of 12 consumer units taken at random from a lot arranged in three composites of four units.

The mean nutrient content of the sample is not less than 80% of the declared label value (adjusted for rounding)

Producer's Risk (%)
Rejection Level < 80% of Label
Between Lot & Between Lab Variability CV = 3%
True Mean
(% of label)
Method Variability
RSDr Table Note 11
(%)
Variability of Nutrients Within Lot (CV Table Note 12 of distribution as % of True Mean)
10% 20% 30% 40% 50%
90 3 0.7 4.9 11.7 17.8 22.7
7 2.8 7.3 13.4 18.9 23.4
15 12.4 15.3 18.9 22.6 25.8
100 3 0.0 0.1 1.6 4.9 8.9
7 0.0 0.5 2.3 5.6 9.5
15 1.9 3.2 5.6 8.7 12.1
110 3 0.0 0.0 0.2 1.2 3.3
7 0.0 0.0 0.3 1.5 3.7
15 0.2 0.6 1.5 3.2 5.5
120 3 0.0 0.0 0.0 0.3 1.2
7 0.0 0.0 0.0 0.4 1.5
15 0.0 0.1 0.4 1.2 2.6

Table Notes

Table Note 11

RSDr: repeatability relative standard deviation

Return to table note 11 referrer

Table Note 12

CV: coefficient of variation

Return to table note 12 referrer

Producer's Risk (%)
Rejection Level < 80% of Label
Between Lot & Between Lab Variability CV = 7%
True Mean
(% of label)
Method Variability
RSDr (%)
Variability of Nutrients Within Lot (CV of distribution as % of True Mean)
10% 20% 30% 40% 50%
90 3 7.6 11.5 16.2 20.7 24.6
7 9.8 13.2 17.4 21.5 25.1
15 16.7 18.8 21.5 24.4 27.1
100 3 0.5 1.5 3.8 7.1 10.8
7 1.0 2.2 4.6 7.8 11.3
15 4.1 5.5 7.8 10.6 13.6
110 3 0.0 0.2 0.8 2.3 4.5
7 0.1 0.3 1.1 2.6 5.0
15 0.9 1.5 2.7 4.5 6.7
120 3 0.0 0.0 0.2 0.7 1.9
7 0.0 0.0 0.2 0.9 2.2
15 0.2 0.4 0.9 1.9 3.4
Consumer's Risk (%)
Acceptance Level >= 80% of Label
Between Lot & Between Lab Variability CV = 3%
True Mean
(% of label)
Method Variability
RSDr (%)
Variability of Nutrients Within Lot (CV of distribution as % of True Mean)
10% 20% 30% 40% 50%
75 3 7.0 16.1 23.7 29.0 32.7
7 12.5 19.2 25.3 29.8 33.1
15 24.4 26.9 29.9 32.6 34.8
70 3 0.1 1.7 6.3 11.8 16.8
7 0.7 3.1 7.7 12.8 17.5
15 6.9 9.4 12.9 16.6 20.2
60 3 0.0 0.0 0.0 0.3 1.2
7 0.0 0.0 0.0 0.4 1.5
15 0.0 0.1 0.4 1.2 2.6
Consumer's Risk (%)
Acceptance Level >= 80% of Label
Between Lot & Between Lab Variability CV = 7%
True Mean
(% of label)
Method Variability
RSDr (%)
Variability of Nutrients Within Lot (CV of distribution as % of True Mean)
10% 20% 30% 40% 50%
75 3 19.5 23.5 27.7 31.2 34.0
7 21.9 25.1 28.7 31.8 34.3
15 28.1 29.8 31.8 33.9 35.7
70 3 3.3 6.1 10.2 14.7 18.8
7 4.8 7.5 11.4 15.5 19.4
15 10.7 12.7 15.6 18.7 21.7
60 3 0.0 0.0 0.2 0.7 1.9
7 0.0 0.0 0.2 0.9 2.2
15 0.2 0.4 0.9 1.9 3.4

Table 3
Producer's Risk and Consumer's Risk for Class II maximum limit

Calories, fat, saturated fat, trans fat, sugars, sodium

The sample is comprised of 12 consumer units taken at random from a lot arranged in three composites of four units.

The mean nutrient content of the sample is not more than 120% of the declared label value (adjusted for rounding)

Producer's Risk (%)
Rejection Level > 120% of Label
Between Lot & Between Lab Variability CV = 3%
True Mean
(% of label)
Method Variability
RSDr Table Note 13
(%)
Variability of Nutrients Within Lot (CV Table Note 14 of distribution as % of True Mean)
10% 20% 30% 40% 50%
110 3 2.2 8.8 16.5 22.5 27.0
7 5.9 11.8 18.2 23.5 27.6
15 17.2 20.1 23.5 26.9 29.7
100 3 0.0 0.1 1.6 4.9 8.9
7 0.0 0.5 2.3 5.6 9.5
15 1.9 3.2 5.6 8.7 12.1
90 3 0.0 0.0 0.0 0.3 1.2
7 0.0 0.0 0.0 0.4 1.5
15 0.0 0.1 0.4 1.2 2.6

Table Notes

Table Note 13

RSDr: repeatability relative standard deviation

Return to table note 13 referrer

Table Note 14

CV: coefficient of variation

Return to table note 14 referrer

Producer's Risk (%)
Rejection Level > 120% of Label
Between Lot & Between Lab Variability CV = 7%
True Mean
(% of label)
Method Variability
RSDr (%)
Variability of Nutrients Within Lot (CV of distribution as % of True Mean)
10% 20% 30% 40% 50%
110 3 12.1 16.3 21.0 25.2 28.7
7 14.5 18.0 22.1 25.9 29.1
15 21.5 23.4 26.0 28.5 30.9
100 3 0.5 1.5 3.8 7.1 10.8
7 0.0 2.2 4.6 7.8 11.3
15 0.0 5.5 7.8 10.6 13.6
90 3 0.0 0.0 0.2 0.7 1.9
7 0.0 0.0 0.2 0.9 2.2
15 0.0 0.4 0.9 1.9 3.4
Consumer's Risk (%)
Acceptance Level <=120% of Label
Between Lot & Between Lab Variability CV = 3%
True Mean
(% of label)
Method Variability
RSDr (%)
Variability of Nutrients Within Lot (CV of distribution as % of True Mean)
10% 20% 30% 40% 50%
125 3 18.8 27.6 33.4 37.0 39.4
7 24.5 30.1 34.5 37.5 39.7
15 33.9 35.6 37.6 39.3 40.8
130 3 4.4 12.7 20.5 26.2 30.2
7 9.2 15.8 22.1 27.1 30.7
15 21.2 23.9 27.1 30.1 32.6
140 3 0.1 1.7 6.3 11.8 16.8
7 0.7 3.1 7.7 12.8 17.5
15 6.9 9.4 12.9 16.6 20.2
150 3 0.0 0.1 1.6 4.9 8.9
7 0.0 0.5 2.3 5.6 9.5
15 1.9 3.2 5.6 8.7 12.1
Consumer's Risk (%)
Acceptance Level <= 120% of Label
Between Lot & Between Lab Variability CV = 7%
True Mean
(% of label)
Method Variability
RSDr (%)
Variability of Nutrients Within Lot (CV of distribution as % of True Mean)
10% 20% 30% 40% 50%
125 3 30.3 33.3 36.1 38.4 40.2
7 32.1 34.4 36.8 38.8 40.4
15 36.4 37.5 38.8 40.2 41.3
130 3 16.1 20.3 24.7 28.6 31.7
7 18.5 21.9 25.8 29.3 32.1
15 25.2 27.0 29.3 31.6 33.7
140 3 3.3 6.1 10.2 14.7 18.8
7 4.8 7.5 11.4 15.5 19.4
15 10.7 12.7 15.6 18.7 21.7
150 3 0.5 1.5 3.8 7.1 10.8
7 1.0 2.2 4.6 7.8 11.3
15 4.1 5.5 7.8 10.6 13.6

Graphs 1.1 - 6.2 visually depict data provided in tables 1 - 3 to facilitate comparisons between different mean nutrient contents and different sources of variation.

Comparison of Scenarios

Class I: Added Vitamins and Mineral Nutrients,
Producer's Risk (Type I error)
(Rejection Level <100%)

Graph 1.1
True Mean=110% of Label
Between Lot & Between Lab Variability CV= 3%

Graph 1.1: Comparison of Scenarios for Class 1: Added Vitamins and Mineral Nutrients, Producer's Risk (Type 1 error), True Mean=110% of label

Graph 1.2
True Mean = 120% of Label
Between Lot & Between Lab Variability CV = 3%

Graph 1.2: Comparison of Scenarios for Class 1: Added Vitamins and Mineral Nutrients, Producer's Risk (Type 1 error), True Mean=120% of label

Class I: Added Vitamins and Mineral Nutrients,
Producer's Risk (Type I error)
(Rejection Level < 100%)

Graph 2.1
Within lab method variability RSDr = 7%
Between Lot & Between Lab Variability CV = 3%

Graph 2.1: Comparison of Scenarios for Class 1: Added Vitamins and Mineral Nutrients, Producer's Risk (Type 1 error), Within lab method variability RSDr = 7%

Graph 2.2
Within lab method variability RSDr = 15%
Between Lot & Between Lab Variability CV = 3%

Graph 2.2: Comparison of Scenarios for Class 1: Added Vitamins and Mineral Nutrients, Producer's Risk (Type 1 error), Within lab method variability RSDr = 15%

Class I: Added Vitamins and Mineral Nutrients,
Consumer's Risk (Type II error)
(Acceptance Level >= 100%)

Graph 3.1
True Mean = 90% of Label
Between Lot & Between Lab Variability CV = 3%

Graph 3.1: Comparison of Scenarios for Class 1: Added Vitamins and Mineral Nutrients, Consumer's Risk (Type 2 error), True Mean=90% of label

Graph 3.2
True Mean = 80% of Label
Between Lot & Between Lab Variability CV = 3%

Graph 3.2: Comparison of Scenarios for Class 1: Added Vitamins and Mineral Nutrients, Consumer's Risk (Type 2 error), True Mean=80% of label

Class I: Added Vitamins and Mineral Nutrients,
Consumer's Risk (Type II error)
(Acceptance Level >= 100%)

Graph 4.1
Within lab method variability RSDr = 7%
Between Lot & Between Lab Variability CV = 3%

Graph 4.1: Comparison of Scenarios for Class 1: Added Vitamins and Mineral Nutrients, Consumer's Risk (Type 2 error), Within lab method variability RSDr = 7%

Graph 4.2
Within lab method variability RSDr = 15%
Between Lot & Between Lab Variability CV = 3%

Graph 4.2: Comparison of Scenarios for Class 1: Added Vitamins and Mineral Nutrients, Consumer's Risk (Type 2 error), Within lab method variability RSDr = 7%

Class II: Calories, Fat, Saturated Fat, Trans Fat, Cholesterol, Sodium, Sugars
Producer's Risk (Type I error)
(Rejection Level > 120%)

Graph 5.1
True Mean = 100% of Label
Between Lot & Between Lab Variability CV = 3%

Graph 5.1 : Comparison of Scenarios for Class 2: Calories, Fat, Saturated Fat, Trans Fat, Cholesterol, Sodium, Sugars Producer's Risk (Type 1 error), True Mean=100% of label

Graph 5.2
Within lab method variability RSDr = 7%
Between Lot & Between Lab Variability CV = 3%

Graph 5.2 : Comparison of Scenarios for Class 2: Calories, Fat, Saturated Fat, Trans Fat, Cholesterol, Sodium, Sugars Producer's Risk (Type 1 error), Within lab method variability RSDr = 7%

Class II: Calories, Fat, Saturated Fat, Trans Fat, Cholesterol, Sodium, Sugars
Consumer's Risk (Type II error)
(Acceptance Level <= 120%)

Graph 6.1
True Mean = 140% of Label
Between Lot & Between Lab Variability CV = 3%

Graph 6.1 : Comparison of Scenarios for Class 2: Calories, Fat, Saturated Fat, Trans Fat, Cholesterol, Sodium, Sugars Consumer's Risk (Type 2 error), True Mean=140% of label

Graph 6.2
Within lab method variability RSDr = 7%
Between Lot & Between Lab Variability CV = 3%

Graph 6.2: Comparison of Scenarios for Class 2: Calories, Fat, Saturated Fat, Trans Fat, Cholesterol, Sodium, Sugars Consumer's Risk (Type 2 error), Within lab method variability RSDr = 7%

B. Acceptance Criteria for Lot Compliance

  1. The nutrient values are assumed to be normally distributed in the lot about the declared/label value. The consumer's risk and the producer's risk are evaluated based on this assumption which should hold for a stable production process. Under this assumption, since a nutrient value must be non-negative, the coefficient of variation (CV) should have an upper bound of 50%. This translates to the probability of any random sample taken from the lot giving a nutrient value less than half the declared value to be less than 0.16, and likewise for being greater than 1.5 the declared value. In other words, fewer than one in six such random samples selected from the lot should provide results that are not within 50% of the declared value. Consequently, if one of our three lab results is not within 50% of the label value, the validity of our assumption of an underlying normal distribution for the nutrient content is highly questionable. Having only three observations for our lot, standard statistical methods for testing normality are not applicable but this criterion should serve a similar purpose.
  2. This criterion serves as the nucleus of the compliance test by requiring the mean of the three lab results to fall within the allowable limit which is the declared value subjected to the appropriate rounding rules (Appendix 3) and subsequent applicable tolerance.
  3. With the growing fortification of foods, there may be safety issues associated with large variability. It would seem reasonable that where a nutrient has been added to a food an overall variability remain within 10% (0.1) of the mean value. This variability then should lie within the limits of a confidence interval for the standard deviation of the lot nutrient content. Recognizing that the variability within a lot as well as that between lots may vary significantly according to the nutrient under consideration, a 99% confidence level is preferred to the more commonly used 95% level of confidence. This allows greater elasticity in the estimate and is less stringent by providing a wider confidence interval. The above is equivalent to having 0.1 being greater than the 99% lower confidence limit, evaluated from the lab results as (s x 0.4344/mean value) where mean value and s are respectively, the declared mean and the standard deviation of the three lab determinations for the lot nutrient content.

C. Glossary

Accuracy:
The closeness of agreement between a test result and the accepted reference value.
Coefficient of Variation (CV):
This quantity expresses the standard deviation as a percentage of the mean. The coefficient of variation of a random variable X having a mean µ and variance variance (letter sigma) is given as coefficient of variation
Label Value (or Declared Value):
The amount of nutrient declared on the label or in advertising.
Lot (or Batch):
A collection of units of product from which a sample is to be drawn and inspected to determine conformity with the acceptability criteria.
Nutrition labelling compliance test:
a collection of identically labelled products produced under conditions as nearly uniform as possible and available for inspection at one time.
Sample:
A subset of units of product drawn from a lot or batch, that is representative of the lot for inspection purposes.
Composite sub-sample:
A subset of the sample units that are combined and mixed to homogeneity.
Repeatability Variance (R):
The variance of independent test results obtained with the same method on identical test items in the same laboratory by the same operator using the same equipment within short intervals of time (RSDr)
Reproducibility Variance (B):
The variance of test results obtained with the same method on identical test items in different laboratories with different operators using different equipment.
Producer's Risk (Type I Error):
the probability that a lot meeting the label claim will be deemed unacceptable.
Consumer's Risk (Type II Error):
the probability that a lot of unacceptable declared values is erroneously accepted as satisfactory
Date modified: