Article ID 1301200103
Author Anders Kallner, Dept. of Clinical Chemistry, Karolinska Hospital, S - 171 76 Stockholm, Sweden
Download as a PDF here
The concept uncertainty in measurement
It is unavoidable that all decisions, all actions and therefore all measurements harbour an inherent uncertainty. For the lay - man/woman the term uncertainty is used in many connections and is a frequently used concept with no scientific meaning. However, metrologists have defined uncertainty and given it a scientific content that is useful for all measurements. The definition ofuncertainty in measurementaccording to ISO is:
'parameter, associated with a result of a measurement that characterises the dispersion of the values that could be reasonably attributed to the measurand'.
This definition may be difficult to apply to practical work and we shall therefore expand on the definition and explain how it can be used for describing how well a measurement procedure performs'.
The result of a measurement shall always give information about an interval within which the results can be expected with a given probability. This can be reported as an interval, as a standard deviation (SD), coefficient of variation or standard error of the mean (SEM). Most commonly the distribution of results is given as one SD. The SD is a description of the precision and can always be calculated from 2 or more results but the interpretation of this value requires knowledge of the properties of the distribution of values. The most probable value within a Gaussian distribution is represented by the mean. Together with the mean, the SD fully defines the shape of a Gaussian distribution. In these distributions the mean defines the position of the distribution on the X - axis and the SD its width and height. The SD is therefore a measure of the random error (the width of the distribution). In a Gaussian distribution about 2/3 of all results will be found between -1 SD and +1 SD counted from the mean of the distribution. Further, about 95 % will be found within the interval -2 SD to +2 SD and about 99 % between -3 SD and +3 SD.
All modern computers and calculators have predefined routines for the calculation of the mean and SD. However, the calculated properties are only estimates of the true standard deviation and mean of the distribution. If these estimates are made from too small a number of observations the estimate will be non - reliable i.e. the uncertainty of the standard deviation and mean will be large.
When characterizing a result we do not only want to describe how precise we have managed to perform the measurement (repeatable or reproducible) but also how correct it is i.e. the deviation of the result from the "true value". We never know the "true value" and it is therefore not possible to describe the trueness of the result. This dilemma is resolved by assigning a value and call it the "true value". This is normally called "conventional true value" (convention is here used in the meaning of agreement) or 'assigned value'. The difference between the mean of many measurements of the same quantity and the conventional true value is the "bias". The statistic "trueness" will therefore describe the systematic error.
The precision of a result (standard error of the mean) can be improved by increasing the number of observations whereas the trueness of the result cannot be influenced this way. In our role as analysts we are interested to find not only the random error but also the systematic error. However, as a consumer of results of measurements we would prefer a concept that would include both the random and the systematic error for a single measurement i.e. both the precision and the trueness. This concept is called accuracy and in laboratory medicine often equivalent to total error
. Theconcept of uncertaintyas defined above will offer a simple and tangible alternative to total error and will also give us a possibility to avoid the use of "error" to describe the variation inherent in measurements.
An accepted scientific approach to solve a problem is to break it down into smaller entities and study each of them. This approach can be used also to estimate the uncertainty in measurements and is usually not too complicated. However, as in all sciences the synthesis of results that should be performed is the difficult part when scientists may make mistakes and end up with grossly erroneous conclusions. The concept of uncertainty as defined by ISO requires that all steps involved in a measurement are defined and evaluated with regard to their uncertainty. This is a great help in characterizing and optimising the performance of a measurement procedure because the sources of uncertainties will be revealed. The chemist can review the result and select the steps that require improvement in an orderly and reproducible manner. All steps should be included, preanalytical, e.g. sampling and sample preparation, as well as analytical, e.g. calibrations, dilutions and postanalytical e.g. transformations and corrections. The accumulated data shall then be combined. The following text will describe how these steps can be systematized and performed in a pragmatic manner.
Formal accreditation of laboratories and measurement procedures or methods according to the ISO standards 15189 (1) and 17025 (2) requires that the uncertainty in measurements is estimated. The preferred method for estimation of uncertainties is described in 'Guide to the Expression of uncertainties n measurements' (GUM) (3).
Besides estimating the uncertainty in measurements to identify areas in which improvements should be focused, there is a point in estimating the uncertainty in all measurement that produce results for the diagnosis and management of diseases. The reason is that the uncertainty in measurements will include a contribution from calibration etc that affect the bias as well as estimates of the imprecision and any pre - and postanalytical uncertainties.
The concept of uncertainty is not a statistical concept in traditional sense i.e. it needs not be associated with a known distribution of the data. The uncertainty shall rather be understood as an interval within which the result can be found with a given probability. Thus, the result will be within the interval but all values within the interval have the same probability to represent the result. This is quite different from other distributions, e.g. the Gaussian distribution where the mean is the most probable value. Therefore, there is not necessarily any symmetry in the concept of uncertainty. There is no value that is more probable or common than any other. It is convenient, though, to let a number represent the distribution and one can choose any representative value from the interval. One is as good as the other but the central value has advantages from a presentation point of view. By these statements we have in fact described the essential features of a 'rectangular distribution'
The calculation or estimation of the uncertainty in measurements is principally very simple. This is done by estimating the uncertainty in each single step of a procedure (input variables, X i ) and combining them in an 'uncertainty budget'. The combination of the individual uncertainties follows similar rules as the propagation of errors in normal statistics giving the 'combined uncertainty' for the procedure. The combined uncertainty is the concept that is closest to 'the total error'.
In the examples in this document we will use a worksheet from Excel 5.0 that simplifies these calculations and do not require any knowledge of the mathematics behind. The worksheet can be used for evaluation of all uncertainty budgets that deal with independent observations.
It is essential when establishing a realistic uncertainty budget to identify the variables that give rise to the uncertainty and their sizes. Therefore one must have detailed knowledge about the procedure of measurement to allow identification and quantification of all reasonable sources. Examples of sources of uncertainty in our field are the measurement of volumes, weighing, reaction temperature, purity of reagents, and value assigned to the calibrator. Also properties of the instrument used e.g. the size and tolerance of the cuvette, if the instrument uses more than one cuvette they may be different, the wavelength etc. If a visual recording is made one must consider the position of the eye in relation to the scale, i.e. one source of uncertainty that can be referred to the operator. When a calibration function is established one must estimate the uncertainty of the value assigned to the calibrator. Possible sources that might influence the calibration function are for instance the assigned value of the calibrator, the fitting of the curve, the dilution of the calibrators etc. When an extraction is included one must estimate the yield. Even if all sources of bias were eliminated an uncertainty of the success of that procedure remains and should be estimated.
All these uncertainty sources must be evaluated and brought into the uncertainty budget. An experienced chemist may directly disregard some of the uncertainty sources because the experience is that they are very small or because those remaining are much larger.
The procedure of measurement shall then be described in mathematical terms to illustrate how the different variables affect each other and how their uncertainties can be joined in a combined uncertainty. This procedure is tricky and requires experience from establishing stochiometric calculations i.e. the qualitative and quantitative description of a chemical reaction and calculations of concentrations and amounts involved.
The estimation of the size of the sources of uncertainty can be done in two different ways called type A and type B. The value and results of both approaches are treated similarly but the type A is recommended whenever possible. In both cases the 'standard uncertainty' is estimated. The standard uncertainty is abbreviated u(x i ).
In this model the estimation is based on the standard deviations derived from repeated measurements. The SD and the standard uncertainty will therefore have the same size. A word of warning: sometimes the variation is given as a coefficient of variation (CV) or confidence interval rather than SD. That information can also be used but it must then be corrected for what the numbers represent e.g. multiply with the result of measurement and divided with 100 if the CV is given in percent or multiplied with the square root of the number of observations if the confidence interval or SEM (standard error of the mean) is given. One must also clarify how many SD the information refers to. As a rule the number is one but sometimes multiples are given, for instance two SD.
Sometimes, and particularly in complex measurement procedures, we do not have access to or we are not able to estimate the variation from repeated experiments, which is a prerequisite for the estimation of the SD. The professional experience, information in the literature or specifications from a manufacturer will usually allow the demarcation of an interval within which a result can reasonably be expected. For instance, it is fairly safe to assume that the European woman is between 130 cm and 210 cm tall, that a litre of milk costs between USD 0,1 and 2 and that one litre of water has a mass between 994 g and 1004 g. The more one knows about the procedure of measurement and the items measured the better will the estimate of interval be. In laboratory medicine it is often important to find a reasonable interval for the volume, the mass or the reading of an instrument. For instance a 1 mL pipette might deliver between 0.90 mL and 1.05 mL, the body mass of a grown up man without excessive fat might be between 65 and 85 kg; based on your experience your might even assume that it is between 69 kg and 76 kg etc.
The interval that is defined in this way will not identify any result within the interval as being more reasonable than any other in the same interval. Thus, the interval represents a rectangular distribution of possible results.
Throwing a single dice will give results belonging to a rectangular distribution. One cannot foresee the number of dots that will come up but a reasonable (the largest possible) interval is 1 - 6. In fact no other results can be obtained. The probability for each result within the interval is the same, provided the dice is fair. The mean of the interval (3.5), however, can never be obtained!
Given an interval there is a somewhat smaller interval within which the probability that a result will occur is increased. This new interval is created from the half - width of the initial interval divided by the square root of 3 (about 1.73) in both directions from the middle of the initial interval. It is about twice as common that results will be within this interval as outside these limits. Using the example with the dice, the inner interval will be from 2 to and including 5. For the dice it means that between 2 and 5 there are four alternatives (2, 3, 4 and 5), whereas outside there are only two, 1 and 6. It is therefore more probable that any of the four dots 2 to 5 will show up than any of the two outside the smaller interval. The probability that any given number of dots within the new interval will be obtained is the same as outside and one cannot predict which. I.e. it is more probable than one of four given alternatives is obtained than one of two.
The width of this inner interval is twice the standard uncertainty 2 u(x i ).The standard uncertainty will correspond approximately to the probability within the interval mean � 1 SD in a Gaussian distribution where about 2/3 of all observations will be found. You can convince yourself about this by playing with a good dice and you would expect 2/3 of all answers to be found between and include 2 and 5 dots.
An example from our own profession is the estimation of the uncertainty of a measured volume using a two - litre measurement cylinder. Suppose we want to measure 500 mL, and assume a reasonable interval to be � 3 % or (485-515) mL. The standard uncertainty is then 15 (half the interval) divided by the square root of 3 i.e. 8.7 mL.
Once the sources of uncertainty have been identified and the sizes of the uncertainties estimated then we face the problem to combine the contributions to an uncertainty for the entire procedure i.e. the combined uncertainty in the result y. This is abbreviated u c (y).
Let us begin with an example:
We want to make a dilution of a sample and are only interested in the final volume. According to the procedure we shall take 10 mL of the sample solution and dilute that to 500 mL. We use the measuring cylinder from the example above and measure the sample with a pipette with a relative uncertainty of 5 %. Thus, we now have each volume defined to its size and accompanying uncertainty. Observe that the uncertainty of the measuring cylinder is given as an interval whereas the uncertainty for the pipette is given as a relative standard uncertainty.
The volumes V 1 and V 2 should be added and thus the 'reaction' could be written
The standard uncertainties are added but not as such but as their squares:
The combined uncertainty is then obtained by taking the square root from the sum, in this example 8.7 mL. Perform the calculations yourself and find that the contribution from the pipette is of minor importance in the combined uncertainty!
The standard uncertainty will thus demarcate an interval where we can estimate that 2/3 of all the results will be found. This is also the usual way to give the variation (usually the standard deviation) of results in scientific literature. If we want to find the interval that is large enough to give a 95 % probability to cover the results in a Gaussian distribution the SD should be multiplied with about 2 for a two - tailed distribution (the size of the factor is depending on the number of observations). To reach a corresponding probability for the combined uncertainty it shall be multiplied by 2. If we wish a larger probability then a larger factor, for instance 3 for about 99 % probability, should be used. This factor is called the coverage factor ( k ) and the result expanded uncertainty U y . The coverage factor shall always be given in the answer together with the uncertainty. In case no coverage factor is given then the combined standard uncertainty covering about 2/3 of the result is given (k=1).
When estimating the combined uncertainty ( u c (y)) the starting point is to define how the different parts of the procedure interact. In our model we assume that the uncertainty sources are independent. In the example above the two volumes were added to reach the total volume. For additions (subtractions), the combined uncertainty is the square root of the sum of the squares of the ingoing standard uncertainties. In case the variables shall be multiplied (divided) the squares of the ingoing relative standard uncertainties shall be added. When the square root is drawn the relative combined uncertainty is achieved.
The mathematical expressions that describe how the standard uncertainties shall be combined become rather complicated when the uncertainty budget for a procedure contains all four operators and a variable may participate as a logarithm or exponent. The generic rule for the combination is based on partial derivatives but we will use an approximate numerical method for solving partial derivatives. This can conveniently be achieved by using a spreadsheet program like Excel�.
Kragten (4) originally described the numerical approximation of partial derivatives. The document from Eurachem (5) describes how this method can be used in a worksheet.
All cells in the worksheet that will be filled by calculations are protected to avoid unintentional changes in the program. Cells that can be changed are marked with a blue border. The number of decimals is fixed to three that will give a sufficient presentation of results within laboratory medicine even if it sometimes exaggerates the precision. You can perform calculations directly in that cell where the value finally will be placed. Never copy the contents of a cell into another, delete and input it again or unforeseeable errors may occur.
It is suggested that you enter different data into the given examples to see how the outcome changes.
Enter the names of the input variables in row 3 of the worksheet. The input variables can be entered in any order. The corresponding names will appear in column B, rows 10 to 18. Enter actual or representative values in row 4 and the standard uncertainty of the results in row 5 or 6 depending on if the standard uncertainty is given in absolute (row 5) or relative terms (row 6). Relative uncertainties shall be given as parts of 1, i.e. 1.5 % should be 0.015.
Then the interrelation between the various components of the budget shall be entered in cell C21 ('Nominal'), i.e. how the final result shall be calculated from the input variables. Often this formula presents itself easily but it can sometimes be rather complicated and require deep thoughts. It may be helpful to establish and solve an equation that describes how to calculate the result. In the dilution example above the expression will be simply the volume of the original sample plus the dilution volume. Compare example 1.
The expression which goes into cell C21 will be a mathematical expression and shall therefore, with the Excel� nomenclature, begin with "=" ("equal sign"). Then, enter the algorithm and drag the contents of C21 as far to the right as variables have been entered. Finally press "enter" and all calculations will be carried out. Note that the cells to the right of the last used column in row 21 shall be empty. If not, it is safe to mark them and press delete.
Depending on how your computer has been configured you may need to press the F9 key to trigger the calculations. If necessary you can change the configuration to 'automatic' under 'Tools-Options-Calculation' and check 'Automatic'.
The combined uncertainty will be given on the last row in both absolute and relative terms. Also the coverage factor (default =2) and the uncertainty interval will be given. Contributions from the different sources of uncertainty are shown on row 23 and graphically in the inserted diagram. The diagram can be freely moved within the surface if you want to study the underlying contents of the table. The scale of the Y - axis can also be changed to improve the presentation.
The following examples are chosen to illustrate an increasing complexity in the calculation and originate in routine laboratory work. The uncertainties and other numbers do not necessarily represent reality but have been chosen to illustrate their influences. Each example is solved on the attached diskette where also you will also find a template for the calculations.
Let us repeat the example above dealing with adding two volumes and use the template.
Enter the nominations "Sample" and "Dilution" in the correct cells in row 3 and the volumes 10 and 490 respectively in row 4. The standard uncertainties are entered in row 5 and 6 (absolute and relative standard uncertainty, respectively). Move the cursor to cell C21 under the label "Nominal" and enter the formula '= C9+C10'. Copy this formula two columns to the right. The combined uncertainty will be shown in cell C25, the relative in E25 and the interval calculated with a coverage factor 2 in the cells K25 - L25. The expanded uncertainty is given in cell I25. compare the result with our manually worked example.
We want to weigh NaCl on an old fashioned scale. The empty vessel weighs 12 g. We add NaCl until 127 g. Weighing within the given interval can be made with an uncertainty of 1,5 %. Calculate the mass of the NaCl.
In cell C21 the formula will be:' =final weight-weight of empty vessel (tare)'. Test the result if the uncertainty is 0.2 g!
We shall calculate the volume of a water bath which has a length of 35 cm, width 25 cm and height 20 cm. The edges are measured using a ruler with an uncertainty of 2 %.
The formula in C25 will be: '=the length x width x height'. Check the result if the uncertainty of measurement in the interval is 0.5 cm!
Let us add a slight complication to example 3. Let us assume that the water bath is half - filled (10 cm) and you want to fill it up to 3 cm (the margin) from the upper rim. How much water should be added? We make the same assumptions regarding the uncertainty of the lengths as above, but since the vessel walls are not quite even we must add the uncertainty this contributes. Let us assume that this uncertainty is multiplicative and optimally 1 but with the uncertainty 3 %.
The formula in C21 will be '"length x width x (the height - marginal - the height at half - filled water bath) x unevenness factor'. Test the results if the uncertainty in all lengths is 0.2 cm!
Let us increase the complexity of example 1 above. Assume that the sample concentration is 320 mmol/L with a standard uncertainty of 3 %. What is the concentration in the final solution and what is its combined uncertainty? Enter the numbers to the template as in example 1 but add the concentration of the sample solution in a column of its own.
The formula will be: '=sample x concentration/(sample + dilution)'.
We want to estimate the uncertainty in an HPLC - method. The sample is diluted 1 + 9 with a solution containing 0.25 mmol/L internal standard (IS). We inject 20 �L of the dilution. The sample peak is 247 mm, the IS - peak is 235 mm. How much substance did the sample contain and which is the combined uncertainty of the result? The standard uncertainty in the sample volume is � 3 % and for the dilution volume � 2 %. The standard uncertainty in weighing of IS is � 1 %. Measuring of the peaks is associated with an interval of � 2 %. The same amounts of IS and sample give the same response on the printer with an uncertainty of � 3 %.
The relation between signal and concentration (i.e. the calibration function) is: signal = concentration x 2-1,5; (Y=2 x X-1,5) in the concentration interval 0,5-5 units. The standard uncertainty in the slope (b) is 2 % in the intercept (a) 5 % and in the signal 0,01 units. Estimate the combined uncertainty in the middle of the interval, i.e. 2,75 units.
We compare the results from measuring the same samples on two different instruments. The results are assumed to be identical, that is the regression function is assumed to be Y = X. Which deviation in Y results can we expect if the standard uncertainty in the estimation of the coefficient is 5 % and the interval of the intercept is -0,5 - + 0,8 at the critical limit 17 mmol/L? The standard uncertainty in the reference method is 3 %.
The absorbance of a sample is measured at 580 nm. The specific absorbance for the substance at this wavelength is 12,5, the absorbance curve is linear between 540 and 620 nm with a slope of 0.001 absorbance units/nm. The sample was diluted 1 + 9 and the absorbance 1.35. Estimate the concentration and its uncertainty if the standard uncertainties in the volume measurements are 2 % and 1,5 % per sample and dilution, respectively, the standard uncertainty of the specific absorbance (u a ) 0,5 %, in the reading 1 %, in the slope (u l ) 5 % and the wavelength of the filter is given as an interval of 580 nm � 5 nm.
There are many factors involved in this example and let us argue like this: We must somehow translate the uncertainty in wavelength to uncertainty in specific absorbance. This can be done by calculating the specific absorbance at another wavelength. Let us assume 560 nm. This wavelength has no uncertainty because we assume it is without uncertainty. At exactly 580 nm the specific uncertainty is 0.125. We can include the uncertainty in the slope of the absorbance curve by (580 - 560) x 0.001 � u l . The specific absorbance at 560 nm will therefore be (0.125 � u a -20 x 0.001 � u l ). We can then formulate the absorbance at 580 nm � 5 nm. It will be (0.125 � u a -20 x 0.001 � u l ) + 580 nm � 5 nm-560) x (0.001 � u l ).
Although the wavelength has a small uncertainty this will be the dominating source of uncertainty. Discuss what happens if the slope increases or if the uncertainty in the volume measurements is changed.
The concept of uncertainty can also be used for other purposes than measurements and it can include for instance preanalytical and post analytical sources of uncertainty. Let us examine this example that is a simplification of an article in "Clinical Chemistry and Laboratory Medicine" (6).
B - Glucose is used in the primary health care for diagnosis and control of diabetes. Which is the smallest difference in results that, with a given probability can be assumed to be different?
The patient shall be fasting. The patient shall be calm and relaxed to avoid that catecholamines and other hormones will give a falsely elevated glucose concentration. The concentration of glucose in erythrocytes is less than in plasma and therefore one also has to control the intake of fluids to avoid falsely low results and increased hematocrit. The capillary sampling is difficult. One cannot avoid a varying addition of interstitial fluid etc. There are instruments on the market that measure P - Glucose and use an algorithm to transfer the value to B - Glucose. The algorithm is based on the assumption that P - Glucose is 15 % higher than B - Glucose, which however is depending on the hematocrit of the patient. Finally we also have uncertainty contributions from the measurement itself. All these uncertainties shall be considered in estimating the combined uncertainty that shall be the basis for estimating the smallest significant difference between two results.
Let us select 6.0 mmol/L as an interesting concentration.
Fasting and fluid intake.Just standing up can cause to up to 15 % changes of the plasma volume. Between day variation of the plasma volume is given in the literature (7) to � 6.5 % based on repeated measurements. Let us assume that this includes the variations in stress, fluid intake and fasting but excluding whether the patient is sitting, standing or resting.
Sampling:Literature (8) postulates an uncertainty of � 3.2 % for S - Glucose also based on repeated measurements. For capillary sampling it is reasonable to add a little, � 5 %.
Measurement:The precision of the measurement is related to the instrument. An instrument like Hem cue can give an imprecision of � 3 - 5 %, many of the other instruments the same order and magnitude if one uses the same batch of reagent strips. Between batches the manufacturers may allow as much as up to 8 % imprecision. Let us assume that we can manage an interval of � 5 % uncertainty that also includes the uncertainty of calibration.
Post analyticalIf the measurement is made with an instrument that really measures B - Glucose this source of uncertainty is not interesting. If, however the instrument measures something different then one must divide with a factor. Some literature suggests 1.17 but it is closer to 1.11 (6). Regardless of the value of the factor it is attached to an uncertainty, let us assume 5 %.
The smallest difference that can be called significant is
where D is the difference, u c (y) is the combined uncertainty and k the coverage factor. The square root is because two values are compared. A convenient abbreviation is D=3 x u c (y).
1. ISO/DIS 15189. Quality management in the medical laboratory, ISO, Geneva 2000.
2. ISO/IEC IS 17025. General requirements for the competence of testing and calibration laboratories. Geneva 1999.
1. Guide to the Expression of Uncertainty in Measurement (GUM). ISO Geneva, Schweiz 1993 (ISBN 92 - 67 - 10188 - 9)
2. J Kragten. Calculating standard deviations and confidence intervals with a universally applicable spreadsheet technique. Analyst 1994; 119: 2161 - 2166.
3. Eurachem, Quantifying Uncertainty in Analytical Measurement, LGC Information Service, Teddington Middlesex 1995 ISBN 0 - 948926 - 08 - 2
4. A. Kallner, J Waldenstrom Does the Uncertainty of Commonly Performed Glucose Measurements Allow Identification of Individuals at High Risk for Diabetes? Clin Chem Lab Med 1999;37:907 - 912
5. C Ricos et al. Current databases on biological variation: pros, cons and progress. Scand J Clin Lab Invest 1999;59(7):491 - 500.
6. X. Fuentes - Arderiu et al. Pre - metrological (Pre - Analytical) Variation of some Biochemical Quantities. Clin Chem Lab Med 1999;37:987 - 989
Three important definitions by ISO :
Trueness,agreement between the average value from many observations and the true value
Precision,agreement between independent results of measurement
Accuracy, agreement between the result of a measurement and a true value of the measurand
Note the difference between Trueness and Accuracy; the latter refers to one measurement whereas the former to the mean of many measurements. Therefore inaccuracy will include both the bias and the imprecision inherent in a specific result.
Terminology and nomenclature are corner stones of science and a citation from L Carroll 'Alice through the looking glass' might be appropriate to consider:
'When I use a word' Humpty Dumpty said in rather a scornful tone, 'it means just what I choose it to mean? neither more nor less'
'The question is,' said Alice, 'whether you can make words mean different things'.
'The question is, said Humpty Dumpty, ' which is to be master ? that's all'.
Copyright © 2002 International Federation of Clinical Chemistry and Laboratory Medicine (IFCC). All rights reserved.