Advancing excellence in laboratory medicine for better healthcare worldwide

Optimal analytical performance for poct by c g fraser

  
Optimal Analytical Performance for POCT

Article ID 1301200106
Author, Callum G Fraser
Biochemical Medicine, Tayside University Hospitals NHS Trust, Ninewells Hospital and Medical School, Dundee DD1 9SY, Scotland

Download as a PDF here

Abstract
Quality specifications for the reliability performance characteristics of laboratory tests, particularly precision and bias, are necessary prerequisites for creation and control of analytical quality. Many strategies have been promulgated for setting these. Recently, the available approaches have been fixed into a hierarchical framework that has now been agreed by experts in the field to be the best current approach to a global strategy to set quality specifications in laboratory medicine. They should be incorporated into quality planning strategies everywhere i rrespective of the settings in which laboratory medicine is practised, including POCT. Models higher in the hierarchy are preferred to lower approaches but lower approaches are better than none and should be used if all that are available.
Introduction
Every analytical method, irrespective of where it is actually performed, can be described fully in terms of its performance characteristics. These are of two types, practicability performance characteristics and reliability performance characteristics. The former include skills required, speed of analysis, volume required, and type of sample required. The latter include precision, bias, limit of detection, and measuring range. It is often suggested that, for point of care testing [POCT], considerations of speed of analysis - expressed as total turnaround time - surpass all others. However, quality specifications for the reliability performance characteristics of laboratory tests, particularly precision and bias, are absolutely necessary prerequisites for analytical quality management. Moreover, such analytical quality specifications should be firmly based upon medical requirements, useable in all laboratories irrespective of size, type or location, generated using simple to understand models, and widely accepted as cogent by professionals in the field.
Quality specifications are required in many facets of the discipline, including generating specifications for new analytical systems, assessing available literature to assist in method selection, evaluating submitted tenders, assessing data generated in method validation, and creating appropriate internal quality control and external quality assessment schemes which guarantee the specified analytical quality. A plethora of papers, reviews, and book chapters dealing with the generation and application of quality specifications has been published over time [1]. However, there still seem to be real dilemmas in deciding on appropriate quality specifications, particularly for precision and bias. Although there are many very logical reasons for this situation, a crucial recent development was that a consensus was reached in 1999 on global strategies to set quality specifications in laboratory medicine [2]. This consensus was based upon a hierarchical approach published just prior to the consensus conference [3].
The hierarchy and its application to the setting of analytical quality specifications for precision and bias are the subjects of this review. Examples used are taken from those quantities often measured in POCT settings.
The hierarchy of strategies to set quality specifications
The hierarchy shown in Table 1 has been agreed by experts in the field to be the best current means to classify the available strategies.
Assessment of the effect of analytical performance on specific clinical decision-making
Quality specifications in specific clinical situations.
Assessment of the effect of analytical performance on general clinical decision-making.
2A. General quality specifications based on biological variation.
 
2B. General quality specifications based on medical opinions.
Professional recommendations.
3A. Guidelines from national or international expert groups.
 
3B. Guidelines from expert individuals or institutional groups.
Quality specifications laid down by regulation or by external quality assessment scheme [EQAS] organisers.
4A. Quality specifications laid down by regulation.
 
4B. Quality specifications laid down by EQAS organisers.
Published data on the state of the art.
5A. Published data from external quality assessment and proficiency testing [PT] schemes.
 
5B. Published individual methodology.
 
Table 1. Hierarchical approach to classification of strategies.
Assessment of the effect of analytical performance on specific clinical decision-making
Clearly, the first choice should logically be the strategy at the top of the hierarchy. Thus, analytical quality specifications should be derived from analysis of the effect of analytical quality on medical decision-making in specific clinical situations. A first example is provided by consideration of cholesterol assays.
If the POCT methodology had a positive analytical bias, then the population distribution would move to the right and "false positive" results would be found irrespective of the clinical decision making criterion used for patient classification. If the clinical strategy was to treat with either lifestyle advice, diet or drugs [which would all entail further laboratory tests and recall], or even to simply repeat of the test, then additional health care resources would be spent and more of the population would be labelled as "at greater risk". In contrast, if the laboratory had negative bias, the distribution would shift to the left; the number of "false negatives" would increase, saving costs on additional testing in the short term, but possibly eventually leading to huge health care costs as the population missed at initial testing succumbed to premature coronary artery disease.
This situation can be assessed more formally [4]. In Figure 1, the distribution of serum cholesterol concentrations in a Danish population is shown [upper panel].



Figure1. The effect of negative [middle panel] and positive [lower panel] bias on the percentage of the population at risk [above 7.0 mmol/L]. From Hyltoft Petersen P, Horder M. Scand J Clin Lab Invest 1992;58 [suppl 208]:79.
The effect of negative and positive biases of 10% on those of at high risk, that is, purely for illustrative purposes here, those with true serum cholesterol concentration of greater than 7.0 mmol/L, can be easily calculated. Subsequently, the calculation can be done for all values of bias. The functional relationship between the decreases and increases in the percentage of the population at high risk and analytical bias is as shown in Figure 2. If the medical needs in terms of allowable percentage misclassification could then be defined, the allowable analytical bias - the analytical quality specification - can be easily calculated.



Figure2. Functional relationship between the percentage of the population at high risk and analytical bias. From Hyltoft Petersen P, Horder M. Scand J Clin Lab Invest 1992;58 [suppl 208]:79.
Investigation of the relationship between medical needs and analytical performance can be done in a similar manner for other quantities and we have explored this in some detail [5]. For example, there is a relationship between the risk of microalbuminuria and the blood glycated haemoglobin concentration. Figure 3 shows the consequences of analytical bias on the risk in an individual with a true glycated haemoglobin of 10.1%.



Figure 3. Influence of analytical bias on the risk of microalbuminuria. From Hyltoft Petersen, et al. Clin Chim Acta 1997;260:200.
If negative bias was present, the reported values would be less than 10.1%, the clinician would imagine that the patient was under reasonable control and not change therapy in any way - the patient actually has a higher glycated haemoglobin concentration, less good glycaemic control and a greater risk of microalbuminuria [and the other sequelae of poor control]. In contrast, if the analytical method had positive bias, the glycated haemoglobin would appear lower: the clinician might congratulate the patient on maintaining good control but, while the risk of microalbuminuria might be lower, the risk of hypoglycaemic episodes might be increased. Thus, deciding the acceptable clinical outcomes could allow clear definition of acceptable analytical performance.
However, a significant problem with this approach is that only very few tests are used in single, well-defined clinical situations. Moreover, quality specifications calculated depend very much on the assumptions made about how the test results are used by clinicians - even for glycated haemoglobin assays [6].
Assessment of the effect of analytical performance on general clinical decision-making
The second strategy in the hierarchy is the creation of quality specifications based on general ways in which clinicians use test results. Quality specifications for precision and bias in monitoring and diagnosis can be based on data on the components of biological variation, namely, within-subject [CV I ] and between-subject [CV G ] variation.
In clinical monitoring, analytical random variation must be kept low so that changes in test results in an individual are significant, with high probability, and that these do not simply reflect analytical random variation. This is particularly important when POCT is considered because, at least traditionally, the analytical performance achieved in sites other than the laboratory were inferior and the results were intrinsically more variable. It should be noted that one of the alleged advantages of POCT is that patients can be monitored closely and frequently. Irrespective of the time scale, monitoring involves comparison of test results from an individual over time.
In the simplest "homeostatic" model, changes in serial results can be due to ?
�         the patient getting better,
�         the patient getting worse,
�         pre-analytical variation,
�         biological variation [within-subject] and analytical variation - changes in bias and inherent precision [CV A ].
Thus, if pre-analytical sources of variation are minimised, then, to assess whether change has occurred, it must exceed the inherent variation due to biological and analytical variation which is defined as the reference change value. The reference change value [RCV] can be calculated as -
2 � Z � [CV A 2 + CV I 2 ]
where Z is the number of standard deviates appropriate to the probability selected [for example, 1.96 for P < 0.05 and 2.56 for P < 0.01].
It is simple to demonstrate the effect of precision on medical decision-making. Taking cholesterol [CV I ~ 6%] as an example, the change required for significance [at P < 0.05] increases with precision as shown in Table 1.
Precision [CV, %]
RCV [%]
2
17.5
4
20.0
6
23.5
8
27.7
10
32.3
 
Table 1. Effect of precision on reference change value [RCV] for serum cholesterol at P < 0.05
For precision, the widely accepted quality specification is that the analytical variation [CV A ] should be less than one-half the average within-subject biological variation [7]. Harris showed that, if CV A < 0.50CV I , then the amount of variability added was about 10% [in reality, 11.8%] which was stated to be "reasonable" [8]. This proposal has been very widely accepted by professionals. Furthermore, this idea has been expanded more recently and three classes of analytical quality,optimum, desirableandminimum, based upon different fractions of within-subject biological variation have been proposed as shown in Figure 4 [9].



Figure 4. Percentage increase in test result variability due to analytical precision [expressed as a ratio of analytical to within-subject biological variation] showing three possible quality specifications based on within-subject biological variation. From Fraser CG et al. Ann Clin Biochem 1997;34:8-12.
Although there are many strategies for the interpretation of laboratory test results in diagnosis, many use population-based reference values, particularly the less experienced. It is often the case that patients have tests done in various locations such as the emergency room, the outpatient clinic, and the ward - in which POCT may be used - and in the laboratory. Clearly, test results should be comparable over location. In consequence, the ideal is that all testing sites serving a homogeneous population should all use the same reference values. For this to be achieved, it has been shown [10] that bias should be less than one-quarter of the group biological variation [that is, BIAS < 0.25[CV I 2 + CV G 2 ] ].Again, three classes of analytical quality,optimum, desirableandminimum, based upon different fractions of within- plus between-subject biological variation, have been proposed as shown in Figure 5 [9].



Figure 5. Percentage of results outside reference limits due to analytical bias [expressed as a ratio of analytical to group (within- plus between-subject) biological variation] showing three possible quality specifications based on biological variation. From Fraser CG et al. Ann Clin Biochem 1997;34:8-12.
These well established approaches have advantages in that data on components of biological variation are available for more than 200 quantities. A recent compilation in the easily available literature makes the data easy to obtain [11], and the data seem independent of study location, number of subjects, length of study, analytical methodology, age of subjects or whether they are in a state of health or have stable but chronic disease. Moreover, data on components of biological variation have been used to define quality specifications for other characteristics and in other laboratory settings [12].
Quality specifications sometimes alleged to be based on "medical needs" have been calculated from the responses of clinicians to a series of short case studies [vignettes] on the general interpretation of test results. Most of these studies have significant deficiencies in design and execution: these problems and potential solutions have been debated again recently [13].However, the best example of this approach is that of Thue et al [14] who derived quality specifications for the precision of analysis of one quantity only [haemoglobin] through a series of vignettes submitted to a large single speciality clinical group [general practitioners in Norway]. This study could be used as a model for future similar vignette studies.
Professional recommendations
Certain national or international professional groups have published quality specifications. For example, the recommendations of the National Cholesterol Education Panel [US] have been used extensively [15] as have the detailed recommendations of the American Diabetes Association [16] for self-monitoring of blood glucose. The latter have evolved over time; a major problem with these particular guidelines is that they seem empirical and it is, in fact, quite difficult to interpret what they actually mean. Moreover, the quality specifications laid down by experts often differ quite markedly.
Additionally, certain quality specifications have been proposed through publication of guidelines based on what could be viewed as best or good laboratory practice. These are often presented or developed at a single consensus conference without significant discussion. However, these guidelines have the advantage that they are usually generated from the very broad experience of either a single expert or an expert group from a single institution.
Quality specifications laid down by regulation or by external quality assessment scheme [EQAS] organisers
The acceptable standards of analytical performance required have been laid down in a number of countries. The best example is the US CLIA'88 legislation [17]that documents acceptable total error for a number of commonly assayed analytes. The major disadvantage of these quality specifications is that, although based on expert views, they tend to be empirical and are clearly influenced by what is actually achievable at the time [the state of the art].
EQAS use a variety of measures of location and allowable dispersion. In Europe, some use statistical analysis of the data returned from the participant laboratories but, more and more, fixed limits are used [18]. Again the problem of quality specifications based upon these fixed limits is that, although often based on expert opinion, they tend to be subjective and are affected by the state of the art.
Published data on the state of the art
Quality specifications could be generated through reference to the performance achieved by groups of laboratories participating in EQA and PT schemes. This has the advantage that many data are often available. However, for a number of obvious reasons, true analytical performance may not be accurately mirrored by this apparent state of the art.
Measures of the quality of analytical performance could be obtained be comparison with attainment documented in published works on similar or other assay methods for the quantity for which quality specifications were required. This has some merit in that many data are often available, but has the real difficulty that published method performance may be the best possible rather than that achieved in practice. Again, performance achieved analytically may bear no relationship to actual medical needs. With regard to POCT, a problem is that many evaluations of technology are done in laboratories with well-trained staff only and are not done by the clinical staff who would actually do the procedures in practice. Moreover, traditionally, for example in a study done by us on cholesterol assays in the Coronary Care Unit [19], it was considered that the state of the art achieved by clinical staff was inferior to that attained by laboratory staff. However, modern technology does seem to allow results to be obtained which are operator independent and after minimal training [20].
Conclusions
A hierarchy of approaches to set analytical quality specifications has been created and approved by expert professionals. The hierarchy should be applied in practice. These simple to understand models are appropriate for all settings in which laboratory medicine is practised, including POCT, and they should be incorporated into quality planning strategies everywhere. As we have stated previously in a review on quality specifications for analyses done in alternate sites including POCT [21], there is no reason why different standards are warranted, and we have tabled general numerical analytical quality specifications based on biological variation for test commonly performed as POCT. Clearly, models higher in the hierarchy are preferred to lower approaches but lower approaches are better than none and should be used if all that are available. New useful models may be developed in the future and these should be incorporated into the hierarchical scheme when widely approved by professionals in laboratory medicine.
References
Fraser CG. Quality specifications in laboratory medicine. Clin Biochem Revs 1996;17:109-14.
Hyltoft Petersen P, Fraser CG, Kallner A, Kenny D, eds. Strategies to set global analytical quality specifications in laboratory medicine. Scand J Clin Lab Invest 199; 59:475-585.
Fraser CG, Hyltoft Petersen P. Analytical performance characteristics should be judged against objective quality specifications. Clin Chem 1999;45:321-3.
Hyltoft Petersen P, Horder M. Influence of analytical quality on test results. Scand J Clin Lab Invest 1992;58[suppl 208]:65-87.
Hyltoft Petersen P, deVerdier CH, Groth T, Fraser CG, et al. The influence of analytical bias on diagnostic misclassifications. Clin Chim Acta 1997;260:189-206.
Lytken Larsen M, Hyltoft Petersen P, Fraser CG. A comparison of analytical goals for haemoglobin A1c assays derived using different strategies. Ann Clin Biochem 1990;28:272-8.
Cotlove E, Harris EK, Williams GZ. Biological and analytic components of variation in long-term studies of serum constituents in normal subjects. III. Physiological and medical implications. Clin Chem 1970;16:1028-32.
Harris EK. Statistical principles underlying analytic goal-setting in clinical chemistry. Am J Clin Pathol 1979; 374:72-82.
Fraser CG, Hyltoft Petersen P, Libeer JC, Ricos C. Proposals for setting generally applicable quality goals solely based on biology. Ann Clin Biochem 1997;34:8-12.
Gowans EMS, Hyltoft Petersen P, Blaabjerg O, Horder M. Analytical goals for the acceptance of common reference intervals for laboratories throughout a geographical area. Scand J Clin Lab Invest 1988;48:757-64.
Sebastian-Gambaro MA, Liron-Hernandez FJ, Fuentes-Arderiu X. Intra- and inter-individual biological variability data bank. Eur J Clin Chem Clin Biochem 1997;35:845-52 [also available at www.westgard.com ].
Fraser CG. Quality specifications in laboratory medicine - current consensus views. Accred Qual Assur 1999;4:410-13.
Fraser CG. Judgement on analytical quality requirements from published vignette studies is flawed. Clin Chem Lab Med 1999;37:167-8.
Thue G, Sandberg S, Fugelli P. Clinical assessment of haemoglobin values by general practitioners related to analytical and biological variation. Scand J Clin Lab Invest 1991;51:453-9.
National Cholesterol Education Program Laboratory Standardization Panel. Current status of blood cholesterol measurement in clinical laboratories in the United States. Clin Chem 1988;34:193-201.
American Diabetes Association: Consensus statement on self-monitoring of blood glucose. Diabetes Care 1994;17:81-6. .
US Dept. of Health and Human Services. Medicare, Medicaid, and CLIA programs: regulations implementing the Clinical Laboratory Improvement Amendment of 1988 (CLIA). Final rule. Fed Reg 1992;57:7002-186.
Ricos C, Baadenhuisjen H, Libeer JC, Hyltoft Petersen P, et al. Currently used criteria for evaluating performance in EQA in European countries and a comparison with criteria for a future harmonisation. Eur J Clin Chem Clin Biochem 1996;34:159-65.
Nelson LM, Clark RS, Fraser CG. A laboratory and in-ward evaluation of cholesterol assays on the Ames Seralyzer. J Autom Chem 1985; 7: 173-6.
Bingham D, Kendall J, Clancy M. The portable laboratory: an evaluation of the accuracy and reproducibility of I-STAT. Ann Clin Biochem 1999;36:66-71.
Fraser CG, Hyltoft Petersen P. Desirable performance standards for imprecision in alternate sites. Arch Pathol Lab Med 1995;119:909-13.
Acknowledgement
The author thanks IM Kennedy for the exemplary perspicacity evidenced in the preparation of the manuscript of this paper.
Correspondence to:
Dr Callum G Fraser, Biochemical Medicine, Tayside University Hospitals NHS Trust, Ninewells Hospital and Medical School, Dundee DD1 9SY, Scotland.
Telephone +44 1382 660111
FAX +44 1382 654333
e-mail callumf@tuht.scot.nhs.uk
Copyright © 2002 International Federation of Clinical Chemistry and Laboratory Medicine (IFCC). All rights reserved.
 
Website developed by Insoft Digital