Gordon Challand and Jacqui Osypiw
Consultant Clinical Biochemists
Department of Clinical Biochemistry, Royal Berkshire Hospital,
Reading, Berkshire RG1 5AN, UK

Download as a PDF here

The need for Departments of Clinical Biochemistry and of Laboratory Medicine to provide both pre- and post-analytical advice has never been greater. With increasing specialisation of Hospital Clinicians; with an increasing number of investigations being requested directly from Primary Care; and with an ever-increasing range of specialised tests being available, there are many possibilities of inappropriate investigation and of incorrect interpretation of unfamiliar tests by clinical staff. Obvious examples are a conclusion of digoxin toxicity from a sample taken too soon after the last dose of digoxin or of a low serum transferrin being interpreted as synonymous with iron deficiency. Our own Department receives some 2000 samples a day (over half of which come from Primary Care), from which more than 1000 reports contain significant abnormalities. It is totally impracticable for our Department to contact requesting Clinicians directly about every report that contains unexpected abnormalities or that may be liable to misinterpretation.

Today, most Departments of Clinical Biochemistry in the UK use a Duty Biochemist.  Duty Biochemists (whether medically or scientifically qualified) are usually senior members of laboratory staff, working on a rota. Their function is to scan reports containing abnormal results (typically many hundreds each day), and identify reports containing unexpected abnormalities. �Unexpected� can be in the context of previous results on the same patient, or abnormalities at variance with the clinical information given about the patient. Faced with such a report, the Duty Biochemist has several options: to let the report go without further action; to ask for analyses to be checked if there is doubt about quality control; to add further tests in the hope of eliciting further useful information; to telephone the responsible clinician; to visit the ward; or to add an interpretative comment to the report (1).

Thirty years ago, few Duty Biochemists added an interpretative comment to a report, but with increasing workload and increasing clinical specialisation, adding an interpretative comment to reports has become common. However few Duty Biochemists have received formal training in adding interpretative comments, they tend to work in isolation with little or no feedback from users, and often never become aware of clinical outcome.

Clinical Pathology Accreditation (UK) Ltd (the national accreditation organisation in the UK) produced, in 1992, standards required for Departments seeking accreditation: standard D4 statedInterpretive reports are accurate, comprehensive and clinically relevantand addedReports should be subject to regular audit.For Clinical Biochemistry, there was no external procedure available to check compliance with this standard.

The idea of circulating a set of results containing abnormalities and asking colleagues what interpretation they would add was conceived in 1997: the internet-based general discussion mailbase of the Association of Clinical Biochemists (acb-clin-chem-gen@jiscmail.ac.uk) was an ideal forum for this (2). The first �Case for Comment� and the summarised responses is given in Table 1. We had assumed that a clear consensus would emerge on an appropriate response, but even on what was thought to be a straightforward case, there was a considerable disparity of response and opinion, and none of the comments matched the one produced by this Department: �Some proteinuria. Low volume of very concentrated urine: adequate water intake?�

Table 1: the first Case for Comment

Table 2. Case 21 for Comment

By 2001, 100 Cases had been distributed through the Internet. Most of these dealt with analytical interpretation, but pre- and post-analytical issues were also covered. There had been more than 400 different individual participants from 29 countries, and �Cases for Comment� has been translated into Italian, French and Mandarin. The Cases were widely acclaimed for their educational value, and we are aware of them being used for teaching in sites ranging from the South Island of New Zealand through Chengdu in China to Prince Edward Island in the Gulf of St Lawrence, Canada. Despite all this, there was little evidence of improvement in commenting practice, and the number of participants on each Case began to the scheme unmanageable. The decision was therefore made to move to a formal EQAS run through a pre-programmed web page; initial funding was granted by Clinical Pathology Accreditation (UK) Ltd. Anonymised peer review was continued, but using assessment of whole comments rather than comment components, to eliminate the subjective element inherent in breaking down comments into individual components. Assessors were asked to score each comment on the basis of its appropriateness, taking into account the results, the clinical information given, and the intended recipient of the report, so that effectiveness of communication was included in the assessment. A positive score loosely equates to the comment �adding value� to the report. The scheme was given Pilot UK NEQAS status, and the first distribution using the new format was made in July 2001 (4). A similar scheme has since been established in Australia (5).

Cases are made available fortnightly through the home page of UK NEQAS (www.ukneqas.org.uk). Each participant (protected by individual password) logs on to the web page, sees the Case, and has 2 weeks to make a succinct comment. Assessors then have 1 week to score each comment on a scale from -1 (inappropriate) to +3 (highly appropriate). The mean score given by the assessors to each comment enables ranking of all comments. The organisers then make a summary of the Case available to participants through the web page: an example of a participant�s summary is given in Figure 1. This includes the Case, the comment made by the participant, the mean score given to this, the distribution of scores given to all participants, the participant�s average score over the previous 6 months, and an outline of the Case which includes examples of low-, median-, and high-scoring comments. Users of our service have agreed with the utility of this marking.

In the first three years, some 70 Cases have been distributed. There are currently around 300 individual and group participants. There have been more than 40 000 visits to the web site. It is widely used as an educational resource (6), and in questionnaires, the scheme has been awarded an 80% rating in terms of its educational value. Some individual participants have been monitored (with their permission), and their scores for each Case have improved over time. The proportion of participants receiving zero or negative scores for each Case has markedly reduced. Nonetheless, doubts have been expressed about the validity of the peer review process used to award marks to each comment.

Peer review was introduced to guide the Organisers on which comments were more or less appropriate. In this process, valid differences in opinion can occur: these can include differences in interpretation (particularly when there is little or no evidence base); in professional or ethical issues; and in the comparative weight given to �good� and �bad� components included within a comment. An otherwise good comment can be ruined by an inappropriate suggestion for follow-up. In addition, often quite minor differences in phraseology can significantly affect the score given to a comment: dogmatic statements of a diagnosis tend to score worse than suggestions of possible diagnoses. This reflects the weight given by assessors to communication as well as interpretational skill. However, clinicians shown the summaries have totally agreed with the ranking given to individual comments quoted in the Case summary.

There is no gold standard regarding the appropriateness of a comment on a Clinical Biochemistry report, nor to what extent the marks given to a series of comments by a participant might be regarded as �poor performance�. Because of this, poor performance in the scheme is solely defined as active participation (i.e. submitting a comment which then receives a Continuing Professional Development credit) in less than 50% of the distributed Cases. However, even passive participation (looking at the Case and its summary) is of educational value. It is debatable to what extent the scheme might be used to identify poorly performing participants on the basis of the numerical scores allocated to their comments, particularly as there is no way of ensuring that the comments made to an EQAS Case reflect the comments which a participant makes in real life (however, exactly the same criticism can be levelled at conventional analytical EQA). Concerns have been expressed about the scheme�s potential uses in a revalidation process and in identifying poorly performing participants. However, there would have to be considerable discussion and widespread agreement with professional and regulatory bodies before any such use could be put into effect. On the positive side, participation in the EQAS is concrete evidence that an individual is submitting himself to an audit process required both for laboratory accreditation and for personal appraisal: as such, it can only be of benefit to the individual and to the community.

Early in 2004, doubts were expressed, through the general discussion mailbase of the Association of the Clinical Biochemists, about the entire utility of a Duty Biochemist scanning reports containing abnormalities. The �antagonists� felt that the possibility of a Duty Biochemist making a mistake through insufficient clinical information or knowledge of the patient was high; and that scanning each day many hundreds of reports containing abnormalities was a questionable use of the time of highly skilled laboratory personnel, which would be better spent in direct contact with Clinicians. In addition, there is no evidence that a Duty Biochemist improves patient care. The �protagonists� felt that to maximise most good to most patients, a proactive Duty Biochemist service for both the pre- and post-analytical phases was essential; and in addition suggested that a formal study to establish benefit to patients would be both unethical and impracticable. There is only a little evidence that interpretative comments change clinical practice for the better (e.g. 7) but there is considerable indirect evidence that they are appreciated, for example views expressed through user questionnaires. Neither side of this discussion has questioned the undeniable educational value of such an EQAS. The topic is obviously of major importance to the future role of professional staff working in Clinical Biochemistry and in Laboratory Medicine, and it is possible that a formal debate on the topic may be held at the next meeting of the International Federation of Clinical Biochemistry (Glasgow 2005).

Acknowledgements

Parts of this article have previously been published in the UK in the Association of Clinical Pathologists� News: they are here reprinted by kind permission of its Editor, Emyr Wyn Benbow. Thanks are also due to Jane French and Finlay MacKenzie for managing all operational aspects of the UK NEQAS for Interpretative Comments in Clinical Chemistry, and to UK NEQAS for permission to reproduce the copyright Case Summary included as Figure 1.

References

1. The provision of interpretative comments on biochemical report forms: a personal view. WJ Marshall and GS Challand; Ann Clin Biochem 37: 758 � 763, 2000.
2. Cases for comment, education and audit. GS Challand; JIFCC 10: 53 � 55; 1998
3. Experience with assessing the quality of comments on Clinical Biochemistry reports. Ping Li and GS Challand; Ann Clin Biochem 36: 759 � 765, 1999
4. Web based external quality assessment of individual skills � the UK NEQAS for interpretative comments in Clinical Chemistry. GS Challand, JC Osypiw, F MacKenzie, JG Middle et al; Abstract S259; Euromedlab, Prague, 2001
5. Quality assessment of interpretative commenting in Clinical Chemistry. EM Lim, KA Sikaris, J Gill, J Calleja et al; Clin Chem 50: 632-637, 2004
6. Cases for Comment on the ACB mailbase: were they educational? JC Osypiw, GS Challand; Proceedings of the ACB National Meeting, Glasgow 2002, A106, p86
7. Do interpretative comments work? MA Giles and GE Curtis. Proceedings of the ACB National Meeting, Birmingham 2004, A93, p116.

Figure 1. A participant�s  summary sheet from the EQAS [Case 132 summary � separate attachment]