Sidelines 14 (3)

Sidelines articles for this issue focus on information quality, information retrieval, information use by health care professionals and developments in web-based resources.

We would also like to draw your attention to some potentially useful ‘beginner’ articles that may be of interest to recommend to clients.

Borrill, Z., Houghton, C., Sestini, P. & Sullivan, P.J. (2003) ‘Retrospective Analysis of Evidence Base for Tests Used in Diagnosis and Monitoring of Disease in Respiratory Medicine’, BMJ 327: 1136-8.

This study assesses the quality of common clinical tests used in one outpatient respiratory medicine clinic over a period of three months.  Examining the case notes of new outpatients referred to the clinic, the first three eligible tests ordered for each patient, along with the question they were being used to answer, were recorded.  These tests were divided into three groups: Group A were tests aimed at making a diagnosis; group B were tests carried out to assess a previously diagnosed condition; and group C was a trial of therapy.  MEDLINE was searched for each test question using a published sensitive search strategy. The best evidence retrieved for each question was graded using criteria devised by the Centre for Evidence Based Medicine.  Only half the tests were used to make a diagnosis and only one fifth of tests to assess a known condition were supported by level 1a-1c evidence.  There was no evidence to support trials of therapy. 

A final diagnosis may be based on a series of different tests.  Patients undergoing a series of tests that include high quality evidence based tests along with inaccurate or unassessed tests may result in an incorrect final diagnosis.  The authors concluded that their study reflects the practice in a single outpatient clinic and that the proportion of evidence based tests in other clinics may be higher.  However, a need for further high quality research into medical tests used within respiratory medicine is required.

Loong, T.  ‘Understanding Sensitivity and Specificity with the Right Side of the Brain’, BMJ 327: 716-9.

Using a series of diagrams this article attempts to explain the concepts of sensitivity, specificity, positive predictive value and negative predictive value. The author takes a hypothetical population of 100 people, of whom 30 have a disease, and demonstrates sensitivity and specificity. Sensitivity refers to how good a test is at correctly diagnosing people who have the disease, while specificity refers to how good the test is at identifying those that are well. Positive predictive value refers to the chance that a positive result will be correct, while negative predictive value is concerned with negative test results. Positive and negative predictive values change according to the prevalence of the disease.  The positive predictive value will fall as the prevalence of the disease falls, while the negative predictive value will rise.  A low prevalence means that the person undergoing the test is unlikely to have the disease, and therefore a negative test is likely to be correct.

Taking a real example of a test for diagnosing systemic lupus erythematosus, that has a sensitivity of 94%, a specificity of 97% and a prevalence of 33 in 100,000, the author demonstrates that while there are many more true negative than false negative results, there are also more false positive than true positive results.  The test has a negative predictive value of 99.9% but a very low positive predictive value of around 1%.  For patients tested with symptoms of the disease the prevalence is higher than 33 in 100,000. Therefore, the positive predictive value will be higher than 1% as these patients are from a population with a higher prevalence of the disease. The author concludes that because of our dependence on the linguistic (left) side of our brain we have difficulty in comprehending how a test with 94% sensitivity can only be correct 1% of the time. 

Lanier, D.C., Roland, M., Burstin, H. & Knottnerus, J.A.  (2003) ’Doctor Performance and Public Accountability’, Lancet 362: 1404-8.

The paper illustrates efforts being made to improve professional performance in health care in three countries: the UK, the USA and the Netherlands.  It was found that traditional approaches to improving performance (including continuing professional education, clinical audit and peer review) were aimed at the individual clinician, and only had a limited effect.  More recently efforts have been made to implement improvements nationally, with attempts to reshape clinical routine and organisational procedures.  Strategies have included the introduction of evidence-based guidance, more effective use of information technologies, and greater involvement of doctors in continuous quality improvement programmes.  It is not yet known how effective these strategies have been, and it is expected that the effectiveness of any strategy will be dependent on the local or national circumstances within which the initiative is applied.

From the experiences of the 3 countries the study suggested that professional leadership is vital if a culture valuing good professional practice is to be engendered.  It also found that government has a substantial effect on professional performance through the introduction of regulations and other forms of centralised supervision.  However, the authors concluded that sustained success would be dependent on the continued inclusion of doctors in any performance improvement strategies and that increased public accountability will fuel further efforts towards improving professional performance.

Rogers, J.E., Wroe, C.J., Roberts, A., Swallow, A., Stables, D., Cantrill, J.A. & Rector, A.L.  (2003) Automated Quality Checks on Repeat Prescribing, British Journal of General Practice 53: 838-44.

Good clinical practice in primary care includes periodically reviewing repeat prescriptions. However, manually checking all repeat prescriptions is impractical.  This study investigated the feasibility of computerising the application of repeat prescribing quality checks to electronic patient records in primary care in the UK. 

A machine-readable drug information resource, based on the British National Formulary (BNF), was installed in 3 general practices in Greater Manchester.  The computer software raised alerts for each repeat prescription when the electronic patient record showed no valid indication for the medication prescribed.  Semi-structured interviews took place to assess clinicians’ reaction to the software. 

There was no valid indication in the electronic medical records for 14.8% of the repeat prescriptions, but 62% of these alerts were incorrect.  A number of reasons were cited for the incorrect alerts, including errors in the drug information resource, locally idiosyncratic clinical coding, the BNF not listing all current clinically accepted indications for a drug, and the inability of the system to recognise an ‘obvious’ indication only inferable by a clinician.  The interviewed clinicians supported the idea behind the software, but could not accept it unless the high number of incorrect alerts could be reduced.  The authors feel that this would only be feasible with a change in clinical coding practice.

Holt, T.A. & Ohno-Machado, L. A Nationwide Adaptive Prediction Tool for Coronary Heart Disease Prevention, British Journal of General Practice 53: 866-70.

The National Service Framework for coronary heart disease recommends that patients with a greater than 30% risk of developing coronary heart disease in the following 10 years should be treated with a similar priority to those with established disease. The challenge to primary care teams is how to identify those at-risk individuals who lack cardiovascular symptoms.

This article discusses the standardised electronic recording of cardiovascular risk factor information collected in primary care using a nationwide data collection system. This can then be used to create a new strategy, using an adaptive prediction model, for targeting primary prevention interventions at high-risk individuals.

An adaptive prediction model uses existing data and classification 'gold standards' to predict in which class a new case belongs. In this example, the adaptive prediction tool would function as a pattern recognition device, using cardiovascular risk factor data from across the country to identify patients at increased risk of coronary heart disease. In principle, the model could improve the accuracy of predictions currently made through the Framingham algorithm over time, by responding to significant trends in the patterns of CHD in the UK as those patterns develop during the 21st century.

Woloshin, S., Schwartz, L.M. & Ellner, A. Making Sense of Risk Information on the Web: Don't Forget the Basics, BMJ 327: 695-6.

This article highlights issues surrounding the use of web-based risk calculators. Such calculators generate tailored risk assessment information based on personal factors such as age, diet, family history, pre-existing medical conditions etc. However, their usefulness depends on their accuracy and whether they are complete and balanced.

Three important aspects of the quality of risk calculators are highlighted: clarity, context and uncertainty.

The issue of clarity means that the user should be made aware of what specific risk is under consideration, for example whether it is the risk of contracting a disease, or of dying from it. The majority of calculators estimate only the chance of developing a disease, not the probability of death from the disease. The context issue relates to the fact that it is important that users are given risk information in context. For example, how does their risk compare to that of other people, or how does the risk of developing one disease compare to the risk of developing another. The uncertainty issue lies in the idea that risk factor calculators should give an indication of the sense of uncertainty inherent in risk prediction, and the strength of evidence behind the factors used.

Examples of good quality web based resources for risk prediction and assessment of these tools are provided in this article.

Bird, D.  (2003) Discovering the Literature of Nursing: a Guide for Beginners, Nurse Researcher 11 (1): 56-70.

This article is aimed at nurses and offers an introduction to literature searching. David Bird breaks the process down into six clearly defined stages, and guides the reader through each aspect of the process.

1. The first stage is to choose appropriate sources of information. This articles concentrates on electronic bibliographic databases. It explains how to chose a database, and how to access some of the major databases.

2. The second stage is to formulate a search strategy. This section starts with a useful paragraph that explains why this is necessary and concludes with an example search strategy.

3. Section 3 covers combining search terms and explains how to use Boolean operators.

4. Stage 4 is about entering the search criteria and explains the differences between systems that use a single search box and those that allow combinations of more than one search set.

5. Stage 5 describes browsing the search results and suggests useful criteria to help the searcher shortlist relevant articles from the results.

6. Finally, stage 6 looks at accessing the full text of items, covers the increasing existence of full text links and suggests alternatives for when these are not available.

The article concludes with a practice search for PubMed. There are several extensive appendices, covering the databases available through the NHS, and a selection of other databases, which are available for free on the Internet.

Cooper, J.E. (2003) Prospects for Chapter V of ICD-II and DSM-V, British Journal of Psychiatry 183: 379-81.

This article discusses the development process of the psychiatric chapter of the eleventh revision of the International Statistical Classification of Diseases and Related Problems in the light of the publication of A Research Agenda for DSM-V and of the appointment of a new Director General of The World Health Organisation (WHO).

A Research Agenda for DSM-V is published by the American Psychiatric Association and so reflects the views of the contemporary research community of America. Despite this, it has several sections that John Cooper feels could make a valuable contribution to the consultation process for ICD-11. In particular the chapter on personality disorders and relational disorders offer a useful discussion of an area that is often avoided because of its difficulty. Cooper disagrees with the suggestion made in A Research Agenda for DSM-V that, with suitable international input, it could be used as a replacement for ICD-11. He suggests that ICD-11 could perhaps be used as a replacement for DSM-V.

The articles ends with a discussion of how WHO might increase the sales potential of Chapter V of ICD-11, and thereby increase the number of people who have access to the ICD classification, and also increase the revenue available to the WHO.

Leung, G.M., Johnston, J.M., Tin, K.Y., Wong, I.O., Ho, L.M., Lam, W.W. & Lam T.H. (2003) Randomised Controlled Trial of Clinical Decision Support Tools to Improve Learning of Evidence Based Medicine in Medical Students, BMJ 327:1090-3.

In this randomised controlled trial Leung et al. aimed to assess the effectiveness of learning Evidence Based Medicine (EBM) through a clinical decision support tool on a handheld computer (InfoRetriever) compared to using a pocket-sized card containing guidelines for clinical decision making. The third intervention was a control.

169 fourth year medical students were randomly divided into three equal size groups: a control group, an InfoRetriever group and a pocket card group. The InfoRetriever group were given InfoRetriever, software designed to give rapid access to current medical evidence and containing several evidence databases, on a Personal Digital Assistant. The pocket card group were given a pocket card containing guidelines on clinical decisions designed to prompt students to apply EBM techniques in their clinical learning. The groups assigned an active intervention also received two interactive teaching sessions.

The authors describe several limitations to their study but conclude that providing students with a handheld computer through which quick access to valid evidence can be obtained “can improve learning in evidence based medicine, increase current and future use of evidence, and boost students' confidence in clinical decision making.^“

De Lusignan, S. (2003) The National Health Service and the Internet, Journal of the Royal Society of Medicine 96: 490-3.

This paper presents a brief summary of NHS core Internet site provision followed by a discussion of the patterns of usage of the Internet by both patients and professionals.

The core information services are listed as the ‘official NHS gateway’, NHS Direct Online, and the National electronic Library for Health. Other websites do get a mention, however, including the National Library of Medicine (which provides access to PubMed), sites to help assess Internet sites such as Health on the Net (HoN) and Judge Health and other patient information sources such as Patients UK, NetDoctor and Contact a Family. 

Patterns in use of the Internet for health information by patients and professionals are summarised from a number of studies identified by a search on MEDLINE and the Internet. The author concludes that the Internet is used in inconsistent and suboptimal ways and presents some suggestions of ways for general practitioners (GPs) to use the Internet more efficiently.

Walsh, K, & Dillner, L. (2003) Launching BMJ Learning: Online Learning Resources Based on the Best Available Evidence, BMJ 327: 1064.

This paper promotes bmjlearning.com, a web based learning resource launched by the BMJ for general practitioners (GPs). The site is currently free but users are required to register. The editorial describes the background to the aims and development of bmjlearning.com and some of the facilities available on the website. For instance, users can record their learning experiences (recommended to help the appraisal process for GPs), test their consultation skills on the interactive case histories, and “read, reflect and respond” to issues listed. The “just in time” modules offer a way for busy professionals to keep up to date. The disadvantage of the site is the current lack of breadth in the topics covered by the learning resources. However, the interactivity of the site goes somewhere to making up for this. Since the article was written further developments have taken place on the site including an events listing which can be searched by area, type of event, target audience and even cost and childcare. Although the site is aimed at GPs it covers many areas of interest to other health professionals.

Back to Inform 14 (3)

This page was last updated on: 15 March 2004