IFM Healthcare Sidelines 14 (2)

This selection of abstracts explores publishing trends and issues around the quantity, quality and style of reporting and retrieving research evidence. The emerging theme is the importance of minimising the potential for publication bias. Achieving this objective will affect many aspects of the research process as well as the systems that record and retrieve research and its results. Ensuring the quality and reliability of routine data collection poses other important issues. Two papers in this column explore how the effective retrieval of statistical and management information through medical information systems depends on the quality and consistency of coding systems.

Yank V, Barnes D. Consensus and contention regarding redundant publications in clinical research: cross-sectional survey of editors and authors. Journal of Medical Ethics 2003;29:109-114.

The International Committee of Medical Journal Editors defines duplicate publication as “the publication of a paper that overlaps substantially with one already published”. Authors and editors were surveyed on the extent of this practice, why it happens, whether it is ever justified, and how it should be prevented. All those surveyed agreed that redundant publication occurs because of pressures on authors to publish and that journals could do more to prevent it. However, the views of authors and editors tended to differ about whether redundant publication can be justified. Authors gave many examples that they considered to be acceptable. For example, publishing a second article in a non-peer reviewed symposia supplement, segmented (salami-sliced) articles, and publishing to a different audience or non-English speaking audience. Publishing research results in more than one paper may introduce bias into the research that forms the basis for evidence-based policy and practice, so this issue has important implications. The survey highlights the need for consensus between authors and editors to prevent redundant publication.

Krzyzanowska M K, Pintilie M, Tannock I F. Factors associated with failure to publish large randomized trials presented at an oncology meeting. JAMA 2003;290(4):495-501.

This study identifies large randomised controlled trials presented at the American Society of Clinical Oncology (ASCO) annual meetings (1989 to 1998) that have yet to be published in full. The objectives of the study were to determine the rate of publication, to quantify the level of bias against publishing non-significant or negative results, and to identify factors associated with the time it takes to publish.

Non-publication and non-dissemination of important trial results can lead to bias in systematic reviews (including meta-analyses) due to the potential overestimation of treatment effects, and can ultimately contribute to inappropriate decision-making in clinical practice.

The study found that 26% of the 510 trials identified (phase 3 randomised trials with a sample size greater than 200) remained unpublished 5 years after being presented. It also found that there is a significant bias against publishing non-significant or negative results. The non-publication of large trial results not only weakens the evidence base, but also has ethical implications as trial investigators are ultimately breaking their contract with trial participants and funding bodies.

Villanueva P, Peiro S, Librero J, Pereiro I. Accuracy of pharmaceutical advertisements in medical journals. Lancet 2003;361:27-32.

This study investigates the accuracy of pharmaceutical advertising in medical journals that make reference to trials purportedly providing evidence to support the claims being made for the product.

Advertisements for antihypertensive and lipid-lowering drugs published in 6 Spanish medical journals in 1997 with at least one bibliographical reference were identified and then reviewed for accuracy. It was found that 44% of the 102 references retrieved did not accurately support the promotional statement made in the advertisement. This was despite the majority of the bibliographic references appearing in reputable medical journals (82%), and the stringent regulatory procedures that oversee medical publications in Spain.

The authors accept that their study was limited to only 6 Spanish journals and 2 types of drug. They also recognise that their study only questions whether the bibliographic reference supports the advertisement, and not whether the claims being made are actually true: each claim would have to be considered as a separate research question and systematically reviewed. Furthermore, the study does not look at whether advertising actually affects physicians’ prescribing behaviour. Despite the limitations of the study the authors suggest that doctors should always be cautious of advertisement claims, even when the claims appear to be evidence-based and supported by references to trials published in reputable medical journals.

Aronson JK. Anecdotes as evidence: we need guidelines for reporting anecdotes of suspected adverse drug reactions. BMJ 2003;326:1346

Discussion around the hierarchy of clinical evidence is highly topical. This editorial expands the debate by emphasising the relevance and importance of anecdotal evidence for reporting adverse events.

The author discusses the need for detailed guidelines for reporting anecdotes of suspected adverse drug reactions. Many anecdotal reports currently lack vital information and there is no uniformity in reporting. A suggested protocol for publishing anecdotal reports is available on the web at bmj.com.

Although the author is concerned with the reporting of anecdotal evidence there have also been reports of incomplete information about adverse events reported as part of formal research. For example, Santiago et al comment on the lack of uniformity in tracking adverse events in randomised controlled trials (RCTs): Santiago LM, Debanne SM, Neuhauser D. Tracking adverse events in RCTs: lack of agreement among regulatory institutions. Quality and Safety in Health Care 2003;12:234-235.

Santiago et al note that this lack of consistency in monitoring adverse events in RCTs exists both nationally and internationally. This is illustrated by examples of the variation in guidelines from the different agencies in the USA National Institutes for Health (NIH). As with Aronson, Santiago et al advocate the use of clear and consistent guidelines in reporting adverse events.

The need for guidelines in reporting adverse events has been recognised, but these papers emphasise the need for consistency across all study designs and the need for greater agreement across regulatory institutions.

Dickersin and Rennie discuss the widespread problem of the lack of information on ongoing clinical trials. Many trials are not registered on publicly accessible databases whilst ongoing. In addition, some may never be published, either because they were not submitted for publication, or due to publication bias - positive results being more likely to be published than negative ones. The results of some trials, therefore, may never be known, and those who participated in them due to a desire to contribute to medical knowledge may not have achieved their aim.

There are inherent difficulties involved in maintaining databases on ongoing research, including industry resistance, the lack of necessary funding for a sustained effort required in maintaining a database, the lack of a mechanism for enforcing contribution to databases, and a general lack of awareness of the importance of the problem.

Initially the way forward may be to merge existing databases, such as the UK National Research Register, the Meta-Register of Controlled Trials, ClinicalTrials.gov, Center Watch and the European Clinical Trials Database. The authors feel that efforts to resolve the problem of wasted and unnecessarily duplicated research should be made throughout the profession, by the public and private sectors, by journal editors, lawmakers and investigators.

Bennett DA, Jull A. FDA: untapped source of unpublished trials. Lancet 2003;361:1402-1403.

A major problem in producing systematic reviews is the reducing the potential for publication bias. Publication bias may be compounded by the inclusion of only published studies within a review, as these have been found to be more likely to show statistically significant results than unpublished studies.

This research, carried out by Caroline Maclean and colleagues, looks at the use of US Food and Drug Administration (FDA) unpublished research within a systematic review.

Searches were carried out to locate trials for inclusion in a review on NSAIDs for dyspepsia. In addition to traditional searches of databases of published research, the researchers also located and analysed FDA reviews of new drug applications, the majority of which are usually unpublished. The FDA research was found to be of similar methodology to published studies, although the quality of reporting was poorer. It was, however, a very resource-intensive exercise to locate and analyse this unpublished data.

The authors conclude that the disclosure of unpublished FDA research in the public domain would help alleviate publication bias, and make an enormous contribution to evidence-based health care.

Healy D, Cattell D. Interface between authorship, industry and science in the domain of therapeutics. British Journal of Psychiatry 2003;183:22-27.

This article looks at the consequences of the emergence of a new form of scientific authorship. Ghost-writing involves the use of unacknowledged writers or editors and is used widely by communications (medical writing) agencies working for pharmaceutical companies.

The authors compare ghost-written articles with those produced in the traditional way. They assess the quantity of material produced by each acknowledged author, journal impact factors, citation rates and literature profiles for each of the two groups. Agency articles tend to be longer, have more authors (and each author was linked to more papers) and a greater citation rate than non-agency articles. These facts merit a discussion of the pros and cons of ghost-writing.

Positive effects of ghost-writing by agencies include a greater likelihood that results will be published results than if authorship is left to researchers. Secondly, the quality of the writing may be better. Thirdly, at least some communications agencies seem to be better than researchers at disclosing vested interests. Finally, data suggest that company-sponsored publications may be better at reporting adverse effects than other publications.

Negative effects of ghost-writing include a lack of recognition for the people who actually write the articles, as well as the possibility that credited authors of ghost-written articles may become opinion leaders in a field in which they actually have little first hand experience. Articles that are produced by pharmaceutical companies may be likely to address questions that are of interest from a marketing point of view, rather than scientifically valid questions. Finally, the authors are concerned about the accuracy of company-authored reports, especially in relation to adverse events.

The article ends by offering a possible solution to these issues, whereby companies make the raw data from the trials publicly available.

Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence b(i)ased medicine – selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 2003;326:1171-1173.

This article explores the potential for bias in systematic reviews (including meta-analyses) that rely on studies sponsored by pharmaceutical companies. It contends that because the companies that sponsor them own the results from these studies, it can be difficult for researchers to identify multiple publication, selective publication and selective reporting of results, which may lead to bias.

The authors conducted an experiment using the material submitted by industry to the Swedish regulatory authorities as part of the drug approval process. The reports all related to the selective serotonin reuptake inhibitor group of drugs. This pool of information was used a gold standard and compared to the information available in published format.

The results of the study showed that multiple publication was frequent, and often hard to spot, though the result of the analysis suggested that this did not cause major bias. Selective publication also existed, so that studies that showed a significant effect were more likely to be published than those that didn’t. In addition, selective reporting, particularly, the tendency to report favourable per protocol analyses, rather than the less favourable intention to treat analyses, could lead to large overestimates of effect.

This article concludes that reliance on published material alone is likely to lead to medical decisions being based on biased evidence.

Brown PJP, Warmington V, Laurence M, Prevost T. Randomised crossover trial comparing the performance of Clinical Terms Version 3 and Read Codes 5 byte set coding schemes in general practice. BMJ 2003;326:1127-1130.

The use of standardised clinical terminology in the form of a knowledge based coding scheme is used to record patient data in electronic records. The NHS has developed the Clinical Terms Version 3 coding scheme, which provides unlimited hierarchical depth, multiple relationships and unambiguous preferred terms.

This paper describes a randomised crossover trial to determine whether Clinical Terms Version 3 provides greater accuracy and consistency in coding electronic patient records than the Read Codes 5 byte set. The main outcome measures were the percentage of coded choices ranked as being exact representations of the original terms; the percentage of cases where the coding choice of paired GPs was identical; and the length of time taken to find a code.

Ten general practitioners recruited from practices in Norfolk conducted the study. Each GP recorded the consultation details of ten consecutive patients in a consultation setting. A framework of headings was provided: reason for encounter, diagnosis, treatment and medical history. The GPs were grouped into five pairs and each GP coded terms using both schemes. They coded the terms from their own records and those of the other doctor.

The authors report that Clinical Terms version 3 performed significantly better than Read codes 5 in the consistency of coding the meaning of concepts. Exact matches were more common using Clinical Terms Version 3, and the pooled proportion with exact and identical matches by paired participants was greater for Clinical Terms version 3 than Read Codes 5. The time taken to code with Clinical Terms was not significantly longer than Read Codes 5. The authors conclude that improved coding accuracy in electronic patient records can be achieved with the use of Clinical Terms Version 3.

Gray J, Orr D, Majeed A. Use of Read codes in diabetes management in a South London primary care group: implications for establishing disease registers. BMJ 2003;326:1130-1132.

This cross sectional study examined the Read codes used in recording information on the management of diabetes in 17 GP practices in one primary care group in South West London. The main outcome measures were the number of codes in use in all of the practices, the variation in the use of codes between practices and the prevalence of Read code use in diabetic patients.

The study identified all patients with diabetes, and all the Read codes associated with their management. Patients were identified across all practices by using the C10 code for diabetes, all its lower level codes and the drugs used in treatment. The proportion of practices that had used each Read code, and the proportion of patients with diabetes who had the code in their electronic patient record was then calculated.

The study found nine separate Read codes groupings and 25 individual diabetes codes were in use across all 17 practices. Only one Read code, C10 Diabetes Mellitus and its sub code, was used by all practices. However, its use varied from 14% to 98% of patients with diabetes. Other Read codes for monitoring the care of patients varied widely across practices and less than half of patients had their type of diabetes coded.

The authors conclude that diabetes registers may be inaccurate because the prevalence of diabetes is underestimated due to many patients not having a diabetes code recorded in their medical record. The use of Read codes for diabetes needs to be standardised and the level of coding improved if valid diabetes registers are to be constructed.

Samanta A, Samanta J, Gunn M. Legal considerations of clinical guidelines: will NICE make a difference? Journal of the Royal Society of Medicine 2003;96:133-138.

With reference to a legal shift in deciding how standards of care are measured, this paper addresses the issue of how lawyers and the courts might use guidelines, such as those produced by NICE, in clinical negligence litigation.

The process for developing guidelines and the potential benefits and limitations of using guidelines as an option for improving the overall quality of clinical care are discussed. Case law from the US and the UK are presented and the Government’s agenda for healthcare and NICE, its policy for healthcare quality and the present climate of medial practice are discussed in turn.

The conclusions that are drawn are based on recent analysis and suggest that guidelines are likely to play an increasing part in the UK law of clinical negligence. NICE guidelines are likely to emerge as ‘a reasonable body of opinion’ for the purpose of medical litigation and doctors who deviate from NICE guidelines may need to explain why they have done so.

Bessell TL, Anderson JN, Silagy CA, Sansom LN, Hiller JE. Surfing, self-medicating and safety: buying non-prescription and complementary medicines via the internet. Quality and Safety in Health Care 2003;12:88-92.

This study aimed to evaluate

The DISCERN rating instrument was used to assess the quality of online consumer health information. Of 104 unique e-pharmacy websites that were investigated, 63 (61%) provided some health information, 41 (40%) provided no information, and 53 (51%) published poor quality information of limited or no benefit.

A case scenario was designed to evaluate the quality of care delivered, along with Internet pharmacy practice standards. The study examined what happened

As consumers may have insufficient access to information and advice to make informed