Randomised trials in alternative/ complementary medicine

Bruce G Charlton MD


Bruce G Charlton MD
Reader in Evolutionary Psychiatry
Department of Psychology
University of Newcastle upon Tyne
NE2 4HH
England

Editor-in-Chief, Medical Hypotheses

Tel: 0191 222 6247
Fax: 0191 222 5622
e-mail: bruce.charlton@ncl.ac.uk

Summary

The increasing trend for applying randomised trials to alternative medicine is part of a larger social project to 'integrate' orthodox and alternative medicine - an aspiration about as sensible as a desire to integrate astronomy and astrology [1]. What seems to be implied by integration is that alternative medicine should take on some of the evaluative trappings of orthodox medicine in order that it share some of the prestige.

Much hope has been pinned upon randomised trials to evaluate the effectiveness of alternative medicine [2]. Thanks to the excessive zeal of the 'Evidence-based medicine' movement, it has become common to believe that randomisation procedures are able to confer automatic validity on any investigation. Consequently it is assumed that randomised trials will be able to generate objective evidence concerning therapeutic effectiveness even when performed outside of science. However, this viewpoint is based on a misunderstanding of both science and randomised trials. At best, randomised trials may serve as a stepping-stone by which individual treatments from alternative medicine are incorporated into the science-based pathological system of orthodox medicine. But randomised trials by themselves are incapable of establishing the validity of systems of alternative medicine because alternative systems are incapable of generating theory-embedded hypotheses that have the precision and formal complexity required to be testable and generalizable. [1, 3, 4].

Instead of resolving questions of effectiveness, randomised trials therefore merely tend to surround alternative medicine with a pseudo-scientific aura that is neither appropriate nor deserved.

Non-scientific pathology

Alternative/ complementary/ fringe medical systems are frequently discussed, but seldom defined - perhaps because there are so many of them. But the difference between alternative and ‘orthodox’ medicine is clear enough - orthodox medicine is based upon scientific pathology (or is working towards this goal), while alternative medical systems have non-scientific pathologies based on spiritual, mystical, or otherwise intuitive insights.

In orthodox medicine, explanations of diseases and symptoms are constrained by scientific concepts from the biological sciences, chemistry and physics - knowledge of illness is embedded in this vast and inter-linked background. By contrast alternative medical systems are unconstrained by existing scientific knowledge - their explanatory pathologies are non-scientific [2].

Of course, individual items of scientific knowledge may turn out to be wrong, and individual interventions of alternative medicine may turn out to be useful. As a consequence, conventional medicine is predicated upon self-correction, which acknowledges the inevitability of error, while alternative medicine aspires to the timeless ‘truth’ of religion, and some systems boast of their ancient lineage and unchanging validity [2,4].

The fact that systems of alternative medicine eschew scientific pathology means that their validity is untestable.

Randomised trials applied to alternative medicine

If any of the alternative medical systems were obviously effective, there would be no call for randomised trials to prove the point. However, despite many decades or centuries of experience, there is no clear cut instance in which any alternative therapy is unequivocally effective and indicated for any particular disease or symptom. In other words, there are no cures of the otherwise incurable - nobody dragged-back from certain death in the way that has happened many millions of times with antibiotics and steroids. Severed limbs are not reattached to bodies, nor diseased internal organs extracted, nor can reliable anaesthesia be induced (despite the propaganda-based claims for acupuncture [4]).

Lacking any clear-cut and replicable evidence of therapeutic effectiveness, the only positive indicators comprise a few randomised trials - for example the famous Lancet report of better-than-placebo effectiveness of a homoeopathic remedy in hay fever [6]. However, well-conducted positive trials of homoeopathy have only been reported for conditions which are distinctive in having a highly unpredictable prognosis - conditions such as hay fever, rhinitis, asthma, excema, back pain, arthritic pain, migraine, chronic fatigue, post-operative ileus, multiple sclerosis and so on [7, 8, 9] - and the same applies to other less frequently investigated alternative systems. In conditions with such widely variable natural histories it takes a great deal of research using many methodologies to establish the effectiveness even of orthodox scientific treatments (and their value often remains uncertain despite huge investments of resources).

The potential clinical usefulness of a trial arises when its results may be generalizable [10]. What makes a result potentially generalizable is that it can be interpreted in the context of a recognised theoretical basis of already existing scientific knowledge. A well-controlled therapeutic trial is a type of scientific experiment, and like any other scientific experiment it may be understood as testing an hypothesis [11]. But a hypothesis must have certain formal qualities if it is to be both useful and testable [12]. In particular the hypothesis must be part of a larger and systematic theory. A scientific theory is like a map - a theory tells you where to look and what to expect enabling observations to act as tests of these predictions [13].

There is no single definitive test of any scientific theory - rather, the truth of a theory is decided on the basis of a range of observations, preferably done by independent investigators (including some who are genuinely but honestly trying to refute the theory), and using a variety of methodologies. A theory becomes established when other scientists incorporate and build upon it in their work [12].

Untestable pseudo-hypotheses

Questions of the general form 'Is X an effective treatment?' are not true scientific hypotheses - rather they are pseudo-hypotheses with a misleading similarity to the genuine article [10, 11]. For example, it would not make sense to ask something as vague as whether beta-blockers are an 'effective treatment’, since - although precise - the question lacks the requisite structure to be a testable. A great deal more would need to be specified. For example, ‘effective’ in what dose, for what condition, and with what outcome? Without such knowledge the benefits of treatment cannot be evaluated. If a beta blocker trial were performed on patients with MI the death rate might be reduced, but if many of these patients were also suffering from asthma, then beta blockers would probably kill more than they saved.

So, an hypothesis needs to be not just numerically precise, but also sufficiently theoretically complex that it entails numerous observable consequences. Lacking scientific pathologies, such hypotheses cannot be generated by alternative medicine systems. This makes randomised trials or meta-analyses uninterpretable when they examine pseudo-hypotheses such as 'Is homoeopathy a placebo response?' or 'Are the clinical effects of homeopathy placebo effects?' [5, 6]. Such questions simply do not have the formal complexity required of scientific hypotheses - they do not entail numerous observable consequences.

Trials addressing such questions merely end-up measuring outcome variables in comparison groups. It is then left to statistical procedures (and statistical assumptions) to try and make sense of the results. Such attempts invariably degenerates into a morass of numerical logic-chopping that leads nowhere [5, 8].

Incorporating individual alternative treatments into orthodox medicine

Some alternative medical treatments are clearly without significant biological effects - others almost certainly do have effects, for instance some herbal medicines. But merely knowing that some herbal medicines do something is not an adequate basis for their therapeutic use - because all effective treatments have significant risks [14].

Before they can be clinically deployed, interventions need to be embedded in a scientific theory and incorporated into the orthodox medical system. Even orthodox treatments of proven effectiveness but lacking an acceptable scientific explanation tend to be regarded as disreputable (for example the perennial unpopularity of electro-convulsive therapy), and may become neglected to the point of extinction (for example the rise and fall of lithium in nineteenth century psychiatry) [15]. Observations become facts only when they are invoked to support a theory, and a convincing theory will usually require minimal observational confirmation before being accepted [5, 16].

When a truly effective intervention emerges within an alternative medical system, its benefits need to be re-explained using ‘orthodox’ theories in order that it can be evaluated. A topical example would be the almost-certainly effective antidepressant herbal medication - St John's Wort. This agent has been slotted into orthodox medicine by treating it as essentially interchangeable with conventional antidepressants such as the SSRIs [17]. But the lack of any agreed scientific theory to explain the action of St John's Wort has apparently deterred conventional psychiatrists from using what is probably an important drug. (Ironically, the widely accepted neurochemical explanations of orthodox antidepressant action are scientifically flimsy, and owe their success to the marketing needs rather than scientific discoveries [18].)

In other words, the effectiveness of an alternative intervention can only formally be established by first incorporating that specific intervention into the scientific explanatory framework of orthodox medicine. But measuring the effect of specific alternative interventions using randomised trials does not help decide on the validity of systems of alternative medical. The proven effectiveness of St John’s Wort when used like an SSRI does not validate alternative systems of herbal medicine.

The proper role of randomised trials

Useful randomised trials come at the end of a long process of therapeutic development because a randomised trial cannot be designed until it is known what needs to be controlled [19]. Knowledge must be accumulated to answer such questions as: what dose of a drug should be given, what are the side effects, should the trial include both men and women and of what age, how specific does the diagnosis need to be, what severity of disease should be recruited, do other drugs or diseases interfere with the outcome, what outcomes should be measured, and so on. Randomisation is in fact on of the least important aspects of a useful randomised trial, in the sense that for a well-controlled experiment, randomisation is merely the icing on the cake [11].

When randomised trials are used in alternative medicine, the proper process of therapeutic development is turned on its head. Instead of being at the end of therapeutic evaluation, trials are placed at the beginning, and used in isolation. Not only is there no accumulated scientific knowledge with which to design a trial, the very possibility of any scientific context is excluded. The problem, therefore, is not that alternative medical systems are scientifically primitive, it is that alternative medical systems are not scientific at all - by self-definition. And outside of science, performing randomised trials is simply an absurdity - as was wittily illustrated by the publication of a 'positive' randomised trial of the power of remote intercessory prayer in the treatment of blood infection [20].

When the constraints of randomised trials are properly understood, so too is the irrelevance of 'positive' trials in alternative medicine. Performing more such trials is merely evidence of intellectual confusion. Instead of resolving questions of effectiveness, randomised trials simply surrounds alternative medicine with a pseudo-scientific aura that is neither appropriate nor deserved.

Acknowledgements. Many thanks to Neville Goodman for his help in surveying the literature.

Fitzpatrick M. Put alternative medicine back in its box. Spiked www.spiked-online.com/ www.spiked-online.com/Articles/00000006D949.htm. Accessed 18.7.2002.

Ernst E. What’s the point of rigorous research on complementary/ alternative medicine? JRSM 2002; 95: 211-213.

Charlton BG. Philosophy of medicine: alternative or scientific? JRSM, 1992;85:436-8.

Skrabanek P. Acupuncture: past, present and future. In False premises: false promises. Tarragon Press: Glasgow, 2000. Pp 29-52.

Vandenbroucke JP. Homoeopathy trials: going nowhere. Lancet 1997; 350: 824.

Reilly DT, Taylor MA, McSharry C, Aitchison T. Is homoeopathy a placebo response? Controlled trial of homoeopathic potency, with pollen in hayfever as model. Lancet 1986; ii: 881-886.

Linde K, Clausius N, Ramirez G et al. Are the clinical effects of homoeopathy placebo effects? A meta analysis of placebo-controlled trials. Lancet 1997; 350: 834-843.

Kleijnen J, Knipschild P, Rietter G. Clinical trials of homoeopathy. BMJ. 1991; 302: 316-323.

Taylor MA, Reilly D, Llewellyn-Jones RH, McSharry C, Aitchison TC. Randomised controlled trial of homoeopathy in perennial allergic rhinitis with overview of four trial series. BMJ 2000; 321: 471-476. & Letters. BMJ. 2001; 322: 169-171.

Charlton BG. The scope and nature of epidemiology. Journal of Clinical Epidemiology, 1996; 49: 623-626.

Charlton BG. Mega-trials: methodological issues and implications for clinical effectiveness. Journal of the Royal College of Physicians of London, 1995; 29: 96-100.

Hull DL. Science as a process. Chicago University Press: Chicago, 1988.

Ziman JM. Real science: what it is, and what it means. Cambridge University Press: New York, 2000.

Ernst E. Herbal remedies: where is the evidence. BMJ 2000; 321: 395-396.

Healy D. The creation of psychopharmacology. Harvard University Press: Cambridge, MA, USA, 2001.

Vandenbroucke JP. Medical journals and the shaping of medical knowledge. Lancet 1998; 352: 2001-2006.

Stevinson C, Ernst E. Hypericum for depression. An update of the clinical evidence. European Journal of Neuropsychopharmacology. 1999; 9: 501-505.

Healy D. The antidepressant era. Harvard University Press: Cambridge, MA, USA, 1997.

Charlton BG. Fundamental deficiencies in the megatrial methodology. Current controlled trials in cardiovascular medicine. 2001; 2: 2-7.

Leibovici L. Effects of remote, intercessory prayer on outcomes in patients with bloodstream infection: randomised controlled trial. BMJ 2001; 323: 1450-1451. & Letters. 2002; 324:10037.

E-mail
bruce.charlton@ncl.ac.uk

HOME

also by Bruce Charlton
Peak Experiences
Cargo Cult Science
The Malaise Theory of Depression
Delirium and Psychotic Symptoms
Public Health and Personal Freedom
Psychiatry and the Human Condition
Awareness, Consciousness and Language
Psychopharmacology and The Human Condition