Infostat, cargo-cult science and the policy sausage-machine: NICE, CHI and the managerial takeover of clinical practice

Bruce G Charlton
Being a final Draft version of
Charlton BG. The new management of scientific knowledge: a change in direction with profound implications. In: NICE, CHI and the NHS reforms: enabling excellence or imposing control? Edited by A Miles, JR Hampton, B Hurwitz. Aesculapius Medical Press: London, 2000. Pp 13-32.

Prof. Bruce G Charlton MD
UEL Centre for Health Services Research,
St Bartholomew’s Hospital,
London
&
Reader in Evolutionary Psychiatry
Department of Psychology
University of Newcastle upon Tyne
NE2 4HH
England

Editor-in-Chief, Medical Hypotheses

Tel: 0191 222 6247
Fax: 0191 222 5622
e-mail: bruce.charlton@newcastle.ac.uk

Infostat, cargo-cult science and the policy sausage-machine:
NICE, CHI and the managerial takeover of clinical practice

In 1993 I published a Lancet Viewpoint entitled ‘Management of science’ in which I pointed-out that purchasing managers in the National Health Service had begun implementing very precise contracts which controlled detailed aspects of clinical practice (Charlton, 1993).

For instance, it became mandatory to specify protocols for beta-blocker and aspirin treatment following myocardial infarction, and for prescription of benzodiazepines, antidepressants and other psychotropic drugs, and for aspects of skin cancer management. The justification for these blanket recommendations was that these interventions were ‘scientifically’ proven to be effective across the board in almost every clinical circumstance - at least according to the NHS Management Executive’s interpretation of the results of large randomized trials (although expert clinicians usually disagreed; eg. Julian, 1995).

But the fundamental problem was structural: the government were creating a managerial structure that separated the power to influence treatment from clinical responsibility for the consequences of that treatment. For example, managers were claiming the right to determine the nature of a drug prescription while doctors remained both morally and legally responsible for the outcome. This seemed self-evidently unethical, and placed doctors in the impossible position of being vulnerable to sacking of they were disobedient and to malpractice suits if they obeyed. The system was also open to corruption, since political influence could be brought to bear on managers, tending to generate protocols using criteria dictated by expediency rather than effectiveness. For example, the call for specified protocols on psychotropic drugs were unsupported by any rational consensus on what such protocols should contain, and the instructions concerning ‘skin’ cancer (and the failure to differentiate between basal cell, squamous cell and melanomatous malignancies) seemed more justified by the immediate demands of public relations than by any body of solid scientific evidence.

As things turn out, I had underestimated the seriousness of this kind of threat to clinical practice, and my article unfortunately proved to be prophetic of a trend which has culminated in the creation of NICE and CHI. The management of science in medicine is now established by statute, and - even worse - the criteria of effectiveness have been conflated with economic considerations of ‘cost-effectiveness’. The stage is set for clinical science to be steamrollered by the demands of power politics.

The evidential basis of NICE guidelines - the number-crunching commissars
NICE and CHI are organized in the form of a statutory arm of the government bureaucracy, as special health authorities with powers intended to influence the clinical practice of doctors and other health workers. The credibility of NICE and CHI depends crucially upon the claim that NICE guidelines will be objectively valid - based on appropriate evidence, properly interpreted, rationally argued, and intellectually compelling. Only if this is true will NICE guidelines posses the necessary legitimacy such that failure to adhere to them would reasonably be interpretable as negligent. But if NICE guidelines are seen to be partisan, irrational, scientifically unconvincing or politically-driven - then NICE guidelines will amount to little more than government propaganda backed up with a big stick.

NICE advertises itself as the application of rational and scientific management to medical practice. In particular the claimed intellectual credibility of NICE and CHI derives from a whole raft of data-driven and statistically-based academic disciplines which have become dominant in the past decade. These include Health Economics, Epidemiology, Evidence-based medicine, the Cochrane Collaboration, and related areas of research concerning ‘clinical effectiveness’, ‘meta-analysis’, ‘unexplained variations in practice’, ‘guidelines’, and so on. All of these activities in the UK (activities which I term ‘Infostat’ - see below) have been created, sustained and directed largely by Department of Health funding. In return for this funding, the practitioner of these arts have performed (sometimes unwittingly) a managerial role - aspiring to the function of commissars for government and the NHS management (qv. the role of intellectuals in Stalinist USSR; Joravsky, 1970. Infostat technicians can be seen as having prepared public and professional opinion for the emergence of NICE and CHI, and the statutory government regulation of clinical practice (Charlton & Miles, 1998).

Clearly, NICE has no special access to evidence, nor does it have any special technique for analyzing evidence, nor are its decision-making processes anything out of the ordinary for a health authority. There is therefore no reason to assume that NICE will perform any better than any other government bureaucracy when it comes to providing objective, rational and independent guidance. Given these facts, the statutory powers of NICE and CHI seem implicitly to assert that there can be no genuine or principled uncertainty about the proper nature of guidelines - since if there were such genuine or principled uncertainty, then the coercive imposition of guidelines would not be justifiable.

Problems with RCTs
For NICE guidelines to be credible, it ought to be clear to any informed and disinterested party that optimal clinical practice - in each specific instance for which guidelines are used - is known, and can be stated clearly and exactly in the form of useable guidelines. Presumably, this implies that randomized controlled trials (RCTs) are intended to be the main source of evidence for NICE, since it is a commonly held belief among the clinically ignorant and epidemiologically unsophisticated that the RCT method is capable of providing precise and unambiguous (‘gold standard’) guidance for clinical practice (Sackett et al, 1985 or, indeed, any publication you care to select from the EBM movement, Effectiveness Bulletin or the Cochrane Collaboration).

Unfortunately for NICE, RCTs cannot provide precise and unambiguous guidance for clinical practice (Feinstein, 1995; Charlton,1996a, 1996b, 1997; Goodman, 1998, 1999). The recent proliferation of ‘evidence-based’ clinical advice and guidance, purports to be objectively valid according to the notion that large randomized trials provide the best guidance for clinical practice. But in reality these RCT-based guidelines have proved to be just as unreliable as any other sources of advice. This was inevitable, since most randomized trials are done on un-representative populations of heterogeneous subjects and employing sub-optimal levels of experimental control. The interpretation of such trials is usually far from straightforward, and may be impossible (Goodman, 1999).

Many bio-statisticians hoped that meta-analysis would solve this problem and provide a method for objective interpretation. But this was also a delusion, since an accumulation of inadequate data simply makes a bigger pile of inadequate data, and the statistical averaging of different trials done in different places by different people for different purposes merely generates a meaningless statistical artifact. To put it bluntly, meta-analysis is a logically incoherent technique of zero scientific credibility (Feinstein, 1995; Charlton, 1996b). It embarrassing for those who understand statistics and clinical medicine to contemplate the innocent enthusiasm of its protagonists, and profoundly worrying that their armour-plated credulity has become the orthodoxy among politicians and managers. Meta-analysis never was adequate or appropriate to the task of decision-making and arbitration wished upon it by the protagonists of evidence-based medicine; and even within its limited statistical purview, the disagreements between supposedly ‘objective’ meta-analyses have been decisively damaging to the credibility of the method (Bailar, 1997).

The failure of RCTs and meta-analysis to deliver objective and authoritative guidelines means that NICE recommendations will inevitably suffer from the same lack of intellectual credibility that afflicts the many other sources of supposedly-definitive guidelines emanating from the Department of Health sponsored guidelines industry (Centre for Reviews and Dissemination, Cochrane Collaboration etc. etc.). NICE guidelines will differ from existing sources of medical advice only because they will be mandatory, and enforced on doctors by sanctions.

How science works

Since science is the major source of ‘reliable knowledge’ (Ziman, 1978), and that is what NICE is claiming to base its guidelines upon, one approach to evaluating the decision-making processes of NICE is to compare the workings of NICE with the workings of science (Hull, 1988).

In scientific practice, rival theories are proposed by the many individuals and groups working in a field of endeavor. Alternative observations, experiments and interpretations emerge from different directions. Disagreement and dissent (whether over fundamentals or over minutiae) is the normal state of affairs in any active subject within science. Yet as well as being a locus of disagreement and dissent, science has been characterized by its being progressive, an accumulation of tested knowledge. Despite disagreement, some scientific theories are accepted and built-upon, while others (indeed, typically the bulk of scientific work produced) is either ignored or rejected. Who decides what is ignored and what is incorporated? Who winnows the wheat from the chaff?

The answer is that no specific person or group ‘decides’ but that scientific 'truth' emerges as in the working practices of other scientists who are active in the field (Hull, 1988). The ultimate judge of scientific validity is to have your work used by other scientists in their work, and the ultimate prestige is to be acknowledged as the originator of work which is being so used. And it is this testing by use, by others, in practice and in competition with other theories, which stands at the root of the objectivity of science.

For example, there was no central committee or individual who decided that Crick and Watson's double helix structure for DNA was correct, and that this 'true' theory should be imposed upon other biologists (Judson, 1979). Rather the structure was used by other biochemists in their own researches. In a nutshell, when it was assumed that the double helix structure was true, scientists were able to predict observations, perform experiments and make manipulations that worked. The truth of the structure was not therefore decided, rather it emerged in practice.

Not every biologist agreed that Crick and Watson were correct, and nobody tried to coerce these people into adopting the double helix theory. There was dissent, and no central committee imposed sanctions on the unbelievers. But the dissent gradually disappeared from discourse of practitioners because, over time, it emerged that those who disagreed were not able to 'knock down' the double helix structure to the satisfaction of other workers in the field, nor were they able to produce a similarly powerfully explanatory alternative structure to the double helix (Judson, 1979). The anti-double helix dissenters were starved of status and influence, and have largely disappeared from the scene by a process akin to natural selection whereby pro-double helix scientists ‘reproduced’ and left scientific descendents, while the anti-double helix scientists failed to reproduce and became (more-or-less) extinct (Hull, 1988).

This is the kind of winnowing process that gives scientific knowledge its validity; the propositions of valid science have been tested by use. Validity is therefore pragmatic with respect to the natural world. If a scientific theory stops producing the goods, stops working when trying to explain and predict the natural world, or if something better comes along - then the scientists who advocate their theory will find that their work, hence their careers, are stalled (so long as they are honest, and do not fabricate or misrepresent their results - and science absolutely depends upon the honesty of its practitioners; Bronowski, 1975). The situation is ripe for the death of that theory, and its replacement.

Scientific knowledge is therefore both tentative (subject to revision) and also objective (in the sense that the decision about validity is made by the community of active scientists using natural world criteria - and not by the person who devises the theory nor by any specific minority group). Although scientists are subject to human frailties, and although there science demonstrates vast influences of prejudice, ignorance and malice - scientific theories are not just a matter of opinion (Hull, 1988). Real scientific theories are imposed on scientists by the practical constraints of the natural world, not by managerial diktat nor by the big stick of an inquisition such as CHI.

Summary of the nature of scientific decision making

1. The cumulative and progressive nature of science depends on a type of social structure for testing theories in practice against the natural world.

2. When science is working properly, arbitration concerning the 'truth' of scientific theories depends upon active scientific practitioners who use that theory in their work.
3. In science power is dispersed among practitioners. Truth emerges from the social structure of science in a way that is beyond the power of any individual or group to determine.

4. The personal motivation driving the system of arbitration is the personal interest of the arbitrators. In science the power to arbitrate goes with personal responsibility for the consequences. To ‘believe’ in a theory is equivalent to staking one’s own practice (hence reputation, hence career) on its validity.

5. Scientific 'truth' is therefore something that is tested against the constraints of the natural world in the practice of its active workers. The bottom line is the test of the natural world.

6. The ultimate scientific sanction to enforce a theory upon the practitioner is whether it is consistent with observed phenomena - the sanction is that the constraints of the natural world will ‘sabotage’ the work of a scientist who uses false theories.

The social structure of NICE and CHI
It has been necessary to spell out the way in which science works, in particular how decision-making depends upon the social structure of science, in order to contrast this with the social structure of NICE and CHI. Here are a few of the differences.

1. NICE and CHI are part of the executive arm of government, performing roles established by statute.

2. NICE and CHI are near the apex of a top-down managerial hierarchy in which the upper echelons audit and control the lower ones. This means that non-practitioners make decisions and enforce those decisions upon practitioners.

3. Power to judge scientific theories in NICE and CHI is centralized and concentrated in the hands of the few who give orders to be acted upon by the many.

4. The personal motivation driving the system of arbitration comprises the usual incentives of officials working in a bureaucratic structure. The career and status of a managerial arbitrators does not depend primarily upon getting the scientific decisions correct; rather, career and status depend primarily upon satisfying the internal demands of the system. The real world consequences of the theory impinge on the practitioners, not the managers.

5. NICE redefines ‘science’ as being whatever is the outcome of its deliberations. Since decisions are in the hands of the few, this decision-making process is readily corrupted by political expediency, external pressure, or self-interest. The ‘bottom line’ is keeping the boss happy.

6. The sanctions imposing the views of NICE derive from the world of politics, not from the natural world.

Managerial takeover of the NHS

From administration to management The creation of NICE and CHI can be interpreted as the most recent and aggressive expression of a managerial takeover of the NHS which has been going on for several decades. Managers can be defined as staff concerned with second-order activities - such as policy, finance and organizational planning - as contrasted with practitioners who perform the core tasks of the organization. As Simon Jenkins has put it: ‘The achievement of the Thatcher reforms was to make [NHS] management centralized and supreme’ (1995).

One of the most important changes since the NHS began, was the transformation of administrators to managers. There was more to this than a mere change in name - it represented a change in philosophy. Administrators are an unavoidable element in large organizations of any size - managers are an optional extra. The role of health service administrators was the secondary (albeit vital) one of running the organizational side of the NHS. Their job was to make a framework for professionals to exercise judgment within, they sought to implement the regulations and ‘oil the wheels’. Broadly, they aspired to the civil service ideal of impartiality and advice rather than interference.

But the role of managers is rather different. Managers are primary. They do not just implement regulations, they make regulations; they do not just make a framework for judgment, they dictate the processes and outcomes of judgment; they are committed not impartial, they give orders rather than offering advice; they commission new wheels rather than oiling existing wheels. In this sense, NICE is managerial, rather than administrative - it is designed to dominate professionals - not to assist them.

Central control versus devolved autonomy
In the old NHS, authority was assumed to lie with the clinical professionals by virtue of their skills, traditions and patient contact, so that once the organizational structure was established, detailed policy (such as it was) was to a great extent implicit and emergent from the interactions of clinicians, administrators and politicians. That is to say, policy was bottom-up rather than top-down. The nineteen eighties, by contrast, saw a trend towards dirigisme with a massive expansion in the powers of central government (Klein, 1989; Jenkins, 1995). The NHS ‘reforms’ brought more government: albeit rationalized as serving the long-term goal of less government. This politicization of the NHS ushered in a new era in which the service was to be driven from above by radical ideas emanating in broad outline from the cabinet and its policy think tanks, and implemented in detail by an executive management empowered to impose these upon an unwilling and resistant professional and ancillary workforce.

The first wave of General Managers were therefore pretty much tools of government policy. They were accountancy-orientated hit men, focused almost exclusively on financial reorganization: implementing cuts in funding, introducing ‘efficiency’-enhancing measures (privatization, contracting-out etc.), and - in general - having an attitude to the NHS that reflected the Prime Minister of the day. This period established the dominance of the way of thinking that sees the NHS primarily in generic organizational and economic terms - and not as specifically a health service.

Initially, at least, managers were engaged upon managing down to the level of clinical consultation, but not beyond. The interface between the public and the health professional remained out of bounds. Clinical decision-making was left largely independent and untouched by direct mechanisms, although of course it was influenced by organizational and resource constraints, and in turn influenced them. Since that time, a path has been followed that establishes financial control by encroaching on clinical independence and seeking increasing control of decision-making. So that over the years the managerial agenda has moved towards direct influence of decision-making processes in the clinical consultation, and pre-determination of the possible outcomes of doctor-patient interactions.

Dominance of General Management
In an important sense, the single unifying thrust of the recent NHS reforms has been the transfer of power from health professionals - principally doctors - to a new class of employees: general managers (Charlton, 1993). General Management, as it was called by the Griffiths report - gets its title from the intellectual rationale which lies behind the decision to put managers in charge of the NHS. It was asserted that the skills of management were generic, ie. nonspecific, transferable between different situations, and applicable equally to private sector retail and public sector service organizations, and to all types of social function.

General Management was the answer to what Griffiths saw as the main problem of the NHS, exactly that nobody was ‘in charge’ (1983). In actuality this matter of nobody being ‘in charge’ of the NHS was only partly accidental and regrettable. It was also partly an inevitable and desirable aspect of a collegial and democratic organization based upon the work of skilled professionals dealing with clients whose views must also be taken into account. The accidental and regrettable aspects of nobody being in charge were largely a consequence of the previous set of Conservative NHS reforms in 1974, which - in seeking to diminish the power of doctors, and achieve a broad representation of interest groups - had only succeeded in creating a gridlock at the level of policy and planning (Klein, 1989). The log-jam certainly needed un-blocking, and personally responsible, dynamic General Managers were a good swap for sclerotic and unaccountable committees. The massively increased program of hospital-building in the past decade is evidence of the consequential improvement in executive planning.

However, the implementers of the Griffiths report seems not to have appreciated that an organization depending upon the skills of autonomous health professionals will not function properly if it has imposed on it a hierarchical, line-managed, heavily-regulated structure imported from businesses based on simpler and more routine activities, and lacking the necessary personal relationship between providers and clients which characterizes medicine. Since 1983, it has been the long-term intention that the primary focus, the very crux of NHS activity, was to be the management of the organization, rather than the personal interactions between clinicians and patients. From this it followed that managers were obviously the best people to run the show. The essence of the successive waves of NHS reforms has been to make possible this managerial takeover.

General Management was imposed upon the NHS (Klein, 1989; Jenkins, 1995). Management did not evolve organically in response to the demands of the system as it was, but was grafted onto the existing system because this was felt to be what the system needed (or if not needed, then deserved). The intention was that General Management transplanted from the private sector would take over from the clinical professionals in a kind of ‘graft versus host’ reaction.

The enhanced role of policy and planning in NHS affairs
There is more to General Management than a transfer of power from clinicians to managers. Alongside, there has been a trend towards explicit, top-down policy and planning. Power is now more concentrated at upper levels and in managerial chains of command, and clinical interactions are increasingly constrained: doctors have less autonomy and patients have less influence over the clinical consultation.

The superficial aspects of this dominance of policy and planning are familiar: they comprise the whole bogus rhetoric of mission statements, aims and objectives, targets, charters, standards, guidelines, protocols, publicity, public relations and packaging - the rising tide of ‘bullshit and bollocks’, as the playwright Alan Plater has succinctly characterized it.

But the deeper role of policy has acted synergistically to increase the rate of policy change in the NHS. Clinical practice is, of course, familiar with change - having reinvented itself several times in response to therapeutic innovation over the past sixty years (Le Fanu, 1999). And such clinical change leads to policy changes, but by an indirect, implicit, bottom-up route. At present, however, there is almost an expectation of rapid changes in policy at the large scale or ‘macro’ level. This is apparently what ‘dynamic’ management is all about - making structural alterations (‘permanent revolution’ as Mao Tse Tung put it, or ‘thriving on chaos’, as management guru Tom Peters puts it; 1989). Changing organizational structures is how managers make their reputations and maintain their power (Charlton, 1995).

High-level strategy synergizes with low-level shell-shock to entrench managerial dominance; and only those whose heads are above the fray are able to safeguard their own interests. The managerial literature therefore, unsurprisingly, regards changing an organization as in itself admirable. We are all familiar with the phony nomenclature whereby every threat is seen as a ‘challenge’, every disaster seen as presenting an ‘opportunity’. All this translates as organizational change creating an increased scope for shrewd managers to attain rapid career enhancement. Meanwhile, clinicians and other practitioners are kept busier than ever on ‘fire-fighting’ tasks: ‘implementing’ reforms, increasing productivity, negotiating rationing of services and in general squaring the circle where central planning meets market forces and patient demands.

Macro-strategy leads to micro-tactics. Frequent shifts in overall direction at high levels create endemic insecurity at the lower levels (re-deployment, redundancy, retraining…). Chaos translates into increased uncertainty at the level of personal goals and inter-personal interactions, with an accompanying narrowing of perspective and shortening of time horizons. The contemporary NHS is characterized by an official v unofficial schism - far-sightedness and bold ambition among the planners, with a survivalist mentality among the planned.

The chaos game is one that only managers and politicians can ‘thrive’ on: professionals and patients are the losers. Those aspects of the NHS reforms destructive of the provision of clinical services are often excused or minimized by policy-minded commentators on the basis that ‘at least’ the reforms initiated change. This perspective presumes that the NHS was so bad that change could only be for the better, and contrasts sharply with the perspective of practitioners and patients who tend to be impressed only by improvements, rather than by change per se. The difference is that clinicians and clients seek evidence that change will be positive, otherwise they support the status quo; managers seek a system where change is the status quo. All groups are, of course, self-interested. But the self-interest of practitioners is closer to the needs of the public - not least because individual patients can exert direct influence by means of the personal relationship that is the clinical consultation.

The problem of legitimacy
General Management was created to be ‘in charge’ of the NHS, and was imposed upon the organization following an era from 1974 when nobody was in charge of the NHS (the health service was supposed to be run by consensus, but in action was blocked by huge committees paralyzed by vetoes; Klein, 1989). However, power in the NHS is importantly conditional upon consent and cannot be wielded unilaterally. To be effective, power must be perceived as legitimate. And legitimacy can seldom be conferred by diktat; the holders of power must be seen as in some way entitled, or else resistance will be excessive.

The legitimacy of General Management was initially seriously lacking. Indeed, how could matters be otherwise given the lack of relevant knowledge and experience of most of the newly appointed managers? Aside from some NHS administrators who underwent the transition to manager, the early cohorts of general managers were pretty much whisked-up out of thin air, under the above-mentioned belief that management skills were generic. This view was not widely shared within the health service, where it was assumed that running a military barracks, a sweet factory and a hospital presented significantly different challenges. Nonetheless, managers were bought in from the military and the private sector - although the salaries and conditions on offer from the NHS were insufficiently attractive to get the best personnel, and the early cohorts of managers were considered to be inadequately competent.

Managerial influence on the NHS was initially dependent upon various crude threats (resource cutbacks, closures, sacking etc.) and the personal qualities of the manager. Probably the most effective managers were those who combined common sense and force of character. Since managerial authority was not backed up by any convincing expertise, and because solidarity among managers was undermined by short term contracts and competition among themselves, there was a desperate search for a technique of management that would reliably impress clinicians and politicians alike, between whom managers were uncomfortably sandwiched in the NHS hierarchy. Rationalist management by Infostat methods came along just in the nick of time.

Outwitting the clinicians
So, from 1983 there has been a frantic scrabble to plaster a veneer of reasonableness over the fait accompli of the managerial takeover of the NHS. There was a dual purpose: underwriting managerial legitimacy and undermining clinical legitimacy. Managers seized-upon ‘Infostat’ techniques as the ideal tool for creating the authority necessary to perform their appointed role. Infostat is my term for the use of information technology and statistics to support and legitimate managerial decision-making.

Clinicians are a problem. Not only are they frequently confident and articulate individuals prepared to argue their corner, but clinicians have traditionally been able to claim a special legitimacy for their views based upon experience, expertise and direct personal contact with patients and this legitimacy has been supported by a broad public consensus. However, the clinical databases stored in the heads of doctors are jealously guarded and may be subjectively interpreted to benefit clinical interests. Managers can only log-into them via the doctor, and since doctors are frequently the major obstacle to NHS management attaining its objectives, it was important for managers dedicated to gaining ascendancy over clinicians to denigrate clinical experience and open-up direct access to alternative sources of data.

The lack of direct contact between patients and managers is, on the face of it, a great and apparently fatal disadvantage to their competence and credibility in assuming the mantle of advocates of public health. Infostat techniques can be seen as a response to this crucial deficiency, in that they aggrandize the significance of the large data base and denigrate the value of personal contact with patients contact (too ‘anecdotal’ and ‘subjective’); and misrepresent the scope of statistical analysis as being the best and proper procedure for managerial decision-making (Charlton, 1996b, 1997). Managers have embarked upon the collection and interpretation of large data bases in the knowledge that such sources of information are either unavailable to clinicians’ or else too inaccessible and unwieldy for busy doctors to interpret.

Much of the increased emphasis on explicit information and statistical analysis as tools of NHS policy can be seen as a rhetorical strategy with which to over-ride the claims of clinical professionals to an unique understanding of health issues. Increasingly, clinical training, experience and patient contact are derided as subjective, and marginalized as anecdotal compared with the awesome weight of ‘comprehensive’ health service data, and the dazzling power of ‘objective’ statistical analysis. Whatever is not large scale, numerical, and statistically-processed, is seen as inadmissible evidence. And politicians and management have - through Department of Health funding of Infostat technologists - ensured that they have a virtual monopoly of such evidence.

The scope and nature of INFOSTAT technologies

‘Infostat’ is a neologism coined to describe managerial technologies which employ ‘info’-rmation and subject it to ‘stat’-istical analysis in order to support decision-making.

Infostat requires large quantities of data in order for it to be a plausible reflection of health service activity and to produce summary averages of impressive statistical precision. Its introduction has therefore been inextricably linked with an demands for more information on more aspects of NHS activity. Insofar as Infostat is designed to legitimate the control of clinical interactions, this requires a greatly increased amount of data collection about clinical affairs. This managerial drive is what lies behind calls for ‘accountability’, ‘transparency’ and ‘explicitness’ in clinical decisions (Charlton, 1999).

Such demands for information are not driven by a clinical need nor for reasons of organizational effectiveness or efficiency (although these goals may be used to excuse the demands for more data). Instead the true drive behind increased information collection is the managerial need for more numerical data in relation to the activity of employees, so that employee activity can be more easily monitored and controlled.

Because data is used rhetorically, this means that quantity of data is much more important than its quality. Indeed politicians and managers are almost indifferent to the real quality of data, and academic criticisms of validity, sampling and statistical appropriateness water of a duck’s back since they are entirely subsidiary to the managerial process. Technical criticisms may provide a focus for dissent, but do not affect the steady onward implementation of Infostat because the goal data collection is not so much a matter of discovering ways of legitimately reducing clinical activities to numerical data, but of the necessity for using numerical data to describe clinical activities. So long as the data has some kind of face validity or superficial plausibility - that is all that is required. The primary issue is that there must be such data, and lots of it.

Hence at every level of the NHS an increased proportion of resources (personnel, time, effort, software, hardware etc.) are now devoted to routine information collection and processing - and the trend continues unchecked. And all this newly available data is potentially available to be fed into Infostat mechanisms and used to support management decisions. Because even though policy is not actually dictated by the outcomes of data collection and analysis, access to such accumulations of data is assumed to be a pre-requisite of competence to determine policy. In this game, statistics always trumps direct personal experience to the point that people are no longer supposed to act upon the evidence of their senses.

Pure statistical analysis is a neutral activity which does not make causal suppositions - ideally it simply clarifies what was already there. But the major pitfalls of statistics arise when it comes to attaching real world entities to mathematical symbols and making statistical manipulations without regard for the relevant causal processes. There is almost unlimited scope for statistical malpractice in health service management because Infostat is apparently applicable to any problem so long as enough data can (somehow or another) be expressed quantitatively (Charlton, 1997). Statistically precise measurement without an understanding of what the numbers represent is a snare to the incautious - and constant a temptation to the unscrupulous - because numerical exactitude is so often mistaken for understanding.

Modern NHS management operates on the basis that bad data is assumed to be better than no data at all. And from the perspective of someone wishing to use statistics as a cosh to stun the clinical opposition, this is quite correct. But from the perspective of someone actually trying to run a good health service then bad data generates wrong answers. The watchword, as always, is GIGA - ‘garbage-in, garbage-out’. No matter how high-powered the statistics, conclusions cannot be stronger than the validity and representativeness of the data base from which they were generated.

The policy sausage-machine

NHS management is trying to create the illusion of a policy sausage-machine. The NHS sausage machine is fed on numerical data and disgorges rational policies. No thinking is necessary - all is predetermined by objective information and statistics. Raw data is collected and loaded into a funnel at the top of NICE; the handle is turned; inputted information is ground-up, mixed and segmented by statistical engines; and clinical guidelines are emitted in discrete but linked gobbets. NICE is portrayed as merely a mechanism for generating rational policies; CHI merely the mechanism for ‘implementing’ these rational policies.

Of course it is all an illusion. The sausage machine is designed by politicians and operated by managers. Data input is selective, analysis is selective, and results are selective. The numerous political pressures and managerial judgments are shielded from critique by an elaborate facade of pseudo-evidence and quasi-mathematical impartiality. The official propaganda for NICE denies the massive role of arbitrary opinion and interest involved in deciding what ought to be done, and concentrates all its efforts on making people do it.

And what of doctors and the other clinicians and practitioners? The role of doctors is equivalent to stacking supermarket shelves and manning the checkouts - merely a question of receiving and distributing the sausages. The doctor’s job is to be an obedient receptive functionary, strictly following protocols; all significant decisions about the nature, content and range of sausages are made elsewhere and by other people.

Although much is made of ‘public accountability’ nowadays, the patient’s job is merely to choose from the range of goods on display (sausages will, of course, be brightly packaged, and each will have a ‘quality assurance’ stamp). The individual patient’s interaction with the policy sausage machine cannot significantly influence clinical processes and outcomes. Personal relationships to the sausage salesmen - I mean doctors - will be at the level of a fast food outlet: compulsorily cheerful, utterly stereotyped. The doctor is not allowed to take notice of individual patients needs, wants and preferences - the doctor can only stick to procedure, and act within a prescribed, predetermined and limited range of responses.

NICE and CHI are our first taste of a world in which politicians and health service managers think they know exactly what doctors ought to be doing (after all the Infostat commissars have analyzed the data and told them exactly - to the third decimal place - what doctors ought to be doing). The only obstacle that politicians and managers perceive in the path of optimal health care is that doctors will not always do what they are told. The medical profession is too strong. The only sensible solution is therefore to destroy the medical profession and force doctors (or if they still refuse, somebody else) to follow instructions. By this account of clinical practice doctors role in the health service is not to exercise autonomous judgment: the doctors role is simply to obey guidelines.

This explains why doctors are - at the time of writing, and according to the Prime Minister’s opening speech at the 1999 Party conference - officially Public Enemy Number One. Doctors are to the present government what the coal miners were to Mrs Thatcher’s government - the symbolic group who stand in the path of the untrammeled will of the state: and whose power and credibility must be broken. In this sausage-machine world, doctors are not an answer to the problem of providing health services; doctors themselves are the problem.

Doctors are no longer a group of people who in their practice actually embody the health service - rather doctors are merely one of several groups of people whose job it is to deliver health services. The actual nature of clinical service is no longer to be under the control of doctors, never mind patients, but will be decided by politicians and managers - allegedly on the basis of ever-increasing quantities of information and statistics. Who could argue with that?

The demand for Infostat

Machievelli and the demand for a managerial crib
Why did the NHS end-up on the path to this nightmare parody of a one stop superstore? The most plausible answer is that present policies arose incrementally but inexorably from the initial decision to impose General Management on the NHS: from the decision that the NHS must have a centralized leadership and hierarchical management in order that it be controllable by the government.

Many of the newly appointed General Managers of the nineteen eighties were inexperienced, inexpert, unconfident and overwhelmed by the grandiose (and often impossible) tasks handed down to them from above. Even worse, managers found themselves blocked and intimidated by the self-confidence (and arrogance) of clinicians from below. Most managers came and went swiftly on their well-paid but short-term contracts. The unscrupulous among them achieved success as hit-and-run merchants - making a name with a new logo and letterhead, then up the ladder before anything could go wrong. Locations and roles changed so frequently that specific local knowledge of health and personnel had no time to develop. In other words, health service managers found themselves in a position analogous to Machiavelli’s ‘Prince’ to whom that infamous book of advice on manipulation and survival was addressed (Machiavelli, 1992).

To condense a passage from the political philosopher Michael Oakeshott (1962): ‘Machiavelli wrote for the new prince of his day who brought to his task only the qualities that had enabled him to gain political power. Lacking education (except in the habits of ambition), and requiring some short-cut to the appearance of education, he required a crib. The project of Machiavelli was, then, to provide a crib to politics, a political training in default of a political education, a technique for the ruler who had no tradition.’

Infostat technologies have provided exactly the crib that General Management needed. In a real sense, General Management was neither personally nor professionally competent to do the job it was given. Yet the job had to be done somehow - the BMW needs replacing, school fees escalate, mortgages must be paid. Managers needed a plausible rationale for their decisions, and arguments to back them up. Such arguments must be of a sort designed if not to convince clinicians, then at least to neutralize, bamboozle, side-step or overwhelm clinical opposition. The ideal of Infostat is not to out-argue opposing viewpoints by decisive logical or scientific evidence, but instead to bury them under a sheer mass of evidence.

In this managerial model, everything hinges upon gaining legitimacy for the Infostat crib. Infostat purports to capture the essence of what clinical practice ought to be. If it did, in fact, do so; then coercive managerial dominance over the consultation would be justifiable and probably desirable. If it does not - and the Infostat crib is merely a more-or-less plausible technical trick - then the NICE claim for legitimacy in controlling the doctor-patient interaction is exploded.

NICE as cargo cult science
Infostat bears some resemblance to science. But the Infostat technologies which led-up to the creation of NICE are actually an example of what Richard Feynman called ‘cargo cult’ science: an activity presenting the facade of science but without the substance (1986).

Feynman described how remote and undeveloped South Seas islands briefly enjoyed prosperity when used as US air bases during the Second World War. When peace came the air force left, and with it the prosperity. On the islands quasi-religions developed with the goal of attracting cargo-bearing aeroplanes to return and restore the golden age. These cults took the trappings of a technological culture - runways, marker flares, control towers, headphones, aerials etc. - but all constructed from wood and stone technology. Everything was assembled in the proper way, the appearance of everything seemed just about right, the people went through the right motions… but the planes never landed.

By analogy, Infostat has many of the trappings of science - researchers, data collection, statistics, large grants, publications and so on and so forth. Its practitioners chatter about concepts such as ‘evidence’, ‘effectiveness’, ‘measurement’ and ‘rigour’. NICE even appoints a Professor of clinical pharmacology as its Chair - someone who wears a white coat, does ward rounds and used to work in a laboratory. But like the cargo cult, it is just a facade. It superficially looks like science but it just doesn’t work - no cumulative, ‘reliable knowledge’ results. No planes land.

The reason is that essential elements are missing from the recipe. As explained above, the processes of science and of NICE are profoundly different - one is democratic the other autocratic, one is orientated toward practice the other to policy, one is tested against the natural world and the other against political expediency. Feynman suggests that the crucial missing attribute in cargo cult science is an attitude of integrity and honesty about real world results, and certainly this integrity is lacking in managerial discourse (as Feynman discovered when he investigated the space shuttle Challenger disaster; 1988).

In contrast to the ethic of honesty which necessarily permeates scientific practice (Bronowski, 1975) NICE already exemplifies a fundamental evasiveness. For instance, by conflating effectiveness with cost-effectiveness in its mission statements, NICE commits the basic dishonesty of implying that these quite distinct variables can routinely be satisfied by a single recommendation. The first NICE decision to disallow the anti-flu drug Relenza likewise used mendacious criteria manufactured on the hoof for the purpose of rationalizing a politically-motivated decision. And what about the acronym NICE? - that name speaks volumes about an organization based upon spin-doctoring rather than patient-doctoring.

The parable of the notebooks
NICE presents itself as a scientific organization, merely implementing the objectively compelling results of data analysis - hence it claims the objectivity and impartiality of a scientific process. But NICE is almost the opposite of science, because it reverses the relationship between theory and practical experience. Science sees theory as a conjectural explanations with testable consequence to be measured up against practical experience under controlled conditions. NICE, by contrast, sees theory as technique and seeks not to explain, but to excuse - specifically to fabricate excuses for the centralized control of medical practice.

Because it is a matter of applying technique to data in order to support decision, Infostat management creates the impression of being able to manufacture knowledge from data. This is reminiscent of the parable told by Karl Popper and quoted by Jacob Bronowski (1975). A man spent his whole life making detailed observations of the world and writing them into notebook after notebook - a meticulous record, leaving out nothing. Humidity levels, racing results, cosmic radiation, the stock market - all were included in his compilation.

Many hundreds of these thick notebooks were bequeathed to the Royal Society on his death, for the use of the premier scientists of the Nation. But the secretary of the Royal Society read the bequest, took one swift glance at the accumulated volumes, then promptly ordered that the whole lot to be dumped into landfill site...

Why? Well, the moral of the story is that the secretary of the Royal Society knew, without even looking, that the contents of the notebooks must be scientifically worthless. Science is in the questions asked and the means by which they are addressed; science is not an undiscriminating process of relentless, cumulative, impartial observation. Science is not a loose-leaf folder of facts, but the structured knowledge of principles that gives rise to the facts - the hidden likeness which explain observations and not the observations themselves. And the application of statistical procedures to such notebooks would not affect the issue - condensed garbage is still garbage.

Infostat management - of which NICE is the most extreme example so far - shares the misguided approach of this unfortunate diarist. NICE is a diabolical engine that manufactures decisions when fed with data. These decisions are legitimated - not by the quality of the evidence nor by the patterns revealed by analysis - but by burying the opposition under a heap of misleading information and statistical obfuscation, and by intimidating dissent through the sanctions deployed by CHI. Real medical knowledge does not come to order, nor can it be accelerated by ‘fast track’ timetables. Its pace is dictated by the natural history of disease and the experience of active practitioners. We know no other valid way.

Of course, neither I nor anyone else can prove conclusively that good medicine cannot be practiced using a pseudo-objective system of data-exused management such as NICE. But such a requirement is absurd - it is logically equivalent to asking for proof that the Loch Ness Monster cannot exist. The present system of medical practice by highly autonomous doctors is already in place, and works to a high level of effectiveness and efficiency, and gives (according to available data) a remarkably high level of public satisfaction. It needs to be improved - but improved by provision of more health services, not by cutting and hamstringing services in order to increase managerial control mechanisms. Doctors are far from perfect, but the public would rather trust the judgment of their doctor than a government bureaucracy.

The effectiveness and efficiency of health care imposed by diktat from NICE and CHI is purely theoretical, and there is no reason to assume it will work. The onus of proof lies with those who suggest that an Infostat-fuelled, guideline-generating sausage machine forms a better model for medical practice than that which currently operates.

Conclusions
NICE is not about science, it is about government and managers attaining the statutory power to control doctors. The aspirations and supposed benefits of NICE do not stand up to a moments consideration. Its scientific rhetoric is merely an excuse for coercion, its scientific rationale merely the trappings of a cargo cult. We already know how to do science (Hull, 1988), and it is not done like NICE. Top-down regulation based on Infostat technologies is a branch a management theory, not of human biology. Political expediency will corrupt science.

NICE is just part of the NHS sausage machine, a mechanism which exists to support policy decisions and which has little incentive to seek the scientific truth. The policy sausage machine was designed and paid for by politicians, and it will be influenced primarily by politicians. Whatever the phony feel-good mission statements about ‘clinical excellence’ and ‘health improvement’, the fact is that NICE is a government funded bureaucracy, and will behave exactly how we expect government funded bureaucracies to behave. In other words, NICE will make recommendations the nature of which will excused by impressive sounding statistics; but the real point of NICE is not what is in the recommendations, but the fact that there are recommendations and the fact that these recommendations will be enforced on doctors by the CHI inquisitors.

Since the real function of NICE and CHI is to control the medical workforce, as long as there are recommendations to be enforced by CHI, then NICE will be serving its political purpose. So NICE will control by persuasion, and CHI by intimidation - by carrots and whips. And these are big whips.

T D Lysenko was a biologist in the Soviet Union to whom Stalin granted the power to enforced his theories upon fellow biologists (Joravsky, 1970). From approximately 1927 to 1964, Lysenko ruled his discipline using the draconian methods of a totalitarian state. Biological science was virtually destroyed, and replaced by data fabrication in the service of propaganda - great agricultural progress was claimed: meanwhile the peasants starved. Disobedient scientists who criticized or resisted were removed from their posts - some were imprisoned and killed.

NICE is not like that - the proposal is only that doctors who disobey NICE recommendations will be harassed, humilated, sacked or struck-off the medical register. CHI will only remove their livelihood, not their lives.

We are assured that NICE and CHI will deploy their powers responsibly, and that sanctions will be used only on those people who really deserve them. The individuals who make such assurances are no doubt sincere. But once NICE and CHI have power over medical practice, we will be reliant entirely on their sense of restraint to prevent abuses.

Probably, the threat of sanctions will be enough to ensure compliance. Like Galileo, doctors will merely be shown the instruments of torture. NICE and CHI can rely on their imaginations to do the rest.

References

Bailar III J. (1997) The promise and problems of meta analysis. New England Journal of Medicine 337, 559-61.

Bronowski J. (1975). Science and human values Harper Colophon, New York.

Charlton BG. (1993a). Management of science. Lancet 342, 99-100.

Charlton BG. (1996a). Megatrials are based on a methodological mistake. British Journal of General Practice 46, 429-431.

Charlton BG. (1996b). The uses and abuses of meta-analysis. Family Practice 13, 397-401.

Charlton BG. (1996). The future of clinical research: from megatrials towards methodological rigour and representative sampling. Journal of Evaluation in Clinical Practice, 2, 159-169.

Charlton BG. (1997) Restoring the balance: Evidence-based medicine put in its place. Journal of Evaluation in Clinical Practice 3, 87-98.

Charlton BG & Miles A. (1998) The rise and fall of EBM. QJM 91, 371-374.

Charlton BG. The ideology of accountability. Journal of the Royal College of Physicians of London 33, 33-5.

Feinstein AR. (1995). Meta-analysis: statistical alchemy for the 21st century Journal of Clinical Epidemiology 48, 71-79.

Feynman RP. (1986). ‘Surely you’re joking Mr Feynman. Unwin, London.

Feynman RP. (1988) ‘What do you care what other people think’. Unwin, London.

Goodman NW. (1998) Anaesthesia and evidence-based medicine. Anaesthesia. 53, 353-68.

Goodman NW. (1999) Who will challenge evidence-based medicine? Journal of the Royal College of Physicians of London. 33, 249-51

Griffiths R (1983). Report of the NHS management inquiry. DHSS, London.

Hull DL. (1988). Science as a process - an evolutionary account of the social and conceptual development of science University of Chicago Press, Chicago.

Jenkins S. (1995). Accountable to none - the Tory nationalization of Britain Hamish Hamilton, London.

Joravsky D. (1970). The Lysenko affair. Harvard University Press: Cambridge, Massachusetts.

Judson HF. 1979 The eighth day of creation: makers of the revolution in biology. Jonathan Cape: London.

Julian DG (1995) Secondary prophylaxis after myocardial infarction. BMJ. 310, 61.

Klein R. (1989). The politics of the National health service (2nd Edition). Longman, London.

Le Fanu J. (1999) The Rise and fall of Western Medicine. Little, Brown and Co, London.

Machiavelli N. (1992). The prince David Campbell, London.

Oakeshott M. (1962). Rationalism in politics and other essays. Methuen, London.

Peters T. (1989). Thriving on chaos: handbook for a managerial revolution Pan, London.

Sackett Dl, Haynes RB, Tugwell, P. (1985). Clinical Epidemiology Little, Brown & co., Boston.

Ziman J. (1978). Reliable knowledge: an exploration of the ground for belief in science Cambridge University Press, Cambridge.

Bruce G Charlton MD
Department of Psychology
University of Newcastle upon Tyne
NE1 7RU
England

E-mail
bruce.charlton@ncl.ac.uk

HOME

also by Bruce Charlton
Quality Assurance Auditing
The Malaise Theory of Depression
Psychiatry and the Human Condition
Awareness, Consciousness and Language
Psychopharmacology and The Human Condition
Injustice, Inequality and Evolutionary Psychology