The irrelevance of statistics-based research to individual patients

Article Sidebar

Requires Subscription or Fee PDF (USD 25)
Published: Jun 30, 2011
DOI: https://doi.org/10.5750/ijpcm.v1i2.64

Main Article Content

James Penston
Scunthorpe General Hospital, North Lincolnshire, UK

Abstract

Randomised controlled trials and epidemiological studies provide the grounds for much of clinical medicine and consequently affect the lives of millions of patients around the world. But this statistics-based research offers little to individual patients.The ability to generalise the results of these studies to the wider population of patients is unsuccessful. External validity is brought into question by the selection involved in the recruitment process and in the participation in clinical trials. Thus, we cannot know with any confidence to whom the results apply and this has obvious implications for individual patients. Yet, there is a further problem: conventional statistical analysis involves the frequentist approach which entails that probabilities only apply to classes. Hence, the results of this type of research are not strictly applicable to individuals.But even if we set aside the difficulties associated with external validity, there is a more important problem which relates to the size of the treatment effect. Large-scale studies entail that any treatment effect detected will be very small. While this is often hidden by the use of relative risk reductions, once the absolute treatment difference is presented, the paltry size of the benefit becomes clear. This has little importance or relevance to the lives of individual patients and, as is argued, has doubtful meaning.Current medical practice encourages patients’ participation in decisions regarding their care and this includes providing them with sufficient information to enable them to make informed choices about their treatment. However, were they to be told about the problems with external validity and the true size of the benefit – not to mention the many other problems with statistics-based research – it is likely that far fewer would accept treatment than is currently the case.

Article Details

Issue
Vol. 1 No. 2 (2011): June
Section
Regular Articles
Author Biography

James Penston, Scunthorpe General Hospital, North Lincolnshire, UK

Consultant Physician/Gastroenterologist

References

. Penston, J. (2010). Stats.con – How we’ve been fooled by statistics-based research in medicine. The London Press.

. Elwood, J.M. (1988). Causal relationships in medicine. Oxford: Oxford University Press.

. Rothwell, P.M. (2005). Treating individuals 1: External validity of randomised controlled trials: “To whom do the results of this trial apply?” Lancet 365, 82-93.

. Fuks, A., Weijer, C., Freedman, B., Shapiro, S., Skrutkowska, M. and Riaz, A. (1998). A study in contrasts: eligibility criteria in a twenty-year sample of NSABP and POG clinical trials. National Surgical Adjuvant Breast and Bowel Program. Pediatric Oncology Group. Journal of Clinical Epidemiology 51, 69-79.

. Hall, J.C., Mills, B., Nguyen, H. and Hall, J.L. (1996). Methodologic standards in surgical trials. Surgery 119, 466-472.

. Shapiro, S.H., Weijer, C. and Freedman, B. (2000). Reporting the study populations of clinical trials. Clear transmission or static on the line? Journal of Clinical Epidemiology 53, 973-979.

. Wilcox, R.G. et al (1988). For the ASSET study group. Trial of tissue plasminogen activator for mortality reduction in acute myocardial infarction. Anglo-Scandinavian Study of Early Thrombolysis (ASSET). Lancet 332, 525-530.

. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. (1988). Randomised trial of intravenous streptokinase, oral aspirin, both or neither among 17187 cases of suspected acute myocardial infarction: ISIS-2. Lancet 332, 349-360.

. ISIS-3 (Third International Study of Infarct Survival) Collaborative Group. (1992). ISIS-3: a randomised comparison of streptokinase vs tissue plasminogen activator vs anistreplase and of aspirin plus heparin vs aspirin alone among 41,299 cases of suspected acute myocardial infarction. Lancet 339, 753-770.

. LATE Study Group. (1993). Late assessment of thrombolytic efficacy (LATE) study with alteplase 6-24 hours after onset of acute myocardial infarction. Lancet 342, 759-766.

. Assessment of the Safety and Efficacy of a New Thrombolytic (ASSENT-2) Investigators. (1999). Single-bolus tenecteplase compared with front-loaded alteplase in acute myocardial infarction: the ASSENT-2 double-blind randomised trial. Lancet 354, 716-722.

. Penston, J. (2003). Fiction and fantasy in medical research. London: The London Press.

. Gruppo Italiano per lo Studio della Sopravvivenza nell’Infarto Miocardico. (1990). GISSI-2: a factorial randomised trial of alteplase versus streptokinase and heparin versus no heparin among 12,490 patients with acute myocardial infarction. Lancet 336, 65-71.

. Gruppo Italiano per lo Studio della Sopravvivenza nell’Infarto Miocardico. (1994). GISSI-3: effects of lisinopril and transdermal glyceryl trinitrate singly and together on 6-week mortality and ventricular function after acute myocardial infarction. Lancet 343, 1115-1122.

. Gilbody, S., Wahlbeck, K. and Adams, C. (2002). Randomised controlled trials in schizophrenia: a critical perspective on the literature. Acta Psychiatrica Scandanavica 105, 243-251.

. Muller, D.W. and Topol, E.J. (1990). Selection of patients with acute myocardial infarction for thrombolytic therapy. Annals of Internal Medicine 113, 949-960.

. Haynes, R.B. and Dantes, R. (1987). Patient compliance and the conduct and interpretation of therapeutic trials. Controlled Clinical Trials 8, 12-19.

. Gomez-Marin, O., Prineas, R.J. and Sinaiko, A.R. (1991). The sodium-potassium blood pressure trial in children: design, recruitment and randomisation. Controlled Clinical Trials 12, 408-423.

. Pablos-Mendez, A., Barr, R.G. and Shea, S. (1998). Run-in periods in randomised trials: implications for the application of results in clinical practice. Journal of the American Medical Association 279, 222-225.

. Yusuf, S. et al. (2000). for the Heart Outcomes Prevention Evaluation Study Investigators. Effects of an angiotensin-converting-enzyme inhibitor, ramipril, on cardiovascular events in high-risk patients. New England Journal of Medicine 342, 145-153.

. Licht, R.W., Gouliaev, G., Vestergaard, P. and Frydenberg, M. (1997). Generalisability of results of randomised drug trials. British Journal of Psychiatry 170, 264-267.

. Gurwitz, J.H., Col, N.F. and Avorn, J. (1992). The exclusion of elderly and women from clinical trials in acute myocardial infarction. Journal of the American Medical Association 268, 1417-1422.

. Bjorn, M., Brendstrup, C., Karlsen, S. and Carlsen, J.E. (1998). Consecutive screening and enrolment in clinical trials: the way to representative patient samples? Journal of Cardiac Failure 4, 225-230.

. Brown, N., Melville, M., Gray, D. et al. (1999). Relevance of clinical trial results in myocardial infarction to medical practice: comparison of four year outcome in participants of a thrombolytic trial, patients receiving routine thrombolysis, and those deemed ineligible for thrombolysis. Heart 81, 598-602.

. Britton, A., McKee, M., Black, N., McPherson, K., Sanderson, C. and Bain, C. (1999). Threats to applicability of randomised trials: exclusions and selective participation. Journal of Health Services Research and Policy 4, 112-121.

. Moore, D.A., Goodall, R.L., Ives, N.J., Hooker, M., Gazzard, B.G. and Easterbrook, P.J. (2000). How generalisable are the results of large randomised controlled trials of antiretroviral therapy? HIV Medicine 1, 149-154.

. Woods, K.I., Ketley, D., Ludman, P.F. et al. (1999). Intravenous beta blockade in acute myocardial infarction. Doubt exists about external validity of trials of intravenous beta blockade. British Medical Journal 318, 328-329.

. Dixon, G., Boyle, R.M. and Norris, R.M. (2000). Clinical trials in acute myocardial infarction versus real life: a limitation of evidence-based medicine. British Journal of Cardiology 7, 709-711.

. Hampton, J.R. (2002). Size isn’t everything. Statistics in Medicine 21, 2807-2814.

. Jarvinen, T.L.N., Sievanen, H., Kannus, P., Jokihaara, J. and Khan, K.M. (2011). The true cost of pharmacological disease prevention. British Medical Journal 342, d2175.

. Downs, S.H., Black, N.A., Devlin, H.B., Royston, C.M. and Russell, R,C. (1996). Systematic review of the effectiveness and safety of laparascopic cholecystectomy. Annals of the Royal College of Surgeons of England 78, 241-323.

. Roberts, C. (1999). The implications of variation in outcome between health professionals for the design and analysis of randomised controlled trials. Statistics in Medicine 18, 2605-2615.

. Cohen, L.J. (1989). An introduction to the philosophy of induction and probability. Oxford: Clarendon Press.

. Best, W.R., Becktel, J.M., Singleton, J.W. and Kern, F. (1976). Development of a Crohn’s disease activity index. National Copoperative Crohn’s Disease Study. Gastroenterology 70, 439-444.

. Gotzsche, P.C. (1989). Methodology and overt and hidden bias in reports of 196 double-blind trials of non-steroidal anti-inflammatory drugs in rheumatoid arthritis. Controlled Clinical Trials 10, 31-56.

. Thornley, B. and Adams, C. (1998). Content and quality of 2000 controlled trials in schizophrenia over 50 years. British Medical Journal 317, 1181-1184.

. Naylor, C.D., Chen, E. and Strauss, B. (1992). Measured enthusiasm: does the method of reporting trial results alter perceptions of therapeutic effectiveness? Annals of Internal Medicine 117, 916-921.

. Forrow, L., Taylor, W.C. and Arnold, R.M. (1992). Absolutely relative: how research results are summarized can affect treatment decisions. American Journal of Medicine 92, 121-124.

. Bobbio, M., Demichelis, B. and Giustetto, G. (1994). Completeness of reporting trial results: effect on physicians’ willingness to prescribe. Lancet 343, 1209-1211.

. Bucher, H.C., Weinbacher, M. and Gyr, K. (1994). Influence of method of reporting study results on decisions of physicians to prescribe drugs to lower cholesterol concentrations. British Medical Journal 309, 761-764.

. Cranney, M. and Walley, T. (1996). Same information, different decisions: the influence of evidence on the management of hypertension in the elderly. British Journal of General Practice 46, 661-663.

. Fahey, T., Griffiths, S. and Peters, T.J. (1995). Evidence based purchasing: understanding results of clinical trials and systematic reviews. British Medical Journal 311, 1056-1059.

. Malenka, D.J., Baron, J.A., Johansen, S., Wahrenberger, J.W. and Ross, J.M. (1993). The framing effect of relative and absolute risk. Journal of General Internal Medicine 8, 543-548.

. Skolbekken, J-A. (1998). Communicating the risk reduction achieved by cholesterol reducing drugs. British Medical Journal 316, 1956-1958.

. Friedmann, P.D., Brett, A.S. and Mayo-Smith, M.F. (1996). Differences in generalists’ and cardiologists’ perceptions of cardiovascular risk and the outcomes of preventive therapy in cardiovascular disease. Annals of Internal Medicine 124, 414-421.

. Lewis, D.K., Robinson, J. and Wilkinson, E. (2003). Factors involved in deciding to start preventive treatment: qualitative study of clinicians’ and lay people’s attitudes. British Medical Journal 327, 841.

. Lewis, D.K. and Barton, S. (2003). Who decides when to start preventive treatment? A questionnaire survey to compare the views of different population subgroups.

Journal of Epidemiology and Community Health 57, 241-242.

. Steel, N. (2000). Thresholds for taking antihypertensive drugs in different professional and lay groups: questionnaire survey. British Medical Journal 320, 1446-1447.

. MacAlister, F.A., O’Connor, A.M., Wells, G., Grover, S.A. and Laupacis, A. (2000). When should hypertension be treated? The different perspectives of Canadian family physicians and patients. Canadian Medical Association Journal 163, 403-408.

. Trewby, P.N., Reddy, A.V., Trewby, C.S., Ashton, V.J., Brennan, G. and Inglis, J. (2002). Are preventive drugs preventive enough? A study of patients’ expectation of benefit from preventive drugs. Clinical Medicine 2, 527-533.

. Penston, J. (2007). Patients’ preferences shed light on the murky world of guideline-based medicine. Journal of Evaluation in Clinical Practice 13, 154-159.