Statistical principles & resources
General statistics resources related to ethics
- Some of Resnik's ethics papers on appropriate use and misuse of statistics:
- Gardenier & Resnik 2002: The misuse of statistics: Concepts, tools, and a research agenda
- Resnik 2000: Statistics, ethics and research: An agenda for education and reform. (Note this paper highlights a number of important issues for ethical statistical practice. However, Resnik is an ethicist, not a statistician. Statisticians will find that some of his statistical comments and recommendations should be revised.)
- DeMets' 1999 paper Statistics and ethics in medical research focusing on the importance of using statistics ethically as well as correctly in study design and data analysis
- Deming's 1965 seminal paper on Principles of Professional Statistical Practice
- Series of eight papers by Douglas Altman in BMJ in the 1980's on Statistics and Ethics in Medical Research
- Misuse of statistics is unethical
- Study design
- How large a sample?
- Collecting and screening data
- Analyzing data
- Presentation of results
- Interpreting results
- Improving the qualitity of statistics in medical journals
- Ten categories of statistical errors (Holmes, 2004) as applied to the physiology literature
False positives and false negatives
- Boffetta commentary on false positive results followed by letters on arguing false negatives are also important (Clapp & Kriebel ; Crosignani) and Boffetta et al's reply. Also JNCI's media memo discussing the Boffetta commentary (url link)
- Science article about policy's oversensitivity to false alarms
- Analysis and categorization of false positives in regulation (Hansen et al 2007)
P-values
- Even the New York Times gets it wrong: See the March 11, 2013 article "Putting a Value to 'Real' in Medical Research" by Nicholas Bakalar and the firestorm of responses on blogs: Andrew Gelman's, Larry Wasserman's, Hilary Parker's, ...
- Schervish (1996) P-values: what they are and what they are not explains how p-values do not quantify evidence, directly
References on statistical significance, p-values, and multiple comparisons
- Statistical Significance
- Hubbard and Armstrong (2006) Why we don't really know what statistical significance means: implications for educators. A review of the confusion between testing and other forms of inference
- Gelman and Stern (2006) The Difference Between "Significant" and "Not Significant" is Not Itself Statistically Significant. Examples of using the p-value as a (poor) summary of an interval
- Mulitple Comparisons
- Rothman (1990) No Adjustments Are Needed for Multiple Comparisons. Be extremely careful if you want to make this argument!
- Perneger (1998) What's Wrong With Bonferroni Adjustments is similar, but takes care to acknowledge the Neyman-Pearson validity of Bonferroni
Reporting Statistics
- See Reporting results
- Tables: see Reporting results
- Fleming's paper "Discerning Hype from Substance" decribing the bias for positive findings in research with strategies for avoiding this bias (as applied to clinical trials but this advice is gnerally applicable.)
Reproducible research
The recent conversation among biostatisticians about reproducible research started with the 2006 Peng et al article in AJE, followed by the 2009 Biostatistics policy. Niels Keiding followed up with a letter to the Biostatistics Advisory Board suggesting their focus on reproducibility is too superficial and trivializes much of the complex work done by statisticians. The Advisory Board asked Dr. Keiding to turn his letter into a commentary published in the July 2010 issue of Biostatistics. In this issue the commentary was preceded by an editorial from Diggle & Zeger and followed by invited comments from Breslow, Cox & Donnelly, Deangelis & Fontanarosa, Donoho, Goodman, and Grove. Peng contributed a discussion of the commentary and Keiding replied.
Other relevant links:
- Joseph Loscalzo's commentary "Irreproducible Experimental Results" discusses the importance of reproducibility in science, clarifies the distinction between reproducible and replicable, and discusses multiple reasons for failure to reproduce experimental results.
- Roger Peng's blog post on the Simply Statistics blog is a brief introduction to the ideas of reproducibility in research
- Here is a follow-up post discussing reproducibility and reciprocity
- Model Transparency and Validation Report of the ISPOR-SMDM Modeling Good Research Practices Task Force
- December 2011 special section in Science on Data Replication and Reproducibility. Here is the introduction. Of particular interest to quantitative scientists, look at Peng's article on reproducibility in computational science, Ioannidis & Khoury's article on validating "omics" research, and Santer et al's discussion of reproducibility in climate science.
- Guidelines in the bioinformatics literature by Huang & Gottardo
- Laine et al 2007 paper in Annals of Internal Medicine
- Delamothe 1996 BMJ editorial "Whose data are they anyway?"
- Boulesteix 2010 paper about overoptimism in bioinformatics research
Subgroup analyses
- Weiss commentary on reporting subgroups
- Example (Rebbeck 2007 and erratum)
- Martin's 1984 comment in The Lancet "Munchausen's statistical grid which makes all trials significant"
Teaching ethics in statistics
- Statistical consulting class website from U Minnesota (S Weisberg) with several ethics lectures
- Lawrence Lesser's paper on teaching statistical ethics. This paper gives some background on ethics, an overview of ethical consideration for several major statistical topics, and it also includes a long list of references.
The role of the hypothesis testing paradigm in FDA drug approval: The carvedilol story (4 papers in Controlled Clinical Trials 1999)
- One of the results papers (Packer et al NEJM 1996) reporting on mortality as an endpoint
- Introduction to the FDA approval process in the context of carvedilol by Fisher & Moye
- Fisher's perspective
- Moye's perspective
- Fisher responds
REFERENCES
Gardenier, John, and David Resnik. "The misuse of statistics: concepts, tools, and a research agenda." Accountability in Research: Policies and Quality Assurance 9.2 (2002): 65-74.
Resnik, David B. "Statistics, ethics, and research: an agenda for education and reform." Accountability in Research 8.1-2 (2000): 163-188.
DeMets, David L. "Statistics and ethics in medical research." Science and Engineering Ethics 5.1 (1999): 97-117.
Deming, W. Edwards. "Principles of professional statistical practice." Encyclopedia of Statistical Sciences (1986).
Altman, Douglas G. "Statistics and ethics in medical research. Misuse of statistics is unethical." British Medical Journal 281.6249 (1980): 1182.
Altman, Douglas G. "Statistics and ethics in medical research: study design." British Medical Journal 281.6250 (1980): 1267.
Altman, Douglas G. "Statistics and ethics in medical research: III How large a sample?." British Medical Journal 281.6251 (1980): 1336-1338.
Altman, Douglas G. "Statistics and ethics in medical research. Collecting and screening data." British Medical Journal 281.6252 (1980): 1399-1401.
Altman, Douglas G. "Statistics and ethics in medical research: V--Analysing data." British Medical Journal 281.6253 (1980): 1473-1475.
Altman, D. G. "Statistics and Ethics in Medical Research. VI--Presentation of Results." British Medical Journal 281.6254 (1980): 1542-544.
Altman, D. G. "Statistics and Ethics in Medical Research. VII--Interpreting Results." British Medical Journal 281.6255 (1980): 1612-614.
Altman, Douglas G. "Statistics and ethics in medical research. VIII-Improving the Quality of Statistics in Medical Journals." British Medical Journal (Clin Res Ed) 282.6257 (1981): 44-46.
Holmes, Tyson H. "Ten categories of statistical errors: a guide for research in endocrinology and metabolism." American Journal of Physiology-Endocrinology and Metabolism 286.4 (2004): E495-E501.
Boffetta, Paolo, et al. "False-positive results in cancer epidemiology: a plea for epistemological modesty." Journal of the National Cancer Institute 100.14 (2008): 988-995.
Crosignani, Paolo. "Re: false-positive results in cancer epidemiology: a plea for epistemological modesty." Journal of the National Cancer Institute 101.3 (2009): 212-213.
LINK: "Skepticism and Greater Awareness of Epidemiology's Limitations Could Reduce Impact of False-Positive Cancer Results." Journal of the National Cancer Institute 100.14 (2008): 975.
Pacala, Steven W., et al. "False alarm over environmental false alarms." Science 301.5637 (2003): 1187-1188.