Mills lists a variety of ways that data can be manipulated (“tortured”) to achieve a goal by a dishonest researcher. A number of questions can screen for data manipulation.
Questions that may trigger more questions:
(1) Did the data come from testing a primary or secondary hypothesis generated before the data was analyzed?
(2) What is the rationale for inclusion and exclusion criteria?
(3) Were key defiinitions (exposure, outcome, cutoff points) clearly stated before looking at the data?
(4) How many statistical tests were performed during the course of data analysis?
(5) Were multiple comparisons used? If so, were appropriate adjustments made?
(6) Are both p values and confidence intervals reported?
(7) Has data for all subgroups and all follow-up points been reported?
To read more or access our algorithms and calculators, please log in or register.
