Mills lists a variety of ways that data can be manipulated (“tortured”) to achieve a goal by a dishonest researcher.


Starting point: available data


Process: Multiple hypotheses are generated until one fits the data.


This approach may involve one or more of the following:

(1) multiple different types of statistical analysis performed until something “significant” is identified

(2) misuse of P

(3) making multiple comparisons without making the needed adjustments

(4) no hypothesis in advance or multiple changes to it


Registering a study in advance is one way of reducing this problem.


To read more or access our algorithms and calculators, please log in or register.