Mills lists a variety of ways that data can be manipulated (“tortured”) to achieve a goal by a dishonest researcher.
Starting point: available data
Process: Multiple hypotheses are generated until one fits the data.
This approach may involve one or more of the following:
(1) multiple different types of statistical analysis performed until something “significant” is identified
(2) misuse of P
(3) making multiple comparisons without making the needed adjustments
(4) no hypothesis in advance or multiple changes to it
Registering a study in advance is one way of reducing this problem.
