A proposed framework for the pre-specification of statistical analysis methods in clinical trials (Pre-SPEC)

Choosing a statistical analysis approach in clinical trials requires us to make a number of decisions. Which patients should we include in the analysis? What statistical model should we use? How should we handle missing data? Each of these decisions will have an impact on the results of our analysis, and different choices could lead to different interpretations of the data.

But too much freedom can be a bad thing when it comes to analysis. The danger is that we may use the trial data to help us choose a method that gives us the answer we want — if we run enough analyses, one of them is bound to give a significant result. The solution to this problem is to pre-specify; we need to choose our analysis method before seeing the data.

However, this concept is not as simple as it first appears. If I say that I’ll use multiple imputation to handle missing outcome data, is this pre-specified? After all, I’ve said what I plan to do. Except that I haven’t really; there are many different ways to do imputation, and I’ve not said which one I plan to use. I could keep running different imputation methods until I got the result I wanted, and claim it was pre-specified.

Pre-specification is not just about saying what we plan to do; it’s also about making sure we can’t use the data to choose a method that gives us the answer we want. We have recently proposed a framework (Pre-SPEC) (https://arxiv.org/abs/1907.04078) for the pre-specification of statistical analyses that is based around this principle. This framework is intended to help trialists plan their own analysis approach, and also to help reviewers and journal editors identify whether trial results may be at risk of bias due to inadequate pre-specification.

Our proposed framework involves five points:

• Pre-specify the analysis methods before recruitment to the trial begins;
• Specify a single primary analysis strategy (if multiple analyses are planned, one should be identified as the primary);
• Plan all aspects of the analysis, including the analysis population, statistical model, use of covariates, handling of missing data, and any other relevant aspects;
• Enough detail should be provided so that a third party could independently perform the analysis (ideally this could be achieved by providing the planned statistical code); and
• Adaptive analysis strategies which use the trial data to inform some aspect of the analysis should use deterministic decision rules

This framework is still a work in progress, so we are interested in what others think; if you have comments or suggestions, drop us a line (https://arxiv.org/abs/1907.04078).

Short bio

Brennan Kahan is a 2019 Doug Altman Scholarship recipient and a medical statistician at the Pragmatic Clinical Trials Unit, Queen Mary University of London. His research interests include design and analysis of clinical trials, and improving transparency in the analysis of clinical trials. He has no conflicts of interest to declare.