When Should You Write your Statistical Analysis Plans?

Recently in the LinkedIn Biostatistics group, one user asked:

A client wants me to prepare final SAP [statistical analysis plan] shortly after protocol and CRFs are finalized for a Phase 3 trial, to submit to FDA prior to start of study. I find this unusual. Any experience doing so? When?

As of the time of this writing, this discussion topic had the most contributions of any topics in the Biostatistics group, so there is clearly interest in the topic.

There are also a wide range of opinions, from the original discussant thought it odd to have a SAP completed before the initiation of the study, to those who prefer to have the SAP completed early. Given the diverse opinions, it is hard to know the right thing to do.

ICH E9 (Section 5.1) gives the following advice:

The statistical analysis plan (see Glossary) may be written as a separate document to be completed after finalizing the protocol. In this document, a more technical and detailed elaboration of the principal features stated in the protocol may be included (see section 7.1). The plan may include detailed procedures for executing the statistical analysis of the primary and secondary variables and other data. The plan should be reviewed and possibly updated as a result of the blind review of the data (see 7.1 for definition) and should be finalized before breaking the blind. Formal records should be kept of when the statistical analysis plan was finalized as well as when the blind was subsequently broken.

Clearly, the SAP needs to be finalized before the breaking of the blind (in an adaptive trial, this means before the first interim analysis is performed). Beyond that, the guidance does not mandate any particular schedule beyond hints that a blinded data review might lead to revisions. Regulatory factors, such as whether the protocol is under a special protocol assessment at the FDA, may affect the timing of the SAP as well.

At Cato Research we prefer SAPs to be written soon after the completion of the protocol and after perhaps a first draft of the case report form. Doing so offers the following advantages:

1. The details of the statistical analysis can be completed and documented when the details of the protocol are still fresh. These details may be forgotten if the analysis plan is put off, especially for multiyear studies.
2. The SAP encourages further reflection on – and revision to, if necessary – the study design. The statistical analysis plan details the methodology that will be used to manipulate the data to draw conclusions concerning the study’s objectives. A lot of time “the devil is in the details,” and it takes thinking about these detailed issues, such as handling of dropouts and missing data, to realize alterations that may need to be made to the study. If alterations to the study are no longer feasible, the project team can use the lead time to decide how to deal with the issues.
3. The SAP can be used as a tool to develop and judge the adequacy of the case report form. An experienced biostatistician will usually be able to say if the data structure from the case report form will meet the needs of the statistical analysis.

As mentioned and encouraged in the guidance, the SAP can be revised if needed based on blinded information. Sometimes this is an inconvenience; however, this inconvenience is small compared to the advantages listed above.

At Cato Research, we will strive to have our SAPs completed before or soon after the enrollment of the first subject in a study.

This is a post by John Johnson, Ph.D. John is a Senior Biostatistician and the Associate Director, Statistics at Cato Research.