Aug 27 2010

Five Keys to Successful Adaptive Trials

Running adaptive trials is not easy. However, in a world of skyrocketing drug development costs, these trials are set to become essential tools in any drugmaker’s repertoire, especially early in clinical development. A poorly done adaptive trial can not only result in trial failure (i.e. failure of the study to produce useful information about the treatment studied), but can make it less likely for a sponsor to run a useful adaptive trial in the future. Here are some keys to adaptive trial success:

1. Be aware of the up front planning required

For most adaptive trials a lot of planning is required. The project statistician will probably be very busy during this time, and so will the best of the project team. Because trial adaptations need to be prespecified in the protocol, the range of possibilities–and the consequences of each–need to be considered. The statistician should be running many clinical trial simulations, even if this is a “textbook” adaptive trial. These simulations include not only the “ideal” situations, but also what happens when thing go wrong. For example, does the trial design work if the dropout rate is higher? What if dropout is related to safety or lack of efficacy? What happens if the treatment causes harm (does worse than control)? Can it stop more quickly?

The project team should be busy as well. The clinical, data management, chemistry, manufacturing, and controls (CMC), and statistical teams should have a “dry run” of interim analysis procedures to anticipate issues. Write down these procedures so that when turnover in staff happens, new staff can be trained on the documents. Be aware that the timelines associated with adaptive trials are usually very tight because it is desirable to run interim analyses on clean data, and this tends to magnify any inefficiencies of the project team, and good planning will expose the worst potential ones. The team can then plan to work around them or resolve them.

2. Consider Bayesian methods

Bayesian methods are well explained in a summary by Donald Berry. These methods enable the flexibility to consider many more kinds of adaptive trials than classical methods. The major downside is that they are more complex to design and execute (because of their flexibility), but the additional operational cost of a Bayesian trial is usually small compared to the savings afforded by enrolling fewer patients and making better decisions.

3. Avoid adaptive trials with one interim analysis to stop for efficacy

These trials are effectively fixed sample size trials requiring more subjects than classical randomized control trials (with a rare exception stopping at the interim analysis) and the added complexity of planning and executing an adaptive trial. If a trial with one interim analysis is required, I usually recommend one that stops for futility only. Be aware that the interim analyses are not going to give the sponsor a lot of information, especially in a pivotal trial where the results of the analysis have to be limited to a few people independent of other operation of the study until the final analysis is performed.

4. Get buy-in

Buy-in from all key stakeholders is very important. If management is not committed to running the adaptive trial, this could sabotage the process. The clinical, regulatory, data management, site staff, and statistical teams all need to be aware of their roles in the process, as well as the tight timelines that are necessary to produce correct and timely decisions. The sponsor must be willing to use the recent advances, such as electronic data capture (EDC) and centralized randomization usually with interactive voice/web response system (IVRS/IWRS), and IVRS/IWRS for patient reported outcomes.

5. Believe the results

Not wanting to believe the results of an interim analysis (especially early stopping for futility) is so endemic that a theory has been created around “non-binding” interim analyses. This is a shame. In most cases I’ve been in where interim analysis results were intentionally ignored, later results confirmed the earlier analyses. It is possible to create adaptive designs that will simply change the randomization allocation during the first part of the trial (for example to randomize subjects more toward interesting doses) and not stop, with the ability to stop later in the trial than it is to set up stopping rules that are eventually ignored.

Bonus. Know when not to use adaptive trials

Adaptive trials are not always appropriate. Overly fast accrual rates will make it difficult to produce timely analyses. Management unwilling to commit to the process will often cancel out the advantages. Adapting an existing non-adaptive trial to address operational issues (such as slow enrollment) is likely to produce biased results.

In conclusion, adaptive trials, when designed and run well, are can lower costs and enable faster and better decision-making in a product development program. Following the keys above will help make your adaptive trial useful.

This is a post by John Johnson, Ph.D. John is a Senior Biostatistician and the Associate Director, Statistics at Cato Research.