Analysis of FDA Guidance on Adaptive Designs

In February 2010, the FDA published a draft guidance on Adaptive Design Clinical Trials for Drugs and Biologics (“the guidance”). This highly anticipated guidance has come out in draft form 4 years after then FDA deputy commissioner Dr. Scott Gottlieb’s famous keynote speech on the topic accelerated interest on the topic. In the intervening years, it seems there has been a huge ambivalence among smaller sponsors to try these kinds of designs, mostly because of regulatory challenges or perceived regulatory resistance to adaptive trials. Hopefully this guidance, when finalized, will allay many of the fears of using these trials and accelerate their adoption. Even before it is finalized, it will serve as a useful tool for sponsors to use in identifying issues of concern to regulators in the planning and execution of adaptive trials.

Adaptive trials have the potential, when planned and executed well, to reduce time, costs, and number of research participants exposed to an unproven treatment. Variations involving adaptive randomization have the potential to enable the building of a high-quality safety database at relevant doses with fewer overall participants. In the cases where a treatment is not effective, a well-planned and executed adaptive trial has the potential to demonstrate failure more quickly. It turns out that regulatory risk is one of the key factors I have seen slowing down the adoption of these trials despite the high interest both from industry and FDA in adaptive trials. In fact, in more than one instance I have seen sponsors abandon an adaptive trial strategy due to regulatory feedback or appearance of high regulatory burden.

Because this 50 page document contains a lot of detail covering a wide range of topics, commentary will be split up into several sections. Also, for purposes of saving on space, any time “treatment” is used it may refer to a small-molecule compound, protein, biologic, radioactive compound, or any other entity undergoing the drug development process.

According to the guidance, an exploratory study is simply any study that does not rigorously control the Type I error rate (ie. rate of falsely rejecting the null hypothesis). In a “learn and confirm” paradigm, these are the studies where the learning about the major efficacy qualities of the treatment take place. Alternatively, they may rigorously control the Type I error rate on a weak endpoint such as a biomarker.

Clearly in exploratory studies there is a lot of room for creativity, and well-planned and executed adaptive trials are especially suited for these kinds of trials.

For dose-ranging and selection, adaptive trials offer great flexibility including situations involving safety factors, inverse-U-shaped dose-response curves, or the potential for a flat dose-response curve (ie. a treatment failure). The widely-publicized ASTIN trial, while employing a strategy that seems exotic by current standards, shows what is possible, and it is noteworthy how quickly the trial demonstrated lack of efficacy despite the flexibility of the methodology.

For treatments where very little is known about efficacy, adaptive trials afford the opportunity to measure essential properties of several different endpoints, the treatment effect on each, and the uncertainty due to population variation. These measures can be successively refined until enough information is obtained to finalize the design of an adequate and well-controlled study. For example, it is possible to adapt different doses and different endpoints.

An adequate and well-controlled (A&WC) study, also known as a pivotal or confirmatory trial, is the subject of much of the guidance. Though adaptive trials tend to show their strength in exploratory trials, they can be effective in A&WC trials as well. Effective management of the interim analysis process, including the restriction of unblinded data and communication between unblinded personnel and other study staff, is essential to the successful execution of an adaptive A&WC trial. In short, the FDA has identified 3 major concerns:

1. Bias from multiple testing. Statisticians are aware of this issue and therefore should be involved early in the planning of any trial using adaptive methods.
2. Difficulty in interpretation when treatment effect is used in the adaptation. This occurs especially when many different doses are used at the start and doses are dropped during the conduct of the study. This is known as selection bias and needs to be handled carefully at the design stage because it is nearly impossible to correct at the end of the study.
3. Operational bias. This simply means that if study personnel, including the investigator’s staff and sponsor, are aware of interim results adjustments could be made that affect the outcome of the trial in an unknown way. Therefore, strong firewalls between unblinded staff and the rest of the study staff need to be built into the protocol and data monitoring committee charter.

These concerns must be addressed up front, usually in the protocol, statistical analysis plan, or both. In many cases the FDA will be willing to grant a special protocol assessment for an A&WC study using these methods, so there are plenty of opportunities to show the agency that these concerns are being addressed.

The FDA is clear in its draft guidance that it believes that time for reflection on study results is invaluable in a drug development program. This often runs counter to sponsor desires, especially using adaptive designs to eliminate lag between Phase 2 and Phase 3 segments of development (“seamless” designs). For example, a rapid drug development program, according to the FDA, might miss hard-to-discover safety issues such as interactions with concomitant therapies, subset differences in treatment effect, or compliance. It is essential to keep these issues in mind when design an adaptive trial and discussing it with the FDA, especially when using seamless Phase 2/3 trials.

Adaptive trials can be a powerful tool in lowering drug development cost and time to market. Like fire, they can have huge benefits when used carefully and correctly, or they can create a dangerous situation if used carelessly. The 2010 draft guidance, though not in final form, should help sponsors understand the issues better and reduce the risk–both regulatory and business–when using adaptive trials in their drug development program.

This is a post by John Johnson, Ph.D. John is a Senior Biostatistician and the Associate Director, Statistics at Cato Research.