In March 2010, the FDA issued a draft guidance on the use of noninferiority (NI) trials. While not in final form, this guidance should help sponsors understand the issues the FDA considers when reviewing NI trials.
In contrast to statements over the last 8 years on the matter, this guidance shows the kinds of arguments FDA wants to see when sponsors justify their NI trials, without being too prescriptive in how those trials are to be designed. While the arguments at first glance seem complex, they are the kinds of arguments they expected to see all along. They even show a couple of examples. Now we know.
Importance of this draft guidance on NI trials
In the industry we have been without a guideline for NI trials for several years, and we have had to justify each noninferiority trial on a case-by-case basis. While most design elements have been easy to justify, the NI margin, or maximum allowable “slippage in efficacy” for the new treatment, had to be argued statistically and clinically anew for each case. Correspondence on these trials sometimes even drew out over the course of months, leading to prolonged delays in study startup. Even personnel experienced in NI trials were often frustrated by the large variation in regulator responses across divisions.
The history of regulatory positions on NI trials gives an insight into this frustration. In 1992, a Points to Consider document was offered to industry which gave specific guidance on noninferiority margins. In 2003, after an anti-infectives advisory committee meeting, the document was withdrawn. The advisory committee met again on the topic in 2006. In October 2007, a 3-page draft guidance for NI trials for antibacterial products was published, but did not seem to come out of draft phase. This guidance basically asked sponsors to justify their study designs on a case-by-case basis or even consider alternatives to an NI trial. Furthermore, it asked sponsors to reconsider ongoing studies, including studies already running under a special protocol agreement.
During this time, the antibiotic under the trade name Ketek was reported to have serious safety side effects. The FDA was scrutinized heavily over its approval of the drug. In response , they noted that:
Noninferiority studies were considered acceptable as the basis for approval. Concurrent with the Ketek review, our thinking on noninferiority studies was evolving. Today, noninferiority studies are no longer considered acceptable for two of the three indications for which Ketek was originally approved. We are applying this new regulatory position to more recently submitted and planned applications.
At the same time, industry is facing pressure to run more “head-to-head” or comparative effectiveness trials, where two active compounds compete. In some therapeutic areas (for example in myocardial infarction), placebo-controlled trials are considered unethical. With independent institutes running comparative effectiveness trials, some sponsors will want to get ahead of the curve. In development programs, some sponsors may want to get preliminary comparative efficacy data against a standard of care. Modern antibacterial development still depends heavily on the use of NI trials, despite the FDA’s attempt to push sponsors away.
Now that we have a detailed guideline of the kinds of data and arguments the agency wants to see, sponsors can structure their protocols and analysis plans to meet agency expectations rather than waste time fumbling in the dark for literature references on how to justify a design for an NI trial.
What to remember when designing and running a NI trial
When an NI trial is prescribed, whether it be for comparative effectiveness studies or ethical reasons, there are several important details beyond the conventional superiority-to-placebo study that must be considered at the design stage. Failure to consider these details carefully may result in a trial whose outcome is impossible to interpret, which wastes time and money and puts patients’ lives at risk without commensurate advance in knowledge.
The NI margin
The NI margin is the size of the difference in treatment effect we are willing to tolerate before declaring the experimental treatment not effective. For example, if we decide on a NI margin of 10%, then the 95% lower confidence limit of the difference between control and treatment must not be greater than -10%. Another way to think about this is that the NI margin is the largest amount of “slippage in efficacy” we can tolerate.
Obviously, this NI margin must be justified. Before it was withdrawn, sponsors could point to a “Points to Consider” document which gave specific guidance. This guidance is not specific, and it looks like the FDA is no longer willing to give specific guidance on the NI margin. However, Section III.A.4 gives a very good explanation on what the margin means, and the combined statistical and clinical arguments that must be made to justify it.
The statistical argument comes from defining the effect of the active control (called M 1 in the guidance). If the NI margin were to be set higher than M1, the whole trial would be meaningless because a successful result could not guarantee that the experimental treatment is any better than placebo.
The clinical argument will more than likely mean that the final NI margin is smaller than M1 (It is assumed that no clinical argument can justify a larger margin, for then no treatment effect can be proven.). The clinical argument can consider any improvements in the safety profile of the experimental treatment, especially if it a drug in a new class in a therapeutic area where drugs in new classes are needed or if the safety profile is known to be better than the active control.
If historical evidence is used to determine M1, the guidance poses many issues to consider, such as the “constancy assumption” (that a drug’s effect is stable over time), the quality of the historical evidence, and variation in previous effect estimates. While the guidance doesn’t offer a specific prescription in dealing with these issues (that would probably be impossible), there is a good sampling of the regulatory challenges any argument for M1 based on historical evidence is likely to face.
In addition, the guidance describes and shows an example of a “synthesis” strategy where evidence from the current trial is directly combined with evidence from previous trials to demonstrate superiority to placebo. This level of guidance will be a big help in designing studies and crafting protocols.
Other issues
The guidance covers many other issues to consider in the design of a noninferiority trial, such as the methods of statistical analysis. The two main methods are a comparison to active control minus the NI margin, and a synthesis method, where the effect of experimental treatment over placebo is inferred.
Alternatives
I have seen cases where the regulatory authority shot down a noninferiority trial and proposed some vague alternatives. As regulatory consultants, we have found this frustrating. Fortunately, this guidance has elaborated on what the FDA considers in making these recommendations and some details on those alternatives:
- Add on therapy strategy where the experimental treatment or placebo is given in addition to the standard treatment.
- Running a superiority trial in a refractory patient population (i.e. one already shown not to respond to standard of care)
- Employing some rescue, early escape, or randomized withdrawal trial. Each of these types has a particular use, and are good when it is undesirable to leave subjects on a placebo for an extended period of time.
Conclusions
This guidance is 66 pages long, and is very thorough in identifying the issues for sponsors to consider in the design of NI trials. It should reduce risk and frustration when proposing NI trials to the agency, or wasted trial when reporting the results to the agency.
