Monday, July 10, 2017

Age Group in Pediatric, Perinatal or Preterm, and Geriatric Subjects

In a previous blog article, I discussed the “Pediatric use and geriatric use of drug and biological products”:
For Pediatric population: according to ICH guidance E11 "Clinical Investigation of Medicinal Products in the Pediatric Population", the pediatric population contains several sub-categories:
  • preterm newborn infants
  • term newborn infants (0 to 27 days)
  • infants and toddlers (28 days to 23 months)
  • children (2 to 11 years)
  • adolescents (12 to 16-18 years (dependent on region))
Notice that in FDA's guidance "General Considerations for Pediatric Pharmacokinetic Studies
for Drugs and Biological Products
for Drugs and Biological Products", the age classification is a little bit different. I am assuming that the ICH guidance E11 should be the correct reference.
Geriatric population:
Geriatric population is defined as persons 65 years of age and older. There is no upper limit of age defined.
Recently, I run into several studies where the age group needs to be further split.

Pediatric population can be further divided into 2-5 years old, 6-11 years old, and 12-16 years old.
This grouping of the pediatric population was used in a pharmacokinetic study in primary immunodeficiency patients in child. Per FDA’s request, the pediatric patients are further divided into groups 2-5, 6-11, and 12-16 years old. FDA asked that the study should contain subjects in each of the sub-groups so that the assessment can be made to see if the pharmacokinetics are consistent across different pediatric sub-groups (or at least there should be no obvious difference among these sub-groups).
An Open-label, Single-sequence, Crossover Study to Evaluate the Pharmacokinetics, Safety and Tolerability of Subcutaneous GAMUNEX®-C in Pediatric Subjects With Primary Immunodeficiency
The age definition in perinatal period is much trickier. In a policy statement by Committee on Fetus and Newborn “Age Terminology Duringthe Perinatal Period”, various definitions of age are described:
  • Gestational age (or “menstrual age”) is the time elapsed between the first day of the last normal
  • menstrual period and the day of delivery
  • “Chronological age” (or “postnatal” age) is the time elapsed after birth
  • Postmenstrual age (PMA) is the time elapsed between the first day of the last menstrual period and birth (gestational age) plus the time elapsed after birth (chronological age). Postmenstrual age is usually described in number of weeks and is most frequently applied during the perinatal period beginning after the day of birth.
In clinical trials with pre-term babies, the study endpoints are usually defined based on the post-menstrual age (PMA). For example, in an article by Bassler et al “Early inhaled budesonide for the prevention ofbronchopulmonary dysplasia”, the primary outcome was a composite of death
or bronchopulmonary dysplasia at 36 weeks of postmenstrual age. While the study treatment will not be given until after the birth, 36 weeks of postmenstrual age is counted from the first day of the last menstrual period. For different babies, the chronological age or observation period (when the death or BPD event is observed) will be different depending on the actual gestational age.  

It can be illustrated in the following diagram.

  • Gestational age = birth date – the first day of the last normal menstrual period
  • Chronological age = assessment date or event date – birth date
  • Postmenstrual age = assessment date or event date – the first day of the last menstrual period
  • Postmenstrual age = gestational age + chronological age

Geriatric population can be further divided: 
In a study with Interstitial Lung Diseases (ILDs) in elderly patients, there are substantial number of patients with age >= 80 years old. The traditional definition of geriatric population using 65 years old as a cut point will not be sufficient. We end up further dividing the geriatric population into 65 - < 80 years old and >= 80 years old. We think this will provide more meaningful sub-grouping to assess the impact of the age group in ILD indication. 

Monday, July 03, 2017

(Bayes) Success Run Theorem for Sample Size Estimation in Medical Device Trial

In a recent discussion about the sample size requirement for a clinical trial in a medical device field, one of my colleagues recommended an approach of using “success run theorem” to estimate the sample size. ‘Success run theorem’ may also be called ‘Bayes success run theorem’. In process validation field, it is a typical method based on a binomial distribution that leads to a defined sample size.  

Application of success run theorem depends on the reliability of the new process (or new device). In medical device trials, the reliability is the probability that an item (i.e. the device) will carry out its function satisfactorily for the stated period when used according to the specified conditions. A reliability of 95% means that a medical device will be functional without problem for 95% of times.

With the success run theorem, we will calculate the sample size so that we have 95% confidence interval to run the device without failure (reliability). Usually, people use 95% confidence interval to achieve 95% reliability. With ‘success run theorem’, the sample size can be calculated as:

                                 N = ln(1-C)/ln( R)

Where N is the sample size needed, C is the confidence interval, and R is the reliability.

With typical 95% confidence interval to achieve 95% reliability, a sample size of 57 will be needed. An excel spreadsheet is built for calculating the sample size using success run theorem.   

The website below contains the explanation how the success run theorem formula is derived. With C = 1 – R^(n+1), we would have N = [ln(1-C)/ln(R)] – 1, slightly different from the formula above.
 How do you derive the Success-Run Theorem from the traditional form of Bayes Theorem?
This derivation above is based on uniform prior for reliability (a conservative assumption) which assumes no information from predicate devices and the same weight to every reliability value to fall anywhere between 0 to 1.

In medical device field, devices evolve and they are constantly being improved. When we evaluate a new device or next generation device, there is usually some prior information that can be based on. Therefore, instead of uniform prior for reliability, Bayesian technique with mixture of beta priors for reliability can be applied. Using mixtures of beta priors for reliability, we will be able to incorporate historical information from predicate device to decrease the sample size requirement.

We have seen this application in the field of automotive electronics attribute testing, but have not seen any application in FDA regulatory medical device testing.


Saturday, June 17, 2017

Statistical Analysis Plan in Clinical Trial Registries

Recently, a question comes up when I search the – the EU clinical trial registry website – the counterpart of in US. Should the clinical trial registries include the statistical analysis plan (for primary and secondary efficacy endpoints)? The statistical analysis plan could include the statistical methods for primary and secondary endpoints, missing data handling, stopping rule for early termination of the study, justification for sample size estimation, and so on.
For in US, Protocol Registration Data Element Definitions for Interventional and Observational Studies requires the inclusion of some details about statistical analyses:
Detailed Description Definition:
Extended description of the protocol, including more technical information (as compared to the Brief Summary), if desired. Do not include the entire protocol; do not duplicate information recorded in other data elements, such as Eligibility Criteria or outcome measures. Limit: 32,000 characters. 
For Patient Registries: Also describe the applicable registry procedures and other quality factors (for example, third party certification, on-site audit). In particular, summarize any procedures implemented as part of the patient registry, including, but not limited to the following: 
  • Quality assurance plan that addresses data validation and registry procedures, including any plans for site monitoring and auditing.
  • Data checks to compare data entered into the registry against predefined rules for range or consistency with other data fields in the registry.
  • Source data verification to assess the accuracy, completeness, or representativeness of registry data by comparing the data to external data sources (for example, medical records, paper or electronic case report forms, or interactive voice response systems).
  • Data dictionary that contains detailed descriptions of each variable used by the registry, including the source of the variable, coding information if used (for example, World Health Organization Drug Dictionary, MedDRA), and normal ranges if relevant.
  • Standard Operating Procedures to address registry operations and analysis activities, such as patient recruitment, data collection, data management, data analysis, reporting for adverse events, and change management.
  • Sample size assessment to specify the number of participants or participant years necessary to demonstrate an effect.
  • Plan for missing data to address situations where variables are reported as missing, unavailable, non-reported, uninterpretable, or considered missing because of data inconsistency or out-of-range results.
  • Statistical analysis plan describing the analytical principles and statistical techniques to be employed in order to address the primary and secondary objectives, as specified in the study protocol or plan. 
In EU, the is mainly based on the EudraCT database. As part of the clinical trial application (similar to IND in US), the sponsor needs to provide the clinical trial protocol information to be entered into EudraCT database.

In the guidance “Detailed guidance on the European clinical trials database (EUDRACT Database)”, it asks for the information regarding the clinical trial design, but there is no mention of the statistical analysis plan.

As a matter of fact, all clinical trial registries across different countries are supposed to meet the requirements by International Clinical Trials Registry Platform (ICTRP) from World Health Organization. In the list of elements for WHO Trial Registration Data Set , there is no mention of statistical analysis plan as part of the registration elements.

No matter what, there seems to be different understanding about the details of the clinical trial to be posted in clinical trial registries. Some companies posted very detail information including how the clinical trial data would be analyzed. Other companies were very restraint and posted as little information as possible.

In terms of the elements regarding the statistical analyses, there are actual more studies in with some details than studies in even though the requirement regarding the inclusion of the statistical analysis plan is mentioned in, not in For example, in a study “A Multicenter, Randomized, Double-Blind, Phase 3 Study of Ramucirumab (IMC-1121B) Drug Product and Best Supportive Care (BSC) Versus Placebo and BSC as Second-Line Treatment in Patients With Hepatocellular Carcinoma Following First-Line Therapy With Sorafenib”, a lot of details about the statistical analyses are provided in the

When I try to see if the interim analysis and its corresponding boundary method are mentioned in, I can clearly see the inconsistencies across different trial sponsors.

Here are some studies that the interim analysis and boundary method are mentioned.
Here are some studies that the interim analysis is mentioned, but the boundary method is not.

Sunday, June 04, 2017

Calculating exact confidence interval for binomial proportion within each group using the Clopper-Pearson method

Clopper-Pearson confidence interval is commonly used in calculating the exact confidence interval for binomial proportion, incidence rate,... The confidence interval is calculated for a single group, therefore Clopper-Pearson method is not for calculating the confidence interval for the difference between two groups. 

In many oncology studies where there is no concurrent control group. For response rate, The exact confidence interval will be constructed (usually through Clopper-Pearson method) and then the lower limit of the 95% confidence interval is compared with the historical rate to determine if there is a treatment effect. 

Here are some examples that Clopper-Pearson method was used to calculate the exact confidence interval: 

Medical and statistical review for Venetoclax NDA:
"For the primary efficacy analyses, statistical significance was determined by a two-sided p value less than 0.05 (one-sided less than 0.025). The assessment of ORR was performed once 70 subjects in the main cohort completed the scheduled 36-week disease assessment, progressed prior to the 36-week disease assessment, discontinued study drug for any reason, or after all treated subjects discontinued venetoclax, whichever was earlier. The ORR for venetoclax was tested to reject the null hypothesis of 40%. If the null hypothesis is rejected and the ORR is higher than 40%, then venetoclax has been shown to have an ORR significantly higher than 40%. The ninety-five percent (95%) confidence interval for ORR was based on binomial distribution (Clopper-Pearson exact method). "
Motzer et al (2015) Nivolumab versus Everolimus in Advanced Renal-Cell Carcinoma
"If superiority with regard to the primary end point was demonstrated, a hierarchical statistical testing procedure was followed for the objective response rate (estimated along with the exact 95% confidence interval with the use of the Clopper–Pearson method)"
Foster et al (2015) Sofosbuvir and Velpatasvir for HCV Genotype 2 and 3 Infection
"Point estimates and two-sided 95% exact confidence intervals that are based on the Clopper–Pearson method are provided for rates of sustained virologic response for all treatment groups, as well as selected sub-groups."
Cicardi et al (2010) Icatibant, a New Bradykinin-Receptor Antagonist, in Hereditary Angioedema
"Fisher’s exact test, with 95% confidence intervals calculated for each group by means of the Clopper–Pearson method, was used to compare the percentage of patients with clinically significant relief of the index symptom at 4 hours after the start of the study drug. Two-sided 95% confidence intervals for the difference in proportions were calculated with the use of the Anderson–Hauck correction."
According to SAS manual, the Clopper-Pearson confidence interval is described as below:
The confidence interval using Clopper-Pearson method can be easily calculated with SAS Proc Freq procedure. Alternatively, it can also be calculated directly using the formula or using R function. 

Using Venetoclax NDA as an example, the primary efficacy endpoint ORR (overall response rate) is calculated as 85 / 107 = 79.4. 95% confidence interval can be calculated using Clopper-Pearson method as following: 

Using SAS Proc Freq:  
With proc freq, we should get 95% confidence interval of 70.5 – 88.6.

data test2;
  input orr $ count @@;
have 85
no 22

proc freq data=test2 order=data;
  weight count;
  tables orr/binomial(exact) alpha=0.05 ;

Using formula:

data test;
  input n n1 alpha;
  phat = n1/n;
  fvalue1 = finv( (alpha/2), 2*n1, 2*(n-n1));
  fvalue2 = finv( (1-alpha/2), 2*(n1+1), 2*(n-n1));
  pL =  (1+   ((n-n1+1)/(n1*fvalue1) ))**(-1);
  pU =  (1+   ((n-n1)/((n1+1)*fvalue2) ))**(-1);
107 85 0.05

proc print;


Using R: 
f1=qf(1-alpha/2, 2*n1, 2*(n-n1+1), lower.tail=FALSE)
f2=qf(alpha/2, 2*(n1+1), 2*(n-n1), lower.tail=FALSE)

Saturday, May 27, 2017

Clinical Trial with Insufficient Sample Size: under power or detecting a trend?

When planning for a clinical trial, an important step is to estimate the sample size (the of patients needed to detect the treatment difference) for the study. In calculating the sample size, it is conventional to have significant level set at 0.05 and statistical power set at 80% or above. Sometimes, we need to design a clinical trial with insufficient sample size. This occurs pretty often in early phase clinical trials, in investigator initiated trials (IITs), and in rare disease drug development process due to the constraints in resource, budget, and available patients who can participate in the study. We could design a study without formal sample size calculation and we would simply state that the sample size of xxx is from the clinical consideration even though we don’t what it means exactly ‘the clinical consideration’.

If there are biomarkers or surrogate endpoints and treatment effects for biomarkers and surrogate endpoints are easier to detect than the clinical endpoints, we could design a proof-of-concept study or early phase study using the biomarkers or surrogate endpoints. The sample size can be formally calculated based on the treatment effect in biomarkers or surrogate endpoints. For example, in solid tumor clinical trials, we could design a study with smaller sample size based on the effect in shrinking the tumor size. In studies of inhaled antibiotics in non-CF Bronchiectasis, the early phase study could use the sputum density of the bacteria count as the endpoint so that the smaller sample size is required to demonstrate the effect before the late stage study where the clinical meaningful endpoint such as exacerbations should be used.

We can run into the situation where there is no good or reliable biomarkers or surrogate endpoints and the clinical endpoint is the only one available. The endpoint for the early phase study and the late phase study is the same. In order to design an early phase study with smaller sample size, we will need to do one of the followings:
  • Increase the significant level (alpha level) to allow greater type I error. Instead of testing the hypothesis at the conventional alpha = 0.05, we can test the hypothesis at alpha = 0.10 or 0.20 – we would say that we are trying to detect a trend. 
  • Lower the statistical power to allow greater type II error – design an underpowered study.

While both approaches have been used in literature, I would prefer the approach with increasing the significant level to detect a trend. Intentionally designing an underpowered study seems to have the ethical concern.

Here are some examples that the clinical trial is to detect a trend using alpha = 0.20 (or one-sided alpha=0.10):

“…It was estimated that for the study to have 90% power to test the hypothesis at a one-sided 0.10 significance level, the per-protocol population would need to include 153 participants in each group. The failure rate was estimated with binomial proportion and 95% confidence intervals. One-sided 90% exact confidence intervals were used to estimate the difference in the failure rates between the two treatments, which is appropriate for a noninferiority study and which is consistent with the one-sided significance level of 0.10 that was used for the determination of the sample size. “

 “A three-outcome (promising, inconclusive, not promising), one-stage modified Simon optimal phase II clinical trial study design with an interim analysis was chosen so that there would be a 90% chance of detecting a tumor response rate of at least 20% when the true tumor response rate was at least 5% at a 0.10 significance level, deeming that a RECIST response rate of less than 20% would be of little clinical importance in ATC.”

 “Assuming a mPFS of 3.5 months for GP and 5.7 months for GV (HR=0.61), a sample size of 106 subjects (53 per group) provided 85% power to detect this difference, using a one-sided test at the 0.10 significance level.”

 “The primary null hypothesis was that CoQ10 reduces the mean ALSFRSr decline over 9 months by at least 20% compared to placebo—in short, that CoQ10 is “promising.” It was tested against the alternative that CoQ10 reduces the mean ALSFRSr decline by less than 20% over 9 months compared to placebo, at one-sided alpha = 0.10

Here are some studies with insufficient power (less than 80% power). Notice that these studies still have 70% power. I can't image people's reaction if we design a study with 50% power. 

"A detailed calculation of sample size was difficult, since few studies have evaluated medications intended to augment local osseous repair in periodontal therapy. However, in one study of a selective cyclooxygenase-2 inhibitor in periodontal therapy, a sample of 22 patients per group was sufficient for the study to have 70% power to detect a 1-mm difference between the groups in the gain in clinical attachment level and reduction in probing depth, with a type I error rate of 5%."
“We estimated that a sample size of 600 would provide at least 70% power to detect a 33% reduction in the rate of the composite of the following serious adverse fetal or neonatal outcomes”

“With the sample of 99 patients, the study would have 70% power at a two-sided significance level of 0.05” 

Thursday, May 04, 2017

Final Version of Protocol Template by FDA/NIH and TransCelerate

Previously, I discussed the protocol template for clinical trials. This week, FDA/NIH and TransCelerate simultaneously released the final version of the protocol template.

FDA/NIH's protocol template is intended for clinical investigators who are writing protocols for phase 2 and phase 3 NIH-funded studies requiring investigational new drug (IND) or investigational device exemption (IDE)  applications, but could also be helpful to other investigators conducting studies of medical products that are not regulated by FDA.

The final protocol template by TransCelerate is for industry-sponsored clinical trials for the licensure.

                            Word Version of Final Template

                            Common Protocol Template Core Template – Basic Word Edition
                            Common Protocol Template – Technology-Enabled Edition

REFERENCE: FDA, NIH & Industry Advance Templates forClinical Trial Protocols | RAPS

Monday, May 01, 2017

Betting on Death: Moral Dilemma

I recently re-read the book “What Money Can't Buy: The Moral Limits of Markets” by Michael J. Sandel. The example used in the book about the viatical industry and the moral dilemma associated with this make me think about the similar dilemma we are facing in the event-driven clinical trials where the event is unfortunate outcome (for example, morbidity  and mortality).
A viatical settlement (from the Latin "viaticum") is the sale of a policy owner's existing life insurance policy to a third party for more than its cash surrender value, but less than its net death benefit. Such a sale provides the policy owner with a lump sum. The third party becomes the new owner of the policy, pays the monthly premiums, and receives the full benefit of the policy when the insured dies.
"Viatical settlement" typically is the term used for a settlement involving an insured who is terminally or chronically ill.
The viatical industry started in the 1980s and 1990s, prompted by the AIDS epidemic. It consisted of a market in the life insurance policies of people with AIDS and others who had been diagnosed with a terminal illness. Here is how it worked: Suppose someone with a $100,000 life insurance policy is told by his doctor that he has only a year to live. And suppose he needs money now for medical care, or perhaps simply to live well in the short time he has remaining. An investor offers to buy the policy from the ailing person at a discount, say, $50,000, and takes over payment of the annual premiums. When the original policyholder dies, the investor collects the $100,000.
It seems like a good deal all around. The dying policyholder gains access to the cash he needs, and the investor turns a handsome profit – provided the person dies on schedule.
With viaticals, the financial risk creates a moral complication not present in most other investments: the investor must hope that the person whose life insurance he buys dies sooner rather than later. The longer the person hangs on, the lower the rate of return.
The anti-HIV drugs that extended the lives of tens of thousands of people with AIDS scrambled the calculations of the viatical industry.
The viatical industry can extend to people with other terminal diseases such as cancer. However the concept is the same and the moral issues are the same: betting the people to die sooner than later.

In clinical trials with event-driven design where the event is bad such as death, cancer recurrence, pulmonary exacerbation, transplantation rejection,…), we may face the same dilemma. While the intention of new treatment is to prevent the bad event from happening, as the trial sponsor, we also hope that these bad events can occur more often so that we can finish the study early and have the study results available earlier.

Suppose there is a cancer clinical trial where the primary efficacy endpoint is time to death and suppose we design a randomized, double-blind study to compare two treatment groups: an experimental treatment group and a control group, we will calculate the sample size to see how many death events are needed to have at least 80% statistical power to show the treatment difference. Then based on the accrual rate and dropout rate, we can further calculate the number of subjects needed to have the desired number of death events. During the study, we can check the aggregate death rate to see if the actual results are in line with the assumptions. If the death rate is below our assumptions, we should be happy since the lower death rate could indicate the experimental treatment works, however, as the trial sponsor, we would not be happy since the lower death rate will indicate the longer trial to accrue the requirement number of death events.

As the trial sponsor, we may want to employ the enrichment strategies to select the population who may be likely to die therefore the death events can be accrued quickly. As mentioned in FDA’s guidance “Enrichment Strategies for Clinical Trials to Support Approval of Human Drugs and Biological Products”, this type of enrichment strategy is called prognostic enrichment. Here is what is said in FDA’s guidance:
A wide variety of prognostic indicators have been used to identify patients with a greater likelihood of having the event (or a large change in a continuous measure) of interest in a trial. These indications include clinical and laboratory measures, medical history, and genomic or proteomic measures. Selecting such patients allows a treatment effect to be more readily discerned. For example, trials of prevention strategies (reducing the rate of death or other serious event) in cardiovascular (CV) disease are generally more successful if the patients enrolled have a high event rate, which will increase the power of a study to detect any given level of risk reduction. Similarly, identification of patients at high risk of a particular tumor, or at high risk of recurrence or metastatic disease can increase the power of a study to detect an effect of a cancer treatment. Prognostic enrichment strategies are also applicable, or potentially applicable, to the study of drugs intended to delay progression of a variety of diseases, such as Alzheimer’s disease, Parkinson’s disease, rheumatoid arthritis, multiple sclerosis, and other conditions, where patients with more rapid progression could be selected; it is possible, of course, that such patients might be less responsive to treatment (i.e., that rapid progression would be a negative predictor of response), and that would have to be considered.  
For any given desired power in an event-based study, the appropriate sample size will depend on effect size and the event rate in the placebo group. Prognostic enrichment does not increase the relative risk reduction (e.g., percent of responders or percent improvement in a symptom), but will increase the absolute effect size, generally allowing for a smaller sample size. For example, reduction of mortality from 10% to 5% in a high-risk population is the same relative effect as a reduction from 1% to 0.5% in a lower risk population, but a smaller sample size would be needed to show a 5% vs. 0.5% change in absolute risk. It is common to choose patients at high risk for events for the initial outcome study of a drug and, if successful, move on to larger studies in lower risk patients.

While this enrichment strategy is good for the trial sponsor and makes the clinical trial smaller, it also gives a bad taste because we are betting that the selected study population will have high death rate and that the patient die sooner.

Friday, April 28, 2017


Last week, I attended the 10th Annual Conference on Statistical Issues in Clinical Trials held in the camps of the University of Pennsylvania. This year, the topic is about "Current Issues Regarding Data and Safety Monitoring Committees in Clinical Trials".

The conference is supposed to discuss the current issues and emerging challenges in the practice of data monitoring committees, however, these issues and challenges discussed are not new. These issues are known for long time and remain as issues.

One thing I like this one-day annual conference is that all presentation slides including the panel discussions are posted online. For DMC discussions this year, the presentation slides are available at:

The early days, the term DSMB (data and safety monitoring board) was used. After FDA issued its guidance "The Establishment and Operation of Clinical Trial Data Monitoring Committees for Clinical Trial Sponsors", the term DMC (data monitoring committee) become popular. In the conference this year, a new term DSMC (data and safety monitoring committee) is also used.

There are several talks about the training for DMC members and the need to train more people who can serve on the data monitoring committee. I felt that the discussion about the training for DMC members is targeting the wrong audience. Instead of targeting the statisticians, the focus should be on those MDs in the medical field. Very often, we have difficulties to find the MDs who can serve on the data monitoring committee, let alone to fine the MDs who have prior experience serving in the data monitoring committee. For a large scale clinical trial, there may be several committees: steering committee, event or endpoint adjudication committee, and data monitoring committee. There is often a shortage of members for these committees. 

Nowadays, the study protocols are getting more and more complicated. The DMC committee members may not understand the complexity of the complicated trial design.  This is especially true for clinical trials using adaptive design and Bayesian design. This poses challenges to the DMC members.

We used to debate whether or not the data monitoring committee should be given the semi-unblinded materials for review where the treatment group is designated as "X" or "Y" instead of the true treatment assignment. The conference presentations said loudly that DMC should have sole access to interim results with complete unblinding (not semi-unblinding) on relative efficacy & relative safety of interventions.

Here are some points from Dr Fleming’s presentation “DMCs:Pomoting Best Practices to Address Emerging Challenges
Mission of the DMC
  • To Safeguard the Interests of the Study Participants
  • To Preserve Trial Integrity and Credibility to enable the clinical trial to provide timely and reliable insights to the broader clinical community
 Proposed Best Practices and Operating Principles
  • Achieving adequate training/experience in DMC process
  • Indemnification
  • Addressing confidentiality issues
  • Implementing procedures to enhance DMC independence
                DMC meeting format
                Creating an effective DMC Charter
                DMC recommendations through consensus, not by voting
                DMC contracting process
  • Defining the role of the Statistical Data Analysis Center
  • Better integration of regulatory authorities in DMC process


Friday, April 07, 2017

Estimate the number of subjects needed to detect at least one event and the rule of three

It is sometimes important to know how likely a study is to detect a relatively uncommon event, particularly if that event is severe, such as intracranial hemorrhage (ICH), progressive multifocal leukoencephalopathy (PML), bone marrow failure, or life-threatening arrhythmia. A great many people must be observed in order to have a good chance to detecting even one such event, much less to establish a relatively stable estimate of its frequency. For most clinical trials, sample size is planned to be sufficient to detect main effects in efficacy and sample size is likely to be well short of the number needed to detect these rare events. Sometimes, we do see the request from FDA for a sample size large enough to detect at least one rare event.

Cempra is a pharmaceutical company in Chapel Hill, North Carolina with a very promising antibiotics drug. However, their NDA submission was not approved by FDA, not due to the efficacy, but due to the safety issue (precisely, due to FDA’s concern about the potential liver toxicity). In FDA’s complete response letter (CRL, in other words, rejection letter), it says:
“To address this deficiency, the FDA is recommending a comparative study to evaluate the safety of solithromycin in patients with CABP. Specifically, the CRL recommends that Cempra consider a study of approximately 9,000 patients exposed to solithromycin to enable exclusion of serious drug induced liver injury (DILI) events occurring at a rate of approximately 1:3000 with a 95 percent probability.”
The request for a study with these many subjects is unusual for a pre-market study. It will certainly not be feasible for an antibacterial drug in community-acquired bacterial pneumonia indication. It is interesting to see that the FDA applied the ‘rule of three’ in proposing the sample size.

The rule of three says: to have a good chance of detecting a 1/x events, one must observe 3x people. For example, to detect at least one event if the underlying rate is 1/1,000, one would need to observe 3,000 people. In Solithromycin case, to detect at least one DILI event if the background rate of DILI event is 1/3,000, the number of subjects to be observed would be 3 x 3,000 = 9,000 subjects.

I had previously written an articleabout the rule of three from the angle of calculating the 95% confidence interval when there is no event occurred in x number of subjects. The same rule can be applied to the situation for estimating the sample size to detect at least one event.    

In one of our studies with a thrombolytic agent, FDA is concerned about the potential side effect of intracranial hemorrhage (ICH). In FDA’s comment on IND, FDA asked us to have a safety database containing at least 560 subjects exposed to the tested thrombolytic agent. They stated:
In order to permit the occurrence of 1 ICH event without necessitating halting the study, the safety database would have to have at least 560 subjects exposed to your XYZ drug with only one ICH event to keep the ICH rate under 1%. Please comment.
In this case, the background rate of ICH is assumed to be 1%. Based on the rule of three, the safety database would be 3 x 100 = 300. However, the number of 560 is coming from the direct calculation assuming a binomial distribution.

Based on an ICH rate of 1%, assuming a binomial distribution, to detect at least one ICH event, 560 subjects will be needed in order to keep upper bound of the exact two-sided 95% confidence interval below 1%.

A small SAS program below can be used for the calculation. Adjust the sample size and then see what the exact 95% confidence intervals are.

 data test;
  input ICH $ count;
  Have 1
  No   560

proc freq data=stopping;
  weight count;
  tables ICH /  binomial (p=0.01) alpha=0.05 cl;
           *p=0.01 indicates the background rate;
  exact binomial*Obtain the exact p-value;

In clinical studies with potential side effect of rare events, if we need to base the safety database on detecting at least one such event, the sample size could be very large. As we see from the calculation or simply apply the rule of three, the sample size largely depends on the assumed background event rate. It is usually the case that the background event rate is from the literature. The literature usually have variety of the results. Using DILI as an example, the background rate from the literature could vary depending on if we talk about the rate in normal population, in community-acquired bacterial pneumonia patients, or in community-acquired bacterial pneumonia patients treated with other antibiotics. 


Wednesday, March 22, 2017

Regenerative (Medicine) Advanced Therapy: from RAT to ART to RMAT

In the United States, the government can promote or incentivize the drug development in certain areas through its policies. In FDA, there are different kind of designations, each providing specific benefits or incentives for speeding up the drug development in specific areas. Here are some of the designations/vouchers,… all with purpose to stimulate the drug development in certain areas.

For granting special status to a drug or biological product (“drug”) to treat a rare disease or condition upon request of a sponsor. This status is referred to as orphan designation (or sometimes “orphan status”). Orphan designation qualifies the sponsor of the drug for various development incentives of the Orphan Drug Act, including tax credits for qualified clinical testing. A marketing application for a prescription drug product that has received orphan designation is not subject to a prescription drug user fee unless the application includes an indication for other than the rare disease or condition for which the drug was designated.
Fast track is a process designed to facilitate the development, and expedite the review of drugs to treat serious conditions and fill an unmet medical need.

A process designed to expedite the development and review of drugs which may demonstrate substantial improvement over available therapy.

These regulations allowed drugs for serious conditions that filled an unmet medical need to be approved based on a surrogate endpoint.

A Priority Review designation means FDA’s goal is to take action on an application within 6 months.

Under Section 529 to the Federal Food, Drug, and Cosmetic Act (FD&C Act), FDA will award priority review vouchers to sponsors of rare pediatric disease product applications that meet certain criteria. Under this program, a sponsor who receives an approval for a drug or biologic for a "rare pediatric disease" may qualify for a voucher that can be redeemed to receive a priority review of a subsequent marketing application for a different product.
Authorizes the FDA to award priority review vouchers to sponsors of certain tropical disease product applications that meet the criteria specified in that section

With incentives to help bring new antimicrobials to market. Antimicrobials or antibiotics drugs can be approved after being designated as a Qualified Infectious Disease Product (QIDP) under the GAIN Act. As part of this QIDP designation, FDA’s review of the drug application is expedited. The designation also qualify the drugs for five years of marketing exclusivity to be added to certain exclusivity already provided by the FDA.

With 21st Century Cure Act passed last December, we now have a new designation – Regenerative Advanced Therapy. It is interesting that Regenerative Advanced Therapy can be abbreviated as RAT – which is not a good name.In one of the webinars, Dr Frank Sasinowski questioned the term and suggested the use of Advanced Regenerative Therapy (ART). Nobody wants to have their innovative therapy being labelled as RAT, everybody wants to have their innovative therapy being labelled as ART.

Yesterday, FDA issued its first Regenerative Advanced Therapy designation to Humacyte - a company in Research Triangle Park, North Carolina for their tissue engineered vessel or human acellular vessel. In its issuance, neither RAT nor ART is used by FDA. Instead, the term is called RMAT (Regenerative Medicine Advanced Therapy). I guess that FDA realizes the term RAT is not a good one. 

With RMAT designation, the sponsor will have the breakthrough and priority review benefits as well as one aspect of fast track (rolling submission). In addition, with RMAT designation, the sponsor will have an opportunity for accelerated approval on basis of invalidated surrogate endpoint and the sponsor will have reduced phase 4 requirements if the product is approved. 

Below are some specific sections from 21st Century Cure Act regarding the Regenerative Advanced Therapy designation:

 (g) Regenerative Advanced Therapy.—
 “(1) IN GENERAL.—The Secretary, at the request of the sponsor of a drug, shall facilitate an efficient development program for, and expedite review of, such drug if the drug qualifies as a regenerative advanced therapy under the criteria described in paragraph (2).
 “(2) CRITERIA.—A drug is eligible for designation as a regenerative advanced therapy under this subsection if— 
“(A) the drug is a regenerative medicine therapy (as defined in paragraph (8)); 
“(B) the drug is intended to treat, modify, reverse, or cure a serious or life-threatening disease or condition; and 
“(C) preliminary clinical evidence indicates that the drug has the potential to address unmet medical needs for such a disease or condition.
 “(5) ACTIONS.—The sponsor of a regenerative advanced therapy shall be eligible for the actions to expedite development and review of such therapy under subsection (a)(3)(B), including early interactions to discuss any potential surrogate or intermediate endpoint to be used to support the accelerated approval of an application for the product under subsection (c).
 “(6) ACCESS TO EXPEDITED APPROVAL PATHWAYS.—An application for a regenerative advanced therapy under section 505(b)(1) of this Act or section 351(a) of the Public Health Service Act may be— 
“(A) eligible for priority review, as described in the Manual of Policies and Procedures of the Food and Drug Administration and goals identified in the letters described in section 101(b) of the Prescription Drug User Fee Amendments of 2012; and 
“(B) eligible for accelerated approval under subsection (c), as agreed upon pursuant to subsection (a)(3)(B), through, as appropriate— “(i) surrogate or intermediate endpoints reasonably likely to predict long-term clinical benefit; or “(ii) reliance upon data obtained from a meaningful number of sites, including through expansion to additional sites, as appropriate. 
“(7) POSTAPPROVAL REQUIREMENTS.—The sponsor of a regenerative advanced therapy that is granted accelerated approval and is subject to the postapproval requirements under subsection (c) may, as appropriate, fulfill such requirements, as the Secretary may require, through— 
“(A) the submission of clinical evidence, clinical studies, patient registries, or other sources of real world evidence, such as electronic health records; 
“(B) the collection of larger confirmatory data sets, as agreed upon pursuant to subsection (a)(3)(B); or 
“(C) postapproval monitoring of all patients treated with such therapy prior to approval of the therapy.
 “(8) DEFINITION.—For purposes of this section, the term ‘regenerative medicine therapy’ includes cell therapy, therapeutic tissue engineering products, human cell and tissue products, and combination products using any such therapies or products, except for those regulated solely under section 361 of the Public Health Service Act and part 1271 of title 21, Code of Federal Regulations.”.    
SEC. 3034. GUIDANCE REGARDING DEVICES USED IN THE RECOVERY, ISOLATION, OR DELIVERY OF REGENERATIVE ADVANCED THERAPIES.(a) Draft Guidance.—Not later than 1 year after the date of enactment of the 21st Century Cures Act, the Secretary of Health and Human Services, acting through the Commissioner of Food and Drugs, shall issue draft guidance clarifying how, in the context of regenerative advanced therapies, the Secretary will evaluate devices used in the recovery, isolation, or delivery of regenerative advanced therapies. In doing so, the Secretary shall specifically address— 
(1) how the Food and Drug Administration intends to simplify and streamline regulatory requirements for combination device and cell or tissue products; 
(2) what, if any, intended uses or specific attributes would result in a device used with a regenerative therapy product to be classified as a class III device; 
(3) when the Food and Drug Administration considers it is necessary, if ever, for the intended use of a device to be limited to a specific intended use with only one particular type of cell; and 
(4) application of the least burdensome approach to demonstrate how a device may be used with more than one cell type. (b) Final Guidance.—Not later than 12 months after the close of the period for public comment on the draft guidance under subsection (a), the Secretary of Health and Human Services shall finalize such guidance.  
SEC. 3035. REPORT ON REGENERATIVE ADVANCED THERAPIES.(a) Report To Congress.—Before March 1 of each calendar year, the Secretary of Health and Human Services shall, with respect to the previous calendar year, submit a report to the Committee on Health, Education, Labor, and Pensions of the Senate and the Committee on Energy and Commerce of the House of Representatives on— 
(1) the number and type of applications for approval of regenerative advanced therapies filed, approved or licensed as applicable, withdrawn, or denied; and 
(2) how many of such applications or therapies, as applicable, were granted accelerated approval or priority review. (b) Regenerative Advanced Therapy.—In this section, the term “regenerative advanced therapy” has the meaning given such term in section 506(g) of the Federal Food, Drug, and Cosmetic Act, as added by section 3033 of this Act.  
Subchapter A of chapter V of the Federal Food, Drug, and Cosmetic Act (21 U.S.C. 351 et seq.) is amended by inserting after section 506F the following: 
“SEC. 506G. STANDARDS FOR REGENERATIVE MEDICINE AND REGENERATIVE ADVANCED THERAPIES. “(a) In General.—Not later than 2 years after the date of enactment of the 21st Century Cures Act, the Secretary, in consultation with the National Institute of Standards and Technology and stakeholders (including regenerative medicine and advanced therapies manufacturers and clinical trial sponsors, contract manufacturers, academic institutions, practicing clinicians, regenerative medicine and advanced therapies industry organizations, and standard setting organizations), shall facilitate an effort to coordinate and prioritize the development of standards and consensus definition of terms, through a public process, to support, through regulatory predictability, the development, evaluation, and review of regenerative medicine therapies and regenerative advanced therapies, including with respect to the manufacturing processes and controls of such products. 
“(b) Activities.— 
“(1) IN GENERAL.—In carrying out this section, the Secretary shall continue to— 
“(A) identity opportunities to help advance the development of regenerative medicine therapies and regenerative advanced therapies; 
“(B) identify opportunities for the development of laboratory regulatory science research and documentary standards that the Secretary determines would help support the development, evaluation, and review of regenerative medicine therapies and regenerative advanced therapies through regulatory predictability; and 
“(C) work with stakeholders, such as those described in subsection (a), as appropriate, in the development of such standards. 
“(2) REGULATIONS AND GUIDANCE.—Not later than 1 year after the development of standards as described in subsection (a), the Secretary shall review relevant regulations and guidance and, through a public process, update such regulations and guidance as the Secretary determines appropriate. “(c) Definitions.—For purposes of this section, the terms ‘regenerative medicine therapy’ and ‘regenerative advanced therapy’ have the meanings given such terms in section 506(g).”.