Estimating the average treatment effects at two different visits and its implementation using MMRM model

Orkambi is a combination drug including Ivacaftor and Lumacaftor and is approved by FDA for the treatment of cystic fibrosis in patients less than 12 years with homozygous F508del mutation. The approval was based on two large pivotal studies with identical study design. The treatment effect is trivial, but statistically significant. One thing that is interesting to me is that the primary efficacy endpoint and the key secondary efficacy endpoint are based on the average of two different visits (visits 16 and 24).

• The primary endpoint was absolute change in ppFEV1 from baseline at week 24 (assessed as the average treatment effects at week 16 and 24).
• Average relative change from baseline in ppFEV1 at Week 16 and at Week 24

Typically, for a clinical trial with fixed treatment duration, the treatment effect will be assessed at one specified time point. In Orkambi pivotal studies, the primary efficacy endpoint was measured at post-baseline visits Day 15, Weeks 4, 8, 16, and 24. The treatment effect would usually be estimated at Week 16 or at Week 24.

In Vertex’s briefing book for FDA advisory committee meeting on May 12, 2015, the rationale of using the average of two visits was indicated as below:

Change in ppFEV1 at Week 24 was assessed as the average of the treatment effects at Week 16 and at Week 24 to provide a more precise estimate of the treatment effect at the end of the treatment period, given the inherent variability in ppFEV1. 

The average treatment effects at Weeks 16 and 24 was obtained from the mixed model with repeated measures (MMRM). It would be naïve if one thinks that the average value of week 16 and week 24 is calculated for each individual subject. According to FDA’s statistical review, the statistical methods were described as below:

For the primary efficacy endpoint, absolute change from baseline in ppFEV1 at Week 24, the primary analysis was to test the difference between each active combination treatment group versus placebo using a mixed model with repeated measures (MMRM). Both on-treatment measurements and measurements after treatment discontinuation (for subjects who discontinued dosing early) were included in primary analyses. The MMRM analysis included subject as a random effect, treatment, visit, and treatment-by-visit interaction as fixed effects, with adjustment for sex, age group at baseline, and ppFEV1 severity at screening. An unstructured covariance structure was assumed to model the within-subject errors. A Kenward-Roger approximation was used for the denominator degrees of freedom. The primary result obtained from the model was the average of the treatment effects at Week 16 and at Week 24.

The summary results were depicted at the following graph.

Based on the description above, the following SAS codes can be used to calculate the average treatment effect at weeks 16 and 24. Notice that in MMRM model, all observations for the change from baseline to all visits (day 15, weeks 4, 8, 16, 24) are used. The last two estimate statements are to calculate the treatment effect at week 16 or at week 24 separately. 

proc mixed data=FEV1;
     class subject treat visit sex agegrp ppFEV1_severity;
      model Chg_ppFEV1 = treat visit treat*visit sex 

                         agegrp ppFEV1_severity/ddfm=kr;
       repeated window / sub = subject type = un;
       estimate 'Average Treatment Effect at Weeks 16 and 24'
                             treat -1 1
                             treat*visit 0 0 0 -0.5 -0.5
                                         0 0 0  0.5  0.5/cl;
       estimate 'Treatment Difference at Week 16'
                             treat -1 1
                             treat*visit 0 0 0 -1 0
                                         0 0 0  1 0/cl;
       estimate 'Treatment Difference at Week 24'
                             treat -1 1
                             treat*visit 0 0 0 0 -1
                                         0 0 0 0  1 /cl;
run;