Sensitivity Analysis of Randomized Trials with Missing Data

utterlypanoramicSecurity

Nov 30, 2013 (3 years and 8 months ago)

71 views

Sensitivity Analysis of

Randomized Trials with

Missing Data

Daniel Scharfstein

Department of Biostatistics

Johns Hopkins University

dscharf@jhsph.edu

Statistical Principles for Clinical Trials

(FDA Guidance
-

E9)


“It is important to evaluate the
robustness

of
the results and primary conclusions of the trial.”


Robustness refers to “the
sensitivity

of the
overall conclusions to various limitations of the
data, assumptions, and analytic approaches to
data analysis.”

FDA Critical Path Initiative


FDA’s
Critical Path

white
-
paper “calls for a joint
effort of industry, academia, and the FDA to
identify key problems and develop targeted
solutions.”


In response to this document, the Office of
Biostatistics at CDER has identified
missing data

as one of these key areas (Robert O’Neil,
personal communication).



FDA: Office of Biostatistics at CDER

(Robert O’Neil)


“Virtually every drug/disease area in clinical trials
has problems with patient data that are missing
because patients dropped out, died, withdrew
due toxicity or aggravation with the trial, failed to
complete forms, and other reasons.”


“The success and failure of a trial and its
interpretation often depends on how these
missing data are dealt with at either the planning
or analysis stage.”


“Current statistical methodologies proposed need
to be evaluated for their ability to address
informative treatment
-
related
missing data
…”

FDA: Office of Biostatistics at CDER

(Robert O’Neil)


“… a concensus on missing data approaches
needs to be developed in order to minimize the
impact of failed studies and remove obstacles to
ambiguous interpretation of product efficacy and
safety conclusions.”


“There are
no

established set of
diagnostics

to
evaluate the severity or impact of missing data
…”


“there is no easily available computer software
…”

Clinical Trial Registration
-

ICMJE


“In return for the altruism and trust that makes
clinical research possible, the research enterprise
has an obligation to conduct research ethically
and to report it
honestly
.”


“Registration is only part of the means to an end;
that end is
full transparency

with respect to the
performance and reporting of clinical trials.”


In my view, the evaluation of the robustness of
trial conclusions is an integral part of honest and
transparent reporting.


Goal


Present a coherent
sensitivity analysis
paradigm

for the presentation of results of
clinical trials in which there is concern about
informative missing data.



ACTG 175


ACTG 175 was a randomized, double bind trial
designed to evaluate nucleoside monotherapy vs.
combination therapy in HIV+ individuals with
CD4 counts between 200 and 500.


Participants were randomized to one of four
treatments: AZT, AZT+ddI, AZT+ddC, ddI


CD4 counts were scheduled at baseline, week 8,
and then every 12 weeks thereafter.

ACTG 175


One goal of the investigators was to compare the
treatment
-
specific means of CD4 count at week
56
had all subjects remained on their assigned
treatment through that week
.


The interest is efficacy rather than effectiveness.


We define a completer to be a subject who stays
on therapy and is measured at week 56.
Otherwise, the subject is called a drop
-
out.


33.6% and 26.5% of subjects dropped out in the
AZT+ddI and ddI arms, respectively.



ACTG 175


Completers
-
only analysis


Treatment

Mean
CD4


SE


95% CI

AZT+ddI

385

8.5

ddI

360

7.7

Difference

25

11.5

(3,48)

p=0.0027

ACTG 175


The completers
-
only means will be valid
estimates if, within treatment groups, the
completers and drop
-
outs are similar on
measured and unmeasured characteristics.


Missing at random (MAR), with respect to
treatment group.


Without incorporating additional information, the
MAR assumption is untestable.


It is well known from other studies that, within
treatment groups, drop
-
outs tend to be very
different than completers.





Sensitivity Analysis Paradigm


Evaluate the robustness of the above
conclusions to deviations from the untestable
MAR assumption.


Three steps

1.
Models

2.
Estimation

3.
Testing


Step 1: Models



For each treatment group, specify a set of models
for the relationship between the distributions of
the outcome for drop
-
outs and completers.


Index the treatment
-
specific models by an
untestable, interpretable parameter (
alpha
),
where zero denotes MAR.


alpha

is called a selection bias parameter and it
indexes deviations from MAR.


Pattern
-
mixture model




Treatment
-
specific imputed distributions of
CD4 count at week 56 for drop
-
outs


Selection model for the probability of being a
completer given the outcome.


The parameter
alpha

is interpreted as the log
odds ratio of dropping out when comparing
subjects whose log CD4 count at week 56 differs
by 1 unit.


alpha
>0 (<0) indicates that subjects with higher
(lower) CD4 counts are more likely to drop
-
out.


alpha
=0.5 (
-
0.5) implies that a 2
-
fold increase in
CD4 count yields a 1.4 increase (0.7 decrease) in
the odds of dropping out.



Step 1: Models


For a plausible range of
alpha
’s
,
estimate the
treatment
-
specific means by taking a weighted
average of the mean outcomes from the
completers and drop
-
outs.

Step 2: Estimation

Treatment
-
specific imputed distributions of
CD4 count at week 56 for drop
-
outs

Treatment
-
specific estimated mean
CD4 at week 56 as function of
alpha


Test the null hypothesis of no treatment effect as
a function of treatment
-
specific selection bias
parameters.


For each combination of the treatment
-
specific
selection bias parameters, form a Z
-
statistic by
taking the difference in the estimated means
divided by the estimated standard error of the
difference.

Step 3: Testing


If the selection bias parameters are correctly
specified, this statistic is normal(0,1) under the
null hypothesis.


Reject the null hypothesis at the 0.05 level if the
absolute value of the Z
-
statistic is greater than
1.96.



Step 3: Testing

Contour Plot of Z
-
statistic

Contour Plot of Z
-
statistic


Extensions


Longitudinal and time
-
to
-
event outcomes


MAR or CAR (with respect to all observable
time
-
independent and dependent data)


Sensitivity analysis with respect to
alpha

Sensitivity Analysis Paradigm

Conjecture


There is information from previously conducted
clinical studies to help in the analysis of the
current trials.


Data from previous trials may be able to restrict
the range of or estimate
alpha
.

Comments on Other Approaches


Likelihood
-
based inference


LOCF

Likelihood
-
based Inference


A parametric model for the outcome
and

a
parametric for the probability of being a
completer given the outcome.


For example, the outcome is log normal.


Estimated
alpha
’s are
-
2.6 (95% CI: [
-
3.0,
-
2.1])
and
-
2.8 (95% CI: [
-
3.3,
-
2.2]) in the AZT+ddI
and ddI arms.


Estimated means are 303 (95% CI: [278,331])
and 297 (95% CI: [271,324]) in the AZT+ddI
and ddI arms.


Need to be certain about log
-
normal assumption.







Treatment
-
specific imputed distributions of
CD4 count at week 56 for drop
-
outs

LOCF

Bad idea


Imputing an unreasonable value.


Results may be conservative or anti
-
conservative.


Uncertainty is under
-
estimated.




Summary


We have presented a paradigm for reporting the
results of clinical trials where missingness is
plausibly related to outcomes.


We believe this approach provides a more honest
characterization of the overall uncertainty, which
stems from both sampling variability and lack of
knowledge of the missingness mechanism.

dscharf@jhsph.edu



Scharfstein, Rotnitzky A, Robins JM, and Scharfstein DO: “Semiparametric
Regression for Repeated Outcomes with Non
-
ignorable Non
-
response,”
Journal of the American Statistical Association
, 93, 1321
-
1339, 1998.


Scharfstein DO, Rotnitzky A, and Robins, JM.: “Adjusting for Non
-
ignorable
Drop
-
out Using Semiparametric Non
-
response Models (with discussion),”

Journal of the American Statistical Association
, 94, 1096
-
1146, 1999.


Rotnitzky A, Scharfstein DO, Su TL, and Robins JM
:
“A Sensitivity
Analysis Methodology for Randomized Trials with Potentially Non
-
ignorable
Cause
-
Specific Censoring,”

Biometrics,

57:30
-
113, 2001


Scharfstein DO, Robin JM, Eddings W and Rotnitzky A: “Inference in
Randomized Studies with Informative Censoring and Discrete Time
-
to
-
Event Endpoints,”
Biometrics
, 57: 404
-
413, 2001.


Scharfstein DO and Robins JM: “Estimation of the Failure Time
Distribution in the Presence of Informative Right Censoring,”
Biometrika
89:617
-
635, 2002.


Scharfstein DO, Daniels M, and Robins JM: “Incorporating Prior Beliefs
About Selection Bias in the Analysis of Randomized Trials with Missing
Data,”
Biostatistics
, 4: 495
-
512, 2003.