No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Chop-Lump Tests for Vaccine Trials
Abstract
This article proposes new tests to compare the vaccine and placebo groups in randomized vaccine trials when a small fraction of volunteers become infected. A simple approach that is consistent with the intent-to-treat principle is to assign a score, say W, equal to 0 for the uninfecteds and some postinfection outcome X > 0 for the infecteds. One can then test the equality of this skewed distribution of W between the two groups. This burden of illness (BOI) test was introduced by Chang, Guess, and Heyse (1994, Statistics in Medicine 13, 1807-1814). If infections are rare, the massive number of 0s in each group tends to dilute the vaccine effect and this test can have poor power, particularly if the X's are not close to zero. Comparing X in just the infecteds is no longer a comparison of randomized groups and can produce misleading conclusions. Gilbert, Bosch, and Hudgens (2003, Biometrics 59, 531-541) and Hudgens, Hoering, and Self (2003, Statistics in Medicine 22, 2281-2298) introduced tests of the equality of X in a subgroup-the principal stratum of those "doomed" to be infected under either randomization assignment. This can be more powerful than the BOI approach, but requires unexaminable assumptions. We suggest new "chop-lump" Wilcoxon and t-tests (CLW and CLT) that can be more powerful than the BOI tests in certain situations. When the number of volunteers in each group are equal, the chop-lump tests remove an equal number of zeros from both groups and then perform a test on the remaining W's, which are mostly >0. A permutation approach provides a null distribution. We show that under local alternatives, the CLW test is always more powerful than the usual Wilcoxon test provided the true vaccine and placebo infection rates are the same. We also identify the crucial role of the "gap" between 0 and the X's on power for the t-tests. The chop-lump tests are compared to established tests via simulation for planned HIV and malaria vaccine trials. A reanalysis of the first phase III HIV vaccine trial is used to illustrate the method.
... Finally, we describe in more details the Chop-Lump test. 2 To test the equality of the distribution of S between the two treatment groups, all zero observations are removed from the treatment group with fewer zeros and an equal proportion of zeros are removed from the other treatment group. This leaves one group with no zeros at all. ...
... We considered the rank version of this test (CH − LW) because it is expected to be more powerful. 2 ...
... In vaccine clinical trials, the VE estimate is the standard statistic for presenting efficacy outcomes. In this article, we proposed a VE statistic based on the BOI score. 1 Even if more advanced methods have been proposed in the literature, [2][3][4][5][6][8][9][10][11] we believe that this approach has an important role from a practical point of view. First, the statistic is meaningful and interpretable in vaccine clinical trials: it is one minus the BOI score ratio and represents the proportional reduction in BOI score due to the vaccine. ...
In recent years, many vaccines have been developed for the prevention of a variety of diseases. Although the primary objective of vaccination is to prevent disease, vaccination can also reduce the severity of disease in those individuals who develop breakthrough disease. Observations of apparent mitigation of breakthrough disease in vaccine recipients have been reported for a number of vaccine‐preventable diseases such as Herpes Zoster, Influenza, Rotavirus, and Pertussis. The burden‐of‐illness (BOI) score was developed to incorporate the incidence of disease as well as the severity and duration of disease. A severity‐of‐illness score S > 0 is assigned to individuals who develop disease and a score of 0 is assigned to uninfected individuals. In this article, we derive the vaccine efficacy statistic (which is the standard statistic for presenting efficacy outcomes in vaccine clinical trials) based on BOI scores, and we extend the method to adjust for baseline covariates. Also, we illustrate it with data from a clinical trial in which the efficacy of a Herpes Zoster vaccine was evaluated.
... where " p is the proportion of infections in the combined samples, " x the overall mean, and s G 2 the sample variance of X in group G. One of the limitations of the BOI test is its low power in the presence of many zeros, in particular if p V & p C 2,6 where p V and p C are the proportions of infected individuals in the vaccinated and [10.5.2014–9:50am] [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16]//blrnas3/cenpro/ApplicationFiles/Journals/SAGE/3B2/SMMJ/Vol00000/140049/APPFile/SG-SMMJ140049.3d (SMM) [PREPRINTER stage] control groups, respectively. ...
... For values of Z between 1 and 5, there is a high linear correlation between Z and À logð1 À ÈðZÞÞ, making the power of Fisher methods similar to the power of the Z-score method. For simplicity, in the following we will approximate the asymptotic power of the Fisher method by the asymptotic power of the Z-score method [10.5.2014–9:50am] [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16]//blrnas3/cenpro/ApplicationFiles/Journals/SAGE/3B2/SMMJ/Vol00000/140049/APPFile/SG-SMMJ140049.3d (SMM) [PREPRINTER stage] where c ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi f1 þ 3ð1 À pÞg=2 p . Figure 1plots the curve of indifference between CH–LW test and FCM test. ...
... [10.5.2014–9:50am] [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16]//blrnas3/cenpro/ApplicationFiles/Journals/SAGE/3B2/SMMJ/Vol00000/140049/APPFile/SG-SMMJ140049.3d (SMM) [PREPRINTER stage] ...
In recent years, many vaccines have been developed for the prevention of a variety of diseases. Many of these vaccines, like the one for herpes zoster, are supposed to act in a multilevel way. Ideally, they completely prevent expression of the virus, but failing that they help to reduce the severity of the disease. A simple approach to analyze these data is the so-called burden-of-illness test. The method assigns a score, say W, equal to 0 for the uninfected and a post-infection outcome X > 0 for the infected individuals. One of the limitations of this test is the potential low power when the vaccine efficacy is close to 0. To overcome this limitation, we propose a Fisher adjusted test where we combine a statistic for infection with a statistic for post-infection outcome adjusted for selection bias. The advantages and disadvantages of different methods proposed in the literature are discussed. We compared the methods via simulations in herpes zoster, HIV, and malaria vaccine trial settings. In addition, we applied these methods to published data on HIV vaccine. The paper ends with some recommendations and conclusions.
... This is a test of the total "burden of illness." This approach creates "lumpy" data because the nonzero Y values may be quite far from 0. Having a bolus of zeroes could seriously inflate the variance if a t-test is used, leading to poor power [1]. Avoiding this problem by using a rank test creates another problem: tied ranks from the zeroes. ...
... To test the incidence null hypothesis where p C and p V are the probabilities of infection in the two arms, form the usual z-score comparing proportions: (1) where , and . A closely related zstatistic compares the observed number of people in the upper left cell of Table 1 to the number expected from its central hypergeometric null distribution: ...
Vaccine benefit is usually two-folded: (i) prevent a disease or, failing that, (ii) diminish the severity of a disease. To assess vaccine effect, we propose two adaptive tests. The weighted two-part test is a combination of two statistics, one on disease incidence and one on disease severity. More weight is given to the statistic with the larger a priori effect size, and the weights are determined to maximize testing power. The randomized test applies to the scenario where the total number of infections is relatively small. It uses information on disease severity to bolster power while preserving disease incidence as the primary interest. Properties of the proposed tests are explored asymptotically and by numerical studies. Although motivated by vaccine studies, the proposed tests apply to any trials that involve both binary and continuous outcomes for evaluating treatment effect. Published 2015. This article is a US Government work and is in the public domain in the USA.
... As was done in example 1, a series of supportive and complementary statistical tests could be done to better understand the possible mechanism of action and under what set of assumptions the proposed tests of efficacy would remain valid. Several authors have in fact proposed testing schemes for HIV vaccine efficacy [10,11,12,13]. Mehrotra et al. [10] compare a number of composite statistics, which amount to different weighting schemes for the two univariate tests, and discuss their relative performance under different possible alternatives. ...
... Gilbert et al. [11] and others consider the framework of potential outcomes to construct sensitivity analyses to better understand the existing evidence for efficacy under different possible scenarios for differences in infected populations on the two arms [12]. Follmann et al. [13] propose a test designed to have good power for a location shift in Y but in a way that would not see power gains under the harmful vaccine scenario described above. These and many other instructive and creative papers in this area demonstrate how careful elucidation of alternatives and study of proposed tests under these alternatives can not only expand the scientific insights to be gained from the data but may be a necessary step to avoiding erroneous inference. ...
Standard statistical theory teaches us that once the null and alternative hypotheses have been defined for a parameter, the choice of the statistical test is clear. Standard theory does not teach us how to choose the null or alternative hypothesis appropriate to the scientific question of interest. Neither does it tell us that in some cases, depending on which alternatives are realistic, we may want to define our null hypothesis differently. Problems in statistical practice are frequently not as pristinely summarized as the classic theory in our textbooks. In this article, we present examples in statistical hypothesis testing in which seemingly simple choices are in fact rich with nuance that, when given full consideration, make the choice of the right hypothesis test much less straightforward. Copyright © 2012 John Wiley & Sons, Ltd.
... An equal number of zeros is removed from both groups, and the test is conducted on the on the remaining scores, which are mostly greater than zero. 37 We note that it would be useful to conduct further investigation of different inference methods, including consideration of type I error control, power, and empirical coverage of confidence intervals. Such investigation might also include additional simulation studies in a variety of settings. ...
Network science methods can be useful in design, monitoring, and analysis of randomized trials for control of spread of infections. Their usefulness arises from the role of statistical network models in molecular epidemiology and in study design. Computational models, such as agent-based models that propagate disease on simulated contact networks, can be used to investigate the properties of different study designs and analysis plans. Particularly valuable is the use of these methods to assess how magnitude and detectability of intervention effects depend on both individual-level and network-level characteristics of the enrolled populations. Such investigation also provides an important approach to assessing consequences of study data being incomplete or measured with error. To address these goals, we consider two statistical network models: exponential random graph models and the more flexible congruence class models. We focus first on an historical use of these methods in design and monitoring of a cluster randomized trial in Botswana to evaluate the effect of combination HIV prevention modalities compared to standard of care on HIV incidence. We then present a framework for the design of a study of booster vaccine effects on infection with, and forward transmission of, SARS-CoV-2 variants. Motivation for the study is driven in part by guidance from the United Kingdom to base approval of booster vaccines with “strain changes” that target variants on results of neutralizing antibody tests and information about safety, but without requiring evidence of clinical efficacy. Using designs informed by our agent-based network models, we show it may be feasible to conduct a trial of novel SARS-CoV-2 vaccines in a single large campus to obtain useful information regarding vaccine efficacy against susceptibility and infectiousness. If needed, the sample size could be increased by extending the study to a small number of campuses. Novel network methods may be useful in developing pragmatic SARS-CoV-2 vaccine trials that can leverage existing infrastructure to reduce costs and hasten the development of results.
... Another problem for vaccines targeting rare events is that most of the participants do not become infected during the trial. Follmann et al. (2009) [61] introduced chop-lump Wilcoxon and t-tests based on BOI to tackle the issue. Similar to BOI, this approach also assigns a score to each subject, 0 for uninfected subjects and S for severity endpoint assessment. ...
In the past decades, the world has experienced several major virus outbreaks, e.g. West African Ebola outbreak, Zika virus in South America and most recently global coronavirus (COVID-19) pandemic. Many vaccines have been developed to prevent a variety of infectious diseases successfully. However, several infections have not been preventable so far, like COVID-19, which induces an immediate urgent need for effective vaccines. These emerging infectious diseases often pose unprecedent challenges for the global heath community as well as the conventional vaccine development paradigm. With a long and costly traditional vaccine development process, there are extensive needs in innovative vaccine trial designs and analyses, which aim to design more efficient vaccines trials. Featured with reduced development timeline, less resource consuming or improved estimate for the endpoints of interests, these more efficient trials bring effective medicine to target population in a faster and less costly way. In this paper, we will review a few vaccine trials equipped with adaptive design features, Bayesian designs that accommodate historical data borrowing, the master protocol strategy emerging during COVID-19 vaccine development, Real-World-Data (RWD) embedded trials and the correlate of protection framework and relevant research works. We will also discuss some statistical methodologies that improve the vaccine efficacy, safety and immunogenicity analyses. Innovative clinical trial designs and analyses, together with advanced research technologies and deeper understanding of the human immune system, are paving the way for the efficient development of new vaccines in the future.
... Logistic regression and the Wilcoxon rank-sum test with inverse probability weighting (67) were used to compare the immune response rates and magnitudes between genotype groups among control vaccine recipients. For immune response variables (e.g., IgA) with some positive responses but many negative responses (response rate of Յ20%), the choplump test (68) was used instead of the Wilcoxon rank-sum test. ...
By analyzing data from the HVTN 505 efficacy trial of a DNA/recombinant adenovirus 5 (rAd5) vaccine regimen, we found that host genetics, specifically Fc gamma receptor genetic variations, influenced whether receiving the DNA/rAd5 regimen was beneficial, neutral, or detrimental to an individual with respect to HIV-1 acquisition risk. Moreover, Fc gamma receptor genetic variations influenced immune responses to the DNA/rAd5 vaccine regimen. Thus, Fc gamma receptor genetic variations should be considered in the analysis of future HIV vaccine trials and the development of HIV vaccines.
... These analyses were performed on the modified Total Vaccinated Cohort (mTVC), which excluded autologous HSCT recipients who did not receive 2 doses or who had a confirmed HZ episode within 1 month of receiving dose 2, and included only patients with HZ who completed at least 1 ZBPI questionnaire. The Chop-Lump [25] test was used to assess the difference in ZBPI severity of illness scores and ZBPI severity of interference scores between the RZV and placebo groups in the mTVC cohort. ...
Herpes zoster (HZ) can have a substantial impact on quality of life (QoL). The vaccine efficacy (VE) of a recombinant zoster vaccine (RZV) was 68.2% (95% confidence interval [CI], 55.6% to 77.5%) in a phase 3 study in adult autologous hematopoietic stem cell transplant (HSCT) recipients (NCT01610414). Herein, we report the impact of RZV on patients' QoL. Autologous HSCT recipients were randomized 1:1 to receive 2 doses of RZV or placebo, given 1 to 2 months apart. QoL was measured by the Short Form Survey-36 and Euro-QoL-5 Dimension at baseline, 1 month, and 1 year postdose 2 and during suspected HZ episodes with the Zoster Brief Pain Inventory (ZBPI). The RZV impact on ZBPI burden of illness and burden of interference scores was estimated. The 2 scores were calculated from the area under the curve (days 0 to 182) of the ZBPI worst pain and ZBPI activities of daily living scores, respectively, assuming a score of 0 for patients not having a confirmed HZ episode. The ZBPI maximum worst pain score was significantly lower in the RZV than placebo group (mean: 5.8 versus 7.1, P = .011). Consequently, the VE estimates for HZ burden of illness (82.5%; 95% CI, 73.6 to 91.4) and burden of interference (82.8%; 95% CI, 73.3 to 92.3) were higher than the HZ VE estimate (ie, 68.2%). RZV showed significantly better QoL scores than placebo 1 week following rash onset among patients with confirmed HZ. In addition to reducing the risk of HZ and its complications, RZV significantly reduced the impact of HZ on patients' QoL in those who developed breakthrough disease.
... These analyses were performed on the modified total vaccinated cohort (mTVC), which excluded subjects who did not receive two doses or who had a confirmed HZ episode within one month of receiving dose 2 and included only HZ patients who completed at least one Downloaded from https://academic.oup.com/biomedgerontology/advance-article-abstract/doi/10.1093/gerona/gly150/5046047 by guest on 28 June 2018 A c c e p t e d M a n u s c r i p t ZBPI questionnaire. The Chop-Lump [16] test was used to assess the difference in ZBPI severity of illness scores and ZBPI severity of interference scores between the RZV and placebo groups in the mTVC cohort. ...
Background
To determine the efficacy of a recombinant zoster vaccine (RZV) in reducing the herpes zoster (HZ) burden of illness, HZ burden of interference with activities of daily living (ADLs) and HZ impact on quality of life (QoL).
Methods
The assessments were integrated in two Phase III trials, ZOE-50 (NCT01165177) and ZOE-70 (NCT01165229). HZ burden of illness and HZ burden of interference with ADLs were assessed by the Zoster Brief Pain Inventory (ZBPI) instrument; QoL by the EuroQol-5 Dimension (EQ-5D) utility index and the SF-36 health survey. We report the ZOE-50 results and a pooled analysis of subjects ≥70 years of age from the trials combined.
Results
The estimated vaccine efficacy (VE) in reducing HZ burden of illness as well as HZ burden of interference was >90% in both the ZOE-50 and the pooled ZOE-70 analysis. In confirmed HZ cases, RZV reduced the maximal ZBPI worst-pain score in the pooled ZOE-70 analysis (p=0.032) and the maximal ZBPI average-pain scores in both the ZOE-50 (p=0.049) and the pooled ZOE-70 analysis (p=0.043). In breakthrough HZ cases, trends for diminished loss of QoL compared to placebo-recipient HZ cases were observed, with differences up to 0.14 on the EQ-5D index at time points during the four weeks following HZ onset.
Conclusions
RZV reduced the HZ burden of illness significantly, particularly due to its very high VE in preventing HZ. For breakthrough HZ cases, the results suggest that RZV mitigated severity of HZ-related pain, burden of interference with ADLs and recipients’ utility loss.
... This approach has previously been established as problematic and discussed in several publications in HIV research outlining the study design challenge of wanting to evaluate treatment only in infected individuals. These papers discuss the drawbacks of the ITTI approach and provide valid analyses incorporating data from all randomized participants (3)(4)(5)(6). ...
... While randomization inference methods are well developed for binary and continuous outcomes, there has been comparatively little work for outcomes with point masses at zero. Existing work tends to focus on testing without consideration of interval or point estimation (Follmann et al. 2009;Hallstrom 2010). ...
While randomization inference is well developed for continuous and binary outcomes, there has been comparatively little work for outcomes with nonnegative support and clumping at zero. Typically, outcomes of this type have been modeled using parametric models that impose strong distributional assumptions. This article proposes new randomization inference procedures for nonnegative outcomes with clumping at zero. Instead of making distributional assumptions, we propose various assumptions about the nature of the response to treatment and use permutation inference for both testing and estimation. This approach allows for some natural goodness-of-fit tests for model assessment, as well as flexibility in selecting test statistics sensitive to different potential alternatives. We illustrate our approach using two randomized trials where job training interventions were designed to increase earnings of participants.
... These testing procedures compare the randomized groups by essentially removing an equal number of zeros (or patients with undetectable titer levels) from each group before performing a permutation test on the remaining, less-diluted sample. 20,21 Last, an approach used in the LEAP study secondary analyses was to examine the upper end of the peanut-specific IgE distribution, rather than the means or medians. The LEAP study evaluated a high-risk cohort and after 5 years of follow-up, 17% developed peanut allergy in the Avoidance (control) group compared with 3% in the Consumption (intervention) group. ...
Clinical studies to prevent the development of food allergy have recently helped reshape public policy recommendations on the early introduction of allergenic foods. These trials are also prompting new research, and it is therefore important to address the unique design and analysis challenges of prevention trials. We highlight statistical concepts and give recommendations that clinical researchers may wish to adopt when designing future study protocols and analysis plans for prevention studies. Topics include selecting a study sample, addressing internal and external validity, improving statistical power, choosing alpha and beta, analysis innovations to address dilution effects, and analysis methods to deal with poor compliance, dropout, and missing data.
... Also, we cannot guarantee that important metabolites will not be discarded even if the group structure is taken into account. Alternatively, an equal number of zero observations can be removed from each group, e.g. the Chop-Lump approach [7] which proved powerful when combined with the Wilcoxon or t-test. However, this approach will further reduce already small group sizes for which metabolomics research is known. ...
Background
ERp is a variable selection and classification method for metabolomics data. ERp uses minimized classification error rates, based on data from a control and experimental group, to test the null hypothesis of no difference between the distributions of variables over the two groups. If the associated p-values are significant they indicate discriminatory variables (i.e. informative metabolites). The p-values are calculated assuming a common continuous strictly increasing cumulative distribution under the null hypothesis. This assumption is violated when zero-valued observations can occur with positive probability, a characteristic of GC-MS metabolomics data, disqualifying ERp in this context. This paper extends ERp to address two sources of zero-valued observations: (i) zeros reflecting the complete absence of a metabolite from a sample (true zeros); and (ii) zeros reflecting a measurement below the detection limit. This is achieved by allowing the null cumulative distribution function to take the form of a mixture between a jump at zero and a continuous strictly increasing function. The extended ERp approach is referred to as XERp.
Results
XERp is no longer non-parametric, but its null distributions depend only on one parameter, the true proportion of zeros. Under the null hypothesis this parameter can be estimated by the proportion of zeros in the available data. XERp is shown to perform well with regard to bias and power. To demonstrate the utility of XERp, it is applied to GC-MS data from a metabolomics study on tuberculosis meningitis in infants and children. We find that XERp is able to provide an informative shortlist of discriminatory variables, while attaining satisfactory classification accuracy for new subjects in a leave-one-out cross-validation context.
Conclusion
XERp takes into account the distributional structure of data with a probability mass at zero without requiring any knowledge of the detection limit of the metabolomics platform. XERp is able to identify variables that discriminate between two groups by simultaneously extracting information from the difference in the proportion of zeros and shifts in the distributions of the non-zero observations. XERp uses simple rules to classify new subjects and a weight pair to adjust for unequal sample sizes or sensitivity and specificity requirements.
Electronic supplementary material
The online version of this article (doi:10.1186/s12859-017-1480-8) contains supplementary material, which is available to authorized users.
... When an interpretation only relies on the second part (positive outcome) of the ZIP or ZINB model, the conclusion could be misleading because the two groups with the positive outcome are not ensured to be comparable by randomization. 36 In this article, our estimates of direct, mediation, and total effects and their comparisons between groups will use information from all the randomized subjects with both parts of the model so that the ignorability of randomization holds. The outcome distribution under ZIP is ...
Different conventional and causal approaches have been proposed for mediation analysis to better understand the mechanism of a treatment. Count and zero-inflated count data occur in biomedicine, economics, and social sciences. This paper considers mediation analysis for count and zero-inflated count data under the potential outcome framework with nonlinear models. When there are post-treatment confounders which are independent of, or affected by, the treatment, we first define the direct, indirect, and total effects of our interest and then discuss various conditions under which the effects of interest can be identified. Proofs are provided for the sensitivity analysis proposed in the paper. Simulation studies show that the methods work well. We apply the methods to the Detroit Dental Health Project's Motivational Interviewing DVD trial for the direct and indirect effects of motivational interviewing on count and zero-inflated count dental caries outcomes.
... However, studies of this nature are inherently difficult, and we are unaware of a 'silver bullet' approach that fully addresses the truncation by death question. We recommend the use of principal stratification methods in conjunction with other approaches (e.g., [31]) to better understand treatment Copyright effects. To this end, the rank-based approaches described in this manuscript are important methods to add to an analyst's tool kit. ...
We describe rank-based approaches to assess principal stratification treatment effects in studies where the outcome of interest is only well-defined in a subgroup selected after randomization. Our methods are sensitivity analyses, in that estimands are identified by fixing a parameter and then we investigate the sensitivity of results by varying this parameter over a range of plausible values. We present three rank-based test statistics and compare their performance through simulations, and provide recommendations. We also study three different bootstrap approaches for determining levels of significance. Finally, we apply our methods to two studies: an HIV vaccine trial and a prostate cancer prevention trial. Copyright © 2013 John Wiley & Sons, Ltd.
... We prefer to use V E rather than than V E * as our definition of vaccine efficacy because when assumption (A-1) fails and the vaccine affects the probability of a non-malaria caused fever, we would like to include this effect in the measure of vaccine efficacy. A vaccine may cause fever by a variety of mechanisms (Follmann, Fay and Proschan, 2009): (a) some vaccines are live but weakened pathogens and the weakened pathogen can produce an infection in a weakened host; (b) vaccine induced antibodies may enhance rather than reduce the chance of disease (Burke, 1992); or (c) vaccines might induce an auto-immune reaction which could hamper the ability of the immune system to fight disease. As an example , consider a vaccine that prevents 10% of malaria caused fevers but causes 10{R P by malaria/(1 − R P by all causes)}% of the people without fevers under the placebo to have a fever. ...
Malaria is a major public health problem. An effective vaccine against malaria is actively being sought. We formulate a potential outcomes definition of the efficacy of a malaria vaccine for preventing fever. A challenge in estimating this efficacy is that there is no sure way to determine whether a fever was caused by malaria. We study the properties of two approaches for estimating efficacy: (1) use a deterministic case definition of a malaria caused fever as the conjunction of fever and parasite density above a certain cutoff; (2) use a probabilistic case definition in which the probability that each fever was caused by malaria is estimated. We compare these approaches in a simulation study and find that both approaches can potentially have large biases. We suggest a strategy for choosing an estimator based on the investigator's prior knowledge about the area in which the trial is being conducted and the range of vaccine efficacies over which the investigator would like the estimator to have good properties.
The outcome in a randomized experiment is sometimes nonnegative with a clump of observations at zero and continuously distributed positive values. One widely used model for a nonnegative outcome with a clump at zero is the Tobit model, which assumes that the treatment has a shift effect on the distribution of a normally distributed latent variable and the observed outcome is the maximum of the latent variable and zero. We develop a class of semiparametric models and inference procedures that extend the Tobit model in two useful directions. First, we consider more flexible models for the treatment effect than the shift effect of the Tobit model; for example, our models allows for the treatment to have a larger in magnitude effect for upper quantiles. Second, we make semiparametric inferences using empirical likelihood that allow the underlying latent variable to have any distribution, unlike the original Tobit model that assumes the latent variable is normally distributed. We apply our approach to data from the RAND Health Insurance Experiment. We also extend our approach to observational studies in which treatment assignment is strongly ignorable.
Intradermal (i.d.) and intramuscular (i.m.) injections when administered with or without electroporation (EP) have the potential to tailor the immune response to DNA vaccination. This Phase I randomized controlled clinical trial in human immunodeficiency virus type 1-negative volunteers investigated whether the site and mode of DNA vaccination influences the quality of induced cellular and humoral immune responses following the DNA priming phase and subsequent protein boost with recombinant clade C CN54 gp140. A strategy of concurrent i.d. and i.m. DNA immunizations administered with or without EP was adopted. Subtle differences were observed in the shaping of vaccine-induced virus-specific CD4+ and CD8+ T cell-mediated immune responses between groups receiving: i.d.EP + i.m., i.d. + i.m.EP, and i.d.EP + i.m.EP regimens. The DNA priming phase induced 100% seroconversion in all of the groups. A single, non-adjuvanted protein boost induced a rapid and profound increase in binding antibodies in all groups, with a trend for higher responses in i.d.EP + i.m.EP. The magnitude of antigen-specific binding immunoglobulin G correlated with neutralization of closely matched clade C 93MW965 virus and Fc-dimer receptor binding (FcγRIIa and FcγRIIIa). These results offer new perspectives on the use of combined skin and muscle DNA immunization in priming humoral and cellular responses to recombinant protein.
Mediation analysis seeks to understand the mechanism by which a treatment affects an outcome. Many conventional and causal mediation approaches have been developed for continuous outcomes. When the outcome of interest is non-continuous (e.g., binary, count, or zero-inflated count), mediation approaches relying on linear models may not be appropriate. This chapter introduces the framework and definition of direct and indirect (mediation) treatment effects on non-continuous outcomes around and through intermediate variables (mediators). Different mediation approaches for non-continuous outcomes are discussed under a variety of settings where post-baseline confounders and unmeasured confounders may be a concern, and under a series of assumptions with and without sequential ignorability. Methods are illustrated with application to a randomized dental caries prevention trial.
Intradermal (ID) and intramuscular (IM) injections when administered with or without electroporation (EP), have the potential to tailor the immune response to DNA vaccination. In this Phase I randomized controlled clinical trial in HIV-1 negative volunteers we investigate whether the site and mode of DNA vaccination influences the quality of induced cellular and humoral immune responses following the DNA priming phase and subsequent protein boost with recombinant clade C CN54 gp140. We adopted a strategy of concurrent ID and IM DNA immunizations administered with or without EP. We observed subtle differences in shaping of vaccine induced virus-specific
CD4+ and CD8+ T cell-mediated immune responses between groups receiving: IDEP+IM, ID+IMEP and IDEP+IMEP regimens. The DNA priming phase induced 100% seroconversion in all of the groups. A single, non-adjuvanted protein boost induced a rapid and profound increase in binding antibodies in all groups, with a trend for higher responses in the IDEP+IMEP group. The magnitude of antigen specific binding IgG correlated with neutralization of closely matched clade C 93MW965 virus and Fc-dimer receptor binding (FcgRIIa and FcgRIIIa). These results offer new perspectives on the use of combined skin and muscle DNA immunization in priming humoral and cellular responses to recombinant protein.
Vaccines have a long history of providing protection to human health. As an example, eradication of the deadly disease, Smallpox, was accomplished in 1980 via massive vaccination programs around the world. Vaccination was, in fact, recognized as the greatest public health achievement of the 20th century by the U.S. Centers for Disease Control and Prevention. In recent years, statisticians have played an increasingly important role as collaborative researchers in vaccine development. This review paper highlights some of the innovative applications of statistical techniques that have contributed to the successful development of several vaccines for preventing diseases such as chickenpox, herpes zoster, rotavirus gastroenteritis, and cervical cancer. Real life examples will be used to illustrate statistical innovations in several areas, including trial design and adaptation, safety monitoring, endpoint measure, and the evaluation of correlates of protection. Many of these methods, while innovative and novel at the time, are now broadly applied across pharmaceutical and vaccine clinical programs. These statistical contributions are a reflection of the statisticians' engagement and leadership, which has resulted in a substantial improvement of the scientific rigor, quality, and operational efficiency in vaccine development. This paper was contributed as part of the 175th Anniversary celebration of the American Statistical Association.
Length of stay in the intensive care unit (ICU) is a common outcome measure in randomized trials of ICU interventions. Because many patients die in the ICU, it is difficult to disentangle treatment effects on length of stay from effects on mortality; conventional analyses depend on assumptions that are often unstated and hard to interpret or check. We adapt a proposal from Rosenbaum that addresses concerns about selection bias and makes its assumptions explicit. A composite outcome is constructed that equals ICU length of stay if the patient was discharged alive and indicates death otherwise. Given any preference ordering that compares death with possible lengths of stay, we can estimate the intervention's effects on the composite outcome distribution. Sensitivity analyses can show results for different preference orderings. We discuss methods for constructing approximate confidence intervals for treatment effects on quantiles of the outcome distribution or on proportions of patients with outcomes preferable to various cutoffs. Strengths and weaknesses of possible primary significance tests (including the Wilcoxon-Mann-Whitney rank sum test and a heteroskedasticity-robust variant due to Brunner and Munzel) are reviewed. An illustrative example reanalyzes a randomized trial of an ICU staffing intervention.
This article is concerned with the assessment of the causal effect of a treatment on a variable that is defined only in the subset of patients who experience a specific event. In this case, the treatment can affect the variable directly, as well as indirectly through its effect on the occurrence and severity of the event. In this article we describe the pure (direct) and hybrid (direct and indirect) causal effects and methods typically used to assess them. When the treatment has a strong effect on the occurrence of the event, we found no method with adequate properties to address the pure causal effect; this remains an intractable statistical problem with no clear solution. Among the valid methods for assessment of the hybrid causal effect, power depends greatly on whether the treatment effect is primarily on the occurrence of the event or on the variable of interest.
and Examples.- Overview of Vaccine Effects and Study Designs.- Immunology and Early Phase Trials.- Binomial and Stochastic Transmission Models.- and Deterministic Models.- Evaluating Protective Effects of Vaccination.- Modes of Action and Time-Varying VE.- Further Evaluation of Protective Effects.- Vaccine Effects on Post-Infection Outcomes.- Household-Based Studies.- Analysis of Households in Communities.- Analysis of Independent Households.- Assessing Indirect, Total, and Overall Effects.- Randomization and Baseline Transmission.- Surrogates of Protection.
Study objective:
To determine the neurocognitive effects of continuous positive airway pressure (CPAP) therapy on patients with obstructive sleep apnea (OSA).
Design, setting, and participants:
The Apnea Positive Pressure Long-term Efficacy Study (APPLES) was a 6-month, randomized, double-blind, 2-arm, sham-controlled, multicenter trial conducted at 5 U.S. university, hospital, or private practices. Of 1,516 participants enrolled, 1,105 were randomized, and 1,098 participants diagnosed with OSA contributed to the analysis of the primary outcome measures.
Intervention:
Active or sham CPAP MEASUREMENTS: THREE NEUROCOGNITIVE VARIABLES, EACH REPRESENTING A NEUROCOGNITIVE DOMAIN: Pathfinder Number Test-Total Time (attention and psychomotor function [A/P]), Buschke Selective Reminding Test-Sum Recall (learning and memory [L/M]), and Sustained Working Memory Test-Overall Mid-Day Score (executive and frontal-lobe function [E/F])
Results:
The primary neurocognitive analyses showed a difference between groups for only the E/F variable at the 2 month CPAP visit, but no difference at the 6 month CPAP visit or for the A/P or L/M variables at either the 2 or 6 month visits. When stratified by measures of OSA severity (AHI or oxygen saturation parameters), the primary E/F variable and one secondary E/F neurocognitive variable revealed transient differences between study arms for those with the most severe OSA. Participants in the active CPAP group had a significantly greater ability to remain awake whether measured subjectively by the Epworth Sleepiness Scale or objectively by the maintenance of wakefulness test.
Conclusions:
CPAP treatment improved both subjectively and objectively measured sleepiness, especially in individuals with severe OSA (AHI > 30). CPAP use resulted in mild, transient improvement in the most sensitive measures of executive and frontal-lobe function for those with severe disease, which suggests the existence of a complex OSA-neurocognitive relationship.
Clinical trial information:
Registered at clinicaltrials.gov. Identifier: NCT00051363.
Citation:
Kushida CA; Nichols DA; Holmes TH; Quan SF; Walsh JK; Gottlieb DJ; Simon RD; Guilleminault C; White DP; Goodwin JL; Schweitzer PK; Leary EB; Hyde PR; Hirshkowitz M; Green S; McEvoy LK; Chan C; Gevins A; Kay GG; Bloch DA; Crabtree T; Demen WC. Effects of continuous positive airway pressure on neurocognitive function in obstructive sleep apnea patients: the Apnea Positive Pressure Long-term Efficacy Study (APPLES). SLEEP 2012;35(12):1593-1602.
In randomized studies, treatment comparisons conditional on intermediate post-randomization outcomes using standard analytic methods do not have a causal interpretation. An alternate approach entails treatment comparisons within principal strata defined by the potential outcomes for the intermediate outcome that would be observed under each treatment assignment. In this paper, we develop methods for randomization-based inference within principal strata. The proposed methods are compared with existing large-sample methods as well as traditional intent-to-treat approaches. This research is motivated by HIV prevention studies where few infections are expected and inference is desired within the always-infected principal stratum, i.e., all individuals who would become infected regardless of randomization assignment.
0W(t)), where 0(t) is an unspecied baseline hazard function, W(t) = w(t, V(t)), w(; ) is a known function that maps (t, V(t)) to Rq, and0 is a q 1 unknown parameter vector. When 0 6 =0 , then drop-out is nonignorable. On account of identiability problems, joint estimation of the mean 0 of Y and the selection bias parameter 0 may be dicult or impossible. Therefore, we propose regarding the selection bias parameter 0 as known, rather than estimating it from the data. We then perform a sensitivity analysis to see how inference about 0 changes as we vary 0 over a plausible range of values. We apply our approach to the analysis of ACTG 175, an AIDS clinical trial.
We outline a framework for causal inference in setting where assignment to a binary treatment is ignorable, but compliance with the assignment is not perfect so that the receipt of treatment is nonignorable. To address the problems associated with comparing subjects by the ignorable assignment--an "intention-to-treat analysis"--we make use of instrumental variables, which have long been used by economists in the context of regression models with constant treatment effects. We show that the instrumental variables (IV) estimand can be embedded within the Rubin Causal Model (RCM) and that under some simple and easily interpretable assumptions, the IV estimand is the average causal effect for a subgroup of units, the compliers. Without these assumptions, the IV estimand is simply the ratio of intention-to-treat causal estimands with no interpretation as an average causal effect. The advantages of embedding the IV approach in the RCM are that it clarifies the nature of critical assumptions needed for a causal interpretation, and moreover allows us to consider sensitivity of the results to deviations from key assumptions in a straightforward manner. We apply our analysis to estimate the effect of veteran status in the Vietnam era on mortality, using the lottery number assigned priority for the draft as an instrument, and we use our results to investigate the sensitivity of the conclusions to critical assumptions. Statistics Version of Record
Vaccine efficacy and effectiveness (VE) are generally measured as 1 minus some measure of relative risk (RR) in the vaccinated group compared with the unvaccinated group (VE = 1 - RR). In designing a study to evaluate vaccination, the type of effect and the question of interest determine the appropriate choice of comparison population and parameter. Possible questions of interest include that of the biologic effect of vaccination on susceptibility, on infectiousness, or on progression to disease in individuals. The indirect effects, total effects, and overall public health benefits of widespread vaccination of individuals within the context of a vaccination program might also be of primary concern. The change in behavior induced by belief in the protective effects of vaccination might influence the estimates of these effects or might itself be of interest. In this paper, the authors present a framework of study designs that relates the scientific question of interest to the choice of comparison groups, the unit of observation, the level of information available for analysis, and the parameter of effect.
The natural history of human immunodeficiency virus type 1 (HIV-1) viremia and its association with clinical outcomes after
seroconversion was characterized in a cohort of homosexual men. HIV-1 RNA was measured by reverse-transcription polymerase
chain reaction (RT-PCR) in stored longitudinal plasma samples from 269 seroconverters. Subjects were generally antiretroviral
drug naive for the first 3 years after seroconversion. The decline in CD4 lymphocyte counts was strongly associated with initial
HIV RNA measurements. Both initial HIV RNA levels and slopes were associated with AIDS-free times. Median slopes were +0.18,
+0.09, and −0.01 log10 copies/mL, respectively, for subjects developing AIDS <3, 3–7, and >7 years after seroconversion. In contrast, HIV RNA slopes
in the 3 years preceding AIDS and HIV RNA levels at AIDS diagnosis showed little variation according to total AIDS-free time.
HIV RNA load at the first HIV-seropositive visit (∼3 months after seroconversion) was highly predictive of AIDS, and subsequent
HIV RNA measurements showed even better prognostic discrimination.
The last case of poliomyelitis in the United States due to indigenously acquired wild poliovirus occurred in 1979; however, as a consequence of oral poliovirus vaccine (OPV) use that began in 1961, an average of 9 cases of vaccine-associated paralytic poliomyelitis (VAPP) were confirmed each year from 1961 through 1989. To reduce the VAPP burden, national vaccination policy changed in 1997 from reliance on OPV to options for a sequential schedule of inactivated poliovirus vaccine (IPV) followed by OPV. In 2000, an exclusive IPV schedule was adopted.
To review the epidemiology of paralytic poliomyelitis and document the association between the vaccine schedule changes and VAPP in the United States.
Review of national surveillance data from 1990 through 2003 for cases of confirmed paralytic poliomyelitis.
Number of confirmed paralytic poliomyelitis cases, including VAPP, and ratio of VAPP cases to number of doses of OPV distributed that occurred before, during, and after implementation of policy changes.
From 1990 through 1999, 61 cases of paralytic poliomyelitis were reported; 59 (97%) of these were VAPP (1 case per 2.9 million OPV doses distributed), 1 case was imported, and 1 case was indeterminate. Thirteen cases occurred during the 1997-1999 transitional policy period and were associated with the all-OPV schedule; none occurred with the IPV-OPV schedule. No cases occurred after the United States implemented the all-IPV policy in 2000. The last imported poliomyelitis case occurred in 1993 and the last case of VAPP occurred in 1999.
The change in polio vaccination policy from OPV to exclusive use of IPV was successfully implemented; this change led to the elimination of VAPP in the United States.
A vaccine is needed to prevent human immunodeficiency virus type 1 (HIV-1) infection.
A double-blind, randomized trial of a recombinant HIV-1 envelope glycoprotein subunit (rgp120) vaccine was conducted among men who have sex with men and among women at high risk for heterosexual transmission of HIV-1. Volunteers received 7 injections of either vaccine or placebo (ratio, 2 : 1) over 30 months. The primary end point was HIV-1 seroconversion over 36 months.
A total of 5403 volunteers (5095 men and 308 women) were evaluated. The vaccine did not prevent HIV-1 acquisition: infection rates were 6.7% in 3598 vaccinees and 7.0% in 1805 placebo recipients; vaccine efficacy (VE) was estimated as 6% (95% confidence interval, -17% to 24%). There were no significant differences in viral loads, rates of antiretroviral-therapy initiation, or the genetic characteristics of the infecting HIV-1 strains between treatment arms. Exploratory subgroup analyses showed nonsignificant trends toward efficacy in preventing infection in the highest risk (VE, 43%; n=247) and nonwhite (VE, 47%; n=914) volunteers (P=.10, adjusted for multiple subgroup comparisons).
There was no overall protective effect. The efficacy trends in subgroups may provide clues for the development of effective immunization approaches.
The first trial of the efficacy of a human immunodeficiency virus (HIV)–1 vaccine was conducted in North America and The Netherlands
between 1998 and 2003. This multicenter, randomized, placebo-controlled trial of a recombinant glycoprotein 120 vaccine included
5403 initially HIV-negative volunteers who were monitored for 3 years. The 368 subjects who acquired HIV-1 infection were
monitored for 2 years by use of the following postinfection end points: plasma HIV-1 RNA level (viral load), CD4+ lymphocyte count, initiation of antiretroviral therapy (ART), and HIV-1–related clinical outcomes. This article reports the
study results that pertain to the effect of vaccination on the postinfection end points. The time until initiation of ART
and the time until virologic failure or initiation of ART were similar in the vaccine arm and the placebo arm. The pre-ART
viral load and CD4+ lymphocyte count trajectories were also comparable between the groups. Evidently, the vaccine did not affect HIV-1 disease
progression
In randomized trials, the treatment assignment mechanism is independent of the outcome of interest and other covariates thought to be relevant in determining this outcome. It also allows, on average, for a balanced distribution of these covariates in the vaccine and placebo groups. Randomization, however, does not guarantee that the estimated effect is an unbiased estimate of the biological effect of interest. We show how exposure to infection can be a confounder even in randomized vaccine field trials. Based on a simple model of the biological efficacy of interest, we extend the arguments on comparability and collapsibility to examine the limits of randomization to control for unmeasured covariates. Estimates from randomized, placebo-controlled Phase III vaccine field trials that differ in baseline transmission are not comparable unless explicit control for baseline transmission is taken into account.
We outline a framework for causal inference in settings where assignment to a binary treatment is ignorable, but compliance with the assignment is not perfect so that the receipt of treatment is nonignorable. To address the problems associated with comparing subjects by the ignorable assignment - an "intention-to-treat analysis" - we make use of instrumental variables, which have long been used by economists in the context of regression models with constant treatment effects. We show that the instrumental variables (IV) estimand can be embedded within the Rubin Causal Model (RCM) and that under some simple and easily interpretable assumptions, the IV estimand is the average causal effect for a subgroup of units, the compliers. Without these assumptions, the IV estimand is simply the ratio of intention-to-treat causal estimands with no interpretation as an average causal effect. The advantages of embedding the IV approach in the RCM are that it clarifies the nature of critical assumptions needed for a causal interpretation, and moreover allows us to consider sensitivity of the results to deviations from key assumptions in a straightforward manner. We apply our analysis to estimate the effect of veteran status in the Vietnam era on mortality, using the lottery number that assigned priority for the draft as an instrument, and we use our results to investigate the sensitivity of the conclusions to critical assumptions.
Background. A vaccine is needed to prevent human immunodeficiency virus type 1 (HIV-1) infection. Methods. A double-blind, randomized trial of a recombinant HIV-1 envelope glycoprotein subunit (rgp120) vaccine was conducted among men who have sex with men and among women at high risk for heterosexual transmission of HIV-1. Volunteers received 7 injections of either vaccine or placebo (ratio, 2:1) over 30 months. The primary end point was HIV-1 seroconversion over 36 months. Results. A total of 5403 volunteers (5095 men and 308 women) were evaluated. The vaccine did not prevent HIV-1 acquisition: infection rates were 6.7% in 3598 vaccinees and 7.0% in 1805 placebo recipients; vaccine efficacy (VE) was estimated as 6% (95% confidence interval, -17% to 24%). There were no significant differences in viral loads, rates of antiretroviral-therapy initiation, or the genetic characteristics of the infecting HIV-1 strains between treatment arms. Exploratory subgroup analyses showed nonsignificant trends toward efficacy in preventing infection in the highest risk (VE, 43%; n = 247) and nonwhite (VE, 47%; n = 914) volunteers (P = .10, adjusted for multiple subgroup comparisons). Conclusions. There was no overall protective effect. The efficacy trends in subgroups may provide clues for the development of effective immunization approaches.
In 2003 Thompson and colleagues reported that daily use of finasteride reduced the prevalence of prostate cancer by 25% compared to placebo. These results were based on the double-blind randomized Prostate Cancer Prevention Trial (PCPT) which followed 18,882 men with no prior or current indications of prostate cancer annually for seven years. Enthusiasm for the risk reduction afforded by the chemopreventative agent and adoption of its use in clinical practice, however, was severely dampened by the additional finding in the trial of an increased absolute number of high-grade (Gleason score >/= 7) cancers on the finasteride arm. The question arose as to whether this finding truly implied that finasteride increased the risk of more severe prostate cancer or was a study artifact due to a series of possible post-randomization selection biases, including differences among treatment arms in patient characteristics of cancer cases, differences in biopsy verification of cancer status due to increased sensitivity of prostate-specific antigen under finasteride, differential grading by biopsy due to prostate volume reduction by finasteride, and nonignorable drop-out. Via a causal inference approach implementing inverse probability weighted estimating equations, this analysis addresses the question of whether finasteride caused more severe prostate cancer by estimating the mean treatment difference in prostate cancer severity between finasteride and placebo for the principal stratum of participants who would have developed prostate cancer regardless of treatment assignment. We perform sensitivity analyses that sequentially adjust for the numerous potential post-randomization biases conjectured in the PCPT.
Antibody-dependent enhancement is a general in vitro property of enveloped viruses. In certain circumstances, antibody-dependent enhancement is a bona fide pathophysiologic mechanism in vivo. There are several examples of virus disease of humans and animals in which incomplete or partial immunity can lead to enhanced infection and/or disease. In some cases, this appears to be attributable to antibody-dependent enhancement. Conversely, there are several examples of viruses for which in vitro antibody-dependent enhancement has been demonstrated, but for which vaccines have been used safely in millions of persons for decades. Finally, antibody-dependent enhancement of HIV is a genuine concern. However, to date there is no direct clinical, experimental, or epidemiological evidence that HIV enhancement can be operative in vivo. Such evidence should be actively sought.
Asymptomatic carriage of malaria parasites occurs frequently in endemic areas and the detection of parasites in a blood film from a febrile individual does not necessarily indicate clinical malaria. In areas of low and moderate endemicity the parasite prevalence in fever cases can be compared with that in community controls to estimate the fraction of cases which are attributable to malaria. In areas of very high transmission such estimates of the attributable fraction may be imprecise because very few individuals are without parasites. Furthermore, non-malarial fevers appear to suppress low levels of parasitaemia resulting in biased estimates of the attributable fraction. Alternative estimation techniques were therefore explored using data collected during 1989-1991 from a highly endemic area of Tanzania, where over 80 per cent of young children are parasitaemic. Logistic regression methods which model fever risk as a continuous function of parasite density give more precise estimates than simple analyses of parasite prevalence and overcome problems of bias caused by the effects of non-malarial fevers. Such models can be used to estimate the probability that any individual episode is malaria-attributable and can be extended to allow for covariates. A case definition for symptomatic malaria that is used widely in endemic areas requires fever together with a parasite density above a specific cutoff. The choice of a cutoff value can be assisted by using the probabilities derived from the logistic model to estimate the sensitivity and specificity of the case definition.
A new efficacy measure is developed for use in prevention trials of interventions which may affect both disease incidence and disease severity. We assign a severity score to each incident case and sum severity scores over all incident cases within each treatment group to create a burden-of-illness score for each treatment group. Efficacy is evaluated by the difference between the burden-of-illness per randomized subject in the two randomized treatment groups. Since the numbers of summands in each burden-of-illness score is a random variable, standard methods of analysis are not directly applicable. The asymptotic distribution and sampling properties of the net reduction in the burden-of-illness score are derived for trials designed to stop either after a fixed length of follow-up or after the occurrence of a fixed number of cases. We illustrate the method with data from a clinical trial of a human rotavirus vaccine.
Consider a randomized trial in which time to the occurrence of a particular disease, say pneumocystic pneumonia in an AIDS trial or breast cancer in a mammographic screening trial, is the failure time of primary interest. Suppose that time to disease is subject to informative censoring by the minimum of time to death, loss to and end of follow-up. In such a trial, the potential censoring time is observed for all study subjects, including failure. In the presence of informative censoring, it is not possible to consistently estimate the effect of treatment on time to disease without imposing additional non-identifiable assumptions. Robins (1995) specified two non-identifiable assumptions that allow one to test for and estimate an effect of treatment on time to disease in the presence of informative censoring. The goal of this paper is to provide a class of consistent and reasonably efficient semiparametric tests and estimators for the treatment effect under these assumptions. The tests in our class, like standard weighted-log-rank tests, are asymptotically distribution-free alpha-level tests under the null hypothesis of no causal effect of treatment on time to disease whenever the censoring and failure distributions are conditionally independent given treatment arm. However, our tests remain asymptotically distribution-free alpha-level tests in the presence of informative censoring provided either of our assumptions are true. In contrast, a weighted log-rank test will be an alpha-level test in the presence of informative censoring only if (1) one of our two non-identifiable assumptions hold, and (2) the distribution of time to censoring is the same in the two treatment arms. We also study the estimation, in the presence of informative censoring, of the effect of treatment on the evolution over time of the mean of repeated measures outcome such as CD4 count.
Consider a randomized trial in which time to the occurrence of a particular disease, say pneumocystis pneumonia in an AIDS trial or breast cancer in a mammographic screening trial, is the failure time of primary interest. Suppose that time to disease is subject to informative censoring by the minimum of time to death, loss to and end of follow-up. In such a trial, the censoring time is observed for all study subjects, including failures. In the presence of informative censoring, it is not possible to consistently estimate the effect of treatment on time to disease without imposing additional non-identifiable assumptions. The goals of this paper are to specify two non-identifiable assumptions that allow one to test for and estimate an effect of treatment on time to disease in the presence of informative censoring. In a companion paper (Robins, 1995), we provide consistent and reasonably efficient semiparametric estimators for the treatment effect under these assumptions. In this paper we largely restrict attention to testing. We propose tests that, like standard weighted-log-rank tests, are asymptotically distribution-free alpha-level tests under the null hypothesis of no causal effect of treatment on time to disease whenever the censoring and failure distributions are conditionally independent given treatment arm. However, our tests remain asymptotically distribution-free alpha-level tests in the presence of informative censoring provided either of our assumptions are true. In contrast, a weighted log-rank test will be an alpha-level test in the presence of informative censoring only if (1) one of our two non-identifiable assumptions hold, and (2) the distribution of time to censoring is the same in the two treatment arms. We also extend our methods to studies of the effect of a treatment on the evolution over time of the mean of a repeated measures outcome, such as CD-4 count.
Medical cost data often exhibit strong skewness and sometimes contain large proportions of zero values. Such characteristics prevent the analysis of variance (ANOVA) F-test and other frequently used standard tests from providing the correct inferences when the comparison of means is of interest. One solution to the problem is to introduce a parametric structure based on log-normal distributions with zero values and then construct a likelihood ratio test. While such a likelihood ratio test possesses excellent type I error control and power, its implementation requires a rather complicated iterative optimization program. In this paper, we propose a Wald test with simple computation. We then conduct a Monte Carlo simulation to compare the type I error rates and powers of the proposed Wald test with those of the likelihood ratio test. Our simulation study indicates that although the likelihood ratio test slightly outperforms the Wald test, the performance of the Wald test is also satisfactory, especially when the sample sizes are reasonably large. Finally, we illustrate the use of the proposed Wald test by analysing a clinical study assessing the effects of a computerized prospective drug utilization intervention on in-patient charges.
Two-part models arise when there is a clump of 0 observations in a distribution of continuous non-negative responses. Several methods for comparing two such distributions are available. These include the straightforward application of the z-test (or t-test), the Wilcoxon-Mann-Whitney rank sum test, the Kolmogorov-Smirnov test, and three tests that use a 2 degree of freedom chi(2) test based on the sum of the test for equality of proportions and a conditional chi(2) test for the continuous responses. This conditional test may be the z-test, the rank sum test, or the chi(2) corresponding to the Kolmogorov-Smirnov test. This study compares the size and power of several of these methods. All tests have the appropriate distribution under the null hypothesis if the distribution of the continuous part has finite moments. If it does not, the z-test has no power to detect any alternatives. It is found that the 2 d.f. tests are superior to the others when the larger proportion of 0 values corresponds to the population with the larger mean. If the reverse holds, the difference in the proportion of zeros reinforces the difference in means and some single-part models (the rank sum or Kolmogorov-Smirnov) do best. In those cases, the two-part models are not far behind, although statistically significantly poorer with respect to power. Published in 2001 by John Wiley & Sons, Ltd.
Many scientific problems require that treatment comparisons be adjusted for posttreatment variables, but the estimands underlying standard methods are not causal effects. To address this deficiency, we propose a general framework for comparing treatments adjusting for posttreatment variables that yields principal effects based on principal stratification. Principal stratification with respect to a posttreatment variable is a cross-classification of subjects defined by the joint potential values of that posttreatment variable tinder each of the treatments being compared. Principal effects are causal effects within a principal stratum. The key property of principal strata is that they are not affected by treatment assignment and therefore can be used just as any pretreatment covariate. such as age category. As a result, the central property of our principal effects is that they are always causal effects and do not suffer from the complications of standard posttreatment-adjusted estimands. We discuss briefly that such principal causal effects are the link between three recent applications with adjustment for posttreatment variables: (i) treatment noncompliance, (ii) missing outcomes (dropout) following treatment noncompliance. and (iii) censoring by death. We then attack the problem of surrogate or biomarker endpoints, where we show, using principal causal effects, that all current definitions of surrogacy, even when perfectly true, do not generally have the desired interpretation as causal effects of treatment on outcome. We go on to forrmulate estimands based on principal stratification and principal causal effects and show their superiority.
First generation HIV vaccines are not likely to provide complete protection from HIV-1 infection. Therefore, it is important to assess a vaccine's effect on disease progression and infectiousness of infected vaccinees in an efficacy trial; however, direct assessment of such vaccine effects is not feasible within current trial designs. Viral load in HIV-infected individuals correlates with infectiousness and disease progression in a natural history setting, and thus is a reasonable candidate for a surrogate outcome in vaccine efficacy trials. We consider comparisons of viral load of infected vaccinees to that of infected trial participants in the control group. Dramatic differences in viral loads between these groups would suggest a vaccine effect on disease progression. However, modest differences, even if statistically significant, could be consistent with an imperfect vaccine effect on susceptibility to infection and not an effect on disease progression, that is, a selection effect of the vaccine. Thus, the usual statistical tests for no difference between groups do not test the biologically and clinically relevant hypothesis. We propose a model for the possible selective effects of a vaccine and develop several test statistics for assessing a direct effect of the vaccine on viral load given this selection model. Finite sample properties of these tests are evaluated using computer simulations.
Vaccines with limited ability to prevent HIV infection may positively impact the HIV/AIDS pandemic by preventing secondary transmission and disease in vaccine recipients who become infected. To evaluate the impact of vaccination on secondary transmission and disease, efficacy trials assess vaccine effects on HIV viral load and other surrogate endpoints measured after infection. A standard test that compares the distribution of viral load between the infected subgroups of vaccine and placebo recipients does not assess a causal effect of vaccine, because the comparison groups are selected after randomization. To address this problem, we formulate clinically relevant causal estimands using the principal stratification framework developed by Frangakis and Rubin (2002, Biometrics 58, 21-29), and propose a class of logistic selection bias models whose members identify the estimands. Given a selection model in the class, procedures are developed for testing and estimation of the causal effect of vaccination on viral load in the principal stratum of subjects who would be infected regardless of randomization assignment. We show how the procedures can be used for a sensitivity analysis that quantifies how the causal effect of vaccination varies with the presumed magnitude of selection bias.
In many experiments, researchers would like to compare between treatments and outcome that only exists in a subset of participants selected after randomization. For example, in preventive HIV vaccine efficacy trials it is of interest to determine whether randomization to vaccine causes lower HIV viral load, a quantity that only exists in participants who acquire HIV. To make a causal comparison and account for potential selection bias we propose a sensitivity analysis following the principal stratification framework set forth by Frangakis and Rubin (2002, Biometrics58, 21-29). Our goal is to assess the average causal effect of treatment assignment on viral load at a given baseline covariate level in the always infected principal stratum (those who would have been infected whether they had been assigned to vaccine or placebo). We assume stable unit treatment values (SUTVA), randomization, and that subjects randomized to the vaccine arm who became infected would also have become infected if randomized to the placebo arm (monotonicity). It is not known which of those subjects infected in the placebo arm are in the always infected principal stratum, but this can be modeled conditional on covariates, the observed viral load, and a specified sensitivity parameter. Under parametric regression models for viral load, we obtain maximum likelihood estimates of the average causal effect conditional on covariates and the sensitivity parameter. We apply our methods to the world's first phase III HIV vaccine trial.
To support the design of the world's first proof-of-concept (POC) efficacy trial of a cell-mediated immunity-based HIV vaccine, we evaluate eight methods for testing the composite null hypothesis of no-vaccine effect on either the incidence of HIV infection or the viral load set point among those infected, relative to placebo. The first two methods use a single test applied to the actual values or ranks of a burden-of-illness (BOI) outcome that combines the infection and viral load endpoints. The other six methods combine separate tests for the two endpoints using unweighted or weighted versions of the two-part z, Simes', and Fisher's methods. Based on extensive simulations that were used to design the landmark POC trial, the BOI methods are shown to have generally low power for rejecting the composite null hypothesis (and hence advancing the vaccine to a subsequent large-scale efficacy trial). The unweighted Simes' and Fisher's combination methods perform best overall. Importantly, this conclusion holds even after the test for the viral load component is adjusted for bias that can be introduced by conditioning on a postrandomization event (HIV infection). The adjustment is derived using a selection bias model based on the principal stratification framework of causal inference.
The last several years have seen significant progress in the development of vaccines against malaria. Most recently, proof-of-concept of vaccine-induced protection from malaria infection and disease was demonstrated in African children. Pursued by various groups and on many fronts, several other candidate vaccines are in early clinical trials. Yet, despite the optimism and promise, an effective malaria vaccine is not yet available, in part because of the lack of understanding of the types of immune responses needed for protection, added to the difficulty of identifying, selecting and producing the appropriate protective antigens from a parasite with a genome of well over five thousand genes and to the frequent need to enhance the immunogenicity of purified antigens through the use of novel adjuvants or delivery systems. Insufficient clinical trial capacity and normative research functions such as local ethical committee reviews also contribute to slow down the development process. This article attempts to summarize the state of the art of malaria vaccine development.
About a quarter-century after the discovery of HIV, there is neither a marketable vaccine nor a credible expectation about when there will be one. Dr. Robert Steinbrook reports on recent setbacks to the development of AIDS vaccines.
Semiparametric methods for inferring treatment effects on outcomes defined only if a post-randomization event occurs
- Y Jemiai
Jemiai, Y. (2005). Semiparametric methods for inferring treatment ef-fects on outcomes defined only if a post-randomization event oc-curs. Ph.D. Thesis, Harvard Department of Biostatistics, Cam-bridge, Massachusetts.
Does finasteride affect the severity of prostate cancer? A causal sensi-tivity analysis Attributable frac-tion estimates and case definitions for malaria in endemic areas
- B E Shepherd
- M W Redman
- D P Ankerst
Shepherd, B. E., Redman, M. W., and Ankerst, D. P. (2008). Does finasteride affect the severity of prostate cancer? A causal sensi-tivity analysis. Journal of the American Statistical Association, to appear. Smith, T., Schellenberg, J. A., and Hayes, R. (1994). Attributable frac-tion estimates and case definitions for malaria in endemic areas. Statistics in Medicine 13, 2345–2358.
An analytic method for randomized trials with informative censoring: Part I
- Robins