Loading...
Organizing Committee
Harvard University
Brown University
Johns Hopkins Bloomberg School of Public Health
Abstract
Estimators of various causal or statistical quantities are usually constructed with a particular target population in mind, that is, the population about which the investigators intend to draw inferences (e.g., decide on the implementation of a treatment strategy or use algorithm-derived predictions). Typically, however, the data used for estimation comes from a population that differs from the target population. How to ensure or evaluate whether the estimates generalize to the target population is a question that has received substantial attention in many scientific disciplines, but with the fields not always connecting with one another on overlapping challenges and solutions. This workshop will bring together experts from different disciplines to present state-of-the-science methods to address generalizability and discuss key challenges, and open problems.
Loading...
Loading participant list in background...
Workshop Schedule
Friday, November 17, 2023
8:30 - 8:50 AM EST
Check In
11th Floor Collaborative Space
8:50 - 9:00 AM EST
ICERM Welcome
Welcome - 11th Floor Lecture Hall
Brendan Hassett, ICERM/Brown University
9:00 - 9:15 AM EST
Organizer Welcome
Welcome - 11th Floor Lecture Hall
Issa Dahabreh, Harvard University
Jon Steingrimsson, Brown University
Elizabeth Stuart, Johns Hopkins Bloomberg School of Public Health
9:15 - 9:45 AM EST
Evaluating Ex Ante Counterfactual Predictions Using Ex Post Causal Inference
11th Floor Lecture Hall
Virtual Speaker
Cyrus Samii, New York University
Session Chair
Jon Steingrimsson, Brown University
Abstract
We derive a formal, decision-based method for comparing the performance of counterfactual treatment regime predictions using the results of experiments that give relevant information on the distribution of treated outcomes. Our approach allows us to quantify and assess the statistical significance of differential performance for optimal treatment regimes estimated from structural models, extrapolated treatment effects, expert opinion, and other methods. We apply our method to evaluate optimal treatment regimes for conditional cash transfer programs across countries where predictions are generated using data from experimental evaluations in other countries and pre-program data in the country of interest.
10:05 - 10:20 AM EST
Coffee Break
11th Floor Collaborative Space
10:20 - 10:50 AM EST
Efficiently transporting average treatment effects using a sufficient subset of effect modifiers
11th Floor Lecture Hall
Speaker
Kara Rudolph, Columbia University
Session Chair
Jon Steingrimsson, Brown University
Abstract
We develop flexible and nonparametric estimators of the average treatment effect (ATE) transported to a new population that offer potential efficiency gains by incorporating only a sufficient subset of effect modifiers that are differentially distributed between the source and target populations into the transport step. We develop both a one-step estimator when this sufficient subset of effect modifiers is known and a collaborative one-step estimator when it is unknown. We discuss when we would expect our estimators to be more efficient than those that assume all covariates may be relevant effect modifiers and the exceptions when we would expect worse efficiency. We use simulation to compare finite sample performance across our proposed estimators and existing estimators of the transported ATE, including in the presence of practical violations of the positivity assumption. Lastly, we apply our proposed estimators to a large-scale housing trial.
11:10 - 11:40 AM EST
Understanding effect heterogeneity in observational and randomized studies of causality
11th Floor Lecture Hall
Speaker
Ivan Diaz, NYU
Session Chair
Jon Steingrimsson, Brown University
12:00 - 12:05 PM EST
Group Photo (Immediately After Talk)
11th Floor Lecture Hall
12:05 - 1:30 PM EST
Lunch/Free Time
1:30 - 2:00 PM EST
Generalizing trial evidence to target populations in non-nested designs: Applications to AIDS clinical trials
11th Floor Lecture Hall
Speaker
Ashley Buchanan, University of Rhode Island
Session Chair
Jon Steingrimsson, Brown University
Abstract
Comparative effectiveness evidence from randomized trials may not be directly generalizable to a target population of substantive interest when, as in most cases, trial participants are not randomly sampled from the target population. Motivated by the need to generalize evidence from two trials conducted in the AIDS Clinical Trials Group (ACTG), we consider weighting, regression, and doubly robust estimators to estimate the causal effects of HIV interventions in a specified population of people living with HIV in the USA. We focus on a non-nested trial design and discuss strategies for both point and variance estimation of the target population average treatment effect. Specifically in the generalizability context, we demonstrate both analytically and empirically that estimating the known propensity score in trials does not increase the variance for each of the weighting, regression, and doubly robust estimators. We apply these methods to generalize the average treatment effects from two ACTG trials to specified target populations and operationalize key practical considerations. Finally, we report on a simulation study that investigates the finite-sample operating characteristics of the generalizability estimators and their sandwich variance estimators.
2:20 - 2:50 PM EST
Extending Inferences to a Target Population Without Positivity
11th Floor Lecture Hall
Speaker
Paul Zivich, University of North Carolina at Chapel Hill
Session Chair
Jon Steingrimsson, Brown University
Abstract
To draw inferences from a sample to the target population, where the sample is not a random sample of the target population, various generalizability and transportability methods can be considered. Many of these modern approaches rely on a structural positivity assumption, such that all relevant covariate patterns in the target population are also observed in the secondary population of which the data is random sample of. Strict eligibility criteria, particularly in the context of randomized trials, may lead to violations of this positivity assumption. To address this concern, common methods are to restrict the target population, restrict the adjustment set, or extrapolate from a statistical model. Instead of these approaches, which all have concerning limitations, we propose a synthesis, or combination, of statistical (e.g., g-methods) and mathematical (e.g., microsimulation, mechanistic) models. Briefly, a statistical model is fit for the regions of the parameter space where positivity holds, and a mathematical model is used to fill-in, or impute, the nonpositive regions. For estimation, we propose two augmented inverse probability weighting estimators; one based on estimating the parameters of a marginal structural model, and the other based on estimating the conditional average causal effect. The standard approaches and the proposed synthesis method are illustrated with a simulation study and an applied example on the effect of antiretroviral therapy on CD4 cell count. The proposed synthesis method sheds light on a way to address challenges associated with the positivity assumption for transporting and causal inference more generally.
3:10 - 3:40 PM EST
Coffee Break
11th Floor Collaborative Space