Upgrading from DesignLibrary 0.1: what changes, what does not, and why
Source:vignettes/upgrading.Rmd
upgrading.RmdThe short version
DesignLibrary 2.0 replaces the designer functions of 0.1 with a
library of self-contained design files, read through
make_design(). It requires DeclareDesign, fabricatr,
estimatr, and randomizr 2.0.
All fifteen 0.1 designers keep their names and their argument names,
with two exceptions: block_cluster_two_arm_designer() loses
six arguments, and three arguments (label_E1 and
label_E2 of process_tracing_designer(), and
args_to_fix of every designer) are accepted but ignored
with a warning. Each now builds a library design, so
two_arm_designer(N = 40) returns the same design as
make_design("two_arm_flexible", N = 40). Most 0.1 scripts
that call a designer and diagnose the result run unchanged.
There are two types of break:
Removed functions and removed arguments stop with an error that names the replacement.
A handful of changes do not error. These aries if there is a changed default, renamed inquiries, or renamed estimator terms or different step names. See Porting an existing script for more.
What does not change
A 0.1 script like this one runs as written. As in 0.1,
library(DesignLibrary) also attaches DeclareDesign,
fabricatr, estimatr, and randomizr.
library(DesignLibrary)
design <- two_arm_designer(N = 100, ate = 0.25)
set.seed(343)
draw_estimates(design)
#> # A tibble: 1 × 12
#> term estimate std.error statistic p.value conf.low conf.high df outcome
#> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <chr>
#> 1 Z -0.0159 0.171 -0.0929 0.926 -0.356 0.324 98 Y
#> # ℹ 3 more variables: estimator <chr>, inquiry <chr>, estimand <dbl>What carries over:
- The fifteen designer names:
two_arm_designer(),two_arm_attrition_designer(),two_arm_covariate_designer(),pretest_posttest_designer(),randomized_response_designer(),mediation_analysis_designer(),multi_arm_designer(),two_by_two_designer(),factorial_designer(),block_cluster_two_arm_designer(),regression_discontinuity_designer(),spillover_designer(),cluster_sampling_designer(),binary_iv_designer(), andprocess_tracing_designer(). - Their argument names and defaults, except for
block_cluster_two_arm_designer()and the defaultNofmulti_arm_designer(). - Inquiry and estimator labels, except for the cases listed below.
- The designs themselves. Seven of the designers became new library
files (
two_arm_covariate,regression_discontinuity_polynomial,spillover,cluster_sampling,binary_iv,process_tracing_bayes, andfactorial) flattened from their 0.1 code, and at the same arguments and seed each draws what its 0.1 designer drew.
What changes, and why
Removed functions
Four aliases that 0.1 had already deprecated are gone, along with two helpers. In each case you get an informative message.
simple_two_arm_designer(N = 50)
#> Error in `.defunct_01()`:
#> ! `simple_two_arm_designer()` was removed in DesignLibrary 2.0.0.
#> ℹ Use `two_arm_designer()`, which takes the same arguments.| Removed | Use instead |
|---|---|
simple_two_arm_designer() |
two_arm_designer(), same arguments |
simple_factorial_designer() |
two_by_two_designer(), same arguments |
simple_iv_designer() |
binary_iv_designer(), same arguments |
simple_spillover_designer() |
spillover_designer(), same arguments |
get_design_code() |
get_code("<id>") |
match.call.defaults() |
match.call() and formals()
|
In 0.1 every designer pasted its own source code into an attribute of
the design it returned, and get_design_code() read that
attribute back. In 2.0 the source of a design is a file in the package,
so there is nothing to paste: get_code() prints the file.
Each designer’s help page names the id of the file it builds.
get_design_code(two_arm_designer())
#> Error in `.defunct_01()`:
#> ! `get_design_code()` was removed in DesignLibrary 2.0.0.
#> ℹ Designs no longer carry their code as an attribute.
#> ℹ Use `get_code()` with a library design id, for example `get_code("two_arm_flexible")`.
#> ℹ `list_designs()` lists the ids, and each designer's help page names the id it builds.
get_code("two_arm_flexible")
#> design <-
#> declare_parameters(
#> N = 100,
#> assignment_prob = 0.5,
#> control_mean = 0,
#> control_sd = 1,
#> ate = 1,
#> treatment_sd = 1,
#> rho = 1
#> ) +
#> declare_model(
#> N = N,
#> u_0 = rnorm(N),
#> u_1 = rnorm(n = N, mean = rho * u_0, sd = sqrt(pmax(0, 1 - rho^2))),
#> potential_outcomes(
#> Y ~ control_mean +
#> (1 - Z) * (u_0 * control_sd) + Z* (u_1 * treatment_sd + ate)
#> )
#> ) +
#> declare_inquiry(ATE = mean(Y_Z_1 - Y_Z_0)) +
#> declare_assignment(Z = complete_ra(N, prob = assignment_prob)) +
#> declare_measurement(Y = reveal_outcomes(Y ~ Z)) +
#> declare_estimator(Y ~ Z, inquiry = "ATE")The description, definitions, and
shiny_arguments attributes went with code. Now
get_args() and design_info() describe a
design’s parameters.
block_cluster_two_arm_designer() takes new
arguments
The 0.1 designer derived its level sizes from N and its
three shock standard deviations from one total sd, through
defaults that referred to each other. Its defaults also produced a
design with no blocking and no clustering (one block of 100 single-unit
clusters) and no effect. The 2.0 designer takes the level sizes and the
shocks directly, and its defaults are a blocked and clustered design
with an effect to find: 20 blocks of 4 clusters of 10 units, each shock
with standard deviation 0.577, and ate = 0.2.
The 0.1 arguments N, sd,
sd_i_1, rho, control_mean, and
treatment_mean are errors that say what to set instead.
block_cluster_two_arm_designer(N = 200, rho = 0.5)
#> Error in `block_cluster_two_arm_designer()`:
#> ! `block_cluster_two_arm_designer()` no longer takes `N`, `rho` (DesignLibrary 0.1).
#> ℹ Set the sample size through `N_blocks`, `N_clusters_in_block`, and `N_i_in_cluster`.
#> ℹ Treatment and control potential outcomes are perfectly correlated; `rho` is not supported.
design <- block_cluster_two_arm_designer(
N_blocks = 5, N_clusters_in_block = 4, N_i_in_cluster = 10, ate = 0.1
)assignment_probs takes one probability rather than one
per block, and verbose is ignored with a warning.
Because the defaults changed,
block_cluster_two_arm_designer() with no arguments is a
different design in 2.0, and nothing warns about it.
multi_arm_designer() has a larger default sample
As in 0.1, the designer declares an inquiry for every pair of arms
and estimates each by a difference in means between its two arms, under
labels like "DIM (Z_2 - Z_1)". The default N
went from 30 to 90, so that each of the three arms has 30 units rather
than 10.
design <- multi_arm_designer(m_arms = 3)
set.seed(343)
draw_estimates(design)[, c("estimator", "term", "estimate", "inquiry")]
#> # A tibble: 3 × 4
#> estimator term estimate inquiry
#> <chr> <chr> <dbl> <chr>
#> 1 DIM (Z_2 - Z_1) Z2 -0.621 ate_Y_2_1
#> 2 DIM (Z_3 - Z_1) Z3 -0.350 ate_Y_3_1
#> 3 DIM (Z_3 - Z_2) Z3 0.271 ate_Y_3_2Two renamings that change no values
mediation_analysis_designer()’s inquiries
Indirect_0 and Indirect_1 are now
Mediator_Effect_0 and Mediator_Effect_1. Each
is the effect of the mediator on the outcome with treatment held at 0 or
at 1.
draw_estimands(mediation_analysis_designer())
#> # A tibble: 7 × 2
#> inquiry estimand
#> <chr> <dbl>
#> 1 FirstStage 0.335
#> 2 Mediator_Effect_0 0.4
#> 3 Mediator_Effect_1 0.4
#> 4 Controlled_Direct_0 0.5
#> 5 Controlled_Direct_1 0.5
#> 6 Natural_Direct_0 0.5
#> 7 Natural_Direct_1 0.5process_tracing_designer() fixes its estimator labels at
"Straw in the Wind" and "Smoking Gun".
design <- process_tracing_designer(label_E1 = "Hoop")
#> Warning: label_E1 and label_E2 are ignored; the estimator labels are fixed.
args_to_fix is ignored
0.1 designers could fix arguments into the returned design so that
expand_design() could not vary them. In 2.0 every parameter
stays redesignable on the design object, and a non-empty
args_to_fix warns.
design <- two_arm_designer(args_to_fix = "N")
#> Warning: args_to_fix is ignored; parameters remain redesignable on the design
#> object.Designs are now all DeclareDesign 2.0 designs
DeclareDesign 2.0 has no separate potential outcomes or reveal steps:
potential outcomes live in declare_model() and are revealed
in declare_measurement(). Some step names differ as a
result. factorial_designer()’s inquiry step, for example,
was "ATE" and is now "inquiry".
names(factorial_designer())
#> [1] "parameters" "model" "assignment" "measurement" "inquiry"
#> [6] "estimator"Inquiry and estimator labels are otherwise those of 0.1. Code
that picks a step by name or by position (design[[3]])
needs checking, since the position of a step can change even
where its name does not.
Sweeping a parameter
expand_design() still works but is deprecated in
DeclareDesign 2.0. Pass a vector to redesign() instead,
which needs no designer function at all.
designs <- redesign(two_arm_designer(), N = c(50, 100))
length(designs)
#> [1] 2One caution: process_tracing_designer() checks that
cor_E1E2_H and cor_E1E2_not_H are compatible
with the clue probabilities, and redesign() does not. An
incompatible value passed through redesign() fails only
when the design is run, with “negative probability” from the measurement
step.
Porting an existing script
Search the script for each pattern in the left column.
| Search for | Replace with | If you do not |
|---|---|---|
simple_two_arm_designer,
simple_factorial_designer, simple_iv_designer,
simple_spillover_designer
|
two_arm_designer, two_by_two_designer,
binary_iv_designer, spillover_designer
|
error naming the replacement |
get_design_code( |
get_code("<id>") |
error naming the replacement |
match.call.defaults( |
match.call() and formals()
|
error |
block_cluster_two_arm_designer( with N,
sd, sd_i_1, rho,
control_mean, or treatment_mean
|
N_blocks, N_clusters_in_block,
N_i_in_cluster, sd_block,
sd_cluster, sd_i, ate
|
error naming the replacement |
block_cluster_two_arm_designer() relying on
defaults |
state the 0.1 sizes and ate = 0 explicitly |
silently a different design |
multi_arm_designer( without N
|
N = 30 for the 0.1 default |
silently a larger sample |
Indirect_0, Indirect_1
|
Mediator_Effect_0, Mediator_Effect_1
|
filter returns no rows |
design[[ with a number, or step names such as
"ATE"
|
names(design) to find the step |
wrong step, or error |
args_to_fix |
drop it | warning |
expand_design( |
redesign(designer(), arg = values) |
deprecation warning |
make_design(design = |
make_design("<id>") |
error |
The rows in bold are the ones to search for deliberately, since nothing will point you to them.
What is new
The big change is that the key files contained in the library are
simple design scripts rather than “designer functions.” This makes it
much easier to inspect a design and to contribute a design. In addition
the designs make use of the the new declare_parameters
functionality from DeclareDesign 2.0 which makes it easier to contribute
modifiable designs.
The library holds 75 designs, including the declarations from Research Design in the Social Sciences.
-
list_designs()browses the library -
make_design()builds a design and changes its parameters -
get_args()lists what can be changed -
get_code()prints the source -
designs_shiny_app()opens the library in a browser -
vignette("DesignLibrary")gives you a tour.