Generates diagnosands from a design or simulations of a design.

```
diagnose_design(
...,
diagnosands = NULL,
sims = 500,
bootstrap_sims = 100,
make_groups = NULL,
add_grouping_variables = NULL
)
diagnose_designs(
...,
diagnosands = NULL,
sims = 500,
bootstrap_sims = 100,
make_groups = NULL,
add_grouping_variables = NULL
)
vars(...)
```

- ...
A design or set of designs typically created using the + operator, or a

`data.frame`

of simulations, typically created by`simulate_design`

.- diagnosands
A set of diagnosands created by

`declare_diagnosands`

. By default, these include bias, root mean-squared error, power, frequentist coverage, the mean and standard deviation of the estimate(s), the "type S" error rate (Gelman and Carlin 2014), and the mean of the inquiry(s).- sims
The number of simulations, defaulting to 500. sims may also be a vector indicating the number of simulations for each step in a design, as described for

`simulate_design`

- bootstrap_sims
Number of bootstrap replicates for the diagnosands to obtain the standard errors of the diagnosands, defaulting to

`100`

. Set to FALSE to turn off bootstrapping.- make_groups
Add group variables within which diagnosand values will be calculated. New variables can be created or variables already in the simulations data frame selected. Type name-value pairs within the function

`vars`

, i.e.`vars(significant = p.value <= 0.05)`

.- add_grouping_variables
Deprecated. Please use make_groups instead. Variables used to generate groups of simulations for diagnosis. Added to default list: c("design", "estimand_label", "estimator", "term")

a list with a data frame of simulations, a data frame of diagnosands, a vector of diagnosand names, and if calculated, a data frame of bootstrap replicates.

If the diagnosand function contains a `group_by`

attribute, it will be used to split-apply-combine diagnosands rather than the intersecting column names.

If `sims`

is named, or longer than one element, a fan-out strategy is created and used instead.

If the packages `future`

and `future.apply`

are installed, you can set `plan`

to run multiple simulations in parallel.

```
design <-
declare_model(
N = 500,
U = rnorm(N),
Y_Z_0 = U,
Y_Z_1 = U + rnorm(N, mean = 2, sd = 2)
) +
declare_inquiry(ATE = mean(Y_Z_1 - Y_Z_0)) +
declare_assignment(Z = complete_ra(N)) +
declare_measurement(Y = reveal_outcomes(Y ~ Z)) +
declare_estimator(Y ~ Z, inquiry = "ATE")
if (FALSE) {
# using built-in defaults:
diagnosis <- diagnose_design(design)
reshape_diagnosis(diagnosis, select = "Power")
}
if (FALSE) {
# Adding a group for within group diagnosis:
diagnosis <- diagnose_design(design,
make_groups = vars(significant = p.value <= 0.05),
)
diagnosis
diagnosis <- diagnose_design(design,
make_groups =
vars(effect_size =
cut(estimand, quantile(estimand, (0:4)/4),
include.lowest = TRUE)),
)
diagnosis
}
# using a user-defined diagnosand
my_diagnosand <- declare_diagnosands(absolute_error = mean(abs(estimate - estimand)))
if (FALSE) {
diagnosis <- diagnose_design(design, diagnosands = my_diagnosand)
diagnosis
get_diagnosands(diagnosis)
get_simulations(diagnosis)
}
# Using an existing data frame of simulations
if (FALSE) {
simulations <- simulate_design(designs, sims = 2)
diagnosis <- diagnose_design(simulations_df = simulations_df)
}
# If you do not specify diagnosands, the function default_diagnosands() is used,
# which is reproduced below.
alpha <- 0.05
default_diagnosands <-
declare_diagnosands(
mean_estimand = mean(estimand),
mean_estimate = mean(estimate),
bias = mean(estimate - estimand),
sd_estimate = sqrt(pop.var(estimate)),
rmse = sqrt(mean((estimate - estimand) ^ 2)),
power = mean(p.value <= alpha),
coverage = mean(estimand <= conf.high & estimand >= conf.low)
)
# A longer list of useful diagnosands might include:
extended_diagnosands <-
declare_diagnosands(
mean_estimand = mean(estimand),
mean_estimate = mean(estimate),
bias = mean(estimate - estimand),
sd_estimate = sd(estimate),
rmse = sqrt(mean((estimate - estimand) ^ 2)),
power = mean(p.value <= alpha),
coverage = mean(estimand <= conf.high & estimand >= conf.low),
mean_se = mean(std.error),
type_s_rate = mean((sign(estimate) != sign(estimand))[p.value <= alpha]),
exaggeration_ratio = mean((estimate/estimand)[p.value <= alpha]),
var_estimate = pop.var(estimate),
mean_var_hat = mean(std.error^2),
prop_pos_sig = estimate > 0 & p.value <= alpha,
mean_ci_length = mean(conf.high - conf.low)
)
```