Learning about what works within a field often takes time, money and effort. Multi-arm designs can make this process somewhat more efficient: multiple treatments are assigned in parallel and compared against the same control group, which reduces sample size and hence costs.

Suppose that we want to learn whether there is racial discrimination in the labor market by conducting an experiment. Companies from our population of size \(30\) are randomly assigned to receive a résumé from a white, black or Latino candidate. Companies are assigned to one of these three conditions with equal probabilities, and résumés only vary race and are otherwise identical. We define our outcome of interest as the difference in callbacks between experimental conditions.

In settings of multiple treatment arms, we could do a number of pairwise comparisons: across treatments and each treatment against control.

## Design Declaration

**M**odel: We specify a population of size \(N\) where a unit \(i\) has a potential outcome, \(Y_i(Z = 0)\), when it remains untreated and \(m\) \((m = 1, 2, ..., M)\) potential outcomes defined according to the treatment that it receives. The effect of each treatment on the outcome of unit \(i\) is equal to the difference in the potential outcome under treatment condition \(m\) and the control condition: \(Y_i(Z = m) -Y_i(Z = 0)\).**I**nquiry: We are interested in the effect of each treatment arm with respect to a comparison group.**D**ata strategy: We randomly assign \(k/N\) units to each of the three treatment arms.**A**nswer strategy: We fit a linear regression model with individual indicators for each of the treatments as covariates. The average treatment effect is equal to the regression coefficients, which are computed by subtracting the mean of the comparison group from the mean of each treatment group.

```
N <- 30
outcome_means <- c(0.5, 1, 2)
sd_i <- 1
outcome_sds <- c(0, 0, 0)
population <- declare_population(N = N, u_1 = rnorm(N, 0, outcome_sds[1L]),
u_2 = rnorm(N, 0, outcome_sds[2L]), u_3 = rnorm(N, 0, outcome_sds[3L]),
u = rnorm(N) * sd_i)
potential_outcomes <- declare_potential_outcomes(formula = Y ~ (outcome_means[1] +
u_1) * (Z == "1") + (outcome_means[2] + u_2) * (Z == "2") +
(outcome_means[3] + u_3) * (Z == "3") + u, conditions = c("1",
"2", "3"), assignment_variables = Z)
estimand <- declare_estimands(ate_Y_2_1 = mean(Y_Z_2 - Y_Z_1), ate_Y_3_1 = mean(Y_Z_3 -
Y_Z_1), ate_Y_3_2 = mean(Y_Z_3 - Y_Z_2))
assignment <- declare_assignment(num_arms = 3, conditions = c("1", "2", "3"
), assignment_variable = Z)
reveal_Y <- declare_reveal(assignment_variables = Z)
estimator <- declare_estimator(handler = function(data) {
estimates <- rbind.data.frame(ate_Y_2_1 = difference_in_means(formula = Y ~
Z, data = data, condition1 = "1", condition2 = "2"),
ate_Y_3_1 = difference_in_means(formula = Y ~ Z, data = data,
condition1 = "1", condition2 = "3"), ate_Y_3_2 = difference_in_means(formula = Y ~
Z, data = data, condition1 = "2", condition2 = "3"))
estimates$estimator_label <- c("DIM (Z_2 - Z_1)", "DIM (Z_3 - Z_1)",
"DIM (Z_3 - Z_2)")
estimates$estimand_label <- rownames(estimates)
estimates$estimate <- estimates$coefficients
estimates$term <- NULL
return(estimates)
})
multi_arm_design <- population + potential_outcomes + assignment +
reveal_Y + estimand + estimator
```

`diagnosis <- diagnose_design(multi_arm_design)`

Estimand Label | Estimator Label | N Sims | Bias | RMSE | Power | Coverage | Mean Estimate | SD Estimate | Mean Se | Mean Estimand |
---|---|---|---|---|---|---|---|---|---|---|

ate_Y_2_1 | DIM (Z_2 - Z_1) | 500 | -0.01 | 0.42 | 0.16 | 0.95 | 0.49 | 0.42 | 0.44 | 0.50 |

(0.02) | (0.01) | (0.01) | (0.01) | (0.02) | (0.01) | (0.00) | (0.00) | |||

ate_Y_3_1 | DIM (Z_3 - Z_1) | 500 | 0.00 | 0.44 | 0.90 | 0.95 | 1.50 | 0.45 | 0.45 | 1.50 |

(0.02) | (0.02) | (0.01) | (0.01) | (0.02) | (0.02) | (0.00) | (0.00) | |||

ate_Y_3_2 | DIM (Z_3 - Z_2) | 500 | 0.01 | 0.42 | 0.56 | 0.96 | 1.01 | 0.42 | 0.45 | 1.00 |

(0.02) | (0.02) | (0.02) | (0.01) | (0.02) | (0.02) | (0.00) | (0.00) |

The diagnosis of our design indicates that the regression provides unbiased estimates of the average treatment effect (ATE) in each arm. These estimates, however, are not so precise; The estimated standard deviation is large yielding wide confidence intervals that contain the true value of the ATEs more than 95% of the time.

## Using the Multi-Arm Designer

In R, you can generate a multi-arm design using the template function `multi_arm_designer()`

in the `DesignLibrary`

package by running the following lines, which load the package:

`library(DesignLibrary)`

We can then create specific designs by defining values for each argument. For example, we can create a design called `my_multi_arm_design`

where `N`

, `m_arms`

, and `outcome_means`

set to 80, 4, and `c(-0.2, 0.2, 0.1)`

, respectively, and other parameters use default values. To do so, we run the lines below.

`my_multi_arm_design <- multi_arm_designer(N = 80, m_arms = 4, outcome_means = c(-0.2, 0.2, 0.1))`

You can see more details on the `multi_arm_designer()`

function, its arguments, and default values, by running the following line of code:

`??multi_arm_designer`