By Clara Bicalho, Sisi Huang, and Markus Konrad
DeclareDesign is a collection of tools to help you “declare” and “diagnose” research designs. In a word, with the DeclareDesign packages you can quickly state the core analysis-relevant features of a research design, and in return you will get a diagnosis that tells you how well your design is likely to perform and how changes in the design could improve performance.
That’s the good news. The bad news is that ‘DeclareDesign’ is all set up in R. Great for some, but a dealbreaker for many.
That’s why we created DDWizard. DDWizard is a (shiny) web interface that lets you select any design from a growing library of templates, the DesignLibrary, customize them, and interrogate them. All without ever having to write any R code.
The wizard has a
Design tab and a
Diagnose tab. The
Design tab lets you load and customize a design; the
Diagnose tab lets you interrogate the design—generating nice plots showing how the design performs compared to other similar designs.
It’s still in beta but you can already do a lot. We would love it if you tried it out and sent us feedback to improve it (see the end of post).
To get started immediately with the simplest design, head to https://eos.wzb.eu/ipi/DDWizard/ and load the “Two Arm” design. Click over to the
Diagnose tab and select “Run diagnoses and update plot” to start learning about a very simple experiment with one treatment condition and one control condition.
Five things to do with the Declare Design Wizard
Here are five examples of things you can do with the wizard.
1 Create a design
The DeclareDesign library has templates (“designers”) for generating many common designs. In each case a complete design usually requires specifying a model of how the world works, inquiries (estimands), data strategies (sampling and assignment) and answer strategies (estimates) (see here for more detail).
Each designer in the library can make a class of designs based on the arguments you provide. The library includes simple experimental designers, as well as designers for factorial and multiarm experiments. It also has designers for observational studies, such as regression discontinuity designs or instrumental variables designs. And it even has some designers for simple qualitative studies, such as process tracing.
Suppose for instance you were interested in a 2-by-2 factorial design. The “Two by Two” designer in the library lets you specify the sample size, mean outcomes and standard deviations in each outcome cell, and the assignment probability for each factor (
prob B). You can also specify how your estimand weights the effects across arms (
weight B). Other designers allow different arguments; in the multiarm designer for example you can specify the number of arms you have and then provide arguments related to each arm. (Click the
Read more button for details on each designer).
Once you plug in the values you want, the interface generates the code for the design and you can start putting it to work.
2. Diagnose and improve your design
For any design selected, you can use the diagnosis tab to further inspect design properties such as power, bias, and root mean squared error (RMSE) under different values of one or more parameters. Interactive graphs and tables make it easy to visualize trends and trade-offs between alternative design specifications.
As you do so you can also generate fancy figures to display your diagnoses, varying up to three design parameters at at time.
To generate plots:
- Provide the values you want (you can give value to any parameters that are not “fixed”—see middle panel).
- Choose the parameter to appear on the x-axis and any other parameters to be displayed in the color aesthetics (optional) or along different plots (optional). (These last two options are only available when multiple parameters are being varied in the diagnosis).
- Download the graph as
.pngOR download the code needed to generate the plot and the relevant data as an
.Rfile. If you do this you can further tailor the design in any way you like.
4. Use it for teaching
Many designers in the DesignLibrary let you illustrate how design-based inferences depend on specific model assumptions. This can be useful for teaching. For example, the “Binary IV” design arguments can be defined in such a way that violates one or more assumptions for a strong instrumental variable. Similarly, the “Mediation” design lets you explore design properties under the violation of sequential ignorability and heterogeneous effects. Simulating and diagnosing such designs helps illustrate the magnitude and direction of bias and other properties when each assumption is violated separately, as well as what estimates are affected by or perform best under these scenarios.
5. Move beyond the library: Use the DDWizard as an interface for your own designs
For more advanced R and DeclareDesign users, the DDWizard can serve as an interactive interface for exploring the properties of any custom-made designer function.
To do this locally:
- Make a copy of the DesignLibrary repository here
- Add new designer function(s) to the library in the
"R/"directory of the package (tips) and rebuild the package locally;
- Download or clone the DDWizard repository on Github here and run
shiny::runApp()on the app directory.
A version of the DDWizard should then run locally with new function(s) from your DesignLibrary version appearing in the dropdown menu below
Even better, you can use pull requests to contribute your designers to the DesignLibrary package in which case they will be available for everyone via the web app (the DDWizard is routinely updated to include the latest version of the DesignLibrary). See instructions here