Randomized controlled trials

As governments seek to incorporate more empirical evidence into their decision-making, many are conducting Randomized Controlled Trials (RCTs), the type of experiments used to approve medical treatments, to test the effectiveness of their programs. Transportation planners could also use RCTs to better model human behavior or evaluate design decisions, policies and programs.

Randomized Controlled Trials (RCTs) are simple experiments where one randomly-selected group receives a treatment of some kind, and is then compared to a another randomly-selected untreated group. Medical researchers have long used RCTs to test the effectiveness of treatments, but the technique is only recently becoming common in public policy.

For example, development economists are adopting RCTs as a tool to evaluate poverty alleviation policies before applying them at a large scale. In some cases, such trials have shown well-intended programs to be ineffective or even harmful; at a minimum, they have shown that human response to policies can be very sensitive to context.

The behavioral parameters in transportation forecasting models, such as the value of time and modal preferences, can also be context-specific. For this reason, transportation planners often observe actual travel behavior to estimate those parameters for a specific project. When the data required for robust revealed-preference studies are not available, planners rely on stated-preference surveys or studies from other cities, which may be less accurate. To test designs, policies or programs that can be implemented on a small scale with relatively little investment, transportation planners can use RCTs to create data that reveal actual behavior in the local context.
As one example from the passenger rail sector, planners could use RCTs to measure the ridership impact of specific amenities, such as on-board Wi-Fi. An agency could mail to a random group of residents a code to use Wi-Fi for free on commuter trains. If the group that received the code then rode the train more than a random group that did not receive a code, the agency might consider providing free Wi-Fi for all its passengers.

As another example, RCTs could be used to evaluate Transportation Demand Management (TDM) programs designed to discourage commuters from driving alone to work. TDM measures, such as subsidized transit passes, showers for cyclists and parking fees, can all be tested with RCTs, enabling transportation management associations to focus on providing the programs proven to work in their context.

In medical research, RCTs can raise serious ethical concerns, since experimental treatments may cause harmful side-effects and effective treatments are withheld from the control group, which could potentially benefit from them. Similarly, RCTs could also cause unrest amongst participants when applied in public policy. Some trials could create unrealistic expectations amongst travelers which may cause complaints if, following the trial, a successful program fails to be rolled out. They may also be misleading if they are applied to programs which require a minimum degree of implementation or other complimentary policies to become effective. For example, a single bike rack is unlikely to cause a huge uptake in cycling. However, a series of bike racks connected by a network of new bike paths are more likely to have an impact. While it’s important to consider these potential issues, they should not be overstated and nullify the benefits of the method.

Transportation planners already utilize a variety of techniques to inform decision-making with empirical evidence. RCTs could be an additional tool for investigating how scalable transportation measures affect travel behavior in a particular context.