The ability to perform causal and counterfactual reasoning are central properties of human intelligence. Decision-making systems that can perform these types of reasoning have the potential to be more generalizable and interpretable. Simulations have helped advance the state-of-the-art in this domain, by providing the ability to systematically vary parameters (e.g., confounders) and generate examples of the outcomes in the case of counterfactual scenarios. However, simulating complex temporal causal events, such as those that existing in driving and vehicle navigation, is challenging. To help address this, we present a high-fidelity simulation environment that is designed for developing algorithms for causal discovery and counterfactual reasoning in this safety-critical context. A core component of our work is to introduce agency, such that it is simple to define and create complex scenarios using high-level definitions. The vehicles then operate with agency to complete these objectives, meaning low-level behaviors need only be controlled if necessary. We perform experiments with three state-of-the-art methods to create baselines and highlight the affordances of this environment. Finally, we highlight challenges and opportunities for future work.

Examples of routes in which cars are causally conntected by "leader-follower" routes:
Car 1 follows Car 2 , Car 3 follows Car 4 , Car 5 follows Car 6

Examples of routes in which cars are NOT causally connected by "leader-follower" routes:

CausalCity is a high-fidelity simulation environment, with AI agent controls for creating scenarios for causal and counterfactual reasoning. This environment reflects the real-world, safety critical scenario of driving. We want a simulation environment that enables controllable scenario generation that can be used for temporal and causal reasoning. This environment allows us to create complex scenarios including different types of confounders with relatively little effort. Within CausalCity you can introduce any number of vehicles, each of which is controlled at a high-level and has basic AI agency to maneuver avoiding collisions, navigating corners, stopping at traffic lights, etc. The high-level controls for each vehicle allow us to define each agent's behavior in a more abstract form controlling their sequence of actions (e.g., turn left at the next intersection, following that merge into the left lane etc.), their speed changes in different legs of the journey, their stopping distance behind other vehicles etc. Furthermore, our simulation can be used to introduce confounders to the environment such as the time of day and the weather conditions, which can be set both changing the visual appearance of the scene but also enabling causal relationships to be introduced (for example between vehicle speed or stopping distance and the amount of water on the roads). Also, traffic lights can be controlled at a low (the timing of each individual light) and high (transition timings for all the lights) levels. All these present opportunities for future work on causal reasoning.

Counterfactuals - Being a simulation CausalCity could be used for developing and testing methods for counterfactual reasoning. A scenario can be replicated several times with different interventions to create "ground-truth" data for a "What would have happened if...?" question.