Decentralized signal control for multi-modal traffic network: A deep reinforcement learning approach

Authors

Jiajie Yu
Pierre-Antoine Laharotte
Yu Han
Ludovic Leclercq

Document Type

Journal Article

Publication Date

2023

Subject Area

mode - bus, mode - car, infrastructure - bus/tram lane, infrastructure - bus/tram priority, infrastructure - traffic signals, operations - traffic, planning - service improvement, planning - methods

Keywords

Artificial Neural Network, Bus Holding, Deep Reinforcement Learning, Multi-Modal Network, Traffic Signal Control

Abstract

Managing traffic flow at intersections in a large-scale network remains challenging. Multi-modal signalized intersections integrate various objectives, including minimizing the queue length and maintaining constant bus headway. Inefficient traffic signals and bus headway control strategies may cause severe traffic jams, high delays for bus passengers, and bus bunching that harms bus line operations. To simultaneously improve the level of service for car traffic and the bus system in a multi-modal network, this paper integrates bus priority and holding with traffic signal control via decentralized controllers based on Reinforcement Learning (RL). The controller agents act and learn from a synthetic traffic environment built with the microscopic traffic simulator SUMO. Action information is shared among agents to achieve cooperation, forming a Multi-Agent Reinforcement Learning (MARL) framework. The agents simultaneously aim to minimize vehicles' total stopping time and homogenize the forward and backward space headways for buses approaching intersections at each decision step. The Deep Q-Network (DQN) algorithm is applied to manage the continuity of the state space. The tradeoff between the bus transit and car traffic objectives is discussed using various numerical experiments. The introduced method is tested in scenarios with distinct bus lane layouts and bus line deployments. The proposed controller outperforms model-based adaptive control methods and the centralized RL method regarding global traffic efficiency and bus transit stability. Furthermore, the remarkable scalability and transferability of trained models are demonstrated by applying them to several different test networks without retraining.

Rights

Permission to publish the abstract has been given by Elsevier, copyright remains with them.

Comments

Transportation Research Part C Home Page:

http://www.sciencedirect.com/science/journal/0968090X

Recommended Citation

Yu, J., Laharotte, P. A., Han, Y., & Leclercq, L. (2023). Decentralized signal control for multi-modal traffic network: A deep reinforcement learning approach. Transportation Research Part C: Emerging Technologies, 154, 104281.