Evaluating Action Durations for Adaptive Traffic Signal Control Based on Deep Q-Learning

Abstract

Adaptive traffic signal control is the control technique that adjusts the signal times according to traffic conditions and manages the traffic flow. Reinforcement learning is one of the best algorithms used for adaptive traffic signal controllers. Despite many successful studies about Reinforcement Learning based traffic control, there remains uncertainty about what the best actions to actualize adaptive traffic signal control. This paper seeks to understand the performance differences in different action durations for adaptive traffic management. Deep Q-Learning has been applied to a traffic environment for adaptive learning. This study evaluates five different action durations. Also, this study proposes a novel approach to the Deep Q-Learning based adaptive traffic control system for determine the best action. Our approach does not just aim to minimize delay time by waiting time during the red-light signal also aims to decrease delay time caused by vehicles slowing down when approaching the intersection and caused by the required time to accelerate after the green light signal. Thus the proposed strategy uses not just information of intersection also uses the data of adjacent intersection as an input. The performances of these methods are evaluated in real-time through the Simulation of Urban Mobility traffic simulator. The output of this paper indicate that the short action times increase the traffic control system performances despite more yellow signal duration. The results clearly shows that proposed method decreases the delay time.