This paper explores a new coordination strategy for an intelligent system of cooperative reinforcement learning agents that operate the signals of a congested traffic network. The enhanced coordination strategy is embedded in the standard Max-Plus algorithm and finds a direction of traffic that each agent should emphasize in the learning process to improve coordination considering one or more corridors. The main objective of the enhanced coordination is to increase network throughput, while also preventing queue spillovers and gridlocks. The proposed strategy reduces conflicting coordination between neighboring intersections from Max-Plus and applies two types of incentives in the learning process: 1) a direct factor in the cost function, and 2) a bias for the reward of all actions based on the largest difference between the coordinated and the competing demands along a corridor. A step-by-step description of the algorithm implementation is provided, and its effects are illustrated using a simulated network with oversaturated conditions. Compared to max-plus alone and no explicit coordination, the enhanced coordination increased network throughput and reduced the proportion of stopped vehicles. Additional scenarios in the same network demonstrated the algorithm performance when permitted turning movements are added, generating noise to the expected coordinating effects, and also when volumes are higher in one direction. Network performance is analyzed in terms of conventional measures such as total network throughput and queue management, as well as percentage of vehicles stopped over time and throughput along each corridor.
↧