Session: CIE-06-01: AL/ML AI and ML for System Engineering

Paper Number: 194273

194273 - Hierarchical Reinforcement Learning With Graph Neural Networks for Vehicle Routing Problem With Time Windows 

Vehicle routing problem with time windows (VRPTW) is widely encountered in operations, remaining computationally challenging due to combinatorial complexity and constraints. Recent advances in reinforcement learning have enabled direct construction of routing solutions from data efficiently. However, most existing methods focus primarily on customer visiting sequences while implicitly assuming fixed route departure times, which limits the exploration to discrete decisions and potentially excludes high-quality solutions. To address this limitation, this paper proposes a hierarchical reinforcement learning framework with graph neural networks (GNN-HRL). The proposed method jointly optimizes customer visiting order and shift start times through a hierarchical decision structure. A graph attention network is employed to encode node and edge information, while two decoders sequentially determine the next customer to visit and the shift start time using a Beta-distribution-based continuous action policy. The model is trained using the REINFORCE algorithm with a baseline reward to maximize the solution quality. Experiments on the Homberger benchmark and a real-world security dispatch dataset demonstrate that the proposed approach improves routing efficiency and generalization performance compared with a state-of-the-art neural routing solver. The results show that incorporating temporal scheduling decisions into the RL framework can significantly enhance solution quality and operational efficiency in VRPTW scenarios.

Presenting Author: G. Gary Wang Simon Fraser University

Presenting Author Biography: Prof. G. Gary Wang is a Professor in the School of Mechatronic Systems Engineering at Simon Fraser University, Canada. His research focuses on AI-driven optimization, metal additive manufacturing, intelligent optimization, and product and process Design. He has made significant contributions to data-driven engineering design and digital engineering methodologies, particularly in integrating artificial intelligence techniques with complex engineering system design.

Prof. Wang has published extensively in leading journals and conferences in design engineering and optimization, and his work has been widely cited in the fields of computational design, surrogate modeling, and engineering decision-making. He actively serves the research community through editorial and conference leadership roles and has supervised numerous graduate students and researchers in advanced engineering design methods.