Session: DAC-09-1: Design of Complex Systems

Paper Number: 143524

143524 - Developing Heuristics for Resource Allocation and Utilization in Systems Design:
A Hierarchical Reinforcement Learning Approach 

This paper presents an approach that integrates systems engineering principles with reinforcement learning (RL) to solve complex design problems under resource constraints. The approach presented in this paper integrates principles from systems engineering and hierarchical RL to address the challenge of learning heuristics. Product design processes often involve decomposing a large system into interconnected subsystems, with resources allocated across teams responsible for optimizing each subsystem. We formulate this process as a hierarchical multi-armed bandit (MAB) problem, where decisions are made at both the system level (allocating budget across subsystems) and subsystem level (selecting heuristics for sequential information acquisition). We employ Thompson Sampling, a Bayesian RL technique, to concurrently learn effective decision-making policies at each level. This hierarchical RL agent aims to maximize the overall system performance while adhering to fixed budgets for design evaluations. To demonstrate the approach, we present an illustrative example of a race car optimization in The Open Racing Car Simulator (TORCS) environment. Results show that the RL agent can learn to allocate resources strategically, prioritize the most impactful subsystems, and identify effective information acquisition heuristics for each subsystem team. The results also indicate that our method converges to high-performing car configurations with greater efficiency when compared to Bayesian Optimization.

Presenting Author: Vikranth Gadi Purdue University

Presenting Author Biography: Vikranth Gadi is a PhD Candidate in the School of Mechanical Engineering at Purdue University.

Authors:

Vikranth Gadi Purdue University
Zoe Szajnfarber The George Washington University
Jitesh Panchal Purdue University