Session: DAC-02-01-Artificial Intelligence and Machine Learning for Challenging Real-World Problems in Design Automation

Paper Number: 90065

90065 - Mean Squared Error May Lead You Astray When Optimizing Your Inverse Design Methods 

When performing time-intensive optimization tasks, such as those in Topology Optimization or Shape Optimization, researchers have turned to Machine Learned (ML) Inverse Design methods -- i.e., algorithms that predict the optimized geometry from input conditions -- to either replace or warm start traditional optimizers. Almost exclusively, such methods are trained and optimized to reduce the Mean Squared Error between a method's output and a ground truth training dataset of optimized designs, this being the obvious choice for traditional supervised learning. While convenient, we show that this choice may be myopic. Specifically, we compare two methods of optimizing the hyper-parameters of both a random forest (RF) and k-nearest neighbors (KNN) model for predicting the optimal topology in a 2D heat sink example. 

We show that under both direct Inverse Design as well as when warm starting further Topology Optimization (TO), using typical Mean Squared Error metrics produces less performant models than a proposed metric that directly evaluates the objective function, though both methods produce designs that are almost one order of magnitude better than a control condition that uses a uniform initialization common in TO. We also illustrate how warm starting TO with predicted solutions impacts both the convergence time, the type of solutions obtained during optimization, and the final designs. Sensitivity analyses on various model parameters demonstrate that the results are not dependent on other model hyperparameters. Overall, our initial results portend that researchers may need to revisit common choices for evaluating ID methods that subtly trade-off factors in how an ID method will actually be used. We hope our open-source dataset and evaluation environment will spur additional research in those directions

Presenting Author: Mark Fuge University of Maryland College Park

Presenting Author Biography: Mark Fuge is an Associate Professor of Mechanical Engineering at the University of Maryland, College<br/>Park, where he is also an affiliate faculty in the Institute for Systems Research and a member of the<br/>Maryland Robotics Center and Human-Computer Interaction Lab. His staff and students study<br/>fundamental scientific and mathematical questions behind how humans and computers can work together<br/>to design better complex engineered systems, from the molecular scale all the way to systems as large as<br/>aircraft and ships using tools from Applied Mathematics (such as graph theory, category theory, and<br/>statistics) and Computer Science (such as machine learning, artificial intelligence, complexity theory, and submodular optimization). He received his Ph.D. from UC Berkeley and has received an NSF CAREER<br/>Award, a DARPA Young Faculty Award, and a National Defense Science and Engineering Graduate<br/>(NDSEG) Fellowship. He has prior/current support from NSF, DARPA, ARPA-E, NIH, ONR, and<br/>Lockheed Martin. You can learn more about his research at his lab’s website: http://ideal.umd.edu.

Authors:

Shai Bernard University of Maryland College Park
Jun Wang University of Maryland College Park
Mark Fuge University of Maryland College Park