Session: CIE-13-01 - SEIKEM: Design Informatics

Paper Number: 90895

90895 - Natural Language Processing for Content Analysis of Communication in Collaborative Design 

We address the problem of content analysis in text-based engineering design communication. Existing methods to characterize communication content in engineering design are manual or qualitative, which is tedious for large datasets. We argue that the traceability and understanding of design communication can be improved by automating tracking information flow and the identification of known topics. Such an approach can exceed current methods by modeling communication content with interaction frequency and representing expert-defined topics. Tagged messages can also enable structured interaction pathways and text-based feedback during design activities. Towards that goal, we formulate the characterization of communication messages as an intent classification task. We identify two intents---Intent 1 captures the presence and flow of information, Intent 2 captures specific topics about design parameters and objectives. We compare the predictive accuracy of convolutional LSTM, character-based convolutional LSTM, XLNet, and BERT models for the intent classification task. The results of our comparison show that the XLNet model predicts Intents 1 and 2 with 88\% and 81\% accuracy, respectively, on text data collected from 40 teams in a design experiment with university students. We analyze the differences in communication patterns between high- and low-performing teams. Time-series studies show that high-performing teams have more responsive communication and a higher consistency of information exchange.

Presenting Author: Joseph Thekinen University of Calgary

Presenting Author Biography: Dr. Joseph Thekinen is an Assistant Professor in the Department of Mechanical and Manufacturing Engineering at the University of Calgary. He received his Ph.D. from the School of Mechanical Engineering at Purdue University. He did his B.Tech and M.Tech from Indian Institute of Technology Kharagpur, with honors such as university silver medal (for topping academics in his department), best Masters thesis award, ABS scholarship, and J.P. Ghose Memorial award. His research focuses on enhancing human-AI partnership in decentralized sociotechnical systems using algorithmic game theory, machine learning, network science, and experimental techniques

Authors:

Sachin Lokesh Plaksha University
Ashish Chaudhari Massachusetts Institute of Technology
Joseph Thekinen University of Calgary
Jitesh Panchal Purdue University