Session: CIE-09-02 AMS/SEIKM: Artificial Intelligence and Machine Learning in Design and Manufacturing

Paper Number: 142360

142360 - Design, Development, and Testing of a Smart Hand Tool: Achieving Work Task Recognition Using Synthetic Data and Edge Intelligence 

This paper describes research toward developing smart hand tools that leverage artificial intelligence (AI) and sensors for use by human workers. Smart hand tools can provide useful feedback that can benefit human workers, contribute to worker training, and broaden participation in the skilled trade workforce. Specifically, the paper focuses on task recognition. Given the challenges of producing enough training data for machine learning (ML) using data purely from human-based testing, this paper shows how data synthetically-generated by a robot can be leveraged in the ML training process. The paper also demonstrates how fine-tuning ML models for individual physical tasks and workers can significantly scale up the benefits of using ML to provide this feedback. Experimental results show the effectiveness and scalability of this approach, including comparing test data size versus accuracy. In order for smart hand tools of the type introduced here to operate in real-time task recognition, as well as providing analytics on efficient and safe tool usage and operation, ML models need to be deployed `on tool'. This paper demonstrates how this can be accomplished by using a tinyML implementation. This paper provides a proof-of-concept for using automated platforms to help train smart tools, which will be essential given the wide range of uses for smart hand tools.

Presenting Author: Rachel New The University of Texas at Austin

Presenting Author Biography: Rachel (Chela) New is a doctoral student in the Walker Department of Mechanical Engineering. She entered the graduate school at the University of Texas in Fall of 2023, after completing a B.S. in Physics, with minors in Math and Computer Science at the University of Toronto in 2023. While at Toronto, Chela was a member of the FSAE team and also the autonomous car team. She also has industry work experience with Harmonic Bionics (Austin) and Endiatx as an embedded systems specialist.

Authors:

Rachel New The University of Texas at Austin
Carlos Salazar The University of Texas at Austin
Jose Bendana The University of Texas at Austin
Sundar Sripada v.s. The University of Texas at Austin
Sandeep Chinchali The University of Texas at Austin
Kenneth Fleischmann The University of Texas at Austin
Raul Longoria The University of Texas at Austin