Session: MR-02-02 - Theoretical & Computational Kinematics (A.T. Yang Symposium)

Paper Number: 90495

90495 - Transforming Hand-Drawn Sketches of Linkage Mechanisms Into Their Digital Representation 

This paper presents a machine learning approach for building an object detector for interactive simulation of planar linkages from hand-drawn or computer-generated sketches. Modern machine learning algorithms for object recognition require an extensive number of training images, but there are no data sets of planar linkages available online. In this paper, we present a method for generating images of sketches similar to hand-drawn ones and their automatic detection using a real-time object detection system based on a convolutional deep neural network called YOLO. The YOLO is trained on a class of real-life photos, which can not be used directly for predicting line drawings of hand-drawn sketches of mechanisms. Therefore, we use the principles of transfer learning and fine-tuning to retrain an existing YOLOv4 network to optimize the weights of this network and detect joints and links of hand-drawn mechanism sketches. A novel algorithm for establishing $n$-bar linkage mechanisms topology from the results obtained through YOLO is also presented. The results show that this algorithm performs well on hand-drawn sketches and suggest potential extension of it to textbook-style sketches and patent drawings. This work could help with conversions of hand-drawn sketches and text book and patent illustrations to their digital representation for effective communication, analysis, cataloging, and classification.

Presenting Author: Anar Nurizada Stony Brook University (SUNY)

Presenting Author Biography: Graduate Research Assistant in Mechanical Engineering at Stony Brook University

Authors:

Anurag Purwar Suny Stony Brook
Anar Nurizada Stony Brook University (SUNY)