Session: CIE-03-01 Artificial Intelligence and Machine Learning in Design and Manufacturing

Paper Number: 72149

Start Time: August 17, 10:00 AM

72149 - Sketch-Based Mechanism Simulation Using Machine Learning 

This paper presents a machine learning approach for building an object detector for interactive simulation of planar linkages from handmade sketches and existing drawings found in patents and texts. While there are drawing apps available to help users make static drawings, including that of a linkage mechanism, it is both educational and instructive for them to see it come to life via automated simulation. Touch- and pen-input devices and interfaces have made sketching a more natural way for designers to express their ideas, especially during early design stages. Moreover, texts and patent drawings provide countless and diverse styles of drawing mechanisms, which upon detection could be animated without manually inputting them in a software. Modern machine learning algorithms for object recognition require an extensive number of training images. However, there are no datasets with bar linkages available online. Therefore, our first goal was to generate images of sketches similar to hand-drawn and use state-of-the-art deep generation models, such as $\beta$-VAE, to produce more training data from a limited set of images. The latent space of $\beta$-VAE was explored by linear interpolation between sub-spaces and by varying latent space's dimensions. This served two-fold objectives -- 1) examine the possibility of generating new synthesized images via interpolation and 2) develop insights in the dependence of latent space dimension on bar linkage parameters. t-SNE dimensionality reduction technique was implemented to visualize the latent space of a $\beta$-VAE in a 2D space. Training images produced by animation rendering were used for fine-tuning a real-time object detection system – YOLOv3.

Presenting Author: Anar Nurizada Stony Brook University

Authors:

Anar Nurizada Stony Brook University
Anurag Purwar Stony Brook University