Analysis of “Visual Similarity of Pen Gestures”
Author:
A. Chris Long, Jr., James A. Landay, Lawrence A. Rowe, and Joseph Michiels
Summary:
Introduction:
People have trouble remember gestures, especially if they are similar, and gestures are difficult to design. These researchers designed a tool that can advise gesture designers. Ran experiments to measure gesture similarity, devised an algorithm to compute similarity, and then created design tool to provide advice.
Related Works:
Noted gesture recognition used in Newton, Palm Pilots, word processors, note-taking, air traffic control, etc. Researchers used single-stroke techniques.
Psychologists attempted to determine how people judged similarity of geometric primitives. Example: for rectangles, people used width and height. However, different people use different similarity metrics and the same person will even use different metrics for different stimuli (i.e might judge triangles by tilt instead of height and width). Apparently “the logarithm of the quantitative metrics…correlate with similarity”.
Discussion also covers multi-dimensional scaling (MDS), which reduces the number of input dimensions for viewing on a plot. Concerns were comparing participants to one another, how many dimensions to use, how to measure distance (Euclidean distance chosen), and assigning meaning to the axes.
User Study 1:
Created a set of empirically determined, dissimilar gestures. 10 participants shown groups of three gestures (aka a “triad” ) and asked to the pick the most dissimilar one from the other two. Goal 1: to determine what geometric properties influenced perceived similarity, and did so through MDS. Goal 2: to produce a model of gesture similarity to predict similarity between two given gestures, and did so with “regression analysis” (appears to be just feature vectors in a vector space). Did regression analysis for each dimension and then compared two dimensions at a time for comprehension sake. Derived equations for each dimension that would plot an arbitrary value from a new gesture. Their derived model reports gesture similarities with correlation 0.74.
User Study 2:
Sought to determine how certain features played into similarity (angle of bounding box, length and area of bounding box, rotation). Same setup and goals as before, but participants didn’t see all combinations of gestures. Did independent analysis of each of the three gesture sets to determine how similarity judgements were affected. Used results of study 1 to predict study 2 as well. Found that the bounding box angle is an important feature, and that alignment/non-alignment with the normal coordinate axes is significant to similarity. The other differences listed above were not significant enough.
Turned the two derived models from each study towards the other and had correlations of 0.56 for the first, and 0.51 for the second. Authors quick to admit this judging human perception with a computer program is hard, but happy with their results.
Author’s Discussion:
Said their models could still be used for a gesture design tool to advise gesture creation. Liked MDS and good for discovering “candidate predictors”, but it was limiting, so regression analysis best for creating model.
Discussion:
I’m still not confident that these researchers found a reliable model for predicting gestures similarities, or something that could be automated and reproduced. The proof of this coming from the fact that the models worked great on the data they were finessed for, but then dropped to the flip of a coin then the data was switched. The paper is older, so I’d be interested to see the tool they developed.
Compliments on their user studies as they appeared to be executed well. I might have argued to let the participants draw each shape of the triad and then decide which was the most dissimilar. However there it would have made the study take longer, and then you can’t measure qualitative feedback like “I chose this shape as the most dissimilar because it felt the most different to draw.”
Comments made elsewhere:
Yes the idea of letting the participants draw the gesture is good. By drawing actually they would be better judging the similarities in gesture. Its just like watching a football game on TV compared to playing the game on the field.
This would have taken longer and of course they had to pay more to the participants