Analysis of “Sketch Recognition User Interfaces: Guidelines for Design and Development”
Comments Made Elsewhere:
Summary:
Paper claims that prior research as either been on sketch UI without much recognition or more solely on the sketch recognition technologies. Evaluates what makes a strong sketch recognition married with HCI, including how to handle situations, editing, errors, etc. Also looks at what evaluation method is appropriate (paper prototypes, etc.). Uses the domain of shapes for a PowerPoint presentation.
Application: Created a PowerPoint slide annotation application. Allowed “online editing mode” where user could hold pen for X milliseconds to switch from sketch to editing/selecting. Also allowed recognized and unrecognized shape creation (require UI element to switch). Shapes are cleaned up when put into PowerPoint.
System evaluation invovled three users asked to draw, draw and type text, draw and edit/delete.
Observations: (1) Don’t give feedback until the user explicitly says they’re done sketching. (2) Have to provide an obvious indication of recognized/unrecognized mode switching, (3) Only recognize one-domain at a time (multiple domain not feasible as of this paper), (4) Have pen-based editing for copy, paste, etc. (5) Support distinct editing and sketching gestures without a modal switch, (6) Use large buttons (which users prefer over keyboard shortcuts), (7) Always have real-time pen response.
Found that it was either paper prototyping or a full system. SkRUIs are different then normal testing because (1) they aren’t command-line based, and (2) there’s lots of freedom relative to just button clicking.
Discussion:
None of these papers seem to mention if the person is using a USB-attached tablet or a screen tablet, which very much adds to the HCI. And three people is not nearly enough in my opinion.
There domain seems relatively simple (relationship diagrams), though they did mention circuit diagram recognition as well. Having the user define the domain (i.e. LADDER) adds a whole other factor. After repeated use of LADDER, functionality liked delayed recognition would seem like a hindrance because I know the rules and how some shapes rely on others for recognition, so I would want them to be sure the system knew they were there.
I like the idea of online editing mode. Thinking of other gestures-based selection initiations besides holding, what if the first few points of a stroke drawn over an existing shape would automatically start dragging that shape. If the user pauses, he didn’t really mean to pick it up and it is released, turning the gesture from editing to sketching. If he did mean to pick it up and continues with his motion for some threshold, the shape will move with the pen until pen-up. Deleting could also occur in a similar way.
Incorporating the lasso/selection-box tool we read about previously would also be a good addition here.
Having immediate recognition on pen-up also seems like a solution to the recognized vs. unrecognized shape creation. If the system has a low confidence in the shape, leave it unrecognized instead of correcting it. The user can then explicitly request re-recognition or un-recognition of the shape (or is this giving to much uneeded functionality to the novice user?).
i did not understand the things you have mentioned about editing. Did you mean, it is difficult to understand the intentions behind the user’s pause.
Incorporating a lasso tool to select would be nice idea but in that case the mode switching should be taken care of.