Analysis of “Multimodal Collaborative Handwriting Training for Visually-Impaired People”
Comments Made Elsewhere:
Summary:
Authors sought to create a collaborative teacher-student system for visually-impaired children to learn handwriting and signatures. Uses haptic and auditory feedback. Most people don’t realize that blind people can’t write. With speech synthesizers, a lot less emphasis is put on it, too. People with visual disabilities learn best through tactial and auditory feedback.
Systems works by teacher drawing the appropriate stroke that is then felt by the student using a haptic device. Previous work had a device that would physically move the person’s hand for them to teach, but it was not as received as well by non-sight participants. A second study did better when combining haptic feedback with auditory pitches based on location. Combining this with real-time control of the haptic feedback by a teacher proved useful (as opposed to a constant-rate computered genereated haptic feedback).
Their system (McSig) allows a teacher to sit at a tablet PC and draw a shape, or create stencils for latter, that the student draws. A device then guides the visually-impaired child along the stroke, using Dutch paper (device that raises when drawing on) and auditory pitch all as multimodal feedback.
Used an open source library from previous work to limit the trajectory of the force feedback to safe levels. Performed usability testing with visually imparied adults. Performed system evaluation with visually impaired children in 20 minute sessions. A student was pre-tested to see if he knew the letter, then guided through it. All letters were single stroke, though multi-stroke was supported. A longevity study is next.
Discussion:
A very interesting and worthwhile application of the sketch domain to teaching. Not much sketch recognition is going on with the computer itself as the author’s commented that the Microsoft handwriting recognition seemed to fail regularly, so the auditory feedback was turned off. But its more sketch recognition of a different type, that of an impaired human through aide of a computer. The problem is still the same (i.e. a human with sight can look at a sketch and understand it, but one without sight can not).
The success of their system is repetition with appropriate feedback, either verbally from a teacher or through a computer generated sound, force, or tactial shape. We, as humans, are able to learn, we just need enough information.
This paper also seems interesting in light of my final project where I desire to teach people to draw. Again, something that can be done with proper repetition, feedback, and the building of confidence (I believe). It’s my hypothesis that most people shy away from drawing because they believe they can’t do it, but given a system that helps them establish spatial relationships with visual feedback and “rewards”, a person could be taught to draw, say, a favorite Disney character.
Here they have a teacher provide real-time drawing feedback to the student, which they found to be important in light of prior research that had the computer provide this feedback in a constant-time after a teacher had input the shape. This seems interesting to me also for my final project, and I’m considering it as a phase to complete for the user.