Introduction to Sketch Recognition

by Tracy Hammond and Kenneth Mock

Summary

The paper discusses pen based computing and sketch recognition with particular attention paid towards educational environments. Background is given on the creation of the field, specifically Ivan Sutherland's early contributions. Tablet PCs are described as well as the distinction between passive (that which requires contact) and active (that which requires a special stylus) digitizers. Also described are different pen input devices (tablets, UMPC, drawing tablets) and the software that can be used with such input means. The paper focuses then on this type of input in educational situations. How lectures can be annotated, edited, and prepared using pen input is discussed.

The paper then moves into a discussion of sketch recognitions and lists domains - art, music, chemistry, mathematics, finite-state machines, UML, COA - in such recognition makes sense. The paper also gives a brief overview of FLUID (LADDER and GUILD), and comments on how sketches can be tied to higher-level functionality. The authors describe how shape definitions are created in LADDER using primitive shapes and constraints. Also noted are two challenges with pen-input: 1) recognization systems being able to handle varied input, 2) administrative concerns. The authors then discuss two case studies of tablet use in classrooms. The paper concludes noting the increase in TabletPC's adoption in educational markets.

Discussion

Both of the case studies take place in Math classes: it would have been better to have each case study focus on a different subject. The paper gives a good overview of current pen input devices, those devices' roles in an education environment, and what is happening in sketch recognition. The work was a bit fluffy, but the author openly admits that.

Something that concerns me about sketch recognition is that it seems heavily domain specific. I have difficulty moving beyond the "that's neat" feeling into viewing it as something that can have a fundamental shift in the way people interact with computers.

Play this game, how would you want to interact with a computer if anything was possible? Paper has some great benefits, but far too many of them I feel are physical not the user interface. By physical I mean the lightness, the contrast. In pen input I lose some of those nice physical qualities but I gain the nice digital ones (the ability to edit, the ability to reproduce, the ability to collaborate over a large distance).