Interactive Learning of Structural Shape Descriptions from Automatically Generated Near-miss Examples Intelligent User Interfaces

by

Tracy Hammond and Randall Davis

Summary

The paper tackles the problem of creating shape descriptors for sketch recognition systems. The paper builds on LADDER, which is a languages to describe how shapes are drawn, displayed and edited. The author point out that recognition is based on what the shapes look like, not how they were drawn. The problem is that descriptors can be under constrained, recognizing shapes that the designer did not intend, or over constrained, not recognizing shapes that the author did intend.The big problem is that it is much easier to draw a shape that to create a formal description.
The authors propose a visual debugger of shape descriptions. The system uses a hand-drawn positive example to build the initial structural description. The system then asks the user to identify near-miss shapes as positive or negative. To check if the description is over constrained a description is created with the constraint negated, and then a shape is generated. If the user views this shape to be correct the constraint is unnecessary. To test for the under constraint a list of possible restraints is created. The negation of the constraint is added and users are again queried. This can determine if a constraint should be included. The paper also describes parameters that the authors used when generating new shapes.
The end goal is to produce descriptions that contain no unnecessary constraints, also aren't missing needed constraints.

Discussion

The section about the users drawing similar examples was spot on. As I did my user study most of the participants would draw the same line over and over again. So figure 1 gets a special place in my heart. I choose this paper to read for my user study report and also my final project. In my final project I need users to draw shapes that will be game pieces. Looking back, in my Plinko description, I softened constraints because the system seemed to be unable to recognize my drawing as what I intended. A system like this would be better, I could draw the shape I was interested in, and then through an iterative process the shape description was perfected. I could see this being a way to get new users started in creating descriptions. Let them draw a shape, build a description and then let them modify it. It gets around the blank page problem. When I am writing there is always a fear of the blank page, I usually write on top of an outline, and that gets me over the initial hump. I could see new users intimidate creating new shapes from scratch.

Citation

Hammond, T. and Davis, R. 2006. Interactive learning of structural shape descriptions from automatically generated near-miss examples. In Proceedings of the 11th international Conference on intelligent User interfaces (Sydney, Australia, January 29 - February 01, 2006). IUI '06. ACM Press, New York, NY, 210-217. DOI= http://doi.acm.org/10.1145/1111449.1111495