Interactive Stickies: Low-Latency Projection Mapping for Dynamic Interaction with Projected Images on a Movable Surface

We propose a fast projection mapping system that enables physics-based interaction with projected images through the movement of a handheld surface onto which the images are projected. With the help of a 2D physics simulator engine, projected dynamic objects interact with static shapes that have been hand-drawn or printed on the surface under the effect of virtual gravitational and inertial forces generated according to the surface movement.

A distinct feature of the proposed system is that the graphics rendering of the projected images and the projection mapping of those images onto the surface take place at different frame rates. The graphics is rendered at up to 60 fps whereas the feedback control for the projection mapping runs at 400 fps, enabling the 2D workspace in which the objects interact to be perceived as being perfectly "stuck" onto the surface without delay.

Our prototype system is bulit upon a low-latency projection hardware and a projection mapping algorithm that we recently reported, and it works well with an uncalibrated projector-camera pair and a markerless planar surface with hand-drawn or printed textures.

• Shingo Kagami and Koichi Hashimoto, Interactive Stickies: Low-Latency Projection Mapping for Dynamic Interaction with Projected Images on a Movable Surface, 47th ACM SIGGRAPH International Conference and Exhibition on Computer Graphics and Interactive Techniques (SIGGRAPH 2020 Emerging Technologies), Virtual Event, U.S.A., August 17, 2020.
  • DOI: https://doi.org/10.1145/3388534.3407291
  • Supplemental video: https://youtu.be/vCk7bIPNT2Q