[English | Japanese]

Animated Stickies: Fast Video Projection Mapping onto a Markerless Plane through a Direct Closed-Loop Alignment

Shingo Kagami and Koichi Hashimoto
IEEE Transactions on Visualization and Computer Graphics
IEEE International Symposium on Mixed and Augmented Reality (ISMAR2019)


Advancement of high-speed projection technologies has enabled projection mapping of video contents onto fast moving surfaces. However, most of existing techniques rely on precise position adjustment or calibration of tracking cameras with respect to the projector, resulting in expensive installation and maintenance costs.

Adopting a closed-loop approach, in which not only the surface texture but also the projected image is tracked by a camera, is effective in removing the need for position adjustment or calibration, but it introduces a difficulty in tracking because of the interference between the surface texture and the projected light. Therefore, this approach has been adopted for high-speed applications only in limited situations such as mapping onto a solid-color surface with a known shape or mapping onto a surface with fiducial markers.

In this study, we designed fiducial patterns to be inserted into a fast flapping sequence of binary frames of the Digital Micromirror (DMD) projector, which allows the simultaneous tracking of the surface texture and a fiducial geometry separate from a single image captured by the camera. Therefore, the proposed method is free from a calibration or position adjustment between the camera and projector.

Our implementation on a CPU runs at 400 frames/s and combining it with our low-latency DMD projector enables arbitrary video contents supplied through HDMI to be "stuck" onto a variety of textured surfaces.