[English | Japanese]

High-Speed Vision and Real-Time Sensory Information Processing

We are investigating systems, algorithms and architectures for robots and information systems to recognize the real world in real time, with high-speed visual processing technologies as the core competence.

Specifically, recent studies include the topics on

  • high-speed and real-time vision systems,
  • high-speed projector-camera systems,
  • real-time sensor networks.

Recent Results

(Not so recent actually... Please check out our YouTube movie list for more recent work)


Decentralized Topology Estimation for Adhoc Vision Sensor Network (2008-)


Establishing a logical network of vision sensors, in which each arc connects a pair of nodes sharing the same field of view, is important for cooperation of networked vision sensors deployed in an adhoc manner. In this study, matching of local image descriptors for the logical network estimation is decentralized and parallelized to multiple sensor nodes so that broadcast is avoided and network traffic is reduced.

  • Shin Kondo, Shingo Kagami, Koichi Hashimoto: Decentralized Topology Estimation in a Vision Sensor Netwrok, SICE Tohoku Chapter Meeting 246-13, 2008. (in Japanese)

Synchronizing Vision Sensors to Intensity-modulated Illumination (2008-)


A method of synchronizing the shutter timing of vision sensors to intensity-modulated illumination is proposed. The synchronization is achieved by constructing a software phase-locked loop within the camera, taking the images as input, and the shutter timing as output.

  • Lei Hou, Shingo Kagami, Koichi Hashimoto: Illumination-based Real-Time Contactless Synchronization of High-Speed Vision Sensors, 2008 IEEE International Conference on Robotics and Biomimetics, pp.1750-1755, 2009. (PDF)

Video Projection with Reduced Motion Blur using a High-Speed DMD Projector (2008-)


It has been recognized that hold-type displays such as Liquid Crystal Displays (LCD) and Digital Micromirror Devices (DMD) displays are disadvantageous in displaying video with fast motion when compared to impulse-type displays. Double-rate driving has been successfully applied in LCDs, but is difficult to be applied in DMD displays because they represent pixel brightness with pulse-width modulation (PWM). In this study, a method for motion blur reduction utilizing high-speed projection by a DMD projector is proposed. Subjective accessment experiments showed its significant difference (p < 0.001).

  • Tomoyuki Inoue, Shingo Kagami, Koichi Hashimoto: Video Projection with Reduced Motion Blur using a High-Speed DMD Projector, 2008 JSME Conference on Robotics and Mechatronics, 2P2-E15, 2008. (in Japanese)

Finding an accelerometer within a camera view (2007-)


Localization of mobile devices such as cellphones and PDAs, which is one of the key technologies in realizing e.g. context-aware services, is still an open problem. In this study, focusing on the fact that many of recent mobile devices are equipped with accelerometers, a method of finding correspondence between signals from an acceleromter and image-space trajectories from a fixed camera is proposed.

  • Osamu Shigeta, Shingo Kagami, Koichi Hashimoto: Identifying a Moving Object with an Accelerometer in a Camera View, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.3872-3877, 2008. (PDF)

High-speed visual feedback control over a high-load network (2007-)


High-speed visual feedback control at 1000 fps over a high-load IP network is demonstrated. The system consists of a high-speed vision system, a host PC that executes motor-control tasks on an RTOS, and an easy-to-construct configuration for real-time communication to deliver the high-speed visual feature information obtained by the vision system to the host PC. Rather than introducing OS-dependent technologies for real-time communication, we have developed a dedicated network processing unit that handles all of the UDP/IP and Ethernet processing instead of the host PC. The visual information packets are forwarded in priority to the other background traffic by off-the-shelf Gigabit Ethernet switches with the IEEE802.1Q/p QoS mechanism.

  • Daisuke Wako, Shingo Kagami and Koichi Hashimoto: Implementation and Evaluation of a Real-Time Network System based on the Standard Ethernet Technology, 2008 JSME Conference on Robotics and Mechatronics, 1A1-F19, 2008. (in Japanese)
  • Shingo Kagami, Shoichiro Saito, Takashi Komuro, Masatoshi Ishikawa: A Networked High-Speed Vision System for 1,000-fps Visual Feature Communication, First ACM/IEEE International Conference on Distributed Smart Cameras, pp.95-100, 2007. (PDF)

A dynamically reconfigurable architecture for high-speed vision systems (2007-)


A new reconfigurable processor architecture specialized for high frame rate visual processing is proposed. This architecture employs a 2-D mesh processing element (PE) array in which the PEs can be configured to operate as SIMD arrays or operation-pipeline trees depending on image processing algorithms so that maximum on-chip memory consumption is reduced. To achieve high on-chip memory utilization, the architecture features that the ALU network and the local memory network can be configured independently.

  • Nao Iwata, Shingo Kagami, Koichi Hashimoto: A Dynamically Reconfigurable Architecture Combining Pixel-Level SIMD and Operation-Pipeline Modes for High Frame Rate Visual Processing, IEEE International Conference on Field-Programmable Technology 2007, pp.321-324, 2007. (PDF)

Nearest-point tracking using a high-speed projector-camera system (2007-)


A 1,000-fps projector-camera system in which projected patterns are adaptively controlled according to image processing results is demonstrated. Adaptive structured light projection enables fast and efficient 3-D information acquisition. The prototype system is applied to tracking of the nearest point of an object, and experimental results show that the system successfully tracked an apex of a fast-moving target object.

  • Tomoyuki Inoue, Shingo Kagami, Joji Takei, Koichi Hashimoto, Kenkichi Yamamoto, Idaku Ishii: High-Speed Visual Tracking of the Nearest Point of an Object Using 1,000-fps Adaptive Pattern Projection, 2007 IEEE International Workshop on Projector-Camera Systems, 2007. (PDF)

3,000-fps shape measurement using a high-speed projector-camera system (2006-)


3D shape measurement based on a structured light method at high frame rates such as 3,350 fps with 256x256 pixels and 500 fps with 1024x1024 pixels is achieved using a high-speed DMD projector and a high-speed camera. The proposed structured light method features tracking of projected patterns in the image space considering discontinuous object surfaces.

[-] Collapse the video

Disassembling a projector:

  • Joji Takei, Shingo Kagami, Koichi Hashimoto: 3,000-fps 3-D Shape Measurement Using a High-Speed Camera-Projector System, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.3211-3216, 2007. (PDF)

Real-time Ethernet incorporating frame preemption (2006-)


A method of incorporating a frame preemption technique, by which a high-priority frame can suspend a low-priority ongoing frame, into Ethernet switches is proposed. It has been shown that sub-millisecond latency which is required in robot control over a network is achieved at the 100-Mbps wire speed.

  • Ryosuke Aoki, Shingo Kagami and Koichi Hashimoto: A Real-Time Network System Interoperable with the Standard Ethernet, 2006 JSME Conference on Robotics and Mechatronics, 2P2-C16, 2006. (in Japanese)

A smart-camera-type high-speed vision platform: VCS-IV (2002-)


We have been developing a smart camera platform for implementing high-speed visual processing over 1,000 fps. It features a modular structure in which individual functions are implemented in stackable separate units so that the system can be easily expanded.

  • Shingo Kagami, Takashi Komuro, Yoshihiro Watanabe and Masatoshi Ishikawa: VCS-IV: A Real-Time Vision System Using a Digital Vision Chip, IEICE Transactions on Information and Systems, Vol.J88-D-I, No.2, pp.134-142, 2005. (in Japanese) (PDF)
  • Shingo Kagami, Takashi Komuro, Masatoshi Ishikawa: A High-Speed Vision System with In-Pixel Programmable ADCs and PEs for Real-Time Visual Sensing, 8th IEEE International Workshop on Advanced Motion Control, pp.439-443, 2004. (PDF)