High-Speed Vision and Real-Time Sensory Information Processing

We are investigating systems, algorithms and architectures for robots and information systems to recognize the real world in real time, with high-speed visual processing technologies as the core competence.

Specifically, recent studies include the topics on

• high-speed and real-time vision systems,
• high-speed projector-camera systems,
• real-time sensor networks.

Recent Results

(Not so recent actually... Please check out our YouTube movie list for more recent work)

Decentralized Topology Estimation for Adhoc Vision Sensor Network (2008-)

Establishing a logical network of vision sensors, in which each arc connects a pair of nodes sharing the same field of view, is important for cooperation of networked vision sensors deployed in an adhoc manner. In this study, matching of local image descriptors for the logical network estimation is decentralized and parallelized to multiple sensor nodes so that broadcast is avoided and network traffic is reduced.

Synchronizing Vision Sensors to Intensity-modulated Illumination (2008-)

A method of synchronizing the shutter timing of vision sensors to intensity-modulated illumination is proposed. The synchronization is achieved by constructing a software phase-locked loop within the camera, taking the images as input, and the shutter timing as output.

Expand the video

Collapse the video

Video Projection with Reduced Motion Blur using a High-Speed DMD Projector (2008-)

It has been recognized that hold-type displays such as Liquid Crystal Displays (LCD) and Digital Micromirror Devices (DMD) displays are disadvantageous in displaying video with fast motion when compared to impulse-type displays. Double-rate driving has been successfully applied in LCDs, but is difficult to be applied in DMD displays because they represent pixel brightness with pulse-width modulation (PWM). In this study, a method for motion blur reduction utilizing high-speed projection by a DMD projector is proposed. Subjective accessment experiments showed its significant difference (p < 0.001).

Finding an accelerometer within a camera view (2007-)

Localization of mobile devices such as cellphones and PDAs, which is one of the key technologies in realizing e.g. context-aware services, is still an open problem. In this study, focusing on the fact that many of recent mobile devices are equipped with accelerometers, a method of finding correspondence between signals from an acceleromter and image-space trajectories from a fixed camera is proposed.

Expand the video

Collapse the video

High-speed visual feedback control over a high-load network (2007-)

High-speed visual feedback control at 1000 fps over a high-load IP network is demonstrated. The system consists of a high-speed vision system, a host PC that executes motor-control tasks on an RTOS, and an easy-to-construct configuration for real-time communication to deliver the high-speed visual feature information obtained by the vision system to the host PC. Rather than introducing OS-dependent technologies for real-time communication, we have developed a dedicated network processing unit that handles all of the UDP/IP and Ethernet processing instead of the host PC. The visual information packets are forwarded in priority to the other background traffic by off-the-shelf Gigabit Ethernet switches with the IEEE802.1Q/p QoS mechanism.

Expand the video

Collapse the video

A dynamically reconfigurable architecture for high-speed vision systems (2007-)

A new reconfigurable processor architecture specialized for high frame rate visual processing is proposed. This architecture employs a 2-D mesh processing element (PE) array in which the PEs can be configured to operate as SIMD arrays or operation-pipeline trees depending on image processing algorithms so that maximum on-chip memory consumption is reduced. To achieve high on-chip memory utilization, the architecture features that the ALU network and the local memory network can be configured independently.

Nearest-point tracking using a high-speed projector-camera system (2007-)

A 1,000-fps projector-camera system in which projected patterns are adaptively controlled according to image processing results is demonstrated. Adaptive structured light projection enables fast and efficient 3-D information acquisition. The prototype system is applied to tracking of the nearest point of an object, and experimental results show that the system successfully tracked an apex of a fast-moving target object.

Expand the video

Collapse the video

3,000-fps shape measurement using a high-speed projector-camera system (2006-)

3D shape measurement based on a structured light method at high frame rates such as 3,350 fps with 256x256 pixels and 500 fps with 1024x1024 pixels is achieved using a high-speed DMD projector and a high-speed camera. The proposed structured light method features tracking of projected patterns in the image space considering discontinuous object surfaces.

Expand the video

Collapse the video

Expand an extra video

Collapse the video

Disassembling a projector:

Real-time Ethernet incorporating frame preemption (2006-)

A method of incorporating a frame preemption technique, by which a high-priority frame can suspend a low-priority ongoing frame, into Ethernet switches is proposed. It has been shown that sub-millisecond latency which is required in robot control over a network is achieved at the 100-Mbps wire speed.

A smart-camera-type high-speed vision platform: VCS-IV (2002-)

We have been developing a smart camera platform for implementing high-speed visual processing over 1,000 fps. It features a modular structure in which individual functions are implemented in stackable separate units so that the system can be easily expanded.

Expand the video

Collapse the video