High-Speed Image Restoration Based on a Dynamic Vision Sensor

초록

Highlights What are the main findings? We show that a Dynamic Vision Sensor (DVS), combined with a conventional image sensor and enhanced by event-driven techniques, can effectively suppress artifacts in motion blur compensation. We demonstrate that the proposed technique significantly improves the image quality of the blurred image. What are the implications of the main findings? The event-based vision sensor can practically complement conventional CIS to achieve motion blur-resilient, high-speed imaging in smartphones without incurring prohibitive power or latency overhead. The demonstrated improvement under realistic low-illumination, fast-motion conditions suggests that future mobile camera designs can leverage DVS-CIS fusion as a viable system-level solution, rather than relying solely on heavier learning-based deblurring or more complex optics.Highlights What are the main findings? We show that a Dynamic Vision Sensor (DVS), combined with a conventional image sensor and enhanced by event-driven techniques, can effectively suppress artifacts in motion blur compensation. We demonstrate that the proposed technique significantly improves the image quality of the blurred image. What are the implications of the main findings? The event-based vision sensor can practically complement conventional CIS to achieve motion blur-resilient, high-speed imaging in smartphones without incurring prohibitive power or latency overhead. The demonstrated improvement under realistic low-illumination, fast-motion conditions suggests that future mobile camera designs can leverage DVS-CIS fusion as a viable system-level solution, rather than relying solely on heavier learning-based deblurring or more complex optics.Abstract We report on the post-capture, on-demand deblurring technique based on a Dynamic Vision Sensor (DVS). Motion blur causes photographic defects inherently in most use cases of mobile cameras. To compensate for motion blur in mobile photography, we use a fast event-based vision sensor. However, in this paper, we found severe artifacts resulting in image quality degradation caused by color ghosts, event noises, and discrepancies between conventional image sensors and event-based sensors. To overcome these inevitable artifacts, we propose and demonstrate event-based compensation techniques such as cross-correlation optimization, contrast maximization, resolution mismatch compensation (event upsampling for alignment), and disparity matching. The results show that the deblur performance can be improved dramatically in terms of metrics such as the Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), and Spatial Frequency Response (SFR). Thus, we expect that the proposed event-based image restoration technique can be widely deployed in mobile cameras.

키워드