Hybrid light field image super-resolution and interpolation method using multi-array cameras

Abstract

Recent advances in camera technologies have led to the design of plenoptic cameras. This camera type can capture multiple images of the same scene using arrays of microlenses, where each microlens has a shifted location providing a separate view of the scene. Such a design results in a superior performance, as compared to traditional cameras, enabling multi-view or multi-focal imaging captured in a single shot. However, the main drawback of the currently available plenoptic imaging technology is limited spatial resolution, which makes it difficult to use in applications where sharpness or high-resolution is essential, such as in the film industry. Although some previous attempts have addressed this issue, they were affected by high computational complexity as well as limited interpolation factor. To resolve this, a novel light field field hybrid super-resolution method is proposed which combines two traditional methods of multi image super-resolution and hybrid single image super-resolution to create hybrid super-resolution image for efficient application to plenoptic images. Furthermore, after this combination, the output of the hybrid super-resolution image is segmented into the objects of interest. Then, super-resolution reconstruction by sparse representation is applied to super resolve the segmented image. This technique helps to increase the resolution of light field images and maintain sharpness after super-resolution. Additionally, block matching super-resolution is proposed to provide a means of enhancement for the resolution of plenoptic images by developing corresponding super-resolution methods which exploit the disparity information, estimated from the light field images, to reduce the matching area in the super-resolution process. The proposed method is denoted as block matching super-resolution super-resolution. Following on, the proposed novel super-resolution method is combined with directionally adaptive image interpolation to preserve sharpness of the high-resolution images. In addition, light field digital refocusing with the proposed super-resolution approaches can be used to record the light field and provide maximum achievable resolution. With this simplification it is easy to explain the method of refocusing and the characteristics of the performance. The complexity of the standard light field camera configuration is also taken into account. The implemented super-resolution approaches that have been proposed are used to super resolve the ‘all-in-focus’ images. This research has narrowed the knowledge gap by creating a working super-resolution and interpolation application in order to allow higher quality to all light field images captured by plenoptic cameras. The significant advantage of this application for computer vision is the super-resolved micro images or the various angles available in a single light field super resolution image which allow depth estimation. Moreover, this research demonstrates a steady gain in the peak signal to noise ratio and structural similarity index quality of the super-resolved images for the resolution enhancement factor 8x8, as compared to the most recent approaches.