IJIRST –International Journal for Innovative Research in Science & Technology| Volume 3 | Issue 03 | August 2016 ISSN (online): 2349-6010
Self-Learning based Single Image SuperResolution Devyani Tambe PG Student Department of Electronics & Telecommunication Engineering Padmabhooshan Vasantdada Patil Institute of Technology, Bavdhan , Pune, India
Prof. S. M. Kulkarni Assistant Professor Department of Electronics & Telecommunication Engineering Padmabhooshan Vasantdada Patil Institute of Technology, Bavdhan , Pune, India
Abstract Images with low quality have low resolution and also have some blocking artifacts. To perform image super-resolution (SR) on this low quality image gives low visual quality of the image. In this paper a self-learning based super resolution technique is used to obtain a low quality SR on single image as well as removal of artifacts which are introduced due to compression. On the low quality image if deblocking is done, then the details may be lost. With self-learning sparse representation for low resolution and high resolution image patches by using learned dictionaries. This method gives far better results in terms of visual quality as compared with other methods of interpolation used. Keywords: Super-Resolutiont, Self-Learning, Sparse Representation, Learned Dictionaries _______________________________________________________________________________________________________ I.
INTRODUCTION
With advanced technological development in fields of multimedia and network, there is always need to exchange data across various devices having different capabilities. However, this is done through the internet and limited by the channel bandwidth and storage capability. Due to this constraint, images sent over the internet are in low quality. Images are degraded as they are downscaled and compression is performed to reduce the size. Downscaling of an image exploits the spatial redundancy while compression exploits the correlation in the frequency and temporal domains. The quality degradation reduces the required bandwidth and storage space required for images, it also leads to significant information loss and various unpleasing visual artifacts. This gives rise to blocking, ringing, or blurring [1]. The process in which a high resolution (HR) image is produced from one or several other low resolution (LR) images is called Super-resolution (SR). The quality of images is improved by the spatial resolution enhancement, given by image superresolution. The image SR recovers a high-resolution image from one or multiple low-resolution input images. There are two categories of approach for image Super-resolution: (i) classical approaches and (ii) example/learning-based approaches. In the first method, which is classical approach, reconstruction-based method, where a set of LR images of the same scene are matched with sub-pixel accuracy to form an HR image [2]. This method is time-consuming and impractical as multiple LR images are required as input. The sub-category of this is classical is frame interpolation [3], which gives overly smooth images with ringing and blurred artifacts. The example/learning-based methods gives the high frequency details of an LR image based on the occurrence prior between LR image patches and HR image patches in a dictionary training set, which gives better result ass compared to the classical approach. As in LR input, example-based methods matching for image patches from a pre-collected training LR image dataset or the same image itself based on self-examples, and using their HR versions to produce the final SR output. However, the HR details given by this method cannot be guaranteed to give the true HR details. Due to this, the performance of this method depends on the similarities between the training set and test set or the self-similarity in the image itself. A coupled dictionary and training approach for SR based on patch-wise sparse recovery is where the learned couple dictionaries relate the HR/LR image patch spaces via sparse representation. The sparse representation of the LR patch can be reconstructed with its underlying HR patch. This method of SR considers that input LR image is degraded only by down-scaling or by blurring operation. However, in a practical environment, where image compression is achieved gives compression artifacts as blocking and ringing. Compression reduces the image size by half without perceptual quality loss in the LR form of an image. However, if SR is directly performed on LR image it gives compression artifacts which are magnified and hence the final output visual perceptual quality will be low. Hence, a high performing SR scheme for low quality image is required for improving the resolution of image which is degraded by down scaling and compression. A unified framework was proposed to improve the resolution and perceptual quality simultaneously of low-quality web image. This method combines learning based SR with adaptive regularization which is decided by JPEG compression quality factor of an image. SR algorithm for example based compressed image also use Discrete Cosine Transform domain. This framework achieves denoising and SR for the compressed image.SR relies on main two operations to achieve results, denoising operation followed by SR operation
All rights reserved by www.ijirst.org
114
Self-Learning based Single Image Super-Resolution (IJIRST/ Volume 3 / Issue 03/ 020)
Fig. 1: Flowchart of self-learning single image SR
The method proposed is a self-learning based sparse representation to obtain low quality SR single image with removal of blocking artifacts. The main advantage of this method is the unified framework to remove blocking artifacts and using sparse based image decomposition for single image SR. This is also a self-contained, as no training image is required. II. SINGLE IMAGE SUPER RESOLUTION The flowchart of proposed self-learning single image based SR framework is shown in fig. 1. In this method, the input LR image I with blocking artifacts and its down-scaled are first decomposed into the low-frequency parts, and the high-frequency parts, and using the BM3D (block-matching and 3D filtering) algorithm. The basic information will be in the LF parts and the blocking artifacts with the other edge and texture details will be in the HF parts of the images. The coupled dictionary training method to learn two sets of coupled dictionaries, and used for SR of blocking and non-blocking patches, respectively. To get the SR of patch-wise sparse reconstruction with the pair of dictionaries, and corresponding HR/LR patch is used for each patch without blocking artifacts. Each patch having blocking artifacts the SR reconstruction is done. The first pair is non-blocking dictionaries consisting of corresponding HR and LR patch, and second is for blocking dictionaries. They can be enlarged and decomposed into HR non-blocking and HR blocking components. Then integrate the non-blocking high frequency component and the bicubic-interpolated to obtain the final SR result of I. Preprocessing As training patches are obtained from the input LR image, there is no need to collect extra training patches. Input image is converted to high frequency domain and certain preprocessing operations are conducted. The image I which has blocking artifacts is downscaled and apply BM3D algorithm to decompose I into LF and HF components. Two sets of HR/LR patch pair are fixed as training samples for dictionaries learning which are then used for SR and also Type equation here.deblocking. There is a patch in higher scale and its corresponding patch in lower scale which has a magnifying factor, as a result a couple of training image is generated. Using blocking artifact detection in HR part on both coupled patch pairs generates “blocking” and “non-blocking” patch pairs. Using this two sets of training samples, there are two sets of dictionaries to be learned. Then SR for each non-blocking input LR as sparse representation. Dictionary Learning Having HR/LR training patch pairs without blocking artifacts generate a couple of dictionaries to model the relationships between HR and LR patches. The set of non-blocking patch pair of LR training patches as the observation space, while the set of HR training patches as the latent space, where the patches have sparse representations with respect to certain dictionaries. Patches (LR) in are observable, while patches (HR) in are what we want to recover.
All rights reserved by www.ijirst.org
115
Self-Learning based Single Image Super-Resolution (IJIRST/ Volume 3 / Issue 03/ 020)
III. EXPERIMENTAL RESULTS JPEG images with blocking artifacts were used to evaluate the proposed SR algorithm for low quality single image. The test LR images used were compressed by JPEG which had a quality factor(QF) between 15 to 25. The QF is not required to be known in advance of the input LR image. For test images which are LR the magnifying factor is 2 the HR/LR patch sizes is either 16x16 / 8x8 and size of learned dictionaries is fixed to 1024.
Fig. 2(a): high resolution image (b): downscaled RGB to gray conversion. (c): High frequency components. (d): low frequency components. (e): Edge detected output. (f) bicubic interpolated output (g) final output SR
REFERENCES Li-Wei Kang, Boqi Zhuang, Chih-Chung Hsu,,Chia-Wen Lin and Chia-Hung Yeh, “Self-learning-Based Low-Quality Single Image Super-Resolution,” IEEE MMSP 2013. [2] M. Y. Shen and C. C. J. Kuo, “Review of postprocessing techniques for compression artifacts removal,” J. Vis. Commun. Image Rep., 1998. [3] W. T. reeman, T. R. Jones, and E. C. Pasztor, “Example-based super-resolution,” IEEE Computer Graphics and Applications, vol. 22, 2002. [4] D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image,” Proc. IEEE Int. Conf. Comput. Vis., Sept. 2009, pp. 349-356. [5] J. Yang, J. Wright, T. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process., vol. 19, 2010. [6] J. Yang et al., “Coupled dictionary training for image super-resolution,” IEEE Trans. Image Process., vol. 21, no. 8, pp. 3467-3478, Aug. 2012. [7] M.-C. Yang and Y.-C. F. Wang, “A self-learning approach to single image super-resolution,” IEEE Trans. Multimedia, vol. 15, no. 3, 2013. [8] T. Q. Pham et al., “Resolution enhancement of low quality videos using a high-resolution frame,” Proc. SPIE, vol. 6077, 2006. [9] G. A. Triantafyllidis et al., “Blocking artifact detection and reduction in compressed data,” IEEE Trans. Circuits Syst. Video Technol., 2002. [10] J. M. Fadili et al., “Image decomposition and separation using sparse representations: an overview,” Proc. IEEE, vol. 98, no. 6, June 2010. [11] L.-W. Kang, C.-W. Lin, and Y.-H. Fu, “Automatic single-image-based rain streaks removal via image decomposition,” IEEE Trans. Image Process., vol. 21, 2012. [12] K. Dabov et al., “Image denoising by sparse 3D transform- domain collaborative filtering,” IEEE Trans. Image Process., vol. 16, 2007 [1]
All rights reserved by www.ijirst.org
116