IJSTE - International Journal of Science Technology & Engineering | Volume 3 | Issue 09 | March 2017 ISSN (online): 2349-784X
Sequence-to-Sequence Similarity for Image Denoising Akhila A S PG Student Department of Electronics & Communication Engineering Marian Engineering College, Trivandrum, India
Dr. M Sasikumar Professor Department of Electronics & Communication Engineering Marian Engineering College, Trivandrum, India
Abstract Image denoising is a continuing field of researches. Despite of several remarkable progresses in image denoising technologies, still there exists so many challenges. None of the denoising algorithm could be widely used for all applications. Fundamental challenge in a denoising technique is that it should adequately smooth a noisy image to efficiently remove the noise, while simultaneously preserving the important structural details within the image. In this work, we present a new similarity concept for image denoising; sequence to sequence similarity. In this concept of similarity, instead of taking pixel averages and block similarities, which are traditional concepts, sequence to sequence similarity is evaluated. This measure not only denoises the image but also improves the image quality by preserving the structural contents in an image. The experimental results show that the proposed method is good in preserving the edge information comparing to other approaches using pixel to pixel similarity and block to block similarity. Keywords: Image denoising, block similarity, sequence to sequence similarity, edge information ________________________________________________________________________________________________________ I.
INTRODUCTION
Image is always subjected to noise during its acquisition, coding, transmission, and processing steps. Noise is very difficult to remove from the digital images without the prior knowledge of noise model. Hence noise removal forms the pre-processing step in the field of photography, research, satellite technology and medical science, where degraded image has to be restored before further processing. Image denoising is the manipulation of the image data to produce a visually high quality image. Generally the existing or current denoising algorithms or approaches are filtering approach, multifractal approach and domain transform approach such as wavelet based approach. But such image denoising causes blurring and introduces artifacts in the original image.Different types of images inherit different types of noise and different noise models are used to present different noise types. Denoising method tends to be problem specific and depends upon the type of image and noise model. Based on noise types and noise models so many denoising techniques are developed, but some time these techniques are not useful for all application. Also most all of them cause loss of content during denoising process. Loss of content causes blurring of image. This is the one of the main issue while developing a denoising algorithm. However time to time we need the reinforcement learning of theoretical and practical idea of noise present in digital images. The objective of this thesis is to find out a solution for the problem of using conventional denoising techniques. Most denoising algorithms make two assumptions [7] about the noisy image. The first assumption is that the noise contained in the image is white noise. This means that the noise contains all frequencies, low and high. Because of the higherfrequencies, the noise is oscillatory or non-smooth. The second assumption is that the true image is smooth or piecewise smooth .This means the true image or patches of the true image only contain low frequencies. These assumptions can cause blurring and loss of detail in the resulting denoised images. Previous methods attempt to separate the image into the smooth part (true image) and the oscillatory part (noise) by removing the higher frequencies from the image. However, not all images are smooth. Images can contain fine details and structures which have high frequencies. When the high frequencies are removed, the high frequency content of the true image will be removed along with the high frequency noise. This will result in a loss of fine detail in the denoised image. Also, nothing is done to remove the low frequency. Low frequency noise will remain in the image even after denoising. Several method have been proposed to remove the noise and recover the true image .Even though they may be different in the use of tools, it must be emphasized that a wide class share the same basic idea that is denoising is achieved by averaging. This averaging may be performed locally or by the calculus of variations or in the frequency domain. This based several denoising techniques have been proposed so far including, the arithmetic mean filter, Gaussian filter [1], Wiener filter [2], neighborhood filtering[19], directional Gaussian smoothers [6]. In all these local averaging is done. The fourth order partial differential equation model is an image denoising method proposed for maintaining the balance of denoising and edge keeping.[4],[5]. Total variation based filtering was introduced by Rudin, Osher, and Fatemi [7] is an effective filtering method for recovering piecewise-constant signals. TV denoising is formulated as the minimization of a non-differentiable cost function[7-8]. Several 2D wavelet-based denoising methods have been developed [8-10]. Gurmeet Kaur, Rupinder Kaur proposed in[8] Image Denoising Using Wavelet Transform And Various Filters, various filtering approach and wavelet based approach and their All rights reserved by www.ijste.org
201
Sequence-to-Sequence Similarity for Image Denoising (IJSTE/ Volume 3 / Issue 09 / 041)
comparative study. These methods usually transform the image into multiple subbands at different orientations and resolution scales and then apply each subband’s wavelet coefficients to a de-noising procedure. Then, the image is reconstructed by performing the inverse transform on these wavelet coefficients to return it back to the spatial domain. In [9] Denoising of an Image using Discrete Stationary Wavelet Transform and various thresholding techniques are proposed. A nonlinear thresholding techniques in wavelet domain such as hard and soft thresholding, using discrete stationary wavelet transform for different wavelets, at different levels to denoise an image and to determine the best one out of them is proposed. A non-parametric method for texture synthesis is proposed by Alexei A. Efros and Thomas K. Leung in [11-12]. The texture synthesis process grows a new image outward from an initial seed, one pixel at a time. A Markov random field model is assumed, and the conditional distribution of a pixel all its neighbors synthesized so far is estimated by querying the sample image and finding all similar neighborhoods. This method aims at preserving as much local structure as possible and produces good results for a wide variety of synthetic and real-world textures. S.M. Smith introduces SUSAN: A New Approach to Low Level Image Processing.[12]. This paper describes a new approach to low level image processing; in particular, edge and corner detection and structure preserving noise reduction .Non-linear filtering is used to define which parts of the image are closely related to each individual pixel; each pixel has associated with it a local image region which is of similar brightness to that pixel The new feature detectors are based on the minimization of this local image region, and the noise reduction method uses this region as the smoothing neighbourhood. These methods are accurate, noise resistant and fast. Kostadin Dabov and etal, in their paper [13] present a novel approach to image denoising based on effective filtering in 3D transform domain by combining sliding-window transform processing with block-matching. Blocks within the image are processed in a sliding manner and utilize the block-matching concept by searching for blocks which are similar to the currently processed one. The matched blocks are stacked together to form a 3D array which exhibit high level of correlation. We exploit this correlation by applying a 3D decorrelating unitary transform and effectively attenuate the noise by shrinkage of the transform coefficients. The subsequent inverse 3D transform yields estimates of all matched blocks. The final estimate is computed as weighed average of all overlapping blockestimates. In [14-15] another filter model is proposed based on fussy set this paper the Fuzzy based Histogram Adaptive Filter is used to denoise an image corrupted with salt and pepper noise. It consists of mamdani fuzzy logic which has different rules. By checking the neighbouring pixels and after implementing these set of rules on these pixels, the output of the pixel is then decided whether this is a noisy picture or a healthy pixel. A. Buades, B. Coll, and J. Morel proposed a non-local algorithm for image denoising[16], which asserts that the de-noising of an image pixel is theoretically a weighted sum of all pixels in the image, with weights proportional to the similarity of local neighborhood pixels to the center pixel. The above methods estimate the noise pixel value based on the information provided in a surrounding local window surrounding local window. Unlike these local denoising methods, non-local methods estimate the noisy pixel and replaced based on the information of the whole image. Many similar algorithms are developed as an extention to the original NLM algorithm. In [22] Ville and Kocher proposed the surebased non-local mean filter, in which, the weights of the central pixel are set to the maximum weight of the surrounding pixels in the filter window instead of a value of one as in the original version. In [24] a new modified NLM was proposed by establishing the weight of the central pixel as zero. Later Wu et al introduced two different new NLM approaches, the James-Stein Type center pixel weight [21] and the probabilistic NLM. These improved NLMs focus mainly on improving visual performance, while some other works focus on reducing the computational cost. A common disadvantage of the above methods is the fact that they usually require some ‘apriori’ knowledge about the noise and the original signal. This information is necessary to select the optimal choice of parameter values in all these. Recently in [31] Panetta et al introduces a third type of similarity called sequence to sequence similarity which mainly for preserving edges. This method which is proven to be much better in reconstructing the edge than other methods has good reponse. In this paper our aim is to develop a denoising algorithm using sequence to sequence similarity concept. II.
SEQUENCE TO SEQUENCE SIMILARITY
The traditional image processing techniques, that we have discussed so far, rely on pixel and block similarity measurements. A new concept of similarity measurement that is sequence-to-sequence similarity[32] is introduced in this section. The main issue with the existing denoising algorithms is that they sometimes cause blurring of the image. Again some algorithm may cause loss of fine structural details such as edge informations. Restored image will be perfect if their edges are clearly retrieved. By considering a third kind of similarity called sequence-to-sequence similarity (SSS), it is proved that we could concentrate more on the edge information. This concept is differs from the other two similarity approaches that is pixel to pixel based and block to block based, in the sence that, this utilizes the high similarity of sequences within the image in different directions instead of using block content similarity. While in block-to-block similarity, a similar pair of blocks with close mean values are foundout. But sometimes it does not always indicate the content is actually similar, as this may result from the noise in the surrounding pixels of the block rather than the central pixel. Hence, with the noise in the surrounding pixels, the original pairs of blocks with high similarity in the noise-free original image will be shown to have lower similarity. And reverse also happens. A similar pair of sequences in the same direction are considered to be indicating the similarity between edge\texture information. . Thus, by evalauvating the similarity between these sequence in same direction it is possible to reconstruct edge sequence. Similar edge information will be captured and utilized to reconstruct the original image. It will have good performance in dealing with edge information, especially for images with the appearance of texture information. The SSS process can be illustrated with the
All rights reserved by www.ijste.org
202
Sequence-to-Sequence Similarity for Image Denoising (IJSTE/ Volume 3 / Issue 09 / 041)
following example. As shown in the figure 1. In the filtering window, we have two sequences. Let’s call it as a surrounding sequence (in blue) and a central sequence (in red), each consisting of pixels in the same direction. The middle point of the central sequence is defined as the center pixel of the filtering window, while the middle point of the surrounding sequence is one of the surrounding pixels in the filtering window. The similarity between these two sequences is called the sequence-to-sequence similarity.
Fig. 1: Sequence to sequence similarity
As shown in Figure1 in the horizontal sequence, the sequences [107 90 67 53 50] and [101 88 70 53 43] in top, the sequence [98 84 67 52 40] in the bottom row are all highly similar to the central sequence [102 86 67 51 43]. By evaluating similarity weights among these highly similar sequences we could reconstruct the center pixel, if it is corrupted. Similarly in the rightoblique direction, the surrounding sequence [188 153 133] and central sequence [183 153 155] can also be a pair exhibiting hi gh sequence-to-sequence similarity and utilizing their similarity nearby sequences can be reconstructed. In order to obtain the edge information in each direction and to use this concept for denoising purpose here, similarity is calculated in all the four directions that is horizontal, vertical, right-oblique and left-oblique directions. Making use of these similarity values a filter is designed as follows. Filter Design The sequence-to-sequence similarity can be represented by a mathematical model, whose inputs are two sequences (A=[đ??´1,đ??´2,‌,đ??´đ?‘›]and B=[đ??ľ1,đ??ľ2,‌,đ??ľđ?‘›]) and whose output (1) To demonstrate the use of sequence-to-sequence similarity for image denoising, we introduce a new SSS-based denoising filter. To contend with the edge information in each direction, similarity is calculated in the four directions: horizontal, vertical, right-oblique and left-oblique. It utilizes the similarity value of two neighboring correlated sequences as a difference weight. Its function can be described as below equation (2) (2) Where, Ai+k1,j+k2 is an image pixel of a noisy image (A), and Wc(i+k1,j+k2 ) is the corresponding weight. For these neighbouring pixels (k1≠0 and k2≠0), the similarity weight has four components, including horizontal similarity weight (Wh), vertical similarity weight (W v), right-oblique similarity weight (Wr) and left-oblique similarity weight (Wl).
For these neighboring pixels (k1=0 or k2=0), the horizontal similarity weight (Wh), vertical similarity weight (Wv) are all equal to zero. For the central neighboring pixels (k1=0,k2=0), all the similarity weights are equal to zero. The four similarity values can be calculated by the Wi+k1,j+k2h=đ?•Š(Ai+k1,j+k2−K:j+k2+K,Ai,j−K:j+K) (4) Wi+k1,j+k2v=đ?•Š(Ai+k1−K:i+k1+K,j+k2,Ai−K:i+K,j) (5) Wi+k1,j+k2l=đ?•Š(Ai+k1−K:i+k1+K,j+k2−K:j+k2+K,Ai−K:i+K,j−K:j+K) (6) Wi+k1,j+k2r=đ?•Š(Ai+k1+K:i+k1−K,j+k2−K:j+k2+K,Ai+K:i−K,j−K:j+K) (7) Where, the function S() is calculated from equation (1). And the sub-index in A indicates the position of each pixel in the sequence to calculate the sequence-to-sequence similarity. III. EXPERIMENT The most common noise emanating from an image acquisition system can be modeled as Gaussian noise. Furthermore, Gaussian noise is the most extensively used noise model in all experiments and requires only a single parameter, which varies the noise level. Hence here also the noise in the image is assumed to be a additive Gaussian noise. This can be represented as All rights reserved by www.ijste.org
203
Sequence-to-Sequence Similarity for Image Denoising (IJSTE/ Volume 3 / Issue 09 / 041)
A(i,j) =I(i,j)+n(i,j) (8) where I(i,j) represents the original image pixel information for pixel index (i,j), n(i,j) is the noise and A(i,j) is the corresponding noisy pixel. Different kinds of images are selected to apply this algorithm. The algorithm performs well in both grayscale and colour images. Computer simulation results for some images with different noise level are shown in figure 3 and figure 4. From the computer simulation results, it can be seen that the edge information is well preserved. And when the level of noise is small (with small varience value), the denoised results are almost similar to the original one. As the noise level increases, the noise begins to affect the content information to a large extent. With the SSS, we still can recover the content information including discernible edges. Also it can be seen that output has better human visual quality.
(a) (b) (c) (d) (e) Fig. 2: Denoising results of different methods;(a)Original image; (b) Noisy image; (c) Output of gaussian filter; (d) Output of NLM; (e) Output of SSS;
(a)
(b)
(c)
(d)
(e)
(f) (g) (h) Fig. 3: Output of SSS filter for gray images : (a) Original image 1; (b) noisy image (varience 0.02) ; (c) SSSoutput ; (d) noisy image (varience 0.25); (e) SSSoutput; (f) Original image 2 ; (g) noisy image (varience 0.02) ; (h) SSSoutput.
(a)
(b)
(f)
(c)
(g)
(d)
(b)
(h
Fig. 4: Output of SSS filter for color images: (a) Original image 1; (b) noisy image (varience 0.02) ; (c) SSSoutput ; (d) noisy image (varience 0.25); (e) SSSoutput; (f) Original image 2 ; (g) noisy image (varience 0.02) ; (h) SSS output.
All rights reserved by www.ijste.org
204
Sequence-to-Sequence Similarity for Image Denoising (IJSTE/ Volume 3 / Issue 09 / 041)
IV. CONCLUSION Denosing plays an important role in improving the quality of images in some application where peak quality of image required( such as medical diagnosing, surveying..etc). There have been several published algorithms and each approach has its assumptions, advantages, and limitations. Most of the known techniques fails to preserve the edge informations during denoising process. But with sequence to sequence similarity consideration, shows better result in edge preservation. Sequence to sequence similarity based filter have good performance in dealing with edge information, especially for images with the appearance of texture information. So this can be applied for processing satellite images, medical images etc. Successful utilization of the similar edge information for each direction makes the method applicable for many applications, such as image restoring, image denoising, and image deblurring. Also PSNR values obtained by SSS filtering is better than NLM. Table 1 shows the PSNR value for image tire.tif using Gaussian filter , an NLM filter and SSS filter(for given noise value.). Corresponding outputs are also shown Noise level Noise level (σ =30) (σ =70) Gaussian -53 -66 NLM 28.0 23.72 SSS 29.3 26.10 ACKNOWLEDGE We would like to thank all reviewers for their helpful comments that helped greatly for this work. REFERENCE [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30]
James C Church, Yixin Chen and Stephen V Rice “A spatial median filter for noise removal in digital images” IEEE, page(s):618-623,2008. L. Yaroslavsky.” Local Criteria And Local Adaptive Filtering In Image Processing: A Retrospective View”. Springer Verlag, 1985. O. Pogrebnyak, V. V. Lukin, “Wiener discrete cosine transform-based image filtering,” Journal of Electronic Imaging, Vol. 21, no. 4, 2012. Chen Yang and Zhu Qiang,, “A Study on Partial Differential Equation Model of Image Denoising Method” International Journal of Signal Processing, Image Processing and Pattern Recognition Vol.8, No.9 (2015), pp.1-12 A C. Bovik, T S. Huang and D C Munson Jr., “A generalization of median filtering using linear combinations of order statistics [J]”, Acoustics, Speech and Signal Processing, IEEE Transactions on, vol. 31, no. 6, (2006), pp. 1342-1350.) D. Charalampidis, “Efficient Directional Gaussian Smoothers,” IEEE Geoscience and Remote Sensing Letters, vol.6, no.3, pp.383-387, July 2009 L. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D, 60:259–268, 19922004. Qiangqiang Yuan; Liangpei Zhang; Huanfeng Shen, “Hyperspectral Image Denoising Employing a Spectral–Spatial Adaptive Total Variation Model,” IEEE Transactions on Geoscience and Remote Sensing, , vol.50, no.10, pp.3660-3677, Oct. 2012 Gurmeet Kaur, Rupinder Kaur “Image Denoising Using Wavelet Transform And Various Filters” International Journal Of Research In Computer Science eISSN 2249-8265 Volume 2 Issue 2pp 15-21(2012). C.Smith, S. Agaian, D. Akopian “A wavelet-denoising approach using polynomial threshold operators,” IEEE Signal Processing Letters, pp. 906-909, 2008 Guangyi Chen , Shen-En Qian, “Denoising and dimensionality reduction of hyperspectral imagery using wavelet packets, neighbour shrinking and principal component analysis,” International Journal of Remote Sensing, Vol. 30, Iss. 18, 2009 Efros and T. Leung. Texture synthesis by non parametric sampling”. In Proc. Int. Conf. Computer Vision, volume 2, pages 1033–1038, 1999. S. Smith and J. Brady.” Susan - A New Approach To Low Level Image Processing”. International Journal of Computer Vision, 23(1):45–78, 1997. Abdullla Al Jumah “ Denoising Of An Image Using Discrete Stationary Wavelet Transform And Various Thresholding Techniques” Journal of signal and information processing , 2013,4,33-41. Iram sami, Abhishek Thakur “Image Denoising For Guassian Noise Reduction In Bionics Using DWT Technique” IJECT Vol 4 Issue Spl-3 April 2013. K. Dabov, A. Foi, V. Katkovnik, K. Egiazarian, “Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering,” IEEE Transactions on Image Processing, vol.16, no.8, pp.2080-2095, 2007 J. Camarena, V. Gregori, S. Morillas and A. Sapena, “A simple fuzzy method to remove mixed Gaussian-impulsive noise from colour images,” IEEE Trans. Fuzzy Systems, vol. 21, no.5, pp. 971-978, 2013. S. Morillas, V. Gregori, and A. Hervs, “Fuzzy peer groups for reducing mixed Gaussian-impulse noise from color images,” IEEE Trans. Image Process., vol. 18, no. 7, pp. 1452–1466, 2009. A. Buades, B. Coll, and J. Morel. “On image denoising methods”.Technical Report 2004-15, CMLA. Buades, B. Coll, J.-M. Morel, “A non-local algorithm for image denoising,” Computer Vision and Pattern Recognition, 2005, vol.2, pp.60-65, 20-25 June 2005 S. Nercessian, K. Panetta, S. Agaian, “A multi-scale non-local means algorithm for image de-noising,” SPIE Defense, Security, and Sensing, 84060J-10, 2012 Yuntao Qian; Yanhao Shen; Minchao Ye; Qi Wang, “3D nonlocal means filter with noise estimation for hyperspectral imagery denoising,” IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp.1345-1348, 22-27 July 2012 L. Zeng, B. Lu, “Region-based non-local means algorithm for noise removal,” Electronics Letters , vol.47, no.20, pp.1125-1127, Sep. 2011 Yue Wu, B. Tracey, P. Natarajan, J.P. Noonan, “James–Stein Type Center Pixel Weights for Non-Local Means Image Denoising,” IEEE Signal Processing Letters , vol.20, no.4, pp.411-414, April 2013 Y. Wu, B. Tracey, P. Natarajan, J. Noonan, “Probabilistic Non-Local Means,” IEEE Signal Processing Letters, vol.20, no.8, pp.763-766, 2013 D. Van De Ville and M. Kocher, “Sure-based non-local means,” IEEE Signal Processing Letters, vol. 16, no. 11, pp. 973-976, Nov. 2009. J. Salmon, “On two parameters for denoising with non-local means,” IEEE Signal Processing Letters, vol. 17, no. 3, pp. 269-272, Mar. 2010. V. Duval, J. Aujol, and Y. Gousseau, “A bias-variance approach for the nonlocal means,” SIAM J. Imag. Sci., vol. 4, no. 2, pp. 760-788, 2011. K.N. Chaudhury, A. Singer, "Non-Local Euclidean Medians," IEEE Signal Processing Letters, vol.19, no.11, pp.745-748, Nov. 2012 ] Lu Lu, Weiqi Jin, Xia Wang, “Non-Local Means Image Denoising with a Soft Threshold,” IEEE Signal Processing Letters, vol.22, no.7, pp.833-837, July 2015.
All rights reserved by www.ijste.org
205
Sequence-to-Sequence Similarity for Image Denoising (IJSTE/ Volume 3 / Issue 09 / 041) [31] Karen Panetta, Long Bao, Sos Agaian, “Sequence to Sequence Similarity Based Filter for Image Denoising” IEEE sensor journel, 10.1109/JSEN. 2016.2548782. [32] C.C. Liu, T.Y. Sun, S.J. Tsai, Y.H. Yu, S.T. Hsieh, “Heuristic wavelet shrinkage for denoising,” Applied Soft Computing, Vol. 11, No. 1, pp.256-264, 2011. [33] R. Yan, L. Shao, Y. Liu, “Nonlocal hierarchical dictionary learning using Wavelets for image denoising,” IEEE Trans. on Image Processing, vol.22, no. 12, pp. 4689-4698, 2013.
All rights reserved by www.ijste.org
206