ISBN:378-26-138420-0298
INTERNATIONAL CONFERENCE ON CIVIL AND MECHANICAL ENGINEERING, ICCME-2014
A Novel Approach to Assess the Quality of Tone Mapped Images P.Manjulamma, M.Tech (DECS) Student,
B.Sreenivasan , Assistant professor,
N.pushpalatha Assistant professor,
Department of ECE, AITS Annamacharya Institute of Technology and Sciences, Tirupati, India-517520 1
putta.manjula@gmail.com 2 srinu325@gmail.com 3 pushpalatha_nainaru@rediff.com
Abstract – This paper deals with the assessment of measuring the quality of tone mapped images by considering the Tone Mapping Operators (TMOs) that convert high dynamic range (HDR) to low dynamic range (LDR) images provide practically useful tools for the visualization of HDR images on standard LDR displays. Different TMOs create different tone mapped images, and a natural question is which one has the best quality. Without an appropriate quality measure, different TMOs cannot be compared, and further improvement is directionless. Subjective rating may be a reliable evaluation method, but it is expensive and time consuming, and more importantly, is difficult to be embedded into optimization frameworks. Here we propose an objective quality assessment algorithm for tone mapped images by combining two concepts one of them is a multi scale signal fidelity measure on the basis of a modified structural similarity index and the other followed by a naturalness measure on the basis of intensity statistics of natural images. Validations using independent subject-rated image databases show good correlations between subjective ranking score and the proposed tonemapped image quality index (TMQI). The proposed measure not only provides an overall quality score of an image, but also creates multi-scale quality maps that reflect the structural fidelity variations across scale and space. Keywords - Tone Mapping Operators, low dynamic range, high dynamic range image, image fusion, image quality assessment, naturalness, perceptual image processing, structural similarity, tone mapping operator I. INTRODUCTION There has been a growing interest in recent years in high dynamic range (HDR) images, where the range of intensity levels could be on the order of 10,000 to 1. This allows for accurate representations of the luminance variations in real scenes, ranging from direct sunlight to faint starlight [1]. With recent advances in imaging and computer graphics technologies, HDR images are becoming more widely available. A common problem that is often encountered in practice is how to visualize HDR images on standard display devices that are designed to display low dynamic range (LDR) images. To
overcome this problem, an increasing number of tone mapping operators (TMOs) that convert HDR to LDR images have been developed, for examples [2]– [5]. Because of the reduction in dynamic range, tone mapping procedures inevitably cause information loss. With multiple TMOs available, one would ask which TMO faithfully preserves the structural information in the original HDR images, and which TMO produces natural-looking realistic LDR images. TMO assessment in the past mostly relied on human subjective evaluations. In [6], perceptual evaluations of 6 TMOs were conducted with regard to similarity and preferences. An overview and a subjective comparison of 8 TMOs were reported in [7]. HDR capable monitor was employed in to compare 6 TMOs in a subjective experiment using a paired comparison method. In, 14 subjects were asked to rate 2 architectural interior scenes produced by 7 TMOs based on basic image attributes as well as the naturalness of the LDR images. A more comprehensive subjective evaluation was carried out in where tone mapped images generated by 14 TMOs were shown to 2 groups of 10 human observers to rate LDR images, concerning overall quality, brightness, contrast, detail reproduction and color. In subjects were asked to choose the best LDRs derived from 2 TMOs with different parameter settings to optimally tune the algorithms. The value of subjective testing cannot be overestimated. However, they have fundamental limitations. First, it is expensive and time consuming. Second, it is difficult to be incorporated into an optimization framework to automatically improve TMOs and adjust their parameter settings. Furthermore, important image structures contained in HDR images may be missing in tone mapped images, but human observers may not be aware of their existence. In this sense, subjective evaluation should not be regarded as a golden standard for the quality of tone mapped images. Typical objective image quality assessment (IQA) approaches assume the
INTERNATIONAL ASSOCIATION OF ENGINEERING & TECHNOLOGY FOR SKILL DEVELOPMENT 92
www.iaetsd.in
ISBN:378-26-138420-0299
INTERNATIONAL CONFERENCE ON CIVIL AND MECHANICAL ENGINEERING, ICCME-2014
reference and test images to have the same dynamic range [12], and thus cannot be directly applied to evaluate tone mapped images. Only a few objective assessment methods have been proposed for HDR images. The HDR visible differences predictor (HDR-VDP) is a human visual system (HVS) based fidelity metric that aims to distinguish between visible (supra threshold) and invisible (sub threshold) distortions. The metric reflects the perception of distortions in terms of detection probability. Since HDR-VDP is designed to predict the visibility of differences between two HDR images of the same dynamic range, it is not applicable to compare an HDR image with an LDR image. A dynamic range independent approach was proposed in [14], which improves upon HDR-VDP and produces three types of quality maps that indicate the loss of visible features, the amplification of invisible features, and reversal of contrast polarity, respectively. These quality maps show good correlations with subjective classifications of image degradation types including blur, sharpening, contrast reversal, and no distortion. However, it does not provide a single quality score for an entire image, making it impossible to be validated with subjective evaluations of overall image quality. The purpose of the current work is to develop an objective IQA model for tone mapped LDR images using their corresponding HDR images as references. Our work is inspired by the success of two design principles in IQA literature. The first is the structural similarity (SSIM) approach [15] and its multi-scale derivations [16], [17], which asserts that the main purpose of vision is to extract structural information from the visual scene and thus structural fidelity is a good predictor of perceptual quality. The second is the natural scene statistics (NSS) approach, which maintains that the visual system is highly adapted to the natural visual environment and uses the departure from natural image statistics as a measure of perceptual quality [18]. Here we propose a method that combines a multi-scale structural fidelity measure and a statistical naturalness measure, leading to Tone Mapped image Quality Index (TMQI). Moreover, we demonstrate that TMQI can be employed for optimizing parameters in TMOs and for adaptively fusing multiple tone mapped images. II. LITERATURE SURVEY A. High-Dynamic-Range Image (HDR) High-Dynamic-Range Imaging (HDRI or HDR) is a set of techniques used in imaging and photography to reproduce a greater dynamic range of luminosity than possible using standard digital imaging or photographic techniques. HDR images can represent more accurately the range of intensity levels found in
real scenes, from direct sunlight to faint starlight, and is often captured by way of a plurality of differently exposed pictures of the same subject matter. NonHDR cameras take photographs with a limited exposure range, resulting in the loss of detail in bright or dark areas. HDR compensates for this loss of detail by capturing multiple photographs at different exposure levels and combining them to produce a photograph representative of a broader tonal range. The two primary types of HDR images are computer renderings and images resulting from merging multiple low-dynamic-range (LDR) or standard-dynamic-range (SDR) photographs. HDR images can also be acquired using special image sensors, like oversampled binary image sensor. Tone mapping methods, which reduce overall contrast to facilitate display of HDR images on devices with lower dynamic range, can be applied to produce images with preserved or exaggerated local contrast for artistic effect. HDR stands for High Dynamic Range. For those who aren’t so acquainted with this high-tech shutterbug lingo, dynamic range is basically just the difference between the lightest light and darkest dark you can capture in a photo. Once your subject exceeds the camera’s dynamic range, the highlights tend to wash out to white, or the darks simply become big black blobs. It’s notoriously difficult to snap a photo that captures both ends of this spectrum, but with modern shooting techniques and advanced post-processing software, photographers have devised ways to make it happen. This is basically what HDR is: a specific style of photo with an unusually high dynamic range that couldn’t otherwise be achieved in a single photograph. You’ve probably seen these types of images scattered across the Web. Depending on how they’re processed, HDR photos can be anything from stunningly accurate reproductions of what your eyes see to mind-blowing surreal works of art that transform reality into a high-def dreamscape. Here are a few examples from HDR. At the most basic level, an HDR photo is really just two (or three, or nine) photos taken at different exposure levels and then mashed together with software to create a better picture. It’s a bit more complicated than that, but not much more – that’s basically the gist of it. Ideally, the photographer takes a range of bracketed photos – that is, photos of the same subject taken with varying shutter speed/aperture combinations in order to produce a set of images with varying luminosity and depth of field. Then, with the help of advanced postprocessing software, the photographer is able to blend the photos together and create a single image comprised of the most focused, well-lit, and colorful parts of the scene.
INTERNATIONAL ASSOCIATION OF ENGINEERING & TECHNOLOGY FOR SKILL DEVELOPMENT 93
www.iaetsd.in
ISBN:378-26-138420-02100
INTERNATIONAL CONFERENCE ON CIVIL AND MECHANICAL ENGINEERING, ICCME-2014
B. Tone Mapping Tone mapping is a technique used in image processing and computer graphics to map one set of colors to another in order to approximate the appearance of high dynamic range images in a medium that has a more limited dynamic range. Print-outs, CRT or LCD monitors, and projectors all have a limited dynamic range that is inadequate to reproduce the full range of light intensities present in natural scenes. Tone mapping addresses the problem of strong contrast reduction from the scene radiance to the displayable range while preserving the image details and color appearance important to appreciate the original scene content. C. Purpose and Methods The goals of tone mapping can be differently stated depending on the particular application. In some cases producing just aesthetically pleasing images is the main goal, while other applications might emphasize reproducing as many image details as possible, or maximizing the image contrast. The goal in realistic rendering applications might be to obtain a perceptual match between a real scene and a displayed image even though the display device is not able to reproduce the full range of luminance values. Another approach to tone mapping of HDR images is inspired by the anchoring theory of lightness perception. This theory explains many characteristics of the human visual system such as lightness constancy and its failures (as in the checker shadow illusion), which are important in the perception of images. The key concept of this tone mapping method (Lightness Perception in Tone Reproduction) is a decomposition of an HDR image into areas (frameworks) of consistent illumination and the local calculation of the lightness values. The net lightness of an image is calculated by merging of the frameworks proportionally to their strength. Particularly important is the anchoring—relating of the luminance to a known luminance, namely estimating which luminance value is perceived as white in the scene. This approach to tone mapping does not affect the local contrast and preserves the natural colors of an HDR image due to the linear handling of luminance. One simple form of tone mapping takes a standard image (not HDR – the dynamic range already compressed) and applies unsharp masking with a large radius, which increases local contrast rather than sharpening. However, HDR tone mapping, usually using local operators, has become increasingly popular amongst digital photographers as a postprocessing technique, where several exposures at
different shutter speeds are combined to produce an HDR image and a tone mapping operator is then applied to the result. There are now many examples of locally tone mapped digital images, inaccurately known as "HDR photographs", on the internet, and these are of varying quality. This popularity is partly driven by the distinctive appearance of locally tone mapped images, which many people find attractive, and partly by a desire to capture high-contrast scenes that are hard or impossible to photograph in a single exposure, and may not render attractively even when they can be captured. Although digital sensors actually capture a higher dynamic range than film, they completely lose detail in extreme highlights, clipping them to pure white, producing an unattractive result when compared with negative film, which tends to retain colour and some detail in highlights. In some cases local tone mapping is used even though the dynamic range of the source image could be captured on the target media, either to produce the distinctive appearance of a locally tone mapped image, or to produce an image closer to the photographer's artistic vision of the scene by removing sharp contrasts, which often look unattractive. In some cases, tone mapped images are produced from a single exposure which is then manipulated with conventional processing tools to produce the inputs to the HDR image generation process. This avoids the artifacts that can appear when different exposures are combined, due to moving objects in the scene or camera shake. However, when tone mapping is applied to a single exposure in this way, the intermediate image has only normal dynamic range, and the amount of shadow or highlight detail that can be rendered is only that which was captured in the original exposure. III. A PERCEPTUAL IMAGE QUALITY ASSESSMENT METRIC-STRUCTURAL SIMILARITY (SSIM) This paper is related to structural similarity index (SSIM) which represents perceptual image quality based on the structural information. SSIM is an objective image quality metric and is superior to traditional quantitative measures such as MSE and PSNR. This project demonstrates the SSIM based image quality assessment and illustrates its validity in terms of human visual perception. It will be very helpful to understand SSIM and its applications by reviewing the paper listed below. The structural similarity (SSIM) method is a recently proposed approach for image quality assessment. It is widely believed the statistical properties of the natural visual environment play a fundamental role in the evolution, development and adaptation of the human visual system. An important
INTERNATIONAL ASSOCIATION OF ENGINEERING & TECHNOLOGY FOR SKILL DEVELOPMENT 94
www.iaetsd.in
ISBN:378-26-138420-02101
INTERNATIONAL CONFERENCE ON CIVIL AND MECHANICAL ENGINEERING, ICCME-2014
observation about natural image signal samples is that they are highly structured. By “structured signal”, mean that the signal sample exhibit strong dependencies amongst themselves, especially when they are spatially proximate. These dependencies carry important information about the structure of the objects in visual scene. The principal hypothesis of structural similarity based image quality assessment is that the HVS is highly adapted to extract structure information in the visual field, and therefore a measurement of structure similarity or distortion should provide a good approximation to perceived image quality. The SSIM index is a method for measuring the similarity between two images. The SSIM index is a full reference metric, in other words, the measure event of image quality is based on an initial uncompressed or distortion free image as reference. The SSIM is designed to improve on traditional metrics like PSNR and MSE, which have proved to be inconsistent with human eye perception. The first approach is called the error sensitivity approach. In that approach the test images data is considered as the sum of the reference image and an error signal. It is assumed that the loss of perceptual quality is directly related to the visibility of the error signal. In most of HVS- based image quality assessment models attempt to weigh and combine different aspect of the error signal according to their respective visual sensitivities, which are usually determined by psychophysical measurement. And the problem with this approach is that larger visible differences may not necessarily imply lower perceptual quality. In the second approach the observation process efficiently extract and make the use of information represented in the natural scene, whose statistical properties are believed to play a fundamental role in the development of the HVS. So, the example of the second approach is the structural similarity based image quality assessment method, which is based on observation that natural images are highly “structured,” meaning that the signal samples have strong dependencies among themselves. These dependencies carry important information about the structure of the objects in the visual scene.
Figure: 1 Structural Similarity (SSIM) Measurement System [6] IV. QUALITY ASSESSMENT METHOD Due to the reduction in dynamic range, TMOs cannot preserve all information in HDR images, and human observers of the LDR versions of these images may not be aware of this. Therefore, structural fidelity plays an important role in assessing the quality of tone-mapped images [19]. On the other hand, structural fidelity alone does not suffice to provide an overall quality evaluation. A good quality tone mapped image should achieve a good compromise between structural fidelity preservation and statistical naturalness, which are sometimes competing factors. A. STRUCTURAL FIDELITY The SSIM approach provides a useful design philosophy as well as a practical method for measuring structural fidelities between images [20]. The original SSIM algorithm is applied locally and contains three comparison components − luminance, contrast and structure. Since TMOs are meant to change local intensity and contrast, direct comparisons of local and contrast are inappropriate. Let x and y be two local image patches extracted from the HDR and the tone-mapped LDR images, respectively. We define our local structural fidelity measure as
where σx , σy and σxy are the local standard deviations and cross correlation between the two corresponding patches in HDR and LDR images, respectively, and C1 and C2 are positive stabilizing constants. Compared with the SSIM definition [15], the luminance comparison component is missing, and the structure comparison component (the second term in (1)) is exactly the same. The first term in (1) compares signal strength and is modified from that of the SSIM definition based on two intuitive considerations. First, the difference of signal strength between HDR and LDR image patches should not be penalized when their signal strengths are both significant (above visibility threshold) or both insignificant (below visibility threshold). Second, the algorithm should penalize the cases that the signal strength is significant in one of the image patches but insignificant in the other. This is different from the corresponding term in the original SSIM definition where any change in signal strength is penalized. To distinguish between significant and insignificant signal strength, we pass the local standard deviation σ through a nonlinear mapping, which results in the σ` value employed in (1). The nonlinear mapping should be designed so that
INTERNATIONAL ASSOCIATION OF ENGINEERING & TECHNOLOGY FOR SKILL DEVELOPMENT 95
www.iaetsd.in
ISBN:378-26-138420-02102
INTERNATIONAL CONFERENCE ON CIVIL AND MECHANICAL ENGINEERING, ICCME-2014
significant signal strength is mapped to 1 and insignificant signal strength to 0, with a smooth transition in-between. Therefore, the nonlinear mapping is related to the visual sensitivity of contrast, which has been an extensively studied subject in the literature of visual psychophysics [21]. Practically, the HVS does not have a fixed threshold of contrast detection, but typically follows a gradual increasing probability in observing contrast variations. Psychometric functions describing the detection probability of signal strength have been employed to model the data taken from psychophysical experiments. Generally, the psychometric function resembles a sigmoid shape [22], [23] and the sensory threshold is usually defined at the level of 50% of detection probability. A commonly adopted psychometric function is known as Galton’s ogive [21], which takes the form of a cumulative normal distribution function given by
where p is the detection probability density, s is the amplitude of the sinusoidal stimulus, τs is the modulation threshold, and θs is the standard deviation of the normal distribution that controls the slope of detection probability variation. It was found that the ratio k =τs/θs is roughly a constant, known as Crozier’s law . Typical values of k ranges between 2.3 and 4, and k = 3 makes the probability of false alarm considerably small [21]. The reciprocal of the modulation threshold τs is often used to quantify visual contrast sensitivity, which is a function of spatial frequency, namely the contrast sensitivity function (CSF) [21]. A CSF formula that fits well with data collected in various psychological experiments is given by [25] A( f ) ≈ 2.6[0.0192 + 0.114 f ] exp[−(0.114 f )1.1] where f denotes spatial frequency. This function is normalized to have peak value 1, and thus only provides relative sensitivity across the frequency spectrum. In practice, it needs to be scaled by a constant λ to fit psychological data. In our implementation, we follow Kelly’s CSF measurement [26]. Combining this with above, we obtain τs( f ) =1/λ A( f ). This threshold value is calculated based on contrast sensitivity measurement assuming pure sinusoidal stimulus. To convert it to a signal strength threshold measured using the standard deviation of the signal, we need to take into account that signal amplitude scales with both contrast and mean signal intensity, and there is A √2 factor between the
amplitude and standard deviation of a sinusoidal signal. As a result, a threshold value defined on signal standard deviation can be computed as τσ ( f ) = µ’ / √2 λ A( f ) where µ is the mean intensity value. Based on Crozier’s law [21], [24], we have θσ ( f ) = τσ ( f )/ k We can then define the mapping between σ and σ` as
In (1), σ`x and σ`y are the mapped versions of σx and σy, respectively. They are bounded between 0 and 1, where 0 and 1 represent completely insignificant and completely significant signal strengths, respectively. The local structural fidelity measure S local is applied to an image using a sliding window that runs across the image space. This results in a map that reflects the variation of structural fidelity across space. The visibility of image details depends on the sampling density of the image, the distance between the image and the observer, the resolution of the display, and the perceptual capability of the observer’s visual system. A single scale method cannot capture such variations. Following the idea used in multi-scale [16] and information-weighted SSIM [17], we adopt a multi-scale approach, where the images are iteratively low-pass filtered and down sampled to create an image pyramid structure [27], as illustrated in Fig. 1. The local structural fidelity map is generated at each scale. Fig. 2 shows two examples of such maps computed at multiple scales for the LDR images created from two different TMOs. It is interesting to observe these fidelity maps and examine how they correlate with perceived image fidelity. For example, the structural details of the brightest window regions are missing in Image (b), but are more visible in Image (a). For another example, there are detailed structures in the top-right dark regions that are not easily discerned in Image (a), but are better visualized in Image (b). All of these are clearly reflected in the structural fidelity maps. At each scale, the map is pooled by averaging to provide a single score:
where xi and yi are the i -th patches in the HDR and LDR images being compared, respectively, and Nl is the number of patches in the l-th scale. In the literature, advanced pooling strategies such as information content based pooling [17] have been shown to improve the performance of IQA algorithms.
INTERNATIONAL ASSOCIATION OF ENGINEERING & TECHNOLOGY FOR SKILL DEVELOPMENT 96
www.iaetsd.in
ISBN:378-26-138420-02103
INTERNATIONAL CONFERENCE ON CIVIL AND MECHANICAL ENGINEERING, ICCME-2014
Fig 2. Framework of multiscale structural fidelity assessment However, in our current experiment, these advanced pooling methods did not result in notable performance gain in the proposed structural fidelity measure. The overall structural fidelity is calculated by combining scale level structural fidelity scores using the method in [16]
where L is the total number of scales and βl is the weight assigned to the l-th scale. There are several parameters in the implementation of our structural fidelity model. First, when computing Slocal, we set C1 = 0.01 and C2 = 10, and we find that the overall performance of the structural fidelity model is insensitive to these parameters within an order of magnitude. Second, to create the fidelity map at each scale, we adopt the same setting as in the SSIM algorithm [15] by employing a Gaussian sliding window of size 11×11 with standard deviation 1.5. Third, as in [16], we assume a viewing distance of 32 cycles/degree, which can represent signals up to 16 cycles/degree of resolution without aliasing, and use 16 cycles/degree as the spatial frequency parameter when applying the CSF in to the finest scale measurement, the spatial frequency parameters applied to the subsequent finer scales are then 8, 4, 2, 1 cycles/degree, respectively. Fourth, the mean intensity value in (6) is set to be the mean of the dynamic range of LDR images, i.e., µ = 128. Fifth, when combining the measures across scales, we set L = 5 and {βl} = {0.0448, 0.2856, 0.3001, 0.2363, 0.1333}, which follows the psychophysical experiment results reported in [16]. Finally, in order to assess the quality of color images, we first convert them from RGB color space to Yxy space and then apply the proposed structural fidelity measure on the Y component only. B.STATISTICAL NATURALNESS A high quality tone mapped LDR image should not only faithfully preserve the structural fidelity of the HDR image, but also look natural. Nevertheless, naturalness is a subjective quantity that is difficult to define quantitatively. A large literature has been dedicated to the statistics of natural images which have important significance to both image
processing applications and the understanding of biological vision. An interesting study of naturalness in the context of subjective evaluation of tone mapped images was carried out in which provided useful information regarding the correlations between image naturalness and different image attributes such as brightness, contrast, color reproduction, visibility and reproduction of details. The results showed that among all attributes being tested, brightness and contrast have more correlation with perceived naturalness. This motivates us to build our statistical naturalness model based on these two attributes. This choice may be oversimplifying in defining the general concept of statistical image naturalness (and may not generalize to other image processing applications that uses the concept of naturalness), but it provides an ideal compromise between the simplicity of our model and the capability of capturing the most important ingredients of naturalness that are related to the tone mapping evaluation problem we are trying to solve, where brightness mapping is an inevitable issue in all tone mapping operations. It also best complements the structural fidelity measure described in above, where brightness modeling and evaluation are missing.
Fig 3 Histograms of (a) means (fitted by Gaussian PDF) and (b) standard deviations (fitted by Beta PDF) of natural images Our statistical naturalness model is built upon statistics conducted on about 3,000 8bits/pixel gray-scale images obtained from that represent many different types of natural scenes. Fig. 3 shows the histograms of the means and standard deviations of these images, which are useful measures that reflect the global intensity and contrast of images. Recent studies suggested that brightness and contrast are largely independent quantities in terms of both natural image statistics and biological computation [32]. As a result, their joint probability density function would be the product of the two. Therefore, we define our statistical naturalness measure as N = (1/K) Pm Pd where K is a normalization factor given by K = max{Pm Pd }. This constrains the statistical naturalness measure to be bounded between 0 and 1.
INTERNATIONAL ASSOCIATION OF ENGINEERING & TECHNOLOGY FOR SKILL DEVELOPMENT 97
www.iaetsd.in
ISBN:378-26-138420-02104
INTERNATIONAL CONFERENCE ON CIVIL AND MECHANICAL ENGINEERING, ICCME-2014
C.QUALITY ASSESSMENT MODEL Given a tone mapped LDR image, we now have two available measurements, structural fidelity S and naturalness N, which are given by above equations respectively. These two quantities can be used individually or jointly as a 2D vector that characterizes different aspects of the quality of the LDR image. However, in most applications, users would prefer to have a single quality score of the image. Therefore, an overall quality evaluation that combines both quantities is desirable. In particular, we define the following 3-parameter function to combine the two components Q = aSα + (1 − a) Nβ where 0 < a < 1 determines the relative weights assigned to the two components, and α and β defines the sensitivities of the two components, respectively. Since both S and N are upper-bounded by 1, this overall quality measure is also Upperbounded by 1. V. VALIDATION OF QUALITY ASSESSMENT METHOD The validation process is conducted by comparing our objective quality assessment results with subjective data. Two evaluation metrics are employed which are given as follows. 1) Spearman’s rank-order correlation coefficient (SRCC) is defined as
where di is the difference between the i -th image’s ranks in subjective and objective evaluations. SRCC is a nonparametric rank-order based correlation metric, independent of any monotonic nonlinear mapping between subjective and objective scores. 2) Kendall’s rank-order correlation coefficient (KRCC) is another non-parametric rank correlation metric computed as
where Nc and Nd are the numbers of concordant (of consistent rank order) and discordant (of inconsistent rank order) pairs in the data set, respectively. The proposed TMQI is the only objective quality measure being tested. To the best of our knowledge, almost no other method has been proposed to compare images with different dynamic ranges. The only exception is the method proposed in [14], which creates probability maps to distinguish between visible (supra threshold) and invisible (sub threshold) degradations. The probability maps are shown to be useful in classifying image distortion types but are not meant to be pooled to produce an
overall quality score of a tone mapped image. As a result, direct comparison with the proposed method is not possible. VI. RESULTS
Fig 4 images generated with different parameter Q, S, & N values
Fig. 5 Run time versus the number of image pixels of the proposed algorithm
Fig 6. images generated with different parameter Q, S, & N values
VII. CONCLUSION An objective model to assess the quality of tone mapped images is developed by combining a multi-scale structural fidelity measure and a statistical naturalness measure. The proposed measure not only provides an overall quality score of an image, but also creates multi-scale quality maps that reflect the structural fidelity variations across scale and space. TMQI is designed to evaluate grayscale images only, but most HDR images of natural scenes are captured in color. One simple method to evaluate tone mapped color images is to apply the TMQI to each color channel independently and then combine them. Color fidelity and color naturalness measures may be developed to improve the quality measure. Simple averaging is used in the current pooling method of the structural fidelity map. Statistical naturalness measure is based on intensity statistics only. Advanced statistical models (that
INTERNATIONAL ASSOCIATION OF ENGINEERING & TECHNOLOGY FOR SKILL DEVELOPMENT 98
www.iaetsd.in
ISBN:378-26-138420-02105
INTERNATIONAL CONFERENCE ON CIVIL AND MECHANICAL ENGINEERING, ICCME-2014
reflects the structural regularities in space, scale and orientation in natural images) may be included to improve the statistical naturalness measure. Using TMQI as a new optimization goal, many existing TMOs may be redesigned to achieve better image quality. Novel TMOs may also be developed by taking advantage of the construction of the proposed quality assessment approach. Finally, the current method is applied and tested using natural images only. The application scope of HDR images and TMOs is beyond natural images. For example, modern medical imaging devices often capture HDR medical images that need to be tone-mapped before visualization. The TMQI and optimization methods may be adapted to these extended applications. REFERENCES [1]. E. Reinhard, G. Ward, S. Pattanaik, P. Debevec, W. Heidrich, and K. Myszkowski, High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting. San Mateo, CA: Morgan Kaufmann, 2010. [2]. E. Reinhard, M. Stark, P. Shirley, and J. Ferwerda, “Photographic tone reproduction for digital images,” in Proc. 29th Annu. Conf. Comput. Graph. Interact. Tech., vol. 21. 2002, pp. 267–276. [3]. G. W. Larson, H. Rushmeier, and C. Piatko, “A visibility matching tone reproduction operator for high dynamic range scenes,” IEEE Trans.Visual. Comput. Graph., vol. 3, no. 4, pp. 291–306, Oct.–Dec. 1997. [4]. F. Drago, K. Myszkowski, T. Annen, and N. Chiba, “Adaptive logarithmic mapping for displaying high contrast scenes,” Comput. Graph.Forum, vol. 22, no. 3, pp. 419–426, 2003. [5]. R. Fattal, D. Lischinski, and M. Werman, “Gradient domain high dynamic range compression,” in Proc. 29th Annu. Conf. Comput. Graph.Interact. Tech., 2002, pp. 249–256. [6]. F. Drago, W. L. Martens, K. Myszkowski, and H.-P. Seidel, “Perceptual evaluation of tone mapping operators,” in Proc. SIGGRAPH Conf.Sketches Appl., 2003, p. 1. [7]. A. J. Kuang, H. Yamaguchi, G. M. Johnson, and M. D. Fairchild, “Testing HDR image rendering algorithms,” in Proc. IS T/SID Color Imag. Conf., 2004, pp. 315–320. [8]. P. Ledda, A. Chalmers, T. Troscianko, and H. Seetzen, “Evaluation of tone mapping operators using a high dynamic range display,” ACM Trans. Graph., vol. 24, no. 3, pp. 640–648, 2005. [9]. A. Yoshida, V. Blanz, K. Myszkowski, and H. Seidel, “Perceptual evaluation of tone mapping operators with real-world scenes,” Proc.SPIE, Human Vis. Electron. Imag., vol. 5666, pp. 192–203, Jan. 2005. [10]. M. ˇ Cadík, M. Wimmer, L. Neumann, and A. Artusi, “Image attributes and quality for evaluation of tone mapping operators,” in Proc. 14th Pacific Conf. Comput. Graph. Appl., 2006, pp. 35–44. [11]. M. Barkowsky and P. L. Callet, “On the perceptual similarity of realistic looking tone mapped high dynamic range images,” in Proc. Int. Conf. Image Process., 2010, pp. 3245–3248. [12]. Z. Wang and A. C. Bovik, Modern Image Quality Assessment. San Rafael, CA: Morgan & Claypool Publishers, Mar. 2006. [13]. R. Mantiuk, S. Daly, K. Myszkowski, and S. Seidel, “Predicting visible differences in high dynamic range
[14].
[15].
[16].
[17].
[18].
[19].
[20].
[21].
[22].
[23].
[24].
[25].
[26].
[27].
[28].
images—model and its calibration,” Proc. SPIE, vol. 5666, pp. 204–214, Dec. 2005. T. O. Aydm, R. Mantiuk, K. Myszkowski, and H. Seidel, “Dynamic range independent image quality assessment,” in Proc. Int. Conf. Comput. Graph. Interact. Tech., 2008, pp. 1–69. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004. Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image quality assessment,” in Proc. IEEE Asilomar Conf.Signals, Syst., Comput., Pacific Grove, CA, Nov. 2003, pp. 1398–1402. Z. Wang and Q. Li, “Information content weighting for perceptual image quality assessment,” IEEE Trans. Image Process., vol. 20, no. 5, pp. 1185–1198, May 2011. Z. Wang and A. C. Bovik, “Reduced- and no-reference image quality assessment,” IEEE Signal Process. Mag., vol. 28, no. 6, pp. 29–40, Nov. 2011. H. Yeganeh and Z. Wang, “Objective assessment of tone mapping algorithms,” in Proc. IEEE Int. Conf. Image Process., Sep. 2010, pp.2477–2480. Z. Wang and A. C. Bovik, “Mean squared error: Love it or leave it? A new look at signal fidelity measures,” IEEE Signal Process. Mag.,vol. 26, no. 1, pp. 98–117, Jan. 2009. E. P. Simoncelli and B. A. Olshausen, “Natural image statistics and neural representation,” Annu. Rev. Neurosci., vol. 24, pp. 1193–1216,May 2001. M. Cˇ adík and P. Slavík, “The naturalness of reproduced high dynamic range images,” in Proc. 9th Int. Conf. Inf. Visual., 2005, pp. 920–925 V. Mante, R. Frazor, V. Bonin, W. Geisler, and M. Carandini, “Independence of luminance and contrast in natural scenes and in the early visual system,” Nature Neurosci., vol. 8, no. 12, pp. 1690–1697, 2005. E. C. Larson and D. M. Chandler, “Most apparent distortion: Fullreference image quality assessment and the role of strategy,” J. Electron. Imag., vol. 19, no. 1, pp. 011006-1–011006-21, 2010. M. Song, D. Tao, C. Chen, J. Bu, J. Luo, and C. Zhang, “Probabilistic exposure fusion,” IEEE Trans. Image Process., vol. 21, no. 1, pp. 341–357, Jan. 2012. T. Mertens, J. Kautz, and F. Van Reeth, “Exposure fusion,” in Proc. Pacific Conf. Comput. Graph. Appl., 2007, pp. 382–390. R. Mantiuk, K. Myszkowski, and H. Seidel, “A perceptual framework for contrast processing of high dynamic range images,” in Proc. 2nd Symp. Appl. Percept. Graph. Visual., 2005, pp. 87–94. R. S. Blum and Z. Liu, Multi-Sensor Image Fusion and Its Applications. New York: Taylor & Francis, 2006.
INTERNATIONAL ASSOCIATION OF ENGINEERING & TECHNOLOGY FOR SKILL DEVELOPMENT 99
www.iaetsd.in