Future Metrology Hub Academic Publications covering work package 2

Page 1

Optics and Lasers in Engineering 122 (2019) 37–48

Contents lists available at ScienceDirect

Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng

2500-Channel single-shot areal profilometer using hyperspectral interferometry with a pinhole array Tobias V. Reichold∗, Pablo D. Ruiz, Jonathan M. Huntley Loughborough University, Wolfson School of Mechanical, Electrical and Manufacturing Engineering, Loughborough LE11 3TU, UK

a r t i c l e

i n f o

OCIS codes: 120.3930 100.3175 120.6650 110.5086 Keywords: Metrological instrumentation Interferometric imaging Surface measurements Phase unwrapping Pinhole array Hyperspectral

a b s t r a c t Surface profilometry techniques such as coherent scanning interferometry or focus variation require long scan times and are thus vulnerable to environmental disturbance. Hyperspectral interferometry (HSI) overcomes the problem by recording all the spatial and spectral information necessary to reconstruct a 2D surface height map in a single shot. In this paper, we present a new HSI system that uses a pinhole array to provide the necessary gaps for the spectral information. It is capable of measuring 2500 independent points, twice the previous maximum number, with a maximum unambiguous depth range of ∼825 μm and a larger maximum surface tilt angle of 33.3 mrad. The use of phase information allows height to be measured to a precision of ∼6 nm, an order of magnitude improvement on previous HSI systems.

1. Introduction A wide range of contact and non-contact techniques now exist to measure absolute surface profiles and roughness. Contact tools such as drag stylus profilers or atomic force microscopy can make measurements irrespective of material composition, are able to cope with modest surface discontinuities and provide axial resolutions down to a few picometers [1]. However, the resultant force on the sample may change its shape or cause surface damage [1,2]. With scanning times per line as long as several minutes, measurements are also vulnerable to environmental disturbances such as temperature and humidity changes or ambient vibrations [3]. Areal scans comprising several line scans take correspondingly longer and are thus even more vulnerable to such influences. Optical surface profilers based on coherence scanning interferometry (CSI) or focus variation avoid some of the problems of contacting instruments, and as a result are finding widespread industrial application [4]. In CSI, a Mirau or Michelson objective with a broadband source is used to scan a sample in the axial direction, the height of a given point on the sample being determined from the point of maximum fringe modulation at the corresponding camera pixel. The broadband nature of CSI allows absolute height measurements of surface steps without phase unwrapping, which would not be possible with monochromatic sources. However mechanical scanning components increase the required acquisition

∗

time, hence the sensitivity to environmental disturbances is high [5]. Single shot measurement approaches are required to remove environmental sensitivities, such as the system proposed by Schwider and Zhou in which a broadband Fizeau interferometer and grating spectrometer were used to measure absolute 1D line profiles [6]. A recent technique known as hyperspectral interferometry (HSI) eliminates the need for any mechanical movement, enabling single-shot areal measurements with low environmental sensitivity. As with CSI, the use of a broadband source removes the depth ambiguity that would be present when measuring discontinuous surfaces with a monochromatic source. A Linnik-type interferometer and a hyperspectral imager are used to spatially separate a set of narrowband interferograms on a 2D photodetector array (PDA) enabling 2D unambiguous optical path difference measurements in a single shot [7,8]. In the original proof of concept [7], an interference filter (etalon) was used to turn the broadband spectrum from a super luminescent diode (SLED) into a frequency comb that illuminates the object and a reference mirror. The object was imaged through the interferometer onto the PDA, while a diffraction grating at the pupil plane of the imaging optics redirected the narrowband interferograms that correspond to each peak of the frequency comb to distinct positions onto the PDA, along the grating’s dispersion axis. Early implementations of HSI yielded a modest number of independent height measurements (∼200), an unambiguous depth range of 350 μm and a depth measurement precision of 80 nm [7,8]. Due to the etalon,

Corresponding author. E-mail address: t.reichold@lboro.ac.uk (T.V. Reichold).

https://doi.org/10.1016/j.optlaseng.2019.05.015 Received 7 March 2019; Received in revised form 7 May 2019; Accepted 14 May 2019 Available online 25 May 2019 0143-8166/© 2019 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license. (http://creativecommons.org/licenses/by/4.0/)

T.V. Reichold, P.D. Ruiz and J.M. Huntley

Optics and Lasers in Engineering 122 (2019) 37–48

optical throughout was compromised to ∼1/15 of the input power and pixel utilisation was only ∼2% of the available PDA. A more efficient implementation of HSI utilised a microlens array (MLA) to significantly increase the number of independent measurement channels [9]. The MLA was placed at the output of an imaging Linnik interferometer at an intermediate plane where the object is imaged with some magnification, effectively spatially sampling the object and the reference mirror. At the focal plane of the MLA, an array of spots appears, each spot being an image of the pupil of the main imaging optics by one of the microlenses. An imaging spectrometer based on a diffraction grating then turns this set of spots onto an array of linear spectra on the PDA. A small rotation between the MLA and the dispersion axis of the grating was introduced to prevent overlap between spectra. The MLA-based HSI system provided 1225 channels (>6× more than the etalon based system), 10× more optical throughput and an unambiguous depth range of 880 μm. However, the MLA acts effectively as a Shack-Hartmann sensor and when the wavefront of the object beam reaches it at an angle (due to local surface slope) the corresponding spot at the MLA focal plane drifts from the spot produced by the reference beam and thus the interference is lost. This means that an MLA-based system can only deal with relatively small surface height gradients.

In the current paper, this shortcoming is addressed by means of an alternative way to spatially sample the interferometer output image. It is based on a pinhole array (PHA) similar to those used in lens-less in-line holographic microscopes [10] or more prominently in spinning Nipkow disk confocal microscopy [11]. Although the optical power throughput is reduced compared to an MLA-based system, a higher channel count and improved tolerance to large surface height gradients is achieved. In addition, the use of phase information is investigated for the first time in an HSI system, which is found to result in over one order of magnitude improvement in depth measurement precision. The paper is structured as follows: Section 2 describes the new proposed optical setup; Section 3 the experimental methodology to study the system performance; Section 4 presents experimental results for three different samples and a discussion and conclusions are offered in Section 5. 2. Experimental setup The optical system is separated into two major subsystems: a balanced Linnik interferometer and a hyperspectral imaging system as shown in Fig. 1. The overall system is identical to the MLA-HSI predecessor as described in [9], except for the interface between the two

Fig. 1. Optical setup. The top part of the figure shows the two wavelengths illumination configuration used for spectral calibration and the broadband configuration used for absolute height measurements. The bottom diagram shows the interferometer and spectrometer subsystems. The main system components are: Superluminescent diode, SLED; 2 × 2 90:10 fibre coupler; fibre Bragg gratings FGB1 and FBG2; variable fibre attenuator, VFA; collimator lens L0 ; Lenses L1 , L2 , L3 ; Beam splitter, BS; reference mirror, R; object surface, O; pinhole array, PHA; imaging lenses L4 , L5 ; transmission diffraction grating, G; and photodetector array, PDA. 38

T.V. Reichold, P.D. Ruiz and J.M. Huntley

Optics and Lasers in Engineering 122 (2019) 37–48

subsystems: the object is imaged here onto an array of pinholes rather than microlenses. The hyperspectral imager in turn images the pinholes rather than the spots of light behind the microlenses. Provided the pinhole size is comparable to the spot size, the PHA-HSI therefore shares many of the same design considerations as the MLA-HSI. 2.1. Light source and interferometer The interferometer’s main light source is a super-luminescent diode (SLED, Superlum S840-HP1 15 mW) with a centre wavelength đ?œ†c = 840 nm and a bandwidth Δđ?œ† = 50 nm at full-width-half-maximum (FWHM). A 90:10 2 Ă— 2 single mode ďŹ bre coupler provides a two-state output for two independent illumination states. State one provides a “two wavelengthâ€? illumination condition by means of two series ďŹ bre Bragg gratings FBG1 and FBG2. This provides two selectively-ďŹ ltered wavelengths đ?œ†1 = 820 nm and đ?œ†2 = 853 nm with a narrow spectral band < 0.01 nm for spectral calibration of the wavelength axis in the spectral images as described in [9]. State two provides a broadband illumination condition by coupling the SLED output directly into the interferometer launch optics. A variable ďŹ bre attenuator (OZ optics neutral density for SM 780HP ďŹ bre) is used to adjust the input power to avoid saturating the PDA for any given exposure time. At the launch arm of the Linnik interferometer, singlet lens L0 (100 mm focal length) collimates the expanded beam, while lenses L1 and L2 in both arms (150 mm focal length) are near-infrared (NIR) achromatic doublets set to illuminate the reference mirror and the object surface with collimated beams. BS is a 50:50 non-polarizing beamsplitter cube. L2 in the reference and object arms and L3 (150 mm focal length) at the recombination arm form bi-telecentric systems that image the reference mirror and the object at the back focal plane of L4 . At that plane, an image of the object interferes with the collimated reference beam so as to form a broadband interferogram [9].

Fig. 2. Encoding of surface height in pinhole-array-based hyperspectral interferometer: (a) object with a surface step proďŹ le. The insert above shows a crosssection through the sample with height distribution h(x, y). Lines indicating cross-sections through the zero optical path dierence and sample datum surfaces are also shown; (b) image of the object on the pinhole array; (c) array of spectra (no reference beam in the interferometer); (d) spectra (shown here in monochrome for clarity) are modulated by fringes which frequency encode surface height from datum (adapted from [9]).

2.2. Hyperspectral imager A pinhole array on a chromium plated glass (50Ă—50 pinholes, square grid, 10 Îźm pinhole diameter, 120 Îźm pitch) is used to spatially sample the superimposed images of the object and reference mirror. The eective ďŹ eld of view is 5.88 mm Ă— 5.88 mm. A given pinhole labelled by indices (m, n) collects light from a small region on the sample surface, centred on a point with coordinates (xm , yn ), where m = 0, 1, 2, ‌, Nx – 1; n = 0, 1, 2,‌, Ny – 1 and Nx , Ny are respectively the number of pinholes along the x and y axes [9]. The pinholes are imaged onto a photodetector array (PDA, Apogee U16M, 4096 Ă— 4096 pixels, pixel size 9 Îźm, array size 36.86 mm array diagonal 52.13 mm) with imaging lenses L4 (Nikkor 50 mm f/# 1.8) and L5 (Nikkor 135 mm f/# 2.8). A blazed near-infrared diraction grating G (300 lines mm−1 ) diracts the collimated beams that L4 produces for each pinhole, thus creating a 1-D spectral signal that ďŹ ts between the images of neighbouring pinholes. The −1 diraction order has the highest intensity for this grating and is captured by L5 to image an array of spectra onto the PDA. The magniďŹ cation m54 for lenses L4 and L5 is equal to 2.709 (as measured using the known pinhole pitch and the distance between pinhole images on the PDA), which in combination with the 300 lines mm−1 diraction grating result in an individual spectrum length of approximately 2.093 mm on the PDA. Therefore, a maximum of âˆź234 pixels are available for each spectrum.

small angle � with respect to the row/column axes of the PDA to avoid cross-talk and spectral leakage of adjacent spectra. A PHA with reduced number of pinholes (Nx = Ny = 5) is used in Fig. 2(b) to illustrate the concept. Provided the diraction grating G in Fig. 1 is set with the grooves parallel to the columns of the PDA array, the spectra align parallel to the rows. Fig. 2(c) illustrates the spectra when only one of the reference or object beams is imaged onto the MLA. When both beams are present on the MLA (Fig. 2(d)), the spectra are modulated with fringes with a spatial frequency that encodes the optical path dierence between the object and reference beams for the corresponding pinhole in the PHA. We denote the sampling points along the wavenumber axis by kp where p = 0, 1, 2,‌, Nk – 1, and Nk is the total number of sample points.

2.4. Reduced sensitivity to sample tilt A drawback of the previously-proposed MLA-based system [9] is that its performance deteriorates when imaging a specularly-reecting sample whose normal is misaligned with respect to the optical axis of L2 . As stated previously this is because the rays from the tilted surface enter each microlens at a ďŹ nite angle causing the focused spots to shift sideways. This is shown schematically in Fig. 3(a) for a single microlens, together with the intensity cross section through the microlens’s point spread function.

2.3. Spatial encoding of spectral information The spatial encoding of spectral information is identical to that for a MLA. The key points from [9] are summarized here to aid comprehension of the concept. Fig. 2(a) shows a schematic view of an object that has a surface step which is imaged onto the PHA (Fig. 2(b)). Both the coordinate system of the sample, and the row/column axes of the PHA, are rotated by a 39

T.V. Reichold, P.D. Ruiz and J.M. Huntley

Optics and Lasers in Engineering 122 (2019) 37–48

nal processing procedures are therefore applicable. After removal of the DC term, each interferogram is multiplied by a Hanning window and Fourier transformed. An iterative bounded Newton-Raphson method is used to locate the Fourier peak from each interferogram. At the peak, frequency and phase information is recorded and the frequency is converted to optical path dierence, z0 , using the following equation: đ?‘§0 (đ?‘š, đ?‘›) = (2đ?œ‹âˆ•Î”đ?‘˜)đ?‘“đ?‘˜ (đ?‘š, đ?‘›),

(5)

fk is the frequency in units of cycles across the spectral bandwidth used, Δđ?‘˜ = đ?‘˜2 − đ?‘˜1 , where k1 and k2 are the minimum and maximum wavenumbers respectively [9]. The ‘two wavelength’ measurement provides the scaling factor that allows pixel location to be converted to wavenumber. z0 can be considered as an estimate of the OPD, which is related to the surface height value as: [ ] đ?‘§0 (đ?‘š, đ?‘›) = 2 â„Ž0 − â„Ž(đ?‘š, đ?‘›) . (6) h0 is the known distance from the sample datum surface to the plane of zero optical path dierence, as shown in Fig. 2(a). 3.2. HSI analysis using phase information HSI implementations so far have used only the frequency of the recorded fringes to infer depth. The phase of the interference signal, as determined from the argument of the Fourier transform peak, has previously been ignored. By contrast, phase in CSI has been used for many years to provide lower-noise estimates of OPD [12]. A general algorithm to estimate OPD from the phase at the transform peak, using intensity measured as a function of wavenumber, has been proposed recently [13] and we therefore investigate its applicability to HSI data. đ?œ™1 will be used to denoted the phase calculated from the real and imaginary parts of the transform at the positive frequency peak corresponding to OPD = z0 . It can be shown that đ?œ™1 is a sum of two terms:

Fig. 3. Susceptibility of (a) MLA- and (b) PHA-based HSI systems to sample tilt for the case of smooth (non-speckled) wavefronts.

If đ?œƒđ?‘š(đ?‘€) represents the angle between the surface normal and optical axis of L3 , the lateral shift of the spot is đ?›żđ?‘Ľ = 2đ?œƒđ?‘š(đ?‘€ ) đ?‘“3

(1)

where f3 is the focal length of a microlens. Interference signal is therefore lost when đ?›żx equals or exceeds the diameter of the point spread function (PSF) of the imaging system, given by đ?‘‘đ?‘ƒ đ?‘†đ??š 3

1.22Îťđ?‘? = , đ?‘ đ??´3

đ?œ™1 = đ?œ™0 + đ?‘˜1 đ?‘§0

đ?œ™0 is a phase oset that arises due to the relative phase shifts on reection from the object and the reference mirror. The second term arises from the fact that the interference signal is shifted by −đ?‘˜1 along the wavenumber axis prior to Fourier transformation. By the Fourier shift theorem, this gives rise to an additional factor exp (ik1 z0 ) at the Fourier peak at OPD = z0 . A single phase measurement is insuďŹƒcient to separate the đ?œ™0 and k1 z0 terms from Eq. (7). If đ?œ™0 is not known a priori, for example from documented material properties or a previous calibration, it can be estimated in a least squares sense from data taken from a region of the sample known to have a constant value of đ?œ™0 [13]. Such an estimator for đ?œ™0 will be denoted đ?œ™Ě‚ 0 , with the estimators for z0 provided by Fourier peak location and phase denoted by đ?‘§Ě‚ (1) and đ?‘§Ě‚ (2) , respec0 0 Ě‚ tively. đ?œ™1 − đ?œ™0 is a wrapped phase, lying in the range –đ?œ‹ to đ?œ‹, but can be unwrapped by using the phase đ?‘˜1 đ?‘§Ě‚ (1) to give the phase estimator 0 for z0 : )] [ ( đ?œ™1 − đ?œ™Ě‚ 0 − đ?‘˜1 đ?‘§Ě‚ (01) 1 (2) Ě‚ đ?‘§Ě‚ 0 = đ?œ™1 − đ?œ™0 − 2đ?œ‹ đ?‘ đ??ź đ?‘ đ?‘‡ (8) đ?‘˜1 2đ?œ‹

(2)

where đ?œ†c is the central wavelength and NA3 is the numerical aperture of the microlenses. The maximum allowable surface slope angle for a MLA-based system, đ?œƒđ?‘š(đ?‘€) , follows from Eqs. (1) and (2) as: đ?œƒđ?‘š(đ?‘€ ) =

0.61Îťđ?‘? 1.22Îťđ?‘? ≈ , đ?‘“3 đ?‘ đ??´3 đ??ˇ3

(3)

where đ??ˇ3 = 100 Îźm is the microlens diameter. The situation for the PHA-based HSI system, on the other hand, is different because the pinhole locations are ďŹ xed. So long as light reaches a given pinhole from the sample, interference with the reference beam will always occur. The limitation on tilt angle is now given by the requirement that light from the sample passes through aperture E1 . Tilt or surface slope in the object causes the focused spot at E1 to move in the plane of the circular aperture. Loss of signal occurs when this spot moves đ??ˇ laterally by a distance 21 from the centre position, where đ??ˇ1 = 20 mm is the diameter of E1 . This corresponds to a maximum permissible surface slope angle đ?œƒđ?‘š(đ?‘ƒ ) =

đ??ˇ1 , 4đ?‘“2

(7)

where NINT is a function that rounds to the nearest integer. The map of OPD resulting from applying Eq. (8) to the fringe pattern from each of the pinholes can then be converted into a height map using Eq. (6). The performance of these two approaches is compared experimentally in the next section.

(4)

where f2 is the focal length of lens L2 . For the systems described in this paper and [9], đ?œƒđ?‘š(đ?‘ƒ ) and đ?œƒđ?‘š(đ?‘€) take the values 33.33 mrad and 10.24 mrad, respectively; the use of the PHA therefore results in an increase in allowable tilt by approximately 3.25Ă—.

4. Experimental results 3. Data analysis The system was tested with three dierent object conďŹ gurations, as shown in Fig. 4. Every time a new sample was loaded into the object arm of the interferometer, its surface was brought to the plane of zero optical path dierence with the reference beam (which we refer to here as zero delay line, ZDL) by means of a translation stage. As the object

3.1. ‘Standard’ HSI analysis Each pinhole produces a 1-D spectral intensity signal in the same way as each microlens in the previous MLA set-up [9]. Identical sig40

T.V. Reichold, P.D. Ruiz and J.M. Huntley

Optics and Lasers in Engineering 122 (2019) 37–48

moves towards the ZDL, the spatial frequency of the spectral fringes is gradually reduced and it approaches 0 at the ZDL location. Fringes due to a global relative tilt between the object and reference mirror surfaces can also be observed (and compensated for) close to the ZDL. These appear across the whole object field of view because the chromium mask on the PHA is not fully opaque at near infrared wavelengths, and are convenient for the removal of any residual tilt against the reference beam. Fig. 4(a) shows the case of a gold-coated mirror that was placed at four different positions relative to the ZDL. Fig. 4(b) shows a pair of gauge blocks with known height difference, which were used to assess the accuracy of step height measurements. Fig. 4(c) illustrates a silver coated mirror mounted on a PZT driven tilting stage (S-334.2SL, Physics Instruments, maximum mirror range ±50 mrad, repeatability = 5 μrad).

Fig. 4. Different object configurations viewed from above. The red dashed line is a cross section through the Zero Delay Line surface.

Fig. 5. Portion of an image of the PHA through the imaging spectrometer when a gold-coated flat mirror is observed using: (a) two-wavelengths illumination configuration and reference wave alone; (b) broadband illumination configuration and reference wave alone; (c) Broadband configuration with reference and object waves (mirror surface at 50 μm from the ZDL); (d) as (c) but with the mirror surface at 100 μm from the ZDL.

41

T.V. Reichold, P.D. Ruiz and J.M. Huntley

Optics and Lasers in Engineering 122 (2019) 37–48

It can be tilted by a known angle đ?›ź tilt with respect to the optical axis to assess the performance of the system in terms of surface slope. Further details of these experiments are provided in Sections 4.2–4.4. 4.1. Two-wavelength illumination Using the two-wavelengths illumination conďŹ guration, an image is taken with light from the reference or object arm alone. Each pinhole in the array produces two spots, as shown in Fig. 5(a). The horizontal arrowed line indicates the image locations of a single pinhole at the two wavelengths đ?œ†1 and đ?œ†2 . The m’ and n’ axes are parallel to the m and n axes, respectively, but with an arbitrary shift between the two coordinate system origins, and demonstrate the rotation of the PHA with respect to the PDA. This image is used for the calibration of the wavelength axis for the later spectral images using the same procedure as described in [9]. As the grating equation is non-linear the distance d between the đ?œ†1 and đ?œ†2 pinhole images vary slightly with pinhole indices as shown in Fig. 6. 4.2. Broadband illumination – gold-coated mirror Fig. 6. Separation between the spots corresponding to đ?œ†1 and đ?œ†2 pinhole images for each of the 50 Ă— 50 pinholes.

A gold-coated mirror mounted on a translation stage was used as the ďŹ rst object. This allowed the noise level in the measurements to be estimated at a range of distances from the ZDL. Using broadband illumination, two images were taken: one with the reference wave, the other with the object wave. This provided the spectrum of each of the two waves and hence allowed the DC and low-frequency terms in the interference signal to be fully removed for subsequent single-shot imaging [9]. A series of interferograms was then taken at 50 Îźm steps from the ZDL location as shown schematically in Fig. 4(a). With increasing OPD,

the modulation frequency of the individual spectra increases as shown in Fig. 5(c) and (d). The horizontal proďŹ le through the fringe modulated spectrum corresponding to the centre pinhole when the mirror was at 100 Îźm from ZDL is shown in Fig. 7(a). The DC term was ďŹ rst removed, then the signal was normalised to remove the inuence of the source

Fig. 7. Interferogram corresponding to the central pinhole when the mirror is at 100 Îźm from the ZDL: (a) Intensity proďŹ le vs wavenumber after DC subtraction, spectral normalisation and application of a Hanning window; (b) Magnitude of the Fourier transform of (a); (c) Same as (b) but with horizontal axis scaled to distance from ZDL. 42

T.V. Reichold, P.D. Ruiz and J.M. Huntley

Optics and Lasers in Engineering 122 (2019) 37–48

Fig. 8. Stacked height maps of a at gold coated mirror moved axially with a translation stage in 50 Οm steps at 50, 100, 150 and 200 Οm, respectively. Table 1 Rms residual values in nm for at gold-coated mirror at three locations relative to ZDL position. Analysis method

Polynomial degree

Δℎ = 50 Οm

Δℎ = 100 Οm

Δℎ = 150 Οm

Δℎ = 200 Οm

Peak location Peak location Phase

1 2 2

122 95.4 5.8

141 123 7.9

157 141 10.1

222 213 12.6

spectral envelope, and ďŹ nally a Hanning window was applied along the wavenumber axis to reduce spectral leakage [9]. The Fourier transform contains a dominant peak, the position of which indicates the distance from the mirror surface to the ZDL, as described in Section 3.1. Fig. 8 shows three 50 Ă— 50 stacked height maps of the mirror at displacements Δℎ = 50, 100 and 150 Îźm from the ZDL. Each map corresponds to a ďŹ eld of view of 5.89 Ă— 5.89 mm2 . A total of 7500 independent coordinates were measured, 2500 for each surface map. The mean displacement measured by the system between the ďŹ rst and second locations (cyan and red surfaces, respectively) is 48.74 Îźm, 51.34 Îźm between the second and third locations (red and green surfaces) and 48.56 Îźm between the third and fourth locations (green and blue surfaces). These values are consistent with the nominal known displacement of 50 Îźm and a positioning accuracy of âˆź 1 Îźm, corresponding to 1/10 of a division on the micrometer of the manually-driven translation stage. The modulation of the spectral interference proďŹ le approaches zero outside the range đ?‘˜ = 7.709 Ă— 106 − 7.263 Ă— 106 m−1 which is equivalent to a wavelength range of 815–865 nm. This is the same as the speciďŹ ed bandwidth of the SLED source and is twice the wavelength range of the previously-described MLA system [9]. The spectral range 815– 865 nm was sampled with 234 pixels, giving an eective inter-pixel increment in wavelength and wavenumber of đ?›żđ?œ† = 0.21 nm, and đ?›żđ?‘˜ = 1.904 Ă— 103 m−1 , respectively. The corresponding theoretical height Ď€ range â„Žđ?‘š = 2đ?›żđ?‘˜ = 0.825 mm.

The measurement error of the system can be characterised by the root mean square (rms) residuals after least squares ďŹ tting of a polynomial surface to the data. The rms values for a ďŹ rst and second degree ďŹ t are shown in Table 1. The residuals from a 2nd degree polynomial ďŹ t, in the range 95–141 nm, are âˆź6Ă— lower than corresponding values for the previously described MLA system [9]. Even lower residuals are produced by the phase analysis method. The extracted phase maps đ?œ™1 (m, n) for 50,100, 150 and 200 Îźm from ZDL are shown in Fig. 9. An average phase oset value đ?œ™Ě‚ 0 = 0.897 rad was calculated using the procedure described in [13]. The ďŹ nal height maps h (m, n) produced after unwrapping using Eq. (8) are shown in Fig. 10. Some of the phase values for Δℎ = 50 Îźm were not unwrapped correctly leading to anomalously high residual values and are not considered further here. The height maps show relatively large deviations from atness (rms residual of 438 and 509 nm with respect to a best-ďŹ t plane); as these are approximately 8Ă— the known deviation from atness for the mirror (đ?œ†/10 at 633 nm) they are likely to be an artefact of the sensor. One possible cause is a defocusing error: if lenses L1 in the reference and object arms have a small axial position errors, the wavefronts arrive at the PHA with slightly dierent curvatures. The reproducible nature of this systematic error, apparent in the very similar proďŹ les in Fig. 10 for the four dierent distances from ZDL position, means that it could in any case be removed by suitable calibration. The main feature of interest is the noise level. At 5.8 nm and 12.6 nm rms residual with 43

T.V. Reichold, P.D. Ruiz and J.M. Huntley

Optics and Lasers in Engineering 122 (2019) 37–48

Fig. 11. Height map for gauge block step sample.

Fig. 9. Raw phase maps đ?œ™1 (m, n) before unwrapping for gold mirror sample at Δℎ = 50 Îźm (a), 100 Îźm (b), 150 Îźm (c), and 200 Îźm (d).

The base block is a 30 mm Ă— 34 mm Ă— 9 mm gauge block onto which two smaller gauge blocks of dimensions 30 mm Ă— 9 mm Ă— 1 mm and 30 mm Ă— 9 mm Ă— 1.05 mm are wrung together. The average step height over the full step width was 49.765 Îźm Âą 40 nm as determined with a TALYSURF CLI 2000 surface proďŹ ler. Fig. 11 shows the measured areal proďŹ le over a 5 mm Ă— 5 mm region straddling the step as measured using the peak location analysis method. As the PHA is rotated relative to the step edge, some of the pinholes

respect to a 2nd degree polynomial for the cases Δℎ = 50 Îźm and 200 Îźm, respectively, this is over an order of magnitude lower than with the standard peak location analysis method. 4.3. Broadband illumination – height step The second sample was made from calibration blocks so as to contain a well-deďŹ ned surface step, as shown schematically in Fig. 4(b).

Fig. 10. Height maps h (m, n) for gold mirror sample after phase unwrapping at (a) Δℎ = 50 Οm, (b) Δℎ = 100 Οm, (c) Δℎ = 150 Οm and (d) Δℎ = 200 Οm. 44

T.V. Reichold, P.D. Ruiz and J.M. Huntley

Optics and Lasers in Engineering 122 (2019) 37–48

als, to obtain an estimate of the step height. Although higher degree polynomials could have been used as alternative extrapolating functions, a simple plane provided a good ďŹ t to the data on both sides of the step, and with a reduced number of parameters is less susceptible to noise-induced errors at the edges of the extrapolated region. The widths of each zone (a), (b), (c) were respectively 1 mm, 2 mm, 0.5 mm for both top and bottom surfaces. (d) is the sum of all (b) and (c). The average step height across the sample is |â„Žđ?‘ đ?‘Ąđ?‘’đ?‘? | = 49.78 Îźm with a maximum deviation of Âą 57.03 nm for the case Δℎ = 50 Îźm. Corresponding values for Δℎ = 100 Îźm were |â„Žđ?‘ đ?‘Ąđ?‘’đ?‘? | = 49.92 Îźm Âą 67.89 nm; and for Δℎ = 150 Îźm, |â„Žđ?‘ đ?‘Ąđ?‘’đ?‘? | = 50.21 Îźm Âą 83.73 nm. Note that this error refers to the variations in step height along the calculated transition line and does not represent the rms height error of the top or bottom surface which varied typically over the range 100–300 nm. Fig. 12. Gauge block step sample setup shown from the side (left); close-up of region (d) with area of interest (a) around the step (right).

4.4. Broadband illumination – tilting surface

sample simultaneously parts of the top and bottom surface at the same time resulting in noisy data in that region. In order to reduce the error in this region and obtain a good estimate of measured height on each side of the step, an extrapolation technique was used as shown in Fig. 12. A region of interest around the diagonal transition line was bounded as indicated by the green dashed lines at position (a) in Fig. 12. The remaining parts of the top and bottom surfaces were ďŹ tted with 1st degree polynomials to reduce noise as indicated by zone (b). Finally, two-way extrapolation was performed which extrapolates the ďŹ tted top and bottom surfaces backwards and forwards respectively as indicated by zone (c), using the ďŹ rst degree best-ďŹ t polynomi-

The third sample was a silver coated mirror mounted on a PZT driven tilting stage. The mirror rotates a known angle about an axis perpendicular to the optical axis. A tilted sample with 5 mrad nominal tilt using the tilting stage, at 50 Îźm from the ZDL, is shown in Fig. 13. A ďŹ rst degree ďŹ t was applied to the surface map and the unit surface normal vector was used to establish the surface tilt relative to a reference mirror pose also at 50 Îźm from ZDL with nominal 0 mrad tilt. The 0 mrad reference plane shown in Fig. 13 (right, (b)) has a slightly negative tilt value if counter clockwise rotation is deďŹ ned to be positive. The angular shift between any two planes can be calculated from the surface normals of the 1st degree ďŹ tted planes. The results for all tilt angles, relative to the measured 0 mrad reference plane, and at dierent osets from the ZDL are shown in Table 2. Below đ?œƒ = 10 mrad, the measurements between dierent osets are consistent with one another to

Fig. 13. Left: Measured height map for silver mirror at 5 mrad apparent tilt angle, 50 Îźm from ZDL location; Right: Schematic side view, XZ plane of measurement; (a) 0 mrad plane at 50 Îźm from ZDL, (b) measured 0 mrad with residual angle đ?œƒ res to (a), (c) is tilted surface plane with đ?œƒ nom = 5 mrad, đ?œƒ is measured tilt angle relative to (b). Table 2 Tilt sample rms height error, and calculated tilt angle đ?œƒ, for three osets Δh from zero OPD position. Nominal tilt angle / mrad 1 2 3 4 5 10 15

Rms height error / nm

Calculated tilt angle / mrad

Δℎ = 50 Οm

Δℎ = 100 Οm

Δℎ = 150 Οm

Δℎ = 50 Οm

Δℎ = 100 Οm

Δℎ = 150 Οm

161 151 180 212 282 628 615

147 164 184 178 185 273 1315

163 154 191 188 192 340 863

0.80 2.11 3.61 4.14 5.58 11.81 15.13

0.78 2.08 3.61 4.18 5.57 11.82 17.71

0.74 1.99 3.59 4.22 5.44 11.80 17.71

45

T.V. Reichold, P.D. Ruiz and J.M. Huntley

Optics and Lasers in Engineering 122 (2019) 37–48

Fig. 14. Measured height proďŹ les for silver mirror at 0, 1, 2, 3, 4, 5, and 10 mrad tilt angle when the mirror is at 50 Îźm from the ZDL.

within 0.14 mrad. Deviations between the requested and the measured tilt angle are larger at up to 0.6 mrad, however the tilt stage is under open loop control so there is no independent veriďŹ cation of the actual tilt angle achieved. Although the errors increase at and above đ?œƒ = 10 mrad, the results demonstrate that the pinhole-array based system is able to measure surfaces at angles of up to 15 mrad, which is 50% more than the upper theoretical limit on surface tilt angle, đ?œƒđ?‘š(đ?‘€) , as calculated in Section 2.4 for a microlens-array based system.

reduced by over an order of magnitude, to values below 10 nm for a at mirror. Funding Funding for this project was provided by the EPSRC Centre for Doctoral Training in Embedded Intelligence (EP/L014998/1), EPSRC Future Metrology Hub (EP/P006930/1) and by Renishaw Plc. Acknowledgments

5. Conclusions

The authors gratefully acknowledge useful discussions with David McKendrick and Nick Weston, Renishaw Plc, and ďŹ nancial support from Renishaw, the EPSRC Centre for Doctoral Training in Embedded Intelligence and the EPSRC Future Metrology Hub.

An improved hyperspectral interferometry system has been developed, and its performance on optically smooth, stepped and tilted surfaces has been explored in this paper. The new system uses an array of pinholes at the input of an imaging spectrometer to separate spatial and spectral information on a large format photodetector array. 2500 channels can be measured simultaneously, which more than doubles the throughput of the previously described system based on a microlens array and corresponds to a pixel utilization greater than 50%. Using a pinhole array solves the problem of wavefront separation at the pupil plane which is present in the microlens array based system, thus allowing a tilt angle acceptance of Âą33.3 mrad without loss of modulation. A pinhole array of 50 Ă— 50 pinholes with a diameter of 10 Îźm and a pitch of 120 Îźm allows coverage of an area of 5.88 Ă— 5.88 mm2 with a maximum unambiguous height range of 825 Îźm. The use of phase information to calculate surface proďŹ le, as opposed to the usual Fourier peak location method, was investigated in an HSI system for the ďŹ rst time. Although systematic errors were signiďŹ cant, random errors were

References [1] Jalili N, Laxminarayana K. A review of atomic force microscopy imaging systems: application to molecular metrology and biological sciences. Mechatronics 2004;14(8):907–45. [2] Pecheva E, et al. White light scanning interferometry adapted for large-area optical analysis of thick and rough hydroxyapatite layers. Langmuir 2007;23(7):3912–18. [3] Barker A, Syam W, Leach R. Measurement noise of a coherence scanning interferometer in an industrial environment. 31th ASPE annual meeting Portlan , Oregon, USA; 2016. [4] International Organization For Standardization. Geometrical product speciďŹ cations (GPS) – Surface texture: areal – Part 604: nominal characteristics of non-contact (coherence scanning interferometry) instruments. International Organization for Standardization; 2013. p. 41. [5] de Groot P. Principles of interference microscopy for the measurement of surface topography. Advances in Optics and Photonics 2015;7(1):1–65. [6] Schwider J, Zhou L. Dispersive interferometric proďŹ lometer. Optics Letters 1994;19(13):995–7. 46

T.V. Reichold, P.D. Ruiz and J.M. Huntley

Optics and Lasers in Engineering 122 (2019) 37–48 Dr. Pablo D. Ruiz received his BA in Physics in 1996 from the National University of Salta and PhD in 2002 from the Rosario Institute of Physics at the National University of Rosario, Argentina. He joined Loughborough University in 2002, where he currently holds a readership in Applied Optics. His research interests include interferometry, lidar, speckle metrology, optical coherence tomography, synthetic aperture interferometry, elastography and the application of optical methods in experimental mechanics.

[7] Huntley JM, Widjanarko T, Ruiz PD. Hyperspectral interferometry for single-shot absolute measurement of two-dimensional optical path distributions. Measurement Science and Technology 2010;21(7):075304. [8] Widjanarko T, Huntley JM, Ruiz PD. Single-shot profilometry of rough surfaces using hyperspectral interferometry. Optics Letters 2012;37(3):350–2. [9] Ruiz PD, Huntley JM. Single-shot areal profilometry using hyperspectral interferometry with a microlens array. Opt Express 2017;25(8):8801–15. [10] Graulig C, Kanka M, Riesenberg R. Phase shifting technique for extended inline holographic microscopy with a pinhole array. Optics Express 2012;20(20):22383–90. [11] Nakano A. Spinning-disk confocal microscopy – a cutting-edge tool for imaging of membrane traffic. Cell Struct Funct 2002;27(5):349–55. [12] de Groot P, Deck L. Three-dimensional imaging by sub-Nyquist sampling of white– light interferograms. Optics Letters 1993;18(17):1462–4. [13] Huntley JM, et al. Absolute distance from phase in spectral interferometry Opt. express (to be submitted Apr. 2019); 2019.

Prof. Jonathan M. Huntley has been involved in the development and application of full-field optical techniques for over 30 years, including 11 years at the Cavendish Laboratory, Cambridge. Appointed Professor of Applied Mechanics at Loughborough in 1999, he has published over 110 journal papers, two granted patent families, is a Fellow of the Institute of Physics, and was awarded the Institute of Physics Paterson Medal and Prize in 2005.

Tobias V. Reichold received a MEng in Aerospace Engineering from the University of Leicester in 2015. The same year he began his PhD in single-shot hyperspectral interferometry at Loughborough University with the Centre of Doctoral Training in Embedded Intelligence CDT-EI. He currently holds the associate member status with the Institute of Mechanical Engineers IMechE. His research interests include surface profilometry, depth-resolved measurements, optical system design and miniaturization.

47

Publisher’s Note: This is to keep you informed that, to match the layout with the online version, this print version of the article had to be reduced with one page and therefore the remainder of this page is blank. The content of the scientiďŹ c article remained the same.

Optics and Lasers in Engineering 106 (2018) 111–118

Contents lists available at ScienceDirect

Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng

A holistic calibration method with iterative distortion compensation for stereo deflectometry Yongjia Xu a, Feng Gao a,∗, Zonghua Zhang a,b, Xiangqian Jiang a a b

EPSRC Center, University of Huddersfield, HD1 3DH, UK School of Mechanical Engineering, Hebei University of Technology, Tianjin 300130, China

a r t i c l e

i n f o

Keywords: Calibration Algorithms Phase measurement Specular surface Optical metrology Three-dimensional shape measurement

a b s t r a c t This paper presents a novel holistic calibration method for stereo deflectometry system to improve the system measurement accuracy. The reconstruction result of stereo deflectometry is integrated with the calculated normal data of the measured surface. The calculation accuracy of the normal data is seriously influenced by the calibration accuracy of the geometrical relationship of the stereo deflectometry system. Conventional calibration approaches introduce form error to the system due to inaccurate imaging model and distortion elimination. The proposed calibration method compensates system distortion based on an iterative algorithm instead of the conventional distortion mathematical model. The initial value of the system parameters are calculated from the fringe patterns displayed on the systemic LCD screen through a reflection of a markless flat mirror. An iterative algorithm is proposed to compensate system distortion and optimize camera imaging parameters and system geometrical relation parameters based on a cost function. Both simulation work and experimental results show the proposed calibration method can significantly improve the calibration and measurement accuracy of a stereo deflectometry. The PV (peak value) of measurement error of a flat mirror can be reduced to 69.7 nm by applying the proposed method from 282 nm obtained with the conventional calibration approach. © 2018 Published by Elsevier Ltd.

1. Introduction The three-dimensional (3D) shape measurement of objects is becoming increasingly important in many applications. Many optical measurement methods have been developed for the 3D measurement of diffuse objects [1–3]. However, the measurement of specular surface remains a challenge because of the reflecting property. Stereo deflectometry is a technique to obtain the form information of freeform specular surfaces [4–7]. Stereo deflectometry system generally consists of a screen and two cameras. The screen is most commonly a standard liquid crystal display (LCD) attached to a computer and used to display two groups of orthogonal sinusoidal fringe patterns. The patterns are captured by the cameras simultaneously through the reflection of a target surface. By dealing with the deformed patterns, the form of the target surface can be calculated. For each camera, the normal vector of a space point can be calculated based on a triangular relationship consisting of the space point, the image point in the camera, and the corresponding point in the screen. By searching the space point and matching the normals in terms of the two cameras, a primary 3D data and the corresponding normal vectors of the target surface can be obtained. Then the optimized overall shape of the object is integrated based on the normal vectors [8–9].

∗

Corresponding author. E-mail address: F.Gao@hud.ac.uk (F. Gao).

The normal calculation accuracy and the matching accuracy of normal vectors are very sensitive to systematic errors so the calibration of the stereo deflectometry system plays an important role. Because the screen of the deflectometry cannot be captured directly by the cameras, in order to build the relationship between the cameras and the screen, extra special equipment are introduced for common deflectometry system calibration, such as calibration target [10,11], flat mirror with marker [12–14], and extra cameras and/or screen [15,16]. These equipment not only make the calibration process complex but also bring new error source. Therefore, Werling [17] researched to calibrate a standard deflectometry setup using a flat mirror. Xiao [18] describes a geometrical calibration approach to optimize the relation between the system components by using a markerless flat mirror. Traditional deflectometry calibration approaches [4,5,12,13,19–23] separate the whole system into several components or steps, where each step has to deal with noisy data and the inaccurate result propagated from the previous calibration step. Moreover, system parameters are optimized based on different cost functions in terms of different steps. The error propagation of calibration steps and the inconsistency of optimization functions result in large systematic deviations for the global calibration parameters. Therefore, holistic calibration approaches are researched to enhance calibration accuracy. Olesch and Faber et al. [24–26] proposed a self-calibration method for deflectometry which can reduce the global error of a flat mirror from 3 μm to under 1 μm. By applying the normal vector of an optical flat mirror as an intermediate variable, Ren

https://doi.org/10.1016/j.optlaseng.2018.02.018 Received 10 January 2018; Received in revised form 23 February 2018; Accepted 27 February 2018 Available online 7 March 2018 0143-8166/© 2018 Published by Elsevier Ltd.

Y. Xu et al.

Optics and Lasers in Engineering 106 (2018) 111–118

Fig. 3. The simulation setup. Fig. 1. The illustration of stereo deflectometry.

search a calibration method that can eliminate distortion more effectively and achieve higher calibration accuracy for stereo deflectometry. This paper proposes a novel holistic calibration method for stereo deflectometry based on the iterative distortion compensation algorithm proposed by Xu et al. [30]. A markless flat mirror is used to reflect the fringe patterns on the screen to the cameras during the calibration process. Initial system parameters are estimated based on the captured patterns. Then an iterative algorithm is proposed to compensate system distortion. During the iterative process, camera imaging parameters, system geometrical relation parameters and system distortion are optimized based on a cost function.

et al. [27] proposed an iterative optimization calibration approach for a stereo deflectometry. The global measurement error of a 2 in. flat mirror is near 100 nm in their experiment. The calibration approaches mentioned above all apply traditional distortion compensation model researched for camera calibration [28,29] to deal with the distortion in deflectometry. However the accuracy of traditional distortion compensation approach is limited by the matching accuracy between the established mathematical model and the real distortion. Though the accuracy of the conventional approach is acceptable for the general calibration requirement of the system with micron level measurement accuracy, it is one of the error source that hinder the stereo deflectometry to reach nanometer level measurement accuracy. Therefore, it is urgent to re-

Fig. 2. The flowchart of the proposed calibration method. 112

Y. Xu et al.

Optics and Lasers in Engineering 106 (2018) 111–118

Fig. 4. The simulated distortion on the pixel coordinate. (a) Radial distortion; (b) eccentric distortion; (c) thin prism distortion; (d) the overall distortion caused by radial distortion, eccentric distortion and thin prism distortion.

Fig. 5. Calibration results of the simulated system. (a) reprojection error with no distortion data; (b) reprojection error with distortion using conventional approach; (c) reprojection error with distortion using the proposed method.

113

Y. Xu et al.

Optics and Lasers in Engineering 106 (2018) 111–118

Fig. 6. Measurement error of the simulated at mirror. (a) result based on the true system parameters; (b) result using conventional approach; (c) result using the proposed method.

Fig. 7. The experiment setup.

2. Principle The components of a stereo deectometry include a LCD screen and two CCD cameras as demonstrated in Fig. 1. The image of the screen after the reection of a at mirror is also represented in Fig. 1 for better description. The calibration task for the stereo deectometry is to obtain an accurate relative position relation of the components and the imaging projection of cameras. Based on pinhole imaging model,A1 andA2 are the cameras’ internal parameters which represent the transformation from camera coordinate system {C1 }, {C2 } to pixel coordinate system {P1 }, {P2 } respectively. [R1 t1 ] and [R2 t2 ] are the coordinate transformation from a screen coordinate system{L} to {C1 } and {C2 } respectively. [Rc tc ] is the transformation from {C1 } to {C2 }, which can be derived from [R1 t1 ] and [R2 t2 ]. Since the screen is not in the view of the cameras, a markless at mirror is applied to guarantee the screen can be captured by the cameras during the calibration process. Two groups of mutually perpendicular phase-shifting fringe patterns are displayed on the screen in sequence. The patterns are coded with dierent fringe frequencies [17] and are captured by the cameras simultaneously through the reection of the mirror. After applying phase-shifting and phase unwrapping technique [32,33], two mutually perpendicular absolute phase maps for each camera are obtained. For one camera pixel m(u, v), the corresponding physical position M′(xw , yw ) in the mirrored LCD coordinate system can be uniquely located based on its phase value (đ?œ‘x , đ?œ‘y ) by using the below

equation: { đ?‘Ľđ?‘¤ = (đ?‘›đ?‘? â‹… đ?‘?∕2đ?œ‹) â‹… đ?œ‘đ?‘Ľ đ?‘Śđ?‘¤ = (đ?‘›đ?‘? â‹… đ?‘?∕2đ?œ‹) â‹… đ?œ‘đ?‘Ś

(1)

where p is the size of LCD pixel pitch and np is the number of LCD pixels per fringe period. Without considering distortions, camera parameter A and the relation [R′ t′] between mirrored screen {L′} and camera coordinate system can be obtained based on the pinhole model from at least three arbitrary calibration poses of the mirrored screen: [ đ?‘ đ?‘š = đ??´ â‹… đ?‘…′

] �′ ⋅ � ′

(2)

The relation [R t] between one camera and the real screen can be calculated with the least-squares solution by at least three mirror reections according to the following equation: { ′ đ?‘… = (đ??ź − 2đ?‘›đ?‘›đ?‘‡ )đ?‘… (3) đ?‘Ąâ€˛ = (đ??ź − 2đ?‘›đ?‘›đ?‘‡ )đ?‘Ą + 2đ?‘‘đ?‘› where n is the mirror normal vector expressed in camera frame, d is the distance from the mirror to the camera center. n can be obtained based on the equation below ⎧ ⎪đ?‘›đ?‘– = (đ?‘’đ?‘–đ?‘˜ Ă— đ?‘’đ?‘–đ?‘— )∕‖đ?‘’đ?‘–đ?‘˜ Ă— đ?‘’đ?‘–đ?‘— ‖ ⎪ ⎨đ?‘›đ?‘— = (đ?‘’đ?‘—đ?‘– Ă— đ?‘’đ?‘—đ?‘˜ )∕‖đ?‘’đ?‘—đ?‘– Ă— đ?‘’đ?‘—đ?‘˜ ‖ ⎪ ⎪đ?‘› = (đ?‘’đ?‘–đ?‘˜ Ă— đ?‘’đ?‘—đ?‘˜ )∕‖đ?‘’đ?‘–đ?‘˜ Ă— đ?‘’đ?‘—đ?‘˜ ‖ 114 ⎊ đ?‘˜

(4)

Y. Xu et al.

Optics and Lasers in Engineering 106 (2018) 111–118

Fig. 8. Reprojection error of the cameras on one calibration position. (a) error along x direction on camera 1 using conventional approach; (b) error along y direction on camera 1 using conventional approach; (c) error along x direction on camera 2 using conventional approach; (d) error along y direction on camera 2 using conventional approach; (e) error along x direction on camera 1 using the proposed method; (f) error along y direction on camera 1 using the proposed method; (g) error along x direction on camera 2 using the proposed method; error along y direction on camera 2 using the proposed method.

where i, j, k represent three arbitrary mirror positions. eis the unit vector that perpendicular to both two mirror normal vectors, and can be calculated according to the below equation:

where đ??´âˆ— , đ??´âˆ— , đ?‘…∗ , đ?‘Ąâˆ— , đ?‘…∗ and đ?‘Ąâˆ—2 represent the optimized value of A1 , A2 , 1 2 1 1 2 R1 , t1 , R2 and t2 . e1 and e2 represent the deviation between m∗ and đ?‘šĚ‚ of camera 1 and camera 2 respectively:

⎧(đ?‘…′ − đ?‘…′ )đ?‘‡ â‹… đ?‘’đ?‘–đ?‘— = 0 đ?‘— ⎪ đ?‘–′ ′ đ?‘‡ ⎨(đ?‘…đ?‘— − đ?‘…đ?‘˜ ) â‹… đ?‘’đ?‘—đ?‘˜ = 0 ⎪( đ?‘… ′ − đ?‘… ′ ) đ?‘‡ â‹… đ?‘’ = 0 đ?‘–đ?‘˜ ⎊ đ?‘– đ?‘˜

đ?‘’1 =

(5)

đ?‘” ∑ đ?‘˜ ∑ ‖ ∗ ‖ ‖đ?‘š1 (đ??´1 , đ?‘…1 , đ?‘Ą1 , đ?‘›đ??żđ?‘– , đ?‘‘đ??ż đ?‘– , đ?‘€1 đ?‘– đ?‘— )− đ?‘šĚ‚ 1 (đ??´1 , đ?‘…1 , đ?‘Ą1 , đ?‘›đ??żđ?‘– , đ?‘‘đ??ż đ?‘– , đ?‘€1 đ?‘– đ?‘— )‖ ‖ ‖ đ?‘–=1 đ?‘—=1

(10)

The remaining unknown parameter d in Eq. (3) can be solved by a linear equation as ⎥đ?‘ĄâŽ¤ đ?‘‡ ′ ⎥ ( đ??ź − đ?‘› đ?‘– đ?‘› đ?‘– ) 2 đ?‘› đ?‘– 0 0 ⎤⎢ ⎼ ⎥ đ?‘Ą đ?‘– ⎤ ⎢ ( đ??ź − đ?‘› đ?‘— đ?‘› đ?‘‡ ) 0 2 đ?‘› 0 ⎼⎢ đ?‘‘ đ?‘– ⎼ = ⎢ đ?‘Ą ′ ⎼ đ?‘— đ?‘— ⎢ ⎼⎢ đ?‘‘ ⎼ ⎢ đ?‘— ⎼ ⎣(đ??ź − đ?‘›3 đ?‘›đ?‘‡3 ) 0 0 2đ?‘›đ?‘˜ ⎌⎢ đ?‘— ⎼ ⎣đ?‘Ąâ€˛đ?‘˜ ⎌ ⎣đ?‘‘đ?‘˜ ⎌

(6)

đ?‘’2 =

đ?‘–=1 đ?‘—=1

(11)

where I is a 3 Ă— 3 identity matrix. Until now, the initial values of system parameters have been estimated. Ideal imaging model is a linear projection. For one camera pixel m, distortion will cause a deviation Δm between the real coordinate and the reprojection đ?‘šĚ‚ : Δđ?‘š = đ?‘šĚ‚ (đ??´, đ?‘…, đ?‘Ą, đ?‘›, đ?‘‘, đ?‘€) − đ?‘š

where M1 and M2 are the corresponding physical point in the screen of camera 1 pixel m1 and camera 2 pixel m2 respectively. đ?‘šâˆ—1 and đ?‘šĚ‚ 1 are the corrected pixel and the reprojection pixel of camera 1 respectively. đ?‘šâˆ—2 and đ?‘šĚ‚ 2 are the corrected pixel and the reprojection pixel of camera 2 respectively. g is the number of calibration poses and k is the number of camera pixels. nL is the mirror normal vector expressed in LCD frame and dL is the distance from the mirror to the LCD center, which unify the normal and position of the mirror in terms of the two cameras into the LCD coordinate system and can be calculated based on the following equation:

(7)

where M is the corresponding point on the real screen. Because distortion is a systematic error, the deviation should be a ďŹ xed value for dierent calibration positions. Dierent Δm can be obtained from the dierent calibration positions. Using Δđ?‘š represents the average value of the Δm. By compensating the distortion with the Δđ?‘š according to Eq. (8), a corrected pixel m∗ matching the linear model can be obtained: đ?‘š = đ?‘š + Δđ?‘š ∗

{

(8)

1

2

1

1

2

đ?‘›đ??ż = đ?‘…−1 đ?‘› đ?‘‘đ??ż = đ?‘‘ − đ?‘› â‹… đ?‘Ą

(12)

Therefore, all system parameters can be adjusted in coordination according to one cost function. The calculated system parameters and the corresponding deviation Δ� which make the cost function converge into the minimization will be saved and applied in the future measurement process. The proposed calibration process is summarized as Fig. 2.

Then A1 , A2 , R1 , t1 , R2 and t2 are iteratively recalculated by minimizing the deviation between m∗ and đ?‘šĚ‚ according to the following function with Levenberg-Marquardt Algorithm: ‖ ‖ ‖ [đ??´âˆ— , đ??´âˆ— , đ?‘…∗ , đ?‘Ąâˆ— , đ?‘…∗ , đ?‘Ąâˆ—2 ] = min(‖ ‖đ?‘’1 ‖ + ‖đ?‘’2 ‖)

đ?‘” ∑ đ?‘˜ ∑ ‖ ∗ ‖ ‖đ?‘š2 (đ??´2 , đ?‘…2 , đ?‘Ą2 , đ?‘›đ??żđ?‘– , đ?‘‘đ??ż đ?‘– , đ?‘€2 đ?‘– đ?‘— )− đ?‘šĚ‚ 2 (đ??´2 , đ?‘…2 , đ?‘Ą2 , đ?‘›đ??żđ?‘– , đ?‘‘đ??ż đ?‘– , đ?‘€2 đ?‘– đ?‘— )‖ ‖ ‖

(9) 115

Y. Xu et al.

Optics and Lasers in Engineering 106 (2018) 111–118

Fig. 9. Slopes of a at mirror calculated based on the two calibration methods. (a) Slopes along x direction using the conventional approach; (b) slopes along y direction using the conventional approach; (c) slopes along x direction using the proposed method; (d) slopes along y direction using the proposed method.

3. Experiments

with the proposed method is shown in Fig. 5(c). Obviously, the reprojection error of the proposed method is similar to the accurate value in Fig. 5(a) and much better than the result of the traditional method. After calibration, a at mirror was simulated to be measured based on the calibration results. A measurement result obtained with the true system parameters is demonstrated in Fig. 6(a) as a reference standard. The PV of the measurement error of the proposed method is 28.3 nm as shown in Fig. 6(c), which is very close to the ideal measurement accuracy in Fig. 6(a). While the PV of the measurement error of the Ren’s method is 513 nm as shown in Fig. 6(b), which is much larger than the proposed method. For a clear comparison, the scales of Fig. 6(a), Fig. 6(b) and Fig. 6(c) are set to the same in z direction.

3.1. Simulation A stereo deectometry system with a screen and two cameras was simulated as shown in Fig. 3 to test the proposed calibration method. The size of the simulated camera image was 1280 × 960 pixels with the principal point at (640,480). Through the reection of a at mirror, two orthogonal absolute phase maps on the screen were displayed in turn and captured by the cameras simultaneously. The system parameters can be calibrated from the cameras pixels and the corresponding physical positions in terms of the mirrored screen coordinate system. In order to simulate the real experimental environment, random location error with a maximum value of 0.005 mm was added to the camera pixels’ corresponding physical locations on the screen. A simulated overall distortion caused by radial distortion, eccentric distortion and thin prism distortion was added to the camera images as shown in Fig. 4. Generally, reprojection error is used as the evaluation criteria of the calibration accuracy. To evaluate the accuracy of calibration methods, a standard reprojection error was acquired when no distortion is in the system, as shown in Fig. 5(a). To make a clear demonstration, the reprojection error was transformed from 2D matrix into a vector. Ren’s approach [27] is a typical calibration algorithm based on traditional distortion compensation model and can achieve competitive measurement accuracy compared to other calibration techniques. Therefore, a calibration result was obtained by using Ren’s approach as a comparison, as shown in Fig. 5(b). The reprojection error calculated

3.2. Experiments and discussion To verify the proposed method, two experiments have been conducted on a stereo deectometry system. Since the measurement accuracy is inuenced by the setup conďŹ guration of the deectometry system [31], the experiments are conducted based on the same hardware system as shown in Fig. 7. The camera is Lumenera CCD sensor (Model Lw235M) with a resolution of 1616 Ă— 1216. The camera lens is a Navitar lens (Model MVL35M23) with 35 mm ďŹ xed focal length. The diagonal full ďŹ eld of view of the lens is 17.9°. A 12.9 in. iPad Pro is applied as the screen of the stereo deectometry. Under the control of Display Duet, the iPad Pro can display fringe patterns generated by a computer. During the calibration process, a 4 in. mirror with đ?œ†/20atness was placed at 9 dierent positions. The global form error of the mirror is very small and can be within 30 nm. The calibration accuracy is aected by 116

Y. Xu et al.

Optics and Lasers in Engineering 106 (2018) 111–118

Fig. 10. Measurement error of a at mirror. (a) Conventional approach; (b) the proposed method.

Fig. 11. Measurement of a concave mirror. (a) the obtained surface; (b) the measurement error.

a number of factors such as phase error, form error of the calibration mirror and imperfect performance of the screen used in the calibration process. The phase error comes from the electronic noise and nonlinear response of the electronic devices used in the calibration process, which has very signiďŹ cant impact on the calibration accuracy. Therefore, comparing with the inuence of the phase error, the inuence of the form error of the used calibration mirror is very small and can be ignored. Absolute phase maps were obtained by using an eight-step phase shifting algorithm and the optimum frequency selection method [32,33]. The calibration result has been compared with the conventional calibration approach [27] by using the same calibration input, as shown in Fig. 8. The reprojection errors calculated based on the traditional approach and the propose method are shown in Fig. 8(a), (b) and Fig. 8(c), (d) respectively. It is clear that there is an obvious form error when using the traditional approach due to inaccurate calibration. The form error will result in a measurement error of the system. In contrast, the reprojection error using the proposed method is close to random noise, except some invalid points. A 2 in. at mirror with đ?œ†/20atness was measured based on the calibration results. The slopes of the surface was calculated based on

the calibration results and shown in Fig. 9 with y–z perspective. Fig. 9(a) and (b) are the slopes calculated based on the traditional calibration approach and Fig. 9(c) and (d) are the slopes calculated based on the proposed calibration approach. Theoretically, the slopes of a at mirror should be a horizontal plane. However, due to inaccurate calibration, the calculated slopes are tilt as shown in Fig. 9(a) and (b), which results in signiďŹ cant reconstruction error of the surface. The error increases signiďŹ cantly with the increase of reconstruction size. In contrast, the proposed calibration method can obviously reduce the slopes error as shown in Fig. 9(c) and (d). A surface was reconstructed with the calculated x, y coordinate value and the normal information based on the integration method proposed by Ren et al. [9]. Since the same integration method was applied, the inuence of the regularization error on the reconstruction result should be the same for the proposed calibration method and Ren’s calibration approach. Because the captured fringe pattern at the edge of the mirror has low quality due to the diraction of light and the manufacture error, the edge of the mirror was not reconstructed. The size of the reconstruction result is near 40 mm. A plane is ďŹ tted base on the surface in order to evaluate measurement results. The PV of the deviation be-

117

Y. Xu et al.

Optics and Lasers in Engineering 106 (2018) 111–118

tween the measurement result and the ďŹ tted plane is 282 nm by using the conventional method, as shown in Fig. 10(a). In contrast, the PV of the deviation using the proposed method is 69.7 nm, as shown in Fig. 10(b). A concave mirror (stock #40-913) from Edmund [34] with đ?œ†/8 surface accuracy was measured based on the proposed method. The eective focal length of the mirror provided by the manufacturer is 635 mm. The focal length tolerance of the mirror is Âą 2%. The measurement result of the mirror is shown in Fig. 11(a). A standard sphere is ďŹ tted base on the measurement result to evaluate the result. The eective focal length of the ďŹ tted sphere is 633.1 mm which is within the focal length tolerance. The PV and the RMS of the deviation between the measurement result and the ďŹ tted sphere are 96.8 nm and 17.1 nm respectively, as shown in Fig. 11(b).

[5] Balzer J, Hofer S, Beyerer J. Multiview specular stereo reconstruction of large mirror surfaces. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR); 2011. p. 2537–44. [6] Fischer M, Petz M, Tutsch R. Model-based deectometric measurement of transparent objects. In: EOSAM, 2014; 2014. p. 15–19. [7] Kaminski J, Lowitzsch S, Knauer M, Häusler G. In: Osten W, editor. Full-ďŹ eld shape measurement of specular surfaces. Berlin Heidelberg: Springer; 2006. p. 372–9. [8] Huang L, Asundi A. Improvement of least-squares integration method with iterative compensations in fringe reectometry. Appl Opt 2012;51:7459–65. [9] Ren H, Gao F, Jinag X. Least-squares method for data reconstruction from gradient data in deectometry. Appl Opt 2016;55:6052–9. [10] Soumelidis A, Fazekas Z, Bodis-Szomoru A, Schipp F, Csakany B, Nemeth J. Specular surface reconstruction method for multi-camera corneal topographer arrangements. Recent Adv Biomed Eng 2009:639–60. [11] Huang L, Ng CS, Asundi AK. Dynamic three-dimensional sensing for specular surface with monoscopic fringe reectometry. Appl Opt 2011;19:12809–14. [12] Huang L, Xue J, Gao B, Mcpherson C, Beverage J, Idir M. Modal phase measuring deectometry. Opt Express 2016;24:24649–64. [13] Breitbarth M, KĂźhmstedt P, Notni G. Calibration of a combined system with phase measuring deectometry and fringe projection. Int Soc Opt Photon 2009;7389:738909. [14] Balzer J, Werling S. Principles of shape from specular reection. Measurement 2010;43:1305–17. [15] Bonfort T, Sturm P. Voxel carving for specular surfaces. In: Proceedings of the Ninth IEEE International Conference on Computer Vision; 2003. p. 591–6. [16] Xiao Y, Su X, You Z. Pose transfer geometrical calibration for fringe-reection optical three-dimensional measurement. Opt Commun 2013;305:143–6. [17] Werling S. Deektometrie zur automatischen sichtprĂźfung und rekonstruktion spiegelnder oberächen. KIT ScientiďŹ c Publishing; 2011. [18] Xiao Y, Su X, Chen W. Flexible geometrical calibration for fringe-reection 3D measurement. Opt Lett 2012;37:620–2. [19] Zhao W, Su X, Liu Y, Zhang Q. Testing an aspheric mirror based on phase measuring deectometry. Opt Eng 2009;48:103603. [20] Huang L, Ng CS, Asundi AK. Fast full-ďŹ eld out-of-plane deformation measurement using fringe reectometry. Opt Lasers Eng 2012;50:529–33. [21] Xiao Y, Su X, Chen W, Liu Y. Three-dimensional shape measurement of aspheric mirrors with fringe reection photogrammetry. Appl Opt 2012;51:457–64. [22] Maldonado AV, Su P, Burge JH. Development of a portable deectometry system for high spatial resolution surface measurements. Appl Opt 2014;53:4023–32. [23] Wedowski RD, Atkinson GA, Smith ML, Smith LN. Dynamic deectometry: a novel approach for the on-line reconstruction of specular freeform surfaces. Opt Lasers Eng 2012;50:529–33. [24] Olesch E, Faber C, Hausler G. Deectometric self-calibration for arbitrary specular surfaces. In: DGaO Proceedings; 2011. [25] Faber C, Olesch E, Krobot R, Häusler G. Deectometry challenges interferometry: the competition gets tougher!. Int Soc Opt Photon 2012;8493:84930R. [26] Faber C. New methods and advances in deectometry. Erlangen ScientiďŹ c Press; 2012. [27] Ren H, Gao F, Jiang X. Iterative optimization calibration method for stereo deectometry. Opt Express 2015;23:22060–8. [28] Zhang Z. A exible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 2000;22:1330–4. [29] https://www.vision.caltech.edu/bouguetj/calib_doc. [30] Xu Y, Gao F, Ren H, Zhang Z, Jiang X. An iterative distortion compensation algorithm for camera calibration based on phase target. Sensors 2017;17:1188. [31] Zhao P, Gao N, Zhang Z, Gao F, Jiang X. Performance analysis and evaluation of direct phase measuring deectometry. Opt Lasers Eng 2018;103:24–33. [32] Towers CE, Towers DP, Jones J. Absolute fringe order calculation using optimised multi-frequency selection in full-ďŹ eld proďŹ lometry. Opt Laser Eng 2005;43:788–800. [33] Zhang Z, Towers CE, Towers DP. Time eďŹƒcient color fringe projection system for 3D shape and color using optimum 3-frequency selection. Opt Express 2006;14:6444–55. [34] https://www.edmundoptics.com/.

4. Summary A holistic calibration method with an iterative distortion compensation algorithm is proposed to calibrate the stereo deectometry. Both simulation and experiments verify the proposed calibration method can eectively eliminate system distortion and signiďŹ cantly improve the calibration accuracy. In comparison with the conventional calibration approach, the measurement accuracy can be considerably enhanced with the proposed calibration method. Acknowledgments The authors gratefully acknowledge the UK’s Engineering and Physical Sciences Research Council (EPSRC) funding of the EPSRC Centre for Innovative Manufacturing in Advanced Metrology (Grand Ref: EP/I033424/1), the EPSRC Future Advanced Metrology Hub (EP/P006930/1), the funding with Grant Ref: EP/K018345/1 and the European Horizon 2020 through Marie Sklodowska-Curie Individual Fellowship Scheme (NO: 707466-3DRM). Supplementary materials Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.optlaseng.2018.02.018. References [1] Zhang Z. Review of single-shot 3D shape measurement by phase calculation-based fringe projection techniques. Opt Lasers Eng 2012;50:1097–106. [2] Zuo C, Huang L, Zhang M, Chen Q, Asundi A. Temporal phase unwrapping algorithms for fringe projection proďŹ lomery: a comparative review. Opt Lasers Eng 2016;85:84–103. [3] Zuo C, Chen Q, Gu G, Feng S, Feng F. High-speed three-dimensional proďŹ lometry for multiple objects with complex shapes. Opt Express 2012;20:19493–510. [4] Knauer MC, Kaminski J, Hausler G. Phase measuring deectometry: a new approach to measure specular free-form surfaces. Int Soc Opt Photon 2004;5457:366–76.

118

Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.