www.as-se.org/ssms
Studies in Surveying and Mapping Science (SSMS) Volume 2, 2014
A General Epipolar-Line Model between Optical and SAR Images and Used in Image Matching Shuai Xing*1, Qing Xu 1, Pingyuan Cui2, Chaozhen Lan1,2, Pengcheng Li1 1
Zhengzhou Institute of Surveying and Mapping, Zhengzhou, China
2
School of Aerospace Engineering, Beijing Institute of Technology, Bei jing, China
*
xing972403@163.com; xq@szdcec.com; cuipy@bit.edu.cn; lan_cz@163.com; lpclqq@163.com
Abstract The search space and strategy are important for optical and SAR image matching. In this paper a general epipolar-line model has been proposed between linear array push-broom optical and SAR images. Then a dynamic approximate epipolar-line constraint model (DAELCM) has been constructed and used to construct a new image matching algorithm with Harris operator and CRA. Experimental results have shown that the general epipolar-line model is valid and successfully used in optical and SAR image matching, and effectively limits the search space and decreased computation. Keywords Photogrammetry; Match; Accuracy; Optical; SAR
Introduction Neither stereoplotting nor block adjustment will succeed without homologous imag e points in photogrammetry. There are usually two ways to achieve homologous i mage points, stereoscopic measuring an d image matching. Stereoscopic measuring is based on stereoscopic viewing and measuring instrument which costs more time an d labour force. Image matching can automatically acquire homologous points by programs and computers. It is so efficient and automatic that it has been wi dely used in photogrammetry. [1]
Lisa Gottesfeld Brown has divi ded i mage matching into three key problems as feat ure space, si milarity measure, search space an d strategy. He also summarized many image matching algorithms. But now most research concentrates on matching with images acquired by the same sensor at different locations, an d good accuracy an d efficiency h as been achieved. But it is still difficult to match i mages acquired by different sensors, especially optical and SAR i mages. Since imaging mechanisms of optical and SAR images are remarkably different, imag e matching algorithms based on pixel grey value are not suitable. But more imag e matching algorithms based on feature have been used to match optical and SAR i mages because features in different imag es are more stable and easily detect ed. [2]
[3]
O. ThÊpaut et al. an d X. Dai et al. have proposed optical and SAR image matching algorithms based on linear [4] features. Paul Maxwell Dare used area features to match optical and SAR i mages. They focused on the first problem [5] in image matching, feature space. Jordi Inglada et al. have summarized si milarity measures suitable for optical an d SAR image matching and proved cluster reward algorithm (CRA) was the best one. But they all have not considered the third problem, search space an d strategy. Epipolar-line constraint is an excellent search strategy in frame remote sensing image matching, and is well used in many commercial digital photogrammetry softwares. But epipolar-line doesn’t exist in linear array push-broom optical and SAR i mages. So in this paper a dynamic approximate epipolar-line constraint model has been proposed, which has been used to con struct a new matching algorithm with Harris operator and CRA, and successfully used in optical an d SAR i mage matching. Two Imaging Models The approximate epi polar-line relationship between linear array push-broom optical and SAR image is derived from their imaging models, so two imaging models will be introduced first . The Scanning Model The linear array push-broom i mage is acquired with linear sensors by scanning the E arth surface. The relationship
26
Studies in Surveying and Mapping Science (SSMS) Volume 2, 2014
www.as-se.org/ssms
between groun d and imag e is rigorous central projection. E ach line of the image has different elements of exterior [8-10] : orientation (EEO). The collinearity equation of i line is expressed in the following way
a1 ( X − X si ) + b1 (Y − Ysi ) + c1 ( Z − Z si ) a3 ( X − X si ) + b3 (Y − Ysi ) + c3 ( Z − Z si ) a ( X − X si ) + b2 (Y − Ysi ) + c2 ( Z − Z si ) 0= −f 2 a3 ( X − X si ) + b3 (Y − Ysi ) + c3 ( Z − Z si )
xi = − f
(1)
Where ai , bi , ci ( i = 1, 2,3 ) are elements of rotation matrix composed of φi , ωi , κ i , which are angle EEOs. F. Le berl Model The geometry relation between ground and i mage points in a SAR image is established by the Doppler and Range [11, 12] . equations proposed by F. Leberl The range equation of slant range image is
( X − Xs ) 2 + (Y − Ys ) 2 + ( Z − Zs= ) 2 ( ys M y + Ds 0 ) 2
(2)
Where Ds0 is the slant range delay, ys is the across-track image coordinate of ground point P , M y is the across-track pixel size, ( X , Y , Z ) is the object space coordinates of groun d point P , ( Xs, Ys, Zs ) is the object space coordinates of radar antenna center and polynomial functions of flight time T .
1 = X S 0 + X V 0T + X a 0T 2 + ⋅⋅⋅⋅⋅⋅ S X 2 Y= Y + Y T + 1 Y T 2 + ⋅⋅⋅⋅⋅⋅ S S0 V0 a0 2 1 Z S 0 + ZV 0T + Z a 0T 2 + ⋅⋅⋅⋅⋅⋅ Z= S 2 = T Kx ⋅ x
(3)
Where X S 0 , YS 0 , Z S 0 are the object space coordinates of radar antenna center of the origin of i mage coordinate, X V 0 , YV 0 , ZV 0 are first order vari ations of Xs, Ys, Zs , X a 0 , Ya 0 , Z a 0 are second order variations of Xs, Ys, Zs , T is flight time of x , x is the along-track image coordinate, K x is scanning time of each line of i mage.
X V , YV , ZV are hypothesized the velocity vectors of radar antenna center at some time, then they can be expressed as = V X Y= V V Z=
∂X S = X V 0 + X a 0T + ⋅⋅⋅⋅⋅⋅ ∂T ∂YS = YV 0 + Ya 0T + ⋅⋅⋅⋅⋅⋅ ∂T ∂Z S = ZV 0 + Z a 0T + ⋅⋅⋅⋅⋅⋅ ∂T
(4)
The Doppler equation is
λ RS
− Xv( X − Xs ) + Yv(Y − Ys ) + Zv( Z − Zs ) =
2
f DC
(5)
Where Rs is the slant range of ground point P , λ is the radar w avel ength, f DC is the Doppler frequency. F. Leberl model is composed of formul a (2) and (5). When
f DC = 0 , they can be lineari zed to achieve EEOs with
GCPs. Construction of a General Epipolar-L ine Model Epipolar-line is an i mportant feature in frame remot e sensing image matching, which is a straight line an d deri ved from collinearity con dition equations. Since homologous image points in a stereoscopic pair must be in their corresponding epipolar-lines, the homologous image point searching can be along the epipolar-line. So computation
27
www.as-se.org/ssms
Studies in Surveying and Mapping Science (SSMS) Volume 2, 2014
can be remarkably reduced and reliability of matching can be i mproved. But there is no rigorous epipolar-line in linear array push-broom optical and SAR imag es because of their imaging mechanism. Considering the performance of epipolar-line constraint, we tried to construct an general epipolar-line based on imaging models of linear array push-broom optical and SAR sensors, and anal yzed the feasibility of i mage matching with it. Here linear array push-broom optical image was regarded as reference image and SAR i mage was reg arded as [6, 7] has been used to con struct the approxi mate epi polar-line. matching image. Projection contrail method Formula (1) can be expressed in the other way:
a1 xl + a2 yl − a3 f u X − X Si =( Z − Z Si ) c x + c y − c f =( Z − Z Si ) w 1 l 2 l 3 + − b x b y b Y − Y = (Z − Z ) 1 l 2 l 3 f = (Z − Z ) v Si Si Si c1 xl + c2 yl − c3 f w
(6)
(5) also can be expressed in the other way when f DC = 0 .
Z=
X V ( X S − X ) + YV (YS − Y ) + ZV Z S ZV
(7)
(6) and (7) are combined then
= Z
[ X V ( X S − X Si ) + YV (YS − YSi ) + ZV Z S ] ⋅ w [ X V ⋅ u + YV ⋅ v] ⋅ Z Si + X V ⋅ u + YV ⋅ v + ZV ⋅ w X V ⋅ u + YV ⋅ v + ZV ⋅ w
(8)
Actually in (3) and (4), X V 0 , YV 0 , ZV 0 and X a 0 , Ya 0 , Z a 0 are so small that they could be neglect ed in the deduction of the general epipol ar-line model. Now (8) i s expressed as
Z = d5 ⋅ xr + d 6
(9)
where
d5 = d6
RV ⋅ K x ⋅ w W
[ X V 0 ( X S 0 − X Si ) + YV 0 (YS 0 − YSi ) + ZV 0 Z S 0 ] ⋅ w [ X V 0 ⋅ u + YV 0 ⋅ v] ⋅ Z Si + W W
RV = X V2 + YV2 + ZV2 W= X V ⋅ u + YV ⋅ v + ZV ⋅ w (9) can be combined with (6) then
X = d1 ⋅ xr + d 2 Y = d3 ⋅ xr + d 4 where
d1 = d2
[ X V 0 X S 0 + YV 0 (YS 0 − YSi ) + ZV 0 ( Z S 0 − Z Si )] ⋅ u [YV 0 ⋅ v + ZV 0 ⋅ w] ⋅ X Si + W W d3 =
d4
RV ⋅ K x ⋅ u W
RV ⋅ K x ⋅ v W
[ X V 0 ( X S 0 − X Si ) + YV 0YS 0 + ZV 0 ( Z S 0 − Z Si )] ⋅ v [ X V 0 ⋅ u + ZV 0 ⋅ w] ⋅ YSi + W W
So the rigorous formula of the general epipolar-line can be expressed as (11) based on (6), (9) an d (10).
28
(10)
Studies in Surveying and Mapping Science (SSMS) Volume 2, 2014
yr =
www.as-se.org/ssms
( D1 ⋅ xr2 + D2 ⋅ xr + D3 − DS 0 )
(11)
My
where
D1 = (d1 − K x ⋅ X V 0 ) 2 + (d3 − K x ⋅ YV 0 ) 2 + (d5 − K x ⋅ ZV 0 ) 2 D2 =2 ⋅ [(d1 − K x ⋅ X V 0 )(d 2 − X S 0 ) + (d3 − K x ⋅ YV 0 )(d 4 − YS 0 ) + (d5 − K x ⋅ ZV 0 )(d 6 − Z S 0 )]
D3 = (d 2 − X S 0 ) 2 + (d 4 − YS 0 ) 2 + (d 6 − Z S 0 ) 2 Dynamic Approximate Epipolar-Line Constraint Model Obviously the approximate epipolar-line between linear array push-broom optical and SAR i mages is not a straight line according to (11). But when we focused on the n eighbor of homologous image points whose length was about 200 pixels, we foun d that the general epipolar-line was very close to a straight line. This means that the general epipolar-line can be substituted by a straight line in a small interval. Then the next problem was how to construct a search space and strat egy based on this straight line. A dynamic approxi mate epipolar-line constraint model (DAELCM) h ad been constructed based on general epipolar-line by projection contrail method. In DAELCM two thresholds d min and d max had been used to calculat e an interval. Then in this interval a fitted straight line had been used to substitute the general epi polar-line, which was called dyn amic approxi mate epi polar-line. The con struction of DAELCM had been fig ured in Fig. 1 in detail. dmax
dmin
Zinitial
EEO of optical image
Coordinates p of image points in optical image
Zmax=Zinitial+dmax
EEO of SAR image
Zmin=Zinitial-dmin
EEO of optical image
Construction of projection rays Gmax(X,Y,Z)
p’max
Gmin(X,Y,Z)
p’min
Construction of the straight line p’minp’max
DAELCM
FIG. 1 CONSTRUCTION OF DAELCM
A New Image Matching Algorith m A new image matching algorithm based on DAELCM h ad been proposed. In the new algorithm, feature point had been chosen as feature space because it was simple, less complex and explicit. Harris operator was a famous feature point extraction operator which had been proposed by C. Harris an d M. J. Stephens. Similarity measure of the new [5] algorithm is CRA . Experimental results have shown that CRA is superior to other similarity measures, such as correlation coefficient, distance to indepen dence and mutual information in optical and SAR i mage matching. Then the new optical and SAR image matching algorithm is construct ed based on Harris operator, CRA an d DAELCM.
29
www.as-se.org/ssms
Studies in Surveying and Mapping Science (SSMS) Volume 2, 2014
Experiment A SPOT-5 and Radarsat-1 satellite remote sensing image had been chosen for experiment. They covered the same area in Beijing, China. The SPOT-5 satellite imag e had been acquired in February 10th 2002, its incidence angle was 2.76 degrees and resolution i s 5m. The Radarsat-1 satellite image had been acquired in November 6th 2002, its along-track resolution was 8.82m an d across-track resol ution was 5.56m. The representations of the same object in SPOT- 5 an d Radarsat-1 images were very different though their resolution was close. So even there are much more textures in detail in urban area, traditional image matching algorithms are hard to succeed. The SPOT-5 imag e had been regarded as a reference image. 90 feature points had been extracted in the SPOT- 5 image. And 35 homologous image points had been measured for matching accuracy eval uation. In experiment we first calculated correspon ding search space in Radarsat-1 imag e of 90 feature points. Here the average elevation had been set 50m according to the topographic of test area, d min and d max had been set -100m and 100m, an d scanning interval was about [-10, 10] pixels. Then corresponding search space of all feature points had been calculated based on DAELCM. We foun d all correct homologous image points in Radarsat-1 imag e were in their search space. We chose three points an d showed the homologous imag e points and their search space in Fig. 2. Finally 48 homologous image points had been matched based on CRA and terrain continuum constraint. Match points of three homologous image points in Fig. 2 have been shown in Fig. 3. The mean square error of 35 check points in x direction was 2.75 pixels an d in y direction was 3.41 pi xels, and the maximum absolute error of matching points in x direction w as 6.41 pixels an d in y direction was 7.35 pi xels. The matching accuracy of the algorithm in this [4] paper is higher than Paul Maxwell Dare in 1999.
(a)
(b)
(c)
FIG. 2 THREE HOMOLOGOUS IMAGE POINTS AND THEIR SEARCH SPACE. THE CROSS IS THE HOMOLOGOUS IMAGE POINT, AND YELLOW POLYGON IS THE SEARCH SPACE.
(a)
(b)
(c)
FIG. 3 MATCH POINTS OF THREE HOMOLOGOUS IMAGE POINTS. GREEN CROSS REPRESENTS HOMOLOGOUS IMAGE POINT, YELLOW DECUSSATION REPRESENTS MATCH POINT.
Conclusion A general epipolar-line model has been proposed an d used to construct a DAELCM in this paper, which is a new search strategy in optical and SAR image matching effectivel y limits the search space and decreases computation. In the experiment the process of matching 90 points costs l ess than 1 second. Th e accuracy of the i mage matching algorithm is equal to stereoscopic measuring, and the algorithm is highly automatic with least manual work. But the number of homologous imag e points achieved by image matching is limited because of the great differences between optical an d SAR i mages. ACKNOWLEDGEMENTS
This work was supported by National Natural Science Foundation Project (41371436).
30
Studies in Surveying and Mapping Science (SSMS) Volume 2, 2014
www.as-se.org/ssms
REFERENCES
[1]
Chang Benyi. “The preliminary research on stereo plotting method of spot imagery.” Agta Geodetica et Cartographica Sinica. 18(3), pp. 183-189. 1989.
[2]
Gong Danchao. “Models and Algorithms on Processing of High-Resolution Satellite Remote Sensing Stereo Images.” Ph. D Thesis, Zhengzhou Institute of Surveying and Mapping. pp. 36-40. 2003.
[3]
He Yu. “SAR digital block triangulation.” Master Thesis, Zhengzhou Institute of Surveying and Mapping. pp. 7-9. 2005.
[4]
Jiaying Liu, Guo Zhang, Deren Li. “Epipolar Resampling of Different Types of Satellite Imagery.” Proceeding of "Mapping without the Sun." ISPRS WG VII/6, VII/7, II/1, Vol. XXXVI Part 7/C54, pp. 272-277. 2007.
[5]
Jordi Inglada, Alain Giros. “On the Possibility of Automatic Multisensor Image Registration.” IEEE Transactions on Geoscience and Remote sensing, 42(10), pp. 2104-2120. 2004.
[6]
Lisa Gottesfeld Brown. “A survey of image registration techniques.” ACM Comput. Surv. 24(4), pp. 325-376. 1992.
[7]
Thépaut, K. Kpalma, and J. Ronsin. “ERS SAR and SPOT images automatic registration in a multichannel consensual segmentation scheme.” IEEE International Geoscience and Remote Sensing Symposium, Proc. IGARSS’98. 1998.
[8]
Paul Maxwell Dare. “New Techniques for the Automatic Registration of Microwave and Optical Remotely Sensed Images.” Ph.D Thesis, University of London. 1999.
[9]
Qian Zeng-bo, Liu Jing-yu, Xiao Guo-chao. “Space Photogrammetry.” Beijing. PLA Press. pp. 63-66. 1992.
[10] X. Dai and S. Khorram. “An feature-based image registration algorithm using improved chain-code representation combined with invariant moments.” IEEE Transactions of Geoscience and Remote Sensing, 37, pp. 2351-2362. 1999. [11] Xiao Guochao and Zhu Cai-ying. “Radargrammetry.” Earthquake Press. pp. 56-58. 2001. [12] Yan Qin, Zhang Zuxun and Zhang Jianqing. “Orientation of remote sensing images taken by CCD from different orbit.” Geomatics and Information Science of Wuhan University. 26(3), pp. 270-274. 2001.
31