Realistic CG Stereo Image Dataset with Ground Truth Disparity Maps

Page 1

社団法人 電子情報通信学会 THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS

信学技報 IEICE Technical Report

Realistic CG Stereo Image Dataset with Ground Truth Disparity Maps Sarah MARTULL Martin PERIS and

Kazuhiro FUKUI

Graduate School of Systems and Information Engineering, University of Tsukuba 1-1-1 Tennodai, Tsukuba-shi, Ibaraki-ken, 305-8573 Japan E-mail:

info@martull.com,

info@martinperis.com,

kfukui@cs.tsukuba.ac.jp

Abstract Stereo matching is one of the most active research areas in computer vision. While a large number of algorithms for stereo correspondence have been developed, research in some branches of the field has been constrained due to the few number of stereo datasets with ground truth disparity maps available. Having available a large dataset of stereo images with ground truth disparity maps would boost the research on new stereo matching methods, for example, methods based on machine learning. In this work we develop a large stereo dataset with ground truth disparity maps using highly realistic computer graphic techniques. We also apply some of the most common stereo matching techniques to our dataset to confirm that our highly realistic CG stereo images remain as challenging as real-world stereo images. Keyword Stereo Vision,Disparity Map, Computer Graphics, Dataset

1. Introduction Stereo vision is a very active research topic, every year several new stereo matching methods are introduced [1]. The goal of these stereo matching algorithms is to accurately

generate

a

dense

disparity

map,

which

describes the difference in location of corresponding features seen by the left and right cameras. To measure and compare the performance of such algorithms it is essential that the ground truth disparity map is known. Several stereo datasets with known ground truth disparity maps are available [2], [3], but the number of stereo pairs is very limited. This limitation has been

Figure 1: "Head and lamp" scene. Left image and ground truth disparity map. •

Non-occluded

area

mask,

near

depth

discontinuity masks and 3D camera position and orientation on each frame.

constraining the progress of research in some branches of

Among the available datasets, one of the most known and

the field. For example, trying to apply machine learning

extended scenes is the “head and lamp” (Figure 1) stereo

(ML) techniques to solve the stereo matching problem has

dataset developed at University of Tsukuba [3].

been very difficult due to the fact that ML usually requires

For this reason and as a tribute to the early efforts of the

large amounts of data with ground truth for learning and

University of Tsukuba in stereo vision, we modeled our

very few is available. Some efforts on using Computer

scene after the original “head and lamp” set of images.

Graphics (CG) synthetic data have been made [4], but the

Giving the chance to appreciate the scene from points of

simplicity of the generated scenes makes the stereo

view not seen until now, thanks to CG simulation.

matching problem unrealistically easy to solve.

2. Modeling process

In this work we created a highly realistic CG dataset

The scene has been modeled using the software

that properly models real-world imperfections, while

Autodesk Maya 2012. The original stereo pair from

providing accurate ground truth. We contribute a dataset

University of Tsukuba was used as image planes and all

with the following properties:

the geometry of the scene has been modeled using them as

Different illumination conditions.

reference. Figure 2 shows an example of an early stage of

1800 full-color stereo pairs per illumination

the modeling process.

condition with ground truth disparity maps (1

3. Generation of disparity maps and other

minute video at 30FPS using an animated

ground truth data

stereo camera).

After modeling the scene, the output of the render

256 levels of disparity.

engine provides for each frame two RGB images and two

This article is a technical report without peer review, and its polished and/or extended version may be published elsewhere. Copyright ©2012

by

IEICE


Figure 2: Partial model of the scene geometry over the original image. Left: Wire view. Right: Geometry view.

Figure 4: Output of Block Matching algorithm. Left: Original stereo pair. Right: CG stereo pair. based on Block Matching [5] and implemented in OpenCV library [6]. We will use the default parameters of OpenCV in our experiments. Figure 4 shows the output of the Block Matching algorithm for both, the original and the new, stereo pairs. If we compare the results in Figure 4, we can observe that our CG dataset is even more complex than the original, as the original dataset has only 16 different disparity values (which is relatively easy to solve for stereo matching algorithms) and each stereo pair on our CG dataset has up to 256 possible disparity values making it more challenging to find the stereo correspondences.

5. Conclusion Figure 3: First frame of the new dataset (left camera view). Upper-left: RGB image. Upper-right: disparity map. Lower-left: non-occlusion mask. Lower-right: near depth discontinuity mask.

In this work we developed a highly realistic CG stereo dataset with several kinds of ground truth information that can be applied to stereo matching and camera tracking problems. We release it in the hope that it will be useful to further the research in the field of Computer Vision. The

grayscale images. The grayscale images were created

dataset can be found at: http://cvlab.cs.tsukuba.ac.jp

using a custom ramp shader and represent the depth map

6. Acknowledge

of the scene. This depth map can be easily transformed into a disparity map knowing the camera intrinsic parameters (established in the camera properties of the CG software) and the following formula for each pixel on the depth image: d = fT/Z Where d is the disparity of the pixel, f is the focal length of the camera, T is the baseline distance of the stereo rig and Z is the depth value of the pixel. In addition to the ground truth disparity map we provide for each frame: a mask for non-occluded areas and a mask for near depth discontinuity areas (Figure 3). We also provide the 3D position and orientation of the camera on each frame, so the dataset can be used also for visual

camera

tracking

algorithm

development

and

evaluation.

4. Experiments We apply a well known stereo matching algorithm to our dataset and to the original dataset. The algorithm is

This work was supported by KAKENHI (23650081).

References [1] D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1/2/3):7-42, April-June 2002. [2] D. Scharstein and R. Szeliski. High-accuracy stereo depth maps using structured light. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR2003), volume 1, pages 195-202, June, 2003. [3] Y. Nakamura, T. Mastsuura, K. Satoh and Y. Ohta. Occlusion detectable stereo – occlusion patterns in camera matrix. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR1996), pages 371-378. [4] T. Frohlinghaus and J.M. Buhmann. Regularizing phase-based stereo. In Thirteenth International Conference on Pattern Recognition (ICPR1996), pages 451-455. [5] K. Konolige. Small Vision System: Hardware and Implementation. Eighth International Symposium on Robotics Research. October, 1997. [6] OpenCV 2.3, http://opencv.willowgarage.com (January 2012).


Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.