Interactive Modeling with Distance Fields by Anders Madsen

Institut für Visualisierung und Interaktive Systeme

Universität Stuttgart Universitätsstraße 38 D–70569 Stuttgart Germany

Diplomarbeit Nr. 2970

Interactive Modeling with Distance Fields Tim-Christopher Reiner

Studiengang:

Informatik

Prüfer:

Prof. Dr.-Ing. Carsten Dachsbacher

Betreuer:

Dipl.-Phys. Gregor Mückl Prof. Dr.-Ing. Carsten Dachsbacher

begonnen am:

17. August 2009

beendet am:

16. Februar 2010

CR-Klassifikation:

I.3.5, I.3.7 Computational Geometry and Object Modeling Three-Dimensional Graphics and Realism

Contents 1 Introduction 1.1 Progressions in Rendering Techniques . . 1.2 A View on Traditional Modeling Software 1.2.1 Representing Curves and Surfaces 1.2.2 Solid Modeling . . . . . . . . . . . 1.3 Motivation to Model with Distance Fields

. . . . .

2 Previous Work and Related Topics 2.1 Modeling with Explicit Distance Fields . . . . . . 2.2 Sculpting . . . . . . . . . . . . . . . . . . . . . . 2.3 Surface Reconstruction . . . . . . . . . . . . . . . 2.4 Rendering Fully Procedural Scenes in 4 Kilobytes

. . . .

9 . 9 . 9 . 9 . 10

. . . . . . . . . . . . . . . . . . . . . . . . .

13 13 14 15 16 16 17 18 20 21 22 23 24 24 26 29 30 31 32 32 33 33 34 34 35 35

3 Dealing with Distance Fields 3.1 Implicit Surfaces at a Glance . . . . . . . 3.2 Euclidean Structure and Notation . . . . 3.3 Fundamental Characteristics . . . . . . . 3.4 Describing Basic Primitives . . . . . . . . 3.4.1 Sphere . . . . . . . . . . . . . . . . 3.4.2 Plane . . . . . . . . . . . . . . . . 3.4.3 Box . . . . . . . . . . . . . . . . . 3.4.4 Box Approximation . . . . . . . . 3.4.5 Cylinder . . . . . . . . . . . . . . . 3.4.6 Cone . . . . . . . . . . . . . . . . . 3.4.7 Torus . . . . . . . . . . . . . . . . 3.5 Essential Modeling Transformations . . . 3.5.1 Rigid Body Transformation . . . . 3.5.2 Uniform and Non-Uniform Scaling 3.5.3 Shear . . . . . . . . . . . . . . . . 3.6 Boolean Algebra . . . . . . . . . . . . . . 3.6.1 Union . . . . . . . . . . . . . . . . 3.6.2 Intersection . . . . . . . . . . . . . 3.6.3 Difference . . . . . . . . . . . . . . 3.7 Effects . . . . . . . . . . . . . . . . . . . . 3.7.1 Morphing and Blending . . . . . . 3.7.2 Twisting . . . . . . . . . . . . . . . 3.7.3 Displacement Mapping . . . . . . . 3.7.4 Infinite Repetitions . . . . . . . . . 3.7.5 Morphological Operations . . . . .

. . . . .

. . . . . . . . . . . . . . . . . . . . . . . . .

. . . . .

. . . . . . . . . . . . . . . . . . . . . . . . .

. . . . .

1 2 4 5 6 6

. . . . . . . . . . . . . . . . . . . . . . . . .

Contents 3.8

3.9

References to Advanced Primitives . 3.8.1 Ellipsoid . . . . . . . . . . . . 3.8.2 Hypertexture . . . . . . . . . 3.8.3 Blend of Objects . . . . . . . Assimilating Explicit Distance Fields

. . . . .

36 36 36 37 37

4 Ray Marching Distance Fields 4.1 The Concept of Ray Marching . . . . . . . . . . . . . . . . . . 4.2 Adaptive Marching Step Size Dependent on Distance . . . . . . 4.2.1 Enhancements . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Strategies to Tackle Deviations from Euclidean Distance 4.3 Ray Marching in a Shader . . . . . . . . . . . . . . . . . . . . . 4.4 Illumination and Shading . . . . . . . . . . . . . . . . . . . . . 4.4.1 Gradient Estimation . . . . . . . . . . . . . . . . . . . . 4.4.2 Phong Illumination Model . . . . . . . . . . . . . . . . . 4.5 Texturing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.1 2D Texture Mapping . . . . . . . . . . . . . . . . . . . . 4.5.2 Procedural and Solid Textures . . . . . . . . . . . . . . 4.6 Shadowing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6.1 Casting Shadow Rays . . . . . . . . . . . . . . . . . . . 4.6.2 Soft Shadows . . . . . . . . . . . . . . . . . . . . . . . . 4.6.3 Ambient Occlusion . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

39 39 41 42 43 43 44 44 45 46 47 50 50 50 51 52

5 Implementation: Interactive Modeling with Distance Fields 5.1 Development Tools and Libraries . . . . . . . . . . . . . 5.1.1 Qt 4.5 . . . . . . . . . . . . . . . . . . . . . . . . 5.1.2 Computational Geometry Algorithms Library . . 5.2 Main Window Framework . . . . . . . . . . . . . . . . . 5.2.1 Central Modeling Widget . . . . . . . . . . . . . 5.2.2 Settings Panel . . . . . . . . . . . . . . . . . . . 5.2.3 Materials Panel . . . . . . . . . . . . . . . . . . . 5.2.4 Shader Panel . . . . . . . . . . . . . . . . . . . . 5.3 The Model-View-Controller Concept . . . . . . . . . . . 5.3.1 Additional Cameras . . . . . . . . . . . . . . . . 5.3.2 Multiple Viewports . . . . . . . . . . . . . . . . . 5.4 The Scene Graph . . . . . . . . . . . . . . . . . . . . . . 5.4.1 Adding New Primitives to the Scene . . . . . . . 5.4.2 Adding Transformations . . . . . . . . . . . . . . 5.4.3 Rearranging Items . . . . . . . . . . . . . . . . . 5.4.4 Shader Generation . . . . . . . . . . . . . . . . . 5.5 Hybrid Rendering . . . . . . . . . . . . . . . . . . . . . . 5.6 Picking . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7 Manipulators . . . . . . . . . . . . . . . . . . . . . . . . 5.7.1 Interactive Translation . . . . . . . . . . . . . . . 5.7.2 Interactive Rotation . . . . . . . . . . . . . . . . 5.7.3 Interactive Scaling . . . . . . . . . . . . . . . . . 5.8 Constructive Solid Geometry . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

55 55 55 56 56 58 59 61 61 62 63 63 64 65 65 66 66 67 70 72 72 74 75 76

. . . . . . . . . . . . . . . . . . . . . . .

Contents 5.9

Applying Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.9.1 Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.9.2 Color Textures . . . . . . . . . . . . . . . . . . . . . . . . . . 5.9.3 Displacement Maps . . . . . . . . . . . . . . . . . . . . . . . 5.10 Visual Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.11 Closing the Gap to Traditional Modeling . . . . . . . . . . . . . . . . 5.11.1 Converting Triangle Meshes into Discretized Distance Fields 5.11.2 Generating Triangle Meshes from Distance Fields . . . . . . .

iii . . . . . . . .

. . . . . . . .

78 78 78 79 81 82 83 86

6 Conclusions 6.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

89 89 90 91

Acknowledgements

Bibliography

List of Figures

1 Introduction Modeling and describing three-dimensional objects is one of the essential topics in computer graphics. A conventional approach to produce a computer-generated image is to render a given scene. Prior to that, various objects have been arranged within the scene, camera settings were specified, and lighting has been set up. Modeling tools can support these early stages of the overall process. As a start, they allow the user to create and design three-dimensional models in an intuitive way, and deliver an interpretable mathematical representation of these objects. In addition, they can help to arrange whole scenes. Hence, modeling tools are of great value to artists in order to express their creativity, and supersede the work of manually writing mathematical object descriptions and arrangements. There are many dedicated modeling programs available, Autodesk Maya for instance, a commercial high-end and very comprehensive modeling package used by many professionals, or Blender, a mature modeling tool released as free software. Most of them use popular and widespread ways to represent and internally store a model. Certainly, the most common way is to use a basic polygonal mesh, which is very robust and flexible, but has its drawbacks. Polygonal meshes render fast and nicely using traditional rendering methods on consumer graphics hardware—but look clumsy in a low resolution or if the viewer zoomed in. This diploma thesis introduces a novel internal representation of objects for modeling tools. We will use distance fields as an implicit way to describe surfaces. A distance field can be expressed by a function that returns the Euclidean distance to the nearest surface for a given point in space. A surface emerges where the distance field evaluates to zero. This characteristic is shared with the related algebraic surfaces, whose equations are true for points on the described surface. Modeling with distance fields yields various questions that have to be discussed. What are the benefits of using them instead of established model representations? Which modeling aspects are simplified, which become more complicated instead? Do distance fields allure with unique features which are impractical to reproduce with traditional approaches? And, how do they perform and how are they related to the other popular ways of modeling? The author developed a small modeling tool based on distance fields as an accompanying part of this thesis, that will help us to find answers to these questions for the time being. It is introduced and thoroughly explained within its own dedicated chapter. Prior to that, we deal with basic principles the modeler is based upon. Another focus of this thesis will be on interactivity, which plays a key role for our modeling tool. For now, it should already be obvious that a way of real-time rendering of distance fields is required, in order to provide an interactive view of the scene the user is currently working on, and to give immediate visual feedback after the user made any changes. An approach where the user has to manually request a time-consuming re-rendering of the scene after making changes would be considered as unproductive and certainly not convenient. Due to the fact that the modeling tool is not based upon polygonal meshes for internal representation of

Introduction

objects, classical rendering methods are not applicable. The rendering technique the modeler uses, called ray marching, reveals that it is suited very well to cope with distance fields. The next section of this chapter introduces various fundamental techniques and terms that will be referred to quite often throughout the thesis. The introduction goes on by stating the motivation to model with distance fields and comparing it to established modeling approaches. We continue with a broad overview of previous work in related topics. The sophisticated main chapters start with a thorough examination of distance fields. Based upon the insight gained, ray marching strategies to render distance fields in real-time are discussed. After that, we complete the main part by giving a broad survey of the novel approach to model with distance fields and all its peculiarities, and eventually by presenting and explaining our modeling implementation. Conclusions, some ideas for future work and a few remarks complete the thesis.

1.1 Progressions in Rendering Techniques Within the extensive field of computer graphics, one can perceive two contrarious approaches throughout the history of image synthesis, also known as rendering, the process of generating an image out of a described scene containing various models. One intention is to produce realistic looking images of high quality, simulating a variety of natural processes and occurrences. Usually, this is based upon a physical description of light transport and diffusion, the way light is absorbed, reflected and refracted at objects and surfaces, and mutual interferences among objects. The outcome can even be photo-realistic and stunning to the observer. Due to their rather long computational time, sophisticated rendering techniques which resemble real physical processes are most suitable for producing single, high-class images—used in publications, illustrations or advertisements for instance. Another field of application is Computer-generated imagery (CGI), where movies are enriched by special effects, as well as fully computer-generated movies in general. In the latter case, it’s quite obvious that image quality is the topmost priority, rather than rendering time1 . On the contrary, there are numerous cases where an oppositional approach is required. Computer games are a self-evident example where one of the major objectives is to provide an interactive frame rate2 . Clearly, image quality has to suffer if rendering time is strongly limited to a couple of milliseconds. Instead of thoroughly following precise physical models, we rely on plausible approximations and extreme simplifications in order to receive a result that is still overall satisfying. The real-time approach consistently used by all modern graphics cards here is classical triangle rasterization. The scene to be rendered this way consists of polygonal meshes, refined into triangle meshes, more precisely. Today’s graphics libraries like OpenGL 1

The first movie using only computer-generated imagery, Toy Story, was released in 1995 by Pixar Animation Studios. According to an interview with Silicon Valley ACM SIGGRAPH, Pixar employees stated that a typical rendering time was 4 hours a frame—rendered at 1536×922—on multiple racks of Sun workstations. For complex scenes, rendering time expanded up to 13 hours for a single frame. 2 The human visual system requires a frame rate of at least 24 frames per second in order to perceive an illusion of flawless motion in movies, television shows and computer games. Note that constructed cases like immediate switching between black and white can require a considerably higher frame rate to imitate a continuous flow of lighting information. Interactive systems, to be discussed later in this thesis, are mostly considered as interactive if frames per second are beyond 15.

1.1 Progressions in Rendering Techniques

or Microsoft DirectX implement a so-called immediate mode, where client commands are directly executed and geometry is rendered to the screen without further delay. Expressed in a sloppy way, the programmer sends every triangle to be drawn to the graphics API, where it is processed throughout the rendering pipeline and eventually rasterized into the framebuffer. Several years ago, the two aforementioned approaches were clearly distinguishable. On the one hand, the ambitious goal to produce realistic images, based upon widely accepted physical models, regardless of a long rendering time, and on the other hand, the rock-solid requirement to deliver interactive frame rates, squeezing in as much realism and complexity as possible. Two widely used families of rendering techniques—radiosity and ray tracing—have traditionally been used for physics-based, high-quality image synthesis. Both rely heavily on the radiometric measure called radiance, being the most important quantity in this case, that characterizes emission and reflection, specifically the amount of light that is received on or emitted from a particular surface. Furthermore, both take global illumination into account, and simulate interdependencies of light between objects. This way, surfaces can receive not only direct illumination from a light source, but indirect illumination from reflected and scattered light as well. A prime example often referred to is light leaking around a corner, or brightened areas under a desk, illuminated from the top. Radiosity is a method based on the theory of thermal radiation. The scene is subdivided into discrete patches and each patch is considered to have its received and emitted amount of light equally distributed along its surface. Illuminated patches can reflect light and thus illuminate other patches in an indirect way. We will refer to radiosity a few times during this chapter and compare its characteristics to those of other rendering techniques, but are not going more into detail, as radiosity doesn’t play any notable role throughout this thesis. Ray tracing generates images by shooting rays out of a camera through the image plane, where the scanned scene is eventually projected. These rays follow the path of light, in a reverse manner for the most simple algorithm, from the camera, reflected at objects, to light sources. The family of ray tracing algorithms plays a significantly more important role than other rendering techniques for the topic covered by this thesis, due to the fact that a variant is going to be used for rendering most of the images to be presented here. The algorithms will be explained later in their own dedicated chapter. It turns out that radiosity, an application of the finite element method due to the discrete nature of its diffuse patches, advanced ray tracing techniques, often tackled by Monte Carlo methods in order to cope with their dimensional complexity, as well as most rendering algorithms, eventually attempt to solve the all-embracing rendering equation. This integral equation describes the conservation of energy during light diffusion and is accepted as the mathematical foundation of all global illumination algorithms—see [29], the 1986 Siggraph paper by Kajiya, one of the pioneers in the field of computer graphics. Note that one is generally not restricted to a single rendering technique. Quite the contrary, a variety of different methods can be utilized to generate a final image. Imagine a real-time walk trough architecture, stored as a triangle mesh and rendered interactively with OpenGL, but with fixed global lighting information at the vertices, precomputed with radiosity methods for instance. In fact, professional rendering applications combine a broad range of techniques. There is another important property of rendering algorithms in computer graphics—they can be categorized into image order and object order algorithms. A prime example of an image

Introduction

order algorithm is ray tracing, which iterates over the pixels in the image plane and shoots one up to several rays through each pixel. On the contrary, object order algorithms iterate over the elements, i.e. geometric primitives, in the scene, like the OpenGL immediate mode, where triangle after triangle is rasterized to the screen. An object order approach is more efficient in most cases, where there are considerably fewer geometric primitives than pixels in the image plane. However, if dealing with complex geometry like in volume rendering, for instance, an image order approach is potentially faster. A new era in computer graphics has begun since programmable graphics hardware was introduced. Modern graphics cards allow the execution of small programs, called shaders in this context, to process the vertex transformations, and most important, to determine the color of each fragment that is projected to the screen. By rasterizing primitives using a fragment shader, which is executed for each fragment—that is pixel-by-pixel—the traditional object order approach is being overlaid by an image order technique. The developed modeling application described in this thesis is based heavily on various fragment shaders and they will play a key role throughout the main chapters. The flexibility and computational power of modern programmable graphics hardware eventually led to the establishment of GPGPU methods, i.e. general-purpose computing on graphics processing units, where graphics cards are used for calculations outside the field of graphics, for example in technical, scientific or economical simulations. Even though this thesis clearly stays within a graphics context, we will quite often utilize widespread GPGPU methods. Reminisce about the two contrarious approaches to image synthesis from the beginning of the chapter: physically-based, realistic rendering versus real-time rendering. Nowadays, the historical boundaries have vanished to a certain amount and refined techniques arose. Powerful modern graphics cards allow an impressive illusion of realism to be added to real-time rendering, whereas the sheer computational power, combined with programmable shaders, cuts down the runtime of evaluating physical models, provided that extensive approximations and simplifications are given. An active research topic today is to incorporate aspects of global illumination into real-time rendering3 . Many modern computer games already feature rudimentary global lighting interactions. One can find ambitious projects like real-time ray tracing 4 on the other side of the spectrum. Today, the question “I wonder if this was possible in real-time?” crosses one’s mind quite often while looking at a stunning rendered image. Presumably, the answer isn’t as obvious as it would have been years ago.

1.2 A View on Traditional Modeling Software During the last section, we went through the history of image synthesis from a computer scientist’s perspective. Besides the technical aspect, however, a good deal of the expressiveness and magnificence of an image is accomplished through an appealing composition. This is where artists come into play. As drawing plain images or textures is usually embraced by 3

Precomputed Radiance Transfer (PRT) being a prime example. Contrary to radiosity methods that evaluate only diffuse and fixed lighting of a scene, PRT handles dynamic changes within the lighting environment. 4 See publications related to OpenRT, the real-time ray tracing project of the Saarland University Graphics Group, for a comprehensive overview.

1.2 A View on Traditional Modeling Software

(a) Approximating a curve,

(b) and a surface.

Figure 1.1: Comparison of a curve and a surface (dashed) and their polygonal representations.

the term 2D art, the key role of modeling software is to support a 3D artist in creating and sculpting models, arranging scenes, adding optional animations, and to provide an interface for rendering. Much of these tasks involves modeling the real world, and plenty of real-world objects are inherently smooth. This does not only apply for natural occurrences, but for man-made objects as well. Think of designing a curved engine hood for a car using computer-aided design (CAD). Analogies in 2D are, for instance, high-quality, scalable true type fonts or vector graphics. A mathematical description is needed in order to process geometry using computers.

1.2.1 Representing Curves and Surfaces There are common approaches to store and represent curves and smooth surfaces. We already introduced polygonal meshes—they approximate surfaces using polygonally bounded planar surfaces. See Figure 1.1 for an example. The smooth curves are now described by discrete sampling points. Apparently, the approximation can be improved by decreasing the spacing between adjacent sampling points, thus, adding more information. However, when zooming in, the misalignment of the linear approximation becomes clearly visible again. These circumstances remain when dealing with basic geometric shapes instead of arbitrary curves. See Figure 1.2, where a circle is approximated by n-sided regular polygons. Although choosing a large enough value for n may yield a sufficient approximation for some cases, the amount of information to store is notably larger than just encoding “circle at center c with radius r”. Professional modeling software allows the use of functions that are of a higher degree than first-degree, piecewise linear polygons for approximation. Still remaining nothing more than approximations, they require less storage and offer an intuitive way for interactive manipulation. The vast majority of these parametric curves are cubic; parametric surfaces being bicubic, respectively. Lower-degree polynomials are unable to provide enough flexibility regarding the control over the shape. Higher-degree polynomials are filigree, tend to wiggle, and are computationally costly. Well-known and downright ubiquitous implementations are non-uniform rational B-spline

n=3

n=4

n=5

n=6

n=7

n=8

n = 36

Figure 1.2: Approximating a circle with n-sided regular, i.e. equiangular and equilateral, polygons.

Introduction

(NURBS) curves and surfaces. They can be used for arbitrary shapes in compact form and offer good flexibility and precision. Besides being indispensable in computer-aided design, NURBS are used for modeling in computer-generated imagery5 . They are in fact a generalization of the widespread Bézier curves and patches, developed independently by Paul de Casteljau and Pierre Bézier [2], originally to design car body parts. Nevertheless, higher-order surface descriptions were unable to gain ground in real-time applications like computer games, where all-dominant polygonal meshes are used to represent surfaces.

1.2.2 Solid Modeling For rendering, it is sufficient to represent just the surface, i.e. the shell or boundary, of the model. Shaded and textured, shell models dissemble that they are hollow inside, and are thereby used in games and movies almost exclusively. However in CAD applications, it is often very important to properly deal with solid objects. Being able to distinguish between the inside and the outside of an object is crucial for engineering processes. In addition, while running physical, medical or technical simulations, for instance, a field of research outside computer graphics, significant incidents occur inside of models. An important technique is constructive solid geometry (CSG), that allows the construction of complex and sophisticated models out of basic primitives by using Boolean operations. Using boundaries representations instead of solid models requires additional topological data and frequent consistency checks after making changes. We will come back to the powerful concept of CSG and solid modeling by the time distance fields are thoroughly explained. It turns out that CSG operations can be implemented with distance fields in an impressively simple way. A very comprehensive and fundamental overview of mathematical descriptions for curves, surfaces, and solid models is given in [18], chapters 11 and 12.

1.3 Motivation to Model with Distance Fields Distance fields are common and established in computer graphics, as the next chapter will show. They are useful in many other topics of research as well. Imagine a thermal simulation for instance, that evaluates whether sensitive components have a sufficient distance to sources of heat. For modeling and rendering, distance fields are most often represented as finite discretized volumes. A decade ago, processing power of ordinary consumer hardware was not strong enough to evaluate implicitly described surfaces on the fly during rendering. In addition, complex geometry can’t be described implicitly in a straightforward way. In 2008, the graphics programmer and artist Iñigo Quilez experimented with fully procedural distance fields to model geometry, together with procedural textures and lighting. He was able to squeeze whole scenes, a ray marcher for rendering, and a routine to present the rendered image to the screen into an executable that fits into 4 kilobytes, see Section 2.4. 5

In fact, the characters in Pixar’s movie Toy Story were entirely modeled using NURBS surfaces. This cleared the way for smooth renderings independent of the zoom level. For the successor, however, Pixar already moved on to subdivision surfaces—see [11] for an introduction.

1.3 Motivation to Model with Distance Fields

Quilez was able to render stunning and impressive images, beyond doubt. However, it is certainly quite tedious and error-prone to encode procedural distance fields by hand, render them, and judge the result before making corrective actions. Possibly, some aspects can only be wangled by trial-and-error methods. Considering that Quilez is a skilled graphics programmer, it becomes clear that common 3D artists are unable to produce comparable scenes using his technique without profound knowledge in computer science and mathematics. At Stuttgart University, the idea of developing a modeling tool based upon distance fields emerged. Intentions were to gain insight about this novel and unusual approach and to fathom new ways of modeling out. In addition, a technically mature modeling tool would be able to assist 3D artists in creating objects using these innovative methods. After dealing with distance fields and the way to render scenes described by them, we will present our modeling tool and the techniques it uses. It turns out that we are able to transfer many aspects of traditional modeling to this new kind of environment.

2 Previous Work and Related Topics In 1982, Jim Blinn demonstrated the value of blending implicit surfaces for both scientific visualization and modeling in [4]. Subsequently, researchers in the United States [48], in Japan [34] and in Canada [54] developed techniques for modeling implicit surfaces. An early work discussing interactive techniques for implicit modeling is [5]. At that time, researchers were challenged in the way that implicit surfaces did not even render near realtime, not even on workstations. Early visualization methods include scanline rasterizations and point-based approximations, besides ray casting. For a more recent overview, browse the papers presented in the 2005 “Modern techniques for implicit modeling” Siggraph course session [35].

2.1 Modeling with Explicit Distance Fields Modeling environments for explicit distance fields were developed several years ago, when graphics hardware was still unable to evaluate implicit descriptions interactively. A notable work is [19]. The authors present adaptively sampled distance fields, which integrate numerous concepts in computer graphics. Apart from representation of geometry and volume data, they allow a broad range of processing operations such as rendering, sculpting, level-of-detail management, surface offsetting, collision detection, and color gamut correction. Adaptively sampled distance fields are amenable to volume ray casting and can be used to produce a variety of effects as shown in Figure 2.1(b), for instance. Of course, relying upon discretization leads to intrinsic geometric aliasing, as depicted in (c).

2.2 Sculpting Among artists, a quite popular way of designing models is sculpting. This technique is based on the metaphor of “virtual clay”, and allows to carve surfaces out of blocks by removing clay, and to extend surfaces by adding more clay. Popular professional applications for sculpting are Autodesk Mudbox and Pixologic ZBrush. Sculpting techniques have been transferred to distance fields as well, see Figure 2.1(a). In this example, an artistic carving of fine detail into a high order surface is shown.

2.3 Surface Reconstruction Apart from modeling, distance fields can be utilized for a variety of other applications. Surface reconstruction is a prime example. Intuitively, the intrinsic measure of proximity can be

Previous Work and Related Topics

(b) rendering of cocaine molecule,

(a) Carving on high order surface,

Figure 2.1: Applications of adaptively sampled distance fields. Images taken from [19].

exploited to determine small gaps and holes. See [7], where smooth shapes are reconstructed out of points via natural neighbor interpolation of distance fields. Later, we will discuss how morphological operations can be used for the same purpose.

2.4 Rendering Fully Procedural Scenes in 4 Kilobytes A great benefit of implicit surfaces is the marginal storage space required for their mathematical expression, contrary to explicit representations. In order to produce a rendering out of these very small descriptions, one could use a dedicated special-purpose programming language like HyperFun1 . Quilez went a step further and compressed the whole ray marching engine together with the scene into an executable that fits into 4 kilobytes. The technique is explained in [44] and the impressive result is depicted in Figure 2.2. Note that these images were not rendered in real-time. According to Quilez, rendering takes around half a minute on the CPU, and half a second on the GPU on recent consumer hardware with a resolution of 1280 × 1024. By looking at these images, one can guess that the manual description of the scenes took quite some time. Think about how convenient it would be to be able to arrange such scenes with the support of a modeling application.

See http://hyperfun.org.

2.4 Rendering Fully Procedural Scenes in 4 Kilobytes

(a) The “Slisesix” rendering.

(b) The “Organix” rendering.

Figure 2.2: Procedurally described scenes rendered by Iñigo Quilez using ray marching.

3 Dealing with Distance Fields Distance fields store geometric information about an object or even a whole scene. For any point in space, the distance field yields the distance of this point to the closest surface. Constructing explicit, discretized distance fields as volumes around objects is quite common in computer graphics. However, these explicit distance fields play a minor role in this thesis. This chapter gives an introduction to describing distance fields in an implicit way, where surfaces will emerge out of it. In addition, basic modeling aspects are discussed, in order to create more complex objects and arranging them in a scene. Then, simple modifications are presented, that can be exploited to create basic effects. To establish a connection to common modeling and graphics, a basic approach to integrate explicit distance fields stored in volumetric grids is addressed at the end of this chapter. First, we start out by catching a glimpse of the idea behind implicit surfaces, and continue with a basic mathematical foundation.

3.1 Implicit Surfaces at a Glance Implicit surfaces are two-dimensional, geometric shapes that exist in three-dimensional space [6]. They are defined according to a particular mathematical form, and a distance field is no more than a special case of an implicit surface. Intuitively, a surface consists of an infinite number of three-dimensional points. All these points on an implicit surface satisfy a particular requirement, represented by a function, which is quite often generically named f . This function takes a three-dimensional point P as its argument. By definition, P is on the surface if f (P ) = 0. This way, f implies a surface rather than explicitly describing it. The term that describes a surface is called an expression. Expressions of implicit surfaces differ significantly from those of parametric expressions, which are common and ubiquitous in traditional computer graphics. See Figure 3.1 for different expressions for the unit circle. Regard that parametric expressions are established for a reason. For instance, parametric surfaces can be easily sampled. In order to obtain discrete samples of the unit circle, one can traverse α or t in small step sizes. (Eventually, this leads to the problem of polygonal approximation discussed at the very beginning of the thesis, see Figure 1.2.) Naturally, it is possible to sample implicit surfaces as well, but not as straightforward. Arbitrary mathematical expressions can be used to describe an implicit surface. A special case is an algebraic surface, that is expressed by a polynomial only, i.e. it can be described by a finite number of terms. In our case, however, implicit surfaces are described by complex combinations of various distance functions—after modeling whole scenes using our modeling

Dealing with Distance Fields y

Equiangular parametric (transcendental trigonometric): P = (cos(α), sin(α)), α ∈ [0, 2π]

P α

x t

Non-equiangular parametric (rational trigonometric): P = (±(1 − t2 )/(1 + t2 ), 2t/(1 + t2 )), t ∈ [−1, 1] Implicit: Px2 + Py2 = 1

Figure 3.1: Expressions for the unit circle. This example is found in the introductory chapter of [6].

tool, for instance. For the ray marching process, it is not even necessary to know the “ingredients” of the distance function. A simple black box function would be sufficient, as we are only interested in the result for a given point in space. For continuous functions, the result for a given point P is quite often a measure of proximity [6] between P and the surface. This measure is, in general, not the Euclidean distance. For algebraic surfaces, the so-called algebraic distance is proportional to the Euclidean distance, if f is linear. Eventually, for the distance fields this thesis is dealing with, the measure corresponds to the Euclidean distance. See especially [6] for further reading. It provides a thorough overview of implicit surfaces. If interested in more mathematical application fields, consult [37].

3.2 Euclidean Structure and Notation Throughout the entire thesis, we will mostly be concerned with the three-dimensional space R3 of Euclidean geometry. For the sake of clarity, however, we will reduce many problems to two dimensions, R2 , for illustrations and examples. Points are determined by three coordinates and we usually refer to them as P , Q, or R, for instance. We will denote vectors by boldfaced letters, usually u, v or w. Elements of R3 are written as v = (v1 , v2 , v3 ). We stick to standard vector space operations in real coordinate systems, hence u + v = (u1 + v1 , u2 + v2 , u3 + v3 ) and a · v = (a · v1 , a · v2 , a · v3 ), with a ∈ R. A canonical basis for R3 consists of three unit vectors pointing in the direction of the x-axis, y-axis, and z-axis, respectively, i.e. ex = (1, 0, 0), ey = (0, 1, 0), and ez = (0, 0, 1). Every other vector in this space can be expressed as a linear combination of these unit vectors. We use a right-handed Cartesian coordinate system, see Figure 3.2(a), for modeling and ray marching, and interpret length specifications as depicted in Figure 3.2(b). Note that these coordinate system specifications do not influence the underlying mathematics in any way. Euclidean space needs more specifications than the described coordinate system provides. For modeling, it is indispensable to be able to measure distances between points and angles between vectors. In addition, length and angle information is essential for shading during rendering, as the next chapter will show. Think of rotating an object by a given angle or calculating the distance of a surface point to a light source, for instance. Again, we stick to 3 the P3 standard Euclidean structures for the inner product, i.e. the dot product on R : u · v = i=1 ui · vi . √ Using the dot product, the length of a vector can be defined asp kvk = v · v. This length function is called the Euclidean norm and is equivalent to kvk = v12 + v22 + v32 on R3 .

3.3 Fundamental Characteristics

y-axis

height

x-axis

y=0 x-z-plane

x z

z-axis (a) The right-handed Cartesian coordinate system,

depth

width

(b) interpretation of length specifications.

Figure 3.2: Right-handed Cartesian coordinate system where the view is along the negative z-axis and the y-axis points upward. Later, the x-z-plane is used as a “ground plane” for modeling—keep in mind that this is purely arbitrary. Width is interpreted as a length on the x-axis, height on the y-axis, and depth on the z-axis. Traditionally, the (x, y, z)-axes are colored (red, green, blue).

3.3 Fundamental Characteristics We describe a three-dimensional distance field using a function d : R3 → R and call it a distance function. For a given point P in the three-dimensional Euclidean space, the distance function returns the Euclidean distance from P to the surface that is described by the distance field. Consequentially, d(P ) returns 0 if P is a point on the surface. A basic example for a distance field and its corresponding distance function follows. Consider a sphere with radius r = 1.5 that is centered at the origin. Let P be a point outside the sphere at (6, 1, 0), Q a point that lies exactly on the surface of the sphere, and R a point inside the sphere at (−0.5, −0.5, 0). We can reduce this example to R2 by disregarding the depth information and only paying attention to the x-y-plane. Then, the distance field describes the boundary of a circle instead of the surface of a sphere. This example is illustrated in Figure 3.3. For P , the distance function d returns the length of the dashed line between P and the circle boundary. For Q, d returns 0. Point R lies inside the circle, hence d returns the length of the dashed line between R and the circle boundary with a negative sign. √ Using the sphere equation (3.1) fromp Section 3.4.1, d(P ) evaluates to 62 + 12 − 1.5, which is roughly 4.58, and d(R) evaluates to (−0.5)2 + (−0.5)2 − 1.5, which is roughly −0.79. y Q

1 R 0

x 1

Figure 3.3: A circle with radius r = 1.5 is centered at the origin. Point P lies outside, Q exactly on the boundary, and R inside the circle. The distances for P and R to the boundary of the circle are illustrated with dashed lines.

Dealing with Distance Fields

We exclusively use signed distance functions, that return a negative value if a given point is inside an object. Note that this differs from the common concept of distance as a length of a line segment between two points, that can’t be negative, and where “distance from P to Q” is interchangeable with “distance from Q to P ”. (There are unsigned distance fields as well, that always deliver positive distances, or possibly clamp to 0 inside objects.) Reminisce about solid modeling and constructive solid geometry from the introduction. Contrary to hollow polygonal meshes, we can easily determine whether a point is inside or outside an object, simply by evaluating the distance function and looking at the sign of the result. Later in this chapter, Section 3.6 shows how to implement CSG using distance fields. It is considerably easier than using polygonal meshes—exploiting the sign of distance fields being one of the reasons. Do not be confused about the distance function d, that usually describes the Euclidean metric, for instance d(u, v) = ku − vk. Whenever a distance between two points is expressed, we stick to the ku − vk expression.

3.4 Describing Basic Primitives Primitives are the meat and potatoes in modeling. This section shows how a sphere, a box, a cylinder, a cone, and a torus can be expressed in such a way that their surfaces emerge out of a valid distance field. These primitives are implemented in the modeling tool that accompanies the thesis, see Figure 3.4. An expression for a plane is given as well, which will be of assistance for the box, the cylinder and cone, and CSG in general.

3.4.1 Sphere Spheres are by far amongst the most prominent primitives in ray tracing applications. Traditional raster graphics struggle with them due to the need of tessellation, whereas ray tracing algorithms easily compute intersections between rays and spheres. Giving an implicit description of the distance field of a sphere is closely related and, again, very simple. (Describing cylinders of infinite length and infinite planes is equally simple, but those primitives are not as visually appealing as spheres.) We start out by making observations on the unit circle. As shown in Figure 3.1, an implicit expression is Px2 +Py2 = 1. Turning this into an implicit function yields fcircle (P ) = Px2 +Py2 −1. For arbitrary radiuses, the function extends to fcircle (P, r) = Px2 + Py2 − r.

Figure 3.4: Five basic primitives that are discussed in theory and implemented in the modeling tool, that is discussed at the end of the thesis. The image was created using the integrated ray marcher.

3.4 Describing Basic Primitives

17 y

k−r kP − 0

1 r 0

P Py x

Figure 3.5: A circle with radius r = 1.5 is centered at the origin. Point P lies outside and its distance to the circle boundary is kP − 0k − r.

Note that fcircle does not return the Euclidean distance to the circle boundary for a given point. Reconsider the recently discussed example in Figure 3.5. For P = (6, 1), we get fcircle (P, 1.5) = 62 + 12 − 1.5 = 35.5. However, the correct distance from P to the origin 0 is q Px2 + Py2 , due to the Pythagorean theorem. Taking the radius of the circle into account, the

correct distance function for a circle is dcircle (P, r) = kP − 0k − r. The approach is identical for a sphere in accordance with analogy. Let p be the vector P − 0, i.e. the vector that points from the origin to P . Hence, we can declare The Sphere Equation

dsphere (P, r) = kpk − r

(3.1)

For a sphere centered at point C, the equation extends to dsphere (P, C, r) = kP − Ck − r. However, we will use rigid body transformations—introduced in the next section—in order to translate (and rotate) objects in the future.

3.4.2 Plane In analytic geometry, the Hesse normal form describes a line in R2 and a plane in R3 . It can be written as r · n0 − s = 0, where n0 is the unit normal vector of the described plane, and s is the distance from the origin to the plane. If P is a point on the surface and r points from the origin to P , the equation is true. Conveniently, the Hesse normal form is used to calculate Euclidean distances. It maps to The Plane Equation

dplane (P, no , s) = p · n0 − s

(3.2)

that returns the distance of P to the surface of the plane described by n0 and s. The distance is signed, so points “behind” the plane have a negative distance. (Usually, but not here, the magnitude of the result is used in order to receive a non-negative, physical distance.) An infinite plane by itself is not a very appealing primitive, in terms of visual attractiveness. However, it is quite valuable when it comes to modeling, as it can be used for restrictions. Imagine an implicit expression of a hemisphere, where a plane is used to model the restriction, say, only points on one side of the plane are considered to be on the surface of the hemisphere.

Dealing with Distance Fields

height / 2

Pb1

x 0

1 z

Pb2

width / 2 depth / 2

Pb3 (a) 2D view of the x-y-plane,

(b) corresponding 3D illustration.

Figure 3.6: An axis-aligned box is centered at the origin. The part of the box that exists in the positive area of the coordinate axes is highlighted blue. Using reflections, the whole box can be reconstructed.

3.4.3 Box Interestingly, the box is a primitive that is rather laborious to describe implicitly than just providing its vertex coordinates explicitly. (In general, even a simple triangle, the basic primitive of the traditional rasterization approach, is more complicated to describe implicitly than just providing its three vertices explicitly.) The unit cube, for instance, has the vertices (±1/2, ±1/2, ±1/2). This expression is already very compact. Furthermore, it is not only an approximation, but a perfect representation, as a box has planar surfaces. If a box is axis-aligned and centered at the origin, we can reduce the problem of finding a distance to its surface by only taking points with positive coordinates into account. See Figure 3.6(a) for an example. Points Pb1 , Pb2 , and Pb3 are congruent with P if mirrored accordingly. Obviously, all four of them have the same distance to the surface of the box. Let |P | = (|Px |, |Py |, |Pz |). Then, the distance of P to the surface is identical to the distance of |P | to the surface. For the above example, |Pb1 | = |Pb2 | = |Pb3 | = P is true. From this it follows that d(Pb1 ) = d(Pb2 ) = d(Pb3 ) = d(P ). Eventually, in general for arbitrary points: d(P ) = d(|P |). Case differentiations are required in order to calculate proper distances. See Figure 3.7, y

Above Inside

Above Right

Right x

(a) Areas of interest in the 2D case,

In Front (b) basic areas of interest for the 3D box.

Figure 3.7: Classification of different areas of interest. Note that arbitrary combinations of these are possible in the 3D case. We tackle the reduced problem—regard that side lengths are twice as long.

3.4 Describing Basic Primitives

Q P

F2 E1

E3 F1

S x (a) Example points and their corresponding distance to the box surface,

(b) declaration of certain faces, edges, and the farthest vertex.

Figure 3.8: Illustration of distances from various example points to the surface of a box. The various faces, edges and the vertex are denoted on the right.

where regions are defined for both the 2D and 3D case. Let w, h, and s be the width, height, and depth of the box, respectively. For a given point P , the “right” condition holds if Px > w/2, the “above” condition holds if Py > h/2, and the “in front” condition holds if Pz > s/2. Arbitrary combinations of these conditions are possible. If neither “right”, “above”, nor “in front” holds, the point is considered to be “inside”. (Or possibly on the surface; distances evaluate to zero in this case anyway.) See Figure 3.8(a) for an example. Point P is considered to be “right” and “above” the boundary of the box. In this case, the distance from P to the surface is equivalent to the distance from P to the corner V . The box has a total of eight corners at (±w/2, ±h/2, ±s/2), but only V = (w/2, h/2, s/2) is of interest, as we reduced the problem to the area of positive coordinates. Hence, d(P ) evaluates to kP − V k. Q and R, however, are only considered as “above” and “right”, respectively. The closest surface point for Q is on the top of the box, so only the y-coordinate has to be taken into account: d(Q) = Qy − (h/2). Analogously, d(R) = Rx − (w/2). Points inside the box, like S, are treated in a different way. Compare their distances to the faces of the box, F1 , F2 , and F3 . Keep in mind that distances inside an object are supposed to be negative, so the maximum of these distances yields the correct result. In the above example, S is closest to the top. The right boundary is further away but the distance from S to the right is actually smaller due to the negative sign. Hence, d(S) = Sy − (h/2), which is negative, because Sy < (h/2). Altogether, The Box Equation is  kP − V k,     k(Px , Py ) − (w/2, h/2)k,     k(Px , Pz ) − (w/2, s/2)k,    k(Py , Pz ) − (h/2, s/2)k, dbox (Pb, w, h, s) = Px − w/2,      Py − h/2,     Pz − s/2,   max(Px − w/2, Py − h/2, Pz − s/2),

if “right”, “above” and “in front”, if just “right” and “above”, if just “right” and “in front”, if just “above” and “in front”, if only “right”, if only “above”, if only “in front”, otherwise (inside box), (3.3)

Dealing with Distance Fields

where P = |Pb|, “right” equals Px > w/2, “above” equals Py > h/2, and “in front” equals Pz > s/2. Note that we significantly reduced the amount of case differentiations by taking the magnitude P = |Pb| of the input point Pb. This way, the box equation checks whether V , F1 , F2 , F3 , E1 , E2 , or E3 contains the closest point to P . Without the reduction of the problem, one has to check for all eight vertices, six faces, and twelve edges of the box.

3.4.4 Box Approximation A way to describe a box using planes and CSG follows. The key idea is to limit the volume of a box using planes. On the x-axis, the box is limited by a “left” and a “right” plane. On the y-axis, the box is limited by a “bottom” and a “top” plane. Eventually, the box is limited by a “back” and a “front” plane on the z-axis. The idea is illustrated in Figure 3.9. Consider (a), where a volume is limited by the “left” and the “right” plane. The “left” plane has −ex = (−1, 0, 0) as its normal vector, and the “right” plane ex , respectively. They are positioned at ±w/2. Obviously, the plane equations (3.2) evaluate to dleft (P ) = −Px − w/2 and dright (P ) = Px − w/2. Hence, dleft (P ) returns a positive value if Px ≤ w/2, else a negative one. dright (P ) returns a positive value if Px ≥ w/2, else a negative one. Section 3.6.2 shows how to implement intersections of distance fields using the max function. Combining them, max(dleft (P ), dright (P )) is negative if −w/2 < Px < w/2. Reminisce about the idea of reducing the problem of distance estimation to only positive coordinates, as in Figure 3.6. Obviously, −w/2 < Px < w/2 is equivalent to |Px | < w/2. Hence, max(dleft (P ), dright (P )) = dright (|P |). Intuitively, this is a box with a width of w, an infinite height, and an infinite depth. Analogously, max(dbottom (P ), dtop (P )) = dtop (|P |) describes a box with infinite width, height h, and infinite depth. Finally, max(dback (P ), dfront (P )) = dfront (|P |) describes a box with infinite width, infinite height, and depth s. Eventually, the box can be described by a final intersection of these three slabs, max(dright (|P |), dtop (|P |), dfront (|P |)). We receive The Box Approximation debox (P, w, h, s) = max(|Px | − w/2, |Py | − h/2, |Pz | − s/2) ,

x z

(a) Limiting the x-range,

(3.4)

x z

(b) limiting the y-range,

x z

Figure 3.9: The key idea to limit the volume of a box by an intersection of six planes.

3.4 Describing Basic Primitives

21 dright (P )

P d(P

x (a) Isocontours of dbox , that correspond to the Euclidean distance,

dtop (P )

y )

x (b) isocontours of debox , the box approximation,

Figure 3.10: Isocontours of a distance field of a box. It turns out that the box approximation does not return correct Euclidean distances, like the distance from P to the corner in (c), for instance.

which is much more compact and elegant than the previous, bulky box equation. Be aware of the drawback that this approximation does not return correct Euclidean distances, but a distance that can be less. Therefore, we use a tilde to label the approximation debox . Figure 3.10 illustrates the drawback of using the box approximation. On the left, the correct isocontours of the box are shown in (a). For every point on a level curve, the distance function evaluates to the same constant. Note that we’re dealing with isosurfaces in R3 , naturally. If the box approximation is used, the isocontours look like in (b), and do not represent correct Euclidean distances. The reason for that is illustrated in (c). Obviously, the distance from P to the “top” plane is greater than the distance to the “right” plane. The box approximation evaluates the maximum of these distances and returns dtop (P ), which is less than the correct result, due√to the Pythagorean theorem. In 2D, the maximum deviation √ possible is a factor of 2, and 3 for the 3D case, naturally. Hence, debox (P, w, h, s) returns a distance a · dbox (P, w, h, s), where a ∈ ( √1 , 1). 3

3.4.5 Cylinder The cylinder features both a smooth surface that consists of the points that have a fixed distance from the cylinder axis, and two flat circular surfaces, the top and the bottom. Based upon the description of a sphere and the idea of limiting volumes by flat planes, the distance field of a cylinder can be expressed in a straightforward manner. It will be sufficient to describe a cylinder with an axis perpendicular to the x-z-plane, that is centered in the origin, like in Figure 3.11(a). Other canonical alignments can be easily deducted from this basic description. One can use rigid body transformations, for instance, for arbitrary alignments in space. It is very simple to describe the cylinder with an infinite height. Remember dcircle , that described the distance field of a circle in two dimensions. Extend this function to 3D by feeding it with the x and z-coordinates and the radius of the cylinder, but disregard the height information of the y-coordinate. This way, every arbitrary slice plane through the distance field that is perpendicular to the x-z-plane looks exactly the same.

Dealing with Distance Fields y

r height

h/2

x z

z (a) Cylinder specifications,

(b) limiting the y-range with planes.

Figure 3.11: Introduction to the cylinder primitive and the idea behind describing it.

However, a cylinder of infinite height might not be very valuable for most cases in modeling. Similar to the idea behind the box approximation, the cylinder can be cropped by CSG intersections with a “bottom” and a “top” plane, as shown in Figure 3.11(b). Obviously, dbottom (P ) = −Py − h/2 and dtop (P ) = Py − h/2. Again, due to symmetry, it is sufficient to crop with the “top” plane only, and the distance evaluates to |Py | − h/2. Note that it is possible to create semi-infinite cylinders by leaving the magnitude out. In the slisesix rendering, Figure 2.2(a), Quilez used semi-infinite cylinders for the monster’s tentacles. Symmetrically cropping a cylinder of infinite height leads us to The Cylinder Equation

decylinder (P, r, h) = max(k(Px , Pz )k − r, |Py | − h/2) .

(3.5)

Indicated by the tilde, the cylinder equation is an approximation to the Euclidean distance as well. Consider a slice through the x-y-plane. Then, the cross-section looks exactly like a box and the problem of underestimated distances is the same as illustrated in Figure 3.10.

3.4.6 Cone A cone can be described as a generalization of a cylinder, by taking an angle of divergence from its oriented axis into account. In an even more complex manner, a cone’s basis could be arbitrarily shaped, and the apex could be positioned freely rather than stuck on the inner axis. We will limit the cone to be of standard shape, as in Figure 3.12. In (a), the trigonometry behind the derivation of the distance from a point P to the cone’s surface is illustrated. This infinite cone is bipartite and symmetric to the “ground” plane, centered at the origin, and oriented along the y-axis. As traceable from the illustration, the distance evaluates to dcone (P, θ) = k(Px , Pz )k cos θ − |Py | sin θ , where θ is the angle of divergence from the y-axis. This isn’t very helpful for modeling yet, however. We want the cone to be expressed in a similar manner as the cylinder, with a given radius r and height h, and to be of finite volume. The result is shown in (b). Using r and h, we can determine θ = arctan(r/h). Then, we clip

3.4 Describing Basic Primitives

sθ

,P x k(P

−

| si |P y

nθ

co )k z

|Py| tan θ

k(Px , Pz )k − |Py| tan θ

|Py|

(a) Geometry for the distance to a cone,

(b) limiting the cone to a finite volume.

Figure 3.12: Introduction to the cone primitive. Illustration (a) was inspired by [23].

the cone with the “ground” plane and a plane parallel to that at height h. Eventually, we can express The Cone Equation decone (P, r, h) = max(k(Px , Pz )k cos(arctan(r/h)) − |Py | sin(arctan(r/h)), Py − h, −Py ) . (3.6)

3.4.7 Torus The torus is a visually appealing primitive that can be described by implicit surfaces—and distance fields—in a comparatively easy way, regard being had to its nature as a product of two circles. The first circle is indicated as the dashed circle in Figure 3.13(a) and acts as a circular inner axis of the torus. Then, the torus can be created by revolving another circle along this axis, which is indicated in (b). Figure 3.13 declares our interpretation of the two radiuses that are used to describe the torus. We refer to the distance from the center to the circular inner axis of the torus as

ut e

router x

rinner

x 0

z (a) View from above on the “ground” x-z-plane,

(b) front view of the x-y-plane.

Figure 3.13: Introduction to the torus primitive and the manner the radiuses are interpreted.

Dealing with Distance Fields

router = ro . The radius of the revolving circle is called rinner = ri and corresponds to the distance from the inner axis to the surface of the torus. (Note that other declarations exist. Sometimes, an interchanged denotation of the radiuses is used. In few cases, both radiuses start at the center and range until the torus is hit first, and the torus is left, respectively, i.e. ro ± ri in our case. The latter can be convenient for some parametric descriptions.) √ An implicit equation for a torus radially symmetric about the y-axis is ( x2 + z 2 −router )2 + 2 y 2 = rinner , which is almost an applicable equation for a distance field, but lacks an additional square root. (Compare this to the introductory difference between fcircle and dcircle .) Step by step, the distance field for a torus can be deduced as follows. For a point Pflat on the ground plane, its distance to the circular inner axis is d(Pflat ) = k(Pflat, x , Pflat, z )k − ro . For arbitrary points P in space, the height has to be respected: d(P ) = k(k(Px , Pz )k − ro , Py )k, via the Pythagorean theorem. Then, the torus can be inflated by subtracting the inner radius from this distance, which eventually results in The Torus Equation

dtorus (P, router , rinner ) = k(k(Px , Pz )k − ro , Py )k − rinner . (3.7)

Again, the torus can be freely positioned in space by applying rigid body transformations, instead of manually taking changes inside the equation. This important modeling technique is to be discussed next, as we finished the introduction of a few selected, basic primitives, by now. Later, the thesis references to further, advanced primitives like ellipsoids.

3.5 Essential Modeling Transformations In traditional 3D modeling, three essential transformations lay the groundwork for modifying and arranging objects within a scene. These are called translating, rotating, and scaling. It is inevitable to be able to express their effects for distance fields, in order to provide a serious modeling tool. However, this thesis takes a slight rearrangement to the taxonomy of modeling transformations. We will unite translations and rotations to rigid body transformations, and, in contrast, split scalings up into uniform and non-uniform scalings. Other affine transformations exist, such as a shear. The process of shearing and its distorting impact on distance fields is discussed as well, as we will encounter the same nuisance when dealing with displacement mapping later. In general, a distance field can be affected by a transformation T by applying the inverse transformation T−1 to the distance field’s domain.

3.5.1 Rigid Body Transformation After an object has been transformed by a rigid body transformation (RBT), it still features the same shape, i.e. the object is treated like a rigid body. RBTs are called Euclidean transformations as well, which already sounds very appropriate for applications on distance fields. The rigid body model requires the Euclidean distances in space between any two coordinates to remain unchanged by applying the transformation. Hence, it is obvious that a correct distance field remains correct after an arbitrary amount of RBTs have been applied.

3.5 Essential Modeling Transformations

0 x 0

(a) Translation on the x-y-plane by vector t,

(b) rotation of a box around the z-axis by α = 30◦ .

Figure 3.14: Rigid body transformations (RBT) embrace both translations and rotations.

We embrace both translations and rotations under the term RBT—intuitively, both transformations preserve Euclidean distances, which is very convenient for our purpose. In general, any reflection, translation and rotation is a global isometry on Euclidean spaces.

Translation Translating objects in order to arrange a scene is certainly the most basic task in modeling, and fortunately, the easiest. A basic translation, such as in Figure 3.14(a), moves an object in space by a translation vector t. We can emulate this for distance fields by simply evaluating the field at a different position, i.e. we shift the domain of the distance field by t: dtranslate (doriginal , P, t) = doriginal (P − t) ,

(3.8)

where doriginal is the distance function of the object prior to the translation. Rotation Rotations can be described by rotation matrices, around the coordinate axes in R3 . These are    1 0 0 cos θ Rx (θ) = 0 cos θ − sin θ , Ry (θ) =  0 0 sin θ cos θ − sin θ

and three of them represent basic rotations    0 sin θ cos θ − sin θ 0 1 0  , Rz (θ) =  sin θ cos θ 0 , 0 cos θ 0 0 1

where θ describes a counter-clockwise rotation in our right-handed oriented space, assumed that we view along the negative z-axis. In Figure 3.14(b), a box is rotated along the z-axis by α = 30◦ , hence, Rz (α) is applied. For arbitrary rotations in space, one can either use a dedicated matrix based upon an arbitrary axis, that remains fixed by the rotation, and an angle, i.e. Rodrigues’ rotation formula, or combine the three basis matrices as Rx (γ)Ry (β)Rz (α), for instance. In this case, α, β, γ

Dealing with Distance Fields

correspond to the yaw, pitch, and roll of an object. Note that the interpretation of these matrices can be subject to many ambiguities, and we stick to a consistent notation therefore. Given a valid rotation matrix R that is conform to the rigid body model of conserving Euclidean distances, we can rotate a distance field as follows: drotate (doriginal , P, ω, θ) = doriginal (Rω (−θ)P ) .

(3.9)

Rotating an object via a simple multiplication with a given matrix is a simple task, whereas obtaining or constructing this valid rotation matrix proves to be difficult. We won’t avoid dealing with so-called gimbal locks, and will introduce quaternions as an internal representation for rotations which is tackled in the implementation section of the thesis. The order of applying rotations and translations to objects matters. Adjacent translations are commutative, but rotations are not. (Matrix multiplication in general is not commutative.) For a rigid body transformation, we rotate at the center of an object first, before applying a translation. Otherwise, it would not be rotated around the object’s center. Eventually, we can combine (3.8) and (3.9) into dRBT (doriginal , P, ω, θ, t) = doriginal (Rω (−θ)P − t) .

(3.10)

Given this equation together with the equations for the basic primitives, we can already describe basic scenes such as depicted in Figure 3.4.

3.5.2 Uniform and Non-Uniform Scaling Scaling is another important modeling technique to balance various objects in terms of their size when arranging a scene. By applying a linear transformation, scaling enlarges or diminishes objects according to a scale factor. For uniform scalings, also known as isotropic scalings, this scale factor is the same for all directions, whereas non-uniform scalings, or anisotropic scalings, have varying scale factors for each axis direction. If a scene already features objects with size proportions as desired, one can zoom in and out using the camera in order to emulate scaling the whole scene. However, if individual objects don’t fit into the scene’s general proportions, one can accommodate them to the rest of the objects by scaling. This is often necessary after importing third-party models into a scene. Some of them may be modeled with kilometers, others with millimeters as base units of measurement, as an exaggerated example. Uniform Scaling Distance fields get along with uniform scalings very well. See Figure 3.15(a) for an example, where a unit circle, or unit sphere in R3 , is uniformly scaled by the scaling factor s = 2. Clearly, the point P = (2, 0) is on the boundary of the enlarged circle, i.e. the distance field must evaluate to 0 there. This can be achieved by evaluating the original distance field of the unit circle with (1/s) · P = (1, 0) as an argument. We’re done for implicit surfaces here, but distance fields require a small correction. Let Q = (3, 0) be a point with distance 1 from the enlarged boundary. Evaluating the original distance function with (1/s) · Q = (1.5, 0) returns

3.5 Essential Modeling Transformations

1 x 0

(a) Uniform scaling,

x 0

(b) non-uniform scaling.

Figure 3.15: Comparison of uniform and non-uniform scaling. The former can be expressed using distance fields in a straightforward manner, whereas the latter proves to be not as simple as that.

a distance of 0.5 to the unit circle. Therefore, we have to multiply the result with the scale factor in turn, to receive the correct Euclidean distance. By scaling the argument and correcting the results in turn, we can express a uniform scaling for distance fields as duniform scale (doriginal , P, s) = doriginal ( 1s P ) · s .

(3.11)

Uniform scaling for explicitly stored triangle meshes can be accomplished by iterating over the vertices and multiplying them with the scale factor. Contrarily, we scale a distance field by altering the domain directly by dividing by the scale factor. A special case is scaling with 0, where every object vanishes. This is the only case where an inverse transformation doesn’t exist. (It is easy to construct inverse transformations for the modeling transformations discussed before.) Note how a division by zero occurs in the above equation when scaling with 0. Consider the distance field to be infinity at every point in space then. This is a good expression for a scene that contains nothing, as any object is unreachable. Non-Uniform Scaling So far, the transformations that have been discussed left a surface unchanged, like the rigid body transformations, or mapped a surface to a similar surface, in a geometric sense. Two objects are called similar if one is congruent to the result of a uniform scaling of the other. Corresponding sides remain in proportion, and corresponding angles are preserved. This, however, is not the case for non-uniform scalings, that actually change the shape of an object. Consider different scaling factors for each coordinate axis, a scaling matrix S then:    sx 0 0 sx 0 S =  0 sy 0  . Obviously: SP =  0 sy 0 0 sz 0 0

sx , sy , and sz . We can construct     0 Px s x Px 0   Py  =  s y Py  . Pz s z Pz sz

Dealing with Distance Fields

For explicit triangle meshes, one can iterate over the vertices and apply the scaling matrix. For distance fields, however, we alter the domain again, using the inverse scaling matrix:     1/sx 0 0 (1/sx )Px S−1 =  0 1/sy 0  . As desired: S−1 P = (1/sy )Py  . 0 0 1/sz (1/sz )Pz See Figure 3.16 for an example, where a unit circle is scaled with sx = 2 and sy = 1/2. (Extend this example to R3 using sz = 1.) A reference how the resulting distance field should look like is shown in (a), where adjacent isocontours have constant distance c everywhere. However, if just evaluating dcircle (S−1 P ), the result looks like (b). (This is already sufficient for general implicit surfaces, in analogy to the case of uniform scaling.) Distances are twice as high as they should be on the y-axis, and just half of the correct distance on the x-axis. Unfortunately, there is no straightforward correction as before, where we simply multiplied the result with the scalar s. If multiplying with the maximum scaling factor, i.e. max(sx , sy , sz ), which is sx = 2 in our example, distances are adjusted correctly on the axis with the highest scaling factor. This is illustrated in (c). However, distances on other axes with lower scaling factors can be significantly higher than they should be, four times as high in our example.

c/2 c

(a) Isocontours corresponding to the correct Euclidean distance (reference),

(b) isocontours after scaling by 2 on the x-axis, and on the y-axis, leaving the result untouched,

1/2

c c/4 c

(d) isocontours after multiplying with the minimum scaling factor, which is 1/2 for the y-axis here.

Figure 3.16: Non-uniform scaling can distort the distances significantly, if applied incautiously.

3.5 Essential Modeling Transformations

On the other hand, if multiplying with the minimum scaling factor min(sx , sy , sz ), which is sy = 1/2 in our example, distances are adjusted correctly on the axis with the lowest scaling factor, see (d). Other distances can be evaluated too low, but never exceed the correct Euclidean distance—this will prove useful for ray marching later. In the example, distances on the x-axis are just one fourth of what they should be. In order to preserve a valid distance field, the result has to be multiplied with a correction factor that is not only dependent on the scaling factors, but on the position of the argument P in space as well. The result will always be within the range of the scaling factors. For now, assume that the correction function σ delivers the required factor. Then, we can express non-uniform scalings as follows: dnon-uniform scale (doriginal , P, sx , sy , sz ) = doriginal (S−1 P ) · σ(P, sx , sy , sz ) .

(3.12)

As explained, using the minimum scaling factor produces naı̈ve results that don’t exceed the correct distance: denon-uniform scale (doriginal , P, sx , sy , sz ) = doriginal (S−1 P ) · min(sx , sy , sz )

(3.13)

≤ dnon-uniform scale (doriginal , P, sx , sy , sz ) . As a general modeling advice, one should try to specify the desired proportions of an object directly, instead of applying non-uniform scalings afterward. In order to model a thin slab, define a box with one dimension minimized, for instance. Note that setting one or more scaling factors to −1, while keeping the others at 1, produces distance preserving reflections.

3.5.3 Shear Shearing is a quite popular kind of linear mapping and transformation in traditional modeling. A shear leaves one axis fixed and shifts points proportional to the distance to the axis. An example for a horizontal shear is given in Figure 3.17(a). Explicit vertices can be sheared by applying a shear matrix, which is an identity matrix with one of the zero-elements replaced by the shear factor λ. y

c c c

(a) Shear,

(b) correct Euclidean distance,

Figure 3.17: Horizontal shear, i.e. parallel to the x-axis, of a box, and selected isocontours.

Dealing with Distance Fields

A shear by λ parallel to the x-axis, that effects the y-component of points in space, as in the above example, can be expressed using the shear matrix        1 λ 0 1 λ 0 Px Px + λPy . S = 0 1 0 . Obviously: SP = 0 1 0 Py  =  Py 0 0 1 0 0 1 Pz Pz As always, we alter the domain for a distance function by using the inverse shear matrix:     1 −λ 0 Px − λPy . S−1 = 0 1 0 . As desired: S−1 P =  Py 0 0 1 Pz Note how the distance is distorted after evaluating dbox (S−1 P ), as in Figure 3.17(c). The distance between two points on the x-axis—or any line parallel to it—is preserved, as well as the distance between two lines that are parallel to the x-axis, though distances to the sheared side surface are too high. This becomes worse the higher the shearing factor λ is. Isocontours of the correct Euclidean distance, however, are shown in the reference (b). Again, we refer to a function σ to correct the distances after shearing. Let’s leave it at that, for the moment; we will encounter the same problem when dealing with displacement mapping. Eventually, for a shear by λ parallel to a particular axis, that affects another particular axis, we can state dshear (doriginal , P, λ) = doriginal (S−1 P ) · σ(P, λ) . (3.14)

3.6 Boolean Algebra Boolean algebra can be implemented with distance fields in a very simple way. Straightforward constructive solid geometry is a prime example of what distance fields are capable of. Moreover, the basic Boolean union turns out to be very essential to modeling with distance fields. Consider creating a scene by filling it with various models. In traditional modeling, this can be accomplished by adding more triangle meshes. During rendering, every distinct triangle mesh is processed. In contrast, the distance field of a scene actually changes if further objects are added to a scene. To be able to state the distance from a point to the surface of the closest object, we have to construct a distance field for the whole scene. This all-embracing distance field can be evaluated by the ray marcher, eventually. Other Boolean operations are intersection and difference. Figure 3.18 provides an overview.

A B

(a) Essential union,

(b) union A ∪ B,

A B

B (d) difference A − B.

Figure 3.18: Boolean algebra on distance fields is used for constructive solid geometry.

3.6 Boolean Algebra

3.6.1 Union Consider a scene consisting of n objects. Let d1 , ..., dn be the distance fields describing these objects. In order to evaluate the distance to the closest surface for a point P in space, the distance from P to every object has to be checked. The closest object will deliver the smallest distance. Hence, the distance field that takes every object into account can be constructed as d(P ) = min(d1 (P ), ..., dn (P )). Consider Figure 3.4, where the primitives were introduced. Actually, this scene consists of five objects that were concatenated using the min function. Then, the ray marcher can render them in a single pass, instead of rendering them in a separate pass each, and assembling the results at the end. The concept of combining every object in the scene to an all-embracing distance field is illustrated in Figure 3.18(a). Without the slightest effort, a union of overlapping objects can be expressed using the min function in exactly the same way. See Figure 3.18(b), where two objects, A and B, are unified. The Boolean union A ∪ B can be expressed using distance functions as dA∪B (P ) = min(dA (P ), dB (P )) .

(3.15)

This can be extended to a union of an arbitrary amount of objects. It should be comprehensible that constructing the union of valid distance fields, using the min function, produces a valid distance field for a scene outside of any object. The distances are neither overestimated, nor underestimated. However, this does not apply to the volume inside the unified objects. See Figure 3.19 for an example. In (a), two boxes and their intrinsic isocontours are shown. If pushed together, as in (b), a correct distance field is produced on the outside, but not inside the unified boxes. The min function returns the original negative values inside of each box, as the other one delivers positive distances there. However, the distances inside the unified boxes should fuse together as in the reference (c). Note that this doesn’t cause inconvenience for us, as we are only interested in ray marching until the surface is hit. It can, however, become a problem for advanced transformations and distortions such as blending, melting, or metamorphosis of objects. This peculiarity of unified objects will be revisited in Section 3.7.5 when discussing opening and closing of volumes. It is quite easy to correct interior distances for simple shapes as spheres and boxes, by calculating the intersection lines analytically. However, this approach becomes impossible for complex, arbitrarily shaped objects. (Consider a union of the Stanford bunny with another model, for instance.) The min and max functions are semi-analytic and result in C 1 discontinuity at any point P where dA (P ) = dB (P ). This is where numerical approximations come into play, in order to replace min and max with smoother functions trying to overcome the

→ ← (a) Boxes are pushed together,

(b) wrong distances inside,

Figure 3.19: Example of corrupted distances inside a unified object.

Dealing with Distance Fields

(a) Minimum,

(b) R-union,

Figure 3.20: Union of two ellipsoids. In (a), the min function is used for unification, like in our modeler. Discontinuities and sharp edges are clearly visible, which is no surprise due to the semianalytic nature of the min function. Next, (b) uses a so-called R-union, a case of the R-functions, that p were introduced by Rvachev in [46]. Here, the expression used is dA ∪ dB = dA + dB + d2A + d2B . One can see distances leaking near the boundary. In (c), an approach called SARDF (Signed Approximate Real Distance Function) by [17] is used, that is based on the mathematical model of [39]. The distances evaluate quite satisfying. These images were taken from [17].

discontinuity. We won’t go further into detail, but refer to Figure 3.20, where an aspect of the work of [17] is presented, on how a solution could look like.

3.6.2 Intersection We tentatively introduced Boolean intersection beforehand, as we clipped infinite volumes using planes—reminisce about the box approximation, the cylinder, and the cone. The max function can be used to express an intersection of distance fields: dA∩B (P ) = max(dA (P ), dB (P )) .

(3.16)

Analogously to the union (3.15), the intersection of more than two distance functions is the maximum of the distances each function yields. Consider the objects A and B in Figure 3.18(c). By taking the maximum, a negative result to indicate the interior is only possible if both distance functions yield a negative result. This is only true for the intersection. The union produces correct distances outside the objects, whereas irregularities occur in the inside. Note that this is the other way round for intersections. Reconsider the isocontours of the box approximation in Figure 3.10(b). It turns out that the intersection can underestimate the Euclidean distance. This can result in a performance handicap for ray marching, albeit we still benefit from correct surface evaluation. Note that there are drawbacks for more sophisticated operations. Later, the thesis brings dilation up as an example where discontinuities caused by intersections tamper with the intended result.

3.6.3 Difference A Boolean difference can be used to subtract objects from others, as in Figure 3.18(c), for instance. We will reduce the difference to the already familiar intersection. The inside and

3.7 Effects

outside of an object can be easily interchanged if described by a distance function; the only thing to do is to swap the sign. Let −B be the volume that extends over the entire space R3 , except where B has its interior. The surface of −B and B is the same, apart from opposite surface normals. Further, d−B (P ) = −dB (P ). One can consider the difference operation A − B as an intersection of A and −B. Hence, a Boolean difference can be expressed for distance fields as follows: dA−B (P ) = max(dA (P ), −dB (P )) .

(3.17)

Note that the problem of discontinuities remains, as we rely on an intersection again. Swapping the sign of a distance field can be exploited for various other things. Consider a sphere with a large radius, that embraces both the whole scene and the camera. Turning the sphere inside out yields a so-called skydome that can be optionally textured with an environment map. Analogously, an inside out box can act as a skybox.

3.7 Effects Up until now, we experienced implicit surfaces and distance fields as a compact way to express primitives, and showed how easy it is to implement CSG with them. Now, this section throws a glance on selected effects; these are quite often significantly easier to implement for implicit surfaces, rather than for polygonal meshes.

3.7.1 Morphing and Blending Morphing between two explicit meshes is easy if a bijective mapping between all vertices is given. It causes problems, if the amount of vertices differ for these objects, though. On the contrary, morphing between two distance fields, or implicit surfaces in general, can be expressed in a straightforward manner: dmorph (dA , dB , P, α) = α · dA (P ) + (1 − α) · dB (P ) ,

(3.18)

where α is the factor to morph between the distance fields. Note that this naı̈ve approach may result in parts of the objects vanishing, as illustrated in Figure 3.21. One has to consider more sophisticated methods like skeleton-based interpolation [36] in order to avoid diminishing intermediate surfaces.

(a) Simple interpolation,

(b) as a “limb” using a skeleton approach.

Figure 3.21: Sequence of morphing between two sticks centered at the bottom. Taken from [6].

Dealing with Distance Fields

Figure 3.22: Blending objects using the smoothstep function. Images from [44].

Another technique is to blend distance fields according to the distance to a particular point in space like the origin, for instance. See Figure 3.22 for an example. In order to evaluate the distance for a point P , a blend factor β is computed first. If P is at the origin, the blend factor is set to zero. If P is further away from the origin than a given threshold, β is set to 1. In between, the distance is passed to a smoothstep function that interpolates the distance using a Hermite polynomial, and returns within the range of [0, 1]. Eventually, we can blend distance fields as follows: dblend (dA , dB , P, k) = βP · dA (P ) + (1 − βP ) · dB (P ) ,

(3.19)

where the blend factor β is dependent on P and computed according to the threshold k. Note that the outcome can overestimate the distance. Consult [23] for a solution.

3.7.2 Twisting The twisting transformation rotates two axes by a linear function a of the third axis. Twisting is defined as follows:   Px cos a(Py ) − Pz sin a(Py )  , Py twist(P ) =  (3.20) Px sin a(Py ) + Pz cos a(Py ) and an example is given in Figure 3.23(a). Note that applying this transformation does not preserve distances.

3.7.3 Displacement Mapping Using distance fields, displacement mapping can be implemented in a straightforward manner. It is sufficient to simply add the amount of displacement for a particular point to its distance evaluation. See Figure 3.23(b) for an example, where Fractional Brownian motion is utilized as a noise function. Whereas this is a fully procedural approach, we show how to map standard textures as displacement maps to surfaces in Subsection 4.5.1. Note that the same distortions in distances appear as in the shear, where jagged edges arise.

3.7 Effects

(a) Twist of the y-axis,

(b) displacement mapping using noise.

Figure 3.23: Twist and noise displacement effects. Images from [44].

3.7.4 Infinite Repetitions Particularly interesting are infinite repetitions of distance fields. This can be achieved by a simple modulo operation on the domain. See Figure 3.24 for examples. In (a), a Menger sponge is expressed by subtracting an infinite grid of sticks from a single cube. In (b), an infinite grid of spheres is shown. Both scenes render in real-time with ease on today’s consumer graphics hardware.

3.7.5 Morphological Operations Distance fields have been found useful for morphological operations of erosion and dilation in [49]. It turns out that we can easily perform erosions and dilations on objects by spheres by simply adding or subtracting the sphere’s radius to the distance. In Figure 3.26, a dilation of a unit cube by a sphere with radius 1/4 is depicted. Note how this fails if using the box approximation; the dilated surface corresponds to an isosurface of the distance field, at the level of the sphere’s radius.

(a) Menger sponge at iteration 5,

(b) infinite grid of spheres.

Figure 3.24: Examples of infinite repetitions, rendered using our ray marcher.

Dealing with Distance Fields

(a) Unit cube,

(b) dilation (1/4),

(d) concept.

Figure 3.25: Dilation on a unit cube by a sphere.

Closing is a dilation followed by an erosion, and opening is an erosion followed by a dilation. Unfortunately, this is not as straightforward to implement as it seems at first sight. Adding a value to a distance, then subtracting it again—or vice versa—is a meaningless identity transformation. This is an example of the problem illustrated in Figure 3.19(b). The distances don’t fuse together as they should in order to provide correct distances in the inside. A correct implementation of closing as proposed in [27], however, gives us a practical algorithm for closing volumetric objects, as depicted in Figure 3.26.

3.8 References to Advanced Primitives This section refers to solutions of implementing advanced primitives like ellipsoids, hypertextures, and blended objects.

3.8.1 Ellipsoid The basic idea of computing the distance of a point to an ellipsoid can be found in [24]. Note that this is no longer straightforward and requires finding the roots of a sixth-degree polynomial. One can tackle this problem numerically by Newton’s method.

3.8.2 Hypertexture Applying noise functions to surfaces can greatly enhance their appearance in terms of realism. Especially organic and natural shapes benefit from sophisticated noise techniques.

Figure 3.26: True 3D distance closure. Images taken from [28].

3.9 Assimilating Explicit Distance Fields

In [23], a proposal is given on how to compute a lower bound on band-limited solid noise functions, which were presented in the famous “Hypertexture” paper [40] by Ken Perlin.

3.8.3 Blend of Objects Unfortunately, the blend equation 3.19 can overestimate distances. Refer to [23] on how to estimate lower bounds for distances to blended objects, including soft metablobbies and superelliptic blends.

3.9 Assimilating Explicit Distance Fields Lastly, there are models left that can’t be described implicitly in a straightforward fashion. However, one can still use explicit distance fields to store the distance and surface information. Consider a grid with explicit stored distances, delimited by a bounding box. If inside the volume, perform tri-linear interpolation to compute the distance for a point in question. If outside the volume, evaluate the distance from the point to the bounding box using the box equation 3.3 and add the explicit distance at the surface point hit on the box. If some of the voxels don’t contain a valid distance, return the distance to the next grid cell. This way, it is guaranteed that the surface is not going to get penetrated.

4 Ray Marching Distance Fields In this chapter, we will act on the prospects that our investigation of distance fields gave us, and present a ray marching technique which exploits a distance field’s characteristics. The ray marcher enables us to actually render a scene; eventually, it will be tightly integrated into our modeling environment in the subsequent chapter. Using techniques out of the family of ray tracing algorithms has quickly become one essential and popular way of rendering realistic images in early computer graphics. Comprehensive literature is found in [20], focusing entirely on ray tracing, and [21], dealing with image synthesis more generally. Throughout the next sections, we will show how to set up a ray marching environment for distance fields and distinguish it from the standard, widespread ray casting and tracing approaches. Next, it is explained how to implement the rendering process on the GPU within a fragment shader. The remaining sections are devoted to shading fragments, with respect to lighting, shadowing, and texturing.

4.1 The Concept of Ray Marching First of all, a camera—the “eye” through which we perceive an image—has to be set up, with its corresponding image plane. This process is illustrated in Figure 4.1 and is to all intents and purposes identical for any algorithm among the ray tracing family. Located at C, the camera views into the direction of v. In our case, v is always normalized. The center of the image plane is located at C + v, and the plane is perpendicular to v. Some applications specify the dimensions of the image plane explicitly in pixels, say 640 × 480, for instance. However, we describe the image plane in terms of a field of view, also known as field of vision. It turns out that this is a great benefit for us when it comes to GPU implementation and hybrid rendering later. Note that the term “field of view” is subject to many ambiguities. In general, it describes the extent of the scene that can be scanned at any given moment. We specify an angular dimension of the image plane, the vertical field of view, and refer to it as fovy, as in Figure 4.1(a). Due to the normalized viewing vector, we can compute the length of the up vector u as tan(fovy/2), see Figure 4.1(b). Given an aspect ratio ar, for instance 4/3, we can compute the length of the right vector r as ar · tan(fovy/2). Obviously, the whole arrangement can be rotated arbitrarily around the axis containing v, so it has to be fixed by specifying a point where u is supposed to point to. The concept of backward light transport is shared by the family of basic ray tracing techniques; start at the camera, and trace the paths which direct light toward it. As the lion’s share of light rays from light sources never find their way directly into the camera, a forward simulation computes a significant amount of dispensable rays that have to be discarded.

Ray Marching Distance Fields

u v

Image plane

fovy/2

C (a) Spanning the image plane,

Im ag ep lan e

(b) corresponding 3D view.

Figure 4.1: Assembling the basic camera setup for ray tracing, casting, and marching techniques.

Contrary to other applications, we didn’t explicitly set the image plane’s dimensions in terms of pixels—the rasterizer will take this over for us. But for the moment, let’s assume we already have a discretized pixel grid on our image plane. To start the actual image synthesis, iterate over all pixels and shoot (at least) one primary ray from the camera through the particular pixel. The term “ray casting” was mentioned first in [45], 1982, and the principle reaches back to [1], 1968. In its most basic form, every object in the scene is tested against all primary rays. Intersections are found analytically, and the foremost object is retrieved to shade the pixel in the image plane the ray was shot through. If no intersections are found, this pixel is shaded using a background color. In 1980, Turner Whitted extended the algorithm to ray tracing in his frequently cited paper [53], a milestone in computer graphics. Secondary rays handle reflections, refractions, and shadows, and are cast after an object has been hit—where ray casting would already have stopped. This allows to take various global illumination effects into account and adds significantly more realism to the outcome. Ray marching is strongly related, but different. Instead of calculating intersections with objects analytically, one marches along the ray in (very) small step sizes. Note that this is obviously not appropriate for rendering basic scenes containing standard objects. It would be utterly meaningless to analytically check for surface intersection at every sampling point along the ray, requiring the effort of a single ray casting check every time. However, there are particular applications appropriate for ray marching. Prime examples are direct volume ray casting and rendering participating media, like smoke, fog, or dust. Assume these volumes are bounded by a bounding box. The process of ray marching can be integrated in regular ray casting then; first, calculate the intersections of a ray with the bounding box, where the ray enters and leaves the volume. Then, switch to ray

Figure 4.2: Ray marching with constant step size through a participating volume.

4.2 Adaptive Marching Step Size Dependent on Distance

marching inside the volume and start marching at the entry point. Sample along the ray and add contributions of the volume for shading, until the bounding box is left. See Figure 4.2 for an illustration. Next, we will extend the ray marching concept with effective surface intersection tests.

4.2 Adaptive Marching Step Size Dependent on Distance The fundamental concept presented in [23] is sphere tracing, which is the ray marching technique that we’re going to utilize. Instead of intersecting rays with implicit surfaces, we evaluate their distance fields at the current position on the ray and move along the distance the distance field returned. See Figure 4.3 for an example. The technique guarantees that the implicit surface is not penetrated. In addition, it is sufficient to have a valid bound on the distance, hence we can use the approximation equations for primitives and transformations. A “hit” occurs if the returned distance is less than a given threshold ε. Still, this appears as a rather tedious approach for determining intersections with basic objects. However, sphere tracing taps its full potential when dealing with deformed and pathological surfaces. Moreover, it avoids problems where the derivative jumps and vanishes. A drawback of sphere tracing is that closely marching along boundaries increases the amount of required steps significantly. One can already spot this in the example figure; simple analytical tests just return a hit or a miss. The worst case occurs, if a ray is marched parallel to an infinite planar surface, with a distance ε—but not less—so that a hit never occurs. We discard such a ray after exceeding a maximum amount of steps or particular ray length. Note that the distinction between “tracing” and “casting” is not necessarily applied to ray marching and sphere tracing. One can decide to stop after a hit, or can cast secondary rays. Our ray marcher is restricted to shadow rays for the time being. In general, the variety of ray tracing algorithms discussed is very easy to understand and implement. Initially, one can already produce appealing renderings without putting too much effort into it. However, naı̈ve and non-optimized implementations are slow. Next, we discuss some acceleration techniques, deal with distorted distances, and eventually explain how to port the ray marcher to the graphics card.

Figure 4.3: Sphere tracing, a hit and a miss. Image taken from [23].

Ray Marching Distance Fields

4.2.1 Enhancements We catch a glimpse of some enhancements that can help to increase efficiency and reduce unnecessary distance evaluations. We won’t go much into detail, though, as some of them are difficult to port to the GPU. Several enhancements exist; refer to [23] for further reading.

Exploiting Convexity and the Triangle Inequality The marching step size can be increased if it is known that an object in question is convex. Let P be a point in space outside, v a unit vector, and d the distance function of a convex object. Then, the line segment d(P ) P, ·v (4.1) −v · ∇d(P ) does not intersect the object, but can possibly touch the surface at its second endpoint. This can be exploited for rays anchored at P viewing into the direction of v. How we compute the gradient is explained shortly. Using this enhancement, the algorithm likely converges quadratically due to its similarity to Newton’s method, which also converges quadratically. A proof of the above statements is found in [23]. Note the behavior of sphere tracing after the miss in Figure 4.3, where distance actually increases, after it diminished more and more. If the object is known to be convex, one can take a possible hit out of the question. This is a direct application of the triangle inequality.

Bounding Volumes Testing with an object’s bounding volume, for example a bounding box, instead of the complicated object itself, is an essential technique in computer graphics. We utilize this when integrating volumes containing explicit distance fields, and compute the distance to its bounding box, as long as the point in question is not inside. It turns out that this is a valuable performance gain during rendering. As bounding volumes are usually convex, one can combine them with the above technique of approaching convex objects.

Space Partitioning In its basic form, space partitioning divides the Euclidean space into several disjoint subsets; it can be exactly determined in which region a point in space lies. Space partitioning is frequently used and a fundamental technique in computer graphics, where it organizes the objects within the scene. Space-partitioning data structures like octrees provide convenient geometric queries and allow to skip empty cells, which can be used to improve rendering time during ray marching.

4.3 Ray Marching in a Shader

Figure 4.4: Gaussian function turned upside down.

4.2.2 Strategies to Tackle Deviations from Euclidean Distance We already stated that a valid bound on distance is sufficient to guarantee that sphere tracing does not penetrate surfaces; now, we give an example how deviations can be handled. These occur after applying displacement maps on object, for instance. A rather inefficient, but effective method is to step just a fraction of the evaluated distance along the ray. Considering that one has to be cautious near the surface rather than far away, one could insert the evaluated distance into a function similar to the one illustrated in Figure 4.4. This function is of the form: 2

1 − e−tx

and ensures that ray marching decelerates drastically when converging to surfaces. Note that one has to find a decent trade-off between rendering time and decelerating ray marching. As a prospective work, the ray marcher should estimate an appropriate factor automatically, that ensures surface integrity.

4.3 Ray Marching in a Shader Inherently, computer graphics is a prime application for parallelization. Especially ray marching—and ray casting in general—can be ported to parallel architectures easily due to the independent nature of each ray. Due to the lack of a stack on the GPU, ray tracing can’t be ported without further efforts for the time being. Historically, graphics hardware has always been sort of parallel. Today, programmable shader allow the programmer to execute custom algorithms on the GPU. In this section, we show how to port the ray marcher to the graphics card. The shader is executed by rendering a full-screen quad, defined by the vertices (±1, ±1, 0), using the texture coordinates (±1, ±1). These are processed by the vertex shader first, which is the most basic one in our case. Using the GLSL function ftransform(), the model-view and projection matrices are applied to the incoming vector, just like in the fixed-function pipeline. The vertices’ texture coordinates are passed on to the fragment shader, and interpolated by the rasterizer. Some implementations construct their camera settings, as illustrated in Figure 4.1, in the vertex shader, however, we compute the view, up, and right vector on the CPU and pass them as uniforms to the fragment shader. Note that the up and right vectors, unlike the view vector, are not normalized. Hence, we can compute the vector that points from the camera

Ray Marching Distance Fields

to the point where the ray intersects the image plane as follows: d = v + s · r + t · u, where s and t are the texture coordinates interpolated by the rasterizer for the current fragment. This way, the whole image plane maps to the rendered quad, which is helpful for rendering arbitrarily sized viewports later. Normalizing this vector d, one can march along the ray by specifying a progress: C + t · d. Starting at t = 0, evaluated distances are simply added. Using dedicated shader programming languages, one can benefit from convenient and frequently used functions like length or distance that operate on vectors, and may be efficiently processed, depending on the hardware environment. There are minor drawbacks when computing on the GPU, though. Performance is best for algorithms featuring a high arithmetic intensity. The ray marching shader performs well under this aspect, with no dependency between single rays as a bonus. However, branching within a shader can significantly slow down compile time and ray marching. This is why we introduced and utilized the box approximation, for instance, that doesn’t require if-then-else constructs. Note that it is generally not straightforward to implement sophisticated data structures on the GPU. This is why we refrain from exploiting the triangle inequality, for instance, as this would require us to implement some sort of stack, where objects can be removed from.

4.4 Illumination and Shading Up until now, our basic ray marcher is able to compute hits with surfaces, and can render the silhouette of the scene. In this section, we show how to shade the scene using a basic Phong illumination model, adding more realism.

4.4.1 Gradient Estimation Shading a fragment according to a surface point’s angle to and its distance from light sources requires the computation of the surface gradient. Whereas this can be easily described by a function for basic implicit objects, it has to be approximated for arbitrary shapes. We use the common and established central differences method to compute the difference quotient for a distance field d:   d(Px + 21 h, Py , Pz ) − d(Px − 21 h, Py , Pz )   1 1  ∇d(P ) ≈ δh [d](P ) =  d(Px , Py + 2 h, Pz ) − d(Px , Py − 2 h, Pz ) d(Px , Py , Pz + 12 h) − d(Px , Py , Pz − 12 h)

(4.2)

and normalize it afterward. Normalization is important for correct shading, as the gradient is used in dot products. Observe that choosing a value for h has significant impact on shading. See Figure 4.5 for an example. Very low spacings produce numerical artifacts, as seen on the leftmost image; they vanish by increasing the spacing. The voxel grid structure can be perceived on the next image, but gradient estimation seems correct. The high frequencies in shading vanish if the spacings are increased further. Note that the geometry doesn’t change—the model still features an

4.4 Illumination and Shading

Figure 4.5: Stanford Bunny imported into a grid with a bounding box of 4×4×4. Gradient estimated using central differences with spacings h = 0.00005, h = 0.001, h = 0.05, and h = 0.2, respectively.

inherent polygonal structure, it is just not visible anymore. Exaggerated spacings result in a blurry appearance as in the rightmost image. One can obtain finite difference approximations for higher order derivatives in an analogous way. In addition, one can consider using a more sophisticated method, like using the Sobel operator, known from image analysis. Keep in mind that computational time increases, obviously. Furthermore, note that already 6 distance field evaluations are necessary to approximate the gradient using basic central differences as in equation 4.2. Compare this to the enhancement 4.1 proposed for convex objects; if a simple and quick gradient evaluation is not given, the intended benefit is actually turned into a drawback.

4.4.2 Phong Illumination Model The Phong illumination model [41] consists of three parts, the ambient, diffuse, and specular components. The ambient component is a very naı̈ve compensation for indirect lighting, and brightens up an object by a constant value. The diffuse part accounts for Lambertian reflection on the object, see Figure 4.6(a). If the light source, where L points toward, is to have any direct effect on shading the point with surface normal N , θ must be between 0◦ and 90◦ . Geometrically, the light’s diffuse contribution is cos θ. Specular reflection can be observed on shiny surfaces, see (b). Phong’s model assumes that maximum specular reflectance occurs if α, the angle between the reflection vector R and the vector V that points toward the viewer, is zero. Furthermore, specular reflectance falls off sharply as α increases, approximated by cosn α, where n is called the specular-reflection exponent, typically varying from 1, a broad and gentle falloff, to several hundred, simulating a sharp, focused highlight.

N L

L θ

(a) Diffuse reflection,

R α

(b) specular reflection using halfway vector.

Figure 4.6: Phong illumination model. Inspired by [18].

Ray Marching Distance Fields

Figure 4.7: Sphere rendered with our ray marcher using the Blinn-Phong illumination model, with the specular-reflection exponent n = 4, n = 16, n = 64, n = 256, and n = 2048.

N and L are normalized, hence we can use the dot product to rewrite cos θ as N · L for the diffuse term. For the specular term, we use the Blinn-Phong formulation using the halfway vector H. Assuming that the angle between N and H is similar to α, we can express the specular-reflection term as (N · H)n , where H = (L + V ) / |L + V |. After a ray hit a surface, we can shade the point in question. Clearly, L can be easily computed by normalizing a vector from the surface point to the light source. We can estimate N using central differences. The view vector V is identical to −d, the inverse direction of the ray. Determining these vectors is sufficient for our shading. See Figure 4.7 for a sphere with several specular-reflection exponents rendered using our ray marcher. Still, rendering spheres using a constant diffuse color produces monotonous images. Next, we show how texturing can be implemented into the ray marcher, yielding more interesting and diversified results.

4.5 Texturing As soon as one immerses into computer graphics, textures will expeditiously attract a great deal of attention. They are an inherent and integral part of computer graphics, and literally omnipresent. Speed and simplicity are the crucial factors for using texture mapping for applying color and detail to 3D models, instead of manually encoding this information into the model geometry itself. Color textures, the most well-known type of textures, can be acquired in various ways. See Figure 4.8 for an overview, using a marble structure as an example. A talented artist can draw them by hand, a time-consuming task not accomplishable by just anybody. The outcome may be appealing and artistic, whereas this method is inappropriate for creating very realistic looking surfaces. Photographs, on the opposite side, are most suitable for capturing themes to be found in nature. They have their drawbacks, though, as they are fixed in many respects. Consider the fixed angle of the motive, fixed spatial resolution, fixed lighting and many other surrounding influences the photographer has no direct control over. Another orthogonal approach is to generate a realistic looking image of natural elements using various mathematical functions, combined in an algorithm and evaluated by the computer. The result is a so-called procedural texture, having its own characteristics. It takes some effort to not make them look too mathematical and sterile. Typically, gradient noise such as Perlin noise and turbulence functions are used to meet the requirements of randomization found in the outside world. Procedural textures have some great benefits nonetheless—the procedural description

4.5 Texturing

(a) Artistic painting1 ,

(b) professional photograph2 ,

Figure 4.8: Classification of three different texture types.

is very small compared to the image result after evaluation, so storage demands are being reduced to an almost unimportant factor. Adjusting variables and ranges inside the algorithm provides great flexibility to alter the outcome, consider lossless zooming by way of example. In addition, the algorithm can be easily extended to 3D in most cases. In this section, we describe the ray marcher’s parameterizations for mapping 2D textures to surfaces, and show how to implement basic procedural textures.

4.5.1 2D Texture Mapping When shading a point on a textured surface, the ray marcher has to sample the texture at a particular position; underlying processes like interpolation of values are taken over by the graphics hardware itself. The only question we have to deal with is actually how to map a point in three-dimensional space, that can be described using two coordinates if it lies on a surface, into the two-dimensional texture space. Next, we introduce common parameterizations for our primitives, and a method to apply them to arbitrary surfaces.

Sphere We utilize an “earth metaphor” of longitude λ and latitude ϕ and transfer it to the sphere. The longitude is an east-west measurement; lines of constant longitude range from the north to the south pole, and λ can be interpreted as an angle around the equator, ranging from 0 to 2π. The latitude measures the location north or south of the equator, with ϕ ranging from −π/2 on the south pole to π/2 on the north pole. Both λ and ϕ are mapped to the texture coordinate range [0, 1]. Textures having twice the width than height are suited best, as λ ranges twice as long as ϕ. Eventually, we map a point 1

Detail of the drawing marble on marble by the artist Jeff Hayes from 2007. Serpentine marble with a high carbonate content, photographed by Andrew Alden in 2008. 3 Procedural marble texture called African jade, generated by a small C++ script using the libnoise library. 2

Ray Marching Distance Fields

λ Figure 4.9: Sphere textured with an earth texture.

P in space, independent of its distance to the origin, to the texture coordinates   Py arccos kP −0k atan2(Pz , Px ) λ ϕ . = , , 2π π 2π π See Figure 4.9 for an example, where an earth texture is applied to a sphere. Obviously, the texture appears distorted, as a whole pixel row is mapped to the singularity of a pole. Note that other, sophisticated parameterizations exist, that avoid this drawback. Box In order to texture a box, we use a simple planar projection and linearly map two selected coordinates in space, depending on the surface normal, to texture coordinates. If the surface normal is most similar to the z-axis, for instance, we project the texture to the x-y-plane. An example is given in Figure 4.10. Obviously, this naı̈ve technique maps the center of a face to (0, 0), as the primitive itself is centered at the origin. One can arbitrarily shift and scale the texture coordinates later on, though. Of course, this is applicable to every parameterization. Cylinder We use a mix between the sphere and the box parameterizations on a cylinder. A longitude λ is computed around the cylinder’s side. The height of a point in space, however, is linearly

Figure 4.10: Box textured with an image; shifted and scaled.

4.5 Texturing

mapped to the second texture coordinate directly. We utilize the same parameterization for the cone as well. Note that in this case, a whole texture row is mapped to the singularity on top of the cone again, like for a sphere’s pole.

Torus For a torus, we map λ to its inner axis. Then, ϕ is wrapped around the circle with radius router revolving around the inner axis, ranging from 0 to 2π this time. As a torus is described by two radiuses independently from each other, we have to take its outer radius into account, to be able to determine which side of the torus a point faces. Consider a torus with an inner radius of 0.1, and a point P at (1, 0, 0), for instance. Then, P faces the exact opposite sides of the revolving circle for the cases router = 0.5 and router = 2. Keeping this in mind, we compute the texture coordinates for a torus as follows:

λ ϕ , 2π 2π

atan2(Pz , Px ) atan2(kP − 0k − router , Py ) , 2π 2π

Arbitrary Surface Based on the idea of two-part texture mapping [3], we can exploit the previous parameterizations for arbitrary surfaces that lack a canonical surface mapping. In order to accomplish this, one has to place a textured bounding object like a sphere or a box around an arbitrary object, so that the objects share the same center. Then, both objects are combined using CSG intersection. The arbitrarily shaped object is now textured using the bounding object’s parameterization. Basically, a surface point receives the texture coordinate from the position where the line from the center through this point intersects the bounding object’s surface. See Figure 4.11 for an example. In (a), one can perceive strong distortions in the region of the head, ears, and feet. In (b), the areas of different planar projections are clearly distinguishable.

(a) With bounding sphere,

(b) with bounding box (detail).

Figure 4.11: Two-part texture mapping on an object that lacks a canonical “rabbit parameterization”.

Ray Marching Distance Fields

(a) Wooden elephant,

(b) 3D checkerboard pattern.

Figure 4.12: Three-dimensional procedural textures applied to solid-looking objects.

4.5.2 Procedural and Solid Textures Of course, one can apply two-dimensional procedural textures to surfaces using the parameterization techniques mentioned before. A great benefit, however, is that procedural textures can easily be extended to 3D and are therefore capable of describing solid textures. Imagine the time and effort required to create a 3D texture of wood by taking photographs of a wooden block and cutting off very thin slabs little by little. How would one accomplish this anyway for liquids, for instance? Solid textures release us from the hassle of dealing with surface parameterization, as a three-dimensional point in space can be mapped to a three-dimensional texture coordinate in a self-evident manner. In addition, objects actually look like they were carved out of a block of solid matter, as in Figure 4.12(a), inspired by early RenderMan works [52]. For starters, a simple checkerboard pattern function is integrated in our ray marcher, see (b). It’s certainly not that impressive, but already clear of distortions. In recent years, a lot of research focused on the novel idea of utilizing texture synthesis techniques in order to compute solid textures out of given input images. This is far beyond the scope of this thesis, but mentioned later in the “Future Work” Section 6.2.

4.6 Shadowing Still, our ray marcher is lacking one important element: shadows. They add a significant amount of realism to scenes that should not be underestimated. Moreover, they support the viewer in perceiving a strong sense of depth and figuring out relationships between objects. Without shadows, scenes look too artificial and sterile.

4.6.1 Casting Shadow Rays The ray marcher can be easily extended by casting shadow rays after a surface has been hit. For every light source, a secondary ray is created at the surface hit point, that is directed toward the light. If the ray hits a surface along the way, the point in question is occluded and therefore not directly illuminated by the light source. Using sphere tracing again, the

4.6 Shadowing

Point light

Extended occluder Penumbra Umbra

(a) Computing the penumbra as in [9],

(b) soft shadows using our ray marcher.

Figure 4.13: Casting soft shadows, concept and implementation.

ray marching process has to be started at a little offset away from the surface; otherwise, the distance evaluates to zero and every point would be considered as occluded. For starters, it is sufficient to reduce the intensity of occluded fragments a bit during shading. In addition, we discard the computation of specular reflection completely for shadowed areas; a dimmed specular reflection for dark areas does not occur in the real world and looks very unnatural and unrealistic. Note that this is actually the first global illumination effect implemented, as mutual interdependencies between objects are taken into account.

4.6.2 Soft Shadows We are able to produce physically plausible shadows for renderings. However, these hard shadows feature high frequencies at their edges and lead to the impression of stiffness. Soft shadows, on the contrary, feature a smooth transition from illuminated into occluded areas, and look much more realistic. Historically, soft shadows were tedious and time-consuming to compute. Early ray tracing algorithms shot several shadow rays in order to sample smooth shadow edges. Triangle rasterization algorithms used a variety of “hacks”, like shaking the light source a bit for a few renderings and blending the results afterward. A simple extension to ray tracing is given in [38]. For a shadow ray, its distance to a spherical or triangular occluder is computed. See Figure 4.13(a) for an illustration of extending an occluder and determining the penumbra. If the shadow ray intersects the extended occluder, the shadow factor is determined according to its distance to the occluder. Our ray marcher is able to determine the distance to occluders literally for free, as the distance field is evaluated during ray marching anyway. We keep track of the minimum distance for a shadow ray while marching along; in the end, if the distance is smaller than a particular penumbra threshold, we shade the pixel according to a shadow factor dependent on the distance. Our novel method has several benefits. First, no further (analytical) computations are

Ray Marching Distance Fields

needed. Second, all possible occluders are taken into account automatically, where [38] has to check for every object that comes into question as an occluder. Finally, we’re not restricted to particular shapes of occluders; arbitrary surfaces are inherently supported. In order to reduce visually perceived overshootings at the penumbra borders, the shadow factor is put into the smoothstep function, the interpolation technique based on Hermite interpolation and frequently used in graphics applications.

4.6.3 Ambient Occlusion Eventually, we finish this chapter by presenting a fast technique for ambient occlusion that is implemented in our ray marcher. Quite often, one is restricted to estimate ambient occlusion in screen space if real-time rendering is required. Our technique, however, uses a three-dimensional measure of proximity for occlusion. The concept behind the technique invented in [16] is illustrated in Figure 4.15. For a surface point P , the distance field is sampled at n points along the surface normal; few points are sufficient, like n = 5 as in the illustration. An ambient occlusion factor ao can be interpreted by comparing the distance of the sampling points to P to the evaluations of the distance field d as follows: n X 1 ao(P ) = 1 − k1 · (i · δ − d(P + k2 · i · δ)) , (4.3) 2i i=1

where k1 and k2 can be used to influence proximity interpretations. The exponential decay ensures that surfaces further away account less than nearby ones. Moreover, this plausible estimation takes self-occlusion and occlusions by any other object of the scene into account, and is thus a global illumination component. Still, the technique

Figure 4.14: Work image of a test scene. The amount of steps N required during ray marching are mapped to the colors ranging from blue (N = 100) over green (N = 200) to red (N = 300). It is clearly visible that sphere tracing slows down for rays that narrowly hit or just missed a surface.

4.6 Shadowing

(a) Open surface,

(b) occlusions near winding.

Figure 4.15: Concept behind ambient occlusion estimation. The dotted lines indicate the distance from the surface to the sampling points. These distances are identical to the distance field evaluations at the sampling points for an open surface, where no ambient occlusion occurs.

is very efficient, as it requires just a few more evaluations of the distance field. A small test scene is rendered in Figure 4.16, where the impact of the introduced shadowing methods is presented. See Figure 4.14 for a work image. One can see that marching near object boundaries decelerates sphere tracing much more than the applied displacement mapping, soft shadows, and ambient occlusion. See how the sphere’s shadow cast on the box is colored red at its boundaries; the shadow ray just misses the sphere on its way to the light source. We are now able to render images of basic scenes with texturing, lighting, and shadowing. In the next chapter, we introduce the modeling environment we built on top of our ray marcher.

(a) With displacement mapping,

(b) with ambient occlusion,

Figure 4.16: Test scene visualizing the shadows produced by the techniques in our ray marcher.

5 Interactive Modeling with Distance Fields — Implementation — The author developed a small modeling tool, which is an integral part of this diploma thesis. The tool is called dfm, the distance field modeler, and is presented in this chapter. Fundamental and known modeling techniques have to be transferred from dealing with polygonal meshes to working directly on distance fields. In addition, we will focus on interactivity. This is not restricted to real-time rendering, providing immediate visual feedback; we take care in providing a user interface that supports the modeling process as good as possible. After all, the key idea of starting this diploma thesis was to release an artist from the perils of programming distance fields by hand. We will introduce our modeling environment by explaining its structure, the development tools used for implementation, fundamental concepts, and the way our ray marcher is integrated. Then, we will depict how we realized standard modeling actions by giving insight into the application flow. Furthermore, we show how to visualize the underlying distance field. Eventually, we close the gap to traditional modeling by providing possibilities to import and export standard triangle meshes.

5.1 Development Tools and Libraries The modeling tool was written “from scratch”, however, it would be downright impossible to implement such a project without the excessive use of major developing libraries. This section marks out the constitutive libraries. The distance field modeler was developed and tested on recent Linux systems, and only requires development packages that are most likely included in recent distributions. In theory, the modeling tool builds on other major operating systems as well, as it is entirely based upon cross-platform and open source software. A modern graphics card supporting the Shader Model 4.0 is required to run the program; a medium-class consumer card like the Nvidia GTX 260 is perfectly fine, and cards that got a bit long in the tooth, like the 8800 GT, are suitable as well.

5.1.1 Qt 4.5 Basically, the distance field modeler is written in C++ using Qt 4.5, a cross-platform application development framework1 . It is widely used for the development of GUI programs and possesses a comprehensive widget set. In addition, Qt provides an excellent interface to 1

See http://www.qt.nokia.com and http://doc.qt.nokia.com/4.5/index.html for the documentation.

Implementation: Interactive Modeling with Distance Fields

OpenGL, the graphics library used. Due to that, the modeler features a dovetail connection to the integrated ray marcher, which is written in GLSL. Qt’s framework is particularly helpful to implement interactive systems, as it provides integrated key routines for handling user input, for example mouse gestures. We utilize dragand-drop, as another example, which is utterly complex and time-consuming to implement on different operating systems manually. Using the tool qmake, which is part of Qt, a makefile can be created, and hopefully, the distance field modeler compiles nicely with gcc, the GNU Compiler Collection.

5.1.2 Computational Geometry Algorithms Library We make use of CGAL, the Computational Geometry Algorithms Library, a C++ library2 providing access to efficient and sophisticated geometric algorithms. CGAL compiles on various platforms and can easily be linked into applications. For the time being, 3D surface mesh generation is the only part of the CGAL library that is utilized.

5.2 Main Window Framework Designing a graphical user interface is rather an art than a precise science. Lengthy and tedious user studies and evaluations are required in order to determine whether a particular GUI is “good”—in terms of attractiveness, usability, efficiency, and many more aspects. We stick to a common and established user interface design, giving us two benefits. First, we keep the user from the hassle of learning to deal with a new, unusual interface. Second, we can compare our modeling tool to traditional modeling applications more easily. (In analogy to the standard user interface, we implemented three-dimensional manipulators the way an experienced artist would expect them.) This way, it is easier to rate whether peculiarities

Menu Bar

Menu Bar Tool Bars

Tool Bar

Dock Widgets

Central Widget

Central Modeling Widget (derived from QGLWidget)

Shader Panel Status Bar (a) The QMainWindow framework,

Settings Panel

Materials Panel

Status Bar (b) default framework of our modeling tool.

Figure 5.1: Default Qt framework and our derivation. Inspired by the Qt reference documentation.

5.2 Main Window Framework

Figure 5.2: Main application window.

during modeling happen because of the characteristics of implicit surfaces, or simply due to the interface. Qt’s QMainWindow class provides a main application window. Figure 5.1(a) illustrates the layout. A QMenuBar and a QStatusBar are placed on the top and the bottom, respectively. Tool bars and dock widgets are implemented in the QToolBar and the QDockWidget class, and can be docked to the left, right, top, and bottom. It is not possible to create a main window without a central widget. (In a pinch, this can be just a placeholder.) The default framework of the distance field modeler is illustrated in Figure 5.1(b). This is just to make a proposal after the application has been started. The tool bar and panels can be freely arranged within the work area. This section will explain all GUI parts in detail. An actual screenshot of the modeling tool with the default arrangement is shown in Figure 5.2. The central modeling widget occupies the lion’s share of the work area and displays a small scene that has been modeled. A sphere in the front is selected, indicated by the yellow boundary and noted in the status bar. The modeler is in its translation mode, and the sphere is ready to get translated. Next to the object’s name in the status bar, the corresponding rigid body transformation is listed. The materials panel is filled with textures used in the scene, and the shader panel displays the actual fragment shader generated for ray marching the scene. 2

See http://www.cgal.org and http://www.cgal.org/Manual/last/doc_html/cgal_manual/contents.html.

Implementation: Interactive Modeling with Distance Fields

5.2.1 Central Modeling Widget The central modeling widget implemented in the distance field modeler is actually a derived QGLWidget class, which provides functionality for displaying OpenGL graphics integrated into a Qt application. But its purpose is not only to display graphics; it’s as important to have access to the functions it provides as a basic interface widget. This way, keyboard and mouse events can be captured and processed, which is essential to interact using manipulators, for instance. In general, being able to utilize Qt’s basic QWidget functions laid the groundwork for the modeling tool as an interactive system.

The Rendering Task Three virtual functions of QGLWidget are reimplemented to perform typical OpenGL tasks; these are initializeGL(), resizeGL(), and paintGL(). First, the rendering context is set up, framebuffer objects (FBO) and multiple render targets (MRT) are defined, as well as textures initialized. Camera and viewport initialization follows. Furthermore, the initial internal state of the modeler is set, i.e. “use the camera mode”, “no object is currently selected”, and so on. Usually, resizeGL() is responsible for setting up the OpenGL viewport and projection after the widget has been resized or initialized. However, we use it to correct the dimensions of the textures for our multiple render targets. There must be a 1:1 mapping between the widget size and the texture size, so that every texel of an MRT texture maps to a pixel of the central modeling widget. The viewport will be set up later anyway, as it depends on whether a single, dual, or quad view is requested by the user. It is useless to set up projection matrices here as well, as they change often during a single rendering pass, between orthographic and perspective projections. (See Section 5.5, “Hybrid Rendering”, on page 67 for details. Basically, the ray marcher is activated by rendering a full screen quad, using an orthographic projection. Eventually, additional elements like manipulators and the grid are rendered using standard OpenGL and a perspective projection.) The latter, paintGL(), is executed for every frame, i.e. by far the most frequent. It iterates over the viewports and sets up the ray marcher by uploading the corresponding uniforms to the graphics card. Then, the scene is rendered into MRTs in the FBO. If not disabled by the user, a grid is drawn over the scene. If an object is selected and the modeler is in one of its transformation states, manipulators are drawn. (Including the picking buffer this time, so that they can be selected.) A tiny orientation cube is rendered to the top right of the active viewport, indicating the camera’s viewing direction and giving visual feedback about the current orientation. Small orientation axes in the bottom left corner serve the same purpose. Eventually, the off-screen buffer is rendered to the screen using a selection shader. The currently selected object is committed to this shader, and the boundary of this object is highlighted yellow by applying a very simple edge detection algorithm on the color picking buffer. (An object is represented in the picking buffer by a constant, distinct value. Hence, it is obvious that edge detection can be applied to the visible part of the object.) As a last drawing step, viewport outlines are indicated, so that the user can keep the viewports apart. The paintGL() function ends by increasing a frame counter that is reset every second. This way, the current frame rate can be displayed at the very left in the status bar.

5.2 Main Window Framework

Interaction Tasks In addition to rendering, the central modeling widget contributes to the interactivity of the modeler to a large extend. Keyboard and mouse events are processed; if the camera mode is active, for instance, the user can traverse the scene using the keyboard or the mouse. By holding down the left mouse button and moving the mouse, the camera rotates around its center of interest. (If the camera is fixed, like the “front”, “left”, and “top” cameras, rotations are disabled.) Using the mouse wheel, the camera zooms in and out. The camera can be translated using the right mouse button, moving the camera’s center of interest. Other modes can be selected by clicking the corresponding icon in the tool bar. The selection mode is used for selecting objects by clicking on them, and deselecting all objects by clicking into an empty area of the scene. The translate, rotate, and scale modes are used for object transformation. In this case, the corresponding manipulators are drawn for the selected object. Using the control key, the mouse maps to the selection mode while in camera mode, and to the camera mode while in all other modes. This allows for a more continuous workflow, as the user can rotate the camera while translating an object, for example, without having to switch frequently between the different modeling states. As already mentioned, Qt made it convenient to implement a strong interconnectivity between rendering and interaction. This is very important for our interactive system. Consider a very basic example, like adding a sphere to the scene, selecting it, and translating it a little. After each step, the user receives direct visual feedback. After adding the sphere, it is placed at the origin and can instantly be seen, as long as the camera is not placed too far away. By clicking on it, the sphere gets highlighted, so the user can verify that it is actually selected. During translation, the user directly perceives the effect on the sphere and can position it precisely. An interactive frame rate is required, otherwise, the user’s experience will be disillusioning. Our modeling tool is able to provide a satisfying frame rate for basic scenes, thanks to the efficient ray marching algorithm and today’s graphics cards. A small introductory scene like in Figure 5.2, with a little displacement mapping, can be rendered with over 100 fps on a Nvidia GTX 260 without further ado.

5.2.2 Settings Panel In the course of time, the modeler’s capabilities expanded. The role of the settings panel is to keep track of various settings and to offer the user a convenient way to interact with them. This dock widget consists of a QToolBox. It displays a column of tabbed widget items, one so-called page above the other, with the current page displayed below the current tab. A screenshot of the various pages of the settings panel is shown in Figure 5.3. A short overview of the different pages follows. The Viewport & Camera page displays settings of the four viewports of the modeler. These are the camera associated to the selected viewport, with the camera’s field of view and its center of interest. There is the possibility of adding and removing additional cameras. Furthermore, the viewport can visualize a work image of the scene, and can hide the grid, if the corresponding “Thermographic” and “Hide Grid” check boxes are checked. See Section 5.3 for details.

Implementation: Interactive Modeling with Distance Fields

Figure 5.3: Settings panel of the distance field modeler offering various possibilities for control.

Next is the Scene page, which is a very important one. It provides a view of the scene graph of the current scene with the possibility of interactively modifying the graph by adding, removing, and repositioning items. Equally important, properties of the selected scene graph item are displayed and can be edited interactively. See Section 5.4 for details. Using the Distance Analysis page, the user can specify a slice plane through the scene, which delivers insight into the underlying distance field. A visualization of the distance is mapped to the slice plane using a customizable transfer function. An image of the slice plane, or its raw data, can be saved and processed subsequently. See Section 5.10 for details. In order to generate a triangle mesh out of the implicitly described scene, or just to export a raw discretized distance field, the user can specify a bounding box and bounding sphere of the export volume using the Mesh Generation page. There are various settings for the mesh generation algorithm—provided by CGAL—on this page as well, that have a severe impact on the mesh generation process. See Subsection 5.11.2 for details. If desired, the distance field modeler provides an interface to import discretized distance fields. Using the Import Slots page, the user can import and manage explicitly stored models. After importing, these volumes can be treated like any other primitive or object in the scene. See Subsection 5.11.1 for details. Dealing with deviations from Euclidean Distance, as discussed back in Subsection 4.2.2, is controlled via the Ray Marching page. Decelerating ray marching is important when applying displacement maps on objects, for instance. Otherwise, surface peaks can be missed out, or a ray ends up inside an object, penetrating the surface. Settings are passed as uniforms to the shader to avoid frequent recompilation.

5.2 Main Window Framework

5.2.3 Materials Panel A scene containing objects that are all shaded in the same, constant color, would appear very dull and monotonous. The materials panel keeps track of all colors, textures, and displacement maps specified and loaded into the modeling environment. Using drag-and-drop, the user can interactively assign materials, by dragging them out of the panel and dropping them on objects in the scene. See Section 5.9 for details.

5.2.4 Shader Panel Underneath the modeling interface operates the integrated ray marcher; basically, this is a fragment shader written in GLSL. It renders the scene according to uniform variables that describe the view and camera settings and direct to the textures. The source code of the compiled fragment shader that is executed every frame is displayed using a QTextEdit that occupies the shader panel. A specialty of the distance field modeler is that it allows direct modification of the ray marching shader, simply by editing and recompiling the source. See Figure 5.4 for an example, where a twist is added to the cylinder evaluation function. Arbitrary modifications are possible, thanks to the flexibility of modern shader programming on graphics cards. One can edit the distance field, the ray marching algorithm, the lighting and shading, and even the whole outline of the approach. Keep in mind that the main purpose of shader editing is to try things out and to quickly check whether an idea is realizable. A user should be aware of automatic shader regeneration after making changes to the scene graph—adding a new primitive, for instance, is already sufficient to overwrite custom shader modifications. If compilation of the modified shader fails, the user is informed about this via a message box that itemizes the lines containing errors including the reasons why the compilation failed at this position. In this case, the modeling tool reverts to a fail safe mode. The user can click the “Regenerate Shader” button to send a request to the scene graph to generate a valid shader again.

−→ manually extending the shader: float cyl(vec3 p, vec2 r) { . p.x += 0.4*sin(2.0*p.y); . p.z += 0.4*cos(2.0*p.y); . return max(sqrt(p.x*p.x + p.z*p.z) - r.x, . abs(p.y) - r.y); }

. . . and recompiling the shader −→

Figure 5.4: Example of direct shader manipulation. Navigate to the distance evaluation function for the cylinder and rotate the domain of the ground plane against the y-axis. The twist will be invariant to rigid body transformations applied to the cylinders. After the compile button has been clicked, the modified shader will be compiled and—if successful—uploaded to the graphics card.

Implementation: Interactive Modeling with Distance Fields

Note that the whole process of twiddling with the shader is interesting rather for the computer scientist than for the modeling artist. A mature, complete, and professional modeling application should provide a broad and convenient interface to make steady modifications to the scene. Nonetheless, the author implemented the shader panel at a very early development stage, and this proved to be very valuable for debugging and programming in general.

5.3 The Model-View-Controller Concept Known to work efficiently, the distance field modeler implements the Model-View-Controller (MVC) architectural pattern. Basically, this concept structures the development process into the three units “Model”, which is the scene graph in our case, “View”, which presents the scene to the user, and “Controller”, which receives input and initiates a response by instructing to modify parts in the scene graph. Actually, one implements an MVC concept quite often if it is intended to be able to add different views or controllers to an existing model at a later date. Take an existing application for example, that is later ported to a different operating system or even to the web. In our case, however, it is a good idea not to mix application logic, the scene graph, input, and presentation into one particular location, for example the central modeling widget. In fact, we interpret the MVC concept as depicted in Figure 5.5. The whole scene, represented by a scene graph, is encapsulated in the model. There are different kinds of views, though. Viewports play the most prominent role of all views; they render the scene using the ray marcher and present it in the central modeling widget, where the user can interact using manipulators. This way, viewports give very important visual feedback. Another kind of view is featured in the Scene page on the settings panel, where the scene graph is visualized directly using a QTreeWidget. Using this view, the user perceives connectivity between the scene graph items, that is usually hard or impossible to interpret or guess out of the final rendering. (Furthermore, the final rendering is not supposed to give revealing information about the internal structure.) A third kind of view, entirely different again, is the direct listing of the fragment shader source generated out of the scene graph, as featured in the shader panel. No underlying information about the scene can be hidden there. User gestures, like selecting or transforming objects in the central modeling widget, or rearranging items in the scene graph tree widget, are transmitted to their corresponding controllers. They interpret the user’s gesture and instruct the model to update its state correspondingly. Note that this is not possible for manual changes inside the fragment shader, as in Figure 5.4. The modeler is not in a position to interpret arbitrary code fragments and abstract them into scene graph items of unpredictable complexity. Using an object-oriented programming language like C++ helped to implement this structured approach, where individual components use method invocations on others. Communication between these objects are implemented using Qt’s signals and slots. The signals and slots mechanism is a central feature of Qt, which the modeler uses to coordinate everything. Basically, Qt utilizes the preprocessor and converts signals and slots to callback functions. Difficulties arise when using a fixed architectural pattern, squeezing everything into a single widget, and trying to extend the modeler to use different cameras and viewports at the same time. Thanks to the MVC concept, the distance field modeler extended quite easily in this

5.3 The Model-View-Controller Concept Request shader

Model Change notification

View • Renders the model, • requests scene and shader, • sends user gestures to controller, • allows controller to select view.

• • • •

Encapsulates scene graph, responds to state queries, generates fragment shader, notifies views of changes.

View selection User gestures

State change

Controller • Defines application behavior, • maps user actions to model updates, • selects view for response, • one for each functionality.

Figure 5.5: Model-View-Controller pattern applied to the distance field modeler. Method invocations are denoted with solid lines, and events, processed with Qt’s signals and slots, with dashed lines. The above figure was inspired by the book [50], chapter 11, “Architecture”.

way. The rest of this section deals with this topic, that is practically implemented in every mature modeling application.

5.3.1 Additional Cameras Four general cameras are already specified after the initial startup. First, the Main Perspective camera is assigned to the Main Viewport. This camera offers a great degree of freedom and can be arbitrarily positioned by the user. Three cameras, Front, Left, and Top, have their center of interest fixed to a plane, a viewing direction always perpendicular to this plane’s normal, and a fixed up vector. The latter cameras are usually orthographic rather than perspective in most modeling applications. All cameras are held in a QList of pointers. New cameras can be created by the user, as many as desired, in order to allow the user to save and revert to a particular camera setting and to increase the flexibility.

5.3.2 Multiple Viewports There is the option to select a single, dual, or quad view of the scene within the central modeling widget. Selectable viewports are Main Viewport, Top-Right, Bottom-Left, and BottomRight. Every viewport has a single camera attached, but not exclusively; due to the pointer list of cameras, multiple viewports can share the same camera. By default, the Main Perspective, Front, Left, and Top cameras are connected to the Main Viewport, Top-Right, Bottom-Left, and Bottom-Right viewports, respectively, after application startup. Viewport-dependent settings are to visualize a work image, and to hide the grid. This way, it is possible to have a normal rendering on the left, and a work image on the right, for instance, using the same camera for both views. Arbitrary combinations are possible, and Figure 5.6 shows some of them. Standard compositions are shown in (a, b, c). In (d), a work image of the scene is visualized. In analogy to a thermographic view, the amount of steps required to eventually shade a pixel

Implementation: Interactive Modeling with Distance Fields

(a) Single view,

(b) dual view,

(d) thermographic view,

(e) comparison with shadows,

(f) custom arrangement.

Figure 5.6: Several arrangements of viewports.

is mapped to a color ranging from black over blue, cyan, green, yellow, to red. A dual view sharing the same camera is shown in (e), with a reduction to just the shadow and ambient occlusion part to the right. Arbitrary combinations are possible, as in (f), and the size of the viewports is alterable.

5.4 The Scene Graph In general, a scene graph is a data structure that represents and arranges a scene. In our case, the scene graph is a tree structure. The root node of the scene graph has exactly two children, “Distance Field” and “Lighting”. The first node contains all primitives of the scene, and all transformations applied to them. The latter lists all light sources and offers general lighting settings. Their individual items are not interchangeable. Primitives are stored as leafs in the tree, i.e. they are not allowed to have any child nodes. Transformations, though, can be arbitrarily positioned, but they are meaningless if they don’t hold primitive child nodes at the end, where the transformation is applied to. A transformation node may have many children, but is restricted to a single parent. Usually, a transformation node applies its operation to all of its child nodes. However, the modeler provides several operations, which are based upon a sequence of arguments. Take the “difference” operation, for instance; it matters whether object A is subtracted from B or vice versa. This operation takes its first child and subtracts the remaining children. Furthermore, the “morph” operation, for instance, doesn’t apply a constant transformation to all of its child notes; quite the contrary, individual weights are assigned to the children, which are blended together, eventually. Associating the effect on a point in space with a geometrical transformation, and concatenating these effects during graph traversal is an efficient and evident way to process such operations. A common feature, for instance, seen in many graphical applications and

5.4 The Scene Graph

supported by our modeling tool as well, is the ability to group related primitives into a compound object. Then, this embracing object can be selected and transformed as easily as an individual, single object. Later, the thesis presents an example of constructive solid geometry by modeling a cup of coffee. A glimpse of what a scene graph actually looks like can be caught in Figure 15(f). (Note that translations, rotations, and scalings are omitted for clarity.) As mentioned just before, the entire cup can be treated as a single object; one can select it and place it on an additional desk, for instance. Other implementations of scene graphs, like in Pixar’s RenderMan, allow nodes to have a relation to any other node including itself, or provide extensions that refer to another node. In theory, our modeling tool could be extended with this functionality quite easily using nodes consisting of pointers to other nodes; however, these relations can’t be visualized in a straightforward manner using the QTreeWidget item and its basic tree model. Without cycle detection, difficulties could arise during shader generation as well, as cycles would result in an infinite shader source. It should already be clear that we simply took the key principle of a scene graph and adopted it to suit our particular needs for the distance field modeler—keep in mind that there is no consensus at all as to what a scene graph should look like.

5.4.1 Adding New Primitives to the Scene An integral part of designing a scene is to add new primitives to it. The user can select a particular primitive out of a QMenu. Every list item has an assigned QAction, that is connected to the scene graph via signals and slots. The scene graph—the “model” in our MVC concept— takes care of the rest, then. A dedicated counter for every kind of primitive exists, that is incremented accordingly. Each primitive is pickable, and receives a unique picking ID. A new primitive node is created and attached to the “Distance Field” node. A new, updated shader is generated, compiled, and uploaded to the graphics card; eventually, all views are informed about the model’s state change and update themselves by rendering the new model. In order to delete a node, it has to be ensured that the particular node doesn’t have any child nodes. Then, the node can be removed from the scene graph and its memory can be freed. Eventually, the shader is updated again and views render the new scene without the deleted node.

5.4.2 Adding Transformations Transformations are added to the scene graph in a similar manner, however, they are not restricted to being leaf nodes. If a transformation results in a new primitive, like CSG or morphing operations, they receive a unique picking ID as well. Some of the transformations have arguments that can be interactively altered using the modeler. For instance, a rigid body transformation is defined by three floats that specify the translation along the axes, and a 3 × 3 rotation matrix; all values change during interactive manipulation. It is not a feasible solution to regenerate and compile a new shader for every frame, if the user alters a transformation by dragging a primitive’s manipulator—the frame rate would drop

Implementation: Interactive Modeling with Distance Fields

considerably below any interactive threshold. Hence, the modeler uses uniform variables in the shader that are defined before rendering a frame. The rigid body transformation, for instance, uses a uniform vec3 and a uniform mat3 to inform the shader about the translation and rotation parts, that are internally stored in the corresponding scene graph node. In order to render a frame, the uniform variables have to be uploaded to the graphics card first. For this reason, the scene graph is asked about all uniforms that have to be taken into account. A uniform registry has been implemented, that keeps track of every uniform variable of the scene. If a transformation is added to the scene graph, it registers its uniforms to this registry. If this transformation is deleted later on, it ensures that its uniforms are removed from the registry.

5.4.3 Rearranging Items On the Scene page, the scene graph is visualized directly as a tree using a QTreeWidget, including its topology, contrary to the views in the central modeling widget. There, it is possible to make direct topological changes inside the scene graph. The QTreeWidget already offers a convenient drag-and-drop solution for internal movement of nodes. Such internal movements can be useful in many ways. Consider grouping a few primitives into a union; these primitives can be dragged into the particular union node then. For a difference between primitives, the order of the sequence of children matters. In order to change “subtract B from A” into “subtract A from B”, drag-and-drop can be used to rearrange “B” and put it before “A”. We already addressed a few restrictions: Transformations and primitives stick to the “Distance Field” node, whereas light sources can only be children of the “Lighting” node. Furthermore, primitives have to be leaf nodes. In order to ensure this, Qt’s default drag-and-drop routine for internal movement within a QTreeWidget was overloaded and extended. The modeler rejects drag-and-drop operations that violate these requirements.

5.4.4 Shader Generation A crucial moment for modeling with the distance field modeler is the process of shader generation for the scene. In a nutshell, the scene is directly transformed into GLSL source code to describe its distance field, then, the resulting shader is compiled and uploaded to the graphics card, and eventually, the views are updated by rendering the scene with the new shader. Basically, the shader is generated by breadth-first traversal of the tree structure of the scene graph. This was implemented by using a queue; while processing a node, put all of its children to the end of the queue and move on. Every primitive and transformation has a particular piece of fragment source, which are concatenated according to the scene topology. The distance field of the scene is produced by concatenating fragment source pieces into a sequence of min operations. (Compare this to equation 3.15 for Boolean union.) During scene graph traversal, these code fragments are appended to the shader source. In order to affect nodes, transformations have to be declared beforehand. Hence, they have to be prepended to the shader source; otherwise, they are being referenced by a parent node

5.5 Hybrid Rendering

without being declared, resulting in a compilation error. Every scene graph node has an assigned piece of source that is appended to the shader, one piece that is prepended to the shader, and optionally, uniform variables that are registered in the uniform registry. Note that it is tricky to generate and concatenate fragment source pieces—they have to be aligned in a strictly defined sequence to receive a valid shader. One has to pay attention to correctly open and close brackets, define functions before they are referenced, and provide the necessary amount of arguments to GLSL functions. The shader is generated quite frequently; after adding, deleting, and rearranging scene graph items, after editing their properties in the property editor on the Scene page, after assigning materials to them, after loading new textures, just to name a few occasions. Eventually, the shader is used by the integrated ray marcher to render the scene.

5.5 Hybrid Rendering We combine image order ray marching of the scene with object order rasterization of modeling additions like the grid and manipulators. In principle, a grid and manipulators could be added as distance fields to the scene as well and ray marched in one go. However, this proves to be not very reasonable. First, these additions don’t really need shading, and shouldn’t cast shadows on other objects. Contrary to the grid, manipulators are supposed to be rendered on top, but if they are included in the distance field, objects in front would occlude them. Next, a grid consists of lines that are rendered with a thickness of exactly one pixel, and the manipulators should always be drawn in the same height, pixel-wise, regardless of zooming and the distance to the camera. This is very hard to achieve using ray marching; especially tracing the grid. In addition, the sphere tracing algorithm and the purpose of drawing the grid are a very bad match—consider the many small marching steps on a ray parallel to a grid line. This is where hybrid rendering comes into play. In fact, each frame is composed using three rendering passes. Figure 5.7 gives an overview. First of all, the ray marching shader is executed off-screen into an FBO to render an image of the scene. This is done in a typical GPGPU fashion by rendering a full-screen quad with this particular fragment shader applied. There are three rendering targets; in addition to the regular image of the scenery, the ID of the surface hit is encoded in the color picking buffer, and, very important for hybrid rendering, the depth component of the fragment is set to the depth of the surface hit. How the surface ID encoding and picking works is explained in detail in the next section. Filling the depth component is not as straightforward as using the standard OpenGL fixed function pipeline. There is the built-in variable fragcoord.z for GLSL programs, however, this corresponds to the constant depth value of the full-screen quad rendered to execute the shader, and not to the desired depth of the surface point hit. In order to solve the visibility problem of merging the scene with modeling elements, we stick to standard z-buffering, invented by Wolfgang Straßer [51] and independently, eight months later, by Edwin Catmull [10] during their PhD theses. Given a near plane and a far plane at depth n and f , we can compute the depth value z 0 after the perspective transformation as follows: f + n 1 −2 · f · n 0 z = + , (5.1) f −n z f −n

Implementation: Interactive Modeling with Distance Fields

Ray Marching Shader

Draw manipulators

Scenery Buffer

Draw grid

Depth Component

Color Picking Buffer

Selection Shader

— Multiple Render Targets —

Frame Buffer Object

OpenGL

Mask grid against depth map

Figure 5.7: Hybrid rendering, sequence of operation diagram. An additional render target is used to implement color picking. A third component, the depth buffer, plays an important role when merging ray marching with standard rasterization techniques.

where z is the depth of the surface hit in camera space and z 0 ∈ (−1, 1), provided that n < z < f . During ray marching, a ray is traced from the camera until a surface is hit. One cannot use the length of this ray as the depth value z—the problem is illustrated in Figure 5.8, where the ray is indicated by a dashed line. The perspective transformation, where the canonical viewing frustum is transformed into the unit cube, has to be taken into account. Hence, one has to compute the distance of a surface hit to the image plane. In the illustration, an example is given for a surface hit at point P . Note that the dashed line is longer than the correct depth z. They are only identical for a point directly in the center of the image plane, and drift apart when moving to the border of the image plane. Reminisce about equation 3.2, which yields the distance to a plane, p · n0 − s. One can construct a plane around the camera, located at C, with the view vector v as the plane normal: p · v − s. Now, s can be found by taking the distance from the camera to this plane, by simply substituting p for c, which results in s = c · v. Eventually, putting it all together: z = p · v − c · v. In OpenGL, the near plane maps to depth 0; the range (−1, 1) can easily be buckled to (0, 1) by multiplying with 1/2 and adding 1/2. Note that the depth map visualized in Figure 5.7 is strongly exaggerated. Due to the extensive non-linearity of the z-values after the perspective transformation, the depth map would usually be perceived as plain white. The non-linearity is useful and reasonable, though; areas near the camera gain considerably

Image plane

5.5 Hybrid Rendering

z = p · v − c · v, where p = P − 0 and c = C − 0.

Figure 5.8: Computing the depth z of P , which corresponds to the distance from P to the image plane. The camera is located at C and views into the direction of v.

more computational precision than areas far away. As the modeling environment is closely coupled with the integrated ray marcher, it is obvious that both have to interpret the perspective transformation in exactly the same way. Therefore, the camera’s position, field of view, view and up vectors, and the near and far planes have to be identical for both ray marching and setting up projection matrices for OpenGL rendering. For ray marching, the corresponding variables are passed via uniforms to the shader, and for OpenGL rendering, a very convenient function exists, that is gluPerspective(fovy, aspect, zNear, zFar ), which sets up the desired perspective projection matrix. OpenGL does not implement the idea of a “camera”; this is emulated by translating and scaling the scene (the “world”) according to the modelview matrix, also called the modeling and viewing transformation. Our modeler uses the convenient gluLookAt(eyex, eyey, eyez, centerx, centery, centerz, upx, upy, upz) function to specify the modelview matrix, passing the same arguments to this function as uploaded as uniforms to the ray marching shader. “Eye” is our camera here, and “center” corresponds to “camera + view”. As expected, our modeler isn’t safe from z-fighting artifacts as in Figure 5.9(b). At least, this is a proof that the ray marcher’s interpretation of depth is correct and identical to the OpenGL standard, as the box in the example is specified to lie exactly within grid lines. There are solutions for z-fighting, but non-sophisticated ones are basically hacks. One could add a little offset to the fragment’s depth, but—how much? Then, z-fighting occurs at shifted positions again, anyway. Another solution is to label the grid as “priority”, however, this fails again due to numerical errors if z-buffer precision is too low. Having dealt with the tricky process of filling the depth buffer, GL_DEPTH_TEST is enabled for the second rendering pass. Then, the grid is drawn and automatically masked against

(a) Sphere merged with the grid,

(b) z-fighting occurs at the edges of a box.

Figure 5.9: Hybrid rendering of a sphere and a box, using image order ray marching, and the grid, using object order line rasterization. Z-fighting occurs in (b), at faces alongside grid lines. Inverted and exaggerated colors are used to emphasize the artifacts.

Implementation: Interactive Modeling with Distance Fields

the depth map. If the Front, Left, or Top camera is used, the grid rotates to match a corresponding ground plane that is parallel to the image plane and perpendicular to the view vector. Note that the grid is not drawn on the color picking buffer. Optionally, manipulators are drawn, including the picking buffer this time, but bypassing the depth test. The size of the manipulators are amplified in the picking buffer, using cylinders instead of lines. This way, picking becomes more convenient for the user, and strictly pixel-perfect clicking is not a requirement anymore. The third and last rendering pass uses a selection shader to render the off-screen scenery into a viewport in the central modeling widget. Given the ID of the selected object as a uniform, the selection shader performs a simple edge detection algorithm on the corresponding shape in the color picking buffer. Eventually, the color of the fragments at the edges are set to yellow. During the whole rendering stage, one could already implement a naı̈ve anti-aliasing technique. The width and the height of the textures attached to the FBO could be doubled. (All of them due to MRT restrictions; they must have the same dimensions and number of bits.) Using bi-linear interpolation for mapping the outcome to the viewport, one can perceive smoothed edges on the rendering. By all means, the picking and depth buffer have to be interpolated using nearest neighbor. A blend between object IDs results in corrupted IDs, possibly mapping to an entirely different object. Geometric interpolation of depth, on the other hand, between two objects that are far away from each other, is pure nonsense. Keep in mind that rendering time increases by factor 4, and that an adaptive anti-aliasing technique is probably a better candidate. Similarly, dividing the texture’s dimensions by 2 results in a rendering speedup of factor 4. Obviously, stretching the scenery to the viewport results in a blurry image, but maybe a complex scene can be rendered in real-time again this way. (Expectedly, the image order ray marching complexity scales with the resolution.)

5.6 Picking Historically, the problem of picking arose in the early seventies when the developing inexpensive raster graphics superseded the involved architecture of vector displays. This development was one of the most significant contributions to computer graphics, without doubt, however, major disadvantages of raster systems arise from the discrete nature of the pixel representation [18]. Apart from the need of scan-conversion and approximation of smooth curves and surfaces to pixels on the raster grid, it is no longer straightforward to determine which primitive actually was responsible for shading a particular pixel. Our modeling tool utilizes picking for two typical applications. First, the user can click on objects or manipulators to select them. Second, materials can be dragged out of the materials panel and dropped onto the desired object in the central modeling widget. There are several possibilities to implement picking, which differ from each other considerably. Traditionally, the OpenGL API provides an integrated mechanism for picking objects. One has to name all relevant objects first and pass this information to OpenGL during rendering. After clicking, the so-called selection mode has to be entered. One has to get the mouse coordinates of the click within the viewport and unproject them to 3D eye coordinates. A pick matrix is created to define a viewport that renders only a small window around the mouse cursor. During the following rendering pass, nothing is actually drawn, but whenever

5.6 Picking

a primitive is hit, its name and its depth are stored in the selection buffer. After rendering, one can exit the selection mode and pop the stack to identify all relevant objects on the small part of the screen around the mouse cursor. (With the object with the lowest depth being the most prominent.) We won’t utilize traditional OpenGL picking due to two severe drawbacks. First, this mechanism got more and more neglected in the course of time. According to programming guidelines on www.opengl.org, a modern OpenGL program should not use the selection mode (or the closely related feedback mode, which is deprecated) at all. It is implemented in software on the CPU side, and on some OpenGL implementations, performance is reported to be poor. However, the crucial factor is that we won’t be able to feed the selection buffer in our GLSL ray marcher. There is no way to set a primitive’s name there, and the integrated gl_PrimitiveID returns a meaningless pick ID of the full-screen quad rendered to execute the fragment shader. Another picking technique frequently used is ray casting. A ray perpendicular to the image plane is shot through the viewport coordinates in question. This has to be implemented separately for traditional rasterization approaches; in our case, on the contrary, a ray marcher already exists. On request, a single ray could be cast in order to compute the pick ID of the surface hit. Using an additional render target, this can be achieved in one go during ordinary ray marching. The modeling tool connects the unique pick ID of primitives with their corresponding piece of shader source; during evaluation of the distance function in the shader, the pick ID is taken along using piggybacking. Eventually, the pick ID of the surface hit is encoded in the picking buffer as follows: gl_FragData[1] = vec4(floor(id / 256.0) / 255.0, mod(id, 256.0) / 255.0, 0.0, 1.0);

This way, up to 65536 different IDs can be encoded using the red and green color channels. Blue and alpha are left blank for future extensions. (Note that the integer range [0, 255] is crunched to the float range [0.0, 1.0] by dividing by 255.) An exaggerated example of how such a picking buffer could look like is given in Figure 5.7. In order to pick an object at a screen space position in question, the modeler passes the GL_COLOR_ATTACHMENT1_EXT buffer to the glReadBuffer function, and requests exactly one pixel using glReadPixels. The pick ID can be easily reconstructed then: ID = r · 256 + g, where r and g are the corresponding values read out of the red and green channels. This method is called color picking and is in fact an elegant solution to the picking problem nowadays. Before the establishment of MRT, one had to execute an additional rendering pass for picking, where every object was drawn in a distinct color; or, the pick ID was encoded in the framebuffer using bitwise operations, which is more or less a “dirty hack” rather than an elegant solution. Even better, due to the separate color buffer, the user can pick objects in the thermographic or any other view that uses a false-color visualization. Filling the whole color picking buffer during a render pass is reasonable, instead of rendering only on a picking request—it can be used for the edge detection algorithm in the selection shader. The yellow outline produced by the selection shader is quite helpful to perceive which object is currently selected. Figure 5.10 illustrates a few examples. In (a), a torus is selected, which overlaps with other primitives. Due to the edge detection, it still becomes clear where its outlines are. A bit ambiguous, however, is the case (b), where a sphere behind a little box is selected. Could the box be selected as well? What if the user zooms in so that the boundary

Implementation: Interactive Modeling with Distance Fields

(a) Selected torus,

(b) selected sphere,

Figure 5.10: Various objects selected via color picking and our simple edge detection algorithm.

of the sphere is no longer visible—is the cube the only selected object then? A first solution would be to shade the whole selected object in a specific color, like in (c). Then, however, objects that are usually in the same color by default—like a lemon, for instance—could be interpreted mistakenly as selected or not selected. Traditional modeling applications may use bounding boxes to indicate selected objects. Based upon polygonal meshes, the geometry of this bounding box computes quite easily by taking the minimum and maximum of the object’s geometric coordinates. When using distance fields, on the contrary, it is possible to stretch primitives to infinity; or clone them endless times. How would one draw a bounding box containing an infinite volume? Using our selection shader is a suitable solution for selection indication, in the author’s view. In addition to picking objects on the central modeling widget, the user can select objects by clicking on the corresponding item in the QTreeWidget that visualizes the scene graph tree in the Scene page. The viewports are automatically updated with the new selected object then. In the next section, manipulators are going to be discussed. In theory, these could be given names in order to utilize the OpenGL selection mode for picking them. This would be very inconsistent, as we already implemented a working and robust color picking environment.

5.7 Manipulators In traditional modeling applications, manipulators provide a visual representation of the available transform commands. To be able to compare the behavior of our modeling tool to that of established applications, we stick to basic manipulator functionality as an experienced user would expect them. Figure 5.11 presents a variety of manipulators in other systems. The translation, rotation, and scaling manipulators of the distance field modeler are presented in Figure 5.12. Note that it is possible to place and align objects in space without the use of manipulators. The user can directly edit RBT and scaling nodes using the property editor in the Scene page. This is especially useful for precisely aligning and orienting objects. However, using manipulators first is very convenient for coarse arrangements.

5.7.1 Interactive Translation Translating objects interactively using the central modeling widget is implemented as follows. If the modeler is in its translation state, it is checked whether the currently selected object

5.7 Manipulators

(a)

(b)

(c)

(d)

(e)

(f)

(g)

Figure 5.11: 3D manipulators in other systems—the visual design varies slightly, and functionality is largely identical. Translation in 3DS Max (a) and Blender (b), rotation widget from XSI (c), and combo-widgets from Houdini (d), Modo (e), and Maya (f, g). Image and idea taken from [47].

has an RBT node as one of its ancestors. If such a node is found, it is used for subsequent translations, if not, an RBT node is created and directly inserted as a parent node for the object: then, the former object’s parent possesses the RBT node as its child. Next, the center of the object has to be computed. This is done by traversing the scene graph and taking every transformation into account; RBT and scaling nodes being the most prominent ones. Note that it is not always obvious where the center of an object is—consider grouping several primitives using a CSG union, by way of example. After that, the translation manipulator is drawn at the center of the object. Basically, it consists of three colored arrows parallel to the coordinate axes. Viewport and camera settings are taken into account, so that the manipulator always occupies the same amount of image space. Using color picking, the modeler determines when the user clicks on an arrow and starts the interactive translation. The active translation axis becomes yellow. In the first instance, it has to be determined where exactly on the manipulation arrow the user has clicked. This could be calculated in image space—compute the point on the translation axis that is nearest to the point clicked in the viewport. However, the modeling tool computes the nearest point in three-dimensional space. See Figure 5.13 for an illustration. Consider a line described by the active translation arrow, i.e. a line that contains it. In the example figure, the translation arrow is parallel to the x-axis, and the line that contains it is indicated by red dots. Construct another line perpendicular to the image plane that goes through the point clicked on the viewport. In the figure, this line is parallel to the view vector v and the position of the mouse cursor is referred to as M . Calculate the line segment that represents the minimum distance between these two lines,

(a) Translation,

(b) rotation,

Figure 5.12: Introduction of the basic object manipulators of the distance field modeler.

Implementation: Interactive Modeling with Distance Fields

A I

φx

)

v Image plane

C (a) Initiating translation along the xaxis at point A on the manipulator,

Im ag ep lan e

(b) mouse position M in screen space is projected to φx (M ) on the line through the active translation axis.

Figure 5.13: First steps of translating a sphere. Eventually, the sphere has to be relocated so that A is at the same position as φx (M ). The camera is located at C and views into the direction of v.

indicated by a dashed line in the figure. Let φ{x,y,z} (P ) be a function on R2 into R3 which maps the viewport coordinate P to its plausible projection on the line described by the translation arrow parallel to the x, y, or z-axis. Then, the intersection of the dashed line segment and the red dotted line is at φx (M ). When initiating a translation by clicking at M , the mouse location is projected onto the translation arrow and saved as point A = φx (M ). (Analogously, φy (M ) or φz (M ) for translations parallel to the other axes.) The mouse cursor is supposed to “stick” on A. The determined initial center of the object in translation is referred to as I. Over the period of moving the mouse over the viewport with the left mouse button pressed, the cursor position M is perpetually mapped to φx (M ). Now, the object has to be relocated according to the user’s mouse movement. The vector pointing from the initial object center I to the “sticking” point A is (A − I). Set the new object center to φx (M ) − (A − I). The corresponding RBT node in the scene graph is updated and the modified uniform uploaded to the graphics card. Viewports are updated; note that a time-killing recompilation of the shader is not necessary, which is absolutely essential for interactivity. Eventually, the translation is finished after the user releases the left mouse button. The highlighted arrow is drawn using its default color again. We described a translation on the so-called “global” axes. Professional modeling suites offer “local” axes aligned to the object, which could be determined by evaluating the RBT node in our case. Another possible alignment is parallel to the image plane boundaries, i.e. “camera” axes, which correspond to the up and right vectors in our ray marcher.

5.7.2 Interactive Rotation Rotations are initiated when the user clicks on a circular rotation manipulator, the wheel, as depicted in Figure 5.14. Again, if no RBT node exists as an object’s ancestor, one is created. Similar to the “sticky” point A during translation, it has first to be determined at which position the user clicked on the manipulator wheel. Consider a line perpendicular to the image plane through the viewport location of the mouse cursor. This line is intersected

5.7 Manipulators

Figure 5.14: Rotation of a cube.

with a plane that embeds the active manipulator wheel. The intersection line is connected with the object center to construct a “base line” to estimate the rotation angle later. During mouse movement, lines from the object center to the intersection of the line through the mouse position and the wheel’s plane are perpetually constructed. Using basic trigonometry, the current rotational angle can be easily computed. Internally, RBT nodes store rotations as quaternions due to simplicity and avoidance of gimbal locks. A new quaternion representing this rotation is constructed and multiplied with the quaternion stored in the RBT node. (Note that this is not commutative.) A rotation matrix is computed out of the quaternion, the model is updated, and uploaded as a uniform to the graphics card. Views are updated without generating a new shader. Eventually, the interactive rotation ends after the user released the left mouse button. Quaternions, their transformation from and to Euler angles, and the computation of the corresponding rotation matrices were, to a certain extent, implemented by the author. Note that Qt 4.6, that wasn’t available yet when this diploma thesis started, will introduce the QQuaternion class. From the Qt snapshots, it already looks promising and, amongst others, features built-in nlerp and slerp, i.e. normalized and spherical linear interpolation between two quaternions, which could be valuable for camera movements, for instance. Later, CGAL was added as a library to the modeler, demanding the boost C++ libraries3 . Boost features comprehensive implementations of quaternion routines as well. Initially, the author decided to avoid boost on purpose not to bloat the modeling tool. Keep in mind that there are other established solutions for interactive rotations. A very common implementation, featured in many CAD applications for instance, is a virtual trackball. See [25] for an overview and mathematical foundations of popular virtual trackballs. An intuitive implementation is the rolling-ball method [22], that maps the mouse movement to the rotation of a sphere on a flat surface that is rolled using the palm of hand accordingly. This movement is context-free, i.e. entirely determined by the translation vector of the mouse cursor, without taking previous rotations into account. See [12] for an early and fundamental study how to rotate 3D objects using 2D control devices.

5.7.3 Interactive Scaling The process of mapping the mouse to scaling manipulators is very similar to the mapping described in the “interactive translation” section before. Instead of RBT nodes, however, 3

See http://www.boost.org.

Implementation: Interactive Modeling with Distance Fields

scaling nodes are required and added where necessary. The initial projection A is saved again. The cursor location is perpetually projected using the φ function. Consider the distance from the object center I to A as being a base factor of 1.0. The stretch factor is proportionally computed as follows: kφ(M ) − Ik , kA − Ik then, the scaling node is updated, the corresponding uniform uploaded to the graphics card, and the viewport updated without recompiling the shader. Reminisce about the circumstances caused by non-uniform scalings and the difficulty to correct distorted distances, as discussed in Subsection 3.5.2 on page 26. Altering the object’s dimensions directly instead of non-uniform scaling should be preferred; the uniform scale, though, is absolutely unproblematic. Best-known implementations offer the user a dedicated part of the manipulator or a modifier key to explicitly equalize all scaling factors.

5.8 Constructive Solid Geometry Throughout the thesis, we pointed out the ease of constructive solid geometry using distance fields several times. In this section, a hands-on example is given at full length in Figure 5.15, where the process of modeling a cup of coffee using the distance field modeler is explained in detail. First, a basic sloped cup is modeled in (a), and extended with a handle to a mug in (b). Next, a thin layer of foam is modeled in (c) as a topping. It is textured using a coffee foam texture; note that fixed lighting occurs due to the photographic nature of the texture. One can conceal the fixed lighting a bit by using bump or displacement maps, in order to arrange the normals for shading in the desired position. The combined result is shown in (d), and a textured rendering in (e). Eventually, (f) shows the corresponding scene graph, which would look more or less the same in traditional modeling suites. Note that rigid body and scale transformations are omitted for clarity. Applicable CSG operations are union, intersection, and difference, as explained in the “Boolean Algebra” Section 3.6 on page 30. Grouping objects can be implemented using the union operation; this way, a new unique pick ID can be assigned to the whole group, that can be transformed like a single object then. Instead of concatenating objects using the min function, intersection and difference operations use the max function to generate their shader source. Rearranging objects within CSG transformations is possible using drag-and-drop on the scene graph widget in the Scene page. A unique feature of our modeler, which distinguishes it from traditional modeling, is that no meshes have to be computed and generated to represent the outcome of a CSG operation. This way, it is possible to interactively translate, rotate, or scale operands of CSG transformations, with the result directly rendered to the screen. It is even possible to interactively transform invisible objects, which were subtracted from others. Fortunately, we’re only interested in rendering shallow objects at the moment. Keep in mind that solid modeling for CAD applications is not as convenient as it looks at first sight, as discussed before; distances can be utterly wrong in the inside. Intersections, on the other hand, may deliver significantly truncated distances. This doesn’t raise a problem for ray marching, but extends rendering time.

5.8 Constructive Solid Geometry

(a) Modeling the cup. Start with a cylinder as the basis. Subtract a cylinder with a smaller radius to model a cup. (One can already drink out of it!) Slightly morph to a cone to get a nice slopy cup.

(b) Modeling the handle. Start with a torus and hyperize it by squaring the domain of the ground plane. Subtract a box to construct a well-formed handle that doesn’t extend into the cup.

(c) Modeling the top layer of foam. Apply a coffee foam texture to a thin cylinder to brew a delicious topping. (At least in a graphics sense.) (d) Combined result.

Union

Difference

Morph

Hyperize Difference

(e) Rendering. Texturing with little displacement to conceal the fixed lighting of the coffee foam.

(f) Corresponding scene graph with CSG, morph, and hyperize transformations.

Figure 5.15: Sequence of operation diagram. A cup of coffee is modeled using CSG operations.

Implementation: Interactive Modeling with Distance Fields

5.9 Applying Materials We already emphasized the importance of the literally omnipresent textures in computer graphics and depicted basic parameterizations in the previous chapter. In this section, it is explained how the modeler keeps track of its textures internally. As a generalization, we use the term material, that embraces colors, textures, and displacement maps in the distance field modeler’s context.

5.9.1 Organization The material panel gathers and administrates all the materials the user adds in a QListWidget. It offers loading textures and displacement maps from files, specifying colors, adding a variety of sample colors, and deleting materials. Colors are simply stored by their RGB values and an optional name. Keeping track of textures, on the other hand, is not as straightforward. When rasterizing polygonal meshes, one can change textures as often as desired, after an arbitrary amount of polygons, during a single rendering pass, and can stick to the main texture unit, as long as multi-texturing is not favored. Keep in mind that the ray marching fragment shader is executed by rendering a single, full-screen quad; only one texture can be bound to a texture unit then. Texture units are usually limited to 8, which makes it nontrivial to pass more than 8 textures simultaneously to the shader. Of course, a mature modeling environment should allow the user to use more than 8 textures at once. Hence, the distance field modeler keeps track of two texture atlases. In former times, a texture atlas was usually a 2D texture containing multiple textures arranged next to each other. (One had to be very cautious not to leak into adjacent textures—during interpolation, for instance.) We use 3D textures as atlases here, one with a width and height of 512 each, and one with 1024 × 512. See Figure 5.16 for an illustration of the organization. Both are bound to texture units, so the ray marching shader can access them. Assigned colors, however, are written explicitly in the shader during shader generation.

5.9.2 Color Textures

id e

The process of loading a texture from file and adding it to a texture atlas is as follows. First, the user selects images out of a QFileDialog. By default, Qt supports reading various image

(a) Rectangular,

(b) wide textures.

Figure 5.16: Texture organization using two 3D textures as atlases.

5.9 Applying Materials

types, ranging from Windows bitmaps over GIFs, JPGs, PNGs, and TIFFs to X11 bitmaps and pixmaps. A selected file is stored in a QImage. A preview icon for the QListWidget item representing the texture is created. According to the image aspect ratio, it is classified as “rectangular” or “wide” and inserted in the texture atlas that suits best, optionally resized to fit 512 × 512 or 1024 × 512 if required. Next, it is converted to a QPixmap and transformed into an OpenGL format using convertToGLFormat(). Eventually, the texture atlases are updated. The modeler keeps track of the textures in main memory, concatenating existing and new textures using the memcpy() function. An atlas is defined at once using the glTexImage3D() function. Alternatively, one could leave the list of pixmaps intact and update new parts of the atlas using glTexSubImage3D() to specify a 3D subimage. Unfortunately, all texture coordinates referring to the “depth” within the atlas change at that time and have to be evaluated again. Texture coordinates are usually referred as s, t, r, and q in a homogeneous coordinate system. In GLSL, r is already used as a selector for “red” in RGBA quadruples, hence the “depth” coordinate of a 3D texture is called p by convention. Consider a 3D atlas containing just two images. Then, these two images have a p-coordinate of 0.25 and 0.75, respectively: the coordinate range [0, 1] is split into halves, with 0.25 and 0.75 being the center of [0, 0.5] and [0.5, 1]. If linear interpolation is enabled, one can receive an evenly balanced blend of both textures by addressing p = 0.5. In general, one can compute p for a texture at the i-th position in an 3D atlas containing n textures as 1 1 pi = +i· . 2n n Obviously, p changes if n changes, so every assigned texture coordinate has to be recomputed again after adding new textures to the materials panel; eventually, the shader is regenerated taking the new coordinates into account, compiled, and uploaded to the graphics card. For each primitive, the user can influence the s and t-coordinates for texture lookup by adding a particular offset and scaling factor. “Wide” textures are appropriate for many parameterizations, for a sphere, cylinder, or torus by way of example, that map one coordinate to the range [−π/2, π/2] and the other to [0, 2π].

5.9.3 Displacement Maps Internally, displacement maps are treated in absolutely the same way as regular color textures. The process of loading them from file, adding them to the atlases, and administrating them using the materials panel is identical to the process described before—yet a single flag is set to indicate that it is considered a displacement map rather than a color texture. Images are added to an atlas in the order of their arrival. The sequence of textures and displacement maps is not important, and can alternate. In fact, even a regular color texture can be used as a displacement map, and vice versa. Instead of determining the diffuse object color, a displacement map adds its intensity to the distance of the object. We already pointed out how simple it is to implement this kind of mapping using distance fields, but be cautious of distortions of distance again. A broad example of applying materials to various objects is given in Figure 5.17. In (a), a cubical pattern is applied to a sphere, (b) shows displacement mapping on a torus, and in (c), a cylinder is used as a basis for modeling a rusty old barrel.

Implementation: Interactive Modeling with Distance Fields

“Antique White” (a) Texturing, coloring, and displacement mapping using a cubical pattern on a sphere. Materials can be dragged out of their panel and dropped on the desired object. (With kind regards to M. C. Escher.)

(b) Displacement mapping on a torus.

(a) (continued) Amplified displacement.

(c) Texturing a rusty old barrel that looks dented. In general, textures don’t map as desired straightaway. In the rightmost image, the texture has been scaled to the right to fit the barrel.

Figure 5.17: Various examples of color and displacement mapping.

5.10 Visual Analysis

5.10 Visual Analysis Quite often, it is helpful to gain insight about the underlying distance field of an object or a whole scene. Especially for development and implementation of new techniques and objects, it can be valuable to receive feedback about the impact on the distance returned. Visualization techniques help to communicate the geometric background of distance fields to the user. A very basic but common technique to visualize three-dimensional data is to restrict the viewer’s attention to a two-dimensional slice plane through the volume. On this slice plane, data is mapped to perceivable geometric primitives and colors that are rendered using computer graphics. On the Distance Analysis page, the distance field modeler offers the user to specify such a slice plane through the x-y-plane, centered at the origin. There are three possible transfer functions that map distance values to color. See Figure 5.18 for an overview. We already used isocontours to illustrate a distance field’s nature quite often. In general, these level sets—isocontours in the 2D, isosurfaces in the 3D, isohypersurfaces in the higher dimensional case—are one of the fundamental ways of visualizing phenomena. In addition, the distance can be visualized by mapping it to the color red on surfaces, that fades away as distance increases. Optionally, the inside can be mapped to blue, that fades away as it is advanced deeper inside of the objects. A mix between those two techniques is to clamp the color values into discretized steps.

Figure 5.18: Distance analysis on a slice plane through the scene.

Implementation: Interactive Modeling with Distance Fields

3 "torus.raw" matrix

2.5

optionally: set terminal postscript color set output "torus.ps"

1.5

0.5

set cbrange [0:3] plot "torus.raw" matrix w image

Figure 5.19: Raw distance export of a torus as torus.raw and gnuplot import. A matrix plot is used to visualize the image. For this thesis, the matrix plot was directly rendered into a postscript file.

Various drawbacks and specifics of our way to deal with distances have been discussed throughout the thesis. In fact, Figure 5.18 recapitulates quite a lot. First, note how the box approximation is responsible for square-cut isocontours, which can be spotted in the upper right, for instance. Compare this to Figure 3.10. Next, see how the nature of the cone as a double cone, placed apex to apex, is still leaking through the whole distance field, despite being clipped internally by a plane using CSG. This can be spotted in the top part; compare this to Figure 3.12. See how the lower right part of the cone even leaks through the sphere’s distance field in the lower right part of the image. (This can only be seen in the image with the isocontours, as the red color already faded away there in the other ones.) Eventually, if observed carefully, one can discover that the object intersections don’t feature correct distances in the inside; we discussed this problem back in Section 3.6. On the implementation side, the distance values for the slice plane are evaluated using the regular ray marching shader in an additional render pass prior to the actual rendering. A uniform Boolean is set, and the image plane of the ray marcher is set to match the requested slice plane. Distances on the image plane are evaluated and directly returned, and no rays are traced until a surface is hit. The outcome is written into a texture and later rendered using an additional, dedicated visualization fragment shader, that is set up according to the desired transfer function using its own uniforms. The user can decide to save the produced image of the slice plane as a PNG file, or can export the raw distance values to a file. These raw data files can, for instance, be processed by gnuplot subsequently. See Figure 5.19 for an example.

5.11 Closing the Gap to Traditional Modeling As the thesis approaches the end of the main chapters, we will complete our modeler by clearing the way for incorporating traditional triangle meshes into our system. This way, we offer the user two benefits: First, the rich supply of available polygonal models can be utilized to enrich scenes. Second, a promising object modeled with our tool can be used in professional modeling applications subsequently. Both features are to be presented in this chapter to bring the modeler and the main chapters down to a round figure.

5.11 Closing the Gap to Traditional Modeling

Direct sources, e.g.

PLY

export Blender

Stanford 3D Scanning Repository

import

ply2vri Common mesh format, export Distance Field Modeler

3ds, off, x, md2, ms3d, raw, obj, . . .

Figure 5.20: Process of computing a df file, a raw discretized distance field.

5.11.1 Converting Triangle Meshes into Discretized Distance Fields In order to import a meshed model into our modeler, a discretized distance field has to be computed first. We utilize the ply2vri command line tool4 for converting triangle meshes in PLY format into signed-distance volumetric grids. PLY was developed at Stanford University, and most models in the Stanford 3D scanning repository 5 are stored in this format. Originally, ply2vri was developed to convert into the vri format, used in VripPack, the Volumetric Range Image Processing Package 6 for volumetrically merging a set of range images. See [13] for an application to surface reconstruction from range images. During conversion to vri, evaluated distances are quantized from double precision floats, which are used internally, to unsigned shorts, ranging from 0 to 65535. (Note that the projects mentioned above got a bit long in the tooth—the paper [13] appeared at SIGGRAPH 1996, for instance—which is sort of a justification for restricting to 16-bit data types.) The author modified the ply2vri script to be able to export raw float distances as well—later, they will be loaded into 3D floating point textures—and maintained the old code a bit so that it compiles using gcc 4. The new format for output is called df, which stores a grid of explicit distance values as floats. A df file has a header of four integers, specifying the x, y, and z range of the volume, and the amount of intersections per unit; then, the raw data is dumped. Three different ways of df file acquisition are illustrated in Figure 5.20. First, one can start by using a PLY model, for example from the Stanford 3D scanning repository, and convert it using ply2vri. In addition, there a dozens of models available in different formats. There exists a variety of tools to convert them to PLY, one example being the modeling application Blender, that is able to import many formats and export them to PLY. A benefit is that Blender is free software; note that Maya, for instance, could serve the same purpose. Then again, one can convert them to a df file. In the end, one can even use our distance field modeler to export a specified volume directly to df. The distance field modeler offers four import slots for df files. Each object is stored in its own 3D floating point luminance texture. (Note that in the case of all slots filled, already 6 out of 8 texture units are occupied, taking the two texture atlases into account.) We refrain from the atlas concept for explicit distance fields. Four df files could, in theory, be stored in a 4

See http://grail.cs.washington.edu/software-data/ply2vri for the project page. See http://graphics.stanford.edu/data/3Dscanrep. 6 See http://graphics.stanford.edu/software/vrip. 5

Implementation: Interactive Modeling with Distance Fields

(a) Distance analysis,

(b) work image,

Figure 5.21: Explicit distance field imported into the distance field modeler. The workload is comparatively high, as it is ray marched in very small steps where no distance information is available. Rightmost, impacts of parameter settings for ply2vri are shown; increased ramp length on the top, soft edges at the bottom, and default settings centered.

single 3D RGBA texture, with one dedicated channel for each distance field. This will cause a lot of fussiness, however, if those files differ in size significantly; a lot of texture space will be wasted, as the texture dimensions are stretched to fit the largest object. If a larger object is appended, the previous ones have to be torn apart and re-padded with “blanks”. We won’t circumvent this problem by concatenating distance fields in depth using a single 3D texture atlas as well. Keep in mind that discretized volume data expands rapidly in size. A volume with a dimension of 12 × 12 × 12 and 32 subdivisions for each unit already occupies 216 MB, if 32-bit floats are used as data types. (That is 123 · 323 · 4 bytes.) Considering that a typical consumer graphics card features about 1 GB of video memory these days, it becomes clear that this is rather not enough for opulent models. We introduced the principle of integrating explicit distance fields in our system back in Section 3.9. First, a rough distance to the volume’s bounding box is given; then, cautious ray marching is appropriate inside the volume. The ply2vri tool converts according to a particular ramp length, which indicates how many voxels in each direction from the surface are taken into account; the rest of the voxels don’t contain valid distance data and are set to the voxel grid size. See Figure 5.21 for an example. In (c), the default conversion settings are illustrated in the center. (Used in (a) and (b).) A doubled ramp length is featured at the top. Note that this is valuable if one has morphological operations like dilation or erosion in mind. At the bottom, “soft edges” were enabled, that fill in the true signed distance reported during conversion at voxels whose closest surface point is at the boundary of a hole in the mesh. (Note how the bottom hole of the Stanford Bunny is filled.) These voxels remain empty using “hard edges”, however. Depending on the model, this can lead to an eroded, ragged look. See Figure 5.22 for an example. The tube appears porous due to laxly edge interpretations of ply2vri. Note that requesting soft edges doesn’t extend

5.11 Closing the Gap to Traditional Modeling

(a) Hard edges,

(b) soft edges.

Figure 5.22: Conversion of the VIS institute logo into an explicit distance field.

conversion runtime. Another parameter for conversion is the voxel resolution, which specifies the length of each voxel depending on the meshed object’s coordinate system. See Figure 5.23 for an example, where conversion artifacts occur. The alien model is quite tall and slim, and extends in depth due to its long tail; yet it features filigree details as the fingers, for instance. If the voxel resolution is set quite high in order to capture these details, conversion suffers from a rather long runtime. Furthermore, many voxels are basically “wasted” on the coarse torso and for empty space surrounding the model. On the other hand, if voxel resolution is too low, artifacts may occur as in (b) and (c), despite the torso, arms, and legs being very well represented. This is independent of deciding for hard or soft edges. Note that the Stanford Bunny is a much more homogeneous and compact model, being sort of “low-maintenance” in this case. For the alien model, however, a hierarchical grid with an adaptive voxel cell resolution would be appropriate—which is no longer straightforward to implement. After importing explicit distance fields, they can be treated like any other object in the

(a) Mesh in Blender,

(b) distance conversion,

Figure 5.23: A slim and rangy alien model imported into the distance field modeler. Conversion artifacts occur at skinny detail regions; the error is already contained in the explicit distance field.

Implementation: Interactive Modeling with Distance Fields

scene, both implicit and explicit. Arbitrary combinations and transformations are possible. Eventually, they can be exported as an explicit distance field again, or converted into a traditional triangle mesh. The latter process is going to be discussed now. In the end, we showed that our modeling tool already provides basic modeling based on explicit surface representations.

5.11.2 Generating Triangle Meshes from Distance Fields The most well-known algorithm to compute a triangle mesh of an isosurface from threedimensional scalar volume data is probably the marching cubes algorithm [32]. One could apply it to a distance field in order to extract an isosurface of level zero that corresponds to an explicit polygonal approximation of the implicit surface. We utilize the more sophisticated surface mesh generation algorithm provided by CGAL. It is mainly based on the work [8], that provides good approximations of normals, areas and curvatures. A notable feature of the algorithm is that the surface needs only to be known through an oracle that, given a line segment, detects whether this segment intersects the surface and returns the intersection points if so. This makes the algorithm useful in a wide variety of contexts, especially for our demands, and for various kinds of surfaces. To launch the meshing process, we have to provide CGAL with a pointer to a function that returns the value of the distance field for a given point. Then, CGAL is able to request distances for particular points and takes over mesh generation for us. Hence, our only task is to write such a function. We already have a complete distance field evaluation function implemented in our ray marching shader. In order to respond to the meshing algorithm’s requests, we could execute the shader after each request and render a single pixel, instructed to evaluate the distance field at the particular point in question. However, this would be kind of exhausting to permanently switch between CPU and GPU operations, causing high overhead for computing a single value. Instead, we ask the user to specify a bounding box around the volume to be meshed. This limitation is passed to CGAL’s mesher as well. Then, we slice through the volume as in

(a) Distance field export,

(b) import into Blender,

Figure 5.24: Surface mesh generation of a distance field.

5.11 Closing the Gap to Traditional Modeling

(a) Multiple objects export,

(b) Blender import,

Figure 5.25: Export of disjoint objects.

the “Distance Analysis” Section 5.10 step by step and concatenate the results to a threedimensional look-up table of the distance within the bounding box. Requests from the CGAL mesher can be satisfied by tri-linear interpolation inside the discretized volume. Whereas this method is not as exact as executing the shader for each request, it is significantly faster. Eventually, the mesh is saved in the object file format (off ). See Figure 5.24 for an example; note that the distance field expression requires tremendously less storage space than the explicit mesh, which requires 28.8 MB (11.3 MB gzip compressed) without further treatment for its approximately 390 000 vertices and 780 000 faces. The algorithm is capable of meshing disjoint objects as well, as demonstrated in Figure 5.25. Whereas meshing works best for curved surfaces, the outcome is rather poor for hard edges, see Figure 5.26. One can reduce the radiuses of the Delaunay balls [14] CGAL uses for triangulation, leading to significantly more faces. However, these can be appropriately merged with subsequent treatments.

(a) Box with 400 vertices,

(b) box with 40 000 vertices,

Figure 5.26: Two different granularities for exporting a box. For (a), Delaunay balls of size 0.1 were used, taking less than one second to export on an Intel E6850. For (b), Delaunay balls were sized 0.01, and export took around 12 seconds.

6 Conclusions We have discussed the characteristics and quirks of distance fields, their qualification for effective ray marching, and our suggestions for implementing a modeling tool based upon distance fields at great length. This chapter concludes the thesis by summarizing our work. First, statements on the results of our efforts on developing a modeling application built upon distance fields are given. Then, we take a prospect on possible future work. Eventually, some final remarks close the thesis.

6.1 Results Our modeling tool, the distance field modeler, is able to provide an interactive environment for modeling basic objects and arranging basic scenes. Just like in traditional modeling, primitives can be created, transformed, arranged, textured and modified in an intuitive way, just like an experienced user would expect it. Basic scenes can be rendered in real-time by the integrated ray marcher, which is closely coupled with the modeling application. This allows for a direct visualization of the distance field and immediate visual feedback during modeling. From the user’s perspective, an artist is able explore the impacts of transformations on distance fields directly, without the requirement of expressing these mathematically by oneself; this task is taken over by the modeler. Technically, we explained how to implement hybrid rendering, by carefully adapting the ray marcher’s camera settings to OpenGL’s modelview and projection matrices and using the z-buffer to solve the visibility problem. Whereas our modeler lacks some of the flexibilities of polygonal meshes, like simply picking a vertex and moving it around, we can express a variety of effects by very simple mathematical operations. Some of them, like infinite repetitions, are not even possible to express explicitly without further extensions. Moreover, a simple twist of a planar box would require the computation of a completely new mesh, with considerably more vertices than before; using implicit surfaces, a basic trigonometric expression is sufficient. In general, we are not restricted to any kind of approximations of curved surfaces. On the contrary, simple objects like spheres have to be tessellated by traditional modelers based on polygonal meshes. Displacement mapping is particularly easy to implement on implicit surfaces, by simply adding height information to the result. Rendering using displacement maps on meshes requires more effort due to the necessary mesh refinement. Basic emulation using bump maps can be implemented in the same fashion by disturbing normals during shading, requiring the exactly same amount of computation. A particular strength of the ray marcher based on distance fields is the way soft shadows and ambient occlusion are implemented. We proposed a technique that softens shadow

Conclusions

edges with almost no additional costs while still casting only a single shadow ray to each light source. By interpreting proximity information inherently stored in distance fields, we estimate plausible ambient occlusion in three-dimensional space. Traditionally, dealing with these global illumination aspects is not considered to be trivial. The modeling tool closes the gap to traditional modeling by providing interfaces to convert meshes to explicit distance fields, and to generate meshes out of distance fields. This way, a bidirectional connection to the well-known world of polygonal surface representations is established, and an artist can profit from comprehensive libraries of meshed models.

6.2 Future Work The field of computer graphics is broad and deep, and by writing a modeling application, one touches several graphics disciplines. A modeler could always be extended in various ways, and a few suggestions for future work follow. Concerning the modeling process itself, one could focus even more on procedural content generation, without the artist having to deal with underlying mathematics. For starters, an integrated editor for procedural textures could be implemented. Not only textures, but geometry as well could be taken into account; think of genetic modeling, by way of example. It turns out that implicit modeling techniques are helpful to model organic, natural looking shapes; these are appropriate for procedural descriptions as well. Using noise, one can emulate deformations found in nature. Parameterizations in the algorithms can be used to vary the outcome. Additionally, one could influence the objects directly using sculpting techniques. As a start, a floating point texture could be mapped to an object, which is editable and acts as a displacement map. Note that sophisticated parameterizations are required; otherwise, sculpted elevations or reductions could either vanish or appear at undesired positions. Our modeling tool lacks support for animation. The concept of specifying key frames and interpolating the scene arrangements in between could be transferred to implicit surfaces as well. Moreover, a time parameter keeping track of an elapsed time could be integrated into procedural descriptions of textures and objects. This would already allow for a variety of basic effects, without putting too much effort into development. Concerning image synthesis, there is still much room for improvement. Due to Phong illumination, objects appear as if they were made out of plastic. One could integrate bidirectional reflectance distribution functions [33] in order to define how light is reflected at surfaces. Solid textures are very applicable due to their canonical mapping to three-dimensional objects. Historically, they were almost exclusively described procedurally. Today, research is taking place in the field of texture synthesis, and recent approaches are given in [15] and [30]. The ray marcher is able to sample opaque surfaces. One could pay regard to translucent surfaces by continuing ray marching, blending further evaluations, and accumulating alpha information until a fragment is saturated. It is not straightforward to emulate global illumination effects like ray tracing, or particle tracing methods like photon mapping [26] are capable of, mainly due to the lack of a stack

6.3 Final Remarks

and complex data structures on the GPU. Work exists on how to port ray tracing and photon mapping entirely to the GPU [42, 43], though these ports don’t outperform well-organized CPU implementations yet. Nonetheless, it will be tempting to see if future hardware releases the programmer of the restrictions of current graphics hardware. After the main rendering step, one could think of subsequent image space post processing. On the one hand, enhancements like anti aliasing would improve the visual quality of the renderings. On the other hand, one could implement further effects like toon shaders, for instance, or any other methods known from image processing. Concerning mathematics and algorithms, one can think of various areas to work on. For sphere tracing, it is sufficient to estimate a lower bound on distances, so ray marching should decelerate up to the point where surfaces are no longer penetrated, but not more. Several strategies are feasible; analytically, numerically, and stochastically. On the CPU side, ray tracing can be greatly improved by space partitioning. There is ongoing research to construct these hierarchical structures in real-time on the GPU. See [31] and [55] for recent techniques.

6.3 Final Remarks Using implicit surfaces is not necessarily less practical than relying upon polygonal models—it is simply different. The surface representations require different techniques for their creation, transformation, and visualization. Historically, they had different applications, but we showed that one is not restricted to this. In the beginning, we had a somewhat critical attitude toward real-time rendering of distance fields, but it turned out that today’s graphics cards provide the necessary computational power with ease. On the other hand, preserving correct distances for transformations turned out to be not that self-evident as we initially expected; fortunately, estimating lower bounds is sufficient. Time will tell if implicit surfaces and distance fields gain ground in more computer graphics applications, thanks to consumer hardware that is able to render them interactively.

Acknowledgements The following persons contributed much to this thesis, and I am deeply grateful. Prof. Dr. Carsten Dachsbacher provided the initial idea to develop a modeling tool based on distance fields. He actively participated in supervision and provided the idea to close the gap to traditional modeling by converting triangle meshes into distance fields and vice versa. I am thankful that I was able to work on a really interesting and motivating topic. Dipl.-Phys. Gregor Mückl supervised me, and he supported me to a great extent. He is developing his own modeling application1 , Moonlight|3D, and thus a very qualified contact. He helped a lot when I was stuck in the perils of hybrid rendering. Uwe Bauknecht, Gregor Mückl, Hannes Mühleisen, and Marius Wernke proof-read this thesis thoroughly, which I appreciate very much. My family has been a great support during the last quarter-century. Thank you very much indeed, for everything you did for me.

The modeling tool and this thesis use icons from the Crystal Project KDE icon theme2 , and textures from the Filter Forge 3 and CG Textures 4 libraries.

http://www.moonlight3d.eu http://www.everaldo.com/crystal 3 http://www.filterforge.com 4 http://www.cgtextures.com 2

Bibliography [1] Arthur Appel. Some techniques for shading machine renderings of solids. In AFIPS ’68 (Spring): Proceedings of the April 30–May 2, 1968, spring joint computer conference, pages 37–45, New York, NY, USA, 1968. ACM. [2] Pierre Bézier. Emploi des Machines à Commande Numérique. Masson et Cie, Paris, 1970. Transl. as Numerical Control—Mathematics and Applications, Wiley, London, 1972. [3] Eric Bier and Kenneth Sloan. Two-part texture mappings. IEEE Computer Graphics and Applications, 6:40–53, 1986. [4] James F. Blinn. A generalization of algebraic surface drawing. ACM Transactions on Graphics, 1(3):235–256, 1982. [5] Jules Bloomenthal and Brian Wyvill. Interactive techniques for implicit modeling. In SI3D ’90: Proceedings of the 1990 symposium on Interactive 3D graphics, pages 109–116, New York, NY, USA, 1990. ACM. [6] Jules Bloomenthal and Brian Wyvill, editors. Introduction to Implicit Surfaces. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1997. [7] Jean-Daniel Boissonnat and Frédéric Cazals. Smooth surface reconstruction via natural neighbour interpolation of distance functions. In SCG ’00: Proceedings of the sixteenth annual symposium on Comp. geometry, pages 223–232, New York, NY, USA, 2000. ACM. [8] Jean-Daniel Boissonnat and Steve Oudot. Provably good sampling and meshing of surfaces. Graphical Models, 67(5):405–451, 2005. [9] Stefan Brabec and Hans-Peter Seidel. Single sample soft shadows using depth maps. In In Graphics Interface, pages 219–228, 2002. [10] Edwin E. Catmull. A subdivision algorithm for computer display of curved surfaces. PhD thesis, University of Utah, 1974. [11] Edwin E. Catmull and James H. Clark. Recursively generated b-spline surfaces on arbitrary topological surfaces. Computer-Aided Design, 10(i6):350–355, November 1978. [12] Michael Chen, S. Joy Mountford, and Abigail Sellen. A study in interactive 3-d rotation using 2-d control devices. In SIGGRAPH ’88: Proceedings of the 15th annual conference on Comp. graph. and interactive techn., pages 121–129, New York, NY, USA, 1988. ACM. [13] Brian Curless and Marc Levoy. A volumetric method for building complex models from range images. In SIGGRAPH ’96: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pages 303–312, New York, NY, USA, 1996. ACM. [14] Boris N. Delaunay. Sur la sphère vide. Bulletin of Academy of Sciences of the USSR, No. 6, pages 793–800, 1934.

Bibliography

[15] Yue Dong, Sylvain Lefebvre, Xin Tong, and George Drettakis. Lazy solid texture synthesis. Computer Graphics Forum (Proceedings of the Eurographics Symposium on Rendering), 27(4):1165–1174, 2008. [16] Alex Evans. Fast approximations for global illumination on dynamic scenes. In SIGGRAPH ’06: ACM SIGGRAPH 2006 Courses, pages 153–171, New York, NY, USA, 2006. ACM. [17] Pierre-Alain Fayolle, Alexander Pasko, Benjamin Schmitt, and Nikolay Mirenkov. Constructive heterogeneous object modeling using signed approximate real distance functions. Journal of Comp. and Information Science in Eng., 6(3):221–229, Sept. 2006. [18] James D. Foley, Andries van Dam, Steven K. Feiner, and John F. Hughes. Computer Graphics. Principles and Practice. Addison-Wesley, Reading, Massachusetts et al., 1997. [19] Sarah F. Frisken, Ronald N. Perry, Alyn P. Rockwood, and Thouis R. Jones. Adaptively sampled distance fields: a general representation of shape for computer graphics. In SIGGRAPH ’00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pages 249–254, New York, NY, USA, 2000. ACM Press/AddisonWesley Publishing Co. [20] Andrew S. Glassner, editor. An introduction to ray tracing. Academic Press Ltd., London, UK, 1989. [21] Andrew S. Glassner. Principles of Digital Image Synthesis. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1994. [22] Andrew J. Hanson. The rolling ball. Graphics Gems III, pages 51–60, 1992. Academic Press Professional, Inc., San Diego, CA, USA. [23] John C. Hart. Sphere tracing: A geometric method for the antialiased ray tracing of implicit surfaces. The Visual Computer, 12:527–545, 1994. [24] John C. Hart. Distance to an ellipsoid. Graphics gems IV, pages 113–119, 1994. Academic Press Professional, Inc., San Diego, CA, USA. [25] Knud Henriksen, Jon Sporring, and Kasper Hornbæk. Virtual trackballs revisited. IEEE Transactions on Visualization and Computer Graphics, 10(2):206–216, 2004. [26] Henrik Wann Jensen and Niels Jørgen Christensen. Photon maps in bidirectional monte carlo ray tracing of complex objects. Computers & Graphics, 19(2):215–224, 1995. [27] Mark W. Jones. Facial reconstruction using volumetric data. In VMV ’01: Proceedings of the Vision Modeling and Visualization Conference, pages 135–150. Aka GmbH, 2001. [28] Mark W. Jones, J. Andreas Bærentzen, and Milos Sramek. 3D distance fields: A survey of techniques and applications. IEEE Transactions on Visualization and Computer Graphics, 12(4):581–599, 2006. [29] James T. Kajiya. The rendering equation. SIGGRAPH Computer Graphics, 20(4):143– 150, 1986. [30] Johannes Kopf, Chi-Wing Fu, Daniel Cohen-Or, Oliver Deussen, Dani Lischinski, and Tien-Tsin Wong. Solid texture synthesis from 2D exemplars. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007), 26(3):2:1–2:9, 2007.

Bibliography

[31] Christian Lauterbach, Michael Garland, Shubhabrata Sengupta, David Lübke, and Dinesh Manocha. Fast BVH construction on GPUs. Computer Graphics Forum, 28(2):375–384, 2009. [32] William E. Lorensen and Harvey E. Cline. Marching cubes: A high resolution 3D surface construction algorithm. SIGGRAPH Comput. Graph., 21(4):163–169, 1987. [33] Fred E. Nicodemus. Directional reflectance and emissivity of an opaque surface. Applied Optics, 4(7):767–773, 1965. [34] H. Nishimura, M. Hirai, T. Kawai, T. Kawata, I. Shirakawa, and K. Omura. Object modeling by distribution function and a method of image generation. Trans. IECE Japan, Part D, J68-D(4):718–725, 1985. [35] James F. O’Brien and Terry S. Yoo, editors. Modern techniques for implicit modeling. ACM SIGGRAPH 2005 Courses, 2005. [36] T. J. O’Donnell and Arthur J. Olson. Gramps - a graphics language interpreter for realtime, interactive, three-dimensional picture editing and animation. SIGGRAPH Comput. Graph., 15(3):133–142, 1981. [37] Stanley J. Osher and Ronald P. Fedkiw. Level Set Methods and Dynamic Implicit Surfaces. Springer, October 2002. [38] Steven Parker, Peter Shirley, and Brian Smits. Single sample soft shadows. Technical Report UUCS-98-019, Computer Science Department, University of Utah, 1998. [39] Alexander Pasko, Valery Adzhiev, Benjamin Schmitt, and Christophe Schlick. Constructive hypervolume modeling. Graph. Models, 63(6):413–442, 2001. [40] Ken H. Perlin and Eric M. Hoffert. Hypertexture. SIGGRAPH Computer Graphics, 23(3):253–262, 1989. [41] Bui Tuong Phong. Illumination for computer generated pictures. Communications of the ACM, 18(6):311–317, 1975. [42] Timothy J. Purcell, Ian Buck, William R. Mark, and Pat Hanrahan. Ray tracing on programmable graphics hardware. ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH 2002), 21(3):703–712, July 2002. [43] Timothy J. Purcell, Craig Donner, Mike Cammarano, Henrik Wann Jensen, and Pat Hanrahan. Photon mapping on programmable graphics hardware. In Proceedings of the ACM SIGGRAPH/EUROGRAPHICS Conference on Graphics Hardware, pages 41–50. Eurographics Association, 2003. [44] Iñigo Quilez. Rendering worlds with two triangles with raytracing on the GPU in 4096 bytes. NVSCENE 08, August 2008. [45] Scott D. Roth. Ray casting for modeling solids. Computer Graphics and Image Processing, 18(2):109–144, February 1982. [46] Vladimir L. Rvachev. On the analytical description of some geometric objects. Reports of Ukrainian Academy of Sciences, 153(4):765–767, 1963. [47] Ryan Schmidt, Karan Singh, and Ravin Balakrishnan. Sketching and composing widgets for 3D manipulation. Computer Graphics Forum, 27(2):301–310, 2008. Proceedings of Eurographics 2008.

Bibliography

[48] Thomas W. Sederberg. Piecewise algebraic surface patches. Computer Aided Geometric Design, 2(1-3):53–59, 1985. [49] Jean Serra. Image Analysis and Mathematical Morphology. Academic Press, Inc., Orlando, FL, USA, 1983. [50] Inderjeet Singh, Beth Stearns, and Mark Johnson. Designing enterprise applications with the J2EE platform. Addison-Wesley Longman Publish. Co. Inc., Boston, MA, USA, 2002. [51] Wolfgang Straßer. Schnelle Kurven- und Flächendarstellung auf graphischen Sichtgeräten. PhD thesis, Technical University of Berlin, 1974. [52] Steve Upstill. RenderMan Companion: A Programmer’s Guide to Realistic Computer Graphics. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1989. [53] Turner Whitted. An improved illumination model for shaded display. Communications of the ACM, 23(6):343–349, 1980. [54] Geoff Wyvill, Craig McPheeters, and Brian Wyvill. Data structure for soft objects. The Visual Computer, 2(4):227–234, 1986. [55] Kun Zhou, Qiming Hou, Rui Wang, and Baining Guo. Real-time kd-tree construction on graphics hardware. ACM Transactions on Graphics, 27(5):1–11, 2008.

List of Figures 1.1 1.2

Comparison of a curve, surface, and their polygonal representations . . . . . . . Approximating a circle with polygons . . . . . . . . . . . . . . . . . . . . . . .

5 5

2.1 Applications of adaptively sampled distance fields . . . . . . . . . . . . . . . . . 10 2.2 “Slisesix” and “Organix” renderings . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19 3.20 3.21 3.22 3.23 3.24 3.25 3.26

Expressions for the unit circle . . . . . . . . . . . . . Right-handed Cartesian coordinate system . . . . . . Circle and distances to it . . . . . . . . . . . . . . . Five basic primitives . . . . . . . . . . . . . . . . . . Distance to circle . . . . . . . . . . . . . . . . . . . . Axis-aligned box centered at the origin . . . . . . . . Classification of different areas of interest for a box . Distances from various example points to the surface Limiting the volume of a box by six planes . . . . . Isocontours of a distance field of a box . . . . . . . . The cylinder primitive . . . . . . . . . . . . . . . . . The cone primitive . . . . . . . . . . . . . . . . . . . The torus primitive . . . . . . . . . . . . . . . . . . . Rigid body transformations . . . . . . . . . . . . . . Uniform and non-uniform scaling . . . . . . . . . . . Non-uniform scaling distorting distances . . . . . . . Horizontal shear . . . . . . . . . . . . . . . . . . . . Boolean algebra . . . . . . . . . . . . . . . . . . . . . Corrupted distances inside unified object . . . . . . . Union of two ellipsoids . . . . . . . . . . . . . . . . . Morphing sequence . . . . . . . . . . . . . . . . . . . Blending objects . . . . . . . . . . . . . . . . . . . . Twist and noise displacement effects . . . . . . . . . Infinite repetitions . . . . . . . . . . . . . . . . . . . Dilation on a unit cube by a sphere . . . . . . . . . . True 3D distance closure . . . . . . . . . . . . . . . .

4.1 4.2 4.3 4.4 4.5 4.6 4.7

Basic camera setup for ray marching . . . . . Ray marching with constant step size through Sphere tracing, a hit and a miss . . . . . . . . Gaussian function turned upside down . . . . Gradient estimation . . . . . . . . . . . . . . Phong illumination model . . . . . . . . . . . Sphere rendered using Blinn-Phong . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . of a box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . volume . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

14 15 15 16 17 18 18 19 20 21 22 23 23 25 27 28 29 30 31 32 33 34 35 35 36 36

. . . . . . .

40 40 41 43 45 45 46

100

List of Figures 4.8 4.9 4.10 4.11 4.12 4.13 4.14 4.15 4.16

Three different texture types . . . . . . Sphere textured with earth texture . . . Box textured with photograph . . . . . Two-part texture mapping . . . . . . . . Three-dimensional procedural textures . Soft shadows . . . . . . . . . . . . . . . Work image . . . . . . . . . . . . . . . . Concept of ambient occlusion estimation Shadows produced by our ray marcher .

. . . . . . . . .

47 48 48 49 50 51 52 53 53

5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11 5.12 5.13 5.14 5.15 5.16 5.17 5.18 5.19 5.20 5.21 5.22 5.23 5.24 5.25 5.26

Qt framework and our derivation . . . . . Main application window . . . . . . . . . Settings panel . . . . . . . . . . . . . . . . Direct shader manipulation . . . . . . . . Model-View-Controller architecture . . . . Arrangements of viewports . . . . . . . . Hybrid rendering . . . . . . . . . . . . . . Computing the depth for z-buffering . . . Hybrid rendering of a sphere and a box . Selected objects . . . . . . . . . . . . . . . 3D manipulators in other systems . . . . . Manipulators of the distance field modeler Translating a sphere . . . . . . . . . . . . Rotating a cube . . . . . . . . . . . . . . . Modeling a cup of coffee . . . . . . . . . . Texture organization using 3D atlases . . Color and displacement mapping . . . . . Distance analysis . . . . . . . . . . . . . . Raw distance export and gnuplot import . Computing a df file . . . . . . . . . . . . . Importing explicit distance fields . . . . . Converting the VIS institute logo . . . . . Conversion artifacts at detail regions . . . Surface mesh generation . . . . . . . . . . Export of disjoint objects . . . . . . . . . Granularities for exporting a box . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

56 57 60 61 63 64 68 69 69 72 73 73 74 75 77 78 80 81 82 83 84 85 85 86 87 87

Declaration — Erklärung The work contained in this thesis has not been previously submitted for a degree or diploma at this or any other higher education institution. To the best of my knowledge and belief, the thesis contains no material previously published or written by another person except where due reference is made.

Ich versichere, dass ich die Arbeit ohne fremde Hilfe und ohne Benutzung anderer als der angegebenen Quellen angefertigt habe, und dass die Arbeit in gleicher oder ähnlicher Form noch keiner anderen Prüfungsbehörde vorgelegen hat und von dieser als Teil einer Prüfungsleistung angenommen wurde. Alle Ausführungen, die wörtlich oder sinngemäß übernommen wurden, sind als solche gekennzeichnet.

Stuttgart, February 15, 2010 (Tim-Christopher Reiner)