Digital Imaging: Anatomy of 3rd dimension

Proceedings of the URECA@NTU 2009-10

Digital Imaging: Anatomy of the 3rd Dimension Tan Cheng Quan School of Art, Design and Media

Asst Prof Ina Conradi Chavez School of Art, Design and Media

Abstract - Drawing on the ideas of recent technological initiatives, such as 3D stereoscopic, and avant-garde film-making and sound cultures, our work explores principles and methodologies of 3-D technology, motion and space. With the aim to craft meaningful, immersive, interactive experiences by integrating stereoscopic 3D into otherwise 2 dimensional forms, creating a new level of depth and artistic imagery.

2 AIMS / OBJECTIVES

Keywords - Art and Animation, 3D Stereoscopy, Stereoscopic Rig, 3D Maya.

Technology, Stereoscopic

The research aims to explore, develop, document stereoscopic workflow methodology and to produce artistic 3D content that can be output to various 3D systems such as traditional anaglyph and the newer polarized systems. My role was focused on the task of stereo-graphing 3D content and developing the stereoscopic pipeline.

Digital Filters,

3 LITERATURE REVIEW / BACKGROUND In the 1950s 3D stereoscopic films enjoyed a revival with the development of single camera stereo solutions with anamorphic lenses. It was however short lived, in theatres projectors needed to be perfectly matched and synchronized which was sometimes beyond the average projectionist's technical skills. It was difficult for post production techniques to be applied to stereoscopic images.

1 INTRODUCTION The objective of this project is to identify and experiment with 3D stereoscopic methods and techniques by utilizing 3D packages and compositing software. Often an artwork is either animated and flat, or has form but is static. Hence, we seek to employ stereoscopy as an avenue to fuse form with animation, to introduce the illusion of depth and form into motion art pieces for implementation in large gallery spaces.

(Figure 1)

It is only in recent years that our technology has developed into all digital production pipelines, allowing practical 3D content creation and screening of stereoscopic films. Many 3D stereoscopic life action and animated features were created and with the recent introduction of the consumer 3D TV, stereoscopic content is introduced in the household. [1]

Using stereoscopic depth as part of the design process, from concept design to 3D animation, would give artist-filmmaker the opportunity to further develop entirely new creative experiences. At the same time the art of animation would converge seemingly separate methodologies to confluence state of equilibrium between the disciplines of art, science and technology.

The result is a unique art form with a heightened sense of emotion and immersion through the element of depth.

For our research, we aim to take advantage of this digital advancement in 3D technology. The use of modern stereoscopy techniques for artistic purpose is still in its infancy and has much room for exploration and development.

3.1 COMMON STEREOSCOPIC FILTERS

The following research is formed as collaboration among faculty, students and researchers from the School of Art, Design and Media and the School of Electrical and Electronic Engineering who share an enthusiasm for painting, emotive animation and 3D stereoscopic fabrication processes.

Anaglyph- Anaglyph employs a pair of complementary colour filters for each eye, with the most common being red and cyan. The red filter will only allow red to pass through, while the cyan filter allows the transmission of blue & green. The viewable image is treated in a way that the left stream contains only red channel and the right stream contains green and blue channels.

Anaglyph is flexible and can be viewed through practically any medium that can display the spectrum of colours relatively accurately, from print to television.

However, it suffers from poor colour fidelity.

Figure 1. ‘Sadness 01-Gloom’, Digital render of S3D Art proposal , NTU’s ADM Gallery Space

1526

Proceedings of the URECA@NTU 2009-10

Figure 2. Red Cyan Anaglyph glasses Figure 4. The RealD projector and Z-Screen mount.

Polarized - In typical polarized system 2 projectors are used to play the movie, one for the left and one for the right eye. Both projection are aligned perfectly so that both images overlap. A polarizing glass is then placed in front of each projector, one angled to allow horizontal light through and the other vertical light.

Similarly the passive eye glasses have one horizontal polarized lens and one vertically polarized lens. Each being side able to allow light of that particular polarized type through to the eye.

The stereoscopic tests were created using a series of software, from Next Limit's Real Flow, to Autodesk Maya and to Adobe After Effects.

Polarized systems are the basis of modern movie projection set ups, compared to anaglyph, polarized lenses are clear and do not distort colours. Light efficiency however is reduced as an active stereo projection reduces light by about 50 percent.

4.1 ARTWORK CREATION

4 METHODOLOGY

I will briefly cover the artwork creation in Real Flow and in detail the pipeline between Maya and After Effects, focusing on the aspects of stereographing 3D content, from the virtual camera set up to the final render.

The abstract animated forms and designs are the creation of fellow URECA student Khoo Yi Hui using Next Limit's Real Flow, a software primarily designed for generating particles designed to primarily simulate realistic liquid like effects.

Active Shutter Glasses - Technically, these are LCS (liquid crystal shutters) they blind the left and right eye in sync with a display that shows left and right image alternately. [2]

In the software, particles are generated using emitters; fields are then created to influence, move and sculpt them, resulting in the creation of abstract shapes and form. Due to the erratic nature of these particles, trial and error is required to get the desired effect. A layer of polygonal mesh is generated over them to materialize its surface and shape, these are then exported out as *.bin files and imported into Autodesk Maya via a plugin.

Crosseye - Not a physical filter per say, it involves having both stream of images displayed side by side while the viewer crosses his/her eye until both images converge into one to create a stereo effect. Not practical for film as it is both difficult and tiring to cross ones eye for long amounts of time.

4.2 STEREO CAMERA SETUP AND RENDER [4] [5] [6] Maya 2009 has a readymade stereoscopic camera rig which was used for the project. The rig consists of 3 cameras, StereoCameraLeft, StereoCameraRight & StereoCameraCenter. These are connected to one another via expressions and the primary controls are located on the Center Camera.

Figure 3. Cross-Eye technique Real D system - Based on polarized stereo, the Real D system is used most of today's 3D theatres and one of the major reasons that brought about a 3D renaissance.

A single projector plays at double rate, alternating between the left and right frame. A mechanized polarized lens in front of the projector switches between clockwise and anticlockwise polarization every other frame. The advantage of Circular polarized lens allow for some degree of head tilting.

Figure 5. Basic Stereo Camera Rig in Maya The first step after creating the camera is to position it for the desired composition, while doing so scrub through the timeline to view the entire sequence. This

1527

Proceedings of the URECA@NTU 2009-10

will assist positioning of the camera correctly for the duration of the animation. After locking in the position of the camera and selecting a focal length, we go on to adjust the cameras stereo specific settings. Setting up a 4 panel layout in Maya is useful for facilitating the Stereoscopic workflow. (Figure 6)

make the object appear smaller and near while a smaller value would make the object appear larger and far.

(Figure 7)

Figure 8. Interaxial Separation 5.0 & Interaxial Separation 1.2 The Zero Parallax determines the point in depth where both image converge. In stereo terms, an object in point of the 0 parallax from the camera will have 0 depth, while anything between the camera and Zero Parallax point will have positive depth (appear to pop out from screen) while anything behind this point will have negative depth (appear to recede behind screen). A visual representation of the Zero Parallax plane and a safe volume cone (Figure 9) can be generated in the controls to assist in the setting. The optimal point judging from the stereo tests is to have the zero parallax at an average of the distance between the furthest object and the nearest object to screen as well as keep the object/s within the safe volume cone.

Figure 6. Panel Layout for Stereo workflow Topleft - stereoCameraLeft, TopRight stereoCameraRight Bottomleft - stereoCamera (Anaglyph), BottomRight Perspective In the attribute editor of the camera rig, we are presented with a wealth of options and settings. The first important task would be to set the Stereo type to off-axis (Figure 7) as it does not suffer from vertical misalignment that is seen in the traditional converged method.

Figure 9. Safe volume cone and Zero parallax plane.

During the camera setup process, several anaglyph preview renders are made and make necessary adjustments from there. It is extremely useful to create full sized previews and tested at its intended location & projection size. Unlike traditional 2D movies, the stereo effect can significantly vary depending on the size of the viewing screen to the distance the audience is sitting from it.

Figure 7. Stereo type set to Off-axis Following which, both the Interaxial Seperation & Zero Parallax settings are main key in getting the desired stereo effect. Setting the right values is crucial, it could mean the difference between good stereoscopy with convincing spatial depth or one that is painful to watch.

Once the desired camera setup is complete. In render settings the renderable camera should be set to stereoCamera (Stereo Pair), this will automatically allow Maya to render both the left and right stereo images separately. Also, add <camera> suffix to the file name so it will be named accordingly from the camera which it was rendered from.

Interaxial Separation determines the distance between 2 cameras, in real world scale it would be set to a value of 6.0-6.5 cm. This is to simulate the average distance between the human eyes. However in the case of our project, the Real Flow mesh is abstract and relative in its size. Hence, value had to be generated via trial and error by using the anaglyph preview window, in project stereotest_RF25, a value of 1.200 is found to be optimum. From our tests it is apparent that Interaxial Separation value is inversely proportional to the apparent size of the 3D object. A large value would

4.3 COMPOSITING & FINAL RENDER FOR ANAGLYPH STEREO [5] The rendered sequences from Maya of the left and right stereo camera are imported into AfterEffects as video footages and placed into their respective compositions.

These compositions are then stacked over one another in

1528

Proceedings of the URECA@NTU 2009-10

a new composition. There are 2 methods for creating the anaglyph effect, both with similar end results.

Figure 12. stereotest_RF21(12 sec)

Figure 10. AfterEffects Workspace

4.3.1 Method 1- 3D Glasses Filter 3D glasses Effect is applied to the layer on top, under its settings, match the Left View and Right View to their corresponding footage and set 3D view to Balanced Coloured Red Blue (the "Blue" is actually Cyan). While viewing with 3D glasses tweak Balance slider until there is minimal cross-talk .This technique also allows interlaced output with and a different upper and lower field.

Figure 13. stereotest_RF22 (12 sec)

4.3.2 Method 2 - Shift Channel Effect This method works by manually isolating colour channels from the Left and Right footages. On the Left footage layer, set Green and Blue channels to off and on the Right footage layer set Red to off. This will isolate the correct colours for the respective Red and Cyan filter lenses when viewing through 3D glasses. The final render settings is found to work optimally with QuickTime's Animation codec. Although the file size is large, it has very accurate colour reproduction and hence minimises ghosting and cross-talk when rendered in anaglyph.

Figure 14. stereotest_RF23 (24 sec)

An alternative method to AfterEffects is to use 3dtv.at's Stereoscopic Player to preview the Stereo movie. It allows for the input of left and right footages in various format and automatically creates a stereo solution from an expansive list of Stereoscopic filters. The viewer however needs to have the player installed to view it.

Figure 15. stereotest_RF25 (20 sec)

5 RESULTS 5.1 STEREO TESTS Figure 16. stereotest_RF26 (16 sec)

Figure 11. stereotest_RF18 (4 sec) Figure 17. stereotest_RF27 (31 sec)

1529

Proceedings of the URECA@NTU 2009-10

now take a 3Dimensional space, unrestricted by the flat screen as the shapes appear to move in and out the screen space. From our results, we can suggest that Stereoscopy is suitable not only for mainstream movies but can serve as a value added element in the visual arts. Following our Stereoscopic tests, Short film "Dreams" was remade into stereoscopic 3D with extremely positive results 5.3.

5.2 DOCUMENTATION Two PDF tutorials, containing detailed step by step instructions of stereoscopic creation from Maya to AfterEffects/Photoshop was documented as a reference/tutorial for future stereo content creators.

Being in its infancy stage, we may question the viability of 3D stereoscopy in the long run. Will it stay as one of the defining elements of film (sound, color, etc.) or is it just a novelty that will die out in the years to come?

Perhaps dĂŠjĂ vu of the 1950s? Although this is hard to answer I expect the enthusiasm and rapid development of digital technology will continue to innovate the 3D scene beyond what we have now. Although after half a century of development, viewers are still required to wear passive/active eyeglasses to effectively watch 3D. This effectively reduces the places where 3D can be shown where it is difficult to hand out or collect the glasses back to the inventory. The solution for this is to have 3D screens that do not require glasses, such technology is in development but currently are limited in its use.

Figure 18. Stereo Camera Setup in Maya - 13 pages

With the industry placing heavy emphasis on 3D technology (similar to the HD revolution a few years back) the future for 3D related content looks positive. It presents a new tool, not only for film makers but also for artists to creatively implement 3D stereo into their works.

Figure 19. Compositing Stereo Anaglyph in Photoshop and AfterEffects - 9 pages

7 CONCLUSION

5.3 STEREOSCOPIC SHORT FILM

This research has successfully accomplished the goal to create stereoscopic works as well as document the methods to create them. From camera setup, compositing to final rendered video we have produced 7 short stereo tests and a full colour 3D short film "Dreams". Having crossed the technical and artistic hurdle of producing these works, our following step is to repurpose the stereoscopic experience format in an immersive, large scale installation. Perhaps even fully interactive 3D that responds to the viewer, the possibilities are endless.

The learning and experimentation process with the Stereo Tests provided excellent research and training required for the successful conversion of abstract short film "Dreams" into stereoscopy. Directed & produced

by Asst Prof Ina Conradi, It was selected and screened Dimension 3 Festival in France, June 2nd 2010.

"Dreams" was rendered in 2 HD video files for the left and right channel and with separate a sound file. It was previewed in 3dtv.at's Stereoscopic Player prior to submission.

ACKNOWLEDGEMENT Asst Prof Ina Conradi Chavez School of Art, Design and Media Khoo Yi Hui

Figure 20. Screen capture from "Dreams", duration 04:36

School of Art, Design and Media We wish to acknowledge the funding support for this project from Nanyang Technological University under the Undergraduate Research Experience on Campus (URECA) programme.

6 DISCUSSION In general, the 3D Stereoscopic video tests have high appeal among viewers compared to their 2D counterparts, as the visual depth significantly increases the immersion experience. The forms that appear flat

1530

Proceedings of the URECA@NTU 2009-10

REFERENCES [1]

[2]

[3]

Mendiburu, Bernard. "Introduction to 3D Cinema."

3D Movie Making: Stereoscopic Digital Cinema from Script to Screen. Amsterdam: Focal/Elsevier, 2009. 7. Print.

Mendiburu, Bernard. "Tools of the Trade" 3D Movie Making: Stereoscopic Digital Cinema from Script to Screen. Amsterdam: Focal/Elsevier, 2009.

56-57. Print.

Cowen, Matt. REAL D 3D Theatrical System.

Digital image. 5 Dec. 2007. Web. 10 June 2010. <http://www.edcf.net/edcf_docs/real-d.pdf>.

[4] Keller, Eric. "Orthographic and Stereo Cameras."

Mastering Maya 2009. Hoboken, N.J.: Wiley, 2009. 92-95. Print.

[5] Stereoscopic 3D in Maya. Digital Tutors, 2009.

DVD. [6] Stereoscopic Camera Fundamentals. Aristomenis Tsirbas. The Gnomon Workshop, 2010. DVD.

1531