Innovations 2011 by Industry Design Inc.

magazine

innovations Fall/Winter 2011

building building a better 3D experience 1

$3.99 CANADA

74470 93740

IN THE SPIRIT OF MARCONI EDGING UP TO ANIMATION’S UNCANNY VALLEY INTERACTING WITH MUSIC AT THE LEADING EDGE OF THE AEROSPACE INDUSTRY SHADOW REACHING

F/W 11

Also

and more...

IN THE SPIRIT OF MARCONI

Using cellphones as mobile relay terminals would improve reception and protect the environment.

WHO TO HELP FIRST IN A DISASTER?

Life-or-death decisions are made easier by a simulator that’s the envy of emergency operations centres everywhere.

director’s desk

innovations fall/winter 2011

Production Editor

Sharon Cavalier ICICS Administrator

Writer Craig Wilson ICICS Communication Writer Design Industry Design www.industrydesign.ca Office ICICS University of British Columbia 289-2366 Main Mall Vancouver, BC, Canada V6T 1Z4 Tel: 604-822-6894 Fax: 604-822-9013 Email

info@icics.ubc.ca

18 SHADOW REACHING

Wall-sized display screens are great for collaboration and teaching, but only with the right “chalk”.

Fall/Winter 2011

}

Sound and video clips add dimension to new compositions.

Virtual-machining software helps aircraft manufacturers get parts right the first time.

INTERACTING WITH MUSIC

AT THE LEADING EDGE OF THE AEROSPACE INDUSTRY

ICICS members are doing important and interesting work, but not enough people beyond the academic community and related industry partners know about it. With this issue of Innovations magazine (formerly the newsletter Focus), we are reaching out to the wider community with a fresh new look and a more accessible style. We know from events we have held recently that people beyond UBC, whether the general public, industry personnel, or other academics, wish to connect with us, but often don’t know where to begin. Redesigning our main publication as a magazine that appeals to the general reader and broadening its distribution base will help bridge this gap. ICICS is a forum, and we encourage interested readers at all levels to engage with us. In this issue, we highlight advances ICICS researchers are making in aircraft manufacturing, electroacoustic music, robotic surgery, 3D video, film animation, collaboration technology, and more. With over 150 researchers from departments across the campus collaborating on projects, we have an embarrassment of riches when it comes to deciding which ones to profile. We hope you will enjoy this slice.

NOVEL GUIDANCE SYSTEM FOR ROBOTIC SURGERY

Higher precision robotic surgery will mean better outcomes for cancer patients.

EDGING UP TO ANIMATION’S UNCANNY VALLEY Camera-based technique can produce a range of facial expressions not yet seen in animated films.

Panos Nasiopoulos ICICS Director

15–17

CONVERTING 2D VIDEO TO 3D / BUILDING A BETTER 3D DISPLAY

Novel glasses-free display technology and 2D to 3D conversion process could change the 3D experience.

innovations magazine

Fall/Winter 2011

When you take a call in your basement or on a mountaintop in the next few years, you may have Diomidis Michalopoulos to thank for it

In the Spirit of Marconi IN DECEMBER 1901, GUGLIELMO MARCONI TRANSMITTED THE FIRST TRANSATLANTIC RADIO SIGNAL, THE LETTER “S” IN MORSE CODE, FROM CORNWALL, ENGLAND TO SIGNAL HILL IN ST. JOHN’S, NEWFOUNDLAND. SEVENTY-THREE YEARS LATER, HIS DAUGHTER FOUNDED THE MARCONI SOCIETY TO PROMOTE AWARENESS OF MAJOR INNOVATIONS IN COMMUNICATIONS.

ecently, the society’s antennae picked up on the work of Diomidis Michalopoulos, a Killam Postdoctoral Fellow at UBC supervised by Electrical and Computer Engineering professor Robert Schober. Michalopoulos was given the society’s Young Scholar Award in 2010 for his innovations in cooperative wireless communications. Recipients are considered to have already had an impact in their field, and must be no older than 27, Marconi’s age when he made his landmark transmission. They are also seen as potential future candidates for the Marconi Award, the equivalent of the Nobel Prize in communications science. Only two other young researchers worldwide were given the award in 2010.

USING CELLPHONES AS RELAY TERMINALS In cooperative communications, relay terminals are used to forward information 4

Fall/Winter 2011

from source to destination terminals. In a cellular network, these might be simpler and consume less energy than large terminal hubs. Michalopoulos’ innovative research looks at using mobile relay terminals in the network, such as cellphones. He won the Young Scholar Award for protocols he developed for selecting relays, based on average channel conditions and specified energy consumption. A network using these protocols would work well in areas of low signal strength, and be able to re-route around obstacles. airness guides the selection of individual phones as relay terminals in Michalopoulos’ protocols; all phones involved ultimately consume equal amounts of power. By sacrificing a little, each user gains a lot. Indirectly, so does the environment: the network would operate at reduced transmission power, without the need for fixed relay stations. Michalopoulos came to UBC in 2009 from Aristotle University of Thessaloniki,

Greece. Robert Schober is happy he did. “I feel very privileged,” he says, “to have Diomidis in my group. He is a truly original thinker, and his work could move the industry in a new direction.” Earlier this year, Michalopoulos was honoured once more when the Canadian government awarded him a Banting Postdoctoral Fellowship. Named after the Canadian co-discoverer of insulin, this is a highly competitive international competition, with only 70 fellowships awarded annually, worth $70,000 per year for two years. hen you take a call in your basement or on a mountaintop in the next few years, you may have Diomidis Michalopoulos to thank for it.

For more information, contact Diomidis Michalopoulos at dio@ece.ubc.ca

Novel

Guidance System for Robotic Surgery

IN LAPAROSCOPIC SURGERY, SURGICAL TOOLS AND A CAMERA ARE INSERTED ON LONG ARMS THROUGH SMALL INCISIONS. The surgeon performs the operation looking at a monitor, with the camera controlled by an assistant. In robotic surgery, also performed through small incisions, the surgeon looks into a console and controls three or four arms with “wrists” on the end that have surgical tools and a 3D camera attached. The wrists provide many more degrees of freedom than are available in laparoscopic surgery, and the system is much more intuitive and precise. ICICS researchers are dramatically improving the guidance system of the state-of-the-art da Vinci surgical robot. Led by electrical engineering professor Tim Salcudean, they are fusing preoperative ultrasound and MRI images with ultrasound and X-ray images taken during surgery, for realtime tool guidance that will minimize tissue and nerve damage. The system will also correct for tissue and target movement and deformation during the operation. The team is focusing on prostate- and kidneycancer treatment, where minimally invasive surgery is crucial. The techniques they are developing could be applied to a number of procedures in the future, with a profound impact on healthcare—better surgical outcomes, shorter hospital stays, faster recovery times. The supplier, Intuitive Surgical, clearly thinks so; they have donated a second robot to the project that complements one purchased through an ICICS-led infrastructure grant. With da Vinci robots now at both UBC and Vancouver General Hospital, UBC is one of only three centres in the world to have two surgical robots dedicated for research and teaching. For more information, contact Tim Salcudean at tims@ece.ubc

innovations magazine

Fall/Winter 2011

robot designer or animator has yet been able to cross the uncanny valley with a creation that’s 100 percent lifelike. The Dolby Research Chair in Computer Science, Wolfgang Heidrich, may be getting close. Heidrich developed the image-processing algorithms underlying high-dynamic range (HDR) display technology invented at UBC by physicist Lorne Whitehead. The resulting display has a contrast ratio much closer to what we perceive in the real world than that of conventional displays. Brightside Technologies Inc., the spinoff company formed to commercialize HDR Technology, was acquired by Dolby Laboratories in 2007. Capturing Detail and Emotion

An array of video cameras and strobe lights captures a wide range of facial expressions.

Edging Up to Animation’s Uncanny Valley

Heidrich’s adherence to lifelike rendering of reality has guided his recent work on facial capture for film animation. Along with PhD student Derek Bradley, Heidrich has developed a novel technique for animating faces, based on capturing real faces with high-definition video cameras. The subject (or actor) sits facing a semi-circular array of paired video cameras, zoomed in to the pore level. Each stereo pair captures a detailed patch of the subject’s face. Pores, hair follicles, and blemishes are used as reference points to create stereo depth images from the two cameras’ adjacent frames, which are then rendered as a 3D mesh of the actor’s face. In traditional capture techniques, markers are worn by the actor as reference points, which is not only awkward but also produces

“Current capture systems are limited when it comes to facial animations. That’s why they mostly get used for creating aliens.”

animation based on geometry rather than detail. Heidrich’s system is markerless, uses relaWHEN CHILDREN WERE SHOWN EARLY VERSIONS tively inexpensive commercial video cameras, OF THE ANIMATED MOVIE SHREK, THEY LOVED THE and produces highly detailed animation that GREEN OGRE WITH THE HEART OF GOLD THE FILM can later be altered—virtual makeup can be WAS NAMED AFTER. applied, for example. “Current capture systems,” Heidrich points ut they started crying when his human love interout, “are limited when it comes to facial animations. That’s est Princess Fiona showed up; the animators made why they mostly get used for creating aliens.” His system, on her less lifelike, and the kids loved her too. Fiona had the other hand, with its level of detail, can capture a range of fallen into the “Uncanny Valley,” a term from robotics expressions. A library of different expressions and positions describing a narrow region where the robot is lifelike enough to could be created for a given actor, and used in different scenes. resemble a human, but with something wrong. The effect on the viewer is revulsion, perhaps because it triggers an evolutionary response related to mate selection or avoidance of disease. No

B 6

Fall/Winter 2011

Merging Capture and Animation Computer simulation is the other main approach currently used in animation, for scenes that capture is unsuitable for, such as a tsunami. Heidrich’s colleague Robert Bridson is a pioneer in physics-based animation, where simulations are guided by the laws of physics. The water scenes in Avatar, for example, were done using fluid simulation software developed by Bridson’s startup company, Exotic Matter AB. Heidrich’s long-term goal is to merge his capture techniques with Bridson’s physicsbased simulations, initially for fluids. “It would be the first systematic attempt,” he says, “to tackle tight integration of visual measurement and physical simulation.” Both Bridson’s work on fluids and Heidrich’s facial capture research are supported by the GRAND Network of Centres of Excellence led by computer scientist and ICICS member Kellogg Booth (see Focus, Spring 2010). Rendering of the eyes remains an obstacle in facial capture, because of their shine. Once that crucial hurdle is overcome, Heidrich’s system may become a key component in spanning the uncanny valley. For more information, contact Wolfgang Heidrich at heidrich@cs.ubc.ca

REALISTIC SIMULATIONS

FOR FILM

Image courtesy of Exotic Matter AB Photorealistic simulations of smoke, liquids, and clothing that obey the laws of physics can be created using software developed by computer scientist Robert Bridson. The software has been used in films such as the Harry Potter series, 10,000 BC, Hell Boy II: The Golden Army, The Dark Knight, and Inkheart. A scene from Avatar created using Bridson’s techniques, in which one of the Na’vi drinks rain water from a leaf, won the “Best Single Visual Effect of the Year” award in 2010 from the Visual Effects Society. Eventually, it may be possible to merge the capture method described in the adjacent article with these techniques to produce highly convincing simulations. Watch for an article on Bridson’s start-up company, Exotic Matter AB, in an upcoming issue.

innovations magazine

Image courtesy of Exotic Matter AB

Fall/Winter 2011

WHO TO

HELP

FIRST

IN A DISASTER

Fall/Winter 2011

DURING DISASTERS, TOUGH DECISIONS NEED TO BE MADE. IF AN EARTHQUAKE KNOCKS OUT MOST OF A CITY’S WATER SUPPLY, HOW MUCH SHOULD BE DIVERTED FROM HOUSEHOLDS TO HOSPITALS? WHAT ROADS SHOULD BE REPAIRED FIRST? WHO NEEDS ELECTRICITY THE MOST? A multidisciplinary team led by power-systems expert José Marti has developed a simulator to help infrastructure managers prepare for disasters, and make the right decisions in the midst of crisis. Disasters such as earthquakes make the linkages among critical infrastructures (power, water, transportation, communications, hospitals, etc.) painfully clear, yet coordinating responses is problematic. For business and security reasons, managers are reluctant to share their infrastructure data. They also naturally want their damaged systems to get the most attention. Marti and his colleagues have devised techniques to protect the privacy of infrastructure data while information is exchanged to enable high-level, real-time decision making. Scenarios can be run in advance to prepare for disasters, with the system learning from the results. Human factors such as where

innovations magazine

a manager’s kids go to school can be taken into consideration. These unique features have caught the attention of researchers and security agencies around the world. Marti is now the key player in an international effort involving 7 countries and 40 collaborators. Closer to home, his system was chosen from over 30 others to assist in planning and real-time decision support during the 2010 Winter Olympics in Vancouver. An Emergency Operations Centre now under construction at UBC is expected to become a global centre for infrastructure interdependencies research. For more information, contact José Marti at jrms@ece.ubc.ca

Fall/Winter 2011 99

Interacting with Music

2000, ICICS broadened membership eligibility to all researchers across the UBC campus. Bob Pritchard and Keith Hamel from the School of Music heeded the call, and were instrumental in designing the sound studio that became part of a major addition to the ICICS building. The result is a state-of-the-art facility that is one of the few studios in the world capable of supporting 64-channel sound. The two composers have used the studio extensively since, producing an impressive body of interactive works and novel music technologies. Hamel’s automated score-following work takes him to Paris regularly, where he works with the score-following team at IRCAM (Institut de Recherche et Coordination Acoustique/Musique), one of the world’s leading electroacoustic music research centres. Pritchard’s interactive works have won awards and been performed around the world.

“Strength confronts life, death, and resurrection.”

Award-Winning Interactive Piece Strength is a 12-minute work composed by Pritchard for alto sax, with interactive video and audio clips processed in real-time by the Max/MSP/Jitter software package. In the work, the camera pans up two rotating male bodies while water runs down them. 10 Fall/Winter 2011

Roses lie against various parts of their bodies, the petals of which fall to their feet at the end of the piece. These images, and the sounds of machinery, are tied together by the saxophone, in a commentary on durability, impermanence, and transformation. “Like most of my work,” Pritchard says, “Strength confronts life, death, and resurrection.” With the help of cinematographer Cathryn Robertson, Pritchard shot Strength on a shoestring budget, using available objects—a wading pool, garden hose, whiskey barrel, hockey sticks—to construct the set, then edited the footage using the ICICS Video Edit Suite. His artistic vision prevailed, and Strength was awarded a Unique Award of Merit by the Canadian Society of Cinematographers in 2007.

Putting the Performer in Charge Until fairly recently, the musician in Pritchard’s interactive pieces synchronized their performance to the audio and video clips. In his current work, technological advances such as score-following allow the performer to control the flow of the piece, which is in keeping with Pritchard’s philosophy of composition. “The music comes first,” he stresses, “then you choose the tools to get at whatever emotion is coming across.” Motion sensors are

one such tool. In a piece for alto flute he is currently composing, sounds and video images, opening the door to a singer creating the musician wears an accelerometer that looks like a watch on her her own audio-visual accompaniment. wrist. Certain gestures she makes while playing cue audio samples. Artisynth, a related project led by Fels, produces artificial Other sounds as well as video clips are triggered by Max/MSP/ speech differently, by bringing together software models of the Jitter messages embedded in the score, which the computer follows vocal anatomy—jaw, lip, tongue, vocal cords, etc.—that we use based on pitch, using Hamel’s to talk. The Artisynth model score-following software. “The “The music comes first, then you choose appears onscreen in DIVAs as performer can be as expressive an animated face, and Pritchard the tools to get at whatever emotion is expects the DIVA team to have as she wants to be,” Pritchard points out, “providing a much a version working shortly that coming across.” better musical experience for will also generate the “voice” both the performer and the produced by the gloves. With audience.” He is now looking at creating a dance piece where the sufficient processing power, the DIVA singer could control her dancer wears accelerometers on her ankles and wrists to trigger own animated, projected chorus, engaging the audience much interactive sounds and video. more effectively than a computer can. Working with electroacoustic music technology, Pritchard believes, “can inform the composer’s ability to compose acoustically. We teach it to undergraduates because it changes the way you listen to sound and manipulate it.” Bringing the Technology controlled by the performer is front and centre in humanities into ICICS can also help us change the way we think DIVAs (Digital Ventriloquized Actors), a current performance about technology. project that builds on groundbreaking speech synthesis work done at UBC by Sid Fels. Fels and his team developed a pair of For more information, contact Bob Pritchard gloves that the wearer can manipulate to control frequency-based at dr.bob@ubc.ca speech synthesis parameters and create speech. Pritchard and Fels then modified the gloves to support song and trigger additional

DIVAs

UBC Toolbox

INTERACTIVE PERFORMANCE SOFTWARE Max/MSP/Jitter is a highly sophisticated but complex program for developing interactive performance works. When Bob Pritchard was asked to teach the music technology module of an interdisciplinary course in 2003, he wanted the students, who had already done modules in creative

innovations magazine

writing, dance, video, and circuitry, to work with audio and video processing. He knew they would never be able to master Max/ MSP/Jitter sufficiently in the short time they had with him, so he teamed up with fellow ICICS member Keith Hamel (Music) and Nancy Nisbet of UBC Visual Arts to

develop a simplified set of modules, which they called UBC Toolbox. They subsequently made the program available for free on the Internet, and it is now in use around the world. Not surprisingly, Pritchard was awarded a Killam Teaching Prize in 2005.

Fall/Winter 2011 11

12 Fall/Winter 2011

AT THE LEADING EDGE OF THE

AEROSPACE INDUSTRY

Helping Aircraft Manufacturers Get It Right the First Time THE AEROSPACE INDUSTRY faces some of the world’s most physics: characteristics and behaviour of the material, tool viexacting manufacturing standards. Parts manufacturers need brations, cutting mechanics, temperature, kinematics, etc. The to be certified by an international association. Detailed records software packages he has developed allow engineers to test the must be kept about a part’s history, so that malfunctions can be interaction between materials and cutting tools before machintraced to the machine it was made on, the cutting tool used, and ing takes place, and make adjustments to maximize productivithe operator involved. Materials must be light and strong, and ty. High-precision parts can be made right, at the optimal speed, can be very expensive—the raw material for a titanium part, for the first time. instance, can cost hundreds of thousands of dollars. There is no room for error, and the competition UBC Expertise Flies High is fierce. “Roughly eighty percent of the Mechanical engineering profesAltintas’ Manufacturing Automation sor Yusuf Altintas has a long hisLaboratory (MAL) at UBC is considworld’s aerospace companies tory of meeting these challenges. ered the best virtual-machining cenuse our algorithms.” In the early 1980s, he developed tre in the world. CutPro, a package software for automating the manuof science-based virtual-machining facture of propellers for the highly algorithms developed at the MAL, successful de Havilland Dash 8. Since then, he has become one has been licensed by over 130 companies and research centres of the world’s foremost experts in virtual machining, or “mill- worldwide. In recent years, Altintas’ methods have been used ing” parts in computer simulations that factor in the relevant by Boeing, Airbus, and Bombardier to cut aluminum wing innovations magazine

Fall/Winter 2011 13

panels; ASCO Aerospace Canada Ltd. to mill titanium flap and slat tracks for a number of different aircraft; Pratt & Whitney Canada to machine business-jet engine impellers; and Hitachi to manufacture hydro turbines. Altintas has held the NSERCPratt & Whitney Canada Chair in Virtual High-Performance Machining since 2002.

Committed to Research “Roughly eighty percent of the world’s aerospace companies use our algorithms,” Altintas says. So do many other industries. They often ask Altintas to consult on specific manufacturing problems, but he usually turns them down. Instead, he turns the problem over to one of his graduate students as a fundamental research question. In exchange for advising the company on the use of his software, Altintas receives equipment and material for the MAL. The machine-tool company Mori Seiki, for instance, donated two machining centres worth $800,000 that Altintas and his students use for testing their algorithms. Top

Keeping Canadian Manufacturing Competitive In his spare time, Yusuf Altintas is Scientific Director of a Natural Sciences and Engineering Research Council of Canada (NSERC) Strategic Network that is developing advanced, science-based virtual machining technology for the Canadian automotive, aerospace, machinery, energy, biomedical, and die-mold industries. 14 Fall/Winter 2011

cutting-tool companies such as Sandvik Coromant (Sweden), Kennametal (USA), and Mitsubishi Materials (Japan) also use his technology, and provide expensive tools in return. “I am very committed to fundamental research,” Altintas asserts, “which in my case embodies teaching graduate students, with the problems being defined by real-world scenarios. Teaching and research are a professor’s primary job.” Altintas’ heart has always been in academia. He turned down a job offer from General Motors Canada to come to UBC in 1986 because, he joked, the fishing was better in BC. A number of years later he confessed to the GM executive who tried to hire him that he’d only caught half a dozen trout since coming to BC. The man did some quick calculations and informed Altintas that those fish had cost him several hundred thousand dollars. His students and the aerospace industry would say they were worth it.

With funding over 5 years of $5 million from NSERC and $350,000 from industry, the Canadian Network for Research and Innovation in Machining Technology (CANRIMT) is made up of 20 academic researchers from 7 universities across Canada, as well as researchers from the National Research Council’s Aerospace Manufacturing Technology Centre and industrial partners Pratt & Whitney Canada, Bombardier Aerospace, ASCO Aerospace Canada Ltd., Automation Tooling Systems Inc., Memex Automation Inc., Origin International Inc., and Promation Engineering Ltd. Altintas

For more information, contact Yusuf Altintas at altintas@mech.ubc.ca

and his colleagues have involved their industrial partners from the start, to keep the research on track. The software tools they develop will be easily integrated into existing CAD/CAM systems, so that parts manufacturing can be optimized virtually without costly shop-floor trials. The end goal is to keep the Canadian manufacturing sector competitive.

3D:

CONVERTING

2D VIDEO TO

Taking Cues from Nature 3D TVs, cellphones, games boxes, and movie theatres are becoming more and more common, but 3D content creation is lagging behind, since it is so difficult and expensive. Some 3D TVs currently on the market can convert 2D content to 3D, but with limited success. A new conversion technique invented in the UBC 3D Innovation Lab, however, is extremely promising, since it mimics nature. The range of cues humans draw upon to build up a 3D view of the environment, and the relationship of cues to one another, is extracted by the system from a 2D video or image to create its 3D counterpart. Those that are inappropriate are rejected. The end result is a highly accurate 3D version of a 2D original. The

innovations magazine

conversion will happen in real-time on a TV, PVR, cellphone, or any other display device where a 3D view is desirable. Navigating using a 3D version of Google maps would be much more intuitive, and who doesnâ&#x20AC;&#x2122;t want to see Eartha Kitt as a 3D Catwoman in the 1960s Batman series? In related work, the researchers are developing a host of techniques that will allow the viewer to comfortably sense depth and benefit from the entire colour gamut, without having to wear glasses. We will feature this work in an upcoming issue. For more information, contact Panos Nasiopoulos at panos@icics.ubc.ca

Fall/Winter 2011 15

building a better 3D display EVOLUTION HAS EQUIPPED US WITH TWO EYES FOR REDUNDANCY, BUT ALSO FOR DEPTH PERCEPTION, SO WE CAN SUCCESSFULLY NEGOTIATE THE WORLD.

wo separate views of a scene are projected onto our retinae, offset slightly because of the distance between our eyes, then fused by the brain to help us perceive depth. Perspective lines, shadows, blocking of background objects by foreground objects, and motionparallax cues, where objects in the foreground seem to shift faster than those in the background as we move our head from side to side, are also important cues. Since about 1860, we have been

16 Fall/Winter 2011

able to mimic this aspect of human vision by looking at two offset photographs of the same scene through a prism stereoscopeâ&#x20AC;&#x201D;with enough staring, they converge into a three-dimensional image. By the mid-1950s, colour 3D movies were the craze. Two images of the same movie were projected onto the screen through polarizing filters. Audience members wore cheap glasses with polarizing filters that let light through according to the image appropriate for

each eye. Telescopes and other props that swung out at the audience were common features of these B movies; character development and plot were a little thin. With recent movies such as Avatar and Clash of the Titans (see animation article on Pages 6â&#x20AC;&#x201C;7), 3D has moved into the mainstream.The audience still has to wear glasses, and much of the content is computer-generated, since 3D filming of real actors and scenes is so difficult, requiring a number of

different cameras pixels for up to 30 viewpoints, Stoeber and his team are do-itshooting simulta- thereby avoiding the loss of resoluneously. For the tion inherent in autostereoscopic yourselfers, and will manufacture same reason, 3D displays. Stoeber, who holds a joint many of the display’s novel comtelevisions already appointment in Mechanical Engi- ponents at UBC, in the Advanced on the market suf- neering and Electrical and Compu- Materials and Process Engineering fer from a dearth of content beyond ter Engineering (ECE), will design Laboratory (AMPEL). systems “Combining MEMS technology video games, which are computer- microelectromechanical (MEMS) that house optical elements with the right optics components, generated. Viewers still need to wear glasses, with lenses wirelessly syn- and rotate rapidly back and forth to electronics, and rendering apchronized to two overlapping images project multiple views of the same proaches,” Stoeber says, “will lead to the next generation of 3D dison the screen, so the correct plays.” Since MEMS-based 3D eye sees the corresponding displays could be mass proimage when it should. Glass“Combining MEMS technology duced, Stoeber believes they es are not required with new with the right optics components, will one day also be used in autostereoscopic designs; biomedical applications such electronics, and rendering however, these displays have as computed tomography low resolution, since pairs approaches will lead to the next (CT) and image-guided surof pixels are required to disgeneration of 3D displays.” gery, as well as in cell phones play each set of offset imand other handheld devices. ages. They also have a limited Filming 3D content for these number of 3D viewpoints, displays would also be much so motion-parallax cues are scene. System-on-a-chip expert easier, since it would require only of minimal use. Shahriar Mirabbasi (ECE) will de- two cameras running simultaneHIGH-RESOLUTION 3D velop advanced micro-circuitry to ously. generate the voltage necessary to roWITHOUT GLASSES tate the MEMS platforms. CompuFor more information, contact Boris Stoeber at team of researchers led ter graphics specialist Sid Fels (ECE) will focus on rendering the images, stoeber@mech.ubc.ca by Boris Stoeber are drawing on their com- using techniques he developed in a bined expertise to ad- previous project that allow informadress these problems. The crux of tion for multiple viewpoints to be their approach is to use individual delivered in a single frame of video.

ICICS/TELUS

People & Planet Friendly Home Initiative An ICICS/TELUS consortium that promotes sustainability while maintaining and improving quality of life. Read more about this unique project in the next issue of Innovations.

innovations magazine

Fall/Winter 2011 17

SHADOW REACHING

THE EARLY NINETEENTH CENTURY, BLACKBOARDS BEGAN REPLACING SLATES IN THE CLASSROOM, AND PUBLIC EDUCATION BECAME A MUCH MORE SOCIAL, COLLABORATIVE EXPERIENCE, MEDIATED BY CHALK IN THE TEACHER’S HAND. WALL-SIZED DISPLAY SCREENS NOW IN LIMITED USE HAVE THE POTENTIAL TO SIMILARLY REVOLUTIONIZE COLLABORATION, WHETHER IN THE CLASSROOM OR A UTILITIES CONTROL ROOM. WHAT’S MISSING IS THE CHALK.

raditional means of interacting with computers by keyboard and mouse are not practical for largescreen displays, which are designed for collaborative work. Information needs to be graphically displayed and manipulated on the screen in a way that makes sense to the audience. With touch screens as large as the 5 m x 3 m display in ICICS’ Interactive Workroom, a step ladder would be needed to reach all areas of the screen. Because these screens have been deployed without a suitable means of interaction, they have remained largely in the realm of research. Garth Shoemaker was Director of Research at Idelix Software, a developer of mapping software, before coming to UBC to pursue a PhD in human–computer interaction.

18 Fall/Winter 2011

His supervisor, computer scientist Kellogg Booth, is an internationally recognized expert in the field. Under Booth’s direction, Shoemaker has applied his practical experience to the missing “chalk” problem. His working principle is that for interaction with large screens to be useful, the audience needs to be able to connect it to the user—to see the user’s hand draw a line around an island on a map, for example. Shoemaker realized that if the user’s shadow were projected onto the screen, the closer they got to the apparent light source, the larger their shadow would become; with it, they could reach all areas of the screen. Having had his eureka moment, he set himself the task of generating a virtual shadow of the user that could interact with the screen.

Collaboration Technology for Large Interactive Wall Screens “This is a recurring theme in my work,” Shoemaker says, FUTURE TRACKING “drawing from the real world and leveraging that in the virtual world.” After consulting with Booth, virtual-reality expert Sid SHOEMAKER BELIEVES THAT TRACKING WILL ONE Fels, computer-animation specialist Michiel van de Panne, DAY BE DONE exclusively with cameras, so the user will not psychologist Jocelyn Keillor (National Research Council), need to wear tracking markers, nor use an input device like a and other psychology and kinesiology researchers, he devised Wiimote. An instructor in a lecture hall, for example, will be a technique for rendering a virtual shadow of the user. Using tracked by depth-sensing cameras and rendered on a large wall visual and magnetic trackers and a virtual light source, the screen. They will interact with it by simply pointing to the area shadow is “cast” onto the of the screen where they screen as if by a real light. want certain information Modified Nintendo Wiimote “This is a recurring theme in my work, to appear, then touching controllers function as input their finger to their thumb drawing from the real world and devices. Shoemaker calls as a “click”. Static, sequential leveraging that in the virtual world.” the technology “Shadow PowerPoint presentations Reaching.” may go the way of the slate. Another advantage of Shoemaker’s research rendering users as virtual has been supported by shadows is that screen detail can be seen through them. “It’s a SMART Technologies of Calgary, manufacturers of interactive delicate balance,” Shoemaker points out. “You want something whiteboards, and by Defence Research and Development that provides the information but doesn’t get in the way.” Icons Canada. It is also part of the on-going research program of the for different tools, such as a virtual pen, are located in various new GRAND Network of Centres of Excellence led by Booth places on the shadow. Data are stored in the shadow’s “stomach,” (see Focus, Spring 2010). Some form of it may be coming soon and shared by passing folders to other users’ shadows. By using to a classroom or utilities control centre near you. more comprehensive tracking techniques, Shoemaker can generate a much richer three-dimensional body model that can For more information, be rendered on the screen in a variety of ways. Collaborators contact Garth Shoemaker at garths@cs.ubc.ca in different cities, for example, may be identified by rendering them in a more lifelike, recognizable way.

innovations magazine

Fall/Winter 2011 19

Bring your friends list to the TV screen.

Introducing Facebook on Optik TV. A first in Canada, only from TELUS. To learn more about Facebook on Optik TV™ and for details about our latest offers visit telus.com/optiktv

UBC’s Institute for Computing, Information and Cognitive Systems (ICICS) is an umbrella organization that promotes collaboration among researchers from the faculties of Applied Science, Arts, Commerce, Education, Forestry, Medicine, and Science, and with industry. ICICS facilitates the collaborative multidisciplinary research of approximately 150 faculty members and 800 graduate students in these faculties. Our members attract approximately $18 million annually in grants and contracts. Their work strengthens Canada’s strategic Science and Technology research priority areas, benefiting all Canadians.

www.icics.ubc.ca

PUBLICATIONS MAIL AGREEMENT NO. 40049168 RETURN UNDELIVERABLE CANADIAN ADDRESSES TO: ICICS, University of British Columbia 289-2366 Main Mall Vancouver, BC V6T 1Z4 info@icics.ubc.ca