N G Ā T O H U | FA S T-S T-S TA TARRTT G GRRAAN NTTSS 22002200
BETWEEN THE REAL AND IMAGINARY WORLDS INTERACTIVE MIXED REALITY IN REAL TIME Dr Fang-Lue Zhang from Victoria University of Wellington Te Herenga Waka will develop a novel framework for enriched immersive experiences in Mixed Reality. Mixed Reality (MR) is a blend of the physical and digital worlds. A given MR experience can sit anywhere on the MR continuum between physical and virtual reality. Using a combination of wearable screens, cameras and physical controls, this technology provides tremendous potential for opportunities in entertainment, education and other enriched experiences. However, there are still important technological limitations to overcome. Recently, researchers have developed 360 degree video technology that allows for the capture and augmentation of entire 3D environments. To build an authentic MR experience, it is necessary to reconstruct physical scenes in a fully interactive virtual reality. However, current approaches are poorly adapted to rapidly evolving scenes containing moving objects captured from a moving camera – all features required 38
of a practical MR application. Dr Zhang proposes to use a Marsden Fund Fast-Start grant to develop a framework for scene reconstruction, which goes beyond these restrictions, allowing for MR experiences that synthesise interactive dynamic content in real time. The proposed research will bring together many strands of cutting-edge technology and innovation, such as deep learning, spherical image synthesis and MR into a single coherent project for the first time. This will inspire a cross-fertilisation of ideas and enhance New Zealand’s international reputation in computer graphics.
