Streaming digital twins into Unreal Engine

Bentley Systems has developed a new technology that streams models from its iTwin platform on demand, into the visually rich environment of Unreal Engine 5.0, writes Greg Corke

Greg Demchak of Bentley Systems demonstrating 3DFT at the Year in Infrastructure (YII) event in London Image courtesy of Bentley Systems

When it comes to real-time visualisation, Bentley Systems has followed a familiar path. Much like Autodesk, it first addressed its needs through acquisition – buying e-on software, the developer of LumenRT, in 2015. However, in recent years, it has slowly turned to the major players - Epic Games (Unreal Engine), Unity and Nvidia (Omniverse). And that’s hardly surprising. These 3D engines / visualisation platforms have massive development resources, can handle colossal datasets, and offer out of the box support for multiple platforms and devices – desktop, VR, AR and MR. Beyond files File-based workflows simply don’t cut it if you want visualisation to become an integral and truly influential part of the design process. When designs change, you have to re-import the model. In a worst-case scenario, you must re-build your entire viz asset. The workflow is just too fragmented.

On the desktop, applications including Enscape, Lumion, Twinmotion and LumenRT have benefitted from ‘live links’ to many of the major CAD / BIM authoring tools. Move a wall in Revit, Archicad, Rhino or SketchUp and, seconds later, see that change appear in the visualisation tool - on the desktop and in VR. Most importantly, models are always up to date.

Bentley is now looking to apply this optimised workflow to huge infrastructure ‘digital twins’, with a new ‘mesh export service’ that streams data from its iTwin platform on demand, into Unreal

Engine 5.0 and (in the future) Unity and Nvidia Omniverse. It’s important to note here that this iTwin service isn’t just for users of Bentley design and construction software, such as Synchro 4D, MicroStation and OpenRoads Designer. The iTwin platform can quickly bring in data from a whole host of third-party software including Revit, ‘‘ 3DFT could be key for Bentley for extending the reach of Rhino, SketchUp, Navisworks and Civil 3D. digital twin data, instantly streaming it on demand to The ‘mesh your visualisation platform of choice, all while export service’ is maintaining a single source of truth in the iTwin platform based around a new Bentley tech’’ nology called 3D fast transmission (3DFT). Rather than having to download a huge dataset in its entirety, it streams only what you need to see at any given moment. Simply reference the iTwin model, and it creates a live link, dynamically streaming the mesh on demand at different levels of detail (LoD). According to Greg Demchak, senior director, Digital

Innovation Lab, Bentley Systems, this dynamic streaming is Bentley’s unique IP. 3DFT also takes care of optimising the models automatically, as Julien Moutte, VP, Technology, Bentley Systems, explains, “You can bring very large and complex models to your users. And the performance is still very acceptable, even on mid-size hardware.”

Of course, Moutte’s definition of ‘midsize’ won’t apply in all cases. Unreal Engine 5.0 is very adept at handling large datasets, but some of the models Bentley customers deal with are colossal, terabytes in size. We can’t imagine many facilities or cities will not require GPUs that sit at the high-end, especially as render quality and resolution increases.

The graphics processing requires a local GPU. Mesh data is streamed to a local workstation, where it is temporarily cached and processed in Unreal Engine.

At the recent Bentley Year in Infrastructure (YII) event in London, Bentley demonstrated the technology with several datasets, including a water treatment plant from Jacobs, a bridge maintenance project, and a huge 500 million polygon model of the ITER Experimental Fusion Reactor in France. This fully interactive experience was derived from a snapshot of a Synchro model, showing a specific moment in the construction timeline with all the temporary scaffolding. It could be experienced in VR, using the fully immersive Meta Quest Pro, and on a powerwall, where users could teleport around the model using an Xbox controller.

The path to applications

Unreal Engine is very different to push button real time viz tools like Enscape and Twinmotion. “It’s not just a rendering engine, it’s a complete dev environment,” explains Demchak. “It’s a C++ toolkit, the UI UX, you can wire up things however you like.”

In terms of development, Bentley has so far provided a means to get huge infrastructure models into Unreal Engine very quickly, and stream any changes in near real time. However, there are no ‘products’ per se. The company is looking for customer feedback before it decides if it will build products on top of Unreal Engine. Of course, AEC firms could also take the lead, using Unreal Engine developers to create their own interactive custom apps for desktop, VR, and MR.

There’s a tonne of directions this could go, from simple collaborative design / review using an iTwin service called issue resolution, to remote bridge inspections streaming in photogrammetry meshes from iTwin Capture. Demchak explains the potential for this to simulate the real world, “You could have avatars, many people walking through the bridge. Instead of a [Microsoft] Teams meeting with 2D graphics of just a face, we can be walking together and identifying issues and talking about it.”

Bentley is also exploring ways to stream IoT data from digital twins into Unreal Engine, pulling data from physical sensors to provide a real time spatial context for the live information.

Indeed, 3DFT could be key for Bentley for extending the reach of digital twin data, instantly streaming it on demand to your visualisation platform of choice, all while maintaining a single source of truth in the iTwin platform.