HxDR: reality capture

HxDR for reality capture

In 2005 Leica Geosystems, maker of laser scanners and surveying devices, was acquired by Hexagon AB. In 2010, Leica launched the revolutionary BLK360 small format scanner. Since then, it has steadily fleshed out a complete BLK ecosystem. Martyn Day visited the company at its HQ in Zurich, Switzerland, to learn about the latest developments

Brands (and people’s perception of them) are interesting things. Historically, Leica Geosystems, being Swiss, would probably fall into my stereotype of high-value engineering, coming at a premium cost. But, given the company introduced the BLK360 laser scanner in 2018 for £15k, bundled with Autodesk’s Recap, this would be wrong.

Leica’s offering was aggressively priced. At the time, its smallest and lowest cost competitor, the Faro Focus3D, came in around £22k. It seemed that Leica was simply testing the market, but with hindsight, that assumption seems to have been completely incorrect!

Leica has gone on to continue to deliver new form factors for the scanning market, building up a range of innovative and beautifully designed scanning products. BLK2GO is a handheld portable scanner, which utilises SLAM (simultaneous localisation and mapping) technology to capture interior and exterior laser scans through the simple act of walking around.

BLKFLY is a drone variant of the BLK2GO – but, in saying that, it completely underestimates the engineering work that has gone into its design (see box out for more info).

Next to be released was the BLKARC, which again leans on the BLK2GO design but is a module for autonomous reality capture, initially destined for Boston Dynamics’ Spot robot.

In just a few short years, Leica has a fully productised its range of portable scanners, which can be carried by hand, flown and even by robot. They all share the same beautiful industrial design and all feed into the company’s Cyclone 3D point cloud processing software, although more and more of the processing (cleaning up, registration) is becoming automated on device. This is a long way from historical tripod-based scanning strategies which take way longer to capture and process. However, the larger nonSLAM scanners still benefit from range and accuracy.

With a now-obvious product evolution plan moving to mobile scanning, the next part of this, seems to be to connect the Leica data capture devices to a central cloud backbone. For this, Leica has developed a cloud-based, WebGL-based service that is called HxDR.

The HxDR platform

Launched in January 2020, the concept of HxDR is to connect the sensor hardware directly to the cloud. Whilst in the cloud, all the processing is carried out, all the holes within the point cloud are filled up to get coherent mesh, and then brought back to the user through a thin client, such as web browser, so the user can view the data, perform measurements, etc. There’s no human intervention; it’s all automatic. As part of the processing, HDxR subdivides and multi-levels the data automatically. Similar to game engine technology, on initial load, you get a low-

2

fidelity version, and then, over time, more and more detail is streamed in. The app never requires the user to fully download the entire dataset, just for viewing purposes, although if you do want to, there is a download button for the assets.

All rendering is done on the client side, not on the cloud server, but it’s optimised for fairly standard laptop graphics. The more video memory your system has, then the more of the data it can cache, smoothing movement.

As HxDR also supports terrestrial scans, not just point clouds and the automated meshing, it also supports panoramic images / viewpoints. The cool thing about using viewpoints is that while they are an image, they still use the 3D data in the background, enabling accurate measurement. Measurements can get stored so other team members can see persistent areas and lengths than have been investigated.

It’s also possible to add 3D annotations inside HxDR and this can be shared inside an asset with multiple collaborators logged into the system. Flying around the model, you can carry out dynamic ‘tours’, which can be shared with someone, or recorded as an Mp4 video. If your scan has an exterior and an interior scan, it’s possible to remove the outer skin and instantly get a ‘doll’s house view’ of the interior space.

HxDR is not just a viewing and markup tool for point clouds / meshes. It’s also a platform. Leica allows users to create custom applications using the power of the HxDR platform. In the example we were shown, a scan from a BLKARC (hosted on the back of the Spot robot), was merged with the CAD model for the office floor, putting the two next to each other to check for differences. This brings the processing close to the dataset, which means you can have your BLKARC running around a construction zone, for instance, and ‘change detection’ can be done automatically in the cloud, with the results being sent directly to the client. The intention of the HxDR platform is to enable any group, both within as well as outside of Hexagon, to deliver their technology and embed that into a more powerful pipeline

HxDR isn’t just for the BLK series of services but for 3D mapping data from all sorts of inputs. Airborne imagery, laser scans, indoor and outdoor terrestrial scan data, as well as mobile mapping data gets seamlessly combined using HxDR.

Users drag and drop their reality capture files to upload them into HxDR, and then the automated meshing functions process the point clouds. This is an extremely significant development, connecting users on site who are scanning, with an ever-present cloud-based processing and display engine.

Utilising handheld SLAM scanners with airborne drone SLAM scanners, data capture will see huge productivity gains. If the processing moves from expert users on desktop workstations, to using the cloud to automatically clean up and register, you’re talking orders of magnitude in productivity savings. Then there’s the additional benefit of the centrally-hosted model which can be viewed and shared by everyone in the project.

From our conversations with architects, who have one major project and need to start with a survey of the existing building or infrastructure, this can take weeks and at significant cost. Theoretically HxDR could provide the scan data to the architect on the same day.

Extending this to technologies not yet perfected, but in development from a number of firms, including Hexagon, the elusive Scan-to-BIM automation could then be the next part of this process. By

1 The BLKARC module: autonomous reality capture with Boston Dynamics’ Spot robot 2 An HxDR supermesh, a combination of data from multiple different sensors, all blended together

using machine learning to identify architectural features in the mesh, such as walls, doors, windows, ceilings, floors, stairs etc. and to build an IFC version from the mesh. I don’t think this is far away - perhaps in the next few years.

Hexagon Content Program

HxDR is also a lot more than automated Cyclone in the cloud. Leica has been busy building a library of scanned cities and landscapes, which customers can license (2D and 3D data), a bit like Ordnance Survey Maps. So far Leica has over 3.6 petabytes of towns, cities and landscapes. This can be used for modelling in context or to drive smart city platforms for urban planning. The quality of the city scans has to be seen to be believed - it’s really excellent, especially considering the amount of reflective surfaces in the city, which is a notoriously unfriendly material for light-based scanners.

While the standard settings are impressive, HxDR is capable of what Leica calls a ‘supermesh’. But what makes supermesh, a supermesh? According to Leica, it’s the combination of data from multiple different sensors, all blended together and rendered on the local laptop. From a bird’s eye view, the models initially look the same, but as you zoom in the detail is incredible; you can see all the cars, details in the facades, and there’s 3D depth to tunnels / parking access.

The creation of supermeshes is also 100% automatic if you have the data, automating the blending between rooves and the facades. The supermesh model of Munich we were shown took up 360 GB on the HxDR system.

Once you have a HxDR city and you are an architect working on a project, you might want to bring in your model to see it in supermesh context. That is also possible. With the excellent fidelity of the city model, you can look at your design from any angle, whether that be street views or purely aerial. With materials in the IFC, the system also produces an environment map to match the shading to the lighting of the city capture. It’s also possible to be inside the proposed building and look out from any window and see what actually would be seen from that view.

In the final demonstration, Leica showed an IFC model of a factory, in a supermesh of the city of Delft in The Netherlands. Not only did you have access to the BIM data, but it was also possible to add animations - in this case a Spot robot walking around the site performing a scan with the BLKARC. But that robot could really be at the factory and what HxDR would be showing would be live data from the IoT sensor on Spot. So, Leica sees HxDR as also a potential platform for dashboarding for digital twin management of sites.

Burkhard Boeckem, CTO of Hexagon explained the evolution of HxDR, “Development started in 2017 with some ideas. By 2018 we set out to build it, and the first version was shown at CES in 2020. There are basically four business tracks to HxDR. One is the content programme, where customers can actually buy parcels with a credit card. Then we have the Capture model, where customers upload, mesh, collaborate and do things with others. Then there’s the what I would call ‘white label’ business, where companies use HxDR as their platform for building their applications - then it’s maybe ‘powered by HDR’. And, finally, where Hexagon builds its own cloud solutions.

“You will see a lot of new products coming out, that have a nice cloud architecture because we think that HxDR has a fantastic cloud architecture. It’s scalable, we’re in close cooperation with Amazon. That’s why we built a corporate platform, so every Hexagon division can benefit.”

Conclusion

Leica’s BLK and HxDR developments have been a multi-year journey to what now is a coherent ecosystem – LiDAR data capture any way you want it, automated pipelines to enable one button registration and clean up - all centred on a cloud service which enables hosting, collaboration and city model licensing, with the digital twin market being lined up in its sights. This is the product of joined up thinking.

Boeckem was the CTO of Leica Geosystems for twenty years but, in 2020 was elevated to CTO of the whole of Hexagon, which is a huge $14.7 billion market cap company with many, many brands. In AEC, it owns firms like Leica, Bricsys, and Intergraph, and has always appeared more of a holding company than a known brand. I asked him what his goal was in his new role and he said ‘convergence’. With so many Hexagon brands’ CTOs reporting in to him, it’s important to not keep reinventing the wheel, where all divisions can benefit.

HxDR is a case in point - a platform for all Hexagon firms to build on. I fully expect to see many more of these kinds of products come out of Hexagon.

■ www.leica-geosystems.com

BLK2FLY: the design of Leica’s autonomous flying laser scanner

The design of Leica’s drone is really something else. To the uninitiated it looks like a BLK2GO with a frame and rotors on it but that belies the huge amount of engineering that has gone into the design, materials and manufacture.

The vast majority of scanning firms, looking to get into drones, would just license or build on top of an existing drone chassis - because what’s the point of reinventing the wheel? It would save a huge amount on development and would mean being able to get to the market quickly. This is not the Leica way.

The company set up a team, hiring from the local Zurich talent pool, which is rich in drone technology developers, and they set about the challenge of creating a flying BLK2GO. Obviously, it’s an expensive scanner and so it had to be reliable and strong, yet light to provide decent flight time. The frame is made of a special carbon material, which is woven, embedding the drone’s essential sensors. Using Finite Element Analysis (FEA) software to explore stresses and strains in the design, material is added in the production process in areas that need it most. It’s a highly optimised design and using cutting edge manufacturing methods.

Lenovo ThinkPad P16

The new ThinkPad P16 packs the high-end performance of a 17-inch mobile workstation into a compact 16-inch form factor, powered by new 12th Generation Intel Core HX processors

he ThinkPad P16 is one of the most

Tambitious mobile workstations to come out of Lenovo in recent years. It marks the convergence of two different form factors – the 15-inch and 17-inch – into an innovative high performance 16-inch chassis.

But it’s not just its size that’s different. With a brand new industrial design, the ThinkPad P16 brings Lenovo’s legendary mobile workstation family bang up to date with a sleek, modern aesthetic and premium materials, including an anodised aluminium ‘storm grey’ cover. With a 16:10 aspect ratio on the display, which includes a 4K OLED option, and thin bezels on each edge, the chassis is equivalent in size to a more traditional 15.6-inch laptop.

Optimised performance

The high-end mobile workstation is designed for the most demanding professionals, including product designers, architects, engineers, and visualisation specialists.

Performance is up front and centre. Powered by the new 12th Generation Intel Core HX processors, the ThinkPad P16 delivers processing performance that can rival desktop workstations.

With up to eight ‘Performance’ cores and eight ‘Efficient’ cores, for a total of 16 physical cores and 24 threads, the processor excels in both single-threaded workflows, such as CAD and Building Information Modelling (BIM), and multithreaded workflows, such as engineering simulation, ray trace rendering and reality modelling.

In addition, the ThinkPad P16 offers a choice of high-performance professional GPUs up to and including the NVIDIA RTX A5500 (16 GB) for the most demanding real-time visualisation and VR workflows.

Other specs include up to 128 GB of DDR5 memory, up to two high performance NVMe SSDs and the very latest in wireless connectivity including Intel WiFi 6E AX211 and 4G.

Cool operator

Despite offering similar core specifications, not all mobile workstations are the same. Due to cooling constraints, processors can be fed different levels of power and can sometimes be ‘throttled’ to stop them getting too hot. This is not the case with the ThinkPad P16.

Thanks to its innovative dual vapour chamber, the powerful mobile workstation excels in its thermal design, delivering exceptional performance while running cool under heavy loads.

Learn how the ThinkPad P16 was designed tinyurl.com/ThinkPadP16