Esri ArcGIS Reality for reality modelling

It’s nothing new to be able to model building sites, cities and countries. However, such activity traditionally requires substantial technical skill, special software, and a significant amount of data wrangling. Now, the availability of rapidly developing reality modelling tools that leverage LiDAR scans and photogrammetry techniques has drastically lowered the cost of producing large-scale models automatically. In turn, this has created an influx of demand for applications that are easy and efficient to use, prompting companies and governments to look for ways to digitise their infrastructure, however big.

Typically, the reality capture models might look stunning but are very dumb. Intelligent data needs to be added, combined with building information and geo - spatial data, such as cadastral surveys. Here the industry’s deep-seated problems with data silos just add to the level of complexity.

ArcGIS Reality is Esri’s new play to offer a tool where reality capture 3D data, BIM and GIS information can be accurately layered, and enable functions such as analysis, asset monitoring and predictive simulation.

Esri is used to handling huge data sets and tiled information with its ArcGIS application. The company is certainly ambitious, as it believes that the ArcGIS platform, when mixed with BIM and reality capture models, and when connected to real-time sensor feeds, will become ‘digital twins of reality’, from buildings and utilities in real-word context, to trainlines and motorways at a country scale.

Questioning reality

AEC Magazine caught up with Esri’s Kurt Schwoppe, sales manager for Imagery and Remote Sensing and Marc Goldman, director of AEC Industry Solutions at Esri, to talk about the new product and the company’s interest in the built-environment space.

We asked what technology Esri was using to build its reality models. “Esri has had a drone solution for a couple of years but recently we bought a company from Germany called nframes, which brought the ability to do high-end processing across city wide, and even country size, areas, dealing with thousands and thousands of images, multiple terabytes of data,” explained Schwoppe. “It’s opened the door for us, so we can run that full spectrum of remotely sensed content to generate digital twins, or at least the visualisation component of digital twins (I know that digital twins are more than just a 3D visualisation).

“This is where we start with reality mapping, enabling organisations that have facilities or assets which were built before the days of 3D CAD to go out and do remote sensing collection with drones or aircraft and bring it into our ArcGIS reality software.”

Format

What is the company’s attitude to files? “3D models are now an inherent part of the ArcGIS platform,” explained Schwoppe. “It’s been a significant focus of ours, and we now support many different industry standard formats. And we have an approved, open format called I3S (Indexed 3D Scene Layer Specification) which we helped specify. It’s a 3D standard which is specifically designed to package up and optimise large data models for streaming over the web or locally. It has support for 3D point, 3D object and 3D mesh layers.

“In the geospatial industry, there’s a there’s a group called the Open Geospatial Consortium (OGC) and they produce open standards for the GIS community. We worked with them to create an open 3D modelling format (i3S) which anybody can use or take advantage of. It’s what all our 3D models get published into.

“We take the raw sources, the drone/ aircraft captures, and push them into these i3S models. Of course, if we need to, we can also write these out in more common formats like OBJ etc. but Autodesk and Esri have embraced this format. We can bring this fusion of CAD with the remotely sensed data, to create a digital twin.

Use cases

Schwoppe thinks that Esri’s ‘special sauce’ is its ability to combine 3D captured data seamlessly with its complete range of 2D GIS capabilities and now this can be greatly expanded.

One of Esri’s flagship products is ArcGIS Urban, which enables cities to be modelled, but up till now had very simplistic 3D (extruded polylines). With the new reality capture technology, customers can now replace the low-poly model with a digital twin of any city. This means digital twins can be used in resolving urban mapping issues, for visualisation requirements or regional planning. City data that looks like the actual city can be combined with traditional GIS data, such as traffic lights, crosswalks, fire hydrants etc.

In oil and gas, Esri has customers that have extensive facilities together with long distance pipelines which might span miles or countries. ArcGIS Reality enables the assets to be modelled in 3D, spanning the whole length of the pipeline, including plant, pumping stations, encroachment areas and GIS mapping data over huge areas.

One of Esri’s customers is the US Corps of Engineers, and they are working on a project in the Houston, Texas area called ‘Texas Coast’. Schwoppe explained, “It will be the largest civil works project in the history of the United States, in real dollars. Even larger than the Panama Canal, as they must build a series of dykes and locks to protect the coast of Houston, as it’s critical for us and our oil industry. The Corps need to 3D map the terrain of that whole area before they build the dykes. We can pull that all together in a single solution.”

Digital Twin

When is a digital twin not a digital twin?

It’s a perplexing issue. A dumb 3D model of a whole city, with no smart information is just a model, even if it’s a seriously cool model.

At AEC Magazine we conclude that one of the essential components of a digital twin solution is the ability to connect it to live sensors to bring the model to life, to help make those important facilities related decisions.

We asked Esri how far they had got with connecting up their models and how their tool would compete with Autodesk Tandem. Marc Goldman responded, “We’re connecting to SCADA (Supervisory Control & Data Acquisition) sensors and IoT sensors. We’re doing heat maps of spaces and spatial analysis.

“There is an opportunity to add value to Tandem, and I’ve worked on both sides of the fence (I’m an [Autodesk] fanboy. But it’s still a building floating in space; there’s no context. You don’t see the street, you don’t see the streetlights, you don’t see the utilities underground, you can’t just turn on a layer and see the traffic flow or the weather patterns. So, imagine all the model capabilities of Tandem, with all the context and layers that Esri GIS can bring in.

“Another important data class in digital twins is laser scan data to capture the asbuilt and compare that to the as-designed, as Goldman explained, “Point clouds in GIS environments tend to be very ‘marshmallowy’, the corners tend to be rounded over and small details like a streetlamp would kind of be a blob. With this new technology, we’re now down to a level of accuracy that someone would really have to think twice about what they were looking at. Is that a 3D mesh? A point? Cloud? A photo? A NeRF? It’s all coming together in ArcGIS Reality.”

Schwoppe added, “If it’s collected correctly and the surveyors dropped the right points, ArcGIS will display 3D geospatial data to sub centimetre accuracy.”

Scanning

A lot of firms that have come into this space have the technology and platform but don’t have the datasets of cities or countries. I asked the team on where the content is coming from.

Goldman explained, “We, Esri, don’t go out and collect data, we might take sensor data, and then our cartographers turn it into a format such that it can be a layer easily added. But our content play tends to be more of a partner play than us ourselves.

Schwoppe added, “Esri has a product called ArcGIS Living Atlas which is a collection of GIS data from around the world, supplying maps, apps and data layers for climate change analysis, farming information, flood prevention, land use, even Covid trends and other natural disasters.

“We are building our own 3D dataset of the world but it’s not at an infinite level of detail. It’s a start.

“Any engineering firm with a drone can go out and get this level of accuracy and visualisation and we can feed that into the GIS. We also have third partners, including Hexagon, which has one of the best cameras nearmap, which we support. Customers can commission a company to go out and fly a route to collect specific data and we can help by providing standards on how they should collect.

“We can also work with you and our partners to go out and collect an area of interest. We also have partners who just collect different areas on speculation, like nearmap. Their 3D models work within our environment directly. We want to provide a software solution that gives the

1 City model of Utrecht, Holland, combining reality capture city data, active construction site data and the architect’s BIM model. Construction progress can be checked against the design and planning data

2 GIS meets BIM and reality capture data to track construction site progress customer the most amount of flexibility.

“We can work with just about any drone or drone camera that’s out there. We can work with LIDAR data, provided it’s in our supported list of file formats. And we can work with any type of aerial camera, whether that’s on our drone, an aircraft, or a satellite.”

ArcGIS Reality doesn’t come with technology to perform clean-up of imported laser scans. Schwoppe and Goldman explained that the best place for that to happen is in the specialist applications which come with the laser scanning systems like Leica Cloudworx, but they do offer tools to clean up errant meshes.

Conclusion

Esri is the long reigning champion of the GIS mapping world. Ever since Esri president and co-founder Jack Dangermond and Autodesk CEO Andrew Anagnost announced a development partnership at Autodesk University 2017, the company has actively pursued connection and convergence with the rest of the 3D AEC world. This was a significant change of mindset. At the time, Dangermond explained, “You cannot design in a vacuum. You cannot build in a vacuum. You cannot operate and manage infrastructure and assets in a vacuum.” Esri sees CAD and GIS as essential tools in managing the

ArcGIS Reality family

There are several distinct products in the ArcGIS Reality family. Some are extensions to already established GIS products, others are new tools aimed at data formatting, editing and production of Reality data.

built environment. In many respects, with Esri’s strong penetration of the planning and city-facing side of things, its focus on combining all datasets in 3D should hopefully drive 3D tools and the understanding of 3D data into planning permission and permit departments, which have resisted model-centric approaches, and BIM data (buildings and civil).

The reality modelling world is growing. We have established players like Bentley Systems, Vu.city and recent entrants from Leica Geosystems (Hexagon) whose highly-capable HxDR service combines capture and an online data store with web-based, highly detailed textured

ArcGIS Reality for ArcGIS Pro is the extension to connect Esri’s desktop GIS software, to pull in images from autonomous drones to generate 3D output for reality mapping.

ArcGIS Reality Studio is a whole new application for collating meshes as well as BIM model data. Every firm seems to be taking slightly different approaches which is handy, depending on the needs of individual companies. Esri brings this GIS part of the jigsaw to the table and with strong integrations with Autodesk will look to take a slice from the emerging digital twin market, at all scales. ■ www.esri.com and generating reality maps from aerial images of cities to even whole country-scale. The application delivers ‘survey-grade’ representations of reality.

Currently, the commercial list price of ArcGIS Reality Suite is $20K for a one-year subscription, which runs on any CPU, with no limit to the number of cores or GPUs. To support times of peak workloads, Esri provides discounted surge pricing for additional licences.

Site Scan for ArcGIS is Esri’s cloud-based end-to-end reality mapping software for drone imagery, simplifying drone program management, imagery data collection, processing, and analysis.

ArcGIS Drone2Map is a desktop application focused on reality mapping from drone imagery, enabling offline processing and inthe-field rapid mapping.