Situational awareness – the 21st Century Warfighter

By John Reardon, Publisher

The United States Army has had centuries of experience advancing the vehicles used in warfighting. If you consider the arrival of the first tank on the battlefield early in the 20th Century, you can imagine the fear it conjured up. This continued through World War II with the Sherman tank and completely changed how mechanized vehicles impacted the battlefield forever. As we move into the 21st Century, the same drive to advance the abilities of military vehicles is occurring. The sophistication that the Army is now employing in the next generation vehicle has the potential of being a generational change that is equivalent to a soldier on horseback facing their first tank.

The Army is moving forward with solutions to address greater preparedness on the battlefield at the United States Army Combat Capabilities Development Command or DEVCOM in Warren, Michigan. This is the home of the Army’s best and brightest minds. They are working on the Electrification of the fleet - striving to have vehicles that can be selfsustained for 7 days without being recharged. They employ many Advance Driver Assistance Systems for autonomous operations and contribute to the driver through augmented reality. They are advancing security around C2 and working to have increased situational awareness. Situational awareness will be our focus.

The first step in the Army’s quest for situational awareness is real-time video capture with a 360-degree view surrounding the vehicle. This will require the solution to handle multiple video feeds in high resolution and with low latency. The OSS solutions capture video at 64 giga transfers per second. A demosaicing algorithm is applied to reconstruct the image utilizing the GPU at incredible speeds to create a viewable image. This architecture allows each camera to be run in parallel with a raw data feed, thus avoiding additional latencies.

Processing Speeds

In reviewing their options, the Army looked to the experience of One Stop System to build an environmentally appropriate solution that incorporated the high-speed performance required but with much lower latency than traditional Ethernet architectures. The OSS solution uses a PCIe Gen 4 architecture that takes multiple video feeds into several Nvidia Jetson AGX Orins for simultaneous distribution to the crew. The use of the Nvidia Jetson AGX Orin affords the Army ample headroom in computing power, along with all the tools within the development kit of the Orin. The Jetson features 8 Arm Cortex A78AE and 64 GB of RAM, able to do 275 trillion operations per second. Having the advantage of a large amount of support for various sensors makes this a powerful choice as needs evolve.

It was clear to the Army that to achieve a low latency video stream that could assist with a vehicle’s many needs in real-time, they would have to move away from Ethernet and the packet data structure used. To achieve the desired results of full motion video with less than a 50-millisecond latency, the Army looked to the leaders in applying PCIe technology from One Stop Systems. Currently shipping PCIe Gen 4 solutions that produces speeds of 16 GBps, the OSS solution could meet the need and evolve as Gen 5 and 6 solutions become available.

The OSS solution was designed to have two primary functions. That of retrieving unaltered, high-speed video feeds from the cameras and pushing this data out over the network to be processed and displayed to the crew in real time. In this case, real time refers to seeing images with latencies similar to seeing the live action with the human eye rather than through a camera for an experience indistinguishable from the real thing. This solution allows the raw data feed to be fed to the GPU for much faster processing without being limited to the ability of each discrete camera. And subsequently, be pushed out to the network via a multi-casted, direct memory access scheme to each node.

In measuring the latency across the network to test the impact of system alterations and software mods, a system-wide benchmark is used to determine overhead at each node. The Army can determine if the desired < 50-millisecond goal is sustained as other features are employed. This occurs by measuring the actual user experience from each display (Glass-to-Glass) and the impact of each additional software modification.

The ability to retrieve real-time video streams and react to them opens a realm of possibilities. By having a 360-degree view, any discrete vehicle can better defend itself and act as a sensor to communicate vital information to the command or others on the battlefield. It can employ a level of speed and accuracy that can monitor concerns that might otherwise be missed by a distracted crew or remote operator. This could be the detection of an IED (Improvised Explosive Device), or it could be a predictive maintenance concern such as a fluid leak. It also has the potential to support a driveby video capability, such as maneuvering the vehicle remotely or augmenting the driver’s threat awareness.

Artificial Intelligence

The leadership of DEVCOM has made it clear that a narrow focus on AI applications will be the process in which this video server will be built. The idea of curtailing what they refer to as “Irrational Exuberance” surrounding

AI and instead being more practical in its application. In other words, they are eager to apply simple video and pattern recognition algorithms to improve situational awareness, but they will be prudent in employing compound, complex algorithms that might take control away from the operator.

Screen Latency

In a situation where millisecond counts, the time it takes to display an image is a significant consideration. The choice of the monitor can cause a delay anywhere from 1 millisecond to more than 15 milliseconds. The high-speed monitors with a faster refresh typically used in civilian gaming applications are those with the lowest latency. Current solutions adapted to the rugged environment tend to use monitors that trail in performance to those used by a gamer. For this reason, advances in environmental consideration for more advanced displays will be needed.

A Unified Network

The OSS solution has created a SWAP-C solution that is practical in its application across the nearly 14,000 ground vehicles employed. It allows a vehicle traveling at speed to be aware of hostile concerns and has sufficient system overhead to maintain a target within its sights. It is the speed and the low level of latency of the OSS solution that makes it possible to use situational awareness in a fire control sequence. The OSS solution supports nodes within the vehicle, such as the Gunner’s station, the Squad leader’s computer, the driver’s computer, and the Commander’s station. By acknowledging that the threat environment is dynamic and becoming more complex from peer and near-peer adversaries, the OSS system offers a unified network to provide situational awareness when needed.

Although the Army has clearly defined the parameters of the OSS video viewing system, using PCIe will enable them to develop the solution to be an integrated part of the battlefield. The bandwidth of the system and future PCIe Gen 5 or 6 architectures will be easily complimented with advances in the Ampere class GPUs. This ability to interconnect will lead to a mesh of sensors across the battlefield from vehicles that will be incorporated into the battle plan.

Conclusion

The OSS solution is a tailored response to a specific need. It meets all the application parameters along with growth potential due to the performance of the Jetson Orin and future iterations of PCIe. There is an elegance in the SWAP-C characteristics of the OSS solution that provides all the needed features without painting the Army into a corner. The solution can scale to future needs as the world evolves. It employs open standards that have already proven themselves as advanced technologies able to scale.

The explosion of technology within the Bradley, the JTLV, and Stryker has enhanced the utility of these vehicles beyond their original charter. The future of these vehicles will continue to borrow from commercial and consumer applications of autonomous vehicles. The Electrification of military vehicles will undoubtedly bring about data center performance at the edge. As the production of these vehicles continues to ramp, the OSS solution should scale towards the performance range of their Rigel Edge Supercomputer platform, OSS’s more prominent and most advanced highperformance computing platform. Giving the Army what they need in a performance computing platform at the edge.