SPECIAL FEATURE
Bringing AI to the Rugged Edge By Jim Ison, Vice President, One Stop Systems
Bridging the gap between sensors and highperformance compute power is a growing challenge, especially in systems where quick, complex decisions are vital The concept of “the edge” in embedded systems has taken on new urgency in the federal space. The need to address the burgeoning amount of high-speed, vital data with powerful AI processing for immediate complex decision and reaction could be likened to the idea of stuffing a data center under the seat of a helicopter (if only!). “The edge” can be defined as “where it’s happening” and in government systems, that’s the field. Traditionally, the problem of linking high-speed sensors and actuators with super-powerful AI resources has been addressed with high-speed data communications. But that has serious limitations in terms of field operations where package size, speed, mobility, and reliability are paramount. This compound challenge involves a combination of truly enormous processing power with vast multiple terabyte data storage, internal data routing, external high-speed data acquisition bandwidth and blazing high-speed networking capability. All this must then be packaged into a standard rack mount with cooling capability for the truly large amount of heat that will be generated. In addition, the packaging must be ruggedized for shock and vibration, capable of a variety of power source connections and designed for easy setup and installation. It literally 16
COTS Journal | November 2021
involves making data center hardware available at the rugged edge. There have, of course, been previous efforts to meet the demands of AI at the edge but so far none of the attempts have managed to meet all the challenges facing true “under the seat” implementation. There have been several that have incorporated sufficient computing power— mostly in the form of graphic GPUs that can handle the parallel inferencing inherent in AI. However, some have not managed to meet the small form factor size requirements. Those that have, have not been able to package enough hardware nor supply the needed cooling. Still missing from
some that may meet many of the needs is the necessary ruggedization required for land vehicles and aircraft. One feature they all do seem to agree on, and implement is the use of a high-performance graphic processor, or GPU. GPUs are not only superior for floating point operations needed in graphics but also for the parallel inference operations required by AI. The GPU that almost all AI at the edge implementations are focusing on is the NVIDIA A100 Tensor Core GPU (Figure 1). The A100 implements seven GPU cores per unit and offers up to 2 terabytes per second of memory bandwidth.
Figure 1: The NVIDIA A100 Tensor Core GPU provides up to 20X higher performance over the prior generation and can be partitioned into seven GPU instances to dynamically adjust to shifting demands. It is available in 40GB and 80GB memory versions.
