PERSPECTIVES

How Connected Worker Technology Helps Address Industry Labor Issues

By David Greenfield

Director of Content/Editor-in-Chief

Challenges associated with attracting and retaining workers to the manufacturing and processing industries have been ongoing for decades. And while some types of automation technologies, such as robots, help keep humans out of the repetitive and potentially injury-inducing work that can make industrial employment less desirable, other types can also help attract and retain human workers.

Chief among these technologies that can help attract workers are connected frontline worker (CFW) platforms. This Industry 4.0 technology is designed to guide workers in their tasks using smart glasses, smartphones, tablets, and PCs, enabling them to handle complex processes.

Dominic Gallello, CEO of Symphony Industrial AI, a supplier of CFW, digital manufacturing, and plant performance technologies, explains that CFW systems “leverage cloud computing, mobile telecommunications, and wearable technology to inform front line workers and o er them state-of-the-art, human-driven instruction support by providing ‘byte size’ work directions to act upon.”

Benefi ts of CFW technology cited by Gallello, include: • Up to 50% savings of time on operations such as changeovers through standard work guidance; • Facilitating autonomous maintenance with step-by-step instruction workfl ows; • 30% faster onboarding of new workers; • Achieving 100% compliance with work duties via automatic execution logs; and • Time savings of up to 70% on inspection process with digital CAPA (corrective and preventative actions) management.

“At the core of CFW platforms are digital workfl ow execution engines that consist of two parts,” notes Peter Verstraeten, CEO of Proceedix, a Symphony IndustrialAI company that supplies digital work instructions and inspection software. “First, the authoring component enables business experts to create and maintain all kinds of workfl ows to provide step-by-step guidance for operators, technicians, and inspectors in various industries. Second, the digital execution system automatically logs all workfl ow execution details to assure 100% transparency and compliance—anytime and anywhere—whether the worker is online or o ine.”

Beyond delivering specifi c work instructions and logging activities, CFW platforms also integrate with existing business and operations systems to provide workers with instructions and inspections from manufacturing information systems, such ERP (enterprise resource planning), MES (manufacturing execution systems), CMMS (computerized maintenance management systems), and LIMS (laboratory information management systems). Such integration also enables CFW platforms to loop worker feedback into these systems.

“This allows for business intelligence platforms to assess performance, check compliance, and identify areas for improvement using this new, correlated data,” says Gallello.

Getty Images

5 tips on how to get the 5 tips on how to get the most out of industrial wearable devices.

Learn how Frito-Lay improves worker ergonomics with wearable technology.

T�e ifferences etween ocal and emote O

By David Greenfield

Director of Content/Editor-in-Chief

Industrial I/O (input/output) modules are critical industrial network components that deliver input signals from a field device, such as a sensor or actuator, to a controller. These modules then route output commands from the controller back to the device. As simple and straightforward as this seems, decisions around how to deploy I/O modules on an industrial network get more complex when you consider the remote and local I/O options available.

These options led one Automation World reader to ask: Can you use remote I/O in the main control/electrical enclosure? Though this question may seem to run counter to the purpose of remote I/O, it is not an uncommon method of remote I/O deployment.

To learn more about industrial I/O options and the best way to deploy them for di erent applications, we connected with Bryan Little of system integrator Avanceon to learn more for a recent episode of the “Automation World Gets Your Questions Answered” podcast.

We began our discussion with an explanation of the di erent industrial I/O types. According to Little: • Local I/O is a term used to refer to I/O modules located in the same rack or chassis as the controller and typically won’t have any computing power onboard due to its proximity to the controller. • Distributed or remote I/O is typically deployed in a location separate from than the main controller. As such, distributed

I/O will typically have some level of computing power onboard to perform data processing, as well as the ability to turn outputs on and o independent of the main processor.

Given that the definition of remote or distributed I/O indicates that the modules aren't located in the main electrical enclosure, the reader question about how to use remote I/O in the main control enclosure would seem to indicate a lack of understanding about the di erence between remote and local I/O. According to Little, the question isn’t as o base as it may seem.

Reasons to put remote I/O in the main enclosure

“There are actually a lot of good benefits” to placing remote I/O modules in the main control cabinet, says Little. Three of the top reasons are: safety, less wiring, and standardization.

On the safety front, preventing arc flash is good reason for putting remote I/O in the main cabinet. With remote I/O in the cabinet, you can have di erent doors inside your panel to create a high-voltage side and a low-voltage side. “You can put remote I/O in that highvoltage side to assist with troubleshooting without having to open those doors,” he says.

Regarding standardization, if all the I/O modules used in the plant are remote I/O, that can help a company standardize on spare parts. “You can trim down the training needed on the same parts; and using the same part in di erent locations helps with engineering drawing standards,” Little says.

Higher cost?

Though remote I/O modules used to cost more than local I/O, Little doesn’t see that as being a di erentiator today. “You can get the same performance from a more compact remote I/O module that may be less expensive than main PLC I/O cards,” he says. “I think the upfront costs really come down to the network infrastructure. But with industry trending towards everything being on some kind of network, that infrastructure is probably already in place. So if you already have Ethernet for your PLC and your HMI, adding an Ethernet remote I/O isn't much of an addition.”

Plus, there are benefits related to future expansion. “A main PLC chassis usually only has a maximum number of I/O slots, but you can always add remote I/O,” says Little.

Wireless remote I/O

As with anything wireless in the industrial control arena, many engineers are skeptical about its dependable use in production operations. But Little says wireless remote I/O could be beneficial depending on the application.

“There are a lot of things going on with data collection now,” says Little, “so if you have a control panel in place and you needed to collect data from it, such as vibration or temperature inside your panel, you can easily add some wireless remote I/O to collect that data. And you wouldn't have to build a whole new infrastructure.”

Since the start of the COVID-19 pandemic, we’ve all become aware of the supply chain issues facing food and beverage and every other industry. Among the many pandemic-era product shortages we experienced was a lack of cream cheese, which went missing from retail shelves just in time to put a damper on the making of holiday treats in late 2021.

As it turns out, this cream cheese shortage was not the product of the typical supply chain issues seen by the industry. Instead, a ransomware attack knocked out core systems and operations at Schreiber Foods for several days in October 2021. Not only was Schreiber on the hook for a reported $2.5 million in ransom, but the attack wreaked havoc on peak production season for the large Wisconsin-based cheese producer. It was unable to conduct business as usual for days—a disruption that trickled down to smaller farmers, cooperatives, and companies that buy ingredients from Schreiber.

And this is just one example. Cyber attacks have hit the food and beverage industry hard this past year. In its “State of Malware” report, Malwarebytes clocked an eye-popping 607% surge in malware detections in the food and agriculture sector in 2020, with things dramatically settling down in first quarter 2021, yet still hovering at a notable 36% increase. One of the most high-profile incidents in 2021 was the strike at JBS Foods, a global meat processor that doled out $11 million in Bitcoin to the REvil ransomware gang and halted operations in more than a dozen U.S. processing plants, prompting meat shortages around the country. Smaller companies were also in the crosshairs. New Cooperative, an Iowa-based farm service provider, and farm co-op Crystal Valley were attacked by the BlackMatter ransomware group, while a U.S. bakery was one of more than 1,000 companies a ected by the Kaseya supply chain attack as it lost access to key systems and had to pause operations.

The flurry of attacks on the sector prompted an o cial September 2021 warning from the FBI’s Cyber Division alerting food, beverage, and agriculture companies to the growing threat, especially as the adoption of smart technologies and Industrial Internet of Things (IIoT) applications increases the potential attack surface. Larger businesses are targeted because of their ability to pay higher ransom demands, the alert cautioned. But smaller entities are not immune because they’re generally viewed as soft targets, especially those in the early stage of digital transformation that aren’t as tech-savvy and versed in cybersecurity best practices.

“Cyber-criminal threat actors exploit network vulnerabilities to exfiltrate data and encrypt systems in a sector that is increasingly reliant on smart technologies, industrial control systems, and internet-based automation systems,” the report reads. “Ransomware attacks targeting the food and agriculture sector disrupt operations, cause financial loss, and negatively impact the food supply chain. Companies may also experience the loss of proprietary information and personally identifiable information (PII) in addition to the reputational damage that can result from a ransomware attack.”

Put Your Cyber Defenses Up efore T�ey Take ou own

By Beth Stackpole

Contributing Editor

A changing landscape

The increasing number of cyber-attacks on the food and beverage industry comes on the heels of threat actors targeting other critical infrastructure sectors, many of which are migrating from closed environments (often referred to as a walled garden) to networks of connected devices, equipment, and systems as part of e orts to digitally transform operations. Leveraging technologies like cloud, IIoT, advanced analytics, and artificial intelligence (AI) and machine learning (ML), manufacturers in this space aim to parlay terabytes of data long collected and stored in industrial equipment and systems into insights that will garner e ciencies, spark innovation, and optimize new business processes. Most share a common goal: to boost quality, improve plant performance and uptime, and enable predictive maintenance.

The global pandemic also sparked major changes to manufacturing operations that increased cybersecurity risks as remote access capabilities were used to accommodate personnel unable to physically be on the plant floor. “What accelerated with COVID-19 was remote access, as not everyone could be on premise,” says Marilidia Clotteau, food and beverage marketing manager for the consumer-packaged goods segment at automation supplier Schneider Electric. “Before, everything was in the plant. But when you start having a mix of on-premise, cloud, and connected devices, there are more potential vulnerabilities. There needs to be constant review and implementation of barriers to ensure the house is well kept, managed, and secure.”

While many in the food and beverage sector are hungry for Industry 4.0 applications to stake out a competitive edge, their existing production environments often aren’t ready to digest new technologies securely. Most industrial control systems—from simple programmable logic controllers (PLCs) to more complex supervisory control and data acquisition (SCADA) and distributed control systems (DCSs)—as well as industrial networks were designed decades before cybersecurity was a major concern. As a result, many lack the basic encryption, authentication, and authorization controls along with automated asset management capabilities that are a staple of enterprise IT platforms. Moreover, the alphabet soup of proprietary protocols employed by industrial equipment, the landscape of siloed systems, and the lack of enterprise-grade monitoring tools makes it much more challenging to safeguard OT networks and assets compared with enterprise IT counterparts.

“Most of the industrial and control verti-

cals weren’t developed with cybersecurity as a first principle, and plants were dependent on the enterprise to protect operations,” notes Mike Lester, director of cybersecurity strategy, governance, and architecture for Emerson Automation Solutions. “There’s now a spectrum of security capabilities and postures you have to deal with, but it hasn’t been the primary focus in this industry. That has been safety and control, and now there’s the cybersecurity wrinkle.”

Though there are federal and state regulations governing plant and equipment safety, they don’t yet extend to cybersecurity protocols, according to Colonel John T. Hoffman, senior research fellow at the Food Protection and Defense Institute (FPDI) based at the University of Minnesota and established by the U.S. Department of Homeland Security to pursue research, innovation, and education programs to reduce food system disruption. In fact, some U.S. Food and Drug Administration (FDA) rules, specifically those approval processes related to when devices or sensors are

Companies in the food and beverage sector are gearing up cybersecurity e orts on the plant floor in the face of escalating attacks. Photo courtesy of Rockwell Automation

changed, are just cumbersome enough to act as a deterrent to upgrading to newer, more secure technology, he contends.

“The mentality in the OT world is: If it’s not broke, don’t fix it. And the result is legacy OT devices scattered through food production connected in many cases in totally illogical ways,” Hoffman says. Consolidation in the industry has made it even more difficult for OT and IT management to have visibility into exactly what equipment is in the plant, let alone have a complete under-

XX Ultrasonic Sensors with NEW easy-to-use software. Ideal for level detection & control, mobile equipment, material handling and hoisting applications.

WOW! That was “Simply easy!”

This engineer just set up several ultrasonic sensors for a new machine line. Despite the varying ranges he had to set, he used a single software application. He set the distances. He adjusted gain. He filtered out anomalies. And those settings will remain for future replacement sensors.

Programmable... ...Flexible... ...and SMART!

standing of potential vulnerabilities.

Not only is the number of devices installed in food and beverage plants an order of magnitude higher than other industries, the equipment is much more bespoke and varied. “There are a lot of very unique devices in the food world—for example, a device that cuts corn flakes into a finished product,” Hoffman explains. “That cutter may be unique to the company, been in use for 25 years, and no one wants to fix something that’s not broken.”

The number of devices installed in food and beverage plants is an order of magnitude higher than other industries, and the equipment tends to be much more bespoke and varied. This can make manufacturers even more hesitant to upgrade legacy systems as needed.

A blueprint for cybersecurity

Outdated equipment might still be functionally operational, but it packs a huge deficit when it comes to modern security controls, which means action is required. While escalating attacks have put cybersecurity concerns on the radar of top executives in food and beverage companies, as well as in other sectors, it’s still not a top concern for too many. In its 2021 industry outlook, tax, audit, and advisory firm Mazars USA found cybersecurity ranked surprisingly low as a top concern for business, with only 10% of respondents reporting they were “very concerned” about threats, up slightly between 2019 and 2021.

Nevertheless, there is a growing mandate to move forward, and the first step is to embark on a security assessment—either on your own or, more likely, with a qualified partner—to evaluate the current state of the organization’s OT infrastructure and to define clear cybersecurity goals. Implementing a controls firewall and network segmentation to ensure safe zones should be a central part of the evolving cybersecurity roadmap, along with standard processes for regular backup of data, including air gapping and storing password-protected backup copies offline.

Central to the FBI’s cybersecurity guidance to manufacturers in this sector is to create a recovery plan to maintain and retain multiple copies of sensitive or proprietary data and servers in a physically separate, segmented, secure location—a strategy that is also critical for business resiliency and continuity. Regular installation of software updates, including patches to operating systems and firmware, needs to be codified into ongoing cybersecurity best practices and system maintenance, the FBI alert says.

In terms of the technology stack, cybersecurity safeguards remain relatively consistent between IT and OT and, in fact, should be coordinated as part of a holistic security plan. Most experts suggest following the National Institute of Standards and Technology (NIST) Cybersecurity Framework, a set of industry standards and best practices to help organizations manage and mitigate cybersecurity risks. Technologies such as asset inventory and identification, network segmentation, endpoint protection, incident response planning, secure remote access capabilities, and real-time threat detection are key pillars of a modern security roadmap.

Longer term, experts expect advanced capabilities such as multifactor authentication and use of AI and machine learning to also become central to plant floor defenses. “You should expect to see more use of AI around user action so if an operator suddenly makes a change that’s outside of range, the system would automatically flag something like that as a potential unauthorized operation,” notes Brian Fenn, COO of Avanceon, an IT service management company and systems integrator.

Beyond the synergies, there are also key di erences in IT/OT security requirements. Enterprise IT security has historically focused on confidentiality, integrity, and availability, but the order is reversed when viewed through the lens of OT priorities. “The plant doesn’t necessarily care about confidentiality as long as operations are safe and they’re still producing,” Emerson’s Lester says.

Specifically, there’s the problem of incident response time—an area where there are stark differences between what constitutes acceptable downtime for IT systems compared with OT systems. “If I’m a consultant responsible for the service level agreement (SLA) of a large food and beverage IT infrastructure, email could be down for eight to 10 hours on a weekend and it’s not overly critical in terms of dollar importance,” explains Brian Deken, connected services commercial manager for Rockwell Automation. “Yet if I’m down one hour on the plant floor, it could be hundreds of thousands of dollars. You need some sort of automated response system as part of realtime threat detection to have more rapid incident response and recovery.”

Bridging the IT/OT divide

Traditionally, security efforts related to OT and plant floor technology have been outside the purview of IT and the enterprise chief information security officer (CISO), if one exists. Even if there’s some coordination, there hasn’t been much in the way of a formal, shared roadmap. That OT/IT divide needs to close in order to adequately safeguard plant floor assets as they are synced up with enterprise applications and potentially cloud platforms as part of ongoing digital transformation.

While responsibility for OT security will vary depending on the food and beverage company, it’s important that IT and OT work together on cybersecurity initiatives. “The better they manage convergence, the better they do here,” Fenn says. “You’re trying to take principles and concepts from the IT space and make sure they’re applied in a way from an OT standpoint that will keep things

running and stable and not cause other issues down the line.”

For example, OT systems, which are typically more isolated and have decadeslong lifecycles, demand stability from a production standpoint so they can’t be managed and updated in the same manner as IT systems, where you can automatically push out an antivirus or applications update, Fenn explains. As a workaround, he suggests setting up a develop and test environment where OT and IT can work through application patches and antivirus updates to keep systems secure without having to take mission-critical production systems o ine as part of the process.

“If you need to take down a legacy system to patch it, every moment it’s down is loss of money,” says Guilad Regev, senior vice president, global customer success, for Claroty, which markets an industrial cybersecurity platform that includes continuous threat detection and secure remote access solutions, among other capabilities. “If you create segmentation and redesign networks, it factors in all the pros and cons.” Read more about the cyber attack at JBS.

Cybersecurity best practices

With the foundational technologies in place, food and beverage companies can begin to execute a cybersecurity roadmap that will ensure the right protections. Following these best practices will ensure the best results:

Conduct a complete risk assessment. It’s important to understand what’s out on the networks and how it’s all interconnected, but it’s also critical to perform a similar deep dive on the changing landscape from a controls perspective and to fully understand all the possible threat vectors. “Identify all scenarios by severity,” says Sree Hameed, industry marketing manager, consumer products for Aveva. Target controls to the machines with the greatest severity and the highest likelihood of breach—for example, the systems and machines that govern recipes, which would cause the highest levels of damage if breached, he explains.

Perform regular OT asset preventive maintenance checks.

Check in regularly with vendors to determine when upgrades are coming out and regularly monitor log fi les to check for abnormalities. In that way, you can spot something in short order when it starts to go awry.

Invest in cybersecurity awareness and training. It’s not enough for enterprise professionals and key plant floor personnel to understand what’s at stake in the event of a cybersecurity breach—the greater organization needs to share the responsibility and be versed in an action plan for risk mitigation. Conducting cybersecurity awareness training a couple of times annually and requiring certification can go a long way in building up cybersecurity competencies across the organization. “Embed training into employee performance to foster a culture of cybersecurity in the organization,” Clotteau says.

Address the information gap with new talent. Industrial engineers in the food and beverage sector understand the unique protocols, proprietary systems, and uptime requirements of OT, but are unfamiliar with common cybersecurity technologies. At the same time, IT gets cybersecurity, but has no real knowledge of plant operations. To complicate matters, there’s lingering distrust between the groups. “Companies need to groom some unicorns,” says the FPDI’s Ho man. “To be an IT security specialist in an OT world—that’s a challenge. Universities can help, but it doesn’t exist right now.” Learn about the key challenges to using automation in packaging and processing operations. Click… Clack… Closed! The Fast Assembly Cable Entry Frame The KEL-FA, (Fast Assembly) cable entry frame offers a fast, tool-less assembly. Provides a strain relief according to EN 62444 and a wide range of cable grommets for any application. UL TYPE IP54 www.icotek.com