Boosting inspection performance with AI and deep learning

BOOSTING INSPECTION PERFORMANCE WITH AI AND DEEP LEARNING

Inspection systems play a crucial role in keeping consumers safe, as product contamination can cause harm and has the potential to result in high costs and damage to brands. Elisabeth Skoda looks at recent developments and explores the role AI can play to make inspection systems more efficient.

A proactive approach

Different types of inspection systems, whether it’s vision inspection, metal detecting or X-ray systems, face a range of challenges and have to find a balance between precision and false rejects.

Smart technologies have increasingly been used in inspection technology not just to identify contaminants, but also to improve efficiency, compliance and profitability.

In the era of the Internet of Things (IoT), everything that can be connected will be connected, points out Phil Brown, Managing Director of Fortress Technology Europe and Sparc Systems. Knitting together the extraction of data and bridging the gap between machines and humans to help predict rather than react to all types of production scenarios. Advanced new systems with data collection and paperless test routines are becoming more prevalent.

“Manufacturers might opt for a combination metal detector and checkweigher capable of pre-configuring every test by retailer code of practice and product being inspected. As demonstrated by the Raptor series recently launched by Fortress and Sparc, integrated sensors can confirm when a check has been conducted, generating a dated digital due diligence report that is signed by the operative on the screen for full transparency,” Mr Brown adds.

In food inspection, the quantity of data tags is fairly small. Yet, this makes it easier to automate the identification of meaningful patterns. When timeseries granular data is monitored side by side across a fleet of machines 24/7, it provides production managers with the level of production oversight previously unattainable.

“Robust communication must be the starting point of any good digital transformation program. We recently launched a new suite of connectivity software. Comprising turnkey packages like Contact Reporter and Contact Manager, the Industry 4:0-ready package also gives factories a choice in networked data retrieval technologies – including OPC/UA Adapter and Ethernet/IP Adapter – to facilitate real-time end-to-end production visibility and enhanced Quality Assurance and due diligence,” says Mr Brown.

A software-driven approach

This year, Fortress Technology Europe and Sparc Systems are set to launch a system that combines two technologies – X-ray and metal detection – to provide a more failsafe inspection solution in line with digital transformation.

“Sparc engineers have conceived a software-driven solution for inspecting dual density products with an X-ray. The software uses two algorithms

working side by side to inspect denser areas of a product, such as bread crust, and the less dense centre. This same technology is now being applied to a compact integrated metal detector-X-ray system. Operating side by side, two inspection heads are mounted on one conveyor and operated by a single touchscreen, sharing electronics, controls, and a reject mechanism.”

Boosting accessibility

Neil Sandhu, SICK’s UK Product Manager for Imaging, Measurement and Ranging, has observed developments towards greater accessibility in inspection technology.

“We have seen an explosion in ease of use that has hugely broadened the accessibility of machine vision systems to packaging operators, opening up so much more scope for them to automate quality inspection and process control tasks. Advancements in processing technology have enabled massive amounts of power to be packed into ever-more compact and affordable, programmable vision sensors. Only a few years ago, 3D was still feared as a ‘black art’ only to be attempted by seasoned experts, while 2D systems still needed a lot of time and cost to configure and commission. Now many 2D solutions are ‘plug and play’ and even creating something bespoke is straightforward using just a webbrowser dashboard, and without the need for programming skills.”

Machine vision hardware has become smaller, more powerful and more robust, but the real advances have come from the software, he adds.

“Development platforms like SICK’s AppSpace aimed at making it as quick and simple as possible to configure a machine vision solution – no matter whether you are an experienced machine-builder or a novice end-user.”

Using AI in pharmaceutical inspection

Syntegon recently installed the first fully validated visual inspection system utilizing Artificial Intelligence (AI) in an automated inspection machine in collaboration with biotechnology company Amgen. The AI utilizes deep learning algorithms which are capable of accurately identifying recurring patterns and deviations.

“When we started development in 2017, AI was already used in many domains, so why not apply it to pharmaceutical inspection? One of the main challenges consisted of transferring this type of application to very complex pharmaceutical processes and developing suitable implementation and validation concepts for this strictly regulated industry. Together with Amgen,

we were able to install the first fully validated visual inspection system utilizing AI in an automated inspection machine in 2020,” says Dr José Zanardi, responsible for vision inspection development and applications at Syntegon.

The AI-based system resulted in both an increase of particle detection rates and a reduction in false detection rates, he explains.

“Amgen uses the system to reliably distinguish pesky air bubbles at the syringe’s rubber stopper from foreign particles, where conventional vision technology may misidentify. Syntegon’s AI-based vision system was able to increase the particle detection rate by 70%, while reducing the false detection rate by 60% (average values in a particular inspection station).”

Dr Zanardi points out that there are a few points to bear in mind when it comes to using AI within inspection systems.

“A decisive factor is courage. The pharmaceutical industry is known for its conservative approach to innovation, due to the very strict regulatory guidelines for process validation – overall a highly positive attribute since the manufactured products have a direct impact on the health and safety of patients. Inspection technology experts can easily perform the required upgrades for visual inspection usage. However, there is one crucial point that must be considered to enable validation: in contrast to

“Knitting together the extraction of data and bridging the gap between machines and humans to help predict rather than react to all types of production scenarios.”

“Where it has previously been very challenging to achieve consistently robust and repeatable quality inspections, for example when packaging naturally-grown produce, they can now be mastered with high levels of reliability and availability.”

many other industries, the neural network must be ‘frozen’ for application in pharmaceutical use once its training phase is finalized. It must be fixed and internal parameters are no longer able to change to make it versioncontrolled for validation.”

Deep learning in vision inspection

Mr Sandhu points out the potential of deep learning when it comes to new developments in inspection technology. The SICK Intelligent Inspection Deep Learning App runs on SICK’s newly launched InspectorP621 2D programmable vision camera. By embedding the Intelligent Inspection App onto SICK’s InspectorP621 deep learning camera, SICK can offer a ready-made package that uses artificial intelligence to run complex vision inspections.

“Where it has previously been very challenging to achieve consistently robust and repeatable quality inspections, for example when packaging naturally-grown produce, they can now be mastered with high levels of reliability and availability. Automation is therefore now practical and affordable for complex imaging tasks such as sorting fruit and vegetables or checking that flow wrapping is correctly sealed,” he adds. “The image inference is carried out directly on the device, so there is no need for an additional PC. As the system training is done in the Cloud, there is also no need for separate training hardware or software, saving on implementation time and cost.”

The app is set to be expanded to anomaly, detection and segmentation functions.

“This will enable a product with a complex shape to be identified in a scene, or to be picked out from a random selection on a conveyor, for example to pick apples randomly placed on a conveyor but leave the pears,” says Mr Sandhu.

The future of AI

Dr Zanardi sees great potential for AI and deep learning in inspection technology, and foresees it will be used more and more in facilities globally, and different inspection processes.

“AI is especially advantageous in ‘Difficult to Inspect Products’ (DIP), i.e. products that pose a challenge to obtain satisfactory inspection results when inspected with traditional image processing techniques. Advantages include an increase in detection rates (i.e. defective products successfully assessed as defective) and a decrease in false reject rates (i.e. good products wrongfully assessed as defective). Achieving these two points simultaneously is a remarkable feature, which is made possible by utilizing AI. Syntegon are developing the AI solution further to implement it in other machine families in our portfolio in the near future. Fewer false rejects will make inspection more efficient. Hence, I am convinced that our project with Amgen is just the beginning of a very exciting journey,” he concludes. n