OCR Update

OCROCR latestlatest andand aa firstfirst ….….

Inthequesttoincreaseefficiency,productivity andsafety,terminaloperatorsareopting fortheuseofartificialintelligence technologiesandmodernautomated controlsystemsfortheirOCRSystems. ThomasFrancisreports…

The capturing and processing of key data detected by Optical Character Recognition (OCR) systems has been in place now for many years.The data of containers and vehicles enhance the container handling process through shorter lead-time and higher data quality.It enables the automatic release and confirmation of work order from the terminal operating system to the container handling truck in real-time.

Those operators who have deployed OCR systems in their terminals have quickly realised the savings and benefits of such a system, including reduced data entry errors, reduced operating costs and increased crane productivity. Over the last couple of years, we have also seen the introduction of artificial intelligence technology for OCR systems. As containers are being handled, at speed, across multiple layers of container handling equipment, key data is being captured.This innovative method of acquiring OCR data in motion allows users to automate container identification both on the landside and quayside.

Spain-based Orbita Ingenieria SL, Ports and Terminals Division,has been delivering solutions for many years by providing automation services and engineering aimed to enhance the processes of its ports and terminals customers, including through its gate, crane, and rail suite offerings as well as expertise in the ocular character recognition space (OCR).

For example,one of their innovative solutions is the CraneCCR system which consists of a vision hardware controlled by an embedded software application.The CraneCCR controls the crane, ensuring a continuous monitoring of the spreader, as well as the identification of different type of movements (loading, unloading, removals, lane changes, gantry movements, etc.). In addition, the CraneCCR hardware has been adapted to optimise image capture conditions.The cameras provide PTZ technology, combining the latest pan, tilt and zoom camera technology with state-of-the-art control algorithms.The system captures container ID numbers, container ISO code, container IMO placards presence, images of all visible container sides, internal terminal truck number (ITTI) and seal presence.

It was no secret that the services offered by Orbita Ingeniera drew the attention of US-based TMEIC Corporation Americas, a subsidiary of Toshiba Mitsubishi-Electric Industrial Systems Corporation (TMEIC), and specialising in crane controls and automation including their MaxView range of advanced crane systems.At the beginning of April,TMEIC announced it completed the acquisition of the Ports andTerminals Division of Orbita Ingenieria SL (Orbita) through its wholly owned subsidiary,TMEIC Port Technologies SL. “Orbita ’ s expertise and experience enhances our Port Automation offering, ” said Declan Daly,Vice President of Industrial Systems at TMEIC Corp Americas.

This certainly would be the case, as the acquisition of Orbita ’ s Ports and Terminals Division will allow TMEIC to expand its offerings to new and existing ports and terminals customers in the form of both automated gate and yard systems as well as enhanced automated crane solutions, bringing TMEIC even closer to providing an integrated solution of IT and OT systems for container terminal automation.

Another company that provides OCR Systems is Sweden-based ABB. Deployed globally since 2002, their Crane OCR system provides near 100% accuracy and unequalled reliability. It automates the process of recording container information of containers loaded and discharged from ship-to-shore (STS) gantry cranes.

More recently,ABB completed a project at the Exolgan container terminal in Argentina and introduced a new module - the QuayPro. ABB installed their Crane OCR with the QuayPro module on all seven Ship-to-Shore (STS) container cranes at the Exolgan Container Terminal, which handles over 40 percent of the total container volume at the Port of Buenos Aires.The new module automatically tracks the location of containers discharged from and loaded to a vessel, significantly increasing the accuracy of BAPLIE (bayplan/stowage plan occupied and empty locations message) files.

According to Richard Micheli, Product Line Manager, OCR,ABB Marine & Ports, this is the first solution of its kind to go live in the industry and the QuayPro application has already improved the container terminal’ s cargo-handling processes since its implementation.

“As we look to respond to increasing container volumes and minimise physical risks to our personnel, we are enhancing our processes to boost productivity and improve safety.ABB’ s OCR technology with QuayPro on our ship-to-shore cranes has proven pivotal in this regard and represents a major milestone on our roadmap to the future, ” said Duncan Glass,Terminal Manager, Exolgan Container Terminal. Adding QuayPro to ABB’ s Crane Optical Character Recognition increases productivity by streamlining the cargo-handling process, with stowage confirmation of containers loaded onto and discharged from a vessel, and the delivery of visual and audio work instructions to the crane operator. By reducing the risk of human error, stow plans are tracked and checked in real time and can be adjusted on the fly.

“Whereas in a traditional set-up, dock clerks move around the terminal and ship to record information manually and prepare a list of instructions for the crane operator,ABB automates the entire process for efficiency, flexibility and reduced risk, ” said Micheli.

“QuayPro also provides instructions for the crane operator on where to place the containers, using business logic to adapt to changing conditions, and record the container ’ s location to facilitate tracking, allowing key tasks to be executed remotely minimises the risk of accidents at the terminal. ”

ABB’ s full scope of supply for the seven ship-to-shore cranes comprises Optical Character Recognition hardware and software enabling complete traceability of damaged containers and seal presence images of the load/discharge of containers; the QuayPro module including the ‘CabView ’ features for accurate, real-time overviews of the bay as well as queued work instructions; and an Exception Management software application, which notifies the exception handler of any critical variations from the stowage plan.

“The solution is completing the process of automated container handling meeting the demands to manage bigger vessels and container volumes more efficiently. It will be a vital component in our objective to help make global container terminals become safer, greener and more productive, ” Micheli added.

With computer technology evolving rapidly, and the implementation of artificial intelligence, the latest generation of Optical Character Recognition systems and the implementation of such a system will bring a significant impact on efficiency, productivity and safety for terminal operators and ports around the globe.

Digital Twins to boost efficiency in container terminals

HolgerSchuett,MDatakquinetport consultingGmbH,Germanyreports...

Container terminals are under pressure, forced to serve bigger and bigger vessels with more cargo per arrival in shorter times.Digitisation is key in any industry and is also taking place in the port and terminal industry. But what do buzz words like visualisation, data mining or cloud application really mean when it comes to harsh day-to-day business, and how can this information be used to support the day-to-day operation at the terminal?

This digital information may be used as a base for optimisation algorithms and for forecasting the up-coming operation.This second step, using the digitised information is called digitalisation. Furthermore, by using AI (artificial intelligence) topics like predictive maintenance may be set into place.

Global Terminal Operator groups (GTOs) can offer a support service to each particular terminal manager by using a well-organised data warehouse combined with a state-of-the-art simulation system.This is exceeding the ordinary dashboard-technology whilst reaching far more towards depicting future opportunities in gaining better utilisation of machinery or inventory, which will lead to higher efficiency of the processes.Once evaluated by a simulation or even an emulation system, and based on continuous data pooling and mining,the results and solutions can be used to improve the inventory-setting, general layout and even strategies in operation.

The Digital Twin

Digitalisation has taken big steps in the industry in previous years. More and more sensors are plugged into machines to provide data of the current state of each single component of the system. Collecting all this data leads to big amounts of information, which has to be handled with special data storage systems. But collecting information should not be the end of the story. New technologies have been developed to analyse this big amount of data to provide algorithms to optimise the operation. One such technology is called the Digital Twin, which has become a buzz-word within the ports and terminal industry.

In 2003 Michael Grieves and JohnVickers developed this definition: “The Digital Twin concept model …contains three main parts: a) physical products in Real Space, b) virtual products in Virtual Space, and c) the connections of data and information that ties the virtual and real products together” Source:AWhitepaper by Dr. Michael Grieves, “DigitalTwin: Manufacturing Excellence through Virtual Factory Replication. ”Available from: https://www.researchgate.net [accessed Apr 06 2022]. Besides the real world and the virtual model there has to be a connection, which secures that the model at each moment has the same state as the real object. Based on this interpretation the digital twin supports various functionalities to different user groups of the Twin.

Container terminals are quite complex systems, which have to be controlled by IT systems.To secure the required productivity of the whole process, all sub-processes have to be synchronised in the best way. The logistic flow of goods in a terminal is most typically controlled by the Terminal Operating System (TOS) and the planning staff in the control room.

With upcoming (process as well as equipment) automation, the algorithms in these systems have to be improved.The base for this improvement is digitalisation, which provides more and more detailed information about the state of the terminal (i.e. PDSystems providing the current position of each object in short periods to name just one example of many).

Enhancing existing simulation and emulation models to be connected to the real terminal states provides a digital twin of the whole terminal.This will allow the planner, as well as the algorithms, to find the best decision at any point of time.

Functionality within the Digital Twin

To provide the adequate view on the data collected, the digital twin will use various functionalities. For example:

Visualisation Most people think in pictures instead of bits and bytes or tables.Thus, for humans, providing a view based on pictures will be the perfect view of the current operation at the terminal. 3D-Animations - known from the gaming industry - have made enormous developments in recent years, which allows the use of this technology for real-time visualisation of all processes at the terminal as a “Live-View ” .

The point of view as well as the angle may be varied by the single user.Thus, in the first idea the visualisation shows the same view as looking out of the window from the control room. But, the use of Augmented Reality (AR) functions will allow the highlighting of specific activities and states additional to the real world view, for example: - colour all equipment working for a specific work area (e.g. one vessel, gate traffic, etc.) with the same colour - colour the equipment according to the current productivity (green if best, yellow if medium, red if too low) - show only boxes planned for the next vessel departure - show IMDG cargo coloured by IMDG codes - colour code equipment by the fuel or battery state OtherViews for terminal staff may be dashboards, showing on one view the current state of all machines of a specific equipment type (again coloured by the state). Selecting a specific piece of equipment in the 3D or in the dashboard is showing its attributes, which may be different for e.g. yard-planners or maintenance staff. Furthermore, the 3D-visualisation may not only be provided at the terminal site but may also be shown at the headquarters.

* By providing these views of the current operation all decisions may be supported for the whole terminal staff in real time. Thus, they are able to make the best decisions at any time, especially increasing efficiency at the terminal based on the information.

Monitoring Whilst in theVisualisation mainly the current state of the terminal with all its objects is shown, the Monitoring is analysing the recent past and shows the results in a dashboard with pre-defined KPIs. In this case the Digital Twin is representing the information as numbers and timelines (instead of 3D).

The results support the planner again in taking the right decisions at any time based on the recent past. Furthermore, the monitoring acts as a base for more sophisticated algorithms in the TOS. For example, selecting the right transport equipment for the next job depends not only on the current position, but also on the lowest utilisation to provide a more balanced usage of the whole fleet.

3D-Animations have made enormous developments in recent years. Use of this technology for real-time visualisation of all processes at the terminal as a “Live-View ” . * In this way the decision support of human planners or of algorithms out of the TOS will be optimised, as they are based on current states and time lines instead of fixed rules defined in advance only.

Simulation Simulation technology has successfully taken its place in the port and terminal industry mainly for planning, testing and training tasks. Based on a digital twin it currently supports day-to-day operations at the terminal.

There are mainly two fields of applications: • Based on the whole information saved in the recent past, “ re-plays ” of historical scenarios are possible.The control staff may rerun the scenario from a recent shift and replay the operation to analyse occurred bottle-necks or in-efficiencies. Furthermore, the planners may run exactly the same scenario with another set of planning parameters to understand and learn how to act better next time.

• Terminal operators may also use simulation technology to forecast the upcoming operation. Based on the current state of the terminal and all its components, the scenario for a simulation run is more or less fully defined. Additional to the states of the yard and all pieces of equipment, the current state of the planning parameters out of the TOS has to be provided to the simulation. Planning parameters include the berth plan, crane split, work queues as well as allocation information of the yard (decking strategies) and the equipment (e.g. pooling).

Based on all this information out of the (enhanced) digital twin, the simulation may start a “ preview ” of the future operation. This preview functionality can be be started by the terminal staff, but it may also run in a scheduled way. In this case warnings will automatically be generated, if pre-defined milestones (e.g. vessel service finished) will not be achieved.Thus, either the control staff or - in a more automated way - the TOS may rethink the current planning and adapt it to the results of the preview.

* Both applications of the simulation technology, the “re-play ” of historic scenarios as well as the “preview ” of the upcoming operation, support the terminal operator and the algorithms in the TOS to find the best decisions and improve the planning of the operation, which will lead to more productive and more efficient

processes.

Predictive analysis based on AI Another upcoming technology in the port and terminal industry is the field of Artificial Intelligence (AI).Again, the data collected within the digital twin are used as a base for AI methods.These methods analyse the big amount of data available and detect correlations between various parameters as well as patterns of data, which will lead to specific situations.

In current projects these results are often used to predict parameters, e.g. eta/etd of vessel arrivals, as well as planning decisions, e.g. number of cranes allocated to the vessels regarding the owner and route. Another field is predictive maintenance. In this case patterns of sensor data are detected which lets terminal staff know to expect a break down of the device within the next hours/days.

* AI methods based on the current state of the terminal support the predictive maintenance, which allows the terminal to maintain the equipment on a case base instead of fixed time intervals only.

Standardisation

Currently, various standardisation initiatives have been founded to guarantee the interoperability and compatibility of systems in the port and terminal industry.

Two of them have a big impact to digital twin development: PortML stands for Port Markup Language. It is a descriptive language or document which describes container port assets, such as yard sets, quay cranes, wharves, and AGVs, as a markup code for modelling and simulation. The Centre of Excellence in Modelling and Simulation for Next Generation Ports (C4NGP, Faculty of Engineering, National University of Singapore) is initiating the creation of PortML for the building of future smart ports digital twins.

A universal PortML enables collaboration, interoperation, communication and formalisation of port designs, configurations and information into structured formats that are compatible with all stakeholders, within the global maritime and port industries. PortML also supports the fast building of reconfigurable digital twins, information sharing and coordination between international ports and port operators, thus bringing about improved labour efficiencies and cost savings for competitive business environments of the future. TIC4.0 is an initiative which plans to standardise the communication between equipment, systems and subsystems at a container terminal.

The main objectives are: - Define and agree on a common language and definitions among the companies involved in the cargo handling industry.

- Develop communication protocols and frameworks that will be adopted by industry players for seamless data communication.

- Facilitate the inter-operability of different information sub-systems of a cargo handling facility.

- Promote the deployment and adoption of selected existing standards and those developed by TIC 4.0 by the sector.

Based on these initiatives the digital twin should be built in a semi-automated manner

Currently, various standardisation initiatives have been founded to guarantee the interoperability and compatibility of systems in the port and terminal industry. Two of them have a big impact to digital twin development PortML and TIC4.0

TwinSim

At the end of 2021 Germany-based Eurogate Group initiated the ‘TwinSim ’ project and invited University Hamburg and akquinet port consulting to join with the aim of developing a digital twin of virtual models of processes and equipment on container terminals.This will result in analysis and situation-based optimisations, improving the processes with regard to efficiency, productivity, resilience, and minimisation of the environmental impact of container terminals.

Eurogate will implement the required technology to some equipment at the Container Terminal Hamburg and will build a data-lake to collect all information. Based on this data-lake the functionalities discussed in this article will be implemented. University Hamburg will analyse the data with the means of AI to optimise the planning and processes. As well as the operational handling processes, the maintenance of the equipment will be a main focus of this analysis. Breakdowns of equipment will be reduced by forecasting them and providing maintenance before it occurs.

Visualisation and Monitoring will be based on AKQUINET’ s 3D-Visualisation “Live-View ” , which is part of the CHESSON family.Within a 4-level architecture a 3D LiveVisualisation will be provided, showing the whole terminal with all its objects and processes. In parallel, dashboards for all types of objects will be provided, showing an overview of all devices of this type. By either clicking on the object in the 3D or selecting a special device in the dashboard, detailed information of the piece of equipment will be shown. Beyond this overview, monitoring data and time-lines may be selected and shown. Based on the CHESSCON Sim- and Emulation family, simulation applications - as described above will be implemented.

About the author

Holger Schuett combines academics, research and professional services in the field of port and terminal processes and IT. He grew up in Germany and received his Diploma in Applied Mathematics at the University of Hamburg. After he obtained his PhD in Automations Technology in the Technical University in Hamburg-Harburg he joined HHLA, the biggest terminal operator in Germany. In HHLA’s IT department he was responsible for the field of simulation and emulation for container terminal’s processes. The major project was the simulation based support for the fully automated terminal in HamburgAltenwerder (CTA). The emulation systems developed in 1999-2002 are still used for software releases as well as optimising terminal’s strategies. After CTA’s going live he joined 2003 the worldwide recognised Institute of Shipping Economics and Logistics (ISL). In 2010 he founded the commercial subsidiary ISL Applications GmbH, today known as akquinet port consulting GmbH. Also in 2010 he joined the University of Applied Science Bremerhaven as a professor for the mastercourse “Integrated Safety and Security Management” . In this position his main areas are the maritime logistics including port and terminal operation- and mathematical methods. In 2022 he retired from the university and the institute but is still working for akquinet port consulting.

*TheTwinSim project is funded by the BMDV (Federal Ministry of Digital and Transport) in Germany within the IHATEC programme.

RBS outline top trends to hit the headlines in 2022

WorldPortDevelopmentpresents...

New arising challenges are advancing the frontiers of technology capacity. Transforming current infrastructures to accommodate mass data has forced a significant push for innovation. Over the past year, we have seen real-life applications of machine learning in terminal planning and decision-making, as well as automation operating on a cloud TOS.This year, the port industry can expect to see changes in existing IT infrastructures to sustain this technology.

1. Artificial Intelligence (AI)/Optimisation It’ s quite difficult to top technology with an endless scope of limitless capabilities. Although we are still in the early stages of true AI / Optimisation, its intelligence of forecasting potential bottlenecks, determining optimal resource allocation, and calculating the most efficient berthing times unbound from inefficiencies, have emerged as a top function within any state-of-the-artTOS.In 2021, the port and terminal sector will strive to scale AI to boost performance while also differentiating their business value.

Augmenting container information complicating both internal and external communication, along with economic pressures to optimise resources, will gauge AI popularity amongst terminals. Recent global events have also driven a downturn in the economy, reinforcing AI as a viable solution to remedy their slump in operations and terminal activity.AsAI arrives at its next stage of a broader integration in the industry, we will witness a shift from input to outcome and quality focus.The sector can also expect to see new legislative reforms regarding the framework for the ethical use of AI. 2. 5G Data facilities are expanding to house the massive data growth while current network capabilities are striving to omit latency in transferring information. Subsequently, network providers have instituted plans of reconstructing network infrastructures to support this enormous demand.Another current practice of 5G in terminals involve regulating traffic light signals to facilitate consistent and safe transport flow. 5G is likely to be a strategy strengthening communication between terminal equipment and devices,TOS, and external staff such as customs and port authorities.

With the pool of network providers deploying 5G network becoming more widespread in 2021, the industry can expect an inter-relationship between 5G and IoT devices. Faster network speeds and enabling more concurrent connections will exercise the full potential of IoT in a terminal. Collecting and consolidating data will depict precision, inherently driving better decisionmaking from planning to forecasting.Therefore, terminals turning to 5G will transpire if their main goal is to improve on data transparency and becoming a smart port where every service is delivered on-time.

3. Cloud and Edge Computing Cloud computing has gained traction in the industry over the years for its power to run a full TOS entirely on the web at lower costs. However, due to the large magnitude of change required switching from an on-premise solution to a true cloud TOS, terminal operators veer from commitment.This year, it is about to change,as cloud server facilities are in the process of being built in theASEAN region. Cloud TOS accessibility heightens with localised server centres creating the opportunity for ports and terminals of any size to leverage autonomy in planning and processes.Therefore, cloud TOS will likely make a breakthrough in the Asian market in 2022.

Recent worldwide events have injected financial stress on the supply chain, straining the entire logistics chain and global economy. Consequently, the deficit in consumer demand has driven the port sector into a state of idleness. Now, there is a significant emphasis on cost savings which will conclude cloudTOS as the most reliable solution. Edge computing, in conjunction with automation on the cloud, is projected to make a breakthrough for increasing computing power.

Acting as a node with its computing, storage, and networking system enables immediate information access for accurate gathering and cleansing of data. Positive sentiments of the cloud are estimated to rise in the industry from IT managers devising a feasible strategy in response to the downturn in the economy.

4. Greener Technologies Duties associated with exercising corporate social responsibility will no longer be optional. The case is especially true, with many individuals finding it difficult to escape the wrath of climate change.As a result, terminals are transitioning towards carbon neutrality and renewable energy sources,while equipment providers are launching electric-powered container handling equipment. Environmental sustainability will take priority and be one of the key factors influencing decision-making within the port sector.

New greener equipment product release will expect to hit the market this year, as the entire industry is taking conscious steps towards reducing their carbon footprint. Numerous countries have entered the Paris Agreement, advocating positive change to avert stark temperature rises or have the economy implode into chaos.The industry can anticipate new government regulations enforcing strict protocol and guidelines which outline the threshold for carbon emissions and wastage. Establishing environmental responsibility will, therefore, highlight resource optimisation as a critical module within a TOS, attributing to a decrease in carbon emissions as well as cutting costs.

“Over the past year, we have seen real-life applications of machine learning in terminal planning and decision-making, as well as automation operating on a cloud TOS. It’ s quite difficult to top technology with an endless scope of limitless capabilities.

5. Automation This trend has been around in the industry for a while now and is not a new concept, but still is a very popular topic of discussion. A continuous increase in vessel sizes has engendered the realisation that operations can no longer be handled effectively and economically without incorporating any automation.This revolutionising technology has transformed conventional terminal operations and developed a large reception due to its ability to induce greater efficiencies in operations as a result of minimal to none human intervention.

Automation comes with many benefits for terminals, including – increases operation efficiency especially when paired with AI planning; reduces costs of planning and labour; increases environmental sustainability especially fuel consumption of CHEs; and enhances ROI with faster turnover rates. Automation can provide all these added benefits especially with a real-time TOS that can optimise decisions and moves within a terminal.

Technological advancements are fuelled by initiatives to develop better and more efficient strategies which accommodate the entire spectrum from the most mundane to the most complex tasks.This year we can expect to see advancements in AI capabilities, Blockchain, 5G applications, cloud operating with the edge, and individuals becoming more actionable on mitigating their carbon footprint.