AI on Cruise Ships: The fascinating ways Royal Caribbean uses facial recognition and machine vision

Where entry requires verifying the identity of an individual, the inevitable result is queuing and chokepoints. AI and facial recognition technologies, writes internationally bestselling author and futurist Bernard Marr, can keep things flowing.

In the travel industry, the primary use cases for artificial intelligence (AI) and machine learning technologies revolve around improving customer experiences.

Chatbots, in particular, have proven popular across this industry, with natural language processing (NLP) applied to the challenges of dealing with customer inquiries and providing personalised travel experiences.

Alongside this, recommendation engines power the most popular online travel portals such as Expedia and Trivago, combining customer data with information on millions of hotel vacancies and airline flights worldwide.

However, when it comes to operators, compared to other industries such as finance or healthcare, the travel industry as a whole is at an early stage when it comes to organisation-wide deployment of smart, self-learning machine technology.

One industry leader that is bucking this trend, though, is cruise operator Royal Caribbean Cruises. In recent years, the world’s second-largest cruise operator has put AI to use to solve several problems.

As far as customer experience is concerned, the overriding goal has been to remove the “friction” often experienced. Until recently, this was seen as an inevitable consequence of having to check in a large number of passengers at a single departure time, rather than deal with a continuous flow of guests arriving and departing, as at a hotel or resort.

The company’s SVP of digital, Jay Schneider, tells me “Our goal was to allow our customers to get ‘from car to bar’ in less than 10 minutes.

“Traditionally it would take 60 to 90 minutes to go through the process of boarding a ship, and as a result, people didn’t feel like they were on vacation until day two – we wanted to give them their first day back.”

A vital tool in achieving this aim was the deployment of facial recognition technology. It uses computer-vision equipped cameras that can recognise travellers as they board, cutting down the need for verifying identity documents and travel passes manually.

This could have been done by providing customers with wearables such as a wrist band; however, the decision was taken to eliminate the need for external devices by using biometric identifiers – faces.

“We wanted to get people on their vacations as quickly as possible, and we didn’t want to have to ship every passenger a wearable – we want you to use the wearable you already have, which is your face.”

Computer vision-equipped cameras are built into the terminals that customers interact with as they board, and sophisticated algorithms match the visual data they capture with photographic identification which is submitted before their departure date.

AI doesn’t stop improving customer experience once guests are on board. Several other initiatives are designed to make passengers more comfortable or help them make the most of their time. These range from personalised recommendations for how they should spend their time on board, to monitoring and managing footfall as people move around the boat and queue to make use of services.

These monitoring systems are also powered by computer vision, but rather than recognising individual faces, they monitor the build-up of bodies as passengers move about, allowing congestion to be detected and dealt with where necessary.

The technology for this application was built in partnership with Microsoft, and involved retro-fitting existing CCTV cameras with smart technology. This avoided the need for ships to be taken out of action while the entire camera network was upgraded with computer vision cameras.

“We have massive ships – we didn’t want to take them out of service, gut them and put sensors in, so we worked with Microsoft to understand how we could leverage our existing and somewhat antiquated CCTV cameras.

“Microsoft was a great partner … we threw our data scientists at the problem, and we’ve been able to take old cameras, as well as fisheye cameras, and detect humans through the use of AI.

“There’s a tonne of use cases – it gives us information on things like table turnover times in restaurants, and we’re going to start using it from this summer to alert crew members when lines are backing up.”

This will mean crew can be redeployed in real time to wherever their services are in demand.

Another initiative is aimed at cutting down on food that goes to waste on board cruise liners. With 65,000 plates of food served daily aboard the vessel Symphony of the Seas, AI helps make decisions about how much food should be stocked to ensure guests don’t go hungry while keeping wastage to a minimum.

“We like to think we’re probably the most sustainability-friendly cruise line – and one of the things we’ve focused on when deploying AI is working towards our goals of improving sustainability. Outside of the cost savings, and improved freshness of the food we serve, it has sustainability benefits … we’ve seen a reduction in food waste as a result of this pilot, ” says Schneider.

The most recent application – which began trials just weeks ago – is Royal Caribbean’s chatbot, styled as a virtual concierge, which allows passengers to ask questions about their voyage, destinations, or how they should spend their time on board.

“The whole idea, again, is to pull people out of lines – we don’t want passengers waiting in line at guest services to get questions answered, we want them to be able to get the information they need right away, ” Schneider tells me.

The chatbot employs NLP and machine learning to understand what the most commonly asked questions are, and become more efficient at providing personalised answers. It uses a “human-inthe-loop” model, meaning that if it can’t work out what a customer wants, a human customer service agent is paged into the conversation. The NLP algorithms are then capable of learning how they could have tackled the question, by monitoring the human agent’s response.

With this, as with its other AI initiatives, Royal Caribbean follows a model of carefully monitored, smallscale trial deployments, before individual initiatives are put into organisation-wide use.

Schneider tells me “We believe we get the best results with this method … test, adjust, scale … rather than ‘ready, fire, aim’ – which the rest of our industry seems to do! So, once we’ve carefully tested it and we’re sure it’s ready to go, we will scale it.”

When it comes to gathering data, cruise operators like Royal Caribbean are in a unique position, as they effectively function as hotels, food and beverage providers, supply chain and logistics operations, shipping operators and entertainment and gaming companies, all rolled into one.

This means customer journeys can be tracked and data gathered across all of these functions, enabling a holistic approach to data-driven customer service.

“As you can imagine, ” Schneider says, “there are any number of opportunities … we’ve focused on yield management in cabin occupancy … the list goes on.

“We’re focused on testing, adjusting and scaling examples of where we can use AI to change the customer and the crew experience. Not everything has been successful, but the vast majority have shown early signs of success, and we’ve been extremely thrilled with the results so far.”