Technology

Augmented reality in sight

By Anne Boswell

A study on augmented reality by Lincoln University could be a gamechanger for farmers using the technology in their farm system.

Agriculture has long been known as an industry at the forefront of innovation and technology – but the latest research undertaken by Lincoln Agritech takes this to a whole new level.

Working with DairyNZ, Lincoln Agritech is investigating augmented reality (AR) to see what difference it could make on dairy farms, including its potential benefits and limitations.

AR is a technology that uses equipment such as headsets, spectacles and earphones to add information to what a person can already perceive with their senses. In agricultural terms, it could provide instant health or performance data about an animal a farmer is looking at, or a paddock’s pasture cover and how much dry matter per cow it contains.

Lincoln Agritech, a multidisciplinary research and development company owned by Lincoln University, has approached key industry players such as farmers, farm consultants and veterinarians to ascertain what data they would find useful when this technology becomes available.

Their response was described by Lincoln Agritech precision agriculture scientist Abbas Jafari as “enthusiastic”.

From these findings, Lincoln Agritech selected three possible use cases to investigate. They are: • How effective is AR in providing information about paddocks to support grazing management? • How can AR help farm staff to learn how to use and maintain complex equipment such as tractors if reading through manuals is too time consuming? • How can AR improve staff training and consistency in body condition scoring of cows?

Group manager of precision agriculture at Lincoln Agritech Dr Armin Werner says finding out the value of this technology to farmers while its development is in a very early stage is critical.

“As researchers we need to quantify the benefits of this technology,” he says.

“For example, we needed to find out how long it would take an inexperienced staff member to complete a body condition score assessment on an animal using a booklet, versus using AR, so we can tell the farming industry whether this technology could benefit them sufficiently.”

Lincoln Agritech intern Joost Scholten, from Wageningen University & Research in the Netherlands, has worked with a Canterbury farmer to develop an app for a Hololens 2 headset that provides useful on-the-spot information about a paddock.

The paddock gate is marked with a QR code and when the farmer looks at the code through the headset, the AR equipment identifies and pulls up useful data about the paddock.

The next step in the research and development process is to work with a Lincoln University dairy farm to quantify how AR can help with cow body condition scoring.

Werner says the benefit to farmers using AR is potentially vast.

“Farmers will be able to access information on whatever part of the farm – whether it be paddocks, animals, or farm equipment – faster, easier and more conveniently, as and where needed,” he says.

“What we also consider a game-changer is the handsfree nature of AR. Currently, staff members arrive at a paddock on a vehicle and have to stop, pull their phone

An example of paddock data shown in the AR headset after the farmer has looked at the paddock’s QR code at the gate.

Lincoln Agritech precision agriculture scientist Abbas Jafari is one of the team investigating augmented reality for use on dairy farms.

“Farmers will be able to access information on whatever part of the farm – whether it be paddocks, animals, or farm equipment – faster, easier and more conveniently, as and where needed.”

Dr Armin Werner

or tablet out and pull up the information they require, which is often on different databases.

“With AR they will be able to ask the system to pull up any information they require, without having to search for it.”

Werner says they hope to have enough data by the end of the year to create a prototypical solution in a new project for industry to test.

“By then we will know whether there is a need to drill deeper into the research and development, and what the next step will be,” he says. “Then there will be the possibility of creating a prototype farmers will use for 6-12 months in a real-life situation.”

Werner says it is important for people to understand that they are currently in a very early stage of the development process.

“The clunky hardware that farmers wear in the preliminary stages is not what the future of AR gadgetry will look like, but it is a very important starting point,” he says. “Eventually it will be a pair of spectacles like many of us already wear, and even a contact lens.

“What influential farmers are telling us is that although they know the technology is in its infancy, they are excited to support us.

“We are in a transition to these future technologies but what we are doing now is preparing the industry so when these technologies are available, they will have the training and knowledge to use them.”

Senior scientist at DairyNZ Callum Eastwood, who initiated and supervises the project, says the researchers from Lincoln Agritech and DairyNZ have “shown the possibilities of AR for dairy farming are really exciting”. n