Mission Critical: Agriculture, Spring Edition

V O L U M E 2 N O . 1 • S P R I N G 2 0 1 2 • A U V S I • 2 7 0 0 S o u t h Q u i n c y S t r e e t , S u i t e 4 0 0 , A r l i n g t o n , VA 2 2 2 0 6 , U S A

Agriculture

Inside this issue:

John Deere’s autonomous tech Detecting microbial invasions with UAVs Japan’s aging farmers turn to robotics Mission Critical

•

Spring 2012

1

Promoting and Supporting Unmanned Systems and Robotics Across the Globe

NOW symposium.auvsi.org

LAS VEGAS

6 – 9 AUGUST auvsishow.org

Conference from 6 - 9 August Tradeshow from 7 - 9 August 500+ Exhibiting Companies 40+ Countries Represented 8,000+ Attendees

REGISTER TODAY! ®

CONTENTS V O L U M E 2 N O . 1 • S P R I N G 2 0 1 2

8

Green giant

Farming mainstay John Deere adds intelligence to many of its vehicles, spanning every area from orchards to the military

4 Essential

16 State of the art

A map of where the seeds have been sown in robotic farming

components

The latest in agriculture robotics

26 Q & A

28 Timeline

A history of technology at John Deere

An U.S. Department of Ag expert talks on the National Robotics Initiative

On the cover: A quadrotor with a multispectral imaging sensor from the University of Florida looks for citrus greening, a disease that ravages local plants. Read more on Page 36. Photo courtesy Reza Ehsani.

Mission Critical

•

Spring 2012

1

30 Market report Potential game changers for the slow but steady growth in high-tech ag

36 Uncanny valley For aerial crop imaging to jump, the FAA has to say how high to fly

38 Testing, testing Carnegie Mellon looks to automate specialty crop farming

18 A blighty wind Virginia Tech researchers are using UAVs to track blankets of microbial invaders that affect everything from plants to animals to even human health

41 Technology gap Needed: better sensors, planning

42 Spotlight Mack Robotics’ BinBot could save lives and litigation

44 End users Keenan Amundsen of the University of Nebraska - Lincoln

31 Far East of Eden When it comes to pairing robotics and agriculture, Japan is the cream of the crop

Mission Critical is published four times a year as an official publication of the Association for Unmanned Vehicle Systems International. Contents of the articles are the sole opinions of the authors and do not necessarily express the policies or opinion of the publisher, editor, AUVSI or any entity of the U.S. government. Materials may not be reproduced without written permission. All advertising will be subject to publisher’s approval and advertisers will agree to indemnify and relieve publisher of loss or claims resulting from advertising contents. Annual subscription and back issue/reprint requests may be addressed to AUVSI.

2

Mission Critical

•

Spring 2012

Editor’s message Editorial Vice President of Communications and Publications, Editor Brett Davis davis@auvsi.org Managing Editor Danielle Lucey lucey@auvsi.org

W

elcome to the second year of Mission Critical magazine, our now not-so-new quarterly publication that

embraces a different aspect of robotics in fields that have spanned driverless cars to

Associate Editor Stephanie Levy slevy@auvsi.org

home and health care.

Contributing Writer Liz Ruskin

Advertising Senior Advertising and Marketing Manager Lisa Fick fick@auvsi.org +1 571 255 7779

botic agriculture. Many of us are raised be-

in robotics, beginning with their Machine

lieving that farming is a generations-old art.

Sync technology, which turns tractors into

While this isn’t false, the farms of the 21st

autonomous machines. The company is

century are full of high-tech gear, like so-

field-testing its future intelligent farming

phisticated irrigation systems or genetically

exploits through the Autonomous Orchard

engineered disease-resistant plants.

Project. The company aims to automate

While automating the farm is still a niche market in the United States, Japan has fully embraced the concept, becoming a giant in robotic farming.

using ground robots to farm the country’s prized strawberry harvest and how more forgiving airspace access laws allow farmers to crop dust over rice fields. In a country where the average farmer is nearly 66 years old, robotics has become more of a necessity than a trendy tech movement for

Executive Vice President Gretchen West AUVSI Headquarters 2700 South Quincy Street, Suite 400 Arlington, VA 22206 USA +1 703 845 9671 info@auvsi.org www.auvsi.org

Mission Critical Associate Editor Stephanie Levy explores the company’s endeavors

Liz Ruskin looks at how the Asian nation is

President and CEO Michael Toscano

ricultural giant John Deere. On Page 31,

This edition takes us to a literal field — ro-

On Page 8, Mission Critical contributor

A publication of

Danielle Lucey

the Land of the Rising Sun.

labor-intensive tasks in all kinds of weather and lighting conditions, so work can still persist all the way up to the next rooster’s crow.

In a feature found on Page 18, I spoke with Virginia Tech crop biosecurity guru David Schmale, whose interdisciplinary work uses autonomous planes to track the movement of a vicious bacteria that affects plant and animal life on farms, causing blights in the fields and even possibly death to livestock.

By tracking this genus of bacteria, which travels in layers in the atmosphere, sometimes from as far away as other countries, farmers could one day potentially prevent

Also, for Japan, a robot-filled story of the

diseases to their livelihood by doing pre-

heartbreaking aftermath of the 2011 earth-

ventative fungicide spraying.

quake and tsunami is coming full circle. A year ago, the country used unmanned systems to monitor the aftermath and search for victims of the disaster, but now, through a new program from the government, Japan intends to use the ravaged shores of Sendai and transform them in into the proverbial Elysian Fields for robotic farming.

Flying the flag for the United States is ag-

The health and productivity of agriculture affects us all, from the price we pay for milk at the grocery store to our own personal health. Delving into agriculture has shed a lot of light on how important it is to ensure this sector is thriving, and securing that future safer, cheaper and easier with robotics is a real possibility. We hope you enjoy learning about it as much as we have.

Mission Critical

•

Spring 2012

3

Essential Components Bovine bonanza: Lely sells 12,500th cow milking robot Lely, a Dutch farm technology company, recently delivered its 12,500th cow-milking robot, which equals hundreds upon thousands of cows that produce millions of gallons of milk a day — numbers increased because of the technology.

SCAN IT

or

Click IT:

To see the winning video, click here or scan this barcode with your smartphone

The Astronaut A4 milking system is essentially a station that cows walk into, where then 3-D cameras target a cow’s udders and then the machine proceeds to milk it.

Afterward, brushes clean the cow and a feeding trough emerges to coax the cow out of the stall.

The Astronaut A4 also treats the milk in the system, ensuring a quality product through a Milk Quality Control sensor located in the arm of the robot, which performs the milking, says Lely. It measures things like protein, fat, lactose and color.

The company says that since the machine uses a “dynamic herd approach,” which predicts how each cow will contribute to the total milk yield, this method increases total yield by 10 to 15 percent versus farms that milk twice a day.

Lely says the robotic system is affordable to any dairy

A cow feeds while getting milked by the Lely Astronaut A4. Photo courtesy Lely.

farm that produces more than 300,000 liters, or 79,252 gallons, of milk per year.

To promote the sale of the Astronaut A4 to its 12,500 customer, the company created a video competition for users to show how the system has impacted their farm.

misread what is actually present on a farm because of the time lapse it takes to create an image. What from the satellite’s point of view may look like an unauthorized building might actually be an innocent haystack.

Testing of the UAVs is occurring in vineyards in the south of France. Wine

Spying the farm: UAVs may soon check up on EU farm subsidy claims

growers can claim about €10,000 per hectare in subsidies. The tests are try-

As a way to check up on who is, and more importantly isn’t, following up on the European Union’s Common Agricultural Policy rules, the EU is turning to satellites and unmanned systems to view farmland from above.

The use of satellites is a few years old, but unmanned aerial systems are currently undergoing trials for the EU to use. The use of UAS would help in countries like Austria, which currently does not use satellites to scan farms because its mountainous terrain casts shadows that engulf a lot of farmland. Satellites can also often

4

Mission Critical

•

Spring 2012

SCAN IT

or

Click IT:

To read a paper from a February European Commission workshop on the matter, scan this barcode with your smartphone. http://www.eda.europa.eu/Libraries/

Essential Components

Strawberry fields for robots: UK’s NPL studies autonomous berry picking Scientists at the U.K.’s National Physical Laboratory have developed an imaging technology that can identify the ripeness of strawberries before they are picked.

This leader-follower shopping cart completes the robotic agriculture cycle in the grocery store. Photo courtesy Chaotic Moon.

Researchers hope to use the technology to develop a new crop-picking robot.

NPL’s technology uses radio frequencies,

ing to determine if the UAVs can have up to accuracy required.

Farm to table: Whole Foods tests follow-me shopping cart

Other tests are occurring in Italy and in

Mobile application studio company Chaotic

Spain’s Catalonia region.

Moon Labs has developed an Xbox Kinect-

10 centimeters’ resolution to determine the

Scanning a farm with a satellite currently costs about one-third of the price of using an inspector on a field visit, according to

powered shopping cart that is designed to

Scientists at NPL say the efficiency of this technology can bring down the cost and waste of harvesting fragile produce.

First, the microwave measurement system maps the structure of the strawberry. This

Agency (RPA), which checks on the status

from an outgrowth of the company’s Board

of subsidy claims. The effort is to ensure

of Awesomeness product — a motion con-

farmers aren’t claiming any subsidy fraud.

trol motorized skateboard — and the larger Board of Imagination.

where there is some doubt about accura-

The similarity with the products lies in the

cy, and then only at the specific fields for

Xbox Kinect sensor they use, which is

which the doubt exists,” an RPA spokesper-

synced with a Windows 8 tablet for motion

son said in an interview with the BBC. “This

control.

reduces cost to the taxpayer.”

can also learn based on past experiences.

they’re shopping.

Called the Smarter Cart, the creation came

saves time, lifts the burden on farmers and

tion to determine ripeness. The software

play follow-the-leader with a person while

numbers from the U.K.’s Rural Payments

“The RPA follows up only on those claims

microwaves, terahertz and infrared detec-

The cart can also track items by scanning barcodes, ensuring the product matches a

data then plugs into an algorithm that allows the technology to determine information about ripeness from a single measurement.

Researchers tested the new technology on strawberries because of the demand for the fruit. They had previously run tests of the equipment on cauliflower, but had to call it off when demand for cauliflower dropped.

“The focus now is strawberries. This is a fairly easy fruit to measure as it has high water content and dry leaves, and micro-

Each EU country inspects about 5 percent

shopper’s list, and it can detect if, for in-

of its farms per year, with many more

stance, an item is not gluten free when that

checking more than that.

has been specified as a parameter.

All this UAV research could play well for the

Whole Foods used its flagship location in

“Strawberries are also a high-value fruit that

future of the systems in Europe, since the EU

Austin, Texas, to test a cart this past Feb-

are very time-consuming to pick, so there is

is currently strategizing if it is possible to

ruary using about 30 shoppers. Chaotic

a stronger business case to implement auto-

loosen the restrictions of UAVs in civil ap-

Moon aims to test more than one cart at a

mated picking technology for strawberries

plications.

time beginning in April.

than with some other crops.”

wave imaging is particularly useful for identifying water levels,” lead scientist Dr. Richard Dudley said in a press release.

Mission Critical

•

Spring 2012

5

Essential Components — continued from Page 5

Pint-size propellers: The making of lightweight agricultural robots The robotics team at Queensland University of Technology in Australia is working on ways to make small, lightweight robots that cooperate in teams, or swarms, to survey large swaths of fields and apply herbicide.

Their goal is to make a swarm of robots with cameras and advanced navigation capabilities that can cover large areas and resupply themselves when necessary. Dr. Ben Upcroft, faculty member at QUT, says the camera on the robot would have image recognition software to “pick out features of the weed which make it different from the rest of the crop.”

The three-year project, which recently received nearly $400,000 in funding from the Australian Research Council, also includes the University of Sydney and Queenslandbased Advanced Agricultural Systems, run

by Andrew Bate.

Bate, who has a grain and cattle farm at Bendee, southwest of Emerald in central Queensland, said the automation of agriculture was a new frontier.

“We’ve already reached peak farmland, so we have to figure out smarter farming systems which increase yield in a more cost-effective and environmentally sustainable way,” Bate said. “Every other industry is already enjoying the benefits of robotics.

This is the revolution farming has to have.”

Harvest Automation secures $7.8 million in funding for plant-carrying robots

don and Tel Aviv, along with backing from existing investors Founder Collective, Life Sciences Partners, Cultivian Ventures and the Massachusetts Technology Development Corp.

Leptron’s Avenger available for ag use Recently receiving a lot of press because of a purchase by Arlington, Texas, police, Utah company Leptron Industrial Robotic

“I’m extremely excited to have the opportunity to work with … Entrée, who I know will bring significant value to Harvest. The strong support in this round from our existing investors is also a great indicator of the positive direction we have set and the collective enthusiasm for the future of the company,” says Charles Grinnell, CEO of Harvest Automation.

Helicopters’ Avenger platform is being promoted for use in agricultural applications.

Available with commercial, off the shelf camera turrets, HD video, infrared and night-vision cameras from FLIR, synthetic aperture radar, and LIDAR packages, the 2-mile-range system could aid in mapping erosion, floods and other land management necessities, says the company.

The earliest prototypes of the robot were developed in 2008 with the company’s initial funding sponsors.

The electric platform was made in conjunction with Guided Systems Technologies, and it can either fly autonomously or

The small robots, about knee height, have a gripper for holding onto a potted plant and place the plants on a deck to carry the pot.

Sensors on the vehicle let the robot know its surroundings, though they don’t use GPS or

through operator control with a joystick.

The platform can fly for about 30 minutes and can sustain in 40 mph winds at up to 12,000 feet.

map their surroundings.

While the robots currently do not have a price, Harvest Automation is aiming for the machines to run between $25,000 and $50,000.

In the summer of 2011, the company tested three of the robots at 11 nurseries and greenhouses across the United States.

“All 11 growers gave us deposits on their first purchases, which was really encouraging,” said Grinnell, chief executive of Harvest Automation, in an interview with The Boston Globe.

North Billerica, Mass., company Harvest Automation recently secured $7.8 million in funding to continue their work on prototype robotic farm equipment that would maneuver plants around nurseries and greenhouses.

The company’s latest round of funding comes from Entrée Capital, based in Lon-

6

Mission Critical

•

Spring 2012

SCAN IT

or

Click IT:

Click this photo or scan this barcode with your smartphone to see Leptron’s Avenger perform grass reclamation work.

Driverless Car

sUMMiT

RegisteR Now! auvsi.org/dcs

12-13 June 2012

MotorCity Casino Hotel • Detroit

Driverless Cars in 2022‌

Making the Vision a Reality

Driverless Car Summit 2012 will bring together leaders from the

FACT

llion uld remove 112 mi Driverless cars co ar ye ch ea ad ro m the drunk drivers fro

FACT

The average car is imm obile for 96% of its lifetime

robotics and automotive communities with a common goal of making driverless cars a reality by 2022.

The event will feature a mixture of inspiring presentations and interactive discussions designed to develop the path toward making driverless vehicles

FACT

a mainstream mode of transportation.

in In 2010, 35,000 people died in car accidents If car 2009. in 0 36,00 from the United States, down m deaths were put on a wall the size of the Vietna . years two every lt rebui be to have Memorial, it would

FACT

Maximum capacity of a highway today: 2,400 vehicles per hour. Driverless cars could more than double that using existing infrastructure.

Supported by:

John Deere answers customer demand for more autonomous agriculture

By Stephanie Levy

I

f you harvest your crops using ra-

“Operators can see the location of all grain

dealership as they set up the new equip-

dio-operated combines, automated

carts in the network and, from the combine,

ment. Moorehead says the learning curve

tractors and the latest GPS technology, you

they can send a ready-to-unload request,”

for the new equipment is not steep. End us-

just might be a robot.

John Deere’s Agricultural Management

ers of Machine Sync, and other automated

Solutions manager Holli Brokaw said in

John Deere products already on the market,

an August 2011 press release highlight-

range from large industrial farms to owner/

ing the technology. “This information en-

operator farms.

And you may not realize it, but those technologies came to the farm through research across platforms in the agricultural, commercial and military sectors.

ables operators and managers to make the best decisions to maximize the harvesting

John Deere has provided technology solu-

fleet, while reducing fuel consumption and

tions in agriculture for 175 years (for more

ground compaction.”

information, see this issue’s Timeline on Page 28). Recently, those solutions have included bringing more automation to the farming process. The company’s transition has mirrored that of the auto industry: John Deere started adding driver assistance technologies to its tractors, but now a boom in automation development and demand may plant the seed for fully autonomous robots

“Since the [farmer] who’s buying this is the guy who’s sitting in the tractor, he has a lot of interest in improving his comfort level, whereas in a factory farm, perhaps the guy

“You can put someone like myself in

buying it isn’t the guy driving the tractor or

there and it will still do well,” says Stew-

the truck,” Moorehead says.

art Moorehead, robotics engineer at John Deere’s Intelligent Mobile Equipment division. “With this coordination by the radio allowing the vehicles to talk to each other and know where the other vehicles are, you can really reduce your downtime.”

Orange you glad it’s autonomous?

To reach the next level of autonomy in agriculture, John Deere launched the Autonomous Orchard Project with the University

Reduced downtime leads to increased ef-

of Florida, Carnegie Mellon University and

ficiency, which Moorehead says accom-

Cornell University. The project currently

Machine Sync goes to market

plishes the end goal of lower costs.

boasts two tractors operating in Florida, as

In the summer of 2011, John Deere rolled

stop the combine and wait in the field for

out its Machine Sync technology. This sys-

a tractor to come so it can unload grain on

tem will drive a tractor autonomously while

them,” Moorehead says. “When it’s time to

a combine unloads grain, or other harvest

harvest this crop you’d like to get it har-

yield, into the wagon. Radios in the com-

vested as quickly as possible, otherwise the

“Tractors are the workhorses of the mod-

bine — which stores the crop during har-

weather might change. Once conditions

ern farm,” writes Anthony Stentz, Cristian

vesting — allow the combine to “talk” to the

are right for the harvest they really want to

Dima, Carl Wellington, Herman Herman

tractor during this operation. All the while,

get it done and get it done now.”

and David Stager in their 2002 paper

on and off the farm.

farmers know where all the combines are located in the field and how full they are.

8

Mission Critical

•

Spring 2012

“It’s very expensive for these farmers to

Buyers of Machine Sync will receive training from workers at their local John Deere

well as 10- and 15-foot mowers. Moorehead showcased the Autonomous Orchard Project as part of his presentation at the AUVSI-NDIA

Transportation

Technology

Transfer Initiative in May 2011.

about the Autonomous Orchard Project, published in Autonomous Robots. “By automating these machines, we can increase

the productivity, improve safety and reduce

and ditches in the field. If one of these ob-

The advantages of orchard automation are

costs for many agricultural operations.”

stacles is detected, the vehicle computes a

logistical and environmental. Improved lo-

safe speed to gradually slow to a stop. In

gistics lead to increased productivity; for

a sprayer capacity alone, the autonomous

example, an autonomous tractor can con-

tractor can follow a path of seven kilome-

duct day and night operations while driving

ters at speeds ranging from 5 to 8 kilome-

at a constant speed. This automation also

ters per hour.

removes people from chemical spraying

The purpose of the Autonomous Orchard Project is to increase the productivity of specialty crop producers by automating labor-intensive tasks in the field. Initially, the project just covered mowing and spraying

and reduces the reliance on fuel and harm-

tasks in the orange groves. By using a force

Moorehead said in his presentation that

multiplication approach, autonomous trac-

it’s key the autonomous farm vehicles de-

tors are capable of driving in a structured

tect and interact with “real obstacles.” That

“That would be good to remove people

orchard environment for most of the time. If

means the tractor can’t stop every time a

from those activities,” Moorehead says.

necessary, they can call a remote supervi-

branch, weed or stray orange falls in its

sor for help via wireless radio.

path. When the tractor stops for a detected

Using multiple sensors — a SICK laser, color and thermal cameras, and a MA/COM radar — the tractors in the Autonomous Orchard Project can detect people, vehicles

obstacle, it informs the remote supervisor who can then examine the data to determine if a real obstacle is actually there. Before motion resumes, the operator verifies that the way forward is clear.

ful chemicals.

The Autonomous Orchard Project continues to work on improved path planning and perception for the vehicles and all involved hope to get more customer use feedback.

Better perception and path planning means making the tractors adaptable to weather

John Deere’s new Machine Sync brings higher levels of autonomy to the harvesting process. Moorehead says more automation leads to “better performance, less spilled grain because of poor station keeping and greater safety.” All photos courtesy of John Deere.

Mission Critical

•

Spring 2012

9

Video cameras on the tractors from the Autonomous Orchard project show obstacles at all angles. And while the tractor may move slowly, it gives the operator more time to work with the machine.

a traditional manned tractor to get the best automated efficiency.

“It may be hard to imagine, but as you’re driving this tractor you’re trying to drive in this very straight line,” Moorehead says.

“You’re trying to drive very close to where you drove the previous path, but you also have to turn around to make sure the impleThe Autonomous Orchard Project takes place at the University of Florida’s Institute of Food and Agricultural Sciences Extension.

ment is okay. It’s actually quite a stressful activity.”

Now stretch that process out over the course of a 10-14 hour day. It’s enough to make anyone tired. Fatigue in the field can

variants like dust, fog, rain and snow.

Moorehead says these next steps should take about three years.

hind the wheel, but a GPS map of the field

lead to manual errors and headland incon-

helps steer the vehicle.

sistencies, especially on irregularly shaped

John Deere’s Intelligent Total Equipment Control, or iTEC Pro, complements AutoTrac. Available on two different tractor

fields of crops. AutoTrac and iTEC Pro reduce or eliminate these risks to efficiency, safety and cost.

In Europe, John Deere is merging its work in

models, iTEC Pro coordinates the turns and

the agriculture and commercial sectors with

“You reduce your fuel usage, chemical us-

subsequent functions of the vehicle. It also

a new autonomous lawn mower. Tango,

age, seed usage, and at the end of the day

positions the tractor and implement cor-

debuted in 2011, is a robotic lawn mower

this all translates into money, but it has oth-

rectly for the next pass through the field.

designed specifically for homeowner use.

er advantages as well,” Moorehead says.

Moorehead recommends installing both on

“The company farms are in tune to this.”

Mark Bodwell, manager of military utility vehicle business at John Deere, says Tango has not yet been introduced in the U.S. because there is already a large commercial

The tractors in the Autonomous Orchard Project have been used in research and development for 10 years.

market for manned lawn care that doesn’t exist in Europe.

“There are a lot of people, a lot of companies you can call in the United States that will mow your lawn for you,” Bodwell explains. “They don’t have that in Europe.

Plus, they have smaller lawns.”

These projects aren’t John Deere’s first foray into autonomous farming. In 2001, the company released its AutoTrac system for agriculture. AutoTrac automates tractor driving in the field and controls the implements on board. There is still a person be-

10

Mission Critical

•

Spring 2012

SCAN IT

or

Click IT:

Click or scan this barcode with your smartphone to read Stentz, Dima, Wellington, Herman and Stager’s Autonomous Orchard Project paper.

A soldier uses a video game console to teleoperate the R-Gator.

SCAN IT

or

Click IT:

Click or scan this barcode with your smartphone to see the R-Gator in action

Gator’s growl

The Gator has a 14-pound adjustable pay-

robotics, its initial offerings were the kinds

Off the farm, John Deere has developed

load capacity and four different levels of

of tools engineers and nerds dream of —

autonomy. At its most basic, the warfighter

“a laptop with all kinds of demands and

can drive the Gator like a regular car. Next,

screens full of numbers,” as Moorehead de-

the driver in the Gator can record a path for

scribes it. John Deere had to strip down the

the vehicle while driving it. At the third level

interface to make it more user friendly.

unmanned ground vehicles for military use.

The company’s Gator vehicle, unveiled in 2004, has already become a popular UGV thanks to its simple user interface and the military’s reputation as an early adopter.

John Deere unveiled the A3 version of the R-Gator at the Association of the U.S. Army

of control, Gator is teleoperational. Finally, the vehicle can be operated with total autonomy. To do all this, the Gator interface

convention in October 2011.

resembles that of a video game. John Deere

“The military has some really challeng-

warfighter straps it to an existing combat

ing problems,” Moorehead says. “Robotics may provide solutions for keeping the soldier out of harm’s way, resupplying the soldiers and helping with reconnaissance.”

also made it portable and wearable. The vest. Bodwell says these capabilities make

“It was a great way for us to take some of the autonomous components we’ve been working on in agriculture and in turf care and apply them to a domain that was, in some ways, more challenging and more rugged,” Moorehead says.

the Gator especially useful for obstacle de-

The Gator has been used in the U.S. Army

tection and avoidance.

as a test vehicle or surrogate for other un-

But when John Deere got its start in military

manned programs, and the Navy has used

Mission Critical

•

Spring 2012

11

John Deere — continued from Page 11 Gator for tasks like perimeter security.

example, lessons learned in fuel saving

start to integrate path planning into find-

Bodwell says NASA also got its hands on

techniques for autonomous farm vehicles

ing more optimal ways to work the field,”

a Gator to use as a surrogate for the Mars

have been applied to John Deere’s work on

Bodwell says.

rover program.

newer models of the Gator.

In 2011, John Deere also partnered with

“Many of these technologies that are very

Boeing to work on the assisted carriage

necessary in robotics — things like path

system program with the British military.

planning that can directly translate into

Bodwell says U.K. soldiers used Gator for

some of these guidance programs — we

a demonstration of their version of the U.S.

“Robotics is about optimization and about finding the shortest path.”

Stephanie Levy is associate editor of Mission Critical.

military’s lighten-the-load tasks.

“The biggest surprise I’ve seen working inside the military is the surprise on people’s faces when they understand the technology that John Deere actually provides,” Bodwell says.

For More Information: http://www.deere.com http://www.deere.co.uk/wps/dcom/en_GB/products/ equipment/autonomous_mower/autonomous_mower.page

Farm to (drawing) table “We don’t view it in our company as: We do this robotics for agriculture and we do

http://www.rec.ri.cmu.edu/projects/auto_spraying/ http://edis.ifas.ufl.edu/ae467

this robotics for the military,” Moorehead says. “For us it’s all robotics, and there’s a very rich transfer of knowledge, lessons learned and technology between our tractor program and the military program.”

Starting small, Bodwell says the planning and algorithms that go into John Deere software apply to both military and agricultural robotics. But in putting together autonomous vehicles, equal examples of technology transfer have happened from agriculture to military and vice versa.

“The motor controller we use for steering [R-Gator] is the same motor controller we use for turf,” Bodwell says. “The vehicle control unit comes off our hay baler. A lot of the actuators we use on the products come off other products within the John Deere enterprise or inventory.”

John

Deere’s

technology

transfer between its military and agricultural robotic platforms can solve common problems across the industries. For

12

Mission Critical

•

Spring 2012

John Deere’s R-Gator.

Putting the cart before the horse: John Deere’s technological evolution

nate human operators in the farming equipment itself.

A

s John Deere celebrates its 175th anniversary this year, the company is “an old company on the edge of technology trying to lead this industry,” according to Darren Havens, division manager of marketing for John Deere.

Speaking to an agriculture-oriented conference last fall sponsored

by AUVSI’s Atlanta Chapter, Havens pointed out that wasn’t always the case.

“We have internal communications where we said, ‘Yeah, we’re going to offer the tractor, but still the horse and plow is the way to go,’” Havens said to big laughs from the audience. “It’s hard for people to get their heads around.”

Now, Havens said the same thing is happening with the introduction of unmanned systems into the agriculture industry. John Deere

“The first ones are really tangible machines that, logistically, people can get their heads around,” Havens said.

John Deere’s biggest challenges involve the use of data, and what it can mean for an end user just bringing automation to the field. John Deere is already working to manage ergonomic data. As a leader in technological development in farming, John Deere has taken on the responsibility of getting customers to understand data, simplifying the data process and standardizing data in the agriculture industry.

“The ability to make that seamless, easy and executable as they go into the next planning season is what they need and desire,” Havens said.

For example, there is currently very little consistency in data standards for autonomous systems in agriculture. Havens said the easiest way to fix that — so farmers can work from system to system or company to

We have internal communications where we said, ‘Yeah, we’re going to offer the tractor, but still the horse and plow is the way to go.’ — Darren Harris, John Deere marketing manager kick-started this movement in 1997 when it purchased the company NavCom. NavCom offers global navigation satellite systems and sensors for agriculture, as well as for undersea exploration.

company — is to make standard prescriptions for the machine’s data.

The goal is to create a software development kit that is as user-friendly as an app on an iPhone.

These sensor capabilities have translated to more efficiency in the field. Havens said that while the technology may still be confusing for the end user, this efficiency, and the results it produces, is something they can get behind.

Havens also discussed the ways in which unmanned aerial vehicles could be used in an agriculture setting. Specifically, in order to effectively use UAVs in the field, Havens said John Deere, and other agbased companies, have to think about who the customer is, what the cost/value benefit would be for using UAVs in the field and the data insights that can be gleaned from UAVs that are not already available through traditional methods. Havens said convincing customers of the cost/value benefit of UAVs will probably prove to be the most difficult challenge.

“This was the ‘aha’ moment for a lot of customers, when they could jump into a vehicle or tractor and it goes on its own,” he said.

“Customers are getting very attuned to that precise placement of whatever they’re doing.”

As a next step, John Deere is looking to coordination of technology through telematics, or the integrated use of telecommunications and informatics, for their next phase efforts. Havens explained it as a piece of technology’s ability to communicate two ways. In 2011, every John Deere tractor, combine and sprayer that went out on the market was equipped with this telematics technology. Havens said this allowed farmers to “monitor that machine’s performance and logistics and manage the data flow from system to system easily.”

For instance, John Deere took its first steps in telematics when it eliminated some of the paperwork necessary to get machines to do a certain type of pass in the field while harvesting. Now, a machine can execute its job in the field, and not only does the farmer know where the machine is at all times, but it also sends a work order back when the task is complete. Ultimately, this technology could be used in conjunction with AutoTrac and iTec Pro to elimi-

“Is there an opportunity for UAVs to play a role in simplifying this process, enabling new technologies to be applied to precision agriculture? I think there is,” he said. “The question is how does it evolve, and how is it applied?”

Through machine optimization, logistics and precision support, John Deere is working to build a more autonomous agriculture industry, even if that means dragging end users along with the change, learning together as the technology evolves.

“You don’t just start with technology,” Havens said. “It’s an integrated and complicated vision that has technology surrounding it.”

“That’s what we have to make with autonomous systems: We have to make information that’s simple. That’s what our operators are expecting. That’s what they use personally when they use technology on a day-to-day basis.”

Mission Critical

•

Spring 2012

13

MARKETPLACE

Participate in AUVSI’s Member Marketplace, a NEW members-only network of free online classified advertisements featuring sections devoted to employment, research and for sale items.

Visit http://marketplace.auvsi.org today, or scan the barcode above to go to website.

MAXIMIZE YOUR VISIBILITY —

BECOME A CORPORATE MEMBER TODAY! · Discounts on Exhibits, Sponsorships, Advertising and AUVSI Products · Access to Members Only networking, education and Select VIP Events · Listing in Unmanned Systems Magazine and AUVSI’s Online Directory · Complimentary Job Listings on AUVSI’s Online Career Center · Research reports and knowledge sharing through AUVSI’s Online Community · Chapters and Regions around the Globe

Join today at www.auvsi.org/membership

Cropping up everywhere STATE OF THE ART

Leola, S.D.: The headquarter of Mack Robotics, South Dakota is home to the company’s Bin Bot invention. The robot can enter grain bins to empty out remaining grain without putting human life at risk. Read more on Bin Bot in Spotlight on Page 42.

Teddington, U.K.:

Cambridge, Mass.: In 2009, researchers from the Massachusetts Institute of Technology’s Offshore Aquaculture Engineering Center focused on building cages able to move under their own power that would aid in fish farming.

Called Aquapods, the technology would enable fish farmers to rotate locations without towing cages behind boats.

Researchers at the National Physical Laboratory have developed an imaging technology that would intelligently harvest crops by looking beneath the leafy layers of that crop and predict precise size identification. The lab estimates that annual waste for crops can mean up to 100,000 pounds in lost revenue during an average year by poor harvest selection.

Ames, Iowa: Iowa-based inventor David Dorhout has created a hexapod robot with a face similar to WALL-E that could work in swarms to perform crop planting, tending and harvesting. To see a video of the massive robot, click here.

Pittsburgh, Pa.: The Carnegie Mellon Robotics Institute heads up the Comprehensive Automation for Specialty Crops (CASC) program, which involves industry and academic partners around the country developing and testing robotic technology for use on farms and orchards. Read more about CASC in Testing, Testing on Page 38.

San Diego, Calif.: San Diego company Vision Robotics Corp. develops robots for automating many chores, one of which is its Scout CAD, that uses six arms to scan trees for orange picking. Once the images are analyzed, the robot will generate at 3-D model, and then the arms can later go back and pick those fruit. The company also has robots planned for grape pruning and apple picking.

16

Mission Critical

•

Spring 2012

Gainesville, Fla.: Researchers from the University of Florida are testing using micro air vehicles to assess the health of citrus groves from disease.

F

rom coast to coast and heartland to heartland, agricultural robotic applications pop up all over the globe. While a lot of research has been done in the throughout the world, Europe is remarkable for its widespread adoption of robotic cow milking, while Japan’s myriad applications position the country as king of the robotic crop.

Maassluis, the Netherlands: The international company Lely makes the Astronaut A4, a machine that automates the process of milking cows; the cows can enter the walk-through robot on their own to be milked. The company, which introduced the first robotic milking system two decades ago, says 12,500 of its Astronaut systems have been installed worldwide.

Hamamatsu, Japan: Yamaha introduces the R-Max automated helicopter in 1990; by 2011 there were nearly 2,400 registered in the country for crop dusting, and they sprayed 30 percent of the country’s rice fields.

Sendai, Japan:

Beer-Sheva, Israel:

Japan’s Ministry of Agriculture plans on turning 600 acres of the area destroyed by the March 2011 earthquake and tsunami into a robotic farm, to advance the future of automation in agriculture.

Researchers at Ben-Gurion University of the Negev, working with European partners, are seeking to develop crop-picking robots able to identify, spray and pick a variety of crops, including grapes and apples that are sometimes hard to see among the surrounding foliage.

Tokyo: The Tokyo University of Agriculture and Technology has tested a “farming robotic suit” that would assist in the backbreaking labor of farm work simply by supplementing the user with an exoskeleton-style suit.

Perth, Australia: The University of Western Australia has had a robotic sheep shearer, called the Shear Magic Robot, since the 1990s, though the product has never been commercially available since it is not significantly faster or more efficient than human shearing.

Kensington, Australia: The University of New South Wales is developing a variety of automated farm equipment, including ground robots that can seed and weed crops.

Mission Critical

•

Spring 2012

17

Virginia Tech researchers seek out mysterious blankets of biomaterials

By Danielle Lucey

A Rascal 110 UAV sits ready for a sampling mission at Virginia Tech’s Kentland Farm. All photos courtesy Schmale Laboratory at Virginia Tech.

S

omewhere, up in the air, live silent

passengers on a breeze. Kicked up

into the sky from as far away as other countries, these infinitesimally small microbes move in blanket-like waves. Not so silent is the havoc they can rain down on plant and animal life below, the results of which

Getting the project off the ground Ross has a background in dynamical systems and efficient space travel leveraged on gravity. By applying these concepts to the ways microbes move through the air, he can model and predict when these blankets

Farm, a campus-owned research farm about 10 miles west of Virginia Tech’s main Blacksburg, Va., campus. The facility has dedicated research areas and houses outreach and teaching space, where projects can bring in and educate stakeholders, members of the public and sometimes visit-

plague farmers, their crops, their animals

of microbes are on the move.

and even human health.

Schmale’s experience with unmanned aer-

functional and staffed in part through uni-

Called Lagrangian coherent structures by

ial vehicles puts the aircraft at the crux of

versity funding.

engineers, these dynamically distinct patches are typically talked about in terms of fluid

Schmale and Ross’ work, the tool that allows them to blend aerobiological monitor-

dynamics, relevant to ocean research. But

ing and plant pathology.

a project out of Virginia Tech hypothesizes

“We like to think about … microbial inva-

that similar structures occur in the air and waft between layers of the atmosphere.

It’s these invisible blankets that brought Virginia Tech researchers David Schmale and Shane Ross together for their interdisciplinary research. Their work aims to determine when and where these waves of microbial

sions,” says Schmale, an associate professor in Virginia Tech’s Department of Plant Pathology, Physiology and Weed Science.

He and Ross believe that pathogens move throughout the atmosphere and invade new territories. “We want to track them and find out where they’re coming from and where

invaders will happen and give farmers an

they’re going.”

early warning system to prepare for their

Virginia Tech is blessed with a large ag-

arrival.

18

ricultural research facility in its Kentland

Mission Critical

•

Spring 2012

ing school kids, says Schmale. The farm is

Most key for Schmale, however, is the facility’s 400-foot paved runway, perfect for the project’s fixed-wing unmanned systems.

“Having the facility has been instrumental to our success,” says Schmale. “I would say in terms of challenges, getting that facility up and running, it didn’t happen overnight.

We started all of our work basically on just a flat area of grass that the research farm gave us, and as we started to acquire resources, the administration started to respond with facilities.”

Going unmanned All this mysterious microbial movement falls under the umbrella of aerobiology, the study of the flow of life in the atmosphere, which is then broken down into three processes, says Schmale — release, horizontal transport and deposition. Fungi, for example, release their spores into the air, which are then swept up into turbulent currents. Then they’re transported over some distance and finally deposited onto another host crop.

Release and deposition happen at the Earth’s surface, and there are many tools — like spore vacuums and Petri plates on stakes — to measure these two easily accessible processes. Horizontal transport, however, requires a way to assess the atmosphere and, as a result, is much harder to understand.

There are a number of ways to do this, says Schmale, using full-size aircraft, balloons, even kites.

“With full-scale aircraft you’ve got issues of at least risking one human life,” says Schmale, echoing that a lot of these missions fall into the dull, dirty and dangerous categories, “as AUVSI likes to reflect on quite a bit. So putting an unmanned system in the kicker is an important part of fulfilling research obligations and being able to do things fairly inexpensively and without the risk of human life.

“It’s true, I mean a lot of the missions we do are very dull. I wouldn’t say they’re dangerous. They’re very dull and very dirty.”

When they sample, the team goes out for a week and flies all day long. And since

A ground-based sampling device to collect Fusarium.

they’re using an autopilot, “the plane’s doing the flying; we’re just sitting back watching.”

Sidestepping using a full-scale aircraft for this and using unmanned systems instead helps reduce pilot fatigue, says Schmale.

on Petri dishes containing a selective agar

Certificates of Authorization from the Fed-

medium.

eral Aviation Administration.

The team is using a SIG Rascal 110 for

The Penguin B comes with the largest price

their National Science Foundation-funded

For their research, Schmale, Ross and their

research, but they also fly a BlueBird Boo-

students use a series of four unmanned aer-

merang, a UAV Factory Penguin B and a

ial vehicles, all readily available as remote-

Senior Telemaster. All of the platforms are

controlled platforms, to gather pathogens

licensed to fly through their four separate

tag. It’s a fiberglass, “more professional, commercial body frame,” meanwhile the others are more hobby-based plywood platforms that aid in the project’s workload, says Schmale.

Mission Critical

•

Spring 2012

19

Airspace Invaders — continued from Page 19 The Rascal 110 is the project’s “tried and

activity still occurs above the altitude where

true platform,” says Schmale. It can fly for

the team is allowed to fly.

45 minutes with their current configuration, “and she carries the wing loading of our plates really, really well, and so we’ve used her exclusively for all our NSF work.”

For

their

sensor-based

applications,

Schmale prefers the Penguin B, “because it’s a pusher, and of course we’re not disturbing airflow. We’re sort of in front of the plane, which allows us to channel things into the fuselage.”

The team has published data on hundreds of their flights, though a very literal ceiling limits the research the project can do.

While a lot of transport blankets occur in the operable airspace under Virginia Tech’s certificates of authorization, much of the

search on farms in the future.

“The idea of a consultant that wants to set

“The pathogens are flying considerably

up a business that’s based in part on our

higher than the FAA will let us actually fly

technology would have to get a COA in

for our COAs,” says Schmale.

“They’re

order to do this, and I don’t know how they

moving through the planetary boundary

would do it,” he says. “You’d have to know

layer and beyond.”

in advance where you’d be flying, and

However, the higher a microbe goes, the more it will be exposed to the elements, says Schmale, like UV radiation, which is hard on things like spores and can kill them.

The work at Kentland Farm is all restricted under the four COAs, which specify both altitude and operational restrictions.

you’d basically have to, more or less, say you’re going to fly anywhere in the contiguous U.S. Really, that’s the biggest restriction right now. There’s nothing more than that.”

Growers, however, are amenable to the idea of UAVs buzzing over their farms, says Schmale. If a plane were able to go up as a wave of microbes were coming and detect

Having the Kentland Farm facility makes

where they were coming from, it could help

it easier for the FAA to clear them, says

a farmer know when to spray a fungicide.

Schmale. The FAA’s current policies, however, could limit implementation of this re-

CropCam in Canada has the farming mar-

The Rascal 110 is outfitted with sampling arms, which open and close during flight.

20

Mission Critical

•

Spring 2012

species,” he says.

One of the most important mycotoxins in the United States, according to Schmale, is deoxynivalenol, also known by the much more descriptive name vomitoxin.

“Symptoms that often occur from domestic animals consuming grain basically contaminated with this toxin will often exhibit symptoms of vomiting, even refuse to eat their feed, particularly in the case of swine, and death can ensue if levels are high enough,” warns Schmale.

Another type of mycotoxin — known as zearalenone — plagues animals like pigs and mimics the hormone estrogen, resulting in reproductive problems for the animals, Preparing sampling devices for unmanned aerial vehicles to collect Fusarium in the lower atmosphere.

like swollen teats and ovaries. “It’s a really big deal,” says Schmale.

Other kinds of Fusarium produce mycotoxins that lace common staple crops for humans, like wheat and corn. There are other species that cause skin diseases in humans and an eye disease called Fusari-

ket cornered in that country, says Schmale,

this group … causes a number of different,

because of its price and reliability. That

important diseases of relevance to plants,

model would also play well in the U.S., he

domestic animals and humans,” says

says.

Schmale.

“Those type of systems if you can get into

It can cause wilts, rots, cankers and blights

the few thousand dollar range, I think grow-

to many kinds of crops.

also creates pulmonary infections, with re-

“You’d be hard-pressed to find a crop that

zil, says Schmale.

ers would be really willing to entertain them, but then it’s the whole idea of actually being able to use it and make something of it in your business. And that’s where, if it’s not a hobby, it’s a business, and the FAA has to be involved.”

didn’t have some sort of Fusarium disease associated with it,” says Schmale. “It’s quite phenomenal.”

There are about 80 biological species of the genus around the globe, and that iden-

Fungus fatale The main culprit in all this pathogen research is the extremely prolific fungus in the genus Fusarium, the sole microbe of interest in the NSF study.

While not all species of Fusarium are dangerous — one is even approved for sale as food by the United Kingdom — many strains of the fungi cause widespread disease.

“Fusarium is one of the most important genera of fungi on the planet Earth, and

tified number likely falls far short of the actual number of species, which Schmale estimates as perhaps twice that number or

um keratitis that particularly affects contact lens wearers that don’t frequently wash their lenses in solution, where the fungus implants onto the cornea outer layer of the eye and grows through its layers. Fusarium cent localized outbreaks occurring in Bra-

Gathering samples To actually collect these microbial samples, the UAVs are outfitted with a series of Petri dishes that open and close at set points in flight.

“We have these sampling arms … that

more.

operated like a clam shell, that open and

Of the dangerous strains of Fusarium, some

Schmale.

produce dangerous toxins, called mycotoxins, commonly referred to as fungusproduced molds. These mycotoxins can

close remote control from the ground,” says

There are inner and outer sampling arms, and the configuration on the Rascal 110

contaminate food and feed, says Schmale.

can hold eight Petri plates. The plates con-

“We do a lot of analytical chemistry to

Fusarium.

detect these toxins in food and feed and ultimately attribute them to certain Fusarium

tain a specific medium that is selective for

“What’s neat about using these plates as a

Mission Critical

•

Spring 2012

21

Airspace Invaders — continued from Page 21 sampling device is that with this medium we

Once they land, they’ll detach the used

can basically select just for Fusarium and

plates, put on new ones and repeat the

kind of ignore all the other stuff we’re not

flights. All the sample analysis happens on

trying to track.”

the ground in their lab.

The arms are closed during takeoff and

Types of sampling surfaces in the plates in-

SPR, that allows users to detect an agent in

landing, so there is no exposure to the

clude things like grease, sticky tape and, in

real time, so the team could identify biologi-

plates until the arms are opened, prevent-

the last couple years, filter papers coated

cal agents while the UAV is flying.

ing any sample contamination.

with glycerin that allow them to collect a

The sampling pattern is all autonomous; meanwhile, the researchers open one set of arms for 10 minutes, close them and repeat the process with the second set of arms.

“We have a sample that’s been collected over the period of 20 minutes, but it’s actually two independent samples, kind of like two separate flights, if you will,” says Schmale.

then inspected using DNA-based analyses back in the lab. The future of this research

Mission Critical

•

Spring 2012

They are also exploring a device from a Washington company that makes a technology called surface plasmon resonance, or

Once identified, the researchers can hypothesize where a sample was flowing from based on the time of year.

is likely this type of multimicrobial analysis,

“That’s a big component of what we do,”

says Schmale.

says Schmale. “Some of the species are

Other sensors on the aircraft include an ionic spore sensor, which takes air funneled through the nose of a pusher platform. The air is charged as it passes through this device, and the particles then are deposited

Members of the Schmale lab conduct an aerial sampling mission.

22

very varied sample of microbes, which are

on a surface of opposite charge.

exclusively localized to a certain region or certain crop type, so if you’re collecting a tropical species, for example, during the winter time here in Blacksburg, Va., then you might question, well, it’s likely coming from some warmer, tropical region.”

The team does back-trajectory analysis to

The students with no prior experience with

“What they might consider as a toy is a

test that hypothesis and looks for seasonal

unmanned systems tend to have a warm re-

tool for us,” says Schmale. “It sort of begs

trends in what they’re collecting. One of his

ception to the technology, he says.

the question of, what is the future of agri-

graduate students is focusing her research on these recurring trends from year to year.

“Everybody sort of gets excited … about things that fly. It’s just really cool,” says

Schmale says he’s very selective about

Schmale. “When you take something that

which grad students get to participate in his

can fly and then you turn it into a tool …

and Ross’ program, which blends students

that has some relevance to the system of

from many backgrounds and majors, some

agriculture, in general students get really

of which have never used unmanned sys-

excited.” Schmale says he’s impressed with

tems before.

how quickly young students take to the top-

“I have students who are trained solely through my department … and then we have students who also have a split role,

culture? And are these unmanned systems a part of it? Of course the answer coming from my side of the fence is yes.”

Danielle Lucey is managing editor of Mission Critical.

ic and the intelligent questions they have.

Schmale also has an RC simulator where high schoolers can try flying for themselves.

who are basically co-advised, but their de-

“Of course they crash and burn pretty

gree will come from an engineering depart-

quickly after they try, but it’s all on a video

ment or from my department, for example,”

game, which is really nice,” he says. “They

says Schmale.

can appreciate how tough it really is.

Mission Critical

•

Spring 2012

23

Need to Learn More? No Time to Leave Your Desk?

Tune into AUVSI Webinars AUVSI webinars are a new and fastgrowing service that delivers expert speakers and hot topics in unmanned systems to your desktop. Topics over the coming year will include all facets of this dynamic industry.

Members receive a special discount on all webinar content. Listen to the webinars live, or download them to your computer later.

Check our website for the latest schedule: http://www.auvsi.org/publications/auvsiwebinarseries.

AUVSI’s quarterly publication that highlights special topics in the unmanned systems industry.

Each is an in-depth focus on one particular issue with information on the defense, civil and commercial applications of the technology as well as new developments and what the future may hold.

Upcoming Issues:

VOLUME 1

NO.2 • S

UMMER 20

11 • AU

VSI • 270

Robots aid Japa

0 South Qu

incy Street

, Suite 400

n

Commercial Robots: Summer Edition Security: Fall Edition Sensors: Winter Edition

V

OLUME

1 NO.1

• SPR

ING 20

11 •

I • 2 AUVS

700 S

incy S outh Qu

, Arlington

, VA 2 2 2 0 6

, USA

Unmanned syste ms fight fires

treet, S

uite 40

0, Arlin

g t o n , VA

22206

, USA

ected DOT’s connork w le vehic rtre Europe’s Sa s in a tr d a ro

Robots help p

olice

Inside this issu

e:

First responder

e

robots

Challeng nd Driver

Bli issue:

Inside this

tion

nsporta Smart Tra

MISSION

CRITICAL

•

Spring 201

1

MISSI ON CRITIC AL

I

If your company has technology in any of these arenas, then you can’t afford to miss this opportunity — book your advertising space today!

Contact Lisa Fick at fick@auvsi.org for more information.

•

Summer 2011

1

Dr. Daniel Schmoldt

Q&A

National Institute of Food and Agriculture Staff Director U.S. Department of Agriculture Dr. Daniel Schmoldt, National Institute of Food and Agriculture staff director at the U.S. Department of Agriculture, has helped get the agency involved in President Obama’s National Robotics Initiative. The USDA has teamed up with three other agencies — the National Science Foundation, NASA and the National Institutes of Health — to collect proposals, small and large, for new uses of robotics in the civil sector. While the farming industry has been slow to pick up robotics in the past, Schmoldt says there are now ample opportunities for unmanned systems to plow — or till, or harvest — their way into the industry.

Dr. Daniel Schmoldt at AUVSI’s Unmanned Systems Program Review 2012. AUVSI photo.

Q:

How has the market for unmanned systems and robotics in agriculture evolved?

labor is probably the biggest cost they have.

There’s really only one significant funding

They see labor-saving technology as critical

authority for our agency where we have suf-

to their survival.

ficient discretion to use funds for something like [the Agriculture and Food Research Initia-

Because of the farm workers unions and things

tive]. It wasn’t increased this year, but at least

like that back in the ‘70s, there was a strong backlash against mechanization, because it was eliminating jobs for farm workers. A lot of the robotics that would have ensued and developed from that really didn’t have a chance, because the people that were doing the engineering and technology development were turned away from that. The funding wasn’t there; the incentives weren’t there. But in the last 10 years or so, there’s been a real strong push from the producers, from the stakeholders, because they have a lot of competition from international producers — for example apples from Chile and China, etc. — so they need to increase their production efforts, and

26

Mission Critical

•

Spring 2012

Q:

it wasn’t decreased, which is a good thing.

This is a presidential initiative, not a congres-

Q:

How is the USDA adjusting financially to fund the NRI?

sional one. There’s no new funding line for this, so every agency has to find some place in their discretionary expenditures and have

Do you think NRI funding will survive the current push for austerity on the federal level?

the appropriate legal authorities to support

In this administration, I think there’s an expec-

this program with their regular appropria-

tation, and maybe even a realization, that

tions. Unfortunately on our end … we have

robotics is one potential driver of economic

a whole bunch of very specific legal-funding

development going forward in the next 10

authorities, and those programs are supposed

to 20 years. It’s a burgeoning industry glob-

to only be working on certain things. It’s very

ally, and especially in Europe and Asia, and

prescriptive.

we’re going to see more and more economic development surrounding robotics. So if we

can advance that with this research and de-

quality tends to improve with newer technol-

jobs that are low skill, low wage, plus dan-

velopment program, the NRI, that certainly

ogy in automation.

gerous and ergonomically unfriendly.

seems to be a good way to support that.

Also there’s the opportunity to introduce sen-

It is already happening to some extent, be-

I think the interest is there and the rationale is

sors that can improve the practice; whether

cause there are new technologies coming in,

reasonably well placed for maintaining this

it’s harvesting or thinning, it can be improved

maybe not so much unmanned systems and

program. The exact levels it will be funded at

by the right sensors that humans don’t really

robotics, but other technologies that are help-

remains to be seen.

have or aren’t very good at. You’re going to

ing with this. It’s going to have to come with

see product quality improve.

the technology; it’s not going to be one or the other first. It’s going have to be introduced

Q:

How accessible are these unmanned systems and robotics to smaller momand-pop farms, as opposed to larger factory farms? What can be done to close this gap?

Obviously that’s very much a concern for us.

almost concurrently.

Q:

What does the testing process look like for emerging unmanned systems and robotic technology in agriculture? Are there any standards?

Q:

What kind of dream technology do you want to see down on the farm in terms of unmanned systems and robotics?

We like to have technology that’s scale neu-

There are some standards, for example

tral: It’s accessible from a cross-perspective

safety standards for agriculture equipment.

When I sort of go into a dream state about

for small- and medium-size producers as well

We have suggested in various meetings and

this, what would be nice to see is crop tend-

as large producers. There’s economy at scale

asked a couple of people for development

ing robots. That is whatever crop you’re talk-

associated with these things.

of standards for unmanned robotic systems in

ing about, whether it’s an orchard or tomato

agriculture. I can’t tell you in what stage that

field, that there would be sort of a swarm

is at right now.

of robots that could fairly continuously move

In some cases it’s not just economics; it’s not just cost. It’s other things relative to their operation — their age, who else they’re work-

If we’re able to help support the development,

ing with, their comfort level with technology

we’d like to see them eventually adopted. If

through the field and just monitor and perform opposite stimuli and coordinate among

“While [automation] eliminates more jobs in the short run, it creates more jobs in the long run. We want to see better jobs and get rid of these other jobs that are low skill, low wage, plus dangerous and ergonomically unfriendly.”

– Dr. Daniel Schmoldt

in general. That gets into the second piece of

there are safety roadblocks that sort of limit

accessibility: Do they have the time to use the

themselves along with the people managing

the adoption, we’ve encouraged the appro-

tech? There’s a learning curve, plus there’s a

the operation. Things get done, and they can

priate people — the people that set standards

time consideration.

look for pests, detect pests while they’re pull-

and deal with the legal team — to see these

ing weeds or cultivating weeds, or whatever

are dealt with in a timely fashion.

the operation might be. There’s just this fairly continuous revisiting of the crop production

Q:

How does more automation in agriculture affect the quality of the end product?

Robotics and the new automation can actually be friendlier, because instead of human

environment. The operator knows what’s go-

Q:

How has the automation of agriculture affected the job market for people working on farms? Are there any new emerging opportunities?

laborers, which get tired, they get fatigued,

While it eliminates more jobs in the short run,

they get distracted and all sorts of other

it creates more jobs in the long run. We want

things, machines do not. The end-product

to see better jobs and get rid of these other

ing and what’s being done and feels very comfortable. They know exactly what product quality they’re going to have and what yield they’re going to have at the end of the growing season.

Mission Critical

•

Spring 2012

27

John Deere

TIMELINE

1943

1837

28

1850

John Deere, a blacksmith and inventor, founds his namesake company.

The first piece of farming technology he makes is a polished-steel plow that helps Midwest farmers plow through sticky prairie soil.

1800

The company’s history dates back to the early days of American westward expansion, giving it ample room to grow with, and lead, technology trends in farming.

Mission Critical

•

Spring 2012

1918

Motorized equipment on the farm becomes popularized, and the most popular piece of motorized equipment is the tractor. John Deere gets into the tractor business after buying out Waterloo Boy Tractors. The company sells Waterloo Boymodel tractors for five years before coming out with its first original Model “D” tractor.

1900

T

he first piece of unmanned equipment in the John Deere family was a horse.

The company makes the Hawkeye Riding Cultivator, the first piece of Deere equipment adapted for riding. At first the driver has to stand on the cultivator, but later models include seats.

1999

John Deere begins work on intelligent mobile equipment, including the use of satellites in farming and taking the first steps in automating agricultural processes.

1950

1863

John Deere makes military tractors, ammunition, aircraft parts and cargo, and mobile laundry units during World War II, in part because “limitation orders” restricted civilian production of farm equipment, repair parts and exports.

2011

2001

2002

John Deere works on the Autonomous Orchard Project with Carnegie Mellon University, Cornell University and the University of Florida.

2004 John Deere officially unveils the Gator vehicle platform to the U.S. military.

Using some automated technologies adapted from agriculture products, warfighters can drive the vehicle or operate it as an unmanned ground vehicle.

It’s most recent A3 model debuted in 2011.

2012

John Deere celebrates its 175th anniversary.

2000

AutoTrac becomes available for John Deere tractors. The technology fuses together satellite position data from the StarFire receiver with feeler data gathered from the row sensor.

John Deere debuts Machine Sync. New Machine Sync technology allows the combine operator to automatically control the location of the tractor and grain cart while simultaneously unloading during field operations.

Mission Critical

•

Spring 2012

29

Ripe for the picking MARKET REPORT

I

n robotic agriculture, just as with any

to Europe, with only 10 U.S. dairy farms

In a series of interviews conducted with

other industry, you reap what you sow.

owning robotics milkers as of 2009. In that

attendees at the 2011 World Ag Expo, a

year, there were more than 2,000 farms in

blog by The Robot Report, a news outlet

the world that employed the milkers. The

that tracks the state of business in robotics,

machines were first commercially available

reported that nearly 50 percent of robotic

to the United States in 2000, but were in ex-

tractors employ unmanned driving system

istence in places like the Netherlands start-

technology, however technologies like sen-

The world’s premiere conference in all

ing back in 1992. Purdue University dairy

sor and motion- or vision-based systems,

things farming is the World Ag Expo, held

specialist Mike Schutz, in an interview with

like berry harvesting robots, have until now

annually in Tulare, Calif. With more than

CBS News, attributes the slow adoption in

been slow to be adopted because of how

1,600 exhibitors from around the world,

the U.S. to the country’s cheap labor and

pricy the systems are.

the show picks 10 technologies every year

regulations that give farmers more incentive

that it deems the best to advance the field

to add cows than to make a smaller number

of agriculture.

of cows more efficient milkers.

In 2012, these Top 10 New Products in-

Another Top 10 New Products winner came

of seed, sprays and labor and the lower

cluded some with technologies from the

from specialty harvest company Oxbo Inter-

costs of these sensors. This environment

world of robotics, which could hold clues

national Corp. The Byron, N.Y., company’s

makes automated technology a prime in-

for the proverbial low-hanging fruit in the

9120 Raspberry Harvester features auto-

vestment for larger producers.

robo-ag world.

mated steering technology.

Swedish company DeLaval’s innovations in

“This machine could probably replace two-

company that is the lead material handling

dairy farming date back to 1894 when it

to three-hundred people out in the field, so

systems integrator of automation technolo-

held patents for early forms of milking ma-

when you compare that to what this is do-

gies, attended this year’s World Ag Expo

chines and the first centrifugal milk-cream

ing to your capital costs compared to the

and observed in his company blog that the

separator. This year, it was recognized for

labor cost you can pay it off in a year or

automation market is ripening.

the world’s first automated rotary parlor for

two,” said Scott Korthuis, a product man-

high-capacity milking, the DeLaval AMR.

ager with Oxbo, in another KVPR interview.

“Helping dairy farmers achieve a level of

The machine also employs Oxbo’s Orbitro-

dling suppliers just by the nature of what

sustainably that allows them to stay in busi-

tor, a selective picking system that seeks out

they do every day with their products, but

ness has been a focus and one of the ways

the best raspberries, in this case the IQF,

now, automation with material handling

that we have chosen to do that is with tech-

or individually quick frozen — the highest

is becoming more important as global de-

nology,” said Ben Kern, western regional

grade raspberry.

mand keeps rising and everyone wants to

Though farms aren’t currently over-

whelmed with robotic equipment, automating the process of farming is certainly a growing trend.

California’s KVPR Valley Public Radio.

of robotic milking technology, compared

SCAN IT

or

Robotic milkmaids all in a row: DeLaval’s Automatic Milking Rotary innovation.

Photo courtesy DeLaval.

Click IT:

Click the photo or scan this barcode with your smartphone to see a video of the DeLaval Automatic Milking Rotary dariy management system

30

Mission Critical

•

Spring 2012

that the industry is slowly moving toward more automation because of the rising cost

Greg McGiffney of Bastian Solutions, a

“The agriculture and food handling businesses are big industries for material han-

be more efficient.”

director for DeLaval, in an interview with

The United States has been a slow adopter

The Robotics Report also says, however,

S trawberries ,

spraying and strength — Robotic farming in Japan

By Liz Ruskin

The objects of their affection: These strawberries were for sale in a mid-range department store west of Tokyo for a little more than $1 per fruit.

All photos by Liz Ruskin.

products, thanks in part to price supports and to policies that have discouraged cheaper imports. Japanese consumers tolerate the higher prices, because they tend to distrust the safety of food from overseas. That, plus sustained government support in a country known for its engineering prowess, seems to make up for the meager economies of scale.

So, filled with a sense of national mission, Japanese engineers

S

have set about creating robots to do farm work. It’s a very Japatrawberries in Japan are prized jewels. They are sold, all

ruby red and perfectly aligned, in small trays in supermar-

kets and in the food halls of Tokyo department stores. Some, created for the high-end gift market, go for as much as $1 to $5 per

nese solution to uniquely Japanese problem, and it has put this country at the forefront of agribot research and development.

Strawberry-harvesting bot

berry. Bruising is not tolerated.

At an agricultural research and development campus north of To-

This level of perfection demands not just tender treatment, but a

harvesting robot that has been in the works since 2003. The ma-

lot of backbreaking labor — nearly 1,000 hours per quarter-acre greenhouse — just to pick, grade and pack. The trouble is, labor is in short supply in Japan. The farming population has shrunk by one-third over the past decade, the result of rural migration to the cities and Japan’s tight immigration policies. The average age of a farmer is now nearly 66 years old. One in four farm workers is

kyo, Dr. Shigehiko Hayashi recently demonstrated a strawberrychine was built to travel on rails through a greenhouse of elevated beds, looking for red objects with its three-camera-lens eyes. When its particular color conditions are met, it recognizes the object as a fruit and homes in. Then, using two side cameras arranged for stereoscopic vision, it determines the 3-D position of the fruit.

over 70.

Next it really shows its smarts, employing a “maturity assessment

The government, rallying the country to boost food self-sufficiency,

robot considers the hue saturation, the relative size of the red and

is investing in a range of automation projects, hoping robots will step up to help Japan increase its agricultural production.

At first glance, Japanese agriculture doesn’t seem well suited to expensive machinery. Compared to the United States, farms here are tiny, just 4.5 acres on average, and often tucked into valleys or hemmed in by city sprawl. But these farms produce high-value

algorithm” to decide whether the fruit is ripe enough to pick. The non-red areas, and the margin between them.

If it decides the fruit is ripe, its pincer-like hand reaches out for the stem, tilting its wrist to match the curve of the vine. It cuts and grasps in one motion. Holding the strawberry by a long stem, it then rotates to place the berry in a harvesting tray. It’s equipped with an optional suction device that can help hold the fruit for

Japan — continued from Page 31 the fruit the robot missed, but the robot could reduce their hours by more than half.

Now on his fourth prototype, Hayashi says he’s getting close to commercial readiness.

The cost per robot is now about $120,000.

He figures he needs to bring it down to $70,000 or lower for it to be practical for a normal farm.

He and his colleagues at IAM-BRAIN have also created a robot to that can grade and pack strawberries to the exacting standards of the Japanese market. While other machines rely on mechanical grippers, the IAM-BRAIN team found such machines couldn’t place the strawberries precisely next to each other on a tray, because the “fingers” tended to dislodge neighboring berries every time a new one was placed.

Theirs, a two-armed contraption, uses suction alone to hold the berry, and it only A strawberry-harvesting robot created at the Japanese R&D institute known as IAM-BRAIN.

holds the fruit by its leafy green cap. To do that, it uses its vision system to determine exactly what angle to use to approach each berry. Its nuanced tilting gives the impres-

added security. Otherwise, the berry itself

Technology Research Advancement Institu-

goes untouched by man or machine.

tion, better known by its catchy acronym,

The robot repeats the process all the way down a row and then comes back up the

IAM-BRAIN. “It can only harvest 60 percent or 70 percent of the ripe fruit.”

sion that it is quizzically inspecting each target. The machine can sort by size and settle each in its proper tray. Dr. Satoshi Yamamoto, a colleague of Hayashi’s, says the current version of the packer would costs

same rails to pick berries from the other

The robot will often fail to detect a ripe ber-

about $86,000. He thinks it would be a

side of the aisle. Then it starts down the next

ry if an immature one is hanging in front of

practical choice for a strawberry farm if

aisle. When its tray is full, it brings it to a

it, he says, or if it is hidden behind leaves.

they can bring the price down to $60,000.

stacker, discharges it and takes an empty

Its success at judging ripeness depends on

one. It continues like that until it has worked

the cultivar. On types with a clear margin

the entire greenhouse.

between the ripe and unripe areas, its

“This machine is not perfect,” says Hayashi, of the Institute of Agricultural Machinery, Bio-oriented

judgment is about the same as a human’s, Hayashi says.

•

Spring 2012

future came early to some corners of the Japanese agricultural scene. Way back in 1983, an agricultural engineering profes-

nine seconds per berry, about the same as

sor from Kyoto University demonstrated

a person. The robot, however, could work

an intelligent robot for tomato harvesting.

well in conjunction with human harvesters,

Tokyo-based Iseki & Co. has had a robotic

he says. It was designed to work all night.

system for grafting seedlings on the market

The cooler air at night helps keep the fruit

since 1993.

illuminate its target.

Mission Critical

If agribots are the future of farming, the

It’s not particularly fast, either. It takes about

firm, and the robot has its own LED lights to

32

Robotic spraying

But the most prevalent unmanned systems at work on Japanese farms are in the air,

Workers, who typically harvest in the early

hovering over rice fields. Unmanned heli-

morning, would still come in early to pick

copters, on the market since 1990, are now

routinely used here for crop dusting, sav-

Only a few have been exported to the U.S.,

farmland on the northeast coast were inun-

ing Japan’s elderly farmers from countless

primarily for research purposes. They were

dated by saltwater and debris.

hours of manual spraying.

introduced to the South Korean market for

More than 2,300 are registered for use here, the vast majority versions of the Yamaha RMAX.

insecticide spraying in 2003, but fewer than 90 were registered by 2010. Korean farms are bigger, so they would benefit from a larger aircraft, says William Park,

The RMAX is roughly the size of a motor-

founder of Korea’s UAV Center and infor-

bike and weighs 128 pounds empty. It will

mation clearinghouse. He also says the un-

carry 66 pounds of pesticides or fertilizer,

manned helicopter had an abysmal safety

as liquid or granules. It can fly slow and

record in Korea, which he attributed to a

low, as little as three yards off the ground,

lack of training.

for an hour.

Compared with manned helicopters, the

Unmanned tractors

RMAX is quieter and produces less chemi-

Among the best uses of robots is to send

cal drift. In 2003 unmanned helicopters for

them where human workers fear to tread,

the first time sprayed more acreage than

and by that yardstick the March 2011 tsu-

manned choppers. Now more than 30 per-

nami and nuclear disaster created ideal

cent of Japan’s fields are sprayed by these

proving grounds for agricultural robots.

UAVs.

About 10,900 hectares of rice fields in the

The RMAX is usually operated with a boxy controller, similar to those used for remote control toys, and is reportedly easy to fly. If the operator lets go of the control stick, the

Fukushima Prefecture will go unplanted this year, due to contamination from the nuclear meltdowns at the Fukushima Dai-ichi power plant. In addition, some 20,000 hectares of

The Japanese Ministry of Agriculture, Forestry and Fisheries announced earlier this year that it is funding a series of projects in the tsunami damage zone in which robots and unmanned tractors will be put to work rehabilitating fields in four communities in Miyagi, the prefecture that surrounds the city of Sendai and borders Fukushima.

The government has committed to spending $52 million on the project over the next six years.

Japan’s Institute of Agricultural Machinery has been busy preparing an unmanned tractor for use on the contaminated fields of Fukushima. The prototype can be operated from a control car a kilometer away.

It has cameras on the front and side and navigates with the help of GPS technology.

Its role, says the institute’s Keiji Hanawa, would be to crush the soil surface to prepare for the removal of the top two inches,

RMAX defaults to a dead-still hover.

But the RMAX, which comes with or without GPS, is no toy. Prices for basic models approach six figures. Yamaha also makes an autonomous version, operated from a base station, that reportedly costs $270,000 or up to a $1 million depending on what equipment it comes with.

Their potential for farmers goes beyond pesticide application. Scientists at IAM-BRAIN developed a “crop eye” for mounting under an RMAX helicopter. It is a growth-measuring device that is designed to fly over fields, taking pictures at red, near-infrared and sunlight wavelengths simultaneously. Image analysis software uses the pictures to determine the nitrogen content of the plants and thus gauge the growth index.

The RMAX and its predecessors were designed in Japan, to meet Japan’s agricultural needs. Its popularity proves it’s well suited to Japan’s small farms. But how well would it work outside of its home country?

This strawberry-packing robot determines which way the berry is lying, then uses suction to pick it up by the green hull. The suctioning end effectuator is the blue object in the upper left.

Mission Critical

•

Spring 2012

33

where most of the nuclear contamination

But the GPS-based systems for auto-steering

an after-market product that farmers can

is concentrated. Radiation levels inside

on the market now are expensive and not

add to their existing tractors.

the sealed cabin of a control car would be

widely accepted by Japan’s farmers.

about half those in the field, he says.

Hanawa’s lower tech approach involves

The strength of youth

setting up a target lamp at the end of a row.

Some Japanese engineers are pursuing a

Cameras mounted on the tractor detect the

more direct approach for Japan’s aged

target, and the automatic steering control

farmers with devices that will help them

points the tractor toward it. As the tractor

regain the strength of youth. Inventors at

advances, the cameras also take a series

both the Institute of Agricultural Machinery

of pictures of small features on the ground.

and the Tokyo University of Agriculture and

The holy grail of agricultural robotics in

The system compares those points to correct

Technology have devised separate robotic

Japan would be a small, accurate autono-

the tractor’s lateral movement. With this sys-

suits to help farmers with the physical de-

mous tractor that’s commercially viable and

tem, a tractor can drive 100 yards without

mands of farming — carrying heavy ob-

simple to operate. Scores of companies

deviating so much as two inches, he says.

jects, crouching to weed or standing for

Is that safe enough?

“We have to do this [remediation work],” says the institute’s spokeswoman, Momoko Fujii. “It may not be perfectly safe, but we have to do it.”

and research facilities here have tried to produce one, with mixed success.

One limitation: The maximum distance is 330 yards, which is just enough for most

Meanwhile, Hanawa is perfecting a self-

Japanese farms and probably laughable to

steering tractor that does not rely on ex-

an American farmer, he acknowledges.

pensive GPS systems. Driving a perfectly straight line — for seeding, ridge forming and creating field boundaries — is key to farming efficiency, especially on small plots.

But, he says, if you use two target lamps and move them alternately, you could maintain a straight line for as long as you like.

He hopes to eventually turn this system into

hours with arms held high to prune, for example. The TUAT version, which Prof.

Shigeki Toyama has been working on for 15 years, looks like something out of the movie “Iron Man.” It is a motorized exoskeleton that is strapped to the body. It employs eight electric motors to give more power to the user’s arm and leg muscles, and it has sensors to detect movement. It can be operated by voice command. One version, according to news agency Agence France-Presse, weighs 30 kilograms and is designed to help farmers pull radishes and other root vegetables. The lighter version weighs 23 kilograms and is intended for harvesting grapes. The inventors hope to bring the price down to about $6,000 once it’s in mass production.

But will it appeal to farmers? That’s a potential barrier to deploying of any kind of agribot in Japan: The farmers they are trying to help are elderly, a population that has been slow to adopt new technologies. Few Japanese people over 65, a government survey shows, even use email.

Liz Ruskin is a freelance journalist living in Tokyo. She has worked as a newspaper journalist for 14 years — as a Washington correspondent for McClatchy Newspapers and, before that, as a reporter for the Anchorage Daily News. She has also worked in radio.

34

Mission Critical

•

Spring 2012

How green is my valley? Hard to tell with flight restrictions UNCANNY VALLEY

P

lants may grow from the ground up, but the view of an orange grove from the top down also provides valuable

information, especially for precision agriculture.

farming,” says Ehsani.

Ehsani hopes to use aerial imaging to show

His project in particular is reviewing the health of Florida produce in the wake of a disease called citrus greening. The bacterial disease, transmitted from plant to plant

early onset of the disease. Though aerial imaging with manned aircraft has been around for decades, Ehsani sees unmanned systems fitting in as another method.

A farm concept that observes and responds

via insects called psyllids, can be difficult to

“If we can monitor the overall health of the

to variations in a field of crops, this agri-

detect, and trees can remain symptomless

tree, and that can be done with a multispec-

cultural management style is Reza Ehsani’s

for years before the infection becomes visu-

tral image, we can from the multispectral

specialty. An associate professor at the Cit-

ally evident. Once visible, it can kill a tree

aerial image identify which trees start to

rus Research and Education Center at the

very quickly.

decline for any reason,” he says. “And we

University of Florida, Ehsani set out to find an alternative, cheap method of assessing

“Greening is … a dangerous concern right

can go look at that particular tree.”

now in Florida, because it’s a disease that’s

The traditional method for identifying citrus

widespread in the state and there is no cure

greening is to have human operators walk

“People use satellite images and aerial im-

for it,” he says. “So to manage it, there are

or drive through each row in a grove and

ages to monitor crop health, crop disease,

two ways. One is identifying the infected

look for symptoms. When they see it, they

and also they can try to predict yield, so

trees and then removing them or try to

mark it and take a sample for later confir-

there’s a lot of different applications that

spray a lot of chemicals to kill the insect

mation. Once identified, then they’d go

aerial imaging can provide to farmers and

that transmits the bacteria.”

back and remove the tree.

crop health from the top down.

A quadrotor with a multispectral imaging sensor.

Prof. Reza Ehsani from the University of Florida hopes to use this method to detect a citrus disease ravaging groves in Florida. Photos courtesy Reza Ehsani.

36

Mission Critical

•

Spring 2012

“This process is recommended if your infec-

with it, you need to get some authorization

you go through the whole process of get-

tion rate is less than 1 percent,” he says.

through the FAA, and that’s a very lengthy

ting certified you cannot really use it for re-

Using this human method, there is also a

process.”

search, so that’s kind of a disappointment,”

40 percent error rate, where citrus greening goes undetected.

His idea is to use a quadrotor with a multispectral camera to seek out the greening and then send in people to monitor those trees. “It can do targeted scouting, so you

The recent ruling that the FAA must provide

he says.

airspace for small UAS by 2015 leaves

Another issue with nearly all farm equip-

Ehsani playing the waiting game.

ment is it has to be economical.

“We surely thought that we were exempt as

“The thing is that other people tried to use

a hobbyist, but it turns out that, really, if I

this type of low-cost platforms … but most of the previous attempts were

send the people to specific

not too successful, because

trees … and check if that

we know that to operate

tree has a sign of symp-

them you have to be a very

toms.”

good RC pilot. … And the

A friend turned Ehsani onto

thing is that if you make a

unmanned systems as a

small mistake you crash. And

hobby two years ago, and

if you crash you not only lose

it occurred to him the tech-

the airplane, but you lose a

nology would translate well

lot of expensive sensors that

to his research.

you have on it like a camera.”

“I looked at it and I thought, ‘Man, this is very easy to

The cost of losing an un-

operate.’ Before that I didn’t

manned asset, though, is

have any background in RC

A resolution comparison between a satellite image of a grove versus a quadcopter.

operating. I’d never even tried it; it sounded difficult.”

Using

targeted

small compared to using manned aircraft.

The cost of aerial imaging

scouting

with manned platforms re-

would likely improve the

quires a minimum amount of

productivity and likely the cost of hiring

go and do it as a hobby on a weekend it’s

area to cover. Ehsani estimates the current

scouts, which is about $100 per acre, says

okay, but if I try to use it really for research

minimum amount is around 10 acres and

Ehsani.

to collect data, I cannot. That’s why we are

the starting cost would be around $5,000

not flying right now.”

or more, depending on how close the grove

All of this is fine, in theory, but the project hit a major hurdle once Ehsani, who is

From the about five hours of flying he did

fairly new to using unmanned systems tech-

perform, Ehsani was pleased with the re-

nology, realized the Federal Aviation Ad-

sults.

ministration largely limits where and when unmanned systems can fly.

“About two years ago we came across … these multi-rotor copters that are commercially available,” says Ehsani. “This is something that’s been used by hobbyists for a while. We said, well, if we could use this, that would be great.”

“Our initial data suggests that it’s very, very useful and it could be a very useful tool for growers to use,” says Ehsani.

is to an airport. These flights would have to be continuous, as often as a couple times a month.

He doesn’t believe unmanned vehicles will replace manned vehicles in these large fields, but if you have a smaller grove, “then these systems could be very, very at-

Right now, commercial use is lacking, but

tractive tools, because now you can start

Ehsani is hoping to show this type of tech-

taking advantage of the benefits of aerial

nology isn’t just for applications that have

imaging at a very affordable rate.”

caused unmanned systems a PR problem, like use by paparazzi. People can benefit

After a handful of flights, Ehsani stopped

from the systems through research like his,

using the UAV because of this barrier.

he says.

“We collected some initial data, but then

“I guess everybody else is in our shoes in

we learned that, really, to use this as a hob-

terms of they will see a lot of potential for

by is okay, but if you try to do any research

this type of platform, but right now unless

Aerial imaging with manned vehicles has been in use since the 1970s, says Ehsani.

“But if you go and talk to growers, you will see that they are not using this technology.

Very, very few growers that I know are using these technologies,” says Ehsani.

CMU-led automation program puts robots in the field TESTING, TESTING

By Brett Davis

W

hen researchers at Carnegie Mellon

culture] extension people,” says Marcel

over and over again. There are some sta-

University wanted to evaluate the

Bergerman, the CASC project manager

tistics, I don’t know the exact numbers, but

usefulness of their ground robot for

and a systems scientist with Carnegie Mel-

about 30 percent of accidents in orchards

orchard growers, they did something radi-

lon’s Field Robotics Center. “Not to engi-

can be somehow related back to ladders,”

cal — they put it in the hands of actual or-

neers, we gave them to the extension peo-

says Bergerman.

chard growers.

ple, and they take these machines to actual

The team leaders of the Comprehensive Automation for Specialty Crops (CASC) program have developed a machine, based

orchards, commercial orchards with growers, and they go in and do the experiments themselves.”

To avoid that, CASC developed an automated version of Toro’s E-Workman platform, named the Autonomous Prime Mover.

They delivered APM systems to agriculture

on a commercial Toro platform, that can

Previously, orchard workers would take a

extension programs and growers last year,

autonomously trundle down orchard rows,

ladder, climb it to perform a task, climb

keeping one in-house for system modifica-

allowing human workers atop it to prune,

back down and move the ladder.

tions and testing. The APM has a scissor

thin fruit, train and tie trees, or harvest.

“Now this is driving slowly down the row,

“For the first time in my career, we devel-

and instead of literally going up the ladder,

oped robots … and we gave them to Penn

down the ladder, and moving the ladder,

State and Washington State, to the [agri-

lift on top of it that can carry two workers, who can control the vehicle if they want but who no longer have to climb up and down ladders.

The APM vehicle in “bin dog” mode.

Photo courtesy CASC.

38

Mission Critical

•

Spring 2012

“It’s an interesting case. It’s autonomous

100%

but manned,” says Sanjiv Singh, the proj-

90%

CASC has developed two generations of the machines, which have grown a little bit simpler over time. The first version had an iPad interface, but “iPads aren’t meant for readability in sunshine” and they can heat up, even in the winter, Singh says.

“We’ve gone back and done something a little bit lower tech but more sturdy, and it will work in any lighting condition,” he says. The interface now sports an E-ink panel, like a Kindle e-book reader, and has more discrete buttons, a slider bar for speed control and a foot pedal.

Increase in worker produc0vity

ect director and a research professor at the Robotics Institute.

Increased Efficiency with self-­‐guided pla;orm compared to Ladder

80% 70% 58%

60% 50%

52%

50%

50%

46% 41%

41%

40%

34%

33% 30% 20%

20% 10% 0%

Tying 1-­‐yr-­‐old apple trees

Applying wire clamps, pinching 2-­‐yr-­‐old apple trees

Tying 3-­‐yr-­‐old apple trees

Tying, pinching, thinning 3-­‐yr-­‐old apple trees

Tying, pinching 3-­‐yr-­‐old apple trees

Thinning 5-­‐yr-­‐old peaches

Thinning 3-­‐yr-­‐old peaches

Applying wire clamps, pinching 2-­‐yr-­‐old apple trees

Hanging drier sheets for sDnk bug control

PalmeIe branch tying

The results of a timed trial between the APM and ladder-using workers.

“You put it at the beginning of the row, flip the switch, and … you press the foot pedal and the vehicle starts creeping in the row,” Singh says. “When it gets to the end of the row, it detects the open space with a laser and stops automatically. The workers atop the scissors lift use a joystick to drive it to the next row. … It’s an autonomous vehicle that does one thing only.”

That one-trick pony has a good trick, however. CASC loaned the vehicles to the agriculture extension programs at the partner universities, which in turn lent them to orchards. In one case, the extension program also set up a John Henry-style competition

decisions. Mobility means a vehicle, such

them more productive, more efficient and

as the APM, that allows autonomous mow-

more safe.”

ing and spraying. Manipulation means higher level functions such as pruning, thinning and harvesting.

CASC is funded at $6 million by the U.S. Department of Agriculture Specialty Crop Research Initiative, established by the

“Let’s jump forward into the future — 10,

2008 farm bill, with matching funds from

20, 30 years. A robot will go out and do

industry and university partners. Those part-

the pruning and the thinning and the har-

ners, in addition to Penn State and Wash-

vesting for you,” Bergerman says. “That’s

ington State, include Oregon State, Purdue,

not our focus right now. The focus is how

the USDA Agricultural Research Service

do you augment human workers to make

Appalachian Fruit Research Station, Vision

between the APM and a man on a ladder.

The video, which was shown at AUVSI’s Unmanned Systems Program Review 2012, showed that the automated platform is capable of making much better time.

CASC The overarching philosophy of the CASC program is to go “from information management to mobility to manipulation,” Bergerman says.

Information is data about crop health and yield, using such things as soil and moisture sensors, automated calipers and counters, and fruit counting and sizing systems, to allow farmers to make better management

CASC leaders Sanjiv Singh and Marcel Bergerman at the Mid-Atlantic Fruit and Vegetable Convention. AUVSI photo.

Mission Critical

•

Spring 2012

39

Two workers thin green fruit at Allan Bros. Orchards in Prosser, Wash. In timed trials in Pennsylvania comparing workers on the vehicle with workers on ladders, CASC obtained up to a 58 percent efficiency increase. Photo courtesy CASC.

Robotics, Toro, DBR Conveyor Concepts,

discuss how farmers might use it and how

Spensa Technologies and Trimble.

which leads to irregularity at the ends of

they feel about it.

the rows.

Its initiatives include developing an auto-

In the first year, Singh says, the planned

mated bug trap that can trap and count

“Somebody asked me about that today

equipment tended to be expensive and the

specific types of insects, an automated tree

when I was giving my talk: ‘I don’t believe

reaction was “pretty muted.” Now farm-

caliper and counter, a harvest augmenta-

your machines are going to be able to deal

ers and growers “have started to get a

tion system, and a scout vehicle that as-

with the ends of rows.’ I said you don’t want

little more excited about it … because they

sesses crop load.

us to deal with that, because you wouldn’t

can actually see connection to their work,”

be able to justify the extra cost associated

Singh says.

with that. You’d be better off making your

The program has included the work of 69 students in its three-year life, including sum-

Price is a huge consideration in the agricul-

mer students and those who have worked

ture market, he says.

with it for its whole existence.

“If you suggest a $5,000 user interface for

“One of the reasons the program was cre-

a military system, it’s not a big deal. No

ated, besides creating new technology, is

one even asks about that. You couldn’t have

to educate a new generation of ag engi-

a $5,000 control interface for an ag robot;

neers,” says Bergerman. “That’s been miss-

it just would not compute,” he says.

ing in the U.S. for a long time now.”

That also means that the machines can’t

End users

be made as smart as the orchard grow-

Singh and Bergerman attend events such as

have said that the machines couldn’t work

the recent Mid-Atlantic Fruit and Vegetable Convention in Hershey, Pa., where they spoke with Mission Critical. They use such events to demonstrate their technology and

40

Mission Critical

•

Spring 2012

rows slightly shorter and giving the vehicle a little bit more room to turn around in the flat area.”

In other words, “If you’re going to automate orchards, you’re going to have to build them to suit,” he says. “This is kind of the educational process. … We build these machines to spur people to ask questions.”

Brett Davis is editor of Mission Critical.

ers might like. Singh says some growers for them because they’re not smart enough to negotiate the ends of tree rows, where

For More Information:

the ground may slope up or down. Grow-

http://www.cascrop.com

ers tend to use as much land as they can,

Future ag robots need better grips, better planning, lower price tags TECHNOLOGY GAP

F

or robotics and unmanned systems to

ments,” the report says. Members of the

make serious inroads into the agricul-

working groups that wrote the report

tural market, toughness, ease of use

“identified novel robotic hands, tactile

and affordability may be the keys.

Systems to be used in the field in agriculture must be tough and able to endure rough weather, like their military counterparts, but

sensing and highly accurate, physically realistic simulators as important enablers for autonomous manipulation.”

• Planning: Robust, dynamic 3-D path

the farmers who will ultimately use them

planning “remains an open problem.

generally don’t have the same deep pock-

An important aspect of this problem

ets as the Pentagon.

is the notion of a robot’s situational

“In the agriculture world, we have this other kind of constraint, because you have some of the characteristics that you have in the military world, but the tolerance for price is much lower,” says Sanjay Singh of Carnegie Mellon University, the project director

awareness, i.e., the robot’s ability to autonomously combine, interleave, and integrate the planning of actions with appropriate sensing and modeling of the environment.”

• Sensing and perception: “Participants

of the multi-partner Comprehensive Auto-

believed that new sensing modalities as

mation for Specialty Crops program (for

well as more advanced, higher resolu-

more on that, see Page 38).

tion, lower cost versions of existing

An added complication is that most agricultural environments are relatively unstruc-

modalities would be areas of important progress,” the report says.

tured, not strictly regulated like a factory

• Mechanical hardware: Safety is key,

floor, and exist in an environment of inex-

and “participants therefore felt that

pensive labor.

inherently safer motors and mechanisms with increased strength-to-weight ratio

Roadmap needs In 2009, more than 140 robotics experts from industry, laboratories and academia teamed to produce a roadmap for the industry, entitled “A Roadmap for U.S Robotics — From Internet to Robotics.”

would represent an important enabling technology.” This area of research also

The Department of Agriculture has several requirements in mind for systems it may fund. It is seeking to back robotics research that will “enhance food production, processing and distribution,” and wants: • Automated systems for inspecting, sorting, processing and handling animal or plant products after they have been harvested or created • Systems for inspecting, sorting and handling plants and flowers in greenhouse or nurseries • Systems for vaccinating and deworming large numbers of live animals • Rapid-sensing systems for detecting defects, ripeness, physical damage, microbial contamination, size, shape and other attributes of plant and animal products • Systems for monitoring air and water quality.

Getting a grip Not all of this new technology needs to be very precise, however, especially for some agricultural use.

includes energy-efficient systems, as

In a paper published in 2006, researchers

robots will have to operate off-tether

from Turin University in Italy noted that agri-

for extended periods, and improved

cultural robots don’t need the precise move-

means of locomotion beyond wheels, as

ments required of factory robots and can

“outdoor environments oftentimes exhibit

They noted that in the field of service ro-

get by with movement errors in the range of

highly variable terrain properties” while

botics, which includes agriculture, improve-

a few millimeters. Such a level of precision

indoor ones may have stairs, ladders,

ments in several key areas would be re-

might lead to a really badly built automo-

ramps, escalators or elevators.

bile, but would be perfectly acceptable for

quired, including in the areas of:

picking fruits and berries and would allow

NRI

robots to be built for much less money.

manipulation systems function well

The National Robotics Initiative, announced

in carefully engineered and highly

last year, builds on the roadmap to acceler-

They also concluded that agricultural ro-

controlled environments, such as factory

ate the development and use of robotics in

floors and assembly cells, but cannot

the United States. One of the government

handle the environmental variability

agencies backing the program, which will

and uncertainty associated with open,

fund promising research, is the U.S. Depart-

dynamic and unstructured environ-

ment of Agriculture.

• Manipulation: “Currently, autonomous

bots might first find a home in greenhouses, which can be built to be much more regular than outdoor farmland. Greenhouse plants can be set up in an orderly way, and the greenhouses themselves could be built to allow robots to move up and down the rows.

Mission Critical

•

Spring 2012

41

Against the grain

In an industry sidelined by safety laws, could Bin Bot dig farmers out of a grain bin conundrum?

By Danielle Lucey

F

arm work may sound like it’s only dull

In 2010, 51 workers were engulfed and 26

his own, and, like quicksand, the grain en-

and dirty. But just how dangerous it can

were killed working in grain bins, accord-

gulfs a person in seconds, or it can fall on

actually be inspired one company to try

ing to a study by Purdue University. This

top of the worker, like an avalanche.

and put a stop to the rising trend of deaths

number is the highest on record and nearly

associated with grain bin accidents.

twice the prior year’s accident number.

Jerome Mack, now president of Leola, S.D.-

Workers die one of a few ways. Many grain

occur, are often catastrophic. According to

based Mack Robotics, lost his best friend in

bins, similar to silos but much smaller, em-

the U.S. Occupational Safety and Health

one of these accidents in 1998. At the time,

ploy a sweeping arm to push along grain

Administration, there have been more than

Mack ran a company that used robotics to

that isn’t emptied from the bin by gravity. If

500 grain bin explosions in the last 35

enable easier swine reproduction through

the arm gets caught, a worker may enter

years, resulting in 677 injuries and 184

artificial insemination. But inspired to keep

the bin and try to push or kick it along. If

fatalities. Workers can also perish falling

the number of grain bin accidents down, he

his clothes get stuck on the arm, it can tear

from tall structures in a grain handling fa-

refocused and formed his own grain robot-

off limbs or rip apart his body.

cility.

Suffocation is the main cause of death

Mack Robotics’ goal is simple: to get the

ics company.

Due to the large amount of dust in a grain bin, explosions are possible and, if they

when a worker attempts to move grain on

The new, explosion-proof version of Bin Bot. All photos courtesy Mack Robotics.

The Bin Bot lowering the sweep after moving it to its desired position.

42

Mission Critical

•

Spring 2012

number of grain bin deaths down to zero.

The company formed one year ago, and given a change in OSHA policy, its timing might prove impeccable.

The robot is 100 percent teleoperated by

demonstrations to allow farmers the chance

one person through wireless communica-

to see if Bin Bot will work at their location.

tions and cameras with lighting supplying a video feed. This is a decrease in the number of people it typically takes to work a

Recognizing the dangerous reality the

grain bin, which is between three and four.

United States’ approximately 13,000 grain

The robots currently cost between $15,000

elevator operators face, OSHA has a zero-

and $17,500 depending on attachment

entry policy with grain bins and heavily

options.

fines employers that allow humans to enter.

The problem? Currently, human entry is the only way to ensure grain bins are thorough-

The company is also working on an updated version, one that’s explosion-proof, useful given the widespread grain bin explosion issue and that it’s an industry standard for much farm equipment.

Though Outtrim says cannot speculate the

Attachment options include a scoop, which is useful for typical housekeeping and

cost savings involved using Bin Bot, that’s not Mack Robotics’ goal.

helps clean up the grain and push it out

“Our ultimate goal is to reduce the number

of the bin. The push attachment can move

of injuries and ultimately deaths that are oc-

In the market of robotic grain cleaners, Kris-

more product than the scoop. Finally there

curring. No matter how big or small [the

tin Outtrim, marketing manager for Mack

is a forklift, which is helpful when a grain

operation], it’s going to be an asset to that

Robotics, says she’s yet to see a similar

sweeper is acting up. It can apply pressure

company because of that effect.”

technology at the trade shows she attends.

to it, and if the grain avalanches, it can lift

ly cleaned and emptied.

So what are farmers to do?

“That is a very, very good question, because that is what these elevator managers and farmers and all these people are

the sweep up.

Though the company is still in a promotional phase with Bin Bot, it, in addition

Danielle Lucey is managing editor of Mission Critical.

to trade shows, has begun setting up live

asking. … What are we supposed to do?” says Outtrim. “For one thing that’s how they’ve always done it, and for another thing nobody’s offered a solution. A lot of the comments we get at the shows are ‘We need something like this because they’re telling us we can’t do this, but nobody is providing a solution.’”

Bin Bot has been in a prototype phase since November 2011. In addition to the inspiration garnered from the loss of his friend, a farmer approached Mack interested in

The Bin Bot does some housekeeping inside a grain bin.

turning his hog solution into a grain bin solution. Though Mack couldn’t turn the one machine into a multifunctional system, it inspired him to create the brand new Bin Bot. Using his previous experience, Mack worked with a team of engineers to make the Bin Bot. One of the requirements was to make it powerful but small, able to fit through a 24-inch-diameter opening, the smallest bin door Mack has seen.

The machine is battery operated, with a 12hour life. The machine is about 6-by-2 feet and weighs 800 pounds.

SCAN IT

or

Click IT:

Click the photo or scan this barcode with your smartphone to see Bin Bot work alongside a sweep

Mission Critical

•

Spring 2012

43

Flying where the green grass grows END USERS

K

eenan Amundsen is a University of

It is also superior to the current industry

Nebraska-Lincoln

profes-

practice of driving cameras over the turf by

sor of turfgrass genetics who is using

mounting them to golf carts and then piec-

assistant

some new technology to make his job more efficient.

ing together the resulting images.

mant longer than cool-season turf species.

Pixobot’s Singh says the helicopter used in the initial tests employed a Canon SD750 point-and-shoot, but the spring iteration will

“With the helicopter, we can just hover

use a Sony Nex 5N, which features a larg-

As a turfgrass breeder, he studies the turf to

over the area. We get a live feed through

er sensor and better resolution.

predict areas prone to stress, which allows

the digital camera, so we can orient the

for site-specific management. He needs to

helicopter the way we want and take the

evaluate his green charges on a variety of

pictures,” Amundsen says. “I can sit at my

visual traits, including shoot density, color

desk and drink my coffee in the air condi-

and uniformity. With 40 acres of small

tioning while the computer is analyzing the

“You have to go through the plots sever-

grass plots to evaluate, it can be a time-

images, and I don’t have to get out there for

al times, and that just adds to the time it

consuming job.

hours at a time.”

takes,” Amundsen says. “With one picture,

“Walking through the plots takes several

The Pixobot will be used this spring to study

days. If we can speed up the process by

the timing of spring green-up of buffalo

taking an image and then analyzing the im-

grass, a warm-season turf, which stays dor-

The plots are also usually rated for one variable at a time, meaning multiple passes are required for a walking evaluation.

you can analyze that picture and pull out all that data from the one image. It greatly improves the efficiency of the whole process.”

age computationally, it dramatically accelerates the whole process,” he tells Mission Critical.

He found a way to do just that at a chance meeting last fall with Vishal Singh, a multimedia design specialist for UNL’s edMedia, who also happens to own a company called Pixobot, which markets a small helicopter for aerial surveilling and other uses.

“We got to talking about his interests and my interests, and it was just a natural fit,” Amundsen says.

They did some testing with the vehicle last year, mounting an off-the-shelf digital camera to the helicopter to photograph the turf from the air.

“The idea is to mount it with a digital camera so that we can hover over our turf areas, taking several static images. Then we bring the pictures back to the office and do the computational analysis to start rating the different plots,” Amundsen says.

The Pixobot sped up the process dramatically, and is a better alternative to fixedwing aircraft, which must make multiple passes, and blimps, which are more prone to wind drift.

44

Mission Critical

•

Spring 2012

Vishal Singh flies the Pixobot.

Photo courtesy Keenan Amundsen.

REACH THE

UNMANNED SYSTEMS AND

ROBOTICS COMMUNITIES Capture the attention of the most influential leaders on a daily, weekly and monthly basis. Advertise in Unmanned Systems magazine, unmanned systems:mission critical, Unmanned Systems eBrief or AUVSI.org.

For more information call +1 571 225 7779 or email marketing@auvsi.org