Pick and Choose

A strawberryharvesting robot developed by Harvest CROO Robotics moves through the rows of a strawberry field.

Photo Courtesy of Harvest Croo Robotics

PICK AND CHOOSE

Robots Are Coming to a Farm Field Near You

BY ERIC TEGLER

Within the next decade, the term “mechanized agriculture” will refer to more than the stereotypical combines, feeders, and irrigation systems we’ve come to associate with farming. A new wave of robotic mechanization is on the cusp of washing over the industry in a drive to significantly increase agricultural productivity even as it reduces the number of human hands involved.

“Robots” have been in field (or above) in the form of self-driving tractors, materials-handling machines, and aerial imaging/sensing unmanned aerial vehicles (UAVs) for more than 10 years. They’ll soon be joined by new artificial intelligence (AI) enabled counterparts that will assume the tasks of weeding, seeding, spraying, and harvesting.

According to a study just released by London-based market research firm TechNavio, the global market for autonomous farm equipment will see compound annual growth of approximately 15 percent by 2023. Advances in AI, machine vision, and manipulative robotics are making such growth possible, but the catalysts for developing agribots are inexorably rising demand for food and rising labor costs.

DEMAND AND LABOR

“We’re going to have to innovate to make up for future [food] demand,” said Manoj Karkee, Ph.D.

Karkee is an associate professor at Washington State University’s Center for Precision and Automated Agricultural Systems. He is an expert in agricultural robotics who leads a research team working on the employment of autonomous UAVs to deter birds from damaging crops.

The food demand that Karkee references is expected from a global population rise of more than 2 billion by 2050, according to the United Nations’ World Population Prospects: The 2017 Revision. Increased wealth in Asia and Africa means that consumption per capita will also increase, further spurring demand. Along with reducing crop waste (15 to 30 percent annually), raising agricultural productivity will be absolutely vital.

Farm laborers pick squash on a farm in Virginia. Issues of labor cost and availability have helped to spur development of agricultural robots.

USDA PHOTO BY LANCE CHEUNG

Researchers, some growers, and the nascent agricultural robotics industry posit a shortage of farm labor as holding productivity back. Karkee points to instances of crops going to waste for lack of labor to harvest them, adding, “Human beings deserve better than the working conditions in agricultural hard labor.”

Nevertheless, the great majority of annual crop production is successfully harvested, and USDA statistics present a picture of a labor force that has remained consistent or grown. The USDA’s Hired Farm Labor – Workers and Payroll table from the 2012 Census of Agriculture put the number for hired farm labor in the United States at 566,469. The Bureau of Labor and Statistics assessed the number of agricultural workers in 2016 to be 856,300, and projects total employment to remain approximately the same through 2026.

Labor cost is generally agreed to account for more than one-third of typical farm overhead. It is demonstrably rising. Nationwide minimum wage increases are in force and laws in California now require overtime pay to workers beyond a 40-hour work week. Given the time-critical nature of harvest pick cycles, overtime is inevitable, further increasing labor cost.

While automation is generally described as a response to worker shortages, increasing pay could actually expand the pool of farm labor. This broader picture suggests that the drive for agricultural automation hinges at least as much on cost as labor availability.

MANIPULATIVE AND MULTITASKING

There are two basic forms of robotics, Karkee explains: navigational robotics and manipulative robotics. Navigational robotics facilitates autonomous movement from place to place while manipulative robotics enables physical motions for specified exercises.

“A lot of applications combine these two,” he said. “Fruit harvesting robots may have navigational systems which move the machine to a desired location combined with a ‘hand,’ which is the manipulation part of it used to pick apples or other fruits.”

Harvesting fruits or vegetables requires deft movement and feel as well as the ability to see and choose which objects to pick and which to leave aside. Around 18 billion apples are picked in Washington state every year, meaning there are at least 18 billion movements of human hands to harvest them, Karkee says.

Humans can pick an apple every second or so, a tough ask for machines that need to identify targets, evaluate sensor data to make a pick/no pick decision, and then – if a “pick” decision is reached – navigate within apple tree structures, subtly avoiding branches and gently reaching for the fruit. Not surprisingly, robotic manipulation is the real challenge in agriculture.

“There’s been a lot of effort in these manipulation areas, and we’re making progress using new artificial learning techniques, faster computational systems, and more robust robotics,” Karkee said.

Indeed, machine learning, image processing, and sensor and data mapping are being integrated into agribots tasked with precision weed control, fertilization and seeding, and crop harvesting. These machines rely on the same PNT (position/ navigation/timing) infrastructure that the American military, utility, and financial sectors depend on and bring with them a significant degree of risk concentration.

Well over 70 companies and organizations have determined to meet the challenge of increasing productivity and grasping business opportunity. Startups like Blue River Technology, FarmWise, Abundant Robotics, Cainthus, and FFRobotics populate an IT-influenced robotics developer market poised to expand. One of these – Harvest CROO Robotics – illustrates the blend of established technology adoption and new innovation.

An apple-picking robot developed at Washington State University (WSU) combines navigational and manipulative robotics to enable the machine to move through an orchard and pick fruit.

IMAGES COURTESY OF MANOJ KARKEE, #AGROBOTICS TEAM, AND WASHINGTON STATE UNIVERSITY

PICK AND PREDICT

Strawberries are tough to harvest. Hidden among the leaves of plants at ground level, they bruise easily and ripen at different times. For every hour a strawberry is off the plant and not refrigerated, it loses a day of shelf life. So they are still harvested by hand. That’s a challenge for growers, including Plant City, Florida-based Wish Farms.

When an H2A farm labor contractor walked out, taking his crew with him, during last year’s harvest, Wish Farms lost $500,000 in fruit in ground. “We could not find the domestic labor in order to pick that fruit,” said Harvest CROO Robotics CEO Paul Bissett. “That was not a fun situation.”

The “CROO” in Harvest CROO stands for Computerized Robotic Optimized Obtainer – a strawberry-picking robot. The company was established in 2013 by Wish Farms’ owner, Gary Wishnatzki, and robotics engineer Bob Pitzer. Backed by a coalition of growers representing 70 percent of the $3.3 billion strawberry industry by value, Harvest CROO Robotics is about 18 months away from placing its strawberry harvester on the market.

Manoj Karkee, Ph.D., demonstrates how an instrumented glove was used to understand the forces and motion involved in manual apple picking – information that then guided the design of robotic hands for the WSU apple-picking robot.

IMAGES COURTESY OF MANOJ KARKEE, #AGROBOTICS TEAM, AND WASHINGTON STATE UNIVERSITY

The CROO system consists of a GPS-enabled, self-driving, crop-straddling vehicle on which 16 robots are mounted. Each robot has paddles that gently move leaves aside to expose the fruit. A pick wheel then circles the plant using lidar, stereoscopic cameras, and AI to locate ripe strawberries. Rubber claws then extend from the wheel to pluck the fruit from the stem. The fruit is then deposited into supermarket-ready packaging via a mechanism still in development.

Bissett says each CROO system is capable of harvesting an acre per hour, the equivalent of a 25- to 30-person picking crew. The robots can also work at night when humans can’t and on weekends when they don’t want to. Picking in cooler temperatures reduces refrigeration requirements and energy consumption. Continuous robotic picking meshes well with three- to four-day pick cycles, revisiting the same acreage as strawberries ripen, without paying overtime to workers.

Robots won’t walk off the job, increasing harvest reliability and hence, the investment risk that farmers are willing to take at the outset of a season. “Increasing the surety of harvest is a big deal,” Bissett said. The harvest-per-acre cost is about $10,000 in Florida, and, “If we can assure that every acre is harvested, yields will go up, costs will go down, and growers will be more likely to make that bet at the beginning of the season,” he said.

Harvest CROO Robotics is currently on the fifth iteration of its platform, with the major design components locked in. Testing with a sixth-generation platform will continue this year, and the company expects to generate revenue from the system, getting berries picked at scale in 2020.

“Assuming we hit those milestones, we’ll move to commercial picking operations sometime over the next year to 18 months,” Bissett said.

FIELD OF OPPORTUNITY

As Harvest CROO Robotics goes to market, it envisions leasing its robots to farmers rather than offering them as capital equipment. This model would essentially replicate the on-demand contract labor that growers currently hire, maintaining flexibility for both farmers and the harvest robot provider, relieving the former of the need to acquire, maintain, and operate such complex systems.

“Rather then selling them a lot of capital equipment which they don’t currently buy,” Bissett explained, “we’re going to say, ‘You can choose traditional harvest services labor or you can have it automated with us.’”

The idea of “agribots as a service,” similar to software as a service, aligns well with the tech startup landscape of the agricultural sector. The lion’s share of research and development carried out thus far has been done by small firms. That’s starting to change, Karkee observes, with larger robotics and agribusiness concerns quietly assessing the agribot market internally.

The Pitzer wheel on the Harvest CROO robot circles strawberry plants to locate ripe berries that it then picks with rubber claws. A CROO system can harvest an acre per hour, the equivalent of a 25- to 30-person work crew. Harvest CROO Robotics envisions leasing – rather than selling – its robots to farmers.

Photo Courtesy of Harvest Croo Robotics

“There has been some effort in India and elsewhere. They contact me from time to time regarding possible investment, mostly looking at the status of artificial intelligence and machine learning development.”

Blue River Technology is a concrete example of the trend. The company has launched a weederadicating robot called See & Spray, which it markets as a safer solution to herbicide resistance, claiming it significantly reduces crops’ exposure to chemicals.

Pulled by a tractor, See & Spray employs deeplearning algorithms similar to facial recognition algorithms with cameras and highly capable processors to see every plant in a field, distinguishing weeds from crops and very precisely spraying herbicide on targeted weeds or applying fertilizer to targeted plants. Targeted herbicide or fertilizer application reduces chemicals cost, while the robot replaces up to eight people carrying out the same task, according to Blue River Technology.

John Deere acquired Blue River Technology in September 2017 for a reported $305 million. Deere joins agricultural equipment makers AGCO, CLAAS, CNH Industrial, and Kubota with significant investments in and/or ownership of agricultural robotics developers.

Despite such notable investment, the field remains largely open with questions of who will lead the market (argibusiness or Silicon Valley) joining questions about which devices (drones, harvesters, sprayers, mobile software) will be most in demand.

For Karkee, the question of which bots will proliferate hinges on which show the most value in the next two to three years. Beyond that, the professor puts wider adoption somewhere around five years.

“That’s just my educated guess,” he said. “There are a lot of uncertainties to deal with.”

The supply and cost of labor will surely play a central role along with input from organized labor. Left basically unexamined as yet is the question of whether a new regulatory framework will emerge around agricultural robots. Their impact on labor conditions and the environment as well as security issues arising from their connectedness will likely draw the attention of regulators.

Obviously, it won’t be just the robots picking and choosing in the field. Industry and broader society have a lot of decisions to make going forward.