Design World 2023 Robotics Handbook by WTWH Media LLC

www.designworldonline.com November 2023

2023 ROBOTICS

HANDBOOK

We came. We saw. We automated. Our storing, conveying and picking robots and solutions make automating your warehouse simpler and quicker than ever. movu-robotics.com

Servo Gearheads for AMR/AGV Propulsion

HPG Gearheads

The Harmonic Gearhead® gearbox design allows for backdrivability while ensuring highefﬁciency performance, maximizing elapsed operational time before recharging is necessary. High-capacity cross roller bearing supports heavy payloads.

High Efﬁciency Up to 95% Efﬁciency

Low Backlash for Life

Innovative ring gear inherently compensates for interference between meshing parts, ensuring consistent, low backlash for the life of the gearhead.

High Load Capacity Output Bearing A Cross-Roller bearing is integrated with the output ﬂange to provide high moment stiffness, high load capacity and precise positioning accuracy.

Easy Mounting to a Wide Variety of Servomotors

Quick Connect® motor adaptation system includes a clamshell style servo coupling and piloted adapter ﬂange.

Flange Output Standard

Flanged output is standard for convenient, stable wheel mounting.

Harmonic Drive LLC | 42 Dunham Ridge | Beverly, MA 01915 | 800.921.3332 | www.HarmonicDrive.net Harmonic Drive®, Harmonic Planetary®, Harmonic Gearhead® and Quick Connect® are registered trademarks of Harmonic Drive LLC.

DW 9x10.875 HPG Ad.indd 1

10/25/23 10:14 AM

Contents

11 • 2023 therobotreport.com

ON THE COVER

Inside the development of Apptronik’s Apollo humanoid PAGE 14

COLLABORATIVE ROBOTICS

SENSORS

3D cameras enable automatic loading of pressed parts Four 3D cameras from IDS Imaging each record a part of the entire image field, resulting in data from multiple perspectives for a higher quality 3D point cloud.

How computer vision increases productivity & sustainability in agriculture Integrating computer vision and other advanced technologies like sensors, robotics, and AI will result in more efficient and more automated machines that can treat every single plant at a micro level.

Commercializing tactile sensors for robot dexterity Addressing practical issues such as usefulness, reliability and affordability.

Welding cobot delivers 10x production boost for marine exhaust manufacturer DeAngelo Marine Exhaust has increased weld speed, lowered lead times, and improved quality using its new cobot colleague.

DESIGN & DEVELOPMENT

Inside the development of Apptronik’s Apollo humanoid The company’s R&D work on exoskeletons and bipedal locomotion for the U.S. government and Department of Defense has it primed for the humanoid era.

Teaching robots new tricks with Generative AI Researchers are exploring how large language models can be applied to robotics design, model training, simulation, control algorithms and more.

30 2

Humanoids getting their feet wet in logistics Humanoids offer the potential flexibility and scalability to be used within existing warehouses alongside humans to help pick, move and sort goods.

November 2023

SOFTWARE

Bridging the gap between research, industrial robotics using ROS Tormach created a ROS-based industrial robotic manipulator and control system that avoids “black box” issues that can plague modern robotics applications.

www.therobotreport.com

THE ROBOT REPORT

231002_CJCI_DW_US.indd 1

10/2/23 5:05 PM

ALL IN

AI-based code reading is just minutes away.

LEARN MORE

Lector One Box is a registered trademark of SICK Sensor Intelligence

WWW.SICK.COM

The new generation of RENCO rotary encoders Motor commutation at the push of a button Are you looking for a rotary encoder that is easy to install and reliable to operate? Then meet the new R35i and R35iL models from RENCO. Once you’ve used their built-in mounting aid for fast and easy installation, simply push a button to electronically adjust the motor commutation

ders Enco ery v for e or mot

sps.heidenhain.com

signals. Afterwards, rest assured in dependable operation thanks to features for monitoring and diagnostics never before implemented in an incremental rotary encoder. That’s what makes these encoders the ideal motor feedback solutions for stepper and BLDC motors.

HEIDENHAIN CORPORATION www.heidenhain.us

DESIGN WORLD Follow the whole team @DesignWorld

EDITORIAL

VP, Editorial Director Paul J. Heney pheney@wtwhmedia.com @wtwh_paulheney

DESIGN GUIDES Brought to you by

Key technology content produced by the editors of Design World. Available for FREE download in individual PDFs. Find these technical design guides on the Design World Design Guide Digital Library: www.designworldonline.com/ design-guide-library

Managing Editor Mike Santora msantora@wtwhmedia.com @dw_mikesantora Executive Editor Lisa Eitel leitel@wtwhmedia.com @dw_lisaeitel Senior Editor Miles Budimir mbudimir@wtwhmedia.com @dw_motion Senior Editor Mary Gannon mgannon@wtwhmedia.com @dw_marygannon Senior Editor Rachael Pasini rpasini@wtwhmedia.com @WTWH_Rachael Associate Editor Heather Hall hhall@wtwhmedia.com @wtwh_heathhall

CREATIVE SERVICES

VP, Creative Director Matthew Claney mclaney@wtwhmedia.com @wtwh_designer Art Director Allison Washko awashko@wtwhmedia.com Senior Graphic Designer Mariel Evans mevans@wtwhmedia.com @wtwh_mariel Graphic Designer Shannon Pipik spipik@wtwhmedia.com @wtwh_shannon

Director, Audience Development Bruce Sprague bsprague@wtwhmedia.com WEB DEV / DIGITAL OPERATIONS

Web Development Manager B. David Miyares dmiyares@wtwhmedia.com @wtwh_webdave Senior Digital Media Manager Patrick Curran pcurran@wtwhmedia.com @wtwhseopatrick Front End Developer Melissa Annand mannand@wtwhmedia.com Software Engineer David Bozentka dbozentka@wtwhmedia.com DIGITAL MARKETING

VP, Digital Marketing Virginia Goulding vgoulding@wtwhmedia.com @wtwh_virginia Digital Marketing Manager Taylor Meade tmeade@wtwhmedia.com @WTWH_Taylor Digital Marketing Coordinator Meagan Konvalin mkonvalin@wtwhmedia.com Digital Marketing Coordinator Francesca Barrett fbarrett@wtwhmedia.com Webinar Coordinator Emira Wininger ewininger@wtwhmedia.com

Executive Editor Steve Crowe scrowe@wtwhmedia.com @SteveCrowe

Editor Mike Oitzman moitzman@wtwhmedia.com @MikeOitzman

Editorial Director Associate Editor Eugene Demaitre Brianna Wessling edemaitre@wtwhmedia.com bwessling@wtwhmedia.com @GeneD5

November 2023

Events Manager Brittany Belko bbelko@wtwhmedia.com Event Marketing Specialist Olivia Zemanek ozemanek@wtwhmedia.com VIDEO SERVICES

Videographer Garrett McCafferty gmccafferty@wtwhmedia.com Videographer Kara Singleton ksingleton@wtwhmedia.com PRODUCTION SERVICES

Customer Service Manager Stephanie Hulett shulett@wtwhmedia.com Customer Service Representative Tracy Powers tpowers@wtwhmedia.com Customer Service Representative JoAnn Martin jmartin@wtwhmedia.com Customer Service Representative Renee Massey-Linston renee@wtwhmedia.com Customer Service Representative Trinidy Longgood tlonggood@wtwhmedia.com Digital Production Manager Reggie Hall rhall@wtwhmedia.com

FINANCE

Digital Production Specialist Nicole Johnson njohnson@wtwhmedia.com

Controller Brian Korsberg bkorsberg@wtwhmedia.com

WTWH Media, LLC TOP 1111 Superior Ave. WORK PLACES 2022 26th Floor Cleveland, OH 44114 Ph: 888.543.2447

2011 - 2020

2013 - 2017, 2021

Digital Design Manager Samantha King sking@wtwhmedia.com Marketing Graphic Designer Hannah Bragg hbragg@wtwhmedia.com Digital Production Specialist Elise Ondak eondak@wtwhmedia.com HUMAN RESOURCES

2014 - 2016

Medical Design & OUTSOURCING 6

Events Manager Jen Osborne josborne@wtwhmedia.com @wtwh_jen

Webinar Coordinator Dan Santarelli dsantarelli@wtwhmedia.com

Accounts Receivable Specialist Jamila Milton jmilton@wtwhmedia.com

EDITORIAL

EVENTS

www.therobotreport.com

Vice President of Human Resources Edith Tarter etarter@wtwhmedia.com

THE ROBOT REPORT

All products available directly on

from

WHAT’S NEW RULAND

RULAND.COM TOOL-LESS ADJUSTMENT COMPONENTS • Adjustable handles and knobs replace standard hardware and can be used to torque components without tools. • Levers can be used with Ruland shaft collars for quick installation and adjustment.

Ruland Manufacturing is proud to support the FIRST Robotics Competition as a Gold Supplier of precision shaft collars & couplings.

PRECISION COUPLINGS • Beam couplings accommodate light weights and high misalignment. • Zero-backlash jaw couplings for vibration dampening and a variety of performance options. • Bellows couplings for high size to torque ratio.

MODULAR MOUNTING SYSTEMS

SHAFT COLLARS • Thousands of size and style options. • Proprietary manufacturing processes ensure superior fit and finish.

• Assorted components allow you to build small assemblies for mounting sensors, conveyor rails, machine guards, and more. • Optional pre-designed kits make it easier to select the right system for your application.

• 8 material options.

VIBRATION ISOLATING COMPONENTS

QUICK CLAMPING COLLARS

• Rubber bumpers are ideal as end stops or mounting feet.

• No tools required to install, remove, and adjust.

• Vibration isolation mounts can be sandwiched between components to dampen shock loads.

• Best suited for systems that require frequent adjustments.

• Both types can have studs or tapped holes.

ruland.com | sales@ruland.com

Collaborative Robotics

DeAngelo Marine Exhaust had nearly given up automating complex, manual TIG welds until the company came across Hirebotics’ Cobot Welder that uses a UR10e cobot arm equipped with a welding torch. | Universal Robots

Welding cobot delivers

10X PRODUCTION BOOST for marine exhaust manufacturer

DeAngelo Marine Exhaust has increased weld speed, lowered lead times, and improved quality using its new cobot colleague. STEVE CROWE EXECUTIVE EDITOR ROBOTICS

November 2023

DeAngelo

Marine Exhaust had nearly given up automating complex, manual TIG welds until the company came across Hirebotics’ Cobot Welder. The Florida manufacturer of exhaust parts has now been able to increase weld speed tenfold, lower lead times, and improve quality using its new cobot colleague, which also helped DeAngelo break into new markets. 65 days. That’s how far behind DeAngelo Marine Exhaust was on lead times at its worst point. Based in the heart of ‘the yachting capital of the world’ in Fort Lauderdale, FL, DeAngelo is a synonym

www.therobotreport.com

for aircra quality welds on marine engine exhaust parts. But the demand for DeAngelo’s exhaust parts exceeded the capacity of the company’s labor-intensive production. And CEO Justin Montes had no luck hiring more manual welders. “What we do here is custom fabrication at the highest level,” he said. “You’re not going to pull in a welder off the street and have him fabricating these systems.” That le the CEO to face his own worst nightmare: letting down the finest boat builders, naval architects and engine companies in the world by delivering too late.

THE ROBOT REPORT

“That’s how I lose the account and never get it back,” Montes explained. “We were at an impasse, experiencing significant growing pains, and knew we simply had to scale up; lower our lead times, lower our price, and increase our capacity.” Montes had heard about ‘these cobots’ coming out and said, “It seemed like a good idea. So, we started sending out parts to seven of the big robotic welding companies to see if they could do a demo weld.” The results were underwhelming. A box of returned demo parts shows products with holes in the weld seam and irregular globs of molten metal. “They were all struggling to match our quality. When I contacted Universal Robots (UR), I was already three months into this and had almost given up,” Montes said. UR referred Montes to welding partner Hirebotics, a company that has incorporated UR’s collaborative robots into the Cobot Welder: a complete solution that includes the UR10e cobot arm equipped with a welding torch, the Beacon app for teaching, a welding power source, and a versatile welding table to perform high-quality welds on a variety of workpieces. “At this point in the game, I really thought I was just wasting my time,” said Montes as he recalls getting a text om a Hirebotics co-founder, letting him know they had received his parts. Four minutes later, he got another text, this time with a picture showing the part, perfectly welded up, no gaps. “I figured this was too good to be true and that the entire text exchange was staged,” said Montes, who decided to fly up to Hirebotics in Nashville, TN, to see it first-hand. When he got there, he was able to use Hirebotics’ app-based interface to teach and run the Cobot Welder in just 20 minutes with no prior robotics experience. “I was running parts like I’ve been running robots my whole life. A er that, I gave them a PO, and we got the robot in here. And that’s when the fun started.” From 2 to 20 inches per minute The Cobot Welder has quickly enabled the company to catch up on lead times and reduce defect rates. “We went om being eight weeks over capacity back to baseline in a matter of days,” said Montes. He attributes this accomplishment to the speed of the cobot that MIG welds 20 inches per minute compared to the 2-5 inches per minute when TIG welded manually. Montes runs his hands over a weave weld done by the cobot and said, “Any welder worth his salt

THE ROBOT REPORT

November 2023

Collaborative Robotics would have a hard time believing this is a MIG weld. Once the operators started toying with the settings, we realized we could get the welds to look however we wanted them to: fat welds, skinny welds, butt joints, lap joints—we are even building fixtures to do seam welding. He also emphasizes the ability to dial the cobot in to do full penetration welds: “This was a very big concern for us. A leak could literally sink a $100 million yacht. But we have found no leaks by the cobot.” ROI: one order paid off the cobot The day the Cobot Welder hit the floor at DeAngelo, the company received a large order om the Coast Guard. The parts were 12 inches in diameter, requiring a circular weld five times. “Perfect fit for the cobot,” Montes said. “We went om months to just days turning that order around. If I put dollars into it, with the departmental overtime and everything, that first order paid off my cobot.” The CEO explained how he was also able to shi large portions of the welding work om fabricators, the highest-paid employees in the shop, to the machine operators that now operate the cobot. “Prior to the cobot, we would throw all the parts needed on a cart, and our fabricators would have to weld them all together,” he said. “Once the cobot was up and running, we created a pre-fabrication department where all these high-volume sub-assembly parts could get welded before they got to the

DeAngelo Marine Exhaust of Fort Lauderdale, FL, occupies over 30,000 sq. feet of manufacturing and engineering facilities suited to state-of-the-art marine exhaust systems and diesel particulate filter systems (DPF) production, service and repair. | Universal Robots

fabricator. From a financial perspective, now that the welding is performed by a machine operator rather than a fabricator, our cost-per-part has plummeted.” Winning over welders When the Cobot Welder first arrived at DeAngelo, not everybody shared Montes’ enthusiasm, he recalled. “We have all these great fabricators that see a robot in the building. So immediately the wall is up: ‘What is this machine?’ they would ask.” He explained how he had them run a few sample parts, and kept dialing in the welder until the welds looked right. That was a big ‘aha moment’ for employees. “These same people that were a aid of the cobot, they are now calling me on the

The Cobot Welder has enabled DeAngelo Marine Exhaust to catch up on lead times and reduce defect rates. The company went om being eight weeks over capacity back to baseline in a matter of days. | Universal Robots

November 2023

www.therobotreport.com

weekends, suggesting new parts to run on the cobot,” said Montes. “You could see their brains just explode; now everybody’s embracing it. It’s been a big morale booster for our work culture.” Stan Sherwin, one of the manual TIG welders at DeAngelo, explained how the cobot saves him time: “When the parts get to me, the sub-assemblies are already welded by the cobot and it makes it a lot easier because I don’t have to try to maneuver it around in a tricky position. The welds by the robot look beautiful. And it’s a lot faster.” Easy programming through app-based interface Camilo Aguilar, production manager at DeAngelo, was one of the employees who greeted the cobot with initial skepticism. “I really had my doubts,” he admitted. “Air-cooled MIG welding on stainless steel and pipes? But I was wrong. To be honest, that’s an awesome machine,” said Aguilar, adding how he’s renamed the cobot ‘my baby.’ Though he had no prior robotics experience before the Cobot Welder arrived, the production manager is now the one teaching the operators how to run it. “The programming is not hard; our operators pick it up so fast,” he said. As Montes put it, there was “no more writing 10-page programs in G-code or yearlong learning curves.” Instead, operators teach the robot by simply grabbing the robot arm and moving it through the desired weld trajectory, adding waypoints in Beacon, the Cobot Welder’s app-based interface that runs on any THE ROBOT REPORT

Collaborative Robotics handheld device or tablet. “If you want to weld a circle, press circle, click three points—it knows that it needs to weld a circle along those three points. You want to do a line? Two points, it’s that easy,” explained Montes, who is working on fixtures that will enable the Cobot Welder to weld 25-50 products in the same cycle.

Beacon app. “I have access to all my production data anywhere in real time through the app,” he said. “Everything is a number; we can see how long the arc was on, how much gas and wire we used. And if it looks like Shift A is more profitable because they ran faster, we can copy those settings over to Shift B.”

Troubleshooting in real time Having experienced long response times requesting tech support for the company’s CNC water jets and laser cutters, Montes was positively surprised by the quick replies he received whenever his team had a question about the Cobot Welder. “Trouble-shooting on the cobot is amazing,” he said. “We run the programming off an iPad Mini we bought and there’s a support feature in the app, where you can get somebody on the line within seconds.” He can also take pictures of the issues and submit them. “I don’t need to be a welder,” he said, “but I can send a picture and say, ‘Hey, is this the right setting?’ And in seconds, you’ve just eliminated being down for days if not weeks.”

Future applications The Florida exhaust maker is just getting started down the cobot path. Montes pulls out a map of the shop floor, pointing out fabrication tables where he envisions a Cobot Welder going. “Our goal is to have a cobot with a UR arm on all the fabrication tables of our seven lead fabricators. As they’re jigging parts up, making sure they’re hitting all their target points, they can have the cobot handle the welding in the meantime and just streamline everything.” Jumping from welding two to 20 inches per minute has also enabled DeAngelo to break into new markets. “When you’re dealing with a big engine manufacturer, they’re not trying to pay premium, custom-fabricated part prices,” said Montes, explaining how he has to bring his costs and lead times down significantly to address that market and compete with manufacturers in India as well. “In order to break into those first-fit OEM manufacturers where you buy an engine with my part already on it, we have

Access to weld data from anywhere DeAngelo’s CEO, who has a black belt in Six Sigma lean management and is a selfprofessed ‘numbers nerd,’ emphasizes the production data provided by Hirebotics’

FLEXCODER MOTOR & POSITION FEEDBACK FOR COBOTS Performance data MSAC200    

Magnetic, rotary, true absolute Flat, compact height 10 mm Resolution up to 20 bit Large hollow shaft for cables and media hoses

www.siko-global.com/p/msac200

Nobody at DeAngelo Marine Exhaust had any experience with robotics programming before the arrival of the Cobot Welder. However, the system’s intuitive interface doesn’t require yearlong learning curves. Instead, operators teach the robot by simply grabbing the robot arm and moving it through the desired weld trajectory. | Universal Robots to boost production up to 10x, which is what the Cobot Welder has already helped us do.” He added with a smile, “Once more engine manufacturers figure out ‘Oh, DeAngelo can do this now’, I don’t just need more cobots; I need a bigger building.” RR

Elevate Cobot Productivity Applied Cobotics, The Cobot Feeder Running 24/7 on the Strength of PBC Linear Products

Steel Shafting

Ball Bearing Pillow Block

CNC Rolled Lead Screw

The biggest issue that this machine is solving is cobot downtime. Without some sort of automated loading and unloading system, cobots will inevitably sit idle, falling far short of their desired potential. The Cobot Feeder is built to continuously feed parts to the cobot. PBC Linear has implemented The Cobot Feeders with cobots on their CNC mills and lathes that produce their signature bearings. With roughly 80 CNC machines in their shop and 15 cobot stations in operation, The Cobot Feeder has boosted cobot production up to 1600%, and created a more flexible manufacturing model while elevating profits.

MTB Series Actuator

The Cobot Feeder is easy to install, operate, and can be purchased by itself or with a cobot. In addition, PBC Linear offers production of custom dunnage trays for your specific parts. Find out more at appliedcobotics.com and pbclinear.com.

6402 E. Rockton Road Roscoe, Illinois 61073 USA +1.800.962.8979 • pbclinear.com

Design & Development

Apollo stands 5 feet 8 inches tall and weighs 160 lbs. Capable of li ing 55 lbs, Apptronik expects the swappable battery to power the robot for 4 hours. | Apptronik

November 2023

www.therobotreport.com

THE ROBOT REPORT

Inside the development of Apptronik’s

APOLLO HUMANOID The company’s R&D work on exoskeletons and bipedal locomotion for the U.S. government and Department of Defense has it primed for the humanoid era. THE ROBOT REPORT STAFF

THE ROBOT REPORT

Humanoids

have been the story of 2023. Humanoids have long been relegated to research labs, but the goalposts began to shi this year. Companies are now creating humanoids to perform various real-world tasks. One of those companies is Austin, Texas-based Apptronik, which has a rich history in legged robotics. It worked on the first generation of NASA’s Valkyrie robot and multiple generations of exoskeletons since its founding in 2016. Apptronik recently unveiled its first commercial humanoid robot, Apollo, and is targeting tasks within warehouses and logistics facilities as the first tasks to go a er. We recently had Apptronik’s CEO and co-founder Jeff Cardenas on The Robot Report Podcast to discuss the company’s history, the development of Apollo and the opportunity for humanoids in the future. This conversation has been edited for clarity and length. To hear the interview in its entirety, subscribe to The Robot Report Podcast anywhere you listen to podcasts. What are the origins of Apptronik? Apptronik started in 2016. We're a spin-out om the University of Texas at Austin om a lab called the Human-Centered Robotics Lab. Two of my co-founders are Luis Cintas, who runs the lab at the University of Texas at Austin, and Nick Paine, who was actually Luis's first student when he came to UT Austin and started the lab.

www.therobotreport.com

November 2023

Design & Development

The roots of Apptronik are really from the lab at the University of Texas at Austin, but they largely come out of the DARPA Robotics Challenge. Luis and Nick worked on the Valkyrie robot with the NASA Johnson Space Center. Valkyrie was one of the most advanced electric humanoids at the time. We now have about 85 full-time employees. We've been working on this problem of getting a humanoid out of the lab and into the real world for the better part of the last decade. Apollo represents our eighth iteration of a humanoid. It's much more robust, designed for

November 2023

affordability and mass manufacturing. It's also much more capable than the robots we've built in the past, taking everything we've learned and building the robot we always dreamed of building. Can you detail Apptronik’s product history leading up to Apollo? What we're doing as a company is iterating on our technology roadmap. How do we get from Valkyrie, which was about 300 pounds, six feet two inches tall and a multimillion-dollar system, to a sub-$50,000, highly robust, scalable, humanoid?

www.therobotreport.com

We were iterating along the way to get towards that future vision. Astra really represented one of the first productized humanoids we'd ever built. So it was the most polished of the systems we built up to that point. And we have done full humanoids prior to that, we'd also done just bipeds, and prior to that a handful of R&D bipeds to crack the dynamic locomotion problem. We realized the real demand was for more dexterous manipulation or more versatility in manipulation. And so when we thought about building the first product, we focused on this idea of building a dual-

THE ROBOT REPORT

So Astra was initially deployed on a wheeled, mobile base, Apollo could but then we started to make eventually be outfitted some breakthroughs on the with dexterous dynamic locomotion ont. hands, but the initial We brought in Gerardo Bledt, applications will only who came om Sangbae require a simple 0-1 Kim's lab at MIT and had DoF hand. worked on the Mini Cheetah | Apptronik quadruped. He had done a lot of great work on model predictive control for quadrupeds. We started to bring in the work that was being done there in quadrupeds and convert that to bipeds. That work started in 2020, and we started to see bipedal locomotion maturing to the point where we felt it could be deployed and start to realize the vision of the full humanoid, which is a dual-arm, bipedal system.

arm, mobile manipulator and building a best-in-class system there that was largely performance optimized. That was Astra, and we delivered a handful of those to customers and had them out in the world. A handful of them have been successful pilots along the way in partnership with some other groups. It was eye-opening for us to get the system out there. It proved out some of the early ideas we had about the viability of the technology and where it was at. Once we felt like we had a good line of sight on the manipulation problem, we started to dial up the mobility problem.

THE ROBOT REPORT

Why are so many companies now developing commercial humanoids? I think there are a number of reasons why humanoids are getting a lot of interest right now. The reason why customers are interested is pretty straightforward. Their entire process is built for humans, and they've adopted robots today. This is largely in manufacturing and logistics, where you've seen the most uptake of robots. There's a lot of spot automation that's happening out there in that domain. There are robots that can do one or two things. The big companies adopting this technology have 10-20 different vendors, and each vendor does one particular task. This is becoming difficult for them to manage. There’s also a lot of work that has to be done to fit existing automation into their workflow. So the idea of a multipurpose, or more versatile, system that can easily retrofit into an existing workflow is attractive. But what's made it particularly attractive to these customers is that they're seeing enough progress that they say, “OK, this is now here. And this makes a lot of sense to us for a number of reasons.” I always look toward another technology trend to see what's going to make sense in a new field. I use the computer analogy to explain what a

www.therobotreport.com

November 2023

CONNECT TECHNOLOGY WITH CONFIDENCE / / WWW . H A RW I N . C O M

Harwin’s connector products are proven to perform in extreme conditions, with shock, vibration and temperature range rigorously tested. WITH OUR QUALITY, SERVICE, SUPPORT, AND HIGHLY RELIABLE PRODUCTS, YOU CAN DEPEND ON HARWIN.

Design & Development

Apptronik said it is beginning pilots for Apollo and is targeting commercial availability by the end of 2024. | Apptronik

THE RIGHT STAMPING. THE RIGHT WASHER.

RIGHT NOW. BOKER’S WILL BE THERE. For over 100 years, nearly every industry across the globe has trusted Boker’s to provide quality components, world-class service and fast delivery.

SUBMIT YOUR PART SPECS &

Get a Fast Quote (800) -WASHERS

bokers.com

humanoid really is. In the early days of computers, you had special-purpose computers. Then you went om more special-purpose computers to main ames. But what it took for computers to reach the market and to reach higher density and true commercialization was the move om the main ame to the personal computer - a general-purpose hardware platform that was a so ware update away om doing something new and different. The PC is where computers had their breakout moment. It seemed crazy back in the 90s when you had Bill Gates saying one day in every home you're going to have a computer on every desk. But look at where we are now; everyone's carrying a computer around in their pocket or on their wrist. I view humanoids as personal computers in the early 1980s. What we have to find are the initial applications for humanoids. For this general-purpose platform to be viable out in the market, you'll add functionality on top of it. I tell customers that you can either have 1,000 robots that each do one thing, or you can have one robot that does 1,000 different things. What are the features and capabilities of Apollo? Apollo is 5 feet 8 inches tall and weighs 160 pounds with the battery. It's got onboard compute, it li s about 55 pounds and is targeted at OSHA (Occupational Safety and Health Administration) specs. Initially, we're targeting it to have a four-hour runtime with swappable batteries. Long term, we want it to have a 22-hour per day, sevenday-a-week uptime. And we believe the way to get there is with swappable batteries. Apollo is one of the first systems we've built with swappable batteries in it. It's taking all the ideas that we've learned om all the different systems that we've built [before this], including humanoids, mobile manipulators and exoskeletons, putting them all together. We're aiming to build Apollo for less than the price of many cars that are on the market today, and we don't need an obscene scale to be able to do that. We're building it in a fundamentally different way om what we've seen [in] other systems that are [currently released]. We're borrowing ideas om other machines that are out in the world that are produced at a scale much more affordable than robots are today. But all that's coming together in Apollo. Apollo is designed to have hands. We've partnered with several other groups on hands in the past. We've partnered with some folks who are really pushing the boundaries of dexterous manipulation. We've also purchased hands om groups that are building more affordable robotic prosthetics. Our initial strategy is to basically partner on dexterous manipulation. The first use cases we're targeting for Apollo require only gross manipulation. In terms of where the technology is today, I think long term a humanoid has to

November 2023

THE ROBOT REPORT

have hands. Most of the work that humans do in the world today does involve some level of dexterity. The exciting thing for us, in terms of where we can apply the robots today with the technology that exists right now, is in the gross manipulation domain, meaning things where you're using two hands to grasp something at once. That's largely in logistics and material handling. You'll see us move across the spectrum om gross manipulation to finding our dexterity over time. We will use different effectors for that as we initially deploy [Apollo], so we don't need a five-finger hand [at launch]. For initial deployments, we can use one degree of eedom in the end effector that allows us to solve all the initial use cases that we're looking at, and then move into finer dexterity over time.

We'll show and demonstrate videos of Apollo with full hands, showing the capabilities and dexterity. The problem when you look at commercialization is the rate. The most interesting approach to dexterous manipulation that we've seen is using teleop to train models, which allows you to get a dexterous manipulator out into the world today. The economics of that are tough when you start to look at teleop initially. We believe autonomy is ultimately going to win the day, and we are focused on autonomy for the initial applications that we're deploying and delivering on over the next one to two years. How did Apptronik decide which types of applications to go a er with Apollo? Just like with any big problem, you break the problem down and start as simply as you can.

We're aiming to build Apollo for less than the price of many cars that are on the market today, and we don't need an obscene scale to be able to do that.

Apollo represents the pinnacle of more than 13 generations of electric actuators developed by Apptronik. | Apptronik

THE ROBOT REPORT

www.therobotreport.com

November 2023

Design & Development Then you build in complexity. The way we view the go-to-market strategy for humanoids is to build a generalpurpose robot that can do many of the tasks humans can do but don’t want to do. But before a system can be generalpurpose, it has to be multipurpose and do at least three to four tasks in production out in the world. Before it can be multipurpose, it's got to at least do one of those tasks reasonably well. So our approach is to start by picking one initial task in material handling. We have a particular application that we're focused on, basically handling cases or boxes, and moving them om one place to another. If you can just get to multipurpose, that's enough for many of the logistics use cases and customers we're talking to today. How large of an opportunity is this box-moving application? To give you an idea of scale, there are about 675,000 unfilled logistics jobs in the U.S. today. So there's a huge number of open jobs in the logistics market. If you just look at case handling, there will be about 200 billion cases that are picked globally by 2025. There's a massive opportunity. The thing that was exciting for us was that we realized the technical barrier to entry was actually much lower than we thought. We thought we were going to have to get to dexterous manipulation for humanoids to be viable. We saw that there was demand if we could just move boxes, cases, or totes in a warehouse. And that's something that we can do today. That started to motivate us towards pushing Apollo to the fore ont of our commercialization plan so that we can get robots out there to start working in real-world use cases and build om there. There's a lot of potential for humanoids, but very few, if any, have been deployed commercially. That’s the litmus test. The next big thing for us is to start moving om demos to pilots and then to real deployments. There's a handful of folks that are pushing that forward, and we have a lot of respect for the other players that are here in the space with us.

November 2023

Apptronik has developed a novel electric actuator for Apollo. How did this technology come to be and how will it impact Apollo? The focus of Nick’s Ph.D. was around electric actuators for legged systems. So this has been his body of work in grad school and is now a big part of the DNA of Apptronik. We made a big bet early on with electric actuation. We thought electric actuation was going to be the future. All of our actuators are force-controlled as well, which we thought was going to be an important thing. We've done over 35 unique electric actuators up to this point. So a tremendous amount of iteration. Some of those are small, medium, and large members of a common family of actuators. So

they're not all completely unique, but you learn a lot. What we've landed on a er all of that is the Goldilocks, if you will, of performance and cost. The new actuators we've put into Apollo have about a third fewer components than the previous actuators we've built, which take about a third of the assembly time. And more importantly, they're solving a big problem in robotics

www.therobotreport.com

200 billion cases will be moved each year by 2025, representing a massive opportunity for Apollo and other humanoids. | Apptronik

THE ROBOT REPORT

readycable

cable assemblies MILLIONS OF FEET IN STOCK

FLEXIBILITY FOR DYNAMIC APPLICATIONS

DESIGNED TO PREVENT CORKSCREWING

NO MINIMUM LENGTH REQUIREMENT

UL-VERIFIED GUARANTEE UP TO 36 MOS.

Meets 20+ manufacturer standards • In stock & ships in as little as 24 hours • No minimum order quantity required • Wide range for lowest cost solution

• Entire assembly from single source • Online tool to estimate cable lifetime • Available fully installed within e-chain

Design & Development around the supply chain. One of the big challenges is being driven by cost and how many units we can deliver to the market. The robotics industry requires high-precision components, in particular harmonic drives. There are only a handful of companies that have ever been able to scale those up to high volumes. So one of the things that we were focused on was: how do you get harmonic drives out of the system altogether? How do we get rid of these proprietary, expensive components that are not good for unstructured applications and are also pretty expensive, relatively speaking? With Apollo, we've finally achieved that. Usually, when you hit everything, there's a trade-off. But the exciting thing about Apollo is we actually have a better system – it's higher performance, we can achieve higher speeds, and it's more robust to external impacts. It's also more affordable than the systems we built in the past. So there are some trade-offs that you make when you increase the complexity of the controls when you're building a system like this. We're not using serial chain rotary actuators, as many of the systems in the past have used. So we had to build new so ware and solve some new problems to bring a system like this online. But the benefits have really outweighed that and a lot of that tech risk is now behind us. From a technology standpoint, we're really proud of what we've come up with and what we've delivered. We really think that it's a system that only we could have built with all the experience that we've had over the last several years, and all the systems we built leading up to it. Safety is a big concern with humanoids. How is Apptronik making Apollo safe to work around? Safety is really important. First and foremost, it's a big focus for us at Apptronik. And we think we're well positioned as we’ve done so much work on high-fidelity force control. For the customers that we're dealing with and initial deployments, the easy way to solve this is to not have humans around the system during the pilot phase. But ultimately,

November 2023

humanoids have to be collaborative to deliver on the value proposition. There are all sorts of new committees coming together to think about safety for humanoids. There are two ways we handle safety. The system itself is forcecontrolled, and so it works like a collaborative robot might work: it can respond to an impact. The second way is with vision, so ware and behaviors. The robot basically has this field of vision around it where if you get close, the robot can slow down. And if you get really close to it, it can come to a stop. A big question is what do you do if the robot falls? Right now we have a safety procedure where the robot basically tucks up when it falls to mitigate the risk. But there's a lot of work that we're doing with our customers and there's different philosophies about how folks want to approach that. Some groups don't want the robot to slow down, while others are more concerned that there are a lot of dangerous machines out in the world today. And if you look at manufacturing or logistics today, think about a forkli or any of the machinery they have in manufacturing relative to a humanoid - you're just thinking about how to mitigate that risk. I think it's a crawl-walk-run situation where, initially, we just want to get humanoids out of labs and into real production environments. We can mitigate how closely they interact with humans. Ultimately, we want them to be seamlessly integrated by working with the industry as a whole. We want to work with the other folks in this domain to determine the safety standards and what makes sense and that will continue to progress over the coming years. RR

Control your entire robotic fleet from a web browser in real-time

CLOUD GROUND CONTROL Cloud Ground Control (CGC) is a cloud-based command and control solution for operators and manufacturers of uncrewed air, land and sea vehicles. Connect your DJI drones via our mobile app, or any other vehicles using CGConnect – our cellular micro-modem.

Scan the QR code or visit www.cloudgroundcontrol.com

Design & Development Researchers are exploring how large language models can be applied to robotics design, model training, simulation, control algorithms and more.

| Adobe Stock

Teaching robots new tricks

WITH GENERATIVE AI STEVE CROWE • EXECUTIVE EDITOR • ROBOTICS

new AI agent developed by NVIDIA Research that can teach robots complex skills has trained a Shadow Robot hand to perform rapid pen-spinning tricks. NVIDIA claimed it is the first time a robot can do this as well as a human can. The trick is one of nearly 30 tasks that robots have learned to accomplish thanks to Eureka, which NVIDIA said autonomously writes reward algorithms to train robots. Eureka has also taught

November 2023

robots to open drawers and cabinets, toss and catch balls, and manipulate scissors, among many other tasks. NVIDIA recently published its Eureka research, which includes a paper and the project’s AI algorithms. Robotics developers can experiment with these algorithms with using NVIDIA Isaac Gym, a physics simulation reference application for reinforcement learning research. Isaac Gym is built on NVIDIA Omniverse, a

www.therobotreport.com

development platform for building 3D tools and applications based on the OpenUSD amework. Eureka, itself, is powered by the GPT-4 large language model (LLM) that was created by OpenAI. “Reinforcement learning has enabled impressive wins over the last decade, yet many challenges still exist, such as reward design, which remains a trial-and-error process,” said Anima Anandkumar, senior director of AI research at NVIDIA and an

THE ROBOT REPORT

Infinite Possibilities. SmartRunner Explorer 3-D Pick and place applications require special technology for detection and positioning. When attached to the articulated arm, the SmartRunner Explorer 3-D with stereo vision enables accurate object placement for an exact gripping point. Pepperl+Fuchs Vision Library Toolbox includes ROS (Robot Operating System) drivers.

For more information, visit pepperl-fuchs.com

Design & Development author of the Eureka paper. “Eureka is a first step toward developing new algorithms that integrate generative and reinforcement learning methods to solve hard tasks.” The ability to spin a

pen in its hand is one

AI trains robots of nearly 30 tasks Eureka-generated reward robots have learned to programs, which enable trialaccomplish thanks to and-error learning for robots, a new AI agent om outperform expert humanNVIDIA called Eureka. written ones on more than | NVIDIA 80% of tasks, according to the paper. This leads to an average performance improvement of more than 50% for the bots. The AI agent taps the GPT-4 LLM and generative AI to write so ware code that rewards robots for reinforcement learning. It doesn’t require task-specific prompting or predefined reward templates. Using GPU-accelerated simulation in Isaac Gym, Eureka can quickly evaluate the quality of large batches of reward candidates for more efficient training. Eureka then constructs a summary of the key stats om the training results and instructs the LLM to improve its generation of reward functions. In this way, the AI is self-improving. It’s taught all kinds of computer graphics, computer vision, selfrobots — quadruped, bipedal, quadrotor, driving cars and robotics. dexterous hands, cobot arms and others — to accomplish all kinds of tasks. Inside the pen-spinning trick The research paper provides inNVIDIA said using the Shadow Robot depth evaluations of 20 Eureka-trained hand to spin a pen helped it investigate tasks, based on open-source dexterity whether EUREKA can be used to solve a benchmarks that require robotic hands novel and challenging dexterous task. This to demonstrate a wide range of complex task is highly dynamic and requires the manipulation skills. robot hand to continuously rotate a pen “Eureka is a unique combination of to achieve some pre-defined spinning large language models and NVIDIA GPUpatterns for as many cycles as possible. accelerated simulation technologies,” said NVIDIA used a curriculum learning Linxi “Jim” Fan, senior research scientist approach to break down the task into at NVIDIA, who’s one of the project’s manageable components that could be contributors. “We believe that Eureka will independently solved by EUREKA; similar enable dexterous robot control and provide approaches have been found successful a new way to produce physically realistic for other coding LLM applications to animations for artists.” decision making. Specifically, NVIDIA first It’s breakthrough work bound to instructed EUREKA to generate a reward get developers’ minds spinning with for re-orienting the pen to random target possibilities, adding to recent NVIDIA configurations. Then, using this preResearch advancements like Voyager, trained policy, NVIDIA fine-tuned it using an AI agent built with GPT-4 that can the EUREKA reward to reach the sequence autonomously play Minecra . of pen-spinning configurations. NVIDIA Research comprises hundreds To demonstrate the importance of of scientists and engineers worldwide, curriculum learning, the researchers also with teams focused on topics including AI,

November 2023

www.therobotreport.com

trained a baseline policy om scratch using EUREKA reward without the first-stage pre-training. Eureka fine-tuning quickly adapted the policy to successfully spin the pen for many consecutive cycles along a specified spinning axis. In contrast, neither pre-trained or learning- om-scratch policies completed even a single cycle of pen spinning. In addition, using this EUREKA fine-tuning approach, NVIDIA said it also trained pen spinning policies for a variety of different spinning configurations. Improving with human feedback NVIDIA said it readily incorporates human feedback to modi its rewards for results more accurately aligned with a developer’s vision. NVIDIA said the reason for this is that humans have something Eureka lacks. "Human designers are generally knowledgeable about relevant state variables but are less proficient at designing rewards using them," NVIDIA said. "This makes intuitive sense as identi ing relevant state variables that should be included in the reward function involves mostly common sense reasoning,

THE ROBOT REPORT

but reward design requires specialized knowledge and experience in RL." Reward engineering is a longstanding challenge in reinforcement learning. The most common reward design method is manual trial-and-error. Inverse reinforcement learning (IRL) infers reward functions om demonstrations, but it requires expensive expert data collection, which may not be available, and outputs non-interpretable blackbox reward functions. Several prior works have studied automated reward search through evolutionary algorithms. These early attempts are limited to task-specific implementations of evolutionary algorithms that search over only parameters within provided reward templates. Recent works have also proposed using pre-trained foundation models that can produce reward functions for new tasks. Most of these approaches output scalar rewards that lack interpretability and do not naturally admit the capability to improve or adapt rewards on-the-fly.

In contrast, EUREKA adeptly generates ee-form, white-box reward code and effectively improves in-context. Just getting started NVIDIA is one of many companies exploring how LLMs and Generative AI can impact robotics development. For example, Toyota Research Institute (TRI) recently unveiled how it is using Generative AI to help robots learn new dexterous behaviors om demonstration. TRI said this new approach “is a step towards building ‘Large Behavior Models (LBMs)’ for robots, analogous to the LLMs that have recently revolutionized conversational AI.” At press time, TRI had already taught robots more than 60 difficult, dexterous skills using the new approach. Some of these skills include pouring liquids, using tools and manipulating deformable objects. These were all realized, according to TRI, without writing a single line of new code; the only change was supplying the robot with new data.

TRI’s robot behavior model learns om haptic demonstrations om a teacher, combined with a language description of the goal. It then uses an AI-based diffusion policy to learn the demonstrated skill. This process allows a new behavior to be deployed autonomously om dozens of demonstrations. TRI’s approach to robot learning is agnostic to the choice of teleoperation device, and it said it has used a variety of low-cost interfaces such as joysticks. For more dexterous behaviors, it is taught via bimanual haptic devices with position-position coupling between the teleoperation device and the robot. Position-position coupling means the input device sends measured pose as commands to the robot and the robot tracks these pose commands using torque-based Operational Space Control. The robot’s pose-tracking error is then converted to a force and sent back to the input device for the teacher to feel. This allows teachers to close the feedback loop with the robot through force and has been

Design & Development critical for many of the most difficult skills we have taught. TRI admitted that “when we teach a robot a new skill, it is brittle.” Skills will work well in circumstances that are similar to those used in teaching, but the robot will struggle when they differ. TRI said the most common causes of failure cases we observe are: • States where no recovery has been demonstrated. This can be the result of demonstrations that are too clean. • Camera viewpoint or background significant changes. • Test time manipulations that were not encountered during training. • Distractor objects, for example, significant clutter that was not present during training. Part of TRI’s technology stack is Drake, a model-based design for robotics that includes a toolbox and simulation platform. Drake’s degree of realism allows TRI to develop in both simulation and in reality and could help overcome these shortcomings going forward. TRI’s robots have learned 60 dexterous skills already, with a target of hundreds by the end of 2023 and 1,000 by the end of 2024.

“Existing Large Language Models possess the powerful ability to compose concepts in novel ways and learn om single examples,” TRI said. “In the past year, we’ve seen this enable robots to generalize semantically (for example, pick and place with novel objects). The next big milestone is the creation of equivalently powerful Large Behavior Models that fuse this semantic capability with a high level of physical intelligence and creativity. These models will be critical for general-purpose robots that are able to richly engage with the world around them and spontaneously create new dexterous behaviors when needed.” Boston Dynamics recently used ChatGPT, and other AI models, as a proof of concept and turned its Spot quadruped, which is typically used for inspections, into a robot tour guide. The demo the team planned required Spot to be able to speak to a group and hear questions and prompts om them. Boston Dynamics 3D printed a vibration-resistant mount for a Respeaker V2 speaker. They attached this to Spot’s EAP 2 payload using a USB. Spot is controlled using an offboard computer, either a desktop PC or a laptop, which uses Spot’s SDK to communicate.

The team added a simple Spot SDK service to communicate audio with the EAP 2 payload. Now that Spot had the ability to handle audio, the team needed to give it conversation skills. They started with OpenAI’s ChaptGPT API on gpt-3.5, and then upgraded to gpt-4 when it became available. Additionally, the team did tests on smaller open-source LLMs. The team took inspiration om research at Microso and prompted GPT by making it appear as though it was writing the next line in a Python script. It then provided English documentation to the LLM in the form of comments and evaluated the output of the LLM as though it were Python code. The Boston Dynamics team also gave the LLM access to its SDK, a map of the tour site with 1-line descriptions of each location, and the ability to say phrases or ask questions. They did this by integrating a VQA and speech-to-text so ware. They fed the robot’s gripper camera and ont body camera into BLIP-2, and ran it in either visual question answering mode or image captioning mode. This runs about once a second, and the results are fed directly into the prompt.

Using its Generative AI approach, Toyota Research Institute has taught robots more than 60 dexterous behaviors like peeling vegetables, using hand mixers, preparing snacks, and flipping pancakes. | Toyota Research Institute

November 2023

www.therobotreport.com

THE ROBOT REPORT

To give Spot the ability to hear, the team fed microphone data in chunks to OpenAI’s whisper to convert it into English text. Spot waits for a wake-up word, like “Hey, Spot” before putting that text into the prompt, and it suppresses audio when it is speaking itself. Because ChatGPT generates textbased responses, the team needed to run these through a text-to-speech tool so the robot could respond to the audience. The team tried a number of off-the-shelf text-to-speech methods, but they settled on using the cloud service ElevenLabs. To help reduce latency, they also streamed the text to the platform as “phrases” in parallel and then played back the generated audio. The team also wanted Spot to have more natural-looking body language. So it used a feature in the Spot 3.3 update that allows the robot to detect and track moving objects to guess where the nearest person was, and then have the robot turn its arm toward that person. Using a lowpass filter on the generated speech, the team was able to have the gripper mimic speech, sort of like the mouth of a puppet. This illusion was enhanced when the team added costumes or googly eyes to the gripper. The team noticed new behavior emerging quickly from the robot’s very simple action space. It asked the robot,

“Who is Marc Raibert?” Raibert is the founder of Boston Dynamics. Yet, the robot didn’t know the answer and told the team that it would go to the IT help desk and ask, which it wasn’t programmed to do. The team also asked Spot who its parents were, and it went to where the older versions of Spot, the Spot V1 and Big Dog, were displayed in the office. These behaviors show the power of statistical association between the concepts of “help desk” and “asking a question,” and “parents” with “old.” They don’t suggest the LLM is conscious or intelligent in a human sense, according to the team. The LLM also proved to be good at staying in character, even as the team gave it more absurd personalities to try out. While the LLM performed well, it did frequently make things up during the tour. For example, it kept telling the team that Stretch, Boston Dynamics’ logistics robot, is for yoga. Moving forward, Boston Dynamics plans to continue exploring the intersection of artificial intelligence and robotics. To them, robotics provides a good way to “ground” large foundation models in the real world. Meanwhile, these models also help provide cultural context, general commonsense knowledge, and flexibility that could be useful for many robotic tasks. RR

A NEW GENERATION OF ROTARY ENCODERS

Meet POSITAL’s Updated and Improved IXARC Encoders New TMR sensor technology and upgraded processors offer enhanced performance and better energy efficiency. Incremental encoders are programmable for extra versatility: Resolution and communications interfaces are can be easily customized to meet your exact requirements. Muliturn absolute models feature updated Wiegand sensors and a new ASIC for optimized batteryfree rotation counting More NEXTGEN product announcements coming soon!

A diagram of the overall system used to turn Boston Dynamics’ Spot quadruped into a tour guide that used ChatGPT and other AI models. | Boston Dynamics THE ROBOT REPORT

www.therobotreport.com

November 2023

www.posital.com

Design & Development

| Adobe Stock

Humanoids getting their feet wet

IN LOGISTICS ASH SHARMA • MANAGING DIRECTOR • INTERACT ANALYSIS

Humanoids offer the potential flexibility and scalability to be used within existing warehouses alongside humans to help pick, move and sort goods.

November 2023

Amazon

recently started to test a humanoid robot, Digit, in its warehouses, raising esh questions about how swi ly humanoids will be used in logistics. Made by Agility Robotics, Digit can grasp and li objects, and it will start by shi ing empty tote boxes as Amazon continues to explore new ways of automating its warehouses. Warehousing offers the perfect combination of repetitiveness and menial work, coupled with a lack of standardization and a lack of uniformity of things handled. For companies operating www.therobotreport.com

fulfillment centers with a wide product mix or fluctuating demand, the flexibility of humanoids could deliver a solution to the ongoing labor and skills crisis in the logistics industry. For example, humanoids could be dropped into the workforce during seasonal peaks in demand without requiring substantial operational changes, unlike other automated technologies where large changes to warehouse workflows are required. One early application for humanoids is likely to be trailer unloading, as it is simple, repetitive and physically demanding. THE ROBOT REPORT

Unlike other robots, humanoids offer the additional benefit of being flexible enough to take on multiple different tasks in a facility, rather than being limited to a single process or workflow. Despite the current media storm around humanoids, this change will not happen overnight, as pilot projects can take many months, or even years, to reach completion. Robots are in high demand within the logistics sector in light of severe labor and skills shortages, and volatile demand. However, while the widespread use of humanoids within warehouses is certainly a possibility, it is not an inevitability and we may well find that other robots are better suited to specific tasks and come with fewer ethical problems, particularly in terms of perceived role replacement. Warehouses ripe for humanoids, but challenges persist Reports claim Amazon’s decision to trial Digit raises concerns about the displacement of human workers. Other automated systems are ‘unhuman’ enough not to cause unease to the same extent as humanoid workers. In response to fears around job loss, Amazon has emphasized the “hundreds of thousands of new jobs” that have been created as a result of its use of robotic systems, including “700 categories of new job types, in skilled roles, which didn’t exist within the company beforehand.” The global warehousing sector, particularly within the U.S., has been facing critical shortages of labor and skills in recent years. A poll published earlier this year by MHI and Deloitte revealed hiring and retaining qualified workers and talent shortages were the leading challenges for U.S. supply chain leaders, with many planning to invest in robotics and automation in the near future. Digit has been designed for warehouse automation where the complexities of human thought and movement are less important than improving throughput and filling skills and labor gaps. With almost 1.5 million humans employed across Amazon’s operations, the company is keen to stress the potential to create jobs through the THE ROBOT REPORT

Amazon is initially testing Digit’s ability to move empty tote boxes onto a conveyor belt. | Amazon

deployment of humanoid robots, as well as replacing the most “menial, mundane and repetitive” tasks. Developed by Amazonbacked Agility Robotics, Digit is described as “a mobile manipulator solution.” Standing 5 9in (175cm) tall, Digit can li and carry up to 35lbs (16kg), crouch, reach, pick up and put down. Tye Brady, chief technologist at Amazon Robotics, told reporters in Seattle that the move does not mean job cuts at Amazon and people are “irreplaceable” to the business because of their “ability to think at a higher level, the ability to diagnose problems.” Other humanoids are in development om companies such as Figure and Boston Dynamics. Tesla’s Optimus Robot can sort objects autonomously as it can self-calibrate its arms and legs, while the commercial launch of Apptronik’s Apollo is expected to take place in late 2024. Videos of the humanoid robot have already shown it walking, case picking, palletizing and unloading trailers. In reality, warehouse automation has been in place for decades now, including autonomous mobile robots (AMRs) working alongside human workers and carrying out many of the menial, physically demanding jobs. Automated storage and retrieval systems are capable of order picking at a much faster rate and are being used alongside human workers to increase throughput. How soon will humanoids become the norm in warehouses? It looks as though the widespread use of humanoids in the warehouse is some way www.therobotreport.com

off despite Amazon’s latest move. Pilots of new technology such as this can take upwards of 18 months and, if successful, rollout tends to be in small steps. However, Amazon has always been a leader in its use of robotics and the rest of the industry tends to follow (or fail!), so this pilot could be the catalyst for humanoid robot uptake in the coming years. Is it possible for the use of humanoids to be hyped too much? And given the representations of humanoid robots we o en see in popular culture, will this shape public opinion? Can robots look too much like people? It is true that the more lifelike humanoid robots are, the more they are labor replacement personified. It is too early to tell whether in the long term humanoid robots will become a common sight in warehouses around the world – much will depend on how successful these early pilots are. Humanoids offer the potential flexibility and scalability to be used within existing warehouse operations alongside the human workforce, helping to pick, move and sort goods. They are the next logical step in the evolution of the multibilliondollar robotics industry, as customers seek solutions that can be used across multiple workflows and applications. The flexibility and scalability humanoids offer could be a possible answer to ongoing labor and skills shortages. However, it does not come without complications and it is yet to be seen whether these will be easily overcome to see their rapid deployment in warehouses. RR November 2023

Sensors

VMT is now using a robot with a vision system to load and unload parts om containers. It said this is a difficult technological challenge because the containers come om different manufacturers, are of different types and can vary in dimension.

3D cameras enable

AUTOMATIC LOADING of pressed parts THE ROBOT REPORT STAFF

Four 3D cameras from IDS Imaging each record a part of the entire image field, resulting in data from multiple perspectives for a higher quality 3D point cloud. 32

November 2023

Robots

do monotonous workflows and less pleasant, repetitive tasks with brilliance. Combined with image processing, they become “seeing” and reliable supporters of humans. They are used in quality assurance to check components, help with the assembly and positioning of components, detect errors and deviations in production processes and thus increase the efficiency of entire production lines. An automobile manufacturer is taking advantage of this to improve the cycle time of its press lines. Vision Machine Technic Bildverarbeitungssysteme GmbH (VMT) om Mannheim, Germany developed a robot-based 3D measuring system, FrameSense, for fully automatic loading and unloading of containers. Pressed parts are safely and precisely inserted into or removed om containers. Four Ensenso 3D cameras om IDS Imaging Development Systems GmbH provide the basic data and thus the platform for process automation. www.therobotreport.com

THE ROBOT REPORT

WISE Design | WISE Guidance | WISE Investment

ROBOT TRANSFER UNITS BUILT ON DUALVEE® MOTION GUIDE TECHNOLOGY Support and transfer robot arms of any size with BishopWisecarver®’s suite of RTUs. We provide reliable, long-lasting RTUs and 7th axis systems for all your favorite brands of cobots and industrial arms.

WE’RE THE WISE CHOICE Choose solutions that excel and endure in the most extreme environments and conditions. Easy to order, quick to turn around, and with all the support you need to complete your project on time.

VISIT WWW.BWC.COM

Sensors

Application Four Ensenso C The workflow FrameSense is cameras capture designed to automate is part of the contours of the many manufacturing operations. stacking containers A component comes out of a for VMT. machine - here a press - and | IDS Imaging runs on a conveyor belt to a container. There it is stacked. As soon as the container is full, it is transported to the next production step assembly into a vehicle. Up to now, employees have been responsible for loading the containers. This actually simple subtask is more complex than one might think at first glance. In addition to the actual insertion process, the first step is to determine the appropriate ee space for the part. At the same time, any interfering factors, such as interlocks, must be removed and a general check of the “load box” for any defects must be carried out. All these tasks are now to be taken over by a robot with a vision system - a the respective technological challenge. This is because container. This the containers also come om different is the only way to manufacturers, are of different types ensure process-reliable and thus vary in some cases in their and collision- ee path guidance of the dimensions. loading robot. All this has to be integrated into the existing production process. Type, shape and position inspection with Time delays must be eliminated and the 3D cameras positioning of the components must be For fully automatic loading and unloading, accurate to the millimeter. the position of several relevant features of To counter this, VMT uses four 3D the containers must be determined for a cameras per system. The four sensors each multi-vector correction of the robot. The record a part of the entire image field. basis is a type, shape and position check of This can consist of two containers, each

measuring approximately 1.5 x 2 x 1.5 meters (D x W x H). Two of the cameras focus on one container. This results in data om two perspectives each for a higher information quality of the 3D point cloud. These point clouds of all four sensors are combined for the subsequent evaluation. In the process, registrations of relevant features of the container take place in ROIs (Regions of Interest) of the total point cloud. A registration is the exact positioning of a feature using a model in all 6 degrees of eedom. In other ROIs, interference contours are searched for which could lead to collisions during loading. Finally, the overall picture is compared with a stored reference model. In When inserting the this way, the containers components into can be simultaneously the comb strips of checked for their the racks, the robots condition and position have to work with in a fully automated millimeter precision. manner. Even deformed or slanted containers | IDS Imaging can be processed. All this information is also recorded for use in a quality management system where the condition of all containers can be traced. The calibration

November 2023

www.therobotreport.com

THE ROBOT REPORT

as well as the consolidation of the measurement data and their subsequent evaluation are carried out in a separate IPC (industrial computer) with screen visualization, operating elements and connection to the respective robot control. The main result of the image processing solution is the multi-vector correction. In this way, the robot is adjusted to be able to insert the component at the next possible, suitable deposit position. Secondary results are error messages due to interfering edges or objects in the container that would prevent filling. Damaged containers that are in a generally poor condition can be detected and sorted out with the help of the data. The entire image processing takes place in the image processing so ware MSS (Multi Sensor Systems) developed by VMT. FrameSense is designed to be easy to use and can also be converted to other components directly on site.

LOW-VOLTAGE GEARMOTORS

for AGV/AMR Applications

Robust 3D camera system On the camera side, VMT relies on Ensenso 3D cameras initially on the X36 model. The current expansion stage of FrameSense is equipped with the Ensenso C variant. The reasons for the change are mainly the better projector performance - thanks to a new projection process - as well as a higher recording speed. In addition, the Ensenso C enables a larger measuring volume. This is an important criterion for FrameSense, because the robot can only reach the containers to be filled up to a certain distance. The specifications of the Ensenso C correspond exactly to VMT's requirements, as project manager and technology manager Andreas Redekop explained. "High projector performance and resolution together with fast data processing were our main technical criteria when selecting the camera. The installation in a fixed housing was also an advantage.” The Ensenso C addresses current challenges in the auomation and robotics industry. Compared to other Ensenso models, it provides both 3D and RGB color information. Customers thus benefit om even more meaningful image data. The housing of the robust 3D camera system meets the requirements of protection class IP65/67. It offers a resolution of 5 MP and is available with baselines om current to approx. 455 mm. This means that even large objects can be reliably detected. The camera is quick and easy to use and addresses primarily large-volume applications, e.g. in medical technology, logistics or factory automation. Outlook By automatically loading and unloading containers and the integrated 3D container inspection, manual workstations can be automated with the help of FrameSense. Against the background of the shortage of skilled workers, the system can thus make an important contribution to process automation in the automotive industry, among others. It meets the prevailing challenges THE ROBOT REPORT

November 2023

Our efficient, high-performance low-voltage brushless DC (EC) gearmotors are available with brakes, encoders, special connectors and wire harnesses. Call today with your #AGV #AMR gearmotor questions! bodine-electric.com info@bodine-electric.com

7 7 3 . 47 8 . 3 515 (USA )

S U M M I T

E X P O

Medtech conversations to inspire the next generation of devices

Sponsorship opportunities are available for future DeviceTalks and Robotics programs.

For more information, contact Colleen Sepich. 857.260.1360 | csepich@wtwhmedia.com

of the industry. Ensenso C provides the crucial basis for data generation and exceeds the requirements of many applications. “The high projector power and large sensor resolutions are particularly advantageous in the field of intralogistics,” said Lukas Neumann, Product Management, VMT. “Here, high-precision components have to be gripped om a great distance with a large measuring volume.” For other stacking or bin-picking applications in classic logistics, he could imagine a similar camera with high projector power but lower resolution and fast recording. So nothing stands in the way of further developments and automation solutions in conjunction with "seeing" robots. RR

PLANETARY Brushless DC Gearmotors

High Performance. Zero Maintenance.

Supported by a high-intensity projector, the Ensenso C projects high-contrast textures onto the object to be imaged | IDS Imaging

RATED SPEED 15 to 658 rpm

TORQUE 11 to 226 lb-in.

GEAR RATIOS 4:1 to 162:1

VOLTAGES 24, 130 VDC

Get more torque, maintenance-free performance, and high efﬁciency with our brushless DC planetary gearmotors. Metric and US models in stock. New encoder shaft kit option now available. bodine-electric.com THE ROBOT REPORT

November 2023

info@bodine-electric.com

| 773.478.3515 (USA)

Sensors

How computer vision increases

PRODUCTIVITY & SUSTAINABILITY in agriculture

Integrating computer vision and other advanced technologies like sensors, robotics, and AI will result in more efficient and more automated machines that can treat every single plant at a micro level. JORGE HERAUD VP OF AUTOMATION & AUTONOMY JOHN DEERE

Computer

vision is extending human senses. Whether it is allowing cars to drive themselves, automating production lines for safety and efficiency, or allowing us to unlock our phones with our faces, computer vision can help make human work more productive and accurate. For centuries, farmers have used their senses to understand and work the land to grow the crops we depend on. But today, farmers must extend their senses even further to provide the food, fuel, and fiber needed for a growing population that is expected to exceed 10 billion by 2050, increasing global food demand by 50%. Adding to the challenge, less than 2% of the U.S. population works in agriculture, and overall employment of agricultural workers is projected to grow just by 1% om 2019 to 2029. This means farmers have less help and more work to do in the coming years. Imagine taking care of an entire football field worth of plants — that is

A John Deere tractor equipped with a sprayer boom on each side of the tractor working in the field.

about an acre. Now multiply that field by about 4,000, and you'll understand the enormous task farmers take on when planting and harvesting thousands of acres. A farm of this size can have more than 750 million plants, each experiencing microenvironments and conditions that govern their success. With so much land to keep track of, farmers need technology to help them understand what's happening at critical junctions. That’s where computer vision comes into play. Helping farmers see beyond the tractor cab Computer vision helps farmers "see" crops in ways the human eye can’t. It allows farmers to manage their limited time and resources — like fertilizer, herbicides, and seeds — and gives them data and insights to make timely, accurate decisions for a healthier, more successful crop. This helps farmers be more productive, profitable, and sustainable. For example, the human eye can’t distinguish a weed om a crop while

| John Deere

November 2023

www.therobotreport.com

THE ROBOT REPORT

The John Deere 8R Autonomous Tractor was launched in 2022. | John Deere

A solution for every challenge.

Y driving a tractor 15 miles per hour over a field. But a sprayer equipped with computer vision can. Using cameras, onboard processors, and millions of training images, a technologically advanced sprayer can determine if a plant it sees in the field is a weed or crop and, in milliseconds, spray only on the weed with herbicide, generating significant cost savings for farmers and increasing the sustainability of their operations. Computer vision also helps farmers see deep inside machines as they perform precise tasks, like harvesting crops at the end of a season. They use large machines that are like factories on wheels to do several things at once. For example, when harvesting corn, the machine will cut the stalk om the ground, separate the ear of corn om the stalk, and remove the individual kernels om the ear. The machine then transfers the kernels to a large cart that takes the crop om the field to storage. If one of these steps isn’t operating at peak performance, it could reduce the total crop and profits for the year. With computer vision, farmers can monitor and automatically adjust each of these crucial steps to minimize the time it takes to complete the task and ensure the best possible crop harvest in all conditions. Enabling farmers to do more with data In addition to helping farmers see more, computer vision also helps farmers do more. Thanks to advanced cameras, tractors can use a 360-degree view to identi objects and measure distances. Onboard processing then evaluates the

THE ROBOT REPORT

images om the cameras to determine if the tractor has a clear path forward. This allows the machine to operate in the field autonomously and lets farmers essentially be in two places at once. They can set up a tractor to perform a simple, routine task in a field while they manage more complex aspects of their operations elsewhere. Computer vision also gives farmers a constant stream of data to inform realtime decision making. When combined with historical data om previous seasons, farmers can identi the strategies that produce the best crop and potential areas for improvement. This data takes the guesswork out of farming. Farmers only get one chance each year to produce the best possible harvest, and computer visionenabled machines and the data they collect are key to making farming more productive to feed, fuel, and clothe our growing population. Creating a path to a better tomorrow while feeding the world today But it’s about more than just productivity. Farmers are stewards of the land. Their livelihood depends on it. Our planet’s health depends on it. Integrating computer vision and other advanced technologies like sensors, robotics, and AI will result in more efficient and more automated machines that can treat every single plant at a micro level, ultimately resulting in more sustainable farming operations. Computer vision is unlocking the fully autonomous farm of the future and allowing farmers to become more productive, profitable, and sustainable. RR

www.therobotreport.com

November 2023

doosanrobotics.com

Sensors

Commercializing

TACTILE SENSORS for robot dexterity

Addressing practical issues such as usefulness, reliability and affordability. HEBA KHAMIS CO-FOUNDER CONTACTILE

recently wrote an opinion piece about the need for tactile sensing for evolving robotic dexterity om toddler to adult. In the piece, I make a case for designing with tactile sensing in mind to be able to achieve truly useful humanoid robots that can learn new skills in dexterity. I received an insightful comment when I shared the article on LinkedIn that identifies the practical impediments to the uptake of existing tactile sensing technologies in robotics: “… The [robotics] community has been aching for something affordable that does something useful and reliably for ages. We’ve had amazing features om a wide range of tactile sensors, but they all miss the barriers in some key way that prevent[s] widespread adoption …”

This commentary om Jeremy Fishel, a reputable voice in tactile sensing and robotics, inspired me to explore the practical aspects of commercializing tactile sensors for humanoids in greater depth; particularly, the issues of usefulness, reliability and robustness, and affordability. Of course, none of these can be considered in isolation, and at the center, one must always be thinking of the objective: dexterity.

Usefulness of tactile sensors in robotics is highly dependent on the application At a foundational level, different applications require different sensing features – temporal and spatial resolution, sensing range and sensitivity, and even the dimensionality of the sensing. Tactile sensing on the torso and arms of a humanoid robot may be required to detect collisions and, possibly, to classi the Contactile’s PapillArray intentions of a person via sensor is a so , silicone physical interactions. For array that can measure example, gently pushing 3D deflection, 3D force the back of a humanoid and 3D vibration at each may signal that we want array element. It can it to move forward. This also measure emergent may be achieved with low properties such as spatial resolution and onetorque, incipient slip and dimensional tactile sensing iction such as pressure sensing. | Contactile On the other hand (pun intended), tactile sensing in the

November 2023

www.therobotreport.com

THE ROBOT REPORT

Say hello to the

palms and fingers of the humanoid robot are required to aid in manipulation. The ideal tactile sensor for robotic dexterity is modular with a scalable size (for customizing the spatial resolution). It has high responsiveness and customizable sampling equency (for tailoring the temporal resolution) and sensing range, and it is multi-dimensional. Naturally, there will be trade-offs between some of these features. It is unlikely that high spatial resolution, high temporal resolution, and high dimensionality can all be achieved simultaneously. And, if they could, there would be major implications for the downstream processing required to make use of the resulting data. This is one aspect of integration complexity that should also be considered when determining usefulness. Other aspects of integration complexity to consider include cable routing, communications, and power supply. Reliability is a must, but robustness can depend on cost and replaceability Reliability can mean a few different things. I take it to mean that applying the same stimulus to a sensor repeatedly will yield the same output. Sensor dri , usually due to temperature (and humidity) changes, electromagnetic interference, and other electrical interference (e.g., capacitance), can thwart the reliability of a sensor and, thus, reduce confidence in the data and the usefulness of a sensor overall. Robustness is somewhat related to reliability in that the sensor must be able to survive repeated stimuli. The number of stimuli the sensor is expected to survive will depend on the application and the cost and effort associated with replacing part of or all of the sensor. For example, in grocery e-commerce fulfillment, a tactile sensor on a binpicking robot may be subjected to 1,000 cycles an hour. It might be acceptable to replace a consumable part of the sensor each day/month/year, if it took only a minute/hour/day and cost only a few cents/dollars/tens of dollars; of course, this must be multiplied by the number of tactile sensors that are deployed in any one solution/facility. THE ROBOT REPORT

H-SERIES: … The [robotics] community has been aching for something affordable that does something useful and reliably for ages. We’ve had amazing features from a wide range of tactile sensors, but they all miss the barriers in some key way that prevent[s] widespread adoption …

The safest cobot on the marketplace, delivered in 4 weeks or less.

Six-axis articulation Force torque sensors on each axis PLe CAT 4 Safety Rating 1700mm Reach 25kg Payload

doosanrobotics.com

Think of affordability om the perspective of the value of the problem the sensor solves. Affordability of a sensor is an interesting metric that must be considered together with the usefulness of the sensor, as well as the value of the problem that the sensor solves. The more useful a sensor is, the more one is willing to pay for it. The affordability of a useful sensor for a robot that is destined for Mars to build scientific outposts is very different to one that will end up in a robot that is sorting packages in a postal facility. Affordability is also something that is more achievable with volume. Some of the earliest accelerometers (resistancebridge-type) were sold in the early 1930s to only a couple of hundred customers for $420 U.S.D. each (the equivalent of $1,827 in U.S. dollars today). Today, you can purchase a 3-axis MEMS accelerometer in single unit quantity for less than $1.50, and accelerometers are in practically every mobile device out there.

www.therobotreport.com

November 2023

41 23-0922-Doosan_1-3-Ad.indd 1

10/18/23 11:52 AM

S U M M I T

E X P O

D I S C O V E R T H E L AT E S T A D VA N C E M E N T S I N

Sponsorship opportunities are available for future DeviceTalks and Robotics programs.

For more information, contact Colleen Sepich. 857.260.1360 | csepich@wtwhmedia.com

Robots must be designed with the right sensor for the right purpose at the right price In robotics, tactile sensing for sensing’s sake is the wrong approach – it will drive up costs, power consumption and processing requirements, and may provide little value in return. Tactile sensing must be targeted and planned. Robotics companies should ask themselves the following: What is the purpose of the sensing and what sensors can be used to achieve that purpose? It may be possible for the same sensor to be used for multiple purposes across several areas of the robot; but if not, then

the right sensor must be used in the right place for the right purpose. Contactile’s sensors are designed for a purpose – robotic dexterity – and as such the practical issues of usefulness, reliability, and affordability are all but solved. What are roboticists willing to pay for a reliable and robust tactile sensor that measures the exact parameters that enable a humanoid robot to achieve human-like dexterity? These tactile sensors could be the difference between a humanoid robot that can learn skills in dexterity like people do, and one that is stuck with toddler-like dexterity. RR

Unleash your automation potential with Doosan Robotics. Effortless precision, unbeatable efficiency.

doosanrobotics.com

Contactile said that if an end effector uses its tactile sensors, the grip parameters are determined in realtime, and an optimal grip force is applied dynamically. Preprogramming the grip force is no longer required - just tell the gripper to grip and the sensor feedback will take care of the rest. | Contactile

THE ROBOT REPORT

www.therobotreport.com

November 2023

Software

Tormach’s open-source, ROS-based robotics platform - which includes the control system, industrial robot hardware, and full access to all system parameters - creates a fast, accessible solution that brings industrial robotics to more researchers, developers, and students. | ORNL/Jill Hemman

Bridging the gap between

RESEARCH, INDUSTRIAL ROBOTICS using ROS

STEVE CROWE • EXECUTIVE EDITOR • THE ROBOT REPORT

Tormach created a ROSbased industrial robotic manipulator and control system that avoids “black box” issues that can plague modern robotics applications. 44

November 2023

The

Robot Operating System (ROS) is a powerful open-source platform for robotics research, but until recently it lacked industrial-quality hardware that is tightly integrated with the ROS so ware stack. Robot equipment manufacturers use proprietary, closed-source so ware and control systems for their manipulators, leaving researchers with a steep hill to climb in order to use ROS on industrial robots. Addressing this need and advancing the capabilities of the ROS development community, Tormach has created a ROS-based industrial robotic manipulator and control system that avoids “black box” issues that plague modern robotics applications. Additionally, Tormach’s control system, PathPilot, uses Python as the robot programming language, creating an intuitive programming interface for robot motion and unleashing the potential of the Python package ecosystem.

www.therobotreport.com

THE ROBOT REPORT

High performance materials with custom converting options

This open-source, ROS-based robotics platform – which includes the control system, industrial robot hardware, and full access to all system parameters – creates a fast, accessible solution that brings industrial robotics to more researchers, developers, and students.

•

The problem with ROS and proprietary robot control systems Robot control manufacturers are hesitant to allow ROS developers to access all the system parameters in their closed-source controls for the following reasons: • They have invested significant resources into developing and testing their proprietary control systems and don’t want to expose the inner workings of their system to external researchers for fear of losing intellectual property. • There are risks associated with exposing the system parameters to external researchers. Untested usage may introduce bugs or other issues that could compromise the safety or reliability of the system.

•

Legal or contractual obligations o en prevent manufacturers om sharing proprietary information with external parties. They may be concerned about potential liability issues if their closed-source control systems are modified by external parties. There is little financial incentive for most manufacturers, and in many cases, there is a strong disincentive: the need to invest in additional documentation, training, and support in astructure to enable researchers to work with their control systems effectively.

For these reasons, integrations between ROS and commercially available robot hardware are limited. While drivers exist to connect ROS to other industrial robots, their low (10 – 100Hz) bandwidth implementations simply drip-feed waypoints to a proprietary, closed-source controller. As a result, the user may not have access to whether or not the

Industry solutions: • •

• • •

Surface protection for (EOAT) Custom PTFE, UHMW or Nylon adhesive backed wear strips or sheets for high impact areas. Custom washers for fastening motion components. Flexible polymer films and silicones for soft robotic applications. Low minimum prototype die cutting.

Custom Converting Custom Roll Slitting High Speed Sheeting Adhesive Lamination Die Cutting

The OpeN-AM experimental platform, installed at the VULCAN instrument, features a Tormach ZA6 robotic arm that prints layers of molten metal to create complex shapes. Studying the 3D-printed welds microscopically with beams of neutrons allows researchers to better understand factors such as stress caused by heating and cooling. | ORNL/Jill Hemman THE ROBOT REPORT

www.therobotreport.com

November 2023

Phone (800) 461-4161

www.cshyde.com

Sensors

Python, a programming language that’s been in use for decades, makes programming a Tormach robot accessible for many. The sheer number of devices and so ware that run on Python means the ZA6 has a seemingly countless number of integrations that are possible. | ORNL/Jill Hemman

robot adheres to timing, velocity, and path accuracy intents. Data like motor torque, current, following error is usually unavailable, and the slow control loop severely limits what researchers can accomplish. Motor and drive feedback to ROS The ROS/HAL hardware and so ware stack offers feedback that can provide valuable control opportunities. The ZA6 provides the following: • Feedback om each joint, standard configuration: position, velocity feedback, torque feedback in SI units, following error, diagnostics-like error

November 2023

•

• •

code, with configuration, drives can also report additional diagnostics like motor/encoder temperatures and error code history. 10 digital inputs + 12 digital outputs (one digital input usable as probe input) HAL can report RT latency Feedback om ROS and MoveIt, especially Cartesian pose

Most of these feedback elements focus on only the lower-level control layers. Higher-level control layers can provide other opportunities, depending on research needs.

www.therobotreport.com

Meet HAL: The open-source Hardware Abstraction Layer The connection between ROS and a robot’s hardware relies on a hardware abstraction layer (HAL). HAL evolved out of the opensource Enhanced Machine Controller (EMC) project that had its origin 25 years ago at the National Institute of Standards and Testing (NIST). Active development of HAL continues today via the LinuxCNC and Machinekit projects because HAL is flexible, 100% open-source, and is used in thousands of machines around the world. HAL consists of modular components (loadable binary modules) that communicate with each other by updating, reading, and writing named pins that connect via named signals. In some ways, HAL is like ROS, but there are important differences: • Using PREEMPT-RT Linux extensions, HAL components written in C execute in a 1kHz real-time thread with minimum jitter. • HAL has many pre-written components designed for low-level hardware control (PWM generators, stepper driver step generators, BLDC and three-phase motor controls, and more. The Tormach robot bridges the gap between ROS with the open-source hal_ ros_control component. The combination of HAL and ROS allows a wealth of robot data to be exposed to the user. All process data is accessible via shell commands, data logger utilities, and a graphical scope. All information on the EtherCAT bus, including torque, current, following error, position, velocity, and more are available at 1 kHz and exposed via HAL to ROS. Since HAL is modular and flexible, users can alter their robot’s HAL configuration using pre-built HAL components or by writing new components in C or Python, allowing easy integration with almost any external device or process. Preconfigured for ROS Previously, using a commercially available robot with ROS required finding and downloading the appropriate driver for the control, a URDF file to describe kinematics; creating a moveit configuration, choosing one or more planners, IK solvers, and

THE ROBOT REPORT

perhaps finding and bringing solid models into Rviz. Configuring a new robot for use with ROS is challenging even for experienced ROS developers. An optimized default ROS configuration for the manipulator, like that provided by Tormach as part of the control, helps alleviate many of these issues. The URDF model (unified robot description format) is defined, motion pipelines are configured, and trajectory planners and kinematics solvers are selected and optimized so that the robot is ready to work out of the box. The robot hardware, user interface, and robot programming language are fully documented and supported by Tormach. The robot’s default ROS configuration will be ideal for most applications, saving months of configuration time, and it’s also open-ended to allow users to develop their own unique configurations at will. Python: the robot’s programming language The lack of an industry-standard robot

programming language led Tormach to choose Python for its ZA6 robot. The Tormach Robot Programming Language (TRPL) uses the Python 3 interpreter and works similarly to other common robot programming languages, with commands for different move types, commands to read and set inputs and outputs, and commands to set and change tool and user ames. It is important to note that any Python 3 program is a valid robot program. The robot’s ability to interpret any Python program means that almost any Python package can be imported to help with more challenging robot tasks. Examples include: • Using the csv and http requests libraries to upload data files recorded by the robot to a web server. • Using opencv to recognize ArUco markers for visual servoing and localization. • Using numpy and kdl to calculate forces in cartesian space om the joint torque feedback and robot Jacobian.

• •

Using Twilio to send text messages om the robot. Using ChatGPT and the Python OpenAI API to conversationally create robot programs.

While the TRPL interpreter simplifies a lot of programming tasks like move commands and offsets, power users who are familiar with ROS are able to access the underlying ROS API directly. Suitable for research and education The PathPilot user interface makes it easy to write simple teach-mode programs to help students learn the concepts they need to be successful in industrial robotics. Unlike other robots designed for the classroom, the ZA6 teaches industrial robot concepts like user ames, tool ames, waypoint programming, and Cartesian-versus-joint angle waypoint types. Another reason to use the robot as a teaching tool is its easy-to-learn user interface. RR

Your partner for innovative manufacturing

Superior encoders for position and motion control With a comprehensive line up of encoder solutions, Renishaw brings the expertise needed to address your manufacturing challenges. Whether your application calls for optical, magnetic or laser technology, our encoders achieve the highest levels of accuracy, durability and reliability. Powerfully positioned for innovative motion control.

www.renishaw.com/encoders Renishaw Inc., West Dundee, IL 60118

usa@renishaw.com

Encoder General Half PageRobotReportWTWH 09_23.indd 1

8/10/23 9:30 PM

Robotics Robotics

UtiliTrak® Compact Linear Guide: Precision in Parallel Motion Maximize efficiency with UtiliTrak®, the ultimate solution for running two systems in parallel. Featuring DualVee® guide wheels and MadeWell® crowned rollers, it ensures precise

Bishop-Wisecarver® 2104 Martin Way Pittsburg, CA 94565 www.bwc.com 925-439-8272

performance even when mounting surfaces aren’t perfectly parallel. UtiliTrak® compensates for mounting errors, eliminating the need for absolute parallelism. The DualVee® guide wheel plate assembly and vee channel drive the motion side, while the MadeWell® crowned roller and open channel side accommodate parallel misalignment. Choose Bishop-Wisecarver®’s UtiliTrak®: easy to order, quick to turn around, and with all the engineering support you need to complete your project on time.

Canon U.S.A., Inc. DC brushless Servo Motors Today’s increasing demands of automation and robotics in various industries, engineers are challenged to design unique and innovative machines to differentiate from their competitors. Within motion control systems, flexible integration, space saving, and light weight are the key requirements to design a successful mechanism. Canon’s new high torque density, compact and lightweight DC brushless servo motors are superior to enhance innovative design. Our custom capabilities engage optimizing your next innovative designs.

Canon U.S.A., Inc. Motion Control Products

We are committed in proving technological advantages for your success.

408-468-2320 www.usa.canon.com

November 2023

www.therobotreport.com

3300 North First Street San Jose, CA, 95134

THE ROBOT REPORT

Robotics Robotics

CGI Inc. Advanced Products for Robotics and Automation At CGI we serve a wide array of industries including medical, robotics, aerospace, defense, semiconductor, industrial automation, motion control, and many others. Our core business is manufacturing precision motion control solutions. CGI’s diverse customer base and wide range of applications have earned us a reputation for quality, reliability, and flexibility. One of the distinct competitive advantages we are able to provide our customers is an engineering team that is knowledgeable and easy to work with. CGI is certified to ISO9001 and ISO13485 quality management systems. In addition, we are FDA and AS9100 compliant. Our unique quality control environment is weaved into the fabric of our manufacturing facility. We work daily with customers who demand both precision and rapid turnarounds.

ISO QUALITY MANAGEMENT SYSTEMS: ISO 9001• ISO 13485 • AS9100 • ITAR SIX SIGMA AND LEAN PRACTICES ARE EMBRACED DAILY WITHIN THE CULTURE

CGI Inc. 3400 Arrowhead Drive Carson City, NV 89706 Toll Free: 1.800.568.4327 Ph: 1.775.882.3422 Fx: 1.775.882.9599 WWW.CGIMOTION.COM

Low Friction Material Fabrication For Industrial Automation Low friction polymers are important for automation applications as they can reduce wear and energy consumption, improve efficiency, and extend the lifespan of equipment. PTFE, UHMW, POM, PEEK, Nylon, and Polyimide films are often used in various components of automated systems, including insulators, bearing liners, slides, gears, and other moving parts. CS Hyde supplies various material types, thicknesses, adhesive options, and specialized converting capabilities to create material solutions for friction, impact, or abrasion related issues formed in the fast-moving environment of industrial automation. Common applications include wear strips, conveyor liners, bumper plates, and wire and cable wrapping.

THE ROBOT REPORT

www.therobotreport.com

CS Hyde Company www.cshyde.com 39655 N. IL Rt. 83 Lake Villa, IL 60046 800-461-4161

November 2023

Robotics Robotics 50

DigiKey www.digikey.com 701 Brooks Avenue South, Thief River Falls, MN 56701 USA sales@digikey.com 1-800-344-4539

Omron’s TM Collaborative Robots Increase production and reduce worker fatigue with Omron Automation’s TM Collaborative Robots. Repetitive tasks such as pick and place inspection, machine tending, assembly, and more are easily handled by the 6 axis TM cobot. Utilizing graphical programming with the TMFlow software, hand guided position recording, and a built-in vision system, users can set up simple applications in mere minutes. Omron’s TM Collaborative Robots are ISO 10218-1:2011 and ISO/TS-15066 compliant and include features like rapid changeover using TMVision and Landmark, advanced collaborative control, and external camera support.

Harmonic Drive

Servo Grade AMR Propulsion Drive Trains The drive wheels on Autonomous Mobile Robot Platforms benefit from the inherent characteristics of harmonic planetary technology, primarily, smoothness of travel, backdrivability, and an efficiency curve that does not degrade over time. The integral cross roller bearing construction contributes to an excellent balance of torque density, stiffness, and radial load support in a compact and reliable package. Available in dozens of standard reduction ratios, these low backlash drive trains include motor adaptation as required.

November 2023

42 Dunham Ridge Beverly, MA 01915 United States www.harmonicdrive.net

www.therobotreport.com

THE ROBOT REPORT

Robotics Robotics

Kona Connectors 60A per contact, High-reliability Connectors Kona expands the Harwin High-Reliability portfolio into new levels of current – 60A per contact. Designed to withstand testing environments and conditions. • 2, 3 or 4 single row configurations • Vertical cable-to-board or cable-to-cable • Vibration and shock resistant, compact power future-proofed • 60A max per contact • 6-finger contact design to maintain electrical contact through high vibration and shock Other useful features include reverse fix screws, polarization, and identification of the #1 position. Harwin provides CAD Models, Test Reports and F.O.C samples to assist you in the design process. For more information, visit harwin.com

Rotary encoders from RENCO The new R35i and R35iL encoders from RENCO are incremental rotary encoders without integral bearings. With resolutions of up to 40,000 measuring steps per revolution, they are particularly well-suited as feedback systems for stepper motors in Closed Loop operation and for exact position measurement in measuring machines such as laser scanners. For particularly limited mounting situations, RENCO offers the R35iL. It is one of the smallest rotary encoders currently on the market, with a height of only 8.6 mm.

HEIDENHAIN CORPORATION 333 East State Parkway Schaumburg, IL 60173 847-490-1191 www.heidenhain.us marketing01@heidenhain.com

HEIDENHAIN produces the RENCO rotary encoders, and also sells them worldwide. This means that RENCO rotary encoders benefit from HEIDENHAIN’s comprehensive know-how in the development and production of encoders. HEIDENHAIN develops and manufactures linear and angle encoders, rotary encoders, digital readouts and numerical controls for demanding positioning tasks.

THE ROBOT REPORT

www.therobotreport.com

November 2023

Robotics Robotics

Easily design a maintenancefree gantry robot system that meets all your requirements for up to 40% less cost Gantry robots have become a critical component for manufacturing and warehouse automation, and igus® provides turnkey and custom gantry systems that are built with self-lubricating plastic liners which are engineered to slide instead of roll, allowing for smoother and quieter operation vs. traditional recirculating ball bearings. Every gantry system that igus® offers provides a lightweight, corrosion-resistant solution that is ideal for pick and place, sorting, labeling, measuring, inspection, and repetitive material handling applications. Due to their modular nature, igus® gantry robots are easily customizable, and free onsite consultation is available to ensure the perfect solution at the best price possible.

igus, inc. 57 Ferris Ave, Rumford, RI 02916 www.igus.net 800.521.2747 sales@igus.com

Interconnect Solutions

LEMO® is the industry leader in the design and production of precise custom interconnect systems. LEMO products are designed and manufactured according to rigorous and controlled processes. Inspection and traceability of products are systematically ensured in compliance with our standards. High-quality LEMO Push-Pull connectors are used in a wide range of challenging application environments, such as medical, test & measurement, research, defense & military, information systems, aerospace & autonomous vehicles, robotics, automotive, industrial control, nuclear, broadcast & audio-video, and communications.

LEMO® has been designing precision connectors for over seven decades. Offering more than 90,000 combinations of products that continue to grow through customer-specific designs, LEMO® and its brands REDEL®, NORTHWIRE®, and COELVER® currently serve more than 100,000 customers in over 80 countries around the world.

November 2023

www.therobotreport.com

LEMO USA, Inc. 635 Park Ct. Rohnert Park, California 94928 Ph: 707.578.8811 E-mail: info_us@lemo.com Website: https://www.lemo.com/en

THE ROBOT REPORT

Robotics Robotics

Fully Automate Your Automation The factory of the future demands higher output with increased flexibility, and cobots can’t do it alone. The Cobot Feeder from Applied Cobotics addresses these demands with high-mix, high-volume production while eliminating cobot downtime and alleviating staffing issues. The bottom-line result is increased output without increased labor. The Cobot Feeder presents a worktable that adjusts vertically, synchronizing with each custom tray on the rack tower. A horizontal loader/unloader provides accurate and repeatable motion while positioning trays in front of the cobot. These features, along with the portable dunnage tray cart, make the Cobot Feeder from Applied Cobotics an essential complement to any collaborative automation system. With a 90-day ROI, small to medium-sized enterprises will quickly achieve lights-out manufacturing, increasing safety, quality—and profits.

PBC Linear 6402 E. Rockton Road Roscoe, Illinois 61073 USA +1.815.389.5600 pbclinear.com

SmartRunner Explorer 3-D for Robotic Gripping Enter the third dimension of automation at the speed of light—the SmartRunner Explorer 3-D generates highprecision 3-D point cloud images in addition to 2-D images. It is optionally available with stereo vision technology or time-offlight (ToF) technology. • • • • • • •

THE ROBOT REPORT

Laser class 1 for eye safety Operating frequency up to 30 Hz Gigabit Ethernet TCP/IP interface Compact dimensions (165 x 67 x 56 mm) Supression of ambient light > 100 klx Raw image size 640 x 480 pixels Universal ROS drivers

www.therobotreport.com

Pepperl+Fuchs, Inc. 1600 Enterprise Parkway Twinsburg, Ohio 44087 USA Phone: 330-486-0002 www.pepperl-fuchs.com sales@us.pepperl-fuchs.com

November 2023

Robotics Robotics

Meet POSITAL’s Next Generation Position and Motion Sensors! POSITAL has launched a major upgrade to its IXARC rotary encoders, kit encoders and TILTIX inclinometers. Introduced as part of POSITAL’s NEXTGEN initiative, the upgrades include new TMR magnetic sensors for encoders, new MEMS accelerometer packages for inclinometers, and updated microprocessors for most products. There is also a new in-house developed ASIC and new Wiegand sensors that support zero-maintenance, battery-free operation of multiturn encoders. These changes deliver improvements to precision and energy efficiency, while maintaining mechanical and electrical compatibility with earlier products. More NeXtGen product announcements are on the way. Stay tuned!

POSITAL Inc. 1 N. Johnston Ave. Suite C238 Hamilton, NJ 08609 USA

For more information, please visit: www.posital.com

Tel: 609-750-8705 info@fraba.com

An encoder series designed for use in harsh environments

Renishaw’s FORTiS™ delivers superior repeatability, reduced hysteresis, and improved measurement performance due to an innovative non-contact mechanical design that does not require a mechanical guidance carriage. It provides high resistance to the ingress of liquids and solid debris contaminants. It features an extruded enclosure with longitudinally attached interlocking lip seals and sealed end caps. The readhead body is joined to a sealed optical unit by a blade, which travels through the lip seals along the length of the encoder. Linear axis movement causes the readhead and optics to traverse the encoder’s absolute scale (which is fixed to the inside of the enclosure), without mechanical contact.

November 2023

Renishaw, Inc. 1001 Wesemann Dr. West Dundee, IL, 60118 Phone: 847-286-9953 email: usa@renishaw.com website: www.renishaw.com

www.therobotreport.com

THE ROBOT REPORT

Robotics Robotics

Ruland Manufacturing Zero-Backlash Couplings for Robotic Systems Ruland Manufacturing offers a variety of zero-backlash servo couplings designed for use in high precision applications like automation and robotics. Ruland offers beam, bellows, disc, oldham, jaw, and newly-released Controlflex couplings in thousands of off-the-shelf combinations and sizes to help designers optimize their systems. Robotic vision systems, material handling robots, and automated guided vehicles have infamously strict requirements that require engineers to balance torque, weight, dampening, and more, all while retaining extremely precise power transmission. Ruland servo couplings excel in demanding applications and can be selected based on a wide variety of performance characteristics. Visit Ruland.com for access to everything you need to make a coupling design decision including: full technical product data, 3D CAD models, installation videos, and eCommerce to make prototyping easy.

Ruland Manufacturing 6 Hayes Memorial Dr. Marlborough, MA 01752 508-485-1000 www.ruland.com email: sales@ruland.com

Compact, rugged motion sensing for any task Silicon Sensing’s gyroscopes, accelerometers and inertial systems offer precise, compact and affordable solutions for robotics applications, in any environment. Our products include: • DMU11 - a low cost, compact, precise, six-degreesof-freedom (6-DOF) device delivering marketleading performance that is calibrated over its full rated temperature range. • PinPoint® - A tiny gyro measuring only 5mm x 6mm and delivering on performance, reliability and price. Available in both flat and orthogonal mounts. • CMS300 - a robust, compact gyro and dual-axis accelerometer delivering precise performance with low power consumption. Available in both flat and orthogonal mount packages.

Silicon Sensing Clittaford Road Southway Plymouth Devon PL6 6DE England

Inertial sensing for any task.

Telephone: 01752 723330 www.siliconsensing.com/products

THE ROBOT REPORT

www.therobotreport.com

November 2023

Ad Index Advanced Navigation ..........................................................23

Bishop Wisecarver ......................................................... 33,48 Bodine Electric Company .............................................35,37 Boker’s ..........................................................................................18 Canon U.S.A. Inc..................................................................11,48 CGI Inc. .................................................................................49,BC Chieftek Precision .................................................................27 CS Hyde Company .......................................................... 45,49 DigiKey Electronics ...........................................................3,50 DOOSAN Robotics Americas ................................ 39,41,43 Harmonic Drive .................................................................... 1,50 Harwin .................................................................................... 17,51 HEIDENHAIN CORPORATION ......................................... 5,51 igus .........................................................................................21,52 LEMO USA ........................................................................52,IBC Parvalux, a maxon company .......................................cover PBC .........................................................................................13,53 Pepperl + Fuchs .............................................................. 25,53

11 • 2023 therobotreport.com

SALES

LEADERSHIP TEAM

Ryan Ashdown

Publisher Mike Emich

rashdown@wtwhmedia.com 216.316.6691

Jami Brownlee

jbrownlee@wtwhmedia.com 224.760.1055

Mary Ann Cooke

mcooke@wtwhmedia.com 781.710.4659

Jim Dempsey

jdempsey@wtwhmedia.com 216.387.1916

Mike Francesconi

mfrancesconi@wtwhmedia.com 630.488.9029

Jim Powers

jpowers@wtwhmedia.com 312.925.7793 @jpowers_media

memich@wtwhmedia.com 508.446.1823 @wtwh_memich

CEO Scott McCafferty

smccafferty@wtwhmedia.com 310.279.3844 @SMMcCafferty

EVP Marshall Matheson

mmatheson@wtwhmedia.com 805.895.3609 @mmatheson

CFO Ken Gradman

kgradman@wtwhmedia.com 773-680-5955

Courtney Nagle

cseel@wtwhmedia.com 440.523.1685 @wtwh_CSeel

POSITAL .............................................................................. 29,54 Renishaw ............................................................................47,54 Ruland Manufacturing ......................................................7,55 SICK, Inc. ...................................................................................... 4 SIKO ..............................................................................................12 Silicon Sensing Systems ................................................9,55 Stow Robotics ....................................................................... IFC

November 2023

www.therobotreport.com

FOLLOW THE WHOLE TEAM

@DESIGNWORLD

THE ROBOT REPORT

Learn More Scan QR Code!

LEMO Robotic Connectors These connectors deliver exceptional performance in harsh environments, ensuring long-lasting durability. Whether in industrial automation, medical robotics, or for Rovers and Legged Robots, LEMO connectors provide the ability to connect sensors, actuators, and control systems seamlessly.

• •

Light weight Durable

• •

Compact Ergonomic

YOUR CUSTOM SOLUTIONS ARE

CGI STANDARD PRODUCTS

Advanced Products for Robotics and Automation CGI Motion standard products are designed with customization in mind. Our team of experts will work with you on selecting the optimal base product and craft a unique solution to help differentiate your product or application. So when you think customization, think standard CGI assemblies. Connect with us today to explore what CGI Motion can do for you.

800.568.GEAR (4327) • www.cgimotion.com