ROBOTIC TOUCH Can robots be made to have tactile sensing?

Robotic touch

Can robots be made to have tactile sensing?

by leong yoke Peng

Three decades ago, Roberta L. Klatzky and Susan J. Lederman, formally established that people are exceptionally good at detecting and recognising objects, even when tactile sensing is the only method ƒ˜ƒ‹Žƒ„Ž‡Ǥ—’’‘•‡–Šƒ–›‘—ƒ”‡”‡ƒ…Š‹‰‹–‘›‘—”„ƒ‰–‘ϐ‹†ƒ‡›Ǥ‘•–‘ˆ—•™‘—Ž†„‡ƒ„Ž‡–‘ϐ‹†–Š‡‡›‡˜‡ without looking into the bag. However, for a robot, the same task proves to be extremely complex; in addition to the ability to search and take out the key, the robot needs to know what tactile sensing (e.g., the feeling of touching a key) means in terms a computer can express and analyse (i.e., numbers and equations).

Aside from intellectual curiosity and personal satisfaction that comes from successfully building a cool robot, we seek to develop intelligent machines or robots because they often perform a task more ‡ˆϐ‹…‹‡–Ž›–Šƒ—•ǤŠ‡›Šƒ˜‡„‡‡™‹†‡Ž›—•‡†ƒ• human replacements in areas such as automotive manufacturing, micro-scale fabrication and exploration of remote environments (e.g., space, Figure 1: A unique feature of a key that human hand recognises is deep sea, and the Antarctic). One main purpose of the row of edges on the key blade. these robots is to explore an environment and/or –‘ƒ‹’—Žƒ–‡•—””‘—†‹‰‘„Œ‡…–•Ǥ‘†‡–‡”‹‡ the next course of action during exploration and manipulation tasks, the robot needs to be able to understand its surroundings and distinguish different objects based on the sensory data it has collected.

ƒ’–‹…‡š’Ž‘”ƒ–‹‘Ȃ—•‹‰–Š‡•‡•‡‘ˆ–‘—…Š–‘‡š’Ž‘”‡‘‡ǯ••—””‘—†‹‰•Ȃ‹•‘‡’‘••‹„Ž‡‘„Œ‡…–‹†‡–‹ϐ‹…ƒ–‹‘ ƒ’’”‘ƒ…ŠǤŠ‹•ˆ‘”‘ˆ‡š’Ž‘”ƒ–‹‘‹•—‹“—‡„‡…ƒ—•‡‹–‹˜‘Ž˜‡•†‹”‡…–…‘–ƒ…–„‡–™‡‡ƒ•‡•‘”ƒ†ƒ‘„Œ‡…–Ǥ Š‹•†‹”‡…–…‘–ƒ…–’‘•‡•ƒ›…ŠƒŽŽ‡‰‹‰’”‘„Ž‡•–‘•‹—Ž–ƒ‡‘—•Ž›ƒ‹’—Žƒ–‡–Š‡•‡•‘”ƒ†‡š’Ž‘”‡ƒ surface. For example, when one does not have a prior knowledge about a surface, a sensor (e.g., a robotic hand) might be pushing too hard on the “...we seek surface destroying the surface to be explored. But, unlike exploration methods that rely on vision and auditory sensors, haptic exploration is to develop not limited by interferences between sensors and objects. When a robot is manipulating an object, the manipulator usually blocks the vision or intelligent auditory sensors from fully detecting the object. In such cases, touch based exploration, in which a sensor makes direct contact with the object, is a machines or better choice. robots because WHY SHOULD WE CARE ABOUT ROBOTIC TOUCH? ‡ϐ‹‹‰ –Š‡”‘„‘–‹…•‡•‡‘ˆ –‘—…Š‹•‡••‡–‹ƒŽ‘–‘Ž› ˆ‘”„—‹Ž†‹‰ they often automated machines which can identify objects and its surroundings during exploration and manipulation. It is also valuable for building a perform a task more intuitive prosthetic hand that can more precisely mimic the object ‹†‡–‹ϐ‹…ƒ–‹‘ƒ„‹Ž‹–›‘ˆƒ”‡ƒŽŠƒ†Ǥ•‹‰–Š‡‡›ƒ•ƒ‡šƒ’Ž‡ǡƒ†‹•–‹…– more efficiently feature of a key that a human hand recognises is the row of edges on the key blade (Figure 1). Although detecting the edges on the key blade using than us.” ‘‡ǯ•Šƒ†…‘‡•ƒ–—”ƒŽŽ›ǡ–Š‡•ƒ‡–ƒ•‹‰Š–„‡†‹ˆϐ‹…—Ž–ˆ‘”’”‘•–Š‡–‹…• —•‡”•Ǥ‘‹’”‘˜‡–Š‡ˆ—…–‹‘ƒŽ‹–›‘ˆƒ’”‘•–Š‡–‹…Šƒ†ǡ™‡‡‡†–‘‡ƒ„Ž‡‹––‘‹†‡–‹ˆ›‘„Œ‡…–ˆ‡ƒ–—”‡••‘–Šƒ– ‹–…ƒ’”‘˜‹†‡–Š‹•‹ˆ‘”ƒ–‹‘–‘ƒ—•‡”ƒ†ˆƒ…‹Ž‹–ƒ–‡‘„Œ‡…–‹†‡–‹ϐ‹…ƒ–‹‘–ƒ••Ǥ‡…‡ǡƒ”‘„—•–‡…Šƒ‹• –‘†‡ϐ‹‡–Š‡”‘„‘–‹…•‡•‡‘ˆ–‘—…Š‹•…”—…‹ƒŽˆ‘”‘”‡˜‡”•ƒ–‹Ž‡ƒ†‹–—‹–‹˜‡’”‘•–Š‡–‹…•Ǥ

HOW DOES A ROBOT FEEL? ‘ †‡ϐ‹‡ –Š‡ ”‘„‘–‹… •‡•‡ ‘ˆ –‘—…Šǡ™‡ ϐ‹”•– ‡‡† to understand what makes a robot a robot. Robots are machines which can collect data about their environment, process the data, and react based on the processed data. Generally, a robot encompasses a couple of sensors, a processor, and a few actuators. Sensors are used to detect and collect data from the surroundings. A processor can be thought as the “brain” of a robot. It analyses the data gathered to gain useful ‹•‹‰Š–•ˆ”‘–Š‡ǤŠ‡ǡ‹–…‘ƒ†•ƒ…–—ƒ–‘”•–‘ react. Actuators are mainly motors which allow the robot to move itself or a part of itself. In principle, a robot does not have to look like human as portrayed in ‘˜‹‡•ƒ†•Š‘™•Ǥ•Ž‘‰ƒ•ƒ‘„Œ‡…–Šƒ••‡•‘”•ǡ ’”‘…‡••‘”•ƒ†ƒ…–—ƒ–‘”•ǡ‹–‹•…Žƒ••‹ϐ‹‡†ƒ•ƒ”‘„‘–Ǥ

Keeping that in mind, how does a robot know what it is touching? Let’s go back to the key example earlier. We are able to distinguish a key from other object „›”‡…‘‰‹•‹‰–Š‡‡†‰‡• ˆ‘—†‘ƒ‡›„Žƒ†‡ǤŠ‹• observation leads to the idea that an object can be detect and identify a geometrical feature on a surface —•‹‰ †ƒ–ƒ ‰ƒ–Š‡”‡† „› •‡•‘”• ‘ ƒ ”‘„‘–Ǥ  ™‘ types of measurement data are usually available from robotic haptic exploration: proprioceptive data (e.g., ϐ‹‰‡”Œ‘‹––”ƒŒ‡…–‘”‹‡•Ȍƒ†–ƒ…–‹Ž‡•‡•‘”†ƒ–ƒȋ‡Ǥ‰Ǥǡ contact normal o r c o n t a c t locations). “In principle, a C o l l o c a t e d robot does not tactile sensors, which provide have to look us with tactile sensor data, like human as a r e m o r e commonly used portrayed in in robotic haptic e x p l o r a t i o n . movies and TV However, it is not always practical shows.” to place tactile sensors at the location of contact. Instead, I focus on

Figure 2: Experimental set up to validate the surface feature detection algorithm. (left) PHANToM OMNI haptic device. (right) Surface traced by the stylus.

‹†‡–‹ϐ‹‡†—•‹‰–Š‡•‡•‡‘ˆ–‘—…Š„›†‡–‡…–‹‰–Š‡ object’s geometrical features, and thus, an object can „‡†‡ϐ‹‡†„›–Š‡…‘ŽŽ‡…–‹‘‘ˆ—‹“—‡ˆ‡ƒ–—”‡•ˆ‘—†‘ ‹–ǤŠ‹•„‘––‘Ǧ—’ƒ’’”‘ƒ…Š•‹’Ž‹ϐ‹‡•–Š‡…‘’Ž‹…ƒ–‡† –ƒ• ‘ˆ –ƒ…–‹Ž‡ ‘„Œ‡…– ‹†‡–‹ϐ‹…ƒ–‹‘ǡ ƒ† ‰‹˜‡• —• ƒ •›•–‡ƒ–‹…™ƒ›–‘†‡ϐ‹‡–Š‡”‘„‘–‹…•‡•‡‘ˆ–‘—…ŠǤ •‹‰–Š‹•ƒ’’”‘ƒ…Šǡ•—”ˆƒ…‡ˆ‡ƒ–—”‡†‡–‡…–‹‘„‡…‘‡• an imperative part of robotic haptic exploration.

SURFACE FEATURE DETECTION AND LOCALISATION Generally, robotic haptic exploration encompasses several research areas including sensor design, control and exploration algorithms, and data interpretation. For example, my research focuses on data interpretation ˆ‘” ‘„Œ‡…–‹†‡–‹ϐ‹…ƒ–‹‘ †—”‹‰ Šƒ’–‹… ‡š’Ž‘”ƒ–‹‘Ǥ Ž–‹ƒ–‡Ž›ǡƒ•’‹”‡–‘†‡˜‡Ž‘’ƒƒŽ‰‘”‹–Š–Šƒ–…ƒ

using the kinematic data to detect a feature on a surface –”ƒ…‡†„›ƒ”‘„‘–‹… ϐ‹‰‡”Ǥ‡”‡ǡ–Š‡‹‡ƒ–‹…†ƒ–ƒ ”‡ˆ‡”•–‘Œ‘‹–ƒ‰Ž‡–”ƒŒ‡…–‘”‹‡•‘ˆ–Š‡”‘„‘–‹…ϐ‹‰‡”Ǥ› work involves developing a new algorithm — impulsive hybrid system optimisation — to detect and localise a surface feature based on the kinematics and dynamics ‘ˆƒ”‘„‘–‹…ϐ‹‰‡”‘”•‡•‘”Ǥ

Š‹••—”ˆƒ…‡ˆ‡ƒ–—”‡†‡–‡…–‹‘ƒŽ‰‘”‹–Š‹•ƒ†˜ƒ–ƒ‰‡‘—• ‹ƒˆ‡™™ƒ›•Ǥ–‹•‡ˆϐ‹…‹‡–ƒ†•…ƒŽƒ„Ž‡Ǥƒ††‹–‹‘ǡ it is also shown to perform well under measurement noise and model noise in simulations and experiments (Figure 2ȌǤ”‹‡ϐŽ›ǡ–Š‹•ƒŽ‰‘”‹–Š–ƒ‡•–Š‡‹‡ƒ–‹… †ƒ–ƒ‘ˆƒ”‘„‘–‹… ϐ‹‰‡” –”ƒ…‹‰ƒ•—”ˆƒ…‡ǡ’‡”ˆ‘”• least-squares estimation, and returns the location of a surface feature. Instead of explaining every theory behind this algorithm, it is more worthwhile to point

(a) Data gathered

(b) Data filtering

(c) Data smoothing

Figure 3: Suppose that a robotic finger traces a surface and collects data as shown in (a). A data filtering-based method estimates whether a surface is encountered at the current step [red dot in (b)] based on data gathered in previous steps. The difference between a feature and noise in data is unclear at this point. In (c), a clear picture of a slope is seen when data for the entire time window are considered in a smoothing-based method. Hence, a feature is considerably more salient when more data is included, so this algorithm focuses on detecting features over windows of data.

out an essential property that allows this algorithm contact me for a copy of my journal paper on this to perform so well even when the data are very noisy. –‘’‹…Ǥ•ƒϐ‹ƒŽ‘–‡ǡ”‘„‘–‹…–‘—…Šȋ‘”‘”‡…‘‘Ž› Performance against noise is extremely important for known as haptics by researchers in this area) is any algorithm involving real ƒ —Ž–‹†‹•…‹’Ž‹ƒ”› ”‡•‡ƒ”…Š ϐ‹‡Ž†Ǥ world measured data because these data are often noisy. “Enabling a Joint efforts from people of different backgrounds (mechanical, electrical, For the purpose of feature robot to touch material, etc.) are instrumental for its development. detection, a data smoothing method is preferable to a data is essential ABOUT THE AUTHOR ϐ‹Ž–‡”‹‰‡–Š‘†‹Šƒ†Ž‹‰ ‘‹•› †ƒ–ƒǤ  †ƒ–ƒ ϐ‹Ž–‡”‹‰ for automated Leong Yoke Peng is currently a ϐ‹”•– ›‡ƒ” ŠǤǤ •–—†‡– ‹ ‘–”‘Ž method, such as multiple hypothesis testing, estimates tactile object & Dynamical Systems at California •–‹–—–‡‘ˆ‡…Š‘Ž‘‰›ǡ‹–‡†–ƒ–‡•Ǥ the likelihood of a feature at a given time based on data identification She obtained both bachelor and master †‡‰”‡‡• ‹ ‡…Šƒ‹…ƒŽ ‰‹‡‡”‹‰ gathered in previous steps. On the other hand, a data and building a ˆ”‘‘”–Š™‡•–‡”‹˜‡”•‹–›ǡ‹–‡† States. When she was at Northwestern, smoothing method estimates the likelihood that a feature more intuitive she did computational robotic research ™‹–Š”‘ˆǤ‘††—”’Š‡›—†‡” –Š‡ was encountered based on data gathered for an entire prosthetic hand.” ƒ„‘”ƒ–‘”›ˆ‘”–‡ŽŽ‹‰‡–‡…Šƒ‹…ƒŽ Systems. Find out more about time window. As illustrated ‘‡ ‡‰ „› ˜‹•‹–‹‰ Š‡” …‹‡–‹ϐ‹… in Figure 3, because this surface feature detection ƒŽƒ›•‹ƒ’”‘ϐ‹Ž‡ƒ–Š––’ǣȀȀ™™™Ǥ•…‹‡–‹ϐ‹…ƒŽƒ›•‹ƒǤ algorithm considers data for the entire time window, com/members/yokepeng/ or her research web page ‹–‹•Ž‡•• •‡•‹–‹˜‡ –‘ †ƒ–ƒ ‘‹•‡Ǥ Š‹•ƒŽ‰‘”‹–Š‹• at https://sites.google.com/site/ypleong90/ a smoothing-based algorithm which has a good performance against measurement noise. COMMENTS/SUGGESTIONS? Enabling a robot to touch is essential for automated Do you have any comments on any of the –ƒ…–‹Ž‡ ‘„Œ‡…– ‹†‡–‹ϐ‹…ƒ–‹‘ ƒ† „—‹Ž†‹‰ ƒ ‘”‡ articles in this issue? Or perhaps you have intuitive prosthetic hand. A bottom-up approach to suggestions for us? tactile sensing is to categorise an object based on —‹“—‡ ˆ‡ƒ–—”‡• ˆ‘—†‘–Š‡‘„Œ‡…–ǯ••—”ˆƒ…‡ǤŠ—• Write to us and we will publish your letter with far, I have developed a new approach for detecting and our editors’ response in the next issue. Email localising a surface feature based on the kinematics and us at: †›ƒ‹…•‘ˆƒ”‘„‘–‹…ϐ‹‰‡”ǤŠ‹•™‘”‹•‘Ž›ƒ–‹’‘ˆ magazine@scientificmalaysian.com –Š‡‹…‡„‡”‰‹ˆ—ŽŽ›†‡ϐ‹‹‰–Š‡”‘„‘–‹…•‡•‡‘ˆ–‘—…ŠǤ However, it provides a feasible preliminary framework for this effort. If you are interested in reading more about the technical details of my research, you may