Journal of Automation, Mobile Robotics and Intelligent Systems, vol. 17, no. 2 (2023) by Ł-PIAP

WWW.JAMRIS.ORG

Indexed in SCOPUS

pISSN 1897-8649 (PRINT)/eISSN 2080-2145 (ONLINE)

VOLUME 17, N° 2, 2023

Journal of Automation, Mobile Robotics and Intelligent Systems A peer-reviewed quarterly focusing on new achievements in the following fields: • automation • systems and control • autonomous systems • multiagent systems • decision-making and decision support • • robotics • mechatronics • data sciences • new computing paradigms • Editor-in-Chief

Typesetting

Janusz Kacprzyk (Polish Academy of Sciences, Łukasiewicz-PIAP, Poland)

SCIENDO, www.sciendo.com

Advisory Board

Webmaster

Dimitar Filev (Research & Advenced Engineering, Ford Motor Company, USA) Kaoru Hirota (Tokyo Institute of Technology, Japan) Witold Pedrycz (ECERF, University of Alberta, Canada)

TOMP, www.tomp.pl

Editorial Office

Co-Editors Roman Szewczyk (Łukasiewicz-PIAP, Warsaw University of Technology, Poland) Oscar Castillo (Tijuana Institute of Technology, Mexico) Marek Zaremba (University of Quebec, Canada)

Executive Editor Katarzyna Rzeplinska-Rykała, e-mail: office@jamris.org (Łukasiewicz-PIAP, Poland)

ŁUKASIEWICZ Research Network – Industrial Research Institute for Automation and Measurements PIAP Al. Jerozolimskie 202, 02-486 Warsaw, Poland (www.jamris.org) tel. +48-22-8740109, e-mail: office@jamris.org The reference version of the journal is e-version. Printed in 100 copies. Articles are reviewed, excluding advertisements and descriptions of products. Papers published currently are available for non-commercial use under the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 (CC BY-NC-ND 4.0) license. Details are available at: https://www.jamris.org/index.php/JAMRIS/ LicenseToPublish

Associate Editor Piotr Skrzypczyński (Poznań University of Technology, Poland)

Statistical Editor Małgorzata Kaliczyńska (Łukasiewicz-PIAP, Poland)

Editorial Board: Chairman – Janusz Kacprzyk (Polish Academy of Sciences, Łukasiewicz-PIAP, Poland) Plamen Angelov (Lancaster University, UK) Adam Borkowski (Polish Academy of Sciences, Poland) Wolfgang Borutzky (Fachhochschule Bonn-Rhein-Sieg, Germany) Bice Cavallo (University of Naples Federico II, Italy) Chin Chen Chang (Feng Chia University, Taiwan) Jorge Manuel Miranda Dias (University of Coimbra, Portugal) Andries Engelbrecht ( University of Stellenbosch, Republic of South Africa) Pablo Estévez (University of Chile) Bogdan Gabrys (Bournemouth University, UK) Fernando Gomide (University of Campinas, Brazil) Aboul Ella Hassanien (Cairo University, Egypt) Joachim Hertzberg (Osnabrück University, Germany) Tadeusz Kaczorek (Białystok University of Technology, Poland) Nikola Kasabov (Auckland University of Technology, New Zealand) Marian P. Kaźmierkowski (Warsaw University of Technology, Poland) Laszlo T. Kóczy (Szechenyi Istvan University, Gyor and Budapest University of Technology and Economics, Hungary) Józef Korbicz (University of Zielona Góra, Poland) Eckart Kramer (Fachhochschule Eberswalde, Germany) Rudolf Kruse (Otto-von-Guericke-Universität, Germany) Ching-Teng Lin (National Chiao-Tung University, Taiwan) Piotr Kulczycki (AGH University of Science and Technology, Poland) Andrew Kusiak (University of Iowa, USA) Mark Last (Ben-Gurion University, Israel) Anthony Maciejewski (Colorado State University, USA)

Krzysztof Malinowski (Warsaw University of Technology, Poland) Andrzej Masłowski (Warsaw University of Technology, Poland) Patricia Melin (Tijuana Institute of Technology, Mexico) Fazel Naghdy (University of Wollongong, Australia) Zbigniew Nahorski (Polish Academy of Sciences, Poland) Nadia Nedjah (State University of Rio de Janeiro, Brazil) Dmitry A. Novikov (Institute of Control Sciences, Russian Academy of Sciences, Russia) Duc Truong Pham (Birmingham University, UK) Lech Polkowski (University of Warmia and Mazury, Poland) Alain Pruski (University of Metz, France) Rita Ribeiro (UNINOVA, Instituto de Desenvolvimento de Novas Tecnologias, Portugal) Imre Rudas (Óbuda University, Hungary) Leszek Rutkowski (Czestochowa University of Technology, Poland) Alessandro Saffiotti (Örebro University, Sweden) Klaus Schilling (Julius-Maximilians-University Wuerzburg, Germany) Vassil Sgurev (Bulgarian Academy of Sciences, Department of Intelligent Systems, Bulgaria) Helena Szczerbicka (Leibniz Universität, Germany) Ryszard Tadeusiewicz (AGH University of Science and Technology, Poland) Stanisław Tarasiewicz (University of Laval, Canada) Piotr Tatjewski (Warsaw University of Technology, Poland) Rene Wamkeue (University of Quebec, Canada) Janusz Zalewski (Florida Gulf Coast University, USA) Teresa Zielińska (Warsaw University of Technology, Poland)

Publisher:

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17, N˚2, 2023 DOI: 10.14313/JAMRIS/2-2023

Contents 3

Quantifying Swarm Resilience with Simulated Exploration of Maze‐Like Environments Megan Emmons, Anthony A. Maciejewski DOI: 10.14313/JAMRIS/2‐2023/10 12

Update on the Study of Alzheimer’s Disease Through Artificial Intelligence Techniques Eduardo Garea‑Llano DOI: 10.14313/JAMRIS/2‐2023/15

Cuban Consumer Price Index Forecasting Through Transformer with Attention Reynaldo Rosado, Orlando G. Toledano‑López, Hector R. González, Aldis J. Abreu, Yanio Hernandez DOI: 10.14313/JAMRIS/2‐2023/11

Analyzis of Rehabilitation Systems in Regards to Requirements Towards Remote Home Rehabilitation Devices Piotr Falkowski, Cezary Rzymkowski, Zbigniew Pilat DOI: 10.14313/JAMRIS/2‐2023/16

Heavy Moving Average Distances in Sales Forecasting Maricruz Olazabal‑Lugo, Luis F. Espinoza‑Audelo, Ernesto León‑Castro, Luis A. Perez‑Arellano, Fabio Blanco‑Mesa DOI: 10.14313/JAMRIS/2‐2023/12

New Model of Photovoltaic System Adapted by a Digital MPPT Control and Radiation Predictions Using Deep Learning in Morocco Agricultural Sector Amal Zouhri, Mostafa El Mallahi DOI: 10.14313/JAMRIS/2‐2023/17

A Compact DQN Model for Mobile Agents with Collision Avoidance Mariusz Kamola DOI: 10.14313/JAMRIS/2‐2023/13 36

Application of the Spherical Fuzzy DEMATEL Model for Assessing the Drone Apps Issues Mamta Pandey, Ratnesh Litoriya, Prateek Pandey DOI: 10.14313/JAMRIS/2‐2023/14

VOLUME 17, N∘ 2 2023 Journal of Automation, Mobile Robotics and Intelligent Systems

QUANTIFYING SWARM RESILIENCE WITH SIMULATED EXPLORATION OF MAZE‐LIKE ENVIRONMENTS Submitted: 11th November 2021; accepted: 24th March 2022

Megan Emmons, Anthony A. Maciejewski DOI: 10.14313/JAMRIS/2‐2023/10 Abstract: Artificial swarms have the potential to provide robust, efficient solutions for a broad range of applications from assisting search and rescue operations to exploring remote planets. However, many fundamental obstacles still need to be overcome to bridge the gap between theory and application. In this characterization work, we demonstrate how a human rescuer can leverage mini‐ mal local observations of emergent swarm behavior to locate a lone survivor in a maze‐like environment. The simulated robots and rescuer have limited sensing and no communication capabilities to model a worst‐case scenario. We then explore the impact of fundamental properties at the individual robot level on the utility of the emergent behavior to direct swarm design choices. We further demonstrate the relative robustness of the simulated robotic swarm by quantifying how reasonable probabilistic failure affects the rescue time in a complex environment. These results are compared to the theo‐ retical performance of a single wall‐following robot to further demonstrate the potential benefits of utilizing robotic swarms for rescue operations. Keywords: Swarm robotics

1. Introduction Swarm robotics is a relatively new domain of research that takes inspiration from cooperative bio‐ logical systems such as locking birds, ant colonies, and schools of ish. Like their biological counterparts, robots in a swarm are governed by local rules, but fre‐ quent interactions with other robots and the environ‐ ment generate a more complex, emergent behavior. Sophisticated foraging strategies and complex colony construction by ants reveal just a few of the poten‐ tial advantages of emergent behavior because these actions are accomplished in a distributed and robust manner. Similar emergent behaviors in arti icial sys‐ tems will be extremely useful in many exploration tasks, particularly in harsh environments with a high probability of robot failure. Disaster scenarios are an especially relevant application domain with an increasing rise in occurrence and economic impact [1] and, currently, no viable robotic solutions. Despite the many important applications and advancements in theory, research in robotic swarms has not yet matured to the point of reliable, real‐world deployment. Two fundamental hurdles between the

potential and reality of swarms are (1) determining what collective behavior will emerge from the swarm and (2) identifying the in luence of local parameters. These challenges are not decoupled. Each robot within the swarm may be equipped with a variety of sen‐ sors and control strategies that can be considered as parameters. Swarms are necessarily composed of a large number of agents [2, 3] so the choice of param‐ eters is immediately scaled by the size of the swarm, which imposes some minor dif iculties. The true dif i‐ culty comes from the collective behavior that emerges after the robots interact with each other as well as the environment. Robot interactions propagate aspects of the local parameters, but there is no closed‐form method for determining how individual behaviors will affect the emergent behavior. Physical implementations would best reveal the full emergent behavior. Swarm test platforms are sur‐ facing and offer improved research development, but current realizations are restricted to table‐top envi‐ ronments, like the kilobot [4] or Zooid [5], often require overhead vision systems for robot position‐ ing such as in [6], and are necessarily constrained by the physical number of robots. For example, Georgia Tech’s impressive Robotarium offers a remotely acces‐ sible testbed for swarm algorithms but only includes 25 physical robots [7]. Beyond environment and size limitations, the initial choices of robot physicality already constrains the range of potential emergent behaviors the swarm can exhibit [8]. Simulation there‐ fore remains a very necessary design step because it can be used to investigate the baseline impact of local parameters on collective properties. The investigation can inform choices for the base physicality of individ‐ ual robots within the swarm and is less constrained by swarm size and environmental constraints. Arguably the most fundamental local parameters dictate how a robot moves and interacts with the envi‐ ronment. Speci ically, a robot’s speed, motion abil‐ ity, and sensing range control how the robot will move through an environment even in the presence of more sophisticated trajectory control like [9]. The effect of these parameters will be ampli ied in a swarm and impact properties of the emergent behav‐ ior. A human observing the swarm will likely not be able to identify subtle changes in the emergent behavior, but group studies where human participants are asked to classify swarm behaviors by observ‐ ing simulated results have shown that people can

2023 © Emmons and Maciejewski. This is an open access article licensed under the Creative Commons Attribution-Attribution 4.0 International (CC BY 4.0) (http://creativecommons.org/licenses/by-nc-nd/4.0/)

Journal of Automation, Mobile Robotics and Intelligent Systems

recognize general patterns [10–12]. Robot density, velocity, and relative cohesion were all notable prop‐ erties that helped human participants classify simu‐ lated emergent behavior. In this foundational study, we consider a human rescuer who can only rely on local observations of the emergent swarm velocity to navigate a simulated dis‐ aster environment in an attempt to locate a lone sur‐ vivor. Robots within the swarm have no communica‐ tion or localization ability. This model allows the in lu‐ ence of fundamental robot features related to motion and sensing to be identi ied. Variations in local swarm parameters are evaluated qualitatively, by observing the resulting motion and area coverage, as well as quantitatively, by the impact of the parameter varia‐ tion on the rescuer’s ability to successfully locate the lone survivor. From this work, we begin to establish important swarm characterization. The robustness of swarms in terms of parameter variation, environmen‐ tal features, and robot failure is quanti ied. We also demonstrate the value of employing swarms in disas‐ ter scenarios to motivate additional development. We irst place our work in the grander scope of swarm robotics by presenting related work in Sec‐ tion 2 before presenting the simulation framework in Section 3. Section 4 presents a summary of results from our investigation of emergent behavior. We also discuss important observations from our research in Section 5 before summarizing our work in Section 6.

2. Related Work Our work is motivated by the absence of robot solutions for disaster scenarios including mine res‐ cue [13] and building collapse [14]. Ongoing DARPA challenges and reports [15] indicate a need for more reliable physical robot implementations. There is promising progress in this domain for single robots [16, 17] and improved path planning to mit‐ igate risk [18]. Swarms can potentially leverage these advancements and further reliability efforts by utilizing a large number of robots for increased robustness. Experiments have shown that emergent swarm patterns can serve as an information storage mechanism [19, 20] which supports the robustness paradigm. Yet, to the best of our knowledge, no work has actually quanti ied the resilience of a swarm. In our investigation, we explore a minimalist swarm to establish a baseline performance to which additional functionality can then be compared. Our approach considers the interests of potential swarm users as indictated in an important study conducted by Carrillo‐Zapta et al. where target users, including ire ighters, identi ied areas where swarms can sup‐ port current activities [21]. Study participants consid‐ ered swarms bene icial for gathering information and supporting communication but emphasized that an actual person should remain ‘in the loop’ for decision making. Decision‐making by the human can be informed by observations of the emergent swarm behavior. As demonstrated by works in swarm expressivity, human 4

VOLUME 17,

N∘ 2

2023

participants were able to classify swarm behavior by ‘unfocusing’ and instead looking at collective behav‐ iors such as robot spacing, velocity, and relative cohe‐ sion [10–12]. We maintain our baseline approach by having our simulated human rescuer move using only the observed swarm velocity. The results from our work focus on maintaining a minimal approach, which can represent a worst‐ case scenario in robot functionality but also serve as a baseline. More sophisticated algorithms can then be compared to these results for design and applica‐ tion purposes. For example, the potential bene its of improved odometry [22], environment analysis [23], velocity control [24], shape formation [8], and local communication for risk mitigation [25] can be evalu‐ ated.

3. Simulation Framework 3.1. Swarm Behavior Model We model a minimalist robot moving in an unknown, two‐dimensional environment. The robot is unable to communicate, has no means for localization, and relies on limited sensing. This model allows us to characterize baseline performance for the swarm but also represents a worst‐case scenario for physical robot deployments. The swarm is composed of 𝑀 such robots where 𝑀 is a user‐de ined parameter allowing the signi i‐ cance of swarm size to be explored. Robots are initially distributed randomly within a user‐de ined radius around the start_center. The position of robot 𝑖 at iteration 𝑘 is denoted as x(𝑖, 𝑘). Using discrete‐time steps, robots attempt to move in a straight line with general desired velocity v𝑑 (𝑘) = 𝑠𝑠 [cos(𝛼𝑘 ), sin(𝛼𝑘 )],

(1)

with 𝑠𝑠 representing the maximum robot speed, while also striving to avoid collisions. Each robot is initial‐ ized with a uniformly distributed starting angle 𝛼1 . Subsequent trajectory angles are a combination of the previous 𝑛 trajectory angles and a Gaussian noise cal‐ culated as 𝛼𝑘 =

1 𝑘−1 ∑ 𝛼 +𝛾 𝑛 𝑎=𝑘−𝑛 𝑎

if 𝑛 < 𝑘

𝛼1

otherwise

(2)

where 𝛾 is a normally distributed random variable centered at zero with user‐de ined standard deviation, 𝜎. We explore the impact of heading control on swarm resiliency by varying 𝜎. The size of the moving win‐ dow, 𝑛, is also varied to determine the importance of memory on the emergent swarm behavior. Once a desired velocity is determined, we ensure the robot does not collide with any objects in its sens‐ ing radius, 𝑟𝑠 , by using a generalized forcing model. The actual velocity for robot 𝑖 at time step 𝑘 is calcu‐ lated as 𝑊

v𝑠 (𝑖, 𝑘) = v𝑑 (𝑖, 𝑘) +

𝑀

f𝑤 (𝑤) + 𝑤=1

f𝑠 (𝑖, 𝑗) (3) 𝑗=1,𝑗≠𝑖

Journal of Automation, Mobile Robotics and Intelligent Systems

where f𝑤 and f𝑠 denote the effect of the environment and other robots on robot 𝑖, respectively. We next de ine the repulsive wall force for each of the 𝑊 walls creating our environment as 𝐴𝑠 𝑒 (𝑟𝑠 −𝑑(𝑤))/𝐵𝑠 n̂ if 𝑑(𝑤) ≤ 𝑟𝑠 f𝑤 (𝑤) = 0 otherwise

(4)

(5)

(6)

We maintain 𝐶𝑠 and 𝐹𝑠 as exploration parameters and k̂ as a unit vector in the direction of the line of impact for the collision. Here we note, (5) also serves as a dis‐ persion model to ensure robots are generally directed further into new regions of the environment. 3.2. Rescuer Model Maintaining our base assumptions of a disaster scenario, the human rescuer likewise has a limited visibility range and cannot modify the behavior of the robots. The rescuer can travel at a maximum speed of 𝑠𝑚𝑎𝑥 , has an observation radius of 𝑟ℎ , and position xℎ (𝑘) at time step 𝑘. During time step 𝑘, the rescuer calculates their desired velocity as 𝑊

v𝑑,ℎ (𝑘) = v𝑜𝑏𝑠 (𝑘) +

𝑓ℎ𝑤 (𝑤)

(7)

𝑤=1

where v𝑜𝑏𝑠 (𝑘) is a velocity inferred from local obser‐ vations of the swarm behavior. The second term in (7) prevents the rescuer from colliding with the wall, analogous to (4) but with de ined scalars 𝐴ℎ = 0.1𝑠𝑚𝑎𝑥

(8)

2𝑠𝑚𝑎𝑥 𝐵ℎ = 2𝑠𝑚𝑎𝑥 / log . 𝐴ℎ

(9)

By the formulation of (7), the rescuer’s desired velocity is primarily determined using local observa‐ tions of the swarm. We implement a simple low model for the rescuer to explore how changes in the emer‐ gent swarm behavior in luence the utility of observ‐ able swarm properties. More speci ically, v𝑜𝑏𝑠 (𝑘) = 𝑠𝑚𝑎𝑥

v𝑒 (𝑘) ‖v𝑒 (𝑘)‖

2023

𝑀

v𝑒 (𝑘) =

v𝑠 (𝑖, 𝑘)𝑡(𝑖)

(11)

𝑖=1

𝑡(𝑖) =

1 if ‖x(𝑖, 𝑘) − xℎ (𝑘)‖ ≤ 𝑟ℎ 0 otherwise

(10)

(12)

is an indicator function ensuring the rescuer only uses swarm velocities which are observable. Finally, the rescuer cannot exceed their maximum speed, 𝑠𝑚𝑎𝑥 , nor will they act if the inferred veloc‐ ity from swarm observation is below some threshold magnitude, 𝑠𝑚𝑖𝑛 , so the actual implemented velocity at iteration 𝑘, denoted as vℎ (𝑘), becomes 𝑠𝑚𝑎𝑥

where 𝑑(𝑖, 𝑗) is now the scalar distance between robot 𝑖 and robot 𝑗, calculated as 𝑑(𝑖, 𝑗) = ‖x(𝑖, 𝑘) − x(𝑗, 𝑘)‖.

where

N∘ 2

and

with n̂ as a unit vector pointing into the environment, perpendicular to the wall, and 𝑑(𝑤) as the minimum distance to the wall segment 𝑤. The choice of coef‐ icients 𝐴𝑠 and 𝐵𝑠 dictates how strongly robots will be repelled from obstacles. Robots in the swarm also need to avoid collisions with each other, so f𝑠 is a similarly repulsive force imposed on robot 𝑖 by all other robots. Speci ically, we de ine 𝐶𝑠 𝑒 (𝑟𝑠 −𝑑(𝑖,𝑗))/𝐹𝑠 k̂ if 𝑑(𝑖, 𝑗) ≤ 𝑟𝑠 f𝑠 (𝑖, 𝑗) = 0 otherwise

VOLUME 17,

v𝑑,ℎ (𝑘) ‖v𝑑,ℎ (𝑘)‖

vℎ (𝑘) = 0 v𝑑,ℎ (𝑘)

if ‖v𝑑,ℎ (𝑘)‖ > 𝑠𝑚𝑎𝑥 if ‖v𝑑,ℎ (𝑘)‖ < 𝑠𝑚𝑖𝑛 otherwise

(13)

with an additional limit of 𝜋/4 on the maximum angle change the rescuer will experience between time steps unless they are near a wall boundary. Equation (11) leverages the pattern recognition ability of people as demonstrated in studies like those by Walker et al. [10]. It also introduces an important but challenging aspect of autonomous exploration, which is determining the ‘optimal’ amount of time to wait for information before acting. Swarm interactions encode important environ‐ mental information, like the presence of openings or obstacles [20]. The encoding process changes prop‐ erties of the emergent behavior but takes an amount of time depending on the robot density and distance to obstacles. We introduced two simple parameters to explore when the rescuer should begin moving in the environment using local observations of the swarm. The 𝑠𝑚𝑖𝑛 parameter speci ies a minimum magnitude of observed velocity the rescuer must observe before acting and represents a coarse consensus within the swarm about a desired direction. The second parame‐ ter, 𝑃𝑇, is a speci ied pause time wherein the rescuer allows the robots time to interact before observing the emergent velocity. 3.3. Environment Description Our goal in this work is to initiate a baseline quanti ication of swarm parameters, so we de ined a starting environment, explored the simulated per‐ formance, then systematically added features to gain insight into how different parameters in luenced the emergent behavior. Figure 1 presents the four main environments discussed in this work. All of the envi‐ ronments are built around a central, square room that is four units wide. The size of the environment serves as a scaling factor that directs the choice of other exploratory parameters like maximum speed and sen‐ sor radius. The simulated robot swarm and rescuer are initially positioned centrally in this room and cannot pass through environment boundaries. Robots con‐ tinue to move about the environment according to 5

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

Table 1. Summary of simulation parameters Param. 𝑀 𝑠𝑠 𝑛 𝜎 𝑟𝑠 𝐴𝑠 , 𝐵𝑠 𝐶𝑠 , 𝐹𝑠

Figure 1. A lone survivor is placed at the end of the top‐left‐most hallway for each of the four simulated disaster environments, denoted by the yellow circle. The rescuer begins in the center of the environment but has limited sensing, as indicated by the red ellipse (3) unless they experience a failure or are within a sensor radius of the survivor’s location, at which point the robot is removed from the simulation. The robots themselves are not attempting to locate the survivor, but the collective low created by having robots no longer explore once they reach the survivor directs the rescuer toward the survivor, as the remaining work shows. Removing robots from the simulation can be physically recreated by having the survivor turn the robots off or using another equivalent strategy.

4. Results 4.1. Preliminary Parameter Characterization For the initial swarm parameter evaluation, we focused on qualitatively assessing the reasonableness of the emergent behavior by animating the swarm exploration process and quantitatively recording the percentage of area explored within a ixed time. We initially simulated the robots in Environment 1. As indicated in Figure 1, the survivor is placed at the end of the west hallway. Robots stop when they are in the vicinity of the survivor. After identifying a reasonable initial operating range for all parameters, we pro‐ ceeded to systematically vary one parameter at a time and evaluate the resulting swarm performance. The experiments con irmed several intuitive correlations: ‐ Increasing the number of robots increased the area explored, but with diminishing rate of return, so that a very large number of robots was needed to generate even a small increase in exploration area ‐ Increased robot speed similarly increased area explored, but again with diminishing returns ‐ Too large a speed resulted in unnatural motion, disproportionate to environment size ‐ The number of past movements stored had a neg‐ ligible effect on the area explored when there were robot interactions ‐ More area was explored when the robots’ motion was less random 6

Signi icance Number of robots in swarm Maximum robot speed Number of stored movements Std. dev. for Gaussian noise Robot’s sensing radius Force coeff. for obstacles Force coeff. for neighbors

Base Value 300 0.4 1 0.1 0.8 1, 0.5 1, 0.5

‐ To represent realistic motion, a small value of 𝜎 should be incorporated into the robots’ motion ‐ The presence of other robots acted to restrict spo‐ radic motion due to collision avoidance ‐ Larger sensor radius generally improved perfor‐ mance From this preliminary set of experiments, we de ined base values for the swarm parameters, summarized in Table 1, that created a relatively luid and ef icient dispersion in the environment. 4.2. Resilience to Rescuer Variation Using the baseline swarm parameter values sum‐ marized in Table 1, we ran 100 different swarm dispersion scenarios for the environments in Fig‐ ure 1. A maximum exploration time of 3000 steps was imposed on all scenarios to ensure the simulation time remained tractable. We then simulated a rescuer who navigates the unknown environment according to (13) in an effort to reach the lone survivor. Each res‐ cuer parameter – threshold speed (𝑠𝑚𝑖𝑛 ), pause time (𝑃𝑇), sensor radius (𝑟ℎ ), and maximum speed (𝑠𝑚𝑎𝑥 ) – was systematically varied to explore the resiliency of the swarm and resulting rescue strategy with respect to variations in the fundamental rescuer behavior. Our priorities in evaluating the rescuer performance align with real‐world disaster scenarios: ensuring the rescuer successfully locates the survivor as reliably as possible, minimizing the danger to the rescuer by reducing the number of steps they take in a poten‐ tially dangerous environment, and reducing the time it takes for the rescuer to locate the survivor. Systematically varying each rescuer parameter revealed fundamental relationships between the properties of a successful rescue and the local emergent behavior. Variation in 𝑠𝑚𝑖𝑛 had negligible impact on our evaluation criteria, but the rescuer often required fewer steps to locate the lone survivor when the 𝑃𝑇 was irst increased. The median rescue times for each environment and a range of 𝑃𝑇s are plotted in Figure 2 along with a itted curve to help visualize the overall trend. Pause times around 300 were generally most bene icial for reducing the number of steps the rescuer needed to locate the survivor, independent of environment. Further increases to the 𝑃𝑇 did not notably reduce the number of steps and instead only increased the rescue time. Decreasing the number of rescuer steps is partially correlated with reduced danger to the rescuer, an important priority in disaster scenarios, but waiting

Importance of Pause Time in Reducing Rescuer Steps

800

Environment 1

700

Environment 2

600

Environment 3

500 400 300 200 100 0

200

400

600

800

1000

VOLUME 17,

Median Steps Rescuer Takes to Reach Survivor

Journal of Automation, Mobile Robotics and Intelligent Systems

Environment 1

2500

Environment 2 Environment 3

2000 1500 1000 500 0

0.2

0.4

0.6

0.8

Maximum Rescuer Speed (smax)

Figure 3. The rescuer often requires the fewest number of steps to locate the survivor when they have the same maximum speed as the robots. Here the median rescuer step count is shown when robots have a maximum speed of 0.4. Travelling faster than the swarm does not offer improved performance but indicates a resiliency to variations in rescuer speed Influence of Swarm Size on Rescue Time

3000

Average Simulation Time for Rescue

for the robots to do preliminary exploration also adds time to the overall rescue, which is undesirable. The trade‐off between acting early or waiting for more information is not new, but disaster scenarios add the challenge of a potentially dynamic environment where uncertainty is never fully resolved. Fortunately, rescuers are highly trained to make decisions based on the speci ic situation, so the results of Figure 2 can be used to primarily inform real‐world rescue strategies. The informative role of the swarm explo‐ ration also aligns with the preferred swarm inter‐ action expressed by participants in the survey by Carrillo‐Zapata et al. [21]. Speed is a similarly important parameter to ensure the rescuer fully leverages information from the swarm. Figure 3 shows the median time required to locate the survivor as a function of rescuer speed for the irst three environments from Figure 1. The survivor is located a speci ic distance away from the center of the environment where the rescuer is ini‐ tially positioned so, theoretically, a faster speed would result in decreasing the lower bound on rescue time. We see that the bene its of travelling faster are initially present in Figure 3 but speeds faster than the swarm speed of 0.4 do not signi icantly improve performance in any of the environments. While the utility of the swarm was resilient to variations in rescuer parameters, the rescuer gen‐ erally required fewer steps to locate the survivor if they implemented a suf icient pause time (𝑃𝑇) and a reasonable speed (𝑠𝑚𝑎𝑥 ). Another important param‐ eter is the number of robots present in the swarm to ensure an informative behavior emerges. Figure 4 demonstrates the effect of increasing the number of robots in Environment 4 on the average rescue time. A suf icient number of robots are clearly needed – the rescuer never located the survivor when only one robot was present, and only found the survivor in

2023

Importance of Rescuer Speed in Reducing Steps

3000

Time Rescuer Pauses Before Initial Movement (PT)

Figure 2. The rescuer requires fewer steps to locate the lone survivor if they first pause and allow the robots to do some preliminary exploration. The median number of steps taken by the rescuer over 100 simulations shows that a 𝑃𝑇 of 300 was generally the most beneficial, but there is a range of comparable values indicating resilience to rescuer 𝑃𝑇

N∘ 2

2500

2000

1500

1000

500

100

200

300

400

500

600

Number of Robots in Swarm

Figure 4. The rescuer required less time on average once more than 300 robots were initially distributed because a sufficient number of robot interactions occurred for informative behaviors to emerge. Using more than 300 robots did not significantly reduce the rescue time, as demonstrated by the average rescue times for Environment 4 47% of the simulations when relying on 50 robots, which resulted in an average of 2400 time steps – but suf icient is dif icult to de ine in advance, partic‐ ularly when the environment itself is unknown, as is the case for most disaster scenarios. Fortunately, the rescue performance for our minimalist scenario is relatively robust with respect to variations in swarm size. Swarm sizes of 200 − 600 robots all resulted in very similar average rescue times as summarized in Figure 4 for Environment 4. 4.3. Resilience to Environment Hazards The harsh terrain of many disaster scenarios results in a high probability of failure for autonomous systems. We have shown that the utility of the emer‐ gent behavior is robust with respect to initial swarm 7

Influence of Failure on Swarm Resilience

Percentage of Successful Scenarios

100

1300 1200

1100 1000

900 800

700 600

500 0 0

0.2

0.4

0.6

0.8

400 1.4

1.2

Failure Value for Swarm Robots (pf)

Median Steps Rescuer Takes to Reach Survivor

Journal of Automation, Mobile Robotics and Intelligent Systems

-3

Figure 5. The median rescue time in Environment 4 (lower red curve) increased as robots experienced larger failure rates, but the rescuer still successfully located the survivor in the majority of scenarios (top blue curve) even when as few as 12 robots remained functional

size but, alternatively, these results can be extended to scenarios where a large number of robots fail during the exploration task. To more fully characterize the impact of robot failure on our hypothetical disaster rescue scenario, we introduce a failure parameter, 𝑝𝑓 , to our simulation model. Prior to moving, a robot will be assigned a uniformly generated random number 𝑅. The robot is removed from the simulation if 𝑅 ≤ 𝑝𝑓 to simulate a realistic, immobilizing failure. The lower curve in Figure 5 shows the median time required to locate the survivor in Environment 4 as 𝑝𝑓 was systematically increased. The top curve indi‐ cates how many of the 100 scenarios resulted in the rescuer successfully locating the survivor. For all 100 scenarios, 300 robots are initially dispersed, travelling at 0.4 units per time step, with a rescuer 𝑃𝑇 of 300 time steps. It is instructive to compare the performance of the swarm in the presence of catastrophic failure from Fig‐ ure 5 to the results for varying the initial swarm size in Figure 4. A 𝑝𝑓 of 0.0014 resulted in the rescuer taking a median of 639 steps. The swarm experienced a 75% failure rate over the course of the simulation, so 225 of the original 300 robots failed by the time the rescuer successfully located the survivor. Although there were only 75 robots operating in the environment near the end of the simulation, the surviving robot behaviors had still been in luenced by interactions with other robots pre‐failure. Information about the environment was distributed through the swarm by these inter‐ actions so that, even with mass failure, the rescuer could still leverage the swarm knowledge to locate the survivor. Interpolating the values from Figure 4, the median time for the rescuer would have been about 2000 steps if the swarm was initially composed of only 75 robots and there were no failures. Amazingly, the rescuer successfully reached the survivor as long as at least 12 of the original 300 robots were still present in the environment, which occurred in 58 of the 100 scenarios. 8

VOLUME 17,

N∘ 2

2023

To further put the swarm resilience into context, we consider the theoretical performance of a single robot that primarily operates using a wall‐following strategy. Environment 4 has a perimeter of 104 units, so a wall‐following robot would have an expected path length of 52 units, thus enforcing the same speed, we expect the wall‐follower to need 130 steps on average to locate the lone survivor. The wall‐following robot is also a singular entity, so the rescuer will only success‐ fully locate the survivor if the robot itself does not fail, but each move in a harsh environment adds a chance for failure. We model the probability, 𝑝𝑛 , of the wall‐ following robot successfully moving 𝑛 steps by using the exponential decay model 𝑝𝑛 = 𝑒 −𝑐𝑛

(14)

where 𝑐 is the probability of failure per unit step. Evaluating (14) with 𝑛 = 130, the wall‐following robot would theoretically need to achieve a 99.58% reliability per step to locate the lone survivor in Envi‐ ronment 4 in as many trials as our minimalist swarm, where 75% of the robots failed and the survivor was still successfully located. Even with 100% step reliability, a wall‐following strategy may simply be ineffective for the particu‐ lar environment of interest due to a physical lack of boundaries or extremely reduced visibility, which even limits the abilities of human rescuers [21]. We further explore the swarm’s resilience to environmen‐ tal variations by adding a room, or cavern, to one end of Environment 4 and placing the lone survivor at the center of this room. The room dimensions are such that neither the robots nor the rescuer can sense the survivor without losing contact with a wall. Figure 6 shows the new environment as well as the resulting rescuer trajectory for one illustrative simulation. The trajectory shows that the rescuer sometimes circles back on their previous path, which adds steps to the total that would not occur with a wall‐following algo‐ rithm; however, the trajectory also shows that the res‐ cuer avoided entering several unoccupied regions of the environment, instead spending extra time in areas that must necessarily be entered in order to reach the survivor. Our minimalist rescuer model only considers the velocities of robots within the rescuer’s sensing radius, but this simple algorithm leverages the emer‐ gent swarm behavior to reduce the likelihood of enter‐ ing unnecessary regions of the environment. Con‐ sider the rescuer’s decision making process when they approach the inal hallway junction before entering the large room where the survivor is located. Figure 7 zooms in on this region of the environment and shows the rescuer’s position as a red ‘x’ with the sensing area illustrated with the purple circle. At this point, the rescuer has been unable to detect the upper hallway. A wall‐following strategy would lead the rescuer into the dead‐end hallway, but several robots have already explored this region and are moving back up, creating a small repulsive wave that directs other robots as well as the rescuer away from the dead end in general.

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

the resiliency of the emergent swarm behavior to perturbations. Even with the minimalist model used, the rescuer successfully located the lone survivor in our cavern environment in 83 out of 100 simulations and required a median of 676 steps. The rescuer also frequently avoided entering unnecessary regions of the environment, thereby further reducing the potential hazards encountered in real disaster scenarios. No change in the swarm behavior was needed to account for a signi icant change in the environment, which further illustrates the swarm’s resiliency, this time with respect to environmental features. Figure 6. In the cavern environment, the survivor (yellow square) is positioned in the middle of a large room where limited sensing prevents both the rescuer and robots from seeing the survivor while maintaining contact with the wall. The survivor would therefore not be successfully located if the rescuer was alone, or reliant on a wall‐following robot with comparable sensing radius, but the dispersion of robots in the swarm directs the rescuer away from the wall and to the survivor. The rescuer also avoids entering unnecessary regions of the environment in this scenario

Figure 7. Some robots have already entered the left‐most branch of the cavern environment and encountered a dead end. Zooming in on the final junction of the cavern environment and showing the robot velocities as green arrows, the returning robots exert a pressure that helps direct the rescuer, who can only detect objects located within the large purple circle, into the correct hallway and away from the unnecessary environment segment

The velocity for each robot is shown as a green arrow. The repulsive force of the wall, combined with the observable swarm motion at the single time step, direct the rescuer away from a potentially hazardous region of the environment and up into the correct hallway. More sophisticated path‐planning algorithms would reduce the amount of rescuer back‐tracking, but our focus in this work was on quantifying

5. Discussion While we present speci ic parameter values in Section 4, we are by no means proposing these as ideal scalars for swarm exploration. Nor are we advocating a simplistic interaction between swarm and simulated rescuer. The real contribution of this work is establishing a baseline quanti ication of swarm resilience, demonstrating important relation‐ ships between robots and a minimalist human res‐ cuer, and illustrating the feasible bene its of leverag‐ ing emergent properties for important applications like locating survivors in disaster environments. Interactions are the primary force driving emer‐ gent swarm behavior, so a suf icient number of robots are necessary to ensure that collective properties evolve. While this statement is partially intuitive and also supported by Figure 4, the in luence of swarm size on individual robot properties is potentially less clear but an extremely important design consideration. We used values from Table 1 to govern the dispersion algorithm. The choice of coef icients was impressively resilient to variations, but testing minimum swarm sizes highlights the value of interactions for producing more complex behaviors. For example, during our preliminary investiga‐ tions, we found that an individal robot did not need to store past steps or implement sophisticated heading control for ef icient area coverage if other robots were in the sensing area. Neighboring robots in the swarm essentially serve as a motion memory and enforce more direct trajectories. When one robot moved, other robots could ill in the space to prevent the robot from undoing its move, which is the traditional role of ‘memory’. Similarly, an individual robot may have very random motion (large value of 𝜎 in our simulation) but the presence of other robots combined with a simple collision avoidance mechanism reduced the robot’s erratic trajectory compared to the robot’s trajectory in isolation. The collective behavior generally resulted in more ef icient local behaviors without any change to the individual robot parameters. Figure 8 qualitatively demonstrates the improved trajectory for a robot in a swarm with 𝑀 = 300 robots (red line) compared to a robot operating on its own (blue line) with the same values from Table 1. Robot interactions also contribute to the desired robustness frequently associated with swarms 9

Journal of Automation, Mobile Robotics and Intelligent Systems

Figure 8. A robot operating with the parameters of Table 1 with swarm size 𝑀 = 1 ineffectively navigates the environment (blue line) but increasing the swarm size improves exploration trajectories without any change to the local rules. The red line is an illustrative robot trajectory when 𝑀 = 300 because information is stored implicitly in the behaviors rather than explicitly with any single robot. Figure 5 demonstrates that the swarm could experience catastrophic failure but still retain suf icient information for the rescuer to leverage and successfully locate the lone survivor. Of the 58 scenarios with 𝑝𝑓 = 0.0014 where the rescuer successfully reached the survivor, the fewest number of robots still functioning in the environment was just 12, but the motion of those 12 robots had bene itted from the initial distribution of 300 robots. The surviving robots were generally guided along a more direct path to the survivor by other robots that had entered dead end hallways and were attempting to return before failing. The rescuer similarly bene‐ itted from the exploration of the failed robots before they were immobilized. One challenge from our imple‐ mentation is that the rescuer required a minimum observed velocity from the swarm before moving. Thus, in the event that all robots within the rescuer’s sensing radius failed, the rescuer was unable to move. In a real‐world disaster scenario, the swarm would most likely only inform the rescuer’s search strategy so that even in the extreme cases where no robots are in the rescuer’s sensing radius, the rescuer can still navigate the environment.

6. Conclusion The distributed nature of swarms offers an intu‐ itive promise of robustness and ef iciency that is espe‐ cially desirable for exploration‐based tasks in harsh environments like locating disaster survivors. In this work, we establish a baseline quanti ication of swarm robustness by irst evaluating the impact of funda‐ mental robot properties on the utility of the emer‐ gent behavior. A simulated human rescuer relies on the locally observable swarm velocity to navigate an unknown disaster environment while attempting to locate a lone survivor. We af irmed that a minimalist 10

VOLUME 17,

N∘ 2

2023

swarm governed by a simple collision avoidance algo‐ rithm had suf iciently complex interactions to encode important environmental information that the rescuer successfully leveraged. The utility of the swarm was robust with respect to variations in fundamental res‐ cuer parameters and changes to the environmental structure. More impressively, the rescuer still found the survivor when the swarm underwent catastrophic failure and lost up to 288 of the original 300 robots. Although simple, the robust success of a simulated rescuer locating a lone survivor af irms the bene its of robot swarms. Local observations of the emergent swarm behavior often reduced the number of regions the rescuer entered and even consistently directed the rescuer into empty spaces that would be inac‐ cessible if relying on limited‐sensing strategies like wall‐following. Properties of the swarm do not need to be optimized for the swarm to still be effective in a variety of environments. While our simulation established an important baseline, we believe more information can be extracted by a human observing the emergent behavior and the additional information will further improve the swarm performance in all interactive applications. AUTHORS Megan Emmons∗ – Colorado State University, Fort Collins, CO 80523, USA, e‐mail: mremmon@rams.colostate.edu. Anthony A. Maciejewski – Colorado State University, Fort Collins, CO 80523, USA, e‐mail: aam@colostate.edu. ∗

Corresponding author

References [1] M. Coronese, F. Lamperti, K. Keller, F. Chiaromonte, and A. Roventini. “Evidence for sharp increase in the economic damages of extreme natural disasters,” in Proceedings of the National Academy of Sciences, vol. 116, no. 43, 2019, pp. 21 450–21 455. [2] M. Brambilla, E. Ferrante, M. Birattari, and M. Dorigo. “Swarm robotics: A review from the swarm engineering perspective,” in Swarm Intelligence, vol. 7, no. 1, 2013, pp. 1–41. [3] E. Şahin. “Swarm robotics: From sources of inspi‐ ration to domains of application,” in International Workshop on Swarm Robotics. Springer, 2004, pp. 10–20. [4] G. Valentini, A. Antoun, M. Trabattoni, B. Wiandt, Y. Tamura, E. Hacquard, V. Trianni, and M. Dorigo. “Kilogrid: A novel experimental environment for the kilobot robot,” in Swarm Intelligence, vol. 12, 2018, pp. 245–266. [5] M. L. Goc, L. H. Kim, A. Parsaei, J.‐D. Fekete, P. Dragicevic, and S. Follmer. “Zooids: Building blocks for swarm user interfaces,” in Proceedings of the 29th Annual Symposium on User Interface

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

Software and Technology. Association for Com‐ puting Machinery, 2016, p. 97–109.

IEEE/RSJ International Conference on Intelligent Robots and Systems, 2020, pp. 2518–2525.

[6] J. Wiech and Z. Hendzel. “Overhead vision sys‐ tem for testing swarms and groups of wheeled robots,” in Journal of Automation, Mobile Robotics and Intelligent Systems, vol. 13, no. 2, Jul. 2019, pp. 15–22.

[17] E. Takane, K. Tadakuma, M. Watanabe, M. Konyo, and S. Tadokoro. “Design and control method of a planar omnidirectional crawler mechanism,” Journal of Mechanical Design, vol. 144, no. 1, July 2021.

[7] D. Pickem, P. Glotfelter, L. Wang, M. Mote, A. Ames, E. Feron, and M. Egerstedt. “The rob‐ otarium: A remotely accessible swarm robotics research testbed,” in IEEE International Conference on Robotics and Automation, 2017, pp. 1699–1706.

[18] X. Xiao, J. Dufek, and R. R. Murphy. “Robot risk‐ awareness by formal risk reasoning and plan‐ ning,” in IEEE Robotics and Automation Letters, vol. 5, no. 2, 2020, pp. 2856–2863.

[8] I. Buckley and M. Egerstedt. “In initesimal shape‐ similarity for characterization and control of bearing‐only multirobot formations,” in IEEE Transactions on Robotics, 2021, pp. 1–15. [9] R. Piotrowski, B. Maciąg, W. Makohoń, and K. Milewski. “Design of control algorithms for mobile robots in an environment with static and dynamic obstacles,” in Journal of Automation, Mobile Robotics and Intelligent Systems, vol. 13, no. 4, July 2019, pp. 22–30. [Online]. Available: https://www.jamris.org/index.php/JAMRIS/ar ticle/view/525 [10] P. Walker, M. Lewis, and K. Sycara. “Charac‐ terizing human perception of emergent swarm behaviors,” in IEEE International Conference on Systems, Man, and Cybernetics (SMC) 2016, 2016, pp. 002 436–002 441. [11] D. St‐Onge, F. Levillain, E. Zibetti, and G. Beltrame. “Collective expression: How robotic swarms convey information with group motion,” in Paladyn, Journal of Behavioral Robotics, vol. 10, no. 1, 2019, pp. 418–435. [12] M. Santos and M. Egerstedt. “From motions to emotions: Can the fundamental emotions be expressed in a robot swarm,” International Journal of Social Robotics, 2020. [13] R. R. Murphy, J. Kravitz, S. L. Stover, and R. Shoureshi. “Mobile robots in mine rescue and recovery,” in IEEE Robotics Automation Magazine, vol. 16, no. 2, 2009, pp. 91–103. [14] E. Ackerman. “Why robots can’t be counted on to ind survivors in the lorida building collapse,” Jul 2021. [Online]. Available: https://spectrum.iee e.org/why‐robots‐cant‐help‐find‐survivors‐in‐ the‐florida‐building‐collapse [15] J. Carlson, R. R. Murphy, and A. Nelson. “Follow‐ up analysis of mobile robot failures,” in IEEE International Conference on Robotics and Automation, vol. 5, 2004, pp. 4987–4994. [16] A. Bouman, M. F. Ginting, N. Alatur, M. Palieri, D. D. Fan, T. Touma, T. Pailevanian, S.‐K. Kim, K. Otsu, J. Burdick, and A. Akbar Agha‐ Mohammadi. “Autonomous spot: Long‐ range autonomous exploration of extreme environments with legged locomotion,” in

[19] M. T. Jack, S. Khuman, and K. Owa. “Spatio‐ temporal patterns act as computational mecha‐ nisms governing emergent behavior in robotic swarms,” International Journal of Swarm Intelligence and Evolutionary Computation, vol. 8, no. 1, 2019. [20] M. Emmons, A. A. Maciejewski, C. Anderson, and E. K. P. Chong. “Classifying environmental features from local observations of emergent swarm behavior,” in IEEE/CAA Journal of Automatica Sinica, vol. 7, no. 3, 2020, pp. 674–682. [21] D. Carrillo‐Zapata, E. Milner, J. Hird, G. Tzoumas, P. J. Vardanega, M. Sooriyabandara, M. Giuliani, A. F. T. Win ield, and S. Hauert. “Mutual shap‐ ing in swarm robotics: User studies in ire and rescue, storage organization, and bridge inspec‐ tion,” in Frontiers in Robotics and AI, vol. 7, no. 53, 2020. [22] M. Palieri, B. Morrell, A. Thakur, K. Ebadi, J. Nash, A. Chatterjee, C. Kanellakis, L. Carlone, C. Guarag‐ nella, and A. akbar Agha‐mohammadi. “Locus: A multi‐sensor lidar‐centric solution for high‐ precision odometry and 3d mapping in real‐ time,” in IEEE Robotics and Automation Letters, vol. 6, no. 2, 2021, pp. 421–428. [23] N. I. Giannoccaro and T. Nishida. “Analysis of the surrounding environment using an innova‐ tive algorithm based on lidar data on a modular mobile robot,” in Journal of Automation, Mobile Robotics and Intelligent Systems, vol. 14, no. 4, March 2021, pp. 25–34. [Online]. Available: http s://www.jamris.org/index.php/JAMRIS/articl e/view/574 [24] S. A. Kumar, J. Vanualailai, B. Sharma, and A. Prasad. “Velocity controllers for a swarm of unmanned aerial vehicles,” Journal of Industrial Information Integration, vol. 22, p. 100198, 2021. [25] E. R. Hunt, C. B. Cullen, and S. Hauert. “Value at Risk Strategies for Robot Swarms in Hazardous Environments,” in Unmanned Systems Technology XXIII, H. G. Nguyen, P. L. Muench, and B. K. Skibba, Eds., vol. 11758, International Society for Optics and Photonics. SPIE, 2021, pp. 158–177. [Online]. Available: https://doi.org/10.1117/12 .2585760

VOLUME 17, N∘ 2 2023 Journal of Automation, Mobile Robotics and Intelligent Systems

CUBAN CONSUMER PRICE INDEX FORECASTING THROUGH TRANSFORMER WITH ATTENTION Submitted: 10th January 2023; accepted: 1st August 2023

Reynaldo Rosado, Orlando G. Toledano‑López, Hector R. González, Aldis J. Abreu, Yanio Hernandez DOI: 10.14313/JAMRIS/2‐2023/11 Abstract: Recently, time series forecasting modelling in the Con‐ sumer Price Index (CPI) has attracted the attention of the scientific community. Several research projects have tackled the problem of CPI prediction for their countries using statistical learning, machine learning and deep neural networks. The most popular approach to CPI in several countries is the Autoregressive Integrated Mov‐ ing Average (ARIMA) due to the nature of the data. This paper addresses the Cuban CPI forecasting problem using Transformer with attention model over univariate dataset. The fine tuning of the lag parameter shows that Cuban CPI has better performance with small lag and that the best result was in 𝑝 = 1. Finally, the comparative results between ARIMA and our proposal show that the Transformer with attention has a very high performance despite having a small data set. Keywords: Consumer price index, Time series forecasting, Transformer with attention, ARIMA, LSTM.

1. Introduction The Consumer Price Index (CPI) is a macroeco‐ nomic indicator that aims is to measure the variations over time of the prices of goods and services, included in the family basket, which respond to the inal con‐ sumption expenditures of households. The CPI, one of the most popular indicators in the ield of social and economic statistics, is a general macroeconomic indi‐ cator and a reference for measuring in lation [24]. The main elements for the construction of the index are: having a representative basket of household expen‐ ditures and a weighting structure that de ines the importance of each of these products in the popula‐ tion’s consumption. The weighting assigned to each good or service determines the effect that the varia‐ tion in its price will have on the CPI [20]. In the case of Cuba, the weighting re lects the data obtained in the National Survey of Household Income and Expenditures (ENIGH), which was conducted between August 2009 and February 2010. The weights of goods and services are therefore based on the consumption expenditures that households have access to at that time. The goods and services that affect Cuba’s CPI are: 01 Food and non‐alcoholic beverages; 02 Alcoholic beverages and tobacco; 03 Clothing; 04 Housing services; 05 Furniture and household items; 06 Health; 07 12

Transportation; 08 Communications; 09 Recreation and culture; 10 Education; 11 Restaurants and hotels; 12 Miscellaneous personal care goods and services [5]. The Monthly Publication of the CPI from the National Of ice of Statistics and Information (ONEI), gives the average variation experienced by the prices of a basket of goods and services, representative of the consumption of the population in a given period. Approximately 33596 prices are collected monthly, in 8607 establishments, located in 18 municipalities throughout Cuba, the urban area of the head munici‐ palities of 14 provinces and 4 municipalities of Havana province, obtaining national coverage. This means that the index to be shown is only representative of the country; it does not exist at the level of regions or municipalities. The basket of goods and services includes 298 items that represent more than 90.0% of household expenditure. The data are published in the form of reports in pdf format, which makes it dif icult to process and analyze them because there is no integrated view of the database [18]. Banking, inancial and government authorities systematically monitor the behavior of this indicator as a measure of in lation. It is a reference for monetary policy decisions in all countries. Public administra‐ tions frequently analyze this indicator to evaluate issues such as retirement, unemployment, average wage, subsidies [9]. Due to the nature of the data and the frequency with which it is captured, the historical collection of data is considered to be univariate time series as a global metric CPI or multivariate where the goods and services are taken into account. In general, several approaches in the CPI fore‐ casting ield, model the problem as a univariate time series, concentrating only on the study of the global indicator. Approaching it as a multivariate problem, taking into account the variation of the prices of each goods or service included in the basket, does not work well since the global index is composed of the weighted aggregation of the prices of each products. The most widely used statistical method for fore‐ casting the CPI as a univariate time series problem has been the family of the Autoregressive Integrated Moving Average (ARIMA) [1, 6, 7, 15–17]. Recently, deep learning techniques for time series forecasting have improved the performance of CPI prediction. Recurrent Neural Networks (RNN) or Long Short‐ Term Memory (LSTM) architectures have the ability

2023 © Rosado et al. This is an open access article licensed under the Creative Commons Attribution-Attribution 4.0 International (CC BY 4.0) (http://creativecommons.org/licenses/by-nc-nd/4.0/)

Journal of Automation, Mobile Robotics and Intelligent Systems

to capture time dependence in data, while handling more than one output variable to estimate more than one time instant. Three examples that show good per‐ formance with simple LSTM [26] model and temporal data at different time intervals are from Mexico [8], Ecuador [19, 21] and Indonesia [13]. A considerable number of studies were done to analyse the CPI data in different countries. For CPI forecasting, ARIMA models have been widely employed due to the nature of this type of time series, which have very few data samples. The evaluation methodology contemplates the analysis of the season‐ ality of the time series and the evaluation and selection of the best forecasting model, following the exhaustive strategy of the walk forward validation. On the other hand, some works have focused on the use of deep neural network models. In this type of approach, the great challenge is to obtain generalized models with high accuracy on small data samples, as in the case of the CPI time series. The aim of this work is to develop a new dataset of Cuba’s monthly CPI and a respective time series forecasting model based in Attention mechanism that considers a mapping to a large data representation in encode‐decode architecture. In the deep learning proposal, a set of attributes must be represented with an adequate treatment of the non‐linearity in the time series relationship. On the other hand, the model should consider a model selection over grid search and a ine tuning of the parameters in the learning process. A comparative baseline study was developed taking in to account ARIMA, and our proposal based on Transformer with attention.

2. Basic Concepts and Notation in Time Series Forecasting A time series is denoted as a collection of val‐ ues of a given variable or set of variables ordered chronologically and sampled at constant time inter‐ vals. It is mathematically de ined as a set of random variables 𝑦𝑖,𝑡 , 𝑡 = 1, 2, … , 𝑛 that describe a physical phenomenon. Time series forecasting models predict future values of a target {𝑦𝑖,𝑡+𝑙 }, 𝑙 = 1, 2, … , 𝑚 for a given variable 𝑖 at the time 𝑡. In the simplest case the one‐step‐ahead forecasting model and univariate data, we can de ine the model as: 𝑗

𝑗

𝑦̂ 𝑡+1 = 𝑓(𝑦𝑡 , 𝑦𝑡−1 , 𝑦𝑡−2 , … , 𝑦𝑡−𝑘 , 𝑊𝑘 )

(1)

Where 𝑦̂ 𝑡+1 is the forecast for univariate data at 𝑗 𝑗 𝑗 𝑗 instant 𝑡 + 1, {𝑦𝑡 , 𝑦𝑡−1 , 𝑦𝑡−2 , … , 𝑦𝑡−𝑘 } are the feature representation space of data that consider a lag win‐ 𝑗 dow of size 𝑘, inally 𝑊𝑘 are the parameters in the model. For those problems in which there is more than one variable, spatially related and whose nature individually shows a temporal relationship, we say that the problem is a multivariate time series. Clas‐ sical statistical or machine learning models need to consider the univariate or multivariate problem dif‐ ferently, however, deep learning models can handle both with high accuracy [10].

VOLUME 17,

N∘ 2

2023

Time series are usually characterized by three components: trend, seasonality and residuals. In real world time series, and particularly in the CPI problem, seasonality can be affected by external agents such as the economic and inancial crisis, the prices of the main products in the world market and emerging sit‐ uations such as the COVID‐19 pandemic. There are many methods that can be used for time series forecasting and there is no seemingly best solu‐ tion for any type of problem. The choice of model should always depend on the data and the nature of the problem to be solved. Some models may be more robust against outliers, but have worse per‐ formance; the best choice depend on the use case or nature of the data. Given the diversity of time series problems across various domains, numerous neural network architectures choices have emerged that obtaining very good results. While in classi‐ cal statistical models such as autoregressive models (ARIMA), feature engineering is performed manually and often some parameters are optimized also consid‐ ering the domain knowledge, Deep Learning models learn features and dynamics directly from the data in an autonomous way, learning more complex patterns and capturing the non‐linearity in the data [2].

3. Deep Learning for Time Series Forecasting The most popular architecture in Deep Learning for Time Series Forecasting are the Recurrent Neural Networks (RNNs), Long Short‐Term Memory (LSTM), Gated Recurrent Unit (GRU) and Encoder‐Decoder Model with attention. In a recent review article [11], LSTM model for time series forecasting is discussed in more detail. The main contribution of this model to recurrent architectures such as RNNs [14, 25] is in the solution of the optimization problem, where classical activation functions tend to gradient vanish‐ ing in interactive propagation to capture long‐term dependence. The Gated Recurrent Unit (GRU) [3] is the newest generation of RNNs and is quite similar to an LSTM. The main difference between a GRU and an LSTM is that a GRU has gate, an update, and reset gate; while an LSTM has three gates: an input, a forget, and an output gate, which allow for changes in the state vector of a cell while capturing the long‐term temporal relationship. When the time series is small, GRU is suggested; on the other hand, if the series is large, it must be LSTM. GRU checks in each iteration and can be updated with short‐term information, however LSTM limits the change gradient in each iteration and in this way does not allow the past information to be completely discarded. This is why LSTM is mostly used for 9 long‐term dependency modeling. In [11] he states that there are no signi icant advantages with respect to the computation time of GRU over LSTM, although it has a smaller number of parameters in the cells. In RNN, LSTM and GRU each input corresponds to an output for the same time step. However in many real problems it is necessary to predict an output sequence given an input sequence of different length. 13

Journal of Automation, Mobile Robotics and Intelligent Systems

This scenario is called sequence to sequence mapping model [22], and is behind numerous commonly used applications like forecast a time vector [4, 27]. Atten‐ tion mechanism gives good results also in the presence of long or small input sequences, as limits cases, which are related to the encode mechanism that controls the size of the representation space. Attention mechanism has also the advantage of being more interpretable that other Deep Learning models, that are generally considered as black boxes since they do not have the ability to explain their outputs. Particularity, in case of CPI where the the data are composed of small sequences, the encode‐decoder attention model could be a very good contribution and uncharted area in the scienti ic literature.

4. Forecasting via Transformer For forecasting CPI, a Transformer‐based architec‐ ture with Self‐Attention mechanisms is applied. The proposal follows the basic fundamental design pro‐ posed in [23], with encoder‐decoder. However, the decoder is conceived in a hidden layer having as input ℎ𝑙𝑎𝑠𝑡 , which represents the output of the last hidden state of the encoder block, and as output a single neuron. Figure 1 represents the design of the proposed architecture based on Transformer. At the input to the model, for the feature representation space 𝑦 we apply a positional encoder layer to process the input to the encoder block. The residual connections of the above mentioned layer are used in the normalization layer. In the encoder block we de ine the number of heads in the Multi‐Head Attention models as 𝑛ℎ𝑒𝑎𝑑𝑠 = 10. 𝑚𝑡 𝑑 To obtain the results, 𝑓0 ∶ IR → IR is considered as an unknown function, where 𝑚 is the dimension of random variables. The forecasting capability of an ̂ estimator 𝑓̂ of 𝑓0 is measured via 𝔼𝐷(𝑓): 1 𝑗 𝑗 𝑗 𝑗 ̂ 𝑡𝑗 , 𝑦𝑡−1 |𝑦 − 𝑓(𝑦 , 𝑦𝑡−2 , … , 𝑦𝑡−𝑘 , 𝑊𝑘 )|22 𝑚 𝑡+1 (2) We it the internal parameters of the model using AdamW [12] as optimizer with 50 epochs. We per‐ form the hyper‐parameter tuning via Grid Search cross‐validation, considering the variables step‐size 𝐷(𝑓)̂ =

VOLUME 17,

N∘ 2

Figure 2. Cuba CPI 2010‐2021 𝐿𝑅 and lag size 𝑝. Nevertheless, we manually set the hyper‐parameter dimension of the embedding 𝑑𝑚𝑜𝑑𝑒𝑙 = 100 on the encoder layer.

5. Results and Discussion 5.1. Dataset The Cuban Consumer Price Index database was collected from the of icial website National Of ice of the Statistic and Information ONEI [18]. This is a monthly time series from January 2010 to December 2020 with very low variability in the data as we can show in Table 1. The values are the the global Cuban CPI averages for 11 grouped of the categories and almost 298 goods and services. It is necessary to clarify that the data sets in the context of the CPI are very short series; learning models that require a lot of data are not effective in this context. Under these conditions we are modelling an appropriated problem as a time series forecast. Figure 2 shows the trends of the series and seasonality. The Augmented Dickey‐Fuller (ADF) test is a pow‐ erful tool used to check the stationarity of the time series. This test can help to choose various parameters such as the size of the windows or the differential order to transform into stationary. The null hypothesis of the ADF test is that the time series is non‐stationary. Therefore, if the p‐value of the test is below the signif‐ icance level (0.05), the null hypothesis is rejected and it follows that the time series is truly stationary. In our time series the result of the ADF test can be found in Table 2. The test result shows that the series is non‐ stationary while the irst differential its stationary. Table 1. Characteristic of the Cuban Consumer Price Index dataset

Figure 1. Transformer‐based architecture design 14

2023

mean std min 25% 50% 75% max

CPI 103.35 1.37 100.12 102.66 103.15 103.89 109.5

Journal of Automation, Mobile Robotics and Intelligent Systems

Table 2. ADF Test over series and the first differential. Lag order d=1

ADF Test, Cuban CPI

ADF Test, First differential of the Cuban CPI

ADF Statistic 0.154 p-value 9.7e‐01 Critical Values 1%: ‐3.488 5%: ‐2.887 10%: ‐2.580 ADF Statistic ‐4.089 p-value 1.01e‐03 Critical Values 1%: ‐3.488 5%: ‐2.887 10%: ‐2.580

Figure 3. Autocorrelation and Partial Autocorrelation in Cuban CPI Autocorrelation refers to how correlated a time series is with its past values whereas the Autocor‐ relation Function (ACF) is the plot used to see the correlation between the points, up to and including the lag unit. Furthermore, the Partial Autocorrelation (PACF) at lag K is the correlation that results after removing the effects of any correlations due to the terms at shorter lags. Figure 3; shows the ACF and PACF of the Cuban CPI for the original and irst differ‐ ential of the series. As we can see the signi icance in the ACF its de ined for a lag 𝑝 ≤ 4 while for the PACF is 𝑞 ≤ 2.

VOLUME 17,

N∘ 2

2023

5.3. Validation and Results In order to validate the model the transformer we de ine the follow methodology in the validation. Firstly, the data were split into training and testing, using data prior to January 1, 2018 to train the models and the last two years to validate the model. The irst experiment was aimed to select the best model using a grid search of the parameters in combination with the ine tuning. The parameters, ine tuning is evaluated at two levels, one of them related to the time series variable representation in the input layer and in the second level we consider the internal parameters of the transformer. In all cases, different runs were per‐ formed with combinations of the parameter settings to obtain the best models at each levels. The testing strategy in case of the transformer model, considers a “multi step ahead” approach procedure while in case of the statistic model like ARIMA, considered in the comparison of the challenge, use “one walk‐forward” procedure, which learns one model in each step. Figure 4 shows the behaviour of the different met‐ rics studied with a set of possible lag sizes 𝑝 = {1, 2, 3, 6, 9, 12}. The best solution can be found in 𝑝 = 1 for all metrics. This result is in correspondence with ADF test and statistic analysis developed in the previ‐ ous section. A second level of parameter tuning was developed internally towards the choice of step size in the AdamW adaptive gradient optimization method. For this, we chose a set of parameters in the range 𝐿𝑅 = {0.00027, 0.00025, 0.00023, 0.00021, 0.00019}. The plot of the behavior of the error metrics shows little sensitivity of the models to the parameter setting, which corresponds to the robustness of the adaptive

5.2. Forecasting Measures As in other similar papers, we use the most com‐ mon metrics for CPI time series forecasting. The Root Mean Squared Error (RMSE), Mean absolute Error (MAE) and Mean absolute Percentage Error (MAPE). Given a test set with a window size to 𝑁, 𝐷𝑡𝑒𝑠𝑡 obser‐ vations and a forecast vector 𝑦̂ 𝑡+𝑙 = 𝑓(𝑦𝑡+𝑙 ), these measures are given as: ∑𝑦𝑡+𝑙 ∈𝐷𝑡𝑒𝑠𝑡 (𝑓(𝑦𝑡+𝑙 ) − 𝑦) ⃗

𝑅𝑀𝑆𝐸(𝑓; 𝐷𝑡𝑒𝑠𝑡 ) = 𝑀𝐴𝐸(𝑓; 𝐷𝑡𝑒𝑠𝑡 ) =

𝑁

1 𝑁

𝑀𝐴𝑃𝐸(𝑓; 𝐷𝑡𝑒𝑠𝑡 ) =

(𝑓(𝑦𝑡+𝑙 ) − 𝑦) ⃗

Figure 4. Fine tuning of the lag size parameter

(3) (4)

𝑦𝑡+𝑙 ∈𝐷𝑡𝑒𝑠𝑡

1 𝑁

𝑦𝑡+𝑙 ∈𝐷𝑡𝑒𝑠𝑡

(𝑓(𝑦𝑡+𝑙 ) − 𝑦) ⃗ 𝑦⃗

(5)

Figure 5. Fine tuning of the step size parameter 15

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

hand, the results evidenced in this work establish a baseline for the CPI database in Cuba and at the same time provide a reference to be used in data sets from other countries.

Figure 6. Forecasting based Transformer Table 3. Challenge of the proposal and ARIMA models ARIMA (1,2,1) ARIMA_BoxJenk Persistence Transformer Best_Transformer

Parameters (𝑝, 𝑑, 𝑞) = (1, 2, 1) (𝑝, 𝑑, 𝑞) = (1, 2, 1) ‐ AdamW, 𝐿𝑅 = 0.00025 AdamW, 𝐿𝑅 = 0.00021

RMSE 0.714 0.657 0.739 0.346 0.337

MAE 0.433 0.426 0.530 0.249 0.231

MAPE 0.004 0.004 0.005 0.717 0.631

gradient as a solver method. Los resultados de la prediccion de los ultimos dos anos a partir del mejor modelo se ilustra en la igura 6, con un buen ajuste del modelo. Similarly, in the experimentation, sev‐ eral ARIMA performances were carried out in order to compare the results of modeling the problem via transformer with respect to the classical approaches used in similar works. For the ARIMA evaluation, dif‐ ferent combinations of the parameters (𝑝, 𝑑, 𝑞) were executed, obtaining (1, 2, 1) as the best combination. In addition, a nonlinear processing based on a Box Jenkins function representation space was included and inally a persistence forecasting model using the previous months was used, which allows us to de ine the worst‐case forecasting scenario. Table 3 summa‐ rizes the results of the models studied in an integrated view. The proposed solution for CPI forecasting in Cuba shows superior results for the all metrics in respect to the classical ARIMA models. This result makes us re lect on the potentialities of deep learn‐ ing and in particular the transformer‐based models with attention mechanisms which have demonstrated in this case study their superiority over the classical approaches. In future studies it is suggested to extend the case studies to the CPI of other countries and to consider the multivariate problem.

6. Conclusion From the tests performed, it can be concluded that the univariate models for the study of the Cuban CPI behave with high ef iciency under the conditions of the evaluation considered and the introduction of the Transformer with attention. The good results of the deep neural networks and particularly the Transform‐ ers show stables results in the CPI dataset conditions (Univariate and very short time series). This result its not common in deep learning in general, where the models need very large samples of data. On the other 16

AUTHORS Reynaldo Rosado – Universidad de las Ciencias Infor‐ máticas, Cuba, e‐mail: rrosado@uci.cu. Orlando G. Toledano-López – Universidad de las Ciencias Informáticas, Cuba, e‐mail: ogtoledano@uci.cu. Hector R. González – Universidad de las Ciencias Informáticas, Cuba, e‐mail: hglez@uci.cu. Aldis J. Abreu – Universidad de las Ciencias Informáti‐ cas, Cuba, e‐mail: ajabreu@uci.cu. Yanio Hernandez – Universidad de las Ciencias Infor‐ máticas, Cuba, e‐mail: yhernandezh@uci.cu.

ACKNOWLEDGEMENTS This work was supported by project PN223LH006013 Plataforma para el análisis de grandes volúmenes de datos y su aplicación a sectores estratégicos.

References [1] A. Banerjee. “Forecasting price levels in india–an arima framework”, Academy of Marketing Studies Journal, vol. 25, no. 1, 2021, 1–15. [2] J. Brownlee, Deep learning for time series forecasting: predict the future with MLPs, CNNs and LSTMs in Python, Machine Learning Mastery, 2018. [3] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio. “Empirical evaluation of gated recurrent neural networks on sequence modeling”. In: NIPS 2014 Workshop on Deep Learning, December 2014, 2014. [4] S. Du, T. Li, Y. Yang, and S.‐J. Horng. “Multivari‐ ate time series forecasting via attention‐based encoder–decoder framework”, Neurocomputing, vol. 388, 2020, 269–279. [5] J. M. García Molina, La economía cubana a inicios del siglo XXI: desa íos y oportunidades de la globalización, CEPAL, 2005. [6] A. Ghazo et al. “Applying the arima model to the process of forecasting gdp and cpi in the jorda‐ nian economy”, International Journal of Financial Research, vol. 12, no. 3, 2021, 70. [7] S. Jere, A. Banda, R. Chilyabanyama, E. Moyo, et al. “Modeling consumer price index in zambia: a comparative study between multicointegration and arima approach”, Open Journal of Statistics, vol. 9, no. 02, 2019, 245. [8] L. M. León Anaya, V. M. Landassuri Moreno, H. R. Orozco Aguirre, and M. Quintana López. “Predicción del ipc mexicano combinando modelos econométricos e inteligencia arti icial”, Revista mexicana de economía y inanzas, vol. 13, no. 4, 2018, 603–629.

Journal of Automation, Mobile Robotics and Intelligent Systems

[9] J. Li, Y. Vidyattama, H. A. La, R. Miranti, and D. M. Sologon. “Estimating the impact of covid‐ 19 and policy responses on australian income distribution using incomplete data”, Social Indicators Research, vol. 162, no. 1, 2022, 1–31.

VOLUME 17,

N∘ 2

2023

[20] R. A. G. Rodríguez. “La visión estructuralista de la in lación. primeras aproximaciones para cuba.”, 2017.

[10] B. Lim, and S. Zohren. “Time‐series forecast‐ ing with deep learning: a survey”, Philosophical Transactions of the Royal Society A, vol. 379, no. 2194, 2021, 20200209.

[21] R. Rosado, A. J. Abreu, J. C. Arencibia, H. Gon‐ zalez, and Y. Hernandez. “Consumer price index forecasting based on univariate time series and a deep neural network”. In: International Workshop on Arti icial Intelligence and Pattern Recognition, 2021, 33–42.

[11] B. Lindemann, T. Müller, H. Vietz, N. Jazdi, and M. Weyrich. “A survey on long short‐term mem‐ ory networks for time series prediction”, Procedia CIRP, vol. 99, 2021, 650–655.

[22] I. Sutskever, O. Vinyals, and Q. V. Le. “Sequence to sequence learning with neural networks”, Advances in neural information processing systems, vol. 27, 2014.

[12] I. Loshchilov, and F. Hutter. “Decoupled weight decay regularization”, arXiv preprint arXiv:1711.05101, 2017.

[23] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. “Attention is all you need”, Advances in neural information processing systems, vol. 30, 2017.

[13] D. P. Manik et al. “A strategy to create daily con‐ sumer price index by using big data in statistics indonesia”. In: 2015 International Conference on Information Technology Systems and Innovation (ICITSI), 2015, 1–5. [14] T. Mikolov, M. Kara iát, L. Burget, J. Černockỳ, and S. Khudanpur. “Recurrent neural network based language model”. In: Eleventh annual conference of the international speech communication association, 2010. [15] J. Mohamed. “Time series modeling and fore‐ casting of somaliland consumer price index: a comparison of arima and regression with arima errors”, American Journal of Theoretical and Applied Statistics, vol. 9, no. 4, 2020, 143–53. [16] T. Nyoni. “Modeling and forecasting in lation in kenya: Recent insights from arima and garch analysis”, Dimorian Review, vol. 5, no. 6, 2018, 16–40. [17] T. Nyoni. “Arima modeling and forecasting of consumer price index (cpi) in germany”, 2019.

[24] W. Wibowo, T. Purwa, E. N. A. Bahri, B. S. S. Ulama, and R. N. Wilantari. “Impacts of earthquakes on consumer price index and in lation: A case study in west nusa tenggara province, indonesia”. In: Journal of Physics: Conference Series, vol. 1863, no. 1, 2021, 012062. [25] R. J. Williams, and D. Zipser. “A learning algo‐ rithm for continually running fully recurrent neural networks”, Neural computation, vol. 1, no. 2, 1989, 270–280. [26] S. Zahara, M. Ilmiddaviq, et al. “Consumer price index prediction using long short term memory (lstm) based cloud computing”. In: Journal of Physics: Conference Series, vol. 1456, no. 1, 2020, 012022. [27] B. Zhang, G. Zou, D. Qin, Y. Lu, Y. Jin, and H. Wang. “A novel encoder‐decoder model based on read‐ irst lstm for air pollutant prediction”, Science of The Total Environment, vol. 765, 2021, 144507.

[18] ONEI. “Índice de precios al consumidor base diciembre 2010”, 2022. [19] J. Riofrío, O. Chang, E. Revelo‐Fuelagán, and D. H. Peluffo‐Ordóñez. “Forecasting the con‐ sumer price index (cpi) of ecuador: A compar‐ ative study of predictive models”, International Journal on Advanced Science, Engineering and Information Technology, vol. 10, no. 3, 2020, 1078–1084.

VOLUME 17, N∘ 2 2023 Journal of Automation, Mobile Robotics and Intelligent Systems

HEAVY MOVING AVERAGE DISTANCES IN SALES FORECASTING Submitted: 16th March 2023; accepted: 20th July 2023

Maricruz Olazabal‑Lugo, Luis F. Espinoza‑Audelo, Ernesto León‑Castro, Luis A. Perez‑Arellano, Fabio Blanco‑Mesa DOI: 10.14313/JAMRIS/2‐2023/12 Abstract: This paper presents a new aggregation operator tech‐ nique that uses the ordered weighted average (OWA), heavy aggregation operators, Hamming distance, and moving averages. This approach is called heavy ordered weighted moving average distance (HOWMAD). The main advantage of this operator is that it can use the characteristics of the HOWMA operator to under‐ or over‐ estimate the results according to the expectations and the knowledge of the future scenarios, analyze the his‐ torical data of the moving average, and compare the different alternatives with the ideal results of the dis‐ tance measures. Some of the main families and specific cases using generalized and quasi‐arithmetic means are presented, such as the generalized heavy moving aver‐ age distance and a generalized HOWMAD. This study develops an application of this operator in forecasting the sales growth rate for a commercial company. We find that it is possible to determine whether the company’s objectives can be achieved or must be reevaluated in response to the actual situation and future expectations of the enterprise. Keywords: Heavy moving average distance, OWA opera‐ tor, Distance measures, Sales forecasting.

1. Introduction In decision‐making, various methods can be used to select the best alternative [1, 2]. Of these tech‐ niques, distance measures are some of the most com‐ mon. These include the Hamming Distance [3], the Euclidean distance [4], the Minkowski distance [5], and more generally, all distance methods [6]. The most common problems that can be solved with distance measures are decision‐making prob‐ lems related to comparing alternatives between the actual data and the objective or ideal results [7,8]. The path with the closest distance to zero will be the ideal choice to select. In recent years many authors have developed new extensions such as the logarithmic dis‐ tance [9], prioritized distances [10], Bonferroni means distances [11, 12], induced Euclidean distances [13], intuitionistic fuzzy induced distances [14], induced heavy aggregation distances [15], probabilistic dis‐ tances [16] simpli ied neutrosophic distances [18], attitudinal distances [18], fuzzy linguistic induced dis‐ tances [19], and many more. The most basic distance measure in decision‐ making is the normalized Hamming distance (NHD). Still, it is also possible to combine this traditional 18

technique with other techniques to generate new scenarios for evaluation. For example, one of the most common aggregation techniques is the ordered weighted average (OWA) operator, developed by Yager [20]. The OWA operator has been shown to be a very general and universal aggregation operator that provides tools for human consistent aggregation as shown in Kacprzyk, Yager and Merigo [21], which provides an account of a wide variety of various opera‐ tors. An example of the power of the ordered weighted averaging operators is Kacprzyk and Zaderozny [22] in which, through the choice of the type of the ordered weighted operator and its weights, various voting pro‐ cedures can be represented, providing a convenient and powerful representation. Also, it has been used to solve different problems [23, 24] such as the clus‐ tering method for classi ication problems [25] portfo‐ lio selection [26], competitiveness [27], econometric forecasting [28], pro it of investment [29], business failure [30], inancial decisions [31], and so on. Among the wide variety of OWA operator exten‐ sions, this paper focuses on the heavy OWA (HOWA) operator. This operator has the advantage that the weighting vector is not bounded by 1, so in this case, it is possible to under‐ or overestimate the results and generate more scenarios to consider [32]. Another technique considered in this paper is the moving aver‐ age (MA), which is useful for representing dynamic information because it can ilter out short‐term luc‐ tuations [33]. This common technique, used in many different time series problems in different ields, is an important method to forecast future scenarios based on historical data [34]. This paper presents a new aggregation operator that uses the three previously speci ied techniques. First, we introduce the heavy ordered weighted moving average distance (HOWMAD) operator. It is a distance aggregation operator that considers a parameterized family of distance aggregation opera‐ tors between the minimum and maximum distances. Some of the most important families and speci ic cases are also presented, including the heavy weighted mov‐ ing average distance (HWMAD) operator, the quasi‐ HOWMAD operator, and the generalized HOWMAD operator. To analyze the usefulness of the HOWMAD oper‐ ator, we use it to forecast sales growth rates of a commercial company, identify whether the objectives of the enterprise can be achieved, and analyze whether it is possible to reformulate the objectives based on knowledge about the future and the information

2023 © Olazabal-Lugo et al. This is an open access article licensed under the Creative Commons Attribution-Attribution 4.0 International (CC BY 4.0) (http://creativecommons.org/licenses/by-nc-nd/4.0/)

Journal of Automation, Mobile Robotics and Intelligent Systems

provided by the historical data. In the example, we compare the results with other distance measures. The remainder of the paper is organized as follows. Section 2 reviews aggregation operators and some distance techniques. Section 3 introduces the HOW‐ MAD operator, and Section 4 develops the generalized HOWMAD operator. Section 5 explains the steps for using heavy moving average operators, and Section 6 presents an application of the HOWMAD operator in sales forecasting. Finally, Section 7 summarizes the main conclusions of the paper.

2. Preliminaries This section brie ly reviews some basic concepts to be used throughout the paper, including the aggrega‐ tion operators, the moving average operators, and the distance techniques. 2.1. Aggregation Operators

VOLUME 17,

OWA(a1 , a2 , … , an ) =

wj bj ,

(1)

j=1

where bj is the jth largest element of the collection ai . Another extension of the OWA operator can be obtained by increasing or decreasing the values in the weighting vector; in this case, the sum of the weight values is not bounded by 1. This operator is called the Heavy OWA (HOWA) developed by Yager [32]. One of its main characteristics is that due to the inclusion of a weighting vector whose sum can range from −∞ to ∞, the results can be drastically under‐ or over‐ estimated according to the information and knowl‐ edge possessed by the decision‐maker. This operator is de ined as follows. De inition 2. A heavy aggregation operator is an extension of the OWA operator for which the sum of weights is bounded by 𝑛. Thus, an HOWA operator is a map 𝑅𝑛 → 𝑅 that is associated with a weight vector w, 𝑛 with 𝑤𝑗 ∈ [0, 1] and 1 ≤ ∑𝑗=1 𝑤𝑗 ≤ 𝑛 such that 𝑛

𝐻𝑂𝑊𝐴(𝑎1 , 𝑎2 , … , 𝑎𝑛 ) =

𝑤𝑗 𝑏𝑗 ,

(2)

𝑗=1

where 𝑏𝑗 is the jth largest element of the collection 𝑎1 , 𝑎2 , … , 𝑎𝑛 and the sum of the weights 𝑤𝑗 is bounded to n or can be unbounded if the weighting vector 𝑛 𝑊, −∞ ≤ ∑𝑗=1 𝑤𝑗 ≤ ∞. One of the characteristics introduced by Yager [32] is a characterizing parame‐ ter, called the beta value of the vector W, which can be

2023

de ined as 𝛽(𝑊) = (|𝑊|−1)/(𝑛 −1). If |𝑊| ∈ [1, 𝑛], it follows that 𝛽 ∈ [0, 1]. Therefore, if 𝛽 = 1, we obtain the total operator, and if 𝛽 = 0, we obtain the usual OWA operator. 2.2. Moving‐average Operators One common technique that can be used to solve time series smoothing problems is the moving aver‐ age, which can be extended to the moving‐average operators [36]. Some applications of these operators have been described in economics and statistics [34]. The moving average can be de ined as follows [37]. 𝑁 De inition 3. Given {𝑎𝑖 }𝑖=1 , the moving aver‐ age of dimension n is de ined as the sequence 𝑁−𝑛+1 {𝑠𝑖 }𝑖=1 obtained by taking the arithmetic mean of the sequence of 𝑛 terms, such that

𝑠𝑖 =

One operator that can be used to aggregate infor‐ mation is the OWA operator introduced by Yager [20]. This operator allows the aggregation of informa‐ tion between the maximum and the minimum, and since its introduction, many applications have been described [35]. The de inition is as follows. De inition 1. An OWA operator of dimension n is a mapping OWA∶ 𝑅𝑛 → 𝑅 with an associated weight n vector W of dimension n such that ∑j=1 𝑤𝑗 = 1 and 𝑤𝑗 ∈ [0, 1], according to the following formula.

N∘ 2

1 𝑛

𝑖+𝑛−1

𝑎𝑗 .

(3)

𝑗=𝑖

Another extension of the usual moving average involves combining it with the HOWMA operator. This operator is called the heavy ordered weighted mov‐ ing average (HOWMA) operator. The main advantage of this new operator is that it is possible to under‐ or overestimate the results of the classical moving average according to the expectations of the decision‐ maker for future scenarios. Therefore, this technique is useful for generating new scenarios, which can help understand different and new alternatives for the future. This operator can be de ined as follows [38]. 𝑁 De inition 4. Given a sequence {𝑎𝑖 }𝑖=1 , the HOWMA operator is de ined as the operator that acts 𝑁−𝑛+1 on the sequence {𝑠𝑖 }𝑖=1 , which is multiplied by a heavy weighting vector, according to 𝑚+𝑡

𝐻𝑂𝑊𝑀𝐴(𝑠𝑖 ) =

𝑤𝑗 𝑏𝑗 ,

(4)

𝑗=1+𝑡

where 𝑏𝑗 is the jth largest element of the collection a1 , a2 , … , an , and W is an associated weighting vec‐ 𝑚+𝑡 tor of dimension m that satis ies 1 ≤ ∑𝑖=1+𝑡 𝑤𝑖 ≤ 𝑛 and 𝑤𝑖 ∈ [0, 1]. Observe that we can also expand the weighting vector to the range −∞ to ∞. Thus, the weighting vector w becomes unbounded: −∞ ≤ 𝑛 ∑𝑗=1 𝑤𝑗 ≤ ∞. It is important to note that, as with the OWA oper‐ ator, it is possible to distinguish between the descend‐ ing HOWMA (DHOWMA) operator and the ascending HOWMA (AHOWMA) operator, according to the same rules for the weighting vector that were applied to the OWA operator. 2.3. Distance Techniques A useful technique to calculate the distance between two elements is the Hamming distance [3]. This can be used to calculate the distance between two sets, which can be applied within fuzzy set theory. To de ine the Hamming distance, it is necessary to de ine the basic properties of a distance measure: 19

Journal of Automation, Mobile Robotics and Intelligent Systems

N∘ 2

2023

b) Commutativity: 𝐷(𝐴1 , 𝐴2 ) = 𝐷(𝐴2 , 𝐴1 );

3. Heavy Ordered Weighted Moving Average Distance Operator

c) Re lexivity: 𝐷(𝐴1 , 𝐴2 ) = 0;

3.1. Main Concept

d) Triangle inequality: 𝐷(𝐴1 , 𝐴2 ) + 𝐷(𝐴2 , 𝐴3 ) ≥ 𝐷(𝐴1 , 𝐴3 ). With these properties, the Hamming distance can be de ined as follows (Merigó et al., 2014). De inition 5. A normalized Hamming distance of 𝑛 𝑛 dimension n is a mapping 𝑁𝐻𝐷 ∶ [0, 1] 𝑥[0, 1] → [0, 1], such that

These distance approaches can be extended using the HOWMA operator. In this way, we obtain the heavy ordered weighted moving averaging distance (HOWMAD) operator. This new extension has the advantage that we can under‐ or overestimate the results according to the decision‐maker’s perception and the knowledge of the future scenarios of the variables. Like the HOWMA operator, this new extension includes a weighting vector whose values can range from 1 to ∞ or even from −∞ to ∞. It is important to note that it is possible to obtain new scenarios and have a better understanding of different situations because of this property. The de inition of the HOWMAD operator is as follows. De inition 8. A HOWMAD operator of dimension m is a mapping 𝐻𝑂𝑊𝑀𝐴𝐷∶ 𝑅 𝑚 𝑥𝑅𝑚 → 𝑅 that has an 𝑚+𝑡 associated weighting vector W, with 1 ≤ ∑𝑖=1+𝑡 𝑤𝑖 ≤ 𝑛 and 𝑤𝑖 ∈ [0, 1], such that

a) Non‐negativity: 𝐷(𝐴1 , 𝐴2 ) ≥ 0;

𝑁𝐻𝐷(𝐴, 𝐵) =

1 𝑁

𝑛

|𝑎𝑖 − 𝑏𝑖 | ,

(5)

𝑖=1

where 𝑎𝑖 and 𝑏𝑖 are the ith arguments of sets A and B, respectively. Another extension can be obtained by combin‐ ing the OWA operator and the normalized Hamming distance. This operator is called Hamming ordered weighted average distance (OWAD) operator and can be de ined as follows [39]. De inition 6. An OWAD operator of dimension n is 𝑛 𝑛 a mapping 𝑂𝑊𝐴𝐷∶ [0, 1] 𝑥[0, 1] → [0, 1] that has an 𝑛 associated weighting vector w, with ∑𝑗=1 𝑤𝑗 = 1 and 𝑤𝑗 ∈ [0, 1] such that 𝑛 (𝑘) (𝑘) 𝑂𝑊𝐴𝐷(⟨𝜇1 , 𝜇1 ⟩, … , ⟨𝜇𝑛 , 𝜇𝑛 ⟩) =

𝑤𝑗 𝐷𝑗 ,

(6)

𝑗=1

where 𝐷𝑗 is the jth largest of the differences |𝑥𝑖 − 𝑦𝑖 |, and |𝑥𝑖 − 𝑦𝑖 | is the argument variable represented in the form of individual distances. It is important to note that the distance de ini‐ tion can be combined with the moving average tech‐ nique; in this situation, we obtain the moving average distance (MAD) operator, which can be de ined as follows. Another extension, the ordered weighted moving average distance (OWMAD) operator, can be obtained using the moving averages, Hamming distances, and OWA operator. The main advantage of this new oper‐ ator is that we can under‐ or overestimate the results according to the decision maker’s perception. The def‐ inition of this operator is as follows [36]. De inition 7. An OWMAD operator of dimension m is a mapping 𝑂𝑊𝑀𝐴𝐷∶ 𝑅 𝑚 𝑥𝑅𝑚 → 𝑅 that has 𝑚+𝑡 an associated weighting vector W, with ∑𝑗=1+𝑡 𝑤𝑗 = 1 and 𝑤𝑗 ∈ [0, 1], such that 𝑚+𝑡

𝑂𝑊𝑀𝐴𝐷(⟨𝑥1 , 𝑦1 ⟩, ⟨𝑥2 , 𝑦2 ⟩, … , ⟨𝑥𝑚 , 𝑦𝑚 ⟩) =

𝑤𝑗 𝐷𝑗 , 𝑗=1+𝑡

(7) where 𝐷𝑗 represents the jth largest of the differences |𝑥𝑖 − 𝑦𝑖 |; 𝑥𝑖 and 𝑦𝑖 are the ith arguments of the sets X and Y; m is the total number of arguments considered from the whole sample, and t indicates the movement of the average from the initial analysis. 20

VOLUME 17,

𝑚+𝑡

HOWMAD(⟨𝑥1 , 𝑦1 ⟩, ⟨𝑥2 , 𝑦2 ⟩, … , ⟨𝑥𝑚 , 𝑦𝑚 ⟩) =

𝑤𝑗 𝐷𝑗 , 𝑗=1+𝑡

(8) where 𝐷𝑗 represents the jth largest of the differences |𝑥𝑖 − 𝑦𝑖 |; 𝑥𝑖 and 𝑦𝑖 are the ith arguments of the sets X and Y; m is the total number of arguments considered from the whole sample; t indicates the movement in the average from the initial analysis; and the values 𝑤𝑖 of the weighting vector can range from 1 to ∞, or even from −∞ to ∞. It is important to note that the characteristics of the HOWMA operator can also be identi ied in the HOWMAD operator, so that we can distinguish between the descending HOWMAD (DHOWMAD) and the ascending HOWMAD (AHOWMAD) operators. Additionally, the HOWMAD operator is monotonic and commutative but not bounded if the range of the weighting vector is from −∞ to ∞. Finally, the characteristics of the beta value of the |𝑊|−1 vector W, which can be de ined as 𝛽(𝑊) = , 𝑛−1 also apply to the HOWMAD operator. If |𝑊| ∈ [1, 𝑛], it follows that 𝛽 ∈ [0, 1]. Therefore, if 𝛽 = 1, we obtain the total operator, and if 𝛽 = 0, we obtain the usual OWA operator. 3.2. Families of the HOWMAD Operator In this section, different types of HOWMAD oper‐ ators are presented. Initially, we consider the cases in which the HOWMAD operator becomes the HWMAD operator, the OWMAD operator, the WMAD oper‐ ator, the MAD operator, among others [40, 41]. To obtain these operators, we must make different changes to the weighting vector: a) The total distance operator when 𝛽 = 1; b) The minimum distance when 𝑤𝑛 = 1 and 𝑤𝑗 = 0, for all 𝑗 ≠ 𝑛 and 𝛽 = 0;

Journal of Automation, Mobile Robotics and Intelligent Systems

c) The HWMAD operator is obtained when the 𝑚+𝑡 weighting vector W satis ies 1 ≤ ∑𝑖=1+𝑡 𝑤𝑖 ≤ 𝑛, but there is no reordering step according to the jth largest element of |𝑥𝑖 − 𝑦𝑖 |; d) The OWMAD operator is obtained when the 𝑚+𝑡 weighting vector W satis ies ∑𝑗=1+𝑡 𝑤𝑗 = 1 and 𝑤𝑗 ∈ [0, 1], or when 𝛽 = 0; e) The WMAD operator is obtained when the weight‐ 𝑚+𝑡 ing vector W satis ies ∑𝑗=1+𝑡 𝑤𝑗 = 1, 𝑤𝑗 ∈ [0, 1] and there is no reordering step according to jth largest element of |𝑥𝑖 − 𝑦𝑖 |; f) The MAD operator is obtained when there is no associated weighting vector; g) The Olympic‐HOWMAD operator is obtained when 𝑤1 = 𝑤𝑛 = 0 and 𝑤𝑗 = 1/(𝑛 − 2) for all other values of j; h) The HOWMA operator is obtained if one of the sets is empty; i) The OWMA operator is obtained if one of the sets is empty and the weighting vector satis ies 𝑚+𝑡 ∑𝑗=1+𝑡 𝑤𝑗 = 1; j) The WMA operator is obtained if one of the sets is empty, the weighting vector satis ies 𝑚+𝑡 ∑𝑗=1+𝑡 𝑤𝑗 = 1, and there is no reordering step according to the jth largest element of |𝑥𝑖 − 𝑦𝑖 |; instead, the reordering is according to the ith initial ordering; k) The uniform distance allocation is obtained when |𝑊| 𝑤𝑗 = ; 𝑛

l) The push‐down allocation is obtained when 𝑤𝑛−𝑗+1 = (1 ∧ (|𝑊| − (𝑗 − 1))) ∨ 0; m) The push‐up allocation is obtained when 𝑤𝑗 = (1 ∧ (|𝑊| − (𝑗 − 1))) ∨ 0.

4. Generalized Heavy Moving‐Average Distances This section presents an analysis of different generalizations of the HOWMAD operator using quasi‐ arithmetic means. This is done because, in the quasi‐arithmetic means, it is possible to obtain the generalization of a particular case [42]. We can obtain new partial cases through this analysis, thus making it possible to consider the aggregation process more completely. The quasi‐HOWMAD (QHOWMAD) oper‐ ator can be de ined as follows. De inition 9. A Quasi‐HOWMAD operator of dimension m is a mapping Quasi-HOWMAD: 𝑅𝑚 𝑥𝑅𝑚 → 𝑅 that has an associated weighting vector W, with 1 ≤ 𝑚+𝑡 ∑𝑖=1+𝑡 𝑤𝑖 ≤ 𝑛 and 𝑤𝑖 ∈ [0, 1], such that Quasi-HOWMAD(⟨𝑥1 , 𝑦1 ⟩, ⟨𝑥2 , 𝑦2 ⟩, … , ⟨𝑥𝑚 , 𝑦𝑚 ⟩) 𝑚+𝑡

=𝑔

−1

𝑤𝑗 𝑔(𝐷𝑗 ),

(9)

𝑗=1+𝑡

where 𝐷𝑗 represents the jth largest of the differences |𝑥𝑖 − 𝑦𝑖 |; 𝑥𝑖 and 𝑦𝑖 are the ith arguments of the sets X

VOLUME 17,

N∘ 2

2023

and Y; m is the total number of arguments considered from the whole sample; t indicates the movement in the average from the initial analysis; 𝑤𝑖 are the ele‐ ments of the weighting vector, which can range from 1 to ∞ or even from −∞ to ∞; and 𝑔(𝐷𝑗 ) is a strictly continuous monotone function. Also, some families of the Quasi‐HOWMAD opera‐ tor are: a) The generalized HOWMAD (GHOWMAD) opera‐ tor, better known as the Minkowski HOWMAD (MHOWMAD) operator, is obtained if 𝑔(𝐷) = 𝐷 𝜆 . range from 1 to ∞, or even from −∞ to ∞; and 𝜆 is a parameter such that 𝜆 ∈ (−∞, ∞). b) The HOWMAD operator is obtained if 𝑔(𝐷) = 𝐷. c) The heavy ordered weighted moving quadratic‐ average distance (HOWMQAD) operator, or Euclidean HOWMAD (EHOWMAD) operator, is obtained if 𝑔(𝐷) = 𝐷 2 . d) The heavy maximum distance is obtained if 𝑔(𝐷) → 𝐷 𝜆 , for 𝜆 → ∞. e) The heavy minimum distance is obtained if 𝑔(𝐷) → 𝐷 𝜆 , for 𝜆 → −∞. f) The heavy ordered weighted moving harmonic average distance (HOWMHAD) operator is obtained if 𝑔(𝐷) = 𝐷 −1 . g) The heavy ordered weighted moving geometric‐ average distance operator is obtained if 𝑔(𝐷) → 𝐷 𝜆 , for 𝜆 → 0. Note that individual distances equal to zero are not considered in the aggregation because they are neutral elements.

5. Forecasting Sales Using Heavy Moving Average Operators 5.1. Theoretical Approach Sales forecasting has become one of the most important aspects of planning for companies and usu‐ ally employs sales control systems, such as rules, policies, and procedures, to reach different corpo‐ rate objectives [43, 44]. However, if the sales objec‐ tives cannot be achieved because they are misaligned with reality, this will negatively affect the sales team. Therefore, a common sales forecasting method con‐ siders representative values by using an averaging technique, such as the arithmetic mean or weighted average, to construct different horizons and scenarios. To obtain a new scenario of the sales in the future, the HOWMAD operator can be used. To use this oper‐ ator correctly, the following steps can be taken to summarize the application. Step 1. Analyze and determine the increase or decrease in the sales growth rate based on the his‐ torical data that can be considered important and will have a signi icant impact on the results, over a certain time (e.g., 6 months, 12 months, 3 years). Step 2. Determine the ideal sales growth rate that the company wants to achieve, based on the objectives set in the manuals or that the administration has set for the future sales growth rate. 21

Journal of Automation, Mobile Robotics and Intelligent Systems

Step 3. The difference between the ideal objective and the real results is considered in this step. From this, we obtain the historical distances of the sales, which will help us forecast the future distance between the ideal and the future sales growth rate. Step 4. Once the difference between the objective and the real data is obtained, we ask the decision‐maker to provide two different weighting vectors, one whose sum is equal to 1 and one that is considered a heavy weighting vector, according to their knowledge and expectations for the future of the company. Step 5. With the historical data and the weighting vectors, we can use different operators to obtain the sales distance forecasts, such as Moving Average Distance (MAD), Weighted Moving Average Distance (WMAD), Ordered Weighted Moving Average Dis‐ tance (OWMAD) and Heavy Ordered Weighted Moving Average Distance (HOWMAD). Step 6. With the results obtained with the different operators, it is possible to analyze whether the objec‐ tive set by the company is reasonable or must be reformulated according to the information obtained using the historical information and the expectations described by the weighting vector. 5.2. Numerical Example In this section, we investigate a real problem in a Mexican commercial enterprise that wants to know the difference between the historical sales growth rates and their objective because they want to know if they must reevaluate this objective according to future scenarios. To do this, we applied the steps de ined in Section 5.1. Step 1: The Mexican commercial enterprise provides monthly sales growth rates from 2015–2021 (see Table 1). Step 2: They indicate that the objective sales growth rate is 0.28% per month, for a total of 3.36% per year. Step 3: We calculate the distance between the objec‐ tive sales growth rate and the real growth in sales. The results are as follows (see Table 2).

VOLUME 17,

2015 0.3 0.42 0.24 0.31 0.36 0.28 0.33 0.1 0.19 0.22 0.24 0.35 3.34

2016 0.25 0.31 0.26 0.27 0.29 0.33 0.25 0.15 0.2 0.23 0.28 0.22 3.04

2017 0.31 0.29 0.31 0.29 0.18 0.25 0.27 0.12 0.22 −0.08 0.18 0.32 2.66

2023

Step 4: To generate different future scenarios, the weighting vector 𝑊 = (0.1, 0.15, 0.05, 0.1, 0.15, 0.2, 0.25) = 1 is provided. Additionally, the heavy weight‐ ing vector 𝑊 = (0.1, 0.15, 0.1, 0.15, 0.1, 0.15, 0.3) = 1.05 is used because, according to the expectations and knowledge of the general manager, the company will have better results due to the company’s future economic scenarios and competition. Step 5: The distance aggregation operators are used to forecast the distances for the next ive years. The results are as follows (see Tables 3–6). In Tables 3–6, we use four different aggregation operators to generate different scenarios for the sales distance over the next ive years. To compare the results obtained by the different operators, the following rules are presented by the decision‐maker: 1) If the distance is negative, the objective can be achieved easily (A). 2) If the distance is positive but less than 0.05, the objective can be achieved but with a medium degree of dif iculty (B). 3) If the distance is positive and more than 0.05, the objective cannot be achieved (C). The results are as follows (See Tables 7, 8). As shown in Table 7, different results can be seen, particularly in January, where according to the MAD and WMAD, the objective of sales can be achieved easily in contrast with the OWMAD and HOWMAD results that indicate a medium degree of dif iculty. In the other months, the same analysis can be seen; in this sense, when we include more information in the operator, the results can differ one from another. In the case of Table 8, it is possible to see that in August and December the analysis of the results between the operators is the same. But in the other months, as in the case of Table 7, the same conclusion can be made. When we include more data into the formulation, the results vary, which is the knowledge and expertise of the decision‐maker being included in the operator. Step 6: Analyzing the data reveals that only in Decem‐ ber is it clear that it is possible to achieve the desired

Table 1. Sales growth rates, 2015–2021 January February March April May June July August September October November December Total

N∘ 2

2018 0.36 0.28 0.33 0.32 0.27 0.23 0.23 0.11 0.24 0.21 0.23 0.29 3.1

2019 0.32 0.22 0.18 0.28 0.16 0.14 0.21 0.17 0.23 0.18 0.25 0.33 2.67

2020 0.24 0.32 0.22 0.27 0.31 0.22 0.18 0.32 0.18 0.34 0.3 0.29 3.19

2021 0.28 0.19 0.3 −0.09 0.15 0.22 0.32 0.18 0.29 0.35 0.28 0.4 2.87

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

Table 2. Distance between objective and real sales growth rates January February March April May June July August September October November December

2015 −0.02 −0.14 0.04 −0.03 −0.08 0 −0.05 0.18 0.09 0.06 0.04 −0.07

2016 0.03 −0.03 0.02 0.01 −0.01 −0.05 0.03 0.13 0.08 0.05 0 0.06

2017 −0.03 −0.01 −0.03 −0.01 0.1 0.03 0.01 0.16 0.06 0.36 0.1 −0.04

2018 −0.08 0 −0.05 −0.04 0.01 0.05 0.05 0.17 0.04 0.07 0.05 −0.01

2019 −0.04 0.06 0.1 0 0.12 0.14 0.07 0.11 0.05 0.1 0.03 −0.05

2020 0.04 −0.04 0.06 0.01 −0.03 0.06 0.1 −0.04 0.1 −0.06 −0.02 −0.01

2021 0 0.09 −0.02 0.37 0.13 0.06 −0.04 0.1 −0.01 −0.07 0 −0.12

Table 3. Sales growth rate distance forecasting with MAD operator January February March April May June July August September October November December

2022 −0.014 −0.010 0.017 0.044 0.034 0.041 0.024 0.116 0.059 0.073 0.029 −0.034

2023 −0.013 0.009 0.014 0.055 0.051 0.047 0.035 0.107 0.054 0.075 0.027 −0.029

2024 −0.020 0.014 0.013 0.061 0.059 0.061 0.036 0.103 0.050 0.078 0.031 −0.042

2025 −0.018 0.018 0.019 0.071 0.053 0.066 0.039 0.095 0.049 0.038 0.021 −0.042

2026 −0.009 0.020 0.029 0.087 0.060 0.068 0.038 0.084 0.050 0.033 0.017 −0.047

2025 −0.017 0.024 0.019 0.089 0.062 0.065 0.033 0.100 0.045 0.040 0.021 −0.047

2026 −0.007 0.016 0.029 0.083 0.054 0.065 0.040 0.079 0.053 0.029 0.015 −0.043

2025 0.012 0.035 0.052 0.115 0.086 0.077 0.060 0.156 0.081 0.156 0.054 0.001

2026 0.013 0.036 0.053 0.115 0.088 0.077 0.060 0.156 0.081 0.157 0.054 0.001

Table 4. Sales growth rate distance forecasting with WMAD operator January February March April May June July August September October November December

2022 −0.005 0.005 0.023 0.089 0.041 0.047 0.026 0.096 0.053 0.024 0.015 −0.041

2023 −0.005 0.015 0.016 0.095 0.058 0.049 0.027 0.095 0.050 0.056 0.021 −0.041

2024 −0.015 0.016 0.006 0.091 0.057 0.054 0.028 0.101 0.046 0.054 0.025 −0.045

Table 5. Sales growth rate distance forecasting with OWMAD operator January February March April May June July August September October November December

2022 0.002 0.019 0.039 0.090 0.065 0.064 0.046 0.142 0.073 0.125 0.045 −0.013

2023 0.006 0.026 0.043 0.097 0.074 0.070 0.051 0.150 0.077 0.137 0.048 −0.007

2024 0.009 0.030 0.047 0.100 0.079 0.073 0.056 0.153 0.079 0.144 0.050 −0.004

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

Table 6. Sales growth rate distance forecasting with HOWMAD operator January February March April May June July August September October November December

2022 −0.001 0.014 0.038 0.106 0.062 0.066 0.045 0.140 0.073 0.135 0.046 −0.015

2023 0.001 0.019 0.038 0.086 0.068 0.067 0.046 0.153 0.077 0.125 0.045 −0.015

2024 0.006 0.026 0.045 0.099 0.077 0.074 0.055 0.159 0.082 0.145 0.050 −0.009

2025 0.009 0.031 0.052 0.116 0.085 0.080 0.060 0.165 0.085 0.159 0.056 −0.004

2026 0.010 0.033 0.054 0.117 0.091 0.081 0.062 0.168 0.087 0.161 0.057 −0.003

Table 7. Comparison between operators from January to June 2022–2026 Month January

February

March

April

May

June

Operator MAD WMAD OWMAD HOWMAD MAD WMAD OWMAD HOWMAD MAD WMAD OWMAD HOWMAD MAD WMAD OWMAD HOWMAD MAD WMAD OWMAD HOWMAD MAD WMAD OWMAD HOWMAD

2022 A A B A A B B B B B B B B C C C B B C C B B C C

objective of the enterprise. This is because the inal distance is negative, which indicates that the values in set Y are higher than those in set X. Because X is the objective sales growth rate and Y is the real growth, the company needs to consider a more realistic objec‐ tive that can be achieved in the future. Additionally, it is possible to construct objectives for each month and the general objective with this information. With the use of these aggregation operators, we can formulate different expectations for the future based on the information provided by the decision‐ maker. In this way, the results can include more infor‐ mation and provide more realistic expectations, given the uncertainty that the commercial markets are fac‐ ing. This idea is seen when the results obtained by the MAD/WMAD/OWMAD/HOWMAD are analyzed and have a signi icant variation. The applicability of each of the aggregation operators analyzed in the paper are based on the complexity of the problem to 24

2023 A A B B B B B B B B B B C C C C C C C C B B C C

2024 A A B B B B B B B B B B C C C C C C C C C C C C

2025 A A B B B B B B B B C B C C C C C C C C C C C C

2026 A A B B B B B B B B C B C C C C C C C C C C C C

analyze, that is because if we use the MAD operator the same result will be obtained always (this because only the historical data is taken into account) but if the decision‐maker wants to add some quantita‐ tive information such the expectation that the sales increases or decreases in a certain month, a new competitor will appear in the same market, a con‐ traction or expansion of the general economy or any other information that is important and are not taken into account in the historical data, the use of differ‐ ent weighting vectors and ordering process can be used. In this case, the WMAD uses a weighting vector provided by the decision‐maker and the association between the attributes and the weights remain the same always, with the OWMAD it is possible to order the attributes and weights association according to an optimist or pessimist expectation of the future and in the HOWMAD the weighting vector can be higher or lower than 1, these will under‐ or overestimate

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

Table 8. Comparison between operators from July to December 2022–2026 Month July

August

September

October

November

December

Operator MAD WMAD OWMAD HOWMAD MAD WMAD OWMAD HOWMAD MAD WMAD OWMAD HOWMAD MAD WMAD OWMAD HOWMAD MAD WMAD OWMAD HOWMAD MAD WMAD OWMAD HOWMAD

2022 B B B B C C C C C C C C B B C C B B B B A A A A

the results according to the expectations, aptitude and other quantitative elements that the decision‐maker believes will happened. It is important to note that the results are related to the information and weighting vectors provided by the experts, and this can be an important study limitation. For example, if we make changes to the experts and select different weighting vectors, the results could be signi icantly different. Additionally, we would like to mention that this limitation is also one of the advan‐ tages of using an operator since it is possible to gener‐ ate a signi icant number of scenarios when analyzing the same problem from the point of view of different experts.

6. Conclusion The paper has introduced a new extension of the OWA operator called the heavy ordered weighted moving average distance (HOWMAD) operator. This operator uses the main characteristics of the HOWMA operator and the Hamming distance technique. With this operator, it is possible to combine historical infor‐ mation with the decision‐makers’ knowledge to fore‐ cast the distance between two data sets. We provided the de inition and some of the main properties of the HOWMAD operator. Additionally, we developed a wide range of families of the HOWMAD operator, such as the HWMAD, OWMAD, WMAD, and MAD operators. Additionally, using quasi‐arithmetic means, several cases are presented at the end of Sec‐ tion 4, including the GHOWMAD, HOWMQAD, and HOWMCAD operators. An application of this new approach in sales fore‐ casting has also been developed. We observe that with

2023 B B C B C C C C C C C C B B C C B B B B A A A A

2024 B B C C C C C C C B C C B B C C B B C C A A A A

2025 B B C C C C C C B B C C B B C C B B C C A A A A

2026 B B C C C C C C C C C C B B C C B B C C A A A A

the HOWMAD operator, we can generate new scenar‐ ios to provide the decision‐maker with new alterna‐ tives and possible plans of action. Under uncertainty, it is very important to analyze many possible alterna‐ tives to determine the best decision. In this case, we provide information to the company to reformulate their sales growth rate objectives to something more achievable based on the historical data and the expec‐ tations of the general manager. In future research, we plan to further develop this operator by adding news aggregators, such as the induced ordered weighted average (IOWA) oper‐ ator, probabilistic ordered weighted average (POWA) operator, weighted averages (WA) operators, fuzzy numbers, interval numbers, or multi‐person tech‐ niques [45–47]. Also, the use of the operator in complex and dynamic decision problems, such as the consensus reaching process [48, 49]. AUTHORS Maricruz Olazabal-Lugo – Unidad Regional Culiacán, Universidad Autónoma de Occidente, Culiacán, Sinaloa, México, e‐mail: mari‐ cruz.olazabal@uadeo.mx. Luis F. Espinoza-Audelo∗ – Tecnológico Nacional de México/Instituto Tecnológico de Culiacán, Sinaloa, México, e‐mail: luis.ea@culiacan.tecnm.mx. Ernesto León-Castro – Faculty of Economics and Administrative Sciences, Universidad Católica de la Santísima Concepción, Concepción, Chile, e‐mail: eleon@ucsc.cl. Luis A. Perez-Arellano – Facultad de Psicología, Uni‐ versidad Autónoma de Sinaloa, Culiacán, Sinaloa, Mex‐ ico, e‐mail: luyz_@uas.edu.mx. 25

Journal of Automation, Mobile Robotics and Intelligent Systems

Fabio Blanco-Mesa – Facultad de Ciencias Económicas y Administrativas, Escuela de Administración de Empresas, Universidad Pedagógica y Tecnológica de Colombia, Av. Central del Norte, 39‐115, 150001, Tunja, Colombia, e‐mail: fabio.blanco01@uptc.edu.mx. ∗

Corresponding author

References [1] J. Gil‐Aluja. Elements for a theory of decision in uncertainty, vol. 32, 1999, Springer Science & Business Media. [2] S. Siddique, U. Ahmad, and M. Akram. “A Decision‐Making Analysis with Generalized m‐ Polar Fuzzy Graphs,” Journal of Multiple-Valued Logic & Soft Computing, 37, 2021, 533–552. [3] R. W. Hamming. “Error detecting and error cor‐ recting codes,” The Bell system technical journal, 29(2), 1950, 147–160. [4] J. M. Merigó, and M. Casanovas. “Induced aggre‐ gation operators in the Euclidean distance and its application in inancial decision making,” Expert systems with applications, 38(6), 2011, 7603–7608. [5] H. Xu, W. Zeng, X. Zeng, and G. G. Yen. “An evolu‐ tionary algorithm based on Minkowski distance for many‐objective optimization,” IEEE transactions on cybernetics, 49(11), 2018, 3968–3979. [6] E. Szmidt, and J. Kacprzyk. “Distances between intuitionistic fuzzy sets,” Fuzzy sets and systems, 114(3), 2000, 505–518. [7] J. Bae, L. Liu, J. Caverlee, and W. B. Rouse. Process mining, discovery, and integration using distance measures, 2006, 479–488. [8] A. Kaufmann, J. G. Aluja, and J. M. F. Pirla. Introducción de la teoría de los subconjuntos borrosos a la gestión de las empresas, 1986, Milladoiro. [9] V. G. Alfaro‐Garcia, J. M. Merigó, A. M. Gil‐Lafuente, and J. Kacprzyk. “Logarithmic aggregation operators and distance measures,” International Journal of Intelligent Systems, 33(7), 2018, 1488–1506. [10] E. Avilés‐Ochoa, E. León‐Castro, L. A. Perez‐Arellano, and J. M. Merigó. “Government transparency measurement through prioritized distance operators,” Journal of Intelligent & Fuzzy Systems, 34(4), 2018, 2783–2794. [11] F. Blanco‐Mesa, J. M. Merigó, and J. Kacprzyk. “Bonferroni means with distance measures and the adequacy coef icient in entrepreneurial group theory,” Knowledge-Based Systems, 111, 2016, 217–227. [12] L. A. Perez‐Arellano, F. Blanco‐Mesa, E. Leon‐ Castro, and V. Alfaro‐Garcia. “Bonferroni prioritized aggregation operators applied to government transparency,” Mathematics, 9(1), 2021, 24. 26

VOLUME 17,

N∘ 2

2023

[13] W. Su, S. Zeng, and X. Ye. “Uncertain group decision‐making with induced aggregation oper‐ ators and Euclidean distance,” Technological and Economic Development of Economy, 19(3), 2013, 431–447. [14] G. Selvachandran, P. K. Maji, R. Q. Faisal, and A. Razak Salleh. “Distance and distance induced intuitionistic entropy of generalized intuitionis‐ tic fuzzy soft sets,” Applied Intelligence, 47(1), 2017, 132–147. [15] J. M. Merigó, and M. Casanovas. “Induced aggre‐ gation operators in the Euclidean distance and its application in inancial decision making,” Expert systems with applications, 38(6), 2011, 7603–7608. [16] B. Juang, and L. R. Rabiner. “A probabilistic dis‐ tance measure for hidden Markov models,” AT&T technical journal, 64(2), 1985, 391–408. [17] R. Şahin, and G. D. Küçük. “Group decision making with simpli ied neutrosophic ordered weighted distance operator,” Mathematical Methods in the Applied Sciences, 41(12), 2018, 4795–4809. [18] D.‐H. Peng, T.‐D. Wang, C.‐Y. Gao, and H. Wang. “Enhancing relative ratio method for MCDM via attitudinal distance measures of interval‐ valued hesitant fuzzy sets,” International Journal of Machine Learning and Cybernetics, 8(4), 2017, 1347–1368. [19] S. Xian, and W. Sun. “Fuzzy linguistic induced Euclidean OWA distance operator and its appli‐ cation in group linguistic decision making,” International Journal of Intelligent Systems, 29(5), 2014, 478–491. [20] R. R. Yager. “On ordered weighted averaging aggregation operators in multicriteria decision‐ making,” IEEE Transactions on Systems, Man, and Cybernetics, 18(1), 1988, 183–190. [21] J. Kacprzyk, R. R. Yager, and J. M. Merigo. “Towards human‐centric aggregation via ordered weighted aggregation operators and linguistic data summaries: A new perspective on Zadeh’s inspirations,” IEEE Computational Intelligence Magazine, 14(1), 2019, 16–30. [22] J. Kacprzyk, and S. Zadrożny. “Towards a general and uni ied characterization of individual and collective choice functions under fuzzy and non‐ fuzzy preferences and majority via the ordered weighted average operators,” International Journal of Intelligent Systems, 24(1), 2009, 4–26. [23] G. Beliakov, H. B. Sola, and T. C. Sánchez. A practical guide to averaging functions, 2016, Springer. [24] L. Canós, and V. Liern. “Soft computing‐based aggregation methods for human resource management,” European Journal of Operational Research, 189(3), 2008, 669–681.

Journal of Automation, Mobile Robotics and Intelligent Systems

[25] C.‐H. Cheng, J.‐W. Wang, and M.‐C. Wu. “OWA‐ weighted based clustering method for classi ica‐ tion problem,” Expert Systems with Applications, 36(3), 2009, 4988–4995. [26] M. Bernal, P. Anselmo Alvarez, M. Muñoz, E. Leon‐Castro, and D. A. Gastelum‐Chavira. “A multicriteria hierarchical approach for portfolio selection in a stock exchange,” Journal of Intelligent & Fuzzy Systems, 40(2), 2021, 1945–1955. [27] M. Muñoz‐Palma, P. A. Alvarez‐Carrillo, E. L. Miranda‐Espinoza, E. Avilés‐Ochoa, and E. León‐Castro, E. (2021). “Multicriteria Analysis Model for the Evaluation of the Competitiveness of the States in Mexico.” In Intelligent and Complex Systems in Economics and Business (pp. 1–20), 2021, Springer. [28] M. Olazabal‐Lugo, E. Leon‐Castro, L. F. Espinoza‐ Audelo, J. Maria Merigo, and A. M. Gil Lafuente. “Forgotten effects and heavy moving averages in exchange rate forecasting,” Economic Computation & Economic Cybernetics Studies & Research, 53(4), 2019. [29] E. León‐Castro, E. Avilés‐Ochoa, and J. M. Merigó. “Induced heavy moving averages,” International Journal of Intelligent Systems, 33(9), 2018, 1823– 1839. [30] V. Scherger, A. Terceño, and H. Vigier. “The OWA distance operator and its application in business failure,” Kybernetes, 2017. [31] L. Jin and R. Mesiar. “The metric space of ordered weighted average operators with distance based on accumulated entries,” International Journal of Intelligent Systems, 32(7), 2017, 665–675. [32] R. R. Yager. “Heavy OWA operators,” Fuzzy optimization and decision making, 1(4), 2002, 379–397. [33] L. Espinoza‐Audelo, E. Aviles‐Ochoa, E. Leon‐Castro, and F. Blanco‐Mesa. “Forecasting performance of exchange rate models with heavy moving average operators,” Fuzzy Economic Review, 24(2), 2019. [34] M. K. Evans. Practical business forecasting, John Wiley & Sons, 2002. [35] R. R. Yager, and J. Kacprzyk. The ordered weighted averaging operators: Theory and applications. Springer Science & Business Media, 2012. [36] J. M. Merigo, and R. R. Yager. “Generalized moving averages, distance measures and OWA operators.” International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 21(04), 2013, 533–559.

VOLUME 17,

N∘ 2

2023

HOWMA operators,” Economic Computation and Economic Cybernetics Studies and Research, 50(40), 2016, 135–150. [39] S. Zeng, W. Su, and A. Le. “Fuzzy generalized ordered weighted averaging distance operator and its application to decision making,” International Journal of Fuzzy Systems, 14(3), 2012, 402– 412. [40] L. Zhou, H. Chen, and J. Liu. “Generalized logarith‐ mic proportional averaging operators and their applications to group decision making”, Knowledge Based Systems, 36, 2012, 268–279. [41] R. R. Yager. “Families of OWA operators”, Fuzzy Sets and Systems, 59(2), 1993, 125–148. [42] J. M. Merigó, and A. M. Gil‐Lafuente. “The induced generalized OWA operator,” Information Sciences, 179(6), 2009, 729–741. [43] E. Anderson, and R. L. Oliver. “Perspectives on behavior‐based versus outcome‐based sales‐ force control systems.” Journal of Marketing, 51(4), 1987, 76–88. [44] C. F. Miao, and K. R. Evans. “The interactive effects of sales control systems on salesper‐ son performance: A job demands–resources per‐ spective,” Journal of the Academy of Marketing Science, 41(1), 2013, 73–90. [45] D. Baez‐Palencia, M. Olazabal‐Lugo, and J. Romero‐Muñoz. “Toma de decisiones empresariales a través de la media ordenada ponderada.” Inquietud Empresarial, 19(2), 2019, 11–23. [46] L. F. Espinoza‐Audelo, E. León‐Castro, M. Olazabal‐Lugo, J. M. Merigó, and A. M. Gil‐Lafuente. “Using Ordered Weighted Average for Weighted Averages In lation,” International Journal of Information Technology & Decision Making (IJITDM), 19(02), 2020, 601–628. [47] M. G. Velazquez‐Cazares, A. M. Gil‐Lafuente, E. Leon‐Castro, and F. Blanco‐Mesa. “Innovation capabilities measurement using fuzzy method‐ ologies: A Colombian SMEs case,” Computational and Mathematical Organization Theory, 27, 2021, 384–413. [48] M. Fedrizzi, and J. Kacprzyk. “An interactive multi‐user decision support system for consen‐ sus reaching processes using fuzzy logic with linguistic quanti iers.” Decision Support Systems, 4(3), 1988, 313–327. [49] E. Szmidt, and J. Kacprzyk. “A consensus‐ reaching process under intuitionistic fuzzy preference relations,” International Journal of Intelligent Systems, 18(7), 2003, 837–852.

[37] J. Kenney. Moving Averages. Princeton: Van Nos‐ trand, 1962. [38] E. León‐Castro, E. Avilés‐Ochoa, and A. M. Gil Lafuente. “Exchange rate USD/MXN forecast through econometric models, time series and 27

VOLUME 17, N∘ 2 2023 Journal of Automation, Mobile Robotics and Intelligent Systems

A COMPACT DQN MODEL FOR MOBILE AGENTS WITH COLLISION AVOIDANCE Submitted: 11th April 2023; accepted: 31st May 2023

Mariusz Kamola DOI: 10.14313/JAMRIS/2‐2023/13 Abstract: This paper presents a complete simulation and reinforce‐ ment learning solution to train mobile agents’ strategy of route tracking and avoiding mutual collisions. The aim was to achieve such functionality with limited resources, w.r.t. model input and model size itself. The designed models prove to keep agents safely on the track. Colli‐ sion avoidance agent’s skills developed in the course of model training are primitive but rational. Small size of the model allows fast training with limited computational resources. Keywords: Q‐learning, DQN, Reinforcement learning.

1. Introduction In order to accomplish a task, an unmanned agent must operate with adequate situational awareness and execute an adequate decision support algorithm. The complexity of both of the above components should be matched – be it a case of an autonomous car or a mere line‐following toy robot. The aim of work presented in this paper was to develop rich representation of road topology, yet cre‐ ate a representation suitable for processing by a rel‐ atively simple decision model of an autonomous and mobile agent. The agent’s main goal is to reach its des‐ tination, by riding on a road and taking turns, simul‐ taneously observing other moving agents in order to avoid collisions. The range of stimuli processed by the autonomous vehicle is determined by their physical observability, the cost and power consumption of measurement and communication equipment, and the cost and power consumption of hardware the decision model is run‐ ning on. Autonomous cars are nowadays by far the most sophisticated civil agents, equipped with LiDAR, radar, cameras, and GPS as well as a huge number of sensors collecting the state of the car itself. These real‐time data must be completed with accurate, up‐ to‐date and rich maps of the neighborhood in order to navigate ef iciently, which poses problems either with storage or bandwidth demand, depending on where the maps actually reside. In order to consume such rich input data streams in timely fashion, adequate processing power is needed. Tesla’s Autopilot processor, NVIDIA DRIVE PX2, consumes 250 watts, which is much more than the power of the car headlights. This igure does not 28

include powering all remaining sensors and commu‐ nication devices. Moreover, as for energy used for hardware manufacturing and operation, contempo‐ rary decision models carry substantial training carbon footprint. For example, the training of a natural lan‐ guage modern neural model consumes over 650 MWh of direct and associated energy (mainly cooling) [1]. The case of an autonomous car can be consid‐ ered an extreme one – yet striving to achieve planned goals but with smaller resources is more and more pronounced. Various neural model reduction tech‐ niques have been proposed [2], and the hardware itself becomes more energy ef icient, including the autonomous car computers as well. But still it is a common approach to throw all training data into a complex neural network model, use a great deal of resources to train it, and perform model compression as the last stage before the deployment. Here we propose an environment and decision model for an autonomous vehicle that uses less resources, due to careful encoding of the agent’s input. With such lean processing infrastructure, model train‐ ing is simple and ready for educational purposes as well as for further development for industry use. 1.1. Related Work Our work contributes to the wide and active area of research for autonomous driving, whose main and most challenging topic is control algorithms for self‐ driving cars. These can be developed in two contrary methodologies: the modular one, encompassing per‐ ception, planning, and control blocks, or the end‐to‐ end one, mingling the above functionalities into a single decision model. The state of the technology achieved through the two approaches is presented in detail in Grigorescu et al. [3], and we will point out key advances therein which are relevant to our research. Modules for perception, high‐ and low‐level planning, and motion control can be accomplished diversely in each class, adequately to available resources (hardware, data, power, money). For example, there are efforts to replace state‐of‐the‐art LiDAR sensors with cheaper 2D stereoscopic cameras and 3D reconstruction models (PointNet, AVOD), trained on real LiDAR measurements. Detected spatial objects can be labelled semantically afterwards with a bunch of models: SegNet, IC‐Net, ENet, and the like, derived from general computer vision architectures such as AlexNet, ResNet, and more.

2023 © Kamola. This is an open access article licensed under the Creative Commons Attribution-Attribution 4.0 International (CC BY 4.0) (http://creativecommons.org/licenses/by-nc-nd/4.0/)

Journal of Automation, Mobile Robotics and Intelligent Systems

Planning involves a range of tasks, starting from high‐level path planning and going down to low‐level behavior arbitration (lane control, collision avoidance, etc.) While path planning usually employs a sort of white‐box model working on map data and a local road occupancy grid, behavior arbitration models are neu‐ ral ones and come in two lavors. Deep reinforcement learning is built on simulated environment, develop‐ ing an optimal policy based on the reward function – yet the policy contains non‐learnable parts that are essential for maintaining vehicle safety. Interestingly, the opposite approach – imitation learning – aims actually to learn correct reward function from real scenarios by human drivers. Both approaches have drawbacks, suffering from data quality: simulation output inadequate to reality or scarcity of corner cases, respectively. Motion control is usually effected by state‐of‐the‐ art model predictive control (MPC) which, after having learned vehicle dynamics properties, takes advantage of optimal control theory in order to provide control signals on a considerable control horizon. Applied iteratively, it can adapt to disturbances and modify control trajectory computed so far. End‐to‐end methodology transfers knowledge from existing models, which makes it possible to consume raw and granular data, such as object locations, including the agent itself, and transform it by a multilayer network into decisions believed to be optimal. PilotNet by Nvidia and AutoPilot by Tesla are examples of such complex, monolithic models. Deep Q‐learning (DQN) is now the key planning algorithm, and also central to our interest here. It is presented in detail in Section 2.2. A neural model is taught to estimate the action‐value function Q of total rewards (immediate and discounted future ones) for a control decision, given the current system state. The framework has been enriched so far by a number of extensions, for instance [4]: ‐ Double Q‐learning: two network models are trained using different batch data, in order to reduce bias in Q estimation; ‐ Prioritized Replay: samples for training are drawn with probability relative to their temporal difference errors, for example, errors of predicted Q values, in order to improve the model where it performed worst; ‐ Dueling Networks: a model of Q gets accompanied with a model of V, a value of state regardless of con‐ trol taken there; both models are merged in order to focus learning on states where the control is really crucial; ‐ Multi‐step Learning: control action takes place only every n‐th step, letting the object evolve; it leads to faster learning in speci ic tasks; ‐ Distributional RL: reward value is modeled as dis‐ tribution over prede ined discrete space, allowing for more insight and resulting in faster training; the

VOLUME 17,

N∘ 2

2023

distribution gets updated by small corrections (Categorical DQN) or gets incorporated into the main model (Quantile Regression DQN; see [5]); ‐ Noisy Nets: a noisy input is added to the network model, allowing agents to learn states where the noise is relevant (high weights) for Q value. DQN models are often successfully extended to provide end‐to‐end driving functionality, as in [6], where raw camera images as well as vehicle speed and orientation w.r.t. road waypoints are the network inputs. To account for vehicle dynamics, the latter inputs go through recurrent layers (LSTM, long short‐ term memory), while visual ones go through typical convolution layers. The authors report good results, also if the visual part is trained from scratch, without transfer learning of any sort.

2. The Solution Let us assume that the goal is to provide an autonomous vehicle with situational data and equip it with a decision model so that it would be able to navigate a road system to reach a destination, while avoiding collisions with other agents. The basic task is therefore staying on the road and complying with traf ic rules. The secondary task is to interact with other vehicles to avoid collisions. Below, we present how topology and traf ic rules are prepared for the agent and describe the decision model used. 2.1. Road Topology Representation In wide range of transport environments, there are two components present: the physical infrastructure and traf ic rules imposed in it. We propose to merge them into a single map, where RGB colors encode both components. An example result of such process is pro‐ vided in Figure 1. Channels in red and blue (in range of luminance 0 to 255) are used to encode recommended speed in east‐west and north‐south directions, respec‐ tively. Speci ically, to make the map more human‐ friendly in appearance, we use only a part of this range. Thus, the horizontal component can vary from 150, which means “fast westward” to 250, meaning “fast eastward”. Luminance value of 200 is therefore the new zero in such a coordinate system. Movement gradients encoded on arbitrary R, G, or B channels are widely used elsewhere – such as in computer graphics, to visualize so‐called graphical lows [7]. The green channel, apparently super luous, has been used to encode indulgence in reward for driving according to the recommended speed and direction. This concept is strictly related to the class of deci‐ sion model discussed later, but it also makes practical sense in the phase of traf ic rules encoding. Consider the intersection that is located centrally in Figure 1. Without such a recommendation‐cancelling signal it would be impossible to encode driving directions for the four possible maneuvers there: two possible turns (north‐to‐west and west‐to‐north) as well as for just riding straight. Therefore, setting G luminance to zero there means no punishment, whatever particular rec‐ ommendations encoded in channels R and B are. 29

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

Figure 2. State components at agent location

Figure 1. Example of road system with color‐coded driving directions And on the opposite, the value of 255 denotes that ine or reward is applied in full. Considering the above rules, the colors of the plane in Figure 1 encode driving recommendations shown with arrows. In particular, the purple intersection RGB color is (200,0,200), that is, allowing unpunished driv‐ ing in any direction. The recommended speed vector has been set to zero there for clarity and fairness w.r.t. the four maneuvers. The off‐road parts color has been set to (200,255,200), which in essence results in max‐ imum punishment, regardless of the driving direction. In Figure 1 they have been given a dotted pattern for better igure readability. The proposed coloring approach carries quite a lot of information in just three channels. A part of the map in an agent’s neighborhood can be fed into the model either physically (if a camera‐equipped agent moves on such colored plane in a lab), or virtually (by retrieving it in real time from a database). It can consti‐ tute an extremely lean alternative to bulky, 3D urban maps, yet it can be created from information stored there. The sizes of current 3D maps for self‐driving cars are measured in terabytes: it is 4 TB in case of San Francisco, many orders of magnitude bigger than in our approach [8]. 2.2. Decision Model The decision model class used was a DQN, being a variant of the Q‐learning approach used widely when the state of an agent cannot be easily discretized [9]. DQN follows the original paradigm in the sense that, in a given state x of the agent, it provides the best known control 𝑎 – w.r.t. the immediate as well as the follow‐ ing steps by the agent. The difference is that while classical Q‐learning retrieves the value from a control‐ by‐state table, DQN actually calculates the discrete control on the spot. This brings considerable implica‐ tions to the learning procedure, described later. 30

We calculate the state information fed into DQN from the actual situation on the road and from the last step, so that it covers the following aspects: 1) agent’s current driving direction vs. recommenda‐ tions on the map, 2) indulgence to driving direction recommendations (channel G), 3) distance to the current goal, 4) presence of other agents or other moving obstacles. Figure 2 presents how the above state components get calculated at the current agent location. The agent is marked with a turtle icon because the graphics comes from Turtlesim simulator [10] used further for experimentation. Component 1 is denoted by two val‐ ues, colinear 𝑐 = 𝑣 cos 𝜑 − 𝜗 and perpendicular 𝑝 = 𝑣 sin 𝜑 − 𝜗, where 𝑣 is the driving recommendation speed (blue arrow) and 𝜑 − 𝜗 is the angle between agent’s actual and recommended driving azimuths, respectively. Together, 𝑐 and 𝑝 describe conformity of agent’s movement to the recommendations. Compo‐ nent 2, denoted 𝑔, is just the value of G channel of the map; component 3, denoted 𝑑, is the Euclidean distance to the current goal (an arbitrary location on the map, which can be via a point of a longer route). Components with signs are presented with arrows in Figure 2, although they are in fact scalar values. In reality, an agent, be it a robot, a driver, or a car, is able to collect the above data not only at its location point but also in the neighborhood – particularly in front of itself. To represent such situational awareness with the means provided so far, we decided to express components 1–4 not exactly at the current agent loca‐ tion, where some of them would be useless, but on a grid in front of it. Consequently, all components become matrices of size 𝑁 × 𝑁 elements, correspond‐ ing to cells of the grid, as shown in Figure 3. Therefore, component 1 values 𝑐 and 𝑝 become actually matri‐ ces C and P, calculated for average plane colors in cells. Coarse grid is used here analogously to pixelized vision organs of insects, which apparently do not hin‐ der them from performing high‐precision tasks, such as dragon ly interception skills [11]. Component 2 is calculated with the same averaging process, resulting in matrix G.

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

We found the approach of much educational value, and offering precise control on implementation details – and inally we preferred it over the popular OpenAI Gym framework [13]. One of the important implementation details is that two twin networks are used, the main network 𝑄 being subject to proper training, and the target network, denoted 𝑄, whose weights get updated from 𝑄 periodically in the pro‐ cess of learning. Consequently, 𝑄 evolves slowly and is used to predict quality of future actions. Therefore, (2) becomes 𝑄(x, 𝑎) = 𝑟(x, 𝑎) + 𝛾 max 𝑄(x′ , 𝑎′ ). ′

Figure 3. State components for a location grid

𝑎

Component 4 distances are calculated for grid cells’ centers, resulting in matrix D. Component 4 is introduced here to complete situational awareness, as binary matrix B, whose elements are set to 1 for cells that contain another agent. The above matrices comprise the current situa‐ tion, S = (C, P, G, D, B). Note that S is constructed so that it presents the neighborhood relative to current agent’s location and position, rather than revealing the location coordinates themselves. This forces the agent’s decision model to depend only on local situ‐ ation and imposes the desired degree of generaliza‐ tion by design. The agent cannot learn the plane “by heart,” remembering speci ic successful trajectories at certain locations. Rather than that, it learns desired behaviors for certain local situations, such as taking a turn on an intersection or passing a turning curve. Learning that behavior once, for one particular inter‐ section or curve, will make it able to cope with any number of similar objects, wherever they are on its route. DQN uses a neural network in order to evaluate Q value, that is, the quality of each of the possible actions, for a given state x of the agent. Then, in the greedy approach which we use, the action that gives best Q value is taken: 𝑢 = max 𝑄(x, 𝑎). 𝑎

(1)

The next detail lies in model input structure, that is, the agent state at step 𝑛. In our approach, such a state is composed not only of the current situation, but also the situation in the previous step, x(𝑛) = (S(𝑛), S(𝑛 − 1)). While such model input may seem super luous, it describes well the irst‐order system dynamics. It also neatly conforms to model input structure, which must be a tensor. Describing dynamic phenomena implicitly in neural network models is quite an established practice. Confront, for instance, two action‐recognition models, kinetics‐i3d [7] and TSM [14]. While object movement directions in input video are calculated separately for the earlier one, and provided explicitly on the input, the latter model infers them by time‐domain convolutions – which is exactly what we opt for. The reward 𝑟(x, 𝑎) and the next state x′ as the effect of action 𝑎 taken in state x, is provided in Q‐ learning by the environment. The environment can be the real word but, for the sake of model train‐ ing ef iciency, the most practical approach is to uti‐ lize some sort of a simulator or emulator. We used Turtlesim [10], with custom extensions made in order to calculate our situation components. The middle‐ ware between DQN routine and the simulator was an Environment class, implemented in Python. The class takes care of proper initialization and actual control of multiple agents, as well as of composing the situation S. The reward 𝑟 gets calculated as a sum of the follow‐ ing components:

The neural network behind Q is simultaneously used to direct an agent, and is trained with a history of recorded agent steps, so that it estimates as well as possible the intermediate consequences of an action:

𝜚𝑓 𝜁𝑟 𝜁𝑣 𝜚𝑡

𝑄(x, 𝑎) = 𝑟(x, 𝑎) + 𝛾 max 𝑄(x′ , 𝑎′ ), ′

𝜁𝑓

𝑎

(2)

where 𝑟(x, 𝑎) is the reward for the current step being taken, which leads to the next state x′ . The joint qual‐ ity of the following actions is again estimated by the model, and the best action is assumed to be taken, as in (1) – but with the discount coef icient 𝛾. In order to make the reinforcement learning de ined by (1) and (2) perform correctly, one has to take care with implementation details. We based our implementation on the framework reported in [12], which allows control of every implementation detail.

(3)

reward for driving according to recommendations penalty for driving reverse to recommendations penalty for speeding reward for approaching the current target (measured by velocity in straight line to target) terminal penalty for falling off the track or crashing with another agent.

All parameters but the last one are based on con‐ tinuous measurements of agent position or movement on the plane. DQN training procedure is, in essence, a standard one. An agent is let to play an episode, that is, perform a number of steps, being controlled by the current model 𝑄, while its steps get recorded as training sam‐ ples. Once there are enough samples memorized, one epoch of 𝑄 model training is performed every couple of episodes. After a number of such training sessions, 31

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

Figure 4. Neural network structure 𝑄 model is transferred onto 𝑄 and used for further decisions. Such a plain scheme can be, however, imple‐ mented with subtle differences regarding things such as initial behavior or extensions to a multi‐agent sce‐ nario, which is particularly our case and our contri‐ bution. All in all, DQN procedure has quite a large number of metaparameters, and full grid search could be prohibitive.

3. Experiments and Results Our structure of neural network for 𝑄 function modeling is shown with all variants in Figure 4. In its basic form it is similar to widely used classi ica‐ tion models used in, for instance, optical character recognition. The input contains road situation (blue) and collision structures (orange). The irst three layers of the main network perform 3D convolutions in 20 ilters, with convolution window of size 2 × 2 × 10, while the input size is 𝑁 × 𝑁 × 10 (the third dimension corresponds to 5‐tuple (C,P,G,D,B), taken twice, at the current step 𝑛 and the previous step, 𝑛−1. The last two layers are dense ones, and the output size is 2 × 3, that is, two values of speed (slow and fast) by three values of direction (turn left, drive straight, or turn right). Model training and evaluation has been performed on a track shown in Figure 3. The agent scenario is to approach targets 0 to 4, in a loop. Once an agent gets close to a target, it is given another one. The plane is colored accordingly, giving an agent hints about the general driving direction – save for the intersection where no driving preference is provided lest it divert any of traf ic streams that should pass the intersection straight ahead. The training is performed on four training seg‐ ments, corresponding to the targets 0–3. An agent is spawned on a track fragment within the rectangular 32

Figure 5. Location of training segments

areas, and directed towards the goal. Both the ini‐ tial position and orientation are subject to random disturbances. The training segments are chosen so that agents can learn skills typical to this track, such as driving straight and taking slight and perpendicular turns in both directions. Note that learning each type of turn is done only in one part of the track, but is expected to be utilized by the agent elsewhere on its route. Among many parameters, we decided to keep reward components constant, with values 𝜚𝑓 = 0.5 [sec/m], 𝜁𝑟 = 𝜁𝑣 = −10 [sec/m], 𝜚𝑡 = 2 [sec/m], 𝜁𝑓 = −10. Importantly, rewards 𝜚𝑓 and 𝜚𝑡 are small w.r.t. all penalties, therefore leaving much freedom of strategy to agents, unless they really violate rules imposed by the 𝜁’s. After extensive preliminary tests, we have found that solution quality is not much sensitive to

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

Table 1. Solution quality in various training and testing settings Training variant I II III

grid 5x5 6 agents 10 agents max mean max mean 0.64 0.34 0.53 0.38 3.27 0.70 0.63 0.27 5.27 1.07 0.61 0.28

grid 7x7 6 agents 10 agents max mean max mean 3.52 0.37 1.05 0.33 3.34 0.98 0.84 0.44 0.60 0.36 0.35 0.23

DQN‐related parameters. Consequently, we decided to keep them close to the values from the original project [12]. In particular, reward discount was set to 0.9, the memory buffer contained 20,000 recent steps, model 𝑄 was trained every 4 steps, and transferred onto 𝑄 after every twentieth training session. The experiments were designed to verify impor‐ tance of the training length and the size of agent’s loca‐ tion grid. The veri ication was measured by the mean number of loops done by an agent, until failure. It was calculated in variants with 1, 6, and 10 agents run simultaneously. The training for multiagent variants was done with 6 agents simultaneously, all learning the same uniform strategy. The learning in multiagent setting can be perceived as an extension of a simpler one‐agent scenario, where a part of the model handles interaction between agents. We trained multiagent models in three variants: I) Uniformly, by training from scratch. In such a case, the agents learn at the same time, traversing their routes and avoiding collision. II) By transfer of some part of a one‐agent model onto multiagent model. Selected weights in the irst convolution layer get initialized from the best single‐agent model, and frozen, while other weights undergo training as in approach I. Part of the model with the frozen weights has a grey color in Figure 4. III) By sequentially training a branched model, whose one branch processes only track‐related data and is similar to the uniform model (I). The other branch processes only collision‐related data, and its train‐ ing starts later, once the irst branch is trained well. The two branches are merged canonically in the second last dense layer. The other branch is drawn in the lower part of Figure 4. Comparison of model performance, in terms of average number of loops by an agent before failure, is given comprehensively in Table 1. Bold numbers rep‐ resent best results in a range of model training epochs (1500, 1750, and so on until 3000), while italics stand for their mean value. Models were trained for the same ield of view of 200 by 200 pixels – divided into grids of 5x5, 7x7, and 11x11 cells, respectively. Additionally, we ran performance checks on a more crowded track, for example, with 10 agents. The results consistently show that increasing the number of agents impairs overall performance. Other results are not consistent across training variants – especially the hypothetical improvement for iner grid resolutions. Interestingly, the solution quality

grid 11x11 6 agents 10 agents max mean max mean 3.05 1.53 1.07 0.65 1.71 0.71 0.55 0.28 4.66 0.78 0.62 0.47

Figure 6. Sample agent trajectories, variant IIIwith 6 agents and grid resolution 5x5 increase is observed clearly only for variant I, and in the mean sense ( igures in green in Table 1.) Otherwise, we may clearly point out that results considered best for models II and III de initely outper‐ form the plain and uniform one (I) – compare igures in red, columnwise. Let us examine the best result ever, obtained by variant III in 5x5 grid, and best mean result obtained by variant II in 7x7 grid. Sample agent traces for the earlier case are shown in Figure 6. We can observe that agents, started at various loca‐ tions, quickly converge to optimal trajectory. In a track section on the left to the intersection, where there is no apparent obstacle in view, agents tend to fan out and take alternative, equally optimal routes. No departures from the track are observed and the only scenario terminations are due to agent collisions. Most of them happen at the intersection or immedi‐ ately in front of it (which is a crash zone size artefact). In this setting, agents plainly did not develop any crash avoidance strategy. Occasional crashes in other parts of the track are result of respawns of agents that happen to take place in too close proximity of another agent and can be eas‐ ily eliminated by improving agent respawn procedure in the testing phase. Sample agent traces in the latter selected case are shown in detail in intersection area in Figure 7. Agents’ goal is to pass the intersection straight ahead. Here 33

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

AUTHOR Mariusz Kamola∗ – Institute of Control and Computation Engineering, Warsaw University of Technology, Warsaw, 00‐665, Poland, e‐mail: Mariusz.Kamola@pw.edu.pl. NASK – National Research Institute, Warsaw, 01‐045, Poland, e‐mail: Mariusz.Kamola@nask.pl. ∗

Corresponding author

ACKNOWLEDGEMENTS The author thanks Mr. Wojciech Dudek, PhD, Warsaw University of Technology, for careful development and maintenance of the Turtlesim simulator, and his advice and help in the course of research reported here. Figure 7. Collision avoidance, variant II with 6 agents and grid resolution 7x7 we can observe that developed strategy is to avoid collision by slowing down and diverging from own course – to either right or left. Skipping right results in almost immediate track fallout and episode termi‐ nation. The decision of skipping left is taken if agent has spotted another agent crossing the intersection slightly later. It turns left sharply, taking course paral‐ lel to it. Consequently, the agent is pushed off the track or strays left, departing from its goal and eventually being terminated.

4. Conclusion Here we presented a complete simulation and reinforcement learning environment, capable of train‐ ing autonomous mobile agents to reach their goals. Our main contributions are twofold. First, a plane col‐ oring scheme is proposed that ef iciently encodes both track shape and basic traf ic rules. It allows feeding the neural network model with very compact input, without need of transfer learning from big general pre‐ trained computer‐vision models. Second, three variants of network architectures and their training procedures are proposed and exam‐ ined, with the aim of somehow decoupling track con‐ trol and collision avoidance agent skills. Experiments show that model structure matters in this regard: for instance, model variant II clearly develops a primitive but rational strategy of avoiding collision by falling off the track. Further improvements, including some sort of self‐developed road code, would probably require reformulation of agent reward, and are a valuable topic for further research. The inherent compact size of the model was a design guideline in our work. Being not a result of quantization or any other compression of a much larger model, it can be used directly by agents not only in the execution phase but also in training or updating the model itself. Practical veri ication of this claim is also an important research direction. 34

References [1] E. Strubell, Ananya Ganesh, and Andrew McCal‐ lum. “Energy and Policy Considerations for Deep Learning in NLP,” Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019. doi: 10.48550/arXiv.1906.02243. [2] Y. Cheng, et al. “Model compression and acceler‐ ation for deep neural networks: The principles, progress, and challenges.” IEEE Signal Processing Magazine vol. 35, no. 1, 126–136, 2018. doi: 10.48550/arXiv.1710.09282. [3] S. Grigorescu, et al. “A survey of deep learning techniques for autonomous driving,” Journal of Field Robotics, vol. 37, no. 3, 362–386, 2020. [4] M. Hessel, et al. “Rainbow: Combining improvements in deep reinforcement learning,” Proceedings of the AAAI conference on arti icial intelligence, vol. 32, no. 1, 2018. [5] W. Dabney, et al. “Distributional reinforcement learning with quantile regression,” Proceedings of the AAAI Conference on Arti icial Intelligence, vol. 32, no. 1, 2018. [6] M. Ahmed, C. P. Lim, and S. Nahavandi. “A Deep Q‐Network Reinforcement Learning‐Based Model for Autonomous Driving,” 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), IEEE, 2021. [7] J. Carreira, and A. Zisserman. “Quo vadis, action recognition? a new model and the kinetics dataset,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. doi: 10.48550/arXiv.1705.07750. [8] “How do self‐driving cars know their way around without a map?”, https://bigthink.com/techn ology‐innovation/how‐do‐self‐driving‐cars‐ know‐ their‐ way‐ around‐ without‐ a‐ map/ (accessed 2023.03.31). [9] M. Sewak. “Deep Q Network (DQN), Double DQN, and Dueling DQN: A Step Towards General Arti‐ icial Intelligence,” Deep Reinforcement Learning:

Journal of Automation, Mobile Robotics and Intelligent Systems

Frontiers of Arti icial Intelligence 2019, 95–108. doi: 10.1007/978‐981‐13‐8285‐7_8. [10] W. Dudek, N. Miguel, and T. Winiarski. “SPSysML: A meta‐model for quantitative evaluation of Simulation‐Physical Systems,” arXiv preprint arXiv:2303.09565 (2023). doi: 10.48550/arXiv. 2303.09565. [11] F. S. Chance. “Interception from a Dragon ly Neu‐ ral Network Model,” International Conference on Neuromorphic Systems, 2020.

VOLUME 17,

N∘ 2

2023

[12] “Self‐driving cars with Carla and Python,” https: //pythonprogramming.net/introduction‐ self‐driving‐autonomous‐cars‐carla‐python (accessed 2023.03.31). [13] OpenAI Gym homepage, https://openai.com/res earch/openai‐gym‐beta (accessed 2023.03.31). [14] J. Lin, C. Gan, and S. Han. “TSM: Temporal shift module for ef icient video understanding.” Proceedings of the IEEE/CVF international conference on computer vision, 2019.

VOLUME 17, N∘ 2 2023 Journal of Automation, Mobile Robotics and Intelligent Systems

APPLICATION OF THE SPHERICAL FUZZY DEMATEL MODEL FOR ASSESSING THE DRONE APPS ISSUES Submitted: 25th July 2022; accepted: 3rd November 2022

Mamta Pandey, Ratnesh Litoriya, Prateek Pandey DOI: 10.14313/JAMRIS/2‐2023/14 Abstract: During the past few years, the number of drones (unmanned aerial vehicles, or UAVs) manufactured and purchased has risen dramatically. It is predicted that it will continue to spread, making its use inevitable in all walks of life. Drone apps are therefore expected to overrun the app stores in the near future. The UAV’s software is not being studied/researched despite several active research and studies being carried out in the UAV’s hardware field. A large‐scale empirical analysis of Google Play Store Platform apps connected to drones is being done in this direction. There are, however, a number of challenges with drone apps because of the lack of formal and specialized app development procedures. In this paper, eleven drone app issues have been identified. Then we applied the DEMATEL (Decision Making Trial and Evaluation Laboratory) method to analyze the drone app issues (DIs) and divide these issues into cause and effect groups. First, multiple experts assess the direct relationships between influential issues in drone apps. The evaluation results are presented in spherical fuzzy numbers (SFN). Secondly, we convert the linguistic terms into SFN. Thirdly, based on DEMATEL, the cause‐effect classifications of issues are obtained. Finally, the issues in the cause category are identified as DI’s in drone apps. The outcome of the research is compared with the other variants of DEMATEL, like rough‐Z‐number‐ based DEMATEL and spherical fuzzy number, and the comparative results suggest that spherical fuzzy DEMA‐ TEL is the most fitting method to analyze the interrela‐ tionship of different issues in drone apps. The findings revealed that highest influenced values feature request (DI9 ) 3.12, Customer support (DI6 ) 2.91, Connection/Sync ((DI4 ) 2./72, Cellular Data Usage ((DI3 ) 2.51, Battery (DI2 ) 2.31, Advertisements ((DI1 ) – 0.3, Cost (DI5 ) – 0.5, Additional cost (D11 ) – 0.5, Device Compatibility (DI7 ) – 0.96, and Functional Error (DI10 ) – 1.2. The outcome of this work definitely assists the software industry in the successful identification of the critical issues where professionals and project managers could really focus. Keywords: Drone apps, Issues, Multi‐criteria decision making, Spherical fuzzy DEMATEL.

1. Introduction As the name suggests, an unmanned aerial vehicle (UAV) is an aircraft that does not have any human pilot on board [1]. Software‐controlled light plans 36

in embedded systems, together with onboard sen‐ sors and GPS, allow drones to be navigated from the ground [2]. A majority of small UAVs employ lithium‐ polymer batteries, while bigger vehicles are powered by plane engines [3]. Cameras are available on sev‐ eral of these drones, allowing the operator to record video or take photos [4]. Licensed pilots are in charge of these drones. Many people like lying drones as a hobby. Drones are also capable of carrying a wide variety of sensors and can go to places where most IoT devices cannot. Predicting the weather, replacing traf‐ ic cameras, spotting forest ires, scanning buildings and landscapes for agricultural and structural health monitoring, and conducting search and rescue opera‐ tions are just some of the many uses for drones [5]. There has been an explosion of drone mobile apps since the introduction of smart drone technology, which allows drones to communicate with onboard computers, data collection devices, smartphones, and the cloud [6]. These programs can be used to con‐ trol and navigate drones, as well as provide a variety of applications that can be used to perform complex tasks autonomously [7]. There has been an increase in mobile app distribution platforms like the Google Play store and the IOS store as a result of the increasing availability of free and open software development kits and online APIs for drone hobbyists. By 2028, the drone mobile app market is anticipated to be worth USD72320 million [8]. Drone software is a new topic that demands extensive research, yet there are no earlier works on drone apps for the App Store. In this article, a signi icant number of Google Play Store drone apps will be studied [9, 10]. The paper’s goal is to identify the most common complaints from mobile drone app users, as well as the developer’s response and the time it takes to respond. Mobile app development is a very new and vibrant ield compared to traditional software development, which is a fairly mature industry [11, 12]. The size, cost, time required for development, and user inter‐ face speci ications for mobile apps are also different from those for traditional software [13]. Because of this, traditional methods of software development are inappropriate for creating mobile applications. This issue is made worse by the lack of formal methodolo‐ gies for app development. However, many of the app analytics offered thus far are not software‐engineering focused. On the other hand, recognizing the causes and effects that affect the mobile app rating can give

2023 © Pandey et al. This is an open access article licensed under the Creative Commons Attribution-Attribution 4.0 International (CC BY 4.0) (http://creativecommons.org/licenses/by-nc-nd/4.0/)

Journal of Automation, Mobile Robotics and Intelligent Systems

project managers a clear understanding of this strat‐ egy and assist them in making decisions under pres‐ sure [14, 15]. The current situation causes a number of prob‐ lems with the creation of apps. Users of the apps have reported these problems on associated distribution sites. Recent studies have focused on app dif iculties based on user feedback or experiences, which can pro‐ vide signi icant information, such as complaints about functionality, privacy concerns, feature requests, and so on [16, 17]. The issues with the drone app form an inter‐ connectivity since these issues are connected to one another and have an effect on the ratings of the app. It’s possible to employ a strategy that involves collec‐ tive decision‐making in order to igure out how these dif iculties are connected to one another. Due to the inherent dif iculties of human experts in communi‐ cating their thoughts or their choices, the linguistic analysis of the decisions made by experts is prefer‐ able when measured quantitatively. To overcome the inherent ambiguity associated with language, a fuzzy variant of DEMATEL for group decision‐making is used. It helps in classifying drone app issues in cause‐ and‐effect groups, to help project managers improve the quality of their decision‐making. The spherical fuzzy‐DEMATEL approaches have been used to solve the complex group decision‐making problems such as strategic planning, e‐learning evolution, and R & D [18, 19]. This paper is further organized as follows: Sec‐ tion 2 describes the literature review on software engineering for mobile user reviews and the associ‐ ated issues in the context of drone apps. Section 3 describes the research methodology. Section 4 elab‐ orates the experimental setup. The research process is discussed in Section 5. Results are discussed in Sec‐ tion 5. Section 6 describes the threats to validity, and Section 7 contains the conclusion and scope of future work.

2. Literature Survey Two subsections make up the literature review: work that leveraged mobile user reviews, and work focusing on drone apps. 2.1. Work Leveraging Mobile User Reviews Research on mobile app reviews was pioneered by Harman et al. [11]. In this paper, the author has investigated the correlation between user reviews and number of downloads. They concluded that develop‐ ers should keep an eye on their user ratings, since they have a substantial association with the amount of downloads. Finkelstein et al. used natural lan‐ guage processing (NLP) tools to examine the link between an app’s customer rating, its pricing, its popularity (based on downloads), and the promised attributes retrieved from each app’s description [12]. Researchers observed a substantial link between app popularity and customer ratings, as well as a modest link between app pricing and features promised [13].

VOLUME 17,

N∘ 2

2023

User reviews were mined for information by researchers in other studies. A study by Nikolas et al. examined low‐rated user evaluations of iOS apps in order to assist mobile app developers better under‐ stand their behavior [14]. According to their research, the most common causes for bad reviews were demands for new fea‐ tures, functional issues, and applications that crashed, whereas privacy and ethical concerns accounted for the majority of the evaluations that had the greatest in luence on an app’s rating. In another study, research has examined the in luence of privacy and ethical issues on the user evaluations of 59 Android appli‐ cations [15]. According to their research very small quantity of the apps have privacy and ethical issues. Hoon and Vasa [16, 17] examined the vocabulary of 8.7 million user reviews from the iTunes Store and found a correlation between review length and the rating it received from the reviewers. In another work, Latent Dirichlet Allocation (LDA) was used to analyze more than 12 million user reviews of over a hundred thousand applications in the Google Play Store [18]. In addition to uncovering 10 distinct concerns, they also discovered a major difference between free and paid applications, since premium apps commonly pro‐ vide a complaint issue concerning the related cost, which is missing in user evaluations of free ones. NLP methods were used to automatically extract the most useful evaluations from a database of mobile appli‐ cations [19]. Only 35.1% of the app reviews included useful information that developers may utilize for app enhancement, since the number of reviews is some‐ times too vast for humans to read or comprehend. Thus, their methodology automates a technique for iltering, grouping, ranking, and visualizing just the relevant parts of the evaluations. Up to 30% of mobile app evaluations may include various themes of infor‐ mation; McIlroy et al. suggested an automated tech‐ nique for categorizing user reviews, which attained an accuracy of 66% and a recall rate that was 65% for 13 distinct categories [20]. Recent research by McIlroy et al. (2015) found that after a response to an app review, users would boost the review rat‐ ing 38.7% of the time by 20% [21]. Using NLP, text analysis, and sentiment analysis methods, Panichella et al. suggested a taxonomy of four categories relevant to software maintenance and evolution activities in order to identify app user evaluations, and a method to automatically classify them [22]. The authors used machine learning to integrate these approaches and tested their classi iers experi‐ mentally, demonstrating that their method may help developers glean information about user intent from feedback [23]. A program called ARdoc automates the categorization of user reviews. Three real‐world mobile app developers and an outside software engi‐ neer veri ied the tool’s performance. With accuracy, recall, and F‐Measure scores ranging from 84% to 89%, ARdoc performed well [24]. Users’ evaluations of mobile applications may be analyzed using a model called User Reviews Model (URM), which was developed by Di Sorbo et al. [25]. 37

Journal of Automation, Mobile Robotics and Intelligent Systems

It was used in conjunction with Panichella et al. to create Summarizer of User Reviews Feedback (SURF), a novel method for capturing the intent of user review‐ ers [26]. In order to propose software improvements, SURF creates summaries from sets of user reviews and groups them based on both, the goal and the subjects discovered in user reviews. In another study, 17 mobile applications were tested by 23 developers and researchers to see whether this method worked. Further, Di Sorbo et al. designed and veri ied SURF to help developers automate the handling of user reviews. Recently, a mobile‐speci ic taxonomy was developed by manu‐ ally analyzing 1,566 user evaluations from 39 mobile applications conducted by Ciurumelea et al. Using the URR (User Request Referencer) methodology, the authors developed a method for automatically classi‐ fying user reviews in their multi‐level taxonomy and then directing developers to the speci ic artefacts that must be adjusted to answer a given user review, as well. They found that processing user reviews by hand might be sped up by up to 75% [27]. As an alterna‐ tive technique, Palomba et al. developed CHANGEAD‐ VISOR, which gathers together various user evalua‐ tions that include change requests and recommends to developers which artefacts to edit in a mobile app in order to meet user input [28]. In this method, reviews are sorted by their content, semantics, and structure using Natural Language Processing (NLP). A valida‐ tion with the creators of ten mobile applications has shown the utility of this technique for mining huge numbers of user reviews, delivering 81% accuracy and 70% recall when advising modi ications. This vali‐ dation was carried out. Additionally, there are several additional mobile app projects that make use of user evaluations to improve their work. However, Martin et al. has compiled a thorough collection of research on mobile applications, and we encourage the reader to check it out [29]. 2.2. Drone Apps Issues The creation of drone applications must inevitably involve some form of software development that is equally applicable to the creation of mobile applica‐ tions. The elicitation of the requirement is indicated to be of utmost importance in the context of the devel‐ opment of drone apps from the perspective of require‐ ment engineering [23]. As a preface, we’ll go over the typical scenario of a drone app being developed in the software business, followed by a discussion of relevant literature on drone apps. In this article, we describe an approach for the systematic examination of issues that makes use of the collaborative efforts of groups of people and collective decision‐making. It gives a distinct viewpoint by rec‐ ognizing the reciprocal interaction among concerns, which, in turn, would aid in comprehending the causal relationship that exists between the issues that have been recognized. It would, in turn, aid in comprehend‐ ing the causal relationship among the issues that have been found. 38

VOLUME 17,

N∘ 2

2023

As the foundational technology upon which to con‐ struct our analysis, we make use of Spherical Fuzzy‐ DEMATEL in addition to other variations of DEMA‐ TEL, such as Rough‐Z – number based‐DEMATEL. The Spherical Fuzzy‐DEMATEL approach is a well‐ established way for determining the interconnections that exist between the many issues or components that are present in a certain setting. This strategy has been implemented in a variety of professional ields, including management, mechanical engineer‐ ing, chemical engineering, and software engineering. As of right now, not a single researcher has given any attention to its use in the area of drone app development. Research has shown that drone app development is a relatively new area compared to other forms of software development. The quality of a drone app may be determined by the number of complaints/issues it is exposed to. The more laws a drone app has, the worse its quality. We’ve seen how the Spherical Fuzzy‐DEMATEL approach may be used in a variety of contexts and believe it’s appropriate for use here in analyzing the interrelationships between various aspects of drone apps. The list of issues in drone apps are presented in Table 1. The DEMATEL method focuses on analyzing the interrelationships between variables and identifying the crucial ones using a visual structural model. Numerous researches on the use of DEMATEL have been conducted in the last ten years, and numerous distinct variations have been proposed in the liter‐ ature. DEMATEL analyses provide better outcomes than existing methods. Interpretive structure model‐ ing, structure equation modeling, and so on, cannot perform this task.

3. Methodology Drone app issues are a dif icult challenge to solve since there are so many interconnected concerns. This means that a multi‐criteria decision‐making (MCDM) approach is needed to evaluate drone app concerns, since it takes compromises and competing objectives into consideration. Interpretive structural modelling (ISM) and analytical hierarchy process (AHP) are two of the most often utilized MCDM methodologies in research. The DEMATEL technique is superior to other multi‐criteria decision‐making techniques such as ISM and AHP because it provides an overall degree of in luence of factors or issues, it divides the factors into cause‐and‐effect groups, and it establishes causal relationships [30]. Additionally, the DEMATEL technique provides an overall degree of in luence of factors or issues. The incorporation of fuzzy logic into the DEMATEL methodology accounts for the hazy and imprecise information that is inherently associated with human judgements [31]. The problem of drone apps is inves‐ tigated using the Spherical Fuzzy‐DEMATEL method‐ ology in this particular piece of research. A number of different industries, including management, informa‐ tion technology, and manufacturing, have all made use of the combination of Spherical Fuzzy‐DEMATEL [32].

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

Table 1. List of issues in drone apps [39–41] S. No. 1

Drone apps issues (DI) Advertisements (DI1 )

Battery (DI2 )

Cellular Data Usage (DI3 )

Connection/Sync (DI4 ) Cost (DI5 )

Customer support (DI6 )

Device Compatibility (DI7 )

Device Storage (DI8 )

Feature request (DI9 )

Functional Error (DI10 )

Additional cost (D11 )

Description & references This type was also prevalent mostly in mobile games. Ads are a waste of time for most people. After purchasing an in‐app purchase to eliminate the adverts, some users say that the ads persist. Users of the app report that their smartphone battery drains quickly. Some services may have been running even after the app was closed, using more battery power. However, a few people have reported issues with drone batteries. While the battery was completely charged, it dropped to a dangerously low level as soon as the drone took off, according to the report. In the background, the app transmits a lot of mobile data, but the user is unaware of the content since the app is not under their control. Some users report that they can only connect to their drone after turning off mobile data. Drone connection issues or loss of sync during light have been reported by certain users, resulting in a loss of live streaming video feeds and/or telemetry data. The price of the app is a point of contention among users. The majority of those who were unable to use the software requested a refund. The software works OK, but some customers say it’s not worth the money they spent for it, even if it does what it says. According to a few, the premium applications were not worth the money since there were free alternatives with equal features. Customers of the Google Play Store complain about the lack of assistance for the app when they contact the company. In spite of a few negative reviews, customer service appears to be doing a good job. According to some, they didn’t get a reply at all. Some customers have also expressed dissatisfaction with customer service. There are reports of customer service responding in a manner that smacks of incompetence. The app’s lack of compatibility with a user’s phone or operating system is a common complaint. These compatibility issues may have arisen due to OS version upgrades, such as when a user changed their Android OS and the app stopped operating, or it might be a hardware incompatibility, such as the app not supporting a certain cell model. Users have reported that the software is too large and takes up too much space on their phone, or that it fails to identify the space and produces a no storage space error message while attempting to save drone‐captured ilms or photographs. This criticism was largely about entertainment, video players, and photography. Feature removal is seldom criticized. Users disliked some software features and suggested removing them. Some reviewers complained that a favorite feature was eliminated. Requests for new features from customers are solicited. Users want the app to be able to handle a wider range of drones, while others want to be able to download trial versions of the software. Many users have expressed dissatisfaction with the way some features operate or don’t work at all. “Video recording on my Galaxy S8 but no sound” review complains about the audio functionality not working. Authorization and registration dif iculties, as well as not being able to go beyond the login screen, have all been reported by several users. The user complains about the hidden cost to enjoy the full experience of the app.

In this article, we discuss a technique for evalu‐ ating the aspects that affect whether or not a drone application is successful. The methodology is illus‐ trated in Figure 1 and explanation of of each major stage of this methodology is given. Acquisition (A): In order to discover areas for improvement, it is necessary to collect data so that we can run several quantitative and qualitative pro‐ cedures to acquire a better understanding of the situation. Identi ication (ID): Data collected in ”A” is critical to identifying potential issues which affect the app rating. We will conduct quantitative and qualitative data analysis based on the collected data. Data can also be converted from a qualitative to a quantitative form.

Relationship Analysis (RA): The number of issues that have been detected in ”ID” can range from a few to a signi icant number. It is expected that no problems will arise as a result of one’s independence and isolation from others. In other words, a problem will have an impact on, or may have an impact on, a number of other problems. As a result, it is critical to do thorough research in order to ind connections between issues. Interpretation (I): It is possible to draw conclu‐ sions from the ”RA” analysis.

4. Experimental Setup Section 3’s methodology is used to put up an exper‐ iment in this section. 39

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

Figure 1. Proposed framework of our research study 4.1. Data Collection and Text Processing

4.2. Issues Identification

We gathered user feedback on a variety of drone apps by polling various online forums. Nearly 120,000 user reviews were gathered. About 45% of all reviews were determined to be useful after the text cleaning procedure. Consequently, there were around 50,000 reviews available for study. We then used Radian 6, a social media listening tool, to do text processing on certain chosen keywords in order to identify distinct types of customer complaints in various mobile appli‐ cation categories. After that, we grouped comparable types of user complaints into problems. Information sources and text‐processing tools are outlined in the following sec‐ tions: 1) Collection of user reviews: We gathered cus‐ tomer/users feedback from a variety of app shops, including the Google Play Store, BlackBerry App Store, and others.

After applying clustering methods to the text data, we found that all of the words that were examined could be classi ied, in a general sense, as belonging to one of the 11 categories of problems shown in Table 1. For instance, the term “request” refers to an absent feature (the request for a new feature), while the keyword “drain” is associated with the consumption of the user’s battery (energy consumption issues). It has been revealed that these problem categories are compatible with indings from past research carried out in the ield of mobile app development.

2) Filtration of non‐English terms and slang: Words like ”kool,” ”nhi,” etc. were cleaned up after going through this process. 3) Text processing: In order to parse the text of tes‐ timonials, POS tagger and word2vec are applied. For example, after iltering and analyzing the data, we discovered that terms like ”the battery,” ”slow,” ”login,” ”hot,” and so on occurred rather often. 40

4.3. Issues Relationship Analysis To put it another way, issues don’t just appear on their own; they interact with one other. Creating a system without boundaries is practically impossible. There must be some sort of a tradeoff in order to exe‐ cute drone software well, and this can be altered to suit the needs of different app types and goals. Because of this, it is critical to comprehend the interrelationships between various types of issues. To identify the issues and other factors that affect the system, Fuzzy‐DEMATEL employs an expert opinion‐based approach [8]. We used the eight‐step DEMATEL method to examine the impact of app issues on each other. First and foremost, we need to identify

Journal of Automation, Mobile Robotics and Intelligent Systems

the problems, which we’ve done in Section 4.2; how‐ ever, the problems that we’ve discovered are in line with those that have been reported in other studies, as shown in Table 2. When it comes to mobile app issues, we used Spherical Fuzzy‐DEMATEL and compared the results to other DEMATEL variants, such as DEMATEL and DEMATEL. Sections 4.4, 4.5, and 4.6 describe the DEMATEL, Spherical Fuzzy‐DEMATEL, and rough‐Z‐ number–based DEMATEL, respectively. Section 4.8 illustrates how Spherical Fuzzy‐DEMATEL can be used to address the problems that have been identi ied. 4.4. DEMATEL Fontela and Gabus invented the DEMATEL approach (the methodology of the Decision‐Making Trial and Evaluation Laboratory) [33]. This is an ef icient way to examine the direct and indirect relationships between the components of a complex system [34]. The foundation of DEMATEL is graph theory, which allows for an ef icient analysis of all system relations as well as the construction of a map between various systems components [35]. The interdependence between components may be better understood by studying the total‐relation matrix [36]. There are several domains where DEMATEL has been effectively used, including supply chain [37], risk assessment [38], service quality analysis [39], management [40], and so on. The fundamental phases of the DEMATEL technique for making judgments in evidence theory are as follows, laying the groundwork [41]: Step 1 – Prepared pair‐wise comparison scale Step 2 – Construction of initial direct relation matrix Step 3 – Construction of normalized direct relation matrix Step 4 – Construction of total relation matrix Step 5 – Separation of sum of row and sum of columns in total relation matrix Step 6 – Calculation of prominence and relevance vector Step 7 – Analysis of interrelationship 4.5. Rough‐Z‐number A rough‐Z‐number is proposed to combine the advantages of the Z number and the rough num‐ ber in order to improve the manipulation capability of uncertainty, reliability, and subjectivity for MCDM applications [42]. This is motivated by the Z‐number’s advantage in representing individual risk assess‐ ment’s uncertainty and reliability and the rough num‐ ber’s advantage in manipulating subjectivity among different evaluation values [43]. The Z‐number is introduced to replace the fuzzy part of the fuzzy rough number in the Z‐number [44]. Alternatively, the Z‐number and rough number can be integrated into it. In general, a Z‐number is an ordered pair of fuzzy numbers; however, this is not always the case. It can be ̃ [45, 46]. Initially, as denoted by the symbol 𝑧 = (𝐴,̃ 𝐵)

VOLUME 17,

N∘ 2

2023

a fuzzy constraint on the values, component 𝐴̃ is used, whereas component 𝐵̃ is a measure of reliability for component A. As a result, component B is used as a measure of reliability for the initial component 𝐴.̃ In the following example, a Z‐number is translated into a classical fuzzy number, which is detailed below. ̃ ∫ 𝑎𝜇𝐵(𝑎)𝑑𝑎 𝛼= (1) ̃ 𝜇𝐵(𝑎)𝑑𝑎 Where ∫ symbol indicates an algebraic integration. A weighted Z‐number is calculated by multiplying the dependability component by the restriction compo‐ nent. The irregular fuzzy number can be converted into regular fuzzy number as following – 𝑎 , 𝑎𝜖[0, 1] √𝛼

𝑧̃ = < 𝑎, 𝜇𝑧 (𝑎) > |𝜇𝑧 (𝑎) = 𝜇𝑧

(2)

4.6. Spherical Fuzzy Sets (SFS) The following is a summary of certain terminology and fundamental operations that are necessary for the present research. The References section contains descriptions of all of the operations that were created for SFSs [47]. De inition 1: An SFS 𝐴̃ 𝑠 for the universe of discourse X may be found by using the formula: 𝐴̃ 𝑠 = {(𝐴, 𝜇𝐴̃ 𝑠 (𝐴), 𝜗𝐴̃ 𝑠 (𝐴) 𝜋𝐴̃ 𝑠 (𝐴)|𝑎𝜖𝐴)}

(3)

Where 𝜇𝐴̃ 𝑠 (𝑎) = 𝐴 → [0, 1], 𝜗𝐴̃ 𝑠 (𝑎)∶ 𝐴 → [0, 1]𝜋𝐴̃ 𝑠 (𝑎)∶ 𝐴 → [0, 1] And 0 <=, 𝜇𝐴2̃ (𝑎) + 𝜗𝐴2̃ (𝑎) + 𝜋𝐴2̃ (𝑎) <= 1∀ 𝑎 𝜖 𝐴 𝑠

𝑠

For each a, 𝜇𝐴̃ 𝑠 , 𝜗𝐴̃ 𝑠 and 𝜋𝐴̃ 𝑠 are referred to as the membership degree and the hesitancy degree of a to 𝐴̃ 𝑠 . Some numerical operations were con‐ structed by studying the relationships between SFS and Pythagorean fuzzy set (PFS). De inition 2: Let A1 and A2 be two distinct uni‐ verses, respectively, and 𝑋̃ 𝑠 = (𝜇𝑋̃ 𝑠 , 𝜗𝑋̃ 𝑠 , 𝜋𝑋̃ 𝑠 ) and 𝑌𝑠̃ = (𝜇𝑌𝑠̃ , 𝜗𝑌𝑠̃ , 𝜋𝑌𝑠̃ ) be two different spherical fuzzy sets from the universe of discourse D1 and D2 . The fundamental operations are described below. 𝑋̃ 𝑠 ⊕ 𝑌𝑠̃ = {(𝜇𝑋2̃ + 𝜇𝑌2̃ − 𝜇𝑋2̃ 𝜇𝑌2̃ )1/2 , 𝑠

𝑠

𝜗𝑋̃ 𝑠 𝜗𝑌𝑠̃ , ((1 − 𝜇𝑌2̃ )𝜋𝑋2̃ 𝑠 𝑠 + (1 − 𝜇𝑋2̃ )𝜋𝑌2̃ − 𝜋𝑋2̃ 𝜋𝑌2̃ )1/2 } 𝑠

𝑠

(4)

Multiplication: 𝑋̃ 𝑠 ⊗ 𝑌𝑠̃ = {𝜇𝑋 𝜇𝑌𝑠̃ , (𝜗𝑋2̃ + 𝜗𝑌2̃ − 𝜗𝑋2̃ 𝜗𝑌2̃ )1/2 , 𝑠

𝑠

((1 − 𝜗𝑌2̃ )𝜋𝑋2̃ + (1 − 𝜗𝑋2̃ )𝜋𝑌2̃ − 𝜋𝑋2̃ 𝜋𝑌2̃ )1/2 } 𝑠 𝑠 𝑠 𝑠 𝑠 𝑠 (5) We can de ine spherical weighted arithmetic mean (SWAM) for aggregation purpose as follows: V = (v1 , v2 ,…Vn ) where v is de ined as a weight and 𝑛 v𝑖 𝜖 [0,1] and ∑𝑖=1 𝑣𝑖 = 1 𝑆𝑊𝐴𝑀𝑣 = 𝑣1 𝑋̃ 𝑠1 + 𝑣2 𝑋̃ 𝑠2 + ⋯ 𝑣𝑛 𝑋̃ 𝑠𝑛

(6) 41

Journal of Automation, Mobile Robotics and Intelligent Systems

Table 2. Language (linguistic term) and spherical fuzzy numbers Linguistic term No in luence weak Moderate Strong

Abb NI W M S

𝜇 0 0.35 0.6 0.85

𝜗 0.3 0.25 0.2 0.15

𝜋 0.15 0.25 0.35 0.45

SI 0 1 2 3

N∘ 2

2023

in luence matrix. T’s matrix form may be found in Equation (9) 0 ⎡ ⎢ 𝜇21𝑇 ,𝜗𝑇 𝜋𝑇 21, 21 T=⎢ … ⎢ ⎢ 𝑇 𝜋𝑇 ⎣𝜇𝑛1𝑇 ,𝜗𝑛1, 𝑛1

𝜇12𝑇 ,𝜗𝑇 𝜋𝑇

12, 12

0 … 𝜇𝑛2𝑇 ,𝜗𝑇 ,𝜋𝑇 𝑛2

𝑛2

… 𝜇1𝑛𝑇 ,𝜗𝑇 𝜋𝑇

1𝑛, 1𝑛 ⎤ … 𝜇2𝑛𝑇 ,𝜗𝑇 𝜋𝑇 ⎥ 2𝑛, 2𝑛 ⎥ … … ⎥ ⎥ … 0 ⎦ (9)

4.7. Spherical Fuzzy‐DEMATEL

Step 5: Calculation of initial direct in luence matrix.

In traditional DEMATEL [48], experts’ hesitation is ignored. This is the irst time that the Spherical Fuzzy version of DEMATEL [49, 50] has been described in the literature for tackling this issue [51]. As a demon‐ stration of the method’s applicability, a case study is provided in the following section.

The aggregated direct in luence matrix (T) must be divided into three submatrices since there are three factors in each comparison. Depending on the degree of membership, non‐membership, and uncertainty of the group being studied, distinct matrices may be used for matrix operations. It will be possible to gener‐ ate the irst matrix of direct impact by normalizing and recombining each of these matrices in the same matrix. In Equation, the normalizing procedure is car‐ ried out (10). In matrix form, the result of this step is indicated in Equation (10).

Step 1: Attribute identi ication and selection of sub‐ ject/ area expert Assumptions are made that there are m specialists in the ield of decision‐making who will weigh in on the issue, as well as other factors that should be taken into account. It’s important to understand the decision‐makers’ degrees of competence and the rationale for their selection. In the same way, the selec‐ tion of attributes should be justi ied, for example, by citing scienti ic studies in the linked literature or using them in real‐world industrial scenarios. There should be an adequate representation for each attribute that contains sub‐attributes. Step 2: Construction of direct in luence matrix An expert, or decision‐makers, are questioned about their preferences or opinions on the effect of attributes on the decision‐making process. The score index (SI) for corresponding values is described in the Eq. (7) [52] SI =

|100 ∗ [(𝜇 − 𝜋) − (𝜗 − 𝜋) ]|

(7)

Step 3: Calculation of experts/decision maker’s weight. It is presumed that m decision‐makers have his/her weight, which represents his/her experience and degree of knowledge. Spherical fuzzy describes the nth expert presented their decision Dn = (𝜇𝑛 , 𝜗𝑛 , 𝜋𝑛 ) and weigh coef icient can be calculated by Eq. (8) [53] 1 − {(1 − 𝜇𝑛 )2 + 𝜗𝑛2 + 𝜋𝑛2 )/3

𝛼𝑛 =

Σ𝑛 (1 −

(8)

2 {(1 − 𝜇𝑛 )2 + 𝜗𝑛2 + 𝐵𝜋𝑛 )/3

𝑚

Where ∑𝑛=1 𝛼n = 1 and 0 <= 𝜇𝑛2 + 𝜗𝑛2 + 𝜋𝑛2 <= 1. Step 4: Calculation aggregation relation matrix T. The spherical weighted arithmetic mean (SWAM) is used to combine the various decision‐makers’ direct impact assessment matrices. After this procedure, the result is the T matrix, which aggregates the direct 42

VOLUME 17,

0 ⎡ ⎢ 𝜇21 𝐴𝜇 = ⎢ … ⎢ ⎣𝜇𝑛1 0 ⎡ ⎢𝜗2𝑛 𝐴𝜗 = ⎢ … ⎢ ⎣𝜗𝑛1 0 ⎡ ⎢𝜋2𝑛 𝐴𝜋 = ⎢ … ⎢ 𝜋 ⎣ 𝑛1

𝜇12

…

𝜇1𝑛

𝜇𝑛2

⎤ 𝜇2𝑛 ⎥ ; … …⎥ ⎥ … 0 ⎦

𝜗12

…

𝜗1𝑛

𝜗𝑛2

⎤ 𝜗2𝑛 ⎥ ; … …⎥ ⎥ … 0 ⎦

𝜋12

…

𝜋𝑛2

…

0 …

…

𝜋1𝑛

⎤ 𝜋2𝑛 ⎥ … ⎥ ⎥ 0 ⎦

(10)

Step 6: Calculation of total in luence matrix (Ti ). (𝜇 ) ⎡ 11,𝜗11 ,𝜋11 ) ⎢ (𝜇 Ti = ⎢ 21,𝜗21 ,𝜋21 … ⎢ (𝜇 ⎣ 𝑛1,𝜗𝑛1 ,𝜋𝑛1 )

(𝜇12,𝜗12 ,𝜋12 )

…

… (𝜇𝑛2,𝜗𝑛2 ,𝜋𝑛2 )

… …

(𝜇1𝑛,𝜗1𝑛 ,𝜋1𝑛 )

⎤ 𝜇2𝑛𝑇 ,𝜗𝑇 ,𝜋𝑇 ⎥ 2𝑛 2𝑛 ⎥ … ⎥ 0 ⎦ (11)

Step 7: Computation of Spherical Fuzzy row and columns sums. The sum of row and columns of Spherical Fuzzy num‐ bers can be obtain by using Eqs. (12) and (13). R𝑖 = Σ𝑛𝑖=1 (𝜇𝑖𝑗𝑇 ,𝜗𝑇 ,𝜋𝑇 )

(12)

𝑛 C𝑗 = Σ𝑗=1 (𝜇𝑖𝑗𝑇 ,𝜗𝑇 ,𝜋𝑇 )

(13)

𝑖𝑗

And Spherical Fuzzy numbers can be defuzzi ied by using Eq. (14) Sd = (2𝜇 − 𝜋)2 − (𝜗 − 𝜋)2

(14)

Journal of Automation, Mobile Robotics and Intelligent Systems

Step 8: Evaluation of prominence and relative vector and preparing network relation map 1) Research Process: The main low of our study is depicted in Figure 1 and explained in detail after‐ ward. Application of Spherical Fuzzy‐DEMATEL on Wearable apps issues analysis Step 1: Attribute identi ication and selection of sub‐ ject/area expert Assumptions are made that there are m specialists in the ield of decision‐making who will weigh in on the issue, as well as other factors that should be taken into account. It’s important to understand the decision‐makers’ degrees of competence and the rationale for their selection. In the same way, the selec‐ tion of attributes should be justi ied, for example, by citing scienti ic studies in the linked literature or using them in real‐world industrial scenarios. There should be an adequate representation for each attribute that contains sub‐attributes. Step 2: Construction of direct in luence matrix An expert or decision‐makers are questioned about their preferences or opinions on the effect of attributes on the decision‐making process. Step 3: The decision‐makers’ competence and knowl‐ edge level are represented by the weights they are assigned. SF depictions of the decision‐makers are provided N1 = (0.6, 0.2, 0.3), N2 = (0.7, 0.4, 0.1) and N3 = (0.5, 0.5, 0.5), Eq. (7) is used to calculate the weight coef icients. The weight of the irst decision‐ maker is calculated as follows: 𝛼1 =

1 − {(1 − 0.6)2 + 0.22 + 0.32 )/3 2 ⃓ {(1 − 0.6 )2 + 0.22 + 0.32 )/3 ⃓ ⃓ ⃓ ⎛ 2 ⃓ + (1 − {(1 − 0.7 )2 + 0.42 + 0.12 )/3 1 − ⃓ ⎜ ⎜ ⃓ ⃓ ⃓ 2 + (1 − {(1 − 0.5 )2 + 0.52 + 0.52 )/3 ⎝ ⎷

= 0.36 Similarly, we can compute weight of other decision makers. Step 4: The aggregation relation matrix is calculated using Eq. (8) 1/2

10 𝑛 2 𝛼𝑛 (1 − (𝜇𝑖𝑗 ) )

𝑇 𝜇12 = 1− 𝑛=1

= 0.87 10 𝑛 𝜗𝑖𝑗 = 0.25

𝑇 𝜗12 = 𝑛=1 𝑚

𝑛 2 𝛼1 (1 − 𝜇𝑖𝑗 ) ) −

𝑇 𝜋12 =

1/2

𝑚

𝑛=1

= 0.48

𝑛 𝑛 2 𝛼1 (1 − 𝜇𝑖𝑗 ) − 𝜋𝑖𝑗 ) ) 𝑛−1

VOLUME 17,

N∘ 2

2023

We will now put into practice the steps of the approach that were outlined in Sections 3.1 and 3.2 in order to determine the extent to which certain mobile app issues are connected to one another, as follows: Step 6: Prominence and relative vectors have been calculated. Sum of rows and columns of total relative matrix is known as prominence and relative vectors respectively [53]. Step 7: The values of prominence and relative vectors have been defuzzi ied using Eq. (14) and resultant values are mentioned in Table 7. 4.8. Assessment Results have been examined using MAE (in Table 9), which provides a numerical measure of how closely two issues are related in terms of their prominence and relative vector. By calculating uncertainty and fuzziness, MAE is able to handle it. MAE =

1 𝑁

𝑁

|𝑤𝑦𝑖 − 𝑤𝑛𝑖 |

(15)

𝑖=1

Where N represents the number of in luenc‐ ing issues, 𝑤𝑦𝑖 represents the values of positive side of issues, 𝑤𝑛𝑖 represents the value of negative side of issue. Table 9 displays the issues MAE after being com‐ puted using both the positive (prominence) and nega‐ tive (relative) sides of the argument. As a result, the lower MAE shows that issues calculated from both the positive and negative sides are more compara‐ ble, based on Spherical Fuzzy‐DEMATEL, than rough Z number–based‐DEMATEL and DEMATEL to deter‐ mine the relationship between cause and effect. To put it another way, SFN‐DEMATEL has the abil‐ ity to improve the overall well‐being of a problem in mobile apps when compared to most existing solu‐ tions. Comparing SFN‐DEMATEL to rough Z number– based‐DEMATEL and DEMATEL, the latter is better suited to identify the issues in drone apps that inher‐ ently have a language assessment process since it bet‐ ter addresses subjectivity. SFN‐DEMATEL was found to be more accurate than rough Z number based– DEMATEL and DEMATEL when it comes to identifying issues in drone app development. 4.9. Comparative Analysis of Spherical Fuzzy–DEMATEL with Rough Z Number–based DEMATEL and DEMATEL There are a few studies on drone app development that have been offered in the literature. However, a handful of these strategies take into account the rela‐ tionships between the most important issues in drone apps and explain why selecting the appropriate issue to improve is drone app. Spherical Fuzzy‐DEMATEL has the ability to obtain the whole relationship between in luential concerns and identify the issues in a drone app, unlike existing approaches. The optimization of the drone app can be reduced to simply ixing the issues that the Spherical Fuzzy‐DEMATEL has revealed. 43

Journal of Automation, Mobile Robotics and Intelligent Systems

N∘ 2

VOLUME 17,

2023

Table 3. Expert evaluation on influences DI1 DI2 DI3 DI4 DI5 DI6 DI7 DI8 DI9 DI10 D11 DI1 0 M; S; M M; M; W M; M; M M; S; M S; S; NI M; S; S S; W; M S; M; NI M; NI; S W; M; S DI2 S; S; NI 0 S; S; M M; S; M M; M; M M; M; NI M; S; M M; M; NI M; S; M S; S; M S; S; M DI3 M; NI; M S; W; M 0 M; M; NI M; S; M S; M; M S; S; NI S; NI; NI S; W; M M; S; M W; M; S DI4 S; S; M S; S; NI W; M; S 0 M; NI; M M; S; M W; M; S M; M; M W; M; S M; M; NI M; S; M DI5 M; M; M M; S; M M; NI; M M; S; M 0 S; S; NI S; S; M M; S; M S; S; M M; M; S W; M; S DI6 M; M; NI W; M; S S; W; M S; S; NI W; M; S 0 M; S; M S; W; M S; NI; NI S; W; M S; S; M DI7 S; W; M S; S; M S; NI; NI M; S; M S; W; M S; S; M 0 S; S; M S; S; NI S; M; M M; S; M DI8 M; M; S M; S; M M; M; M M; NI; M S; NI; NI M; M; S S; W; M 0 S; M; M S; S; NI M; S; M DI9 W; M; S S; M; M M; S; M S; S; NI S; S; M M; S; M M; NI; M M; M; S 0 S; NI; NI M; M; M DI10 M; S; M M; M; S S; M; M W; M; S M; S; M M; M; M S; NI; NI W; M; S M; NI; M 0 M; S; M D11 M; S; M M; NI; M M; S; M M; M; NI M; S; M M; NI; M M; S; M M; S; M M; M; NI M; S; M 0

Table 4. Aggregation direct influence matrix DI1 DI2 DI3 DI4 DI5 DI6 DI7 DI8 DI9 DI10 D11

DI1 0 0.5 0.476 0.672 0.7867 0.4057 0.601 0.397 0.381 0.763 0.8166

DI2 0.4236 0 0.7179 0.8109 0.1965 0.3263 0.2262 0.8102 0.4215 0.1604 0.1965

DI3 0.2279 0.3217 0 0.4251 0.5631 0.8419 0.7451 0.1332 0.7621 0.7642 0.8101

DI4 0.8751 0.5571 0.8664 0 0.9164 0.4251 0.2254 0.8723 0.6623 0.6523 0.9106

DI5 0.5543 0.2529 0.8166 0.3434 0 0.3425 0.3255 0.1261 0.1212 0.1238 0.8723

DI6 0.6181 0.7134 0.1261 0.2279 0.2529 0 0.8101 0.3255 0.9201 0.1238 0.2529

DI7 0.7175 0.9106 0.8101 0.3223 0.1261 0.6212 0 0.2746 0.2005 0.3005 0.7206

DI8 0.2465 0.3903 0.7206 0.9202 0.6309 0.6801 0.5235 0 0.4262 0.8016 0.8723

DI9 0.9164 0.4251 0.8723 0.6623 0.476 0.672 0.8101 0.3223 0 0.7134 0.8166

DI10 0.8109 0.1965 0.2529 0.8166 0.7134 0.1261 0.5571 0.8101 0.3223 0 0.1261

D11 0.9106 0.8101 0.2529 0.8166 0.3434 0.8166 0.672 0.7867 0.4057 0.9202 0

DI3 0.8166 0.4236 0 0.1442 0.2746 0.2465 0.3903 0.7206 0.9202 0.6309 0.4215

DI4 0.1261 0.2279 0.1261 0 0.1261 0.9164 0.4251 0.8723 0.6623 0.476 0.5543

DI5 0.8101 0.8751 0.3255 0.381 0 0.8109 0.1965 0.2529 0.8166 0.7134 0.7621

DI6 0.7206 0.5543 0.2746 0.4215 0.763 0 0.7134 0.1261 0.2279 0.2529 0.1261

DI7 0.8723 0.6181 0.1261 0.7621 0.1604 0.7175 0 0.8101 0.3223 0.1261 0.8101

DI8 0.2529 0.4236 0.3255 0.6623 0.7642 0.412 0.2279 0 0.2746 0.2005 0.9201

DI9 0.8664 0.7621 0.2746 0.1212 0.6523 0.2005 0.8751 0.2746 0 0.2005 0.3255

DI10 0.8166 0.6623 0.1261 0.9201 0.1238 0.8166 0.4236 0.1261 0.1212 0 0.2746

D11 0.6181 0.7134 0.8101 0.3223 0.1261 0.9164 0.4251 0.6181 0.1261 0.7621 0

Table 5. Total influence matrix DI1 DI2 DI3 DI4 DI5 DI6 DI7 DI8 DI9 DI10 D11

DI1 0 0.473 0.243 0.541 0.1261 0.6181 0.7134 0.1261 0.2279 0.2529 0.7621

DI2 0.8664 0 0.3843 0.449 0.3255 0.7175 0.9106 0.8101 0.3223 0.1261 0.1212

Table 6. Computation of prominence and relational values r DI1 DI2 DI3 DI4 DI5 DI6 DI7 DI8 DI9 DI10 D11

𝝁 0.7435 0.79 0.831 0.7209 0.7177 0.3223 0.1261 0.1261 0.3255 0.2529 0.7621

𝝑 0.0135 0.0196 0.032 0.0184 0.0187 0.1261 0.2746 0.8664 0.3255 0.2529 1.1264

c 𝝅 0.5651 0.5266 0.5235 0.4786 0.4886 0.9106 0.9106 0.9106 0.2746 0.8664 0.2529

Score 1.0567 0.8723 0.6181 1.1264 0.397 0.2529 1.8101 0.397 1.5623 0.5651 0.1261

𝝁 0.8312 0.8114 0.8125 0.8105 0.8304 0.2529 0.8166 0.9106 0.9106 0.8664 0.6001

𝝑 0.0186 0.0167 0.0176 0.0166 0.0186 0.1261 0.6309 0.3903 0.3903 0.8166 0.476

𝝅 0.6 0.6001 0.6002 0.6001 0.6002 0.8101 0.476 0.4251 0.4251 0.2746 0.8101

Score 1.056 0.476 0.672 0.8101 0.8419 0.3255 0.8419 0.2529 1.7664 0.6623 0.2529

r+c

r–c

2.16 3.57 2.84 1.97 2.95 2.77 1.98 2.62 2.21 1.57 0.1261

‐0.3 2.31 2.51 2.72 ‐0.5 2.91 ‐0.96 2.5 3.12 ‐1.2 ‐0.5

Journal of Automation, Mobile Robotics and Intelligent Systems

Table 7. Rank factors of cause and effect issues Drone apps issues DI1 DI2 DI3 DI4 DI5 DI6 DI7 DI8 DI9 DI10 D11

Rank 6 5 4 3 7 2 8 5 1 9 7

Cause

Effect

DEMATEL

DI1 DI6 DI4 DI3 DI2 DI11 DI9 DI5 DI7 DI10 DI8

Rough Z number based DEMATEL DI9 DI6 DI4 DI3 DI8 DI11 DI1 DI5 DI7 DI10 DI2

Spherical Fuzzy DEMATEL DI9 DI6 DI4 DI3 DI8 DI11 DI1 DI5 DI7 DI10 DI2

Table 9. The MAE of issues importance calculated from positive side, negative side DEMATEL

MAE

0.32

Rough Z number DEMATEL 0.29

Spherical Fuzzy DEMATEL 0.15

As a result, optimizing these dif iculties can have a positive impact on the overall success of drone apps. Spherical Fuzzy‐DEMATEL vs. rough Z number– based‐DEMATEL: When attempting to describe lin‐ guistic phenomena, a fuzzy system is the most effective method, as opposed to a Rough z number based DEMATEL, which might lead to misunderstand‐ ing or misinterpretation. Spherical Fuzzy‐DEMATEL vs. DEMATEL: In the evaluation of linguistic words, the evidence theory is not well suited for hypothesizing that each compo‐ nent of a discernment frame must be mutually exclu‐ sive, even though both Spherical Fuzzy‐DEMATEL and DEMATEL are capable of handling the subjectivity of expert evaluations. Therefore, as compared to rough Z number–based DEMATEL and DEMATEL, Spherical Fuzzy‐DEMATEL is more relevant to identi ication of the issues in drone apps which inherently have a linguistic assessment.

N∘ 2

2023

Table 10. Pair‐wise comparison of the importance of DEMATEL, rough Z number–based DEMATEL, and Spherical Fuzzy‐DEMATEL using the Spearman correlation coefficient DEMATEL

Table 8. Cause‐effect classification and ranking of issues using DEMATEL, Rough Z number DEMATEL and Spherical Fuzzy –DEMATEL Category

VOLUME 17,

DEMATEL Rough Z number based DEMATEL Spherical Fuzzy DEMATEL

Spherical Fuzzy DEMATEL

1 0.7709

Rough Z number based DEMATEL 0.7708 1

0.8873

0.8709

0.8174 0.7544

This is because rough Z number–based‐DEMATEL and DEMATEL do not take into account fuzzy logic. Table 8 presents a cause‐and‐effect breakdown of various issues using Spherical Fuzzy‐DEMATEL, rough Z number–based DEMATEL, and DEMATEL as a basis. Drone app issues identi ied by Spherical Fuzzy‐DEMATEL, rough Z number–based‐DEMATEL, and DEMATEL all include the identical DI9 as well as the other four DI’s discovered by Spherical Fuzzy‐ DEMATEL: DI6 , DI4 , DI3 , and DI8 . Furthermore, the Spearman rank correlation coef‐ icient is calculated between each pair of methods in Table 10 to accurately re lect the similarity of promi‐ nence and relative vectors of methods. As a result of the fact that a higher Spearman cor‐ relation coef icient indicates a more signi icant rank‐ ing similarity of methods, the ranking of similarity of Spherical Fuzzy‐DEMATEL is higher up on the list than either of the other two methods.

5. Results and Discussion The conclusion of this research indicates com‐ plex interdependencies between numerous issues and underlines the multifaceted nature of drone apps. As a result, the impact and sensitivity of the issues cat‐ egories suggest that they may be classi ied as cause‐ and‐effect issues. These indings might be used by the developer of the drone apps to determine which issues demand urgent care and which ones can be put off for a later time. The indings of this study might have a signi icant impact on the quality of drone apps and their popularity among app users. Developers may utilize the indings in this research to enhance the entire process of their app development, even if the study does not explicitly provide a solution. Issues with drone apps are shown in a causal‐effect diagram (Figures 2(a) and 2(b)). Figure 2(a) shows the impact of various kind of issues on a drone app’s rating. With the strength of 3.12, Feature request (DI9 ) seems to be most in luential issue, whereas battery consumption (DI2 ) is the most in luenced issue, with strength of −1.2. 45

Journal of Automation, Mobile Robotics and Intelligent Systems

3.5

VOLUME 17,

N∘ 2

2023

Promenence vector

3 2.5 2 1.5 1 0.5 0

-0.5 -1 Relative vector

-1.5

Figure 2(a). Cause‐effect order diagram of drone app issues using Spherical Fuzzy‐DEMATEL

3.5

Prominence vector

3 2.5 2 1.5 1 0.5 0 1

-0.5

Relative vector -1 -1.5 Spherical fuzzy DEMATEL

Rough z number based DEMATEL

DEMATEL

Figure 2(b). Comparative cause‐effect order diagram of drone app issues

The most in luential drone app issue is repre‐ sented by the issue with the highest rank, while the issue with the lowest rank has the greatest in luence. Figure 2(b) depicts the comparison of DEMATEL, Z number‐based DEMATEL, and Spherical Fuzzy‐ DEMATEL. The Spherical Fuzzy‐DEMATEL strategy has fared the best out of the three methods. Prominence and relative vectors are both represented by positive and negative values, respectively. Few studies in the literature have already looked at various aspects of drone apps. The importance of examining the interrelationship among different issues of drone app development must be exam‐ ined in order to effectively apply drone app develop‐ ment methods. Drone app issues are intertwined in important ways. This work will assist project man‐ agers to make better drone app development choices. The causal diagram provides useful interrelationship 46

information that can be used to plan the app’s launch and deploy it as quickly as feasible.

6. Threats to Validity There are a few potential threats with this research. In speci ic, our study only includes 15 different issues, but in the future, we may include many more issues. Incorporating the indings of this study into multi‐criteria decision‐making approaches, such as ANP and fuzzy ANP, might be helpful in determining which approach would be most suited to investigate the drone app’s issues.

7. Conclusion Developing drone apps has become an essential element of software development. Due to a lack of formal scienti ic methods for the creation of drone apps, many issues arise in the corporate world. Drone

Journal of Automation, Mobile Robotics and Intelligent Systems

app development, unlike its predecessors, web apps and desktop applications, is amateurish and in the midst of an evolutionary process. There are several issues with drone applications, and this study aims to identify them. Using the technique described here, every strategic choice in the creation of a drone app may be veri ied to ensure that the quality of the app is not compromised. This study might assist software companies in determining the main cause of app issues. A total of eleven issues with drone apps have been identi‐ ied in this research. These issues, as was indicated before, are: advertisement (DI1 ), battery consump‐ tion (DI2 ), cellular data usage (DI3 ), connection/ sync (DI4 ), cost (DI5 ), customer support (DI6 ), device com‐ patibility (DI7 ), device storage (DI8 ), feature request (DI9 ), functional error (DI10 ), and additional cost (DI11 ). The Spherical Fuzzy‐DEMATEL approach was used to divide these problems into cause‐and‐effect groups after recognizing them. As previously noted, the reason group includes functional errors, feature requests, connection and sync, user interfaces, bat‐ tery drainage, performance, and security. Aside from costs, uninformativity, and app crashing, there are eight other dif iculties in the impact category, such as device compatibility issues and installation problems. When Spherical Fuzzy‐DEMATEL is compared to other techniques such as DEMATEL and Z number‐based DEMATEL, a superior result is achieved. Although the results of prominence and relative vectors are sim‐ ilar, the two quantitative values of these two view‐ points are clearly distinct. It was necessary to measure performance using the mean absolute error (MAE). According to the indings, “feature request” is the most signi icant issue associated with drone apps, although “battery consumption” is the issue that is most in lu‐ enced by the other issues. Since this investigation and its indings are founded on the accumulated expertise of the software engineering experts, gaining further experience will have the impact of reducing the in lu‐ ence of biases. In the future, research might focus on inding and adding new problems caused by drone applications, and then iguring out how the effects of these new problems interact with those already known.

List of Abbreviations AHP ANP DEMATEL DI GPS IoT ISM LDA MAE MCDM NLP

Analytical Hierarchy Process Analytical Network Process Decision Making Trial and Evaluation Laboratory Drone app issues Global Positioning System Internet of Things Interpretive structural modelling Latent Dirichlet Allocation Mean Absolute Error Multi‐Criteria Decision‐Making Natural Language Processing

VOLUME 17,

PFS SFN SFS SWAM SURF UAV URR URM

N∘ 2

2023

Pythagorean fuzzy Set Spherical fuzzy numbers Spherical fuzzy Set Spherical Weighted Arithmetic Mean Summarizer of user reviews Feedback Unmanned Aerial Vehicles User Request Referencer User Reviews Model

AUTHORS Mamta Pandey – Mahatma Gandhi Institute of Technology, Hyderabad, India, e‐mail: mamtapandey_cse@mgit.ac.in. Ratnesh Litoriya∗ – Medi‐Caps University, Indore (Madhya Pradesh), India, e‐mail: litoriya.ratnesh@gmail.com. Prateek Pandey – Jaypee University of Engineer‐ ing and Technology, Guna (Madhya Pradesh), India, e‐mail: pandeyprat@yahoo.com. ∗

Corresponding author

References [1] P. A. Devi, C. I. Priyadarshini, C. Avvari. “Design of folded wing mechanism for Unmanned Aerial Vehicle (UAV),” Materialstoday. 2022. Proceed‐ ings in press, doi: 10.1016/j.matpr.2022.04.660. [2] S. Bhusal, M. Karkee, U. Bhattarai, Y. Majeed, and Q. Zhang, K. “Automated execution of a pest bird deterrence system using a programmable unmanned aerial vehicle (UAV),” Computers and Electronics in Agriculture, 2022, 198, 106972. [3] N. Eleftheroglou, S. Mansouri, T. Loutas, P. Karvelis, G. Georgoulas, G. Nikolakopoulos, D. Zarouchas. “Intelligent data‐driven prognostic methodologies for the real‐time remaining use‐ ful life until the end‐of‐discharge estimation of the Lithium‐Polymer batteries of unmanned aerial vehicles with uncertainty quanti ication,” Applied Energy, 2019, 254, 113677. [4] S. G. Bolanos, A. Q. Roman, and G. E. Alvarado. “Low‐cost UAV applications in dynamic tropical volcanic landforms,” Journal of Volcanology and Geothermal Research, 2021, 410, 107143. [5] G. Wang, Y. Han, X. Li, P. Chen, W. C. Hoffmann, X. Han, S. Chen, and Y. Lan. “Field evaluation of spray drift and environmental impact using an agricultural unmanned aerial vehicle (UAV) sprayer,” Science of the Total Environment, 2020, 737, 139793. [6] R. Kumar, and A. K. Agrawal. “Drone GPS data analysis for light path reconstruction: A study on DJI, Parrot & Yuneec make drones,” Forensic Science International: Digital Investigation, 2021, 38, 301182. [7] E. Gallo, and A. Barrientos. “Reduction of GNSS‐ Denied inertial navigation errors for ixed wing 47

Journal of Automation, Mobile Robotics and Intelligent Systems

autonomous unmanned air vehicles,” Aerospace Science and Technology, 2021, 120, 107237. [8] “Unmanned Aerial Vehicle (UAV) Market Size to Reach USD 72320 Million by 2028 at a CAGR of 14.4%” | Valuates Reports.

N∘ 2

2023

[20] M. Pandey, R. Litoriya, and P. Pandey. “Val‐ idation of existing software effort estimation techniques in context with mobile software applications,” Wireless Personal Communications, 110(4), 2020, 1659–1677.

[9] https://www.prnewswire.com/in/news‐releas es/unmanned‐aerial‐vehicle‐uav‐market‐size‐ to‐reach‐usd‐72320‐million‐by‐2028‐at‐a‐ca gr‐of‐14‐4‐valuates‐reports‐870953616.html.

[21] M. Pandey, R. Litoriya, and P. Pandey, “Identify‐ ing causal relationships in mobile app issues: An interval type‐2 fuzzy DEMATEL approach,” Wireless Personal Communications, 108(2), 2019, 683–710.

[10] K. Andersen, M. H. Frederiksen, M. P. Knudsen, and A. D. Krabbe. “The strategic responses of start‐ups to regulatory constraints in the nascent drone market,” Research Policy, 49(10), 2020, 104055.

[22] M. Pandey, R. Litoriya, and P. Pandey. “Impact of various critical factors on Mobile App develop‐ ment based on reviews: An Investigative Study,” Technia, 9(1), 2016, 1097–1105.

[11] M. Pandey, R. Litoriya, and P. Pandey. “Identify‐ ing causal relationships in mobile app issues: An interval type‐2 fuzzy DEMATEL approach,” Wireless Personal Communications, 108(2), 2019, 683–710. [12] M. Pandey, R. Litoriya, & P. Pandey. “Mobile APP development based on agility function,” Ingénierie des Systèmes d’Information, 23(6), 2018. [13] S. McIlroy, N. Ali, H. Khalid, and A. E. Hassan. “Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews,” Empirical Software Engineering, 21(3), 2016, 1067–1106. [14] M. Pandey, R. Litoriya, and P. Pandey. “Perception‐based classi ication of mobile apps: A critical review,” Smart computational strategies: Theoretical and practical aspects, 2019, 121–133.

VOLUME 17,

[23] M. Pandey, R. Litoriya, and P. Pandey. “Mobile APP development based on agility function Mobile APP development based on agility func‐ tion.” [24] M. Harman, Y. Jia, Y. Zhang. “App store mining and analysis: Msr for app stores,” In: Proceedings of the 9th IEEE working conference on mining software repositories, MSR ’12. IEEE Press, 2012, pp. 108–111. [25] A. Finkelstein, M. Harman, Y. Jia, W. Martin, F. Sarro, Y. Zhang. “Investigating the relation‐ ship between price, rating, and popularity in the Blackberry world app store,” Inf Softw Technol 87, 2017, 119–139. [26] G. Lee, and R. Santanam. “Determinants of Mobile Apps’ Success: Evidence from the App Store Mar‐ ket,” Journal of Management Information Systems 31(2), 2017, 133–170.

[15] M. Pandey, R. Litoriya, and P. Pandey. “Applica‐ tion of fuzzy DEMATEL approach in analyzing mobile app issues,” Programming and Computer Software, 45(5), 2019, 268–287.

[27] J. Nikolas, A. S. Fogarty, K. Boydell, and Chris‐ tensen. “The Reviews Are in: A Qualitative Con‐ tent Analysis of Consumer Perspectives on Apps for Bipolar Disorder,” In: Proceedings of the 10th IEEE consumer communications and networking conference, CCNC ’13, (2017), pp. 149–157.

[16] M. Pandey, R. Litoriya, and P. Pandey. “Mobile applications in context of big data: A survey,” In 2016 Symposium on Colossal Data Analysis and Networking (CDAN) (pp. 1–5). 2016, March, IEEE.

[28] L. Hoon, R. Vasa, J. G. Schneider, K. Mouzakis. “A preliminary analysis of vocabulary in mobile app user reviews,” In: Proceedings of the 24th Australian computer-human interaction conference, OzCHI ’12. ACM, 2012, pp. 245–248.

[17] M. Pandey, R. Litoriya, and P. Pandey. “Empirical analysis of defects in handheld device applica‐ tions,” In International Conference on Advances in Computing and Data Sciences (pp. 103–113). 2019, April. Springer, Singapore.

[29] R. Vasa, L. Hoon, K. Mouzakis, A. Noguchi. “A pre‐ liminary analysis of mobile app user reviews,” In: Proceedings of the 24th Australian computerhuman interaction conference, OzCHI ’12. ACM, 2012, pp. 241–244.

[18] M. Pandey, R. Litoriya, and P. Pandey. “Applica‐ bility of machine learning methods on mobile app effort estimation: Validation and perfor‐ mance evaluation,” International Journal of Software Engineering and Knowledge Engineering, 30(01), 2020, 23–41.

[30] B. Fu, J. Lin, L. Li, C. Faloutsos, J. Hong, N. Sadeh. “Why people hate your app: making sense of user feedback in a mobile app store,” In: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’13. ACM, 2013, pp. 1276–1284.

[19] M. Pandey, R. Litoriya, and P. Pandey. “Novel approach for mobile based app development incorporating MAAF,” Wireless Personal Communications, 107(4), 2019, 1687–1708.

[31] P. M. Vu, T. T. Nguyen, and H. V. Pham, H. V. “Mining User Opinions in Mobile App Reviews: A Keyword‐based Approach,” 2013, https://arxiv. org/pdf/1505.04657.pdf.

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

[32] L. Zhang, X. Y. Huang, and Y. K. Hu, “CSLabel: An Approach for Labelling Mobile App Reviews,” 32(6), 2017, 1076–1089.

[45] W. Zhang, Y, Deng. “Combining con licting evi‐ dence using the DEMATEL method,” Soft Computing, 23, 2019, 8207–8216.

[33] K. Phetrungnapha, T. Senivongse. “Classi ication of Mobile Application User Reviews for Gener‐ ating Tickets on Issue Tracking System,” 2019, 12th International Conference on Information & Communication Technology and System (ICTS).

[46] W. Zhang, Y. Deng. “Combining con licting evi‐ dence using the DEMATEL method,” Soft Computing, 23, 2019, 8207–8216.

[34] L. Padgaonkar, S. Jain, S. Ajgaonkar, R. Londhe, and B. S. Balbudhe. “Mobile Application Review Classi ication Using Machine Learning Approach,” International Journal of Innovative Research in Science, Engineering and Technology, 8(5), 2019, 5806–5809. [35] A. P. Widyassari, S. Rustad, G. F. Shidik, E. Noer‐ sasongko, A. Syukur, A. Affandy, D. De Setiadi. “Review of automatic text summarization tech‐ niques & methods,” 34(4), 2022, 1029–1046. [36] K. Kalaichelavan, H. Malik, N.. Husnu, and S. Shreenath. “What Do People Complain About Drone Apps? A Large‐Scale Empirical Study of Google Play Store Reviews,” Procedia Computer Science, 170, 2020, 547–554. [37] T. Zhang, J. Chen, X. Zhan, X. Luo, D. Lo, and H. Jiang. Where2Change: Change Request Local‐ ization for App Reviews, IEEE Transactions on Software Engineering, 47(11), 2590–2616. [38] H. Khalid, E. Shihab, M. Nagappan, and A. E. Hassan, “What Do Mobile App Users Complain About?”, IEEE Software 32(3), 2021, 1–1. [39] R. Vasa, L. Hoon, K. Mouzakis, and A. Noguchi. “A Preliminary Analysis of Mobile App User Reviews,” Proceedings of the 24th Australian Computer-Human Interaction Conference, 2012, doi: 10.1145/2414536.2414577. [40] C. Iacob, and R. Harrison. Retrieving and analyzing mobile apps feature requests from online reviews, 10th Working Conference on Mining Software Repositories (MSR), 2013, doi: 10.1109/MSR.2013.6624001. [41] F. Paloma, M. L. V´asquez, G. Bavota, R. Oliveto, M. D. Penta, D. Poshyvanyk, and A. D. Lucia. “Crowdsourcing User Reviews to Support the Evolution of Mobile Apps,” 2017, 137, 143–162.

[47] W. Liu. “Analyzing the degree of con lict among belief functions,” Artif Intell 170(11), 2006, 909– 924. [48] M. L. Tseng. “A causal and effect decision making model of service quality expectation using grey‐ fuzzy dematel approach,” Expert Syst Appl 36(4), 2009, 7738–7748. [49] H. S. Lee, G. H. Tzeng, W. Yeih, Y. J. Wang, and S. C. Yang. “Revised DEMATEL: Resolving the Infeasibility of DEMATEL,” 37(11), 2013, 6746– 6757. [50] J. I. Shieh, H. H. Wu. “A DEMATEL method in iden‐ tifying key success factors of hospital service quality,” Knowledge-Based Systems, 23(3), 2013, 277‐282. [51] M. Yazdi, F. Khan, R. Abbasi, and R. Rusli. “Improved DEMATEL methodology for effective safety management decision‐making,” Safety Science, 127, 2020, 1047–125. [52] W. C. Wang, Y. H. Lin, C. L. Lin, C. H. Chung, M. T. Lee. DEMATEL‐based model to improve the performance in a matrix organization, Expert Systems with Applications, 39(5), 2012, 4978– 4986. [53] Y. Li,Y. Hu, X. Zhang, Y. Deng, S. Mahadevan, “An evidential dematel method to identify critical success factors in emergency management,” Appl Soft Comput J 22, 2014, 504–510. [54] Y. Lin, C. Wang, C. Ma, Z. Dou, X. Ma. “A new combi‐ nation method for multisensor con lict informa‐ tion,” J Supercomput 72(7), 2016, 1–17. [55] J. Hu, and G. N. Zhu. “A Rough‐Z‐number‐ based DEMATEL to evaluate the co‐creative sustainable value propositions for smart product‐service systems,” 38(8), 2021, 3645– 3679. [56] P. Zdzislaw. “Rough sets, rough relations and rough functions,” Fundamental Informaticae, 27(2–3), 1996, 103–108.

[42] D. Ferreira, J. Goncalves, V. Kostakos, and A. K. Dey. “Contextual Experience Sampling of Mobile Application Micro‐Usage,” MobileHCI 2014, Sept. 23–26, 2014, Toronto, ON, CA.

[57] J. Wen, X. Chunhe, L. Yu, and T. Yongchuan. “Ranking Z‐numbers with an improved ranking method for generalized fuzzy numbers,” Journal of Intelligent & Fuzzy Systems, 32(3), 2017, 1931– 1943.

[43] E. Falatoonitoosi, S. Ahmaed, and S. Sorooshiyan. “Expanded DEMATEL for determining cause and effect group in bidirectional relations,” The Scienti ic World Journal, 2014, 1–8.

[58] B. Kang, D. Wei, Y. Li, and Y. Deng. “A Method of Converting Z‐number to Classical Fuzzy Num‐ ber,” Journal of Information & Computational Science, 9(3), 2012, 703–709.

[44] G. Shen, S. G. Sun, Y. Zhang, Z. Wang, B. Chen, C. Ma. “System failure analysis based on DEMATEL‐ISM and FMECA,” Journal of Central South University, 21, 2014, 4518–4525.

[59] M. Tarokh, M. Cross, and M. Lee. “Erratum to: Fuzzy logic decision making for multi‐robot security systems,” Arti icial Intelligence Review, 34, 2010, 289. 49

Journal of Automation, Mobile Robotics and Intelligent Systems

N∘ 2

2023

[60] A. Shahzaib, A. Saleem, M. Tahir, G. Fazal, M. Tariq. “Spherical fuzzy sets and their appli‐ cations in multi‐attribute decision making prob‐ lems,” Journal of Intelligent & Fuzzy Systems, 36(3), 2019, 2829–2844.

[64] S. Gul. “Extending ARAS with Integration of Objective Attribute Weighting under Spheri‐ cal Fuzzy Environment,” International Journal of Information Technology & Decision Making, 20(3), 2021, 1011–1036.

[61] S. L. Si, X. Y. You, H. C. Liu. “DEMATEL Technique: A Systematic Review of the State‐of‐the‐Art Lit‐ erature on Methodologies and Applications,” Mathematical problems in Engineering, 2018, doi. 10.1155/2018/3696457.

[65] A. Shahzaib, A. Saleem, A. Muhammad, Q. Muhammad, K. Marwan. “Spherical fuzzy sets and its representation of spherical fuzzy t‐norms and t‐conorms,” Journal of Intelligent & Fuzzy Systems, 36(6), 2019, 6089–6102.

[62] B. Chang, C. Chang, C. H. Wu. “Fuzzy DEMATEL method for developing supplier selection cri‐ teria,” Expert Systems with Applications, 38(3), 2011, 1850–1858.

[66] K. Gundogdu, Fatma, and K. Cengiz. “A novel VIKOR method using spherical fuzzy sets and its application to warehouse site selection,” Journal of Intelligent & Fuzzy Systems, 37(1), 2019, 1197– 1211.

[63] S. Yuksel, H. Dincer, S. Eti, and Z. Adali. “Strat‐ egy improvements to minimize the drawbacks of geothermal investments by using spherical fuzzy modelling,” International Journal of Energy Research, 2022, doi: 10.1002/er.7880.

VOLUME 17,

VOLUME 17, N∘ 2 2023 Journal of Automation, Mobile Robotics and Intelligent Systems

UPDATE ON THE STUDY OF ALZHEIMER’S DISEASE THROUGH ARTIFICIAL INTELLIGENCE TECHNIQUES Submitted: 11th January 2023; accepted: 18th February 2023

Eduardo Garea‑Llano DOI: 10.14313/JAMRIS/2‐2023/15 Abstract: Alzheimer’s disease is the most common form of demen‐ tia that can cause a brain neurological disorder with progressive memory loss as a result of brain cell damage. Prevention and treatment of disease is a key challenge in today’s aging society. Accurate diagnosis of Alzheimer’s disease plays an important role in patient management, especially in the early stages of the disease, because awareness of risk allows patients to undergo preventive measures even before irreversible brain damage occurs. Over the years, techniques such as statistical modeling or machine learning algorithms have been used to improve understanding of this condition. The objective of the work is the study of the methods of detection and progres‐ sion of Alzheimer’s disease through artificial intelligence techniques that have been proposed in the last three years. The methodology used was based on the search, selection, review, and analysis of the state of the art and the most current articles published on the subject. The most representative works were analyzed, which allowed proposing a taxonomic classification of the studied meth‐ ods and on this basis a possible solution strategy was proposed within the framework of the project developed by the Cuban Center for Neurosciences based on the con‐ ditions more convenient in terms of cost and effectiveness and the most current trends based on the use of artificial intelligence techniques. Keywords: Alzheimer disease, Detection, Progression, Artificial intelligence, Deep learning.

1. Introduction Alzheimer’s disease (AD), as one of the most com‐ mon forms of dementia, is a neurodegenerative dis‐ ease that causes progressive cognitive decline and memory loss. In terms of neuropathology, AD manifests with neuron loss and synaptic loss in the cerebral cortex and in speci ic subcortical regions in a progressive manner that ultimately leads to death [1]. The prevention and treatment of AD is a chal‐ lenge within the problem of aging in today’s soci‐ ety. It is estimated (2018 Alzheimer’s disease facts and igures) that AD accounts for 60 to 70% of age‐ related dementia, affecting some 114 million people in 2050.

Due to its nature and long evolution, the care and treatment of patients with AD adds more economic burden to their family members. In addition, the psy‐ chological burden of caring for people with AD is very severe and, as a result, many families or care‐ givers experience high levels of emotional stress and depression. For now, there is no cure for AD, the available treat‐ ments offer relatively little symptomatic bene it and are palliative in nature. Therefore, achieving effective and ef icient intervention through early detection and diagnosis of AD is of great importance. AD can be diagnosed but not predicted in its early stages, since the prediction is only applicable before the disease manifests itself. Despite being a very recent area of research, a large number of advanced techniques are cur‐ rently reported in the literature [2]. Cognitive func‐ tion assessment techniques have been described as important indicators of dementia [3–5], such as the Mini Mental State Examination (MMSE), the cognitive subscale of the AD (ADAS) and the Rey Auditory Verbal Learning Test (RAVLT), which are used as preliminary screening tools in the detection of collaboration and vocabulary memory in patients with AD. The brain scan technique [6] is very popular, and is based on a magnetic resonance imaging (MRI) machine to obtain tomographic images with the aim of identifying struc‐ tural and functional brain abnormalities, including dementia, and which allows estimating the volume and density of the brain components of the patient with AD. Structural imaging modalities of mild cogni‐ tive impairment (MCI), normal, and AD are illustrated in Figure 1 [7]. There are also a number of important biomark‐ ers with signi icant characteristics throughout the progression of AD disease. Biomarker identi ication

Figure 1. MRI images of MCI and AD compared to that of a healthy brain

2023 © Garea-Llano. This is an open access article licensed under the Creative Commons Attribution-Attribution 4.0 International (CC BY 4.0) (http://creativecommons.org/licenses/by-nc-nd/4.0/)

Journal of Automation, Mobile Robotics and Intelligent Systems

approaches for monitoring disease progression have received increasing attention in the pathophysiology of AD. Finally, techniques based on IoT (Internet of Things) or generalized health techniques have emerged [8–12], which have been designed and devel‐ oped to collect personal health data from patients with AD and further monitor the progression of this disease. In [2] a study of the state of the art of model‐ ing techniques for AD progression is carried out. The authors report that among the key techniques for the care of patients with AD, a fundamental aspect is to understand the progress of the disease and try to identify stable and sensitive biomarkers for accurate monitoring of AD progression. For example, atrophy of the temporal structures is considered a valid diagnos‐ tic marker in the stage of mild cognitive impairment. On the other hand, as the disease progresses in AD patients, changes in their cerebral cortex can be cap‐ tured on magnetic resonance images. Regression models [6, 13, 14] have previously been used in AD studies to explore correlations between cognitive measurement and magnetic reso‐ nance imaging changes. Therefore, the way to estab‐ lish a model of AD progression based on cognitive scores has drawn attention due to its importance in the early diagnosis of AD. Over the years, techniques such as statistical modeling or machine learning algorithms have been used to improve understanding of a wide range of health conditions. Common statistical models have been used in the simulation of AD disease progres‐ sion [2]. The main dif iculty of statistical models is their high dependence on too many assumptions, which means that the prediction method does not consider external factors for the time series. On the contrary, machine learning models have been good at dealing with the above problems. They do not require some strict assumptions. In addition, the factors in the real world are more complex and the machine learning model allows determining more decisive fac‐ tors in the iterative training process. Prediction of AD disease progression can be applied to machine learn‐ ing regression models using multitask learning, time series, and deep learning. However, in the last two to three years, research in this line has continued to produce a large number of publications, most of which focus on the use of deep learning techniques. In this paper we will try to offer a general overview of the most recent results based on their classi ication and analysis. The work is struc‐ tured as follows: in Section 2.1 we will analyze the most relevant studies of the state of the art that have been published on the subject in the last three years; in Section 2.2 we present a classi ication scheme of the methods studied, describing and analyzing the most representative works of each taxonomic unit. In Section 2.3, based on the analyses carried out, we discuss the results and outline an idea of facing the problem, taking into account the conditions and char‐ acteristics of the task; inally, we give the conclusions of this work. 52

VOLUME 17,

N∘ 2

2023

2. Content For the study, an in‐depth investigation was car‐ ried out in the main databases such as IEEE Xplore, ACM Digital Library, and ScienceDirect, looking for works related to the detection and modeling of AD progression through deep learning. Some of the most popular articles were selected as representative, mainly those published in the last three years. Next, an evaluation and analysis of them was carried out, which allowed us to propose a classi ication that we present in Figure 2 of Section 2.2, but irst we will stop at the analysis of the articles that refer to the most advanced states of the art recently published on the subject and that collect in a synthesized way the main advances achieved. 2.1. Recent Surveys in the Study of AD In [2] the trends in the construction of prediction models for AD until 2019 are summarized, analyz‐ ing three main techniques: multitasking model, time series model, and deep learning. In particular, the basic structural elements of the most representa‐ tive multitask learning algorithms are discussed and a multitask disease prediction model based on longi‐ tudinal time is analyzed. The multitask learning approach belongs to sta‐ tistical machine learning, which originates from the school of frequency statistics. The main strategy of this model is based on optimization through the construc‐ tion of a loss function. The stepwise adjustment of the model parameters gradually reduces the prediction and the actual error of the cognitive score in the pro‐ cess of training the samples using gradient descent. Sparse learning methods select important features (biomarkers) in multiple iterations. The authors focus the analyses referring to the time series model on the fact that if the progression of the disease is conditioned by its manifestations in different periods of time, it can be modeled based on these events based on the use of the Markov Random Fields (MRF) model. The main strategy of this model is to calculate the posterior probability and translate it into the problem of inding the integral. The authors review a paper [15] in which the hidden Markov model (HMM) is able to model disease progression more granularly than currently de ined clinical stages and uncover more granular disease stages compared to corresponding stages in clinical diagnoses. Regarding deep learning techniques, the authors focus on the analysis of the use of recurrent neural net‐ works (RNN). Although RNNs have great advantages when dealing with time series data, they suggest that there are still many dif iculties to overcome when per‐ forming disease prediction in the medical ield. A typ‐ ical drawback is that RNNs cannot handle sequential data with missing parts, since RNNs model a nonlin‐ ear relationship between consecutive data. Further‐ more, RNNs require a ixed time interval. According to the authors, this assumption is not reasonable in the ield of health care testing because the frequency of clinical testing varies with changes in the patient’s condition. However, in the work the authors analyze

Journal of Automation, Mobile Robotics and Intelligent Systems

three strategies to solve this problem that focus on the completion of missing data, the establishment of ixed durations, and the optimization of the model. Finally, the authors emphasize that the multitask‐ ing model plays an important role in terms of improv‐ ing predictive performance, since relevance within similar tasks is exploited. They conclude that the prob‐ lem of longitudinal data necessary for the progression of AD is a challenging task, whether for the time series model or for the deep learning model; each particular model has its own advantages in improving the quality of the data. In [16], a systematic review of publications using deep learning approaches and neuroimaging data for the diagnostic classi ication of AD was carried out, tak‐ ing as reference articles published between January 2013 and July 2018. Of 16 studies that met the full inclusion criteria, 4 used a mix of deep learning and traditional machine learning approaches, and 12 used only deep learning approaches. Combining traditional machine learning for classi ication and stacked auto‐ encoder (SAE) for feature selection yielded accuracies of up to 98.8% for EA classi ication and 83.7% for the prediction of the stage of mild cognitive impair‐ ment (MCI), which is a prodromal stage of AD. Deep learning approaches, such as CNNs or RNNs, which use neuroimaging data without preprocessing for feature selection, have yielded accuracies of up to 96.0% for AD classi ication and 84.2% for MCI stage prediction. The best classi ication performance was obtained when multimodal neuroimaging and luid biomarkers were combined. Additionally, the authors re lect and conclude that not all problems can be solved with deep learning. This is because deep learning that extracts attributes directly from the input data without prepro‐ cessing for feature selection has dif iculty integrating different data formats as input. Because weighting for input data is done automatically within a closed network, adding additional input data to the network causes confusion and ambiguity. However, a hybrid approach places the additional information in parts of the process that develops with classical machine learning methods and the neuroimaging in parts of the process that develops with deep learning methods before combining the two results. The authors propose that deep learning will be advanced by overcoming these problems and present‐ ing speci ic solutions for each problem. As more and more data is acquired, research using deep learning will have a greater impact. The expansion of 2D CNN‐ like networks into 3D CNN‐like networks is important, especially in the study of AD, which deals with multi‐ modal neuroimaging. Furthermore, Generative Adversarial Networks (GANs) [17] can be applied to generate synthetic med‐ ical images for data augmentation. Furthermore, rein‐ forcement learning [18], a form of learning that adapts to changes in data as it makes its own decisions based on the environment, may also demonstrate applicabil‐ ity in the ield of medicine. EA research using deep learning is still evolving for better performance and

VOLUME 17,

N∘ 2

2023

transparency. As multimodal neuroimaging data and computational resources grow rapidly, research on diagnostic classi ication of AD using deep learning is shifting toward a model that uses only deep learning algorithms rather than hybrid methods, although it is necessary to develop methods to integrate completely different data formats. In [19], an analysis of the most relevant studies examining AD using MRI data, machine learning, and deep learning techniques is performed with various AD datasets. The article takes a tour of the history of the discovery of AD and the techniques that have been used for its detection through magnetic reso‐ nance imaging and its different modalities. It is a very instructive article for specialists who are beginning to study the subject. In general, based on a literature review carried out, the authors found that published articles tend to focus on two main areas of research, namely biomarkers and neuroimaging, but with a growing interest in the analysis of images. This study reviewed some of the important data sets related to AD. In [20], the authors carry out a fairly detailed analysis of how Alzheimer’s disease affects different parts of the brain, the stages of Alzheimer’s disease, the neuroimaging techniques that are being used to visualize the effects, the resources of relevant data sets and how the different Arti icial Intelligence‐based methods developed over the years are able to identify changes in the brain and then formulate the de ining characteristics. Finally, they perform a comparison of all the methods discussed with a relevant perfor‐ mance metric. In [21], some of the recent approaches that use machine learning and deep learning algorithms to pre‐ dict AD early and contribute to its therapeutic devel‐ opment are explored. These approaches were catego‐ rized in terms of learning technique and data modality used. In addition, they were discussed from different aspects and their strengths, limitations and results were compared. 2.2. Classification of Artificial Intelligence Methods for the Study of AD Published in the Last Three Years Figure 2 shows the scheme that includes the clas‐ si ication proposed in this study. Deep Learning (DL) has been described as “a new area of research in Machine Learning (ML), which has been introduced with the aim of bringing ML closer to one of its original objectives: arti icial intelligence.” The DL structure generally comprises more than two levels of abstraction and representation to help understand texts, images, and sounds [22]. On the other hand, one way to see the classi ication of DL methods in the detection and progression of AD is from the nature of the data it processes. In gen‐ eral they can be divided into three classes: Methods that use images, methods that use clinical data and biomarkers, and methods that use mixed data from images and clinical data. 53

Journal of Automation, Mobile Robotics and Intelligent Systems

Figure 2. Classification of Artificial Intelligence methods for the study of AD published in the last three years Methods using only image data In [23] the authors present the irst automatic end‐to‐end deep learning framework for prediction of future patient disability progression based on multi‐ modal brain magnetic resonance imaging of patients with multiple sclerosis (MS). The model uses parallel convolutional pathways, an idea introduced by the popular Inception net‐ work, and was trained and tested on two large clin‐ ical test datasets. As a result, they obtained a 3D CNN network model with parallel convolutional layers to predict future progression in MS patients using magnetic resonance imaging. The results also indicate that supplementing the model with enhancer lesion labels, if available, further improves prediction accu‐ racy. Finally, the proposed model includes an estima‐ tion of the uncertainty. A study is presented in [24] in which the authors discuss that accumulation of abnormal tau in neu‐ ro ibrillary tangles (NFTs) occurs in AD and in a spectrum of tauopathies. These tauopathies have diverse and overlapping morphologic phenotypes that obscure classi ication and quantitative assessments. The authors propose the application of deep learn‐ ing in the neuropathological evaluation of NFT in post‐ mortem human brain tissue to develop a classi ier capable of recognizing and quantifying tau content. The histopathological material used in the study was obtained from 22 autopsy brains of patients with tauopathies. They used a custom web‐based comput‐ ing platform integrated with an internal information management system to manage the images they called whole slide images (WSI) and expert annotations as ground truth. They used fully annotated regions to train a fully convolutional neural network (FCN) against expert annotations. As a result, they found that the network was able to identify and quantify NFTs with a variety of intensities and diverse morphologies. In [25], the authors implemented a CNN for early diagnosis and classi ication of AD using magnetic res‐ onance imaging, using ADNI 3 imaging class with a total number of samples for network training of 1512 patients mild, 2633 normal and 2480 with 54

VOLUME 17,

N∘ 2

2023

AD. A signi icant accuracy of 99% was achieved. The model performed well compared to many other related works. In addition, they also compared the result with their previous inding in which classical machine learning algorithms were applied using the OASIS dataset, showing that when dealing with a large amount of medical data, deep learning approaches may be a better option than traditional machine learning. In [26], the authors conducted the research with the aim of building a classi ier for diagnosis of AD based on brain imaging through transfer learning using a large and diverse data set. MRI data from more than 217 imaging sites were collected to con‐ stitute the largest brain MRI sample reported to date, according to the authors (85,721 images from 50,876 participants). They then used an Inception‐ResNet‐ V2 network to build a highly generalizable gen‐ der classi ier. The gender classi ier achieved 94.9% accuracy and served as the base model in trans‐ fer learning for the objective diagnosis of AD. After transfer learning, the itted model for EA classi i‐ cation achieved 91.3% cross‐validation accuracy for excluded sites (not used in network training) in the dataset Alzheimer’s Disease Neuroimaging Initiative (ADNI) and 94.2%/87.9% accuracy for direct tests on two independent data sets not used in training (AIBL/OASIS). When this AD classi ier was tested on brain images of patients with mild cognitive impairment (MCI) who had not been used in training, MCI patients who even‐ tually progressed to AD were 3 times more likely to be classi ied as AD than patients with MCI who did not reach AD (65.2% vs. 20.6%). Predicted AD clas‐ si ier scores showed signi icant correlations with dis‐ ease severity. In summary, the proposed AD classi ier could offer a biomarker that could be integrated into AD diagnostic practice. The trained model, code, and preprocessed data are freely available to the research community. The code to train and test the model is available at https://github.com/Chaogan‐Yan/Brai nImageNet. In [27], the authors implement a CNN for AD detec‐ tion and classi ication of MRI images. The methodol‐ ogy starts with basic preprocessing techniques, such as image resizing and pixel normalization, and then the extracted features are integrated into a one‐ dimensional vector that is sent to the CNN with the corresponding labels. Four labels (classes) are used according to the four stages of AD considered, which are (Not demented, Very mildly demented, Mildly demented, and Moderately demented). The evaluation of the prediction model showed an ef icient result of the model for only ten epochs; the accuracy of the model was 97%. In [28], the authors discuss the problem that while several experiments have recently used machine learning approaches for computer‐aided diagnosis of AD, a bottleneck in diagnostic performance has been found in most of the previous studies, mainly due to the inborn defects of the selected learning models.

Journal of Automation, Mobile Robotics and Intelligent Systems

To solve this bottleneck, the authors propose a deep learning architecture. Compared to previous work‐ lows, the proposed approach is capable of assessing a variety of groups in a single environment involv‐ ing fewer labeled learning samples and limited prior domain knowledge. A substantial improvement in ef i‐ ciency was achieved in the description of all diagnostic classes. Recently in [34] an optimized vision transformer called OViTAD is introduced to predict healthy brains, MCI, and AD using data from rs‐fMRI and struc‐ tural MRI (sigma = 3.4 mm). The prediction pipeline included two separate pre‐processing steps for the two modalities, cutting‐level vision transformer train‐ ing and evaluation, and a post‐processing step based on voting for the majority concept. Experimental results showed that the optimized vision transformer proposal outperformed and was on par with vision transformer‐based methods, and the number of train‐ able parameters was reduced by 30% compared to the standard vision transformer. The mean performance of OViTAD over three replicates was 97% ± 0.0 and 99.55% ± 0.39 for both modalities of multiclass classi‐ ication experiments, which outperformed most CNN‐ based models and the existing deep learning. This study showed that vision transformers could outper‐ form and compete with state‐of‐the‐art algorithms to predict various stages of Alzheimer’s disease with less complex architectures. Methods using clinical data and/or biomarkers In [29], the results obtained in tests performed to verify the performance of metabolites in blood to classify AD in comparison with CSF biomarkers are reported. This study analyzed samples from 242 cog‐ nitively normal (CN) and 115 people with demen‐ tia type AD using plasma metabolites (n 5,883). DL, Extreme Gradient Boosting (XGBoost), and Random Forest (RF) methods were used to differentiate AD from NC. These models were internally validated using nested cross‐validation. This study showed that plasma metabolites have the potential to match the AUC of well‐established AD CSF biomarkers in a rel‐ atively small cohort. Further studies in independent cohorts are needed to validate whether this speci ic panel of blood metabolites can separate AD from con‐ trols, and how speci ic it is for AD compared to other neurodegenerative disorders. In [30], the authors assume that multitask mod‐ eling improves the performance, robustness, and sta‐ bility of AD progression detection. However, they state that multimodal multitasking modeling has not been evaluated using time series and the deep learn‐ ing paradigm, especially for the detection of AD pro‐ gression. To this end, in this work, they propose a robust deep learning assembly model based on a CNN and a bidirectional long‐short‐term memory (BiLSTM) network. This multitask multimodal model jointly predicts multiple variables based on the fusion of ive types of multimodal time series data plus a basic knowledge set (BG). Predicted variables include the AD multiclass progression task and four critical

VOLUME 17,

N∘ 2

2023

cognitive scores regression tasks. The proposed model extracts local and longitudinal features of each modal‐ ity using a stacked CNN and BiLSTM network. At the same time, local features are extracted from the BG data using a feedback neural network. The resulting features are merged into a deep network to detect common patterns that are used together to predict classi ication and regression tasks. To validate the pro‐ posed model, the authors performed six experiments in ive ADNI Initiative modalities with data from 1536 subjects. The results of the proposed approach achieve state‐of‐the‐art performance for multiclass progres‐ sion and regression tasks. Methods that combine images and clinical data In [31], the authors propose the use of a multi‐ modal recurrent network to predict the probability of conversion from MCI to AD. They developed an integrative framework that combines not only cross‐ sectional neuroimaging biomarkers at baseline, but also longitudinal biomarkers of cerebrospinal luid (CSF) and cognitive performance obtained from ADNI. The results showed that 1) the prediction model for MCI to AE conversion yielded up to 75% accuracy (area under the curve (AUC) = 0.83) when a single data modality was used separately; and 2) the predic‐ tion model achieved the best performance with 81% accuracy (AUC = 0.86) when incorporating longitudi‐ nal data from multiple domains. In [32], the authors discuss that with the advent of the DL paradigm, it has been possible to extract high‐level abstract features directly from magnetic resonance images that internally describe the data dis‐ tribution in low‐dimensional manifolds. The authors propose a new exploratory analysis of AD data based on deep convolutional autoencoders. The aim was to ind links between cognitive symptoms and the underlying neurodegenerative process by merg‐ ing information from neuropsychological test results, diagnoses, and other clinical data with image fea‐ tures extracted solely through a decomposition based on MRI data. In the work, the distribution of the characteristics extracted in different combinations is analyzed and visualized through regression and clas‐ si ication analysis, and the in luence of each coor‐ dinate of the autocoding variety on the brain is estimated. Image‐derived markers can then predict clinical variables with correlations greater than 0.6 in the case of neuropsychological assessment variables such as MMSE or ADAS11 scores, achieving a classi‐ ication accuracy greater than 80% for the diagnosis of EA. In [33], they discuss that the fusion of multiple data modalities can provide a holistic view of the analysis of AD status. Therefore, on this basis, they use DL to com‐ prehensively analyze images (MRI), and genetic data (single nucleotide polymorphisms (SNP)) and clinical test data to classify patients into AD, MCI, and con‐ trols (CN). The authors use stacked automatic denois‐ ing encoders to extract features from clinical and genetic data, and 3D convolutional neural networks (CNN) for image‐type data. They also developed a 55

Journal of Automation, Mobile Robotics and Intelligent Systems

new data interpretation method to identify high‐ performance features learned by deep models with clustering and perturbation analysis. Using the ADNI dataset, they showed that deep models outperform classical models. They further demonstrated that multi‐modality data integration outperforms single‐ modality models in terms of accuracy, recall, and mean F1 scores. The developed models identi ied the hippocampus, the amygdala brain areas, and the Rey Auditory Verbal Learning Test (RAVLT) as salient features, which are consistent with the known AD literature. An articulated and deep learning framework for predicting clinical scores in AD is presented in [34]. Speci ically, the feature selection method combining cluster LASSO and correntropy is used to downsize and detect features of AD‐related brain regions. The authors explore multilayer recurrent neural network regression independently to study the internal con‐ nection between different brain regions and the time correlation between longitudinal data. The proposed joint deep learning network studies the relationship between MRI and clinical score, and predicts clinical score. Predicted clinical score values allow clinicians to make early diagnosis and timely treatment of dis‐ ease. In [35] the Multimodal Mixing Transformer (3MT) was presented, for classi ication based on multi‐ modal data. The authors use it in their work to classify patients with Alzheimer’s disease (AD) or who are cognitively normal (CN) using neuroimaging data, gender, age, and Mini‐Mental State Examination (MMSE) scores. The model uses a cross‐attention cas‐ cade mode transformer architecture. Auxiliary out‐ puts and a mode drop mechanism were incorporated to ensure a level of mode independence and robust‐ ness. The result is a network that allows the com‐ bination of an unlimited number of modalities with different formats and the full utilization of the data: it handles all missing data combinations while main‐ taining classi ication performance. 3MT was tested on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset and achieved a testing precision of 0:987, 0:0006. To test its generalizability, 3MT was applied directly to the lagship Australia Imaging and Lifestyle Biomarker Aging (AIBL) study after training on the ADNI dataset, and achieved a test precision of 0:925, 0:0004 without ine tuning. 2.3. Discussion After analyzing the described literature, we can conclude that the task of estimating the progression of AD continues to be an open problem today, where the use of the time series model and the deep learning model have their own advantages in improving the quality of the data that are used as input but still have limitations. Future work in this direction is moving towards a model that uses only deep learning algo‐ rithms instead of hybrid methods. In this sense, the open problems to be solved focus on the development of methods that allow the integration of different data formats in deep learning networks. 56

VOLUME 17,

N∘ 2

2023

Regarding the diagnostic task, there is a great diversity of approaches proposed in the last three years based on DL. Almost all of them have striven to offer the best model that could ef iciently employ the medical data set and diagnose disease success accurately, especially in its early stages. The data that are generally used are biomarkers and neuroimaging, but with a growing interest in image analysis without discarding biomarkers. The type of data determines the type of DL algo‐ rithm to be used and the level of complexity of the proposed model. Most of the DL‐based approaches that use biomarker data tend to merge them with other modalities to improve prediction accuracy, so this combination increases their complexity. Although CNNs have shown promising results, they have a very strong suitable inductive bias. Several authors have attempted to incorporate multimodal data by combining a CNN feature extractor with some form of modality information injections. However, these methods assume no missing data, which means that retraining is required for each missing data sce‐ nario. For example, a model trained on image, MMSE, and age data is not capable of making predictions with MMSE alone. Therefore, the most common solution has been to discard samples with missing modali‐ ties, with the risk of underusing data. This problem requires an approach that can automatically handle various missing data situations. Apparently, due to the latest works posted in the arXiv repository [34, 35], the trend in this sense is towards the use of transformer‐type networks that have shown high performance in other areas of knowledge. Proposal of a research framework oriented to the development of AD diagnosis and progression model based on artificial intelligence The irst problem to face in our research would be the task of diagnosing AD, taking into account the availability of MRI images both in international databases and in own databases. In this case, we think that an approach based on the use of images could give good results in the short term, giving the possibility of classifying the images into classes such as AD and mild cognitive impairment and normal cognition. For this, training could be carried out based on the transfer of a CNN network in one of the variants that have reported the best results in the literature. In Figure 3 we present the general scheme of the projected model oriented to the development of the AD progression estimation In the case of the disease progression task, the approach that has the most prospects for development and that is in the state of the art are the combined methods of image analysis and clinical data based on transformers network type. These methods have the advantage of dealing with time series data by applying some of the strategies recommended in the literature to solve the problem of illing in missing data, estab‐ lishing ixed durations, and optimizing the model.

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

longitudinal data of each measurement made to the patient. Semantic image branch

Figure 3. Preliminary scheme of the progression prediction model Table 1. Structure of a STensor STensor (𝑛 = 1, 𝑡1 … 𝑡𝑖 ) 1. T1 (𝑡1 … 𝑡𝑖 ). MRI images of the hippocampi; 2. T2 (𝑡1 … 𝑡𝑖 ). Attributes of the hippocampi on MRI images; 3. T3 (𝑡1 … 𝑡𝑖 ). Cognitive score; 4. T4 (𝑡1 … 𝑡𝑖 ). Neuropathological data; 5. T5 (𝑡1 … 𝑡𝑖 ). Evaluation data; 6. T6 (𝑡1 … 𝑡𝑖 ). Demographics

The proposal could be framed in the multimodal modality with a single task (detection of the degree of progression in classes (Normal, DCI and EA). Multimodal data structure In mathematics, a tensor is an algebraic object that describes a multilinear relationship between sets of algebraic objects related to a vector space. Objects that tensors can map between include vectors and scalars, and even other tensors. In the Tensor low framework (https://www.tens orflow.org), all calculations involve tensors. A tensor is an n‐dimensional vector or matrix that represents all data types. All values of a tensor contain an identical data type with a known (or partially known) form. In our strategy we propose to use the concept of SuperTensor (STensor) and Tensor for the structuring of the multimodal data that the model will use to perform the EA prediction task. The STensor of patient n: Contains the data derived from the processing of MRI images and the clinical data of patient n under study at all evaluated time points (𝑡1 , 𝑡2 , … , 𝑡𝑖 ). Tensor 𝑇𝑘(𝑡1 … 𝑡𝑖 ). It contains the data of each one of the modalities within the Stensor of the patient n under study at all the evaluated moments of time (𝑡1 , 𝑡2 , … , 𝑡𝑖 ). 𝑘 = {1, 2, 3, 4, 5, 6}. A Tensor n will then be made up of 6 tensors that are constituted as follows (Table 1). As an example, Figure 4 shows the scheme for obtaining Tensors 1 and 2. These two tensors will be responsible for containing the segmented images of the patient’s hippocampi (tensor 1) and the data extracted from them (tensor 2) at each of the moments of time. The rest of the tensors will be made up of the

Medical images are naturally associated with rich semantics about human anatomy, re lected in a large number of recurring anatomical patterns, offering unique potential to foster the learning of deep seman‐ tic representations and produce semantically more powerful models for different medical applications. In [36] the Semantic Genesis model was intro‐ duced. Based on the fact that medical images are nat‐ urally associated with rich semantics about human anatomy, which is re lected in a large number of recur‐ ring anatomical patterns, the authors offer unique potential to foster the learning of deep semantic rep‐ resentation. and produce semantically more powerful models for different medical applications. In this sense, the authors propose a self‐ supervised learning framework that allows models to directly learn the common visual representation of the image data, and take advantage of the semantic‐ enriched representation of consistent and recurring anatomical patterns, taking into account the wide set of unique properties that medical imaging offers. The results presented by these authors demon‐ strate that Semantic Genesis is superior to publicly available 3D models, pre‐trained by self‐monitoring or even fully‐monitored, as well as 2D ImageNet‐based transfer learning. For our research we propose an adaptation of this model. The model is conceptually simple: an encoder‐ decoder structure with jump connections in the mid‐ dle and a classi ication head at the end of the encoder. Figure 4 shows the general outline of our proposal. 1) Self-discovery of anatomical patterns: The models begins building a set of anatomical patterns from the MRI hippocampus images, as is shown in Figure 4. To extract deep features of each image of a patient, an auto encoder network is trained with train‐ ing data which self‐learns a mapping of each patient. As shown in Figure 4, due to the consistent and recurring anatomies in these patients, that is, each coordinate contains a unique anatomical pattern, it is

Figure 4. Conformation process of tensors 1 and 2 from the image of the hippocampi and the features extracted from them 57

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

Figure 7. Self‐classification and self‐restoration of anatomical patterns

Figure 5. Proposal of model for the extraction of deep features from the hippocampus images based on Semantic Genesis approach [36]

anatomical patterns from the transformed ones. As shown in Figure 7, the restore branch encodes the input transformed anatomical pattern into a latent space and decodes it to the original resolution, with the goal of recovering the original anatomical pattern of the transform. This makes it possible to obtain very robust characteristics for each class of patient. Multimodal data branch

Figure 6. Extraction of patches (anatomical patterns) based on their coordinates feasible to extract similar anatomical patterns based on the coordinates. Therefore, patches of a number C of random but ixed coordinates are cut out in this small set of discovered patients, sharing similar semantics. Similarity calculation is performed at the patient level rather than at the pattern level to ensure balance between diversity and consistency of anatom‐ ical patterns. Finally, pseudo‐labels are assigned to these patches based on their coordinates, resulting in a new dataset, where each patch is associated with one of the C classes (Figure 5). 2) Self-classi ication of anatomical patterns: After self‐discovery of a set of anatomical patterns, representation learning is formulated as a C‐way multiclass classi ication task. The goal is to encour‐ age models to learn from the anatomical patterns that recur in patient images, fostering a deeply semantically enriched representation. As illustrated in Figure 6, the classi ication branch encodes the input anatomical pattern into a latent space, followed by a sequence of fully connected (fc) layers, and predicts the pseudolabel associated with the pattern. 3) Self-restoration of anatomical patterns: The goal of self‐restoration is for the model to learn differ‐ ent sets of visual representation by recovering original 58

As a second stage, we propose a scheme where a separate CNN subnet would be used to learn each modality. CNN for time series introduces 1D convolu‐ tion (Conv1D), which can learn univariate time series data. The convolution is performed separately along the time dimension for each input tensor. Depending on the number of ilters, the CNN expands each uni‐ variate time series into more abstract and informative features, called feature maps, that are more suitable for Transformer prediction. The CNN sub‐network is applied to extract local features in each time series function. Furthermore, the reference data contained in ten‐ sor 6 plays the background role to improve the accu‐ racy and con idence of the learning process. These reference data are the static characteristics of the patient, such as demographics and some statistical characteristics extracted from their longitudinal time series data. The results of this deep feature extraction step are merged together with the results of semantic feature extraction from the hippocampi images. Finally, the merged deep features of each time series are passed to the transformer network to obtain the prediction of the degree of progression.

3. Conclusion In this work we present an update on the meth‐ ods for the study of Alzheimer’s disease using arti‐ icial intelligence techniques; we refer to the most recent works published in the last three years; and we present a taxonomic classi ication of the methods studied according to the type of data with which they work. We carried out the analysis of the articles that contain the methods of the state of the art based on the taxonomic classi ication, reaching the conclusions that allowed us to make a proposal on how to face the problem based on the analysis of the advantages and disadvantages of the methods studied, which focuses on several solution variants based on deep learning

Journal of Automation, Mobile Robotics and Intelligent Systems

techniques, and which can be implemented in the short and medium term. Each variant then requires a process of evaluation, implementation, and adapta‐ tion to the characteristics of our research. As part of the discussion of this work, we proposed a strategy for predicting the progression of AD based on the use of DL techniques with the aim of developing future research by our team. The scheme will combine hippocampal MRI data to extract informative imaging features and ive time‐series modalities under a CNN‐ Transformer design. State‐of‐the‐art results have shown that advanced network design can lead to signi icant improvement in the monitoring of AD patients. The proposed pre‐ liminary design might be able to optimize multiclass classi ication by simultaneously learning and merging discriminant features from images, time series, and BG data. The resulting model can provide promising performance. Systematized experiments in the state of the art suggest that no single modality can be suf i‐ cient to assess AD progression on its own. In addition, they pointed out the importance of merging the char‐ acteristics learned from these modalities. AUTHOR Eduardo Garea-Llano∗ – Neuroinformatic Depart‐ ment. Cuban Neuroscience Center, Havana 16600, Cuba, e‐mail: eduardo.garea@cneuro.edu.cu. ∗

Corresponding author

ACKNOWLEDGEMENTS This work was supported by the Cuban Neuroscience Center and the Ministry of Science, Technology, and Environment of Cuba within the framework of the project “Development of models to study the progres‐ sion of the disease in brain dysfunction pathologies.”

References [1] Z. S. Khachaturian. “Diagnosis of Alzheimer’s dis‐ ease,” Arch. Neurol., vol. 42, no. 11, pp. 1097– 1105, Nov. 1985. [2] X. Wang, J. Qi, Y. Yang, and P. Yang. “A Sur‐ vey of Disease Progression Modeling Techniques for Alzheimer’s Diseases,” 2019 IEEE 17th International Conference on Industrial Informatics (INDIN), 2019, pp. 1237–1242, doi: 10.1109/ INDIN41052.2019.8972091. [3] M. F. Folstein, S. E. Folstein, P. R. McHugh. “Mini‐ mental state. A practical method for grading the cognitive state of patients for the clinician,” J Psychiatr Res., vol. 12, no. 3, pp. 189–98, 1975. [4] M. Schmidt. “Rey Auditory Verbal Learning Test: A Handbook,” RAVLT, 1996. [5] L. W. Chu, K. Chiu, C. Hui, K. Yu, W. J. Tsui, and P. Lee. “The reliability and validity of the Alzheimer’s Disease Assessment Scale Cognitive Subscale (ADAS‐Cog) among the elderly Chinese

VOLUME 17,

N∘ 2

2023

in Hong Kong,” Ann Acad. Med. Singapore, vol. 29, no. 4, pp. 474–85, Jul. 2000. [6] G. B. Frisoni, N. Fox, C. Jack, P. Scheltens, M. Thompson. “The clinical use of structural MRI in Alzheimer disease,” Nature Reviews Neurology, 2010. [7] K. R. Baskaran, V. Sanjay. “Deep learning based early diagnosis of Alzheimer’s disease using Semi Supervised GAN,” Annals of the Romanian Society for Cell Biology, pp. 7391–7400, 2021. [8] E. G. Spanakis, et al. “MyHealthAvatar: Personal‐ ized and empowerment health services through internet of things technologies,” MOBIHEALTH 2014, 2015. [9] Z. Deng, P. Yang, Y. Zhao, X. Zhao, and F. Dong. “Life‐Logging Data Aggregation Solution for Interdisciplinary Healthcare Research and Collaboration,” 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing, 2015, pp. 2315–2320, doi: 10.1109/ CIT/IUCC/DASC/PICOM.2015.342. [10] C. Xie, P. Yang, and Y. Yang. “Open Knowledge Accessing Method in IoT‐Based Hospital Infor‐ mation System for Medical Record Enrichment,” in: IEEE Access, vol. 6, pp. 15202–15211, 2018, doi: 10.1109/ACCESS.2018.2810837. [11] J. Qi, P. Yang, A. Waraich, Z. Deng, Y. Zhao, and Y. Yang. “Examining sensor‐based phys‐ ical activity recognition and monitoring for healthcare using Internet of Things: A system‐ atic review,” Journal of Biomedical Informatics, vol. 87, pp. 138–153, 2018. doi: 10.1016/j.jbi. 2018.09.002. [12] P. Yang, et al. “Lifelogging Data Validation Model for Internet of Things Enabled Personalized Healthcare,” in IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 48, no. 1, pp. 50–64, Jan. 2018, doi: 10.1109/TSMC.2016. 2586075. [13] J. Wan, et al. “Sparse Bayesian multi‐task learn‐ ing for predicting cognitive outcomes from neu‐ roimaging measures in Alzheimer’s disease,” 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 940–947, doi: 10.1109/CVPR.2012.6247769. [14] J. Wan, et al. “Identifying the Neuroanatomical Basis of Cognitive Impairment in Alzheimer’s Disease by Correlation‐ and Nonlinearity‐Aware Sparse Bayesian Learning,” in IEEE Transactions on Medical Imaging, vol. 33, no. 7, pp. 1475– 1487, July 2014, doi: 10.1109/TMI.2014.231 4712. [15] R. Sukkar, E. Katz, Y. Zhang, D. Raunig, and B. T. Wyman. “Disease progression modeling using Hidden Markov Models,” 2012 Annual 59

Journal of Automation, Mobile Robotics and Intelligent Systems

International Conference of the IEEE Engineering in Medicine and Biology Society, 2012, pp. 2845– 2848, doi: 10.1109/EMBC.2012.6346556. [16] J. Taeho, N. Kwangsik, A. J. Saykin. “Deep Learning in Alzheimer’s Disease: Diagnostic Classi ication and Prognostic Prediction Using Neuroimaging Data,” Frontiers in Aging Neuroscience, vol. 11, 2019, p. 220. doi: 10.3389/fnagi.2019.00220. [17] I. Goodfellow, J. Pouget‐Abadie, M. Mirza, B. Xu, D. Warde‐Farley, S. Ozair, et al. “Generative adversarial networks,” in Communications of the ACM, vol. 63, no. 11, pp. 139–144, November 2020. doi: 10.1145/3422622. [18] R. S. Sutton, and A. G. Barto. “Reinforcement learning: An introduction,” Robotica, vol. 17, no. 2, pp. 229–235, 1999. [19] S. Al‐Shoukry, T. H. Rassem, and N. M. Makbol. “Alzheimer’s Diseases Detection by Using Deep Learning Algorithms: A Mini‐Review,” in IEEE Access, vol. 8, pp. 77131–77141, 2020, doi: 10.1109/ACCESS.2020.2989396. [20] R. Jain, A. Aggarwal, V. Kumar. “Chapter 1 – A review of deep learning‐based disease detection in Alzheimer’s patients,” Editor(s): Hemanth D. Jude, Handbook of Decision Support Systems for Neurological Disorders, Academic Press, 2021, pp. 1–19. doi: 10.1016/B978‐0‐12‐822271‐ 3.00004‐9. [21] M. Ghada, A. Fadhl, and G. H. Algaphari. “Machine learning and deep learning‐based approaches on various biomarkers for Alzheimer’s disease early detection: A review,” IJSECS vol. 7, no. 2, pp. 26– 43, 2021. doi: 10.15282/ijsecs.7.2.2021.4.0087. [22] L. Deng, and D. Yu. “Deep Learning: Methods and Applications,” Foundations and Trends in Signal Processing, vol. 7, no. 3–4, 197–387, 2013. [23] A. Tousignant, P. Lemaître, D. Precup, D. L. Arnold, T. Arbel. “Prediction of Disease Progression in Multiple Sclerosis Patients using Deep Learning Analysis of MRI Data,” Proceedings of Machine Learning Research, 102, pp. 483‐492, 2019. [24] M. Signaevsky, M. Prastawa, K. Farrell. “Arti icial intelligence in neuropathology: deep learning‐ based assessment of tauopathy,” Lab Invest 99, 1019–1029 (2019). doi: 10.1038/s41374‐019‐ 0202‐4. [25] A. W. Salehi, P. Baglat, B. B. Sharma, G. Gupta, and A. Upadhya. “A CNN Model: Earlier Diagnosis and Classi ication of Alzheimer Disease using MRI,” 2020 International Conference on Smart Electronics and Communication (ICOSEC), 2020, pp. 156– 161, doi: 10.1109/ICOSEC49089.2020.9215402. [26] Lu B., et al. “A Practical Alzheimer Disease Classi ier via Brain Imaging‐Based Deep Learning on 85,721 Samples,” bioRxiv preprint doi: 10.1101/2020.08.18.256594; this version posted April 13, 2021. 60

VOLUME 17,

N∘ 2

2023

[27] M. Alshammari, and M. Mezher. “A Modi ied Convolutional Neural Networks For MRI‐based Images for Detection and Stage Classi ication of Alzheimer Disease,” 2021 National Computing Colleges Conference (NCCC), 2021, pp. 1–7, doi: 10.1109/NCCC49330.2021.9428810. [28] H. Shamsul, et al. “A Deep Learning Model in the Detection of Alzheimer Disease,” Turkish Journal of Computer and Mathematics Education, vol. 12, no. 10, pp. 4013–4022, 2021. doi: 10.17762/tur‐ comat.v12i10.5113. [29] D. Stamate, et al. “A metabolite‐based machine learning approach to diagnose Alzheimer‐ type dementia in blood,” Alzheimer’s & Dementia: Translational Research & Clinical Interventions, 5, pp. 933–938, 2019. doi: 10.1016/j.trci.2019.11.001. [30] E. Shaker, A. Tamer, S. M. Riazul I, S. K. Kyung. “Multimodal multitask deep learning model for Alzheimer’s disease progression detection based on time series data”, Neurocomputing, vol. 412, pp. 197–215, 2020. doi: 10.1016/j.neucom. 2020.05.087. [31] G. Lee, K. Nho, B. Kang, et al. “Predicting Alzheimer’s disease progression using multi‐ modal deep learning approach.” Sci Rep, 9, 1952 (2019). doi: 10.1038/s41598‐018‐37769‐z. [32] F. J. Martinez‐Murcia, A. Ortiz, J. ‐M. Gorriz, J. Ramirez, and D. Castillo‐Barnes. “Studying the Manifold Structure of Alzheimer’s Disease: A Deep Learning Approach Using Convolutional Autoencoders,” in IEEE Journal of Biomedical and Health Informatics, vol. 24, no. 1, pp. 17–26, Jan. 2020, doi: 10.1109/JBHI.2019.2914970. [33] B. Lei, et al. “Predicting clinical scores for Alzheimer’s disease based on joint and deep learning,” Expert Systems with Applications, 187, 2022. doi: 10.1016/j.eswa.2021.115966. [34] S. Sarraf, A. Sarraf, D. D. DeSouza, J. Anderson, M. Kabia. “OViTAD: Optimized Vision Trans‐ former to Predict Various Stages of Alzheimer’s Disease Using Resting‐State fMRI and Structural MRI Data”. doi: 10.1101/2021.11.27.470184. doi: bioRxiv preprint. [35] Z. Zhang, F. Khalvati. “Introducing Vision Transformer for Alzheimer’s Disease classi ica‐ tion task with 3D input”. 2022. arXiv preprint arXiv:2210.01177. doi: 10.48550/arXiv.2210. 01177. [36] F. Haghighi, T. Hosseinzadeh, M. R., Z. Zhou, M. B. Gotway, J. Liang. “Learning Semantics‐ Enriched Representation via Self‐discovery, Self‐ classi ication, and Self‐restoration.” In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science, vol. 12261. Springer, Cham. doi: 10.1007/978‐3‐030‐59710‐8_14.

VOLUME 17, N∘ 2 2023 Journal of Automation, Mobile Robotics and Intelligent Systems

ANALYZIS OF REHABILITATION SYSTEMS IN REGARDS TO REQUIREMENTS TOWARDS REMOTE HOME REHABILITATION DEVICES Submitted: 12th August 2022; accepted: 18th January 2023

Piotr Falkowski, Cezary Rzymkowski, Zbigniew Pilat DOI: 10.14313/JAMRIS/2‐2023/16 Abstract: The contemporary international pandemic proved that a flexible approach towards work, trade and healthcare is not only favorable but a must. Hence, the devices enabling home‐rehabilitation became one of the urgent needs of the medical market. The following overview is a part of an R&D project aimed at designing an exoskeleton and developing methods enabling effective home reha‐ bilitation. It contains a comparison of current devices in terms of their kinematics, applications, weights, sizes, and integration with selected ICT technologies. The data is analyzed regarding conclusions from qualita‐ tive research, based on in‐depth interviews with phys‐ iotherapists and questionnaires organized beforehand. The investigation assesses whether commercial and developed devices enable feedback from a patient by all possible means; hence, if they could allow effective tel‐ erehabilitation. Moreover, their capabilities of increasing engagement and accelerating improvements by super‐ vising techniques and measuring biomechanical param‐ eters are evaluated. These outcomes are a base to set the constraints and requirements before designing an exoskeleton dedicated to home treatment. Keywords: Home rehabilitation, Exoskeleton, ICT technologies, Market overview, Rehabilitation robotics, Remote rehabilitation, UX analyzis.

1. Introduction The modern world transforms continuously. Thus, new approaches towards common activities are devel‐ oped. The pandemic in 2020 caused an urgent need to transfer processes, such as working [73], learn‐ ing [46], treating or training [49], into the online envi‐ ronment. The situation got so bad in some of the countries, Poland, among others, that patients were not permitted to leave their houses even for neces‐ sary physiotherapy sessions. What is worse, a sce‐ nario of repeating such an emergency is relatively possible. As a result, not only do post‐COVID patients require rehabilitation, but also the ones suffering from motor diseases, whose therapy was restricted [30]. To avoid these situations in future, treatment shall be easily transferable to the patients’ houses and possible to continue even without the physical presence of a physiotherapist. As proven to be effective for kine‐ siotherapy, rehabilitation robots may be used for this purpose [76].

Even though there are a lot of robot–aided rehabil‐ itation devices, an effective home rehabilitation with‐ out a great effort of a physiotherapist is not possible yet. However, a fusion of medicine and engineering should enable such a remote treatment in an ef icient way [67]. What is more, almost 30% of examined phys‐ iotherapists stated that the remote–home–therapy should be the main direction of physiotherapy devel‐ opment. Moreover, new trends in healthcare ICT tech‐ nologies [25], advanced control methods for complex goal functions [37, 38, 62, 66], and the miniaturized, lightweight design of the rehabilitation robots [35, 36, 58] brought new needs for such devices. Therefore, as a revival of the RENUS project [52], the ExoReha system is being designed by the Łukasiewicz Research Network – Industrial Institute for Automation and Measurements PIAP. The initial phase described within this paper is an in–depth overview and comparison of possible com‐ petition – the current or signi icant by different means robot–aided rehabilitation devices for human extrem‐ ities, both the upper and the lower. They are assessed in terms of their kinematics, applications, weights, sizes, and integration with selected ICT technologies. To gather relevant practical insights, the results of the literature research are taken into consideration regarding the outcomes of the initial interviews with therapists. The details on the analyzed devices are compared according to the same criteria. These are selected to enable the evaluation of possible imple‐ mentation into home rehabilitation.

2. Requirements Towards Systems for Home Rehabilitation The requirements presented within the following section are based on in–depth interviews with the young Polish physiotherapists [39, 40]. Hence, they shall be interpreted only to analyze needs in robot– aided motor therapy practices in the countries of cen‐ tral Europe. Nevertheless, most of the conclusions on technical requirements for the devices apply to home rehabilitation international practices. The qualitative research de ined the following needs and problems. They should shape the direction of development for the rehabilitation devices. 1) The younger physiotherapists are eager to use additional weight support for their patients. However, robot–aided rehabilitation devices are

2023 © Falkowski et al. This is an open access article licensed under the Creative Commons Attribution-Attribution 4.0 International (CC BY 4.0) (http://creativecommons.org/licenses/by-nc-nd/4.0/)

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

relatively expensive and hardly accessible in the local markets. Therefore, to increase their popularity, the designs should be relatively portable and inexpensive to build. Moreover, the devices have to be created to help the rehabilitators, not take over their duties fully. 2) One of the main dif iculties in providing effective treatment is keeping the patient’s constant engage‐ ment. It may be challenging while the rehabilita‐ tion sessions are long and consist of monotonous movements. The physiotherapists claim to involve multimedia and ICT technologies in the therapy willingly. However, according to their observa‐ tions, such means of technology can only motivate a patient for a short time. Thus, the rehabilita‐ tion devices should be designed to enable con‐ nection with different interactive systems. By this approach, patients could stay entertained while training without exchanging the device so often. Moreover, an entertaining aspect of VR may stay in synergy with high precision in motion assessment provided by other technologies [57]. 3) Over 58% of the respondents declared that goal– oriented functional therapy is a future of rehabil‐ itation. Thus, the robots for such treatment shall enable the mobilization of multiple degrees of free‐ dom (DOFs) simultaneously. With this approach, the patient may focus on the motion required for daily–life activities and follow the most genuine patterns within rehabilitation sessions. 4) All of the physiotherapists agree that motor ther‐ apy without visual and audio feedback is not pos‐ sible. Therefore, the systems dedicated for home rehabilitation should be either remotely moni‐ tored by a professional (e.g. presentation of the current con iguration and applied moments of forces, and direct contact with a patient via cam‐ era), or need to gather information on patient’s condition with additional sensors, and process data with more advanced algorithms. For these reasons, rehabilitation devices are mainly presented in terms of their portability, automation level, virtual or augmented reality appli‐ cation, and involvement of different ICT technologies. However, they shall be compared according to their purpose of application. The analyzed devices were chosen based on their high accessibility in Europe or potential applicability for remote home treatment in terms of technology advancement. They were sought with Google Scholar, ResearchGate and IEEE Xplore Digital Library using the following phrases: rehabili‐ tation robot, home robot–aided rehabilitation, robot– aided motor therapy, and rehabilitation exoskeleton.

3. Commercial and Developing Devices for Robot–aided Rehabilitation of Upper Extremity 3.1. Physio by Gridbots Physio is an Indian commercial device for the reha‐ bilitation of an upper extremity (see Fig. 1a). 62

Figure 1. Contemporary rehabilitation robotic systems for upper extremities – a) Physio [11], b) ReoGo [17], c) Burt [3], d) InMotionARM [8], e) Armeo Power [1], f) EksoUE [6], g) ARMin [2]

A patient may use it to perform their training by grabbing a grip at the robot’s end–effector and lead‐ ing the programmed trails. Physio’s parallel kinematic structure enables the 2D plain motion of the hand (2 DOFs, Degrees of Freedom), but it does not activate any particular joint directly. The device is supported with intelligent algo‐ rithms based on machine learning techniques. These enable the online learning performance of a patient and adjust the therapy towards their needs. The robot may reach within a rectangular envelope of 1000x1200 mm, and work under a load of up to 50 N. Its mass is 15 kg, and it can be transported within the package dimensions of 300x400x800 mm. However, it also requires a dedicated controller and an HMI (Human–Machine Interface) display. The device is cobot compliant [11]. 3.2. ReoGo by Motorika ReoGo is a commercial robotic system for an upper limb therapy (see Fig. 1b). It is dedicated mostly for post–stroke patients and the ones suffering from neu‐ rological diseases [41, 77].

Journal of Automation, Mobile Robotics and Intelligent Systems

To train with the device, a user has to sit and attach their forearm to the end–effector of the machine (or a hand, while the other handle is assembled). The kinematic structure of the ReoGo enables the 2D or 3D motion of the extremity (2–3 DOFs). However, none of the particular joints is activated directly. The device may track and assess a patient’s per‐ formance. The system offers workouts based on a library of exercises and games. Nevertheless, while using ReoGo, the physiotherapist plays a key role, as they are responsible for designing and personalizing treatments. The ReoGo is relatively mobile due to its mass of 79 kg and integrated wheels. Its overall dimensions stay within 1010x580x900 mm [17]. 3.3. Burt by Barret Medical Burt is an easy–setup commercial robotic system for an upper extremity rehabilitation (see Fig. 1c). It is dedicated to recovering post–stroke patients. To train with the device, a sitting user has to attach their forearm to the end–effector of the Burt’s manipula‐ tor and leads along the programmed trail. The device enables the 3D motion of the limb (3 DOFs), but none of the joints is activated directly. The manipulator may operate within the human–sized work volume (with a reach of 1050 mm). Burt may track and gather data on patients’ improvements over time. However, a physiotherapist is still required to program the whole treatment. Not only does the system provide manual therapy, but it also works on the attention, memory and visual neglect of a patient. The methods applied for the ther‐ apy include a game environment. The Burt may operate with the maximum velocity of 1.5 m/s and under the load of up to 45 N. It is relatively mobile due to its mass of 80 kg and the integrated wheels [3, 19]. 3.4. InMotionARM by Bionik InMotionARM is a commercial system for reha‐ bilitation of an upper extremity (see Fig. 1d). It is dedicated to neurological patients, post–stroke among others [50] [45]. To train with the device, a user must place their forearm in a brace on the robot’s end– effector. The system is designed to support the leading motion of a patient’s hand on a plane (2 DOFs), but it does not necessarily need to be active (the device may also be used in the treatment of totally immobile people) [59]. Even though, none of the user’s joints are activated directly. The device may track users’ performance and send reports wirelessly. However, it does not create nor modify workouts automatically. A physiotherapist’s assistance is still needed then. The InMotionARM consists of a manipulator, a con‐ trol cabinet, an HMI display, and a desk. It may also be enriched with the hand rehabilitation module. The total mass of all the devices is 271 kg, but as the system is placed on a set of wheels, it may be transferred [8].

VOLUME 17,

N∘ 2

2023

3.5. ArmeoPower by Hocoma ArmeoPower is an exoskeleton–type commercial robotic system for an upper extremity rehabilita‐ tion (see Fig. 1e), clinically tested for post–stroke patients [47]. The training with the device is pos‐ sible after attaching an arm and a forearm to the braces and grabbing a handle on the end–effector. The system supports or resists motion along the pro‐ grammed trails. Moreover, it may support the weight of a limb. The ArmeoPower activates up to 6 DOFs directly within a range of a human’s reach [26]. It is also equipped with a 400 mm long electric lifting column to adjust the comfortable training height. The system tracks a patient’s progress and assesses the support needed at further stages. A physiotherapist’s role is limited to creating an initial training set and then adjusting the machine according to the reports. To increase the motivation of a user, the ArmeoPower offers a library of game–like exercises. The system consists of a robot, a control system, and a lat–screen HMI display. It may also be enriched by a hand–rehabilitation module. The ArmeoPower may be transported as it is mounted on the mobile platform, and its overall dimensions are held within 2050x780x1660 mm. The total mass of the device is approximately 205 kg. A room to perform the training should not be smaller than 2.70x3.50x2.00 m. The ArmeoPower may be used for rehabilitation of patients up to 135 kg of weight [1]. 3.6. EksoUE by Ekso Bionics EksoUE is a non–actuated upper–extremity com‐ mercial rehabilitation exoskeleton; thus it is not a robot in a strict sense (see Fig. 1f). However, it is used as a device applied for the treatment of people affected by strokes and neurological or orthopedic diseases to increase their joint reach. The construction supports 5 DOFs per limb and directly activates the shoulder and elbow joints. Its principle of work is based on spring forces, so it can only assist the motion of a patient. The device may provide lift assistance of even 6.8 kg per side. As the construction is drive–less, implementing it into treatment requires constant monitoring by a physiotherapist or even performing parallel manual therapy. The exoskeleton is easy to put on, making it a plug–and–play type device. The mass of the exoskeleton is approximately 5.5 kg. Due to its weight, size, and light design, the EksoUE is a truly portable solution [6]. 3.7. ARMin ARMin is a rehabilitation system for upper extrem‐ ity developed at ETH Zurich university (see Fig. 1g). It is dedicated to the neurorehabilitative training of patients and, above all, to the research on motor learn‐ ing, and therapy [2]. Working out with the device is possible after attaching an arm and a forearm to the segments of an exoskeleton and grabbing a handle on the end–effector by a patient. The ARMin enables 6 DOFs motion and activates joints of a limb directly within their anatomical range [63]. 63

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

The device tracks the changes during the treat‐ ment process and assesses a patient’s needs. The ARMin system was also prepared and tested to be used for game therapy with biosignals involved. However, as the device is still in progress with clinical testing, it is not available for wide use [68]. The system consists of an exoskeleton on a vertical column and a control cabinet. It can also be attached to an HMI display (e.g. computer screen). Due to its relatively big size and no wheels assembled, the ARMin requires additional equipment to be transported to another place [2].

4. Commercial and Developing Devices for Robot–aided Rehabilitation of Lower Extremity 4.1. Lokomat by Hocoma Lokomat is a commercial, treadmill–based, sta‐ tionary rehabilitation system for lower extremities, applicable for post–stroke and neurological ther‐ apy [51, 56] (see Fig. 2a). It offers the natural patterns of gait [28] and the bodyweight support of a patient (up to 85 kg). To train with the device, a patient needs to get fastened in the lift and have their lower limbs attached to the exoskeleton (however, conventional training without exoskeleton on is also possible) [33]. The therapy is based on walking on the treadmill, which may be speeded up to the velocity of 3.2 km/h (up to 10 km/h without gate orthosis). The device activates hip and knee joints directly (2 DOFs per limb) but does not lock other natural DOFs. Therefore, it is possible to be used for function–oriented rehabilita‐ tion [74]. The system tracks a user’s performance and presents this to the physiotherapist and the patient. A therapist is required to set up the irst workout plan and then adjust it based on the reports from the device. However, they may organize more than one session at the same time. Also, the Lokomat offers a wide range of game–like exercises, which are designed to increase the motivation of patients. The system consists of an exoskeleton mounted on the pelvic support moving along the vertical col‐ umn, a bodyweight support lift, a treadmill, and a display for augmented performance feedback. Its mass is approximately 1000 kg, and its overall dimensions vary up to 3500x2140x2460 mm. Moreover, the sys‐ tem requires a minimum room of 5x4x2.6 m. Thus, it is not a transferable solution and may be used only in specialist clinics. The Lokomat is suitable for patients up to 135 kg of weight [10]. 4.2. ReoAmbulator by Motorika ReoAmbulator is a commercial stationary, treadmill–based system for lower extremity rehabilitation (see Fig. 2b) [33]. It is dedicated to gait training, with possible bodyweight support. A patient has to attach their thighs and calves to the mechanical legs to begin a workout on the treadmill. However, conventional gait training without any additional robotized support is also possible. 64

Figure 2. Contemporary rehabilitation robotic systems for lower extremities – a) Lokomat [10], b) ReoAmbulator [15], c) G–EO [16], d) HAL [7], e) ReWalk Personal 6.0 [18], f) ReStore [18], g) EksoNR [5], h) MotionMaker [9] , i) Anklebot [20] , j) PHYSIOTHERABOT [42], k) RENUS–1 [14]

The integrated treadmill may move with a velocity of up to 3.5 km/h (up to 10 km/h without mechanical legs applied). Every mechanical leg activates 2 DOFs (1 DOF at heap and knee joints each) directly and supports the vertical motion of the pelvis. What is more, the construction of these results in a complex motion of extremities mimicking natural gait patterns. The ReoAmbulator tracks and analyzes the per‐ formance of a user and provides a physiotherapist with this information, so they are responsible only for setting up a training and then modifying it according to the received reports. Also, a machine gives real– time visual and audio feedback, which may be helpful for immediate improvement of technique. The system also involves virtual reality and game–like exercises to maximize users’ engagement in treatment. The system consists of two exoskeletons (mechan‐ ical legs) mounted on the modules moving along the vertical column, a bodyweight support lift, a treadmill, and two displays, one for the patient and one for the physiotherapist.

Journal of Automation, Mobile Robotics and Intelligent Systems

The treadmill ramp is designed to allow accessing it with a wheelchair. The mass of a whole system equals approximately 960 kg while its overall dimen‐ sions reach up to 4050x1310x2750 mm, dependent on the chosen modules. Even though the device is equipped with wheels, it is typically used stationary in specialist clinics. To obtain lexibility of use, the producer of the device declares the time of adjusting it to the features of the user to be no longer than 10 minutes. The RoboAmbulator can be used for the treatment of patients up to 150 kg of weight and 90– 200 cm tall [15]. 4.3. G–EO by Reha Technology G–EO is a commercial stationary system dedicated to gait training (see Fig. 2c). It enables a diversity of motion patterns such as walking, climbing stairs or slopes, and backward trajectories; all these with the optional dynamic bodyweight support [64]. To train with the device, a patient has to attach their feet to the holders at the end effectors. The system can perform gait–like movements up to 2.3 km/h while activating 3 DOFs per limb. However, none of the extremities’ joints is mobilized directly. To receive the most natural effect of treatment, the G–EO mimics genuine patterns of human motion [23]. Even though the system gathers and processes diverse data on treatment, it requires a physiother‐ apist to operate it. Nevertheless, limiting persons responsible for rehabilitation based on climbing–like exercises is more advantageous than conventional therapy. The G–EO may be additionally equipped with an FES (Functional Electrical Stimulation) module muscles or a heartbeat and blood oxygen tracking system. Also, the device uses virtual scenarios to make the rehabilitation process more entertaining. The system consists of two manipulators attach‐ able to the patient’s feet and equipped with tactile sensors, a bodyweight support harness, a construction rack, a computer of the physiotherapist, acting as an HMI, and the control system with the patient’s display for the virtual reality. The mass of a whole setup is approximately 900 kg, while its overall dimensions are held within the 4060x1240x2800 mm space. The G–EO is generally used stationary in specialist clinics, as it is relatively heavy and requires constant moni‐ toring by a therapist. It is suitable for patients up to 150 kg of weight and 90–200 cm tall [16, 65]. 4.4. HAL by Cyberdyne HAL is a commercial cyborg–type device. The pro‐ ducer does not exactly designate its application; how‐ ever, it may be used for medical purposes (see Fig. 2d). Besides the overall wearable robot, the HAL is also available in a few different versions. One of them is an exoskeleton (one–legged or two–legged) for reha‐ bilitating lower extremities for patients with muscu‐ loskeletal ambulation disabilities. The device activates 2 DOFs per limb directly (one in a hip joint and one in a knee joint) [27, 72]. The exoskeleton’s motion depends on the bio– electrical signal (BES) control scheme, so the device

VOLUME 17,

N∘ 2

2023

tries to follow the intentions of the move triggered by the user. Thus, the robot teaches a patient how to activate the areas of the neurological system responsi‐ ble for speci ic movements [29, 61]. As the HAL is not strictly a rehabilitation device, it does not require the operation of the physiotherapist. However, it may be bene icial, as the system enables manipulating opera‐ tions of the exoskeleton with a detachable controller. Hence, this method may be used for robot–aided motor therapy. Also, the device allows monitoring the status of a user graphically. The system consists of a complete exoskeleton with a freely detachable controller. It is a truly portable solution, as the mass of a whole system is approximately 9 kg while its overall dimensions are 430x470x1230 mm. The HAL is suitable for patients of 150–200 cm height, and a mass of 40–100 kg, wearing shoes 23–30 cm long [72]. The exoskeleton could be used for home rehabilitation; however, it is relatively expensive, and its battery enables only one–hour of operation [7]. 4.5. ReWalk Personal 6.0 by ReWalk ReWalk Personal 6.0 is a commercial exoskeleton– type robot dedicated to gait–support (see Fig. 2e) [33]. However, thanks to its ability to mimic natural walk‐ ing patterns, the device may also be used for post– stroke therapy. The device is controlled by sensing subtle changes in the patient’s centre of gravity. It may activate 2 DOFs per limb (one–legged and two–legged versions are available) directly and assist everyday motion at home or in the community, as well as be used for motor treatment. The maximum speed of gait achievable with the assistance of the system is 2.6 m/s [54]. The exoskeleton is designed for people with spinal injuries, as the ReWalk company also offers wear‐ able devices ReStore, dedicated explicitly to rehabili‐ tation purposes. However, compared to them, ReWalk Personal 6.0 is applicable for the cases of immobile patients [78]. As the device is applicable for everyday activities support, it does not require the assistance of a physiotherapist. It may be treated as a rehabil‐ itation device as it stimulates the brain to recall the motion stimuli and connect them with the particular gait swings. The whole system is packed in the exoskeleton. As its construction is based on a lightweight exoskeleton, the ReWalk Personal 6.0 could be applied in home rehabilitation; however, it is relatively expensive and available only in some countries (mainly in Germany, Italy and the United Kingdom in Europe). The wear‐ able device is suitable for patients 160–190 cm tall, and up to 100 kg of weight [18, 55]. 4.6. ReStore by ReWalk ReStore is a commercial soft exoskeleton–type device for post–stroke rehabilitation (see Fig. 2f). It is only applicable to the therapy of patients able to walk with the support of any additional device. The soft exoskeleton is placed like a calf orthosis and activates directly 1 DOF of the ankle joint [48]. 65

Journal of Automation, Mobile Robotics and Intelligent Systems

The ReStore is designed to support gait by improv‐ ing its symmetric technique and increasing the speed. As the solution is relatively simple and safe, no assis‐ tance of physiotherapy is required [24]. It is driven based on the data from the motion sensors to syn‐ chronise the limp extremity’s swing with the one non– injured. The system consists of a soft exoskeleton, and its control system is placed on the belt connected with the wiring. However, the device is also remotely accessi‐ ble with a mobile application. It is the most portable rehabilitation robot, designed only for one segment of a limb. However, it does not give a possibility of holis‐ tic and more advanced therapy, and it is still mainly used locally in the specialist European and American clinics [18, 24]. 4.7. EksoNR by Ekso Bionics EksoNR is a lower–body commercial rehabilitation exoskeleton for gait training; thus, in contrast to the EksoUE it is a typical rehabilitation robot (see Fig. 2g). It may be applied for motor therapy and posture sup‐ port for people suffering from various diseases. Its construction enables activation of 2 DOFs per side when mobilizing directly, one at the hip and one at the knee joint. The maximum speed of gait achievable with the assistance of the system is 1.6 m/s [54]. The exoskeleton gathers data on speed, distance, and gait training time. Afterwards, this may be pro‐ cessed and used to improve treatment and correct the common technique mistakes. Also, the intelligent software adapts to the users’ needs and optimizes the workouts to increase their effectiveness. As the device is designed to be operated by a physiotherapist, the system allows adjusting swing support and other parameters of strides. The system consists of an exoskeleton and a con‐ trol panel with a display to present data on perfor‐ mance to the patient and the therapist. The EksoNR is relatively compact and has a mass of 25 kg itself; however, it is still only used in specialist clinics. The exoskeleton is suitable for patients of 150–195 cm tall and up to 100 kg of weight [5, 72]. 4.8. MotionMaker by Swortec MotionMaker is a stationary system for rehabil‐ itation of lower extremities, dedicated for disabled people (see Fig. 2i) [33]. Originally, it was developed at the Swiss Federal Institute of Technology Lausanne, and then the concept has transformed into a start–up. The device is designed to train both limbs while sitting by performing programmed routines with parallel Func‐ tional Electrical Stimulation (FES) [9]. To begin the session, a patient has to attach their extremities into two exoskeletons – by their feet, calves and thighs. The system activates 3 DOFs per extremity when mobi‐ lizing directly, one at the hip, knee and ankle joints respectively [71]. The device must be operated by the physiothera‐ pist. Due to its university background, besides clinical application, it has also been used as a research device. 66

VOLUME 17,

N∘ 2

2023

The design of the MotionMaker has been devel‐ oped, and in 2011 the company presented a new rehabilitation robot, WalkTrainer, dedicated to gait training [9]. The original system consists of the main device with two exoskeletons and a control system with an HMI display. Even though they all are attached to the mobile platform, the MotionMaker is rather a sta‐ tionary solution due to its size and mass of 210 kg. However, it is placed on wheels, making the device possible to relocate. Its overall dimensions are equal to 1520x750x1580 mm; however, it requires 2x4x2.2 m free room to operate. The exoskeletons it people 140– 195 cm tall, and of weight up to 135 kg [4]. 4.9. Anklebot Anklebot is a lower extremity rehabilitation exoskeleton–type robot constructed at the Massachusetts Institute of Technology (see Fig. 2j). It is dedicated to people suffering after strokes. To use it, a patient has to attach their calf to the brace and place a foot in the dedicated shoe holder. Afterwards, the device moves the foot along its natural trajectories within the ergonomic range of an ankle. The robot activates directly 2 DOFs of this joint [75]. The device was widely used for research purposes. Among others, it contributed to determining the stiff‐ ness of ankles for people with paralyse. However, due to its early stage of market–readiness, the Anklebot requires operating by a therapist [69, 70]. The system consists of an exoskeleton with its con‐ trol system. The mechanism itself is relatively low– weight and could be used nearly anywhere [75]. Due to its construction, it may be applied for training in sit‐ ting, lying or standing positions. Moreover, it could be used for gait training with a treadmill. Nevertheless, it is still in the test phase. The company Bionik is trying to develop the product as InMotion Ankle and wants release it to sell. So far, the device has completed pre– clinical tests [20]. 4.10. PHYSIOTHERABOT PHYSIOTHERABOT is a Turkish system designed at the Yildiz Technical University (see Fig. 2h). It consists of two devices – PHYSIOTHERABOT, an exoskeleton for motor rehabilitation of both extremities, and PHYSIOTHERABOT/W1, an exoskeleton for the physiotherapy of a wrist and elbow. The universal robot requires attachment of either a forearm or a thigh and a calf to the braces to begin treatment. The wrist–and–elbow– rehabilitation device must be attached to the forearm and grabbed by the handle. Both the machines may activate 3 DOFs of a limb directly and be used for either active or passive rehabilitation [21]. The rehabilitation process takes place in a sitting position. Operating the system is possible even with a professional staff involved only remotely via the HMI system. Both the devices gather the data and use them in a feedback loop to self–adjust [22]. The PHYSIOTHERABOT consists of an exoskele‐ ton attached to the seat, its control system and an HMI based on a PC. The PHYSIOTHERABOT/W1 set

Journal of Automation, Mobile Robotics and Intelligent Systems

is the same apart from the seat and the different construction of an exoskeleton. Both parts of the sys‐ tem may be used separately. The mass and sizes of the devices make them capable of transport to the patients’ houses. However, they are not approved for commercial use yet. Thus, they are still treated as research equipment to develop advanced control tech‐ nologies for medical devices, also involving AI–based algorithms and EMG tracking [12, 13, 22, 31]. 4.11. RENUS RENUS is a Polish post–stroke rehabilitation sys‐ tem designed at the Industrial Institute for Automation and Measurements PIAP (see Fig. 2k). It consists of two devices – a manipulator, RENUS–1, dedicated for the upper extremity and a manipulator, RENUS–2, dedicated for the lower extremity [53]. They both may be used for active or passive treatment. To do so, a patient has to either grab a handle or place a foot in the shoe–holder; respectively, for the device. Each of the machines is capable of activating 3 DOFs of a limb indirectly [34]. Rehabilitation of an upper limb may be realized in either sitting or standing position, while the training of a lower limb requires remain seated. Using the RENUS system may be done only under constant supervision of a professional therapist [52]. The devices are relatively big and heavy. Moreover, they have never completed the clinical trial tests. Due to these, they were treated only as research equipment to assess the capabilities of robot–aided treatment. Hence, they cannot be used as the system for home rehabilitation.

5. Comparison of the Systems The presented devices are compared in terms of their potential for home rehabilitation. As intended by the questioned physiotherapist, the systems should be capable of multi–joint mobilization to recall natural movement patterns. Also, they are expected to enable remote control over the physiotherapy process and constant monitoring of the patient. Moreover, they should be usable in the limited space of lats. The main factors enabling these are their kinematics structure, size, weight, transport dif iculty, commercial avail‐ ability, minimum room size needed, ICT technologies implemented, and requirements for patients and phys‐ iotherapists. These are presented in the Tables 1–5. Even though there are multiple commercial sys‐ tems available, there were no standardized trials con‐ ducted to systematically compare their ef icacy in recalling life functions to the patients. This means assessment of their applicability for remote home applications can be based only on their functions and investigated needs of the physiotherapists [39]. The colors of cells in the tables depend on an impact of a certain parameter on the device’s applica‐ bility for home rehabilitation. ‐ Green cells contain favorable parameters; ‐ Yellow cells contain parameters, which may hinder home–rehabilitation with the device;

VOLUME 17,

N∘ 2

2023

Table 1. Comparison of the rehabilitation systems in terms of their kinematics structure (abbr.: physio...–physiotherabot) Device Physio ReoGo Burt InMotionARM Armeo Power EksoUE ARMin PHYSIO.../W1 RENUS–1 Lokomat ReoAmbulator G–EO HAL ReWalk ReStore EksoNR MotionMaker Anklebot RENUS–2

Extremity Upper Upper Upper Upper Upper Upper Upper Upper Upper Lower Lower Lower Lower Lower Lower Lower Lower Lower Lower

DOFs 3 2–3 3 2 6 5 (10) 6 3 3 2 (4) 2 (4) 3 (6) 2 (4) 2 (4) 1 2 (4) 3 (6) 2 3

PHYSIO...

Upper/Lower

Activation of Joints Indirect Indirect Indirect Indirect S, E, W (1) S, E S, E, W (1) E (1), W Indirect H (1), K H (1), K Indirect H(1), K H(1), K A (1) H (1), K H (1), K, A (1) A (1) Indirect S (1), E (1), W (1) H (1), K, A (1)

Table 2. Comparison of the rehabilitation systems in terms of their availability and offered rehabilitation schemes (abbr.: physio...–physiotherabot) Device Physio ReoGo Burt InMotionARM Armeo Power EksoUE ARMin PHYSIO.../W1 RENUS–1 Lokomat ReoAmbulator G–EO HAL ReWalk ReStore EksoNR MotionMaker Anklebot RENUS–2 PHYSIO...

Motion Plane Plane Any Plane Any Any Any Dłoni Any Gait Gait Different types of gait Gait (learning by practicing) Gait (learning by practicing) Ankle joint (gait) Gait (learning by practicing) Any Ankle joint Any Any

Position Any Sitting Any Sitting Any Any Any Any Any Standing Standing

Availability Commercial Commercial Commercial Commercial Commercial Commercial Commercial R&D R&D Commercial Commercial

Standing

Commercial

Standing

Commercial

Standing

Commercial

Standing

Commercial

Standing

Commercial

Sitting Sitting Sitting Sitting

Commercial Commercialised R&D R&D

‐ Red cells contain parameters which hinder and may even prevent the devices from being applied for home rehabilitation.; ‐ Grey cells contain parameters with no data available. As presented, the contemporary devices dedi‐ cated to robot–aided rehabilitation do not necessar‐ ily meet all the desired parameters to be applied for home rehabilitation. The most signi icant one is the lack of remote connection to the control sys‐ tems (see Table 4). This is no surprise, as the machines are designed to be used in the clinics. There, physiotherapists may adjust the therapy parameters 67

Journal of Automation, Mobile Robotics and Intelligent Systems

N∘ 2

Table 3. Comparison of the rehabilitation systems in terms of their physical properties and requirements (abbr.: physio...–physiotherabot, n/a no–data, v. difficult–very difficult)

Table 5. Comparison of the rehabilitation systems in terms of requirements towards patients and physiotherapists (abbr.: physio...–physiotherabot, n/a–no data)

Device Physio ReoGo Burt InMotionARM Armeo Power EksoUE ARMin PHYSIO.../W1 RENUS–1 Lokomat ReoAmbulator G–EO HAL ReWalk ReStore EksoNR MotionMaker Anklebot RENUS–2 PHYSIO...

Device Physio ReoGo Burt InMotionARM Armeo Power EksoUE ARMin PHYS.../W1 RENUS–1 Lokomat ReoAmbulator G–EO

Paralysed patients After modi ication Yes Yes Yes Yes With assistance Yes Yes After modi ication Yes Yes Yes

HAL

Yes

ReWalk

With assistance

ReStore EksoNR MotionMaker Anklebot RENUS–2 PHYS...

With assistance With assistance Yes Yes Yes Yes

Mass [kg] 15 79 80 271 205 5.5 N/A N/A N/A 1000 960 900 9 N/A N/A 25 210 3 N/A N/A

Size [cm] 30x40x80 101x58x90 N/A N/A 205x78x166 N/A N/A N/A N/A 350x214x246 405x131x275 406x124x280 43x47x123 N/A N/A N/A 152x75x158 N/A N/A N/A

Transport Possible Possible Possible Possible Possible Possible Dif icult Dif icult Dif icult V. dif icult V. dif icult V. dif icult Possible Possible Possible Possible Possible Possible Dif icult Dif icult

Room [m] N/A N/A N/A N/A 2.7x3.5x2.0 – N/A N/A N/A 5.0x4.0x2.6 N/A N/A – – – – 2.0x4.0x2.2 – N/A N/A

Table 4. Comparison of the rehabilitation systems in terms of involved ICT technologies (abbr.: physio...–physiotherabot, n/a–no data) Device Physio ReoGo Burt InMotionARM Armeo Power EksoUE ARMin PHYS.../W1 RENUS–1 Lokomat ReoAmbulator G–EO HAL ReWalk ReStore EksoNR MotionMaker Anklebot RENUS–2 PHYS...

Performance feedback Yes Yes Yes Yes Yes No Yes Yes No Yes Yes Yes Yes Yes No Yes N/A N/A No Yes

VR/game

Biosignals

Yes Yes Yes Yes Yes No Yes Yes No Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes

No No No No No No Tested Tested No No No Yes Yes Yes No No Yes No No Tested

Remote connection N/A N/A N/A Yes N/A No N/A Yes No No N/A N/A N/A N/A Yes No N/A N/A No Yes

according to their observations. As to providing a possibility of telerehabilitation, the safe remote con‐ nection technology needs to be implemented to the devices designed for that aim [67]. Another inding regarding the current rehabilitation–devices market is a tendency to implement virtual reality (VR) tech‐ nologies (see Table 4), even though these do not yet guarantee a constant level of entertainment. Thus, they are not solving the need to increase a patient’s engagement within the whole therapy. However, as the market tends to apply VR and people are more familiar with this, it may be good to keep this trend and enhance the quality of user experience. Based on the questionnaire research results, the devices for home remote motor therapy of extremities should be designed to: ‐ Provide stable remote connection to adjust the workout parameters and monitor the patient’s measurable progress (e.g., by analyzis of tracked 68

VOLUME 17,

2023

Physiotherapist’s duties Initial workout, supervision Workout, supervision Workout, supervision Workout, supervision Workout, supervision None, or additional manual therapy Workout, supervision Initial workout, remote supervision N/A Workout, supervision Workout, supervision Workout, supervision Training in operation and parameters adjustment Training in operation and parameters adjustment None Setting the parameters Workout, supervision N/A Workout, supervision Initial workout, remote supervision

biosignals or information on sensed forces). So far, only four of the analyzed devices has such function‐ ality. In the future, the rapid development of the digital twin and VR technologies may contribute to creating the methodology of remote manual robot– aided therapy. It could be based on dragging a vir‐ tual copy of the rehabilitated extremity and causing corresponding motion of the exoskeleton used by the patient; ‐ Enable motion along with the natural patterns. Therefore, they should not constrain degrees of free‐ dom in particular joints, but not necessarily by sup‐ porting every degree of freedom with drives. Only seven analyzed devices were not over–constraining natural DOFs of the extremity; ‐ Offer comprehensive treatment capabilities thanks to the implemented ICT technologies rather than powerful mechanical components. So far, only four of the analyzed devices have biosignals tracking implemented, while all but two use VR or another game environment. The mechanical design should enable transporting the device to the patients’ lats and using them onsite. Therefore, they should it the standard housing doors and not exceed the load capacity of the loors by weight. Hence, devices with treadmills or additional lift bars are not recom‐ mended. It is worth noticing that the devices for rehabili‐ tation of lower extremities are more likely to be used for home therapy. This may be, because most of them are dedicated to gait training, which needs to be per‐ formed daily, if it is to be effective. Therefore, their technological advancement is usually also superior to the devices for upper limbs. For this reason, the mar‐ ket shall be more open to new rehabilitation systems for upper extremities, especially portable ones, involv‐ ing innovative ICT technologies. Additionally, the devices for upper extremities, which are the most complex in terms of implemented

Journal of Automation, Mobile Robotics and Intelligent Systems

technologies, are also the most complex in their con‐ struction. Hence, the ICT sophistication and porta‐ bility are not likely to characterize one system. It is signi icant, as these parameters are required to enable effective telerehabilitation – so far, rather presented as advanced conceptual studies [40] or research [32], but without real–life implementation .

6. Conclusion The overview of technologically advanced or widely available in Europe rehabilitation robots of extremities was presented to assess their applica‐ bility for remote home kinesiotherapy. The main requirements for them were based on prior in–depth interviews and questionnaire research. Considered parameters of the robots were gathered in tables and analyzed in terms of corresponding to the de ined needs. According to the observations, there is no com‐ mercial and widespread device enabling full remote home rehabilitation of upper extremities. The sys‐ tems offered for the clinics typically suffer from a lack of remote connection and technologies allowing telerehabilitation. As the role of a local operator is sig‐ ni icant, a physiotherapist may not fully use the advan‐ tage of intelligent algorithms adjusting workouts to maximize their effectiveness. Therefore, the presented technological gaps must be completed with the high‐ est priority. Nevertheless, the policy of constant devel‐ opment of ICT technologies shall be continued, as their effectiveness is con irmed with multiple studies; also for post–stroke and home therapy in particular [44,60, 80]. The presented insights on the market of rehabili‐ tation robots shape the direction of its development. As the concept of telerehabilitation becomes a real‐ ity, the devices shall enable the effective transfer of multiple and various sensory stimuli to the operator. Thanks to this, the physiotherapists will get suf icient information to monitor and lead a workout. More‐ over, the devices need to be both portable and afford‐ able. Only with these the rehabilitation systems may be leased to the patients. Thus, the production costs need to be decreased. It may be realized by replacing redundant sensors and non–required activated DOFs with intelligent algorithms. Also, with the technol‐ ogy advancement, the commonly used devices, such as smartphones, computers or smartwatches, could enrich the rehabilitation system with everyday col‐ lected data. Even with this, the devices would remain signi icantly more expensive than single therapy ses‐ sions. Thus, they will be still dedicated to the people who need long–term or intensive therapy, post–stroke among others [43]. Based on the conducted research, the main trends for the future include robotization of the task–oriented motor treatment and providing teleoperation. The technology likely to be used for such application is a digital twin, possibly combined with advanced sensors and VR/AR [40]. With aris‐ ing interest on Metaverse, also this platform can con‐ tribute to the development of telehealth. Moreover, the appropriate safety means have to be developed. The technologies expected to be involved for this

VOLUME 17,

N∘ 2

2023

purpose are EMG tracking, simple wearable devices (related with mHealth), and AI analyzes of the treat‐ ment [79]. The neural networks can be used both in diagnoses and supporting control over treatment [38]. To provide truly available solutions, the devices offered for the different local markets should differ one from another. It is a consequence of the differ‐ ences in economic situation among world regions, and even more critical – differing technology aware‐ ness of their societies. Moreover, some of the dis‐ eases restrain the variety of exercises allowed. These aspects may be considered by designing the system holistically but as a set of freely attachable modules. With this approach, the system’s applicability is not limited to particular cases, which contributes to the popularization of robot–aided treatment. AUTHORS Piotr Falkowski∗ – ŁUKASIEWICZ Research Network – Industrial Research Institute for Automation and Measurements PIAP, Al. Jerozolimskie 202, Warsaw Warsaw University of Technology, 02‐486, Plac Politechniki 1, Warsaw, 00‐661, e‐mail: piotr.falkowski@piap.lukasiewicz.gov.pl. Cezary Rzymkowski – Warsaw University of Tech‐ nology, 02‐486, Plac Politechniki 1, Warsaw, 00‐661, e‐mail: cezary.rzymkowski@pw.edu.pl. Zbigniew Pilat – ŁUKASIEWICZ Research Network – Industrial Research Institute for Automation and Measurements PIAP, Al. Jerozolimskie 202, Warsaw, e‐mail: zbigniew.pilat@piap.lukasiewicz.gov.pl. ∗

Corresponding author

References [1] “Armeo power website [online]”. https://ww w.hocoma.com/solutions/armeo‐ power/. Accessed: 2020‐10‐21. [2] “Armin website [online]”. https://sms.hest.ethz. ch/research/current‐research‐projects/armin‐ robot. Accessed: 2020‐10‐23. [3] “Barrett technology website [online]”. https:// medical.barrett.com/. Accessed: 2020‐10‐22. [4] “The datasheet of motion maker by swortec [online]”. http://www.swortec.ch/index.p hp/products/motionmaker. Accessed: 2020‐ 11‐02. [5] “Eksonr by ekso bionics website [online]”. https: //eksobionics.com/eksonr/. Accessed: 2021‐ 01‐15. [6] “Eksoue by ekso bionics website [online]”. https: //eksobionics.com/eksoue/, Accessed: 2020‐ 10‐23. [7] “Hal by cyberdene website [online]”. https://ww w.cyberdyne.jp/. Accessed: 2021‐01‐15. [8] “Inmotionarm by bionik labs website [online]”. https://www.bioniklabs.com/products/inmoti on‐arm. Accessed: 2020‐10‐23. 69

Journal of Automation, Mobile Robotics and Intelligent Systems

N∘ 2

2023

[9] “An interview with hans peter gmünder. pwc [online]”. https://magazine.pwc.ch/en/item‐ detail‐view/partake‐in‐life. Accessed: 2021‐ 01‐15.

clinical trial of the safety, reliability, and feasibility of exosuit‐augmented post‐stroke gait rehabilitation”, Journal of neuroengineering and rehabilitation, vol. 17, no. 1, 2020, 1–11.

[10] “Lokomat by hocoma website [online]”. https: //www.hocoma.com/solutions/lokomat/. Accessed: 2021‐01‐15.

[25] P. Bontje, R. Kruijne, M. Pol, K. Inoue, R. Kobayashi, Y. Ito, and M. Van Hartingsveldt. “Developing an international research of health‐care ict applied for rehabilitation and daily living support between japan and the netherlands”, Assistive Technology, vol. 34, no. 2, 2022, 140–147.

[11] “Physio by gridbots website [online]”. https:// www.gridbots.com/physio. Accessed: 2020‐ 10‐10. [12] “Physiotherabot project website [online]”. http: //ytubiomechatronics.com/portfolio‐item/low er‐limb/, note = Accessed: 2021‐11‐05. [13] “Physiotherabot/w1 website [online]”. http://yt ubiomechatronics.com/portfolio‐item/physiot herabot‐w1/. Accessed: 2021‐11‐05. [14] “Presentation of prototypes of rehabilitation robots by piap (in polish) [online]”. https:// www.youtube.com/watch?v=FSBf5kPEz3k. Accessed: 2021‐01‐15. [15] “Reoambulator by motorika website [online]”. ht tp://motorika.com/reoambulator/. Accessed: 2021‐01‐15. [16] “Reoambulator by motorika website [online]”. ht tps://www.rehatechnology.com/en/. Accessed: 2021‐01‐15. [17] “Reogo by motorika website [online]”. http://mo torika.com/reogo/. Accessed: 2020‐10‐22. [18] “Rewalk website [online]”. https://rewalk.com/. Accessed: 2021‐01‐15. [19] C. Adans‐Dester, A. O’Brien, R. Black‐Schaffer, and P. Bonato. “Upper extremity rehabilitation with the burt robotic arm”, Archives of Physical Medicine and Rehabilitation, vol. 100, no. 12, 2019, e208–e209. [20] J. Ahn, and N. Hogan. “Walking is not like reach‐ ing: evidence from periodic mechanical pertur‐ bations”, PloS one, vol. 7, no. 3, 2012, e31767. [21] E. Akdoğan, and M. A. Adli. “The design and con‐ trol of a therapeutic exercise robot for lower limb rehabilitation: Physiotherabot”, Mechatronics, vol. 21, no. 3, 2011, 509–522. [22] E. Akdogan, and M. E. Aktan. “Impedance control applications in therapeutic exercise robots”. In: Control Systems Design of Bio-Robotics and Biomechatronics with Advanced Applications, 395– 443. Elsevier, 2020. [23] E. Andrenelli, M. Capecci, L. Di Biagio, L. Pepa, L. Lucarelli, C. Spagnuolo, P. Guidoni, P. Ser‐ a ini, F. Morgante, and M. Ceravolo. “Improving gait function and sensorimotor brain plasticity through robotic gait training with g‐eo system in parkinson’s disease”, Annals of Physical and Rehabilitation Medicine, vol. 61, 2018, e79–e80. [24] L. N. Awad, A. Esquenazi, G. E. Francisco, K. J. Nolan, and A. Jayaraman. “The rewalk restore™ soft robotic exosuit: a multi‐site 70

VOLUME 17,

[26] R. S. Calabrò, M. Russo, A. Naro, D. Milardi, T. Bal‐ letta, A. Leo, S. Filoni, and P. Bramanti. “Who may bene it from armeo power treatment? a neuro‐ physiological approach to predict neurorehabil‐ itation outcomes”, PM&R, vol. 8, no. 10, 2016, 971–978. [27] B. Chen, H. Ma, L.‐Y. Qin, F. Gao, K.‐M. Chan, S.‐W. Law, L. Qin, and W.‐H. Liao. “Recent devel‐ opments and challenges of lower extremity exoskeletons”, Journal of Orthopaedic Translation, vol. 5, 2016, 26–37. [28] Y. Cherni, M. Hajizadeh, F. Dal Maso, and N. A. Turpin. “Effects of body weight support and guidance force settings on muscle synergy during lokomat walking”, European Journal of Applied Physiology, vol. 121, no. 11, 2021, 2967–2980. [29] O. Cruciger, T. A. Schildhauer, R. C. Meindl, M. Tegenthoff, P. Schwenkreis, M. Citak, and M. Aach. “Impact of locomotion training with a neurologic controlled hybrid assistive limb (hal) exoskeleton on neuropathic pain and health related quality of life (hrqol) in chronic sci: a case study”, Disability and Rehabilitation: Assistive Technology, vol. 11, no. 6, 2016, 529–534. [30] S. De Biase, L. Cook, D. A. Skelton, M. Witham, and R. Ten Hove. “The covid‐19 rehabilitation pandemic”, Age and ageing, vol. 49, no. 5, 2020, 696–700. [31] U. Demir, S. Kocaoğlu, and E. Akdoğan. “Human impedance parameter estimation using arti icial neural network for modelling physiotherapist motion”, Biocybernetics and Biomedical Engineering, vol. 36, no. 2, 2016, 318–326. [32] I. Díaz, J. M. Catalan, F. J. Badesa, X. Justo, L. D. Lledo, A. Ugartemendia, J. J. Gil, J. Díez, and N. García‐Aracil. “Development of a robotic device for post‐stroke home tele‐rehabilitation”, Advances in Mechanical Engineering, vol. 10, no. 1, 2018, 1687814017752302. [33] I. Díaz, J. J. Gil, and E. Sánchez. “Lower‐limb robotic rehabilitation: literature review and challenges”, Journal of Robotics, vol. 2011, 2011. [34] J. Dunaj, W. J. Klimasara, and Z. Pilat. “Human‐ robot interaction in the rehabilitation robot renus‐1”. In: International Conference on Systems, Control and Information Technologies 2016, 2016, 358–367.

Journal of Automation, Mobile Robotics and Intelligent Systems

[35] D. Eguren, M. Cestari, T. P. Luu, A. Kilicarslan, A. Steele, and J. L. Contreras‐Vidal. “Design of a customizable, modular pediatric exoskeleton for rehabilitation and mobility”. In: 2019 IEEE international conference on systems, man and cybernetics (SMC), 2019, 2411–2416. [36] P. Falkowski. “Light exoskeleton design with topology optimisation and fem simulations for fff technology”, Journal of Automation, Mobile Robotics and Intelligent Systems, 2021, 14–19. [37] P. Falkowski. “An optimisation problem for exoskeleton‐aided functional rehabilitation of an upper extremity”. In: IOP Conference Series: Materials Science and Engineering, vol. 1239, no. 1, 2022, 012012. [38] P. Falkowski. “Predicting dynamics of a rehabili‐ tation exoskeleton with free degrees of freedom”. In: Conference on Automation, 2022, 223–232. [39] P. Falkowski, T. Osiak, and A. Pastor. “Analy‐ sis of needs and requirements of kinesiother‐ apy in poland for robot design purposes”, Prace Naukowe - Politechnika Warszawska. Elektronika z. 197, Postępy robotyki. T. 2, 2022. [40] P. Falkowski, T. Osiak, J. Wilk, N. Prokopiuk, B. Leczkowski, Z. Pilat, and C. Rzymkowski. “Study on the applicability of digital twins for home remote motor rehabilitation”, Sensors, vol. 23, no. 2, 2023, 10.3390/s23020911. [41] S. Faran, O. Einav, D. Yoeli, M. Kerzhner, D. Geva, G. Magnazi, S. van Kaick, and K.‐H. Mauritz. “Reo assessment to guide the reogo therapy: Reliabil‐ ity and validity of novel robotic scores”. In: 2009 Virtual Rehabilitation International Conference, 2009, 209–209.

VOLUME 17,

N∘ 2

2023

[46] N. Iivari, S. Sharma, and L. Ventä‐Olkkonen. “Digi‐ tal transformation of everyday life–how covid‐19 pandemic transformed the basic education of the young generation and why information manage‐ ment research should care?”, International Journal of Information Management, vol. 55, 2020, 102183. [47] M. Iosa, A. Martino Cinnera, F. Capone, A. Cru‐ ciani, M. Paolucci, V. Di Lazzaro, S. Paolucci, and G. Morone. “Clinical interpretation of work‐ ing volume and weight support in upper limb robotic neurorehabilitation after stroke”, Applied Sciences, vol. 11, no. 24, 2021, 12123. [48] L. J. Jasinski. “Structural exoskeletons and soft fabric exosuits for assistive walking”. In: Wearable Robotics, 311–333. Elsevier, 2020. [49] B. A. Jnr. “Use of telemedicine and virtual care for remote treatment in response to covid‐19 pandemic”, Journal of medical systems, vol. 44, no. 7, 2020, 1–9. [50] G. J. Kim, J. Hinojosa, A. K. Rao, M. Batavia, and M. W. O’Dell. “Randomized trial on the effects of attentional focus on motor training of the upper extremity using robotics with individuals after chronic stroke”, Archives of physical medicine and rehabilitation, vol. 98, no. 10, 2017, 1924–1931. [51] J. H. Kim. “Effects of robot‐assisted therapy on lower limb in patients with subacute stroke”, Journal of the Korea Academia-Industrial cooperation Society, vol. 17, no. 7, 2016, 459–466. [52] W. Klimasara, J. Dunaj, P. Stempniak, and Z. Pilat. “Renus‐1 and renus‐2, the assisted robots sys‐ tem for after stroke mobility rehabilitation”, Prace Naukowe Politechniki Warszawskiej. Elektronika, no. 175, t. 1, 2010, 55–62.

[42] P. A. Gómez, M. D. Rodríguez, and V. Amela. “Design of a robotic system for diagnosis and rehabilitation of lower limbs”, arXiv preprint arXiv:1710.08126, 2017.

[53] W. J. Klimasara, A. Bratek, M. Pachuta, and Z. Pilat. “Systemy mechatroniczne w rehabilitacji ruchowej”, Pomiary Automatyka Robotyka, vol. 13, no. 2, 2009, 577–586.

[43] T. Gueye, M. Dedkova, V. Rogalewicz, M. Grunerova‐Lippertova, and Y. Angerova. “Early post‐stroke rehabilitation for upper limb motor function using virtual reality and exoskeleton: equally ef icient in older patients”, Neurologia i Neurochirurgia Polska, vol. 55, no. 1, 2021, 91–96.

[54] K. Kong, J. Choi, K.‐W. Park, J. Park, D.‐H. Lee, E. Song, B. Na, S. Jeon, T. Kim, H. Choi, et al. “The history and future of the walkon suit: A powered exoskeleton for people with disabilities”, IEEE Industrial Electronics Magazine, 2021.

[44] M. Gustavsson, C. Ytterberg, and S. Guidetti. “Exploring future possibilities of using informa‐ tion and communication technology in multidis‐ ciplinary rehabilitation after stroke–a grounded theory study”, Scandinavian journal of occupational therapy, vol. 27, no. 3, 2020, 223–230. [45] Y.‐w. Hsieh, K.‐c. Lin, C.‐y. Wu, T.‐y. Shih, M.‐w. Li, and C.‐l. Chen. “Comparison of proximal ver‐ sus distal upper‐limb robotic rehabilitation on motor performance after stroke: a cluster con‐ trolled trial”, Scienti ic reports, vol. 8, no. 1, 2018, 1–11.

[55] D. Kuhn and B. Freyberg‐Hanl. “Exoskelett: Ther‐ apiesystem oder hilfsmittel zum behinderung‐ sausgleich”, Trauma und Berufskrankheit, vol. 20, no. 4, 2018, 254–259. [56] H. Y. Lee, J. H. Park, and T.‐W. Kim. “Comparisons between locomat and walkbot robotic gait train‐ ing regarding balance and lower extremity func‐ tion among non‐ambulatory chronic acquired brain injury survivors”, Medicine, vol. 100, no. 18, 2021. [57] M. F. Levin. “What is the potential of virtual real‐ ity for post‐stroke sensorimotor rehabilitation?”, Expert review of neurotherapeutics, vol. 20, no. 3, 2020, 195–197. 71

Journal of Automation, Mobile Robotics and Intelligent Systems

N∘ 2

2023

[58] Y. Liu, X. Li, A. Zhu, Z. Zheng, and H. Zhu. “Design and evaluation of a surface electromyography‐ controlled lightweight upper arm exoskeleton rehabilitation robot”, International Journal of Advanced Robotic Systems, vol. 18, no. 3, 2021, 17298814211003461.

[69] A. Roy, H. I. Krebs, J. E. Barton, R. F. Macko, and L. W. Forrester. “Anklebot‐assisted locomo‐ tor training after stroke: A novel de icit‐adjusted control approach”. In: 2013 IEEE International Conference on Robotics and Automation, 2013, 2175–2182.

[59] S. Macovei, and I. Doroftei. “A short overview of upper limb rehabilitation devices”. In: IOP Conference Series: Materials Science and Engineering, vol. 145, no. 5, 2016, 052014.

[70] A. Roy, H. I. Krebs, S. L. Patterson, T. N. Jud‐ kins, I. Khanna, L. W. Forrester, R. M. Macko, and N. Hogan. “Measurement of human ankle stiffness using the anklebot”. In: 2007 IEEE 10th International Conference on Rehabilitation Robotics, 2007, 356–363.

[60] M. N. Marwaa, H. K. Kristensen, S. Guidetti, and C. Ytterberg. “Physiotherapists’ and occu‐ pational therapists’ perspectives on informa‐ tion and communication technology in stroke rehabilitation”, Plos one, vol. 15, no. 8, 2020, e0236831. [61] K. Miura, M. Koda, K. Tamaki, M. Ishida, A. Marushima, T. Funayama, H. Takahashi, H. Noguchi, K. Mataki, Y. Yasunaga, et al. “Exercise training using hybrid assistive limb (hal) lumbar type for locomotive syndrome: a pilot study”, BMC Musculoskeletal Disorders, vol. 22, no. 1, 2021, 1–8. [62] M. Mohammadi, H. Knoche, M. Thøgersen, S. H. Bengtson, M. A. Gull, B. Bentsen, M. Gaihede, K. E. Severinsen, and L. N. Andreasen Struijk. “Eyes‐free tongue gesture and tongue joystick control of a ive dof upper‐limb exoskeleton for severely disabled individuals”, Frontiers in Neuroscience, vol. 15, 2021, 739279. [63] T. Nef, M. Guidali, V. Klamroth‐Marganska, and R. Riener. “Armin‐exoskeleton robot for stroke rehabilitation”. In: World Congress on Medical Physics and Biomedical Engineering, September 7-12, 2009, Munich, Germany, 2009, 127–130. [64] I.‐A. Nițică, and E. Nechifor. “Antrenamentul mersului cu ajutorul sistemului robotic g‐eo evo‐ lution”, Journal of Physical Rehabilitation and Sports Medicine, no. 2, 2020, 20–29. [65] I.‐A. Nițică, and E. Nechifor. “Antrenamentul mersului cu ajutorul sistemului robotic g‐eo evo‐ lution”, Journal of Physical Rehabilitation and Sports Medicine, no. 2, 2020, 20–29. [66] N. Pavón‐Pulido, J. A. López‐Riquelme, and J. J. Feliú‐Batlle. “Iot architecture for smart control of an exoskeleton robot in rehabilitation by using a natural user interface based on gestures”, Journal of Medical Systems, vol. 44, no. 9, 2020, 1–10. [67] J. Prvu Bettger, and L. J. Resnik. “Telerehabilita‐ tion in the age of covid‐19: an opportunity for learning health system research”, Physical Therapy, vol. 100, no. 11, 2020, 1913–1916. [68] N. Rehmat, J. Zuo, W. Meng, Q. Liu, S. Q. Xie, and H. Liang. “Upper limb rehabilitation using robotic exoskeleton systems: A systematic review”, International Journal of Intelligent Robotics and Applications, vol. 2, no. 3, 2018, 283–295. 72

VOLUME 17,

[71] C. Schmitt, and P. Métrailler. “The motion maker™: a rehabilitation system combining an orthosis with closed‐loop electrical muscle stimulation”. In: 8th Vienna international workshop on functional electrical stimulation, no. CONF, 2004, 117–120. [72] R. A. Søraa, and E. Fosch‐Villaronga. “Exoskele‐ tons for all: The interplay between exoskeletons, inclusion, gender, and intersectionality”, Paladyn, Journal of Behavioral Robotics, vol. 11, no. 1, 2020, 217–227. [73] P. Soto‐Acosta. “Covid‐19 pandemic: Shifting dig‐ ital transformation to a high‐speed gear”, Information Systems Management, vol. 37, no. 4, 2020, 260–266. [74] S. Straudi, G. Severini, M. Da Roit, L. D. M. Piz‐ zongolo, C. Martinuzzi, and N. Basaglia. “The dose of robot‐assisted gait therapy may in lu‐ ence functional recovery in a multidisciplinary rehabilitation program: an exploratory retro‐ spective study”, International Journal of Rehabilitation Research, vol. 43, no. 2, 2020, 175–182. [75] K. Swaminathan, and H. I. Krebs. “Analysis of the anklebot training as a method for reducing lower‐limb paretic impairment a case study in electromyography”. In: 2015 IEEE International Conference on Rehabilitation Robotics (ICORR), 2015, 555–558. [76] K. Takahashi, T. Takebayashi, S. Amano, Y. Uchiyama, K. Domen, and K. Hachisuka. “Constrained‐induced movement therapy transfer the function gained by upper arm robotic therapy into daily activities”. In: STROKE, vol. 51, 2020. [77] T. Takebayashi, K. Takahashi, S. Amano, Y. Uchiyama, M. Gosho, K. Domen, and K. Hachisuka. “Assessment of the ef icacy of reogo‐j robotic training against other rehabilita‐ tion therapies for upper‐limb hemiplegia after stroke: Protocol for a randomized controlled trial”, Frontiers in neurology, vol. 9, 2018, 730. [78] R. B. van Dijsseldonk, I. J. van Nes, A. C. Geurts, and N. L. Keijsers. “Exoskeleton home and com‐ munity use in people with complete spinal cord injury”, Scienti ic reports, vol. 10, no. 1, 2020, 1–8. [79] J. Wilk, and P. Falkowski. “A concept of detecting patient hazards during exoskeleton‐aided

Journal of Automation, Mobile Robotics and Intelligent Systems

remote home motor rehabilitation”, Prace Naukowe - Politechnika Warszawska. Elektronika z. 197, Postępy robotyki. T. 2, 2022. [80] H. Zheng, R. Davies, T. Stone, S. Wilson, J. Ham‐ merton, S. J. Mawson, P. Ware, N. D. Black,

VOLUME 17,

N∘ 2

2023

N. D. Harris, C. Eccleston, et al. “Smart rehabilita‐ tion: Implementation of ict platform to support home‐based stroke rehabilitation”. In: International Conference on Universal Access in HumanComputer Interaction, 2007, 831–840.

VOLUME 17, N∘ 2 2023 Journal of Automation, Mobile Robotics and Intelligent Systems

NEW MODEL OF PHOTOVOLTAIC SYSTEM ADAPTED BY A DIGITAL MPPT CONTROL AND RADIATION PREDICTIONS USING DEEP LEARNING IN MOROCCO AGRICULTURAL SECTOR Submitted: 12th May 2022; accepted: 26th April 2023

Amal Zouhri, Mostafa El Mallahi DOI: 10.14313/JAMRIS/2‐2023/17 Abstract: Solar energy is an essential factor in Moroccan sustain‐ able development, especially in solar pumping in the agricultural sector. It is therefore difficult to dissociate the energy system of a society from its economic develop‐ ment and social development. Solar radiation prediction is useful in giving us a global overview on maintaining the integrity of solar systems. Access to database use makes this process more flexible. Solar forecasts can be generated using various available data sources. There are two major pillars of this data: the exploitation of his‐ torical solar radiation data, and the exploitation of other meteorological factors. On the other hand, the choice of data can have an impact on the choice of the model and the approach employed. In this paper we suggest an idea that aims to monitor in real time the situation of solar radiation in Morocco, using Long Short‐Term Memory for deep learning models compared with Artificial Neural Networks and Deep Neural Networks to predict the solar radiation with regard to solar pumping in the Moroccan agricultural sector. Keywords: Morocco sustainable development, Solar pumping, Deep Learning, Renewable energies, Solar radiation.

1. Introduction Morocco has been making great strides in renewable energy, and green process in sustainable development, and especially in solar pumping for its agricultural sector [1]. Recently, in the literature several intelligent algorithms have been used in renewable energy, and for solar pumping prediction for sustainable development [2–5]; Indeed, some Arti icial Neural Networks models have been studied for predictions of solar radiation [6–9]. These have the advantage of giving a fast and accurate tracking of the MPP [10–16]. The controller in renewable energy effectiveness depends on the algorithm used and the neural network trained. Articles over the last years were studied renewable energy, and green process in sustainable development using Machine, and deep learning [16–18]. Some authors discuss in different papers the renewable energy using ANN for MPPT control is presented [19, 20]. 74

The algorithms are based on Deep Learning and Machine Learning Approaches or in Empirical and machine learning models for predicting daily global solar radiation from sunshine duration [21–23]. In general, countries are making great strides towards cleaner energy that is more environmentally friendly. The agricultural sector consumes a large share of energy in Morocco, contributes the most to gross domestic product, and is one of the sectors that employs the most people, especially in rural areas. In recent years, agriculture has made great strides in incorporating renewable energy into its businesses. In 2010, Morocco’s Solar Energy Agency (MASEN) was established, with the goal of providing the coun‐ try with a clean energy source that replaces 90% of imported energy [24]. Renewable energy is expected to exceed 52% of the country’s total energy consump‐ tion and production by 2030. The country invests in both solar and wind energy, especially the Noor Solar Station. The investment is estimated at 2 billion euros, all built over the last ive years with a total area of 300 hectares [25]. For Morocco, solar energy is an important eco‐ nomic issue in line with the choice of sustain‐ able development. The unchecked cost of traditional energy, which has a negative impact on the environ‐ ment, doubles the stake. Solar energy is clean, inex‐ haustible, and bene icial alternative energy consistent with sustainable development. Morocco’s determina‐ tion to be a lag bearer is an advantage in pursu‐ ing energy autonomy, not only for our brothers in Africa, but also for other partners in the world who want to provide services and products and share their experiences. As part of its strategy to promote renewable ener‐ gies, Morocco prioritizes the expansion and sustain‐ able development of these energies. With abundant solar resources, “potential of 2,600 kWh/m2 /year”, and a strategic location in the center of the energy hub (connected to the Spanish power network via two 400 kV/700 MW lines), Morocco offers several investment opportunities in the ield of solar energy in thermodynamics and photovoltaics. The Moroc‐ can solar energy project aims to build a total of 2000 MW of solar power capacity in 2020 at ive sites: Ouarzazate, AinBni Mathar, Foum Al Oued, Boujdour and Sebkhat Tah.

2023 © Zouhri and El Mallahi. This is an open access article licensed under the Creative Commons Attribution-Attribution 4.0 International (CC BY 4.0) (http://creativecommons.org/licenses/by-nc-nd/4.0/)

Journal of Automation, Mobile Robotics and Intelligent Systems

Solar radiation is the radiation or energy we receive from the sun. For energy systems, a thorough understanding of the availability and variability of solar radiation intensity is fundamental and crucial. This is the fuel of any solar energy system. Due to the growing demand for electricity, several countries have been targeting the renewable energy production mar‐ ket. Morocco is launching a range of projects that will hit about 2000 MW of electricity and plans to make 42% of its energy renewable by 2020 and bring it to 52% by 2030. The kingdom has a high sunshine rate: around 3000 hours of sunshine per year. All projects implemented or under construction offer the country the chance to be the leader in the MENA region in this ield. One of the most well‐known projects in Morocco is in Noor, Ouarzazate, which currently has four factories at different stages of development: Noor I is a 160 MW cylindrical‐parabolic mirror with 3 hours of ther‐ mal storage and an annual production of 520 GWh; Noor II is a 200 MW cylindrical‐parabolic mirror with 7 hours of thermal storage and an annual produc‐ tion of 699 GWh; Noor III is a 150 MW tower with 8 hours of thermal storage with an annual production of 515 GWh and Noor IV of 72 MW with an annual production of 125 GWh [25]. Noor has other proposed projects such as Noor Midelt, Noor Ta ilalet, Noor PV II, Noor Atlas , Outat El Haj, Ain Beni Mathar, Boud‐ nib, Bouanane and Boulemane) and Noor Argana. A study and analysis of previous solar radiation studies in Morocco are presented in this article. Section 2 presents the solar pumping in the agricultural sector. Next, Section 3 deals with methods for the acquisition of the necessary data for AI models and with arti icial intelligence and its axes. Section 4 deals with model‐ ing precise assessment methods. Section 5 provides presentations and algorithms used in solar radiation publications in Morocco. Finally, Section 6 shows the results and discussion.

2. Solar Pumping in the Agricultural Sector In Figures 1 and 2, the requested sample concerns 500 farmers, 277 of whom use photovoltaic power (PPV) for irrigation and 223 who use other conven‐ tional sources of energy. To constitute a representative sample is to ensure that the essential components

VOLUME 17,

N∘ 2

2023

Figure 2. Inventory of the solar pumping in the agricultural sector Table 1. Reference population distribution without PPV Without PPV Parent population Sample Con idence threshold Expected proportion of a response Margin of error

1,710,000 2,800 95% 0.5 2%

ha ha

Table 2. Reference population distribution with PPV With PPV Parent population Sample Con idence threshold Expected proportion of a response Margin of error

1,710,600 3,300 95% 0.5 1.7%

ha ha

of its reference population appear in the sample, in identical proportions. Under this condition, the results observed in the sample can be extrapolated to its entire reference pop‐ ulation (see Tables 1 and 2). The margin of error is: 𝑦=

𝑡𝑝2 × 𝑃 × (1 − 𝑃) × (𝑁 − 𝑛) (𝑁 − 1) × 𝑛

n: sampling size. N: size of the target population, actual or estimated. P: expected proportion of a response from the population or actual proportion. In the case of a multi‐criteria study (our case), it can be set at 0.5 by default. tp: Sampling con idence coef icient. The values are associated with con idence intervals. y: margin of sampling error

Figure 1. Spire of solar pumping

Three teams of investigators were trained before the surveys began. The training focused on the opera‐ tion of a solar installation and on how to address the different questions to the farmers. 75

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

Table 3. PPV in the different regions of Morocco Region 1. Tanger‐Tetouan‐Al Hoceïma 2. L’Oriental 3. Fez‐Meknes 4. Rabat‐Salé‐Kenitra 5. Béni Mellal‐Khénifra 6. Casablanca‐Settat 7. Marrakech‐Sa i 8. Darâa‐Ta ilalet 9. Souss‐Massa et le Sud Sum

With PPV 0 58 23 8 25 14 66 29 54 277

Without PPV 13 5 15 38 16 36 46 20 34 223

Sum

‐ Numerical Weather Prediction Method: This method used the Autoregressive Moving Aver‐ age (ARMA) and Autoregressive Integrated Mov‐ ing Average (ARIMA) techniques with numerical weather data to forecast the solar radiation.

13 63 38 46 41 50 112 49 88 500

‐ Satellite Image techniques: This technique is based on the re lection of correctly measured light from the cloud to satellites. ‐ Ground based image techniques: The strategy used the total sky imager (TSI) to present a clear view of cloud shadows for forecasting the solar radiation.

3. Artificial Intelligence Models In our research, we would concentrate on Arti i‐ cial Neural Network forecasting, generally in the tech‐ niques of Arti icial Intelligence, but what are those methods? And what are their effects on predicting solar radiation? Arti icial Intelligence (AI) is a branch of com‐ puter science that deals with the development of smart machines capable of executing tasks that usu‐ ally require human intelligence. With strong predic‐ tion and automation capabilities, AI can operate with excellence in several areas. Machine Learning (ML) goes deeper, as AI tries to emulate human thought. Machine Learning as a branch of AI enables machines to learn without relying on instruction. In fact, in order to recognize patterns, the “machine” is an algorithm that analyzes a volume of data which would be unmanageable for a human being. Machine learning, in other words, allows the machine to be educated to automate tasks that are dif‐ icult for a human being, and it can make predictions with this learning. Deep Learning (DL) may be viewed as a form but more complex of Machine Learning. Deep Learning is a series of algorithms that simulate the human brain’s neural networks. The computer learns by itself, but in

‐ The dif iculty of estimating energy and water energy and water consumption among farmers; ‐ Lack of reliability of farmers’ statements about lower costs and/or the increase in their income fol‐ lowing the installation of PPV systems (Table 3); The number of farmers visited in each region was chosen according to two criteria (Figure 3): ‐ The irrigated area of the region (Data sources: MADRPM 2015/2016) ‐ The presence of the PPV in the region (Data source: ield experience) Due to the rapid rise in implementation and high penetration of solar power in electricity grids world‐ wide, forecasting of solar radiation production has become a crucial need [1]. We show here the different types of solar radiation forecasting methods available: ‐ Stochastic Learning techniques: In order to fore‐ cast shifts in sun angles, these methods are based on recent data from photovoltaic power plants or radiometer outputs.

Distribution of the sample offer (distributors/installers) by type of sector (Formal/Informal)

Number 51 27 10 9 3 100

Financial ins tu ons: 9%

Manufacturer: 3%

Distributors: 10%

Installer: 51%

Installer/Distri butors: 27%

Figure 3. The number of farmers with and without financial institutions 76

2023

‐ Arti icial Neural Network: ANN deals with meteo‐ rological variables taken as inputs to forecast vari‐ ous time scales of solar radiation.

The three teams visited the 500 farmers in a random manner, respecting randomness while also respecting the distribution of the sample by region and by category (with and without PPV). Some dif i‐ culties were encountered in the ield: ‐ The dif iculty of convincing farmers with PPV to participate in this study;

Entity Installer Installer/Distributors Distributors Financial institutions Manufacturer Total

N∘ 2

Journal of Automation, Mobile Robotics and Intelligent Systems

phases or layers, in this technology. The model’s depth would depend on the number of layers in the model.

4. DataSET There are numerous approaches for source data methods that can be used to forecast solar radiation, some of which are: ‐ The Prediction of Worldwide Energy Resources (POWER) is a NASA project with the goal of observ‐ ing, understanding, and modeling the Earth system to discover how it is changing, to better predict change, and to understand the consequences for life on Earth. The project was initiated to improve upon the current renewable energy data set and to create new data sets from new satellite systems. The POWER project targets three user communities: renewable energy, sustainable buildings, and agro‐ climatology. They provide two different datasets: meteorology (starting from January 1, 1981, to now) and the solar radiation data (from July 1, 1983, to now). ‐ The Copernicus Atmosphere Monitoring Service (CAMS) provides consistent and quality‐controlled information related to air pollution and health, solar energy, greenhouse gases, and climate forcing, everywhere in the world. CAMS offer information services based on satellite Earth observation, in situ (non‐satellite) data, and modeling. ‐ METEONORM software is a unique combination of reliable data sources and sophisticated calcula‐ tion tools. It provides access to typical years and historical time series when we can choose from more than 30 different weather parameters. The database consists of more than 8 000 weather sta‐ tions, ive geostationary satellites, and a globally calibrated aerosol climatology. On this basis, sophis‐ ticated interpolation models, based on more than 30 years of experience, provide results with high accuracy worldwide. ‐ Local laboratory: Many universities around the world have already established local laboratories where various sensors (pyrometers, anemometers, pluviometers, radiometers, and thermo‐hygrometers) are mounted to capture different meteorological and solar parameters.

5. Model Accuracy Evaluation Typically, most research papers using methods of Arti icial Intelligence use an evaluation algorithm to measure how effective the model is. In this section we will present some of the most used evaluations of model accuracy: ‐ Root Mean Square Error (RMSE) is the standard deviation of the residuals (prediction errors), it is widely used to validate experimental indings in climatology, forecasting, and regression analysis. It detects how oriented the data is around the best it line. Where Xobs observes values, Xmodel models values at time/place i.

VOLUME 17,

N∘ 2

2023

‐ R-Squared (R2, or the decision coef icient) is a statistical measure which speci ies the proportion of variance in the dependent variable that can be explained by the independent variable. R‐squared, in other words, shows how well the data matches the regression model (the goodness of it). 2

𝑅 =1−

𝑛

∑𝑖=1 (𝑦𝑖 − 𝑦̂ 𝑖 ) ∑𝑖=1 (𝑦𝑖 − 𝑦𝑖 )

where n is the number of measurements, 𝑦𝑖 the value of the measurement, 𝑦̂ 𝑖 the corresponding predicted value and 𝑦 the mean of the measurements. ‐ Mean Absolute Error (MAE) measures the average magnitude of the errors in a set of forecasts, without considering their direction. It measures accuracy for continuous variables. 𝒏

MAE =

∑𝒊=1 |𝒚𝒊 − 𝒙𝒊 | 𝒏

Where 𝑦𝑖 is the prediction and 𝑥𝑖 is the true value. ‐ The mean square error (MSE) uses the squared difference between measured and forecast values. 𝒏

MSE =

∑𝒊=1 (𝒀𝒊 − 𝒀𝒊 )2 𝒏

Where 𝑌𝑖 is the vector of observed values and 𝑌𝑖 is the predicted values. The authors used various instruments and sensors to gather meteorological parameters (Humidity, Tem‐ perature, and Wind Speed). As a result, the authors found that the NARX model with ive inputs provides the best ef iciency after 10 tested models (Table 4). In the Benguerir region, some authors [25–29] used multilayer perception (MLP) to forecast global horizontal irradiance for a hot semi‐arid atmosphere. For this reason, the authors used data obtained from the weather station located in Green Energy Park, Benguerir, Morocco, formed by different meteorological data (temperature, relative humidity, barometric pressure, wind speed, wind direction, precipitation, and others). Using the correlation coef icient, they ind that the global irradiance at the top of the atmosphere and the solar zenithal angle are the most correlated astronomy parameters, and the temperature is the most correlated meteorological parameter with solar irradiance. Author researchers developed an arti icial neural network (ANN) model with a multilayer perceptron (MLP) technique to measure the monthly average global solar irradiation on the horizontal surfaces of the Souss‐Massa region in Morocco. To train the model, the authors used data from 175 locations spread across the Souss‐Massa region for 10 years (1996–2005) provided by the NASA geo‐satellite database and Google Maps. The authors have chosen a set of 24 different climate locations to achieve a stable design of the ANN model and validate it for the remaining 151 sites. As a result, the optimal model has 25 nodes in the hidden layer with an RMSE of 0.234 and a correlation coef icient of R 0.988. 77

Journal of Automation, Mobile Robotics and Intelligent Systems

Table 4. List of reference papers on solar radiation in some laboratories Reference

Models

Arti icial neural network optimization for monthly average daily global solar radiation prediction Recurrent neural network model

23 24

Proposed works

Solar radiation estimation methods using ANN and empirical models solar radiation with using ensemble learning algorithm LSTM LSTM

LSTM

Year

Performance Indicator 2016 Mean Absolute Percentage Error (MAPE), range between 1.67% and 4.25%

2017 Training ratio 90; Validation ratio 5; Testing ratio 5, MSE (3.29 10‐3) R 0.983 MSETEST (5.27 10‐3 )RTEST0.961 2019 RMSE values of many other similar models range from 2.05 to 4.70 MJ m‐2 day‐1 2019 RMSE between 4.6 and 14.6% in average

2020 R2 : 91,6% 2020 MSE/Autumn: 0.0019 MSE/Winter: 0.00301 MSE/Spring: 0.00322 MSE/Summer: 0.0015 2022 MSE/Autumn: 0.0005 MSE/Winter: 0.00401 MSE/Spring: 0.00110 MSE/Summer: 0.0040

The authors tested the models for clear and unclear days, the results are very acceptable for clear days with an NMBE of 0.015%, an NRMSE of 0.10% and a correlation coef icient of 0.99, for unclear days the accuracy was an NMBE of 0.14%, an NRMSE of 0.39% and a correlation coef icient of 0.96. 5.1. Random Forest (RF) Random Forest is a machine learning method and a tweaked algorithm based on a decision tree, includ‐ ing a variety of decision tree ittings for various sub‐ samples of the initial data set at the training level to produce decision trees for computation and to arrange trees for the inal outcome. 78

VOLUME 17,

N∘ 2

2023

Bounoua et al. [30] tested 22 empirical and 4 machine learning models to measure Global Solar Radiation at ive locations in Morocco (Oujda, Mis‐ sour, Erfoud, Zagora, and Tan‐Tan). The models tested were the Multilayer Perceptron Arti icial Neural Net‐ work Model (MLP) and three‐set methods (Boost‐ ing, Bagging, and Random Forest) with some mea‐ sured meteorological variables and some astronomical parameters (ambient air temperature, relative humid‐ ity, wind speed, etc.) were used to train these models. The indings achieved suggest that the temperature and geographic factors model were the more accurate with R: 72.38–93.46%; nMAE: 6.96–17.94%; nRMSE: 9.89–22.39%. The Random Forest (RF) method has also proven to be the highest performing in all stations between the four machine‐learning methods. 5.2. Back Propagation Neural Network (BPNN) Back‐propagation is an integral aspect of Arti icial Neural Network. It tries to adjust the network weights using the error rate of the past epoch. Proper tuning of the weights helps to reduce the error rate and makes the model accurate by increasing its generalization. Aghmadi et al. [27] used BPNN with the Empirical Mode Decomposition (EMD) to improve the accuracy of solar radiation estimation and simplify the energy management system. A one‐hour data for the year 2018 of Measured Direct Normal Irradiance (DNI) col‐ lected from a meteorological ground station located in Rabat, Morocco, is used. Three concept reliabil‐ ity and performance quality criteria are used: MAE, MAPE, and RMSE. The tests of the EMD‐BPNN hybrid approach revealed an RMSE of 28.13% and a MAE of 20.99%, much less than other traditional approaches such as the conventional neural network or the ARIMA time series. 5.3. Deep Neural Network (DNN) A Deep Neural Network is a neural network with a certain degree of sophistication, a technol‐ ogy designed to simulate the behavior of the human brain – speci ically, the identi ication of patterns and the passing of inputs through different layers of simu‐ lated neural connections to predict performance using advanced mathematical modeling to process data in complex ways. Jallal et al. [33] suggested a Deep Neural Network capable of handling the non‐linearity and dynamic behavior of meteorological data and providing accu‐ rate real‐time predictions of hourly global solar radi‐ ation. The neural network used hourly data on global solar radiation and meteorological parameters based on the METEONORM data sets of the city of El Kelaa des Sraghna, Morocco. The writers used the Elman neural network (ENN) with the Levenberg‐Marquardt Optimizer. With a 99.38% correlation coef icient (R), the Deep Neural Network when implemented proves to be very effective and accurate. 5.4. Long Short‐Term Memory (LSTM) Soufene et al. [34] suggested LSTM enables the simulation of very long‐term dependencies. It is based

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

on a memory cell and three gates (Forgotten Gate – Input Gate – Output Gate). The complete activity of the LSTM can be outlined in three steps: ‐ Detect knowledge from the past, drawn from the memory cell via the forgotten gate; ‐ Choose, from the current entrance, the ones that will be useful in the long term, via the input gate. These would be applied to the memory cell; ‐ Collect valuable short‐term information from the current cell state to produce the next hidden state via the output gate [34]. Benamrou et al. [23] suggest a very short‐term prediction of horizontal global solar irradiation using the LSTM model, using two separate data sources. The irst was obtained from the Al‐Hoceima, Morocco, Meteorological Station for the duration (2015–2017) and the second was a set of satellite‐ derived data retrieved from the CAMS dataset around the Al‐Hoceima Meteorological Station for the same period. The authors used the RFE (Recursive Function Elimination) approach to ind the desired features of the model. Three scenarios were suggested to model solar irradiation using various algorithms (XGBoost, Random Forest, and SVR) to train the features. As a result, XGBoost provided the best output model with an R2 coef icient of 0.916. Bendali et al. [26] propose a hybrid approach to re ine the forecasting of solar irradiance using a Deep Neural Network with genetic algorithm. For this pur‐ pose, the authors evaluated Long Short‐Term Memory (LSTM), Gate Recurrent Unit (GRU), and Recurrent Neural Network (RNN) models. The genetic algorithm was used to ind the most suitable number of window sizes and the number of neurons in each layer. For this work, the Global Horizontal Irradiance (GHI) time series of Fes, Morocco, was used from 2016 to 2019 as a data set derived from the METEONORM Platform. The combination of the genetic algorithm and the LSTM showed the best results for the four seasons of the year evaluated with the MSE and MAE.

Figure 4. Photovoltaic panel

Figure 5. Model of a photovoltaic cell

Figure 6. Photovoltaic generator block diagram

6. Modeling a Photovoltaic Generator 6.1. Modeling a Cell The model of photovoltaic cell equivalent as I = Iph − Id − Ish

(1)

The physics of the PV cell is very similar to the clas‐ sical p‐n junction diode. When the junction absorbs light, the energy of the absorbed photons is trans‐ ferred to the electron system of the material, resulting in the creation of charge carriers that are separated at the junction. The charge carriers may be electron‐ion pairs in a liquid electrolyte, or electron hole pairs in a solid semiconducting material (Figure 4). The charge carriers in the junction region create a potential gra‐ dient, get accelerated under the electric ield, and cir‐ culate as the current through an external circuit. The current squared times the resistance of the circuit is the power converted into electricity.

The remaining power of the photon elevates the temperature of the cell. A number of modules make up a typical photo‐ voltaic panel that can be connected in a string con igu‐ ration in order to achieve desired current and voltage at the inverter input. A number of photovoltaic panels connected in a string con iguration is typically known as a photovoltaic array. Current versus voltage (I‐V) characteristics of the PV module can be de ined in sunlight and under dark conditions. In the irst quadrant, the top left of the I‐V curve at zero voltage is called the short circuit current. This is the current measured with the output terminals shorted (zero voltage). The bottom right of the curve at zero current is called the open‐circuit voltage. This is the voltage measured with the output terminals open (zero current). 79

Journal of Automation, Mobile Robotics and Intelligent Systems

Figure 5 represents the model of a photovoltaic cell, and the block diagram (Figure 6) comprising four parameters can present the equivalent electrical dia‐ gram of the photovoltaic generator (GPV). Two input variables, which are the insolation in the plane of the panels E, the junction temperature of the cells Tj, and two output variables: current supplied by the GPV Is, voltage at the terminals of the GPV versus different illuminations and temperatures, we use the following model:

VOLUME 17,

N∘ 2

2023

Figure 7. PV generator scheme in MATLAB‐SIMULINK

𝐼𝑐𝐶 (𝑇) = 𝐼𝑐𝑐 (𝑇𝑟𝑒𝑓 ) ⋅ [1 + 𝛼(𝑇 − 𝑇𝑟𝑒𝑓 )] 𝐺 1000

(Amps)

𝐼𝑝ℎ = 𝐼𝑐𝑐

𝐼𝑠𝑎𝑡 (𝑇) = 𝐼𝑠𝑎𝑡 (𝑇𝑟𝑒𝑓 ) 𝑇𝑟𝑒𝑓 ⋅ 𝑇

3 𝑛

exp

𝑞.𝐸𝜃 1 1 ⋅ − 𝑛𝑘 𝑇 𝑇𝑟𝑒𝑓

(Volts)

(2)

Figure 8. Characteristic I‐V of the PV cell

With: n is the quality factor of the diode, normally between 1 and 2, k is the constant of Boltzmann k = 1, 𝛼 is the coef icient of variation of the current. Standard illumination, G Standard temperature, T Maximum power Pmax Voltage at Pmax or optimal voltage (Vopt) Current at Pmax or Optimal current (Iopt) Short‐circuit current Isc Open circuit voltage Vco Number of cells in series Forbidden band energy Temperature coef icient Isc Temperature coef icient Vco Power temperature coef icient Saturation current Isat

1000 W/m2 25∘ c 60W 17.1 V 5.5 A 3.8 A 21.1 V 36 1.12 ev 65 mA/∘ c −80 mV/∘ c (0.5+−0.05)%/∘ C 20 nA

6.2. Modeling a Module An elementary cell does not generate enough volt‐ age: between 0.5 and 1.5, according to technology. It usually takes several cells in series to generate a usable voltage. The module voltage is therefore: Vm = Ns ∗ V Vm : the voltage of the module. Ns : number of cells in series per module 6.3. Model of a Photovoltaic Chain For modules mounted in series and in parallel one can write: Ichaine = I ∗ Np Vchaine = Vm ∗ Ns−module With: Ichaine : the current delivered by a module chain Photovoltaic (A). Np : number of modules in parallel. Ns_module : number of modules in series. Vchaine : the voltage at the terminal of the chain (V). 80

6.4. Model of PV Solar The photovoltaic generator scheme in the Matlab‐ Simulink environment represented by: The simulation results of the photovoltaic genera‐ tor are represented by Figures 7 through 14. These ig‐ ures represent the current‐voltage and power‐voltage characteristics for different illuminations. Figures 7 and 8 show the in luence of illumination on current‐voltage and power‐voltage characteristics. At a constant temperature, it is found that the current undergoes a signi icant variation, but against the volt‐ age varies slightly. Because the short circuit current is a linear function of illumination while the open circuit voltage is a logarithmic function. 6.5. Current‐Voltage Characteristic Figure 8 shows the in luence of illumination on the characteristic I = f(V). At a constant temperature, it is found that the current undergoes a signi icant vari‐ ation, but against the voltage varies slightly. Because the short‐circuit current is a linear function of illumi‐ nation while the open circuit voltage is a logarithmic function. 6.6. Power‐Voltage Characteristic Figure 9 shows the curve I = f(V) of a typical pho‐ tovoltaic module under constant conditions of irra‐ diation and temperature. The standard irradiation adopted for measuring the response of photovoltaic modules is a radiant intensity of 1000 W/m2 and a temperature of 250∘ C. It is dif icult to give a source of current or voltage to a photovoltaic module over the full extent of the current‐voltage characteristic. Therefore, the photo‐ voltaic module is considered as a source of power with a point Pm. It is important to note that some solar regulators realize an adaptation of impedance so that at every moment one is close to this point P where the

VOLUME 17,

N∘ 2

2023

(Watts)

Journal of Automation, Mobile Robotics and Intelligent Systems

(Volts)

Figure 11. Characteristic power‐voltage for different values of the temperature 𝑃 = 𝑓(𝑉); 𝐸 = 1000 W/m2

(Watts)

Figure 9. P‐V characteristic of the PV cell

(Amps)

Figure 10. Current‐voltage characteristic for different temperature value 𝐼 = 𝑓(𝑉); 𝐸 = 1000 W/m2 power is found to be maximal. It is therefore interest‐ ing to place oneself on this point to get the most energy and thus make the most of the peak power installed. 6.7. Influence of Temperature The in luence of temperature on the characteristic I = f(V). It is essential to understand the effect of changing the temperature of a solar cell on the charac‐ teristic 𝐼 = 𝑓(𝑉) in Figure 9. The current depends on the temperature since the current increases slightly as the temperature increases, but the temperature has a negative in luence on the open circuit voltage. When the temperature increases the open circuit voltage decreases. Therefore, the maximum power of the gen‐ erator is decreased. Figures 10 and 11 illustrate the variation of the power delivered by the generator as a function of the voltage for different values of the temperature, which allows us to deduce the in luence of the temperature on the characteristic P = fct(V). 6.8. Influence of Solar Radiation Figure 12 illustrates the variation of the power delivered by the generator as a function of the voltage for different values of the temperature, which allows us to deduce the in luence of the temperature on the characteristic P = fct(V). Figures 13 and 14 represent the characteristic I‐ V of a module re lecting the in luence of different

(Amps)

Figure 12. Characteristic power‐current for different values of the temperature 𝑃 = 𝑓(𝑉); 𝐸 = 1000 W/m2

(Volts)

Figure 13. Current‐voltage characteristic for different radiation values 𝐼 = 𝑓(𝑉); 𝑇 = 25 CIn

radiation at a ixed temperature: the current of the module is proportional to the radiation, while the open circuit voltage changes slightly with the radia‐ tion. The optimum power is also proportional to the radiation. In Figure 14, we represent the variation of the power delivered by the generator as a function of the voltage for different illumination values, which allows us to deduce the in luence of the illumination on the characteristic P. This paper provided an analysis of forecasting the solar radiation using arti icial intelligence techniques in Morocco. As seen in Table 4, a number of machine learning and deep learning methods have been used. The most widely used methods are machine learning algorithms and, in particular, ANNs. The methods used vari‐ ous model precision performance for different data sources; we can detect the best performance with a decision coef icient R2 of 99.12%, using data obtained from a local laboratory in the Marrakech region. 81

Journal of Automation, Mobile Robotics and Intelligent Systems

VOLUME 17,

N∘ 2

2023

AUTHORS Amal Zouhri∗ – Sidi Mohamed Ben Abdellah Univer‐ sity, Faculty of Sciences, Department of Physics, Fez, Morocco, e‐mail: amal.zouhri@usmba.ac.ma. Mostafa El Mallahi – Sidi Mohamed Ben Abdellah University, High Normal School, Fez, Morocco, e‐mail: elmallahi@usmba.ac.ma. ∗

Figure 14. Power‐voltage characteristic for different radiation values 𝑃 = 𝑓(𝑉); 𝑇 = 25 C

We also note the rise in the use of deep learning approaches in recent years, as can be seen in Table 4, in particular Deep Neural Networks and Long Short‐ Term Memory. As well as the ANN case, the best performance had a decision coef icient R2 of 99.38% using the METEONORM datasets for the Elkelaa des Sraghna region. From our study, we have seen different choices of geographical, meteorological, and solar input param‐ eters. This choice is the most critical consideration for the reliable and accurate estimation of solar radiation. Unless there are few studies working on this prob‐ lem, we may take, for example, the cross‐correlation function CCF as used by Ettaybi et al. [35], to calcu‐ late the correlation between the clear‐sky index and each meteorological parameter to determine which one will be used to train the model. In the other hand, Benamrou et al. [23], use the Recursive Fea‐ ture Elimination (RFE) approach with XGBoost algo‐ rithm to ind the best features to be used for model learning.

7. Conclusion Renewable energy has been highlighted as a cru‐ cial strategic source for green development in the world. Morocco has an immense solar energy capacity; the Kingdom is implementing a number of policies and initiatives to meet the optimistic goal of 2030 by achieving 52% of overall electricity generation using solar. In conclusion, efforts towards greener energy will always be ongoing as technology doesn’t stop advancing and evolving. Since the agricultural sector is one of the most important contributors to our national GDP, it is necessary to ind new ways to decrease costs and increase ef iciency, all the while making sure to maintain eco‐friendly processes and make well‐ informed decisions. In order to encourage potential studies in this area, our paper provides an updated summary of predict‐ ing solar radiation papers in Morocco. Indeed, due to advances in the AI methods, the ef iciency and avail‐ ability of daily data, and the development of actual solar energy projects, the example of Noor projects as we’ve seen in the introduction involve more and more studies and applications for solar radiation and for energy systems in general. 82

Corresponding author

References [1] M. Azerouia, A. El Makrini, H. El Moussaoui, and H. El Markhi. “Renewable Energy Potential and Available Capacity for Wind and Solar Power in Morocco Towards 2030,” Journal of Engineering Science and Technology Review, 2018. [2] Y. A. Al‐Sbou, and K. M. Alawasa. “Nonlinear autoregressive recurrent neural network model for solar radiation prediction,” Int. J. Appl. Eng. Res., vol. 12, 2017, 4518–4527. [3] E. F. Alsina, M. Bortolini, M. Gamberi, and A. Regattieri. “Arti icial neural network optimisa‐ tion for monthly average daily global solar radi‐ ation prediction,” Energy Convers. Manag. 120, 2016, 320–329. [4] A. Angstrom. “Solar and terrestrial radiation. Report to the international commission for solar research on actinometric investigations of solar and atmospheric radiation,” Q. J. R. Meteorol. Soc., 50, 1924, 121–126. [5] J. Annandale, N. Jovanovic, N. Benade, and R. Allen. “Software for missing data error analy‐ sis of Penman‐Monteith reference evapotranspi‐ ration,” Irrig. Sci., 21, 2002, 57–67. [6] V. Z. Antonopoulos, D. M. Papamichail, V. G. Aschonitis, and A. V. Antonopoulos. “Solar radia‐ tion estimation methods using ANN and empir‐ ical models,” Comput. Electron. Agric., vol. 160, 2019, 160–167. [7] K. Basaran, A. Özçift, and D. Kılınç. “A new approach for prediction of solar radiation with using ensemble learning algorithm,” Arab. J. Sci. Eng., vol. 44, 2019, 7159–7171. [8] M. Behrang, E. Assareh, A. Ghanbarzadeh, A. Noghrehabadi. “The potential of different arti icial neural network (ANN) techniques in daily global solar radiation modeling based on meteorological data,” Sol. Energ., vol. 84, 2010, 1468–1480. [9] M. Benghanem, A. Mellit, S. Alamri. “ANN‐based modelling and estimation of daily global solar radiation data: a case study,” Energy Convers. Manag., vol. 50, 2009, 1644–1655. [10] F. Besharat, A. A. Dehghan, A. R. Faghih. “Empiri‐ cal models for estimating global solar radiation: a review and case study,” Renew. Sust. Energ. Rev., vol. 21, 2013, 798–821.

Journal of Automation, Mobile Robotics and Intelligent Systems

[11] Z. Boussaada, O. Curea, A. Remaci, H. Camblong, and N. Mrabet Bellaaj. “A nonlinear autoregres‐ sive exogenous (NARX) neural network model for the prediction of the daily direct solar radi‐ ation,” Energies, vol. 11, 2018, 620. [12] L. Breiman. “Random forests,” Mach. Learn., vol. 45, 2001, 5–32. [13] L. Breiman. “Bagging predictors,” Mach. Learn., vol. 24, 1996, 123–140. [14] M. V. C. Calça, M. R. Raniero, D. M. Z. Fernando, S. A. Rodrigues, and A. Dal Pai. “Outliers detection in a quality control procedure for measurements of solar radiation,” IEEE Lat. Am. Trans., vol. 17, 2019, 1815–1822. [15] J. Chen, G. Li. “Estimation of monthly average daily solar radiation from measured meteorolog‐ ical data in Yangtze River Basin in China,” Int. J. Climatol., 33, 2013, 487–498. [16] J. L. Chen, L. He, H. Yang, M. Ma, Q. Chen, S. J. Wu, and Z. Xiao. “Empirical models for estimating monthly global solar radiation: a most compre‐ hensive review and comparative case study in China,” Renew. Sust. Energ. Rev., vol. 108, 2019, 91–111. [17] R. Chen, E. Kang, S. Lu, J. Yang, X. Ji, Z. Zhang, and J. Zhang. “New methods to estimate global radiation based on meteorological data in China,” Energ. Convers. Manag., vol. 47, 2006, 2991– 2998. [18] M. Despotovic, V. Nedic, D. Despotovic, S. Cve‐ tanovic. “Evaluation of empirical models for pre‐ dicting monthly mean horizontal diffuse solar radiation,” Renew. Sust. Energ. Rev., vol. 56, 2016, 246–260. [19] D. Elizondo, G. Hoogenboom, and R. McClendon. “Development of a neural network model to pre‐ dict daily solar radiation,” Agric. For. Meteorol., vol. 71, 1994, 115–132. [20] P. N. Jyothy, and M. R. Sindhu. “An Arti icial Neu‐ ral Network based MPPT Algorithm For Solar PV System,” 2018 4th International Conference on Electrical Energy Systems (ICEES), Chennai, India, 2018, pp. 375–380, doi: 10.1109/ICEES.2018. 8443277. [21] J. Fan, L. Wu, F. Zhang, H. Cai, W. Zeng, X. Wang, and H. Zou. “Empirical and machine learning models for predicting daily global solar radiation from sunshine duration: a review and case study in China,” Renew. Sust. Energ. Rev., vol. 100, 2019, 186–212. [22] Y. Feng, N. Cui, Q. Zhang, L. Zhao, and D. Gong. “Comparison of arti icial intelligence and empir‐ ical models for estimation of daily diffuse solar radiation in North China Plain,” Int. J. Hydrog. Energ., vol. 42, 2017, 14418–14428. [23] B. Benamrou, M. Ouardouz, I. Allaouzi, and M. B. Ahmed. “A Proposed Model to Forecast

VOLUME 17,

N∘ 2

2023

Hourly Global Solar Irradiation Based on Satel‐ lite Derived Data, Deep Learning and Machine Learning Approaches” Journal of Ecological Engineering, vol. 21, no. 4, 2020, 26–38. [24] W. Bendali, Y. Mourad, S. Ikram, and B. Moha‐ mmed. “Deep Learning Using Genetic Algorithm Optimization for Short Term Solar Irradiance Forecasting,” Conference: International Confer‐ ence on Intelligent Computing in Data Sciences (ICDS) At: Fes, December 2020. [25] Y. Stitou. “Case Study: Masen NOOR Ouarzazate Solar Complex,” The Center for Mediterranean Integration, 2017. [26] W. Bendali, Y. Mourad, S. Ikram, and B. Moha‐ mmed. “Deep Learning Using Genetic Algorithm Optimization for Short Term Solar Irradiance Forecasting,” Conference: International Conference on Intelligent Computing in Data Sciences (ICDS) At: Fes December 2020. [27] A. Aghmadi, S. El Hani, H. Mediouni, N. Naseri, and F. El Issaoui. “Hybrid Solar Forecasting Method Based on Empirical Mode Decompo‐ sition and Back Propagation Neural Network,” E3S Web of Conferences, 2020 2nd International Conference on Power, Energy and Electrical Engineering (PEEE 2020), vol. 231, no. 02001, 2021. [28] Y. Stitou. “Case Study: Masen NOOR Ouarzazate Solar Complex,” The Center for Mediterranean Integration, 2017. [29] M. Azeroual, A. El Makrini, H. El Moussaoui, and H. El Markhi. “Renewable Energy Potential and Available Capacity for Wind and Solar Power in Morocco Towards 2030,” Journal of Engineering Science and Technology Review, vol. 11, no. 1, 2018, 189–198. [30] Z. Bounoua, L. O. Chahidi, and A. Mechaqrane. “Estimation of daily global solar radiation using empirical and machine‐learning methods: A case study of ive Moroccan locations,” Sustainable Materials and Technologies, vol. 28, 2021, e00261. [31] O. El Alani, H. Ghennioui, and A. Ghennioui. “Short term solar irradiance forecasting using arti icial neural network for a semi‐arid climate in Morocco,” 2019 International Conference on Wireless Networks and Mobile Communications (WINCOM), 2019. [32] O. Nait Mensour, S. Bouaddi, B. Abnay, B. Hlimi, and A. Ihlal. “Mapping and estimation of monthly global solar irradiation in different zones in Souss‐Massa area, Morocco, using arti icial neu‐ ral networks,” International Journal of Photoenergy, vol. 2017, no. 8547437, 2017. [33] M. A. Jallal, A. El Yassini, S. Chabaa, A. Zeroual, and S. Ibnyaich. “AI data driven approach‐ based endogenous inputs for global solar radiation forecasting,” Ingénierie des systems 83

Journal of Automation, Mobile Robotics and Intelligent Systems

d’Information, vol. 25, no. 1, 2020, 27–34. doi: 10.18280/isi.250104. [34] S. Alouini, and S. Calcagno. “Les réseaux de neu‐ rones récurrents: des RNN simples aux LSTM,” OCTO BLOG, 2019.

VOLUME 17,

N∘ 2

2023

[35] H. Ettaybi, and K. El Himdi. “Arti icial neural networks for forecasting the 24 hours ahead of global solar irradiance,” AIP Conference Proceedings, vol. 2056, no. 1, 2018, 020010. doi: 10.1063/1.5084983.