Cognitive Coordination for Cooperative Multi-Robot Teamwork

(1)

Cognitive Coordination for Cooperative

Multi-Robot Teamwork

(2)

(3)

Cognitive Coordination for Cooperative

Multi-Robot Teamwork

Proefschrift

ter verkrijging van de graad van doctor aan de Technische Universiteit Delft,

op gezag van de Rector Magnificus prof. ir. K.C.A.M. Luyben, voorzitter van het College voor Promoties,

in het openbaar te verdedigen op dinsdag 12 mei 2015 om 15:00 uur

door

Changyun WEI

Master of Science in Mechanical Engineering, Hohai University, China, geboren te Wuwei, Gansu province, China.

(4)

Dit proefschrift is goedgekeurd door de promotor: Prof. dr. C. M. Jonker

Copromotor: Dr. K. V. Hindriks Samenstelling promotiecommissie:

Rector Magnificus, voorzitter

Prof. dr. C. M. Jonker, Technische Universiteit Delft, promotor

Dr. K. V. Hindriks, Technische Universiteit Delft, copromotor

Prof. dr. F. Brazier, Technische Universiteit Delft Prof. dr. J-J. Ch. Meyer Universiteit Utrecht

Prof. dr. M. A. Neerincx, Technische Universiteit Delft

Dr. M. V. Dignum, Technische Universiteit Delft

SIKS Disseveration Series No. 2015-16.

The research reported in this thesis has been carried out under the auspices of SIKS, the Dutch Research School for Information and Knowledge Systems.

This work is supported by the China Scholarship Council. Published and distributed by: Changyun Wei.

E-mail: weichangyun@hotmail.com

ISBN 978-94-6186-455-0

All rights reserved. No part of the material protected by this copyright notice may be reproduced or utilized in any form or by any means, electronic or mechanical, in-ducing photocopying, recording or by any information storage and retrieval system, without written permission of the author.

(5)

(6)

(7)

C

ONTENTS

1 Introduction 1 1.1 Multi-Robot Teamwork. . . 2 1.1.1 Cognitive Robots. . . 2 1.1.2 Coordinated Teamwork. . . 3 1.2 Research Questions. . . 5 1.3 Dissertation Outline . . . 7 1.4 List of Publications . . . 11 References . . . 12

2 Cognitive Robot Architecture 17 2.1 Introduction. . . 19

2.2 Related Work . . . 20

2.3 Cognitive Robot Control Architecture . . . 21

2.3.1 Overall Design of the Architecture . . . 22

2.3.2 System Architecture and Components . . . 23

2.3.3 Decoupled Architecture Layers. . . 26

2.3.4 Knowledge Acquisition and Representation. . . 27

2.3.5 Information and Control Flow. . . 31

2.3.6 Synchronization of Perceptions and Actions . . . 32

2.4 Navigation Task as An Example. . . 34

2.5 Conclusion. . . 37 References . . . 37 3 Impact of Communication 41 3.1 Introduction. . . 43 3.2 Related Work . . . 44 3.3 Multi-Robot Teamwork. . . 45

3.3.1 Search and Retrieval Tasks. . . 45

3.3.2 The Blocks World for Teams. . . 46

3.4 Coordination Protocols. . . 47

3.4.1 Baseline: No Communication. . . 48

3.4.2 Communicating Robot Teams. . . 50

3.5 Experimental Design. . . 54

3.5.1 Data Collection. . . 54

3.5.2 Experimental Setup . . . 55 vii

(8)

viii Contents

3.6 Results . . . 56

3.6.1 Baseline Performance. . . 56

3.6.2 Speedup with Multiple Robots. . . 59

3.6.3 Communication Performance. . . 59

3.7 Conclusion. . . 62

References. . . 62

4 Multi-Robot Cooperative Pathfinding 65 4.1 Introduction. . . 67

4.3 Models of Decentralized Cooperative Pathfinding . . . 69

4.3.1 Problem Formulation. . . 69

4.3.2 Heuristic Estimated Shortest Paths. . . 70

4.3.3 Coordinated Network. . . 71

4.4 Decentralized Multi-Robot Altruistic Coordination. . . 72

4.4.1 Decision Making for Planning Next-Step . . . 72

4.4.2 Altruistic Coordination in Deadlock Situations . . . 73

4.4.3 Transformation of Deadlock Situations. . . 76

4.4.4 Algorithm of Altruistic Coordination. . . 79

4.4.5 Cooperative Teams. . . 80

4.5 Experiments and Results. . . 81

4.5.1 Experimental Setup. . . 81

4.5.2 Results. . . 81

References. . . 85

5 Multi-Robot Task Allocation 89 5.1 Introduction. . . 91

5.2.1 Dynamic Foraging Problems. . . 92

5.2.2 Task Allocation. . . 93

5.3 Multi-Robot Task Allocation for Foraging. . . 94

5.3.1 Model . . . 94

5.3.2 Problem Formulation. . . 98

5.4 An Auction-Inspired Approach for Foraging . . . 98

5.4.1 Standard Sequential-Single-Item Auctions. . . 98

5.4.2 Extended SSI Auctions for Foraging. . . 100

5.5 A Prediction Approach for Foraging Task Allocation. . . 104

5.5.1 Implicit Coordination Framework. . . 104

5.5.2 Communication for Team Awareness and Intentions . . . 106

5.5.3 Predicting Unallocated Tasks. . . 106

5.5.4 Plan Generation. . . 108

(9)

Contents ix

5.6 Experiments. . . 112

5.6.1 Simulator: The Blocks World for Teams . . . 112

5.6.2 Experimental Design . . . 113 5.6.3 Results. . . 115 5.7 Conclusions. . . 117 References . . . 117 6 Interdependence Language 121 6.1 Introduction. . . 123 6.2 Related Work . . . 124 6.3 Language Specification. . . 126 6.3.1 Agent Concept . . . 126

6.3.2 States, Properties and Goals. . . 128

6.3.3 Activity Concept. . . 129

6.4 Interdependence Modelling. . . 133

6.4.1 Interdependence Relationships. . . 133

6.4.2 Hard Interdependence. . . 135

6.4.3 Soft Interdependence. . . 137

6.4.4 Resilience & Efficiency in Diagrams . . . 138

6.5 Case Studies. . . 139

6.5.1 Search and Retrieval Tasks. . . 139

6.5.2 Sokoban - Pushing Boxes. . . 142

6.5.3 DARPA Robotics Challenge (DRC). . . 144

6.5.4 Resilient Analysis . . . 145

References . . . 147

7 Conclusion 151 7.1 Contributions. . . 152

7.2 Limitations and Future Work . . . 154

Summary 157

Samenvatting 159

Acknowledgments 161

(10)

(11)

1

I

NTRODUCTION

Multi-robot teams have potential advantages over a single robot in many appli-cations. Some tasks are beyond the capabilities of an individual robot working alone, e.g., moving a very heavy box, and other tasks can be done more effi-ciently if more robots engage in it, e.g., moving a very long but not heavy box. The use of multiple robots may yield performance gains, but in order to realise such gains the robots need to systematically coordinate with each other. Oth-erwise, team performance can be impaired due to, for example, interference. The dissertation presents our research on cognitive coordination for coopera-tive multi-robot teamwork. This chapter gives a general overview of the disser-tation, introduces the background and motivation of this research, and formu-lates the key research questions.

(12)

2 1. Introduction

Over the past several decades, robots have been designed to assist and supplement humans in diverse areas, such as service robots in both public and private environ-ments [23,47], surveillance [37,52], underwater, space and planetary exploration [1, 38,49], rescue [7,10,14], and so forth. Robots are in particular expected to replace humans to execute missions in hazardous environments, such as a collapsed building after an earthquake, a nuclear and radiation accident, and in a fire, where the work-ing conditions might be inhospitable to humans. In such practical field applications, the major reason why we need a multi-robot system is that the robots can be devel-oped with various sensors and actuators to serve different functionalities (i.e., hetero-geneous robot teams [27,41]) in a complex task [18], and, moveover, when a single robot can complete a task, a team of robots may be more efficient [4] to do so. In ad-dition, a team of robots is believed to be more robust and reliable than a single robot in many areas [2,3,44].

1.1. Multi-Robot Teamwork

In this dissertation, we are interested in human level intelligent multi-robot systems, although multi-robot systems have also been explored by bio-inspired, swarm-based approaches in robotics domains [6,50]. Since robots are developed to assist and work together with humans [21,28], social deliberation should be taken into consideration in a multi-robot system [2]. Social deliberation requires that the robots are capable of generating long term plans to achieve a global or team goal, rather than just dealing with the problems at hand [25]. As the robots in a team may be interdependent on each other to perform a task, they need high-level concepts of coordination to coop-eratively or collaboratively work together For example, in order to push a heavy box by two robots, they should reach an agreement on where to stand and how to push. Obviously, it is not advisable to push the box in opposite directions.

1.1.1. Cognitive Robots

To develop an individual robot with human level intelligence, there is still a big gap between raw sensor readings and rational decision making, as well as between pa-rameter control and abstract action commands. Decision making for a robot is the process of reasoning about situations, in which the robot needs to plan appropri-ate actions out of multiple alternatives. Computational models for reasoning, how-ever, need structured information [5]. Structured information is typically extracted from raw sensor data that are collected in the form of quantitative measures of the world. For instance, a robot equipped with cameras can capture environmental im-ages, which yield raw data about the environment. Such raw images are useless for a robot per se unless the robot can extract relevant meanings from the images to in-terpret the environment. With regard to robot’s control, abstract action commands produced by means of cognitive reasoning need to be parsed for robots to execute

(13)

1.1. Multi-Robot Teamwork 3

actuators in real-time.

Having structured information of raw sensory readings is also important for robots to maintain team awareness. From the point of view of an individual robot, it needs in-terpreted information to understand and know what is happening in the environment so that it can make rational decisions. From the point of view of a team, robots can use structured information to inform each other of what it believes about the world. Communication in a team is facilitated by having structured information expressed in a common language. In order to coordinate activities in a team, the awareness of other team members is needed [53,56]. Team awareness of a robot consists of what its teammates know about each other, including what they are doing and going to do. To this end, being able to interpret the dynamics of the world at the knowledge level facilitates team awareness.

Knowledge representation is a process to establish a connection between real-world sensory data and symbolic representations [58]. Cognitive robots are considered as a special sort of knowledge-based systems [35], in which the robots need to repre-sent the captured sensory data into knowledge-level percepts, i.e., what the robots know or believe about the world, and also be capable of reasoning. Knowledge-level representations can facilitate maintenance of mental attitudes such as beliefs and goals of robots. The current state of the world is considered as the beliefs of a robot, while the desired state of the world is represented as goals. As a distributed system, robots in a team can communicate their beliefs and goals with each other to help them in con-structing the team awareness. However, it is still a challenge to map low-level sensory data to high-level symbolic knowledge representations [55,58], because the sensors of robots usually provide noisy, incomplete and quantitative data, and robots often work in dynamic environments.

1.1.2. Coordinated Teamwork

Multi-robot teamwork has been studied in a variety of settings and applications, and the primary concern of a multi-robot system is to deal with dynamic environments due to the presence of the others [19]. This is because multiple robots may interfere or interact with each other, so a robot in a team cannot foresee what its environment will be because other robots may change the environment. We can say that the need for coordination in a robot team stems from interdependence relationships between the robots. More specifically, one robot performing an activity may influence another robot’s activity. For instance, even if the robots are homogeneous, a team of them can be more efficient than a single robot [4]. If a task requires a team of heterogeneous robots to work together, it means that they have to help and assist one another in order to achieve the team goal.

In order to achieve good team performance, the robots in a team all need to well coordinate their activities. The research on multi-robot coordination has been studied in many domains. RoboCup Soccer [29] is an example providing a standard platform

(14)

4 1. Introduction

to study various technologies for multi-robot teamwork. The robots in the four legged and middle-size leagues are decentralized and have to use all the sensors on board, while the robots in the small size league use a camera on the top of the filed to capture each other’s positions, so the coordination mechanisms are centralized. Other do-mains include multi-robot exploration, searching, flocking, formation, mapping and coverage, in which the robots need to coordinate their movements (i.e., motion coor-dination) in the environment. The exploration and searching tasks are similar, where the robots need to search for targets of interest in the environment [9], and the robots aim at minimizing the exploration time for finding the targets. The RoboCup Rescue competition [30] describes a searching scenario: the robots need to search victims in an unknown environment. The flocking and formation tasks [43,59] require the robots to move together in a flock or in a particular shape. Multi-robot localization and mapping tasks require the robots to simultaneously localize while building the map of the environment [20]. The exploration and searching tasks are different from coverage tasks, in which the robots are supposed to find the optimal way of scanning all the en-vironment [32]. Real applications for coverage include cleaning, lawn moving, and so forth. Robots in some of the tasks (e.g., rescue) discussed above need high-level con-cepts of coordination, while the others can be considered as a sort of real-time control systems (e.g., formation).

This dissertation studies the multi-robot teamwork in the context of search and

re-trieval, which is known as foraging in robotics [11,12,33,48]. In a foraging task, a team of robots is required to search targets of interest in the environment and also deliver them back to a home base. Many practical applications require search and retrieval such as urban search and rescue robots [14,15,26], deep-sea mining robots [60], and autonomous warehouse robots [24]. Coordination for a foraging task concerns how to direct the movements of the robots and how to distribute the workload more evenly in a team. Requiring both searching and delivering makes a foraging task more com-plicated than a pure searching, exploration or coverage task. Foraging robots have to consider not only where to explore but also when to explore because the robots may need to switch their working mode between exploring targets and retrieving targets. Off-line static planning methods cannot be used to find an optimal solution for a for-aging task any more.

Temporal constraints on the team goal of a foraging task make it even more com-plicated, but they are useful to model more practical applications. Many bio-inspired, swarm-based approaches have been proposed to deal with basic foraging problems [11, 33]. In those approaches, typically, robots minimally interact with one another, as in [11], and if they communicate explicitly, only basic information such as the loca-tions of targets or their own localoca-tions are exchanged [45]. Teammates are even simply treated as obstacles so as to avoid the need for coordination [51]. In such a foraging task, the targets are not distinguished, which reduces the need for cooperation and coordination. In order to model more practical applications, we study a more general foraging problem where various types of targets can be distinguished. In addition, the

(15)

1.2. Research Questions 5

various types of targets of interest can be modelled with ordering constraints in a way that they have to be delivered to the home base in a particular order. Differentiating targets is useful for modelling features of the application context, e.g., how badly a vic-tim is wounded, how valuable a mining resource is, or how urgent a product should be picked up. The temporal ordering constraints on the team goal require that the robots should cooperatively coordinate with each other in order to achieve good team performance.

This dissertation studies the foraging teamwork with respect to robot autonomy without humans, and thus human-robot foraging is beyond the scope of the disser-tation. Multi-robot foraging systems have also been used for studying human-robot interaction in field applications such as search and rescue [17,40,42], where robots and humans work together as a team to search for and rescue victims in a diaster area. Though we only look at automated robot teams, human teammates are also possible to be added to the foraging teamwork studied in this dissertation because the pro-posed cognitive robot control framework (see Chapter2) supports structured knowl-edge representations and human-level intelligence. In order to facilitate multi-robot coordination in foraging tasks, in theory, centralized robot teams can be developed because a centralized controller can be more easier to collect all relevant information, maintain team awareness, as well as direct the activities of the robots. In practice, cen-tralized systems are usually inefficient and do not well respond to dynamic environ-ments in large robot teams [13,31]. Therefore, this dissertation investigates decentral-ized multi-robot systems for foraging, in which the robots are fully autonomous and able to make their own decisions and coordinate their activates via explicit communi-cation. Many theories proposed in the dissertation are domain-independent and thus are not limited to foraging tasks but also applicable to other areas.

1.2. Research Questions

The overall aim of this dissertation is to develop a cognitive robot team that can

coop-eratively coordinate with each other to perform a general search and retrieval task. In

order to develop such a cooperative robot team, the following research questions are formulated:

Research Question 1: how to develop a cognitive robot that can perform in dynamic environments and think abstractly and deliberatively?

Robots are embedded systems [8] that operate in dynamic, noisy environments and need to make real-time responses to the environments. However, it is difficult for a computational model for reasoning to directly utilize (potentially noisy and incom-plete) sensory data to make rational decisions because cognitive reasoning requires meaningful structured information with symbolic representations. In addition, cog-nitive reasoning typically produces abstract action commands, which cannot be used

(16)

6 1. Introduction

for parameter control of robots. A cognitive robot control architecture is thus needed to bridge the gap between the low-level behavioral robotic control and the high-level cognitive reasoning. Cognitive robot architectures are concentrated on combining robotics with cognitive agents, with the aim of ensuring that the robots can perform in dynamic environments, as well as have the abilities of thinking abstractly and de-liberatively.

Research Question 2: what is the impact on team performance of various

commu-nication strategies in cooperative robot teams for foraging?

Although it is believed that the use of multiple robots may yield significant perfor-mance gains compared to the perforperfor-mance of a single robot [12,19], multiple robots may also interfere each other, which can decrease team performance. Therefore, ef-fective coordination protocols should be developed to realise such performance gains. As no clear conclusion has been drawn on what kind of communication is most suit-able for robot teams [39], there is a need for gaining a better understanding of the impact of more advanced communication strategies where multiple robots can coor-dinate their activities by explicitly communicating with each other.

Research Question 3: how can a decentralized robot team plan individual paths while

avoiding stationary obstacles as well as teammates in highly congested environments?

When multiple robots engage in teamwork in a shared workspace, there is an in-herent risk that the robots may frequently block each other. As a result, the use of mul-tiple robots may result in conflicts or interference. In particular, deadlock situations have to be taken into consideration in highly congested environmental configurations for decentralized robots to plan their individual paths. Finding an optimal solution to such a multi-robot cooperative pathfinding (or path planning) problem is NP-hard and intractable [54]. Centralized solutions that are based on global search can guar-antee completeness, but do not scale well with large robot teams [16] and cannot be solved in real-time [46]. Decoupled (also called distributed) solutions are fast enough for real-time applications, but they usually cannot guarantee completeness [36] and suffer from deadlock situations. In a decentralized team, the robots can plan their individual paths towards their respective destinations, but they have to avoid colli-sions with teammates. Coordination mechanisms need to be developed to resolve and break deadlock situations.

Research Question 4: how can a team of robots efficiently share workload in a

dy-namic foraging task with temporal constraints on the team goal?

Task allocation has been extensively addressed in various multi-agent/robot do-mains, with the aim of finding an allocation of tasks to robots that minimises the

(17)

1.3. Dissertation Outline 7

overall team cost. In general, however, even if the locations of the targets are initially known and only an optimal solution for a multi-robot routing problem [31,34,61] needs to be found, the problem is NP-hard [13,31,34,62]. The dynamic foraging task allocation problem is also harder than the typical multi-robot exploration [9,13,57], where robots only need to search and locate the targets but do not need to deliver them back. The robots in the foraging tasks have to decide not only where to explore but also when to explore because they may need to frequently switch their modes between exploring targets and retrieving targets. It means that exploration and re-trieval need to be interleaved, e.g., before arriving at a location, a robot does not know whether it should continue exploring another unvisited location or deliver an object found at the location. Therefore, task allocation approaches to the dynamic foraging problem need to be developed so that the robots can share their workload more evenly and efficiently.

Research Question 5: what kind of a modelling tool can be built to facilitate the

identification of the need for coordination in the early design phase of a cooperative

team?

Since a multi-robot team has potential advantages over a single robot in many ar-eas, there is a need to design a robust and efficient multi-robot system. The awareness of interdependence relationships between all the robots in a team is an important re-quirement for designing such a system because the activities performed by one robot may have an influence on the other [22]. Designing a heterogeneous robot team in particular needs to identify the interdependence relationships as the robots serve dif-ferent functions in the team. Interdependence relationships in teamwork can reflect on what has to be coordinated in order to achieve the team goal. In the early design phase of a cooperative team, developers should be able to easily identify interdepen-dencies between team members.

1.3. Dissertation Outline

This dissertation consists of seven chapters. After introducing the background and motivation of this research, the main contributions are presented in Chapters2-6. The research questions formulated in Section1.2are answered in these chapters. Each of these chapters addresses one research question, and the corresponding findings and contributions are discussed in each chapter. The general structure of these chapters is shown in Figure1.1, which also briefly depicts the relations between the chapters. The work in these chapters is based on the publications listed in Section1.4, and each chapter is self-contained and can be read independently.

• Chapter 2, based on [wei-4], presents an agent-based cognitive robot control architecture, which is a generic framework for bridging the gap between em-bedded robotic systems and cognitive agents. Cognitive agents are responsible

(18)

8 1. Introduction d es ig n a n d a n a ly se a m u lt i-a g en t sy st em

Embedded robotic system: perception processing and behavioural control, e.g.,

recognition, localization. co g n it iv e a g en ts co n tr o l ro b o ts Ch6: Interdependence Language

Agent-based cognitive control:

symbolic reasoning and decision

making; manage the robot’s mental states.

Ch5: Task Allocation Ch3: Communication

Ch2: Cognitive Robot Architecture

Ch4: Cooperative Pathfinding

studies multi-robot task

allocation problem for dynamic foraging with ordering constraints

on the team goal.

studies the design and analysis of the resilience and efficiency of a cooperative team that is in particular complex due to comprehensive interdependence relationships between team members.

studies the role of communication in the coordination of cooperative robot teams and its impact on performance in search

and retrieval tasks.

studies multi-robot cooperative

pathfinding (path planning) problem in highly congested environments for decentralized robots.

(19)

1.3. Dissertation Outline 9

for reasoning and action selections, and an agent serves as a cognitive reason-ing layer in the architecture so that a robot is capable of reasonreason-ing about the real-time situations. Perception processing and behavioral control are handled in lower level layers of the control architecture. A cognitive agent in the archi-tecture can be used to control a real or simulated robot. Using agents programs to create the cognitive layer for robotic control is flexible and facilitates reuse. It becomes relatively easy to adapt the control at the cognitive layer itself to vari-ous domains. This chapter answers the 1st research question.

• Chapter3,4and5focus on the high level concepts of coordination mechanisms for cooperative multi-robot teamwork, which is why they are put into a cloud in Figure1.1. For the sake of reproducibility, accessibility and data collection, the experimental studies in these chapters are performed by means of simulated robots. Cognitive agents are designed to control the simulated robots, playing the same role in directing real robots in the robot control architecture discussed in Chapter2. Another reason why we use the simulated robots in these Chap-ters is that, if we use real robots to execute the tasks, too many reliable low-level robotic functions have to be implemented, which is not our focus. For instance, developing a perfect localization function for a real robot is still a challenge task in the robotic community. Moreover, we cannot avoid hardware failures if we use real robot platforms. Debugging a team of real robots with immature plat-forms and low-level functions will spend excessive time. In Chapter3,4and 5, we assume that the simulated robots have perfect localization function, and there is no hardware failure during executing the tasks.

• Chapter3, based on [wei-6], presents an empirical study that investigates the role of communication in the coordination of cooperative robot teams and its impact on performance in search and retrieval tasks. This chapter addresses the 2nd research question, and the findings are used in Chapter4and5. • Chapter 4, based on [wei-1] and [wei-7], studies the multi-robot pathfinding

problem in highly congested environmental configurations. In Chapter3, we as-sume that the simulated robots can pass through each other successfully, which is not always the case for real robots. In particular, if the robots are located in highly congested environment configurations, a coordination mechanism has to be developed so that they can avoid collisions. This chapter presents a novel decentralized domain-independent approach, which allows the robots to make real-time responses to dynamic environments and can resolve a set of bench-mark deadlock situations in a shared workspace. This chapter addresses the 3rd research question.

• Chapter 5, based on [wei-2] and [wei-8], studies the foraging task allocation problem with temporal constraints on the team goal, where a team of robots

(20)

10 1. Introduction

needs to search and retrieve targets of interest in a specific order from an un-known environment. In Chapter3, we only investigate the performance of com-munication but does not look at how to optimize the distribution of the work-load in a team. Two task allocation approaches are proposed and evaluated to address the 4th research question.

• Chapter6, based on [wei-3], presents a formal graphical language as a modelling tool for designing and analysing a cooperative team. The proposed graphical language is domain-independent, so it can be used to evaluate the resilience and efficiency of the cognitive agent teams discussed in Chapter3,4and5. This chapter addresses the 5th research question.

Finally, Chapter7concludes this dissertation by summarizing the main contribu-tions and findings and answering the corresponding research quescontribu-tions formulated in Chapter1. In addition, limitations and possible directions for further work are also discussed in this chapter.

(21)

1.4. List of Publications 11

1.4. List of Publications

[wei-1] Changyun Wei, Koen V. Hindriks and Catholijn M. Jonker. Altruistic Coordi-nation for Multi-Robot Cooperative Pathfinding. Journal of Applied Intelligence (accepted).

[wei-2] Changyun Wei, Koen V. Hindriks and Catholijn M. Jonker. Dynamic Task Allo-cation for Multi-Robot Foraging. Journal of Artificial Intelligence Research (sub-mitted).

[wei-3] Changyun Wei, Koen V. Hindriks, M. Birna van Riemsdijk and Catholijn M. Jonker. A Formal Language of Interdependence in Teamwork. Journal of

Au-tonomous Agents and Multi-Agent Systems (submitted).

[wei-4] Changyun Wei and Koen V. Hindriks. An Agent-Based Cognitive Robot Archi-tecture. In Mehdi Dastani, JomiF Hübner, and Brian Logan, editors,

Program-ming Multi-Agent Systems, volume 7837 of Lecture Notes in Artificial Intelligence,

pages 54-71. Springer Berlin Heidelberg, 2013.

[wei-5] Changyun Wei, Junchao Xu, Chang Wang, Pascal Wiggers and Koen V. Hin-driks. An Approach to Navigation for the Humanoid Robot NAO in Domestic En-vironments. In Ashutosh Natraj, Stephen Cameron, Chris Melhuish, and Mark Witkowski, editors, Towards Autonomous Robotic Systems, volume 8069 of

Lec-ture Notes in Artificial Intelligence, pages 298-310. Springer Berlin Heidelberg,

2014.

[wei-6] Changyun Wei, Koen V. Hindriks and Catholijn M. Jonker. The Role of Commu-nication in Coordination Protocols for Cooperative Robot Teams. In Proceedings

of the 6th International Conference on Agents and Artificial Intelligence, pages

28-39. SciTePress, 2014.

[wei-7] Changyun Wei, Koen V. Hindriks and Catholijn M. Jonker. Multi-robot Coop-erative Pathfinding: A Decentralized Approach. In Moonis Ali, Jeng-Shyang Pan, Shyi-Ming Chen and Mong-Fong Horng, editors, Modern Advances in Applied

Intelligence, volume 8481 of Lecture Notes in Artificial Intelligence, pages 21-31.

Springer International Publishing, 2014.

[wei-8] Changyun Wei, Koen V. Hindriks and Catholijn M. Jonker. Auction-Based Dy-namic Task Allocation for Foraging with a Cooperative Robot Team. In Nils Bulling, editor, Multi-Agent Systems, volume 8953 of Lecture Notes in Artificial

Intelligence, pages 1-16. Springer International Publishing Switzerland, 2015.

[wei-9] Changyun Wei, Koen V. Hindriks and Catholijn M. Jonker. Performance of Communicating Cognitive Agents in Cooperative Robot Teams. Agents and

(22)

12 References

References

[1] Gianluca Antonelli and G Antonelli. Underwater robots. Springer, 2014.

[2] Tamio Arai, Enrico Pagello, and Lynne E Parker. Editorial: Advances in multi-robot systems. IEEE Transactions on multi-robotics and automation, 18(5):655–661, 2002.

[3] Ronald C Arkin. Behavior-based robotics. MIT press, 1998.

[4] Tucker Balch. The impact of diversity on performance in multi-robot foraging. In

Proceedings of the third annual conference on Autonomous Agents, pages 92–99.

ACM, 1999.

[5] Chitta Baral. Knowledge representation, reasoning and declarative problem

solv-ing. Cambridge university press, 2003.

[6] Gerardo Beni. From swarm intelligence to swarm robotics. In Swarm Robotics, pages 1–9. Springer, 2005.

[7] Andreas Birk and Stefano Carpin. Rescue robotics—a crucial milestone on the road to autonomous systems. Advanced Robotics, 20(5):595–605, 2006.

[8] Thomas Bräunl. Embedded robotics: mobile robot design and applications with

embedded systems. Springer, 2008.

[9] Wolfram Burgard, Mark Moors, Cyrill Stachniss, and Frank E Schneider. Coor-dinated multi-robot exploration. Robotics, IEEE Transactions on, 21(3):376–386, 2005.

[10] Daniele Calisi, Alessandro Farinelli, Luca Iocchi, and Daniele Nardi. Multi-objective exploration and search for autonomous rescue robots. Journal of Field

Robotics, 24(8-9):763–777, 2007.

[11] Alexandre Campo and Marco Dorigo. Efficient multi-foraging in swarm robotics. In Advances in Artificial Life, pages 696–705. Springer, 2007.

[12] Y. Uny Cao, Alex S. Fukunaga, and Kahng Andrew B. Cooperative mobile robotics: Antecedents and directions. Autonomous Robots, 4:1–23, 1997.

[13] Prithviraj Dasgupta. Multi-robot task allocation for performing cooperative for-aging tasks in an initially unknown environment. In Innovations in Defence

Sup-port Systems-2, pages 5–20. Springer, 2011.

[14] Angela Davids. Urban search and rescue robots: from tragedy to technology.

(23)

References 13

[15] G De Cubber, D Serrano, K Berns, K Chintamani, R Sabino, S Ourevitch, D Do-roftei, C Armbrust, T Flamma, and Y Baudoin. Search and rescue robots devel-oped by the european icarus project. In 7th International Workshop on Robotics

for Risky Environments, 2013.

[16] Vishnu R Desaraju and Jonathan P How. Decentralized path planning for multi-agent teams with complex constraints. Autonomous Robots, 32(4):385–403, 2012. [17] Barzin Doroodgar, Maurizio Ficocelli, Babak Mobedi, and Goldie Nejat. The search for survivors: Cooperative human-robot interaction in search and rescue environments using semi-autonomous robots. In IEEE International Conference

on Robotics and Automation (ICRA), pages 2858–2863. IEEE, 2010.

[18] Gregory Dudek, Michael RM Jenkin, Evangelos Milios, and David Wilkes. A tax-onomy for multi-agent robotics. Autonomous Robots, 3(4):375–397, 1996. [19] Alessandro Farinelli, Luca Iocchi, and Daniele Nardi. Multirobot systems: a

clas-sification focused on coordination. IEEE Transactions on Systems, Man, and

Cy-bernetics, 34(5):2015–2028, 2004.

[20] John W Fenwick, Paul M Newman, and John J Leonard. Cooperative concurrent mapping and localization. In IEEE International Conference on Robotics and

Au-tomation (ICRA), volume 2, pages 1810–1817. IEEE, 2002.

[21] Terrence Fong, Illah Nourbakhsh, and Kerstin Dautenhahn. A survey of socially interactive robots. Robotics and autonomous systems, 42(3):143–166, 2003. [22] Brian P Gerkey and Maja J Matari´c. A formal analysis and taxonomy of task

al-location in multi-robot systems. The International Journal of Robotics Research, 23(9):939–954, 2004.

[23] Christian Hofner and GK Schmidt. Path planning and guidance techniques for an autonomous mobile cleaning robot. In Proceedings of the IEEE/RSJ/GI

Inter-national Conference on Intelligent Robots and Systems (IROS), volume 1, pages

610–617. IEEE, 1994.

[24] Michael ten Hompel and Thorsten Schmidt. Warehouse Management:

Automa-tion and OrganisaAutoma-tion of Warehouse and Order Picking Systems (Intralogistik).

Springer-Verlag New York, Inc., 2006.

[25] Luca Iocchi, Daniele Nardi, and Massimiliano Salerno. Reactivity and delibera-tion: a survey on multi-robot systems. In Balancing reactivity and social

deliber-ation in multi-agent systems, pages 9–32. Springer, 2001.

[26] James S Jennings, Greg Whelan, and William F Evans. Cooperative search and rescue with a team of mobile robots. In Proceedings of 8th International

(24)

14 References

[27] David Jung and Alexander Zelinsky. Grounded symbolic communication be-tween heterogeneous cooperating robots. Autonomous Robots, 8(3):269–292, 2000.

[28] Kazuhiko Kawamura, Robert T Pack, and M Iskarous. Design philosophy for ser-vice robots. In Systems, Man and Cybernetics, 1995. Intelligent Systems for the 21st

Century., IEEE International Conference on, volume 4, pages 3736–3741. IEEE,

1995.

[29] Hiroaki Kitano, Minoru Asada, Yasuo Kuniyoshi, Itsuki Noda, Eiichi Osawa, and Hitoshi Matsubara. Robocup: A challenge problem for ai. AI magazine, 18(1):73, 1997.

[30] Hiroaki Kitano and Satoshi Tadokoro. Robocup rescue: A grand challenge for multiagent and intelligent systems. AI magazine, 22(1):39, 2001.

[31] Sven Koenig, Pinar Keskinocak, and Craig A Tovey. Progress on agent coordina-tion with cooperative auccoordina-tions. In AAAI, 2010.

[32] Chan Sze Kong, New Ai Peng, and Ioannis Rekleitis. Distributed coverage with multi-robot system. In IEEE International Conference onRobotics and

Automa-tion (ICRA), pages 2423–2429. IEEE, 2006.

[33] Stephan Krannich and Erik Maehle. Analysis of spatially limited local communi-cation for multi-robot foraging. In Progress in Robotics, pages 322–331. Springer, 2009.

[34] Michail G. Lagoudakis, Evangelos Markakis, David Kempe, Pinar Keskinocak, Anton Kleywegt, Sven Koenig, Craig Tovey, Adam Meyerson, and Sonal Jain. Auction-based multi-robot routing. In Proceedings of Robotics: Science and

Sys-tems, pages 343–350, 2005.

[35] Hector Levesque and Gerhard Lakemeyer. Cognitive robotics. Handbook of

knowledge representation, pages 869–886, 2007.

[36] Ryan Luna and Kostas E Bekris. Efficient and complete centralized multi-robot path planning. In IEEE/RSJ International Conference on Intelligent Robots and

Systems (IROS), pages 3268–3275. IEEE, 2011.

[37] Luiz S Martins-Filho and Elbert EN Macau. Trajectory planning for surveillance missions of mobile robots. In Autonomous Robots and Agents, pages 109–117. Springer, 2007.

[38] Maja J Mataric and Gaurav S Sukhatme. Task-allocation and coordination of mul-tiple robots for planetary exploration. In In Proceedings of the 10th International

(25)

References 15

[39] Yogeswaran Mohan and SG Ponnambalam. An extensive review of research in swarm robotics. In World Congress on Nature & Biologically Inspired Computing, pages 140–145. IEEE, 2009.

[40] Robin R Murphy. Human-robot interaction in rescue robotics. IEEE Transactions

on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 34(2):138–

153, 2004.

[41] Robin R Murphy, Christine L Lisetti, Russ Tardif, Liam Irish, and Aaron Gage. Emotion-based control of cooperating heterogeneous mobile robots. Robotics

and Automation, IEEE Transactions on, 18(5):744–757, 2002.

[42] Illah Nourbakhsh, Katia Sycara, Mary Koes, Mark Yong, Michael Lewis, and Steve Burion. Human-robot teaming for search and rescue. Pervasive Computing,

IEEE, 4(1):72–79, 2005.

[43] Reza Olfati-Saber. Flocking for multi-agent dynamic systems: Algorithms and theory. IEEE Transactions on Automatic Control, 51(3):401–420, 2006.

[44] Lynne E Parker. Distributed intelligence: Overview of the field and its application in multi-robot systems. Journal of Physical Agents, 2(1):5–14, 2008.

[45] Lynne E Parker. Distributed intelligence: Overview of the field and its application in multi-robot systems. Journal of Physical Agents, 2(1):5–14, 2008.

[46] Lynne E Parker. Decision making as optimization in multi-robot teams. In

Dis-tributed Computing and Internet Technology, pages 35–49. Springer, 2012.

[47] Erwin Prassler, Arno Ritter, Christoph Schaeffer, and Paolo Fiorini. A short history of cleaning robots. Autonomous Robots, 9(3):211–226, 2000.

[48] Paul E Rybski, Amy Larson, Harini Veeraraghavan, Monica Anderson, and Maria Gini. Performance evaluation of a multi-robot search & retrieval system: Experi-ences with mindart. Journal of Intelligent and Robotic Systems, 52(3-4):363–387, 2008.

[49] K Schilling and C Jungius. Mobile robots for planetary exploration. Control

Engi-neering Practice, 4(4):513–524, 1996.

[50] Amanda JC Sharkey. Robots, insects and swarm intelligence. Artificial

Intelli-gence Review, 26(4):255–268, 2006.

[51] Dylan A Shell and Maja J Mataric. On foraging strategies for large-scale multi-robot systems. In IEEE/RSJ International Conference on Intelligent Robots and

(26)

16 References

[52] Guangming Song, Hui Wang, Jun Zhang, and Tianhua Meng. Automatic docking system for recharging home surveillance robots. IEEE Transactions on Consumer

Electronics, 57(2):428–435, 2011.

[53] Luc Steels. When are robots intelligent autonomous agents? Robotics and

Au-tonomous systems, 15(1):3–9, 1995.

[54] Pavel Surynek. An optimization variant of multi-robot path planning is in-tractable. In AAAI, 2010.

[55] Moritz Tenorth and Michael Beetz. Knowrob—knowledge processing for au-tonomous personal robots. In IEEE/RSJ International Conference on Intelligent

Robots and Systems (IROS), pages 4261–4266. IEEE, 2009.

[56] Claude F Touzet. Robot awareness in cooperative mobile robot learning.

Au-tonomous Robots, 8(1):87–97, 2000.

[57] Craig Tovey, Michail G Lagoudakis, Sonal Jain, and Sven Koenig. The generation of bidding rules for auction-based robot coordination. In Multi-Robot Systems.

From Swarms to Intelligent Automata Volume III, pages 3–14. Springer, 2005.

[58] Emil Vassev and Mike Hinchey. Knowledge representation for cognitive robotic systems. In 15th IEEE International Symposium on

Object/Component/Service-Oriented Real-Time Distributed Computing Workshops, pages 156–163. IEEE,

2012.

[59] Lovekesh Vig and Julie A Adams. Multi-robot coalition formation. IEEE

Transac-tions on Robotics, 22(4):637–649, 2006.

[60] Junku Yuh. Design and control of autonomous underwater robots: A survey.

Au-tonomous Robots, 8(1):7–24, 2000.

[61] Xiaoming Zheng and Sven Koenig. K-swaps: cooperative negotiation for solving task-allocation problems. In Proceedings of the 21st international jont conference

on Artifical intelligence, pages 373–378. Morgan Kaufmann Publishers Inc., 2009.

[62] Robert Zlot and Anthony Stentz. Market-based multirobot coordination for com-plex tasks. The International Journal of Robotics Research, 25(1):73–101, 2006.

(27)

2

C

OGNITIVE

R

OBOT

A

RCHITECTURE

This chapter introduces a new cognitive robot control architecture in which the cognitive layer can be programmed by means of the agent programming lan-guage GOAL. The architecture exploits the support that agent-oriented pro-gramming offers for creating cognitive robotic agents, including symbolic knowl-edge representation, deliberation via modular, high-level action selection, and support for multiple, declarative goals. The benefits of the architecture are that it provides a flexible approach to develop cognitive robots and support for a clean and clear separation of concerns about symbolic reasoning and sub-symbolic processing. We discuss the design of our architecture and the issue of translating sub-symbolic information and behavior control into symbolic representations needed at the cognitive layer. An interactive navigation task is presented as a proof of concept.

(28)

18 2. Cognitive Robot Architecture

This chapter is based on the following publication:

Changyun Wei and Koen V. Hindriks. An Agent-Based Cognitive Robot Architecture. In Mehdi Dastani, JomiF Hübner, and Brian Logan, editors, Programming Multi-Agent Systems, volume 7837 of Lecture Notes in Artificial Intelligence, pages 54-71. Springer Berlin Heidelberg, 2013.

(29)

2.1. Introduction 19

2.1. Introduction

The main motivation for our work is the need for a flexible, generic, high-level con-trol framework that facilitates the development of re-taskable robot systems and pro-vides a feasible alternative to the usual task- and domain-dependent development of high-level robot control. As cognitive robots are supposed to handle complex reason-ing problems in dynamic environments [28], we believe that agent-oriented program-ming offers such an approach as it supports the programprogram-ming of cognitive agents. Us-ing agent programs to create the cognitive layer in a robot control architecture is natu-ral and provides sevenatu-ral benefits. It becomes relatively easy to adapt the control at the

cognitive layer itself to various domains. This approach is flexible and, if functionality

of other layers is generic and can be used in multiple task domains, facilitates reuse. An agent-based approach, moreover, provides support for autonomous, reactive, and proactive behaviors and also endows a robot with the required deliberation mecha-nism to decide what to do next [6]. Of course, generality may come at a trade-off and does not imply that a generic architecture will always perform better than a dedicated robot control architecture [23].

Designing and developing a cognitive robot control architecture poses several chal-lenges. Robots are embedded systems that operate in physical, dynamic environ-ments and need to be capable of operating in real-time. A range of perception and motor control activities need to be integrated into the architecture. This poses a chal-lenge for a cognitive, symbolic architecture as “it can be particularly difficult to gen-erate meaningful symbols for the symbolic components of cognitive architectures to reason about from (potentially noisy) sensor data or to perform some low-level tasks such as control of motors” [17]. Ideally, moreover, such an architecture should pro-vide support for the integration or exchange of new and different sensors and behav-iors when needed. Given the complexity and the number of components needed in a robot control architecture, one also needs to consider how all the processing compo-nents in the system communicate and interact with each other [18].

The main focus and contribution of this paper is to provide a simple but also generic and flexible solution to integrate the knowledge representation and reasoning capabilities needed for a cognitive robot [32,33]. As perception and action need to be tightly coupled for a cognitive robot to be able to operate effectively [28], we also dis-cuss this relationship. We propose an agent-based cognitive robot control architecture that integrates low-level sub-symbolic control with high-level symbolic control into the robot control framework. We use the agent programming language GOAL[20,22] for implementing the cognitive layer, whereas low-level execution control and processing of sensor data are delegated to components in other layers in the proposed architec-ture. GOAL, among others, supports goal-oriented behavior and the decomposition of complex behavior by means of modules that can focus their attention on relevant sub-goals. GOALhas already been successfully applied to control real-time, dynamic environments [21], and here we demonstrate that it also provides a feasible approach

(30)

for controlling robots. In our approach, the cognitive layer is cleanly separated from the other layers by using the Environment Interface Standard (EIS; [7]). As a proof of concept, we will use a task, in which a robot will navigate in an office environment in order to deliver a message to one of our colleagues, to show how the agent-based cognitive control can be realized in physical robots. The main contribution of our proposed architecture is that it provides:

• a decoupled framework for combining low-level behavioral robot control with high-level cognitive reasoning,

• a generic interface for mapping sensory information to symbolic representa-tions needed at the cognitive layer, and

• a flexible mechanism in the cognitive layer for synchronizing percepts and ac-tions.

The paper is structured as follows. Section2.2briefly discusses some related work. Section2.3presents and discusses the design of the cognitive robot control architec-ture. Section2.4presents a proof of concept implementation. Section2.5concludes the paper and discusses future work.

2.2. Related Work

Cognitive robots are autonomous and intelligent robot systems that can perform tasks in real world environments without any external control, and are able to make deci-sions and select actions in dynamic environments [30]. It remains a challenge, how-ever, to integrate the symbolic problem solving techniques such as knowledge repre-sentation, reasoning, and planning developed within artificial intelligence with sub-symbolic functions such as perception and motor control on a robot [14]. It has been argued, however, that knowledge representation is important to enable robots to rea-son about its environment [28,32,33]. Here we briefly discuss some related work that either explicitly aims at developing a cognitive robot architecture or uses some kind of symbolic representation for controlling a robot.

The work that is most similar in spirit to our own is that of [31] and [12]. In [31] the high-level language Golog is used for controlling a robot. Golog supports writ-ing control programs in a high-level, logical language, and provides an interpreter that, given a logical axiomatization of a domain, will determine a plan. [12] proposes teleo-reactive programming for controlling a robot. A teleo-reactive program con-sists of multiple, prioritized condition-action rules. However, neither Golog nor teleo-reactive programming provide a BDI perspective on programming agents. Moreover, these papers do not discuss the robot control architecture that allows the symbolic framework to control the robot.

CRAM[6] is a software toolbox designed for controlling the Rosie robot platform developed at the Technische Universität München. It makes use of Prolog and

(31)

in-2.3. Cognitive Robot Control Architecture 21

cludes a plan language that provides a construct for concurrent actions. The CRAM approach also aims at providing a flexible alternative to pre-programmed robot con-trol programs. [27] proposes to use a common sense ontology for defining predicates for high-level control of a robot that is integrated into the CRAM architecture. One difference between the CRAM and our approach is that the reasoning and planning components are two separate components in CRAM whereas they are integrated into an agent architecture in our robot control framework. Moreover, we propose the use of an explicit interface component for connecting the low-level and high-level control in our architecture that allows for a clean separation of concerns.

It has been argued that building robot systems for environments in which robots need to co-exist and cooperate with humans requires taking a cognitive stance [11] . According to [11], translating the key issues that such robot systems have to deal with requires a cognitive robot control architecture. Taking a cognitive stance towards the design and implementation of a robot system means that such a system needs to be designed to perform a range of cognitive functions. Various cognitive architectures, such as ACT-R [1] and SOAR [25], have been used to control robots. These architec-tures were not primarily aimed at addressing the robot control problem and in this sense are similar to agent programming languages, the technology that we advocate here for controlling robots. SOAR has been used to control the hexapod HexCrawler and a wheeled robot called the SuperDroid [17]. ADAPT (Adaptive Dynamics and Ac-tive Perception for Thought) is a cogniAc-tive architecture based on SOAR that is specif-ically designed for robotics [9]. The SS-RICS (Symbolic and Sub-symbolic Robotic In-telligent Control System) architecture for controlling robots is based on ACT-R; SS-RICS is intended to be a theory of robotic cognition based on human cognition [3,23]. This work mainly focuses on integrating a broad range of cognitive capabilities into an architecture instead of on more pragmatic issues related to programming cogni-tive robotics and reuse. Unlike [26], we are not mainly concerned here with the long-term goal of developing robotic systems that have the full range of cognitive abilities of humans based on a cognitive architecture such as SOAR. Our work is oriented to-wards providing a more pragmatic solution for the robot control problem as discussed above. Still our work may contribute to the larger goal that [26] sets as there appear to be quite a few similarities between BDI-based agents and cognitive architectures such as ACT-R and SOAR).

2.3. Cognitive Robot Control Architecture

This section introduces our cognitive robot control architecture. We discuss several issues including the processing and mapping of sensory data to a symbolic represen-tation, the translation of high-level decisions into low-level motor control commands, and the interaction of the various architecture components.

(32)

2.3.1. Overall Design of the Architecture

A high-level overview of our layered architecture is shown in Figure2.1. The architec-ture consists of four layers including a symbolic, cognitive layer realized by means of the agent programming language GOAL, a middle layer for controlling robot behavior (written in C++), and a hardware control layer (using URBI1, a robotic programming language). The Environment Interface Standard (EIS) layer provides the technology we have used to manage the interaction between the symbolic and sub-symbolic lay-ers. We argue here that EIS provides a tool that can be extended to deal with the issue of translating sub-symbolic sensory data that consists typically of noisy, incomplete, and quantitative measurements into symbolic representations that are needed in the symbolic cognitive layers to support reasoning. Note that the environment layer is not part of the architecture but refers to the physical environment that the robot operates in.

The main functionality for controlling a robot is placed in the behavioral control layer. In addition to the functions such as (object) recognition, navigation, localiza-tion, path planning and other common functions, this layer is also responsible for communicating with the higher-level symbolic components, including the interpreta-tion of symbolic messages that represent acinterpreta-tions and making the robot perform these actions in its physical environment.

Robot Platform Environment Urbi Server ( Urbiscript ) Robotic Behavioral Control

Environment Interface Cognitive Control

TCP/IP

Agent-based Language: GOAL

Functional processing

(Processing sensory data for desired information)

XML Schema Mapping

(Mapping features, objects, faces, etc.)

quantitative value symbols

Symbolic action commands

Behavior execution

(Monitoring and executing behaviors)

Parsing commands

(Parsing function name and its parameters)

singles for invoking components Symbolic messages

Information flow

Capturing sensors

(Capturing raw sensory data)

Binary data

Control flow

Robot SDK function execution

(Executing SDK functions and controlling motors)

Singles for invoking SDK functions

Figure 2.1: The overall design of the architecture

The interface layer acts as a bridge between the behavioral and cognitive control layers. Because these layers use different languages for representing sub-symbolic and symbolic information, respectively, we need an interface to translate between these representations. The cognitive control layer acts as a task manager for the robot and provides support for managing the robot’s mental state which allows the robot to keep

(33)

2.3. Cognitive Robot Control Architecture 23

track of what is happening while executing a task.

2.3.2. System Architecture and Components

A robot control architecture provides an organizational structure for software compo-nents that control a robotic system [8]; such architectures are specialized because of the unique requirements that embedded systems such as robots impose on software. Here we will discuss the detailed architecture shown in Figure2.2that we propose that matches the layered architecture framework of Figure2.1.

Navigation

Sensing / Sensors Acting / Actuators

Other Robots Environment Memory Working Memory Global Memory Messages Interpreter Environment Configurator Features Configuration Objects Configuration Localization Odometry Debug Monitor Sensor Information GUI

Communication GUI Manual Control GUI Camera Calibration GUI

Grid Map GUI Communication Action Execution Events Configuration Environment Mode Perception

Robot Hardware Platform

Environment

Robot Platform

Urbi Server (Urbiscript Execution Cycle)

Hardware

Environment Local Software

[C++ Robotic Behavior Layer ] Information Processing, Knowledge Processing, Action Execution

[Java Environment Interface Layer] Bridge of Communication, Environment Management [GOAL Agent ] Reasoning, Decision Making, Action Selection

C++ Application Java Application GOAL Agent

Grid Map Path Planning Knowledge Processing Perception Deliver Action Receiver Information Processor Wireless Connection Environment Management Action Environment Interface Reasoning Decision Making Objects Recognition Features Matching Features Extraction Behaviors Motions Action Perception Knowledge Belief Maintaining Goals Information Fusion

(34)

Robot Platform

The architecture has been implemented on the humanoid NAO robot platform from Aldebaran Robotics.2 We have used the URBI middleware [5] that provides the

urbis-cript language for interfacing with the robot’s hardware. We have chosen URBI instead

of the Robot Operating System (ROS3) platform because it does not include the or-chestration layers present in URBI that provide support for parallel, tag-based, and event-driven programming of behavior scripts. When the application is executed, an urbiscript program, which initializes all API parameters of sensors and motors (e.g., the resolution and frame rate of camera images), is sent to configure the robot plat-form.

Behavioral Control

The behavioral control layer is written in C++, connecting the robot hardware layer with higher layers via a TCP/IP connection. This layer is mainly responsible for in-formation processing, knowledge processing, communication with the deliberative reasoning layer and external robots, and action and behavior executions. All of these components can operate concurrently. The main functional modules include:

• Sensing, for processing sensory data and receiving messages from other robots.

Sensors that have been included are sonar, camera, microphone, an inertial sen-sor, and all sensors monitoring motors. Because the memory space required for camera images is significantly bigger than that for other sensors, the transmis-sion of images is realized via a separate communication channel.

• Memory, for maintaining the global memory and working memory for the

be-havioral layer. In global memory, a map of the environment and properties of objects or features extracted from sensor data are stored. We have used a 2D grid map for representing the environment. This map also keeps track which of the grid cells are occupied and which are available for path planning. The working memory stores temporary sensor data (e.g., images, sonar values) that are used for updating the global memory.

• Information Processor, for image processing. This component provides

sup-port for object recognition, feature extraction and matching, as well as for infor-mation fusion. Inforinfor-mation fusion is used to generate more reliable inforinfor-mation from the data received from different sensors. For example, the odometry navi-gation component is only able to provide a rough estimate of the robot’s position due to joint backlash and foot slippage. To compensate for this, the robot has also been equipped with the capability to actively re-localize or correct its posi-tion by means of predefined landmarks. We have used the OpenCV [10] library

2_{http://www.aldebaran-robotics.com/}

(35)

to implement algorithms and methods for processing images captured by the camera. Algorithms such as Harris-SIFT [4], RANSAC [15], and SVM [19] for ob-ject recognition and scene classification have been integrated into this module. • Environment Configurator, for interpreting and classifying events that occur

in the environment. For example, when the output of the left sonar exceeds a certain value, this module may send a corresponding event message that has been pre-configured. This is useful in case such readings have special meaning in a domain. In such cases, an event message may be used to indicate what is happening and which objects and features such as human faces and pictures have been detected. This is a key component in our approach for mapping sub-symbolic data to sub-symbolic representations.

• Navigation includes support for localization, odometry, path planning and map-ping components, which aid the robot in localizing itself and in planning an

op-timal path to a destination in a dynamic, real-time environment. Several trans-formations between layers are required for navigation. The cognitive layer uses a 2D Grid-map Coordinate System (GCS) which is also used by the the path plan-ning component. When following a planned path, after each walking step, the odometry component is used for keeping track of the robot’s actual position and for providing the real-time coordinates of the robot which are used for planning its next walking step. The odometry sensors, however, provide the robot’s coor-dinates in a so-called World Coordinate System (WCS) that needs to be mapped to the robot’s position in the 2D Grid-map Coordinate System (GCS) for path planning. At the lowest layer that executes the walking steps, moreover, the GCS position has to be transformed into the Local Coordinate System (LCS) that the robot platform uses for actual motor movements.

• Communication provides outputs from the Environment Configurator to the

interface layer and receives action messages from this same layer. The commu-nication component mainly is a technical component that acts as a Server/Client infrastructure and uses the TCP/IP protocol for actual message delivery. The be-havior layer in the architecture initiates and starts up a unique server to which a cognitive layer can connect as a client (thus facilitating the swapping of control from one robot to another).

• Debugger Monitor provides several GUIs that are useful for debugging robot

programs, enabling developers to visualize sensory data and allowing them to set specific function parameters. This component also includes a Wizard of Oz

interface to conduct human-robot interaction experiments.

• Action Execution, for instructing a robot to perform concrete behaviors and

actions. The actions include motion movements such as walking and turning around while the behaviors include predefined body gestures such as sitting down, standing up and raising the arms of a humanoid robot.

(36)

Environment Interface

The interface layer between the behavior and cognitive layers has been built using a software package called Environment Interface Standard (EIS; [7]). The core compo-nents in this layer are an Environment Model component that establishes a connec-tion between the cognitive layer with the behavioral control layer, an Environment

Management component that initializes and manages the interface, and an Environ-ment Interface component that provides the actual bridge between the cognitive and

behavioral layer.

The benefit of using EIS for implementing the interface layer is that it already provides a well-defined and structured language for representing knowledge, the so-called Environment Interface Language (EIL). Moreover, various agent platforms sup-port EIS which implies that our architecture may be reused to connect languages other than GOALwithout much effort to the robot platforms that are supported by the archi-tecture. We thus obtain immediately a clear, simple but sufficiently expressive target language for mapping sub-symbolic data maintained in the behavioral layer to sym-bolic percepts used in the cognitive layer. Because this language is already supported by GOAL, this allows us moreover to focus completely on the problem of mapping sub-symbolic data to the EIL language. We discuss a generic scheme to do this in Sec-tion2.3.4below.

Deliberative Reasoning and Decision Making

Knowledge representation and reasoning is an essential component for a cognitive robot that allows such a robot to keep track of what is happening in its environment and to make rational decisions to change that environment [32,33]. The cognitive control layer provides support for reasoning and decision making. In our architec-ture, we employ the GOALagent programming language for programming the high-level cognitive agent that controls the robot. GOALis a language for programming logic-based, cognitive agents that use symbolic representations for their beliefs and goals from which they derive their choice of action. Due to space limitations, we do not describe GOAL agent programs in any detail here but refer the interested reader for more information to [20,22]. In Section2.4, we will use a navigation task to illus-trate how a GOALagent program can be used for deliberative reasoning and decision making for controlling a robot.

2.3.3. Decoupled Architecture Layers

Similar to most layered architectures, we also distinguish reactive and deliberative control in our architecture. In our architecture these layers are loosely coupled and connected through a separate interface layer. We have deliberately chosen for this setup. Alternatively, these layers could have been more tightly coupled. Tight coupling has some benefits such as a more robust integration and a reduced communication overhead which often leads to higher performance than loosely coupled systems [24].

(37)

These benefits are obtained by using a memory component that is shared and directly accessible by all other components. This also avoids the need to “duplicate” informa-tion at different layers, although one should note that also in a tightly coupled setup mappings between sub-symbolic and symbolic data are needed. The disadvantages of a more tightly coupled approach are an increased complexity of the architecture and a higher interdependence between architecture components. As a consequence, it becomes more difficult to extend such architectures in order to be able to handle a range of different tasks [24].

Our choice to opt for decoupled layers is motivated by our objective to design an architecture that is as flexible as possible. A benefit of decoupling is that it provides for a clean separation of concerns. Moreover, decoupling of these layers facilitates the more or less independent programming of high-level, cognitive agents that control a robot as well as of lower-level behavioral functions. An agent programmer, for exam-ple, does not need to spend time handling object recognition. Similarly, a behavior programmer who codes in C++ does not have to consider decision making nor even master the agent programming language used in the cognitive layer. The main chal-lenge that needs to be faced in a decoupled approach is how to translate or transform sub-symbolic data to symbolic representations, an issue to which we turn next.

2.3.4. Knowledge Acquisition and Representation

We have designed a generic method for transforming sub-symbolic data to symbolic representations that can be used in the cognitive layer. The main function of the in-terface layer is to perform the actual transformation. The inin-terface layer thus needs to be provided with the functionality to process all sub-symbolic data that is sent to this layer by the behavioral layer. Of course, the type of data that needs to be processed and transformed depends on the robot platform that is used and the sensors available on that platform. In our case, we have listed all the available sensors and associated functionality for processing raw sensor data in the first two columns of Table2.1. The key issue is how to interpret and map the sub-symbolic data obtained through sensors to a symbolic representation that is useful for decision making in the cognitive layer. In essence, what we need in order to obtain a transformation is a coding scheme. Such a coding scheme should not only indicate how to map data to a particular representa-tion but also allow to specify when such a translarepresenta-tion should be performed.

The solution that we propose as a generic solution for creating a coding scheme for transforming data into symbols is to use a standard template. To this end, we have used XML to create such a standard message template with the following structure: