A Representation Of Robotic Behaviors Using Component Port Arbitration Ali Paikan, Giorgio Metta and Lorenzo Natale Abstract— Developing applications considering reactiveness, scalability and re–usability has always been at the center of attention of robotic researchers. Behavior–based architectures have been proposed as a programming paradigm to develop robust and complex behaviors as integration of simpler modules whose activities are directly modulated by sensory feedback or input from other models. The design of behavior based systems, however, becomes increasingly difficult as the complexity of the application grows. This article proposes an approach for modeling and coordinating behaviors in distributed archi- tectures based on port arbitration which clearly separates representation of the behaviors from the composition of the software components. Therefore, based on different behavioral descriptions, the same software components can be reused to implement different applications. I. INTRODUCTION Behavior–based systems (BBSs) have been devised to program robot applications that do not rely on models of the environment and for which reaction to sensory feedback is crucial. However, BBSs are difficult to design when the task involves interaction of large number of software components. Perhaps, this is one of the reasons why behavior–based approaches are not widely applied to the complex robotic applications. The crucial problem is to represent behaviors and components separately so that the latter can be reused more freely. Best practices in robotics [1] promotes the idea that composition and coordination of software component should be separated during the software development life– cycle. Moreover, a proper abstract representation of the behaviors is crucial for the development of complex robotic application. In modern robotic middlewares, coordination of software modules is more difficult since components run asynchronously and are distributed across a network of computers. We introduce a mechanism for modeling and coordination of behaviors based on port arbitration [2]. Coordination between modules is achieved by defining a set of rules that specify how to arbitrate conflicts between modules that run concurrently and compete for the same resources. In our approach, building an application out of reusable software is done in two phases (Figure 1). First, software components are configured and interconnected in the (distributed) system. Second, a behavioral model is developed which describes the desired behavior of the system. Coordination is then defined by extracting a set of rules from the behavioral model. These The work in this paper was supported by the FP7 EU project Xperi- ence(No. 270273). A. Paikan, L. Natale and G. Metta are with Ital- ian Institute of Technology (IIT), Genova, Italy. Emails: {ali.paikan,lorenzo.natale,giorgio.metta}@iit.it. Look FollowFace ReachObject GraspObject ResetHand Search ~ ${grasped} XML XML Look FollowFace ReachObject GraspObject ResetHand Search ~ ${grasped} XML XML Arbitration Rule Extractor Look FollowFace ReachObject GraspObject ResetHand Search ~ ${grasped} XML XML Components Configuration Behavioral Description Final Application Fig. 1. Application generation from behavioral description. rules determine how data is allowed to travel across the network of components and therefore implicitly define which components are inhibited or free to run. Based on different behavioral description, the same software components can be reused to implement different robotic applications. II. RELATED WORK Our proposed idea has some similarities with the coordi- nation models and languages which are originated from the work of Gelernter and Carriero [3]. A coordination model provides a framework in which the interaction of active and independent entities can be expressed using a specific lan- guage. An example of coordination languages is Linda [4]. This language defines a mechanism to coordinate concurrent computations by means of messages which are formated in tuple structure and can be added to the computation environ- ment. It has been argued that coordination models similar to Linda 1 have the fundamental drawback of intermixing com- putation with coordination [5]. In contrast to Linda which requires computations to make use of specific primitives for coordination, Reo [6] provides a paradigm for composition of distributed software components and services based on the notion of connectors. Reo enforces an exogenous channel– based coordination model that defines how designers can build connectors, out of simpler ones. Application designers can use Reo for compositional construction of connectors that coordinate the cooperative behavior of components in a component–based system. A comprehensive and detailed survey of coordination languages is also given in [5]. Despite coordination languages have had a large devel- opment in the context of parallel and distributed system during the last two decades, to our knowledge, there have not been enough workaround for their realization over available robotic frameworks or study of the required component 1also known as endogenous languages arXiv:1412.4847v1 [cs.RO] 16 Dec 2014 models for their implementation. In contrast, our proposed coordination model exploits component data–flow ports and the connections among the components which commonly can be found in every nowadays robotic frameworks. III. PORT-ARBITRATED COORDINATION The idea of the port arbitration and its applications in robotics has already been discussed in our previous works [2], [7] and [8]. However, in the following, we shortly describe the concept of component coordination using port arbitration mechanism. A simple software component can be seen as a computa- tional unit with a set of data–flow ports for communication. It has a set of input ports to receive information from other components and a set of output ports to stream out the data. A component checks for the condition in which to become active (e.g. upon receiving data), processes and sends the results through its output ports. We make the following assumptions for each component: • The preconditions in which a component gets activated are local to the component itself and they are not visible to other modules. In other words components cannot directly activate or deactivate others. • Data is streamed out if and only if the component is active. For example, an object detector sends object position information through its output port only if the object has been detected. We focus on the typical scenario of a publish–subscribe architecture. The output of a component can be connected to one or more input ports of other component. Interestingly it is also possible to connect multiple outputs to the same input of a component. In the example from Figure 2, Object Detector is a component which processes the streamed images from the robot camera and produces the 3D position of the object when detected. Its output is connected to Gaze Control which receives a 3D position in the robot root frame and controls the head of the robot to gaze at the target point. The output port of Object Detector is also connected to the Arm Control component which moves the robot arm’s end- effector to reach the target point received from its input port. A crucial aspect is that multiple outputs can be connected to the input port of a component. Without proper coordination among the components, data from different components can be delivered to an input port at any time, potentially causing conflicts. For example, the outputs of Object Detector and Rest Arm are both connected to the input port of Arm Control. These components conflict when they are both active and concurrently send position data to Arm Control. To solve this problem, every input port has an arbitrator which can be configured with a set of rules to properly arbitrate the data received from multiple sources. We propose a mechanism to describe the behavioral model of the task. This model allows to derive the necessary rules that properly configure the arbitrators to implement the task. We define the ingredients of our port-arbitrated coordination system as follow: /collision:o Collision Detector /Object/pos:o /collision:o mux arbitrator /RestArm/pos:o Arm Control /Arm/pos:i Gaze Control /Gaze/pos:i Object Detector /Object/pos:o /Object/img:i Face Detector /Face/pos:o /Face/img:i Random Look /RandomLook/pos:o /RestArm/pos:o Rest Arm Fig. 2. An example of different components and the connections among them. The shaded box represents how an arbitrator is employed in the input port of a component to select between multiple connections to the same port. • A pair of source and destination names which identifies the connection from an input port to an output port (e.g. {/Face/pos:o, /Gaze/pos:i}). • An active connection is a connection which has recently delivered data. When data arrives to an input port from a connection, the latter becomes active and remains active for a constant time T. The connection will be inactive if no more data arrives within time T. Notice that the activation of a connection is defined solely in terms of the data it delivers to the port and irrespectively of the result of the arbitration. A. What are the rules and how they are used in arbitrator? Each input port has an arbitrator which selects a single connection among the ones that are active at each time. This “winner” connection delivers data to the component whereas data from the other connections gets discarded. This concept is drawn in Figure 2 for the case of three connections. As it is shown in the figure (shaded part), the port arbitrator is implemented as a multiplexer that let, at most, one active connection deliver its data to the component at each time. In Figure 2, Rest Arm periodically sends the resting position of the arm through RestArm/pos:o to Arm Control. This causes the robot to park and keep the arm in the resting position. To grasp an object we want to hand over control of the arm to another component, Object Detector, that sends the position of the object to be grasped to Arm Control. This can be done by inhibiting the connection from /RestArm/pos:o to /Arm/pos:i in the arbitrator of /Arm/pos:i. In other words we want to specify a rule so that the connection {/RestArm/pos:o, /Arm/pos:i} can be selected only if connection {/Object/pos:o, /Arm/pos:i} is inactive, formally: /RestArm/pos : o ∧¬/Object/pos : o ⇒Select(/RestArm/pos : o) Suppose now we add another component which is re- sponsible for stopping the the arm upon collision. This component is called Collision Detector in Figure 2 and it sends status messages through the port /collision:o when it detects that the arm collides with an object (e.g. using tactile or torque sensors). The desired behavior can be achieved by adding rules in /Arm/pos:i so that activation of /RestArm/pos:o and /Object/pos:o is inhibited by /collision:o, i.e.: /RestArm/pos : o ∧¬/Object/pos : o ∧¬/collision : o ⇒Select(/RestArm/pos : o) /Object/pos : o ∧¬/collision : o ⇒Select(/Object/pos : o) Notice that if no rules specifically select a connection, the latter can never deliver data to the component. In our example, no rule is written for /collision:o in Arm Control; so Collision Detector will never deliver data to this component. As it is, its activation state is only used in the evaluation of the rules of other connections. When data arrives from a connection to an input port, the corresponding arbitrator has to decide whether to ac- cept or discard it. First the port updates the state of each connection (i.e. active or inactive). Based on the activation states, the required Boolean inputs for the arbitrator are generated (i.e. True for the active and False for the inactive connections). The rules are structured in Binary Decision Diagrams (BDD) [9] in the arbitrator. By reasoning on the BDD, the arbitrator evaluates whether the connection discards or delivers data to the component. IV. MODELING THE BEHAVIORS In general, behavior-based robotic controllers consist in a collection of behaviors and are implemented as control laws to achieve and/or maintain goals [10]. In our approach, behaviors can be described as a set of rules for the port arbitrators. In Figure 2, Face Detector sends the position of the detected face to Gaze Control to follow it. In other words, to implement a behavior called Follow Face, the connection /Face/pos:o to /Gaze/pos:i should exist and be selected by the port arbitrator in /Gaze/pos:i. To imple- ment another behavior we call Look Around, the connection {/RandomLook/pos:o, /Gaze/pos:i} should be se- lected by the port arbitrator in /Gaze/pos:i to deliver the random position data generated by Random Look to the Gaze Control. Desired behaviors, therefore, can be implemented by selecting connections which are required to deliver data among specific components. At the behavioral description level, we concentrate only on the connections among the components and the necessary rules that select these con- nections in the arbitrators. The rules can be provided by Condition: Search and Track Inhibition: Condition: Configuration: /RsetArm/pos:o -> /Arm/pos:i Rest Arm Inhibition: Condition: Be Curious Inhibition: Condition: Configuration: /RandomLook/pos:0 -> /Gaze/pos:i Look Around Inhibition: Condition: Configuration: /Face/pos:o -> /Gaze/pos:i Follow Face Inhibition: Look Around Condition: Configuration: /Object/pos:o -> /Gaze/pos:i /Object/pos:o -> /Arm/pos:i Track Object Inhibition: Be Curious Rest Arm /collision:o ¬ Fig. 3. An example of the behavioral model that uses the components from Figure 2 to implement the behavior Search and Track. This behavior allows the robot to look around in search for a face or an object. When the robot detects a face it tracks it with the gaze. When it detects an objects it follows it with the gaze and reaches for it. The overall behavior is implemented by coordinating simpler behaviors. Correct coordination is implemented by inhibitions among behaviors (red arrows). E.g., the red arrow from Follow Face to Look around gives higher priority to the first behavior whose activation inhibits the second. See also the description in the text. specifying configuration of the connections required for im- plementing the behavior, under which condition the behavior can be activated and the list of behaviors it should inhibit. In Figure 2 we have shown an example of the composition of some components and their connections. Based on them, in Figure 3 we depict an example of behavioral description for a task in which the robot searches around, follows human faces and tracks an object with the hand. In the figure, Follow Face, Look Around, Rest Arm and Track Object represent behaviors. Follow Face and Look Around are grouped together to describe, by composition, another (meta behavior) behavior called Be Curious. Be Curious implements a behavior that let the robot randomly look around or follow a human face if a person appears in the scene. The red arrow from Follow Face to Look around gives higher priority to the first behavior whose activation inhibits the second. Rest Arm describes a behavior that keeps the arm in the resting position. Track Object implements tracking of an object with the gaze and reaching for it with the hand. It also inhibits Rest Arm and the meta behavior Be Curious to prevent them from interfering during tracking. The condition “¬ /collision” in Track Object implies that robot can track an object only in absence of collisions. Track Object, Rest Arm and Be curious are further grouped to describe another meta behavior we called Search and Track. A. Behavior Specification A behavior (or a meta behavior) has the following prop- erties: Configuration of a behavior is the list of connections which should be selected by the port arbitrators to implement the behavior. For meta-behaviors, configuration is as a list of behaviors or other meta-behaviors. For example, in Figure 3, the configuration property of Track Object implies that to follow an object with the head, /Object/pos:o should feed data to Gaze Control at /Gaze/pos:i. Tracking with the hand is achieved by sending /Object/pos:o to Arm Control at /Arm/pos:i. Notice that here we focus only on the connections which define the behavior of the system, but other connections are required for proper functioning of some modules (e.g. Face Detector and Object Detector require connections from the robot cameras) For simplicity we do not consider these connections here. Condition is an optional property which specifies in, first– order logic, a constraint that should be verified for the behavior to be activated. The condition ¬ /collision : o of Track Object requires that all the connections specified in its Configuration should be selected only if the port /collision:o is inactive (i.e. it is not sending messages). In a meta-behavior the Condition affects all its child be- haviors, i.e. conditions from all parent meta-behaviors in a hierarchy are conjuncted and inherited by all child behaviors. Inhibition, specifies inhibitions between behaviors or meta-behaviors. Specifying inhibitions allows coordinating behaviors that are competing for the same resources. In Figure 3 we define the behavior Look Around which is implemented by connecting ports of Random Look to Gaze Control. We also define Follow Face and Track Object. These behaviors compete to control the gaze of the robot by sending commands to Gaze Control at /Gaze/pos:i. Conflicts are avoided by further specifying the overall behavior of the robot and assigning inhibitions. In Figure 3 Follow Face in- hibits Look Around. In more details this tells the arbitrator in /Gaze/pos:i that connection {/RandomLook/pos:o, /Gaze/pos:i} should not be selected when connec- tion {/Face/pos:o, /Gaze/pos:i} is active, because Face Detector is sending data to Gaze Control. A behavior can also inhibits a meta-behavior. In this case, the behavior inhibits all the behaviors in the meta-behavior. In Figure 3, Track Object inhibits Be Curious, i.e. it inhibits Follow Face and Look around. In practice this corresponds to assigning decreasing priorities to Track Object, Follow Face and Look around to avoid conflicts in Gaze Control. Similar rules are applied if a meta-behavior inhibits another behavior or another meta-behavior. For the sake of modularity and reusability, behaviors can only inhibit other behaviors within the same meta-behavior. For example in Figure 3, Follow Face is not allowed to inhibit Rest Arm. V. EXTRACTING RULES FROM BEHAVIORAL MODEL In the previous section we have described how behaviors are modeled using connections between ports. In this section we explain how the necessary rules for the arbitrators are extracted from the behavioral model. Every behavior has a list of connections specified by its Configuration property. The properties Conditions and Inhibition determine an extra set of constraints that are applied to the port arbitrators of its connections. For example in Figure 3, Look Around is inhibited by Follow Face. Both behaviors have a connection to /Gaze/pos:i; Thus the following rule is added to the port arbitrator in /Gaze/pos:i: /RandomLook/pos : o ∧¬/Face/pos : o ⇒Select(/RandomLook/pos : o) Look Around is also inhibited by Track Object (through the inhibition to Be Curious). Therefore the previous rule is updated with “¬/Object/pos : o” to reflect the new constraint: /RandomLook/pos : o ∧¬/Face/pos : o ∧¬/Object/pos : o ⇒Select(/RandomLook/pos : o) A behavior’s Condition and the conditions that are inher- ited from the parent groups are also added in the same way to the port arbitrators of all of the connections specified in Con- figuration. For example, the constraint “¬/collision : o” is added to the rules for /Object/pos:o in the arbitrators at /Gaze/pos:i and /Arm/pos:i. To summarize, the algorithm to extract the arbitration rules from the behavior model is easily done in two steps for each behavior i in the model. First: the Condition of i is updated with all the conditions it inherits from the parent meta-behaviors. This condition is added as an extra constraint to the port arbitrators of all the connections specified in Configuration. Second: further conditions are extracted from all inhibitors of i and added to the rules of the corresponding port arbitrators. /Gaze/pos:i Look Around Follow Face /RandomLook/pos:o /Face/pos:o Look Around Listing 1. The representation of "Be Curious" behavior in XML format. The behavioral model can be represented using Extensible Markup Language (XML). Listings 1 illustrates the repre- sentation of Be Curious behavior in the XML format. The model are used by a third–party tool from the YARP [11] robotic framework to extract the arbitration rules and up- date the configuration of connections. The concept is also illustrated in Figure 1. YARP offers a method to describe the configuration of components and their connections in XML format which known as application description file. In short, an application description file contains all the required modules, their configuration (i.e., parameters), the way they are interconnected, and the necessary information for their deployment. The required information for orchestration (co- ordination) of these modules can be represented in another XML file (similar to Listing 1) using our behavioral model based on port–arbitrated mechanism. The final application, can be generated by extracting the arbitration rules from the behavioral description file and using them to configure the component connections from the application description file. Therefore, based on different behavioral model, the same components the their configuration can be used for the development of different robotic applications. VI. DISCUSSIONS Different behavior selection mechanisms are compared in [12], [13] and [14]. An alternative approach to compet- itive action selection is a cooperative mechanism in which recommendations from multiple behaviors are combined to form a control action that represents their consensus. An example of this type of mechanism is DAMN [15]. It uses a centralized arbitrator to fuse the collected commands from different behaviors and select the action which best satisfies the prioritized goals of the system. Nowadays, due to heterogeneity of data types and the complexity of the control systems, the proposed methodology is practically limited to low-level control. However, in our approach, behaviors can also help each other to become active by enabling the relevant connections. This has already been discussed in detail in [2]. Even though coordination based on port arbitration can cover a wide variety of robotic applications, we have experi- enced certain limitations in the system. First, since arbitration is usually done on the data from unidirectional connection to an input port, it cannot be easily used in a service–oriented system where interactions between modules are bidirectional and done using blocking remote procedure calls. Moreover, a robotic task might require performing a sequence of actions synchronized with the internal state of components. This can be also made using port arbitration, nevertheless, delegating this responsibility to a dedicated, external component can be preferable in favor of simplicity and performance. VII. CONCLUSION This work–in–progress article has introduced a mechanism based on port arbitration for modeling and coordination of robotic behaviors. We have shown how robotics tasks can be represented using our behavioral description model and coordinated in a distributed component–based framework using port–arbitrated mechanism and without using any central coordinator. Remarkably, We demonstrated that in our framework, based on different behavioral descriptions, sev- eral robotic applications can be hierarchically implemented using the same reusable software components. Future works will involve investigating proper methods for the validation of the model and checking the consistency of the rules which are extracted from the behavioral description during the development of the tasks. ACKNOWLEDGMENTS The research leading to these results has received fund- ing from the European Community’s Seventh Framework Programme (FP7 ICT) under grant agreement No. 270273 (Xperience) and 611832 (WALK-MAN). REFERENCES [1] D. Brugali and P. Scandurra, “Component-based Robotic Engineering Part I : Reusable building blocks,” vol. XX, no. 4, pp. 1–12, 2009. [2] A. Paikan, G. Metta, and L. Natale, “A port-arbitrated mechanism for behavior selection in humanoid robotics,” 16th International Confer- ence on Advanced Robotics (ICAR), 2013, pp. 1–7, 2013. [3] D. Gelernter and N. Carriero, “Coordination languages and their significance,” Communications of the ACM, vol. 35, no. 2, p. 96, 1992. [4] D. Gelernter, “Generative communication in Linda,” ACM Transac- tions on Programming Languages and Systems (TOPLAS), vol. 7, no. 1, pp. 80–112, 1985. [5] F. Arbab, “What do you mean, coordination,” Bulletin of the Dutch Association for Theoretical Computer Science (NVTI), vol. 19, 1998. [6] ——, “Reo: a channel-based coordination model for component com- position,” Mathematical Structures in Computer Science, vol. 14, no. 3, pp. 329–366, 2004. [7] A. Paikan, P. Fitzpatrick, G. Metta, and L. Natale, “Data Flow Port Monitoring and Arbitration,” Software Engineering for Robotics, vol. 5, no. 1, pp. 80–88, 2014. [8] A. Paikan, V. Tikhanoff, G. Metta, and L. Natale, “Enhancing software module reusability using port plug-ins: an experiment with the icub robot,” International Conference on Intelligent Robots and Systems (IROS), 2014. [9] R. E. Bryant, “Graph-based algorithms for boolean function manip- ulation,” Computers, IEEE Transactions on, vol. 100, pp. 677–691, 1986. [10] M. J. Mataric, “Behavior-based robotics,” Tech. Rep. [11] G. Metta, P. Fitzpatrick, and L. Natale, “YARP: Yet Another Robot Platform,” International Journal on Advanced Robotics Systems, vol. 3, no. 1, pp. 43–48, 2006. [12] P. Pirjanian, “Behavior coordination mechanisms-state-of-the-art,” In- stitute for Robotics and Intelligent Systems, School of Engineering, University of Southern California, Tech. Rep. IRIS-99-375, 1999. [13] J. M. MacKenzie, Douglas C and Arkin, Ronald C and Cameron, “Multiagent mission specification and execution,” Robot colonies, pp. 29–52, 1997. [14] M. Scheutz and V. Andronache, “Architectural mechanisms for dy- namic changes of behavior selection strategies in behavior-based systems,” Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, vol. 34, no. 6, pp. 2377–2395, 2004. [15] J. K. Rosenblatt, “DAMN: a distributed architecture for mobile navi- gation,” Journal of Experimental & Theoretical Artificial Intelligence, vol. 9, no. 2-3, pp. 339–360, Apr. 1997.