Gianluca Baldassarre, Vito Trianni and Stefano Nolfi

1. Objectives & summary of the obtained results

The original goal of this research is to study how a group of robots that can physically assemble together can develop coordinated and cooperative behaviour that allow them to solve problem that a single individual robot would be unable to solve.

The methodologies developed in the attempt to purse such a goal and the analysis of the obtained results, however, lead to a series of results that were unexpected at the beginning of the project.

One of this results was the realization that embodiment, in agents that are physically assembled toghether, represent an extremely powerful source which can be exploited to exhibit coordinated behavior. One effective way to do that is to allow each agent to sense the direction and the intensity of the force exherted on its body by the entire group.

Another results was the realization that artificial evolution combined with a selection criterion (fitness) that evaluate the behaviour exhibited by the group is an extremely powerful method for developing cooperative behaviours in embodied agents.

A third results was the realization that these robots display remarkable generalization skills as well as an ability to spontaneously produce new effective behaviour in new environmental conditions never experienced above. The analysis of the causes of such remarkable characteristics elucidated the fact that the behaviour exhibited by a robot or by a collection of robot is a multi-level and multi-scale phenomena in which behavioural processes, at a certain level of organization, interact between themselves spontaneously leading to the production of higher level behavioural processes (see below).

2. The robots

The pictures below show the hardware of one of the 35 robots that were manifactured within the Swarm-bot european project (Mondada et al., 2004). Each robot (s-bot) consists of a turret and a chassis that can actively rotate with respect to each other. The bottom part (the chassis) includes the tracks and the teethed wheels, and four infrared sensors. The top part (the turret) includes the rigid gripper, one omni-directional camera, four microphones, two speakers, 16 infrared sensors, and a 3-axes accelerometer. A traction sensor placed between the turret and the chassis detect the direction and the intensity of the traction force that the turret exerts on the chassis.

.

Two views of an s-bots. The robots are also provided with several additional sensors which are not used in the experiments reported in this document.

A swarm-bot consisting of several connected s-bots should move as a whole and reconfigure its shape when needed. For example, it might have to adopt a different shape in order to go through a narrow passage or overcome an hole that a single robot will be unable to pass. The behaviour of the swarm-bot should result from self-organizing principles emerging from simple rules followed by each individual s-bot rather than via a global template.

3. Evolving coordinated motion

In a series of experiments we studied how a swarm-bot consisting of several physically assembled s-bots can display coherent behavior which allow it to move in an environment consisting of an open area. In particular we considered the case of a swarm-bot formed by four s-bots assembled into a linear structure (see figure).

Since the orientation of each individual chassis might vary, s-bots should first negotiate a common direction and then move along such direction by later compensating further disalignments occurring during motion.

The behavior exhibited by four evolved individuals assembled into a linear structure in hardware.

The behavior exhibited by four evolved individuals assembled into a linear structure in hardware during few trials. The white line indicate the direction and the intensity of the traction perceived by each robot.

After some initial explorations, we realized that an important pre-requisite for solving this problem consist in providing each robot with the possibility of feeling whether and how the other robots are pulling or pushing it. This was realized by providing each robot with a a traction sensor placed at the junction between the chassis and the turret that returns the direction (with respect to the chassis' orientation) and the intensity of the force of traction that the turret exerts on the chassis (Baldassarre, Nolfi and Parisi, 2003). This sensor, in fact, allow the robots to exploit the fact that the forces exherted by each individual robot are physically integrated at the levels of their attached bodies. From the point of view of each individual robot, therefore, the traction sensor provides a compact information on what all the robots forming the swarm are doing, on the average.

We then provided each individual s-bot with a simple neural controller including four sensory neurons (plus a bias unit) directly connected to two motor neurons (see Figure). The sensory neurons encode the intensity of traction along four directions (front, rear, left and right). The bias neuron is always activated with 1. The activation state of the two motor neurons is used to set the desired speed of the two motors controlling the wheels and the turret-chassis motor. The difference between the two desired speed is also used to control the turret-chassis degree of freedom in order to counter-balance the rotation produced by the wheels\92 motion. In this way, the turret-chassis motor helps the two motors controlling the wheels to turn the chassis especially in those situations in which one or both the wheels partially or totally loose contact with the ground.

The architecture of the s-bots' neural controllers

The connection weights of the neural controllers have been evolved for the ability to produce coordinated movemenent. Each genotype encodes each connection weights of a single neural controller which is duplicated four times and embodied into the four s-bots (i.e., the s-bots forming a swarm-bot have homogenous controllers). The performance of the controller is given by the average performance over five different trials. The fitness (performance) of each swarm-bot is calculated by measuring the average distance travelled by the swarm-bot during several trials each lasting 150 time steps (15s). The distance travelled is calculated by measuring the distance between theh barycentre of the swarm-bot at the beginning and at the end of the trial. The robots have been evolved in simulation. The evolved neural controllers have been then loaded on the real robots and tested on a real environment.

By analysing the evolved robots we can recognize an initial phase in which the robots start to move by pulling and pushing the other robot in different directions and turn until the orientation of all robots converge over a certain direction and then a phase in which the swarm-bot keep moving along the direction which emerged from the negotiation while compensating further disalighments which originate during motion. See the videos above.

The analysis of the evolved neural controllers (Baldassarre, Parisi & Nolfi, 2004) indicates that this behavior is the result of two simple individual tendencies: a conformistic tendency (i.e. is the tendency to follow the direction of the traction that provides an indication of the average direction toward which the other robot are moving) and a stubborn tendency (i.e. the tendency to persevere on the current direction in certain conditions in order to push the other s-bots toward such direction).

As can be seen from the video below, the same neural controllers evolved in the condition described above display an ability to coordinate and display a coordinated-movement behaviour independently from the number of robots assembled together, the way in which they are assembled, and from the fact that the links between robot are rigid or not. The robots also display an ability to coordinate and navigate on irregular terrain. Finally, the same robot assembled to an external object display a coordinated object pulling-pushing behaviour.

A video showing how the robots evolved in the condition described above display an ability to coordinate and to move together in a variety of situation that differ from that experienced during the rvolutionary process.

4. Evolving coordinated motion and coordinated phototaxis

In another series of experiments (see Baldassarre, Parisi and Nolfi, 2006) we studied a slightly more complex scenario in which an swarm-bot, formed again by four s-bots assembled into a linear structure, was evolved for the ability to display a coordinated motion behaviour (when the environment did not include any light target) and a coordinated phototaxis behavior (when a light target was present).

As in the case of the experiment reported in the previous section each s-bot was provided with a simple neural controller which included 8 sensory neurons directly connected to 2 motor neurons. The first 4 sensory neurons encode the intensity and the traction detected by each robot along four orthogonal directions (front, rear, left, and right). The other 4 sensory neurons encode the intensity of the light detected by 4 light neurons located along the same four orientations (front, rear, left, and right). As in the case of the experiment described in section 3, the two motors neurons encoded the desired speed of the two active wheels of the track and the angular rotation of the turret-chassis motorized joint.

The fitness of the swarm-bot was calculated by summing the average distance travelled by the swarm-bot (i.e. the euclidead distance between the intial and finall position of the barycentre of the swarm-bot) during the trials in which a light target was not present in the environment and the average distance travelled toward the light target (in the trials in which the environment contained a light target). The evolved robots display an ability to coordinate and to display a coordinate motion and a coordinated phototaxis as requested (see the video below). So, as you can see, when the robots perceive the light gradient generated by a light bulb, they converge toward a single direction which correspond to the direction of the light and start moving toward that direction. When a light gradient is not present, the robots converge toward a direction which arise as the result of the negotiation and then they keep moving along such direction.

A video showing rhe behavior display by the evolved robots in simulation in the condition experienced during the evolutionary phase. The first part of the video show a trial in which a light target (the yellow ball) is present. The second part show a trial in which the environmnet do not contains any light target.

In addition to that we observed that the evolved robots display a remarkable generalization ability as well as an ability to produce, in new environmental condition, new type of behaviours that have not been exhibited or rewarded during the evolutionary process. More precisely, as for the case of the experiment reported in the previous section, these evolved robots display an ability to generalize their skill with respect to the number of assembled robots, the shape with which robots are physically assembled, and the type of links (i.e. whether the link are rigid or flexible and thus allow individuals robots to change their relative position with limits). In addition to that, however, when situated in new environmental condition the swrm-bot also display other behavioural skills such us the ability to avoid obstacles and the ability to re-arrange their shape to overcome narrow passages.

Some of these behaviours can be observed in this video in which the same neural controllers evolved in the conditions described above have been embodied into 8 robots assembled in a circular structure and placed in a maze-like environment including walls, cylindrical obstacles, and a light target located on the botoom-left side and visible only locally due to the occlusion of the light gradient caused by the walls.

At the beginning we were puzzled by this result. We were not able to explain why such remarkable generalization ability was observed. By analysis the evolved robots, however, we later come up to the conclusion that it could be explained by considering the multi-level and multi-scale structure of robots\92 behaviour (Nolfi, in press).

Indeed, when we analyse the evolved robots we can see how, first of all, they display a series of simple low-level behaviours which arise by the robots/environmental interactions mediated by simple control rules:

• #1. A move-forward behaviour which consists of the individuals\92 ability to move forward when the robot is coordinated with the rest of the team, is oriented toward the direction of the light gradient (if any), and does not collide with obstacles. This behaviour results from the combination of: (a) a control rule which produces a move forward action when the perceived traction has a low intensity and when difference between the intensity of the light perceived on the left and the right side of the robot is low, and (b) the sensory effects of the execution of the move forward action selected mediated by the external environment (which does not produce a variation of the state of the sensors until the conditions that should be satisfied to produce this behaviour hold).
• #2. A conformistic behaviour which consists of the individuals\92 ability to conform its orientation with that of the rest of the team when the two orientations differ significantly. This behaviour results from the combination of: (a) a control rule that makes the robot turns toward the direction of the traction when its intensity is significant, and (b) the sensory effects produced by the execution of this action mediated by the external environment, that lead to a progressive reduction of the intensity of the traction until the orientation of the robot conforms with the orientation of the rest of the group.
• #3. A phototaxis behaviour which consists of the individuals\92 ability to orient toward the direction of the light target. This behaviour results from the combination of: (a) a control rule that makes the robot turns toward the direction in which the intensity of the light gradient is higher, and (b) the sensory effects produced by the execution of this action mediated by the external environment, that lead to a progressive reduction of the difference in the light intensity detected on the two sides of the robot until the orientation of the robot conforms with the direction of the light gradient.
• #4. An obstacle-avoidance behaviour which consists of the individuals\92 ability to change direction of motion when the execution of a motor action produced a collision with an obstacle. This behaviour results from the combination of: (a) the same control rule that led to behaviour #2 and that makes the robot turn toward the direction of the perceived traction (which in this case is caused by the collision with the obstacle, while in the case of behaviour #2 it is caused by the forces exerted by the other assembled robots), and (b) the sensory effects produced by the execution of the turning action mediated by the external environment, that make the robot turn until collisions no longer prevent the execution of a moving forward behaviour.

The combination and the interaction between these four behaviours produce the following higher-level collective behaviours that extend over a longer time span:

• #5. A coordinated-motion behaviour that consists in the ability of the robots to negotiate a common direction of movement and to keep moving along such direction by compensating further misalignments originating during motion. This behaviour emerges from the combination and the interaction of the conformistic behaviour (which plays the main role when robots are misaligned) and the move-forward behaviour (which plays the main role when robots are aligned).
• #6. A coordinated-light-approaching behaviour which consists in the ability of the robots to co-ordinately move toward a light target. This behaviour emerges from the combination of the conformistic, the move-forward, and the phototaxis behaviours (which is triggered when the robots detect a light gradient). The relative importance of the three control rules which lead to the three corresponding behaviours depends both on the strength of the corresponding triggering condition (i.e. the extent of lack of traction forces, the intensity of traction forces, and the intensity of the light gradient, respectively) and on a priority relation among behaviours (i.e. the fact that the conformistic behaviour tends to play a stronger role than the phototaxis behaviour).
• #7. A coordinated-obstacle-avoidance behaviour which consists in the ability of the robots to co-ordinately turn to avoid nearby obstacles. This behaviour arises as the result of the combination of the obstacle avoidance, the conformistic and the move-forward behaviours.

The combination and the interaction between these behaviours leads in turn to the following higher-level collective behaviours that extend over still longer time spans:

• #8. A collective-exploration-behaviour that consists in the ability of the robots to visit different areas of the environment when the light target cannot be detected. This behaviour emerges from the combination of the coordinated-motion behaviour and the coordinated-obstacle-avoidance behaviour that ensures that the assembled robots can move in the environment without getting stuck and without entering into limit cycle trajectories.
• #9. A shape-re-arrangement behaviour which consists in the ability of the assembled robots to dynamically adapt their shape to the current structure of the environment so to pass through narrow passages especially when the passages to be negotiated are in the direction of the light gradient. This behaviour emerges from the combination and the interaction between coordinated motion and coordinated-light-approaching behaviours mediated by the effects produced by relative differences in motion between robots resulting from the execution of different motor actions and/or from differences in the collisions. The fact that the shape of the assembled robots adapt to the current environmental structure so as to facilitate the overcoming of narrow passages can be explained by considering that collisions produce a modification of the shape that affects the relative positions of the colliding robots, in particular with respect to the axis of the narrow passage.

The combination and the interaction of all these behaviours leads to a still higher-level behaviour:

• #10. A collective-navigation-behaviour which consists in the ability of the assembled robots to navigate toward the light target by producing coordinated movements, exploring the environment, passing through narrow passages, and producing a coordinated-light-approaching behaviour (Figure 8).

This analysis illustrates two important mechanisms that explain the remarkable generalization abilities of these robots. The first mechanism consists in the fact that the control rules that regulate the interaction between the agents and the environment so as to produce certain behavioural skills in certain environmental conditions, will produce different but related behavioural skills in other environmental conditions. In particular, the control rules that generate the behaviours #5 and #6, for which evolving robots have been evolved in an environment without obstacles, also produce behaviour #7 in an environment with obstacles. The second mechanism consists in the fact that the development of certain behaviours at a given level of organization that extend for a given time span will, through their combination and interaction, automatically lead to the exhibition of related higher-level behaviours extending over longer time spans (even if these higher-level behaviours have not been rewarded during the adaptation process). In particular, the combination and the interaction of behaviours #5, #6 and #7 (that have been rewarded during the evolutionary process or that arise from the same control rules that lead to the generation of rewarded behaviours) automatically lead to the production of behaviours #8, #9, and #10 (that have not been rewarded). Obviously, there is no warranty that the new behaviours obtained as a result of these generalization processes will play useful functions. However, the fact that these behaviours are related to the other functional behavioural skills implies that the probabilities that these new behaviours will play useful functions are significant.

In principle, these generalization mechanisms can also be exploited by agents during their adaptive process to generate behavioural skills which play new functionalities and which emerge from the combination and the interaction between pre-existing behavioural skills playing different functions.

 

References

Nolfi S. (in press). Behavior and cognition as a complex adaptive system: Insights from robotic experiments. In C Hooker (Ed.), Philosophy of Complex Systems, Handbook on Foundational/Philosophical Issues for Complex Systems in Science. Elsevier.

Baldassarre, G., Nolfi, S. (2009). Strengths and synergies of evolved and designed controllers: a study within collective robotics. Journal of Artificial Intelligence. Vol. 173, pp. 857-875, pdf

Baldassarre G., Trianni V., Bonani M., Mondada F., Dorigo M. & Nolfi S. (2007). Self-organised coordinated motion in groups of physically connected robots. IEEE Transactions on Systems, Man, and Cybernetics, 37(1):224-239. pdf 

Baldassarre G., Parisi D., Nolfi S. (2006). Distributed Coordination of Simulated Robots Based on Self-Organisation,  Artificial Life. 12(3):289-311. pdf

Baldassarre G., Parisi D., Nolfi S. (2004). Coordination and behavior integration in cooperating simulated robots. S. Schaal, A. Ijspeert, A. Billard, S. Vijayakumar, J. Hallam and J-A. Meyer (Eds.), From Animals to Animats 8: Proceedings of the VIII International Conference on Simulation of Adaptive Behavior, pp. 385-394. MIT Press, Cambridge, MA. pdf

Baldassarre G., Nolfi S., Parisi D. (2003). Evolution of collective behavior in a team of physically linked robots. Applications in Evolutionary Computing, Lecture Notes in Computer Science, Berlin: Springer Verlag, pp. 581-592. pdf

Mondada F., Pettinaro G., Guigrard A., Kwee I., Floreano D., Denebourg J-L, Nolfi S., Gambardella L.M., Dorigo M. (2004). Swarm-bot: A new distributed robotic concept. Autonomous Robots. vol. 17 (2-3), pp. 193-221. pdf