Coordinating Autonomous Planning Agents
with Temporal Constraints
Pieter Buzing
a∗J. Renze Steenhuisen
aCees Witteveen
a aFaculty of Electrical Engineering, Mathematics and Computer Science,
Delft University of Technology, P.O. Box 5031, 2600 GA Delft, The Netherlands,
email:
{J.R.Steenhuisen, C.Witteveen, P.C.Buzing}@tudelft.nl
Abstract
We address the problem of coordinating autonomous agents that have to come up with a joint plan and schedule for a set of tasks. These tasks are interdependent and temporally constrained and are distributed over the agents. We start with investigating the scheduling part of the problem: How to achieve, given the constraints, a joint solution to the distributed scheduling problem where each of the agents is allowed to determine its own schedule for the set of tasks assigned to it. Here, the problem is to ensure that whatever individual schedule each agent comes up with, these schedules always can be combined into a global schedule satisfying all constraints. Next, we investigate a more complex problem where, before scheduling, the agents are allowed to make their own plan to achieve the tasks assigned to them. That is, they are allowed to choose autonomously some additional precedence constraints to their assigned tasks. We have to provide the necessary coordination mechanisms to ensure that no conflicts will arise between the resulting plans. The solution we provide is to reduce the temporal constraints of the individual tasks to a set of precedence constraints between tasks and virtual tasks and we solve the resulting coordination problem for tasks with precedence constraints. The solution thus obtained enables us to solve the plan coordination problem with temporal constraints.
1
Introduction
Typically, multi-agent planning and scheduling problems require a set of autonomous planning agents to come up with a joint plan and schedule for achieving a set of tasks. Usually, none of the participating agents is capable of solving all tasks by itself. Therefore, each agent is assigned a partition of the tasks to carry out. To complete its part of the job, each agent has to come up with a plan and schedule to carry out these tasks. Moreover, it is required that these individual plans and schedules are compatible in the sense that together they constitute a feasible plan/schedule for the total set of tasks. In particular, if we assume that the agents involved are self-interested and require to determine their plans and schedules autonomously, we have to ensure that coordination mechanisms are provided to ensure the compatibility of plans and schedules.
With respect to task coordination, it is common to classify sets of tasks with respect to the level of coordination they require [5]. Sometimes, the set of tasks to perform is decomposable into a set of independent tasks that can be assigned to agents. Each of the agents is able to construct an independent plan for its subset of tasks and the coordination problem is trivially solved. Typical tasks that can be solved in this way are reconnaissance tasks and simple pick-up and delivery tasks. Note that here coordination, if necessary at all, is only needed in solving task allocation problems (who will do which tasks). Such tasks can be solved by loosely-coordinated teams of agents.
Other tasks, however, not only require coordination during task allocation, but also during planning. For example, if there are constraints between tasks assigned to different agents, those agents have to coordinate during their planning in order to construct a plan that satisfies these
∗This research is supported by the Technology Foundation STW, applied science division of NWO and the
technology programme of the Ministry of Economic Affairs.
constraints. In order to solve such tasks we need moderately-coordinated agents. Typical problems in this category are monitoring tasks and multi-modal transportation tasks.
Finally, there is a set of tasks that require coordination also in the execution phase of the plan. Such tasks are said to be tightly coupled and require a tightly-coordinated team of agents (e.g., platooning vehicles or moving in formation). Usually, such tightly-coupled sets of tasks contain temporal constraints and simultaneity constraints.
In previous work on coordination [1], we have concentrated on solving multi-agent planning problems where
• agents are self-interested and non-cooperative,
• while the tasks to solve require moderately-coordinated agents, and • no temporal constraints were assigned to the tasks.
Basically, our approach to solve such problems comes down to enabling loosely-coordinated agents to solve tasks that require moderately-coordinated agents by providing adequate coordination mech-anisms that respect the autonomy of the participating agents. For example, we showed how to solve multi-modal transportation systems that require a joint plan for a set of interrelated tasks by providing the participating agents (companies) with a coordination mechanism that enables them to solve their part of the planning problem independently from the others. The coordination mechanism ensures that every plan (solution) an individual agent is providing can seamlessly be integrated in the joint plan to be constructed. This research approach can be summarised by the equation
moderately-coupled agents = loosely-coupled agents + coordination mechanisms.
Our future research will aim at solving multi-agent planning problems that require temporal (semi-) tight coordination between agents to solve them. Therefore, we want to extend our current work on coordination to interdependent tasks with temporal constraints. The introduction of time con-straints forces us to take into account coordination of scheduling besides plan coordination, and enables us to prepare for the next step in coordination research (i.e., the development of coordina-tion mechanisms for tightly-coupled tasks).
First of all, we will concentrate on what we consider a basic coordination problem raised by the addition of temporal constraints on tasks: How to ensure that a distributed scheduling pro-cess executed by a set of autonomous agents will result in a joint schedule that satisfies both the given precedence constraints and the temporal constraints. The basic idea here is to remove all dependency constraints between tasks belonging to different agents by translating them into suit-able disjoint temporal constraints. As a result, we are suit-able to decompose the total problem into independent scheduling problems, one problem for each agent. The agents can determine their own schedule times for the tasks assigned to them individually and are guaranteed that every such schedule can be seamlessly integrated into an overall schedule.
Next, we show how to integrate both plan coordination and scheduling coordination into one framework by solving the problem how a joint schedule can be found for temporally constrained tasks, even if the agents are able to plan their part of the tasks independently from the others. A nice feature of this integrated approach is that it enables us to reduce coordination problems with temporal constraints to coordination problems without temporal constraints, thereby enabling us to reuse previously-developed methods to solve plan coordination methods.
2
Coordination for Distributed Scheduling
We start with investigating a simple distributed scheduling problem where a number of interdepen-dent tasks with temporal constraints has been assigned to a number of agents. The agents want to decide autonomously when to schedule the tasks assigned to them. Each such a schedule, of course, has to satisfy the local dependency constraints and temporal constraints. The problem is how to ensure that whatever local schedule is chosen, there will be no conflicts between them.
• a set of tasks T = {t1, t2, . . . , tn} to be executed,
• a partially-ordered precedence relation ≺ defined on T , where t ≺ t′ indicates that task t has
to start before t′ can start,
• for each task t ∈ T a temporal interval (time window) I(t) = [lb(t), ub(t)], with lb(t) ≤ ub(t), constraining the starting time of t to a time point between lb(t) ∈ Z and ub(t) ∈ Z,
• a set of agents A = {A1, . . . , Am}, and
• a function g : T → A assigning each task t ∈ T to an agent Ai∈ A. The set g−1(Ai) of tasks
allocated to agent Ai is denoted by Ti.
We will use the following notations to denote special time windows and relations between such time windows:
1. [ ] denotes the empty time window, 2. I(t) < I(t′) holds if ub(t) < lb(t′),
3. I(t) ⊓ I(t′) = [max{lb(t), lb(t′}, min{ub(t), ub(t′)}], and
4. c ∈ I(t) if lb(t) ≤ c ≤ ub(t).
Furthermore, we will call a precedence constraint t ≺ t′
an intra-agent constraint if and only if t, t′∈ T
i for some i. Such a precedence constraint is called an inter-agent constraint if and only if
t∈ Ti and t′ ∈ Tj for i 6= j.
Each of the agents Ai independently chooses a schedule si: Ti→ Z for its set of tasks Ti, that
is each agent Ai determines a starting time si(t) for all the tasks t ∈ Ti such that the following
requirements are satisfied.
1. For every t ∈ Ti, si(t) ∈ I(t), that is the starting time should be allowed by the time interval
of the task. 2. For every t, t′
∈ Ti, if t ≺ t′ then si(t) < si(t′), that is starting times should respect local
precedence constraints.
3. For every t ∈ Ti and t′ ∈ Tj with i 6= j, if t ≺ t′ then si(t) < sj(t′), that is starting times
determined by different agents should respect inter-agent precedence constraints.
The reader should be aware that the requirements above for feasible schedules together with the precedence constraints on the tasks induce some additional constraints on the time windows of the tasks related to each other via a precedence constraint.
The following rule has to be applied here: If t ≺ t′
then lb(t′
) := max(lb(t′
), lb(t) + ǫ) and ub(t) := min(ub(t), ub(t′
) − ǫ), where ǫ is the minimal separation time between tasks.
Without loss of generality, we assume ǫ = 1.
To give a simple example, suppose task t has to be scheduled in the interval I(t) = [10, 70] and task t′has to start in the interval I(t′) = [5, 50] and we have the precedence constraint t ≺ t′. This
immediately implies that t′ cannot start earlier than max(5, 10 + 1) = 11 and t cannot start after
min(70, 50 − 1) = 49.
By applying this rule to all pairs of tasks we can obtain a normalised instance. In the sequel, we assume, without loss of generality, that the given instances are normalised and have at least one solution (i.e., for all t we have ub(t) ≥ lb(t) and for all t, t′, t ≺ t′ implies that lb(t) + 1 ≤ lb(t′) and
ub(t) + 1 ≤ ub(t′
)).1
Obviously, because of requirement 3, this distributed scheduling problem requires coordination between the agents. Assuming that all the agents Ai are self-interested and desire autonomy in
de-termining their scheduling function si, we could try to solve this scheduling problem by establishing
a coordination mechanism that guarantees the existence of a global solution satisfying requirements 1-3, whenever each agent has established a local solution satisfying requirements 1 and 2.
1Note that here, we only consider absolute time windows for tasks. The reader might note that a simplified
One such coordination mechanism can be easily found. The basic idea is to get rid of all inter-agent precedence constraints t ≺ t′where t ∈ T
iand t′∈ Tjbelong to the tasks of different agents, by
separating the corresponding time windows: Let I(t) = [lb(t), ub(t)] and I(t′) = [lb(t′), ub(t′)]. We
have to find two time points δ1and δ2such that lb(t) ≤ δ1≤ ub(t), lb(t′) ≤ δ2≤ ub(t′) and δ1< δ2.
Then the intervals are changed to I(t) = [lb(t), δ1] and I(t′) = [δ2, ub(t′)], thereby removing the
possible overlap. This interval separation encodes requirement 3 indirectly by enforcing a smaller value for si(t) than for sj(t′) for all locally feasible schedules si() and sj().
Remark In general, the problem to remove all dependencies between coupled systems with tem-poral constraints is called the Temtem-poral Decoupling Problem and is extensively discussed in Luke Hunsberger’s PhD thesis [4].
The following algorithm determines a set of separated intervals for the tasks connected via inter-agent precedence constraints such that coordination of local schedules is ensured. The algorithm starts with the minimal tasks t in the precedence order that have a successor, separates their time window from the time windows of the tasks dependent on t and then moves upwards in the partial order.
Algorithm 1Algorithm to separate overlapping intervals.
Require: a normalised instance (T, ≺, I, A, g) of a distributed scheduling problem; Ensure: a set of disjoint intervals for tasks t ∈ T connected via precedence relations;
1: remove all local (i.e., intra-agent) precedence constraints from ≺;
2: whilethere exist tasks t, t′∈ T such that t ≺ t′ and I(t) ⊓ I(t′) 6= [ ] do 3: find a task t ∈ min≺{T } such that there exists a task t′ with t ≺ t′; 4: let I(t) = [l1, u1]; 5: let u′ 1:= min{lb(t ′ ) | t ≺ t′ } -1; 6: I(t) := [l1, u′1]; 7: T := T − {t}; 8: end while
Example There are two agents A1 and A2, each having two tasks to perform. There are two
inter-agent precedence constraints between the tasks and each task has a time window for starting times (see Figure 1a). If each of the agents is allowed to determine a feasible schedule for the tasks it has to achieve, we might end up with an infeasible total schedule. For example, the schedules s1(t1) = 43, s1(t3) = 40, s2(t2) = 43, s2(t4) = 40 are locally feasible but do not satisfy the
precedence constraints between the agents. In Figure 1b, Algorithm 1 has been applied to separate the time windows. It is not difficult to see that now every locally feasible schedule also constitutes a globally feasible schedule.
It is easy to see that this algorithm terminates, and that all the temporal intervals are disjoint after termination. Moreover, if the original instance has a solution, the transformed instance will have a solution, too. Note that this algorithm is a centralised one. However, it can easily be transformed into a distributed algorithm by using a blackboard for maintaining the inter-agent dependency constraints.
Remark Besides the algorithm given above, other heuristics might be used to separate time windows as well. Hunsberger [4] provides a number of heuristics to find good values for δ1 and δ2. Note
that the order in which the inter-agent constraints are tackled will influence the result. More sophisticated algorithms will most likely be based on domain specific objectives, like “prefer earliest possible starting times”, “maximise flexibility for the least flexible task”, or “the number of adjusted time intervals of each agent should be equal”. Here are some other ideas for further improvements: 1. Reducing a time window can have cascading effects. This means that decoupling an inter-agent relation can overconstrain other constraints. To avoid this we can apply a minimal change strategy: Take an inter-agent constraint t ≺ t′ and adjust the lower bound of t′or the
[29,50] [19,45] [25,55] [11,55] t1 t2 t3 t4 [29,50] [19,28] [25,55] [11,24] t1 t2 t3 t4 A1 A2 A1 A2
a
b
Figure 1: Separating time windows to coordinate distributed scheduling.
2. Assuming that the problem instance is normalised, for each of the inter-agent constraints we determine the overlap: overlap(t, t′) = ub(t) − lb(t′). This overlap is a measure for the amount
of temporal space the agents together have to give in. If we start the separation process with the smallest overlapping inter-agent constraint and then propagate the effects, we only remove a single constraint, enforce minimal restriction of the time intervals, and possibly make other inter-agent constraints superfluous.
It is now easy to see that whenever there exists a global scheduling s : T → Z satisfying all the constraints, there also exists a coordination mechanism guaranteeing that a set of local solutions sican be found that constitutes a distributed solution to the coordinated scheduling problem. The
following result is an easy consequence of the preceding discussion.
Proposition 1 Every instance (T, ≺, I, A, g) of a multi-agent scheduling coordination problem that is normalised and, for every t, t′
such that t ≺ t′
and g(t) 6= g(t′
), satisfies I(t) ⊓ I(t′
) = [ ] has the property that every set of local schedules {si}Ai∈A satisfying requirement 1 and 2 also satisfies
requirement 3.
3
Coordinating Planning and Scheduling
The basic coordinated scheduling approach does not recognise the possibility that some agents would like to impose their own ordering on starting the tasks they have been assigned to, before they wish to coordinate their scheduling activities. For example, suppose an agent Ai wants to
execute its tasks in such a way that it can optimally use its resources. Now, Ai acknowledges
that in its subset of tasks in principle there is no objection to start t and t′
concurrently (since no constraint forces it to do otherwise), but due to resource constraints known to agent Ai, the agent
decides to start t before t′
. So it wishes to impose an additional precedence t ≺ t′
on its own set of tasks Ti. The problem is that if we allow agents to make such decisions autonomously, there is a
possibility that they never will be able to find a coordinated schedule for the total set of tasks. To see this, consider the following example.
Example There are two agents A1 and A2, each having to complete two temporally constrained
[20,50] [19,45] [19,55] [18,55] t1 t2 t3 t4 [20,50] [20,45] [19,55] [21,55] t1 t2 t3 t4 a b
Figure 2: Autonomous planning of agents might cause failing of coordinated scheduling. In general, we can state the autonomous planning+scheduling problem as follows: Given a coordination problem (T, ≺, I, A, g) with temporal intervals, how to ensure that a feasible joint schedule can be found for all the tasks in T if every agent Aiis free to compose its own ordering ≺i
on the tasks Tiassigned to him, while respecting the precedence relation ≺ and the given temporal
intervals?
This problem closely resembles the plan coordination problem we have investigated in [1, 6]. The only difference between the latter problem and our current problem is the incorporation of temporal constraints for tasks. We will first give a brief overview of this plan coordination problem and then we show how our current problem including temporal constraints can be solved by reducing it to the plan coordination problem.
The distributed plan coordination problem can be stated as follows: We have a set of tasks T together with a precedence relation ≺. There is also a set of agents A = {A1, A2, . . . , Am}, each
agent Ai having to achieve a partition Ti. We assume that each agent is an autonomous planner
and is able to find a suitable plan (Ti,≺i) where each ≺i is an ordering respecting the given
precedence relation ≺. We say that there exists a feasible joint plan of the agents if the structure (T, (≺ ∪ ≺1∪ · · · ∪ ≺m)+) is partially ordered (i.e., if the individual orderings ≺i together with ≺
do not induce a cycle). Here, if ρ is any relation, ρ+ denotes its transitive closure.
In [1], we have investigated this problem and we have shown that it can be solved by adding a minimal set of precedence constraints, called a coordination set, ∆ =Smi=1∆i to the precedence
relation ≺ in such a way that it ensures that every locally determined partial order ≺i respecting
≺ ∪ ∆i can be used to constitute a global partial order (≺ ∪ ∆ ∪ ≺1∪ · · · ∪ ≺m)+ that describes
a joint plan to execute the tasks in T .
Example As an example, consider Figure 2 and neglect the temporal intervals for the tasks. If agent A1 and A2 are free to plan autonomously, they might come up with a jointly infeasible
plan. This happens if agent A1 adds t3 ≺1 t1 and agent A2 adds t2 ≺2 t4, thereby creating the
cycle t1− t2− t4− t3− t1 (see Figure 2b). A solution is to add for example a set of constraints
∆ = {t1≺ t3}. This constraint prevents inter-agent cycles to occur.
The idea is to solve our planning+scheduling problem by first reducing an instance with temporal constraints to a plan coordination instance without temporal constraints. Thereafter, we solve the plan coordination problem by identifying a set of suitable constraints and add these constraints to the given instance. Finally, the agents are allowed to plan autonomously and after planning we solve the resulting scheduling problem by our distributed scheduling algorithm. Summarising, the procedure is as follows:
• Determine a solution of the plan coordination problem, that is find a coordination set ∆. • Let the agents plan autonomously, that is determine their orderings ≺i respecting ≺ ∪ ∆.
• Solve the resulting scheduling problem (T, ≺1∪ ≺2∪ · · · ∪ ≺mI, A, g) with temporal intervals
by applying the basic coordination algorithm.
Here, it suffices to fully describe step 1: The reduction of the coordination problem with temporal constraints into a coordination problem without temporal constraints. The idea is the following: Given an instance (T, ≺, I, A, g), we first collect all the begin and end points of the intervals I(t). More precisely, for each task t, the time interval I(t) = [lb(t), ub(t)] is coded into two values s and s′, where s = lb(t) − 1 and s′ = ub(t) + 1. We collect the total set of all these values in the set
S = {s1, s2, . . . , sp} and, without loss of generality, we may assume that s1< s2<· · · < sp. Now,
we construct the following coordination instance (T′
,≺′
, A′
, g′), where
1. the set of tasks T′ is extended with the set S: T′= T ∪ S.
2. the precedence relation ≺′equals ≺ extended with the total ordering imposed on S. Moreover,
for each task t, associated with time points s and s′, two additional precedence constraints
s ≺′ t and t ≺′ s′ are constructed. The result then is ≺′=≺ ∪ {(s
i, sj) | 1 ≤ i < j ≤ p}
∪ {(s, t), (t, s′) | t ∈ T, s = lb(t) − 1, s′= ub(t) + 1}.
3. we add a special agent AS taking care of the set of virtual tasks S: A′ = A ∪ {AS}.
4. g′
is g extended with g(s) = AS for every s ∈ S.
Example Given Figure 2, the result of the reduction to the plan coordination instance is depicted in Figure 3a. By adding the constraint t4≺ t2it is impossible to create an inter-agent cycle. After
normalisation, the resulting schedule coordination instance is given in Figure 3b.
t1 t2 t3 t4 18 17 46 51 56 19 [20,50] [19,45] [19,55] [18,49] t1 t2 t3 t4 a b AS A1 A2 A1 A2
Figure 3: Reducing a temporal coordination problem to a plan coordination problem.
4
Concluding Remarks
In previous work, we addressed the distributed plan coordination problem. This problem amounts to finding a minimal set of additional constraints that allow a number of agents (planners) to autonomously plan their share of tasks, while guaranteeing a conflict-free global plan. However, this approach only considered the coordination stage and the planning stage: No attention was given to the scheduling phase (i.e., the actual assignment of execution times).
This paper extends our coordination formalism with a mechanism for temporal constraints. Information about the execution time windows of tasks can now be taken into account. We provide a basic coordination algorithm to fully decouple distributed plans. This means that each agent has complete freedom to schedule its tasks at times consistent with his local constraints; The resulting joint schedule will always be valid.
Since removing all inter-agent constraints (and thus overconstraining the agents’ plans) is often not necessary in practise, we also describe a more sophisticated approach. In short, our method comes down to these steps:
1. Reduce the coordination problem with temporal constraints to a coordination problem with-out temporal constraints.
2. Determine a solution of the plan coordination problem (i.e., find a set of additional constraints that make the union of agent plans conflict free).
3. Let the agents plan autonomously, that is determine their local orderings of actions.
4. Solve the resulting scheduling problem with temporal intervals by applying the basic coordi-nation algorithm.
4.1
Future Work
The framework that we presented here only handles absolute time windows for tasks. In the near future, we will also incorporate relative constraints, making it compliant with the STP theory. This means that durative tasks can be modelled. Also, the relative distance between tasks can be made variable, instead of the constant value of 1.
On top of this, we will progress to the area of tightly-coupled multi-agent plans. More specifi-cally, we will provide methods to encode relations like simultaneity (i.e., when tasks t and t′
have to occur at the same time) in our framework.
References
[1] Pieter Buzing, Adriaan ter Mors, Jeroen Valk, and Cees Witteveen. Coordinating self-interested planning agents. Autonomous Agents and Multi-Agent Systems, 12(2):199–218, March 2006. [2] Rina Dechter. Constraint Processing. Morgan Kaufmann, San Francisco, CA, USA, 2003. [3] Rina Dechter, Itay Meiri, and Judea Pearl. Temporal constraint networks. Artificial Intelligence,
49:61–95, 1991.
[4] Luke Hunsberger. Group Decision Making and Temporal Reasoning. PhD thesis, Harvard University, Cambridge, MA, USA, June 2002.
[5] Nidhi Kalra, Anthony Stentz, and David Ferguson. Hoplites: A market framework for com-plex tight coordination in multi-agent teams. Technical Report CMU-RI-TR-04-41, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA, August 2004.
