Distributed Control for Coordination of Operations in Large Ports

(1)

FACULTY OF MECHANICAL, MARITIME AND MATERIAL ENGINEERING

Department of Marine and Transport Technology Mekelweg 2

2628 CD Delft

Specialization: Transport Engineering and Logistics Report number: 2012.TEL.7740

Title: Distributed Control for Coordination of Operations in Large Ports Supervisor: Dr. Rudy Negenborn

(2)

(3)

FACULTY OF MECHANICAL, MARITIME AND MATERIAL ENGINEERING

Department of Marine and Transport Technology Mekelweg 2

2628 CD Delft

Student: Herbert Schro¨er Assignment type: Literature Supervisor: Dr. Rudy Negenborn Report number: 2012.TEL.7740 Specialization : TEL Conﬁdential : No

Creditpoints (EC): 10

Subject: Distributed Control For Coordination Of Operations In Large Harbors Large harbors are complex systems with hundreds to thousands of dynamically moving compo-nents. In order to maximize the beneﬁt and performance of harbors, advanced control algorithms could be developed. Such algorithms are aimed at automatically employing equipment (gantry cranes, trucks, quay cranes, AGVs, yard vehicles, etc.) in such a way that, e.g., transfer delays and lay times of ships are minimized, while throughput and proﬁt are maximized. However, a major problem with such control algorithms is the computational time required to determine which actions should be taken. One approach to address this is by using so-called distributed control, in which there is not one large controller operating a harbor, but in which there are multiple smaller controllers (also referred to as agents), that together determine which actions should be taken.

Recently (since the beginning of 2000) approaches for distributed control of large harbors have started to appear in the literature. In this literature study you will search for this literature and analyze it, resulting in an overview/classiﬁcation of what has been done (and why), and in particular at what has not been done, but what is promising to be looked at in the future. Questions that you will address in this literature study are:

• Why could distributed control improve harbor operation?

• How could the large operational control problem of a harbor be divided into smaller op-erational control problems for smaller parts of the harbor?

• How could intelligent agents be employed to solve the smaller operational control problems? • What is the information that such agents use for making decisions?

• And, how do the agents compute which action to take?

• What are the advantages/disadvantages of the approaches proposed in the literature, and do you see ways in which existing approaches could be combines or extended in order to obtain improved approaches?

Based on your literature survey, it is expected that you conclude with a recommendation for future research opportunities and potential for more ideas and/or applications. The report must be written in English and must comply with the guidelines of the section. Details can be found on the website.

For more information, contact Rudy Negenborn (8B-1-05; r.r.negenborn@tudelft.nl). The professor,

(4)

(5)

English summary

Because of the increasing demand of global shipping, many port are pressured to expand. How-ever, due to land constraints that many ports face expanding outwards is not always a feasible option. One possible option is to improve the efficiency and productivity. A good way to im-prove efficiency and productivity is by using optimization. Advanced control algorithms can be used to maximize the performance of port facilities and therefore minimize the throughput times of ships and containers. A possible way to effectively apply these algorithms is by using distributed control. In distributed control, decision making is split over multiple smaller con-trollers who work together in order to reach a common goal. This literature research focusses on distributed control in container terminals, since almost all of the found literature regarding distributed control in large ports addresses these system borders.

Distributed control is a way of dividing large control problems into multiple smaller control problems by the means of multi-agent systems. Within these multi-agent systems, autonomous agents work together to solve problems that are beyond the individual knowledge of each problem solver. Distributed control seems to be an adequate solution for the efficient management of a container terminal because it’s flexible, adaptable to the environment, versatile and robust. The large operational control problem of a container terminal can be divided into smaller oper-ational control problems for smaller parts of the container terminal by the means of subsystems. Different multi-agent system architectures for container terminals have been investigated. An improved architecture, which inhabits all the best properties of the proposed architectures found in literature, has been presented in this research.

The three major subsystems of the container terminal (berth, transfer and yard) have been investigated separately. Each subsystem should have a management agent at the top with below it a number of resource agents. These resource agents represent the physical resources in the system. The management agents are tasked with determining the optimal schedule for their speciﬁc scheduling problem, and to act as a middle agent for the resource agents working below them. They use Contract Net Protocol to create a bidding system that is used to assign tasks to the best suitable resource agents.

The individual agents in the container terminal’s multi-agent system should be able to commu-nicate with other agents by the means of asynchronous messages which are based upon the FIPA Agent Communication Language standard. The agents make their decisions using optimizing algorithms. All kinds of algorithms could be used. Which one is most suitable highly depends on the problem the agent is supposed to be able to solve. Knowledge based control methods like neural networks and fuzzy reasoning should be used in order to make sure the agent makes the best decisions possible. Agents calculate estimated handling times using properties and states of associated agents and knowledge bases, and use these values to create optimal work schedules.

(6)

(7)

Dutch summary

Vanwege de toenemende vraag naar wereldwijde scheepvaart, worden veel havens onder druk gezet om uit te breiden. Echter, vanwege de land beperkingen waar veel havens mee te maken hebben is uitbreiden naar buiten niet altijd een haalbare optie. Een mogelijke optie is om de eﬃcintie en de productiviteit te verbeteren. Een goede manier om dit te doen is door gebruik te maken van optimalisatie. Geavanceerde controle algoritmes kunnen worden gebruikt om de prestaties van havenfaciliteiten te maximaliseren en daardoor de doorlooptijden van schepen en containers te minimaliseren. Een mogelijke manier om deze algoritmes eﬀectief toe te passen is met behulp van gedistribueerde besturing. Bij gedistribueerde besturing wordt de besluitvorming verdeeld over meerdere kleinere controllers die samenwerken om een gemeenschappelijk doel te bereiken. Dit literatuuronderzoek richt zich op gedistribueerde besturing in container terminals, aangezien bijna alle gevonden literatuur met betrekking tot gedistribueerde besturing in grote havens zich richt op deze systeemgrenzen.

Gedistribueerde besturing is een manier om grote besturingsproblemen op te splitsen in meerdere kleinere besturingsproblemen door middel van multi-agent systemen. Binnen deze multi-agent systemen werken autonome agenten samen om problemen op te lossen die buiten de individuele kennis van de probleem oplossers liggen. Gedistribueerde besturing lijkt een geschikte oplossing te zijn voor het eﬃcint beheer van een container terminal want het is ﬂexibel, aan te passen aan de omgeving, veelzijdig en robuust.

Het grote operationele besturingsprobleem van een container terminal is te verdelen in kleinere operationele besturingsproblemen voor kleinere delen van de container terminal door middel van subsystemen. Verschillende multi-agent systeem architecturen voor container terminals zijn onderzocht. Een verbeterde architectuur, die de beste eigenschappen van de voorgestelde architecturen in de literatuur bevat, is gepresenteerd in dit onderzoek.

De drie belangrijkste subsystemen van de container terminal (berth, transfer en yard) zijn afzon-derlijk onderzocht. Elk subsysteem moet bovenaan een management agent hebben met onder hem werkend een aantal resource agenten. Deze resource agenten vertegenwoordigen de fysieke middelen in het systeem. De taak van de management agenten is het bepalen van de opti-male planning voor hun speciﬁeke planningsprobleem en het fungeren als midden-agent voor de resource agenten die onder hen werken. De agenten maken gebruik van Contract Net proto-col. Dit protocol maakt het mogelijk om een biedingssysteem te creren waarmee taken worden toegewezen aan de resource agent met het beste bod.

De individuele agenten in het container terminal multi-agent systeem moeten kunnen commu-niceren met andere agenten door middel van asynchrone berichten die zijn gebaseerd op de FIPA Agent Communication Language. De agenten maken hun beslissingen met behulp van optimal-isatiealgoritmes. Allerlei algoritmes kunnen worden gebruikt. Welke het meest geschikt is is sterk afhankelijk van het probleem dat de agenten moeten kunnen oplossen. Kennis gebaseerde controle methoden zoals neurale netwerken en fuzzy redenering moeten worden gebruikt om te zorgen dat de agenten de best mogelijke beslissingen maken. Agenten berekenen geschatte behandeltijden met behulp van eigenschappen en status informatie van de betrokken agenten en

(8)

(9)

Abbreviations

ACL = Agent Communication Language AGV = Automated Guided Vehicle BDI = Belief-Desire-Intention CNP = Contract Net Protocol CT = Container Terminal

CTLOS = Container Terminal Logistics Operation System FIPA = Foundation for Intelligent Physical Agents GA = Genetic Algorithm

GBM = Generic BDI Module

JADE = Java Agent DEvelopment framework KAM = Knowledge Acquisition Module

KQML = Knowledge Query Manipulation Language MAS = Multi Agent System

QC = Quay crane

RTGC = Rubber Tired Gantry Crane SC = Straddle Carrier

(10)

(11)

Introduction

1.1 General introduction

There has been an increasing demand in global shipping as a result of increased international trades due to globalization. Looking at the port of Rotterdam, there was a small dip in 2009 because of the economic crisis, but throughput is rising again since 2010 [2]. Because of the increasing demand of global shipping, many port are pressured to expand. However, due to land constraints that many ports face expanding outwards is not always a feasible option. As a result, many ports are examining other ways to cope with the potential surge in cargo handling demands [56]. One possible option is to improve the efficiency and productivity. This would not only make sure the port is able to meet the growing demands but it would also reduce operational costs and it would be better for the environment, as energy and resources will also be used more efficient. A good way to improve efficiency and productivity is by using optimization. Advanced control algorithms can be used to maximize the performance of port facilities and therefore minimize the throughput times of ships and containers. However, these control algorithms have one major downside; they require a lot of computational time to decide what actions should be taken. A possible way to solve this problem is by using distributed control. In distributed control, decision making is split over multiple smaller controllers who work together in order to reach a common goal.

This literature research focusses on distributed control in container terminals, since almost all of the found literature regarding distributed control in large ports addresses these system borders. One research was found on planning vehicle transhipment in a seaport automobile terminal using distributed control [11] and one research on aligning the operations of barges and terminals through distributed control [9] [8], all others regard container terminals.

Maritime intermodal container terminals (CTs) are complex hub system in which multiple trans-port modes receive and distribute freight to various destinations [39]. The operations carried out in container terminals include some of the most complex tasks in the transport industry. Traditionally, the entire terminal is controlled by centralized software, which limits the expan-sion and reconfiguration capabilities of the system. Distributed control seems to be an adequate solution for the efficient management of a container terminal because it’s flexible, adaptable to the environment, versatile and robust.

(14)

1.2 Research objective

The objective of this research is to get an overview of all the research that has been done in the ﬁeld of distributed control for coordination of operations in container terminals. The following research questions will be answered:

• What is distributed control?

• How could distributed control improve container terminal operation?

• How could the large operational control problem of a container terminal be divided into smaller operational control problems for smaller parts of the container terminal?

• How could intelligent agents be employed to solve the smaller operational control problems? • How do the agents decide what actions to take?

• How do the agents communicate with other agents? • What information do the agents use for decision making?

1.3 Structure of the report

Chapter 2 will explain what distributed control is and how it could improve container terminal operation. Chapter 3 will show how the large operational control problem of a container terminal can be divided into smaller operational control problems for smaller parts of the container terminal by the means of the CT’s multi agent system architectures. Chapter 4 will zoom in on the container terminal and shows how intelligent agents could be employed to solve these smaller operational control problems. Chapter 5 will zoom in even further and will look at what goes on within the individual agents; how they communicate, how they decide what actions to take and what information they use. Each chapter will be ended with a discussion and possible improvements. This research will be concluded in Chapter 6.

(15)

Chapter 2

Distributed control

2.1 Multi-agent systems

Distributed control is a way of dividing large control problems into multiple smaller control problems by the means of multi-agent systems (MASs). A MAS can be deﬁned as a loosely coupled network of problem solvers that interact to solve problems that are beyond the individual knowledge of each problem solver [10]. These problem solvers are better known as agents. They are autonomous, which means that they can perform desired tasks without external guidance. In a MAS the separate agents don’t possess enough information or capabilities for solving the overall problem. By communicating with each other and performing their own separate tasks they can solve the overall problem together. In a MAS there is no system global control, the data are decentralized and computation is asynchronous [45].

There is a lot of research being done in the ﬁeld of MASs, because they bring a lot of new abilities compared to centralized systems. These new abilities include the following:

• MASs are able to solve problems that are too large for a centralized system, either because of resource limitations or because of the risk of system failure. If one part of a centralized system fails, the whole system is down. When this would happen in a MAS, the rest of the agents would still be able to continue their tasks.

• MASs allow for the interconnection of multiple legacy systems without having to rewrite the entire software. This can be done by building an agent around the software of the existing legacy system which enables it to communicate with other systems

• MASs can provide solutions for systems where similar type agents, who perform the same kind of task, need to communicate with each other. For example scheduling agents that have to compare schedules with each other.

• MASs can provide solutions for systems where information sources or expertise are dis-tributed throughout the system.

The main advantages of multi-agent systems are in the ﬁeld of performance enhancement. MASs can enhance the performance of a system in the ﬁeld of:

• Computational eﬃciency: diﬀerent computations can be done simultaneously.

• Reliability: the failure of one agent will not fail the whole system. Other agents will automatically be allocated to perform the failed agent’s task.

(16)

• Robustness: the system can handle uncertainties because the suitable information is ex-changed among agents.

• Maintainability: the system is easier to maintain because of its modularity.

• Responsiveness: anomalies can be handled locally and don’t have to be propagated through the whole system.

• Flexibility: agents with different abilities can adaptively organize to solve a problem. • Reuse: agents with a specific function can be reused in different agent teams to solve

diﬀerent problems.

A lot of research is being done in the ﬁeld of Multi Agent Based Simulation (MABS). This type of simulation diﬀers from other kinds of computer-based simulation in that (part of) the simulated entities are modeled and implemented in terms of agents [15] [12]. Research by Davidsson et al. [5] shows that agent based modeling is very suitable for simulating transport logistics.

2.1.1 Individual agents

The individual agents can be divided into three categories; deliberative, reactive and hybrids.

Deliberative agents

Deliberative is a term used for agents with a belief-desire-intention (BDI) architecture. BDI agents are practical reasoning architectures, in which the process of deciding what to do resem-bles the kind of practical reasoning that humans use in their everyday lives [48]. They reason about matters such as nonlocal eﬀects and other agents. The basic components of a BDI ar-chitecture are data structures representing the beliefs, desires, and intentions of the agent, and functions that decide what to do (deliberation) and how to do it.

Reactive agents

Reactive agents are agents that decide what to do without reference to their history. Their decision making is entirely based on the present state of their environment. This means that they do need to have suﬃcient information available in their local environment in order to be able to make an acceptable action [51]. Complex global behavior can emerge through simple interactions with other agents. This makes these type of agents very robust, fault-tolerant and fast, since they don’t have to revise their world model as it changes. Reactive agents are especially useful in rapidly changing environments because, unlike deliberative agents, they are not slowed down by sophisticated reasoning. A downside of reactive agents is that they are unable to consider nonlocal information and predict the eﬀects of their decisions on the global behavior of the system. This could lead to unpredictable and unstable system behavior.

Hybrids

Most agents are neither purely deliberative nor purely reactive. These hybrid agents combine aspects of both types. They are build up out of a number of software layers where each layer deals with a diﬀerent level of abstraction.

(17)

Literature Assignment Herbert Schro¨er

2.1.2 Architectures

In multi-agent systems, different architectures can be used. An architecture provides a frame-work for agent interactions through the definition of roles, behavior expectations, and authority relations. If the system operates in an open-world environment, the agents must be able to dynamically enter and exit the architecture. To make this happen, the agents must be able to locate each other. This is mostly done with the help of middle agents; an agent that is looking for another agent with a particular capability will communicate this to the middle agent, and the middle agent will find the right agent for the job.

Diﬀerent types of architectures have been been explored. These include the following:

• Hierarchy: Agents in the MAS interact through vertical communication between superior and subordinate. The superior agents exercise control over resources and decision making. • Community of experts: Each agent in the MAS is a specialist in some particular area.

They interact through horizontal communication.

• Market: Agents in the system compete for tasks or resources through bidding and con-tractual mechanisms. The only variable they interact with is price, which is used to value services.

• Scientiﬁc community: Solutions to problems are locally constructed. Afterwards they are communicated to other problem solvers that can test, challenge and reﬁne the solution.

2.1.3 Task allocation

Task allocation is a way of assigning responsibility and problem solving resources to the agents in the MAS. Minimizing task interdependencies bring two main benefits: by decreasing commu-nication between the agents it improves problem solving efficiency, and by minimizing potential conflicts it improves the chances for solution consistency. Task allocation can be performed in two ways:

• Static allocation: all tasks are made in advance by the designer, which creates a nonadap-tive problem solving organization. This option is very limiting and inﬂexible.

• Dynamic allocation: this utilizes the Contract Net Protocol (CNP) developed by Davis and Smith [6]. In the protocol the agents can dynamically take the role of manager or contractor. When an agent needs a certain job to be performed it takes the roll of manager and it send out a task description to all suitable agents for the job. The suitable agents (contractors) reply with a bid for the task. The “manager” then allocates the task to the agent with the best bid. This makes for a very ﬂexible method of task allocation.

2.1.4 Planning

For a single agent, planning is a process of constructing a sequence of actions considering only its goals, its capabilities, and environmental constraints. However, in a MAS also the other agents in the system need to be considered. Therefore, planning in a MAS environment also needs to consider the constraints that other agents’ activities place on an agent’s choice of actions, the constraints that an agent’s commitment to others place on its own choice of actions, and the unpredictable change of its environment caused by other unmodeled agents.

Most research on multi-agent planning assumes individual agent based planning. An early solution was complete planning before action. In order to produce a coherent plan, the agents

(18)

must be able to recognize subgoal interactions and avoid them or resolve them. A newer solution is partial global planning. This method does not assume distribution of subproblems. Instead, it lets nodes coordinate in response to current situations. Agents communicate plans and goals with each other, so a receiving agent can form expectations about the future behavior of a sending agent. With this information the receiving agent can alter its own local planning in order to achieve the common goals.

Another ﬁeld of multi-agent planning research focusses on teamwork based planning. This type of planning is especially suited for dynamic environments where team members could fail or be presented with new opportunities.

2.1.5 Managing resources

When resources are limited, it is necessary to allocate these resources to the multiple agents as eﬀective as possible. Various methods have been developed to solve this problem. Some of these methods are operation research-based, others are economics-based. In economics-based methods, agents are assumed to be self-interested utility maximizers. In markets, agents that control scarce resources agree to share them with other agents in return of resources owned by the other agents. This all in order to achieve a common goal. The exchange prices are publicly know. The communication during auctions goes through a central auctioneer so that agents have to exchange a minimal amount of information.

In the future most agents will probably be self-interested. This means they choose a course of action which maximizes their own utility. Care needs to be taken in that resources will not be overused. This problem is mostly solved by introducing pricing or taxing schemes. Also, the agents need to be designed in a way so that they won’t be able to exhibit oscillatory behavior, and that they are not untruthful in order to maximize their own utility.

2.2 Container terminal eﬃciency

In order to keep up with the competition, container terminals need to operate more and more efficient. One way to improve efficiency is by full automation. Automation could improve operational efficiency by eliminating the human mistake. Another way is by optimization. An optimized port operation system is able to maximize the utilization of port facilities and provide a cheaper solution to achieve better port efficiency [56]. Optimization can be done by the means of distributed control.

2.2.1 Container terminal operations

The operations carried out in container terminals include some of the most complex tasks in the transport industry [41]. This is due to:

• The great diversity of entities acting in the container import and export processes. • Interaction with a dynamic environment.

• The distributed nature of the problem which is formed by a set of independent systems, but whose individual decisions directly aﬀect the performance of the others.

Existing centralized and sequential applications for container terminal management are now in-suﬃciently ﬂexible to respond to changing management styles and highly dynamic loading/un-loading requirements [4]. Traditionally, the entire terminal is controlled by centralized software,

(19)

which limits the expansion and reconﬁguration capabilities of the system. Hierarchical organi-zations are used to put the resources into permanent, tightly coupled sub-groups, within which information is processed sequentially by a centralized software supervisor. The result of this is that large parts of the system could be shut down by a single point of failure. Also, this system contributes to plan fragility and increased response times.

Distributed control seems to be an adequate solution for the eﬃcient management of a container terminal because it’s ﬂexible, adaptable to the environment, versatile and robust.

2.2.2 The ideal container terminal

The ideal container terminal is a container terminal that is as efficient as possible. In order to reach this goal, the terminal has to be fully automated and the entire terminal has to be one multi-agent system. In order to keep the MAS as fast and efficient as possible, different properties have to be considered. These include the following:

• Every entity in the system has to be an individual agent.

• The whole system needs to be divided into multiple subsystems so that occurring problems can be resolved locally.

• In subsystems where there are more agents that are able to perform the same tasks, like the transfer system between quay and stack, there has to be a task allocation system which is regulated by a local supervisor agent in order to minimize communication between the individual agents.

• No agents has to own information or resources that it doesn’t need to fulﬁll its own tasks. • Task allocation and planning need to be executed as ﬂexible as possible.

2.3 Summary

This chapter has discussed the meaning of distributed control and multi agent systems, and what its advantages are compared to centralized systems. Various aspects of multi agent sys-tems were mentioned: individual agents, architectures, task allocation, planning and managing resources. It was explained why distributed control seems like an adequate solution for the eﬃcient management of container terminals. The chapter has been concluded with a number of properties that have to be considered in order to design an ideal multi agent system for a container terminal.

The next chapter will give an overview of multi agent system architectures for container terminals as proposed in literature. The chapter will be concluded by comparing the proposed systems to the ideal container terminal discussed in Section 2.2.2.

(20)

(21)

Chapter 3

Architectures

This chapter gives an overview of the research that has been done in the ﬁeld of multi-agent system architectures for container terminals. First a description of a typical container terminal will be given.

3.1 Typical design of a container terminal

The set of operations to be conducted in a container terminal (CT) is very extensive, but the alternative approaches share some common systems:

• Marine side interface: At the berth containers are loaded and unloaded to and from the ships by Quay Cranes (QCs).

• Transfer system: Containers are transferred between the quay and the stack. This can be done by diﬀerent systems; e.g. Automated Guided Vehicles (AGVs) or (manned) trucks. In general these transport systems are referred to as shuttles.

• Container storage system: Containers are taken from the trucks or AGVs by Yard Cranes and stored in the yard, and vice versa.

• Land side interface: This system handles the container transfer between the storage yard and the land transportation modes.

A layout of a typical container terminal is given in Figure 3.1. This system is called an indirect transfer system. There are also CTs with a direct transfer system. In this case both transfer and container storage are done with the same type of equipment. Straddle Carriers (SCs) are used to pick up a container at the berth and directly place it at the right position in the yard.

3.2 Literature

Yin et al. [56] propose a distributed agent system for the planning and scheduling of a container terminal with an indirect transfer system. The distributed agent system comprises four types of agents; a Port Planning Manager, a Berth Control Agent, a Shuttle Allocation Agent and a Yard Storage Agent. The overall architecture of the system can be seen in Figure 3.2. The architecture shows a hybrid of a community of experts and a hierarchy, as the Port Planning Manager stands above the other agents.

(22)

Figure 3.1: Layout of a typical container terminal [41]

Figure 3.2: System architecture as proposed by Yin et al. [56]

All the necessary information for the scheduling of port facilities is stored in the Port Planning Manager’s data source. Besides holding this information, the Port Planning Manager also pro-vides a negotiation and communication platform for the Berth Control Agent, Shuttle Allocation Agent and Yard Storage Agent. This is used for resolving conflicts between the agents under supervision of the Port Planning Manager. The Berth Control Agent is used to determine the near optimal schedule for the berth usage with the aid of a Genetic Algorithm (GA)-enhanced dynamic scheduler. It also coordinates the activation of the Shuttle Allocation Agent, and con-trols the communication between the Shuttle Allocation Agent and itself in order to derive a feasible or near optimal schedule for both the berths and the shuttles. Based on the berth sched-ule and the shuttle allocation, the Shuttle Allocation Agent is tasked to plan the yard storage allocation schedule and the truck or train schedule for transporting containers out of the yard. The workflow of this distributed agent system can be seen in Figure 3.3. The Port Planning Manager keeps track of all the active agents in the system. When a new ship arrives it verifies and filters the data it receives from the ship. The valid ship arrival information is then channeled to the Berth Control Agent. This also triggers the rescheduling of the Berth Control Agent, Shuttle Allocation Agent and Yard Storage Agent. The Berth Control Agent will be discussed in more detail in Chapter 5.

(23)

Figure 3.3: Workﬂow of the distributed agent system [56]

uses a set of hypothetical data to illustrate the capability of the proposed system to optimize the berth schedule and the need and shortage of shuttles for a typical container terminal. The capability of the system is evaluated using a dataset comprising 5 days, with in total 42 ship arrival and loading and unloading records. The given results are a berth allocation schedule and the shuttle needs per time interval, generated by the distributed agent system. This shows that the system can successfully generate the schedules for both the Berth Control Agent and the Shuttle Allocation Agent. However, no clear insight was given in how these data were obtained. Also, it is unclear how optimal these schedules actually are as they are not compared to any other data. The used software has also not been mentioned. According to the authors it is up to the end user wether the system will be fully automatic or merely used as a decision tool. Rebollo et al. [41] present a system architecture which is based on the multi-agent system paradigm for solving complex problems. It is applied to solve the management problem for a container terminal with a indirect transfer system. The presented architecture is shown in Figure 3.4. This is a clear example of a community of experts; each agent is a specialist in some area and there’s only horizontal communication.

A Ship Agent will be created for each ship docking in the port. This agent will hold the load proﬁle of the designated ship. Its goals are to minimize the gantry crane idle time, to maximize its utilization, to minimize the ship’s load/unload time, and to minimize the derived costs from the stowage process. The diﬀerent active Ship Agents co-ordinate with each other as a whole in order to minimize the possible blockages between the assigned cranes.

Each gantry crane in the terminal has its own Stevedore Agent. It uses heuristic search algo-rithms to obtain the most appropriate scheduling to manage the container stowage in the ship’s load/unload sequences. The Stevedore Agent coordinates with the active ship agents and the suitable service agents in order to minimize the number of necessary service agents and the number of empty moves they need to make.

Each shuttle has its own Service Agent. Its main goal is to determine the appropriate allocation for the arriving containers and the suitable conﬁguration of the portion of the yard it controls. To do this, the agent must know the yard map and the properties of its container. Also, it has

(24)

Figure 3.4: System architecture as proposed by Rebollo et al. [41]

to coordinate with the other Service Agents in order to resolve any conﬂicts. There is no middle agent to guide this process. The goal of the Service Agents is to maximize the stacking density in their yard portion. It uses non-supervised learning techniques to reach this goal.

Each system transtainer is modeled as an autonomous agent whose goal it is to eﬃciently perform the stacking operations of the containers in the yard. Its goal is to minimize its number of empty moves. Therefore it must obtain the most accurate sequence for the container movement to/from its correct position in the yard. It waits for stacking requests from the service agents, who provide the transtainer agent with information about what container needs to be moved to/from where. Each terminal gate, which controls the landside interface, is controlled by a Gate Agent. It is tasked in assigning service agents to the incoming (bringing a container) or outgoing (taking a container) trucks and checking the incoming container’s data.

A prototype of this system is currently being developed, so no results have been presented. Yan et al. [53] [52] present a multi-agent based system for resource allocation and operation scheduling problems in container terminals to optimize their productivity. The presented archi-tecture is shown in Figure 3.5. The indirect transfer system has ﬁve kinds of agents identifying physical resources: the Ship Agents, the Berth Agents, the Quay Crane Agents, the Truck Agents and the Yard Crane Agent. Besides those, there are three agents acting as distributed controllers: the Quay Management Agent, the Transfer Management Agent and the Yard Man-agement Agent. The User Agents are used for providing interfaces for the manMan-agement agents’ communication with the outside. With this architecture every part of the terminal has a User Agent, a management agent and several resource agents (see Figure 3.6). The management agents communicate with each other. This makes the architecture a hybrid of a community of experts and a hierarchy.

A Ship Agent is created for every new arriving ship. It will send a request to the User Agent for berth and quay allocation. Each berth has its own Berth Agent. It has its own agenda to notify its status message to the Quay Management Agent. The quay cranes all have their own Quay Crane Agent. Its task is to report the crane’s busy/idle status to the Quay Management Agent. Each truck has its own Truck Agent. They keep contact with, and are under control

(25)

Figure 3.5: System architecture as proposed by Yan et al. [53]

Figure 3.6: Agent group model [53]

of, the Transfer Management Agent. The yard cranes have their own Yard Crane Agent. They communicate their status to the Yard Management Agent.

The Quay Management Agent is created after receiving a resource allocation request by a Ship Agent. It’s responsible for the ﬁnal decisions on berth allocation and assigning the necessary quay cranes. Its goals are to obtain the most cost beneﬁcial berthing and to assign the right quay cranes according to their working status dynamically.

The Transfer Management Agent holds the status information of all the trucks in the system. After receiving a request from the Quay Management Agent or the Yard Management Agent it will assign the trucks needed for the container transfer between the quay and the yard. The

(26)

Transfer Management Agent also has guiding driving rules for trucks to avoid collision and congestion.

The Yard Management Agent is responsible for the allocation of the yard cranes needed for the stacking/pick-up operation. It will guide the yard cranes to make sure they work eﬃciently, with minimized empty moves and restacking.

A preliminary version of the system is currently under development, so no results have been presented. The system is meant for implementing it as the integral management system of a container terminal.

Hui et al. [23] present a multi-agent theory based system for resource allocation and operation scheduling problems in CTs with an indirect transfer system. The presented architecture is shown in Figure 3.7. The agents in the outer ring represent all the physical entities in the system. An agent is created for every one of them. They only communicate with their designated management agent. There is one of each management agent. They collect information from the agents under their management and use it to assign tasks and resolve conﬂicts. The management agents can communicate with the other management agents and with the Core Agent. The Core Agent represents the control center of the CT. It assigns tasks to and resolves conﬂicts among the management agents. The Communication Agent represents communication devices and provides data transmission services for the system. The Database Agent represents the system’s databases, of which information can be requested from the agents in the system.

(27)

A simulation of the multi-agent system has been developed using JavaEE. The simulation results are compared to those from literature [58], where the same terminal model is used. Results show that the MAS model has a much lower completion time than the same instance with a GA algorithm and static scheduling. The results seem unreliable, since the compared results are from diﬀerent researches.

Yu et al. [57] proposes a framework of a CT schedule system based on MAS for an indirect transfer system. The architecture of the system is shown in Figure 3.8. The system has four schedule agents: a Berth Schedule Agent, a Quay Crane Schedule Agent, a Yard Crane Schedule Agent and a Container Truck Schedule Agent. The schedule agents can communicate with each other and with a Blackboard subsystem. The Blackboard is used to store the global database which contains the information and hypotheses needed, generated and shared by the agents. The Blackboard controller is used to supervise, control and choose proper data. The proposed system has not been validated.

Figure 3.8: System architecture as proposed by Yu et al. [57]

Thurston et al. [46] propose a distributed agent architecture for port automation. The architec-ture is shown in Figure 3.9. The direct transfer system consists of four different types of agents. Three of them all represent a physical resource; Quay Crane Agents, Straddle Carrier Agents and Traffic Agents. A Traffic Agent controls a cell of the yard highway that contains more than one entry point, like a crossing. The fourth type of agent is the Area Manager Agent. It assigns jobs to any Straddle Carrier Agent in the area it’s in charge of.

Figure 3.9: System architecture as proposed by Thurston et al. [46]

When a new container comes in the Quay Crane Agent sends a message to all Area Manager Agents requesting bids for the job. The Area Manager Agent then selects the suitable Straddle Carrier Agent out of all the Straddle Carrier Agents in his area. The Area Manager Agent can make time reservation for use of a certain crossing with the Traffic Agents. The Straddle Carrier Agents are not coupled to one specific Area Manager Agent, but can be coupled to new ones when they enter a different area.

(28)

The system has been prototyped in Java, yet no results or clear veriﬁcations have been provided. Ye et al. [55] present an architecture for a Container Terminal Logistics Operation System (CTLOS). The architecture of this indirect transfer system is shown in Figure 3.10. The system is divided in two types of agents: decision agents and operation agents. The decision agent controls every corresponding operation agent and also communicates with the other decision agents. Agents communicate with each other via a communication and negotiation network, but also through a Blackboard an Mailbox system.

Figure 3.10: System architecture as proposed by Ye et al. [55]

Part of the system has been simulated and results have been presented that show a slightly lower handling time for the MAS system. However, no clear insight has been given in how these results were obtained and are therefore unreliable.

Hanwu et al. [14] propose a multi-agent CTLOS based on the theory of constraints. The basic idea of the theory of constraints is that the output of a system is determined by its bottlenecks. Therefore, bottlenecks must be found and improved.

The architecture of this indirect transfer system is shown in Figure 3.11. As can be seen from the Figure, no horizontal communication takes place so this is a clear hierarchy. The information agent acts as a database where information from the other agents is stored. The bottleneck agent finds out bottlenecks by simulation software. When it finds a bottleneck it informs the information agent. The task agent receives information from the outside and translates that into tasks for the other agents. The agents on the bottom row are all resource agents. They send information of the resources to the dispatching agents. The dispatching agents then make scheduling decisions for the resource agents. Places where bottlenecks are found are prioritized in scheduling. The proposed system is purely theoretical, so no results or verifications have been given.

Henesey et al. [17] describe an agent based simulator for evaluating operational policies in the transshipment of containers in a container terminal. A simulator tool called SimPort is used to simulate a CT with a direct transfer system. The used architecture is shown in Figure 3.12. The big diﬀerence between this system and the other discussed literature is that this system is not meant to be used for automation, but merely as a simulation tool to test diﬀerent strategies. The port captain, ship captain, stevedore, and terminal manager are actual managers modeled

(29)

Figure 3.11: System architecture as proposed by Hanwu et al. [14]

as agents. This architecture is coupled to a simulator and communicate by sending actions and receiving observations.

Two sets of experiments have been performed in order to evaluate stacking conﬁgurations and transshipment operational policies. The results show that the developed simulation tool is able to study the impact of diﬀerent policies for sequencing, berthing, and stacking on the performance of CTs.

The simulator tool SimPort is used by Henesey et al. for evaluating all kinds of operational policies in transshipping containers in container terminals [19] [20]. The tool has been extensively veriﬁed and validated [16].

(30)

3.3 Discussion and improvements

As discussed in Section 2.2.2, a container terminal has to own certain properties in order to operate as eﬃcient as possible. Looking at the discussed literature, not all presented CT archi-tectures satisfy these properties. Various properties have not been mentioned, as the level of detail among the literature diﬀers quite strongly.

Most presented systems have an architecture that consists of a hybrid between a hierarchy and a community of experts. There is one agent at the top and below it there is a community of experts. Mostly the top agent is able to give tasks to other agents in the system, but in some cases it’s merely used as a communication platform. It’s best if this agent acts as a middle agent for the management agents in the community of experts so it can quickly resolve occurring conﬂicts.

The community of experts part of the hybrid architecture makes sure the system can be divided into various subsystems. The way the system is divided diﬀers, but there are a minimum of three main subsystems to be recognized ([56],[53]): berth, transfer and yard. Though in some cases ([23],[55]) subsystems are created for almost every type of equipment or task. More subsystems means that problems can be resolved even more local, but it could also increase communication. Also, some of the suggested subsystems seem unnecessary; like a container management agent [23] (a container shouldn’t have to be able to perform any tasks) or using both a Yard Management Agent and a Yard Crane Scheduling Agent [55]. It does seem logical to have separate Berth Control and Quay Crane Scheduling agent as suggested by Yu et al. and Hanwu et al. as these are two separate scheduling problems. If there would be a local problem in the Berth Control Agent, the Quay Crane Scheduling Agent would still be able to function on its own and ﬁnish (un)loading the ships that are already berthed.

An exception to the hybrid architecture is presented Rebollo et al. [41], which only consists of a community of experts. This system uses no middle agents. Because of this, a lot more communication is needed and agents poses more information than they need. This is of course unwanted.

A completely different approach is given by Thurston et al. [46]. A difference with the other systems is that this one uses a direct transfer system whereas the others use an indirect transfer system. The weakest point of this system is there is not an overall yard management agent. Instead there are multiple area management agents, which all compete for an incoming container. When an area management agent gets the job it still has to find the right straddle carrier for it in its own area. This does not only lead to more communication, but also lacks flexibility. The architecture proposed by Henesey et al. [17] is so different that it’s hard to compare it to the others. It’s not meant for automation and it has not really been made clear how information flows between the cranes and straddle carriers.

When combining all above discussed factors, an improved system architecture can be made. This architecture is shown in Figure 3.13. Like in most of the discussed literature, an indirect transfer system is considered.

There is one Port Planning Manager at the top which is connected to a database that holds all the port’s scheduling information. The Port Planning Manager provides a communication and negotiation platform used to resolve conflicts between the different management agents. Below the Port Planning Manager is a community of experts containing four different management agents: a Berth Management Agent, a Quay Crane Management Agent, a Transfer Management Agent and a Yard Management Agent. They are all used to determine the optimal schedule for their specific scheduling problem, and to act as a middle agent for the resource agents working below them. They are able to communicate with each other and with the Port Planning Manager.

(31)

Figure 3.13: Improved system architecture

Every physical resource in the system is represented by a resource agent. There are six types of resource agents: Ship Agents, Berth Agents, Quay Crane Agents, Shuttle Agents and Yard Crane Agents. They are only able to communicate with agents of their own kind and with their designated Management Agent. This in order to have as little communication as possible and so that problems can be resolved locally. The containers in the system won’t have their own agent, as they are just data in the system and don’t have to be able to perform tasks.

3.4 Summary

An overview has been given of all MAS architectures for container terminals proposed in litera-ture. The proposed architectures have been compared to the ideal container terminal discussed in Section 2.2.2. The chapter has been concluded with an improved architecture which inhabits all the best properties of the proposed architectures found in literature.

The next chapter will zoom in on the three major subsystems of the container terminal: berth, transfer and yard.

(32)

(33)

Chapter 4

Subsystems

This chapter will give an overview of the solutions found in literature on how intelligent agents could be employed to solve the smaller operational problems. The chapter is divided into three parts, each of of which considering one of the three major subsystems: berth, transfer and yard.

4.1 Berth

Lokuge et al. [30] propose a MAS for berthing with BDI agents using neural network and adaptive neuro fuzzy inference for intelligent planning. The main agents in the proposed system are shown in Figure 4.1. There are three types of agent: Vessel Agents for each incoming ship, Berth Agents for each berth, and a Schedule Agent. The Vessel Agents only haves to communicate their own properties to the other agents. The Schedule Agent handles many important tasks such as berth-assignment, rescheduling, vessels shifting, etc. The Berth Agents are responsible for all the operations at the berths and make every eﬀort to improve productivity and achieve set targets by the SA. The agents have a hybrid BDI architecture extended with a Knowledge Acquisition Module (KAM) This KAM will be discussed in more detail in Chapter 5.

Figure 4.1: Main agents in the proposed system as proposed by Lokuge et al. [30] A scheduling scenario at the Jaya container terminal port of Columbo has been simulated. Various membership functions and IO surfaces that are used have been given. Results show that the BDI agent system has a much lower average gross berth productivity prediction error than the present system of the Jaya CT. Also the average vessel waiting time becomes much lower.

(34)

Li et al. [29] [27] present a MAS model based on Harvard architecture as a solution to the Berth Allocation Problem in CTs. Harvard architecture means that the architecture has physically separate storage and signal pathways for instructions and data. The presented architecture is shown in Figure 4.2.

Figure 4.2: Agent-based model for the Berth Allocation Problem using Harvard architecture as proposed by Li et al. [29]

The objective functions of the Berth Allocation Problem are shown in Formulas 4.1 and 4.2.

minf1 = N ∑ i=1 (tbei− tbsi) (4.1) minf2= N ∑ i=1 (thei− thsi) (4.2) Where:

N = the total number of container ships arriving in the port in a planning period. tbei = the ﬁnal departing berth time of ship i.

tbsi = the initialized mooring berth time of ship i.

thei = the started handling time of ship i.

thsi = the terminated handling time of ship i.

The objective functions intend that the total berthing and handling or arriving ships should be minimized. Besides the objective functions, the system should also consider various constraints. The system has been implemented on AnyLogic 6.5.0. and SQL Server 2008, and results of a pilot study show that a real life scenario can be recreated. However, no results have been given about how well the system works.

Zhao et al. [60] present a coordinated scheduling strategy for berth and quay crane allocation based on MAS theory. The presented architecture of the MAS is shown in ﬁgure 4.3. The Ship Agents, QC Agents and Berth Agents all represent physical resources. The Management Agent gives out tasks, deﬁnes schedules and acts as a negotiator for the other agents. The Commu-nication Agent is purely for sending messages between the Ship Agent and the Management

(35)

Agent. The Berth and QC Agents don’t communicate with each other; all communication goes through the Management Agent. The system uses the Contract Net Protocol combined with a set of coordination rules. The Management Agent only sends out requests to agents that satisfy basic needs instead of to all agents. This reduces communication and calculation.

Figure 4.3: Architecture of the MAS as proposed by Zhao et al. [60]

A simulation model of the MAS has been developed on JavaEE. Simulations have been run and results are compared to a GA strategy proposed by Han et al. [13]. Results show that the proposed MAS system results into a reduction of around 4% in average linger time in terminal compared to the GA strategy. Besides part of the programming code, no clear insight has been given in how these results were obtained.

Sun et al. [44] propose a hybrid MAS structure model for berth allocation. The Contract Net Protocol is used as negotiation strategy to realize interaction and collaboration among agents. Three types of agent are considered: a Berth Allocation Agent, Berth Agents and Vessel Agents. The structure of the system is shown in Figure 4.4.

(36)

Bidding is used for assigning the appropriate berth for an incoming vessel. The system doesn’t use the ﬁrst in ﬁrst out principle, but instead considers a number of factors to calculate the berthing priority for each vessel. The system considers factors like the position of the vessel, the vessel properties and the emergency of the tasks. After the vessel with the highest priority has been selected, the Berth Allocation Agent requests bids for the job from the Berth Agents. The Berth Agent with the best bid will be awarded the task, and the berthing plan will be sent to both the selected Berth Agents and the Vessel Agent.

The system has been implemented with C# as the development language and SQL Server as the database. Simulation results should indicate that the distributed control can simplify the complexity of the system, and improve the adaptability, robustness and scalability. However, none of these simulation results have been given.

4.2 Transfer

Henesey et al. [21] [18] present a MAS based simulator for evaluating AGV systems for container terminals. Two types of AGVs are modeled: the IPSI AGV and the T-AGV. The IPSI AGV makes use of cassettes that can be detached from the AGV. Because of this the QC doesn’t have to wait for an AGV to arrive; it can place its container on the cassette, which can later be picked up by the AGV. All entities in the system the AGVs, QCs, cassettes, containers and buffers -are modeled as agents. The model is implemented using DESMO-J, an open source library for JAVA. The entities use the Contract Net Protocol to coordinate tasks. This protocol implies that one agent will take the role of manager, which initiates a job to be performed by one or more agents. In this case the buffer agents act as managers and the AGV agents are bidding for jobs. Each QC has a buffer agent and a pre-specified number of AGVs assigned to it.

Simulations have been run using data provided by industrial partners. Results show that ship service times get close to their smallest values when using 3 IPSI AGV with 2 or more cassettes per AGV. More cassettes give a higher utilization rate and may prevent from having to use more AGVs, which costs more money. It has been shown that multi agent based simulation is very suitable for simulating these kind of problems.

Ye et al. [54] use a MAS as the basis for an intelligent truck dispatching system using the Contract Net Protocol (CNP) and fuzzy reasoning. The system consists of a Container Truck Scheduling Agent and multiple Container Truck Agents. Each Container Truck has its own Container Truck agent. The decision process is shown in Figure 4.5.

The CNP makes the bidding system possible. This does result in a lot of communication. In order to reduce the communication and improve eﬃciency, the bidirectional negotiation mecha-nism was adopted. Because of this the Container Truck Scheduling Agent only sends requests to Agents that apply for the job. Fuzzy reasoning is used in bid inviting, bidding and in deciding who the winner is. The Fuzzy model uses the following input information: traveled distance, priority of handling operation, the length of CT queues at loading locations and the number of scheduled CTs. The working of the system has been shown through a case, however details on the membership functions and knowledge sets have not been given.

Li et al. [28] present a system for modeling and simulation of yard trailer dispatching at CTs based on MAS. The modeling is aimed at moderating traﬃc jams at the quay side and storage yard, and minimizing the total working hours of the QCs. The dispatching system during export is shown in Figure 4.6. The system is continuously in pursuit of minimizing the completion time for every step in the process, and consequently the handling time of the ships.

(37)

Figure 4.5: Decision process as proposed by Ye et al. [54]

Figure 4.6: Dispatching system during export as proposed by Li et al. [28]

and Java. Result have been compared to results of a similar system found in literature [59]. Results show that a semaphore mechanism has a faster completion time (1:50 hours) than a Genetic Algorithm (1:56 hours) and static scheduling (2:11). No clear insight has been given in how these results were obtained and since not all data are from the same research, the results seem very untrustworthy.

4.3 Yard

Peng et al. [40] present a distributed MAS that mainly focusses on the communication between Automated Transfer Cranes and AGVs. As can be seen in Figure 4.7, the agents in this subsys-tem are divided in two categories: operation agents and resource agents. Each Yard Crane is controlled by a Yard Crane Agent and each vehicle is controlled by a Yard Vehicle Agent. The operation agents have the ability to compute and assign the initial schedules for the resource

(38)

agent in the area it’s in charge of.

Figure 4.7: Agent infrastructure as proposed by Peng et al. [40]

The system uses market-based control to ﬁnd the best agent for a certain task. Auctions are used where agents are able to bid and raise their bid until no other agent is willing to bid any longer. The task is awarded to the agent with the best bid.

The simulated loading process begins with the assigning of the initial schedules for the Yard Crane Agents by the Yard Crane operation agent. The Yard Crane Agents will plot the shortest path to their assigned containers. The Yard Vehicle operation agent will assign the initial schedules for the Yard Vehicle Agents, based on the initial schedule for the Yard Crane Agents. After this, a process will start where the yard crane operation agent continues to monitor terminal conditions and predict possible conﬂicts. When conﬂicts arise, new negotiations will be initiated in order to get a new solution. Schedules will be altered based on the new solution. Bid prices are estimated as shown in Formulas 4.3,4.4 and 4.5.

P = Ei Ti (4.3) Ei= ∆Mi· Pu (4.4) Ti = Qi· t (4.5) Where: P = bid price.

Ei = eﬀect during the expected operation time when a unit of equipment i is added.

Ti= expected operation time of equipment i.

Pu = amount of cost saved when the processing time of performing operations is reduced by a

unit of time.

Mi = processing time reduced by adding a unit of equipment i required.

Qi has not been speciﬁed.

The system has been simulated using a container terminal yard physical environment which communicates with the multi-agent system. The communication interface between the two systems is built on Java programming language using the Java Remote Method Invocation facility. This facility makes it possible for two Java based programs to communicate with each other. No results or veriﬁcations of the system have been presented.

Shu et al. [42] present a distributed MAS that focusses on a Yard Schedule Agent. The Yard Schedule Agent communicates with several resource agents: Yard Crane Agents, Container Agents, Yard Agents, and Truck Agents. The negotiation model of the Yard Schedule Agent is shown in Figure 4.8.

When a new container comes in, the Yard Schedule Agent will check the working condition of the Yard Schedule Agents that could place the container on a suitable location. If there are more options then it will consider the number of yard crane tasks (TaskNum), task complexity

(39)

Figure 4.8: Yard Schedule Agent negotiation model as proposed by Shu et al. [42]

(TaskRank), and the driver eﬃciency (HandleEf). Also the density of the current operations in a certain yard block (BayLayout) will be requested from the Yard Agents. Each Yard Crane Agent will make a bid based on these values. After evaluation, the Yard Schedule Agent will award the task to the Yard Crane Agent with the best bid. This is informed to the Truck Agent so it knows where to go.

Bid prices are calculated as shown in Formulas 4.6 and 4.7.

T =

T askN um∑ i=1

[T imeConsum(DischargeP osi)∗ T askRanki] (4.6)

F = f (T askN um, T askRank, f (BayLayout), HandleEf ) = T

HandleEf (4.7) Where:

T = time needed. F = bid price.

The system is being implemented using C++ programming. No results or veriﬁcations of the system have been presented.

Keﬁ et al. [26] present a multi-agent model to simulate, solve and optimize the amount of storage space for handling container departures within a CT. The model is developed to minimize the expected total number of rehandles. The MAS consists of multiple Container Agents and an Interface Agent. A Container Agent is created for each container. When a set of containers needs to leave, a message is sent to all containers agents blocking their path. The blocking containers’ agents will then send a “Search-Place” message to their acquaintances to request a new appropriate slot. The containers will be moved to the new slots until all container agents are satisﬁed.

The system has been implemented using the JADE platform. Simulations have been run, com-paring it to a centralized system. Results show that the MAS performs slightly better than the

(40)

centralized system. Also a diﬀerence has been shown between using a non informed algorithm and an informed algorithm. In the former, the determination of a new slot is done randomly. In the latter, a tested heuristic is applied. The results show that using an informed algorithm could signiﬁcantly reduce the number of unproductive movements.

Huynh et al. [25] introduce an agent-based approach to model yard cranes for the analysis of truck turn time with respect to service strategy. The cranes are modeled as utility-maximizing agents. When available to work, each crane agent will evaluate the utilities of all trucks and will pick the truck that has the greatest utility to serve. Three diﬀerent truck utility functions have been developed: distance-based, time-based, and time-and-distance-based.

Distance-based utility function

U (i) = 0 - distance to crane - Penalty· other crane in path - Penalty · turn required? - Penalty · change heading? - Penalty · not closest crane

Time-based utility function

U (i) = truck wait time - Penalty · other crane in path - Penalty · turn required? - Penalty · change heading? - Penalty · not closest crane

Time-and-distance-based utility function

U (i) = truck wait time - distance to crane - Penalty · other crane in path - Penalty · turn required? - Penalty · change heading? - Penalty · not closest crane

Fixed penalties are given if there’s another crane in its path or closer to the truck, and when the truck has to change direction or heading. Variable penalties are given when a crane chooses to serve another truck with a higher utility while heading towards its intended truck.

The model has been implemented in NetLogo, an agent-based simulation platform and program-ming language. Discrete simulations have been run using the three diﬀerent utility functions. Results show that average wait times with the distance-based utility function are four times lower than with the other ones. This is because the time based functions make the cranes make long runs from one end to another without servicing the trucks in between. Diﬀerences become smaller when more cranes are used because the travel distances become smaller, though maximizing the distance-based utility function remains the best option.

Hoshino et al. [22] present an agent cooperation system focussed on the communication between AGVs and Rubber Tired Gantry Cranes (RTGCs). Different selection methods have been described. In the first selection method the AGV selects a RTGC based on relative distance, in the second situation it selects a RTGC based on its assigned workspace. The RTGC selects an AGV right after the AGV arrives at the stack or after it leaves the Quay side. Also a distinction is made between random container storage and planned container storage. For planned container storage the total moving distance of the RTGCs is minimized and the workloads on the locations and work paths are equalized. This leads to several different management models which are shown in Table 4.9. Since storage planning is based on the workspace of the RTGC, distance-based RTGC selection can only be used in combination with random storage. Also, under the workspace-based selection, it is clear that RTGC selection and call-out when the AGV goes into the work path is more efficient. This leaves 6 models that are being tested.

Results show that model 6 is the most eﬃcient system management model, as it requires the least amount of necessary agents for a certain throughput and therefore also the lowest costs. So it seems like it’s best to use storage planning, let each RTGC have its own workspace and to let the RTGC know an AGV is coming as early as possible. No clear insight was given in how

(41)

Figure 4.9: Management models as proposed by Hoshino et al. [22]

Distributed Control for Coordination of Operations in Large Ports

English summary

Dutch summary

Abbreviations

Table of Contents

Chapter 1

Introduction

1.1

General introduction

1.2

Research objective

1.3

Structure of the report

Chapter 2

Distributed control

2.1

Multi-agent systems

2.2

Container terminal eﬃciency

2.3

Summary

Chapter 3

Architectures

3.1

Typical design of a container terminal

3.2

Literature

3.3

Discussion and improvements

3.4

Summary

Chapter 4

Subsystems

4.1

Berth

4.2

Transfer

4.3

Yard