Agent-based models for policy makers
Émile J. L. Chappin, Alfredas Chmieliauskas and Laurens J. de Vries e.j.l.chappin@tudelft.nl
Delft University of Technology
Faculty of Technology, Policy, and Management Energy and Industry Section
P.O. Box 5015
2600 GA Delft, the Netherlands
Abstract: In order to support policy decisions, we have developed a modelling platform called AgentSpring, which facilitates the development of agent-based models in a modular and structured manner, using
state-of-the-art IT development principles and tools. An attractive web-based interface allows for the interaction with policy makers. A model named d13n was developed on the topic of decarbonization of the power generation sector. For this model, relevant applications for policy makers – CO2 reduction,
cross-border effects of policies, security of supply – have been identified. Each of those questions can be tackled by developing specific scenarios within the same modelling platform and with the same model core. Keywords: Agent-based modelling, energy systems, policy support
1 Introduction
Recently, agent-based models have been proposed in order to support policy makers in their decisions (cf. Farmer and Foley, 2009), among others in the context of an energy transition (Chappin, 2011). Traditional simulations and models face limitations as the complexity of the systems which we want to change
increases. As Jean-Claude Trichet, President of the European Central Bank said in a speech on November 18 2010: “[In] the face of crisis, we felt abandoned by conventional tools.” The main traditional tools
underlying policy support focus stronlgy on the (macro) economy. In the words of Alan Kirman: “The idea that the economy is essentially on an equilibrium path from which it is sometimes perturbed seems simply to be the wrong departure point. I claim that we have to start from the vision of the economy as a system of interacting agents whose actions, beliefs and decisions are constantly and mutually influenced” (Kirman, 2009, p. 11). A similar argument holds for many policy questions which are typically related to at least some aspects of economy, technology, and actor behavior.
So far, the application of agent-based models (ABMs) for supporting policy is limited. Many argue that there is a need for ABMs with sufficient richness and operationability to be applicable to real policy questions. This means that suitable platforms are needed to facilitate the development of such models and to promote the interactions of policy makers with with the modellers and with the models themselves.
Tradional approaches, which Trichet calls ‘conventional tools’, are part of a strict regime (cf. Holtz et al., 2008). Too often, policy decisions are based on models that originate from traditional paradigms the applicability of which is debatable. The use of equilibrium modelling in a dynamic world is limited. However, new approaches are challenging, because new modelling techniques such as ABM are not established in the way that traditional ones are (e.g. Lejour et al., 2006; Schäfer and Jacoby, 2006). A second shortcoming of the way in which models are conventionally used for policy support is that policy makers are presented with model outcomes – which depend strongly on the choice of scenarios and many other assumptions – and typically do not have access to the models themselves, as it takes technical expertise to run and interpret them. Although sometimes it is possible to bring policy makers or analysts into the
modelling arena (i.e. participatory modelling, serious gaming), such an approach is not always feasible or effective. In case of ABMs, a complication is that they may actually be scary to policy makers, as they might confront them with the fact that for many of their objectives, they have limited control options. However, the economic crisis and the fundamental change that the energy sectors will need to undergo in the coming decades have made it clear that there is a need for models that represent the complex, dynamic nature of these systems.
In this paper, we present a new platform called AgentSpring which we developed in order to improve the accessibility of our agent-based models to policy makers and analysts. A second, import objective for this platform is to make models more flexible, so different policy questions can be addressed more easily with the same general model. We have applied this platform to a model of two two interconnected electricity sectors with a common CO2 market that resembles the Europe’s emissions-trading scheme (EU-ETS). With
this example, we show how this platform leads to a type of model that is both more flexible with respect to new policy questions and more accessible to policy makers and analysts.
Fig. 1: Snapshot of the user interface of AgentSpring with the model running
2 A new modelling framework
Policy decisions are often about systems that deal with both the social as well as the technical (cf. Ottens et al., 2006). The consequences of policy intervention typically materialize by changing the behavior of actors regarding their options, assets and decisions. That is a core reason in favour of ABM, but it also highlights that the scope of models for policy decisions is relatively large. The models need to be rich
enough in order to properly represent the social, the technical and socio-technical components and their interactions. For policy support, elaborate and diverse behavior of agents has to be possible. Models are data driven and have to incorporate extensive behavior algorithms (Chmieliauskas et al., 2012).
The desire to open up models to a community of researchers, public and private problem owners, and the general public is an important approach to ventilate research results to the public. It changes the role of models and simulations in the debate, and allows the end user to explore, validate and experiment with the tools that researchers develop. In addition, extendability and reusability of code is important, because it allows developed models to become a basis around years of policy-supporting modelling research. 2.1 AgentSpring framework
There are many ABM frameworks in existence, some more popular than others. Although it may have been possible to use and modify existing frameworks, we have taken up the opportunity to build the AgentSpring framework that would leverage off the new and powerful open source libraries and changing software development paradigms. AgentSpring is developed as an open-source tool. This implies that anyone can use and contribute to the platform. AgentSpring is available online1. AgentSpring is based on Java technologies
and runs on all popular operating systems (Linux, Windows, and Mac). AgentSpring gets its name from and makes use of Spring Framework – a popular software development framework, that promotes the use of object oriented software patterns (Johnson et al., 2009). One such pattern calls for separation of data, logic and user interface (Krasner and Pope, 1988). Although the latter is an old concept, most modeling
frameworks mix the three. This may be reasonable for creating smaller models, but for a base electricity and CO2 model (see the application section) it will be ineffective in the long run. Developing and using
AgentSpring enabled us to build a model that is better maintainable and expandable. 2.2 User interface
AgentSpring is characterized by a web-based user interface. See figure 1 for a snapshot of a running model. AgentSpring runs as a local webserver (typically located at http://localhost:8080/agentspring-face). This setup also allows AgentSpring to be run on a dedicated server that is securely opened up for external visits. The interface allows to start, pause and stop the model, to change and create graphs by writing queries and to observe a textual log. Additionally, the interface can be used to select various predefined scenarios and to change key parameters in the model. A model in AgentSpring can also be controlled from command line, with or without running the AgentSpring user interface.
2.3 The system captured in a database
AgentSpring makes use of a database to contain the state of the modelled system. The modelled system is captured in a so-called graph database, which is a database that uses a graph structure of nodes, edges, and properties to represent and store information (Eifrem, 2009).
The complete state of the system at any point in time is considered a graph of objects and their relationships. AgentSpring allows the graph to scale to hundreds of agents, millions of things and relations between them. The application of a database in ABM is promising as it allows for a different representation of the system modeled: the structure of the system – the objects and their interactions/relations – emerges and evolves. Capturing the data and persisting it in a database, makes it flexible to save and search. It enables efficient selection and finding by performing appropriate queries.
1AgentSpring can be found at https://github.com/alfredas/AgentSpring. At the time of writing, the current
An example of a query could be to find all electricity spot markets for which the property valueOfLostLoad is higher than 500 €/MWh load lost and on which the loadDurationCurve contains at lest 15 segments (see figure 2 for the relational diagram for this example, which is part of the documentation). Traditionally, this would be solved maintaining a list of all spot markets in the model, looping over them and checking piece by piece for both conditions. A query, however, will be easier to compose, shorter in code, and it will be much faster. These advantages become even more relevant when queries span various types of objects. It also allows for thinking differently about extracting information, both for analysis of a running model as well as for the behavior of agents themeselves. An example of a more complicated query would be one that calculates the average efficiency of all PowerPlants that have a PowerGeneratingTechnology that uses fuels emitting CO2, of which the EnergyProducer agent – the owner – has a positive cash balance.
Fig. 2: The graph of possible relations for an electricity spot market, a relatively small example. The grey box is the starting point. Solid arrows refer to an ‘is a’ relationship. Dashed arrows are either property or a relation to another object. In this example an ElectricitySpotMarket is a DecarbonizationMarket, which is a DecarbonizationAgent. An ElectricitySpotMarket has (or can have) a property called valueOfLostLoad, which is a double precision number. It also has the property loadDurationCurve, which is a set of SegmentLoad objects. Graphs like these are part of the documentation (see below).
2.4 Types of classes and other files
AgentSpring uses various types of Java classes and other files.
Domain classes are the definitions of things and their properties. For instance it contains the classes Agent and PowerPlant.
Role classes capture pieces of behavior, such as InvestInPowerPlantRole, that can be executed by specific types (or classes) of Agents (which are in the domain, EnergyProducer in this case). Behavior typically results in new or changed information or objects that are persisted in the database with the help of repositories.
Repository classes contain functions that deal with the interaction of typical model code and the database. For instance, findAllOperationalPowerPlants is a function in the PowerPlantRepository, that executes a query to the database for all power plants, checks which ones are operational (and are not unavailable, under construction or decommissioned), and returns the result. Repositories also assist in updating current information or storing new information.
relations between objects. An example of data is parameters of power plants, and a price trend for coal. An example of a relation that is captured in the scenario is the relation between a market and the substance that can be traded on this market. The coal substance is, therefore, in the scenario connected to a supplying agent, the market and a price trend. One could understand that, by simply defining different substances and connections in the scenario, one could develop very different models, with the same code base: scenarios contain a wide variety of types of data and are a strong element in the development of modular model suites. 2.5 Modelling agents and their behavior
AgentSpring makes use of the concepts of roles to encode agent behavior in a modular way. Agents play their roles in the simulation by executing their coded behavior. Models are made by linking agents to such roles and composing a script that defines the set of behaviors in the context of social situations. This makes AgentSpring particularly suited to modeling complex socio-technical systems. AgentSpring decouples agents, their behaviors and their environments. That enables to reuse the pieces, to compose consistent new pieces. Experience has shown that only modular and reusable models can accommodate changing scope and new research questions.
The roles that make up the behavior of agents have the following properties: A role is enacted by a specific class of agents.
A role encodes a piece of behavior.
Input for roles are the properties of the agent enacting the role, but also other parts of the system. Queries are used to access the graph database and retrieve the information needed for the behavior to be executed.
The outcome of the behavior that is captured in a role implies a change in something in the state of the system. This is then stored in the graph database.
A role can initiate other roles, i.e. a hierarchy of roles can be developed.
Roles do not interact with each other directly (apart from iniating other roles, see the previous bullet).2.6 Development and documentation
Typical software development practices enable version control of the model code through a subversion server (https://svn.eeni.tbm.tudelft.nl/d13n for the model described). This is connected to trac, a
wiki-enabled interface to communicate between developers. Another practice in Java coding is on writing documentation. Online documentation is generated based on the structure of the code and the documentation written as part of the code. See figure 3 for a snapshot of the online documentation2. The documentation is
intuitive enough to find your way and grasp both the structure and details of the model in multiple ways. Links between classes are visualized and linked and for each function a graph is made what other functions it uses and it is used by. The documentation supports modellers and the community around models to
understand and explore the structure of the model. It also enables a platform to think about changes and different scenarios.
2The documentation, generated with Doxygen, is located at
https://svn.eeni.tbm.tudelft.nl/d13n/documentation/doxygen-doc/html/index.html, access can be granted on request.
Fig. 3: Snapshot of the documentation of the model as a website
3 Decarbonization of the power sector: d13n
There is a growing consensus that Europe’s electricity sector must be nearly or completely carbon-free by the middle of this century. This will need to be achieved with a combination of a substantial amount of renewable energy and perhaps nuclear power and/or the use of fossil fuels with carbon capture and sequestration. The current approach is to regulate CO2 emissions through the EU-ETS and provide
additional stimulus for renewable energy. The latter policy is implemented at the national level, as a result of which there is considerable heterogeneity in these policies (although there appears to be a tendency towards feed-in tariffs). In addition, countries have specific policies regarding the use of nuclear fuels, the
combustion of coal and carbon capture and sequestration. The resulting variety of electricity market policies is further compounded by differences in basic electricity market design, for instance with respect to
transmission regulation, congestion management and the balancing mechanism. In order to explore those topics, we developed a basic model of two interconnected European electricity markets. Below, a broad overview is given of the agent-based model – which has been named d13n3. The model contains several
behavioral modules, of which an overview can be found in figure 4. Below, a high-level description is provided. More details are available (cf. de Vries and Chappin, 2012).
agent behavior model engine
medium time scale behavior long time scale behavior
select scenario start initialize simulation selection scenario: agents, behavior, data, exogenous variables, policies CO2 auction and electricity spot market invest in generation capacity dismantle generation capacity clear fuel markets determine dispatch plan
short time scale behavior
determine fuel mix market coupling dispatch generators end simulation finished time controller ready to simulate to analysis progress to dashboard database containing model state sign long-term contracts
Fig. 4: Overview of the behavior in the model and the AgentSpring engine.
3.1 Model description
The model contains two physically connected electricity price-zones (i.e. two European countries). The main agents in the model are the electricity generation companies. In the model, they make decisions about the price at which they sell their electricity, their willingness to pay for CO2 credits and they decide on
investment and disinvestment. The agents purchase fuels at exogenously determined prices, i.e. they are price takers in these markets. The agents base their generation dispatch on the prices of fuel, electricity and CO2, while for their investment decisions they also consider estimates of future prices, the costs of different
generation technologies and, if the modeler desires, other factors (such as risk aversion or a preference for specific generation technologies).
The electricity and CO2 markets are the main arenas in which the agents interact. In order to simulate the
realities of European electricity markets, the model contains multiple electricity markets (two for now) with limited interconnector capacities between them. Furthermore, there is a single CO2 market. The electricity
markets are modelled as power exchanges; they allow for long-term contracts as well. The power exchanges are cleared simultaneously with the CO2 auction. A market coupling algorithm for the allocation of
interconnector capacity is included. An iterative process is used to simulate arbitrage between the electricity and CO2 markets.
When agents construct a new power plant, they can choose from a range of generation technologies (gas, coal, wind, nuclear, etc.). Innovation of these technologies is simulated as a gradual decline of costs and improvement of performance (such as fuel efficiencies). To the extent possible, these trends have been calibrated with empirical data. Established technologies, such as gas, coal and nuclear power, develop more slowly than newer technologies such as wind energy.
The model has been developed to test (combinations of) carbon policies and renewable energy policies in interconnected markets, given different assumptions regarding investment behavior. The baseline carbon policy is an emissions trade scheme like exists in the European Union. A minimum price can be included in this scheme. Instead, or complimentarily, a carbon tax can be implemented. Renewable energy policy instruments can be added. As relative prices change, the agents’ preference for generation technologies shifts. The key question is which sets of policies lead to the desired levels of CO2 abatement and how can
costs most likely be minimized, given the range of scenarios. 3.2 Modular behavior
The behavior of agents is modular although the behavior is not completely independent. The advantage of this setup is that each of the modules can in principle be exchanged by a different version (with different behavior). Therefore, it is possible to develop new experiments by expanding the model in one of the many possible directions. When developing it as is, many of the likely development trajectories have been taken into account. A short description of each behavioral part is given.
Sign long-term contracts. Energy producers engage with each other for long-term contracts. Producers make offers for non-intermittent power plants, for different durations (1, 5 and 10 years) and for different parts of the load (base, peak, etc.). The consumer in a market makes a selection based on the offers. A small preference for longer durations is included and its willingness to pay is affected by recent spot prices.
Invest. Energy producers invest based on an expected net present value (NPV) of each of the available options. In this NPV, the expected level of production (and costs and revenues) is assessed by clearing a hypothetical future spot market.
Dismantle. Energy producers dismantle capacity if the technical lifetime has passed or if it is making a continuous marginal loss.
Commit to long-term contracts. Depending on actual developments, energy producers determine which power plants they use to commit to the long-term contracts in their portfolio.
CO2 auction and electricity spot market. With an iterative algorithm, varying the CO2 auction price,
both the CO2 auction and electricity spot market of both countries are cleared.
Determine fuel mix. For each of the power plants an appropriate fuel is derived, using linear programming. A minimum fuel purity/quality for biomass in coal-fired power plants is included.
Unit dispatch. Based on the activities on the markets, the energy producers dispatch their units. Market coupling. The two countries are linked through a limited interconnector. If the
interconnector capacity is insufficient (i.e. producers/consumers would like to transfer more than the capacity allows for), the markets are cleared again, with the exact transfer of the interconnector through the market coupling mechanism. The reults is that the spot prices in the countries are different.
3.3 Application agenda
A basic model has been finished, that is, a model in which a basic set of functionality has been implemented and tested. We foresee a range of relevant applications, tackling many different questions regarding policy, supporting ad-hoc policy questions within a European country, or on the level of the European Union. Supporting policy makers, however, requires to apply the model to quite different contexts, depending on the questions at hand. Many more of these questions can be identified, but at least these questions are relevant: What policies are needed in order to guarantee security of supply, both on the short term as well as in
the long run, when intermittent resources become abundant?
A central question is what the combined effect is of different policy instruments (in particular carbon policy and renewable energy policy) upon an electricity market, in isolation and in combination with neighboring markets with different policies.
What happens when two interconnected electricity markets, both participating in the EU-ETS, have different renewable energy policies? What if one of these decides to phase out nuclear power? What are potential cross-border effects of differences in such policies between countries?
What policies need to be in place in order to drive CO2 emissions down at acceptable cost to society?
What would be the effect of a minimum price for CO2? What if this were implemented in only one
country?
Under what conditions can we find a reasonable equilibrium between long-term contract volumes and spot prices, more or less as they are found in reality?
How can the investment algorithm be calculated to provide for a reasonably stable and reasonably sane portfolio management for energy companies. Under what conditions can such an investment strategy survive? What strategies can be used to enter the market and succes, in terms of market share and profitability? What long-term strategy works for power producers to keep up their market shares and profitability, given the (slow) transformation/transition to a renewable energy system, in which prices and system structure are different?
Earlier work has proved that experiments can be designed and executed successfully (cf. Chappin, 2011) for questions as the above. Such questions may be answered by developing base power generation model and designing proper experiments. The challenge is to develop applicable scenarios and extensions that transform the model as it is now into a simulation suite with the capability to tackle those questions.
4 Conclusions and outlook
The modelling platform AgentSpring, which we developed, facilitates the development and use of agent-based models in a modular and structured manner, using state-of-the-art IT development principles and tools. An attractive web-based interface allows for the interaction with policy makers. AgentSpring coordinates the interactions between the model, the database and the user interfaces. A strong aspect of AgenSpring is that it makes it possible to greatly extend the functionality of the scenarios that are used in the simulation and to develop modular pieces of agent behavior.
The platform was applied to a model of interconnected electricity markets, which was developed to simulate the long-term impact of carbon policy. In this model, the scenario data does not only contain time series such as fuel prices and demand growth data, but also the types fuels (which can therefore be changed without
changing the model), the types of electricity generation facilities and their characteristics, policy parameters (such as the CO2 emissions cap and a CO2 tax) and the load-duration curve. It is also possible to load data
from external data sources, such as online databases. This extended functionality of the scenario makes it possible to quickly address different policy questions and different external conditions with the same modelling platform and with the same model core. AgentSpring can also be used to coordinate the runs on the high performance cluster. In the electricity market model, this was used to run 500 Monte Carlo type simulations in parallel. Thus a runtime of, for instance, 15 minutes is not an obstacle to performing thousands of runs.
Besides making models more flexible, the second objective for AgentSpring was to make them more accessible to policy makers and analysts. To this end, an attractive and accessible user interface was developed, with as an important feature that the database can be queried from this interface. This greatly improves the flexibility of the interface, as new output graphs and tables can be created there. However, this feature is not fully developed and requires some programming skills; our intention is to further automate this so policy analysts can use the model (for instance online) themselves to explore different scenarios. A second point of attention is that we would like to make the process of deploying AgentSpring in combination with a model still less technically demanding. Possible extensions of AgentSpring are modules for the participation of human players (replacing some of the agents in the models) and for the representation of geographical maps and infrastructure networks. Finally, we foresee the inclusion of a link to relevant semantic datasources.
Acknowledgements
This work was supported by the Energy Delta Gas Research program, project A1 – Understanding gas sector intra-market and inter-market interactions, by the Knowledge for Climate program, project INCAH – Infrastructure Climate Adaptation in Hotspots and by Climate Strategies, project Decarbonization of the Power Sector.
References
Chappin, E. J. L. 2011. Simulating Energy Transitions, PhD thesis, Delft University of Technology, Delft, the Netherlands. ISBN: 978-90-79787-30-2.
http://chappin.com/ChappinEJL-PhDthesis.pdf
Chmieliauskas, A., Chappin, E., Nikolic, I. Dijkema, G. 2012. New methods in analysis and design of policy instruments, in A. V. Gheorghe (ed.), System of Systems, Intech.
de Vries, L. J. Chappin, E. J. L. 2012. Decarbonization of the power sector, Technical report, Delft University of Technology. forthcoming.
Eifrem, E. 2009. Neo4j - the benefits of graph databases, no: sql (east) .
Farmer, J. D. Foley, D. 2009. The economy needs agent-based modelling, Nature 460(7256): 685–686. Holtz08 Holtz, G., Brugnach, M. Pahl-Wostl, C. 2008. Specifying “regime” – a framework for defining and describing regimes in transition research, Technological Forecasting & Social Change 75(5): 623–643. Johnson, R., Hoeller, J., Arendsen, A. Thomas, R. 2009. Professional Java Development with the Spring Framework, Wiley-India.
Kirman, A. 2009. What’s wrong with modern macroeconomics – the economic crisis is a crisis for economic theory, CESifo Conference Centre, Munich, 6–7 November 2009.
http://www2.econ.iastate.edu/tesfatsi/CrisisInEconTheory.AKirman2009.pdf
smalltalk-80, J. Object Oriented Program. 1(3): 26–49.
Lejour, Veenendaal, Verweij van Leeuwen2006Lejour06 Lejour, A., Veenendaal, P., Verweij, G. van Leeuwen, N. 2006. WorldScan: a Model for International Economic Policy Analysis, CPB Netherlands Bureau for Economic Policy Analysis, The Hague. 111.
Ottens, Franssen, Kroes Van De Poel2006Ottens06 Ottens, M., Franssen, M., Kroes, P. Van De Poel, I. 2006. Modelling infrastructures as socio-technical systems, International Journal of Critical Infrastructures 2(2–3): 133–145.
Schäfer, A. Jacoby, H. D. 2006. Experiments with a hybrid CGE-MARKAL model, Energy Journal 27: 171–177.
