Forecasting design and decision paths in ship design
using the ship-centric Markov decision process model
Austin A. Kanaa,∗
aFaculty of Mechanical, Maritime and Materials Engineering, Delft University of
Technology, Mekelweg 2, 2628 CD, Delft, the Netherlands
Abstract
This paper introduces a decision-making model for forecasting design and de-cision paths in ship design by applying eigenvector analysis to the ship-centric Markov decision process (SC-MDP) model. This paper uses the concept of composite reducible Markov processes to identify various independent design absorbing paths. An absorbing path represents the long term behavior of a temporal decision process. This method identifies the set of absorbing paths by decomposing the process into sets of inherently independent parts and thus also gives insight into the structure and relationships of the decision process. This is done by examining the set of principal eigenvectors. Two metrics are intro-duced. First, the set of principal eigenvectors is used to identify all independent design absorbing paths without the need for full examination of all initial con-ditions. Second, through the use of the Moore-Penrose pseudo-inverse, the set of principal eigenvectors is used to estimate the optimal life cycle strategy of the decision process. A case study is presented involving life cycle planning for ballast water treatment compliance of a notional container ship to show the utility of these methods and metrics.
Keywords: Decision making, ship design, Markov decision process, eigenvector
analysis, ballast water compliance
∗Corresponding author, email: A.A.Kana@tudelft.nl, work performed at University of
1. Introduction
This paper presents an eigenvector approach to the ship-centric Markov de-cision process (SC-MDP) model designed to forecast design and dede-cision paths of maritime engineering design decisions. Specifically, this paper shows how the set of principal eigenvectors stemming from the SC-MDP model can be used as
5
a leading indicator to identify and quantify the set of viable paths the design will converge to in the long term. These paths are defined as the absorbing paths of the process. This method decomposes the decision process into inherently in-dependent parts that then provide insight into the absorbing paths. This paper follows in a series of publications aimed at exploring the applicability of the
SC-10
MDP model to ship design and decision making. This introduction first lays out the problem background that lead to the initial need for the SC-MDP model, and second it highlights previous SC-MDP work to help place this particular paper in context.
Kana et al. (2016b) described many of the problems associated with
mak-15
ing sound maritime engineering design decisions. They discussed how maritime design decisions are inherently sequential in nature and are influenced by un-certainty. The decisions made early in the process can have a disproportional impact on the final design, despite the lack of detailed information that is present early on (DeNucci & Hopman, 2012; Andrews et al., 2006). Uncertainty exists
20
not only in regards to the lack of detailed information, but also in regards to future impacts of these decisions as there are currently no standard metrics for defining the future impact of design decisions or quantifying their costs (ONR, 2011). These decisions inevitably reduce design freedom moving forward and may cause design lock-in (Niese et al., 2015; Mavris & DeLaurentis, 2000). Poor
25
decision making can have negative impacts on the final design (DeNucci & Hop-man, 2012), and can possibly lead to design changes later in the process. The costs of these design changes become exceedingly higher later in the design and life cycle of the vessel (Keane & Tibbitts, 1996). This causes a need for making sound decisions early, even in the face of uncertainty.
These decisions also have relationships and dependencies that may not be immediately obvious to the decision maker. These dependencies may influence other decisions, or they may relate to the interplay between the design problem statement and the design generation (Kana et al., 2016b). For instance, how does the decision to require the installation of a specific ballast water system
35
during the construction of the vessel affect the opportunity to choose a different technology at some later point during its life cycle? This initial decision may be related to a strategic partnership between the company and the vendor, or it may be dependent on available technologies that meet a specific regional regulation. This is particularly challenging when making decisions regarding
40
technologies that may still be under development. Understanding how decisions relate to each other and understanding their future impact on the final design is thus important during the design process and life cycle of the vessel.
Maritime design is also extremely sensitive to externalities, both throughout the design process, and throughout the life cycle of the vessel. These
exter-45
nalities may include upcoming regulations, changing economy, or posturing of economic competitors (Kana & Harrison, 2017). Due to the long time frame and high expense of the ship design and production process, decisions about vessels must be made well in advance without complete knowledge of future developments. Failure to properly navigate this landscape can have significant
50
ramifications for the vessel or economic viability of the company. Being able to simulate various future scenarios to test their impact on design decisions made today could be very beneficial.
To approach this problem of evaluating design decision in the face of tem-poral uncertainty, Niese & Singer (2013) developed the SC-MDP model. The
55
model was originally created to study life cycle decision making in the face of evolving environmental regulations. To do this, the SC-MDP model was devel-oped to generate and analyze time domain ship design data under uncertainty. The SC-MDP model is defined as applying Markov decision processes to ship design and decision making. The model has previously been used to study a
60
has been used to study optimal decision paths for vessel technologies in the face of uncertain environmental policies, such as ballast water treatment compliance (Niese & Singer, 2013), the Energy Efficiency Design Index (EEDI) (Niese et al., 2015), and Emission Control Area (ECA) regulations (Kana et al., 2015). The
65
previous research worked to quantify probabilistically what the best sequence of decisions that one should take to minimize their costs when regulations change. Niese & Singer (2014) also studied the changeability of a vessel design throughout its life cycle in the face of uncertainty external pressures. They introduced new metrics on quantifying when costs are incurred and how much,
70
and when, active management may be necessary for a specific ballast water system on a given ship. Niese et al. (2015) then later analyzed initial ship design alternatives and the presence of design lock-in given uncertain future scenarios. This previous work performed by Niese and his co-authors was based around performing simulations through the SC-MDP model to examine how
75
certain decisions may constrain future opportunities and to discern differences in seemingly similar solutions.
Kana et al. (2016a) recognized that one limitation of these analysis tech-niques is that in many complex situations there are a vast number of possible paths available to the decision maker. Here, simply obtaining the final result
80
does not provide sufficient insight, especially if it not clear how those results were obtained (Klein et al., 2009). In these cases, forecasting specific decision paths to gain an understanding of the structure, relationships, and sensitivities of these decisions may prove to be invaluable when trying to obtain specific results. For instance, how do you filter all the available design and decision
85
options down to only those that are technically and economically viable? Is it possible to do this without full enumeration of all possible design options and all possible initial conditions? For these reasons, Kana et al. (2016a); Kana & Singer (2016) introduced a means to perform eigenvalue analysis to the SC-MDP model. This was done to quantify changes in individual decisions and to
90
forecast the number of independent design and decision paths the process may follow.
This paper extends this work by introducing temporal eigenvector methods to gain a deeper understanding of the driving forces behind the different decision making scenarios, as well as quantifying their differences. To forecast future
im-95
plications of the decision process, this paper discusses the concept of absorbing paths. An absorbing path represents the long term behavior of a non-stationary decision process. More than one absorbing path may exist for the whole de-cision process, each one being dependent on the initial state of the system. These specific initial states of the system are considered the initial conditions of
100
the system in this paper. Sensitivities to initial conditions has been known for decades to be a challenge when studying path dependent systems (Liebowitz & Margolis, 1995). Niese et al. (2015) discussed the importance of identifying the presence of multiple absorbing paths. They discussed that differing absorbing paths may mean that differing decision sequences may be viewed as only locally
105
optimal. They were able to identify the multiple paths via simulation studies. This paper, on the other hand, claims that these differing paths are in fact de-pendent on where the system initially starts. Also, this paper uses eigenvector analysis as a leading indicator metric to identify these multiple absorbing paths without the need for potentially costly simulations and recursive investigation
110
of the initial conditions. By developing a leading indicator metric for the design absorbing paths, the structure and dependencies of the decision process may become more clear.
As no single model can handle all aspects of design decision making or types of marine design vessels, (Andrews, 2016; Seram, 2013; Reich, 1995) this papers
115
helps to provide one unique perspective on approaching this difficult problem. A case study discussing life cycle planning for ballast water treatment compliance is presented to demonstrate the significance of the set of principal eigenvectors in forecasting future scenarios and on identifying various inherently independent design absorbing paths.
2. Methods
The methods presented in this paper involve the following four primary steps. Each step is presented in more detail below.
1. Obtain the decision policy and associated expected utilities by solving the standard ship-centric Markov decision process.
125
2. From the set of decisions, develop a series of representative transition matrices, M, for each decision epoch. The eigenvectors are then generated from M.
3. Identify the absorbing paths of the decision process by decomposing M
into its set of principal eigenvectors. These principal eigenvectors
de-130
fine the absorbing paths. This follows the concept of composite reducible Markov processes.
4. Use the Moore-Penrose pseudo-inverse of M to generate an estimation
for the optimal behavior of the decision process. This step highlights
the relationship between the principal eigenvectors of the system and its
135
physical behavior.
2.1. The Markov Decision process
Markov decision processes are a mathematical model designed to handle dy-namic sequential decision-making problems under uncertainty. They represent uncertain systems, can differentiate actions, and can handle temporal system
140
variations. An MDP consists of a set of states, S, a set of actions, A, a set of probabilities, T , of transitioning between different states, and a set of rewards, R, received after landing in a given state, s, after taking a specific action, a. The objective of an MDP is to identify the decision policy that maximizes the cumulative, long term expected utility of the system. This policy takes into
145
account both the outcomes of current decisions and future opportunities. The expected utility of the MDP can be obtained via Equation 1, known as the Bellman equation, where U is the expected utility, γ is the discount factor, and
s0 is the future state. U (s) = R(s) + γ max a X s0 T (s, a, s0)U (s0) (1)
The decision policy, π, is found by taking the argument of Equation 1, as defined
150
in Equation 2 (Russell & Norvig, 2003).
π(s) = arg max a
X
s0
T (s, a, s0)U (s0) (2)
2.2. Obtaining the Eigenvectors of the MDP
To obtain the set of principal eigenvectors, the decision policy must be con-verted into a series of representative transition matrices, M. In an MDP, each action is represented by an action transition matrix that details the
probabil-155
ity of transitioning between states if a particular action is taken. The decision policy, π(s), identifies the optimal action for each state. To generate M, select the row associated with a given state from its optimal action transition matrix and place it is the associated row in the representative transition matrix. This is repeated for all states. The result is a matrix that is an amalgamation of all
160
action transition matrices, except this new matrix consists only of the optimal actions determined by the MDP and decision policy. This process creates a matrix that is, by definition, square stochastic. In this sense, M can be thought of as a Markov Chain transition matrix. Since the decision process is time de-pendent, there will be a unique M for each discrete time step. Kana & Singer
165
(2016); Sheskin (2011) present examples of this step in detail.
A visual representation of this step is given in Figure 1. Here an example MDP is provided with 4 actions and 6 states. In this visual example, Action 1 is optimal for states 1 and 6, Action 2 is optimal for states 2 and 3, Action
3 is optimal for state 4, and Action 4 is optimal for state 5. The optimal
170
actions are selected from Equation 2, and M is built from the various action
transition matrices. For these transition matrices, pi,jrepresents the probability
however, this step is repeated for each time step of the process, thus creating a new M for each time step.
175
Figure 1: Visual representation of an example MDP that shows how M is generated from a series of action dependent transition matrices.
Eigenvector analysis is then performed on M (Equation 3), where wi are
the eigenvectors and λi are the eigenvalues. Since M is time dependent, the
eigenvalues and eigenvectors are also time dependent. In general, the eigenvalues are not distinct after following this process; however, the dominant eigenvalue will always equal one (Anton & Rorres, 2005). When the dominant eigenvalue
180
is repeated, there are multiple principal eigenvectors for the entire process. To analyze these situations, a discussion of composite reducible Markov processes is necessary.
wiM = λiwi (3)
2.3. Composite Reducible Markov Processes
Kana et al. (2016a) present an initial discussion on the use of composite
185
reducible Markov process applied to ship design and decision making. They use the set of dominant eigenvalues to quantify the number of unique viable design paths. This paper extends those methods to include two new analy-sis techniques. The first discusses the use of the set of principal eigenvectors
to group specific aspects of the design and decision process together. This
190
technique identifies the specific decisions and aspects of the design which are inherently independent from each other. The physical meaning of this is that this technique decomposes the one decision process into multiple independent decision processes. This method identifies the specific absorbing paths. The second technique uses the set of principal eigenvectors to estimate the optimal
195
life cycle behavior through time, and is discussed in the following subsection. Reducible Markov processes are defined by closed and transient states. Tran-sient states are those in which the process may pass into and through, but will not remain in for the long term. Closed states are those in which that once the process enters those sets of states, it will never leave. These closed states are
200
where is the system will converge to in the long term (Gebali, 2008). Compos-ite reducible Markov processes are those where there are more than one set of closed states for the process (see Figure 2).
Figure 2: Visual depiction of a reducible Markov process with two sets of closed states (from Gebali (2008)).
The set of closed and transient states can be identified by examining the set of principal eigenvectors (Theorem 1). The proof of the theorem can be found
205
in Gebali (2008).
Theorem 1: Let M be the transition matrix of a reducible Markov chain whose eigenvalue λ = 1 corresponds to an eigenvector w. The closed states of the chain correspond to the nonzero elements of w and the transient states of the chain correspond to the zero elements
of w.
Essentially, the nonzero elements of each individual principal eigenvector represents one particular steady state distribution. The specific absorbing path that the system ends up in is dependent on which specific state the system starts in. This is what this paper means by initial condition dependence. By
associat-215
ing the absorbing paths with the closed states of the Markov process, Theorem 1 can be used to justify the use of the principal eigenvectors in identifying long term absorbing paths of the decision process. For each decision epoch, there are a given number of principal eigenvectors equal to the number of dominant eigenvalues for that decision epoch. Each eigenvector is analyzed separately,
220
as each one represents one independent absorbing path. For ship design, this information may inform the decision maker that certain design or decision paths may not be reachable given a specific starting state. In situations such as these the designer needs to be very careful in how they select their starting state.
Traditional analysis using these techniques involve stationary Markov
pro-225
cesses, meaning the transition probabilities do not change with time. This paper extends those studies to examine their applicability to non-stationary processes. By studying non-stationary processes with this method, the decision maker is able to gain insight into not only the instantaneous impact of their decisions on future absorbing paths, but they can also gain an understanding of how those
230
absorbing paths may change and evolve over time.
2.4. Using the Principal Eigenvectors to Estimate the State Vector
The other technique proposed involves using the set of principal eigenvectors to estimate the optimal behavior of the system. This method is used to high-light the relationship between the set of principal eigenvectors and the physical
235
behavior of the system. Mathematically, the objective is to combine the set of principal eigenvectors in such a manner that it estimates the behavior of the
state vector as close as possible. The state vector, st, is a vector that gives the
the elements in stmust equal one. The state vector can be used to describe the
240
behavior of the system through time. To calculate st, simply multiply it by Mt,
as given is Equation 4. s0M0= s1 s1M1= s2 s2M2= s3 .. . (4)
The goal of this step is to minimize the difference between the state vector,
st, and set of principal eigenvectors, [w]. This is given in Equation 5, where α
is a set of scaling factors for the principal eigenvectors. Equations 5, 6, and 7
245
as presented are for a single time step.
min||s − [w] ∗ α|| (5)
To solve for α, set the parameters equal to each other (Equation 6) and solve
for α using the pseudo-inverse of [w] (Equation 7). Here [w]† is the
Moore-Penrose pseudo-inverse of [w]. This is a generalized form of the inverse that can handle non-square matrices. Details on how to calculate the Moore-Penrose
250
pseudo-inverse can be found in Bishop (2009). Again, this step is repeated for
each time step, thus estimating each stthrough time.
s = [w] ∗ α (6)
α = [w]†∗ s (7)
This method finds the α that minimizes this distance. The pseudo-inverse is necessary because [w] is not square. If [w] were square, the actual inverse could be used and α would be able to solve this equation exactly. However, since the
pseudo-inverse is used, this technique is only an estimation. The following case study is given to show an application of this method on a specific ship design problem.
3. Case Study: Life Cycle Planning for Ballast Water Treatment
Compliance
260
A case study is presented that involves life cycle planning for ballast water treatment compliance in the face of uncertain future scenarios. This case is an extension of the one presented in Kana et al. (2016a); Niese & Singer (2014, 2013). As such, only a brief description of the case setup is presented, as the full details are left to those other references. The focus of the study in this
265
paper is on the new methods and metrics that focus on identifying the specific absorbing paths (Subsection 4.1), and also on relating the eigenvector spectrum of the system with the physical life cycle behavior of the system (Subsection 4.2). These new methods provide insight into the selection process of a ballast water system on a given container ship.
270
For ship owners and operators, life cycle planning for ballast water treat-ment compliance and the decisions behind which technology to select remains difficult due to the interplay of stochastic degradation, technology development, and multiple levels of environmental policy-making (Kana et al., 2016a). Mul-tiple governing bodies regulate ballast water, many times with varying strength
275
between geographic regions and with various implementation schedules. Also, there are a vast number of possible ballast water systems, either available to-day or under development which vary across a range of treatment technologies (Lloyd’s Register, 2012). Not only will these technologies differ in terms of tech-nology readiness and regulatory compliance, but they will also vary in terms of
280
deterioration and life cycle performance. These factors make the planning and selection of proper technologies complicated and difficult. Understanding the viable decision paths, while taking into account the various inter-dependencies and temporal uncertainties should aid decision makers navigating this difficult
landscape. The following case study aims to explore some of these factors using
285
the SC-MDP model because of it ability to handle uncertain, temporal decision making problems in the maritime domain.
For this case study, a notional 150,000 DWT container ship with a 20 year
life span is considered. It is routed on the trans-pacific route. The vessel
has been placed in service prior to the 2004 IMO Ballast Water Management
290
(BWM) Convention, and at some point during its life cycle the regulation will come into force and a decision must be made regarding which ballast water technology to select. There are 10 ballast water systems under consideration that vary in terms of performance, cost, availability, and regulatory approval. System 1 is a commercially available exchange system already installed, while
295
systems 2-10 represent various ballast water treatment systems that will become available sometime during the lifespan of the vessel. The original case setup, including inputs, stochastic variables, and economic parameters have all been tested and validated by the previous publications on this specific case study. The scenario was modeled as an MDP, with the following states, actions, transition
300
probabilities, and rewards.
3.1. The Markov decision process
The following section details how the states, actions, transition probabilities, and rewards are defined for this case study.
3.1.1. States
305
The states consist of the ten ballast water systems, their deterioration, and their availability. Each system has six availability states: unavailable, commer-cially available, basic approval, final approval-Tier 1, final approval-Tier 2, and final approval-Tier 3. Tier 2 regulations are roughly 10x more stringent than Tier-1, and Tier 3 regulations are roughly 100x more stringent than Tier-1.
310
Each system may be in one of four deterioration levels, defined as the percent-age of total deterioration: 0%, 33%, 66%, and 100%. This leads to 240 total
states, each one defined as system availability deterioration (Niese & Singer, 2013).
3.1.2. Starting State
315
The process is assumed to be dependent of the specific state the systems starts in, meaning there is initial condition dependence. For this paper, the model assumes equal probability of starting in any given state. The [1x240]
element initial state vector, s0, is given in Equation 8. The objective is to let
the model choose the most natural path for the design process to follow, as
320
opposed to pre-determining its trajectory. The problem with, and importance of, selecting the appropriate starting state was discussed by Niese et al. (2015), who studied the starting states’ implications on future design realizations.
s0= [ 1 240, 1 240, ... 1 240] (8) 3.1.3. Actions
There are 12 actions which include: take no action, perform maintenance on
325
the installed system, or replace the system with another ballast system. Each action is represented by its own action transition matrix. This results is 12 independent [240 x 240] matrices.
3.1.4. Transition Probabilities
The transition probabilities are defined as follows:
330
• Transitioning between ballast systems is deterministic and is selected based on the decision policy calculated by the SC-MDP model (Equation 2). • Transitioning between approval states is based both on the regulatory
environment and the commercial availability. It is assumed the BWM convention is held one year after the ship enters service. Five years after
335
the ship enters service, testing procedures regarding treatment efficacy are available. The same year the legislation is ratified by member States. It
is also assumed that the legislation enters force nine years after the con-vention. Development of the treatment technologies will begin after the BWM convention is held. Also, each system will meet a different
thresh-340
old of regulatory compliance and will become available at different times. Figure 3 shows the availability of the various systems according to both commercial availability and regulatory compliance. For this figure, the deterioration states for each system and availability have been combined. This results in the 60 representative states that are presented in the
fig-345
ure. This was done for two reasons. First, visually it is easier to show 60 states as opposed to 240 states, and second, from a decision perspective, the ballast systems themselves and their availability are the most impor-tant. The deterioration is important for the MDP model, but less so for visualizing which systems are available and when.
350
• Transitioning between deterioration levels is modeled by an exponential distribution, and varies depending on the treatment method. The specifics of this distribution are described by Niese & Singer (2013).
3.1.5. Rewards
The objective is to minimize costs (Equation 9). These costs vary between
355
ballast water systems and they increase as the systems deteriorate through time. The costs are calculated for each time step and the objective is to minimize to total accumulated life cycle costs. The specific cost functions of each system and their deterioration behavior is given in Niese & Singer (2013).
cost = min(captial + install + operating + maintenance) (9)
3.2. Obtaining the Eigenvectors of the MDP
360
The MDP is run for 20 time steps, simulating a 20 year life cycle of the vessel. The optimal decision policy was solved for each time step using Equation 2. From the decision policy, the individual states and their respective optimal
Time 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 % of states
Figure 3: Ballast water system commercial availability and regulatory compliance. Shading represents the percent likelihood a system will be located in that specific state.
This process resulted in 20 M matrices, one for each year, and each one [240 x
365
240] in size. As it is hard to visualize the specifics of all of these matrices, due to their size and number, the reader is referred to Figure 1 to see this step for an example MDP. From here, the principal eigenvectors were then calculated from this set of M matrices. This resulted in 20 sets of principal eigenvectors. Each time step was treated independently, and thus analysis could then be performed
370
4. Results
Two sets of results are presented. First, the set of principal eigenvectors was examined to study the design absorbing paths. Second, the relationship between the principal eigenvectors of the MDP and the temporal behavior is discussed
375
using the Moore-Penrose pseudo-inverse technique presented previously in the Methods section.
4.1. Analysis of the Set of Absorbing Paths
The eigenvectors associated with the set of dominant eigenvalues were exam-ined as a leading indicator metric for forecasting all possible absorbing paths.
380
For the following figures, each column visually shows the values of one particular principal eigenvector. Again, the 240 states have been reduced to 60 representa-tive states by combining the four deterioration states. As stated in Theorem 1, the non-zero elements of the principal eigenvectors represent the closed states of the process, which means they represent the long term behavior of the decision
385
process. The zero elements represent the transient states, which means that the system will not end up in those states in the long term. For instance, if princi-pal eigenvector i for decision epoch t displays System j, this means that System j represents the only long term design that the decision process will converge towards for a specific starting state. Each eigenvector represents a different,
390
independent absorbing path. A sensitivity study was performed varying the affect of the strength of the regulation as to its affect on the absorbing paths.
Figure 4 shows the results for Tier 1 strength. While this process was done for the entire life cycle, years 1, 5, 8, and 20 are presented to represent the temporal variations of the absorbing paths. For year 1, the only path the design
395
will follow involves installing System 1 in the long run. For year 1, System 1 is the only system that is available. This trend continues until year 5 when two paths become apparent: one for System 1 and one for System 3. Even though System 3 may become commercially available in year 3 (Figure 3), it does not become a viable path until year 5. This shows a time lag between
system availability and system viability. As the number of paths increase, the number of ballast systems that become viable increases. For year 8, six different paths are possible, representing Systems 1, 2, 3, 5, 9, and 10 being viable options in the long run. For year 20, all ballast systems become viable except ballast System 1, which became unavailable due to regulatory requirements in year 10.
405 Absorbing paths for year 1 1 indexed principal eigenvector System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval Absorbing paths for year 5 1 2 indexed principal eigenvector System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval Absorbing paths for year 8 1 2 3 4 5 6 indexed principal eigenvector System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval Absorbing paths for year 20 1 2 3 4 5 6 7 8 9 indexed principal eigenvector System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 % of states
Figure 4: The set of principal eigenvectors representing the various independent absorbing paths: Tier 1 regulatory strength. Notice that the number of possible unique paths increases through time.
The set of long term absorbing paths changes when the regulatory strength changes. The results for a Tier 3 regulatory strength are given in Figure 5. Through year 5, the results are similar to that of the Tier 1 policy. However, for year 8, only five paths are identified, one fewer than for Tier 1. Ballast System 5 is no longer a viable option for the Tier 3 policy. Even though the
410
Tier 3 regulation does not come into force until year 10, this analysis projects two years prior that ballast System 5 will not be viable in the long run. This instance shows how this method is a leading indicator forecasting future design
opportunities. This is a subtle, but significant contribution of this method. For year 20, the only ballast systems that are viable represent those that meet Tier
415 3 requirements. Absorbing paths for year 1 1 indexed principal eigenvector System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval Absorbing paths for year 5 1 2 indexed principal eigenvector System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval Absorbing paths for year 8 1 2 3 4 5 indexed principal eigenvector System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval Absorbing paths for year 20 1 2 3 4 5 indexed principle eigenvector System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 % of states
Figure 5: The set of principal eigenvectors representing the various independent absorbing paths: Tier 3 regulatory strength. Notice the number of unique paths for year 20 is less than it is for the Tier 1 policy, due to many ballast systems not being regulatory compliant.
Next, a study was performed to show that the different absorbing paths identified using the principal eigenvectors match the behavior of the decision process when the starting states are varied. To show this, the starting state was changed so that at a given year there was equal probability of landing in
420
any state. The model was then run to see how the process evolves through time given this new set of conditions. Year 8 was chosen for this validation study. Thus, at year 8, the system is run assuming that the prior year there is equal probability of being in any state. This is different from the original analysis where the process was started at year 1.
Time 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval 0 0.1 0.2 0.3 0.4 0.5 0.6 % of states
(a) Varying initial starting states and running time domain simulation
Absorbing states for year 8 1 2 3 4 5 6 indexed principle eigenvector System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 % of states
(b) Using the set of prin-cipal eigenvectors
Figure 6: The absorbing paths identified using the set of principal eigenvectors is the same as those by varying the starting states: Tier 1 strength.
Figure 6 shows the results for Tier 1 regulatory strength. For this case there are six different absorbing paths identified by the set of principal eigenvectors. When the starting states for year 8 are changed so that there is equal probability of being in each state those same six paths can be identified using the state vector. The probability of landing in one absorbing path over another, however,
430
is not equal. For example, it is more likely that System 9 will be the preferred choice over System 2, 3, 5, or 10. System 1 appears as a long term absorbing path in the eigenvectors even though System 1 is not viable for the whole lifespan of the vessel. Since these eigenvectors represent an instantaneous snapshot of how the design may progress, it is unaware that shortly there after the regulation
will change, making System 1 not available. Time 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 % of states
(a) Varying initial starting state and running time domain simulation
Absorbing states for year 8 1 2 3 4 5 indexed principal eigenvector System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 % of states
(b) Using the set of prin-cipal eigenvectors
Figure 7: The absorbing paths identified using the set of principal eigenvectors is the same as those by varying the starting states: Tier 3 strength.
Figure 7 shows how the set of possible absorbing paths change when the regulatory strength is changed to Tier 3. Only five different absorbing paths are identifiable for this regulatory strength. Unlike the situation with Tier 1, System 5 is no longer a long term possibility. System 9, while clearly evident
440
using eigenvector analysis, is barely visible when manually varying the initial conditions. There is a very small possibility it will be selected in the long run. Also, similar to System 1, System 9 becomes unavailable at year 9 when the regulation changes. This study showed that the eigenvectors do represent the
various possible absorbing paths the design may follow, and that these paths
445
are dependent of the which states the systems starts in.
4.2. Relationship between Optimal States and Principal Eigenvectors
As discussed above, the set of principal eigenvectors can be used to estimate the non-stationary behavior of the system through time using the Moore-Penrose pseudo-inverse. Figure 8 displays the optimal behavior of the system calculated
450
both using the state vector (left) and using the pseudo-invere estimation method from the set of principal eigenvectors (right). The plot on the left is an optimal states plot, which shows the best decision path to follow according to the MDP model. This metric is described in detail in Kana et al. (2016a, 2015). This paper only shows the results for Tier 1 compliance, however, the results are
455
consistent for all regulatory and other scenarios. As shown, combining the
eigenvectors in this fashion provide a close approximation of the decision paths. Even though the state vector was used to help develop this estimation, this principal eigenvector re-composition of the best decision path highlights the relationship between principal eigenvectors and the physical decision process.
460
5. Discussion and Conclusion
The results presented in this paper can be useful for ship designers and decision makers for several reasons. First, the eigenvector techniques presented provide one unique perspective into forecasting independent long term design absorbing paths. This helps give insight into the structure of the decision process,
espe-465
cially in regards to inherent independencies within the process. Understanding how the independent absorbing paths change and evolve throughout the life cy-cle of the vessel provides ship designers with new information to help base their decisions today in the face of uncertain future scenarios. Second, the eigenvector methods are inherently a leading indicator highlighting the impact of decision
470
making. The eigenvector methods presented represent the long term absorbing paths the design may follow without the need for full time domain simulations
Time 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 % of states
(a) Optimal lifecycle strategy according to standard SC-MDP model. Time 1 2 3 4 5 6 7 8 9 1011121314151617181920 System1 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System2 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System3 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System4 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System5 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System6 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System7 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System8 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System9 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval System10 Unavailable Available Basic Approval Tier1 Approval Tier2 Approval Tier3 Approval 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 % of states
(b) Estimation using the set of princi-pal eigenvectors and the Moore-Penrose pseudo-inverse.
Figure 8: Lifecycle decision paths as determined by the two methods for Tier 1 compliance.
and exhaustive perturbation of all starting conditions. The focus of this paper was less on what the one final design is, but instead this analysis has focused on the paths that lead the decision process to that point, and identifying the
475
set of viable decision or life cycle paths that a design or vessel may follow. A few limitations of the method should be mentioned, however. While the principal eigenvectors can identify that a system may be dependent on where the system starts, they do not immediately show which starting conditions may lead to which absorbing paths. Also, quantifying the probability that the process will
480
follow one specific absorbing path over another is not immediately clear using these techniques. However, despite these limitations, these methods and metrics
still provide value, and addressing these limitations has been identified as an area of future work.
To conclude, this paper presented a method for applying principal
eigen-485
vector analysis to the SC-MDP model for a temporal, non-stationary problem involving ballast water system considerations in the face of changing environ-mental policies. The set of principal eigenvectors were used to identify various independent absorbing paths and processes within the SC-MDP model. Insights were gained that show the full set of viable ballast water technology options
490
available for the vessel throughout its life cycle, taking into account uncertainty in technology availability and deterioration. The strength of the regulation had an effect on which systems were viable, and when they were to become viable. Eigenvector methods provided new insights into this difficult decision problem that may be difficult to discern using traditional analysis techniques.
495
Acknowledgments
The authors would like to thank Ms. Kelly Cooper and the U.S. Office of Naval Research for their support of this research.
References
Andrews, D. (2016). Does one size fit all? Or do different warship designs require
500
different ship design methods? In Warship 2016: Advanced Technologies in Naval Design, Construction, and Operation (pp. 1–20). Bath, UK: The Royal Institution of Naval Architects. ISBN: 978-1-909024-55-7.
Andrews, D., Keane, R., Lamb, T., Sen, P., & Vassalos, D. (2006). State of the art report: Design methodology. In 9th International Marine Design
505
Conference (IMDC 2006). Ann Arbor, USA.
Anton, H., & Rorres, C. (2005). Elementary Linear Algebra: Applications Ver-sion. (Ninth ed.). Hobokoen, NJ: Wiley.
Bishop, C. M. (2009). Pattern Recognition and Machine Learning . USA: Springer.
510
DeNucci, T., & Hopman, H. (2012). Capturing configuration rationale in com-plex ship design: Methodology and test case results. In 11th International Marine Design Conference (IMDC 2012) (pp. 233–250). Glasgow, UK vol-ume 1.
Gebali, F. (2008). Analysis of computer and communication networks. chapter
515
Reducible Markov Chains. Victoria, B.C.: Springer.
Kana, A. A., Brefort, D. C., Seyffert, H. C., & Singer, D. J. (2016a). A decision-making framework for planning lifecycle ballast water treatment compliance. In 13th International Symposium on Practical Design of Ships and Other Floating Structures (PRADS’2016). Copenhagen, Denmark.
520
Kana, A. A., & Harrison, B. M. (2017). A Monte Carlo approach to the ship-centric Markov decision process framework for analyzing desgin decisions un-der uncertainty. Ocean Engineering , 130 , 40–48.
Kana, A. A., Knight, J. T., Sypniewski, M. J., & Singer, D. J. (2015). A Markov decision process framework for analyzing LNG as fuel in the face of
525
uncertainty. In 12th International Marine Design Conference (IMDC 2015) (pp. 297–308). Tokyo, Japan volume 2. ISBN: 978-4-930966-04-9.
Kana, A. A., Shields, C. P. F., & Singer, D. J. (2016b). Why is naval design
decision-making so difficult? In Warship 2016: Advanced Technologies in
Naval Design, Construction, and Operation (pp. 27–33). Bath, UK: The
530
Royal Institution of Naval Architects. ISBN: 978-1-909024-55-7.
Kana, A. A., & Singer, D. J. (2016). A ship egress analysis method using spectral Markov decision processes. In 13th International Symposium on Practical Design of Ships and Other Floating Structures (PRADS’2016). Copenhagen, Denmark.
Keane, R., & Tibbitts, B. (1996). A revolution in warship design: Navy-industry integrated product teams. Journal of Ship Production, 12 , 254–268.
Klein, G. L., Pfaff, M., & Drury, J. L. (2009). Supporting a robust decision space. In AAAI Spring Symposium - Technical Report (pp. 66–71). volume SS-09-09.
540
Liebowitz, S. J., & Margolis, S. E. (1995). Path dependence, lock-in, and history. Journal of Law, Economics and Organization, 11 , 205–226.
Lloyd’s Register (2012). Ballast Water Treatement Technologies and Current System Availability. Technical Report Lloyd’s Register.
Mavris, D. N., & DeLaurentis, D. (2000). Methodology for examining the
si-545
multaneous impact of requirements, vehicle characteristics, and technologies on military aircraft design. In Proceedings of the 22nd Congress of the In-ternational Council on the Aeronautical Sciences (ICAS) (pp. 145.1–145.10). Harrogate, UK.
Niese, N. D., Kana, A. A., & Singer, D. J. (2015). Ship design evaluation subject
550
to carbon emission policymaking using a Markov decision process framework. Ocean Engineering , 106 , 371–385.
Niese, N. D., & Singer, D. J. (2013). Strategic life cycle decision-making for the management of complex systems subject to uncertain environmental policy. Ocean Engineering , 72 , 365–374.
555
Niese, N. D., & Singer, D. J. (2014). Assessing changeability under uncertain exogenous disturbance. Research in Engineering Design, 25 , 241–258. ONR (2011). ONR BAA 11-022 Assessing total ownership cost . Technical
Report U.S. Department of Navy Office of Naval Research.
Reich, Y. (1995). A critical review of general design theory. Research in
Engi-560
Russell, S., & Norvig, P. (2003). Artificial Intelligence: A Modern Approach. (2nd ed.). Upper Saddle River, NJ: Prentice Hall.
Seram, N. (2013). Decision making in product development - a review of the literature. International Journal of Engineering and Applied Sciences, 2 ,
565
1–11.
Sheskin, T. J. (2011). Markov Chains and Decision Processes for Engineers and Managers. New York: CRC Press.
