OPTIMIZING THE PROCESS TO FORECAST DEMAND BY MERGING EXPERTS AND MODELS

The author is grateful to Dr. W.W.A. Beelaerts van Blokland for his assistance regarding operation managerial issues and comments on earlier drafts.

O

PTIMIZING THE PROCESS TO FORECAST DEMAND

BY MERGING EXPERTS AND MODELS

E.T. Labrujere (1366602)

ME2110-10: Literature Assignment

Faculty of Mechanical Engineering, Technical University of Delft

3 February 2014

| Summary 2

SUMMARY

Demand forecasting is a crucial aspect of the planning process in supply-chain companies. The most common approach to forecast the demand at manufactures involves the use of computerized forecasting systems to produce the first forecasts. Subsequently these forecasts will be judgmentally adjusted by the company´s experts of demand planning to take into account exceptional circumstances expected over the planning horizon. Making these adjustments is necessary to increase the accuracy of the final forecast. But when the forecast has to be done in a general situation, the model forecast performs better than after the adjustment of the expert. In order to investigate what changes should be done to the current forecasting process existing of a model and an expert, over 25 papers are criticized. Seven different improvements are subtracted from these papers and discussed in detail. In order to implement these improvements, the current process is analyzed and suggestions based on previous research are given. The paper ends with one suggestion for a new lay-out of the forecasting process that takes all the earlier discussed improvements into account. Further research to evaluate this suggested model is recommended and some first steps to do this are explained.

| Introduction 3

1 INTRODUCTION

Logistics and supply chain management is not new to the world. In the ancient history the building of the pyramids and the temples of the Romans were realized by this type of management. It is in the recent past that organizations have become aware of the importance of studying this chain of interaction. In Figure 1 a typical supply chain is schematized to provide insight in the flows of information and material among the different elements of the supply chain.

The small black arrows show the flow of information, whereas the thick open arrows show the flow of material/products.

Raw Material

Supplier Manufacturer Distributer Retailer

C u st o m er

Figure 1: The supply chain

In order to be able to deliver enough products to the customer (indicated by the open arrow between retailer and customer) while maximizing profit, it is desirable to have a matching demand and capacity. The green lines from customer to retailer illustrate the demand of the different customers. The green arrow between the retailer and the distributor illustrate the information that is used to determine the level of replenishment needed in order to have enough capacity. The green line from the distributor to the manufacturer illustrates the information that includes the order of the distributor. The green line from the manufacturer to the raw material supplier indicates the order of raw material. All the information flows are dependent of the customer demand, which is unknown in the specific case of retailers. The prediction of the demand is called demand forecasting, and could be seen as optimization of the information flows. Demand forecasting is fundamental to an organization. Without forecasts operations can only respond retroactively, leading to lost orders, inadequate service and poorly utilized production resources [1]. When the demand forecast is over- optimistic, too much stock will lead to extra warehousing costs [2].

An example where demand forecast is essential to the success of the organization is at Procter & Gamble (P&G). P&G is a multinational consumer goods manufacturer with its headquarters in the United States. With over 300 brands and tens of thousands of suppliers in 140 countries in the world [3], demand forecasting is an extremely important and difficult task. In order to meet customers demand in each country, it is important to estimate the amount of products that will be sold. This forecasting task is difficult due to the inter-related nature of the available information, the presence of outliers, level and trend shifts and the impacts of the market and general economic environment, stated Fildes & Beard in 1992 [4].

In order to plan and manage the supply chain, organizations typically set up a unit responsible for forecasting the demand. The most common approach to forecasting demand in support of supply chain planning involves the use of a statistical software system which incorporates a simple unvaried forecasting method, such as exponential smoothing, to produce a statistical

| Introduction 4 forecast. These forecasts will hereafter be called the model forecasts. The forecasts produced by the software system are reviewed, and may be adjusted by the company’s experts to take into account exceptional circumstances expected over the planning horizon, or possibly to correct perceived inadequacies in the system forecast. A forecast done by a company’s expert will be called experts’ forecast.

Regardless of industry, or whether the company is a manufacturer, wholesaler, retailer, or service provider, effective demand forecasting helps organizations identify market opportunities, enhance channel relationships, increase customer satisfaction, reduce inventory investment, eliminate product obsolescence, improve distribution operations, schedule more efficient production, and anticipate future financial and capital requirements [5].

To evaluate a forecasting method, the accuracy of a forecast has to be determined. Forecast accuracies in the supply chain are typically measured using the Mean Absolute Percent Error or MAPE [6]. Statistically MAPE is defined as the average of percentage errors of a method and the goal of forecasting is to decrease this percentage.

Extensive research is done on forecasting and the adjustments done by the experts, where the focus has been on effects of the types of judgmental adjustment on the accuracy. However little is known about implementations possible to improve the adjustment of the expert while taking the model-based forecast into account. This paper gives an overview of the process used to judgmentally adjust a model-based statistical forecast. In the first section an overview of past research is given. After that the process to establish a forecast typical for manufacturer like P&G is described and divided into three subjects; the process to form a model-based forecast, the process where the expert adds knowledge and the process to merge the model with the expert knowledge to a final forecast. In the 4th _{section the theoretical improvements derived by} research are listed. In the 5th _{chapter possible practical implementations are found and} discussed. Also one final suggestion is done for a model that strives to achieve all the improvements as listed in chapter 4. In the last and final section (chapter 6) a conclusion is drawn and recommendations for further research are given.

| Literature review and hypotheses 5

2 LITERATURE REVIEW AND HYPOTHESES

Combining the model-forecasts with the expert’s knowledge began when Schultz started examining the organizational factors leading to a successful implementation of a new forecasting method in 1984 [7]. He found out that an improvement in forecasts could be reached if managers were included in the derivation of a forecasting process. Schultz let the basis for many research to follow.

In 2002 the need for interference of managers in forecasting was examined by Goodwin [8]. He emphasized the shortcomings of statistical methods. He states that if there is fast changing relevant past data available, the statistical methods that extrapolate past patterns into the future are too conservative to react to this fast changes. He also states that past data have to be ‘massaged’ because it can contain unusual events. As one of the biggest deficiencies of statistical methods, he mentions the difficulty of taking into account special events that are known to be occurring in the future.

In 2006 a review of the past 25 years is written [9]. It is reported that there has been a phenomenal growth of interest in judgmental approaches to forecasting and a significant change in attitude of the part of researchers to the role of judgment. Where judgmental adjustments were seen as the enemy of accuracy, the desire to learn how to blend judgment with statistical models to estimate the most accurate forecasts is the new trend. Actually Franses states that experts will always be tempted to adjust model-based forecasts, and most do so often [10]. He saw that model-based forecasts are rarely taken as the final forecast. In 2009 Franses and Legerstee [11] state that experts adjust model-based forecast in 90% of the cases.

In 2007 [12] Stekler addresses the point that ‘while we have been well aware of all of these judgmental elements in the forecasting process, they are rarely addressed or even analyzed’. In 2008 [10] Franses addresses the point that the modeler and the expert are typically not one and the same person and that it would be interesting to examine the relation between the modeler and the expert.

Additionally, now it is clear from literature that combining models with experts leads to improvement in forecasting Nikolopoulos et Al. evaluated the benefits of the intervention of managers to the final forecasts by combining the actual sales to both the system and final forecast [13]. The findings support the case that adjustments do improve accuracy.

Since 1990 there has been a lot of research on how to improve this combined forecast. Blattberg and Hoch focused on ways to combine the statistical model with the expert’s intuition; a 50% model + 50% experts approach [14]. The results of Franses’ and Legerstee’s research [15] indicate that experts have too strong a tendency to downplay the contribution of the model and put too much weight on their own contribution. And these are only a few examples of the difficulties of combined forecasting.

A lot of improvements are suggested in literature, but little is known about how to implement these improvements into the forecasting design. Therefore this report focuses on ways to meld the improvements found in literature into the forecasting design in order to form a better final demand forecast. This is done by looking at the lay-out of the process and examining a mathematical solution. The literature question of this paper is; ‘What improvements can increase

| Literature review and hypotheses 6

that the final forecast consists of the combination of the input of a statistical model and an expert?’.

And additionally to this suggestions will be done in order to solve the sub question; ‘What could

| Working principles of Forecasting 7

3 WORKING PRINCIPLES OF FORECASTING

In this chapter a description of forecasting is given, in order to understand how to forecast demand. In the first section a basic understanding of forecasting is provided. In the second section a description of the principles of a model forecast are given in case of forecasting the demand. In the third section the input of experts in the forecasting process is explained. This chapter ends with a description of the process used in practice to form a final demand forecast. The final forecast is a combination of the model and the input of an expert.

3.1 I

Forecasting is a process of forming an estimation of the outcomes of events that did not take place. The outcomes of these events have not been observed, and a method is used to get to an estimation of the outcome of these events. When specifying this for demand forecasting; demand forecasting is a process of forming an estimation of the demand for the coming period.

The ability to form good forecasts has been highly valued throughout history. Since a forecast involves uncertain future events, the forecasts are usually not perfect. The objective of forecasting is to reduce the forecast error; to produce forecasts that are seldom incorrect and that have small deviations.

A general overview of a process of forecasting used for demand planning is shown in Figure 2. In the process past data is used as an input to form a forecast that forms the baseline for planning the coming demand. The past data exists of the previous items sold to the customer and other data; historical as well as non historical. When the forecast is done, the accuracy is evaluated by calculating the error of the forecast as a function of the final forecast and the actual sales. The forecasting process issubject to requirements set by the company.

Forecast

Performance Requirements

Data:

sold items last year # customers historical data non historical data

Demand forecast

Figure 2: The process of forecasting

In the following sections the black box ‘Forecast’ as illustrated in Figure 2 is further explained and divided into smaller steps; a statistical model, judgmental adjustments and a combination of those two; the final forecast.

| Working principles of Forecasting 8

3.2 A

-

In a supply chain demand planners must anticipate the future behavior of many critical variables before they can make a decision. Their decisions depend on forecasts, and they expect these forecasts to be accurate; a forecast model is needed to make such predictions.

Forecasting methods

According to Reid and Sanders [16] forecasting can be best divided into two main methods; forming a qualitative model and the quantitative model.

Qualitative or quantitative

Forecasts based on subjective techniques are called qualitative forecasts. These forecasts are achieved by taking the opinion and judgment of consumers and experts into account, mostly used when past data is not available. Examples of these methods include the Delphi method, marked research, scenario building, statistical surveys and historical life-cycle analogy.

Qualitative forecasting techniques are generally more subjective than their quantitative counterparts. Quantitative forecasting methods are used to forecast data as a function of past data. To form a quantitative forecast, there are three conditions to meet [17], as listed below;

1. Past information is available.

2. The information is suitable to quantify into a numerical data set.

3. It is assumed that the patterns found in the past data are repeatable and an estimation for the future.

If all the listed conditions can be met, a quantitative model can be used to forecast. A quantitative model is preferred because of its objective character.

Quantitative methods can be divided into two categories; time series models and causal models. Although both are mathematical, the two categories differ in their assumptions and in the manner in which a forecast is generated.

Time Series Models

A time series model is a forecasting model using the past data and patterns of past data as an input for the forecast. Examples of these models use methods like; simple moving average, exponential smoothing and linear trend line.

Causal Models

Forecasting by using the causal models are based on the search for variables that are of influence to the demand planning. An example is to predict the consumption of oil due to an increasing oil price. Searching for a relation between these two variables is typical for forecasting using the causal model. Examples of this type are linear regression and multiple regression.

Patterns of time series

In the supply chain quantitative models are the ones used to forecast. Mostly the forecasts are derived by making use of time series models. These are used since all the conditions are met and this is the most objective method.

| Working principles of Forecasting 9 In the book Operations Management written by Reid and Sanders [16] a detailed description of different forecasting models is given. Since past data is available, the methods used for the models forecast are of the time series type. Time series analysis assumes that all the information needed to generate a forecast is contained in the time series of the data. The forecaster looks for pattern in the data and tries to obtain a forecast by projecting that pattern into the future. The easiest way to identify patterns is to plot data and examine the resulting graphs. There are four basic patterns, which are present in a time series of data;

1. Level or horizontal. A level or horizontal patterns exists when data values fluctuate around a constant mean. This is the simplest pattern and the easiest to predict.

2. Trend. When data exhibit an increasing or decreasing pattern over time, it exhibits a trend. The trend can be upward or downward. The simplest type of a trend is a straight line, or linear trend.

3. Seasonality. A seasonal pattern is any pattern that regularly repeats itself and is of a constant length. Such seasonality exists when the forecasted variable is influenced by seasonal factors such as the quarter or month of a year or day of the week.

4. Cycles. Patterns that are created by economic fluctuations such as those associated with the business cycle are called cycles. These could be recessions, inflation, or even the life cycle of a product. The major distinction between a seasonal pattern and a cyclical pattern is that a cyclical pattern varies in length and magnitude and therefore is much more difficult to forecast than other patterns.

Despite all the types and methods to compose a forecast, there will always be random variation. Random variation is unexplained variation that cannot be predicted. In a formula, this will look like;

The first four components of the data are part of a pattern that is possible to forecast, only the weight of each pattern is unknown. Random variation cannot be predicted at all. Some data have a lot of random variation and some have little. The more random variation a data set has, the harder is it to forecast accurately. The forecasting models try to eliminate as much of this random variation as possible.

Forecasting models: a mathematical description

It is extensively discussed in literature what models are best to forecast. As Bala states in 2010 [18]; “The common practices and various literatures include time series decomposition, exponential smoothing, time series regression and autoregressive and integrated moving average (ARIMA) models. Out of these models, seasonal ARIMA model has been the most acceptable forecasting model that results in maximum accuracy and it has been successfully tested in many practical applications.” Fildes et. Al [2] concluded that among four companies, three companies use forecasting systems that are based on variants of exponential smoothing.

The working principles

In this section a short introduction into the working principles of moving average techniques and smoothing techniques is given.

Forecasting level; moving average techniques

Averaging forecasts are based on a method to take a specific average of the available sales to forecast the demand. The most sophisticated weighted average procedure to obtain a forecast is exponential smoothing. To make a forecast for the next time period, three pieces of information are needed; The current period’s forecast, the

| Working principles of Forecasting 10 current period’s actual value and the value of a smoothing coefficient that varies between 0 and 1. The equation in mathematical terms is quite simple;

Where = next period’s forecast, t+1

= actual value in period, t = forecast for current period, t = smoothing coefficient

Note that depending on which value is set for ‘ ’, more weight is placed on either the current period’s actual or the current period’s forecast. In this manner the forecast can depend more heavily either on what happened most recently or on the current period’s forecast. Values of ‘ ’ that are low – say, 0.1 or 0.2 – generate a rather stable forecast because the model does not put much weight on the current period’s actual demand.

Forecasting trend; trend-adjusted smoothing

The most common trend model is trend-adjusted exponential smoothing. This model uses three equations; the first smoothes out the level of the series, the second smoothes out the trend and the third generates a forecast by adding up the findings from the first two equations.

Where = Forecast including trend for the next period, t+1

= actual value in period, t

= exponential smoothed average of the time series in period t = exponential smoothed trend of the time series in period t = smoothing coefficient of the level

= smoothing coefficient of the trend

Similar to the moving average technique, values for and need to be found. Forecasting seasonality

Seasonality is expressed as the amount of deviation of the actual value from the average or mean of the data. Multiplicative seasonality is seasonality expressed as a percentage of the average. The percentage by which the value for each season is above or below the mean is a seasonal index. The procedure to compute seasonality consists of 5 steps;

1. Calculate the average demand for each season.

Where D= average demand per season per year

= annual demand of year t

| Working principles of Forecasting 11 2. Compute a seasonal index for each season of every year for which data is available.

Where = seasonal index

= seasonal demand

3. Calculate the average seasonal index for each season.

Where = seasonal index per season of next year, t+1

= seasonal index per season per year i = amount of years

4. Calculate the average demand for the next year.

Where = annual demand of next year, t+1

= average demand per season of next year, t+1

5. Multiply next year’s average seasonal demand by each seasonal index.

Where = Forecast including trend for the next period, t+1

After taking all the five steps into account, the forecast is calculated that includes seasonality.

Measurement of accuracy

As already mentioned in the introduction, the evaluation of a forecasting method can be done by determining the accuracy of a forecast. A simple method to evaluate the forecast accuracy in the supply chain is by using the Mean Absolute Percent Error or MAPE [6]. Statistically MAPE is defined as the average of percentage errors of a method.

The MAPE is calculated by;

Where = Mean Absolute Percent Error to measure the accuracy of

the forecast method = actual value in period, t = forecast for current period, t = number of points forecasted

If small inaccuracies want to be measured as well, the MSPE can be calculated, by;

The MSPE eliminates the positive/negative problem by squaring the errors. The result tends to place more emphasis on the larger errors and therefore, gives a more conservative measure than the MAPE.

| Working principles of Forecasting 12

Forecasting models: Implementation of the computer

A theoretical overview of the forecasting methods is given, but how are these models constructed in practice? How do Fast Moving Consumer Goods (FMCG) distributors construct these statistical forecasts?

Independent of the statistical model that is chosen, the computations required to estimate parameters and perform the analysis are tedious enough that computer implementation is essential. In most supply chains a warehouse management system is used to constructs these model forecasts. Examples of such are enterprise systems vendors like SAP, PeopleSoft and Oracle. An application of such a software vendor enables demand planners to accurately understand, predict, and manage the balance between inventory and customer service. An application like this pulls sales data from the enterprise system to generate forecasts based on special algorithms and past experience as Davenport and Brooks state in 2004 [19]. The software helps to increase productivity while reducing order and delivery costs. As an example it turned out that Procter and Gamble and Coca Cola use a SAP forecast and replenish system. Most forecasting and replenishment applications provide a demand forecast and standard analytics content to monitor the quality of the processes and of the solution implementation. Global forecasting calculations are done fully automated without the necessity to manual profile. Global sales and consumption history is evaluated and extrapolated into the future while automatically considering;

 Trends

 Seasons and seasonal pattern

 Past and future demand influencing factors such as; price changes, companywide promotions, holidays, emergency situations (such as hurricanes).

 Related products (cannibalization).

Normally forecasting applications calculate safety amounts according to a very flexible safety stock policy.

The development among the enterprise systems vendors are spearheaded not by the major ones, but more among the smaller, more focused software solution providers such as i2 and Manugistics [19].

As already mentioned in the Case of P&G the model forecast is computed through the forecast and replenishment system of SAP. Most companies don’t take the model forecast as the final forecast, since there is more data available to take into account next to historical data. This is data known by experts only and needs to be added to the model forecast before the final forecast is formed. This process of adjusting the model forecast will be discussed in next section.

| Working principles of Forecasting 13

3.3 J

A

In practice, forecasts derived by the models as described in the previous section are modified by the analyst upon considering information that is not available from the historical data. The process of reviewing and adjusting the output of a model done by the experts is called judgmental adjusting. This section will go into detail on how adjustments are achieved, the process of making a judgmental adjustment and the research done on this subject.

Where and by whom is the adjustment done?

In order to plan and manage the supply chain, organizations typically set up a unit responsible for forecasting. A forecaster is responsible for providing forecasting expertise where necessary, usually through a demand and planning department. The organizational pressures and priorities will differ from organization to organization, where in the supply chain the emphasis will be on the accuracy of demand planning.

The forecasts will usually be revised, monthly as fresh information is received throughout the forecasting model described in the previous section. In a monthly meeting the sales forecaster, the marketing responsible and the demand planner will meet and will exchange all the available useful information. In the end the final responsibility for a forecast lies within the hands of the demand planner, where the input from the meeting and the statistical model are taken into account to form the final forecast. The demand planner is responsible to form the final forecast and to communicate the final value of the forecast to the market and sales persons.

Need for adjustment

As described in the introduction, there is need for this kind of adjustments on the statistical model. Nikolopoulos et al. [15] evaluated the benefits of the intervention of experts to the final forecast. His findings support the case that adjustments do improve accuracy. In practice the demand planner reviews and adjusts the results of a forecasting model mainly triggered by exceptions rather than reviewing all results generated.

Exceptions where the demand planner should adjust the statistical model according to the company proving the software are,

 Situations like when indicating missing stock updates or inconsistent master data

 Situations related to business reasons such as exceptions on order proposal (items) which may deal with reasons why an order proposal needs review, e.g. purchase order value or order proposals of new vendors.

The solution can be configured in a way that order proposals which fulfill certain conditions or bear a certain exception need to be released manually.

| Working principles of Forecasting 14

3.4 T

F

;

Now all the processes are described to form a forecast, the last step to the final forecast can be reviewed. This section provides the reader an overall picture of the theory behind the final forecast and clarifies the process as it is seen today by a schematic overview and a descriptive formula.

Final forecasting; general process today

As described in the previous sections forecasting starts with a statistical system’s forecast. The expert takes the information of the system’s forecast into consideration and adds his non historical knowledge to form the final forecast. This final demand forecast is send to both the raw material suppliers and the retailers.

A better overview in the process of forming a final forecast is shown in Figure 3. As is shown in the figure, the expert takes the model forecast into account MFt+1, and the forecast he gives EFt+1 is the final demand forecast FFt+1.

Model calculation Adjustment by expert Dt MFt+1 EFt+1 = FFt+1 Dt, Kt+1

Figure 3: The situation today

In practice companies can differ in the process of estimating the final forecast. According to Franses (2013) it is typical for companies that there is an analyst who aims to evaluate the quality of this expert’s forecast [20]. The study that Fildes et. Al did in 2009 [2] concluded that the forecast was established as follows;

At the start of each forecasting period, the statistical ‘system’ forecasts are produced using the computer software. After this there is a meeting involving forecasting, marketing, production and sales personnel, where the final forecast is formed including various pieces of marketing and other information. The final forecasts were not subsequently changed by more senior management.

To compare the study of Fildes [2] and Franses [20] to this report, it is important to use identical words. What Fildes states can be understood by saying; the statistical system forecast is captured in the name ‘model’ and ‘the forecasting, marketing, production and sales personnel’ is summarized by the word ‘expert’. Franses adds another factor to the system called the analyst. He states that the experts’ forecast is evaluated by the analyst, the person who is responsible for the accuracy of the statistical model.

At most manufacturers the responsibility of the model is in hands of a software producing company. At P&G this company is SAP. In this particular case the analyst and the expert do not communicate and therefore it is assumed that the general demand forecasting process manufacturers use in practice can be described as Fildes studied. From now on it is assumed

| Working principles of Forecasting 15 that in the forecasting process of a manufacturer, the statistical model itself cannot be changed and therefore the model forecast is assumed to be most accurate and taken as an acknowledgement. But the parameters used to form the model forecast can certainly be communicated to the expert and a better understanding of the model can be achieved.

A mathematical interpretation

To get a better insight of the most up to date research done on final forecasting, this section provides a mathematical formula that is assumed to express the final forecast.

Frances established a formula that contains all the variables, input, output and decision parameters that influence the final forecast ( ). The mathematical term that Franses produced in 2013 [20] include the expert’s forecast (for, say, sales at t+1 a forecast for t+1 is established) and the associated model forecast ;

,

Where = forecast including model for the next period, t+1

= expert’s forecast including model for the next period, t+1 = unknown parameter

= unknown parameter

= model forecast for the next period, t+1

= unpredictable part called ‘Intuition’ for the next period, t+1

As Franses states the unknown parameters ‘ ’ and ‘ ’ can be determined, while is unpredictable. As the influence of the model and the expert both are better understood, the final forecast can be improved.

| Theoretical improvements 16

4 THEORETICAL IMPROVEMENTS

Over the last 25 years a lot of research has been done to gain a higher accuracy of the final forecast. This section describes the most important theoretical improvements that are found through research. In chapter 5 suggestions to realize these improvements are explained.

4.1 A

1. Right weight for model and expert

In 2007 Franses and Legerstee [15] started doing research on this topic. Their results indicate that experts have too strong a tendency to downplay the contribution of the model and put too much weight on their own contribution. In their research they found prove that if the expert would impose less weight on their own judgment (downplaying it from 63% towards 50%) the overall forecast quality would seriously improve. Also they suggest that much better forecasts can be created by assuming a smaller weight to the experts forecast and give more credit to the model. Blattberg and Hoch [14] focused on ways to combine the statistical model with the expert’s intuition and found evidence of improvements when using a 50% model + 50% expert.

2. Prevent optimistic forecasting

Franses states in 2009 that managers adjust model-based forecasts in about 90% of all cases where 54% of all the forecasts is upwards [11]. Fildes states in 2009 that positive adjustments, which involve adjusting the forecast upwards, were much less likely to improve accuracy than negative adjustments [2]. They were also made in the wrong direction more frequently, suggesting a general bias towards optimism. An adjustment in the upward direction can lead to less accurate results.

3. Avoid small adjustments and stimulate large adjustments

Fildes states that relatively larger adjustments tend to lead to greater average improvements in accuracy, whereas smaller adjustments often damage accuracy. In 2008 Franses found that it is important that and that boundaries to adjust have to be set [10]. Fildes found that if the information on which the adjustment is based, is viewed as unreliable, the forecasters are likely to hedge their bets by reducing the size of the adjustment [2].

4. Avoid double counting

In 1993 double counting is noted by Goodwin and Wright [21]. In 2002 [8] they state that a model must be obtained which contains judgmental estimates both of time series components and the effect of explanatory variables that have not been accounted for in the time series forecasts. If this cannot be obtained, so called double counting is occurring. This means that some of the effects of a variable will already be taken into account by the statistical method and any judgmental forecaster who makes an adjustment for the full effects of the missing variable, will be double counting some of these effects. Bunn and Salo [22] suspect that many of the casual adjustment processes common in practice are susceptible to double-counting bias. In a survey it is found that the respondents take recent sales data as input for their forecasts [20]. This is clearly an example of a parameter that is accounted for in the model ánd by the expert. And if forecasters would be assisted in the decomposition of market intelligence into key drivers which can be compared to the ones accounted for in the model, the likelihood of double counting will be reduced [2].

5. Judge in case of a 100% solid motivation

Goodwin found out that in 35% of the situations where the expert adjusts the model forecast, it leads to less accurate results than by simply accepting the statistical forecast [23].Based on the observations of forecasting in companies and discussions with forecasters, it is suggested that

| Theoretical improvements 17 forecasters often inker with the system forecasts merely to demonstrate that they have reviewed the forecasts, and are attending to the task [2]. This does not lead to better results but decreases the accuracy of the combined forecast.

6. Base a judgmental adjustment on objective data

When judgmental forecasters have access to non-time-series information (e.g. information about promotion campaigns) they tend to use it inconsistently or see pre-conceived relationships that do not exist [8].This shows that judgmental adjustment is done on objective data. It is also found that the expert depends the judgment strongly on past adjustments (habit information) instead of past model based forecast errors; about three times as much [11].

Also group processes can distort the judgments of individual group members what will lead to individual group members changing their forecast as a result of hearing the other individuals’ forecasts [8]. To avoid this strict rules have to be followed to avoid the exchange of forecasts but stick to the exchange of non historical data by the ‘forecasting personnel’.

7. Eliminate skepticism of experts on models

Skepticism of experts is already touched upon in the first improvement mentioned. To downplay the role of the model and increase the weight on their own forecast, leads to pour results. To increase the forecasts, skepticism of the experts on the model should be eliminated. This can be done by raising a better understanding of the model, so that experts will trust the model forecast.

Also it is suggested that decision makers are more likely to accept forecasts if they have a sense of ownership of the forecast, because they have contributed to the process that derived them [8]. Fildes and Goodwin [24] found out that when respondents rate the importance of their judgment an average of 4,1 is found (where 1 is not important and 5 is very important).

| Theoretical improvements 18

4.2

;

To realize the improvements, as mentioned above, the lay-out of the process can be adjusted. Goodwin first started doing this by dividing the process of forecasting into two methods [8]. He called them the voluntary integration and the mechanical integration.

Voluntary integration

In the process of voluntary integration the expert receives all the information and details of the statistical forecast and is free to completely ignore or completely accept the statistical forecast or merely to take some account of it in making the judgmental forecast. An important aspect of this method is the fact that the forecaster will be the last one to adjust and thus to form the final forecast. A simplified picture of a process designed following voluntary integration is shown in Figure 4. Model calculation Adjustment by expert Dt MFt+1 EFt+1 = FFt+1 Dt, Kt+1

Figure 4: Voluntary integration

The figure shows that the process is dependent of the model and the expert. The model forecast MFt+1 is based on the previous historical data available Dt, like the actual sales. The experts forecast EFt+1 is based on MFt+1 while taking the experts knowledge Kt+1 for this forecast and the historical data Dt into account. This expert’s knowledge consists of ‘non historical data’, based on exceptional situations as mentioned in the section ‘Judgmental Adjustment’, and other not known knowledge as Goodwin [23] and Franses [15] found. The FFt+1 is equally to the EFt+1 as there will be no further adjusting after the experts’ adjustment.

Mechanical integration

In the process of the mechanical integration; the “integrated” forecast is obtained through the application of a statistical method to the judgmental forecast [8]. The mechanical integration takes place where the final forecast is determined by a statistical method. For example, this may involve the use of statistical methods to identify systematic biases in the judge’s past performance, followed by attempts to remove these biases from subsequent forecasts by correcting them [26] or to avoid double counting. Alternatively, the judgmental forecasts might be combined with the statistical forecasts using a mathematically computed simple or weighted average. In yet another approach, the forecaster might be replaced by a statistical model of his or her past judgmental strategy [27]. In general this process is characterized by the fact that the final forecast will be formed by a statistical method in contrast with the voluntary integration where the expert will take the definitive decision. A simplified picture of a process designed following mechanical integration is shown in Figure 5.

| Theoretical improvements 19 Model calculation Adjustment by expert Dt MFt+1 Kt+1, Dt FFt+1 EFt+1

Figure 5: mechanical integration

The figure shows that the process is designed with 2 stages. In the first stage the calculations are done by the model MFt+1 with the historical data taking into account Dt and sent to the expert. The expert adjusts this by making use of the available historical data Dt and his own knowledge Kt+1 and communicates the experts’ forecast EFt+1 back to the model. The second stage consists of the final adjustment done by the model to form the final forecast FFt+1.

One big danger of designing a system, as shown in Figure 5, is touched upon by Goodwin [8]. He states that correcting judgmental forecasts by use of statistical methods may improve the forecast in the short term. The long term effect however may be to cause the forecasters to devote less effort to producing forecasts that they know will be corrected. It can also cause them to make pre-emptor adjustments to their forecast in an attempt to negate the correction.This can lead to decreasing accuracy instead of increasing.

| Suggestions to implement the theory 20

5 SUGGESTIONS TO IMPLEMENT THE THEORY

Now the possibilities to improve are transparent, solutions need to be found to achieve all the improvements. In paragraph 5.1 suggestions are done per improvement in a process form as well as a mathematical form. In paragraph 5.2 one overall solution is found. This final model is based on mechanical integration which includes suggestions for a final model. It is important that it is clear that the suggestions done in this chapter are based on both research as well as experience of the writer. To get a better insight in the academic level of the suggestions, research has to be done.

5.1 S

In this section the theory as mentioned in the previous section is studied in more detail and theoretical implementations of the improvements are found. To concretize the improvements two types of implementations are suggested; adjustments to the process and a solving algorithm. All these solutions are further explained and add to one big final design and mathematical formula as shown in the next chapter.

1. Right weight for model and expert

Process

To decrease the weight the expert gives to his own adjustment, it is important that the expert is aware of the influence of his judgmental adjustments on the accuracy of the overall forecast. This can be achieved by more communication between the expert and the analyst, where the analyst can provide the expert with more updates on the performance of his previous adjustments [15]. Another approach to increase more awareness on the influences of the expert on the combined forecast can be gained by giving training and workshops. Creating awareness among the experts that they decreases the accuracy of the forecast by adjusting the model forecast can lead to more improved weights [11].

Mathematical

In a mathematical formula the weight given to an expert’s forecast and the model forecast could be expressed as shown in formula 1.

, 1)

Where = forecast including model for the next period, t+1

= expert’s forecast including model for the next period, t+1 = model forecast for the next period, t+1

= Weight given to the expert’s forecast, between 0 and 1

Previous research has examined what the right values for the model and experts’ forecast could be. First of all literature learned us [28] that if there is a general/normal situation to predict without a lot of external disturbances, the model is more accurate in first place, then after adjustments of the expert. Also it is found in literature that if is smaller than 0.4 the accuracy of the overall forecast is increased. This means that an increase in the overall forecast can be stimulated by adding the condition;

2)

In practice this means that the experts’ forecast counts for 40% of the final forecast and the model forecast for 60% in normal situations.

| Suggestions to implement the theory 21

2. Prevent optimistic forecasting

Process

One study asked the experts to consider why they forecasted. The experts were asked to write reasons why their desired outcome might not occur [25]. As a result their forecasts were more accurate because the forecasters were less optimistic.

Also optimistic forecasting can be avoided by simply increasing the awareness of the tendency that experts are used to forecast optimistic [8]. This type of communication can be improved by advertising the relative performance of the adjustments of the expert. A practical implementation can be found in a pop-up window on the monitor of the expert. After adjusting the statistical model, the expert is exposed by a massage like “Be aware that the forecast now is 15.4% LESS ACCURATE that the statistical forecast provided to you!” Unfortunately research showed [8] that the experts continued to rely on their own forecasts.

Mathematical

In mathematical terms optimistic forecasting can be avoided by adjusting the weight given by the expert if optimistic forecasting is occurring. In mathematical terms this can lead to an extra term like the one expressed in formula 3).

3)

In this case, if EF is bigger than MF and could be optimistic, ‘e’ is raised by a power with a value lower than -1. This leads to a value for close to 0. If EF is smaller than MF and thus not optimistic but more likely to be a serious attempt to forecast in the right direction, the power for ‘e’ will be between -1 and 0. This leads to between 0.37 and 1. If EF is lot smaller than MF (large downwards adjustment; the opposite direction of optimistic forecasting) the expression in formula 3 leads to a weight ( ) between 0.9 and 1. This will lead to almost a total implementation of the experts’ forecast instead of the model forecast. This brings us to the third improvement mentioned, where big adjustments should be taken more seriously than small adjustments.

3. Avoid small adjustments and stimulate large adjustments

Process

To find a practical implementation to improve the accuracy by avoiding the small adjustments, it is important that the expert adjusts in case of a 100% solid motivation as is explained in improvement 5. The implementations in the process to achieve this improvement will be discussed further in improvement 5.

Another solution to avoid large and small adjustments, and increase the overall accuracy, is by changing the process. This is done by asking the forecaster is he/she wants to change with the default being no change. This removes the presumption that a change is expected and hint that change is the exception rather than the rule [23].

Mathematical

To avoid small adjustments and stimulate large adjustments the weight as expressed by in formula 1 has to be adjusted. If a small adjustment is done, the value of must be decreased to something in the range of 0 to 0.4 and if the adjustment done is large, the value of must be increased to 0.4 – 1. As is touched upon in the last phrase of the previous paragraph (improvement 0), avoiding small adjustments and stimulate large adjustments can be achieved by implementing formula 3.

| Suggestions to implement the theory 22

4. Avoid double counting

Process

To avoid double counting the expert must document the reasoning behind his adjustments to the analyst. In this way that the analyst has ground to evaluate the performance of the adjustments. A control loop between the analyst and the expert is advised to achieve this. As a result of this loop, an analysis of the correlation between the statistical and judgmental forecast errors can be done. In this way the so called double counting can be detected and avoided by the analyst. Also, if the expert understands better what variables are taken into account in the statistical model established by the analyst, double counting can be prevented; the expert can use his non-historical data only to adjust the statistical forecast instead of taking variables double into account. This technical knowledge can be gained through training and workshops on statistical forecasting methods that are used to establish a statistical forecast [9].

5. Judge in case of a 100% solid motivation

Process

Goodwin [23] found that to decrease the percentage of incorrect adjusting, whereas the model would have let to a better accuracy, it is important that the expert only adjusts the forecast if he/she is absolutely sure that adjustments will lead to higher accuracy. This can be achieved by a set up where the expert explicitly has to send a request if he/she wants to adjust the model. In this way no change would be the default according to Goodwin, who believes that change must be the exception rather than the rule. He found out that in this way the harmful adjustments were reduced without reducing the propensity to make adjustments when they were appropriate. Still under these circumstances, 35% of the forecasts were still adjusted when simply accepting the statistical forecast would have led to greater accuracy.

6. Base a judgmental adjustment on objective data

Process

There are two ways to eliminate the adjustments based on objective data.

The first one is raise the understanding of the experts about market intelligence. If more awareness of how the market works is raised, the objective data that influences the adjustments can be eliminated [11].

The second way is to eliminate the adjustments of the expert by making use of an algorithm that mathematically corrects the biases [21]. There should be a feedback loop of information from the expert to the analyst to communicate the motivation behind the experts’ forecast. On basis of the motivation the expert gives a control function can compare the parameters the model uses and the parameters the expert uses. In this way the algorithm can decide how to take the experts’ forecast into account because he knows where double counting is taking place or subjective data is used. This can be processed into a weight for the forecasts of the model and the expert. With full documentation analysts can seek to make better models, experts can better understand how improvements can be made, and, finally, forecast evaluation becomes easier [11].

To increase the elimination of objective adjustments, a group process can be better designed. At the moment the manufacturers forecast in a group and influence each other. To avoid this strict rules have to be followed to avoid the exchange of forecasts but stick to the exchange of non

| Suggestions to implement the theory 23 historical data by the ‘forecasting personnel’. An advice would be to let all the experts forecast individually and let a model decide on the ‘final’ experts’ forecast [30] and [21].

Mathematical

To let the model calculate the ‘final’ experts’ forecast derived by different forecasters, the rules written in formula 4 have to be taken into account.

4)

Where = overall expert’s forecast including model for the next

period, t+1

= the experts’ forecast of expert i = amount of experts involved

As is formula 4 shows the average of the different experts´ forecasts is determined, and taken as the final experts´ forecast.

7. Eliminate skepticism of experts on models

Process

To eliminate the skepticism of experts, it is important to increase the trust the expert has of the model forecast. Better understanding of the model, leads to better final accuracies in the normal situations. This can be done by setting up a connection between the expert and the analyst to increase the communication [9] and improve the forecaster´s technical knowledge [21].

| Suggestions to implement the theory 24

5.2 O

S

In the previous chapter different options to improve the system are summarized. Two types of lay-out for the processes are discussed. Also a few options to achieve the improvements in process form and a mathematical solution are found. To improve the overall accuracy of the forecast, these improvements need to be implemented into one final forecasting process. This section looks at the implementations in two steps. First it will be explained how a final process design is found. At last a final mathematical solution, to include into the final process, will be introduced.

The Final process

In order to be able to achieve as much of the improvements as possible, a combined model is developed and shown in Figure 6. This process consists of a model and an expert, like in the earlier mentioned models. This process is a mechanical integration, since the final decision of the FFt+1 is given by a model; the algorithm.

This design consists of a model forecast based on the historical data (Dt) and an expert forecast based on the model forecast (MFt+1), the historical data (Dt) and the knowledge the expert has (Kt+1). Control Model calculation A lg o rit h m c alc u la tio n Adjustment by expert Requirements:

ΔModel, ΔExpert, ΔAlgo Accuracy = f(At+1, FFt+1)

MFt+1 EFt+1 K*t+1 Dt Dt, Kt+1 FFt+1 MFt+1

Figure 6: The final model

The model and the expert communicate their forecasts (MFt+1 and EFt+1) to the algorithm. Parallel to that the expert chooses from a list of parameters to communicate the main reason for the adjustment. These three parameters (K*t+1, MFt+1 and EFt+1) will be sent to the model that consists of an algorithm. The algorithm calculates the final forecast (FFt+1).

This whole process is controlled by the analyst who is responsible for the accuracy of the final forecast (FFt+1). This accuracy is a function of the actual sales for the forecasted period (At+1) and the final forecast (FFt+1).

| Suggestions to implement the theory 25 The role of the controller is to evaluate the model and the algorithm by analyzing the Δmodel, Δexpert and a Δalgorithm. In practice a control like this will be a forecasting support system monitored by the analyst [29]. The system calculates the values for Δmodel, Δexpert and a Δalgorithm. The analyst is responsible to help the expert to gain technical knowledge of the model in order to eliminate the skepticism of the expert on the model and to avoid double counting. He is also responsible for the communication of the performance of the expert to the expert. By raising awareness of the accuracy of the experts’ adjustments, the expert can improve his own forecasting.

The Algorithm

This final process consists of an algorithm. This algorithm is derived by taking as many of the requirements found in chapter 5 as possible into account. This algorithm is a suggested solution and has to be tested before usage.

1)

3)

5)

Where = forecast including model for the next period, t+1

= expert’s forecast including model for the next period, t+1 = model forecast for the next period, t+1

= weight given to the expert’s forecast, between 0 and 1

In the final solution as presented in 1), the weights given to the experts’ forecast and the model forecast are taken into account to determine the final forecast. Alpha is calculated by formula 3) as explained in section 5.1. Formula 5) gives an average of the group’s decision on the demand forecast and eliminates the affect group members can have on each other’s forecast.

To eliminate the double counting, a control loop should be designed which compares the parameters the expert uses and the parameters the model uses as an input. This control function should also meet the conditions as found before, and thus .

| Conclusion 26

6 CONCLUSION

This chapter is devoted to give a final opinion on forecasting when a model and an expert have to be merged. Also suggestions for further research are provided.

6.1 C

It is believed that a lot of improvements can be done in order to improve the accuracy of the forecasting process. In general it is concluded that the expert should not be the final decision maker, but a mechanical integration is more advisable. Also, the maximum weight to give the experts’ forecast needs to be determined per forecasting process. Researchers believe that giving the expert a general weight of about 40% leads to a better final accuracy.

One of the improvements is found in applying the right weight to the model forecast and the experts´ forecast. In the current forecasts experts give too much credit to their own forecast instead of the model forecast, leading to a lower accuracy compared to a forecast with an increased weight for the model. Also optimistic forecasting should be prevented in order to get better forecasting results. Next to this forecasters should only adjust the model when they are 100% sure which can be found in large adjustments by the forecaster. It is therefore believed that small adjustments should be avoided and that it is important that experts should only adjust sometimes not always. Since it is found that a decrease in accuracy of a forecast is caused by the fact that actual sales are counted twice, double counting should be avoided. Better technical knowledge on the model of the expert can prevent this. Also, better understanding of how the model works and how the expert influences the overall accuracy of the forecast, must be gained. A feedback loop and an evaluation of the functioning of the expert will stimulate self-learing. If the expert adjusts based on objective data, the accuracy will be higher. Since it is proved, that in general situations simply accepting the statistical forecast would lead to a higher accuracy, it is important to let the experts adjust in exceptional situations only. This can be achieved by raising the trust of the model and the sense of ownership of the forecast.

To implement all these improvements, a recommendation for a new lay-out of the process is given, which includes the use of an algorithm. Further research is necessary to prove if the suggested implementations, and the suggestion for a new model, are correct.

6.2 F

R

In this paper an increase in the accuracy of the overall forecast is found by taking the model forecast as an acknowledgement. The influence of the model on the final forecast should be investigated further. Also the impact of mechanical integration, as implemented in the suggested model, on the expert has to be examined.

Also, the implementations that are described in chapter 4 and 5 are subtracted from the literature improvements, are suggestions and no facts. Therefore the accuracy has to be further researched in order to make sure that the implementations improve the accuracy. So the recommendations done in the 5th _{chapter should be tested before implemented.}

Also the overall suggestion for a new model and the algorithm should be verified by applying it to test data. To evaluate the algorithm the absolute accuracy is calculated, as shown in formula 6. The accuracy is determined by taking the square of the difference between the actual sales and the forecast, in order to take small errors into account as well.

| Conclusion 27

6)

To decrease this error the derivatives of (as shown in formula 7) have to be found and minimized.

If the mathematical term 6) and 7) are combined, the expression in formula 8 is found.

7)

To evaluate this expression the overall derivate ( ) must be analyzed. Also the different performance requirements (Δmodel, Δexpert and Δalgorithm) should be tested.

| Reflection 28

7 REFLECTION

This chapter compares this report with other research done on this topic within the Netherlands. In this chapter a brief overview of the research on this topic in the Netherlands is provided.

Also more insight is given into the relevance of this research by giving the reader an understanding of the increase in terms of profit a company can reach by improving their demand forecast.

The main goal of this chapter is to give the reader a better understanding of the relevance of this paper within its context.

7.1 R

N

As shown in the introduction, a lot of research has been carried out over the world on forecasting the demand for manufacturers. It is

When looking at universities within the Netherlands, it is most likely to find this type of reports written at the more quantitative oriented universities with an interest in supply chain management and forecasting. During the search through the online-databases (of for example Narcis) most of the relevant research finds its origin at the department of econometrics and marketing research at the Erasmus University in Rotterdam, where Prof DR. PH. H. B. F. Franses produces the most notable research (over 20 papers in the field of demand forecasting as a combination of experts and models). Other institutes that do relevant research in this field are the Rotterdam school of management and the Erasmus institute of management, where business analytics and logistics are common research areas. It is advised to search the databases of these institutes before starting research on forecasting at the Supply Chain.

Almost all universities in the Netherlands create something interesting, when looking at more general topics like forecasting or combining different datasets to forecast events. Especially at the University of Amsterdam and the Maastricht University these are common topics. The question arises whether or not the findings can be applied to supply chain forecasting. For example research is found like bringing together a broad variety of moral and non-moral beliefs and combine these into one coherent belief system to forecast. The relevance of this research for a model and an expert to forecast demand in a supply chain is doubtful.

At the Delft University of Technology the most logic places to find researchers on this topic would be within faculties that have a logistic or business orientation or are excellent on statics. The most interesting faculties that have the potential of conducting relevant research are Air Transport Operations (Aero Space Engineering), Transport Engineering and Logistics (Mechanical Engineering), Engineering and Probability, Risk and Statistics (Applied Mathematics) and Transport, Infrastructure and Logistics (Technology Policy and Management). At these faculties a lot of forecasting studies done have a focus on a specific field different than the supply chain; maintenance at KLM, forecast the revenue for transavia.com, long-term forecast at harbors, forecasting the weather, forecasting the water levels and so on. Also papers on the behavior of humans are written like one at the TRAIL research school; Extended prospect theory: findings on choice behavior from economics and the behavioral sciences and their relevance for travel behavior.When searching for papers that are related to forecasting demand, there are no matching papers found for the Delft University of Technology. One of a series of

| Reflection 29 studies that are interesting to further examine are executed by Goossens and Cooke. Around 1990 they did research on how to develop protocols for selecting and weighting the decisions of experts. These papers were conducted within the faculty of Electrical Engineering, Mathematics and Computer Sciences at the department of Safety Science. This research done is very interesting, and it could be relevant to find the applications of their findings on the methods the experts use at FMCG field, but it has to be mentioned that this research was done in around 1990 and that more relevant research from other researchers is available.

When looking at the department TEL of Mechanical Engineering, studies are done like the evaluation of inventory management. Also research to investigate the use of tools like RFID to improve the data gathering to do more accurate prognostics on logistic requirements has been done in 2007. A study to design a forecasting and planning tool for KPN in order to forecast the planning at CEVA is conducted at TEL or modeling and forecasting in the dry bulk shipping market. All these studies are focused on the development of tools for very specific companies or fields, where none of the studies are oriented on combining demand forecasts for FMCG.

7.2 P

To estimate the overall increase in the accuracy that can be reached by implementing new rules to combine forecasts is very difficult. The differences per company and also per period vary so much, that a general decrease of the error cannot be determined. To indicate the potential increase in accuracy, the method to calculate the error has to be studied first. A potential increase in accuracy could be measured by a percentage of the error that exists.

So although there are researchers that have given an estimation of the increase that could be reached per method, giving an overall increase in performance would be too much of a gamble and is therefore left open to further research.