• Nie Znaleziono Wyników

Rolling horizon predictions of bus trajectories

N/A
N/A
Protected

Academic year: 2021

Share "Rolling horizon predictions of bus trajectories"

Copied!
12
0
0

Pełen tekst

(1)

OPT-i An International Conference on Engineering and Applied Sciences Optimization M. Papadrakakis, M.G. Karlaftis, N.D. Lagaros (eds.) Kos Island, Greece, 4-6 June 2014

ROLLING HORIZON PREDICTIONS OF BUS TRAJECTORIES

Masoud Fadaei Oshyani1, and Oded Cats2

1

Royal Institute of Technology - KTH Teknikringen 10, Stockholm, Sweden

masoud.fadaei@abe.kth.se

2

Delft University of Technology – TU Delft Delft, The Netherlands

o.cats@tudelft.nl

Keywords: Prediction, Bus departure time, Optimization, Travel time and Genetic algorithm. Abstract. Bus travel times are subject to inherent and recurrent uncertainties. A real-time

prediction scheme regarding how the transit system evolves will potentially facilitate more adaptive operations as well as more adaptive passengers’ decisions. This scheme should be tractable, sufficiently fast and reliable to be used in real time applications. For this purpose, a heuristic hybrid scheme for departure time estimation is proposed in this study. The predic-tion generated by the proposed hybrid scheme consists of three travel time components: schedule, instantaneous and historical data sources. Genetic algorithm is applied in order to specify the contribution of each data source component to the prediction scheme. The pro-posed scheme was applied for a trunk bus line in Stockholm, Sweden. In addition, the current-ly deployed scheme was replicated in order to compare the performance of both schemes. The results suggest that the proposed scheme reduces the overall mean absolute error by almost 20%. Moreover the proposed scheme provides better predictions except for very long term predictions where both schemes yield the same performance.

(2)

1 INTRODUCTION

The provision of real-time travel information becomes an industry-standard among transit operators and authorities worldwide. Real-time predictions concerning how the transit system evolves facilitate more adaptive operations as well as more adaptive travelers’ route choice. The primary objective of this study is to develop an accurate and reliable prediction scheme for bus arrival times. By proposing a series of incremental and applicable improvements, we aim to contribute to closing the large gap between the currently deployed systems and the ad-vanced state of the art

A real-time prediction method should be tractable, sufficiently fast and reliable to be used in real time applications. In developing the proposed prediction scheme, it is assumed that in-stantaneous data concerning vehicle positions is available for all buses.

Due to several underlying sources of uncertainty such as traffic congestion, driving style, passenger demand or traffic incidents (e.g. accidents, extreme weather), buses’ departure time may differ from the scheduled time. The extent of this inherent uncertainty in bus operations is an important determinant of level-of-service and service efficiency. As a result, information concerning future states of the transit system is valuable for both passengers and operators. The former can use this information to refine their expectations and therefor undertake travel decisions that will decrease their overall travel time. Furthermore, operators are interested in anticipating deviations from the planned service in order to support decisions concerning mit-igation measures, such as fleet management strategies or real-time holding control [1].

The commonly deployed real-time information generation scheme is solely based on the current bus delay, implying that the current delay would be maintained at all downstream stops. This prediction scheme does not incorporate any real-time information apart from the location of the approaching bus. A more elaborative prediction scheme could take into ac-count historical travel time and recent downstream travel time information, anticipated head-ways and their impact on down-stream dwell times as well as trip chain considerations and other factors effecting travel time.

To predict bus departure time several models have been proposed by previous studies, such as regression models [2, 9], Artificial Neural Networks (ANN) [2, 3, 7], Kalman filtering (KF) [3, 8, 10], Interacting multiple model [6], Support Vector Machine (SVM) [2, 5] and statisti-cal pattern recognition [4]. Most of the previous studies represent methods describing the rela-tionship between bus-travel time and elements influencing it.

Chien et al. presented two different ANN prediction methods, link-based and stop-based. They described that stop-based ANN is more suitable where there are multiple intersections between pairs of stops. Whereas link-based is more appropriate for line with fewer intersec-tions between consecutive stops [7]. Yu et al. have instructed four different methods based on SVM, ANN, k-nearest-neighbors algorithm (k-NN) and linear regression and found that the SVM model in more robust for bus arrival time prediction [2]. Chen et al provided a dynamic prediction method consisting of two elements, ANN and KF-based models [3]. Cathey et al. used a KF to assign a set of AVL data to vehicle state estimation [8]. The statistical pattern recognition was compared with KF-based methods by Vu et al [4]. They reported that their model is more reliable when dealing with unusual bus operation events.

Most of the abovementioned prediction methods deploy computationally expensive ma-chine learning models. These models involve a special training phase (‘knowledge generaliza-tion’) which may hinder their deployment in real-time. Moreover, the solution obtained for these methods does not provide information on the underlying relation between travel time factors (inputs) and downstream conditions (outputs).

(3)

The primary objective of this study is to enhance a scheme which is commonly deployed by public transport agencies for generating real-time information by embedding instantaneous and historical travel time data of the respective time-window. In particular, downstream travel times could be estimated based on the travel times of the previous buses that traversed a seg-ment. The integration of various data sources in the hybrid prediction is determined by apply-ing a Genetic algorithm, while the prediction is generated on a rollapply-ing horizon basis with each prediction referring to the remaining bus trajectory. The proposed scheme was applied for a trunk bus line in Stockholm, Sweden. In addition, the currently deployed scheme was repli-cated in order to compare the performance of both schemes. The results suggest that the pro-posed scheme reduces the mean absolute error by almost 20%.

The current as well as the proposed prediction methods are presented and formulated in the following section. Section 3 describes the case study followed by the results of the application. The paper is concluded with a discussion of model assessment and directions for further de-velopments.

2 MODEL DEVELOPMENT

The commonly deployed scheme for real-time information generation is based on the dif-ference between actual and scheduled bus arrival times. The latest delay of a bus is assumed to be maintained along the line and hence result with the same delay at all downstream stops. Hence, the current prediction scheme does not incorporate any instantaneous information apart from the current schedule deviation of the bus. In contrast, the hybrid scheme considers both instantaneous and historical data regarding travel times in addition to scheduled travel times. In this scheme, the predicted travel time to nay down-stream stop is computed by inte-grating these three elements: scheduled, instantaneous and historical travel times.

The weight assigned to each component in the prediction scheme is a parameter that is es-timated based on an optimization process. Moreover, the number of buses to be considered in the instantaneous component and the reference time interval for the historical data need are estimated. The estimation of these parameters is performed using a Genetic Algorithm (GA).

The prediction scheme is applied with the operator’s perspective in mind. Hence, the mod-el generates predictions for departure from each stop of the remaining trajectory. This real-time information generation approach enable operators to better manage and control their fleet and apply proactive measures to improve the service for passengers. Predictions are generated in an event-based process upon a bus visit at stop. Note that commonly provisioned passenger information concerning the remaining waiting time until the next arrival per stop could be di-rectly derived from bus trajectory predictions but is not considered in this paper [1].

2.1 Problem description

Each bus is assigned to a line-direction, , to serve passengers. A line-direction consists of an ordered set of stops, , where is the ith stop and is the last stop on .

A set of bus trips denoted by is assigned to run on line-direction . The planned service specifies the fixed scheduled departure time of each trip at each stop which could be represented as , where is the matrix of all scheduled departure times for the respective line-direction. The corresponding observed departure time is denoted by , where is the matrix that is updated in real-time with observed departure lines for a given line-direction. In addition, a subset of these stops, denoted by ̂ , are designated as time

(4)

point stops (TPS). TPS is a stop where buses are instructed to hold in order to maintain the timetable in case they are ahead of schedule ( ).

The unit of analysis used in the development of a new prediction scheme is the distance traversed between each pair of consecutive stops and is referred as ‘road segment’. A road segment consists of a set of street links and intermediate intersections but is merely defined by its start and stops.

The predicted trajectory generated upon visiting stop at time includes the predicted departure times from stops { . The predicted departure times for trip are represented as a multi-dimensional array, , of | | | |X| | which is initialized with null and is then progressively updated as trips are performed, visit stops and predictions are generated. The predicted departure time of trip from stop made upon visiting stop is thus denoted by . Note that for a given trip, the | |X| | will remain half

empty with a diagonal cut as predictions are made only for downstream stops.

2.2 Current Prediction Scheme

The fundamental assumption of the current prediction scheme is that the current schedule deviation will be maintained unless the vehicle runs ahead of schedule and there is an inter-mediate TPS. Hence, the predicted departure time of trip from stop generated at time is calculated as follows: { ̂ ( 1 ) 2.3 Hybrid scheme

The prediction generated by the proposed hybrid scheme consists of three travel time com-ponents: , and which correspond to schedule, instantaneous and

histor-ical data sources, respectively. First, similarly to the current scheme, the remaining travel time is derived from the scheduled departure times as follows:

( 2 )

Where is the time at which the prediction is made.

Second, travel time information from downstream buses is embedded in the prediction scheme. In particular, the travel time of a segment that connects two consecutive stops could be estimated based on the travel times of previous buses that traversed the segment (on the same date). The number of buses which should be taken into account is a parameter of the prediction scheme and subject to estimation.

As mentioned in Section 2.1, travel times are analyzed at the level of the road segment connecting a pair of consecutive stops. The predicted departure time from a downstream stop is thus based on the summation of instantaneous travel time data from all the road seg-ments between the current bus location - stop – and stop . Note that the most recent data available is utilized for each road segment in order to obtain the most up-to-date information. This implies that the set of buses that are used as a reference for each road segment may vary. For a given road segment, the median value of the observed travel times pertaining to the last

(5)

buses is obtained. The median is taken rather than the average value as a statistic of ob-served travel times in order to avoid effects of extremely low and high values. The prediction scheme for the instantaneous part is summarized in the following formula:

∑ [ ] ( 3 )

Third, information from historical travel time is incorporated in the hybrid method. The historical data is retrieved based on certain common attributes that the current prediction state shares with historical data records. Common attributes could refer to time frames – seasonal, day of the week, time of the day – or service characteristics, such as extend of delay or de-mand levels. In this study, historical information refers to data from the same day of the week from the previous week, and for a similar time of day period.

The time window is determined by , where is the reference time point and a toler-ance parameter. The historical trip data for a prediction made for the road segment that starts at stop where is determined thus as follows:

̂ { } ( 4 )

In order to refer to the most relevant historical data for each road segment, the time win-dow is applied dynamically.

Since the departure times from downstream stops is unknown during the prediction process, scheduled departure times in the timetable are considered as the reference time point. Hence, for any stop , the reference time equals the prediction time plus a summation over all road segments connecting to . This summation is made for median of all observed travel time on the respective day and the defined time period. Hence, the historical part of departure time prediction generated for trip at stop is

∑ [ ] ̂

( 5 )

For simplicity, we consider historical information only from the previous week. However, this time range could be easily expanded to a wider time horizon.

The hybrid prediction scheme integrates scheduled travel time, instantaneous data and his-torical data regarding bus travel time according to the following equation:

( 6 )

Unless there is at least one TPS between and , and the bus is ahead of time. In this situation, the predicted departure time will be the same as the scheduled departure time. The function yields the remaining travel time to stop . Genetic algorithm is applied in order to specify the contribution of each data source to the prediction scheme as described in Section 2.5.

2.4 Performance Analysis

The performance of each scheme is evaluated based on the available AVL data. The per-formance analysis is designed to compare the two aforementioned prediction schemes from the operator perspective.

The mean-absolute error (MAE) is calculated as a performance measurement to assess the average discrepancy between actual and predicted departure times. This measurement is com-puted for all predictions that were made at stop for stop . The MAE is defined as follow: | ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅| ( 7 )

(6)

In order to have an aggregated performance measurement for all predictions made, the MAE value for each scheme could be summarized over all stops and their respective upstream predictions, as follows: ∑ ( ∑ ) ( 8 )

This aggregated performance measurement obtains a single value to evaluate each scheme. Moreover, this performance indicator could also be used as part of an optimization process in order to find the best values for the hybrid scheme parameters.

2.5 Genetic Algorithm

Genetic algorithm (GA) is a meta-heuristic optimization method which is based on the law of natural selection. A potential solution of an optimization problem is represented as a vector of parameters like a set of genes in a chromosome. Each chromosome is evaluated by an ob-jective function, which corresponds to the criteria of the optimization problem. Then a fitness function delegates a fitness rate to each chromosome as a measurement of its closeness to the final solution.

GA is an iterative based method which consists of three main steps: selection, genetic op-eration and replacement. As an initialization step, the initial population is generated either randomly or based on output of another optimization method. First, the solutions are selected based on their fitness values to reproduce. The fitness value is defined based on the objective function of the optimization problem. The fitness value represents the closeness of chromo-somes to the optimum point. A chromosome with higher fitness has higher chance to be cho-sen. Second, the chosen solutions produce a set of solutions in the subsequent generation through the genetic operation including crossover and mutation. Third, according to a specific replacement strategy the entire chromosomes are replaced by newly generated ones and con-stitute a new population.

This iterative process will continue until completing a pre-determined number of iterations or objective function value. The genes corresponding to the best chromosome are the final solution obtained from GA.

3 CASE STUDY

The performance of the prediction schemes was evaluated by applying them on trunk bus line 1 in Stockholm, Sweden (Figure 1). Line 1 is one of four high-demand trunk lines that connect the various districts of Stockholm’s inner-city. This line connects the main eastern harbor to a residential area in the western part of the city through the commercial center. In this study, the eastbound route is considered. This route has 33 stops, of which three TPS marked on the map at stops 10, 17, 24.

Automatic vehicle location (AVL) data was available for this study from all buses running on Line 1. The AVL devices report the time and location of bus arrival and departure times from each stop along the line. AVL data of Line 1 for weekdays from Dec 1, 2011 to Jan 31, 2012, excluding seasonal holidays, was used in this study. The data was provided by Stock-holm transit agency (SL).

Each AVL record represents related information of a bus stop visit: trip ID, line number, vehicle ID, scheduled departure time and, observed arrival and departure times. The dwell time is derived from the difference between observed arrival and departure times. In our case study, only departure times are scheduled and there is no data regarding scheduled arrival times and consequently scheduled dwell times.

(7)

Figure 1 Case study - Bus line 4, Stockholm

There are a few issues that require further consideration for processing AVL data. In the dataset, there are a number records pertaining to partial trips. In other words, there is a possi-bility that a bus runs along only a subset of the line’s stops. These records require special treatment in order to ensure their correct inclusion in the prediction scheme. In addition, some trips have incomplete information for some stops. This happens occasionally in case the bus drive thorough the stop without stopping. In such cases, records are added with interpolated departure times proportional to the stop position. Alternatively, bus drivers are seldom in-structed by the operator’s dispatchers to terminate their trip and disembark passengers, a fleet management strategy known as short-turning. In these unusual cases, no records were added.

4 RESULTS

First, the current scheme is applied for entire data and corresponding predictions are made. Second, the proposed genetic algorithm is run and the optimum values for the unknown pa-rameters are obtained. Then the performance measurement (MAE) is computed for the both prediction schemes. Values of MAE provide possibility for comparing the performance of the two presented prediction schemes.

The optimization problem has been solved in order to calculate the values of unknown pa-rameters ( ). This process has been done through running the specified GA for the problem. After 60 iterations the GA returns: Figure 2 illustrates how the measure evolves through 80

itera-tions. The gray line shows the average for all 40 chromosomes in each iteration. The

blue dots represent the minimum value of at each iteration and the black line shows

the minimum value up till each iteration.

The initial generation was generated randomly. The figure shows that even the first itera-tion returns a good soluitera-tion and the improvement in the objective funcitera-tion value after 80 it-erations is only a little more than one second. The value for the current scheme is

90.94 seconds while it is 74.36 at the first iteration and 73.16 seconds after 80 iterations for the hybrid scheme. The proposed scheme hence outperforms the current scheme even for ran-domly generated parameters without selecting the optimal values. The optimal solution yields an improvement of 19.5 percent in the aggregated MAE value.

(8)

The figure also shows that the final solution is achieved at iteration 14. Therefore there is a possibility to implement our genetic algorithm for a smaller number of iterations and decrease the computation time.

Figure 2 Average and minimum objective function value per generation

The optimal set of parameters was specified in the hybrid method and corresponding dictions for entire data have been computed. Figure 3 shows the mean-absolute error for pre-dictions regarding arrival times at stops 2 to 32. These values are calculated regardless of the time and stop at which predictions have been made for the both prediction scheme.

The number of predictions is increasing from the second stop to downstream ones. For ex-ample, predictions for the second stop have been only made at the first stop; however, predic-tions for the third stops are a combination of predicpredic-tions which have been made at the first and second stops. Based on the figure, prediction performance and accuracy is improved for all stops although it varies from 0.2 to 37.3 second for stops 10 and 32, respectively.

Figure 3 MAE for the current and hybrid schemes along the route

Figure 4 shows the overall distribution of mean error for both prediction schemes. Both scheme results yield a symmetric distribution. The averages are 16.9 and 38.5 for the current and the hybrid schemes which shows a systematic increase in underestimation of time to arri-val for the hybrid scheme. However, standard deviation decreased from 155.9 to 130.3 which indicates that the hybrid scheme is more reliable. The overall improvement is illustrated, since the mean error distribution for the hybrid scheme is narrower and varies less.

73 74 75 76 77 78 79 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 M in

Number of generation (iteration)

The best objective function value of the generation Average objective function value of the generation The best objective function till the current iteration

0 20000 40000 60000 80000 100000 120000 140000 0 20 40 60 80 100 120 140 160 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 M A E (s e c)

Stop order number

(9)

Figure 4 Mean error distribution of the current and hybrid scheme

Figure 5 provides a spatial investigation of the distribution presented in figure 4. It shows mean error of predicted departure times for the all stops. The figure also illustrates lower and upper limits for 95% of calculated mean error values. The mean error for the hybrid scheme exercises less variation and obtains average values closer to zero. In the current scheme, the performance is clearly related to the presence of TPSs (stops 10, 17 and 24).

Figure 5 Accuracy of the current and hybrid scheme along the route

Figure 6 demonstrates the improvement in the hybrid scheme compare to the current scheme from MAE perspective. This figure shows the difference in MAE when comparing the current and the hybrid schemes. MAE is separately calculated for all stops with respect to the stop at which the predictions have been made. Each cell in the figure represents percent-age of improvement in MAE for predictions which have been made at stop for stop . The hybrid scheme shows more robust predictions for all cells except for the few blue cells. However, for these cells the performance decreases only marginally - in the worst case the performance decreases by 3.5 seconds ( .

The figure shows a pronounced improvement in performance following each TPS. It could be caused by higher adaptability of the hybrid scheme with respect to the extent to which drivers adhere to the timetable are able to recover the planned schedule.

(10)

Figure 6 MAE difference between the current and hybrid schemes

The performance of prediciton schemes may vary as a function of the prediction horizon – elapsed time between the prediction generation time and the relaized arrival time. Figure 7 presents the performance of both prediction schemes with respect to prediction horizon. The horizental axes represents the number of intermediate stops between and . Note that the number of observations vary for different prediction horizons – 31 intermediate stops are pre-sent only for predictions made at first stop for the last stop; whereas, horizon equal to one is a case for all stops exept the first stop. In adition, the number of predictions for each horizon is shown in the figure.

(11)

Figure 7 The performance measurement for the current and hybrid schemes respect to prediction horizon Overall, the hybrid scheme provides better predictions except for very long term predic-tions where both schemes yield the same performance. There is a high improvement for mid-term predictions. This may suggest that a potential area for improving the performance of the hybrid scheme is to calculate our GA parameter separately for different prediction horizons.

5 CONCLUSIONS

In this paper we present a heuristic hybrid scheme for transit departure time prediction. The hybrid scheme consists of three elements for calculating travel time prediction. It takes scheduled, instantaneous and historical travel time data. All these data are provided through existing timetables and recorded AVL data. The hybrid scheme is compared versus the cur-rently implemented scheme in Stockholm transit system.

The paper presents results from different perspectives in order to give a comprehensive performance evaluation regarding the both prediction schemes. Both schemes were employed for real-world data of a bus line in Stockholm transit network. The results show that the hy-brid scheme has better performance for short-term and mid-term predictions than the current scheme. The hybrid scheme is also generally more reliable and accurate but induces a larger systematic underestimation of bus arrival time compared with the current one.

In this paper, only one set of weights is calculated for the whole of the bus line. This could be further enhanced by computing these weights dynamically. A follow-up study will validate the proposed method by applying it on independent datasets. Moreover, considering headway through the prediction process could be another direction for further research.

The proposed hybrid scheme could be valuable for both passengers and operators. Passen-gers will receive more reliable departure predictions and it could lead to better trip planning decreasing their travel time. Moreover, operators could anticipate deviations from the planned service in order to support decisions concerning mitigation measures.

REFERENCES

[1] O. Cats and G. Loutos, Real-Time Bus Arrival Information System – An Empirical Evaluation. The 16th International IEEE conference on Intelligent Transportation Sys-tems (ITSC), The Netherlands, 2013.

0 20000 40000 60000 80000 100000 120000 140000 -20 0 20 40 60 80 100 120 140 160 180 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 M A E Prediction horizon

(12)

[2] Bin Yu, William H.K. Lam, Mei Lam Tam, Bus arrival time prediction at bus stop with multiple routes, Transportation Research Part C: Emerging Technologies, Volume 19, Issue 6, 1157-1170, 2011.

[3] M. Chen, X. Liu, J. Xia, S.I. Chien, A Dynamic Bus-Arrival Time Prediction Model Based on APC Data. Computer-Aided Civil and Infrastructure Engineering, 19, 364-376, 2004.

[4] N.H. Vu, A.M. Khan, Bus running time prediction using a statistical pattern recognition technique. Transportation Planning and Technology, 33, 7, 625-642, 2010.

[5] Y. Bin, Y. Zhongzhen, Y. Baozhen, Bus Arrival Time Prediction Using Support Vector Machines. Journal of Intelligent Transportation Systems, 10, 4, 151-158, 2006.

[6] Z. Ma, J. Xing, M. Mesbah, L. Ferreira, Predicting short-term bus passenger demand us-ing a pattern hybrid approach. Transportation Research Part C: Emergus-ing Technologies, 39, 148-163, 2014.

[7] S. Chien, Y. Ding, C. Wei, Dynamic Bus Arrival Time Prediction with Artificial Neural Networks. Journal of Transportation Engineering, 128, 5, 429-438, 2002.

[8] F.W. Cathey and D.J. Dailey, A Prescription for Transit Arrival/Departure Prediction using AVL Data. Transportation Research C, 11C, 3-4, 241-264, 2003.

[9] P. Tétreault, A. El-Geneidy, Estimating bus run times for new limited-stop service using archived AVL and APC data. Transp. Res. A, 44(6), 390–402, 2010.

[10] R.P.S. Padmanabanl, K. Divakarl, L. Vanajakshi1, S. C. Subramanian, Development of a real-time bus arrival prediction system for Indian traffic conditions. IET Intell. Transp. Syst., 4(3), 189–200, 2010.

Cytaty

Powiązane dokumenty

[r]

Porównanie pod wzglêdem innych cech rysunków dzieci z badanych grup Analiza porównawcza rysunków wykazuje, ¿e w zestawieniu z grup¹ kontrol- n¹, w rysunkach dzieci z

As an example of a graph achieving the lower bound, the complete graph K n has cover time determined by the Coupon.. ∗ Research supported in part by NSF

Piotr Fast (redaktor naczelny), michał Głuszkowski, Justyna Pisarska, Joanna Darda-Gramatyka, Paweł Łaniewski (sekretarz redakcji) Adiustacja tekstów obcojęzycznych.

In the management of risk four stages can be dis- tinguished (Figure 6) identifi cation quantifi cation decision reduction control.. In these the decision is not so much a phase, but

*) aparaty Fuji stawiają na funkcjonalność aparatów analogowych, większość ustawień jest dostępna bezpośrednio za pomocą odpowiednich pokręteł. Nie dysponujemy typowymi

FREQUENCY DISTRIBUTION OF CHOICE BEHAVIOUR STRATEGIES The differences in the relative frequency of the 1988 and 1997 answering sequences might show whether or not the

moments and quantiles of the empirical distribution, so they are estimators of the corresponding theoretical values. sample mean = estimator of the expected