Continuous rating of perceived visual-inertial
motion incoherence during driving simulation
Cleij Diane 1,2, Venrooij Joost 1, Pretto Paolo 1, Pool Daan M. 2, Mulder Max 2, Bülthoff Heinrich H. 1
(1) Max Planck Institute for Biological Cybernetics, Spemannstr. 38, 72076, Tübingen, Germany, {diane.cleij, joost.venrooij,paolo.pretto,heinrich.buelthoff}@tuebingen.mpg.de
(2) TU Delft, Kluyverweg 1, 2629 HS, Delft, The Netherlands {d.m.pool, m.mulder}@tudelft.nl
Abstract – Motion cueing algorithms (MCA) are used in motion simulation to map the inertial vehicle motions
onto the simulator motion space. To increase fidelity of the motion simulation, these MCAs are tuned to minimize the perceived incoherence between the visual and inertial motion cues. Despite time-invariant MCA dynamics the incoherence is not constant, but changes over time. Currently used methods to measure the quality of an MCA focus on the overall differences between MCAs, but lack the ability to detect how quality varies over time and how this influences the overall quality judgement. This paper describes a continuous subjective rating method with which perceived motion incoherence can be detected over time. An experiment was performed to show the suitability of this method for measuring motion incoherence. The experiment results were used to validate the continuous rating method and showed it provides important additional information on the perceived motion incoherence during a simulation compared to an offline rating method.
Keywords: driving, simulation, motion, cueing, evaluation
Introduction
Motion-based vehicle simulators are used for a wide variety of applications. They are an increasingly important tool for pilot training, research and vehicle system development. However, one of the main challenges in motion-based simulation is to cope with the typically limited workspace of the simulator. To map the vehicle inertial motions onto the simulator motion space, use is made of a Motion Cueing Algorithm (MCA). As the simulator motion space is much smaller than the vehicle motion space, this process inherently results in differences between the visual and inertial motion cues presented in the simulator. These differences can make it difficult for the observer to integrate the visual and inertial motion cues into a coherent percept of motion, which leads to a decrease in simulation realism. MCAs are therefore tuned to minimize the differences between motion cues.
The perceived motion coherence or incoherence, however, does not only depend on the absolute differences between motion cues, but also on the integration process of these motion cues in the human brain. Similar sized cueing errors therefore do not necessarily result in the same perceived motion incoherence. For example, scaling errors are often perceived as more coherent than similar sized false cues [Gra97]. This results in the perceived motion incoherence changing over time. MCAs are often designed to minimize this
perceived motion incoherence by including models of human perception [Ell05, Tel02, Bas11]. To test the resulting overall MCA quality, human-in-the-loop experiments are often performed. During these experiments the participants are subjected to vehicle simulations with different MCA tunings [Dam10, Val09, Ko12]. By analysing differences in control behaviour and/or answers to questionnaires differences in MCA quality are detected. Such methods are designed to measure general differences in MCA quality, but are not capable of measuring the time-varying aspect of the MCA quality, caused by changes in perceived motion incoherence over time. Without this time dimension one can only speculate as to which aspects of the simulation are actually influencing MCA quality. This information is of great importance to better understanding which motions are perceived as incoherent and with that improve MCA quality. Currently, however, there is no measurement method available with which the time time-varying aspect of the perceived motion incoherence can be measured.
The goal of the experiment described in this paper is to investigate whether continuous rating (CR) methods, used in other research fields such as 3D television quality and music analysis, can be a useful tool for the analysis of the time-varying aspects of the perceived motion incoherence in motion simulation. In the following sections a background of the CR method is described, together with a discussion on how such a method can be validated. This is followed the experiment
description and the presentation of the results that were obtained. In the discussion section the results are further analysed and explained and the suitability of the CR method for measuring time varying motion incoherence is discussed.
Continuous Rating Method
Continuous rating (CR) refers to an online subjective rating, based on the method of magnitude estimation [Ste56]. This method allows the measurement of the perceived intensity of any physical stimulus. In the online version the observer is asked to continuously assign a value (magnitude) to certain sensory stimuli via a dedicated control interface.
In the field of 2D/3D television CR methods are used to assess the quality of the visuals by rating the visual stimuli on its visual comfort. In [Lam11] this method is used to relate the measured visual comfort to disparity and motion, while in [Fre99] the influence of 3DTV properties, such as perceived depth, on the feeling of presence are rated. In the field of music analysis CR is also used. In [Eer02] the method is used to measure the predictability of music over time, while in [Cow11] a CR method is used to relate levels of emotion to specific aspects of music. Finally, in [Sch08, Gir05], a CR method is used in a car motion simulator and a fixed base car simulator respectively. In these experiments strain/workload was rated continuously in different driving situations.
The rating method described in this paper is based on the rating methods described in the previously mentioned papers. The method consists of a training and a measurement section. The training section is split in two procedures. These procedures are based on the rating interface training used in [Liu12] and the training to familiarize the participant with the stimuli range as is done in [Lam11] and [Fre99]. In the following paragraph these sections are further explained.
Method
The CR is performed using a dedicated rating interface consisting of a rotary knob to express the rating and a rating bar displayed on the screen serving as visual feedback on the current rating. The rating interface is shown in Figure 1.
A maximum rating of 1, given by turning the rotary fully to the right, will result in a fully coloured rating bar. A minimum rating of 0 will result in a fully black rating bar. The participants are asked to provide a rating higher than zero when they perceive any incoherence between the visual and inertial motion cues. The height of the rating depends on the intensity of this incoherence. The highest rating should be given to the highest incoherence perceived during the full simulation trial.
Before starting with the measurement section the participants are trained to use the rating interface and to familiarize themselves with the full range of motion incoherences that are present during the full simulation trial. The training section is therefore split up in two procedures: the rating interface training (RInt Training) and the coherence range training (CohR Training). During the RInt Training the participants familiarize themselves with the rating interface via a simple control task, where they are asked to follow a second automatically adjusted rating bar. Subsequently, in the CohR Training, the participants familiarize themselves with the full range of motion incoherencies that can occur in measurement section. They also familiarize themselves with the task of rating this incoherence continuously. To this end the participants will be asked to continuously rate the motion incoherence during a vehicle simulation. The training should be repeated several times to check if the participant can provide a consistent rating. At the end of this training the participants should thus have learned to use the full range of the rating bar, i.e. when no motion incoherence is felt provide a rating of 0 and when the maximum motion incoherence during the simulation trial is felt provide a rating of 1.
In the measurement section participants are asked to continuously rate the motion incoherence in the simulation trial, using the rating interface as was trained during the CohR Training. For verification of consistency it is beneficial to rate the simulation trial multiple times.
Method Validation
The most important properties of a measurement method are reliability and validity of the measurements. In the following paragraphs these properties are further explained.
Reliability
The reliability analysis is done to answer the following question: can participants provide consistent continuous ratings throughout simulation trials? To this end the reliability analysis will focus on within-subject consistency between 3 consecutive ratings of the same simulation trial. For
Figure 1: Rating interface consisting of a rating device and a rating bar.
Rotary knob Rating Bar
each participant the reliability estimate Cronbach’s Alpha [Cro51] is calculated, which is a metric for the expected correlation between the ratings of the 3 trials. A value for the Cronbach’s Alpha of 0.7 or higher is generally used to indicate acceptable reliability. If acceptable reliability is obtained, the mean of the ratings over the 3 trials is used for the validity and sensitivity analysis. If acceptable reliability is not obtained, the corresponding participant is excluded from further analysis as he/she is assumed to not have been able to perform the CR task.
Validity
The validity analysis will be done to answer the following question: is the CR method actually measuring perceived motion incoherence? To answer this question it is assumed that the retrospective offline rating, which is commonly employed in MCA analysis, is indeed measuring MCA quality caused by changes in perceived motion incoherence over time. The initial question then becomes:
1) Is there a significant relation between the offline and continuous ratings?
If so, it can be assumed that the CR method is indeed measuring perceived motion incoherence. The CR for all participants should also be consistent enough to detect differences between MCAs, which leads to a second question:
2) Can the CR method be used to detect significant differences between MCAs?
To answer these questions an additional rating, referred to as the retrospective offline rating (OR), will be done per simulation segment. Each segment consists of the motions during one manoeuvre calculated with one specific MCA. The OR is done after observing a simulation segment and refers to the overall MCA quality during this segment. In order to compare the two rating methods the CR has to be summarized in one value. Research on CR of video quality described in [Han01] shows that the offline rating is best predicted from the CR by the peak impairment, i.e. the maximum impairment during the observation sequence. It is assumed that this is similar for the rating of motion incoherence and therefore the CR will be summarized by the maximum incoherence rating during a simulation segment.
To answer question (1) a correlation between the offline and continuous ratings needs to be calculated. For this calculation the ratings of both methods are normalized per participant and the mean over all participants per segment is calculated. The correlation coefficient between the offline and continuous rating over the resulting data points is then calculated. The correlation coefficient that is obtained will be tested for significance as mentioned in [How13] via the t-test calculated with
Eq 1. Here N is the amount of test items and r the correlation coefficient.
To answer question (2), repeated measures ANOVAs and post hoc tests are performed for each manoeuvre to determine if the CR can be used to find significant differences between MCAs. To check if differences between MCAs were expected, a similar analysis is done on the OR. Ideally the post hoc analysis would show significant differences between the same MCAs.
Method Evaluation
The main difference between the OR and CR method is that the CR has an extra time dimension. The method evaluation therefore focusses on the additional information that can be obtained from this extra dimension. To this end the manoeuvres are split up in specific time bins that are related to different parts of the manoeuvre motion. One time bin during a curve driving manoeuvre can, for example, be the initial change of lateral acceleration, i.e. driving into the curve. A second time bin can be the constant lateral acceleration, i.e. driving through the curve. For this preliminary evaluation the following questions, related to the temporal resolution and sensitivity of the CR respectively, will be answered.
1) Can the CR be used to detect the source of the differences in MCA quality in a time bin smaller than the complete manoeuvre? 2) Does the CR reveal significant differences
between MCAs that were not found using the OR?
To answer these questions, repeated measures ANOVAs and post hoc tests are used to compare the maximum of the CR per MCA for each time bin.
Experiment
The experiment was performed to determine if a CR method can be used to measure time-varying perceived motion incoherence. For this experiment participants were passively observing a car simulation in a motion simulator. During the simulation different levels of motion incoherence were induced by varying MCA settings for different manoeuvres. The experiment lasted approximately 2 hours including breaks.
Apparatus
The experiment was performed in the CyberMotion Simulator at the Max Planck Institute for Biological Cybernetics. This is an eight degrees-of- freedom motion simulator derived from an industrial robot manipulator (Kuka GmbH, Germany). The visuals
𝑡 =𝑟√𝑁 − 2
and vehicle inertial motions were generated using the vehicle simulation program CarSim (Mechanical Simulation, US) The rating interface, shown in Figure 1, consisted of a rotary knob (SensoDrive GmbH, Germany) to express the rating and a rating bar rendered on the dashboard of the virtual vehicle for visual feedback on the current rating.
Participants
In total 16 participants, 15 male and one female, aged between 22 and 38 partook in the experiment. The levels of motion cueing expertise and motion simulation experience ranged from novice to expert.
Task
The participants were first trained to use the control interface and familiarize themselves with the simulation via the RInt and CohR Trainings as described in the Method section. For the CohR Training two simulation trials were rated, after which the within subject consistency was visually checked. If a low consistency was detected a third training trial was given. After a short break the CR Measurement started where the participants were asked to observe and continuously rate 3 simulation trials. A rating trail contained 9 segments, each being a different combination of manoeuvre and MCA. After a second break the retrospective offline rating procedure was started. During this procedure the participants were asked to observe 9 short simulation trials, containing only one segment, and provide one offline rating after each trial using the rating interface. The same simulation segments were used throughout the experiment. For the CohR Training a fixed segment order was used. For the CR measurement the segment order was varied and always different from the order of the training trials. For the offline rating each trial always consisted of the same initial acceleration and final deceleration and one of the 9 segments. The order of these trials was randomized per participant.
Independent variables
The independent variables in this experiment were manoeuvre (3 levels) and MCA (3 levels), which were all embedded in a simulation trial, resulting in 9 different simulation segments. The following manoeuvres were used in in the simulation:
Manoeuvre CD: Curve Driving at 70 km/h
Manoeuvre BA: Braking from 70 km/h to full stop and again Accelerating to 70 km/h
Manoeuvre BCDA: Braking from 70 km/h to 50 km/h while entering the curve, Curve Driving at 50 km/h and Accelerating again to 70 km/h when exiting the curve.
With this choice of manoeuvres, the simulation consists of motion incoherencies in different motion channels. As shown in Figure 2, the manoeuvres CD and BA focus on the lateral and longitudinal
acceleration respectively, while the BCDA manoeuvre combines both accelerations.
The MCAs used in the simulation are modelled as classical washout filters. Between MCAs only the parameters for motion scaling and tilt rate limiting differ to obtain specific cueing errors.
MCAScal: Scaling
o Motion scaling (gain=0.6). Results in scaling and small rotational errors (<4 deg/sec)
MCATRL: Tilt Rate Limiting
o Rotation rate limiting to 1 deg/sec. Results in missing/false cues and unnoticeable rotational errors
MCANL: No Limiting
o No tilt rate limiting or scaling is applied. Results in large rotational errors (<7 deg/sec) In Figure 3 the motion errors for the different MCAs are shown for the manoeuvre CD.
Dependent Variables
The dependent variables were the CR of the simulation trial, repeated 3 times, and the retrospective offline rating of the 9 simulation segments.
CD BA BCDA
Figure 2: Longitudinal and lateral vehicle acceleration for the 3 manoeuvers used in a simulation trial.
MCAScal MCATRL MCANL
Figure 3: Lateral accelerations for visual and the inertial motion cues for 3 different MCAs during manoeuver CD.
Results
The reliability and validity results presented in this section are used to validate the method. The results related to temporal resolution and sensitivity are used to provide a first evaluation of the CR. The results are presented using boxplots showing the median and 25 and 75 percentile values.
Reliability
To test if participants can rate consistently over time, the continuous ratings were performed 3 times. The within-subject consistency was determined by the Cronbach’s Alpha of these three trials, using each time step as a separate measurement. All but one participant had an alpha of 0.7 or higher (mean 0.814, STD 0.084). Therefore the participant with an alpha of 0.617 was excluded from further analysis.
Validity
As mentioned in the Method Evaluation section, the validity analysis is done to answer the following questions: 1) is there a significant relation between the offline and continuous ratings? 2) can the CR method be used to detect significant differences between MCAs? For question (1) the correlation coefficient between the means of the standardized offline and continuous ratings over all participants is calculated for each of the 9 segments. Figure 4 shows the resulting mean ratings for both rating methods.
The correlation coefficient between the two rating methods is r=0.88. The corresponding t-test (two-tailed, N=9), mentioned in the Method Evaluation section, suggests that the found relation between the two methods is significant (p<0.01).
To answer question (2) repeated measures ANOVAs were performed on the results of both the offline and continuous rating to determine if significant differences between MCAs could be found. In the following paragraphs the corresponding results are shown.
Manoeuvre CD
An ANOVA on the CR of the manoeuvre CD showed a significant difference between MCAs
(p<.01). The post hoc test of the CR shows this difference is found between MCATRL and MCANL
(p<.01). A similar analysis was done on the OR where also a significant difference between MCAs was found (p<.05). However, the OR shows that this difference is found between MCATRL and
MCAScal (p<.05). Figure 5 does show similar trends
in the CR and OR rating where MCATRL is rated
most incoherent.
Manoeuvre BA
For this manoeuvre no significant differences between MCAs were found in the CR. The analysis of the OR also didn’t show any significant differences.
Manoeuvre BCDA
An ANOVA on the CR for the manoeuvre BCDA shows significant differences between MCAs (p<.001). The post hoc test shows that MCATRL is
rated significantly different from both MCAScal
(p<.05) and MCANL (p<.01). A similar analysis was
done on the OR where also a significant difference between MCAs was found (p<.001). The post hoc test also shows MCATRL is rated significantly
different from both MCAScal (p<.001) and MCANL
(p<.001). Figure 6 shows that MCATRL is rated most
incoherent for both CR and OR.
Evaluation
For the evaluation of the CR the manoeuvres were divided into several time bins chosen to extract specific part of the manoeuvre motion. These time bins are visualized using shaded colour bars on both the time series and the boxplots. For each time bin a repeated measures ANOVA was done on the maximum of the CR to detect differences between MCAs. Due to limited space only the results of two of the manoeuvres are used. The results of CD BA BCDA
Figure 4: Mean of the ratings for all participants per simulation segment for the two rating methods.
Figure 5: Boxplots for the resulting offline and continuous rating of manoeuver CD per MCA.
Figure 6: Boxplots for the resulting offline and continuous rating of manoeuver BCDA.
manoeuvre BCDA are used to answer the question related to the temporal resolution of the CR: 1) can the CR be used to detect the source of the differences in MCA quality in a time bin smaller than the complete manoeuvre? The results of manoeuvre BA are used to answer the question related to the CR sensitivity: 2) does the CR reveal significant differences between MCAs that were not found using the OR?
Temporal Resolution
In the validity analysis significant differences between MCAs were found for manoeuvre BCDA. In this paragraph the time series of the corresponding CR is analysed, to detect if the source of these differences can be found in time bins smaller than the complete manoeuvre.
Manoeuvre BCDA can be split up in 5 time bins as shown in Figure 7. The time bins are including the start of the deceleration (purple), driving into the curve (grey), driving through the curve with constant acceleration (orange), start of the acceleration (blue) and finally driving out of the curve (red).
ANOVAs were done on the CR of each of these time bins. Only for the time bins “driving into” (p<.05) and “driving out of” the curve (p<.001) significant differences between MCAs were found. Post hoc analysis shows that driving into the curve MCATRL is rated significantly different from MCAScal
(p<.05). The post hoc analysis for the time bin of driving out of the curve shows a significant difference between MCATRL and both MCAScal
(p<.01) and MCANL (p<.001). Figure 8 shows that in
both cases MCATRL was rated most incoherent.
Using the CR method it can thus be detected that the offline rating was dominated by the parts of the motion when driving into and out of the curve. Another noteworthy observation from this figure is that the false cue when driving out of the curve seems to be rated more incoherent than the missing cue when driving into the curve.
Sensitivity
In the validity analysis no significant difference was found between MCAs for manoeuvre BA. In this paragraph the time series of the corresponding CR is analysed to determine if significant differences between MCAs can be found when looking more into detail at the specific time bins. Manoeuvre BA can be split up in 6 time bins as shown in Figure 9. The time bins are including the deceleration (purple), the bump concluding the full stop (grey), the vehicle being stopped (orange), the bump caused by the start of the acceleration (blue), the acceleration (red) and finally the vehicle driving at constant velocity (green).
Figure 7: Manoeuver BCDA. Top/Middle: motion cues for 3 different MCAs. Bottom: Mean of the CR over all
participants per MCA.
Figure 8: Boxplots for resulting maximum of the CR over the time bins of the manoeuver BCDA.
Figure 9: Manoeuver CD. Top/Middle: motion cues for the 3 different MCAs. Bottom: Mean of the CR over all
ANOVAs on the maximum of the CR over each time bin showed significant differences for the “car stopped” time bin (p<.05), the “acceleration” time bin (p<.05) and the “constant velocity” time bin (p<.01). Post hoc analysis only shows a significant difference between MCANL and MCAScal (p<.05) for
the “acceleration” time bin and between MCATRL
and MCAScal (p<.05) and MCANL (p<.05) for the
“constant velocity” time bin. Figure 10 shows that during acceleration MCANL is rated more incoherent
than MCAScal and during constant velocity MCATRL
is rated most incoherent. These results show that the CR method is more sensitive to differences between MCAs than the offline rating method.
Discussion
The results showed that participants with and without prior experience or knowledge on motion simulation can be consistent when rating motion incoherence continuously. From the 16 participants only one participant did not pass the statistical test for consistency (Cronbach’s Alpha<0.7). This result is surprising as for most methods currently used to determine MCA quality, motion simulation expertise is required.
A comparison of the means over all participants of the OR and maximum CR per segment showed a significant linear relation (r=.88, p<0.01) between the two rating methods. This result suggests that the CR indeed measures perceived motion coherence. A second analysis, comparing the maximum CR of each MCA per manoeuvre, showed that the between subject consistency was high enough to detect differences between MCAs. Only for manoeuvre BA no differences between MCAs was found. However, a similar analysis on the OR indicated that for this particular manoeuvre the differences in overall MCA quality were simply not significant enough to be detected. The post hoc tests showed that for both the OR and the CR the MCATRL, which results in large false cues, was rated
most incoherent. This finding is consistent with literature on the severity of cueing errors [Gra97]. However, the results of the CD manoeuvre, where a slight difference between the continuous and offline rating was observed, suggest that using the maximum of the CR cannot fully explain the offline rating of a segment. Further analysis of the time
series data of the CR could result in a more accurate algorithm to predict an offline rating over a specific time frame. Overall, the reliability and validity results show that the CR method results in a valid measure for perceived motion incoherence. The main benefit of the CR over the OR method is the extra time dimension of the CR. The results showed that this extra dimension increases the temporal resolution of the MCA quality measurement, i.e. variations in the MCA quality could be detected within one manoeuvre. This increase in temporal resolution in turn resulted in higher measurement sensitivity. For the CR time series analysis the results per manoeuvre were divided into time bins, chosen to extract specific parts of the manoeuvre motion. The results showed that for manoeuvre BCDA, the sources of the differences between MCAs in the OR were the start and end of the curve. Here MCATRL was rated
significantly more incoherent than the other MCAs. For driving through the curve, on the other hand, the MCAs were not rated significantly different. The CR results also showed that the false cue, induced by MCATRL, was rated more incoherent than the
missing cue, induced by this same MCA. This finding is consistent with literature [Gra97]. These results suggest that the CR is indeed measuring the time-varying aspect of perceived motion incoherence.
The extra time dimension also caused the CR to be more sensitive to differences between MCAs than the OR. The OR results showed no significant difference between MCAs for manoeuvre BA. The CR analysis on specific time bins, however, showed significant differences between MCAs during the acceleration and at constant velocity. During acceleration MCANL was rated more incoherent
than MCAScal, which could be explained by the
increased sense of being rotated induced by MCANL. After the acceleration, where the vehicle
again had a constant velocity, MCATRL was rated
significantly more incoherent than the other two MCAs. The false cue induced by MCATRL is a likely
cause for this difference. The method evaluation results show that the extra time dimension of the CR indeed provides important information on the perceived motion incoherence that could not be obtained with the OR.
The current experiment included only a passive driving simulation. CR during an active driving task was investigated in [Sch11], where drivers were asked to continuously rate the subjective strain during an active driving task as well as during a passive replay of this simulation. In future research a similar experiment should be done to determine if the CR method can be used to measure motion incoherence during active driving tasks.
Figure 10: Boxplots for resulting max of CR over the time bins of the manoeuver BA.
Conclusion
This paper describes a first experiment using a continuous rating method to measure time varying motion incoherence. The results show that participants with different backgrounds and expertise in motion cueing and motion simulation are able to continuously rate motion incoherence during a driving simulation in a motion simulator in a consistent manner. The similarities between the results of the retrospective offline and continuous rating methods show that both methods are indeed rating the same underlying variable, i.e. motion incoherence. The main benefit of the continuous rating method has been demonstrated by the higher temporal resolution of the method: the continuous rating method can be used to detect the source of the overall differences in quality between MCAs within one manoeuvre. This increase in temporal resolution also resulted in a higher measurement sensitivity: differences between MCAs could be detected with the continuous rating method, when no differences were found using the offline rating method. To conclude, this paper shows that the continuous rating method results in a valid measure for perceived motion incoherence and provides important information on the time-varying aspect of this measure.
References
Baseggio M., Beghi A., Bruschetta M., Maran F. and Minen D. An MPC approach to the design of motion cueing algorithms for driving simulators, 14th
International IEEE Conference on Intelligent Transportation Systems, Washington, DC , 2011. Cowie R., Camurri A. and Glowinski D. SIEMPRE deliverable D4.1 - Results from the first series of experiments and first evaluation report, SIEMPRE
Project Consortium, 2011.
Cronbach L. J. Coefficient alpha and the internal structure of tests, Psychometrika, vol. 16, no. 3, pp. 297-334, 1951.
Damveld H. J., Bonten J. L. G., Happee R., van Paassen M. M. and Mulder M. Motion filter design for driver observation in hexapod car simulators, Systems Man
and Cybernetics, Istanbul, 2010.
Eerola T., Toiviainen P. and Krumhansl C. L. Real-time prediction of melodies: Continuous predictability judgements and dynamic models, 7th International
Conference on Music Perception and Cognition, Sydney, 2002.
Elloumi H., Bordier M. and Maizi N. Integrated model-based control with human perception models in driving simulation, IMACS world congress , Villeneuve d'Ascq, 2005.
Freeman J., Avons S. E., Pearson D. E. and IJsselsteijn W. A. Effects of Sensory Information and Prior
Experience on Direct Subjective Ratings of Presence,
Presence, vol. 8, no. 1, pp. 1-13, 1999.
Girard J. M., Wilczyk M., Barloy Y., Simon P. and Popieul J. C. Towards an on-line assessment of subjective driver workload, Driving Simulation
Conference, North America, Orlando, 2005. Grant P. R. and Reid L. D. Motion Washout Filter Tuning: Rules and Requirements, Journal of Aircraft, vol. 34, no. 3, pp. 145-151, 1997.
Hands D. S. and Avons S. E. Recency and Duration Neglect in Subjective Assessment of Television Picture Quality, Applied Cognitive Psychology, vol. 15, no. 6, p. 639–657, 2001.
Howell D. C. Statistical Methods for Psychology, Wadsworth: Cengage Learning, 2013.
Ko S. F. and Grant P. R. Development and Testing of an Adaptive Motion Drive Algorithm for Upset Recovery Training, AIAA Modeling and Simulation
Technologies Conference, Reston, 2012.
Lambooij M., IJsselsteijn W. A. and Heynderickx I. Visual discomfort of 3D TV: Assessment methods and modeling, Displays, vol. 32, no. 4, pp. 209-218, 2011.
Liu T., Cash G., Narvekar N. and Bloom J. Continuous mobile video subjective quality assessment using gaming steering wheel, Sixth International Workshop on
Video Processing and Quality Metrics for Consumer Electronics (VPQM), 2012.
Schießl C. Subjective strain estimation depending on driving manoeuvres and traffic situation, Intelligent
Transport Systems, IET, vol. 2, no. 4, pp. 258-265, 2008. Stevens S. S. The Direct Estimation of Sensory Magnitudes: Loudness, The American Journal of
Psychology, vol. 69, no. 1, pp. 1-25, 1956.
Telban R. J., Cardullo F. and Houck J. A nonlinear, human-centered approach to motion cueing with a neurocomputing solver, AIAA Modeling and Simulation
Technologies Conference and Exhibit, 2002.
Valente Pais A. R., Wentink M., van Paassen M. M. and Mulder M. Comparison of Three Motion Cueing Algorithms for Curve Driving in an Urban
Environment, Presence, vol. 18, no. 3, pp. 200-221, 2009.
