Temporal Aspects of User Experience:
Models and Methods Beyond a Single
Use Situation.
Abstract
User Experience (UX) is an ongoing process and should not be limited to a single use situation. However, this is unfortunately often the case in HCI research. The goal of this workshop is to deepen and expand available knowledge with respect to temporal dynamics of UX. Relevant aspects will be the understanding of how UX is evolving over time, in particular across products and contexts, as well as the exploration of methods that can evaluate long-term UX in practice. For this, the
organizers will prepare a booklet that introduces existing models and methods for long-term UX as a mutual basis for discussing critical experience phases that should be taken into consideration in product development. The booklet also summarizes pre-notes from participants which will be collected prior to the workshop. In addition, during an interactive group activity, participants will reflect on the diversity of long-term UX of different product types. Participants from academia as well as from industry are invited in order to tackle the question of how to make research results and methods applicable in practice.
Author Keywords
User Experience, dynamics, long-term usage, evaluation methods
Sari Kujala Aalto University School of Arts, Design and Architecture PO Box 31000 FI-00076 Aalto, Finland Sari.Kujala@aalto.fi Michael Minge
Technische Universität Berlin Graduiertenkolleg Prometei Franklinstr. 28/29, Sekr. FR 2-6 10587 Berlin, Germany mmi@zmms.tu-berlin.de Anna E. Pohlmeyer
Delft University of Technology Department of Industrial Design Landbergstraat 15
2628 CE Delft, The Netherlands A.E.Pohlmeyer@tudelft.nl
Marlene Vogel
Technische Universität Berlin Graduiertenkolleg Prometei Franklinstr. 28/29, Sekr. FR 2-6 10587 Berlin, Germany mvogel@zmms.tu-berlin.de
ACM Classification Keywords
H.1.2. User/Machine Systems: Human Factors; H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.
Introduction
Based on a survey of 275 UX researchers and practitioners conducted by Law’s et al. [1], user experience is most often characterized to be dynamic, context-dependent, and subjective. Many researchers also highlight the importance of emotions [2, 3, 4, 5] and the holistic and phenomenological nature of UX [6, 7]. Yet, the understanding of temporal aspects in user experiences is still evolving.
Recently, several studies have shown that the
significance of different UX qualities changes over time [8, 9, 10]. For example, Karapanos et al. [10]followed six participants for one month after the purchase of an Apple iPhone and found that, while the importance of its novelty and social meanings faded away quickly, different sources of hedonic quality, such as the participation of the product in daily rituals and cherished activities, gradually emerged.
It is also suggested that long-term UX – how the user’s experience and relationship with a product evolve over time from early learning and enthusiasm to its
becoming a part of daily life – is more important than single momentary experiences [11]. It is long-term user experience that makes people continue to use a product and recommend it to others – not individual experiences at a certain point in time [11]. Norman [12] and Karapanos et al. [13] argue for the importance of the memories of an experience in contrast to the actual experience itself, as memories
will be reported to others. The memories of the UX with one system (“retrospective experience”) are said to affect a user’s expectations relating to a new system and thereby also the likelihood of (re-) use [14]. Creating customer loyalty demands more sustained elicitation of feedback over longer periods of time. This workshop seeks to extend the knowledge of the dynamic nature of UX and ways of measuring it by an in-depth, interactive discussion and experience exchange.
Relevance to the field
The importance of temporal aspects, also with regard to product development and assessing UX, has been pointed out by a number of researchers [3, 10, 13, 14, 15].
The development and changes within as well as between the single elements of the triad user-system-context should be taken into account [14]. In addition to a variety of possible contexts, it is not only the user that changes over time (e.g. learn); the system too might mature (e.g. adapt) and eventually age (e.g. break). Furthermore, the dynamics of UX are influenced by the interconnection of user-system-context. A long-term relationship with all its ups and downs can result, sometimes even comparable to interpersonal love stories [16]. In order to design products that keep the user engaged in a long-term relationship, more insights and empirical evidence is needed with regard to relevant experience phases, in addition to means that enable assessing the dynamics of UX.
To conclude, the dynamic nature of UX is broadly acknowledged [1]. However, to date it is unclear how
the UX of different product/service types and within different contexts of use is evolving over time. As the identification of factors affecting UX over time and of the critical experience phases is essential for product development, the aim of this workshop is to investigate the long-term UX of different product types, and possibilities of longitudinal assessment.
There is a wide variety of UX evaluation methods [17, 18], but most of the existing studies and methods focus on very short-term experience intervals. Karapanos et al. [13] initiated a discussion on different evaluation approaches and their temporal horizon (cross-sectional, repeated sampling, longitudinal, retrospective) by highlighting changes of users’ experiences and behavior over time.
There are also some methods available to address long-term UX, e.g. Day Reconstruction Method [19], UX Curve [11], Experience Sampling Method [20], Ecological Momentary Assessment [21], Repeated Evaluation Technique [22]. However, selecting and using the appropriate method for a certain situation is challenging. Different methods may be suitable to different stages, products and contexts. In the workshop, practical experiences of using different methods will be taken into account. Furthermore, the costs and benefits of these methods for a variety of products and contexts will be explored.
In sum, the scope of the workshop is twofold. On the one hand, the focus is to understand and describe how people experience products over the course of time and what might be suitable methods for evaluation in different stages, for diverse products and contexts.
On the other hand, the role of influencing factors that might change too and will probably lead to different experiences in different stages of usage will also be discussed. Is it possible to identify characteristic phases for prolonged UX? And how do they differ for various products (services) i.e. hedonic, pragmatic and ubiquitous objects?
Activities planned
The aim of the workshop is to bring together current approaches of long-term UX and to reflect their strengths and weaknesses by creating and discussing usage scenarios and possible evaluation solutions for different products. Participants are encouraged to reflect the workshop theme over a longer period: The organizers will provide a brief opinionaire prior to the workshop in order to tailor the event itself to the background, current understanding of temporal UX, experience with different methods, and expectations of the participants. Thus, input for the discussion will be partly based on notes that participants submit before the conference, i.e. pre-notes. Responses will be collected in form of a booklet. This booklet will also include background information on the topic and will therefore serve as a base for the discussions on-site. The workshop itself consists of two parts, an introduction and an interactive group activity. First, goals and theme of the workshop as well as the participants themselves will be introduced. A theoretical and methodological
introduction will be provided by the organizers. Moreover, a summary of the group’s pre-notes will be presented and different perspectives highlighted.
During the second part of the workshop, participants will be engaged in an interactive group activity. The task will be to anticipate prolonged usage scenarios for different products. In iterative steps, sub-groups will reflect on changes in behavior, perceptions, and emotions and will develop an evaluation plan.
To conclude the workshop, an open, but structured discussion and outlook will follow.
Outcomes of the workshop
This workshop will be distinctly different from sessions of the main conference – there will be no paper presentations of individual work. The focus is rather to create an overarching overview of the topic by
providing background information that becomes mutual and by collectively generating new ideas in an
interactive format.
Consequently, the workshop outcomes will be:
an overview of current perspectives on temporal aspects of UX. For this, the organizers provide a booklet that integrates the participants’ pre-notes as well as theoretical and methodological input
raising awareness of the diversity of long-term UX for different products and in different phases
sharing and documenting experiences regarding the costs and benefits of methods measuring temporal aspects of UX
creating a long-term network between researchers and practitioners with experience and/or an interest in the temporal dynamics of UX, hence, between the participants.
Additional information
Intended audience
We welcome a maximum of 18 and a minimum of six participants from academia as well as from industry who are working in the field of User Experience. A variety of disciplinary backgrounds (industrial design, communication design, computer science, psychology, marketing, neuroscience, etc.) is welcomed.
Duration of the workshop
Half-day
Organizers’ background
The organizers are UX researchers from academia with a special interest in temporal, dynamic aspects of UX and prolonged system usage. They have different disciplinary backgrounds (psychology, design, human-computer interaction) and have experience in
organizing interactive workshops and special interest groups (e.g. at CHI, NordiCHI, Design and Emotion).
Needed facilities
Conference room with space for roughly 20 participants
AV-Media, including a beamer
Space to hang up flip chart papers
References
[1] Law, E., Roto, V., Hassenzahl, M., Vermeeren, A., Kort, J. Understanding, Scoping and Defining User Experience: A Survey Approach. In Proc. CHI 2009, ACM Press (2009), 719-728.
[2] Forlizzi, J. and Battarbee, K. Understanding
experience in interactive systems. In Proc. DIS 2004, ACM Press (2004), 261-268.
[3] Hassenzahl M. and Tractinsky, N. User experience - a research agenda. Behaviour & Information
Technology 25, 2 (2006), 91-97.
[4] Isomursu, M. Tähti, M. Väinämö, S. and Kuutti, K. Experimental evaluation of five methods for collecting emotions in field settings with mobile applications. International Journal of Human-Computer Studies, 65, 4 (2007), 404-418. [5] Mahlke, S. Understanding users' experience of
interaction. In Proc. ECCE 2005, (2005), 251-254. [6] Wright, P. and McCarthy, J. Technology as
Experience. Cambridge, MA, USA: MIT Press, (2004). [7] Swallow, D., Blythe, M., and Wright, P. Grounding
experience: relating theory and method to evaluate the user experience of smartphones. In Proc. European association of cognitive ergonomics, (2005), 91-98.
[8] Fenko, A., Schifferstein, H.N.J., and Hekkert, P. Shifts in sensory dominance between various stages of user-product interactions. Applied ergonomics 41, 1(2010), 34-40.
[9] Karapanos, E., Hassenzahl, M., and Martens, J. B. User experience over time. CHI 2008 extended abstracts, ACM Press (2008), 3561–3566. [10] Karapanos, E., Zimmerman, J., Forlizzi J., and
Martens, J.B. User Experience over Time: An Initial Framework. In Proc. CHI 2009, ACM Press (2009), 729-738.
[11] Kujala, S., Roto, V., Väänanen-Vainio-Mattila, K., and Karapanos, E. UX Curve: A method for evaluating long-term user experience. Interacting with Computers, 23 (2011), 473-483.
[12] Norman, D. A. Memory is more important than actuality. Interactions, March/April (2009), 24-26. [13] Karapanos, E. Zimmerman, J. Forlizzi, J., and
Martens, J. B. Measuring the dynamics of
remembered experience over time. Interacting with Computers, 22, 5 (2010), 328-335.
[14] Pohlmeyer, A.E. Identifying Attribute Importance in Early Product Development. Technische Universität Berlin, Ph.D. thesis (2011).
[15] Courage, C., Jain, J., Rosenbaum, S. Best practices in longitudinal research. In: Proc. CHI 2009, ACM Press (2009), 4791–4794.
[16] Russo, B., Boess, S., and Hekkert, P. ‘What’s Love Got to Do With It?’ The Experience of Love in Person-Product Relationships. The Design Journal, 14, 1 (2011), 8-27.
[17] Bargas-Avila, J. and Hornbæk, K. Old Wine in New Bottles or Novel Challenges? A Critical Analysis of Empirical Studies of User Experience. In Proc. CHI 2011, ACM Press (2011), 2689-2698.
[18] Vermeeren, A., Lai-Chong Law, E., Roto, V., Obrist, M., Hoonhaut, J., and Väänänen-Vainio-Mattila, K. User experience evaluation methods: current state and development needs. In Proc. NordiCHI Conference, (2010).
[19] Kahneman, D., Krueger, A.B., Schkade, D.A., Schwarz, N., and Stone, A.A. A survey method for characterizing daily life experience: The Day Reconstruction Method. Science, 306 (2004), 1776– 1780.
[20] Csikszentmihalyi, M., and Larson, R. Validity and reliability of the experience sampling method. Journal of Nervous and Mental Disease, 775 (1987), 526-536.
[21] Schiffman, S., Stone, A.A., and Hufford, M.R. Ecological Momentary Assessment. Clinical Psychology, 4 (2008), 1–32.
[22] Carbon, C.C. and Leder, H. The Repeated Evaluation Technique (RET). A Method to Measure Dynamic Effects of Innovativeness and Attractiveness. Applied Cognitive Psychology, 19 (2005), 587-601.
