Application Sharing
Services
Design and Performance
PROEFSCHRIFT
ter verkrijging van de graad van doctor aan de Technische Universiteit Delft,
op gezag van de Rector Magnificus prof. dr. Ir. J.T. Fokkema, voorzitter van het College voor Promoties,
in het openbaar te verdedigen op dinsdag 25 april 2006 om 13:00 uur
door
Jenny Joyce BEUMER doctorandus in de cognitieve ergonomie
Prof. dr. J.H.T.H. Andriessen Samenstelling promotiecommissie: Rector Magnificus, voorzitter
Prof. dr. J.H.T.H. Andriessen, Technische Universiteit Delft, promotor Prof. dr. F.W.G. van den Anker, Universiteit Hamburg
Prof. dr. H.G. Sol, Rijksuniversiteit Groningen
Prof. dr. W. Veen, Technische Universiteit Delft
Prof. dr. ir. G.J. de Vreede, University of Nebraska at Omaha
‘There are more things in heaven and earth, Horatio, Than are dreamt of in your philosophy.’
William Shakespeare, Hamlett, Act 1. Scene V
Published and distributed by: Jenny Joyce Beumer
Van der Kamlaan 76 2625 KP DELFT The Netherlands
Phone: +31 (0) 15 380 54 94
English editor: Nick Rowling
Printing and binding: Druk.Tan Heck, Delft Cover picture: Ans Beumer
Jenny Joyce Beumer
Application Sharing Services: Design and Performance Doctoral Dissertation, Delft University of Technology ISBN-10: 90-6824-021-8
ISBN-13: 978-90-6824-021-4
Keywords: new technologies, design, performance, application sharing, remote support, distance rehabilitation, remote control
Copyright © 2006 by Jenny Joyce Beumer
My thanks to everybody who helped me with preparing my dissertation. It would not have been possible for me to finish it without all of the help I received. First, I will thank my husband Jelle. You gave me all the emotional support I needed, and took the care for our children Anneke, Saskia and Herbert. Moreover, you gave me a lot of content related help: you did the data analysis and found some excellent literature references. I want to thank my promoter and co-promoter, Erik and Marion: for content related help during the research and the fine-tuning of my dissertation. Erik, thank you for the long discussions that gave direction to the content of the dissertation. I was always looking forward to these appointments. Furthermore, I was really impressed by the many times that you read my dissertation to fine-tune the document. I thank you for all your efforts. I would also like to thank my previous supervisors. Ab de Haan, you helped me with the first part of my research: the foundation, the first experiments, writing the first articles and you motivated me to carry out this research. Harry Bouwman, you helped me a lot with structuring my thoughts and helped me to learn how to write scientific publications. I want to thank my father: thank you for all your corrections concerning language, in English and in Dutch. You read my complete dissertation several times in different phases. You really did a great job. Thanks to my student assistants Marijke, Erik, and Eline and Elise for typing out audio and video tapes, analyzing data and searching on the Internet and in libraries.
Many thanks are due for the two organizations that participated in the casestudies. I would like to thank Stichting Het Loo Erf, the national rehabilitation center for visually impaired persons, and the sponsoring fund ‘Vrienden van het Loo Erf’, Rob Meijer, director of the center, Herman Philipsen, project leader, Bert Ebbe, technical support, Annemiek
Wildenberg and all the employees and rehabilitants of the center. Without your help it would not have been possible to conduct this research. I also wish to thank the sponsoring organization Center for their additional support.
I would like to thank the Dutch Inland Revenue Services (Dutch IRS), Harm Jan van Burg, commissioner, for arranging the financial support of the Dutch IRS, Maarten Visser, project leader, for all your assistance during the research: contacting the necessary stakeholders, coordinating meetings and for your availability every time it was required. My thanks also go to all stakeholders of the Dutch IRS for their input, and thanks to all employees and taxpayers’ volunteers who participated in the experiment.
Blindenvereniging’ for their financial support. For the finishing touch I would like to thank Miranda Aldham Brearry for the English corrections, Nick Rowling for his English translations, Tamrat, who had just finished his Ph.D. at TPM, Helen and Lieneke, secretaries, and Ype, my brother in law, you made a lot of corrections, thank you. Finally, thanks to my sister Riëtte, and all my friends and relatives who supported me in practical ways at home and emotionally. You all made this dissertation possible. Thank you all very much.
1 Introduction ... 14
1.1 Setting the scene ... 14
1.2 Research question ... 17
1.2.1 Sub question A1: ... 17
1.2.2 Sub question A2: ... 18
1.2.3 Sub question B: ... 18
1.3 Composition of the dissertation ... 19
1.3.1 Chapter 2: Orientation on ASAS services... 19
1.3.2 Chapter 3: Method: ASAS services design and performance .... 20
1.3.3 Chapter 4: Scenario’s for ASAS based services ... 20
1.3.4 Chapter 5: Experiment based evaluation... 20
1.3.5 Chapter 6: Scenario based evaluation... 20
1.3.6 Chapter 7: Discussion and conclusions... 20
2 Orientation on ASAS services ... 22
2.1 Orientation on ASAS technology ... 22
2.1.1 ASAS technology in Computer Supported Collaborative Work and GroupWare Systems ... 24
2.1.2 ASAS technology in education and instructional systems ... 25
2.1.3 ASAS technology in E-CRM and call centers... 26
2.2 Choosing ASAS technology... 29
2.3 Factors influencing performance in future ASAS services ... 32
2.3.1 ASAS services and the expected performance effects... 33
2.3.2 Factors influencing performance ... 34
2.3.3 Hypotheses ... 36
2.4 Discussion and conclusions ... 38
3 Method: ASAS service design and performance... 40
3.1 Towards a new design approach ... 40
3.2 Designing ASAS services: EXSE - Early eXperiment and Scenario Based Evaluation ... 43
3.3 Performance testing ... 46
3.3.1 Testing the hypothesis – methodology... 47
3.3.2 Organisational performance related... 48
3.2.3 Individual performance... 53
3.4 Discussion and conclusions ... 57
4 Scenario’s for future ASAS based services ... 60
4.1 Method... 60
4.1.1 Procedure... 60
4.1.2 The interviews ... 61
4.2.2 Services and Additional Scenarios for VICWs Support with ASAS
Services... 64
4.3 Casestudy two: Support for Dutch tax payers ... 68
4.3.1 The Dutch Inland Revenue Services ... 68
4.3.2 Preferred ASAS technology features ... 69
4.3.3 Services and scenarios for taxpayer support... 71
4.4 Discussion and conclusions ... 74
5 Experiment based evaluation ... 78
5.1 Choosing ASAS technology for the 2 casestudies... 78
5.2 Experiment one: rehabilitation at a distance ... 79
5.2.2 Results... 82
5.2.3 Conclusions ... 86
5.3 Casestudy 2: Supporting taxpayers at a distance ... 87
5.3.1 Method ... 87
5.3.1.7 Analysis... 91
5.3.2 Results... 92
5.4 Summary and conclusions...100
6 Scenario based evaluation ... 102
6.1 Method...102 6.1.1 Hypotheses ...102 6.1.2 Subjects ...103 6.1.3 Interviews ...104 6.1.4 Questionnaire ...104 6.1.5 Workshops ...105
6.2 Casestudy 1: Support for Dutch Visually Impaired Computer Workers ...106
6.2.1 Introduction ...106
6.2.2 Results concerning hypotheses ...106
6.2.3 Conclusions ...110
6.3 Casestudy 2: the Dutch Inland Revenue services ...111
6.3.1 Introduction ...111
6.3.2 Results concerning hypotheses ...111
6.3.3 Conclusions ...115
6.4 Discussion and conclusions ...116
7: Discussion and conclusions ... 120
7.1 Introduction...120
7.2 Audio-Supported Application Sharing services...120
7.3 Performance effects...121
7.3.1 Performance effects as measured in experiments ...122
7.3.2 Expert judgements over performance effects ...124
7.4 Design method EXSE ...126
7.4.1 Comments about the design method ...128
7.5 Answering the main question ...130 7.6 Generalisability...132 7.7 Follow-up research ...133 7.8 Practical implications ...133 7.9 In conclusion ...135 References... 138 List of abbreviations ... 143
Appendix 1: Available Application Sharing Software ... 145
Appendix 2: Design related interview protocols and questionnaires ... 150
Interview protocol Case-study 1: Support for Dutch Visually Impaired Computer Workers ...150
Interviewprotocol revalidant 5 april ’01 ...150
Interviewprotocol medewerker Loo Erf 12 april ’01...151
Interview protocol Case-study 2: Support for Dutch tax payers...154
Evaluation questionnaire Casestudy 2: Support for Dutch tax payers..156
Vragenlijst Project Multimedia Support voor belastingplichtigen ...156
Nut en mogelijkheden van ASAS-Technologie bij het geven van Hulp Bij Elektronische Aangifte (HUBEA) ...156
Verwachte invloed van ASAS-technologie op kwaliteit van hulp bij aangifte ...156
Verwachte invloed van ASAS-Technologie op efficiency ...157
Waargenomen NUT van ASAS-Technologie...157
Overige vragen ...158
Appendix 3 Design requirements for ASAS services ... 159
Design requirements casestudy 1 ...159
Technology related requirements...159
Task related requirements...160
User related requirements...161
Official related requirements ...161
Organisation related requirements ...161
Design requirements casestudy 2 ...163
Technology related requirements...163
Task related requirements...164
User related requirements...165
Organisation related requirements ...166
Introduction related requirements ...166
Samenvatting ... 169
Ontwerp van ASAS-diensten...172
Resultaten ...172
Individuele taak prestatie...172
Voor- en nadelen voor de organisatie...173
De ontwerpmethode...174 Generaliseerbaarheid...174 Vervolgonderzoek ...175 Tenslotte...176 Summary ... 177 Method ...178
Performance effects of ASAS services ...179
Design of ASAS services...180
Results...181
Individual task performance ...181
Advantages and disadvantages for the organization ...182
Design method EXSE...183
Generalisability ...184
Follow-up research...185
In conclusion ...185
1 Introduction
Sunday, 22.00 o’clock: a visually impaired computer user phones a friend: “I have been struggling all weekend. Last month I decided that I wanted to use mail at home, but my Braille line and speech module didn’t want to show and read the mail messages. I phoned the supplier of my computer aids. He told me I had to buy another version of my Braille software, so I did. I had a lot of problems: first, it took me a long time to install the software: I had to make a lot of choices and I really did not know what to do. Then, after I installed this version, I could no longer use the Dutch language on my speech synthesizer. It was talking an illiterate type of English. Finally, after installation, I wanted to start the Internet Explorer, but I couldn’t establish any connection due to a modem problem. It is not installed, and I couldn’t find it in the list of modems. It took me my whole weekend! It’s driving me crazy, can you please help me?”
1.1 Setting the scene
I encountered many computer problems during my rehabilitation period at the Dutch national rehabilitation center for visually impaired people ‘Stichting Het Loo Erf’, and observed that other visually impaired persons had similar problems. It is quite easy to imagine how difficult it is for a visually impaired person to work with a computer. First they have to learn to work with the aids, and only then they can start to learn a specific application, such as Microsoft Word ©. When using an application, Visually Impaired Computer Workers (VICWs) will be constantly hindered by their handicap: they always 'see' only a small part of the screen at a certain moment. In Braille, they read a half to a full line at a time; using a speech synthesizer, they hear all the information in succession, and with a magnification package they see again only a small part of the screen at a time. This means that VICWs have little or no overview of the screen, so that certain actions are less self-evident for them than for a well-sighted computer user (Beumer & Haan, 2000; Beumer, Jameson, & Waterham, 2000). As a result of the rapid growth in new computer applications and computer adaptations for VICWs there are long waiting lists for places at special computer training rehabilitation centers, and there is thus a need to find ways to decrease these waiting lists.
A new technology concept called Audio Supported Application Sharing (ASAS) can be used in services to solve these computer problems at a distance. An ASAS service enables a trainer at a rehabilitation center to help a VICW by telephone and to ‘look over their shoulder’, because a copy of the digital information presented to the VICWs, e.g. a certain application or a web page, is sent to the trainer, who can see the problem directly. The trainer can point and click at an informationitem presented on the screen of the VICW (application sharing).
acquired the knowledge they need to work with the computer aids and applications they can contact the rehabilitation center on an incidental basis if they have problems with an application or a computer adaptation tool. Society profits from the rapid rehabilitation of the VICW, their brief absence from the labor market, and from a cheaper rehabilitation process. The use of ASAS services is not restricted to helping VICWs, but can also be expanded to encompass all computer users who have problems using their computer. ASAS services can be used to support helpdesks when clients phone in with computer related problems. They can be used to support distance education. They can be used for homecare advice for the sick, handicapped and elderly, and to support those who cannot go out to shop or visit friends and family. ASAS services can be used to support such groups digitally and to help them to become less isolated and dependent on others.
Although the ASAS technology comes as standard with such frequently used systems as Windows 2000 (Microsoft NetMeeting ©) it is infrequently used. This is a pity, because it could have a significant impact. Implementing the ASAS service concept to support VICWs at a distance would, for instance, give them the possibility to learn the skills they need at their place of work, where they can apply them immediately. Implementing ASAS services when providing helpdesk services, like the helpdesk of the Dutch Inland Revenue Service, would allow taxpayers to show their problems with their digital tax return form to a tax official. The tax official would then be able to see immediately what the taxpayers problem is and be able to provide a solution.
Three reasons can be given to explain why ASAS services are not, as yet, universally implemented to support people requiring computer aided helpdesk/rehabilitation services. First, ASAS services are too new and too complex to know in advance how they will be envisioned. One reason for this might be that the organisational communication processes will change when ASAS services are increasingly commonly used in the future. Secondly, little is known about the way an organisation or individual users may benefit from an ASAS service. Three, even if it is known how the ASAS services will be envisioned, and how it may influence an organisations’ or users’ efficiency, effectiveness and satisfaction, it is difficult to persuade the various stakeholders involved in an organisation to adopt the new service. This is because they have to change their current communication processes and many people do not like changes.
The development of a new service requires an extensive participatory design phase (De Sanctis & Poole, 1994). The reason for this is that the precise features of the service will depend on the future context in which it will be used. This context is not always a continuation of the current situation under consideration. Using a new service can sometimes change how an organisation will realize organisational goals e.g. its banking via the internet, and therefore changes the role of the organisation (Orlikowsky, 1992). Benefits, cannot always be predicted before a scenario has been developed (Van den Anker, 2003).
service concept will be incorporated in the organisation. This result will be obtained by using stakeholder interviews, the internal reports of the organisation in question and, if needed, a technology analysis in which the characteristics of the available technology are compared and matched to the organisational needs of the organisation in question. The scenarios will then be evaluated and further developed during interviews using an iterative process. During this process ideas extracted from previous interviews will be extended, or discarded, again using internal reports and a technology analysis if necessary. Throughout this process the interviewee will be free to give input to the process of designing the scenario(s). The second problem will be tackled by performing an experiment and prototype evaluation of the service-concept. The envisioned ASAS service concept described above will be used as input for this experiment. The experiment results will be used to show the organisation how it can benefit from the new services. This will be achieved in two different ways: at the level of individual behavior, and at the level of organisational performance. Measuring efficiency, effectiveness and satisfaction at the individual level requires task performance to be assessed. In this case, the tasks will be related to the user problems determined in the scenario, and the new service will be evaluated to see if its use decreases time needed to carry out specific tasks (efficiency), decreases incorrect answers (effectiveness) and increases the happiness of the persons who have to perform the tasks. These measurements will be obtained during an experiment based evaluation. Measuring at the organisational level requires that the productivity of the organisation is evaluated. In this situation efficiency, effectiveness and satisfaction can be defined as less money, time and manpower being needed to perform the services an organisation provides (efficiency), better and increased service can be provided using the new service (effectiveness) and the stakeholders involved in the organisation are more pleased with the system (satisfaction). Such measurements will be made during a scenario-based evaluation.
The third problem of implementing the new services, deals with persuading the various stakeholders of an organisation to implement and use the new services, will also be solved implicitly using the participatory design approach, described above. The involvement of people from operational to management levels in such a process creates a broad base. One may expect that the stakeholders will become enthusiastic about performance at the individual level when they interact with the questioner during the participatory design process as resistance towards the system will be reduced due to a better design (Markus, 1983; Muller and Kuhn, 1993). One may also expect that managers will be enthusiastic about performance at the organisational level.
In this dissertation the performance effects for two new ASAS services are described. To do so, it was necessary to design the services. Two casestudies are presented: one was carried out at the Dutch National Rehabilitation Center for visually impaired persons and the other at the Dutch Inland Revenue Services.
1.2 Research question
The main research question deals with measuring performance effects of two future ASAS services. Action research will be used for the design of these services. This will be combined with experiments to measure the differences in performance regarding the current way of support and the ASAS service support. That is, the future service in which the current support is extended with application sharing.
The main research question is formulated as follows:
What are the optimal ASAS service features in relation to its context and how do these features influence individual and organisational performance?
This research question needs to be divided into sub questions due to its complexity. This will allow step by step answering of the research question until the final, complete answer is obtained.
The first two sub questions are context related. The first context related sub question consists of two parts. In the first part the object of research is defined: the ASAS service and its features. In the second part the main aspects of the ASAS service are considered with help of two casestudies.
1.2.1 Sub question A1:
What are the existing ASAS service concepts, in which contexts can these services be used?
The criteria that ASAS services should meet will be analyzed to answer this sub question. Then, an analysis will be presented of the ‘state of the art concerning ASAS services’ and to what degree these services meet the criteria. ASAS service(s) will be described which match the context under consideration together with the method followed to determine an optimal ASAS service. Two different service contexts, or more specifically organisations, will be selected for this research given the above-mentioned ASAS service context match.
Second, a description will be presented of how the service context can be envisioned. Aspects related to the context will be specified. Taking these aspects into account, a description will be given of how the service will probably change communication processes due to the introduction of the ASAS service. This will be achieved using a participatory design approach, this information will then be used to formulate a future scenario.
1.2.2 Sub question A2:
Do ASAS services allow consulting with higher effectiveness, efficiency and satisfaction and do they change interaction intensity compared to present forms of service provision?
To answer this sub question, the available literature will be reviewed to give an explanation for the expected effects of new technologies such as ASAS technology on efficiency, effectiveness, satisfaction with and interaction intensity, and how aspects related to the context under consideration influence this. A theoretical model will be built to act as the foundation for the research presented in this dissertation.
Secondly, the sub question deals with analyzing ASAS service performance effects on efficiency, effectiveness, satisfaction with and interaction intensity. Parts of the future application will be tested in an experiment-based evaluation in which the features of the ASAS service and its related context will be manipulated. Individual effects on effectiveness, efficiency, satisfaction and interaction intensity will be determined.
The analysis of ASAS service performance will be further refined with stakeholder opinions about the future service using interviews, workshops and questionnaires. The scenarios and experiment fragments will be presented in the workshop with the aim of refining the contextual factors. Stakeholders will be asked to discuss the possibilities for generalizing the ASAS and applicability of ASAS services.
The determination of performance effects connects all the parts of the research presented in this dissertation. The effects of participative development of the new service will promote a rich qualitative description of the service developed, given that so many different stakeholders, from workers to management, all with their own knowledge and different perspectives on the organisation will have participated in this process. The experiments will be used to test the performance and to visualize the ASAS service, this will deliver quantitative data about the expected efficiency, effectiveness, satisfaction with and interaction intensity when using ASAS. This data can then be used to help visualize new services. Sub question B also deals with the effects of all this information on the perceptions of the stakeholders of the organisation.
The last question is process related. It deals with the question of how this research will be performed, and is related to choosing a design approach.
1.2.3 Sub question B:
What is the appropriate design approach for this research?
In the next section of this chapter a description is given of where the different research findings discussed above can be found in this dissertation to show how the different research parts are linked together to answer the central research question.
1.3 Composition of the dissertation
Chapter 2: Orientation on ASAS services
Chapter 4: Scenarios for ASAS based services Chapter 1:Introduction
Chapter 6: Scenario based evaluation
Chapter 7: Discussion and conclusions Chapter 5: Experiment based evaluation
Dutch National Rehabilitation Center for visually impaired persons Dutch Inland Revenue Services
Chapter 3: Method: ASAS services design and performance
Figure 1-1 Composition of the dissertation
1.3.1 Chapter 2: Orientation on ASAS services
First, the precise meaning of ASAS services will be explained. Second, an overview of the domains in which ASAS services can be applied is given in this chapter. An explanation is given of the domains and their characteristics in which an ASAS service will be helpful. Confirmation is provided regarding why an ASAS technology was chosen to be the topic of research and the design requirements are given for such a service. These requirements are used to choose the ASAS technology for the experiments discussed in chapter 5.
1.3.2 Chapter 3: Method: ASAS services design and performance
This chapter discusses participatory design approaches. It is argued that some elements are missing in the existing design approaches. This asks for a new approach used in this dissertation, called Early eXperiment and Scenario based Evaluation (EXSE). An explanation is given of how EXSE is based upon an earlier developed approach and of the stepwise approach of EXSE. EXSE uses a combination of research approaches. This combination reflects the two parts of the basic research question. The “what” part asks for the development of a new service which uses qualitative data and the “how” part asks for an experiment (Yin, 1994). A partial answer on sub question B is provided in this chapter.
1.3.3 Chapter 4: Scenario’s for ASAS based services
In this chapter the design process for the two future ASAS services is described, and the chapter concludes with a discussion of the possible generalizations of ASAS service use towards other services in other domains. An answer to sub question A1 is provided in this chapter.
1.3.4 Chapter 5: Experiment based evaluation
In this chapter experimental tests of Prototypes of the two ASAS services are described. It analyses the effects of ASAS services on efficiency, effectiveness and satisfaction, and interaction intensity. A partial answer to sub question A2 is provided in this chapter.
1.3.5 Chapter 6: Scenario based evaluation
In this chapter the analysis of ASAS service performance is further refined with stakeholder opinions about the future service by means of interviews, workshops and questionnaires. In the workshop the scenarios and experiment fragments are presented with the aim to refine contextual factors. Stakeholders will discuss the possibilities of generalization and applicability of ASAS services. Second, a critical note will be placed to the development of stakeholder opinions. It will be discussed how their opinions changed through the participatory design approach.
1.3.6 Chapter 7: Discussion and conclusions
2 Orientation on ASAS services
In this chapter an overview will be presented of Audio Supported Application Sharing (ASAS) services, their technological development over the years, the technologies and their use based on scientific research. l start with an orientation on ASAS service technology. An overview will be given of the domains in which ASAS services can be applied and how these influence the technological design guidelines. This will partially provide an answer to sub question A1: Which ASAS service concepts exist and in which context can this service be used?
Then, evidence drawn from the literature will be presented concerning the expected effect of ASAS services. This information will be used to a model which factors will affect the efficiency, effectiveness, satisfaction and interaction intensity of future ASAS service use, and the role of certain factors in this context. This will partially provide an answer to sub question A2: Does an ASAS service allow consulting with higher effectiveness, efficiency, satisfaction and does using one change interaction intensity in comparison to the current way of consulting?
2.1 Orientation on ASAS technology
Digital information is becoming increasingly integrated in our daily life. We can communicate with each other via e-mail and exchange documents, photographs, video fragments and use webcams during a telephone call (‘nice for the kids…’). Digital information is used in education to communicate with other students and teachers and it is used for commercial purposes and for public services: delivering information direct to the citizens at home without visiting an office at any time a citizen wishes to obtain such information.
The technology used for these systems is more and more communication mediated. That is, these systems are used for communication between two or more persons and during this communication by text, video and audio can also be sent to the other participant(s). The focus of this research is a special sort, or way, of computer mediated communication: communication facilitated by ASAS technology. The services that use ASAS technology are called ASAS services. ASAS stands for Audio Supported Application Sharing. A lot can be said about this technology, and this will be discussed further in this chapter, but main characteristics of this technology are a shared audio connection and the possibility of sharing applications at distance.
should be able to use different ways of connecting to the data (modem, Internet, mobile, et cetera), and access to data should also be independent of operating systems (Windows, Linux, MacOS et cetera).
The focus of this study is application sharing in combination with audio support. Application sharing software originates from data sharing software. Application sharing can be formulated as follows ( Wolf, 1995):
'The basic concept of application sharing is multiplexing of output streams from applications to terminals and multiplexing (filtering) of user input. The term terminal refers to any high-resolution graphics display and its driver or display server software. Sharing is supported on the application level or window level: either single windows are shared or all windows of an application.'
This development can be observed in different domains, all with their own goals for using this technology. One domain uses application sharing to assist computer supported collaborative work. Computer-based systems are developed especially designed to support groups of people working together, called 'GroupWare' (H. Ishi, 1997, p 435). This domain focuses mainly on the interaction aspects of the system being built. Learning effects, technical specifications and the relationship between different participants (customer or agent), receive minor attention in such kinds of software.
A second domain deals with application sharing to develop and use instructional systems. Developing such systems ask for a clear understanding of the learning process for the developer to understand what the interaction of the official and client with the instructional system should look like. Application sharing can be used to facilitate this form of learning (J.J. Beumer et al., 2000).
A third domain deals with application sharing to enable customer relationship management electronically (e-CRM). Electronic customer relationship management uses a set of technologies and flexible implemented practices that can be constantly changed to service customers better (R.A. Gable, 1993). It uses some of the work of other domains: i.e. uses application sharing to improve telephones and helpdesk improvement.
A fourth domain deals with the use of application sharing to improve telephones. It is not primarily concerned with the interaction between customers and agents, but focuses mainly on the technical part of the system. One of the developments in this field is call centers. A call center is a telephone contact point for a company or organisation that can be reached by a number of customers using one dialed telephone number.
A fifth domain deals with application sharing as a means to enable better computer-support. Computer support is commonly realized via ‘helpdesks’. Helpdesks are a practical example of how application sharing can be used. Helpdesks have introduced the concept of application sharing in a lot of organisations. Customers can phone the helpdesk with questions and problems. The members of the helpdesk can solve these problems without leaving their own office. Help desks use web enabled call centers most of the time. This technology is combined with the technology used in GroupWare and Instructional Systems.
of attention will then be used to choose a specific ASAS technology for an experiment, this will be discussed in chapter 5.
2.1.1 ASAS technology in Computer Supported Collaborative Work and GroupWare Systems
Developing and testing systems and interfaces for systems that allow people to share information with others who are working on different computers is not new. Stefik et al. (1987) developed and tested one of the first multi-user interfaces in which people in the same room could share information presented at their own computers with others. This led to the current situation in which systems have been built and evaluated to allow geographically scattered users to work together. This includes seeing and hearing each other via a video and audio link, and sharing and manipulating information presented on the computer of the other persons (Ishi, 1997). In this section some examples are presented and some important interface aspects are described, that influence the interaction between users. These form the points of attention for an ASAS service.
Points of attention
• The importance of What You See Is What I See, called WYSIWIS (Stefik et al., 1987) was first described when the first GroupWare systems were built and tested. WYSIWIS ensures that information presented on the computer screen of one participant is also visible to the other participants’ screens. If one of the participants alters the visible information, the other participants can also see this change.
• More of flexibility in WYSIWIS however, is sometimes needed according to Stefik et.al (1987). For example, if a lot of people are working together, and all the cursors of all the participants are visible at one time, this can be a source of confusion. If only the telepointer – used when a person wants to point out something to all the other participants – is shown, and not all the individual cursors, it increases the overview, although it departs from the WYSIWIS principle. Tang and Minneman (1990) developed and tested a video interface for a collaborative drawing tool for persons working in the same room. Using several experiments, they found a number of aspects that they used to produce some design guidelines, these are given below.
Points of attention
• Hand gesturing is important to explain something to another person, other people. • Users understand the drawings that are created better when time relationships are well timed. That is, when someone wants to show something to others, the technology enables this and does not disturb this due to delays in presenting screen changes to other users.
can be transmitted influences overall task performance. In the same study Hayne, Rendergast and Greenberg (1992) gave concrete indications for the size of the cursor for several users working together. For two users, this is 64 x 64 pixels. If more than one cursor is used the cursors should be clearly different.
Some discussion can be found about voice connection in GroupWare systems. Johansen (1992) describes how and when computer-augmented teamwork is used now and might be used in future. He describes how the development of GroupWare will lead to, for instance, the integration of telephone connections. Johnson distinguished four situations: collaborative work at the same location, geographically scattered locations, working simultaneously, or at different times. When people are working at geographically scattered locations simultaneously – the case for our ASAS service – Johanson claims that conference calls will increase. Johnson also claims that conference calls using PC graphics and image will become commonplace. He furthermore believes that video conferencing will show continual gradual growth, with some use of computer aids, with an emphasis on portable units and – eventually – desktop video.
2.1.2 ASAS technology in education and instructional systems
The literature about ‘instructional systems’ focuses mainly on how graphically scattered groups learn when communicating via electronic devices. The way officials and clients communicate influence the learning process and this, in turn, is influenced by the use of different devices like e-mail, chat and conferencing. Most of the time, little or no attention is paid to how object information can be used to support the learning process. Object information is information about a physical object, for instance a text document. So, most of the time literature about instructional systems neglects the importance of sharing applications.
In this research the stress is placed on a different focus than most literature about instructional systems. First, it concerns how two persons communicate - the official (the ‘agent’) and the client (the ‘customer’, also called ‘client’) - and it is mainly not concerned with larger groups. Second, a hierarchy exists between official and client because the official gives the instruction and the client has something to learn. Third, object information in terms of sharing applications has a central position in this research.
Some studies taken from the field of education, however, can be used for this purpose. Ellis (1992), for instance, distinguishes different cognitive theories that are relevant in the design of multimedia instruction. One of these theories is the contextualistic theory. This theory examines the interactions between the client, the environment and the system being used for instruction. Contextualistic theory is relevant for multimedia instruction, because it pays explicit attention to the clients’ interaction with the instruction system. Support via ASAS technology can be considered a specific example of this type of multimedia support. The following implications for multimedia instruction can be drawn from this theory.
• Use imagery to help internalizing the information.
• Pay explicit attention to the logic and thinking connections in the information provided.
Ratcliffe et al. (1999) have developed, tested and evaluated an extended system that is used to support students in an academic environment. This system combines video and audio communication, including text-based communication, video conferencing, and application sharing to facilitate help desk questions, questions to tutors and questions about the Internet. The answers are given in real time and are not restricted to a specific geographical location. Ratcliffe et al. observed students and advisors in real time, gave them questionnaires, held interviews and analyzed information logged by the system. Radcliffe et al. developed a number of points of attention using their results. Only the points of attention not previously discussed are presented below.
Points of attention
• One of the first design requirements described by Ratcliffe et al. is that the actual use of the system is dependent on the user friendliness of the system. That is, ease with which an action can be performed. This is not only dependent on the system, it is also user specific. So, users must be able to customize the system to their needs.
• During the development stage, Ratcliffe et al. found some technical criteria that proved to be crucially important:
o How easy is it to plug the system into the existing network?
o How compatible is the technology with standard sound and video cards? o How much bandwidth is available?
2.1.3 ASAS technology in E-CRM and call centers
Due to the shift from a production-oriented economy to one based on services (Cussack, 1998), Customer Relationship Management (CRM) currently receives more attention than it had in the past. One method of implementing CRM electronically is to use call centers, as they enable 'efficient and more personalized' services (Peter Keen: http://www.peterkeen.com).
First a word about the positive effects of call centers on e-CRM. This has been made possible by web technology, which can be used to create new options for accessing call centers (Melsoet, 2000). This allows customers to telephone the call center and to seek contact via other means of communication, such as fax, e-mail, or by contacting a company via its web pages.
In the next part of this section the technologies used at a call center will be divided into three technical layers: network layer, the equipment layer, and the report and registration layer. This will aid the description of the technologies involved when a call center is used as an ASAS service (figure 2-1), and it will enable the derivation of new points of attention for ASAS technology.
Figure 2-1 Call center as an ASAS technology Network layer
The first layer is the network layer, which provides physical access to call centers. The traditional telephone network transmits single phone calls to the call center. Earlier in this chapter I described how networks are used in the case of application-sharing. A call center makes it possible to combine telephone support with call-back facilities and video conferencing. Call-back facilities allow a customer to click on a button (the call-back button) when on the Internet; this indicates to the call center that the customer has a question and wishes to be called back. The call center can then phone the customer, either via the Internet (i.e. voice over IP), or on a second telephone line (ISDN, ADSL, or mobile). Video conferencing involves the transmission of a video image of the agent or customer. Two lines, mobile and fixed, or ISDN or ADSL, are also used in this case.
Point of attention
Equipment layer
The second layer, the equipment layer, is used after a phone call has passed the network layer. In this layer the phone calls are switched from the central call-in point to an agent. Automatic Call Distribution (ACD) switches the phone calls to a free operator known as an agent (Bayer, 1987). Computer Telephone Integration (CTI) is used to link the computer functions to the telephone functions. CTI includes applications that show details of the incoming calls on screen.
ACD takes on added complexity if web technology is used. Usually, incoming messages are first divided by a server into two communication means: telephone calls, other communication types (e.g. fax, e-mail, chatting and application sharing) (Steul, 2000). This distinction is often made, as a classic PBX server can be used to handle ordinary telephone traffic. Currently, however, a server is being developed that can handle phone calls alongside fax, e-mail and chatting.
One current trend is virtual phone call answering (Steul, 2000). This development started as interactive voice response (IVR): after dialing a call center, customers are not immediately switched to an agent, but are first asked to answer taped questions. The answers are used to help to match a customer to an agent who can answer his or her question.
The virtual interaction component has currently been increased: information can be received when keys on the telephone are pressed, but the system also works with speech recognition. For example, when customers phone the Dutch railroad company, a voice computer asks them at which station they wish to start their journey, and at what time. The voice computer can deal with a low level of interaction, recognizing place names and times dictated by the customer. If it is unable to interpret what was said, the system can also ask the customer to repeat their answers.
Points of attention
• It is important to notice how the incoming calls are switched to a free agent. • See if interactive voice response (IVR) can be used.
• Look into the possibility of using speech recognition to recognize the answers given by a customer if IVR is used.
One of the characteristics of call centers using CRM is that waiting times are reduced by the use of Automatic Call Distribution (ACD). The intelligence behind ACD makes it possible for customers to be informed about the current waiting time. If they are in a hurry and hear that they will have to wait too long, they can choose to use IVR instead.
Report and registration layer
The report and registration layer consists of software applications that withdraw information from the equipment layer, from the actual conversation with the agent, and from information previously stored in the report and registration layer.
automatic computer registration when the call passes the equipment layer, and by the notes made by the agent during the phone call. For example, the servers that distinguish which medium needs to be used – such as e-mail or voice over IP – can also perform automatic registration tasks. An agent registers information about the customer’s questions.
Most systems work with a kind of shorthand that is used by the agent to enter and retrieve customer data. If a customer calls more often, real-time reports can be used to draw up a customer history, showing how often a customer calls, and when and why.
Points of attention
• Look at the possibility of gathering information in real time. o Look which kind of information is registered automatically.
o Look at the possibility for the agent to enter information about a call in a database during the call.
• Look at the possibility to use a customer’s history database.
The report and registration layer has another positive effect on e-CRM. Contact with the customer can be further personalized by linking real-time details with history and number identification – because, for example, the agent knows what a customer commonly calls for. Sometimes it is possible for a fixed agent to handle calls from a particular customer. This can be particularly useful for dealing correctly with important customers, or when a defaulter calls.
Second, the report and registration layer makes it possible for the company to make direct product deliveries to the client or deliver product information. The agent can also obtain information on stocks and orders, which, if necessary, can be passed on to the client. The ASAS technology used in chapter 5 best matches with the points of attention described above. The ASAS technology used for this match is described in Appendix 1. This comparison is based on the points of attention relevant for the domain in which it has to be applied, the general points of attention described below and the specific ASAS technology criteria needed for the casestudies in which it will be applied (see chapter 5).
2.2 Choosing ASAS technology
In the previous section a description was given of how ASAS technology has been developed in different domains and the points of attention that can be drawn from this. The reasons why the ASAS technology became the topic of the research reported in this dissertation was are to be found in an experiment performed in 1998. In this experiment Beumer et al.
(2000b, 2000c) undertook an experiment aimed at determining the specific points of attention that should be designed into a system intended for the provision of support to Visually Impaired Computer Workers (VICWs) dealing with computer problems.
The experiment involved 32 tasks given to 16 pairs of people. Each pair consisted of a sighted person who performed the tasks together with either a sighted person or a VICW. The persons in the experiment were homogeneous with regard to age, education, nationality and other relevant background variables. Gender was balanced. The communication strategies used to perform the tasks were analyzed. With regard to a support tool, two points of attention were established.
Point of attention
• The official (agent) should be able to see what the client is doing on his or her screen in order to manipulate the information presented there.
Motivation: Officials spent most of their time looking at how the client performed the task. This was how coordination of the task-related communication was achieved.
Point of attention
• The audio connection is of great importance to official and client alike.
Motivation: The official should be able to give instructions, and the client should be allowed to answer, both to give feedback and to ask questions when necessary.
A support tool, the Audio-Supported Application Sharing technology (ASAS), was defined as a result of the experiment. It involves application sharing at a distance, accompanied by telephone support in one form or another (figure 2-2).
Beumer and Van den Hooff (2000a) confirmed the importance of these points of attention. On the basis of several cognitive and communication theories, they clarify the added value of application sharing during telephone support. They argue that application sharing provides a rich context for information processing to take place, and that it therefore supports the cognitive processes whereby information is stored and retrieved. Support should be assisted, as this makes it possible for operations to be explained and exemplified – for example, relevant actions can actually be shown to the user; imagery and movement can be used to help the user internalize the information; and the logic and thinking connections can also be shown.
Making reference to the media richness theory, Beumer and Van den Hooff (2000) also argue the added value of application sharing with telephone support. This can offer information at varying degrees of richness, which will in turn support effective communication in a variety of tasks.
The two general points of attention for ASAS technology described above – application sharing and telephone support – are a prerequisite.
It is clear that these two points of attention need further refinements.
The innovation characteristics are used as are described by Andriessen (1989) to describe the more specified ASAS technology points of attention. Andriessen describes how the processes of investigation, research, consideration and decision-making are needed before a new system can be introduced into an organisation. During this process, the organisation is influenced by three groups of variables: the characteristics of the innovation (such as costs, relative advantage and possibility to experience), the characteristics of the organisation (notably its innovativeness), and the characteristics of the organisation's environment (e.g. competition, coalitions, and support).
Points of attention
• The costs of the technology should be taken into account. This can be divided into two components: buying the ASAS technology and integrating audio support in the system.
o Costs incurred to buy the application-sharing technology. Using ‘free-ware’, software freely available via the Internet, can reduce these costs. Examples include VNC (Virtual Network Computing) and NetMeeting. o Integrating telephone support in the system can be realized using a
traditional telephone together with a multimedia p.c. This possibility use: ISDN or ADSL, i.e. one line for sending speech, one for sending
data.
traditional telephone (sending speech) combined with a cable modem, i.e. sending data.
mobile telephone, sending speech, combined with a traditional telephone, sending data.
traditional telephone, sending both speech and data via the Internet.
• Preferably ‘free-ware will be used’. Free-ware' application-sharing software can be tested and used most easily. The software can be downloaded free from the Internet.
• Look if enough communication means are available. For instance:
o a mobile telephone needed to provide the combination of a traditional telephone and a mobile telephone.
o a cable modem, an ISDN or ADSL connection is needed to provide a cable connection or ISDN or ADSL connection respectively.
• Look at the relative advantage. The relative advantage of the system consists of the advantages greater than the disadvantages given a specific situation. It depends on:
o userfriendliness of the system, that is, how easy it is to perform a desired operation. For instance, is it clear how the technology should be started and closed, is it clear how to perform an operation for another user, and how many actions are needed to achieve this goal.
o operating systems used, when a technology is chosen that uses another operating system than the one available, the relative advantage decreases. The common Operating systems used are Windows NT or Windows 2000, Linux and Macintosh. For instance, Timbuktu can be used on the Macintosh and NetMeeting for Windows. Platform-independent software can be used for all operating systems. Examples include Virtual Network Computing and Timbuktu.
o different situations require different networks. For instance, when large amounts of data must be transported, the relative advantage is increased if ISDN is used. Some application-sharing technology use ISDN, like PictureTel Live 200, others use the Internet (e.g. NetMeeting).
o relative advantage is also dependent on the quality of the audio connection. At this moment, voice quality is much better when using ISDN and ADSL. A mobile telephone connection has less quality, and the quality of cable is unpredictable at present. At present, voice via the Internet has the worst quality.
The points of attention for ASAS technology were discussed in this section. These points of attention will be used to choose an appropriate ASAS technology for the experiments, chapter 5.
This technology is the starting point for designing an ASAS service: the environment in which the ASAS technology will function. Although the technology has a central role in this environment, the service is also dependent on the different circumstances in which it will function. These circumstances are determined by different factors. Changing the value of these factors will affect performance. In the next section I will discuss how performance is expected to change and which factors values are expected to change ASAS service performance most.
2.3 Factors influencing performance in future ASAS services
other words the added value of ASAS service use during training and support will be explained with help of the literature. Then the factors that influence this performance will be discussed in sub-section 2.3.2.This will lead to a research model and additional hypotheses. 2.3.1 ASAS services and the expected performance effects
The added value of ASAS services during training and support can be explained with help of cognitive psychological theories and communication theories (Beumer and van den Hooff, 2000). Within the cognitive theories it is claimed that the more information available about the context in which a certain operation takes place, the easier it is for a user to perform the operation (Tulving, 1983). Tulving asked subjects to learn words and sentences while they were listening to music. The words and sentences could better be recognized if the music was presented. These results have been confirmed by several studies (d’Ydelwalle, 1992). d’Ydelwalle tested how people are facilitated by subtitles when they have to recognize photographs and sounds on television. Another research study has shown how pictures can be used to help people to recognize words faster (Schiffers, Meijer & Levelt, 1991).
It can be argued that the added value of ASAS technology during support can be explained using communication theories like the work of Patty & Caccioppo (1996). They distinguish two routes of information processing:
1) The central (or cognitive) route, aimed at processing arguments, focusing on the message’s content.
2) The peripheral (or affective) route, aimed at processing peripheral stimuli, focusing on the message’s form.
The theory predicts that both routes are of importance in information processing. Offering the possibility of processing information via both routes would lead to faster and better information processing. Paralinguistic and non verbal signals are primarily processed along the peripheral route, which is addressed by using different modes of communication – more than just text. ASAS services provide the opportunity to address both the central and the peripheral route, enabling optimal communication – both content and form (aimed more at relational information) are communicated.
Another relevant communication theory according to Beumer and Van den Hooff that can predict the added value of ASAS technology are contextualistic theories. Using these theories it can be argued that better learning is provided due to explicit attention to the context in which learning takes place. Within these theories explicit attention is paid to the client’s interaction with the instruction system. Implications for multimedia instruction from this theory are, among others (Ellis, 1992): use imagery to help internalizing the information, pay explicit attention to the logic and thinking connections in the information provided, and use an event as the focus of the process (as an anchoring device).
2.3.2 Factors influencing performance
Figure 2-3 shows how the use of ASAS services affects performance during support. These effects are influenced by five factors (Beumer and Van den Hooff, 2000). These are:
1) characteristics of the task 2) user characteristics 3) environmental factors
4) characteristics of the design strategy 5) technological characteristics Use of ASAS services Performance task characteristics environmental factors user characteristics technological characteritics introduction strategy
Figure 2-3 ASAS service factors affecting performance
1) Characteristics of the task
The task determines the medium to be used. According to Daft and Lengel (1984, 1986), a richer medium will be preferable for a more complex task as this decreases the uncertainty about the task. In terms of this information richness theory, using an ASAS service during support provides a communication channel that can offer information of differing richness, which can then be used to support effective communication in a variety of tasks.
2) User characteristics
We can take either an intra-personal or an interpersonal perspective. When looking at user characteristics, intra-personal factors have to do with cognitive psychology, with a focus on learning and remembering in individuals. In this context, differences between users depend on differences in individual information processing. These differences, in turn, depend on age, handicaps, computer literacy and the like.
Handicaps and computer literacy are factors that may cause people to need more help when performing tasks. Beumer (2000) illustrates this in a study in which two groups of users had to perform computerized tasks: well-sighted and visually impaired persons (VIPs). VIPs needed more help and more time to complete the tasks. ASAS technology can be used to help them at a distance.
S.J. Westerman et all (1995) state that cognitive capabilities are influenced by age, they confirmed empirically that people older than 50 year old need more time to process information and perform tasks. P. Verhaeghen & T.A. Salthouse (1997) confirmed these results with a meta-analysis in which 91 studies were compared.
From an inter-personal perspective, we look at the processes that take place when information is exchanged, or in other words, when interaction takes place. Therefore the ability of users (both agent and customer) to fit medium to task is important (Daft and Lengel, 1984, 1986); as is the way they influence each other in these choices, and the way their respective social environments influence them in this, this affects the entire innovation process (Rogers, 1983; Andriessen, 1989).
3) Environmental factors
The environment is an important factor in two ways. First as the social environment in which the adoption, introduction and incorporation of an ASAS service takes place. Negative experiences with a new technology (in this case: the ASAS service) in the past influences stakeholders’ opinions about the expected performance effects of the new service. Second, environmental factors are important as the physical environment in which communication tasks are performed. Noise and interruptions disturb these processes, and application sharing can compensate for such disturbances (Gathercole & Hitch, 1993).
4) Characteristics of the design strategy
The design strategy must be geared to each of these factors. Current design approaches like participative design approaches take as given that the involvement of different stakeholders will lead to better system design and a more positive view of stakeholders concerning future effects on performance (see chapter 3 for more details).
5) Technological characteristics
stored, the easier it is to retrieve it (Tulving, 1983). So, it can be expected therefore that ASAS services will increase performance as a result of this phenomena.
These 5 factors give us the general idea that performance are influenced when using an ASAS service during training and support.
Using this information, figure 3 can be specified further (figure 2-4).
Figure 2-4 Prediction of ASAS service factor effects on performance
2.3.3 Hypotheses
Using this model as a basis some hypotheses were derived, that were then tested in the research study reported here. These are:
1. Technology related hypothesis: efficiency, effectiveness and satisfaction is higher if ASAS service is added to traditional way of consulting.
2. Task related hypothesis: the differences found in hypothesis 1 are larger for complex problems in comparison with simple problems.
constraints (due to e.g. computer illiteracy or handicaps), in comparison with subjects who are not confronted with these constraints.
4. Design related hypothesis: stakeholders will evaluate the service performance effects more positively at the end in comparison with the beginning of the service design process.
5. Communication related hypothesis: with ASAS services, subjects will communicate less than without ASAS.
Rational
In a previous experiment concerning communication of persons with visual constrains we formulated in-depth hypothesis with regard to visual constrains and how communication changes due to these limitations (Beumer et al., 2000). We discussed how these communication changes will occur due to humans limited cognitive capacity (see also Clark and Brennan, 1991; Levelt, 1981, 1989; McDonald, & McGurk, 1978; Sacks et al. 1974). This will in turn influence individual interaction intensity, and this will influence individual and organisational goals (see chapter 4 for more detail).
The hypothesis in the Beumer et al., (2000) experiment covered the following hypotheses. Visually Impaired Persons (called VIPs) look less at each other than sighted people (SP's) and will use more auditory grounding to compensate for the fact that they cannot see the facial expression of the speaker (hypothesis 1). We also hypothesized that the sighted speakers have to exchange more information when engaged in a conversation (hypothesis 2). Furthermore VIP’s ask more questions (hypothesis 3) and make more errors while performing tasks (hypothesis 4). With regard to turn taking we hypothesize that when VIP’s participate in a conversation, more simultaneous conversation takes place (hypothesis 5). With regard to informal communication we hypothesized that speaker and listener use more informal communication when a VIP participates (hypothesis 6). We also expected that more time might be needed when a VIP participates in a (formal and informal) conversation (hypothesis 7). The hypotheses were tested in an experiment.
These hypotheses can be generalized and will be transformed for the experiment, described in chapter 5, to explain communication changes in two situations: changes in technology condition (that is, telephone support, vs. observation based ASAS and intervention based ASAS) and changes in individual performance due to computer literacy and different experiences with digital tax return.
The hypotheses concerning changes in the technology condition are formulated as follows. 5.1 Hypothesis: more time will be spent on tax related information when telephone
consultancy is used than when ASAS services are used.
The hypothesized reason is that tax agents need more time (more words and clarifications) to make their meaning about tax related issues clear to the subject, when using an ASAS service. This information is content related: related towards tax matters.
That is, tax agents need more time (more words and clarifications) to make their meaning about tax diskette related issues clear to the subject. This information is process and technical related: related towards filling in the right information at the right position.
5.3 Hypothesis: More time will be spent on issues of situation awareness when telephone consultancy is used rather than when ASAS services are used.
Situation awareness is awareness of the physical location of certain information, presented on the digital tax return.
5.4 Hypothesis: More time will be needed for informal communication when telephone consultancy is used rather than an ASAS service is used.
Informal communication is talk about non tax related topics and talk that is not about the digital tax return.
5.5 Hypothesis: More turn taking will take place when telephone consultancy is used instead of ASAS services.
Turn taking is defined as the switch in communication from tax agent to subject and vice versa.
Three hypotheses are not listed above, which were listed in the experiment discussed in Beumer et al. (2000). One hypothesis concerned the efficiency of increased media richness. This hypothesis is already answered above. The other two hypotheses concerned the relation between questions and errors made. These hypotheses were left out because it appeared to be not possible to define ‘questions’ and ‘errors’ unambiguously. For instance, that subjects sometimes formulated a question as a real question, and sometimes the question was implicitly interwoven in the question handling.
2.4 Discussion and conclusions
In this chapter an overview of domains in which ASAS service are used, or will be used in the near future was presented. The actuality of these services was shown and how the ASAS service is increasingly being integrated into call centers. It also shows how difficult this technology is to integrate into actual work, for instance in education.
