The role of organizational strategy in the user-centered design of mobile applications
Eshet, Eyal; De Reuver, Mark; Bouwman, Harry
Publication date 2017
Document Version
Accepted author manuscript Published in
Communications of the Association for Information Systems
Citation (APA)
Eshet, E., De Reuver, M., & Bouwman, H. (2017). The role of organizational strategy in the user-centered design of mobile applications. Communications of the Association for Information Systems, 40(1), 315-331. [14].
Important note
To cite this publication, please use the final published version (if applicable). Please check the document version above.
Copyright
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons. Takedown policy
Please contact us and provide details if you believe this document breaches copyrights. We will remove access to the work immediately and investigate your claim.
C
ommunications of the
A
ssociation for
I
nformation
S
ystems
Accepted Manuscript
The role of organizational strategy in the user-centered design of mobile
applications
Eyal Eshet
Turku Centre for Computer Science (TUCS)
Faculty of Social Sciences, Business and Economics, Åbo Akademi University, Finland
eeshet@abo.fi Mark de Reuver
Faculty of Technology, Policy and Management, Delft University of Technology, The Netherlands
Harry Bouwman
Faculty of Technology, Policy and Management, Delft University of Technology, The Netherlands
Received date: 02/02/2016 Accepted date: 4/12/2016
Please cite this article as: Eshet, Eyal; de Reuver, Mark; Bouwman, Harry: The role of organizational strategy in the user-centered design of mobile applications, Communications of the Association for Information Systems
(forthcoming), In Press.
This is a PDF file of an unedited manuscript that has been accepted for publication in the Communications of the
Association for Information Systems. We are providing this early version of the manuscript to allow for expedited
dissemination to interested readers. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered, which could affect the content. All legal disclaimers that apply to the Communications of the Association for
Information Systems pertain. For a definitive version of this work, please check for its appearance online at
C
ommunications of the
A
ssociation for
I
nformation
S
ystems
Research Article ISSN: 1529-3181
The role of organizational strategy in the user-centered
design of mobile applications
Eyal Eshet
Turku Centre for Computer Science (TUCS)
Faculty of Social Sciences, Business and Economics, Åbo Akademi University, Finland
eeshet@abo.fi Mark de Reuver
Faculty of Technology, Policy and Management, Delft University of Technology, The Netherlands
Harry Bouwman
Faculty of Technology, Policy and Management, Delft University of Technology, The Netherlands
Abstract:
Gathering insights on users and their usage context is at the core of the User-Centered Design (UCD) approach. The efficient and effective usage of these insights is strategically important to organizations. With the proliferation of mobile applications, gaining timely and relevant insights is increasingly challenging due to the heterogeneous and dynamic context of use, the abundant availability of information on usage behaviour as well as the intense time constraints imposed by the highly competitive mobile market. This paper develops a research model that considers strategy foci as motivators affecting the efficient and effective usage of insights on users and context. Mediating effects of UCD resources, like time and financial constraints, organizational practices, as well as UCD competences are examined. To test the model a survey was conducted with 100 mobile practitioners and PLS was used to estimate the model. The model shows that focus on an innovation strategy greatly affects data usage on user and their context, directly and indirectly, i.e. mediated by organizational practices and UCD competences. Strategies with a focus on cost have no direct effect on the usage of user insights, but lead to negative impacts on UCD competences and organizational practices.
1 Introduction
User-centered design (UCD) is a well-established approach to the design and development of usable and useful interactive systems. According to the approach, the design of interactive systems is largely informed by insights on users and their context of use. Gaining these insights is strategically important to organizations, as emphasized by research in various disciplines, e.g. Management of Information Systems (MIS) (Robey and Markus, 1984), Strategic Management (Boland, 1978), Software Engineering (Schmidt et al. 2001), Human-Computer Interaction (Gould and Lewis, 1985), the emerging fields of User Experience (UX) design (Hassenzahl and Tractinsky, 2006) and Interaction Design (Sharp et al. 2007) as well as international standards (ISO 9241-210, 2010). Commonly, organizations use an idiosyncratic approach to collect and analyze user data. Regardless of the approach, an efficient and effective use of this data, the dependent variable in our research, is important to the success of the project, and therefore relevant to the implementation of the organizational strategy. Understanding the influence of different strategies and work procedures on the UCD practice is also relevant to IS design management.
The new mobile era, labeled the ‘mobile apps era’ (Eshet and Bouwman, 2014), introduces challenges to the UCD practice of mobile interactive systems (i.e. mobile applications), specifically regarding designers’ effort to gain relevant insights on users and context of use. Firstly, in contrast to the typically single context of use in stationary computing (e.g. desktop and web-applications), the use of mobile computing devices (e.g. mobile phone) is characterized by multi-context of use due to the mobility of people and the devices they carry with them (Henfridsson and Lindgren, 2005). In addition, the affordability of mobile devices leads to an increased diversification of users, and new classes for situated activities that are made possible with mobile computing (Johnson, 1998). Consequently, the diversity of users and variety of use contexts put a strain on the effective collection and analysis of relevant user data. Secondly, rapid technological developments in recent years, for instance in embedded sensors and wireless technology, allow the real-time collection and analysis of actual usage behaviour data. The so-called ‘Big Data’ leads to a continuous flow of data about user behaviour, which is further fuelled by insights from a growing amount of market research companies. With the excess availability of usage data, the efficient use of such insights, i.e. understanding the relevant from the ordinary, becomes more difficult. Lastly, application stores like those provided by Apple and Google have democratized the development and distribution of mobile applications, which resulted in an increasingly competitive and dynamic mobile market
(Bergvall-Kåreborn and Howcroft, 2011). Moreover, Agile approaches are increasingly adopted for the design and development of mobile applications (Eshet and Bouwman, 2014), though the Agile principle on short system delivery cycles limits the time to understand users (Seffah et al. 2005). Given the competitive pressures and resource constraints for UCD practice, the effective and efficient usage of user and context data becomes more strategically important.
Organizational strategy and its relation to a specific aspect in the UCD practice of mobile applications, i.e. the efficient and effective usage of data on users and context of use, has not been a topic of research, to our knowledge. The new mobile era introduces challenges beyond BYOD (bringing your own devices) or channel strategies, affecting a company into its capillaries. Understanding how an outside-in perspective on organization strategies (De Wit and Meyer, 2010) is combined with an inside-out strategies focused on providing resources and building competences in order to support the UCD practice is utmost important. From an outside-in perspective views on cost focused strategies vis-à-vis strategies focused on innovation are relevant to consider (Christensen, 1997; Porter, 1985; Tidd et al. 2005). From an inside-out perspective the availability of UCD resources and capabilities, as extensively discussed in strategic management and IS literature needs attention (Barney, 1991; Mata et al. 1995; Wernerfelt, 1984). Research on the relation between IS and resources and capabilities are rather high level and focus in a generic way on IT assets, processes or IS Capabilities (Wade and Hulland, 2004) rather than on specific practices. IS research has paid extensive attention to ‘Design Science” (Cross, 2001; Hevner et al. 2004; Peffers et al. 2007; Sein et al. 2011), though research on the effect of organizational strategy on the UCD practice has yet to be developed within IS research. Research in Human-Computer Interaction (HCI) emphasizes the relation between the UCD practice and resources and competences as well as management level support (e.g. Rosenbaum et al. 2000; Venturi et al. 2006), though a relation between the organizational strategy and the UCD practice has not been established. Particularly with the increasing dominance of mobile-based applications, attention to the UCD practice is utmost important, let alone with the emergence of Big Data.
Seen the identified gap in literature, the objective of this study is to obtain insights into the impact of organizations’ strategies on the UCD practice. Given the abundance availability of data on users and context, in this paper we focus on how innovation and cost focused strategies affect a specific aspect of the UCD practice, that is, the efficiency and effectiveness usage of data on users and context during the
design and development of mobile applications. Moreover, following HCI-based views on the UCD practice, the relation between organizations’ strategies and the UCD practice is examined using UCD resources and competences, as well as organizational practices, as mediating effects. By that, this paper is unique because we contribute to how time and financial constraints, UCD capabilities as well as organizational practices mediate between strategy and the usage of data on users and context. We also contribute to strategic management literature and to “Design Science” approaches in IS by connecting a strategy focus with the UCD practice of mobile applications. As far as we are aware, a quantitative research focusing on this relation has not been introduced yet.
We conducted a survey among user experience (UX) designers and interaction designers, software developers, project managers and owners who are active in the design and development of mobile applications to collect their perceptions and views. The study was carried out when the use of mobile applications became a common practice in work and non-work activities of people in many western countries. Relating organizational strategic aspects to the UCD practice is important for managers. By doing so we also relate IS research with a focus on strategy to research on the UCD practice in HCI. The remainder of this paper is organized as follows. In the next section the core concepts of the paper are defined. In addition, hypotheses are developed based on relevant literature, and a research model is provided. Section three explains the overall research methodology. Section four presents the study results, followed by a discussion of the results in section five. The final section concludes the paper, points out limitations and suggests follow up research.
2 Core Concepts and Hypotheses Development
2.1 Core Concepts
2.1.1 UCD Practice
UCD is a formalized and standardized approach (ISO, 2010) to the design and development of usable and useful interactive systems. According to the approach, the development of usable and useful interactive systems requires a focus on users and context of use throughout the design and development process. Practically, this focus is operationalized through UCD activities, such as interview, observation, usability testing, and market research (Sharp et al. 2007). Hence, UCD practice refers to the set of activities that aim at gaining insights on users, context of use, and usage behaviour during the design and development
of interactive systems. Those who are involved in the UCD practice use different job titles, commonly interaction designer, user experience (UX) designer, usability engineer, and user researcher (Sharp et al. 2007).
2.1.2 Mobile Application
A mobile application is considered here from a sociotechnical perspective. Socially, mobility is an attribute of the human rather than the application, and is roughly defined as people’s ability to move between locations that vary in their spatial, temporal, and social settings (Henfridsson and Lindgren, 2005). People carry their computing devices with them while moving, and use the devices in varied multi-contextual settings. Accordingly, mobile devices are considered here as devices that are carried-on by people (Dix et al. 2000), in particular smartphones and tablet computers. The software program that runs on these mobile devices is defined here as ‘mobile application’. Technically, a mobile application can take the form of a native platform application (i.e. mobile app), web-based solution (i.e. HTML5) or any hybrid solution.
2.1.3 Organizational Strategy
Porter (1985) describes strategy in terms of the activities an organization chooses to perform, either different activities from rivals or similar activities in different ways, in order to withstand competition in the long run. These activities are not merely support the organization in dealing with the external competition, but essentially guide the internal operations, action, and behaviour of the organization and its employees (Mintzberg, 1987). In this study we limit the scope of strategies to two distinct approaches - cost-leadership strategy and innovation-focused strategy.
2.1.4 Organizational Work Practices
Apart from a strategy, organizations have internal cultures with distinct characteristics that have an impact on the daily work practices, including on the UCD practice (Iivari, 2006). In this study we consider organizational culture in terms of established work practices that contribute to creativity, open collaboration and sharing of ideas within the work environment.
2.1.5 UCD Competences
A competence can be described as a qualification that makes someone fit to perform a particular activity (Ritter and Gemünden, 2004), in this case, activities that relate to the UCD practice. Hence, UCD
competences refer to the qualified capabilities and experience of practitioners to deal with the UCD practice.
2.1.6 UCD Resources
Resources denote everything that is controlled by the organization, such as assets, capabilities, organizational processes, knowledge and information, which enable the organization to cost-effectively implement its strategy (Barney, 1991). In this study, we limit UCD resources to the financial and time resources that are allocated to the UCD practice.
2.1.7 Efficient and effective usage of data on users and context of use
Data on users and context of use refers to the insights gained through the UCD practice. In this study, we use the dependent variable to evaluate the efficient and effective usage of this data, the extent to which the data provide understanding on the context of use, as well as the role of the data in the success of the mobile application.
2.2 Hypotheses Development
2.2.1 Cost-Leadership Strategy
Organizational strategy can be conceptualized from two perspectives, i.e. outside-in and inside-out (De Wit and Meyer, 2010). One approach that is typical for outside-in views in Strategic Management literature is Porter’s (1985) approach on strategic positioning. Porter emphasizes two core types of competitive advantage i.e. cost and differentiation. In combination with the scope of activities, this leads to three strategies: cost leadership, differentiation or focus. Treacy and Wiersema (1993) refined Porter’s ideas by focusing on operational excellence, product leadership, and customer intimacy. In practice, a focus on cost and operational excellence leads to cost awareness and optimization to reduce costs. Hereafter, this strategy is labeled ‘cost-leadership’.
The cost-leadership strategy implies that also with regard to the UCD practice, cost awareness is key, and hence, has an impact on the collection and usage of data on user behaviour and user context. Usage of existing secondary data and low cost alternatives, like using freely available Internet reports on mobile usage, for gaining insight on user behaviour and usage context would be favoured in a cost leadership strategy. Since there are no estblished theoretical insights on the relation between strategy and the UCD
practice, we anticipate a direct and positive impact of cost-leadership strategy on the efficient and effective usage of data on users and context of use.
Hypothesis 1: Cost-leadership strategy is positively related to the efficient and effective usage of data on users and context of use.
In response to outside-in models, Barney (1991) developed the resource-based view on strategy, focusing on how rare resources and capabilities give firms a competitive advantage. If resources and capabilities are rare, hard to imitate, and there are limited alternatives this will contribute to reinforcing a company’s strategy. Hence, resources and capabilities play an important role in dealing with the critical contingencies that firms are facing. The competences and experience of UCD practitioners may be part of these critical resources and capabilities.
In IS literature, research into resources and capabilities is quite extensive, though on a high-level. In this study, we focus on resource and capabilities that are specifically relevant for the UCD practice. Insights on the work of practitioners, who are involved with the UCD practice, show that their work is affected by resource constraints, in terms of time and budget (Monahan et al. 2008; Rosenbaum et al. 2000; Vredenburg et al. 2002), the integration of design practice techniques within an organization (Bygstad et al. 2008; Gulliksen et al. 2006), organizational culture (Iivari, 2006), organizational work practices in terms of internal communication (Rosenbaum et al. 2000; Venturi et al. 2006), and management support (Gulliksen et al. 2006; Rosenbaum et al. 2000; Venturi et al. 2006). In addition, the competences of designers, i.e. skill acquisition and the degree of expertise, are an important element to consider (Hertzum and Jacobsen, 2001; Gulliksen et al. 2006; Suwa and Tversky, 2001).
In this study we consider the concepts of organizational work practices, UCD competences, and UCD resources as defined in the section 2.1. These concepts are mediating the relation between organizational strategies and the efficient and effective usage of data on users and context of use. Stimulating creative- and collaboration- focused work practices require stimulation of competences as well as significant time and budget resources. Existing HCI studies and literature mainly highlight the challenges in promoting and implementing UCD activities within an organization (Gulliksen et al. 2004; Rosenbaum et al. 2000; Venturi et al. 2006) and provide guidelines for dealing with the institutionalization of the UCD practice (Mayhew,
1999; Schaffer, 2004). Since there is no existing literature in this area, hypothesis formulation is based on generic insights from innovation literature relating to business strategy.
With regard to organizational work practices, we anticipate the practices to be more regulated with a cost-leadership strategy, leaving less room for creativity, open collaboration, and sharing of ideas.
Hypothesis 1a: Cost-leadership strategy is negatively related to organizational work practices.
Similarly, we anticipate a negative impact of cost-leadership strategy on UCD resources and UCD competences.
Hypothesis 1b: Cost-leadership strategy is negatively related to UCD resources. Hypothesis 1c: Cost-leadership strategy is negatively related to UCD competences.
2.2.2 Innovation-Focused Strategy
Similarly to Porter’s differentiation strategy and Treacy and Wiersema’s product leadership view, emerging and transformational strategies (Mintzberg et al. 2009) with a focus on the impact of disruptive or incremental innovation (Christensen 1997; Tidd et al. 2005), adopt an outside-in approach. What these strategic approaches have in common is how companies and organizations respond to changes in their external environment, like technology innovation, changes in consumer demand or competitor behaviour. These strategic approaches are labeled in this study ‘innovation-focused’ strategy. The changes in consumer demand and behaviour, as is in the case with mobile applications, may affect an organization and require a response from top-management, such as by offering new products and services.
In innovation-focused strategy, making use of UCD activities requires in depth engagement with users, such as by conducting contextual interviews, participatory observation, focus groups and analyses of data collected via device-embedded sensors. In principal these are time and resource consuming activities and therefore less effective and efficient. Therefore, we anticipate a direct and negative impact of innovation-focused strategy on the efficient and effective usage of data on users and context of use:
Hypothesis 2: Innovation-focused strategy is negatively related to the efficient and effective
With regard to the mediating concepts of organizational work practices, UCD competences, and UCD resources, we anticipate an innovation-focused strategy to leave more room for creativity, open collaboration and sharing of ideas as well as a positive impact on UCD resources and UCD competences.
Hypothesis 2a: Innovation-focused strategy is positively related to organizational work practices. Hypothesis 2b: Innovation-focused strategy is positively related to UCD resources.
Hypothesis 2c: Innovation-focused strategy is positively related to UCD competences.
2.2.3 UCD Resources
Applying UCD activities to gain understanding on users and the context of use is much dependent on the availability of financial and time resources. Several studies observed that budget and time constraints have a significant influence on the UCD practice (e.g. Monahan et al. 2008; Rosenbaum et al. 2000; Vredenburg et al. 2002). These resources may have a determinant role on the approach that practitioners take during the UCD practice, for instance how much effort, if any, is put into user and context studies and how evaluation is carried out. Thus, we anticipate the availability of UCD resources to have a direct and positive impact on the efficient and effective usage of data on users and context of use.
Hypothesis 3: UCD resources are positively related to the efficient and effective usage of data on users and context of use.
On a more general level it can be assumed that UCD resources also impact UCD competences and organizational work practices. More budget and time also implies that there will be more opportunities to develop UCD competences and more attention to new ideas and to feedback, as well as joint teamwork. Therefore, we anticipate a direct and positive impact of UCD resources on the mediating roles of organizational work practices and UCD competences.
Hypothesis 3a: UCD resources are positively related to organizational work practices. Hypothesis 3b: UCD resources are positively related to UCD competences.
2.2.4 Organizational work practices
Prior studies observed that internal communications within the organization (Rosenbaum et al. 2000; Venturi et al. 2006), management support (Gulliksen et al. 2004; Rosenbaum et al. 2000; Venturi et al. 2006), and involvement of cross-functional teams (Rosenbaum et al. 2000) are key success factors for he
UCD practice in organizations. Iivari (2006), examining the role of organizations in facilitating user involvement, observed that organizational culture have influence on UCD practice. Hence, we anticipate a direct and positive impact of organizational work practices on the efficient and effective usage of data on users and context of use.
Hypothesis 4: Organizational work practices are positively related to the efficient and effective usage of data on users and context of use.
2.2.5 UCD Competences
Last, the competences of practitioners who are involved in the UCD practice are relevant. Prior studies have observed that the knowledge and experience of UCD practitioners largely affect the outcome of usability evaluation (Hertzum and Jacobsen, 2001), and the approach and implementation of the UCD practice (Gulliksen et al. 2006). Suwa and Tversky (2001) observed the superiority of experienced designers over novices in generating new ideas from external representations, such as sketches. However, merely having a competence is not enough; the competence should be put into an efficient and effective use that generates value to the organization (Ritter and Gemünden, 2004). Highly competent and experienced practitioners, besides being more aware to the importance of UCD practice in bringing value to the organization, are better equipped to create such value. Experienced practitioners are more informed about the overall importance of and the different possibilities to understand users and the context of use. Moreover, experienced practitioners are more aware of the need for financial and time resources to the UCD practice. Hence, we anticipate a direct and positive impact of UCD competences on the efficient and effective usage of data on users and context of use.
Hypothesis 5: UCD competences are positively related to the efficient and effective usage of data on users and context of use.
2.3 Research Model
Based on the hypotheses formulated above, Figure 1 presents the research model in this study. Next, we explain how the data was collected and how the model was tested.
Figure 1. The research model
3 Methodology
To test our research model, a survey method with the PLS-SEM technique was selected. Surveying by means of a questionnaire is commonly used for testing hypotheses (Johnson and Turner, 2003; Lazar et al. 2010). SEM is especially useful to test models that include mediation, and makes it possible to test the structural and measurement parts of a model at the same time.
3.1 Sample
Data was collected by means of an online questionnaire among practitioners who are involved in the design and development of mobile applications. This was emphasized both in the questionnaire invitation and in the introduction to the questionnaire. Acknowledging the fact that the UCD practice takes place in a complex organizational environment that involves multiple stakeholders with distinct backgrounds and worldviews (Suchman, 2002), different expectations (Krippendorf, 2006), and different criteria for project success evaluation (Baxter and Sommerville, 2011), we were interested in responses from practitioners in the following three project roles: (a) UCD specialists (e.g. UX designer, interaction designer, usability engineer), who implement the UCD practice and make use of its gathered data; (b) project managers, who are familiar with the organizational strategy and have decision-making power on the allocation of UCD competences and UCD resources; and (c) software developers, who also make use of the data that is gathered from the UCD practice. Practitioners in these project roles are both on the management level (i.e. project managers) as well as on operational level (i.e. UCD specialists and software developers). Especially in small firms, operational-level practitioners are commonly also involved in strategy making and decisions. Therefore, the broad range of project roles justifies that our respondents are knowledgeable both on the UCD practice and on organizational strategy. However, the parametric size of these project roles’ populations is difficult to estimate, as there are no relevant databases. In particular those involved in the UCD practice have numerous job titles and educational backgrounds. Thus, data
was collected based on non-probabilistic sampling, which is considered a valid and common practice (Lazar et al. 2010) in cases where it is not possible to apply a strict random sampling.
Potential respondents were contacted through multiple channels, particularly individual e-mail invitations and the mailing lists of professional communities (e.g. HCI, UXPA, IXDA). In addition, a link to the questionnaire was posted on relevant and active discussion forums (e.g. LinkedIn groups, Google+ communities), and on Twitter, using relevant hash (#) tags. The survey was available online for 4 weeks during February – March 2013.
We received a total of 100 responses from 20 countries, mainly from practitioners based in Finland (46%), US (12%), Sweden (9%), Netherlands (5%) and Israel (5%) with other countries representing a smaller percentage. Most of the respondents work as UX designers (33%), project managers (19%), software developers (8%) and project owners (7%).
Respondents work primarily in companies (as opposed to freelancers) of different sizes. About 28% of the companies have 10 or fewer employees, 24% have 11-50 employees, 7% have 51-100 employees, 18% have 101-1000 employees, and the remaining 23% have more than 1000 employees. The median size of companies is 46 and the mode is 5. In terms of business sectors, software (36%) and usability/UX consulting (18%) were the two largest categories, with other sectors, including education, telecommunications, design, technology research and gaming representing a single-digit percentage. At face value, the respondents work in environments in which we would expect them to be active – small, medium-sized and large enterprises mainly in the software business and usability/UX consulting. Although we cannot claim representativeness, we assume that the respondents represent a common sample for our population.
While other studies (e.g. Clemmensen et al. 2013; Ji and Yun, 2006) found differences between usability specialists and software developers, we extensively tested difference between the three groups (UCD specialists, project managers, and software developers) for the core constructs based on ANOVA, but were unable to find any significant differences. Therefore, we conclude that the sample is homogeneous enough to conduct SEM.
3.2 Measures
The measures for the core concepts in this study are based on established items from existing studies, as presented in Table 1. Furthermore, the questionnaire was pilot-tested following Dillman’s (2000) three-stage recommendations. Questions were grouped into four sections: (1) Addressing the context of use in mobile application design – the perceived importance of contextual aspects and the use, purpose, and perceived effectiveness of methods to gather data on users and the context of use; (2) Utilizing the collected data – usage and evaluation of methods for data analysis and informing the design; (3) Organizational settings – business sector, size, organizational practices, strategy and competitive environment, and (4) Demographic data – geographical location, experience with UCD practice, and main role in mobile application projects.
Confirmatory factor analysis, using Warp PLS, showed acceptable levels of convergent and discriminant validity. Convergent validity was acceptable for all constructs. Factor loadings exceeded -70 and all average variance extracted (AVE) were above .60 (Fornell and Lacker, 1981). Construct reliability was acceptable as Composite Reliability is above .80, exceeding the .60 benchmark. Multi-collinearity was not significant since the average of full collinearity VIF equals 1.099, and full collinearity VIF equals 1.445, were way below the 3.3 benchmark.
Table 1: Confirmatory Factor Analysis
Discriminant validity is acceptable, as the average squared correlation of any pair of constructs does not exceed the average of the respective average variance extracted (see Table 2).
Table 2: Interconstruct correlations and Square Root of AVE
4 Results
We trimmed the original research model in Figure 1 by omitting insignificant paths. The final structural regression model as shown in Error! Reference source not found.Figure 2, presented a good fit (Tenenhaus GOF equals .432). Overall, the explained variance of evaluation of data usage is moderate (R2= .42).
Figure 2. Structural Regression Model
With regard to our hypotheses, we address the implications in the discussion section. Table 3 gives an overview of the hypotheses: seven are accepted, four rejected, and two hypotheses show an opposite direction. We will discuss the results in more detail in the next section.
Table 3: Overview of Accepted and Rejected Hypotheses
5 Discussion
In this section, we discuss the results and the hypotheses in greater detail. First of all, it is interesting to see that the model itself, which relates the organizational context and innovation strategies to the UCD practice, particularly with regard to mobile applications and how UCD specialists, project managers and software engineers use data on users and the dynamic context of mobile use, has a high predictive value and explains the efficiency and effectiveness of the UCD practice to a significant extent. To our knowledge, this is one of the first attempts to connect the broader stream of research on strategy and innovation to UCD practice, and also one of the first approaches that go beyond qualitative or descriptive models with regard to the UCD practices, to develop an explanatory model. However, the model remains limited in the sense that it is focused on the UCD practice, while the proof of the pudding will be in the connection of the model with evaluation of the actual usage of mobile applications. In other words, our study only tested the efficiency and effectiveness of the UCD practice in informing the design of mobile applications, not their usage. This means that we can only claim to contribute to how the UCD practice leads to better user informed designs, while taking strategy orientation and how design teams operate within constraints of an organizational setting into account. Next, we discuss the specific insights.
There is no main effect between a cost-leadership strategy and the efficiency and effectiveness in which data on users and use context is used in the UCD practice of mobile applications (hypothesis 1). The effect is mediated by the availability of UCD resources, organizational work practices and UCD competences (hypotheses 1 a-c). It is striking that only the relation with UCD resources is positive, contrary to our expectations (hypothesis 1b), while the other two relations are negative (hypotheses 1a and 1c). This implies that cost focused companies are prepared to invest but don’t positively contribute to favorable organizational work practice and UCD competences. This finding confirms traditional insights from strategic management literature regarding the cost focused strategies in which optimization is core.
In contrast to cost-leadership strategy, there is a relation between an innovation-focus strategy and efficiency and effectiveness in which data on users and use context are used in the UCD practice of mobile applications (hypothesis 2). Strikingly, there is no relation between an innovation-focused strategy and the availability of UCD resources (hypotheses 2b). Apparently it is more important to develop positive organizational work practices and UCD competences than providing budgets and time for the UCD practice (hypotheses 2a and 2c). This finding confirms insights from innovation management, in which specifically the discussion on open innovation and knowledge management illustrates the importance of having an open exchange of information, insights and knowledge. Facilitating internal communication within an organization was previously emphasized as a key to the UCD practice (Rosenbaum et al. 2000; Venturi et al, 2006). The connection between an outside-in strategy focus with an inside-out perspective with a focus on UCD resources and competences and organizational work practices proves to be fruitful. We will discuss the relation between the latter in more detail.
Although earlier studies emphasize the significant influence of budget and time-related constraints on the UCD practice (e.g. Monahan et al. 2008; Rosenbaum et al. 2000; Vredenburg et al. 2002), this does not affect the UCD practice with regard to the efficient and effective usage of data on users and context (hypothesis 3). Organizations rely mainly on the practitioners’ UCD competences and organizational work practices. However it is striking that there is a negative relation between UCD resources (time and budget) and organizational work practices. This implies that due to temporal and financial constraints UCD practitioners rely on open communication and collaboration (hypothesis 3a, but then with a reversed direction). The impact of UCD resources on UCD competences shows a moderate positive contribution (hypothesis 3b).
Creation and sharing of ideas and knowledge within the organization, in order to streamline the UCD practice proves to be highly relevant (hypothesis 4). This relation is the strongest. Design teams that are open and sharing are more likely to make use of all kind of data sources that are relevant to them in a more effective way.
Our results show that the UCD competences of practitioners are essentially the key determinant of an effective data usage (hypothesis 5), at least in the context of mobile application design. The hypothesis supports the idea that competent practitioners are more likely to make efficient and effective use of the resources, by finding alternative solutions to a design problem and capitalizing on earlier experiences.
However, the actual qualifications and experience of practitioners with regard to the design and development of mobile applications were not studied. In future research, a more extensive operationalization of these two concepts, in relation to a practitioner's specific competences may better explain their role in the UCD practice.
Our study contributes to strategic management literature by showing how different high-level strategic orientations affect the specific UCD practice on the operational level. Our tested hypotheses provide a basis for theorizing on how strategic decisions in organizations affect the day-to-day work practices, particularly with regard to the UCD practice. Moreover, we contribute to IS literature by showing how the UCD practice in IS design and development is affected by both strategic orientation as well as the availability of resources and competences in organizations. Given the complex, multifaceted nature of organizations and their operation, our study also contributes an interdisciplinary approach by combining theories from strategic management, IS management and the HCI disciplines.
On a practical level, our findings imply that practitioners, especially those in managerial positions, in organizations that design and develop mobile applications, should take steps to stimulate organizations and ensure they have practitioners with relevant skills and experience to produce better user-informed designs in a timely manner. As we explained before, understanding users and their context of use during the UCD practice in the mobile apps era requires new competences. In the highly competitive mobile business market, obtaining such competences may be part of the rare capabilities that give an organization a competitive advantage.
6 Conclusion, Limitations and Future Research
In this paper, we have shown that there are links between the organizational strategy and the way practitioners, particularly UCD specialists such as UX and interaction designers, deal with data on user and context of use when designing mobile applications. The innovation-focused strategy has a direct impact on the way designers work, i.e. their work practices, and individual UCD competences, but also with regard to how efficient and effective they use data sources on users and the context of use. However, when dealing with the dynamic user context and needs, practitioners rely on their competences and experience from earlier projects, mainly due to project resource limitations.
This is the first study that relates organizational context and innovation strategies with UCD practices, particularly with regard to the way in which practitioners make use of data on users and the context of use. To our knowledge, this is also the first study that tries to develop more explanatory models with regard to the UCD practice in the HCI discipline. The connection between an outside-in strategy focus with an inside-out perspective that focuses on UCD resources, UCD competences, and organizational work practice, proves to be fruitful. It is clear that the latter play an important mediating role.
Due to this rather innovative character, we have tried to establish a theoretical basis in strategic and innovation management literature, as well as a focus on resource-based views. In doing so, we think that an important under-researched interdisciplinary domain is addressed. Developing explanatory models, instead of descriptive and more qualitative models, that take a strategy perspective as a starting point, and pays attention to moderating resources and capabilities in explaining a specific UCD practice, may extend our knowledge on the UCD practice and more generally on the design practice. Research focusing on outcome expectations from the UCD practice, both with regard to constraints and competences, needs to be extended. The same goes for the way information on users and their context exactly plays a role in the UCD practice. An efficient usage of the data may be hindered by the quality of the data, a lack of precision or even by information overload. With regard to further research we see two distinct avenues. On the one hand, we want to advocate the development of more sound research models and a more detailed and precise operationalization of core concepts to be tested in large-scale surveys. These research models should take the actual usage of mobile, or any other type of, applications, into account. On the other hand, we want to propose more detailed qualitative research into everyday design practice. More extensive research will provide deeper insights, which again are open to testing by more quantitative research.
On a practical level, this research highlights the importance of having an open innovative strategy, i.e. encouraging the contribution of ideas and facilitating the communication of information within an organization, to achieve a better informed design. Moreover, practitioners with relevant skills and experience in terms of understanding user needs and context of use can be critical assets, giving organizations a competitive advantage.
The limitations of this study are related to the data we collected, as well as the model we tested. One of the main issues with regard to research into those who are involved in UCD practice is that data is
collected based on a convenience sample and that is highly dependent on the willingness on the part of practitioners to participate. We put a lot of effort in collecting data by addressing respondents in several ways. To further research in this domain, it is necessary to involve more UX and interaction designers, project managers and software developers, not only in the interest of science, but also because research that is based on more representative samples can help improve their working practices.
With regard to the limitations of the model, we did not consider the socio-spatial context in which the designed application is being used. For instance, whether the application is used for specific work-related activities in relatively stable and predictable context of use or for non-work activities in more diverse and dynamic context of use. Including a factor on the context of use would allow a more thorough analysis on the tendency of UCD practitioners to collect user and context data, as well as analyzing the interactions between organizational factors and context of use factors and their influence on UCD practitioners. Moreover, developing and testing alternative models is necessary and expedient. With this paper, we wanted to connect strategic and innovation management research to research into UCD practice, and in this way open new interdisciplinary research venues.
Acknowledgments
The authors greatly thank the reviewers and the editor for their extremely useful and constructive comments on earlier drafts of this paper. The authors also wish to thank Balsamiq Studios and Rally Software for giving away a few licenses of their software to motivate participation in the study. The research work for this study was funded by TUCS - Turku Centre for Computer Science and by a grant from the Foundation for Economic Education in Finland (Liikesivistysrahasto).
References
Barney, J. (1991). Firm Resources and Sustained Competitive Advantage. Journal of Management, 17(1), 99–120.
Baxter, G., & Sommerville, I. (2011). Socio-technical systems: From design methods to systems engineering. Interacting with Computers, 23(1), 4–17. http://doi.org/10.1016/j.intcom.2010.07.003 Bergvall-Kåreborn, B., & Howcroft, D. (2011). Mobile Applications Development on Apple and Google
Platforms. Communications of the Association for Information Systems, 29, 565–580.
Boland, R. J., Jr. (1978). The Process and Product of System Design. Management Science, 24(9), 887– 898.
Bygstad, B., Ghinea, G., & Brevik, E. (2008). Software development methods and usability: Perspectives from a survey in the software industry in Norway. Interacting with Computers, 20(3), 375–385. Christensen, C. M. (1997). The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail.
Harvard Business Press.
Clemmensen, T., Hertzum, M., Yang, J., & Chen, Y. (2013). Do Usability Professionals Think about User Experience in the Same Way as Users and Developers Do? In P. Kotzé, G. Marsden, G. Lindgaard, J. Wesson, & M. Winckler (Eds.), Human-Computer Interaction – INTERACT 2013 (pp. 461–478).
Springer Berlin Heidelberg.
Cross, N. (2001). Designerly Ways of Knowing: Design Discipline Versus Design Science. Design Issues,
17(3), 49–55.
De Wit, B., & Meyer, R. (2010). Strategy Synthesis: Resolving Strategy Paradoxes to Create Competitive
Advantage (3 edition). Australia: Cengage Learning.
Dillman, D. A. (2006). Mail and Internet Surveys: The Tailored Design Method -- 2007 Update with New
Internet, Visual, and Mixed-Mode Guide (2nd ed.). Wiley.
Dix, A., Rodden, T., Davies, N., Trevor, J., Friday, A., & Palfreyman, K. (2000). Exploiting space and location as a design framework for interactive mobile systems. ACM Trans. Comput.-Hum. Interact., 7(3), 285–321.
Fornell, C., & Larcker, D. F. (1981). Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. Journal of Marketing Research, 18(1), 39–50.
Gould, J. D., & Lewis, C. (1985). Designing for usability: key principles and what designers think.
Commun. ACM, 28(3), 300–311.
Gulliksen, J., Boivie, I., & Göransson, B. (2006). Usability professionals—current practices and future development. Interacting with Computers, 18(4), 568–600.
Gulliksen, J., Boivie, I., Persson, J., Hektor, A., & Herulf, L. (2004). Making a difference - a survey of the usability profession in Sweden. In Proceedings of the third Nordic conference on Human-computer
interaction (NordiCHI’04) (pp. 207–215). ACM Press.
Hassenzahl, M., & Tractinsky, N. (2006). User experience - a research agenda. Behaviour & Information
Technology, 25(2), 91–97.
Henfridsson, O., & Lindgren, R. (2005). Multi-contextuality in ubiquitous computing: Investigating the car case through action research. Information and Organization, 15(2), 95–124.
Hertzum, M., & Jacobsen, N. E. (2001). The Evaluator Effect: A Chilling Fact About Usability Evaluation Methods. International Journal of Human-Computer Interaction, 13(4), 421–443.
Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design Science in Information Systems Research.
MIS Quarterly, 28(1), 75–105.
Iivari, N. (2006). “Representing the User” in software development—a cultural analysis of usability work in the product development context. Interacting with Computers, 18(4), 635–664.
ISO. (2010). ISO 9241-210:2010 Ergonomics of human-system interaction-Part 210: Human-centred
design for interactive systems. Geneva: International Organization for Standardization.
Ji Yong Gu, & Yun Myung Hwan. (2006). Enhancing the Minority Discipline in the IT Industry: A Survey of Usability and User-Centered Design Practice. International Journal of Human-Computer Interaction,
20(2), 117–134.
Johnson, B., & Turner, L. A. (2003). Data collection strategies in mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Handbook of Mixed Methods in Social & Behavioral Research (pp. 297–319).
Johnson, P. (1998). Usability and Mobility; Interactions on the move. In Proceedings of the First Workshop
on Human-Computer Interaction with Mobile Devices.
Krippendorff, K. (2006). The Semantic Turn: A New Foundation for Design. CRC Press.
Lazar, J., Feng, J. H., & Hochheiser, H. (2010). Research Methods in Human-Computer Interaction. John Wiley & Sons.
Mata, F. J., Fuerst, W. L., & Barney, J. B. (1995). Information Technology and Sustained Competitive Advantage: A Resource-Based Analysis. MIS Quarterly, 19(4), 487–505.
Mayhew, D. J. (1999). The Usability Engineering Lifecycle: A Practitioner’s Handbook for User Interface Design. Morgan Kaufmann.
Mintzberg, H. (1987). The strategy concept I: Five Ps for strategy. California Management Review, 30(1), 11–24.
Mintzberg, H., Ahlstrand, B., & Lampel, J. (2009). Strategy Safari: The Complete Guide Through the Wilds
of Strategic Management. Pearson Prentice Hall.
Monahan, K., Lahteenmaki, M., McDonald, S., & Cockton, G. (2008). An investigation into the use of field methods in the design and evaluation of interactive systems. In Proceedings of the 22nd British HCI
Group Annual Conference on People and Computers: Culture, Creativity, Interaction - Volume 1
(pp. 99–108). Swinton, UK: British Computer Society.
Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S. (2007). A Design Science Research Methodology for Information Systems Research. Journal of Management Information Systems,
24(3), 45–77.
Porter, M. E. (1985). Competitive Advantage: Creating and Sustaining Superior Performance. Simon and Schuster.
Ritter, T., & Gemünden, H. G. (2004). The impact of a company’s business strategy on its technological competence, network competence and innovation success. Journal of Business Research, 57(5), 548–556.
Robey, D., & Markus, M. L. (1984). Rituals in Information System Design. MIS Quarterly, 8(1), 5–15. Rönkkö, M., & Peltonen, J. (2012). Software Industry Survey. Aalto University, Software Business Lab.
Retrieved from http://www.softwareindustrysurvey.org/
Rosenbaum, S., Rohn, J. A., & Humburg, J. (2000). A toolkit for strategic usability. In Proceedings of the
SIGCHI conference on Human Factors in Computing Systems (CHI’00) (pp. 337–344). ACM Press.
Schaffer, E. (2004). Institutionalization of usability: a step-by-step guide. Addison-Wesley Professional. Schmidt, R., Lyytinen, K., & Mark Keil, P. C. (2001). Identifying software project risks: An international
Delphi study. Journal of Management Information Systems, 17(4), 5–36.
Seffah, A., Desmarais, M. C., & Metzker, E. (2005). HCI, Usability and Software Engineering Integration: Present and Future. In A. Seffah, J. Gulliksen, & M. C. Desmarais (Eds.), Human-Centered Software Engineering — Integrating Usability in the Software Development Lifecycle (pp. 37–57). Springer Netherlands.
Sein, M., Henfridsson, O., Purao, S., Rossi, M., & Lindgren, R. (2011). Action design research. MIS
Seyal, A. H., Awais, M. M., Shamail, S., & Abbas, A. (2004). Determinants of Electronic Commerce in Pakistan: Preliminary Evidence from Small and Medium Enterprises. Electronic Markets, 14(4), 372–387.
Sharp, H., Rogers, Y., & Preece, J. (2007). Interaction design: beyond human-computer interaction (2nd edition). Wiley.
Suchman, L. (2002). Located accountabilities in technology production. Scandinavian Journal of
Information Systems, 14(2).
Suwa, M., & Tversky, B. (2001). Constructive perception in design. In J. Gero & M. L. Maher (Eds.),
Computational and cognitive models of creative design V (pp. 227–239). University of Sydney,
Australia: Key Centre of Design Computing and Cognition.
Tidd, J., Bessant, J., & Pavitt, K. (2005). Managing Innovation: Integrating Technological, Market and
Organizational Change. John Wiley & Sons.
Treacy, M., & Wiersema, F. (1993). Customer intimacy and other value disciplines. Harvard Business
Review, 71(1), 84–93.
Venturi, G., Troost, J., & Jokela, T. (2006). People, Organizations, and Processes: An Inquiry into the Adoption of User-Centered Design in Industry. International Journal of Human-Computer
Interaction, 21(2), 219–238.
Vredenburg, K., Mao, J.-Y., Smith, P. W., & Carey, T. (2002). A survey of user-centered design practice. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 471– 478). New York, NY, USA: ACM.
Wade, M., & Hulland, J. (2004). Review: The Resource-Based View and Information Systems Research: Review, Extension, and Suggestions for Future Research. MIS Quarterly, 28(1), 107–142.
About the Authors
Eyal Eshet is a post-doc researcher at IAMSR, Åbo Akademi University, Turku, Finland. He has several
years of work experience in software programming and design in industrial context, which shaped his scope of research interest on the design practice. He is currently finalizing his dissertation on the User-Centered Design of mobile systems.
Mark de Reuver is Assistant Professor at ICT Section, Faculty of Technology Policy and Management
Delft University of Technology, The Netherlands. He received his PhD degree in 2009 from Delft University of Technology. Mark has published more than 65 journal and conference articles in the area of mobile service innovation, platform governance, mobile service business model and smart living.
Harry Bouwman is a Finnish Distinguished Professor at IAMSR, Åbo Akademi University, Turku, Finland,
and an associate professor at ICT Section, Faculty of Technology, Policy and Management, Delft University of Technology, The Netherlands. His research is focused on ICT and organizations; ICT Management; strategy, business models, and enterprise architecture; and on mobile cloud computing and mobile services.
Copyright © 2015 by the Association for Information Systems. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and full citation on the first page. Copyright for components of this work owned by others than the Association for Information Systems must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists requires prior specific permission and/or fee. Request permission to publish from: AIS Administrative Office, P.O. Box 2712 Atlanta, GA, 30301-2712 Attn: Reprints or via e-mail from publications@aisnet.org.
