Index of /rozprawy2/11081

Pełen tekst

(1)AGH University of Science and Technology in Kraków Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering. P H D D ISSERTATION M ETHODS FOR MODELING SELF - ADAPTIVE MOBILE CONTEXT- AWARE SYTEMS. AUTHOR : Szymon Bobek. S UPERVISOR : Grzegorz J. Nalepa, Ph.D.. Kraków 2016.

(2) Akademia Górniczo-Hutnicza im. Stanisława Staszica w Krakowie Wydział Elektrotechniki, Automatyki, Informatyki i In˙zynierii Biomedycznej. ROZPRAWA D OKTORSKA M ETODY MODELOWANIA ADAPTOWALNYCH SYSTEMÓW ´ MOBILNYCH SWIADOMYCH KONTEKSTU. AUTOR : Szymon Bobek. P ROMOTOR : dr hab. inz˙ . Grzegorz J. Nalepa, prof. AGH. Kraków 2016.

(3) iii. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(4) iv. Monice i Wojtusiowi. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(5) Abstract This dissertation is concerned with methods for modelling self-adaptive mobile context-aware systems. Providing adaptability in such systems is crucial to assure that they work as expected by the user. Mobile environment changes very fast, either due to the user mobility, evolution of user habits, or because of the dynamic nature of the environment itself. Therefore, constant updates and modifications of knowledge are obligatory to provide optimal system performance. The primary research problem of this dissertation is to propose methods for uncertain knowledge modelling and mediation in mobile context-aware systems, to support the user in adjusting the system to his or her personal preferences and habits, and to improve management of uncertain and incomplete knowledge in order to enhance the self-adaptation capabilities of the system. To solve this research problem, the dissertation makes the following original contributions. First of all, an analysis of challenges and requirements for designing and developing mobile context-aware system was conducted. Based on that, the uncertainty modelling mechanism that uses certainty factors and probabilistic interpretation of rule models was proposed, alongside with reasoning algorithms that allows for processing uncertain knowledge bases. This was followed by the proposal of three types of adaptability methods: datadriven, model-driven and user-driven that exploit respectively the strengths of machine learning methods, rule-based modelling language, and user feedback. Finally, mediation techniques and protocols were proposed in order to assure the system intelligibility. The theoretical part of the dissertation was followed by the description of the software toolkit which was developed during the research. This toolkit provides a comprehensive practical support for building mobile context-aware systems, and was used to perform evaluation of the results of the thesis. In order to evaluate the results of the dissertation the qualitative and quantitative evaluation of the selected methods and tools were conducted. The former focuses on the feasibility study of the rule-based recommender system for museum visitors with methods presented in this dissertation. In particular the uncertainty handling mechanisms and short-time adaptability features are presented there. The latter concentrates on mediation techniques evaluation on the example of indoor navigation system, and the study of efficiency of reasoning engines.. v.

(6) Streszczenie Przedstawiona rozprawa doktorska dotyczy metod modelowania adaptowalnych systemów mobilnych ´ s´wiadomych kontekstu. Srodowiska w których działaja˛ tego typu systemy charakteryzuja˛ si˛e wysoka˛ dynamika˛ zachodzacych ˛ w nich procesów i w efekcie podlegaja˛ nieustannym zmianom, zarówno w wymiarze krótkoterminowym jak i długoterminowym. Jest to spowodowane fizyczna˛ mobilno´scia˛ uz˙ ytkownika, zmiana˛ jego nawyków i potrzeb, a takz˙ e dynamika˛ samego otoczenia. W zwiazku ˛ z tym, zapewnienie metod wspierajacych ˛ adaptowalno´sc´ stanowi kluczowy element budowania inteligentnych systemów mobilnych. Głównym problemem badawczym niniejszej pracy jest zaproponowanie metod modelowania i mediacji wiedzy w mobilnych systemach s´wiadomych kontekstu. Metody te powinny wspierać uz˙ ytkownika w dostosowywaniu działania systemu do jego potrzeb i preferencji, poprzez uwzgl˛ednienie dynamiki i niepewno´sci danych kontekstowych. W celu realizacji załoz˙ eń wyznaczonych przez problem badawczy, autor rozprawy prezentuje w niej nast˛epujace ˛ osiagni˛ ˛ ecia stanowiace ˛ oryginalny wkład w dziedzin˛e. W pierwszej kolejno´sci przeprowadzono analiz˛e wymagań dla procesów projektowania, budowania i uruchamiania mobilnych systemów s´wiadomych kontekstu, która posłuz˙ yła za punkt wyj´scia do dalszych prac. Bazujac ˛ na wnioskach z analizy wymagań, zaproponowano dwa mechanizmy modelowania i przetwarzania regułowej wiedzy niepewnej, oparte odpowiednio o algebr˛e współczynników niepewno´sci i interpretacj˛e probabilistyczna˛ modeli regułowych. W dalszej kolejno´sci opracowano trzy metody wspierajace ˛ adaptowalno´sc´ systemów s´wiadomych kontekstu oparte odpowiednio na metodach uczenia maszynowego, operatorach czasowych, oraz mediacji wiedzy. Trzecia metoda obejmuje dwa podej´scia do mediacji kontekstu, oparte o bezpo´sredni i po´sredni udział uz˙ ytkownika w procesie wnioskowania. W celu ewaluacji zaproponowanych metod został stworzony zestaw narz˛edzi do budowania mobilnych systemów s´wiadomych kontekstu. Wyniki teoretyczne jak i praktyczne pracy zostały poddane ewaluacji wzgl˛edem sformułowanych na samym poczatku ˛ rozprawy wymagań. W ewaluacji uwzgl˛edniono studium wykonalno´sci, a takz˙ e testy wydajno´sci i poprawno´sci działania systemów zbudowanych w oparciu o metody i narz˛edzia zaproponowane w niniejszej pracy.. vi.

(7) Contents Abstract. v. Streszczenie. vi. List of Figures. xi. 1 Introduction. 1. 1.1. Motivation and scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 1. 1.2. Objectives and plan of work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 3. 1.3. Original contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 6. 1.4. Exclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 7. 2 Context-aware systems development 2.1. 2.2. 10. Acquisition phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 13. 2.1.1. Software architectures for context acquisition and delivery . . . . . . . . . . . . . . .. 14. 2.1.2. Context acquisition frameworks for mobile devices . . . . . . . . . . . . . . . . . . .. 15. Modelling phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 17. 2.2.1. Knowledge engineering approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 18. 2.2.2. Machine learning approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 23. 2.3. Processing phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 24. 2.4. Requirements for building context-aware systems in dynamic environments . . . . . . . . .. 26. 2.5. 2.4.1. Uncertainty handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 28. 2.4.2. System adaptability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 29. 2.4.3. Mediation and intelligibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 30. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 30. 3 Handling uncertainty in rule-based knowledge representation. 32. 3.1. Classification of sources of uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 33. 3.2. Methods for modelling and handling uncertainty . . . . . . . . . . . . . . . . . . . . . . . . .. 36. 3.2.1. Probabilistic approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii. 37.

(8) viii. CONTENTS. 3.3. 3.4. 3.5. 3.6. 3.2.2. Fuzzy logic approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 37. 3.2.3. Machine learning approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 38. 3.2.4. Certainty factors approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 39. Certainty factors algebra and XTT2 knowledge representation . . . . . . . . . . . . . . . . .. 39. 3.3.1. Certainty factors algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 40. 3.3.2. XTT2 knowledge representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 41. Introducing certainty factors algebra into rules . . . . . . . . . . . . . . . . . . . . . . . . . .. 43. 3.4.1. Introducing certainty factors to a single rule . . . . . . . . . . . . . . . . . . . . . . .. 43. 3.4.2. Using certainty factors in sets of rules . . . . . . . . . . . . . . . . . . . . . . . . . . .. 47. 3.4.3. Use case scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 49. Probabilistic interpretation of XTT2 models . . . . . . . . . . . . . . . . . . . . . . . . . . .. 50. 3.5.1. Transformation into Bayesian network . . . . . . . . . . . . . . . . . . . . . . . . . .. 50. 3.5.2. Learning probability distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 52. 3.5.3. Inference in the probabilistic models . . . . . . . . . . . . . . . . . . . . . . . . . . .. 53. 3.5.4. Use case scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 55. Summary and further challenges in handling uncertainty in user-centric systems . . . . . . .. 56. 4 Adaptability of mobile context-aware systems. 58. 4.1. Classification of adaptability types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 59. 4.2. Data-driven adaptability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 61. 4.3. 4.4. 4.5. 4.2.1. Batch learning for sensors usage patterns discovery . . . . . . . . . . . . . . . . . . .. 62. 4.2.2. Incremental knowledge discovery for building models of user habits . . . . . . . . .. 65. 4.2.3. Introducing uncertainty into Hoeffding trees . . . . . . . . . . . . . . . . . . . . . . .. 69. 4.2.4. Summary and practical examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 76. Model-driven adaptability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 77. 4.3.1. Statistical operators for capturing dynamics of mobile environment . . . . . . . . . .. 78. 4.3.2. Time-based operators for reducing temporal ambiguities . . . . . . . . . . . . . . . .. 79. 4.3.3. Summary and practical examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 80. User-driven adaptability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 82. 4.4.1. Intelligibility for explicit user feedback . . . . . . . . . . . . . . . . . . . . . . . . . .. 83. 4.4.2. Mediation for implicit user feedback . . . . . . . . . . . . . . . . . . . . . . . . . . .. 84. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 84. 5 Semantic context mediation 5.1 S. Bobek. Objectives of mediation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Methods for modeling self-adaptive mobile context-aware sytems. 86 87.

(9) CONTENTS. ix. 5.2. 91. 5.3. 5.4. Implicit user feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1. Context representation and knowledge compilation . . . . . . . . . . . . . . . . . . .. 92. 5.2.2. Mediating user context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 95. Explicit user feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 98. 5.3.1. Introducing semantics into rule-based models . . . . . . . . . . . . . . . . . . . . . .. 5.3.2. Building an ontology for contextual models . . . . . . . . . . . . . . . . . . . . . . . 102. 5.3.3. Semantic queries for intelligibility improvement . . . . . . . . . . . . . . . . . . . . . 105. 5.3.4. Triggering mediation and verifying its results . . . . . . . . . . . . . . . . . . . . . . 107. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111. 6 Toolkit for building mobile context-aware systems 6.1. 6.2. 6.3. 6.4. 99. 113. Acquisition of contextual information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 6.1.1. AWARE framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116. 6.1.2. Learning middleware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117. Modelling rule-based mobile context-aware systems . . . . . . . . . . . . . . . . . . . . . . . 118 6.2.1. C ONTEXT V IEWER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118. 6.2.2. HMR+ rule language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123. 6.2.3. Adaptation of models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126. Execution of context-aware models with H EA RTD ROID . . . . . . . . . . . . . . . . . . . . 127 6.3.1. Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128. 6.3.2. Inference modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129. 6.3.3. Integration with context-aware systems . . . . . . . . . . . . . . . . . . . . . . . . . . 130. 6.3.4. Time handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132. 6.3.5. C ONTEXT S IMULATOR for simulating mobile environments . . . . . . . . . . . . . . 134. Feedback loop in system development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 6.4.1. Collecting feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136. 6.4.2. Building and updating models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136. 6.4.3. Adjusting context providers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136. 6.5. AMVCBC architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137. 6.6. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139. 7 Use cases and evaluation 7.1. S. Bobek. 140. Qualitative evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 7.1.1. Assumptions and use case scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142. 7.1.2. Modelling adaptable recommender system in dynamic environment . . . . . . . . . . 143. Methods for modeling self-adaptive mobile context-aware sytems.

(10) x. CONTENTS. 7.1.3 7.2. 7.3. Results of simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147. Quantitative evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 7.2.1. Utilizing user feedback for ambiguity resolution in indoor navigation system . . . . 150. 7.2.2. Efficiency of reasoning of rule engines for mobile devices . . . . . . . . . . . . . . . 153. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157. 8 Concluding remarks and future work. 161. 8.1. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161. 8.2. Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163. A Context-aware models for H EA RTD ROID. 165. A.1. Context-aware application recommender in HMR+ . . . . . . . . . . . . . . . . . . . . . . . 165. A.2. Context-aware application recommender in XTT2 . . . . . . . . . . . . . . . . . . . . . . . . 173. A.3. H EA RTD ROID Reasonign log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 A.3.1 Compact debug log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 A.3.2 States transitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188. Bibliography. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems. 204.

(11) List of Figures 1.2. Proposal of Adaptable Model, View, Context-Based-Controller (AMVCBC) Architecture . .. 5. 2.1. Three general phases of building context-aware systems. . . . . . . . . . . . . . . . . . . . .. 12. 2.2. How three phases of building context-aware systems concern four requirements for mobile context-aware systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 13. 2.3. An example of CML model [70] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 21. 2.4. An example of a Petri Net model of the context process [76] . . . . . . . . . . . . . . . . . .. 23. 2.5. An example of Bayesian network for human activity reasoning [84] . . . . . . . . . . . . . .. 24. 2.6. Trade-off between different modelling and architectural approaches with respect to the mobile context-aware systems requirements (4R). . . . . . . . . . . . . . . . . . . . . . . . . . .. 26. 2.7. Phases of building mobile context-aware systems. . . . . . . . . . . . . . . . . . . . . . . . .. 27. 3.1. Visualization of epistemic and aleatoric sources of uncertainty. . . . . . . . . . . . . . . . . .. 34. 3.2. Classification of sources of uncertainty in mobile context-aware systems. . . . . . . . . . . .. 34. 3.3. Fragment of an XTT2 model for context-aware system for applications recommendation and profile switching. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 3.4. 42. Fragment of an XTT2 model for context-aware application for recommending points of interests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 49. 3.5. Bayesian network representation of a model presented in Fig. 3.3 . . . . . . . . . . . . . . . .. 51. 4.1. Probability of GPS sensor being active, learned on samples from five days [109] . . . . . . .. 65. 4.2. Difference in power consumption for device with and without learning algorithm implemented, with speed threshold set to 5km/h [23] . . . . . . . . . . . . . . . . . . . . . . . . . .. 65. 4.3. An output from Hoeffding decision tree sampled every 30 minutes for 24 hour time span. . .. 67. 4.4. Decision tree for prediction of mobile network usage with respect to user location, time and activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 68. 4.5. Rule representation of a decision tree presented in Fig. 4.4 encoded with THE XTT2 format. 68. 4.6. Decision tree generated with uncertain data for the dataset used for building a tree from Fig. 4.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi. 74.

(12) LIST OF FIGURES. xii. 4.7. XTT2 table generated from the uncertain decision tree depicted in Fig. 4.6. . . . . . . . . . .. 75. 4.8. Dependency between erroneous readings and the accuracy of decision tree algorithms. . . .. 75. 4.9. An output from uncertain decision tree sampled every 30 minutes for 24 hour time span. . .. 76. 4.10 Actions XTT2 table from Fig. 3.3 in Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . . . .. 81. 4.11 Actions XTT2 table from Fig. 4.10 enhanced with parametrized statistical operators . . . . .. 81. 4.12 Fragment of the XTT2 model that controls accelerometer sampling rate based on the dynamic of the sensor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 82. 5.1. Workflow of the mediation process with implicit and explicit user feedback. . . . . . . . . .. 89. 5.2. XTT2 model for energy saving with respect to predicted mobile network usage. . . . . . . .. 89. 5.3. Sequence diagram for the explicit mediation. . . . . . . . . . . . . . . . . . . . . . . . . . . .. 90. 5.4. Sequence diagram for the implicit mediation. . . . . . . . . . . . . . . . . . . . . . . . . . . .. 91. 5.5. Objectives and types of semantic context mediation. . . . . . . . . . . . . . . . . . . . . . . .. 92. 5.6. Architecture of the implicit semantic mediation system . . . . . . . . . . . . . . . . . . . . .. 93. 5.7. Example of a decision tree from a question forest . . . . . . . . . . . . . . . . . . . . . . . . .. 96. 5.8. Fragment of a diagram representing class relatinoship in the ontlogy for XTT2 (bold nodes) and AWARE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101. 5.9. Fragment of an upper ontology for XTT2 and AWARE with grey boxes representing individuals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102. 6.1. Architecture of AWARE framework [57] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116. 6.2. The concept of C ONTEXT V IEWER’s architecture . . . . . . . . . . . . . . . . . . . . . . . . . 120. 6.3. C ONTEXT V IEWER user interface in graphical display mode . . . . . . . . . . . . . . . . . . 121. 6.4. Workflow of the process of automated discovery of uncertain XTT2 models. . . . . . . . . . 127. 6.5. Architecture of H EA RTD ROID inference engine . . . . . . . . . . . . . . . . . . . . . . . . . 129. 6.6. State interpretation in H EA RTD ROID inference engine. . . . . . . . . . . . . . . . . . . . . . 132. 6.7. Architecture of C ONTEXT S IMULATOR application . . . . . . . . . . . . . . . . . . . . . . . . 134. 6.8. Phases of building mobile context-aware systems supported by tools and methods developed within this work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135. 6.9. Dependencies between the methods developed within this thesis and AMVCBC) architecture components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138. 7.2. An example of an ambiguous location estimation captured by the mediation evaluation tool [92]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152. 7.3. An example of a mobile microlocation mediation application asking the user for help. . . . . 153. 7.4. Floor layout of the experiment environment with three sample routes [92] . . . . . . . . . . . 154. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(13) xiii. LIST OF FIGURES. 7.5. Results of time efficiency benchmark of most popular rule-engines. . . . . . . . . . . . . . . 156. 7.6. Failures of inference for models with a lot of rules.. 7.7. Results of memory efficiency benchmark. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157. 7.8. Ranking of the rule engines, with respect to the key features presented in Table 7.4 and their. . . . . . . . . . . . . . . . . . . . . . . . 156. weighting given in Table 7.5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 8.1. Coverage of gaps identified in Sec. 2.4 by methods developed within the research described in this dissertation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(14) Chapter 1. Introduction This chapter introduces the reader to the contents of the dissertation. It consists of four sections: Section 1.1 presents the background of the research, defines the scope of the thesis and argues why it is an important issue in the field of computer science and artificial intelligence [132, 58]. Section 1.2 specifies the goal of the research and describes the steps taken to achieve it. Section 1.3 emphasizes the original contribution of the thesis. Section 1.4 discusses the issues that were deliberately not addressed in this thesis, and left for further research.. 1.1. Motivation and scope. Research in the area of pervasive computing [152] and ambient intelligence [103] aims to make use of context information1 to allow devices or applications to behave in a context-aware, thus “intelligent” way. Context-aware systems have been studied in several fields and developed for over 30 years. However, they are still identified by Gartner, alongside cloud computing, business impact of social computing and pattern based strategy, as being one of the broad trends that will change IT and the economy in the next 10 years2 . In this work, the primary emphasis is put on, what is called by Gartner, the Smart Advisors systems, narrowed to the concept of mobile context-aware personal assistants. This class of systems is currently between the Innovation Trigger and Peak of Inflated Expectation phases on the Gartner hype cycle diagram, as shown in Fig. 1.1. This means that such systems are in the very early phase of commercial development, requiring lot of additional research and prototype implementations, but the interests and hopes associated with this technology are very high. Most of the existing frameworks for building context-aware systems (e.g. ContextToolkit [45], SOCAM [67], CoBrA [34]) use static context models that do not change over time. In other approaches [59], 1. In this dissertation the primary focus will be put on mobile context-aware systems. Hence, context information will be considered in terms of any data that can be obtained from mobile device sensors or inferred based on it. 2 See: http://www.gartner.com/newsroom/id/1470115.. 1.

(15) 1.1. Motivation and scope. 2. Figure 1.1: Gartner hype cycle of emerging technologies as of July 2015.3. that take into consideration issues of imperfectness of static model, usually machine learning (ML) methods are used for providing adaptability to the system. However, the classic batch ML algorithms need time and data to adjust to a new situation. Due to their generalisation nature they are insensitive to rapidly changing data and need time to adjust to new environment conditions. Although most recent research in the area of knowledge discovery from data streams aims at addressing these issues [91, 157], there is still need for methods that will improve their capabilities of adaptation to sudden changes. Most importantly, to avoid the delays needed for automatic concept drift detection [61], the machine learning models should be supported by the meta knowledge that will allow for quick tuning of the system to unseen situations. Hence, there is a need to introduce modelling methods that will capture both high dynamics of a mobile environment and long-term changes in user habits and preferences. Providing mechanisms for dealing with incomplete or imperfect information is one of the most important factors in designing reliable systems. In mobile context-aware systems the incompleteness of information is understood in terms of missing data, while imperfectness refers to ambiguous or uncertain context. These problems are nowadays addressed mainly by probabilistic approaches like Bayesian networks. However, there is a gap between the conceptual models of context and the numerical models that are used by machine learning methods. Such models cannot be directly modified by the user, as this requires expert knowledge 3. Source: http://www.gartner.com.. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(16) 1.2. Objectives and plan of work. 3. in probability theory and machine learning. In many context-aware systems that operate in highly dynamic environments, once observed data cannot be treated as certain for unlimited period of time. For instance user activity observed five minutes ago, may not be certain now. What is even more important, the degree of uncertainty changes itself as it depends on many contextual factors. For instance uncertainty of the user location caused by outdated GPS readings will be higher during a day, as it is more likely that the user location will change very often, causing location estimates validity expire more quickly. This gives motivation to develop new knowledge management methods that will allow for modelling the dynamics of uncertainty and that will fill the gap between conceptual and numerical models. We observe a paradigm shift in terms of users of context-aware systems, where the user is no longer just a client of the "black box" system, but rather a conscious operator [45]. This moves a role of the user from the data receiver also to the data provider, but at the same time requires from him or her to understand how the system works. Methods that allow to model dialogue between the user and the system are called mediation [45]. Although there were attempts to incorporate mediation techniques for dealing with imperfectness of knowledge in context-aware systems [2, 131], there is no approach that would allow for building context-based mediators that will allow not only to resolve ambiguous situations but also and enhance adaptability by incorporating user in the reasoning process. This gives the motivation to develop methods for designing mediation that will allow including user into the context-based inference process and thus improve intelligibility and adaptability of the system.. 1.2. Objectives and plan of work. Providing adaptability of mobile context-aware systems is crucial to assure it works along with user expectations. Mobile environment in which the system operates is characterised by high dynamics of the processes that occur therein. The environment changes very fast due to user mobility, but also the objectives of the system itself evolves, as the user changes his or her needs and preferences. Therefore, constant model updates and modifications are one of the primary requirement for the mobile context-aware systems. In stationary environments, the system tuning was performed on the initial phase of the system deployment either by the expert or by the user, and did not change over time. Additionally, the stationary solutions were designed upon an assumption that all the contextual data needed for the reasoning is always available and certain. Finally, the classic approaches for building context-aware systems [6] did not include user in the process reasoning, which limits the degree of the user understanding and trust in the system (so called intelligibility [98]). All of the above are the main reasons why the existing context-aware frameworks provide limited support for adaptability in dynamic environments and reasoning based on incomplete data. They also do not provide support for modelling mediation and intelligibility of the system during the design phase. Hence, the main objectives of the research were to propose methods for uncertain knowledge modelling and mediation S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(17) 1.2. Objectives and plan of work. 4. in mobile context-aware system, to support user in adjusting the system to his or her personal preferences or habits, and to improve management of uncertain and incomplete knowledge management in order to enhance the self-adaptation capabilities of the system. To achieve these objectives, a set of methods that assures the system intelligibility, robustness, privacy and efficiency were investigated within this dissertation. The work plan consists of the achieving following sub-goals: 1. Formulation of new methods for supporting user in adapting the system to his or her needs and preferences in dynamic environments based on machine learning and logic-based approach. 2. Improvement of knowledge management methods for imperfect or incomplete context that will allow for modelling dynamics of the uncertainty and provide efficient reasoning under incomplete or missing data. 3. Formulation of mediation techniques for improving system intelligibility and adaptability, and help in resolving uncertainty in data. To combine the above methods into one complete framework for building mobile context-aware systems, an architecture for such solutions is required. Figure 1.2 shows the proposal of Adaptable Model, View, Context-Based Controller architecture (AMVCBC) for context-aware applications. It can be defined as an extension of standard Model View Controller software architectural pattern [74], that includes context and adaptability as a part of the model. The adaptable model layer of the architecture is responsible for discovery and adaptation to user long-term preferences and habits (profiles) but also should provide mechanisms allowing to react on dynamically changing environmental conditions. The context-based controller layer provides mechanisms for context-based mediation between a user, and other system components that will allow the system to resolve vagueness and incompleteness of background knowledge data. This layer should also provide an input for adaptable model layer that will support adaptability of the system by taking into consideration user feedback and other mediators (probabilistic, ontology-based, etc.). The deliverables of the research presented in this dissertation form a complete set of algorithms, tools and methodologies that can be combined into the framework based on the AMVCBC pattern. The discussion on the aforementioned research objectives was partitioned in this dissertation into six chapters, organised as follows. The comprehensive analysis of the state-of-the art solutions in the area of mobile context-aware systems was given in Chapter 2. In Sec. 2.1 the architectural solutions for context acquisition and dissemination were discussed. The context-modelling approaches were discussed and compared in Sec. 2.2, while in Sec. 2.3 methods for processing contextual data were described. Based on this survey, in Sec. 2.4 the main challenges in building mobile context-aware systems were identified, and the requirements for the mobile context-aware system were formulated. These requirements alongside with the research areas that concern them, determined the contents of the rest of this dissertation. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(18) 5. 1.2. Objectives and plan of work. Context provider. Context provider. Intelligent Middleware. Context provider. Background Knowledge. Application Model. Reasoning. Mediation Protocols. Bayes Mediator. Reasoning Protocols. VIEW. Rule-based Mediator. Application. CONTEXT-BASED CONTROLLER. ADAPTABLE MODEL. Mediation Model and Mechanism. Human Mediator. Figure 1.2: Proposal of Adaptable Model, View, Context-Based-Controller (AMVCBC) Architecture. Chapter 3 covers issues of modelling and handling uncertainty in mobile context-aware systems. Section 3.1 identifies the main sources of uncertainty in mobile context-aware systems, while Sec. 3.2 discusses methods for modelling and handling this different types of uncertain context. Such an overview of uncertainty handling methods is followed by the two methods developed as a result of the research presented in this dissertation. In particular this includes certainty factors approach presented in Sec. 3.3 and 3.4, and probabilistic interpretation of decision tables, given in Sec. 3.5. In Chapter 4 methods for providing long-term and short-term adaptability of the mobile context-aware systems were given. Section 4.1 provides a classification of different adaptability types concerned in this work. Following sections elaborate more on each of these types. Section 4.2 discusses usage of machine learning methods and rules for improving adaptability of mobile context-aware systems. Section 4.3 describes statistical and time-based operators in rules for achieving short-term adaptability. Finally, Sec. 4.4 provides a brief introduction to the user-driven adaptability, which is discussed in details in Chapter 5 alongside with mediation techniques. In particular this chapter covers definition of mediation objectives given in S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(19) 1.3. Original contribution. 6. Sec. 5.1 and description of two types of mediation techniques. These techniques include implicit and explicit user feedback, discussed in details respectively in Sec. 5.2 and 5.3. Tools for context acquisition, analysis, modelling and processing that were designed and implemented as a part of this dissertation, were described in Chapter 6. In particular, Sec. 6.1 focuses on the AWARE framework used in this dissertation as a basic context provider system. In Sec. 6.2 the HMR+ language for modelling adaptable mobile context-aware systems was presented, together with a C ONTEXT V IEWER toolkit for context visualisation and initial preprocessing. Section 6.3 describes the runtime and simulation environments formed by H EA RTD ROID inference engine, and C ONTEXT S IMULATOR simulation framework. In Sec. 6.4 the applicability of the methods and tools designed within this dissertation to enhance the self-adaptability of the mobile context-aware systems was concluded. The qualitative and quantitative evaluation of the selected methods and tools was presented in Chapter 7. The former focuses on the feasibility study of the rule-based recommender system for museum visitors with methods presented in this dissertation. In particular the uncertainty handling mechanisms and short-time adaptability features are presented there. The latter concentrates on mediation techniques evaluation on the example of indoor navigation system, and the study of reasoning engines efficiency. Finally, the summary of the research presented in this dissertation along with the future plans was given in Chapter 8.. 1.3. Original contribution. The following results are considered to be the most important original contributions of this thesis: 1. Analysis of challenges and requirements for designing and developing mobile context-aware systems – formulation of a set of requirements that need to be addressed in order to build a high quality mobile context-aware systems was given in Sec. 2.4. 2. Uncertainty handling mechanism based on modified certainty factors algebra and rule-based notation – extension to the XTT2 knowledge representation that allows to model uncertain knowledge and the inference algorithm for uncertain reasoning was given in Sec. 3.4. 3. Uncertainty handling mechanism based on probabilistic interpretation of rule-based notation – the Bayesian representation of the XTT2 decision tables and hybrid inference algorithm for uncertain reasoning was described in Sec. 3.5. 4. Pattern discovery algorithm from uncertain streaming data – algorithm for uncertain XTT2 tables generation, based on the Hoeffding tree algorithm and the uARFF data format for handling uncertain training instances were presented respectively in Sec. 4.2 and Sec. 6.2.3. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(20) 1.4. Exclusions. 7. 5. Extension of the rule representation formalism, called HMR+, that introduces time-based and statistical operators to the XTT2 notation – a set of statistical and temporal operators that can be used in conditional parts of rules was described in Sec. 4.3. 6. Implicit and explicit mediation mechanisms for intelligibility improvements and ambiguity resolution in mobile context-aware systems – two complementary methods for modelling dialogue between the system and the user was described in Sections 5.2 and 5.3. 7. Toolkit for mobile context-aware system design and development – a set of tools that includes web application for context visualisation and initial systematization (C ONTEXT V IEWER), and runtime framework for execution and deployment of mobile context-aware system (H EA RTD ROID inference engine and C ONTEXT S IMULATOR simulation framework) were described in Chapter 6.. 1.4. Exclusions. As context awareness constitutes a wide research area [6], this thesis is focused on modelling contextaware systems in mobile environments. Therefore, the context definition used in the dissertation is limited to the context provided by the mobile device sensors (GPS, WiFi, user activity, etc.), external sources that describe environment conditions (weather forecast, road traffic, etc.) and user provided data such as profiles or interests. Other context-aware systems that use for instance eye tracking for user profile discovery [64], or EEG signal analysis for emotions discovery [75] are beyond of the scope of the discussion presented in this thesis. However, methods described hereby can be directly used also for such systems. Although providing usable, high quality human-computer interaction for context-aware systems is of a key importance in building user-centric applications, issues concerning designing user interfaces are far beyond the scope of this work. The primary focus in this dissertation was to provide methods for inferring and selecting what to present to the user, not how. Therefore, no human-computer interaction issues were addressed here. Finally, although processing streaming contextual data may imply usage of time operators, the temporal logics were not used in this work to model time-related issues. Temporal modalities were introduced in this dissertation mainly to deal with noisy data, and to model trends in context streams. Therefore, they can be considered as syntactic sugar for statistical calculations over a sliding windows. Powerful temporal logic mechanics were considered not suitable for such purposes. The work described in this dissertation is supported by the "Uncertain and incomplete knowledge modelling for mediation in mobile context-aware systems" (K NOW M E) research project funded from NCN (National Science Center) resources for science, reference number 2014/13/N/ST6/01786 (PRELUDIUM 7)4 . 4. See: http://glados.kis.agh.edu.pl. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(21) 1.4. Exclusions. 8. The author was also involved in the number of other projects that allowed him to get knowledge related knowledge engineering system and semantics, and gain experience in this area: 1. 2008-2009 – H E K AT E (MNiSW N516 024 32/2878): Hybrid Knowledge Engineering5 . 2. 2009-2011 – R EBIT (POIG 1.3.1): Business and Technological Rules Management6 . 3. 2010-2012 – BIMLOQ (MNiSW N516 422338): Business Models Optimization for Quality7 . 4. 2012-2015 – P ROSECCO (PBS1/B3/14/2012): Processes Semantics Collaboration for Companies8 . Partial results of the research presented in this dissertation were published in (or submitted to) the international journals and conference proceedings.. JCR Journal papers 1. Szymon Bobek and Grzegorz J. Nalepa. Uncertain context data management in dynamic mobile environments. Future Generation Computer Systems. submitted, April 2016 2. Szymon Bobek and Grzegorz J. Nalepa. Uncertainty handling in rule-based mobile context-aware systems. International Journal of Approximate Reasoning. submitted, April 2016 3. Szymon Bobek, Grzegorz J. Nalepa, Antoni Lig˛eza, Weronika T. Adrian, and Krzysztof Kaczor. Mobile context-based framework for threat monitoring in urban environment with social threat monitor. Multimedia Tools and Applications, 2014. Online first at: http://dx.doi.org/10.1007/ s11042-014-2060-9 4. Grzegorz J. Nalepa and Szymon Bobek. Rule-based solution for context-aware reasoning on mobile devices. Computer Science and Information Systems, 11(1):171–193, 2014. Conference proceedings referenced in Web of Science ´ z˙ yński, and Grze1. Lukas Köping, Marcin Grzegorzek, Frank Deinzer, Szymon Bobek, Mateusz Sla gorz J. Nalepa. Improving indoor localization by user feedback. In Information Fusion (Fusion), 2015 18th International Conference on, pages 1053–1060, July 2015 2. Szymon Bobek and GrzegorzJ. Nalepa. Compact representation of conditional probability for rulebased mobile context-aware systems. In Antonis Bikakis, Paul Fodor, and Dumitru Roman, editors, Rules on the Web. From Theory to Applications, Lecture Notes in Computer Science. Springer International Publishing, 2015 ´ z˙ yński, and Grzegorz Jacek Nalepa. Capturing dynamics of mobile 3. Szymon Bobek, Mateusz Sla context-aware systems with rules and statistical analysis of historical data. In Leszek Rutkowski, Marcin Korytkowski, Rafal Scherer, Ryszard Tadeusiewicz, Lotfi A. Zadeh, and Jacek M. Zurada, ed5. See: http://hekate.ia.agh.edu.pl See: http://www.rebit.zarz.agh.edu.pl 7 See: http://bimloq.ia.agh.edu.pl 8 See: http://prosecco.agh.edu.pl 6. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(22) 1.4. Exclusions. 9. itors, Artificial Intelligence and Soft Computing, volume 9120 of Lecture Notes in Computer Science, pages 578–590. Springer International Publishing, 2015 4. Szymon Bobek, Olgierd Grodzki, and Grzegorz J. Nalepa. Indoor microlocation with BLE beacons and incremental rule learning. In Cybernetics (CYBCONF), 2015 IEEE 2nd International Conference on, pages 91–96, June 2015 5. Szymon Bobek and Grzegorz J. Nalepa. Incomplete and uncertain data handling in context-aware rule-based systems with modified certainty factors algebra. In Antonis Bikakis, Paul Fodor, and Dumitru Roman, editors, Rules on the Web. From Theory to Applications, volume 8620 of Lecture Notes in Computer Science, pages 157–167. Springer International Publishing, 2014 ´ z˙ yński. Challenges for migration of rule-based 6. Szymon Bobek, Grzegorz J. Nalepa, and Mateusz Sla reasoning engine to a mobile platform. In Andrzej Dziech and Andrzej Czyz˙ ewski, editors, Multimedia Communications, Services and Security, volume 429 of Communications in Computer and Information Science, pages 43–57. Springer Berlin Heidelberg, 2014 7. Szymon Bobek, Krzysztof Porzycki, and Grzegorz J. Nalepa. Learning sensors usage patterns in mobile context-aware systems. In Maria Ganzha, Leszek A. Maciaszek, and Marcin Paprzycki, editors, Proceedings of the Federated Conference on Computer Science and Information Systems – FedCSIS 2013, Krakow, Poland, 8-11 September 2013, pages 993–998. IEEE, Sept 2013 Other conference papers ´ z˙ yński, and Grzegorz J. Nalepa. Mod1. Szymon Bobek, Sebastian Dziadzio, Paweł Jaciów, Mateusz Sla eling and Using Context: 9th International and Interdisciplinary Conference, CONTEXT 2015, Lanarca, Cyprus, November 2-6,2015. Proceedings, chapter Understanding Context with ContextViewer – Tool for Visualization and Initial Preprocessing of Mobile Sensors Data, pages 77–90. Springer International Publishing, Cham, 2015 ´ z˙ yński, Szymon Bobek, and Grzegorz J. Nalepa. Migration of rule inference engine to 2. Mateusz Sla mobile platform. challenges and case study. In Grzegorz J. Nalepa and Joachim Baumeister, editors, Proceedings of 10th Workshop on Knowledge Engineering and Software Engineering (KESE10) co-located with 21st European Conference on Artificial Intelligence (ECAI 2014), Prague, Czech Republic, August 19 2014, volume 1289 of CEUR Workshop Proceedings, 2014. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(23) Chapter 2. Context-aware systems development The notion of context has been important in conceptualization of computer systems for many years. However, providing its constructive and precise definition proves to be a non-trivial task. A general observation is that context is about evolving, structured, and shared information spaces, and that such spaces are designed to serve a particular purpose [38]. Schilit et al. [136] narrows this definition to be where you are, who you are with, and what resources are nearby. A similar definition was given in [156], where context is defined as individuality, activity, location, time, and relations. The common sense definition of context was also given by Bolchini et al. [26], who describes it as a set of variables that may be of interest for an agent and that influence its actions. A different approach was presented in [33], where authors do not give a precise definition but rather define categories of contextual information, such as: computing context, physical context, time context and user context. In this work mostly the Dey’s [48] definition will be used, that describes context as any information that can be used to characterize the situation of an entity. For the sake of clarity the entity is defined as a mobile user or device, while information that characterise its situation is any information that can be directly obtained from the mobile device sensors (so called low-level context), or that can be inferred based on this data (high-level context). In particular this definition includes the low-level contextual information provided by the AWARE framework1 , microlocalization based on Bluetooth Low Energy (BLE) beacons, and user profiles in terms of domain specific preferences, interests, etc. The precise definition of the low-level context that will be considered in this dissertation is presented in Table 2.1. Obtaining the low-level contextual information is performed in the first phase of building context-aware systems, which is defined as Acquisition phase. The successive phases are respectively: Modelling phase and Processing phase [46]. Fig. 2.1 depicts these phases together with detailed information about their roles in the development process. Classic approach for building context-aware systems assumes that the context 1. AWARE is an open source Android instrumentation framework for logging, sharing and reusing mobile context. It was released in 2013 for the research purposes by the University of Oulu. For more details see http://www.awareframework.com. 10.

(24) 11 Sensor Accelerometer. HW x. Type SW. Description H. sensor profile and acceleration force along the device’s axis, including gravity, in m/s2 Applications x foreground and background applications used on the device Battery x battery and power events (e.g. reboot, shutdown) data Bluetooth x built-in Bluetooth sensor information and performs interval scans forvisible neighbor Bluetooth devices. Communication x users’ communication usage (i.e. phone call states, messages states) ESM x user-provided data from Experience Sampling Method (ESM) questionnaires; the ESM can be triggered by other context events, time or remotely using the AWARE Server dashboard Installations x applications added, removed or updated on the mobile device Light x sensor profile and ambient luminance, in lux Locations x x network and/or GPS locations from the mobile device; the best estimated location is provided Microlocations x x microlocations estimated with a usage of Bluetooth Low Energy beacons Network x network usage (i.e. airplane mode, Wi-Fi, mobile network, Bluetooth, GPS, Internet availability) Proximity x sensor profile and the distance between the device and an object, in near/far (binary) or centimeters (cm) Screen x x screen status (e.g. on or off) and user’s locking and unlocking events Telephony x mobile phone’s telephony capabilities (e.g. network speed, type) and GSM/CDMA towers and geighbour towers , if available Traffic x network traffic (i.e. packets and bytes) on Wi-Fi, Bluetooth, and mobile network Wifi x built-in Wi-Fi sensor information and performs interval scans for visible Activity x detect the user’s current physical activity, such as walking, driving, or standing still; this is performed by the Google Activity recognition API Weather x gathers information about the current weather in a current location of a user, as well as the forecast for the location of the user User Profile x domain specific user profile, defining user interests, preferences (eg. in terms of recommendations), etc. HW – Hardware sensor, SW – Software sensor, H – Human provided context. Table 2.1: Explicit definition of contextual information that will be used in the work. is obtained in the acquisition phase, modelled (automatically or by knowledge engineer) in the modelling phase, and finally executed in the processing phase. Such an approach, when applied to mobile context-aware systems, exposes serious drawbacks connected with the nature of environment and the system dynamics. It is worth noting, that the work presented in this dissertation focuses on the aspect of designing mobile context-aware systems. The nature of mobile environment in which such systems are immersed, implies important assumptions regarding process of their development. Most of the existing solutions were crafted for the purpose of stationary context-aware systems, which assume that the model of the environment and the user preferences are well defined a priori and do not change over time. In mobile systems this assumption does not hold, exposing the evolutionary nature of the models. The models are influenced by constantly streaming data, which additionally are neither certain nor always available. Therefore, such raw contextual data needs to be appropriately collected, processed and applied to the model iteratively, which impacts classic three-phased approach for designing context-aware systems shown in Fig. 2.1. In fact, this approach needs to be redefined to meet the requirements of mobile context-aware systems. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(25) 12. Acquisition Phase. Modelling Phase. Processing Phase. Collecting Context. (Re)Building Models. Executing Models. Figure 2.1: Three general phases of building context-aware systems.. The deep analysis of literature allowed us to formulate four main requirements (4R) that should be met by every mobile context-aware system in order to assure its high quality and to cope with such drawbacks [21, 20]. These four requirements are: 1. Intelligibility – mobile context-aware system should allow the user to understand and modify its performance. 2. Robustness – mobile context-aware system should be adaptable to changing user habits or environment conditions, and should be able to handle uncertain and incomplete data. 3. Privacy – mobile context-aware system should assure the user that his or her sensitive data are secured and not accessible by the third party. 4. Efficiency – mobile context-aware system should be efficient both in terms of resource efficiency and high responsiveness. Although the requirements are rather general, it can be shown that not all the phases of building contextaware system equally refer to them. Fig. 2.2 shows which of the requirements are particularly important for which of the development phase. Based on these dependencies, a detailed elaboration on the extensions to the three-phased approach from Fig. 2.1 was presented later in this chapter, in Sec. 2.4 (see Fig. 2.7). The rest of this Chapter is organised as follows. In Sec. 2.1 the main issues regarding acquisition of context in mobile environments are discussed. Most important architectures for context dissemination were provided, and comparison of context-management frameworks were given. Sec. 2.2 presents the overview of context modelling techniques, while Sec. 2.3 concentrates on the processing phase. The three mentioned sections describe the problem of collecting, modelling and processing context in a wide perspective, not limited to mobile environments, but point out the areas where the classic approaches are not suitable for the mobile use cases. This discussion is synthesised in Sec. 2.4, where the four requirements (4R) motivate S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(26) 13. 2.1. Acquisition phase. Intelligibility. Robustness. Efficiency. Privacy. Acquisition Phase. Modelling Phase. Processing Phase. Figure 2.2: How three phases of building context-aware systems concern four requirements for mobile context-aware systems.. the existence of three gaps (3G) present in the classic approaches for building context-aware systems, and define the main goal of this thesis. Summary of the chapter and short guide to the rest of the dissertation was given in Sec. 2.5.. 2.1. Acquisition phase. This phase is responsible for delivery of low-level contextual information to the system both for the purpose of building a model and as an input for the processing phase. The context acquisition layer mostly affect the robustness, efficiency and privacy issues, as depicted in Fig. 2.2. These three general requirements can be unfold into more specific ones, as presented below: 1. Data confidentiality. The context providers layer should not send any confidential data to the external servers, but perform all the reasoning locally. 2. Responsiveness. Context acquisition has to work under soft real-time constraints. Mobile environment is highly dynamic, and the context providers should follow rapid changes of context in such an environment. 3. Resource efficiency. The system should consume as little resources as possible to work transparently in the background. 4. Energy efficiency. Most of the sensors, when turned on all the time, require a lot of energy. This may reflect on usability of the system, e.g. on a mobile phone device, and ecological aspects regarding energy saving. Therefore the acquisition layer should consume as little energy as possible. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(27) 2.1. Acquisition phase. 14. 5. Robustness. The acquisition layer should be able to provide information on the level of certainty of the data it delivers. It should also be able to work even when some of the context providers is missing, broken or intentionally turned off by the user. These requirements are highly dependant on the architecture chosen for the context provider layer, as it will be shown in the following chapter. Moreover, it is still very hard to meet all the requirements simultaneously, as there are obvious trade-offs between some of them (eg. trade-off between energy efficiency and responsiveness). The following chapter compares most common architectural solutions with respect to their requirements they suppose to meet.. 2.1.1. Software architectures for context acquisition and delivery. There are several commonly used approaches for modelling architecture of context-aware system. In this section a short overview of most common approaches will be given. Also a comparison of described architectures is presented in Table 2.2. Full circles in the table represent full support for a particular feature, whereas empty circles represent low or no support.. Direct sensor access It is the simplest architecture. It can be described as a system in which a part responsible for reasoning about a context fetches information directly from sensors or other information source. This approach is usually not very energy efficient, however it preservers privacy issues, since no communication with external servers is usually needed, and the interpretation of the sensor data as well as reasoning is performed directly on the host device [53].. Middleware infrastructure It incorporates additional layer that encapsulates the sensor layer and is usually responsible for filtering and initial interpretation of data gathered by the latter. This approach eases extensibility and reusability of hardware dependent code responsible for sensing information. The middleware approach is one of the most common one in context-aware systems area. Comprehensive comparison of existing middlewares was presented in [82].. Centralized context server In this architecture, there is an entity that acts as a context server which may be responsible for gathering contextual data from clients, and reasoning with this data. This approach is especially useful when a context-aware system is composed of many mobile devices with limited resources. The server relieves mobile agents from performing reasoning tasks. On the other hand, one has to consider privacy issues connected with sending private contextual data to remote server, quality of service issues, etc. This approach is also characterized with rather low responsiveness that stems from a possible lack of network connection or communication delays. An example of framework using context server approach is described in [34]. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(28) 15. 2.1. Acquisition phase. Energy efficiency Data confidentiality Resource efficiency Responsiveness Robustness. Direct sensor Access m l m l m. Middleware. Service Oriented. Centralized. Distributed. l l w l w. w w w m w. w m w m m. w m l m l. Table 2.2: Comparison of common context-aware architectures against features important in mobile environments. Distributed context agents In a distributed architecture, each component of the system acts as an autonomous context-aware agent that performs reasoning based on data gathered by itself and by other agents that at the moment it communicates. Agents may be implemented on different devices and can be given ability to represent different features of the environment in which they interact with other agents. Different agents may be designed to consider different contexts and can give different interpretation of the same contextual information. This type of architecture was described in details in [8, 73].. Service oriented architecture Such an architecture aims to allow building large-scale systems with loosely coupled elements. In context-aware applications, this architecture is used mainly in pervasive environment, where variety of context information from many different sources has to be processed. This architecture usually does not preserves privacy nor energy efficiency issues since usually it assumes communication over the web between each of its elements. An example of a framework built in service oriented paradigm is SOCAM [67], described briefly in the next section. Architectures presented in this section and compared in the Table 2.2 are used by the numerous of frameworks for context acquisition, discussed in the next section.. 2.1.2. Context acquisition frameworks for mobile devices. Context management frameworks can be divided into four groups: 1. Frameworks that offer collecting, storing and distributing context within the system. 2. Frameworks that provide visualisation of selected contextual information. 3. Frameworks for modelling and processing context. 4. Hybrid approaches that combine features of former three. Systems that allow for collecting and storing contextual information are one of the most popular tools for building context-aware applications. They provide basic programming interface for accessing context in a real time, as well as mechanisms for storing historical data. Examples of such frameworks are SDCF [4], AWARE [56], JCAF [7], SCOUT [149], and ContextDriod [142]. These frameworks do not provide any mechanisms for visualisation nor preprocessing of data. They serve as a middleware between the physical S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(29) 2.1. Acquisition phase. 16. sensors or native API and the context-aware application. They usually store the data locally to preserve the privacy issues, but some of them like AWARE provides mechanisms for synchronization with external databases. This allows to transfer complex data analysis from mobile devices to the more powerful machines, integrate the knowledge from many sources and exchange this knowledge between the mobile devices in the distributed manner. The other type of context-management frameworks are systems which focus on the visualisation of selected contextual data. In most cases these systems are commercial applications like: Ubidots2 or Valarm3 . However, there are also open source solutions like Nimbits4 and Freeboard5 . The visualisation frameworks give more insight into the raw contextual data by plotting them on diagrams, or augmenting them on the map. They do not provide any tools for automated analysis of data, nor any runtime. Exception here are the Ubidots and Nimbits frameworks, which allows for including simple rule-based triggers that are fired when a context defined by some preconditions is met. These frameworks are mostly designed as cloud-based solutions which communicate with many mobile devices and provide web interface for visualising data. Frameworks that support modelling context-aware systems and provide runtime environments for executing contextual models form the third class of context management systems. One of the most popular runtime for stationary context-aware application is ContextToolkit [48]. It supports rule-based modelling language and execution runtime. Other examples of rule-based frameworks crafted especially for the needs of mobile environments are Context Engine [89], SWAN [116] and H EA RT [24]. The last one is used in this thesis as a part of the runtime environment. A different approach for modelling and executing context models was presented by Brezillon et.al. [29]. They proposed a structure called Contextual Graph. It is a directed acyclic graph that represents the actions to undertake according to the context. The action nodes represent actions to undertake to achieve a goal while the event nodes describe the possible contextual issues of a given event. The last group of context-management frameworks are hybrid solutions like CoBrA [34] for building smart meeting rooms, GAIA [130] for active spaces or SOCAM [67] and mobileGaia [37] – middlewares architecture for building pervasive systems. They combine features of frameworks for collecting storing and distributing context with features of frameworks for modelling and processing context. They do not provide tools for visualisation and data analysis. What is more, they are usually built for very narrow, specific solutions, and can not be considered as frameworks for more general purpose than initially assumed. The exception is MUSIC [59] framework – an open platform for development of self-adaptive mobile applications, which includes a methodology, tools and middleware. 2. See http://ubidots.com. See https://play.google.com/store/apps/details?id=net.valarm.android.pro. 4 See http://nimbits.com. 5 See https://freeboard.io. 3. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(30) 2.2. Modelling phase. 17. Although the hybrid and integrated solutions provide usually very comprehensive approach for building context-aware systems, they are less flexible to changes in their architectures. The existing context acquisition frameworks mostly suffer from lack of mechanisms that provide high energy efficiency and robustness while providing high responsiveness and privacy. These two factors have to be resolved in order to provide a solution that meets all the requirements defined for the acquisition phase. The acquisition phase ends once the current context is ready to deliver to the next stage which is responsible for modelling raw, low-level contextual data by assigning it more semantics. Methods for modelling low-level context for deriving higher level information are discussed in the next section.. 2.2. Modelling phase. Contextual information delivered by the acquisition phase is hard to process without a prior interpretation. Thus, an important challenge in context-aware systems concerns context representation. This can be divided into two categories: • Knowledge engineering approach – in which a full model is given a priori. Knowledge management is reduced to storing a model in a system memory and executing it during the processing phase. • Machine learning approach – in which a model is not given, but rather learned from data. In this case, knowledge management is a non-trivial task, as it is responsible for shaping a model and thus providing correct behaviour of a system. There is a gap between these two approaches. Model based approach can provide reliability and – in some cases – allows for system verification, but the model itself is static and does not change over time. Machine learning provides adaptability, but the model is often subsymbolic, which makes it difficult to adjust and understand by the user. In this section most popular approaches from knowledge engineering field and machine learning area for context modelling are presented. The summary and the comparison are also given in Table 2.3. Following aspect of context modelling methods were taken into consideration: • Formalization – whether or not there are some formal foundation of the modelling language that allows for automatic verification of models. • Simplicity – whether or not the model is simple enough to support fast development and to be understandable for the end user. • Expressiveness – whether or not model is expressive enough to capture dynamic features of mobile context-aware systems. • Support for inference – whether or not there exists reasoners that allow for executing the model, preferably on the mobile platform. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(31) 18. 2.2. Modelling phase. Formalism Simplicity Expressiveness Reasoning support Handling uncertainty Adaptability support Design tools support. KeyValue w l m m m m w. Logic. Rules. l w w w w m m. w w w l w w l. Objectoriented m w l m w m l. Machine learning m m w w l l w. Ontologies. Processes. l m l w m m l. m w l m m m l. Table 2.3: Comparison of context-modelling techniques against features important in mobile context-aware systems. Full circles represent full support, whereas empty circles represent low or no support. • Handling of uncertainty – whether or not the language provides a support for handling different types of uncertainties. • Adaptability support – whether or not the model is able to adapt/evolve over time. • Design tools support – whether or not there exists tools that improve design or rapid prototyping of models.. 2.2.1. Knowledge engineering approach. Key-Value Representation One of the simplest way to define context is to use pairs of a form: key-value. The key is usually a name that defines a context property. For instance it can denote location (e.g. room) or time (e.g. daytime). The complex contexts can be represent as a union of several keys. The value represents current state of the context property (e.g. kitchen, morning, etc.) The ActiveBadges [146] based system called Watchdog described in [136] uses the key-value context representation. A simple example might be: badge location event-type action. The context in the Watchdog system is represented by three keys: badge denoting an electronic ID, location denoting a location of a person wearing a badge and an event-type that describes activity of a person. Remaining parameter represents an action that should be performed when previous three keys match actual context. An example of a context state might be following: Coffee Kitchen arriving "play -v 50 /sounds/rooster.au". We can read this as a statement: When a person that wears a badge Coffee arrives to the Kitchen, then play sound. Another example of a system that uses key-value modelling is FAWIS [40]. It is a framework for representation and translation of context information in adaptive Web-based applications. In FAWIS, context is a collection of profiles, that can be user account, network contention type, hardware or user browser. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(32) 2.2. Modelling phase. 19. The reasoning in key-values models is usually supported by a simple matching engine. When the keys match the actual context values, an action is triggered. Key-value model does not provide formalization and visualization of the model, nor provide design tools. It does not incorporate hierarchy nor any sophisticated structure into model which is flat. The inference is supported usually by a very simple matching algorithm that does not allow for more sophisticated reasoning. However, it is very simple to implement.. Logic-based Models These approaches are widely used in context-aware systems. They enable automated inductive and deductive reasoning to be done on contextual information and due, to their strong formalization, allow for verification and validation of context models. There are several approaches that use logic to represent context. First order logic allows for an expressive description of context using boolean operators and existential and universal quantifiers [100, 129]. Fuzzy logic and probabilistic logic is used to handle uncertainty of the environment and to deal with the imperfections of the data [128]. Context lattices approach is used to represent low-level semantics on domain knowledge and sensor data, and to derive high-level semantics on human activities [151]. Description Logic [5] is usually used in combination with ontologies. It models concepts, roles and individuals, and their relationships. It also provides simple reasoning capabilities that resolves classification tasks [72]. An example of a system that uses first order logic to describe context can be found in [129]. The approach presented in that paper uses Prolog programming language for context representation. An example of a context description statement is presented below in pseudo code: #People(Room 2401, ">=",3) AND Application (PowerPoint, Running) => RoomActivity(20401,Presentation) #People(Room 2401, ">=",3) AND NOT EXSIST x Application(x, Running) => RoomActivity(20401,Meeting). Logic-based models provide strong formalization, though their flexibility might be limited. There is also lack of tools that provide visualization of models defined in logic languages. There exist a lot of programming languages and reasoners for expressing and processing knowledge encoded with logic based languages. However, dedicated reasoners are rarely available for mobile platforms.. Rules for Context Representation Rule-based systems have been in use for several decades in various branches of engineering. Hence, they have also been used in context-aware applications, both as a representation of models and as a support for reasoning [44, 54, 144]. A rule-based system consists of three main elements: knowledge base, fact base and inference engine. Knowledge base is considered as a set of rules, S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.

(33) 2.2. Modelling phase. 20. usually of a form of production rules: IF <conditions> THEN <action>. Fact base contains information used to check which rules conditions are satisfied. Inference engine implements mechanism that allow for processing of rules within a knowledge base. One of the most popular tools for context-aware applications that implements rule-based approach is Context Toolkit [48]. An example of a rule written in a Context Toolkit notation looks as follows: <Reference name="Off"> <Query name="lightOff"> (OR (EQUAL presence 0) (GREATER brightness brightnessThreshold)) </Query> <Outcome outAttribute="light">0</Outcome> <ServiceInput service="LightService" function="lightOff" /> </Reference>. The example rule can be read as follows: If there is no person in a room or brightness exceeds some threshold, then turn off the light in the room. Context Toolkit uses custom rule language and inference engine. However, there are several commonly used rule-based environments that provides advanced reasoning mechanisms and complex rule languages. Examples of such are: Clips6 , Jess7 , Drools8 . Although there were attempts to use these tools for contextaware applications [9, 54], they are still not popular in this area. Rules incorporate more powerful reasoning mechanisms than those available in key-value approach. They allow for assertions of new facts to knowledge base that can later be used as an input for other rules making the knowledge base more dynamic. They provide self-explanation mechanism that is crucial for implementing ineligibility of a system [45]. Rule-based systems provide more advanced methods for selecting rules that should be processed, improving efficiency of the system.. Object oriented models They play an important role in software engineering, e.g. UML diagrams are widely used for supporting software development, ERD diagrams are irreplaceable in designing relational databases schemes, and BPMN is used in process modelling. In context aware systems, Context Modelling Language (CML) [70] developed by Henricksen et al. is an example of such approach. CML is based on Object-Role Modelling language which was developed for conceptual modelling of databases. It provides a graphical representation of different classes and sources of context facts, relations between them and 6. See: http://clipsrules.sourceforge.net. See: http://herzberg.ca.sandia.gov. 8 See: http://www.jboss.org/drools. 7. S. Bobek. Methods for modeling self-adaptive mobile context-aware sytems.