Index of /rozprawy2/11011

Pełen tekst

(1)AGH University of Science and Technology Faculty of Computer Science, Electronics and Telecommunications Department of Telecommunications. Ph.D. Thesis Piotr Boryło. Provisioning of Energy-Aware Cloud Services Over Optical Networks Supervisors: Prof. dr hab. in˙z. Andrzej Jajszczyk Dr in˙z. Artur Lason´. Kraków 2015.

(2)

(3) To my wife, family and friends..

(4)

(5) Acknowledgements. First and foremost, I would like to express my gratitude towards my supervisor, Professor Andrzej Jajszczyk for his valuable advice, patience and motivation to present the results at leading telecommunication conferences and in journals. I have been very fortunate to have had the opportunity to derive a great deal from his experience and to be introduced to seminal researchers as his student. There are no words to express my gratitude to Artur Lasoń for invaluable help in both scientific and everyday life. Without his brilliant ideas, critical remarks and pertinent suggestions this dissertation would not have been made. I am deeply grateful for his readiness to help at any time and valuable advice. I have been fortunate to work with Andrzej Szymański who has inestimably contributed to most of my publications by providing valuable comments and constructively criticizing my ideas. I would also like to thank him for sharing and familiarizing me with simulation framework and good practises in performing simulation-based research. I also wish to thank all my workmates for being friendly and helpful. They have created the most enjoyable and pleasant working atmosphere. I am especially indebted to Jacek Rząsa and Piotr Chołda for supportive remarks and numerous discussions. Their experience and wisdom have significantly enriched my research. Last but not least, I would like to thank my family and friends. I am especially thankful to my wife Ewa for her patience and love. I also wish to thank my parents for their invaluable support and encouragement. All of them deserve my deepest appreciation..

(6)

(7) Abstract. This dissertation focuses on energy-aware cloud service provisioning in multitenant cloud architecture. The work presents an in-depth investigation of the option to reduce carbon dioxide emission by cloud infrastructures comprising data centers interconnected via optical networks. As a result, a solution is proposed in order to solve the problem of dynamic anycast routing in optical networks with energy-aware cloud services provisioning. The solution is composed of three mechanisms: anycast strategies, schemas for fitting the strategies to cloud services, and models of cooperation between IT infrastructure providers and network operators. These building blocks make it possible to respect requirements imposed by cloud paradigms and multi-tenant architecture. The performance of the proposed mechanisms and strategies is investigated using discrete-event computer simulations in two network topologies. A comprehensive study was carried out on all of the proposed mechanisms. An incremental approach to composing the complete solution was used. Such an approach has two advantages. First, it is possible to achieve a better understanding of the composing mechanisms and the complete solution. Second, the most prominent combination of building blocks is easier to determine. Simulation results show that the solution is able to meet the assumed requirements and successfully reduce carbon dioxide emissions without significant deterioration of network performance. The mechanisms are well placed in modern cloud architecture, making them easily deployed. Keywords: carbon footprint reduction, cloud services, optical networks, data centres, anycast strategies, fitting schemas, joint IT and network resource provisioning, and cooperation models.

(8)

(9) Streszczenie. Niniejsza rozprawa doktorska jest poświęcona świadczeniu usług w chmurze obliczeniowej z uwzględnieniem pochodzenia wykorzystywanej energii. W pracy zbadano możliwość redukcji emisji dwutlenku węgla przy założeniu, że chmura obliczeniowa jest złożeniem dwóch infrastruktur utrzymywanych odpowiednio przez operatora sieci optycznej oraz dostawcę zasobów obliczeniowych w centrach przetwarzania danych. W pracy przedstawiono rozwiązanie problemu dynamicznego rutingu w komutowanej sieci optycznej z jednoczesnym świadczeniem usług w centrach przetwarzania danych z uwzględnieniem pochodzenia wykorzystywanej energii. Proponowane rozwiązanie składa się z trzech mechanizmów: strategii kierowania ruchem typu jeden do jednego z wielu, schematów dopasowywania strategii do różnych usług oferowanych przez chmurę obliczeniową oraz modeli współpracy pomiędzy operatorem infrastruktury obliczeniowej i sieciowej. Poszczególne mechanizmy składowe pozwalają na spełnienie wymagań narzucanych przez architekturę chmury usytuowaną na styku działań niezależnych operatorów. Wydajność zaproponowanych mechanizmów i strategii została przebadana za pomocą symulacji komputerowej w dwóch topologiach sieciowych. Szczegółowym badaniom poddano wszystkie proponowane mechanizmy. Fakt zastosowania przyrostowego podejścia w tworzeniu kompletnego rozwiązania pozwolił lepiej zrozumieć zarówno każdy z mechanizmów składowych jak i całe rozwiązanie. Dodatkowo, dzięki takiemu podejściu mogła zostać skutecznie wyznaczona najbardziej wartościowa kombinacja mechanizmów składowych. Uzyskane wyniki pozwalają na stwierdzenie, że zaproponowane rozwiązanie spełnia narzucone wymagania i pozwala na redukcję emisji dwutlenku węgla bez istotnego pogorszenia wydajności sieci. Mechanizmy wchodzące w skład rozwiązania cechują się istotnym potencjałem wdrożeniowym z uwagi na fakt, że są dobrze umiejscowione w nowoczesnej architekturze chmury obliczeniowej..

(10) x. Streszczenie. Słowa kluczowe: redukcja emisji dwutlenku węgla, usługi w chmurze, sieci optyczne, centra przetwarzania danych, strategie typu jeden do jednego z wielu, schematy dopasowywania, jednoczesny przydział zasobów infrastruktury obliczeniowej i sieciowej, modele współpracy.

(11) Contents. Contents. xi. List of Figures. xv. List of Tables. xxi. Abbreviations. xxiii. List of symbols. xxv. I. Introduction and background. 1. 1 Introduction 1.1 Scope and thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Structure of the dissertation . . . . . . . . . . . . . . . . . . . . . . 2 Area of research 2.1 Cloud computing architecture 2.1.1 Optical networks . . . 2.1.2 Computing resources . 2.2 Energy-awareness . . . . . . . 2.3 Problem statement . . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. 3 5 6 8 11 11 14 16 19 20. 3 Related work 25 3.1 Network approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.2 Computing infrastructure approach . . . . . . . . . . . . . . . . . . 28.

(12) xii. Contents. 3.3. Cooperation between cloud providers and network operators . . . . 29. 4 Performance assessment 4.1 Statistical basis . . . . . . . . . . . . . . . . . . . 4.2 Simulation details . . . . . . . . . . . . . . . . . 4.2.1 Estimation of the transient phase length . 4.2.2 Validation of the batch length . . . . . . . 4.2.3 Data gathering . . . . . . . . . . . . . . . 4.3 Simulator environment . . . . . . . . . . . . . . . 4.3.1 Tools . . . . . . . . . . . . . . . . . . . . 4.3.2 Simulator architecture . . . . . . . . . . . 4.3.3 Request generation . . . . . . . . . . . . . 4.3.4 Routing and wavelength assignment . . . 4.3.5 Lightpath setup and teardown . . . . . . 4.3.6 Cloud request handling . . . . . . . . . . 4.4 Network architectures and simulation parameters. II. . . . . . . . . . . . . .. . . . . . . . . . . . . .. . . . . . . . . . . . . .. . . . . . . . . . . . . .. . . . . . . . . . . . . .. . . . . . . . . . . . . .. . . . . . . . . . . . . .. . . . . . . . . . . . . .. . . . . . . . . . . . . .. . . . . . . . . . . . . .. Anycast strategies for optical networks. 5 Introduction of energy-aware strategies 5.1 The single . . . . . . . . . . . . . . . . . 5.2 The random . . . . . . . . . . . . . . . . 5.3 The closest . . . . . . . . . . . . . . . . 5.4 The randomGreen . . . . . . . . . . . . 5.5 The closestGreen . . . . . . . . . . . . . 5.6 The closestGreenWithPenalty . . . . . .. 33 33 35 35 36 36 37 37 37 38 38 38 39 40. 47 . . . . . .. 49 50 51 51 52 53 54. 6 Assessment of energy-aware strategies 6.1 The penalty parameter value . . . . . . . . . . . . . . . . . . . . . 6.2 Energy-aware strategies . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Conclusions related to anycast strategies . . . . . . . . . . . . . . .. 57 57 63 73. III. 75. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. Fitting energy-aware strategies to cloud services. . . . . . .. 7 Fitting schemas introduction 77 7.1 The compound fitting schema . . . . . . . . . . . . . . . . . . . . . 78 7.2 Reference fitting schemas . . . . . . . . . . . . . . . . . . . . . . . 78 8 Assessment of fitting schemas 81 8.1 The penalty parameter value . . . . . . . . . . . . . . . . . . . . . 81.

(13) xiii. Contents. 8.2 8.3. Fitting schemas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Conclusions related to the fitting schemas . . . . . . . . . . . . . . 94. IV Cooperation between network operator and cloud provider 95 9 Introduction of cooperation 9.1 The overlay model . . . . 9.2 The augmented model . . 9.3 The peer model . . . . . .. models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 10 Assessment of cooperation models – fitting schemas pairs. 97 98 99 100 103. 11 Assessment of cooperation models 111 11.1 Input parameters values . . . . . . . . . . . . . . . . . . . . . . . . 112 11.2 Cooperation models . . . . . . . . . . . . . . . . . . . . . . . . . . 117 11.3 Conclusions related to the cooperation models . . . . . . . . . . . . 124. V. Finale. 125. 12 Conclusions 127 12.1 Achievements and contributions . . . . . . . . . . . . . . . . . . . . 130 Bibliography. 132.

(14)

(15) List of Figures. 2.1. Considered architecture (based on [81]), VTN: Virtual Tenant Network, OVSDB: Open vSwitch DataBase Protocol, LISP: Locator/Identifier Separation Protocol, BGP: Border Gateway Protocol, PCEP: Path Computation Element Communication Protocol, SNMP: Simple Network Management Protocol . . . . . . . . . . . 13. 2.2. Types of request flows . . . . . . . . . . . . . . . . . . . . . . . . . 22. 4.1. The topology of the NSF network . . . . . . . . . . . . . . . . . . . 40. 4.2. The topology of the ItalyNet. 6.1. Total blocking probability of both traffic types in the NSF network with VgDC ∈ {4, 11} . . . . . . . . . . . . . . . . . . . . . . . . . . 59. 6.2. Total blocking probability of both traffic types in the NSF network with VgDC ∈ {9, 11} . . . . . . . . . . . . . . . . . . . . . . . . . . 59. 6.3. Total blocking probability of both traffic types in the ItalyNet with VgDC ∈ {7, 15, 20} . . . . . . . . . . . . . . . . . . . . . . . . . . . 60. 6.4. Total blocking probability of both traffic types and brown kilowatts needed to handle 1 Gb/s of DC requests obtained using the closestGreenWithPenalty strategy in the NSF network with VgDC ∈ {4, 11} and an anycast traffic scaling factor equal to 1.79 . 60. 6.5. Total blocking probability of both traffic types and brown kilowatts needed to handle 1 Gb/s of DC requests obtained using the closestGreenWithPenalty strategy in the NSF network with VgDC ∈ {9, 11} and an anycast traffic scaling factor equal to 1.79 . 61. . . . . . . . . . . . . . . . . . . . . . 41.

(16) xvi. List of Figures. 6.6. 6.7 6.8 6.9 6.10 6.11 6.12 6.13 6.14 6.15 6.16 6.17. 6.18. 6.19. 6.20. 8.1. Total blocking probability of both traffic types and brown kilowatts needed to handle 1 Gb/s of DC requests obtained using the closestGreenWithPenalty strategy in the ItalyNet with VgDC ∈ {7, 15, 20} and an anycast traffic scaling factor equal to 1.45 . . . . . . . . . . Brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with VgDC ∈ {4, 11} . . . . . . . . . . . . . . . . . . . Brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with VgDC ∈ {9, 11} . . . . . . . . . . . . . . . . . . . Brown kilowatts needed to handle 1 Gb/s of DC requests in the ItalyNet with VgDC ∈ {7, 15, 20} . . . . . . . . . . . . . . . . . . . . Total blocking probability of both traffic types in the NSF network with VgDC ∈ {4, 11} . . . . . . . . . . . . . . . . . . . . . . . . . . Total blocking probability of both traffic types in the NSF network with VgDC ∈ {9, 11} . . . . . . . . . . . . . . . . . . . . . . . . . . Total blocking probability of both traffic types in the ItalyNet with VgDC ∈ {7, 15, 20} . . . . . . . . . . . . . . . . . . . . . . . . . . . Blocking probability of DC traffic requests in the NSF network with VgDC ∈ {4, 11} . . . . . . . . . . . . . . . . . . . . . . . . . . Blocking probability of background traffic requests in the NSF network with VgDC ∈ {4, 11} . . . . . . . . . . . . . . . . . . . . . Blocking probability of DC traffic requests in the NSF network with VgDC ∈ {9, 11} . . . . . . . . . . . . . . . . . . . . . . . . . . Blocking probability of background traffic requests in the NSF network with VgDC ∈ {9, 11} . . . . . . . . . . . . . . . . . . . . . Brown kilowatts needed to handle 1 Gb/s of DC requests as a function of VgDC set in the NSF network using the closestGreenWithPenalty strategy . . . . . . . . . . . . . . . . . . . . . . . . . . Total blocking probability of both traffic types as a function of VgDC set in the NSF network using the closestGreenWithPenalty strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Brown kilowatts needed to handle 1 Gb/s of DC requests as a function of VgDC set in the ItalyNet using the closestGreenWithPenalty strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Total blocking probability of both traffic types as a function of VgDC set in the ItalyNet using the closestGreenWithPenalty strategy. 61 65 65 66 66 67 67 68 69 69 70. 71. 71. 72 72. Total blocking probability of both traffic types and brown kilowatts needed to handle 1 Gb/s of DC requests obtained using the compound fitting schema in the NSF network with VgDC ∈ {2, 9}, service type ratio 1:5:25, assumed SaaS energy consumption equal to 2.7 kW/(Gb/s) and anycast traffic scaling factor equal to 1.5 . . 83.

(17) List of Figures. 8.2. 8.3. 8.4. 8.5. 8.6. 8.7 8.8 8.9 8.10. 8.11 8.12. 8.13 8.14. 8.15. Total blocking probability of both traffic types and brown kilowatts needed to handle 1 Gb/s of DC requests obtained using the compound fitting schema in the NSF network with VgDC ∈ {9, 11}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 1 kW/(Gb/s) and anycast traffic scaling factor equal to 1.5 . . . Total blocking probability of both traffic types and brown kilowatts needed to handle 1 Gb/s of DC requests obtained using the compound fitting schema in the ItalyNet with VgDC ∈ {7, 13, 20}, service type ratio 1:1:1, assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) and anycast traffic scaling factor equal to 1.35 . Brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with VgDC ∈ {2, 9}, service type ratio 1:5:25 and assumed SaaS energy consumption equal to 2.7 kW/(Gb/s) . . . . Brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with VgDC ∈ {9, 11}, service type ratio 1:2:4 and assumed SaaS energy consumption equal to 1 kW/(Gb/s) . . . . . Brown kilowatts needed to handle 1 Gb/s of DC requests in the ItalyNet with VgDC ∈ {7, 13, 20}, service type ratio 1:1:1 and assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) . . . . . Total blocking probability of all traffic types in the NSF network with VgDC ∈ {2, 9} and service type ratio 1:5:25 . . . . . . . . . . . Total blocking probability of all traffic types in the NSF network with VgDC ∈ {9, 11} and service type ratio 1:2:4 . . . . . . . . . . . Total blocking probability of all traffic types in the ItalyNet with VgDC ∈ {7, 13, 20} and service type ratio 1:1:1 . . . . . . . . . . . . Brown kilowatts needed to handle 1 Gb/s of DC requests obtained using the compound fitting schema in the NSF network with VgDC ∈ {2, 11} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Total blocking probability of all traffic types obtained using the compound fitting schema in the NSF network with VgDC ∈ {2, 11} . Brown kilowatts needed to handle 1 Gb/s of DC requests obtained using the compound fitting schema in the ItalyNet with VgDC ∈ {7, 13, 20} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Total blocking probability of all traffic types obtained using the compound fitting schema in the ItalyNet with VgDC ∈ {7, 13, 20} . Brown kilowatts needed to handle 1 Gb/s of DC requests obtained using the compound fitting schema in the ItalyNet with VgDC ∈ {7, 15, 20} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Total blocking probability of all traffic types obtained using the compound fitting schema in the ItalyNet with VgDC ∈ {7, 15, 20} .. xvii. 84. 84. 86. 86. 87 87 88 88. 89 90. 90 91. 91 92.

(18) xviii. List of Figures. 10.1 Brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with VgDC ∈ {9, 11}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) and augmented cooperation model . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 10.2 Brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with VgDC ∈ {9, 11}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) and peer cooperation model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 10.3 Total blocking probability of all traffic types in the NSF network with VgDC ∈ {9, 11}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) and augmented cooperation model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 10.4 Total blocking probability of all traffic types in the NSF network with VgDC ∈ {9, 11}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) and peer cooperation model 106 10.5 Brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with VgDC ∈ {2, 9}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 10 kW/(Gb/s) and augmented cooperation model . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 10.6 Brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with VgDC ∈ {2, 9}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 10 kW/(Gb/s) and peer cooperation model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 10.7 Total blocking probability of all traffic types in the NSF network with VgDC ∈ {2, 9}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 10 kW/(Gb/s) and augmented cooperation model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 10.8 Total blocking probability of all traffic types in the NSF network with VgDC ∈ {2, 9}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 10 kW/(Gb/s) and peer cooperation model . 108 10.9 Brown kilowatts needed to handle 1 Gb/s of DC requests in the ItalyNet with VgDC ∈ {7, 15, 20}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) and augmented cooperation model . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 10.10Brown kilowatts needed to handle 1 Gb/s of DC requests in the ItalyNet with VgDC ∈ {7, 15, 20}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) and peer cooperation model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109.

(19) List of Figures. xix. 10.11Total blocking probability of all traffic types in the ItalyNet with VgDC ∈ {7, 15, 20}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) and augmented cooperation model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 10.12Total blocking probability of all traffic types in the ItalyNet with VgDC ∈ {7, 15, 20}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) and peer cooperation model 110 11.1 Total blocking probability of all traffic types and brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with VgDC ∈ {9, 11}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 5.4 kW/(Gb/s), anycast traffic scaling factor equal to 1.75, penalty = 1.2 and preferenceMode set to mostOccupied (upper row) and leastOccupied (lower row) . . . . . 11.2 Total blocking probability of all traffic types and brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with VgDC ∈ {2, 9}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 10 kW/(Gb/s), anycast traffic scaling factor equal to 1.75, penalty = 1.1 and preferenceMode set to mostOccupied (upper row) and leastOccupied (lower row) . . . . . 11.3 Total blocking probability of all traffic types and brown kilowatts needed to handle 1 Gb/s of DC requests in the ItalyNet with VgDC ∈ {7, 15, 20}, service type ratio 1:2:4, assumed SaaS energy consumption equal to 5.4 kW/(Gb/s), anycast traffic scaling factor equal to 1.35, penalty = 1.2 and preferenceMode set to mostOccupied (upper row) and leastOccupied (lower row) . . . . . 11.4 Brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with VgDC ∈ {9, 11}, service type ratio 1:2:4 and assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) . . . . 11.5 Brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with VgDC ∈ {2, 9}, service type ratio 1:2:4 and assumed SaaS energy consumption equal to 10 kW/(Gb/s) . . . . 11.6 Brown kilowatts needed to handle 1 Gb/s of DC requests in the ItalyNet with VgDC ∈ {7, 15, 20}, service type ratio 1:2:4 and assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) . . . . . 11.7 Total blocking probability of all traffic types in the NSF network with VgDC ∈ {9, 11}, service type ratio 1:2:4 and assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) . . . . . . . . . . . . 11.8 Total blocking probability of all traffic types in the NSF network with VgDC ∈ {2, 9}, service type ratio 1:2:4 and assumed SaaS energy consumption equal to 10 kW/(Gb/s) . . . . . . . . . . . . .. 114. 115. 116. 119. 119. 120. 120. 121.

(20) xx. List of Figures. 11.9 Total blocking probability of all traffic types in the ItalyNet with VgDC ∈ {7, 15, 20}, service type ratio 1:2:4 and assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) . . . . . . . . . . . . 121 11.10Background traffic blocking probability in the NSF network with VgDC ∈ {9, 11}, service type ratio 1:2:4 and assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) . . . . . . . . . . . . . . . . . 123 11.11DC traffic blocking probability in the NSF network with VgDC ∈ {9, 11}, service type ratio 1:2:4 and assumed SaaS energy consumption equal to 5.4 kW/(Gb/s) . . . . . . . . . . . . . . . . . . . . . . 123.

(21) List of Tables. 4.1 4.2 4.3. Topology parameters for investigated networks . . . . . . . . . . . 41 Relationships between simulation parameters . . . . . . . . . . . . 44 IT resources in DCs . . . . . . . . . . . . . . . . . . . . . . . . . . 45. 6.1. Determined penalty parameter values . . . . . . . . . . . . . . . . . 62. 7.1. Summary of fitting schemas . . . . . . . . . . . . . . . . . . . . . . 79. 8.1. Summary results for different simulation scenarios obtained using the compound fitting schema . . . . . . . . . . . . . . . . . . . . . 93.

(22)

(23) Abbreviations. ARWA BGP CRUD DC DSS EON GMPLS ICT IT LISP LP MPLS NSF OVSDB PaaS PCEP RWA SaaS SDI SDN SNMP StaaS VM VTN WAN WCC WDM. Anycast Routing and Wavelength Assignment Border Gateway Protocol Create, Read, Update, Delete Data Center Decision Support Systems Elastic Optical Networks Generalized Multiprotocol Label Switching Information and Communication Technology Information Technology Locator/Identifier Separation Protocol Lightpath Multiprotocol Label Switching Network Science Foundation Open vSwitch DataBase Protocol Processing as a Service Path Computation Element Protocol Routing and Wavelength Assignment Software as a Service Software Defined Infrastructure Software Defined Networking Simple Network Management Protocol Storage as a Service Virtual Machine Virtual Tenant Network Wide Area Network Wavelength Continuity Constraint Wavelength Division Multiplexing.

(24) xxiv. Abbreviations.

(25) List of symbols. arwa(s, D). The operation of invoking the anycast strategy fitted to the cloud service associated with current LR from source s to set of possible destinations D. arwa. The anycast strategy invoked by the arwa(s, D) operation. cd. The total amount of resources in DC d ∈ VDC. dcth. The threshold utilized by the peer cooperation model, expressed as a fraction of the total DC capacity (cd ). dreal. The reachable destination selected by the SDN controller assuming no preference of any DC. e. The link from the set of physical network links (e ∈ E). ew. The wavelength w on link e. h. The number of lags tested. ht. The mean value of request holding time. iat. The mean value of request interarrival time. leastOccupied The mode that denotes the preferred DCs with available resources less than the assumed threshold mostOccupied The mode that denotes the preferred DCs with available resources more than the assumed threshold netth. The threshold which expresses the increase in the average lightpath length acceptable by the network operator.

(26) xxvi. List of symbols. p. The link forming a part of the path P ; the sequence of links p represents adjacent links comprising path P. pw. The wavelength w on link p. preferenceMode The preferenceMode may be assigned one of two values, mostOccupied and leastOccupied, in order to express cloud operator preferences in the peer cooperation model rd. The amount of resources currently available in DC d ∈ VDC. s. The standard deviation of a sample. tα/2. The (1 − α/2) quantile of the t-Student distribution with N − 1 degrees of freedom. x ¯. The sample mean. x. A sample. zα/2. The (1 − α/2) quantile of the normal distribution. CDC. The set containing information on the total amount of IT resources in each DC (cd for all d ∈ VDC ). CR. The cloud service request. D. The subset of network nodes associated with the DCs (D ⊆ VDC ). Dg. The subset of network nodes associated with the green DCs (Dg ⊆ VgDC ). Dg. The subset of network nodes associated with the brown DCs (Db ⊆ VbDC ). E. The set of physical network links. Eproc. The energy consumption of a DC handling a PaaS task. G(V, E). The graph denoting the physical network. H0. The null hypothesis. Ha. The alternative hypothesis. LR. The lightpath request. LRr. The amount of IT resources required to serve a cloud service request associated with the particular LR.

(27) List of symbols. xxvii. LAC (s, d). Lightpath availability check between the source node s and the destination node d. M. The number of samples. N. The number of observations in a sample. Ps. The power consumption of the server. P. The network path between selected network nodes. QLB. The Ljung-Box test statistics. RDC. The set containing information on IT resources currently available in each DC (rd for all d ∈ VDC ). Tproc. The processing time (in hours). V. The set of physical network nodes. VDC. The set of network nodes associated with the DCs. VgDC. The set of network nodes associated with the green DCs. VbDC. The set of network nodes associated with the brown DCs. VC. The set of client nodes which are not associated with any DC. VrDC. The subset of network nodes associated with the DCs that have sufficient resources to handle particular requests. VrgDC. The subset of network nodes associated with the green DCs that have sufficient resources to handle particular requests. VrbDC. The subset of network nodes associated with the brown DCs that have enough resources to handle particular requests. 1−α. The confidence coefficient. α. The significance level. χ21−α,h. The α quantile of the chi-square distribution with h levels of freedom. µ. The population mean. ρˆ. The sample autocorrelation at lag j. σ. The population standard deviation.

(28)

(29) Part I. Introduction and background.

(30)

(31) 1. Introduction. The popularity of cloud computing is on the rise. Both industry and society are increasingly using modern cloud services. The most common domestic examples include sharing documents, photos, videos and forms with friends and colleagues. Video on demand is another well-known service. Other examples of cloud services are related to the business nature of cloud computing. The most popular are infrastructure provisioning at any scale, starting from single page web applications through larger multi-tier deployments to major-scale computing tasks. These services reduce operational costs and entry barriers, as well as eliminate the need to maintain IT infrastructures. It is also common for companies to host their internal services in the cloud; example services include email, spreadsheets, and questionnaires. Additionally, if employees use their personal computers as terminals connected to virtual machines, it is reasonable to host the machines in the cloud. Employees are then independent from underlying hardware and able to work more flexibly, while the negative impact of failures is negligible in most cases. Finally, the majority of critical data is distributed and stored electronically, which needs to be backed up. Storage infrastructure within the cloud is a convenient environment for such backups. While considering the provisioning of cloud services, we must not underestimate the importance of network infrastructure which is necessary to ensure access to IT resources. Due to the global nature of modern cloud computing, a great deal of effort is required to investigate networks within data centers as well as accessing wide area networks. The network becomes an integral part of the cloud, and therefore a convergence of IT and telecommunication infrastructures is needed to effectively provide services to end users. However, cloud and network infrastructures are usually administered by different entities, which further complicates the problem. Optical data transmission is usually indicated as the most appropriate for providing cloud computing services, although all optical.

(32) 4. 1. Introduction. networks are appropriate due to their high throughput. Additionally, thanks to the Software Defined Networking (SDN) concept, where the control and data planes are clearly separated, it is possible to for the centralized network controller and cloud orchestration software to work together. Cloud computing infrastructure is considered a part of the Information and Communication Technologies (ICT) sector. The major expansion of the ICT in recent years is largely due to the growing popularity of cloud computing. In turn, the ICT sector has an increasing impact on the natural environment. On the one hand, this may have a positive effect by reducing the need to travel, since many people are now able to work remotely and communicate via teleconferences. However, the ICT sector itself consumes vast amounts of energy to power the equipment necessary to run modern services, therefore its impact on the natural environment is becoming increasingly pronounced. According to [115] the ICT sector consumed 8% of global energy in 2008, and this share is expected to grow to 20% in 2020. Additionally, the sector produces about 2-4% of global CO2 [106]. The increasing energy consumption by the ICT sector may also be partially attributed to the growing popularity of cloud computing services. Such services require vast computing power, which in turn results in extremely high power demands of data centers (DCs). As such, DCs are important contributors to overall energy consumption, therefore greening DCs may provide a noticeable improvement on the negative impact of the ICT sector on the environment. This is a powerful motivation for the implementation of the concept of greening of the Internet introduced in [48]. The general concept of improving energy efficiency and decreasing the negative impact on the natural environment is a hot topic and concerns almost every aspect of our lives. There are numerous reasons for introducing green mechanisms into IT and network infrastructures. The first and perhaps the most obvious is the need to reduce the negative impact of the infrastructure on the natural environment. Following the idea of greening, the cloud also affects public relations and marketing. A cloud provider advertising itself as being environmentally-friendly may have a better reputation and as such be more attractive to certain customers. However, economic reasons are the most powerful drivers. The vast power requirements of the equipment directly translate to high operational costs related to electricity. On the one hand, costs may be reduced by improving energy efficiency, while on the other, tax regulations introduced in numerous countries (e.g. the UK, China) and being considered in several others mean that energy obtained from renewable resources is cheaper. Thus, using renewable energy may also reduce operational costs [2]. At the same time, network operators attract the attention of cloud providers by delivering green mechanisms and being willing to cooperate in the field of energy efficiency. This may be significant, since revenues from simple data transmission are decreasing. Thus, telecommunication operators must take.

(33) 1.1 Scope and thesis. 5. advantage of emerging technologies, e.g. SDN, to provide more profitable services, including green services. Driven by these factors, this dissertation focuses on energy-aware cloud service provisioning. The infrastructure consists of DCs interconnected via optical networks. Data centers are associated with selected network nodes. Selected DCs are powered using energy from renewable sources, while the others use energy obtained from conventional sources. Energy obtained from renewable sources is assumed not to contribute to carbon dioxide emissions, while energy obtained from other sources contribute proportionally to its amount. The underlying network is assumed to be based on optical wavelength division multiplexing. The physical network topology is given, and does not change over time. There are no wavelength converters installed in the infrastructure, and no rerouting of existing lightpaths is possible. As one of the paradigms of cloud computing is to offer services in an on-demand fashion, incoming requests are unpredictable and the infrastructure must be studied under the dynamic traffic scenario. Therefore, network and IT resources (assumed to be controlled by separate entities) must be provisioned for each incoming request and current state of the resource occupancy. Another paradigm of cloud computing is to offer the same services in different DCs simultaneously. Thus, it is possible to choose one of many possible DCs to provide the requested service; a corresponding routing schema is known as anycast [33]. Anycast may take into account network occupancy, the type of available energy in specific sources, and IT resource availability in each DC.. 1.1. Scope and thesis. This dissertation proposes a solution for the problem of on-demand energy-aware cloud service provisioning throughout optical networks. The solution is aware of requirements constrained by cloud computing paradigms and architecture. First of all, the solution must be able to choose the destination DC in an energyaware manner. Additionally, the level of preference should be customizable in order to meet the infrastructure conditions. The energy-aware anycast strategies proposed in this dissertation meet the requirements. Secondly, the proposed solution must be able to adjust to different types of cloud services in order to effectively utilize energy obtained from renewable sources. Three types of cloud services are introduced and considered. To meet this requirement, we propose a schema for fitting these anycast strategies to cloud service types. The aim of the fitting schema is to reflect cloud service properties. The results show that the schema brings significant improvements. Finally, the solution must be aware of the multi-tenant cloud architecture where IT and network infrastructure are administrated by separate entities. Different levels of cooperation should.

(34) 6. 1. Introduction. be investigated, and interfaces for exchanging data between the entities should be clearly defined in order to increase the appeal of the solution for real life application. Therefore, I proposed cooperation models between network provider and cloud operator, and investigated models of cooperation between separate entities. Additionally, I assessed different values of input parameters for one of the cooperation models. The improvements encourage the use of these techniques while considering cloud architectures with limited network and IT resources and selected DCs powered from renewable energy sources. The solution was assessed using extended simulations, and two different physical network topologies were investigated. For each topology, the dissertation investigates different traffic loads, various placements of DCs powered from renewable energy sources, as well as changing input parameters of the solution. The results are assessed with regard to carbon dioxide emissions and the impact on the architecture performance. The result shows the efficiency of the solution in comparison to the reference approaches. It reduces the carbon footprint without significantly degrading network performance. Therefore, the following thesis of this dissertation has been proposed and proved: It is possible to reduce carbon dioxide emissions of a cloud infrastructure comprising data centers interconnected via optical networks using energy-aware resource provisioning strategies. The problem of on-demand energy-aware cloud service provisioning throughout optical networks is an emerging topic for network operators and cloud providers. The proposed solution is carefully placed in the cloud architecture assuming the coexistence of network and IT infrastructures. Therefore, the solution may be considered for use in real-life situations. Additionally, the simplicity of the introduced mechanism means that organizations may benefit from a reduction of the carbon footprint without an increase in complexity.. 1.2. Publications. Some of the achievements presented in this dissertation, as well as general considerations related to the topic, have been published in three journal articles and two conference papers. The list of relevant publications is as follows: [14] P. Borylo, A. Lason, J. Rzasa, A. Szymanski, and A. Jajszczyk. Usługi w chmurze korzystające z sieci optycznych o minimalnym zużyciu energii. Przegląd Telekomunikacyjny, Wiadomości Telekomunikacyjne, 86(10):1323– 1330, 2013..

(35) 1.2 Publications. 7. [15] P. Borylo, A. Lason, J. Rzasa, A. Szymanski, and A. Jajszczyk. Anycast Routing for Carbon Footprint Reduction in WDM Hybrid Power Networks with Data Centers. In Proc. IEEE International Conference on Communications (ICC 2014), pages 3720–3726, Sydney, Australia, June 2014. [16] P. Borylo, A. Lason, J. Rzasa, A. Szymanski, and A. Jajszczyk. Assessment of green anycast strategies in hybrid power networks having regular topologies. Przegląd Telekomunikacyjny, Wiadomości Telekomunikacyjne, 83(8-9):748– 754, 2014. [17] P. Borylo, A. Lason, J. Rzasa, A. Szymanski, and A. Jajszczyk. Fitting Green Anycast Strategies to Cloud Services in WDM Hybrid Power Networks. In Proc. IEEE Global Communications Conference (GLOBECOM 2014), pages 2633–2639, Austin, TX, USA, Dec. 2014. [18] P. Borylo, A. Lason, J. Rzasa, A. Szymanski, and A. Jajszczyk. Green Cloud Provisioning Throughout Cooperation of a WDM Wide Area Network and a Hybrid Power IT Infrastructure. Springer Journal of Grid Computing (minor revision, under 2nd review), 1(1):1–34, Nov. 2015. All the papers underwent a thorough review process before their final publication. [14] is a survey presenting the concept of cloud computing services provided by optical networks. A taxonomy of issues associated with delivering distributed services in an energy-efficient way was proposed along with an incentive for minimizing energy consumption and reducing the carbon footprint of the ICT sector. The main contribution of [15] consists of three anycast routing strategies aimed at reducing greenhouse gas emissions by processing energy-intensive requests in data centers powered from renewable energy sources. The proposed strategies were compared against reference approaches with respect to CO2 emission and their impact on network performance. It is shown that they are able to significantly reduce greenhouse gas emissions and, simultaneously, improve network performance. All studies in this paper were performed under the simplification that a single example service is delivered by a data centers able to handle an unlimited number of requests. [16] provides a comprehensive assessment of both well-known anycast strategies and those proposed in [15]. The assessment was performed in regular network topologies, as it is assumed that this creates an opportunity to draw more general and valuable conclusions and to fully understand novel strategies. The knowledge acquired improves further result analysis. The presented results concerned the impact of the strategies on CO2 emission and network performance. The.

(36) 8. 1. Introduction. aforementioned assumption about a single service being offered and an unlimited amount of resources in each data center remains valid. In [17] three types of cloud services are introduced and a schema for fitting anycast strategies introduced in [15] to those services is proposed. The fitting is performed based on network resource requirements and energy consumption of services, as well as the properties of anycast strategies. The solution is compared with cases when the same strategy is applied to each cloud service. It is shown that the proposed schema is able to significantly reduce greenhouse gas emissions without significant deterioration of network performance. Finally, [18] focuses on provisioning cloud services through the cooperation of the wide area all optical networks interconnecting the data centers equipped with hybrid power IT infrastructure. It is assumed that the network is controlled by a centralized controller consistent with the idea of the Software Defined Networking. At the same time, it is assumed that cloud orchestration software controls the cloud infrastructure and operates it in an automated manner. The aforementioned assumption about unlimited capacity of IT infrastructure is no longer valid in this thesis. Models of cooperation between network operators and cloud service providers are proposed and analyzed in terms of impact on cloud performance and reduction of the carbon footprint. Carbon dioxide emissions can be reduced significantly by joining cooperation models with green anycast strategies and fitting schema proposed in [15] and [17], respectively. Efficient usage of green energy does not deteriorate cloud performance.. 1.3. Structure of the dissertation. This dissertation is divided into five parts. The first part provides a theoretical background for the research conducted in the dissertation. Chapter 1 is a general introduction which states the thesis. Chapter 2 introduces cloud computing architecture, describes the energy-aware approach, and provides a formal problem statement. Chapter 3 presents work related to green cloud computing provisioning issues. The most important research is introduced in brief in order to reflect the dominant research areas. Chapter 4 deals with performance assessment tools and research methodology, and presents the simulator environment. The second part presents the results of PhD candidate’s research in the field of anycast routing strategies for all optical networks. The energy-aware strategies proposed mean the solution is able to elastically balance the preferences of the chosen data centers. Chapter 5 describes and references the proposed anycast strategies. Simulation results validating the proposed mechanisms are provided in Chapter 6 with regard to the likelihood of network blocking and the carbon footprint. The third part (Chapters 7 – 8) focuses on fitting anycast strategies to the.

(37) 1.3 Structure of the dissertation. 9. cloud service type. Chapter 7 introduces the compound fitting schema and three reference schemas. Results provided in Chapter 8 show that fitting anycast strategies to the properties of cloud service types may result in a reduction of the carbon footprint without significantly degrading network performance. Therefore, the compound fitting schema proposed in this dissertation allow the solution to effectively utilize energy obtained from renewable sources by adjusting to different service types. The fourth part investigates the problem of cooperation between cloud providers and network operators. Three models of cooperation are introduced in Chapter 9. Simulation results provided in Chapter 10 assess fitting schemas assuming different cooperation models. As a result, the best fitting schema is found for each cooperation model. These pairs are compared in detail in Chapter 11 with regard to the likelihood of network blocking and carbon footprint reduction. The cooperation models make the solution applicable in multi-tenant architecture where the IT and network are administrated by separate entities. Finally, Chapter 12 concludes the dissertation and summarizes the research. Some of the achievements presented in this dissertation were separately funded by the NCBR and CHIST-ERA ERA-Net SwiTching And tRansmission project or by the Dean under Grants No. 15.11.230.134 or 15.11.230.196..

(38)

(39) 2. Area of research. The goal of this Chapter is to present background research into energy-aware cloud service provisioning through optical networks. A detailed description of cloud computing architecture is provided in Section 2.1, including terminology and relationships between optical networks and computing infrastructure (see Sections 2.1.1 and 2.1.2, respectively). The description covers issues related to the operation and control of optical networks as well as the orchestration and abstraction of computing infrastructure. Section 2.2 outlines the incentives for using the energy-aware approach. Finally, the problem is stated formally in Section 2.3.. 2.1. Cloud computing architecture. One of the most descriptive definitions of cloud computing was provided in [39]: A large-scale distributed computing paradigm that is driven by economies of scale, in which a pool of abstracted, virtualized, dynamicallyscalable, managed computing power, storage, platforms, and services are delivered on demand to external customers over the Internet. One of the main paradigms of cloud computing is to offer numerous different services in an on-demand fashion. This is why incoming requests concerning different services are unpredictable, and cloud architecture must be investigated under a dynamic traffic scenario [33]. It is necessary to conduct CRUD (create, read, update, delete) of resources in seconds. Software automating the provisioning and operation of a cloud is needed in order to meet this requirement. Numerous cloud orchestration frameworks are available under open source as well as commercial licenses. A common purpose of such frameworks is to provide an abstraction of IT resources available in the cloud and to CRUD these resources in an automated.

(40) 12. 2. Area of research. manner. Example solutions include EC2 [4], OpenStack [80], OpenNebula [79], and CloudStack [29]. Good scalability, independence from underlying hardware, versatility, modular architecture and a user friendly interface are some of the factors that need to be considered when choosing a specific product. Network resources also need to be provided in an on-demand fashion. The concept of SDN is perfectly suited to the requirements, since in the SDN architecture the control plane is separate from the switching plane. Such a control plane is implemented in software, denoted as an SDN controller and deployed separately from network devices. Traffic is abstracted using the concept of flows, which is perfectly suited to wavelength switching in WDM networks. Each new flow is recognized first, then routed and handled, and finally torn down. The controller communicates with hardware (OpenFlow enabled devices) and virtual devices through the OpenFlow Protocol in order to deliver forwarding rules [76]. Additionally, the SDN controller may be able to communicate through Northbound API with the aforementioned cloud orchestration software [54] for dynamic and coordinated resource provisioning. In [110] the authors investigate SDN management requirements and challenges. Translation of high-level rules into network configuration is indicated as the one of the most important issues. Therefore, cooperation between cloud operators and network providers can take advantage of such translation. High-level requests to provision cloud services through optical networks are translated into particular configuration commands. In [53] the main components of SDN are provided, as are SDN-related taxonomies. Additionally, the authors propose a taxonomy that classifies research and sheds a light on future research directions. Northbound API is mentioned as a significant opportunity for cloud orchestration software to manage network services and virtualized network resources. Simultaneously, cooperation between SDN controllers and cloud orchestration software is noted as the most important application for SDN. Examples of SDN controllers include OpenDayLight [81], Floodlight [38], Beacon [94], and Big Cloud Fabric Controller [11]. Relationships in this architecture are presented in Figure 2.1. Another important paradigm of cloud computing is to offer the same services in different DCs simultaneously, in spite of its on-demand manner. Thus, it is possible to choose one of many possible DCs to provide the requested service [33]. This assumption is realistic, since numerous algorithms and solutions for data replication have been proposed, e.g. [13] or [65]. A corresponding routing schema is known as anycast and is applied to each cloud service request. Therefore, in order to handle anycast requests, an additional decision regarding the selected destination needs to be made based on numerous factors, including availability of network resources, DC occupancy, and availability of green energy availability. Evaluation of anycast strategies is one of the main topics of this dissertation..

(41) 2.1 Cloud computing architecture. 13. Fig. 2.1: Considered architecture (based on [81]), VTN: Virtual Tenant Network, OVSDB: Open vSwitch DataBase Protocol, LISP: Locator/Identifier Separation Protocol, BGP: Border Gateway Protocol, PCEP: Path Computation Element Communication Protocol, SNMP: Simple Network Management Protocol. When considering cloud architecture components, it must be stressed that the SDN controller and cloud orchestration software perform different functions: the former provides network resources, while the latter provides IT resources in DCs. Therefore, requests for cloud services must be processed by both entities in order to provision both types of resources. The cooperation of a cloud with network infrastructure requires corresponding integration of cloud orchestration with network control. Cooperation models between these entities are one of the topics of this work. The issue of integration of network infrastructure with IT resources has been described as important in numerous papers published in renowned journals and magazines. For example, the authors in [54] provide a detailed definition of Software Defined Networking (SDN) and its interfaces as well as a list of its key attributes. One of the major features noted in the paper is the ability of SDN controllers to cooperate with cloud orchestration software in order to quickly provision cloud applications and operate them in an automated manner. This allows networks to support virtualization and migration of virtual machines, and to evolve towards application-oriented networks. An additional effort is.

(42) 14. 2. Area of research. needed to bring these features to Wide Area Networks (WANs). [1] presents the requirements, design alternatives and potential management architectures for Wide Area SDN (WA-SDN). Again, communication between SDN controllers with cloud applications is noted as one of the most important use cases. In a wider context, cloud orchestration software may need to operate on IT infrastructures of more than one operator. Numerous middleware solutions aimed at using multiple infrastructures cooperating under different categories are available, e.g. [112]. exhaustive survey on existing solutions is provided in [84]. The issue concerns more than just scientific papers. For example, B4 is an inter-data center private WAN connecting 12 Google sites (data centers) across the globe [52]. B4 WAN, and the networks within the sites, are controlled by a set of SDN controllers. They all exchange information in order to fully utilize bandwidth available in the WAN and to control applications and site networks. However, only the largest cloud providers are able to build their own private WANs, while others need to cooperate with various network operators to improve service quality and to reduce costs. This problem has been noted by equipment vendors. For example, the concept of the Seamless Virtual Cloud presented by Cisco at numerous conferences (e.g. IEEE Network Operations and Management Symposium 2014, Cisco Live - Milan 2014) assumes that IT resources and WAN infrastructures are seamlessly bounded together and are available for tenants in an on-demand fashion. The framework provides an abstraction of relevant networking technologies in a WAN and in data centers. To sum up, applicationdriven wide area networks become a part of the cloud instead of simply being a way of accessing cloud applications and features. One important issue of the introduced cloud architecture is related to the interpretation of cloud request. The statement that a cloud customer generates a request does not literally mean that a single person generates the request. An aggregation layer is needed in order to group requests originating at the same node and concerning the same type of service. Such an aggregation is reasonable as reservation of personal wavelength in Wide Area Network for every single customer request is nonrealistic and results in inefficient spectrum utilization and unacceptable signalling overhead. For example, the whole traffic related to the same type of cloud service can be aggregated for the office, campus, or conference venue. However, the issue of aggregation is a separate and complex problem which should be investigated separately, and it is therefore out of the scope of this dissertation.. 2.1.1. Optical networks. The underlying wide area network is assumed to be a dynamic all-optical WDM network, since optical data transmission is usually indicated as the best way of.

(43) 2.1 Cloud computing architecture. 15. providing cloud computing services. Therefore, to setup a connection in such an all optical network, a path and unoccupied wavelength on the path must be found between the source and destination nodes. I assume there are no wavelength converters in the network, therefore the wavelength continuity constraint must be met for each lightpath being setup. In dynamic optical networks, lightpaths are setup and torndown in an on-demand fashion. Therefore, this is a suitable solution for cloud services. Additionally, eliminating the electrical layer from the network architecture ensures low energy consumption of the network nodes. In comparison to the energy consumption of data centers optical network energy consumption is negligible and is ignored in the subsequent studies. Introducing the SDN concept in wide area networks is an emerging approach, as SDN was first proposed in local networks. Many papers have been published concerning issues related to scalability of SDN to large-scale networks. The centralized SDN controller has full knowledge of the network topology and its state (information about existing lightpaths). An interesting approach is presented in [51], a survey that considers important SDN-related topics such as applications, quality of service, security and integration with wireless and optical networks, as well as future research trends. SDN support for multi-tenancy is noted as one of the main advantages of SDN leading to DC and network optimization. SDN allows various tenants maintaining DCs to deploy their applications across multiple sites. Furthermore, benefits of adopting SDN in optical networks are introduced. First of all, reconfiguration of optical networks is significantly easier using SDN. Moreover, SDN simplifies software upgrades of optical switches and enables virtualization technologies. Finally, SDN is able to reduce the cost of optical hardware by simplifying the implemented logic (the only task is to follow the flow tables). Requirements for making transport networks ready to provision cloud services are provided in [30]. The authors state that multi-tenant networks must work in an on-demand fashion and flexibly cooperate with IT resource providers. Different kinds of cooperation are investigated, since a fully integrated control plane is not relevant. In [7] an all optical network testbed is proposed. Reconfigurable optical add drop multiplexers are controlled by the extended version of the OpenFlow protocol. The solution is compared with the Generalized Multiprotocol Label Switching (GMPLS) technology. The results show that the SDN concept can be introduced into all optical networks, and indicate that the concept is faster than GMPLS. In order to serve a cloud service request, an SDN controller has to find a path between the source and one of the possible destinations using an appropriate anycast strategy fitted to a given service type. This way the network is customized for application purposes. Additionally, during the decision process, the SDN controller may take into account further information provided by the cloud orchestration software enabling the cooperation of the two clearly separate entities..

(44) 16. 2. Area of research. Each anycast or unicast request is associated with a lightpath request in the optical network. Fixed-alternate routing is used to handle lightpath requests. Fixed routing means that a path is assigned to each source-destination pair and does not change in time (see [77], [28] or [97]), while alternate denotes that an ordered set of paths is prepared for each source destination pair (see [86] or [93]). Therefore, fixed-alternate routing describes the case where an ordered set of paths is prepared at the start of network operation and remains the same throughout the network life [98]. The following algorithm (proposed in [98]) is used to compute the set of alternate paths. First of all, the metric of all links in the topology is set to 1. The shortest path algorithm is performed in order to obtain the first path between two selected nodes. Next, for all links that belong to that path, the metric is set to the number of nodes in the network. Then, the shortest path algorithm computes the second path. Again, for all links that belong to the second path, the metric is set to the number of nodes in the network and the third path is computed using the shortest path algorithm. It should be noted that alternate paths do not have to be disjoint. This is the result of low connectivity in the investigated networks. Thus, the algorithm makes it possible to re-use links; however, this solution has a low priority due to the metrics assigned to such links, since these metrics are higher than the length of any other path in the network. Additionally, the number of alternate paths for each source-destination pair must be tuned. Based on results provided in [98], this number was set to three for each pair.. 2.1.2. Computing resources. Not only optical networks but also IT resources within a DC must be modelled. It is common for a DC to have heterogeneous physical machines, with different amounts of RAM (Random Access Memory) and different CPUs (Central Processing Units). This heterogenous infrastructure may be abstracted using cloud orchestration software responsible for resource allocation, virtualization and consolidation. Furthermore, numerous task schedulers may be used in order to effectively allocate such DC resources, e.g. [36, 57, 78] or MapReduce [31]. One of the most popular implementations of MapReduce is included in the Apache Hadoop library which is a framework that allows for distributed processing of large data sets and scales up to the thousands of machines [6]. Therefore, IT resources available in each DC may be represented as a single value reflecting available RAM and computing power. A specific amount of those resources is consumed while serving a DC request and is released afterwards. A third kind of DC resources that needs to be considered is storage space, which is commonly provided using dedicated equipment. A shortage of storage space is generally unacceptable in a DC. Additionally, storage resources are not usually released after tearing down a request, further complicating the.

(45) 2.1 Cloud computing architecture. 17. case. Thus, it was decided to treat the storage resources as unlimited and omit them from the abstraction. Not only resources inside DC are heterogenous but also cloud request being served consumes different amounts of IT resources. Therefore, in addition to modelling of IT resources within a DC, it is necessary to model resource consumption of a particular request. Standardization of job submission process in cloud infrastructure is an emerging and still open issue (see [102]). Three types are considered in the dissertation: PaaS (Processing as a Service), StaaS (Storage as a Service) and SaaS (Software as a Service). Those types of services have been described in detail in [8], but I focus solely on their requirements for network and data center resources as well as their energy consumption model. It is believed that for each service type a different anycast strategy may be fitted. Practice to divide all requests into a fixed number of classes is a common approach. Such an idea of categorization of cloud service requests is applied, for example, by Google [44]. These three properties are crucial for appropriate fitting of anycast strategies to cloud service types and for proper evaluation of the proposed cooperation models. PaaS refers to a group of services in which a cloud customer is able to process computationally demanding tasks in a DC. It means that a PaaS request occupies network resources only for the time needed to send task related data to the DC, and consumes huge amount of energy and IT resources inside the DC. According to the model proposed in [41], energy consumption of a DC handling a PaaS task may be expressed by the following equation: Eproc = 1.5 · Tproc · Ps , where Ps is the power consumption of the server (355 W), Tproc is the processing time (in hours) and the scaling factor represents the power consumption overhead (e.g., air conditioning). Thus, following considerations presented in [41], an exemplary service of encoding a video file requires Eproc = 100 Wh to process 10 Gb (1.25 GB) of data. The processing time usually exceeds the time of network resource occupancy, but if we assume that the whole task may be computed in a parallel way in the time equal to the network resource occupancy we get Eproc = 36 kW/(Gb/s). It must be emphasized that this assumption does not affect the total amount of the consumed energy, it only decreases the time in which the energy was consumed. StaaS represents cloud services that allow customers to store their data in the cloud. Thus, network resources are occupied for the time needed to transfer the data. Energy and IT resources needed to handle a StaaS request are related only to the Input/Output memory operations (requiring some CPU and RAM resources) and are marginal. This assumption is valid as, in general, power consumption of storage equipment is independent of the amount of currently stored data. The.

(46) 18. 2. Area of research. average StaaS energy consumption suggested in [40] is equal to 0.141 W/(Gb/s) and this value is reasonable from the point of view of this dissertation.. SaaS spans numerous different cloud services, therefore, properties of SaaS requests are very hard to define. Some exemplary cloud services assigned to the SaaS group are: dedicated web application, virtual desktop, virtual machine with dedicated software, etc. As a result, a SaaS request may, for example, occupy network resources for a very long time while consuming only a little amount of IT resources and, subsequently, energy in a DC (as in case of virtual desktop service). However, at the same time, another SaaS request may also hold network resources for a similar time (required to make the use of dedicated software) and consume energy and IT resources comparable to the PaaS service by running computationally intensive tasks using the aforementioned dedicated software. The authors in [40] proposed an estimate of the average energy consumption in a DC for a SaaS request equal to 2.7 W/(Gb/s). This value is proper for SaaS services with very low energy requirements. Knowing a variety of SaaS services it seems reasonable to analyze different cases of energy consumption. Therefore, average energy consumption values assumed in this work are 1, 2.7, 5.4 and 10 kW/(Gb/s), i.e., much higher than the value proposed in [40], but still lower than for the PaaS services. However, the solution proposed in this paper is able to adjust even to the value suggested in [40]. The presented assumptions about cloud services are consistent with those provided in my previous work [17]. Additionally, it is assumed that cloud service requirements for IT resources are proportional to their energy consumption. Such an assumption seems to be reasonable in order to provide a consistent model of cloud services. A separate issue concerns the proportions in which the aforementioned services contribute to the total DC traffic. It is assumed that those proportions may vary in a great deal and depend on, e.g., specific cloud services being offered, customer types, time of a day, etc. In modern clouds, PaaS is probably the least popular as PaaS requires the cloud operator to strongly adjust to user needs, for example a proper codec must be available to provide a service of video encoding. Also, security issues may decrease attractiveness of PaaS, as a user must explicitly share data needed to perform tasks in a DC. On the other hand, SaaS appears to be the most frequently used as SaaS spans numerous services and offers them to clients in the most flexible way (for both cloud operators and users). Therefore, in subsequent studies I investigate different proportions in which the total DC traffic is divided between different cloud service types..

(47) 2.2 Energy-awareness. 2.2. 19. Energy-awareness. In the context of cloud computing, a great deal of previous works was focused on improving energy efficiency of data centers, as DCs are one of the main contributors to the overall energy consumption and carbon footprint of the ICT sector [62, 64]. Data centers contribute to the worldwide energy consumption significantly. The energy consumption of typical DC is about 38 MWh per day which is equal to the energy consumption of 3500 households. The amount of energy consumed by all data centers doubled from year 2000 to 2005, and increase by 56% in the following 5 years perspective (to around 330 billion kWh). It is projected to grow to more than 1000 billion kWh by 2020. As a result power consumption of data centers is between 1.1% and 1.5% of total electricity used worldwide. Most of the statistics were found in [72] and [73]. The issue of energy-awareness is of such importance, that the Green Grid Association, solely devoted to improve the efficiency of information technology and data centers throughout the World was founded [101]. The negative impact of DCs on the natural environment and the amount of energy they consume may be significantly decreased using some of the available solutions. Among the other, the most popular approaches are as follows [2]: • Placing DCs in locations where electricity is obtained from the sources which generate lower quantities of carbon dioxide and other greenhouse gases (e.g. hydropower, solar and wind power stations). • Placing DCs in locations where Mother Nature may effectively help to improve energy efficiency, for instance, where there is cold enough to use air side economizers that use outside air to chill a DC. • Spending less energy on cooling by rising the operational temperatures of a DC. • Reusing heat being generated by the IT infrastructure and drawing upon recycled water rather than fresh one for cooling purposes. • Utilizing modern and efficient chillers with lower leakage of their refrigerant. • Optimizing airflows in DCs by, for example, isolating hot and cold airflows. • Choosing greener gear which consumes less energy, generates less heat and matches the hot- and cold-aisle designs prevalent in modern DCs. For the purpose of this thesis it is assumed that green DCs are those powered from renewable energy sources. This approach is a common practice of the largest IT players, like for example: Apple [47], Facebook [37, 47] or Google [45, 47]..

(48) 20. 2. Area of research. However, this discussion and solutions may be easily adopted to other variants of green DCs, including the ones listed above. Usually, it is not possible to power all DCs using renewable energy. Green energy is available only in selected locations and transmitting it from distant sites is not worthwhile due to high transmission loses. Thus, a hybrid power scenario, where a single cloud operator may have DCs powered from renewable energy mixed with those powered from conventional energy sources, is a very realistic one. In such a case it is reasonable to utilize those green DCs first, allowing the equipment in other DCs to enter the low energy consumption state, which reduces the overall carbon dioxide emission [58].. 2.3. Problem statement. The problem investigated in this dissertation may be briefly described as dynamic anycast routing in optical networks with energy-aware cloud services provisioning. A corresponding static problem that may be formulated is NP-hard. The complexity results from the fact that the RWA problem, which is only a part of the considered problem, was proved to be NP-complete (for the proof see [28]). As I focus on the dynamic problem the static problem will not be formulated and investigated in greater detail. The network is assumed to be an all optical network interconnecting hybrid power IT infrastructure. DCs are associated with selected network nodes. Network and IT resources are administrated by separate entities. In the formal problem definition, graph G(V, E) denotes the physical network, where V is the set of nodes, and E is the set of links. Each fiber e ∈ E carries a fixed number of wavelengths indexed using w. Symbols ew = 1 and ew = 0 denote unoccupied and occupied wavelength w on link e, respectively. VDC describes the set of nodes being associated with the DCs. DCs powered from renewable energy sources are called green and network nodes associated with the green DCs are denoted by VgDC . On the other hand, DCs powered from traditional energy sources are called brown and nodes directly connected to them are denoted by VbDC . In all further equations g and b lower indices will always denote green and brown, respectively. Nodes that are not associated with any DC are termed client nodes, and denoted by VC . The following relations are met: VgDC ∩ VbDC = ∅. (2.1). VgDC ∪ VbDC = VDC. (2.2). VDC ∩ VC = ∅. (2.3). VDC ∪ VC = V. (2.4).

(49) 21. 2.3 Problem statement. For each data center d ∈ VDC , cd and rd denote total amount of resources and resources currently available, respectively. CDC and RDC describe the sets containing cd and rd for all d ∈ VDC , respectively. The network controller has full knowledge about the network topology and its state, including information about existing lightpaths, the location of DCs (VDC ). At the same time, cloud orchestration software has knowledge about power source of DCs, resources available in each DC and cloud operator’s preferences. Dynamic RWA algorithms operate on a lightpath request (LR). Two types of LRs are assumed: 1. A unicast LR from a source node s ∈ V to a destination node d ∈ V . 2. An anycast LR from a source node s ∈ VC to one from the set of possible destinations d ∈ D ⊆ VDC . Anycast LR are used to handle DC traffic. When a cloud service request (CR) arrives to the cloud orchestration software it is further passed to the SDN controller as anycast LR. From the SDN controller point of view, a path P between s and d must be found to serve any LR. Path P is composed of adjacent links and each fiber p being part of the path P , p ∈ P , must meet the wavelength continuity constraint (WCC): ∃w ∀p ∈ P pw = 1 (2.5) where w is a wavelength that may be chosen to serve the LR over the path P . Unicast LRs are served using alternate routing with three, precomputed paths. The details of the algorithm for unicast LRs may be found in [98], [99], [100] as well as in Section 2.1.1. In this work unicast lightpath requests are used solely for handling the background traffic which does not interact with the cloud orchestration software. As the cloud infrastructure is not involved in serving unicast LRs, the path between s and d must only meet WCC (2.5). As anycast schema may be defined as one-to-one-of-many routing, for each anycast LR the RWA algorithm needs to choose a destination d ∈ D and then set up a lightpath from the source node s to the chosen node d. The strategies for anycast LRs handling are described in detail in Chapter 5. As mentioned earlier, each anycast LR is associated with a cloud request, CR, which consumes IT resources within a DC associated with the chosen node d. Thus, the following constraint must be met: rd ≥ LRr. (2.6). where, LRr is the amount of resources required to serve a cloud service request associated with the particular LR. After tearing down the request, both network as well as IT resources are released. One must note that in some scenarios cd , and therefore rd , are assumed to be equal to infinity. Thus, the 2.6 constraint is.