Improving P2P Applications by Breaking the
Architecture Symmetry
Improving P2P Applications by Breaking the
Architecture Symmetry
Proefschrift
ter verkrijging van de graad van doctor aan de Technische Universiteit Delft,
op gezag van de Rector Magnificus prof.dr.ir. J.T. Fokkema, voorzitter van het College voor Promoties,
in het openbaar te verdedigen op dinsdag 9 december 2008 om 10:00 uur door Paweł Jacek GARBACKI
doctorandus in de informatica geboren te Warschau, Polen
Samenstelling promotiecommissie:
Rector Magnificus voorzitter
Prof.dr.ir. H.J. Sips Technische Universiteit Delft, promotor Prof.dr.ir. M. van Steen Vrije Universiteit Amsterdam, promotor Dr.ir. D.H.J. Epema Technische Universiteit Delft, copromotor Prof.dr.ir. R.L. Lagendijk Technische Universiteit Delft
Prof.dr. K. Aberer EPFL, Switzerland
Prof.dr. P. Felber Universit´e de Neuchˆatel, Switzerland Dr. Ch. Gkantsidis Microsoft Research, UK
Published and distributed by: Paweł J. Garbacki E-mail: pawelg@gmail.com
ISBN:978-90-8570-305-1
Keywords: peer-to-peer, heterogeneity, asymmetric architecture Copyright c 2008 by Paweł J. Garbacki
All rights reserved. No part of the material protected by this copyright notice may be reproduced or utilized in any form or by any means, electronic or mechanical, inclu-ding photocopying, recorinclu-ding or by any information storage and retrieval system, without written permission of the author.
Printed in The Netherlands by: W¨ohrmann Print Service
Advanced School for Computing and Imaging
The work described in this thesis has been carried out in the ASCI graduate school. ASCI dissertation series number 170.
This research was supported by a grant of the Netherlands Organisation for Scientific Research (NWO).
To my parents,
CONTENTS i
Contents
1 Introduction 1
1.1 P2P applications . . . 2
1.1.1 File transfer, live streaming and video on demand . . . 2
1.1.2 Search for content . . . 3
1.1.3 Resource management in content delivery networks . . . 4
1.2 Heterogeneity of the Internet . . . 5
1.3 Problem statement . . . 7
1.4 Asymmetric P2P architectures . . . 8
1.5 Research contributions and thesis outline . . . 9
2 A case study of the BitTorrent P2P network 11 2.1 The BitTorrent content distribution system . . . 12
2.2 Measurement framework . . . 14 2.2.1 Pre-processing . . . 15 2.2.2 Measurement . . . 16 2.2.3 Post-processing . . . 17 2.3 Measurement setup . . . 17 2.4 Measurement results . . . 18 2.4.1 Resource heterogeneity . . . 18 2.4.2 Configuration heterogeneity . . . 20 2.4.3 Social heterogeneity . . . 23 2.5 Summary . . . 26
3 Collaborative bandwidth exchange 27 3.1 Bandwidth exchange incentives in P2P networks . . . 28
3.2 Related work . . . 29
3.3 The amortized tit-for-tat protocol . . . 30
3.3.1 Mechanisms of ATFT . . . 30
3.3.2 Collaborative downloads . . . 32
3.3.4 Benefits of ATFT . . . 35
3.4 Analysis of ATFT . . . 36
3.4.1 System model . . . 37
3.4.2 Selection of the optimal number of helpers . . . 37
3.4.3 Download speedup . . . 39
3.4.4 Finding bartering partners . . . 39
3.4.5 Incentive for contribution . . . 41
3.4.6 Borrowers set size . . . 43
3.5 Performance evaluation . . . 44
3.5.1 Experimental setup . . . 44
3.5.2 Collaborative downloads evaluation results . . . 46
3.5.3 Amortization evaluation results . . . 51
3.6 Summary . . . 55
4 Optimizing peer relationships in a super-peer network 57 4.1 Organizing peer relationships . . . 58
4.2 Related work . . . 59
4.3 The architecture of SOSPNET . . . 60
4.3.1 Architecture overview . . . 60
4.3.2 System model . . . 61
4.3.3 Two-level caching . . . 62
4.3.4 The search protocol . . . 62
4.3.5 The insert protocol . . . 65
4.3.6 Balancing the load among super-peers . . . 65
4.3.7 Discussion . . . 68
4.4 Performance model . . . 68
4.4.1 Notation . . . 68
4.4.2 Models of the semantic structure . . . 69
4.4.3 Optimal caching performance . . . 72
4.5 Performance evaluation . . . 74
4.5.1 Experimental setup . . . 75
4.5.2 Results . . . 75
4.6 Summary . . . 81
5 Managing resources with brokers 83 5.1 Resource brokerage in latency-aware P2P networks . . . 84
5.2 Related work . . . 85
5.3 The problem of broker placement . . . 86
CONTENTS iii
5.3.2 Formal background . . . 88
5.3.3 Problem statement . . . 90
5.3.4 An example . . . 92
5.4 The broker-placement algorithm . . . 92
5.4.1 Solution outline . . . 93
5.4.2 The algorithm . . . 93
5.4.3 The computation of the region weights . . . 95
5.5 Algorithm analysis and improvements . . . 97
5.5.1 Worst-case complexity . . . 97 5.5.2 Optimizations . . . 98 5.6 Performance Evaluation . . . 99 5.6.1 Experimental setup . . . 99 5.6.2 Performance results . . . 101 5.7 Summary . . . 106 6 Conclusion 109 6.1 Conclusions . . . 109
6.2 Suggestions for future work . . . 111
Bibliography 111
Summary 127
Samenvatting 131
Acknowledgements 135
1
Chapter 1
Introduction
The peer-to-peer (P2P) paradigm is the basis for the design of a new wave of distributed Internet applications. P2P applications are built of the resources of their users, with users providing service to one another. In this sense, a P2P design is often contrasted with a client-server design where the roles of the service provider (server) and the service consumer (client) are clearly separated at the architectural level. In P2P applications, the roles of client and server are merged into the single role of peer. The Merriam-Webster Online dictionary [8] defines a peer as “one that is of equal standing with another.” This definition reflects well the role of a peer in a traditional, symmetric P2P architecture which imposes a functional equality of peer roles.
From a practical point of view, the architectural symmetry of a P2P system achieved by assigning peers similar roles may be, however, shifting the balance too far to one end of the spectrum where the other end is occupied by the client-server architectures. In a highly heterogenous environment as the Internet, some diversity of peer roles may be required to fully exploit the differences in peer capacities instead of ignoring them. Assigning different roles to peers breaks the architectural symmetry of traditional P2P applications, and the resulting architectural asymmetry may lead to significant improvements in their robustness and performance. The relevant heterogeneity aspects of the Internet environ-ment as well as the nature of the architecture asymmetry reflected in peer role diversity depend on the type of the application. In this thesis we address the challenge of designing and analyzing asymmetric architectures for the dominant P2P application types which are file downloading, live video streaming, video on demand, content search, and content delivery networks.
The remaining part of this chapter is organized as follows. Section 1.1 describes the main application domains of P2P technology. Section 1.2 categorizes the heterogeneity aspects of the Internet environment. Section 1.3 gives the problem statement of this the-sis. Section 1.4 describes the existing asymmetric P2P architectures, while Section 1.5 presents the research contributions and outlines the structure of this thesis.
1.1
P2P applications
The application domain of P2P technology is fairly broad [20, 89, 94, 117, 124]. The most common uses of this technology include file transfer, live streaming and video on demand, content search, and content delivery networks. In this section we briefly describe those uses and give examples of existing P2P applications. More details on the individual P2P applications are provided in later chapters of this thesis.
1.1.1
File transfer, live streaming and video on demand
Early P2P networks such as Napster [10] and Gnutella [116] used simple, single-source, ftp-like protocols for transferring files between peers. The content shared in the early P2P networks consisted mostly of small-size music files. With the later increase of the popularity of large-size movie files, the bandwidth required to move files around in a P2P network became the bottleneck. The absence of protocol-enforced incentive mech-anisms resulted in the plague of free-riders [16], i.e., users exploiting (bandwidth) re-sources of others while refusing to contribute their own rere-sources. The revolution came with the BitTorrent [40] protocol — the first widely used P2P data transfer protocol that enforces bandwidth contributions from the peers. BitTorrent employs a technique called tit-for-tat that effectively limits a peer’s download bandwidth based on its upload bandwidth. Furthermore, peers in BitTorrent obtain data from multiple sources at a time which allows for data exchange between peers with different link capacities, fully utilizing those capacities. The topology formed by BitTorrent peers has the structure of a random mesh. Subsequent P2P file transfer protocols target some of the limitations of BitTorrent. Slurpie [121] improves the download source selection strategy of BitTorrent in order to have a more efficient bandwidth utilization. Avalanche [58] employs a network coding technique to increase the diversity of the data across peers and thus increases the chance that two randomly selected peers can establish a data exchange connection.
The continuous increase in Internet bandwidth capacity offerings for end users opens up possibilities for live streaming and video on demand (VoD). Video streaming over the Internet has been one of the most widely investigated research areas over the last few years [22, 33, 65, 142]. Most research efforts concentrate on designing video distribution systems that can support a large number of users. Multicasting has been proposed to provide a scalable video streaming service, even in the presence of non-homogeneous receivers [73, 86, 105]. Multicasting is a natural paradigm for live video streaming, but today’s Internet does not offer native support for multicasting. In the absence of such support, there have been many proposals to use an overlay multicast for live streaming [3, 32, 38, 64, 133, 143]. Also mesh-based P2P networks have been used for this purpose [92, 146].
1.1 P2P applications 3
The concept of multicast has been extended for supporting near video on demand (nVoD) services. In the simplest approach one periodically starts a new broadcast [18]. More elaborate systems propose to divide the video into segments and distribute each segment in a different multicast channel [14, 65, 137]. A different approach has been sug-gested in the BASS [47] system which builds an nVoD service on the BitTorrent protocol. BASS assumes the existence of a streaming server and uses the P2P network to alleviate the load imposed on the server. Even though BASS reduces the load at the server by a sig-nificant amount, the design of the system is still server oriented, and hence, the bandwidth requirements at the server increase linearly with the number of users. The BiToS [138] system, which is also based on BitTorrent, offers serverless VoD. The main idea of BiToS is to divide the missing pieces into the “high-priority set” and the “remaining-piece set,” and to request the pieces in the remaining-piece set with lower priority. The basic idea of piece prioritization has been extended with a more efficient piece distribution scheme in the Give-to-Get system [95]. Also mechanisms such as network coding and optimized overlay topology management have proven their efficiency in a VoD environment [21].
1.1.2
Search for content
Early P2P protocols such as Gnutella [116] and Freenet [39] concentrated mainly on efficient content location. The content was usually present in the form of files shared by the users of a P2P network. The P2P protocols for searching are traditionally divided into the two classes of structured and unstructured approaches.
Structured P2P networks assume that each content item is attached a unique identifier or key. The items can then be located based on the value of the corresponding key. Due to the conceptual similarity with an algorithmic data structure called hash table, structured P2P networks are often referred to as distributed hash tables (DHTs). Each peer in a structured P2P network is responsible for the content items that have keys that belong to a certain subspace of the key space. The assignment of key subspaces to individual peers determines the topology of the overlay interconnecting the peers. By exploiting some specific properties of the key space, structured P2P networks optimize searching. Typically, the cost of search in a structured P2P network measured in the number of hops, i.e., the number peers consulted while routing the query, is a sublinear function of the number of peers in the system. If the key specified in a search query exists and the system operation is not disrupted, guarantees can be provided that the query will be resolved successfully. The details of the assignment of keys to peers and the search query routing algorithm differ between various designs. The Chord [129] system organizes peers into a logical ring. The Content Addressable Network (CAN) [115] maps keys to points in a multi-dimensional Cartesian space. Pastry [118] and Tapestry [147] represent keys with numerical values and employ a longest prefix routing technique for locating
content. The use of structured P2P networks reaches beyond file sharing. The Chord system architecture has been applied for file mirroring in the Cooperative File System (CFS) [46] in which multiple providers of content store and serve each other’s files. A Chord-based DNS [42] provides a host name lookup service that maps host names to IP addresses and other host-related information. CAN has been used successfully in large scale storage management systems such as OceanStore [76] and Farsite [17].
Unstructured P2P networks do not route queries based on abstract keys but rather employ a generic search technique such as flooding, random walks or expanding ring time-to-live (TTL) search. Each visited peer checks if the locally stored content meets the criteria of the search query. Compared to the functionality of searching for keys offered by structured P2P networks, the queries in unstructured P2P networks can be more ex-pressive. In particular, keyword searches or range queries, which are difficult to support in a structured network, are trivial to realize in an unstructured network. However, the unstructured approaches pay for the querying flexibility with performance and reliability. A hard guarantee of locating a content item can be provided only if the search query is propagated to all peers in the system. A number of unstructured P2P network architec-tures have been proposed to date but only a few of them have become popular. A typical example of an unstructured search protocol is flooding with a TTL implemented in the Gnutella [116] network. Several optimizations to the original Gnutella protocol intended to improve its load balancing, scalability, and response time have been proposed in [35]. The loose topological dependencies between peers in an unstructured network open up possibilities for organizing the network topology to exploit the semantic properties of the content for more efficient searching. Semantic clustering employed in systems as Prox-sem [31] and Edutella [100] increase the query locality by grouping together content items which are semantically related.
1.1.3
Resource management in content delivery networks
File transfer, live streaming, VoD, and content location can be collectively described as mechanisms for content management. Apart from content management, also computer resource management can be provided with a P2P solution. In many applications, com-puter resources such as disk space, computing power and bandwidth available at one peer have value for other peers. Typical examples of such applications are content delivery networks (CDNs) [124, 134] that lower the costs of Web hosting while improving the scalability. Moreover, CDNs increase the resilience to a sudden overload following pub-lication, a phenomenon that has been nicknamed the Slashdot effect, after a popular web site that periodically links to under-provisioned servers, driving unsustainable levels of traffic to them. CDNs place the replicas of Web sites on the peer-managed resources of their users and distribute the load in order to improve the response time. Individual Web
1.2 Heterogeneity of the Internet 5
sites are usually small in size compared to the multimedia content exchanged in file shar-ing P2P networks. Therefore, not the storage disk space or the transfer bandwidth, but rather the latency of accessing the Web sites becomes the bottleneck for which the system is optimized. The Globule [6] CDN assigns to each peer serving Web site replicas and to each client requesting those replicas a point in a metric space where the distance between points approximates the internode latency [130]. The latency-space abstraction allows for the easy computation of client request concentration points and placement of Web site replicas in the proximity of these points [131]. The clients are sent to close-by replicas with DNS redirection.
An alternative to Globule, the Coral [49] CDN, is built on top of a key/value indexing infrastructure called distributed sloppy hash table (DSHT). DSHT is similar to a tradi-tional DHT but it provides a weaker consistency model. Coral allows clients to locate nearby cached copies of Web sites without querying more distant nodes. It also prevents hot spots in the infrastructure, even under degenerate loads by using node clustering pro-vided with an epidemic algorithm. Another CDN built on top of a DHT is the SCAN [36] network. SCAN combines dynamic replica placement with a self-organizing application-level multicast tree to meet the service quality and the server resource usage constraints. It utilizes an underlying distributed object routing and location system which is imple-mented on top of the Pastry DHT [118]. One of the main goals of SCAN is to keep the number of replicas at a minimum to reduce the replication overhead. A number of sys-tems [127, 128] have been designed specifically to address the problem of the Slashdot effect or, in a broader sense, of flashcrowds, which are large bursts of requests over a short time period. These systems focus solely on mitigating the client request load and keep the optimization of the request latency out of their scope.
1.2
Heterogeneity of the Internet
Most of the P2P applications are designed with the Internet as the intended deployment platform. The peer role equality promoted by the pure P2P designs contrasts, however, different heterogeneity aspects of the Internet environment. A central concept in the Inter-net philosophy is the diversity of computer and Inter-network resources. The distributed own-ership of Internet resources results in a multitude of often very different configurations of hardware and software components. The wide scope of the applicability of Internet technology results in a user base which is socially diverse. Below we identify instances of the resource, configuration and social heterogeneity of the Internet and discuss their impact on the performance and robustness of P2P applications.
Resource heterogeneity. The support for heterogenous computer and network
re-sources is the fundament upon which the Internet is built. The processing speed, mem-ory capacity and disk space differ across the Internet-connected computers.
Compute-intensive P2P applications should be able to adjust the volume of the workload to the processing speed and memory capacity. The content exchange P2P protocols have to re-spect disk-space limits. Apart from the computer resources, also the network properties have to be taken into account in the design of P2P applications. Depending on the type of the application, the relevant bottleneck property of the network is either the bandwidth or the latency. The bandwidth is crucial for applications where peers exchange high-volume data such as file sharing P2P networks. The latency, on the other hand, plays an important role in applications providing live user experience such as video streaming and systems in which the communicating peers exchange content items of relatively small sizes as in CDNs.
Configuration heterogeneity. Many components of the Internet infrastructure,
indi-vidual computers, and P2P applications, can be configured. The ownership of the Internet network infrastructure is partitioned among Internet service providers (ISPs). A P2P ap-plication aware of network partitioning can optimize its topology to reduce the volume of the traffic crossing ISP borders. The reduction of cross-ISP traffic has an important eco-nomical benefit as the contracts between ISPs are negotiated based on the amount of traffic coming in and out of their networks. A network configuration aspect which is controlled by the infrastructure owners as well as the users of individual computers is the connec-tivity determining whether two peers can communicate directly. Direct communication is possible only if at least one of the communicating peers can accept external connections. Due to security risks, many Internet-connected computers are shielded with firewalls and network address translators (NATs) that block incoming connections. Firewalls and NATs are either operated at the ISP level or they are integrated with the operating systems of the individual computers. Enabling communication between shielded peers requires tech-niques such as connection relaying or firewall traversing. Although Internet end users have little influence on the configuration of the network infrastructure, they have full con-trol over the P2P application. The application uptime is determined by the users who decide for how long the application will run on their computers. The uptime determines the availability periods of the resources controlled by the P2P application. Peers with long uptimes acting as well known access points can be used for bootstrapping newcomers.
Social heterogeneity. The user-centric nature of P2P systems requires addressing
the differences in social aspects of the Internet with the same attention as resource and configuration heterogeneity. Social aspects of Internet communities such as the nature of the relations between users, their content interests, expert knowledge, social commitment, and language, geographical and cultural backgrounds differ across individual users. The nature of the social relationships between users can be specified in some P2P applications. A closer relation usually implies a higher level of trust between users which translates in the context of a P2P network to less restricted access to the user resources. A seman-tic correlation between user interests is known to exist in content sharing P2P networks.
1.3 Problem statement 7
Clustering of users with similar interests improves the performance of content search and provides a basis for content recommendation. The expert knowledge and commitment to the community can be exploited by assigning specific roles to selected users in a P2P sys-tem. In some file sharing communities, a group of committed users is assigned the role of moderator. The responsibility of the moderators is to remove inconsistent metadata from the system. By expressing the social commitment in a form of a ranking such as top content injectors or top uploaders, P2P applications stimulate user contributions. The lan-guage, geographical and cultural diversity of the communities, if identified and properly interpreted, can improve P2P system usability.
1.3
Problem statement
The heterogeneity of the Internet environment, if ignored or not handled properly in P2P application design, can have a negative impact on application performance. Some P2P designs try to reduce the negative influence of heterogeneity by masking it from the applications and the users. In this thesis we take a different approach by not masking het-erogeneity but rather using it in favor of the application. We address the research problem of exploiting Internet heterogeneity by breaking the architectural symmetry of traditional P2P applications in an effort to improve their performance and robustness. This high-level problem statement can be detailed in the following series of finer-grained questions of assessing the level of the heterogeneity in real-world P2P networks, and of proposing architectural modifications specific to different application types.
How heterogenous is the environment of Internet-deployed P2P networks?
As-sessing the extent of the heterogeneity of the environment in which P2P networks are deployed is required to motivate the work presented in this thesis and to provide realistic parameters for the models used to evaluate the solutions that we will devise. The assess-ment should be done by measuring existing P2P networks. The deployassess-ment size of the popular P2P networks poses a practical challenge of building a measurement infrastruc-ture able to deal with systems consisting of millions of peers.
How can we exploit asymmetric architectures to improve file sharing, live stream-ing and VoD? The heterogeneity of peer bandwidth capacities as well as the diversity of
bandwidth usage patterns open up possibilities for bandwidth utilization improvements. In particular, we are interested in the performance impact of the discrepancy in the band-width capacity among peers as well as between the upload and download links of the same peer. The diversity of bandwidth usage patterns opens up possibilities for idle peers to transfer their bandwidth resources to peers downloading data. Such bandwidth transfer requires, however, rethinking of the resource-exchange model of current content transfer protocols.
How can we make asymmetric content search networks more efficient and ro-bust? The research on improving the performance of content search by leveraging the
heterogeneity of peers in terms of available bandwidth and processing power has resulted in asymmetric architectures based on super-peers. The super-peer architectures proposed to date (see Section 1.4) use a fixed topology where each peer is connected with a single, randomly selected super-peer. Building a super-peer network with a dynamically organiz-ing topology and super-peer selection optimized for search performance and robustness is a challenging research problem.
How do the asymmetric architectures fit into resource management in CDNs?
The management of peer resources in a CDN can be seen as a task required by the main functionality of the system, which is serving Web sites. It makes much sense to delegate this task to selected peers with higher capacities that can accept additional work without adversely affecting the performance of serving Web sites. The design of an architecture for resource management in CDNs needs to address the issues of resource discovery, allocation and load balancing.
1.4
Asymmetric P2P architectures
Asymmetric architectures have been an active field of research in P2P systems. The concept of leveraging the heterogeneity of peers by exploiting high-capacity nodes in the system design has proven to have great potential [145]. The resulting architectures break the symmetry of pure P2P systems by assigning additional responsibilities to high-capacity nodes called super-peers. In a super-peer network, a super-peer acts as a server to
client (ordinary, weak) peers. Weak peers submit requests to their super-peers and receive service from them. Super-peers are connected to each other by an overlay network of their own, submitting and answering requests on behalf of the weak peers.
KaZaa [83] and Morpheus [9], which are both based on the FastTrack [4] protocol, are widely used file sharing systems that make use of super-peers. Although FastTrack is a proprietary technology with no detailed documentation, it is known that FastTrack peers are automatically elected to become super-peers if they have sufficient bandwidth and processing power (a configuration parameter may allow users to disable this feature). A central bootstrapping server provides new peers with a list of one or more super-peers to which they can connect. Super-peers index the files shared by peers connected to them, and proxy search requests on behalf of these peers. All queries are therefore initially directed to the super-peers.
An extension of the basic Gnutella [116] system has an architecture based on
ultra-peers[123], which are conceptually equivalent to super-peers. Any new peer with enough
bandwidth and CPU power immediately becomes an ultrapeer and establishes connections with other ultrapeers, forming an overlay network. A new ultrapeer is required to
estab-1.5 Research contributions and thesis outline 9
lish a predefined minimum number of connections to client peers within a specified time. If this minimum is not reached, the ultrapeer turns into a regular client peer and can try to become an ultrapeer again after some period of time.
Super-peer architectures integrate naturally with P2P systems based on semantic clus-tering [87] such as the Edutella [100] network. Systems for semantic clusclus-tering create a logical layer on top of the base P2P network topology by grouping peers with similar content interests. The clustering is performed by matching the semantic information pro-vided by the peers to clusters, with each cluster being maintained by a super-peer. In addition to controlling the internal structure of the cluster, super-peers are responsible also for routing messages between peers from different clusters.
Super-peer architectures have also been proposed for structured P2P networks [57, 97]. Such architectures group nearby peers based on some criterion such as network latency or adjacency in the key space, and organize the communication between groups using a super-peer layer. To find a peer that is responsible for a key, the top layer overlay network routes among the super-peers to determine the group responsible for the key. The responsible group then uses an intra-group overlay network to determine the specific peer that is responsible for the key. The lookup time in structured super-peer networks depends on the size of the state maintained by each super-peer and on the total number of super-peers. Some architectures [97] are even able to guarantee a constant-time lookup.
1.5
Research contributions and thesis outline
In this thesis we investigate the possibilities of leveraging the inherent heterogeneity of the Internet environment in the asymmetry of P2P application design. The scope of our research spans from the measurement, analysis, and modeling of deployed P2P networks, through optimizations of existing P2P designs, to proposing novel P2P architectures. Below we describe the main contributions of this thesis.
Measuring P2P networks (Chapter 2). We describe the design, implementation and
deployment of a framework for measuring large-scale P2P networks, which we have used to measure the BitTorrent network and the popular content indexing site Suprnova.org. Over a period of three years we have collected traces spanning in total several months which we analyze to identify patterns and extract outliers. The analysis builds a case for the characterization of the heterogeneity of P2P networks and provides parameters for models used in the performance evaluation of various solutions proposed in this thesis. The content of this chapter is based on our research published in [68, 111].
Improving bandwidth utilization with bandwidth exchange (Chapter 3). We
pro-pose a protocol called amortized tit-for-tat (ATFT) that improves the bandwidth utiliza-tion in file sharing, live streaming and VoD P2P networks. The ATFT protocol provides incentives for peers with idle bandwidth to support ongoing downloads, effectively
in-creasing the bandwidth capacity of the P2P system. ATFT implicitly addresses in its de-sign the issue of peers with asymmetric (ADSL) links. We implement ATFT making it the integral part of Tribler, a popular P2P client compatible with the BitTorrent protocol. An analytical and experimental study indicates that with realistic settings, ATFT reduces the download time more than threefold compared to the current state-of-the-art transfer pro-tocols. The content of this chapter is based on our research published in [51, 53, 56, 112].
Improving search performance in super-peer networks (Chapter 4).
Address-ing the limitations of the existAddress-ing super-peer networks, we propose a self-organizAddress-ing super-peer network architecture (SOSPNET). Unlike other super-peer networks, SOSP-NET adapts the topology of inter-peer connections to reflect the semantic correlation between the content items and the peers in an effort to improve the locality of search requests. In addition to the improved search performance, SOSPNET offers simple so-lutions to the generally difficult problems of fault tolerance and load balancing. Using a trace-based model of a P2P network, we show that SOSPNET achieves close-to-optimal search performance and offers a better robustness than alternative architectures for content searching. The content of this chapter is based on our research published in [52, 54].
Improving resource management in CDNs (Chapter 5). We propose an
architec-ture for resource management in CDNs in which high-capacity peers take the role of resource broker and become a mediator between the resource providers and the resource requesters. We propose a method for deciding on the sets of resources that are managed by specific brokers which allows to locate easily resources with certain latency proper-ties. We then combine this method with an algorithm that controls the load imposed on the brokers preventing them from becoming overloaded. We show that this algorithm is optimal up to a linear factor in the number of resources in the system. The content of this chapter is based on our research published in [55].
11
Chapter 2
A case study of the
BitTorrent P2P network
A proper motivation for the mechanisms proposed in this thesis requires a thorough under-standing of the properties of the environments for which those mechanisms are intended. The Internet environment, with its heterogeneity aspects relevant for P2P networks, can be reliably characterized by studying actually deployed P2P networks. At the time of writing this thesis, the most popular P2P network was the BitTorrent system. BitTorrent, originally designed to transfer files, has been successfully employed for other types of P2P applications including live video streaming and video on demand. A number of ser-vices built around the BitTorrent system provide the functionality missing in the original BitTorrent design including content search. We believe that the widespread popularity and the universal applicability of BitTorrent makes it a representative P2P system and an excellent subject for a measurement study characterizing P2P environments.
The distributed nature of BitTorrent and its weak dependency on central components preclude the possibility of collecting the system data at a single point. To meet the scale requirements of collecting data from individual peers, we have designed a custom mea-surement framework and deployed it on a distributed super computer. The framework is built as a composite architecture that consists of components that collect data from the central services for peer and content discovery as well as from the individual peers. The data collected during two measurement periods in total span a period of several months.
The remaining part of this chapter is organized as follows. Section 2.1 describes the basis of the BitTorrent system and the accompanying content discovery services. Sec-tion 2.2 outlines our measurement framework while SecSec-tion 2.3 describes the deployment details and the configuration of the framework during the measurements. Section 2.4 presents and interprets the data collected during the measurements. Section 2.5 summa-rizes this chapter.
file pieces seeder leecher piece transfer
Figure 2.1: File piece exchange in a BitTorrent swarm.
2.1
The BitTorrent content distribution system
BitTorrent [40] is a content distribution protocol optimized for the dissemination of large content items (i.e., files) concurrently to many receivers. The files distributed in the Bit-Torrent network are divided into pieces, which are individually exchanged between peers. Piece exchange is controlled by the tit-for-tat mechanism that enforces fairness in the bandwidth contributions of the peers. BitTorrent tit-for-tat implements a bartering econ-omy in which a peer selects to upload pieces to a limited number of peers that recently contributed the most pieces to the uploading peer. Peers uploading pieces at a higher rate can therefore expect higher download speeds. The dependency between the upload and the download rates of peers builds incentives for data contributions, and BitTorrent is his-torically the first widely deployed P2P content transfer protocol that does provide such incentives.
BitTorrent uses specific terminology to describe the system elements. The set of peers downloading the same file at the same time is referred to as a swarm. The peers that have obtained the entire file but choose to stay in the network and serve the content to other peers for free are called seeders. The peers whose downloads are still in progress are named leechers. The exchange of the file pieces between peers in BitTorrent system is illustrated in Figure 2.1.
Each file distributed in BitTorrent has an associated metadata file with a name tradi-tionally appended with the extension .torrent. Hence, the metadata files are often referred
2.1 The BitTorrent content distribution system 13
Tracker points to
User
new peer
Peer 1 Peer 2 Peer n Peer 1
Peer 2 Peer n
get torrent file
get peers
connect to peers
Web site mirrors
torrent file torrent files server
Web site se arch for conten t points to points to
Figure 2.2: Content and peer discovery in BitTorrent system.
to as torrents. In addition to the typical information included in content metadata such as the content file names and sizes, a torrent contains the identity (i.e., the IP address and the port number) of a tracker. A tracker is a component of a BitTorrent network which provides peer discovery functionality. More precisely, a tracker maintains a list of all online and connectable (non-firewalled) peers downloading or seeding a certain set of content files. Upon request from a peer who is interested in some file, a tracker returns a random sample from the list of peers it maintains. For reliability, the torrent can con-tain a list of several alternative trackers maincon-taining information about the corresponding swarm. Usually, trackers are responsible for peer discovery in multiple swarms. The ob-vious scalability bottleneck inherent to the centralized nature of a tracker has resulted in a recent extension to the base BitTorrent protocol that replaces trackers with the Kadem-lia DHT [93] for distributed peer discovery [44]. Despite the superior scalability offered by the DHT-based solution, presently a vast majority of the BitTorrent swarms are still managed by centralized trackers.
Another important type of meta information included in the torrents are the SHA1 hashes for all pieces of the content. Building hash-based content validation into the proto-col provides a mechanism for preventing content pollution. Separation of the hashes from the content itself by embedding them in torrents has implications on the way fake content filtered is out from the BitTorrent network. In principle, if a torrent has been obtained from a trusted source, a peer can use it to reliably validate every downloaded piece. The sources for torrents are usually torrent file servers which are accessible through dedicated Web sites. Those Web sites are often mirrored for better scalability. A common technique employed to remove fake files from the Web sites is content moderation. On a moderated site, a community of selected, highly trusted users called moderators manually inspect the
submitted torrents for their consistency with the described content and decide which files are posted online. In the example of the Suprnova.org [13] site, moderation has proven to be very reliable in fake content filtering. Unfortunately, the manual metadata inspection is expensive and does not scale well with the content availability, and hence, not many among the torrent distribution sites are currently moderated. Nevertheless, most of those Web sites offer a possibility for posting comments on the torrents which can be used for marking fake files.
Apart from fake content filtering, the main function of the torrent distribution Web sites is to provide a content location service. BitTorrent does not support any content search functionality on the protocol level, so the Web sites take the role of the gateways to the system, allowing users to locate the content they are interested in. In addition to a typical keyword search, the Web sites support also the categorization of the files. The cat-egories are usually fixed and they represent the type of the file content. The catcat-egories are usually defined hierarchically. For instance, the generic category “Movies” can be divided into more specific subcategories as “Comedy,” “Action,” “Adventure,” etc. The content categorization allows for locating content by top-down browsing rather than file-name searching. The browsing is more handy in situations when the user, rather than looking for a specific file, is interested in a certain content type. The interactions between different BitTorrent components involved in content and peer discovery are shown in Figure 2.2.
2.2
Measurement framework
The extent of control decentralization in P2P systems as BitTorrent makes the measure-ments of those systems a challenging task. In principle, gathering detailed peer statistics is possible only by tracking individual peers. The resource consumption of the measuring system is thus proportional to the size of the measured P2P system and the variety of the measured aspects. To reduce the otherwise preventive costs, it is common among the measurement efforts to concentrate on a sample of the peers and to capture only certain aspects of the P2P system. Another way of reducing the cost of the measurements is to modify the P2P client software by adding functionality for collecting the data directly from the peers. The data collected at each peer can then be archived in a central repos-itory, of which maintenance is basically the only cost for conducting the measurements. Despite its obvious advantages including the first-hand accuracy of the collected data and the low operational cost, the intrusiveness of this approach is not acceptable for many users. User reluctance to any software collecting information about the local interactions, commonly known as spyware, is fully understandable from the perspective of privacy pro-tection. Collecting peer statistics locally is in most of the cases not an option and more indirect measurement methods are needed.
2.2 Measurement framework 15
Figure 2.3: The measurement framework.
designed a distributed measurement framework that captures the properties of the BitTor-rent network without intruding on the network operation. Our measurement framework does not require any changes in the BitTorrent client, the tracker, or the torrents distri-bution Web site. All information about the network is obtained (ab)using the standard communication protocol of the peers, trackers and Web sites.
Our measurement framework consists of several components involved in the process of data collecting and analyzing, presented in Figure 2.3. The process itself is divided into three stages: pre-processing, measurement, and post-processing. During pre-processing, high-level statistics describing the BitTorrent swarms are collected from the Web sites. Based on those statistics, swarms are selected for the measurements that involve direct contact with the tracker and with individual peers. The data collected during the measure-ments is analyzed during post-processing. The components involved in the three stages of the data collecting and analysis process take the form of four data repositories, four custom data-collecting scripts, and one modified BitTorrent client. We will now describe the components involved in each stage of the measurement process in the order of their appearance in the figure going from left to right.
2.2.1
Pre-processing
As we have explained in Section 2.1, BitTorrent uses Web sites as metadata directories. The Web site statistics are collected by the WebCrawler script. WebCrawler traverses the torrents distribution Web sites, parses their content and extracts the relevant informa-tion. A typical Web site describes a torrent with the associated content name, the date and time when the torrent was posted, the size of the content file, the number of seed-ers and leechseed-ers, and the category classifying the content. A screenshot of a table with content information as presented on a typical torrents distribution site is shown in
Fig-Figure 2.4: Information organization on a typical torrents distribution Web site.
ure 2.4. The data collected by WebCrawler is stored in the Web site stats repository. The information exposed on the Web sites does not provide a complete description of the associated swarms. In particular, the identities of the trackers required to establish con-nections with the BitTorrent peers are not available on the Web. To fill this information gap, WebCrawler downloads also the torrents linked from the Web pages and stores them in the torrents repository. Even when limiting ourselves to the files indexed by a single torrents distribution site, the total number of the peers downloading those files can be pro-hibitively large for measurement software which has only a limited amount of resources available. The statistics collected in the Web site stats and the torrents repositories allow to select the swarms that are relevant for a particular measurement objective.
2.2.2
Measurement
The measurement itself uses two independent approaches to collect peer statistics. These approaches differ in the method of discovering peers in the swarm and they can be de-scribed as the active start and the passive start measurements. The active start measure-ments involve the GetPeers script that connects to the tracker and explicitly requests the list of peers in the swarm. Since the protocol of the tracker allows for requesting only a small number of randomly selected peers, the GetPeers script continuously polls the tracker for new peers at a constant time interval. The list of the discovered peers is passed to the PingPeers script that contacts individual peers and obtains statistics such as the number of file pieces they possess and their uptime. The peers that are not connectable as a result of being located behind a firewall or a network address translator (NAT), are not registered at the trackers, and even if they were, the PingPeers script would not be able to establish a connection with them. The discovery of non-connectable peers is the task of the passive start measurements. The passive start measurements register at the tracker a number of BitTorrent peers with modified code called ListenPeers that act as honeypots, waiting for other, also non-connectable peers to establish a connection with them. The code modifications, apart from preventing ListenPeers clients from downloading any data
2.3 Measurement setup 17
from the swarm, enable the logging of the IP addresses of the discovered peers. The peers discovered during the active or the passive start measurements are probed from multi-ple network locations by the TrackPeers scripts with the purpose of characterizing the topology of the Internet links interconnecting the peers and measuring the correspond-ing round trip times (RTTs). The operation of the TrackPeers scripts can be described as a multi-source traceroute in that the measurements are performed concurrently from multiple network locations.
2.2.3
Post-processing
The data describing peers collected during the measurements is analyzed in the post-processing stage. Post-post-processing aggregates and correlates the peer-level statistics into a system-level view of the P2P network. The data on the piece download progress is used to estimate the peer download bandwidth. The IP addresses of the peers are mapped to the geographical locations of their origin. The connectivity graph depicting the topology of the network interconnecting the peers is constructed. The network graph is then seg-mented based on the Autonomous Systems (AS) adhesion of the network routers and the estimated RTTs are attached to the network connections.
2.3
Measurement setup
We have performed two independent measurements of the BitTorrent network. The first,
long-runmeasurement spans a period of 8 months from June 2003 to March 2004, while
the second, short-run measurement was performed from May 5 to May 11, 2005. The ob-jective of the long-run measurement was to observe the trends in the BitTorrent network at a coarse granularity but over a long period of time. The short-run measurements, on the other hand, capture more detailed network statistics but over a short time period. An important decision that could affect the credibility of the measurements is the selection of the torrents distribution site. In the case of both the long-run and the short-run mea-surements, we simply select the largest (in terms of the number of torrents) Web site. At the time of performing the long-run measurements, this was Suprnova.org with around 50,000 torrents, and during the short-run measurements this was Piratebay.org [12] with nearly 120,000 torrents.
We have used two platforms for the deployment of our measurement framework. The components performing active start measurements have been deployed on 100 of the 400 nodes available on our Distributed ASCI Supercomputer (DAS) [24], located in the Netherlands. For the passive start measurements, we have used 50 PlanetLab [106] nodes to run ListenPeers, and 300 PlanetLab nodes for the deployment of the probes used by the multi-source traceroute invoked by the TrackPeers scripts. The 300 traceroute nodes
were scattered around the world, but were mostly part of the GREN network [25]. Scrip-troute [125] was of inestimable help for distributed software deployment and operation on PlanetLab.
2.4
Measurement results
In this section we present and interpret the data collected during our measurements of BitTorrent and the accompanying content discovery services, in particular analyzing the heterogeneity aspects of the Internet environment including the resource, the configura-tion, and the social heterogeneity. The interpretation of the data, apart from illustrating the complications to overcome in the design of P2P applications, shows also the potential of the heterogeneity of the environment that an application can exploit in its favor. Our measurement data consist of detailed traces gathered over a period of 8 months of more than two thousand global components. In addition, for one of the most popular files we followed all 90,155 downloading peers from the injection of the file until its disappear-ance (several months). In a period of several days we measured the bandwidth of 289,580 peers downloading over a thousand newly injected files. This makes our measurement effort one of the largest ever conducted.
2.4.1
Resource heterogeneity
The resource heterogeneity aspects captured in our measurements include the bandwidth distribution of the peers and the RTTs between the peers and our infrastructure nodes. The mapping of IP addresses of the peers to their geographical locations allows us to present the bandwidth-related statistics separately for different continents. The results shown in this section are based on the data collected during the short-run measurements of BitTorrent/Piratebay.org.
In P2P networks designed for transferring large files, such as BitTorrent, the bottle-neck resource is bandwidth. The limits on the bandwidth capacities of Internet connec-tions are determined in the contracts between the users and their local ISPs. The selection of the bandwidth capacities offered in the contracts depends on the quality of the local Internet infrastructure. One would expect that this quality differs across regions, coun-tries and continents. We have measured the bandwidth of 289,580 peers downloading a thousand popular files available from the Piratebay.org Web site on May 5, 2005. Fig-ure 2.5 presents the distribution of number of users (peers) per bandwidth, for different continents. Note the logarithmic scale on the vertical axis. We find several groups with similar bandwidth characteristics: (i) Europe and North America, (ii) Asia, (iii) South America and Oceania, and (iv) Africa. In each group we can observe a power-law rela-tion indicated by the linear correlarela-tion between the download speed and the number of
2.4 Measurement results 19 1 10 100 1000 10000 100000 0 1000 2000 3000 4000 5000 >6000 Number of peers Average bandwidth [kbps] Europe North America Asia South America Oceania Africa
Figure 2.5: Bandwidth distribution across continents (Piratebay.org).
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0 1000 2000 3000 4000 5000 6000 7000 8000 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Fraction CDF RTT [msec] Fraction CDF 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0 200 400 600 800 1000 1200 1400
Figure 2.6: Distribution of the round trip time (RTT) between peers (Piratebay.org).
downloads at that speed visible in Figure 2.5.
A widely adopted metric for expressing the latency between two Internet nodes is the round-trip time (RTT). We measured the RTTs using traceroute between the measure-ment framework nodes executing the TrackPeers scripts and the contacting peers. We report here only the results for the complete routes discovered by the scripts. Since the traceroute measurements were conducted from well-connected PlanetLab nodes, there is
0.1 1 10 100
1 10 100 1000
Average uptime period [days]
Average-uptime ranking Trackers
Web site mirrors
torrent file servers
Figure 2.7: The uptime ranking of the three types of global components in BitTor-rent/Suprnova.org .
a positive bias in the reported results: the average RTTs are smaller than normal. How-ever, the fact that we used globally spread sources reduces this effect. Figure 2.6 shows the detailed distribution of RTT. More than 75% of the measured RTTs fall in the 100ms-1s category. Less than 10% of the measured RTTs are outside the 2.2 seconds margin. Since BitTorrent is not latency-limited, we conclude that most BitTorrent peers are close to each other, if the distance metric is the RTT.
2.4.2
Configuration heterogeneity
The configuration heterogeneity is illustrated in our measurements by example of upti-mes. The analysis of the the availability periods of Web site mirrors, torrent file servers, and BitTorrent trackers gives an indication of the reliability of the central system compo-nents. The uptime of the seeders reflects the P2P software usage patterns of the individual system users. Data analyzed in this section has been collected during the long-run mea-surements of BitTorrent/Suprnova.org.
The BitTorrent/Suprnova.org architecture is vulnerable because of potential failures of the four types of global components: the main Suprnova.org Web site, its mirrors, torrent file servers and trackers. The main Suprnova.org server sometimes switched IP number and was down several times. The various mirrors rarely survive longer than a few days due to the high demands of over 1,200,000 daily visitors (Oct 2004), and sometimes, fewer than five mirrors were up. Occasionally, no torrent file servers were available, blocking
2.4 Measurement results 21 0.1 1 10 100 1000 1 10 100 1000 10000
Peer uptime [hours]
Peer uptime ranking
Content lifetime
Figure 2.8: The uptime of the seeders distributing the PC game “Beyond Good and Evil” (Suprnova.org).
all new downloads. In general, trackers are a frequent target for denial-of-service attacks and are costly to operate due to GBs of daily bandwidth consumption. Figure 2.7 shows the results of the availability measurements of 234 Suprnova.org mirrors, 95 torrent file servers, and 1,941 BitTorrent trackers (Suprnova.org itself is not shown). In the figure we plot the average uptime in days for these global components ranked according to decreasing uptime. Only half of the Suprnova.org mirrors had an average uptime of over 2.1 days, which is a good indication of their (un)availability. In addition, only 39 mirrors had a continuous uptime period longer than two weeks. We can conclude that reliable Web hosting of Suprnova.org pages is a problem. As shown in the figure, the torrent file servers are even less reliable. A few trackers show a high degree of availability, with one tracker even showing a continuous uptime period of over 100 days. Half of the trackers had an average uptime of 1.5 day or more, and the 100 top ranking trackers had an average uptime of more than 15.7 days.
The availability of individual peers over a long time period is an important system property with an impact on the content accessibility. We measured peer availability for over three months. On December 10, 2003, the popular PC game “Beyond Good and Evil” from UbiSoft was injected into BitTorrent/Suprnova.org and on March 11, 2004 it died. We followed this content and obtained 90,155 peer IP numbers using our GetPeers scripts. Of these IP numbers, only 53,883 were not behind firewalls and could be traced by our PingPeers scripts. We measured the uptime of all non-firewalled peers with a one minute resolution. Figure 2.8 shows the results of these measurements. Here we plot the
20 40 60 80 100 120 140 160 1 10 100 1000
File lifetime in days
Number of seeds after 10 days
Figure 2.9: The content lifetime versus the number of seeders after 10 days for over 14,000 files (Suprnova.org).
peer uptime in hours after they have finished downloading and stayed online for at least one hour, with the peers ranked according to decreasing uptime. The longest uptime is 83.5 days. Note that this log-log plot shows an almost straight line between peer 10 and peer 5,000. The sharp drop after 5,000 indicates that the majority of users disconnect from the system within a few hours after the download has been finished. This sharp drop has important implications because the actual download time of this game spans several days. Seeders with a high availability are rare. Only 9,219 out of 53,883 peers (17%) have an uptime longer than one hour after they finished downloading. For 10 hours this number has decreased to only 1,649 peers (3.1%), and for 100 hours to a mere 183 peers (0.34%).
In BitTorrent, the availability of content is unpredictable. When the popularity drops and the last peer/seeder with certain content goes offline, the content dies. Figure 2.9 shows the content lifetime of all large files (at least 500 MB) on BitTorrent/Suprnova.org we have followed. Each file is represented as a data point with on the horizontal axis the number of seeders for the file 10 days after its injection time, and on the vertical axis its content lifetime. An important observation is that the number of seeders is not an accurate predictor for the content lifetime. Files with only a single seeder can still have a relatively long content lifetime surviving many files with several hundreds of seeders.
2.4 Measurement results 23 0 100000 200000 300000 400000 500000 600000 12/06/03 12/13/03 12/20/03 12/27/03 01/03/04 01/10/04 Number of downloads Time [month/day/year] Web site mirrors fail
tracker fails tracker fails WebCrawler fails all movies games music
Figure 2.10: The number of users downloading or seeding content of different types (Suprnova.org).
2.4.3
Social heterogeneity
The social heterogeneity is characterized in our measurements with the data describing the variation of the number of downloads over time and the activity of moderators and content submitters. The high level of detail of the collected statistics allows to categorize the downloads based on the type of the downloaded content.
The number of users over time on BitTorrent/Suprnova.org gives a good indication of both the general performance and the dynamics of the system. We show the popu-larity of BitTorrent/Suprnova.org in terms of the number of downloads over time and its dependence on technical failures in the system. Figure 2.10 shows the total number of downloads, and the number of downloads of three types of content (games, movies, and music) in progress in BitTorrent around Christmas 2003. We have selected this month for presentation because it shows a large variance in the number of downloads due to sev-eral BitTorrent/Suprnova.org failures. The consecutive data points have been connected with a line when there was no overall systems failure. The lowest and highest number of downloads in Figure 1 are 237,500 (on Christmas day) and 576,500 (on January 9). Our WebCrawler script requests all pages every hour from one of the active Suprnova.org mirrors. There are three things to be noted in Figure 2.10. The first is the daily cycle; the minimum and maximum (at around 23:00 GMT) number of downloads occur at roughly the same time each day, which is similar to the results found in [119]. The second is the clear difference in the popularity of the content of different types; movies are downloaded more frequently than games which are more popular than music. The third is the large
0 500 1000 1500 2000 2500 3000 3500 4000 12/21 12/23 12/25 12/27 12/29 12/31 01/02 01/04 Number of downloads Time [month/day] seeds PingPeers
tracker and Suprnova.org
Tracker FutureZone.TV Suprnova.org HTML PingPeers scripts seeders only
Figure 2.11: Flashcrowd effect of “Lord of the Rings III” (Suprnova.org).
variation due to failures of either the mirroring system across the Suprnova.org mirrors, of the mirrors themselves, of the torrent servers, or of the trackers. For example, on De-cember 8 and 10, gaps occurred due to failures of the mirroring system and of 6 out of 8 Suprnova.org mirrors. On Christmas day, a large tracker went offline for 98 hours. The failure of this single tracker alone reduced the number of available movies from 1675 to 1017, and resulted in a sharp reduction in the number of downloads. From January 5 to 10, the mirroring system was also offline a few times, causing Suprnova.org to be unusable and the mirrors not being updated, which is visible in the figure as a few gaps in the “all” line. The figure suggests that users are not discouraged by such failures. We conclude that the number of active users in the system is strongly influenced by the availability of the global components.
We now focus on the system’s reaction to the sudden popularity of a single (new) file, a phenomenon called the flashcrowd effect. Figure 2.11 shows the number of downloads for a single file as a function of time (the “Lord of the Rings III” movie with size 1.87 GB). We have selected this file because it uses a tracker (FutureZone.TV) which provides access to detailed statistics, which we collected every five minutes with our GetPeers script. The statistics from Suprnova.org were fetched by WebCrawler every hour, and are in agreement with the total tracker results to such an extent that the corresponding lines in Figure 2.11 overlap almost completely. The bottom two lines show the number of peers contacted by the PingPeers scripts and the number of seeders as reported by the tracker. During the first five days, no peer finished downloading the file and the injector of the file was continuously online. This long time period provides a clear opportunity
2.4 Measurement results 25 1 10 100 1000 1 10 100 1000 10000
Amount of content inserted [.torrents]
User ranking Moderated submitters
Unmoderated submitters
Moderators
Figure 2.12: The activity of different content submitters on Suprnova.org to prevent pol-lution (Suprnova.org).
to identify copyright violators who inject the initial copy of the file into the network. Only on December 23, 2003 there was a problem with the tracker for a few minutes, which is not visible in the Suprnova.org data. The results from the PingPeers script show a significantly lower number of downloads, which is due to the firewall problem (40% of the peers were firewalled). The gaps in the PingPeers results were due to disk quota problems on the measurement framework. From the measurements we conclude that the global BitTorrent/Suprnova.org components are capable of efficiently handling very large flashcrowds.
We have also evaluated the integrity of both the content itself and of the associated metadata, which is a notorious problem in P2P systems. In order to test the integrity of metadata, we donated to Suprnova.org an account for hosting a mirror. By installing a logging feature in the HTML code, we have registered each torrent download and could have easily corrupted the metadata. We conclude that using donated resources for hosting metadata entails substantial integrity and privacy risks. As to the integrity of the content, P2P message boards and other sources strongly indicate that BitTorrent/Suprnova.org is virtually pollution free. However, a direct measurement of fake or corrupted files is diffi-cult; manually checking the content of many files is not really a viable option. Instead, we actively tried to pollute the system. We created several accounts on different computers from which we tried to insert fake files. We failed; the moderators filtered out our fake files. The system of moderators seems to be very effective in this sense.
needed. Figure 2.12 shows the numbers of files that have been injected by the 20 mod-erators, the 71 unmoderated submitters, and the 7,933 moderated submitters that were active between June 2003 and March 2004. The ten most active moderated submitters injected 5,191 files, versus 1,693 for the unmoderated submitters and 274 for the mod-erators. We are surprised that a mere 20 moderators were able to effectively manage the numerous daily content injections with such a simple system. Unfortunately, this system of moderation relies on global components and is extremely difficult to distribute.
2.5
Summary
In this chapter, we have characterized the heterogeneity aspects of the Internet environ-ment relevant for P2P systems based on a measureenviron-ment study of a large P2P network. For the study we have selected the BitTorrent system which, at the time of writing this thesis, was the most popular P2P system with the most significant share in the Internet backbone traffic. To meet the scale requirements of the measurement, we have designed a distributed framework able of collecting data from individual peers. With this framework we have gathered data describing the resource, the configuration and the social hetero-geneity.
The analysis of the collected data reveals significant differences in peer capacities, network configurations and user behavior patterns which could be exploited by P2P ar-chitectures. The heterogeneity of peer bandwidth settings suggests a need for data distri-bution protocols able to fully utilize available link capacities. The direct dependency of the content availability on the seeds should stimulate the research on seeding incentives which are currently not present in BitTorrent. Central components for content and peer location in a form of Web sites and trackers provide a high level of metadata integrity with only a handful of moderators and allow for semantic categorization of the content. However, this comes at a price: system availability and scalability is hampered by the global nature of these components. Decentralization of the moderation, location and se-mantic categorization services is an obvious direction in the evolution of P2P systems like BitTorrent.
27
Chapter 3
Collaborative bandwidth exchange
P2P systems that rely on the voluntary contribution of bandwidth by the individual peers may suffer from freeriding. To address this problem, incentive mechanisms enforcing fair-ness in resource sharing have been designed. The principle of the incentive mechanisms in current content sharing P2P networks such as BitTorrent is to have peers exchange con-tent of mutual interest. As a consequence, a peer can actively participate in the system only if it shares content that is of immediate interest to other peers. This restriction can be lifted by using bandwidth rather than content as the resource upon which incentives are based. Bandwidth, in contrast to content, is independent of peer interests and so can be exchanged between any two peers.
Incentives for bandwidth exchange between peers can be provided by means of a pro-tocol designed for this task. In this chapter, we present the design of such a propro-tocol called amortized tit-for-tat (ATFT) and discuss its application to file sharing, live stream-ing and video on demand (VoD). ATFT uses the mechanism of collaborative downloads which allows to form groups of peers that collaborate in downloading a file on behalf of a single group member. A peer in ATFT can use its currently idle bandwidth to help other peers in their ongoing downloads, and get in return help during its own downloads. In this sense, a peer’s bandwidth contributions are amortized over time with the consump-tion of the bandwidth of other peers. We find that with realistic bandwidth limit settings, ATFT improves the download speed by up to a factor of 3.5 in comparison to BitTorrent. The remaining part of this chapter is organized as follows. Section 3.1 introduces the concept of bandwidth exchange as the alternative to content exchange. Section 3.2 dis-cusses the related work. Section 3.3 describes the realization of the bandwidth exchange concept by the ATFT protocol. Sections 3.4 and 3.5 present the results of an analytical and an experimental evaluation of ATFT. Section 3.6 summarizes this chapter.
