Benefits of Transparent Content Negotation in HTTP

Mark Stemm

Friday 9/4 1:25 pm 219 Phillips (note change in place and time)

Over the past several years, the popularity of the World Wide Web (WWW) has steadily increased. Unfortunately, as a result of overloaded servers and lack of bandwidth between clients and servers, response times observed by end users have also steadily increased. Response times are often so unbearable that the popular press has relabeled the "World Wide Web" the "World Wide Wait". In this work, we propose the use of content negotiation to allow clients and servers to tune response times to meet user demands. In our system, a client can negotiate content to match the bandwidth available between the server and itself. Clients obtain information about the available bandwidth from a SPAND performance prediction server. When the available bandwidth is low or the original page is large, the client requests a smaller version of the document to reduce the response time. Our experiments show that the client is able to identify the subset of transfers that will perform poorly and that transferring a smaller amount of information on these connections helps improve the overall response time. In our system, servers that are limited by the bandwidth leaving their site reduce the quality and thereby the size of the documents being transferred. This allows the servers to serve many more requests when they are under heavy load. Our measurements show that the benefits can be significant, but are very dependent on the ratio of negotiable to non-negotiable content. For example, in a typical mix of 60% image or other negotiable content, a server can increase the number of requests it handles by 50%. When 90% of the content is negotiable, a fourfold increase in Web server throughput is possible.

This is joint work with Randy Katz at UC Berkeley and Srini Seshan at IBM Research.