![[UNINETT]](uninett.gif)
Webcaching is storing copies of web documents close to the user, either as part of the functionality in the web client, or as separate webcaching servers. This document mainly discusses the use of several co-operating web cache servers in a mesh spanning different ISPs.
Webcaching is application level routing, and as web traffic forms a considerable part of the total traffic (mesurements in UNINETT indicate that 50% of traffic flowing in the network is web traffic) may influence the traffic pattern.
ISPs normally have established peering with other ISPs at a national and international level. Web caching heavily influences the flow of network traffic. ISPs may benefit from putting in place agreements with other ISPs on web caching, these may follow established agreements on peering and traffic exchange. Policies on co-operative web caching should be public.
An introduction to external routing issues is found in [ripe-081] and [ripe-181]. Following the terminology of [ripe-081] networks mean physical networks, layer 3 entities, not organisations. The organisations operating networks are called network operators. Network operators are divided into Internet Service Providers (ISP) and customers. An ISP is a network operator who operates a network to provide Internet services to different organisations, the customers. This distiction is not always clearcut, as an ISP may be the customer of another ISP, as is normally the case for international connections; or a customer may have customers (a university may regard its institutes as customers to the central university Internet infrastructure).
An Autonomous System (AS) is a group of IP networks having a single clearly defined routing policy which is run by one or more network operators. Inside an AS routing is done using one or more Interior Routing Protocols (today: OSPF [OSPF]), routing decisions are normally derived from technical parameters like topology, link speeds and load. ASes echange routing information with other ASes using Exterior Routing Policies (today: BGP4 [BGP4]), routing decisions are often based on policy rather than purely technical parameters.
Exchange of routing information between ASes is subject to routing policies, where routes are announced only by those willing to route traffic to a network, that information may and may not be accepted by the recipient of the announced routing information. Only if the announced routes are accepted does traffic flow, and in order for traffic to flow in both directions must routes between the two fringe networks be announced and accepted in both directions.
Multilevel webcaches are two or more co-operating webcaches, often arranged in a hierarchy where a first level webcache is close to the user (local webcaching) and upper level webcaches have first level webcaches as their clients (regional or national webcaches.
Web cache meshes are several co-operating servers that together form a mesh doing multilevel webcaching. A mesh may span a university or a network, or several networks. A first level webcache in a mesh has one or more neighbors and parents, misses are resolved through parents. A request for a document not in cache goes to neighbors and parents, using the Internet Cache Protocol (ICP).
One-level webcaching is a simple routing issue, all incoming web traffic is directed through a single point before it goes to the end users. Multilevel webcaching is a complex routing issue if more than AS is involved. Todays manual routing tables (like in Squid config files) of parents and neighbors may lead to several different routes for web traffic, and these may differ from the normal routes between users and servers. Building web caching meshes implements webrouting.
Webcaching meshes does route traffic, and if they span several ASes they need to implement webrouting both on a purely technical basis (with load, speed and topology as parameters) and on a policy level.
Interior routing is a technical problem with a purely technical solution, a technical solution to the building of webcahcign meshes has to be found. This problem is solvable, but does not neccesarily have a simple solution (if the solution is to be scaleable [W3C95]). Web traffic routing spanning several ASes need to consider the policy level in addition to the technical issues.
We should stop treating co-operative webcaching as interior routing and start paying attention to exterior routing policies. Todays practice cannot continue if webcaching is widespread, today webcaching does not constitute a threat to "normal" traffic flow as few meshes span several ASes. As we aim to change this, we will have to find a way for web caches to communicate policies.
Seen from the ISPs side there are several problems with webcaching
ISPs normally have established peering with other ISPs at a national and international level. Web caching heavily influences the flow of network traffic. ISPs may benefit from putting in place agreements with other ISPs on web caching, these may follow established agreements on peering and traffic exchange. Policies on co-operative web caching should be public.
Co-operative webcaching may reduce load on expensive and overloaded links, and sharing webcachign infrastructure may prove to be as benificiary as sharing other form of infrastructure (the case of NORDUnet is a good example of the benefits of sharing international infrastructure for more ISPs)
We will have to find a way for web caches to communicate policies on how they fit into meshes that are built within an AS or spanning several ASes. The Registration database at NLANR is a good starting point for distributing policies, as shown by the templates for the UK state who a cache is willing to be neighbor or parent to.
Meshes spanning more than one AS must consider the implications their mesh has for routing. Meshes contained in one AS may concentrate on issues of topology, link speed and load when they configure parents/neighbors, just like what is done in internal routing protocols (OSPF). Reconfiguration and joining a mesh should probably be done inside an AS, meshes spanning more than one AS should try to be as stable as possible.
As long as a web cache serves only local users at one location, network topology is a small issue. When you start to build meshes and hierarchies, topology comes into play. If you locate the servers of your meshes without consideration for the topology, you may end up generating more network traffic than without web caching. Some links are critical, one prime example is inter-continental links, another is international links. One reason for this is pricing, such links are normally very expensive, and international bandwith is not plentiful.
![[UNINETT map]](uninett100996.gif)
UNINETT backbone on September 10, 1996
Experiences from other applications than web calls network topology to attention. CUSeeMe is a multipart videoconferencing application for Macintoshes and PC, where connections are made to a reflector, the reflector sends video streams. One of the more popular reflectors is placed in Halden, Norway. When two US users talk to each other they often connect via the reflector in Halden, causing each video stream to cross the Atlantic twice. Changing this usage pattern has proved close to impossible (as long as the Halden server is better than US servers).
I cite this CUSeeMe example to show that if you let end users decide for themselves where to connect, certain parts of the network may more or less collapse. The other lesson to be learned from this is that if the persons who set up the service had known first thing about the network topology (and that their service was going to be extremely popular) they would at least have set up a reflector in a more central network point (which they did not have the permission to do, at the time), not at the very fringes of a network causing them to upgrade their connection continously. Setting up a server in a central point of the network needs permission, this may not always be easy to obtain.
![[Oslo-Halden image]](oslo-halden.0496.gif)
Scaling is one more issue where topology knowledge is needed in order to build services and meshes of servers. An ISP is likely to know both what links will be upgraded and where bottlenecks are going to persist - and ISPs might be able to influence link upgrades.
A webcache sitting by a stream of information, should chose its parents upstream, upstream being in the direction of the server it gets documents from. Make sure to use web caches that are actually upstream of the cache AND downstream of the information downloaded. Webtraffic flows from server to client, the first-level cache should be placed close to the client and upperlevel caches should be placed upstream. A first level web cache for an UNINETT institution should use as its parent an upper level web cache to get documents from outside Norway only if it knows that the upper level cache actually does provide access to documents outside Norway. It makes no sense for a web cache located close to Oslo to use a web cache in Tromsų as its parent for documents outside Norway when the international connection is in Oslo (and Tromsų is far away, even in networked terms).
Web caches should be placed where more networks join together [Bekker96], on campus or at an access point or at an Internet exchange point. Webcache servers should be placed as closely to the actual flow of webtraffic as possible. This means that the natural place for a national top-level cache is at the international connection, preferably at the same place as the national internet exchange point.
If there is a national Internet exchange point, this is a good location for webcaching. Either agree on running a common web cache, as has been successfully done in NewZealand [Neal96], or have agreements between web caches placed close to the exchange point. A factor that makes an Internet exchange point a good place for upper level webcaches, is that traffic exchange agreements are already in place and you may not need to draft another agreement on web traffic exchange.
These arguments also hold for international Internet exchange points. Several webcaches around an Internet exchange point will normally be set up as neighbors, unless one wants to route traffic through another network, this will normally not be the case.
In UNINETT, webcaching is one of several caching activites: alex and leafnode newsserver are two others. This is part of a strategy that aims to provide fast reliable access to Internet services for the Norwegian academic user community.
Experience from the Tromsų University gives a 60% hitrate for the end user, reducing web traffic to less than half. This is in a two-level webcache hierarchy, with 35-60% hitrates at the first level webcache and 15-25% hit rates at the second level webcache. Hitrates vary, the average seems to be 45-55% [Smith96, Neal96].
Webcaching speeds up response times for end users, especially for documents placed on slow servers or behind congested lines. In the worst case, the web documents are given to the end user after fetching them at the origin server with extra latency added by a chain of web caches (each web cache add a RTT to the cache, searching contents and timeout).
Experience shows that webcaching is percieved by the users as speeding up web access, best performance is for congested links [Smith96].
ISPs increasingly publish statistics on network traffic, this is needed to evaluate quality of service. Statistics of round trip times may help in location of web cache servers, and in finding bottlenecks in the network. In some situations response times are quick, but cost is high, and this information may be provided by the ISP.
We should stop treating co-operative webcaching as interior routing and start paying attention to exterior routing policies. Todays practice cannot continue it webcaching is widespread, today webcaching does not constitute a threat to "normal" traffic flow as few meshes span several ASes. As we aim to change this, we will have to find a way for web caches to communicate policies.
The lesson to learn from traditional routing is that purely technical routing decisions and policy decisions are different and may require two different sets of configuration and setups in order to work in an internet. If webcaching meshes span several ASes special consideration must be given to routing policies.
When a mesh aggregates web traffic, this aggregated traffic should be dealt with in close co-operation with the ISP. The two main reasons for this is topology knowledge and routing issues.
[Bekker96] Henny Bekker, Ton Verschuren, Ingrid Melve Requirements and Recommendations of Web caching services DESIRE Technical Report D4.1, April 1996
[Neal96] Donald Neal 1996 The Harvest Object Cache in New Zealand Computer Networks and ISDN Systems, volume 28, p 1415 - 1430
[ripe-081] Tony Bates, Jean-Michel Jouanigot, Daniel Karrenberg, Peter Lothberg, Marten Terpstra Representation of IP Routing Policies in the RIPE Database, February 1993
[ripe-181] Tony Bates, Elise Gerich, Laurent Joncheray, Jean-Michel Jouanigot, Daniel Karrenberg, Marten Terpstra, Jessica Yu Representation of IP Routing Policies in a Routing Registry (ripe-81++), October 1994
[Smith96] Neil G. Smith 1996 The UK national Web cache - The state of the art Computer Networks and ISDN Systems, volume 28, p 1407 -1414
[W3C95] The Web Consortium Problem Statement on Propagation, Replication and Caching
[BGP4] Yakov Rekhter, Tony Li A Border Gateway Protocol 4 (BGP-4), RFC 1654, July 1994
[OSPF] John Moy OSPF Version 2, RFC 1583, March 1994