Review & intRoduction
AnycAsting & PiAs towARds A globAl iP AnycAst seRvice by HitesH bAllAni, PAul FRAncis & discussion oF otHeR AnycAsting exPeRience
Rahat AT Anycast Proxy (AP)
RAP JAP Address Anycast Client (AC)
IAP
JAP Anycast Target (AT)
outline
RFID & Discussion on Papers
outline The AnyCast Concept The PIAS Paper PIAS: Basic Idea What does PIAS solve A few Design details
Some Other Discussion & Reference
tHe AnycAst concePt inteRnet & FRom 6000 level couRse
AnycAst tecHnology “Innovative door towards communication�
IP Anycast, with its innate ability to find nearby resources in a robust and efficient fashion, has long been considered an important means of service discovery. The growth of P2P applications presents appealing new uses for Anycast. AT Anycast Proxy (AP)
RAP JAP Address
IAP Anycast Client (AC)
JAP Anycast Target (AT)
AnycAst tecHnology: bAsic concePt
Anycast is a network addressing and
routing scheme whereby data is routed to the "nearest" or "best" destination as viewed by the routing topology The term is intended to echo the terms unicast, broadcast and multicast.
AnycAst tecHnology: bAsic concePt In Unicast, there is a one-to-one association between network address and network endpoint: each destination address uniquely identifies a single receiver endpoint
In Broadcast and Multicast, there is a one-to-many association between network addresses and network endpoints: each destination address identifies a set of receiver endpoints, to which all information is replicated.
Unicast
Broadcast
Multicast
AnycAst tecHnology: bAsic concePt
In Anycast, there is also a one-tomany association between network addresses and network endpoints: each destination address identifies a set of receiver endpoints, but only one of them is chosen at any given time to receive information from any given sender
Anycast
In the past, Anycast was suited to connectionless protocols (generally built on UDP), rather than connection-oriented protocols such as TCP that keep their own state. But Anycast is generally used as a way to provide high availability and load balancing for stateless services such as access to replicated data; for example, DNS service is a distributed service over multiple geographically dispersed servers
exAmPle
Router 2 Client
Server Instance A
Router 1 Router 3
Router 4
Server Instance B
Example provided through PCH
exAmPle
192.168.0.1
10.0.0.1
Router 2 Client
Server Instance A
Router 1 Router 3 192.168.0.2
Router 4
Server Instance B 10.0.0.1
exAmPle
192.168.0.1
10.0.0.1
Router 2 Client
Server Instance A
Router 1 Router 3 192.168.0.2
DNS lookup for http://www.server.com/ produces a single answer: www.server.com.
IN
A
10.0.0.1
Router 4
Server Instance B 10.0.0.1
exAmPle
192.168.0.1
10.0.0.1
Router 2 Client
Server Instance A
Router 1 Router 3 192.168.0.2
Mask /29 /32 /32
Next-Hop 127.0.0.1 192.168.0.1 192.168.0.2
Server Instance B 10.0.0.1
Routing Table from Router 1: Destination 192.168.0.0 10.0.0.1 10.0.0.1
Router 4
Distance 0 1 2
exAmPle
192.168.0.1
10.0.0.1
Router 2 Client
Server Instance A
Router 1 Router 3 192.168.0.2
Mask /29 /32 /32
Next-Hop 127.0.0.1 192.168.0.1 192.168.0.2
Server Instance B 10.0.0.1
Routing Table from Router 1: Destination 192.168.0.0 10.0.0.1 10.0.0.1
Router 4
Distance 0 1 2
exAmPle
192.168.0.1
10.0.0.1
Router 2 Client
Server Instance A
Router 1 Router 3 192.168.0.2
Mask /29 /32 /32
Next-Hop 127.0.0.1 192.168.0.1 192.168.0.2
Server Instance B 10.0.0.1
Routing Table from Router 1: Destination 192.168.0.0 10.0.0.1 10.0.0.1
Router 4
Distance 0 1 2
exAmPle What the routers think the topology looks like: 192.168.0.1
Router 2 10.0.0.1
Client
Router 1
Server Router 3 192.168.0.2
Routing Table from Router 1: Destination 192.168.0.0 10.0.0.1 10.0.0.1
Mask /29 /32 /32
Next-Hop 127.0.0.1 192.168.0.1 192.168.0.2
Distance 0 1 2
Router 4
tHe PiAs PAPeR
towARds A globAl iP AnycAst seRviceon by HitesH bAllAni, PAul FRAncis
PRoxy iP AnycAst seRvice Ever since the it was proposed in 19931, IP anycast is termed as a powerful IP addressing mode that can be used for service discovery as well as direct support of RPC-like services (for instance DNS) and connectionless network services (for instance IPv6 transition). In spite of its potential usefulness, IP anycast is not widely deployed. The primary reason for this is the difficulty of deployment. The Authors propose a Proxy IP Anycast Service (PIAS) that would make it easy for end-users to deploy anycast-based services. With PIAS, IP anycast is deployed once as an infrastructure service. IP anycast destinations simply register with the nearest anycast proxy (which is discovered through IP anycast), and a tunnel is formed from the proxy to the anycast destination Ref: 1 RFC 1546 – Host Anycasting Service," by Partridge, Mendez et al. November 1993
About tHe AutHoRs
Hitesh Ballani Computer Science Dept Cornell University, New York, NY 14853 Sixth year graduate student in the Computer Science department at Cornell. His research interests are in computer networks
Paul Francis Associate Professor Computer Science Dept. Cornell University He joined the Computer Science department at Cornell in 2003 after many years in industry labs such as Bellcore, NTT Research Labs in Tokyo, and ACIRI Focused on network routing and addressing problems, with a particular interest in large and self-configuring systems
About tHe AutHoRs They made Number of Publications, Talks, Reports, etc. on Anycasting at numerous conferences. They are given chronologically – Publications Towards a Deployable IP Anycast Service
Hitesh Ballani and Paul Francis Proc. of First Workshop on Real, Large Distributed Systems (WORLDS'04) San Francisco, CA, Dec 2004. [pdf] [bibtex] Towards a Global IP Anycast Service
Hitesh Ballani and Paul Francis Proc. of ACM SIGCOMM (SIGCOMM'05) Philadelphia, PA, Aug 2005. [pdf] [bibtex] A Measurement-based Deployment Proposal for IP Anycast
Hitesh Ballani, Paul Francis and Sylvia Ratnasamy Proc. of Internet Measurement Conference (IMC'06) Rio de Janeiro, Brazil, Oct 2006. [pdf] [bibtex] (Best Paper Award)
Talks "Towards a Deployable IP Anycast Service", WORLDS'04 "IP Anycast : Affinity and Proximity Measurements", DNS-OARC'05 "Towards a Global IP Anycast Service", SIGCOMM'05 "A Measurement-based Deployment Proposal for IP Anycast", IMC'06 [pdf]
Technical Reports Understanding IP Anycast Hitesh Ballani and Paul Francis Cornell CIS Technical Report TR2006-2028, May 2006 [pdf]
PRoxy iP AnycAst PAPeR “Towards a Global IP Anycast Service” - (SIGCOMM'05)
The primary contribution of the authors’ paper is a new IP anycast architecture, PIAS which overcomes these problems while largely maintaining the strengths of IP anycast. The paper also presents a detailed design of PIAS and evaluates its scalability and efficiency through simulation. The authors also present preliminary measurement results on anycasted DNS root servers that suggest that IP anycast provides good affinity.
Towards a Global IP Anycast Service (SIGCOMM'05)
wHAt is iP AnycAst ? CLIENT B
CLIENT A
packets destined to the anycast address are delivered to the nearest such host
packets destined to the anycast address are delivered to the nearest such host
MEMBER 2
One-to-Any communication with no changes to routing and clients
MEMBER 1
Assign the same IP address
to members of the group
Robust and efficient service discovery •Query-Reply Services : DNS Root-Servers etc. •Routing Services : IPv6 transition (6to4) etc.
But its use has been limited?
Towards a Global IP Anycast Service (SIGCOMM'05)
limitAtions oF inteR-domAin iP AnycAst
• Wastes address space • Does not scale by number of groups • Difficult to deploy obtain an address preffix a certain level of expertise
•Is limited by IP routing inability to offer load-based selection
Towards a Global IP Anycast Service (SIGCOMM'05)
some teRminology Join AP (JAP) - the nearby proxy with which the target registers; must monitor the health of its targets, determine when they are no longer available Ingress AP (IAP) - A proxy that receives packets directly from a client is referred as IAP; IAP relation is established using native IP anycast.
AT (AP)
RAP JAP Address
JAP
IAP
AC
Rendezvous Anycast Proxy (RAP) - For any given group, it designate a small number of APs (three or four) to maintain a list of JAPs for the group. When acting in this role, they are called the RAP
AT
Towards a Global IP Anycast Service (SIGCOMM'05)
wHAt is PiAs A practical anycast deployment architecture • addresses native IP Anycast limitations • offers new features • opens new anycast usage avenues Key Insight
CLIENT
Native IP Anycast
PIAS
MEMBER
Towards a Global IP Anycast Service (SIGCOMM'05)
PiAs: bAsic ideA Deploy Anycast Proxies All proxies advertise the same prefix
Anycast Proxy (AP)
Towards a Global IP Anycast Service (SIGCOMM'05)
PiAs: bAsic ideA Group Members register with proxies
IP Anycast Member (group 1) Anycast Proxy (AP)
Towards a Global IP Anycast Service (SIGCOMM'05)
PiAs: bAsic ideA Client (C) => Group 1 (green group)
C
C
Client IP Anycast Member (group 1) Anycast Proxy (AP)
Towards a Global IP Anycast Service (SIGCOMM'05)
PiAs: bAsic ideA Native IP Anycast delivers packets to proxies
C
C
Client IP Anycast Member (group 1) Anycast Proxy (AP)
Towards a Global IP Anycast Service (SIGCOMM'05)
PiAs: bAsic ideA Proxies tunnel to appropriate member
C
IP Tunnel
C
Client IP Anycast Member (group 1) Anycast Proxy (AP)
Towards a Global IP Anycast Service (SIGCOMM'05)
PiAs: bAsic ideA Different client might go to a different member
C
IP Tunnel
C
Client IP Anycast
C
Member (group 1) Anycast Proxy (AP)
Towards a Global IP Anycast Service (SIGCOMM'05)
PiAs: bAsic ideA Multiple groups can register
IP Tunnel
C
Client IP Anycast Member (group 1) Anycast Proxy (AP) Member (group 2)
Towards a Global IP Anycast Service (SIGCOMM'05)
wHAt does PiAs solve? • Address Usage • Effort Amortization • Ease-of-Use • Backwards Compatible • Selection Criteria
Efficient use of address space Thousands of groups per IP address in prefix Group address - [IP-Address]:[Port]
Towards a Global IP Anycast Service (SIGCOMM'05)
wHAt does PiAs solve? • Address Usage • Effort Amortization • Ease-of-Use • Backwards Compatible • Selection Criteria
Amortization of effort Deployment effort spread across thousands of groups
Towards a Global IP Anycast Service (SIGCOMM'05)
wHAt does PiAs solve? • Address Usage • Effort Amortization • Ease-of-Use • Backwards Compatible • Selection Criteria
Ease of join/leave No interaction with routing
Towards a Global IP Anycast Service (SIGCOMM'05)
wHAt does PiAs solve? C
• Address Usage • Effort Amortization • Ease-of-Use • Backwards Compatible • Selection Criteria
No changes to clients just as native IP Anycast
Towards a Global IP Anycast Service (SIGCOMM'05)
wHAt does PiAs solve? C
• Address Usage • Effort Amortization • Ease-of-Use • Backwards Compatible • Selection Criteria
Multiple selection criteria for example, load balance, proximity Group members can be clients for the group!
Towards a Global IP Anycast Service (SIGCOMM'05)
wHAt does PiAs solve? • Address Usage • Effort Amortization • Ease-of-Use • Backwards Compatible • Selection Criteria
All this just by proxying? • decoupled issues from routing • can be easily addressed in proxy infrastructure
Towards a Global IP Anycast Service (SIGCOMM'05)
PiAs : design cHAllenges Scalability by no. of groups, group size/dynamics no. of proxies
Robustness and fast-failover
Towards a Global IP Anycast Service (SIGCOMM'05)
A Few design detAils •
Scale by the number of groups
All proxies cannot keep state for all groups Each group’s membership is tracked by a few designated proxies – Rendezvous Anycast Proxy (RAP) for the group AT Anycast Proxy (AP)
RAP JAP Address
JAP
Anycast Client (AC)IAP
•
Anycast Target (AT)
RAP
Scale by group size and group churn
Add a tier to the membership management hierarchy Join Anycast Proxy – the proxy contacted by the target when it joins the group Feeds approximate number of targets associated with it to the group RAPs
JAP 1
AT
AT
JAP N
AT
AT
Towards a Global IP Anycast Service (SIGCOMM'05)
A Few design detAils AT Anycast Proxy (AP)
INITIAL PACKET PATH 4 SEGMENTS LONG
RAP JAP
JAP Address
IAP
Anycast Target (AT)
Anycast Client (AC)
AT Anycast Proxy (AP) RAP JAP IAP Anycast Client (AC)
Selection at the RAP and JAP allows us to offer high level features such as proximity and load balance
Anycast Target (AT)
SUBSEQUENT PACKET PATH – 3 SEGMENTS LONG
Towards a Global IP Anycast Service (SIGCOMM'05)
new AnycAst APPlicAtions Anycast service offered by PIAS •I practical •I easy-to-use •I scales by group number/size/dynamics •I group members can be clients too
Applications •I Peer discovery : network games, p2p applications etc. •I Reaching an overlay network : querying OpenDHT, global RON, i3 etc.
Towards a Global IP Anycast Service (SIGCOMM'05)
PiAs : Possible PRoblems •I Stretch Stretch = PIAS path length/Direct path length
•I Affinity Same client to same ingress
•I Proximity Is native IP anycast based proximity useful?
Towards a Global IP Anycast Service (SIGCOMM'05)
APPlicAtion-lAyeR AnycAst is AlReAdy out tHeRe Other Advantages of PIAS • use for low-level protocols • proximity is a lot easier easier management • faster failover • no extra round-trip
Towards a Global IP Anycast Service (SIGCOMM'05)
summARy •I Practical anycast deployment architecture •I Addresses native IP and application-layer anycast limitations •I Opens new usage avenues Currently deploying PIAS & publicly usable in the near future http://pias.gforge.cis.cornell.edu
some otHeR discussion ReFeRence inteRnet & FRom otHeR PAPeR, ARticles
use oF AnycAst AddRessing in dns A number of the Internet root nameservers are implemented as clusters of hosts using anycast addressing. The C, F, I, J, K, L and M servers exist in multiple locations on different continents, using anycast address announcements to provide a decentralized service. This has accelerated the deployment of physical (rather than nominal) root servers outside the United States. RFC 3258 documents how anycast is used to provide authoritative DNS services.
Akamai, Autonomica*Community DNS, DNSMadeEasy, Dynect, EasyDNS, Netriplex, Neustar Ultra Services (formerly UltraDNS), and many other authoritative name service providers have switched to an IP anycast environment to increase query performance and redundancy
woRldwide Root seRveR distRibution - beFoRe AnycAst
All root servers - not only F Ref: Jo達o Damas Internet System Consortium
woRldwide Root seRveR distRibution - now
All root servers - not only F Ref: Jo達o Damas Internet System Consortium
use oF AnycAst AddRessing in iPv6 tRAnsition In IPv4 to IPv6 transitioning anycast addressing may be deployed to provide IPv4 compatibility to IPv6 hosts. This allows multiple providers to implement 6to4 gateways without hosts having to know each individual provider's gateway addresses.
PeRFoRmAnce-tuning AnycAst netwoRks • Server deployment in anycast networks is always a tradeoff between absolute cost and efficiency. • The network will perform best if servers are widely distributed, with higher density in and surrounding high demand areas. • Lower initial cost sometimes leads implementers to compromise by deploying more servers in existing locations, which is less efficient.
exAmPle Geographic plot of user population density
exAmPle Geographic plot of user population density
Server deployment
exAmPle Geographic plot of user population density
Server deployment Traffic Flow
exAmPle Geographic plot of user population density
Server deployment Traffic Flow
exAmPle Geographic plot of user population density
Server deployment Traffic Flow
exAmPle Geographic plot of user population density
Server deployment Traffic Flow
exAmPle Drawing traffic growth away from a hot-spot
exAmPle Drawing traffic growth away from a hot-spot
exAmPle Drawing traffic growth away from a hot-spot
exAmPle Drawing traffic growth away from a hot-spot
exAmPle Drawing traffic growth away from a hot-spot
exAmPle Drawing traffic growth away from a hot-spot
Topological watershed
exAmPle Drawing traffic growth away from a hot-spot
some tHougHts Local and global nodes In some situations of anycast deployment on the Internet there is a difference between local and global nodes. Local nodes are often more intended to provide benefit for the direct local community. Local node announcements are often announced with the no-export BGP community to prevent peers from announcing them to their peers (i.e. the announcement is kept in the local area). Where both local and global nodes are deployed, the announcements from global nodes are often Autonomous System prepended (i.e. the AS is added a few more times) to make the path longer so that a local node announcement is preferred over a global node announcement
deniAl oF seRvice AttAcks And AnycAst
Is there a chance that Anycast on the Internet can help to distribute DDoS attacks and reduce their effectiveness. As traffic is routed to the closest node, a process over which the attacker has no control, the DDoS traffic flow will be distributed amongst the closest nodes. This often means that not all nodes will be affected. This is often an important reason to deploy anycast addressing. But more research needed to be done know or to be sure of this type of questions.
ReFeRence http://pias.gforge.cis.cornell.edu/publications.php http://en.wikipedia.org/wiki/Anycast http://www.planet-lab.org/taxonomy/term/17 https://gforge.cis.cornell.edu/plugins/scmcvs/cvsweb.php/web/publications.php? rev=1.15;cvsroot=cvsroot%2Fpias;sortby=log http://www.cse.ohio-state.edu/~prasun/888/assigned.htm http://www.caida.org/workshops/wide/0503/slides/050311.wide-anycast.pdf http://www.caida.org/workshops/dns-oarc/200507/slides/oarc0507-Terzis.pdf http://www-users.itlabs.umn.edu/classes/Fall-2007/csci5211/lecture%20notes/csci5211Mobile-IP-multicast-anycast-6up.pdf http://www.authorstream.com/presentation/Cannes-14736-dns-pres-minda-bgp-anycastMeasurement-effects-experiences-JPFrom-query-jp-Status-new-york-Entertainment-pptpowerpoint/ http://www.uknof.org.uk/uknof2/Damas-root-servers.pdf http://www.pch.net/resources/papers/ipv4-anycast/ipv4-anycast.ppt http://www.pch.net/resources/papers/anycast-services/DNS-Hosting-v09.ppt http://www.sanog.org/resources/sanog5-woody-anycast-v10.pdf