ANycast.V2.3

Review & intRoduction

AnycAsting & PiAs towARds A globAl iP AnycAst seRvice by HitesH bAllAni, PAul FRAncis & discussion oF otHeR AnycAsting exPeRience

Rahat AT Anycast Proxy (AP)

RAP JAP Address Anycast Client (AC)

IAP

JAP Anycast Target (AT)

outline

RFID & Discussion on Papers

outline The AnyCast Concept The PIAS Paper PIAS: Basic Idea What does PIAS solve A few Design details

Some Other Discussion & Reference

tHe AnycAst concePt inteRnet & FRom 6000 level couRse

AnycAst tecHnology “Innovative door towards communication�

IP Anycast, with its innate ability to find nearby resources in a robust and efficient fashion, has long been considered an important means of service discovery. The growth of P2P applications presents appealing new uses for Anycast.

AT Anycast Proxy (AP)

RAP JAP Address

IAP Anycast Client (AC)

JAP Anycast Target (AT)

AnycAst tecHnology: bAsic concePt

Anycast is a network addressing and

routing scheme whereby data is routed to the "nearest" or "best" destination as viewed by the routing topology The term is intended to echo the terms unicast, broadcast and multicast.

AnycAst tecHnology: bAsic concePt In Unicast, there is a one-to-one association between network address and network endpoint: each destination address uniquely identifies a single receiver endpoint

In Broadcast and Multicast, there is a one-to-many association between network addresses and network endpoints: each destination address identifies a set of receiver endpoints, to which all information is replicated.

Unicast

Broadcast

Multicast

AnycAst tecHnology: bAsic concePt

In Anycast, there is also a one-tomany association between network addresses and network endpoints: each destination address identifies a set of receiver endpoints, but only one of them is chosen at any given time to receive information from any given sender

Anycast

In the past, Anycast was suited to connectionless protocols (generally built on UDP), rather than connection-oriented protocols such as TCP that keep their own state. But Anycast is generally used as a way to provide high availability and load balancing for stateless services such as access to replicated data; for example, DNS service is a distributed service over multiple geographically dispersed servers

exAmPle

Router 2 Client

Server Instance A

Router 1 Router 3

Router 4

Server Instance B

Example provided through PCH

exAmPle

192.168.0.1

10.0.0.1

Router 2 Client

Server Instance A

Router 1 Router 3 192.168.0.2

Router 4

Server Instance B 10.0.0.1

exAmPle

192.168.0.1

10.0.0.1

Router 2 Client

Server Instance A

Router 1 Router 3 192.168.0.2

DNS lookup for http://www.server.com/ produces a single answer: www.server.com.

IN

A

10.0.0.1

Router 4

Server Instance B 10.0.0.1

exAmPle

192.168.0.1

10.0.0.1

Router 2 Client

Server Instance A

Router 1 Router 3 192.168.0.2

Mask /29 /32 /32

Next-Hop 127.0.0.1 192.168.0.1 192.168.0.2

Server Instance B 10.0.0.1

Routing Table from Router 1: Destination 192.168.0.0 10.0.0.1 10.0.0.1

Router 4

Distance 0 1 2

exAmPle

192.168.0.1

10.0.0.1

Router 2 Client

Server Instance A

Router 1 Router 3 192.168.0.2

Mask /29 /32 /32

Next-Hop 127.0.0.1 192.168.0.1 192.168.0.2

Server Instance B 10.0.0.1

Routing Table from Router 1: Destination 192.168.0.0 10.0.0.1 10.0.0.1

Router 4

Distance 0 1 2

exAmPle

192.168.0.1

10.0.0.1

Router 2 Client

Server Instance A

Router 1 Router 3 192.168.0.2

Mask /29 /32 /32

Next-Hop 127.0.0.1 192.168.0.1 192.168.0.2

Server Instance B 10.0.0.1

Routing Table from Router 1: Destination 192.168.0.0 10.0.0.1 10.0.0.1

Router 4

Distance 0 1 2

exAmPle What the routers think the topology looks like: 192.168.0.1

Router 2 10.0.0.1

Client

Router 1

Server Router 3 192.168.0.2

Routing Table from Router 1: Destination 192.168.0.0 10.0.0.1 10.0.0.1

Mask /29 /32 /32

Next-Hop 127.0.0.1 192.168.0.1 192.168.0.2

Distance 0 1 2

Router 4

tHe PiAs PAPeR

towARds A globAl iP AnycAst seRviceon by HitesH bAllAni, PAul FRAncis

PRoxy iP AnycAst seRvice Ever since the it was proposed in 19931, IP anycast is termed as a powerful IP addressing mode that can be used for service discovery as well as direct support of RPC-like services (for instance DNS) and connectionless network services (for instance IPv6 transition). In spite of its potential usefulness, IP anycast is not widely deployed. The primary reason for this is the difficulty of deployment.

The Authors propose a Proxy IP Anycast Service (PIAS) that would make it easy for end-users to deploy anycast-based services.

With PIAS, IP anycast is deployed once as an infrastructure service. IP anycast destinations simply register with the nearest anycast proxy (which is discovered through IP anycast), and a tunnel is formed from the proxy to the anycast destination Ref: 1 RFC 1546 – Host Anycasting Service," by Partridge, Mendez et al. November 1993

About tHe AutHoRs

Hitesh Ballani Computer Science Dept Cornell University, New York, NY 14853 Sixth year graduate student in the Computer Science department at Cornell.

His research interests are in computer networks

Paul Francis Associate Professor Computer Science Dept.

Cornell University He joined the Computer Science department at Cornell in 2003 after many years in industry labs such as Bellcore, NTT Research Labs in Tokyo, and ACIRI Focused on network routing and addressing problems, with a particular interest in large and self-configuring systems

About tHe AutHoRs They made Number of Publications, Talks, Reports, etc. on Anycasting at numerous conferences. They are given chronologically – Publications Towards a Deployable IP Anycast Service

Hitesh Ballani and Paul Francis Proc. of First Workshop on Real, Large Distributed Systems (WORLDS'04) San Francisco, CA, Dec 2004. [pdf] [bibtex] Towards a Global IP Anycast Service

Hitesh Ballani and Paul Francis Proc. of ACM SIGCOMM (SIGCOMM'05) Philadelphia, PA, Aug 2005. [pdf] [bibtex] A Measurement-based Deployment Proposal for IP Anycast

Hitesh Ballani, Paul Francis and Sylvia Ratnasamy Proc. of Internet Measurement Conference (IMC'06) Rio de Janeiro, Brazil, Oct 2006. [pdf] [bibtex] (Best Paper Award)

Talks "Towards a Deployable IP Anycast Service", WORLDS'04 "IP Anycast : Affinity and Proximity Measurements", DNS-OARC'05 "Towards a Global IP Anycast Service", SIGCOMM'05 "A Measurement-based Deployment Proposal for IP Anycast", IMC'06 [pdf]

Technical Reports Understanding IP Anycast Hitesh Ballani and Paul Francis Cornell CIS Technical Report TR2006-2028, May 2006 [pdf]

PRoxy iP AnycAst PAPeR “Towards a Global IP Anycast Service” - (SIGCOMM'05)

The primary contribution of the authors’ paper is a new IP anycast architecture, PIAS which overcomes these problems while largely maintaining the strengths of IP anycast.

The paper also presents a detailed design of PIAS and evaluates its scalability and efficiency through simulation.

The authors also present preliminary measurement results on anycasted DNS root servers that suggest that IP anycast provides good affinity.

Towards a Global IP Anycast Service (SIGCOMM'05)

wHAt is iP AnycAst ?

CLIENT B

CLIENT A

packets destined to the anycast address are delivered to the nearest such host

packets destined to the anycast address are delivered to the nearest such host

MEMBER 2

One-to-Any communication with no changes to routing and clients

MEMBER 1

Assign the same IP address

to members of the group

Robust and efficient service discovery •Query-Reply Services : DNS Root-Servers etc. •Routing Services : IPv6 transition (6to4) etc.

But its use has been limited?

Towards a Global IP Anycast Service (SIGCOMM'05)

limitAtions oF inteR-domAin iP AnycAst

• Wastes address space • Does not scale by number of groups • Difficult to deploy obtain an address preffix a certain level of expertise

•Is limited by IP routing inability to offer load-based selection

Towards a Global IP Anycast Service (SIGCOMM'05)

some teRminology Join AP (JAP) - the nearby proxy with which the target registers; must monitor the health of its targets, determine when they are no longer available Ingress AP (IAP) - A proxy that receives packets directly from a client is referred as IAP; IAP relation is established using native IP anycast.

AT (AP)

RAP JAP Address

JAP

IAP

AC

Rendezvous Anycast Proxy (RAP) - For any given group, it designate a small number of APs (three or four) to maintain a list of JAPs for the group. When acting in this role, they are called the RAP

AT

Towards a Global IP Anycast Service (SIGCOMM'05)

wHAt is PiAs A practical anycast deployment architecture • addresses native IP Anycast limitations • offers new features • opens new anycast usage avenues Key Insight

CLIENT

Native IP Anycast

PIAS

MEMBER

Towards a Global IP Anycast Service (SIGCOMM'05)

PiAs: bAsic ideA Deploy Anycast Proxies All proxies advertise the same prefix

Anycast Proxy (AP)

Towards a Global IP Anycast Service (SIGCOMM'05)

PiAs: bAsic ideA Group Members register with proxies

IP Anycast Member (group 1) Anycast Proxy (AP)

Towards a Global IP Anycast Service (SIGCOMM'05)

PiAs: bAsic ideA Client (C) => Group 1 (green group)

C

C

Client IP Anycast Member (group 1) Anycast Proxy (AP)

Towards a Global IP Anycast Service (SIGCOMM'05)

PiAs: bAsic ideA Native IP Anycast delivers packets to proxies

C

C

Client IP Anycast Member (group 1) Anycast Proxy (AP)

Towards a Global IP Anycast Service (SIGCOMM'05)

PiAs: bAsic ideA Proxies tunnel to appropriate member

C

IP Tunnel

C

Client IP Anycast Member (group 1) Anycast Proxy (AP)

Towards a Global IP Anycast Service (SIGCOMM'05)

PiAs: bAsic ideA Different client might go to a different member

C

IP Tunnel

C

Client IP Anycast

C

Member (group 1) Anycast Proxy (AP)

Towards a Global IP Anycast Service (SIGCOMM'05)

PiAs: bAsic ideA Multiple groups can register

IP Tunnel

C

Client IP Anycast Member (group 1) Anycast Proxy (AP) Member (group 2)

Towards a Global IP Anycast Service (SIGCOMM'05)

wHAt does PiAs solve?

• Address Usage • Effort Amortization • Ease-of-Use • Backwards Compatible • Selection Criteria

Efficient use of address space Thousands of groups per IP address in prefix Group address - [IP-Address]:[Port]

Towards a Global IP Anycast Service (SIGCOMM'05)

wHAt does PiAs solve?

• Address Usage • Effort Amortization • Ease-of-Use • Backwards Compatible • Selection Criteria

Amortization of effort Deployment effort spread across thousands of groups

Towards a Global IP Anycast Service (SIGCOMM'05)

wHAt does PiAs solve?

• Address Usage • Effort Amortization • Ease-of-Use • Backwards Compatible • Selection Criteria

Ease of join/leave No interaction with routing

Towards a Global IP Anycast Service (SIGCOMM'05)

wHAt does PiAs solve?

C

• Address Usage • Effort Amortization • Ease-of-Use • Backwards Compatible • Selection Criteria

No changes to clients just as native IP Anycast

Towards a Global IP Anycast Service (SIGCOMM'05)

wHAt does PiAs solve?

C

• Address Usage • Effort Amortization • Ease-of-Use • Backwards Compatible • Selection Criteria

Multiple selection criteria for example, load balance, proximity Group members can be clients for the group!

Towards a Global IP Anycast Service (SIGCOMM'05)

wHAt does PiAs solve?

• Address Usage • Effort Amortization • Ease-of-Use • Backwards Compatible • Selection Criteria

All this just by proxying?

• decoupled issues from routing • can be easily addressed in proxy infrastructure

Towards a Global IP Anycast Service (SIGCOMM'05)

PiAs : design cHAllenges Scalability by no. of groups, group size/dynamics no. of proxies

Robustness and fast-failover

Towards a Global IP Anycast Service (SIGCOMM'05)

A Few design detAils •

Scale by the number of groups  

All proxies cannot keep state for all groups Each group’s membership is tracked by a few designated proxies – Rendezvous Anycast Proxy (RAP) for the group AT Anycast Proxy (AP)

RAP JAP Address

JAP

Anycast Client (AC)IAP

•

Anycast Target (AT)

RAP

Scale by group size and group churn 

Add a tier to the membership management hierarchy  Join Anycast Proxy – the proxy contacted by the target when it joins the group  Feeds approximate number of targets associated with it to the group RAPs

JAP 1

AT

AT

JAP N

AT

AT

Towards a Global IP Anycast Service (SIGCOMM'05)

A Few design detAils AT Anycast Proxy (AP)

INITIAL PACKET PATH 4 SEGMENTS LONG

RAP JAP

JAP Address

IAP

Anycast Target (AT)

Anycast Client (AC)

AT Anycast Proxy (AP) RAP JAP IAP Anycast Client (AC)

Selection at the RAP and JAP allows us to offer high level features such as proximity and load balance

Anycast Target (AT)

SUBSEQUENT PACKET PATH – 3 SEGMENTS LONG

Towards a Global IP Anycast Service (SIGCOMM'05)

new AnycAst APPlicAtions Anycast service offered by PIAS •I practical •I easy-to-use •I scales by group number/size/dynamics •I group members can be clients too

Applications •I Peer discovery : network games, p2p applications etc.

•I Reaching an overlay network : querying OpenDHT, global RON, i3 etc.

Towards a Global IP Anycast Service (SIGCOMM'05)

PiAs : Possible PRoblems •I Stretch Stretch = PIAS path length/Direct path length

•I Affinity Same client to same ingress

•I Proximity Is native IP anycast based proximity useful?

Towards a Global IP Anycast Service (SIGCOMM'05)

APPlicAtion-lAyeR AnycAst is AlReAdy out tHeRe Other Advantages of PIAS • use for low-level protocols • proximity is a lot easier easier management • faster failover • no extra round-trip

Towards a Global IP Anycast Service (SIGCOMM'05)

summARy •I Practical anycast deployment architecture •I Addresses native IP and application-layer anycast limitations •I Opens new usage avenues Currently deploying PIAS & publicly usable in the near future http://pias.gforge.cis.cornell.edu

some otHeR discussion ReFeRence inteRnet & FRom otHeR PAPeR, ARticles

use oF AnycAst AddRessing in dns A number of the Internet root nameservers are implemented as clusters of hosts using anycast addressing. The C, F, I, J, K, L and M servers exist in multiple locations on different continents, using anycast address announcements to provide a decentralized service.

This has accelerated the deployment of physical (rather than nominal) root servers outside the United States. RFC 3258 documents how anycast is used to provide authoritative DNS services.

Akamai, Autonomica*Community DNS, DNSMadeEasy, Dynect, EasyDNS, Netriplex, Neustar Ultra Services (formerly UltraDNS), and many other authoritative name service providers have switched to an IP anycast environment to increase query performance and redundancy

woRldwide Root seRveR distRibution - beFoRe AnycAst

All root servers - not only F Ref: Jo達o Damas Internet System Consortium

woRldwide Root seRveR distRibution - now

All root servers - not only F Ref: Jo達o Damas Internet System Consortium

use oF AnycAst AddRessing in iPv6 tRAnsition In IPv4 to IPv6 transitioning anycast addressing may be deployed to provide IPv4 compatibility to IPv6 hosts. This allows multiple providers to implement 6to4 gateways without hosts having to know each individual provider's gateway addresses.

PeRFoRmAnce-tuning AnycAst netwoRks • Server deployment in anycast networks is always a tradeoff between absolute cost and efficiency.

• The network will perform best if servers are widely distributed, with higher density in and surrounding high demand areas.

• Lower initial cost sometimes leads implementers to compromise by deploying more servers in existing locations, which is less efficient.

exAmPle Geographic plot of user population density

exAmPle Geographic plot of user population density

Server deployment

exAmPle Geographic plot of user population density

Server deployment Traffic Flow

exAmPle Geographic plot of user population density

Server deployment Traffic Flow

exAmPle Geographic plot of user population density

Server deployment Traffic Flow

exAmPle Geographic plot of user population density

Server deployment Traffic Flow

exAmPle Drawing traffic growth away from a hot-spot

exAmPle Drawing traffic growth away from a hot-spot

exAmPle Drawing traffic growth away from a hot-spot

exAmPle Drawing traffic growth away from a hot-spot

exAmPle Drawing traffic growth away from a hot-spot

exAmPle Drawing traffic growth away from a hot-spot

Topological watershed

exAmPle Drawing traffic growth away from a hot-spot

some tHougHts Local and global nodes In some situations of anycast deployment on the Internet there is a difference between local and global nodes. Local nodes are often more intended to provide benefit for the direct local community. Local node announcements are often announced with the no-export BGP community to prevent peers from announcing them to their peers (i.e. the announcement is kept in the local area).

Where both local and global nodes are deployed, the announcements from global nodes are often Autonomous System prepended (i.e. the AS is added a few more times) to make the path longer so that a local node announcement is preferred over a global node announcement

deniAl oF seRvice AttAcks And AnycAst

Is there a chance that Anycast on the Internet can help to distribute DDoS attacks and reduce their effectiveness. As traffic is routed to the closest node, a process over which the attacker has no control, the DDoS traffic flow will be distributed amongst the closest nodes. This often means that not all nodes will be affected. This is often an important reason to deploy anycast addressing. But more research needed to be done know or to be sure of this type of questions.

ReFeRence http://pias.gforge.cis.cornell.edu/publications.php http://en.wikipedia.org/wiki/Anycast http://www.planet-lab.org/taxonomy/term/17 https://gforge.cis.cornell.edu/plugins/scmcvs/cvsweb.php/web/publications.php? rev=1.15;cvsroot=cvsroot%2Fpias;sortby=log http://www.cse.ohio-state.edu/~prasun/888/assigned.htm http://www.caida.org/workshops/wide/0503/slides/050311.wide-anycast.pdf http://www.caida.org/workshops/dns-oarc/200507/slides/oarc0507-Terzis.pdf http://www-users.itlabs.umn.edu/classes/Fall-2007/csci5211/lecture%20notes/csci5211Mobile-IP-multicast-anycast-6up.pdf http://www.authorstream.com/presentation/Cannes-14736-dns-pres-minda-bgp-anycastMeasurement-effects-experiences-JPFrom-query-jp-Status-new-york-Entertainment-pptpowerpoint/ http://www.uknof.org.uk/uknof2/Damas-root-servers.pdf http://www.pch.net/resources/papers/ipv4-anycast/ipv4-anycast.ppt http://www.pch.net/resources/papers/anycast-services/DNS-Hosting-v09.ppt http://www.sanog.org/resources/sanog5-woody-anycast-v10.pdf