Campus Network Best Practices: Core and Edge Networks Dale Smith University of Oregon/NSRC dsmith@uoregon.edu This document is a result of work by the Network Startup Resource Center (NSRC at http://www.nsrc.org). This document may be freely copied, modified, and otherwise re-used on the condition that any re-use acknowledge the NSRC as the original source.
Campus Network Rules • • • • • •
Minimize number of network devices in any path Use standard solutions for common situations Build Separate Core and Edge Networks Provide services near the core Separate border routers from core Provide opportunities to firewall and shape network traffic
Core versus Edge • Core network is the “core” of your network – Needs to have reliable power and air conditioning – May have multiple cores – Always route in the core
• Edge is toward the edges of your network – Provide service inside of individual buildings to individual computers – Always switch at the edge
Minimize Number of Network Devices in the Path • Build star networks
• Not daisy chained networks
Edge Networks (Layer 2 LANs) • Provides Service to end users • Each of these networks will be an IP subnet • Plan for no more than 250 Computers at maximum • Should be one of these for every reasonable sized building • This network should only be switched • Always buy switches that are managed – no unmanaged switches!
Edge Networks • Make every network look like this: Fiber link to core router
Edge Networks Continued • Build Edge network incrementally as you have demand and money • Start Small: Fiber link to core router
Edge Networks Continued • Then as you need to add machines to the network, add a switch to get this: Fiber link to core router
Edge Networks Continued • And keep adding switches to get to the final configuration Fiber link to core router
Edge Networks Continued • And keep adding switches to get to the final configuration Fiber link to core router
Edge Networks Continued • Resist the urge to save money by breaking this model and daisy chaining networks or buildings together • Try hard not to do this: Fiber link to core router
Link to another building Link to adjacent building
Edge Networks Continued • There are cases where you can serve multiple small buildings with one subnet. • Do it carefully. Copper or fiber link to core router • Two basic models: Fiber link to core router
Switch in core location
Fiber circuits to small buildings Cat5e or fiber
Cat5e or fiber
Selected Layer 2 Topics • • • • • • •
Collision versus Broadcast Domain VLANs ARP – how it works DHCP - How it works Spanning Tree Link Aggregation Failure modes – 100 Mbs and Gigabit Duplex mismatch
Collision vs. Broadcast Domain • Similar issues – affects performance of LAN • Hubs (Repeaters) – Every packet goes to every port, irrespective of destination of packet – Every port is half duplex – Can only be one packet in transit – two transmitters = Collision
Collision vs. Broadcast Domain • Hubs/Repeaters Hub
• •
Hub
Only One Packet at a time Every packet (even unicast) goes to every port
Collision vs. Broadcast Domain • Hubs/Repeaters Hub
Hub
Two Transmitters = Collision
Collision
Collision vs. Broadcast Domain • Switches – Switches learn where hosts are eavesdropping on traffic and building a forwarding table – Switches forward packets to correct port – Can only be many packets in transit – Broadcasts must go to all ports
Collision vs. Broadcast Domain • Switches Switch
• •
Switch
Many packets can be in flight – store and forward Unicast Packets go to intended destination
Collision vs. Broadcast Domain • Switches Switch
•
Switch
Broadcasts go to all ports (notice this looks like the hubs picture some slides ago)
Collision vs. Broadcast Domain • Switches Switch
•
Switch
Switches need to know about multicast
VLANs • Virtual LANs – reduce scope of broadcast domain and separate traffic • Tagging – identifying the VLAN associated with a packet. Ports are configured as Tagged or untagged. • Trunking – Carrying traffic for multiple VLANs on a single link. Must use tagging.
VLANs • Tagging on Trunks – must tag Single link carrying 3 VLANS
ARP • Address Resolution Protocol • Builds a mapping of IP address to Ethernet Address • ARP Protocol – Broadcast ARP Request (who has this IP?) – Owner of IP address in ARP Request issues ARP reply
• Pathology: anyone can issue an ARP reply at any time
ARP
10.0.0.1 00:00:11:00:00:aa
10.0.0.2 00:00:11:00:00:bb
10.0.0.3 00:00:11:00:00:cc
DHCP • Dynamic Host Configuration Protocol • Used to assign IP address and provide basic IP configuration to a host. • Simple protocol – Client broadcasts a DHCP DISCOVER – Server(s) unicast back a DHCP OFFER – Client selects an offer and sends a REQUEST – Server sends back a DHCP ACK to client
• Managed switches can block rogue DHCP
Spanning Tree • Eliminates loops in Layer 2 networks • Several flavors – Original Spanning Tree 802.1D – Rapid Spanning Tree (RSTP) 802.1w – Multiple Spanning Tree (MSTP) 802.1s and 802.1Q-2003
• Modern managed switches can do all of the above • Lots of discussion about this Tuesday
Link Aggregation • Bonds multiple channels together to provide more bandwidth • Issues: – Compatibility – How traffic is scheduled 3 separate links aggregated as one
Failure Modes • • • •
ARP spoofing Loops in your network Rogue DHCP servers Duplex mis-match – 100Mbs – late collisions and CRC – 1000Mbs – can’t establish link
• Need managed switches to correct these
Core Network
Routing versus Switching Layer 2 versus Layer 3 • Routers provide more isolation between devices (they stop broadcasts) • Routing is more complicated, but also more sophisticated and can make more efficient use of the network, particularly if there are redundancy elements such as loops
Switching versus Routing These links must be routed, not switched
Core Network • Reliability is the key – remember many users and possibly your whole network relies on the core
• May have one or more network core locations • Core location must have reliable power – UPS battery backup (redundant UPS as your network evolves) – Generator
• Core location must have reliable air conditioning • As your network evolves, core equipment should be equipped with dual power supplies, each powered from separate UPS • Border routers separate from Core • Firewalls and Traffic Shaping Devices • Intrusion Detection • Intrusion Prevention • Network Address Translation
Core Network • At the core of your network should be routers – you must route, not switch. • Routers give isolation between subnets • A simple core: Border Router
Firewall/ Traffic Shaper
Core Router
All router interfaces on a separate subnet
Fiber optic links to remote buildings
Central Servers for campus
Where to put Servers? • Servers should be on a high speed interface off of your core router • Servers should be at your core location where there is good power and air conditioning Border Router
Firewall/ Traffic Shaper
Core Router
All router interfaces on a separate subnet
Fiber optic links to remote buildings
Servers in core
Border Router • Connects to outside world • RENs and Peering are the reason you need them • Must get Provider Independent IP address space to really make this work right Internet Exchange
REN
Campus Network
Putting it all Together Firewall/
Border Router
REN switch
Traffic Shaper
Core Router Core Servers
Fiber Optic Links
Fiber Optic Links
Notes on IP Addressing • Get your own Public IP address space (get your V6 block when you get your V4 one) • Make subnet IP space large enough for growth • Use DHCP to assign addresses to individual PCs • Use static addressing for switches, printers, and servers
More Complex Core Designs • One Armed Router for Core VLAN Trunk carrying all subnets Core Router
Core Switch Core Servers
Fiber Optic Links
Fiber Optic Links
Complex Core Designs • Multiple Core Routers Border Router
Firewall/ Traffic Shaper
Core Switch Local Internet exchange switch Core Router
Fiber Links to remote buildings
Core Router
Alternative Core Designs • Wireless Links versus Fiber Firewall/
Border Router
REN switch
Traffic Shaper
Core Router Core Servers
Fiber Optic Links Wireless Links
Layer 2 and 3 Summary • • • •
Route in the core Switch at the edge Build star networks – don’t daisy chain Buy only managed switches – re-purpose your old unmanaged switches for labs
Questions? This document is a result of work by the Network Startup Resource Center (NSRC at http://www.nsrc.org). This document may be freely copied, modified, and otherwise re-used on the condition that any re-use acknowledge the NSRC as the original source.
Symbols to use for diagrams