Philmyers by IABM

BCE IP Broadcast Facility Phil Myers - Product Manager Snell Advanced Media

Introduction to BCE and RTL • BCE (Broadcasting Centre Europe) is the in-house systems integration division of the RTL (Radio Television Luxembourg) group of companies • Major European telecom, ISP, IT service provider that offers technical services to more than 400 clients in Television, Radio, Telecommunications and IT across more than 15 countries • RTL Group is a leading European entertainment network, with interests in 60 television channels and 31 radio stations across 10 countries – a pioneer in European radio and television

The “ • Project to consolidate all of its operations at its new headquarters in Luxembourg • Consists of seven new buildings with a combined cost of 105 million euros • Original decision was taken in 2013 to build the new premises when the RTL group realised the need to invest in more advanced technology in order to expand its business operations • The existing site in Luxembourg has already been sold for property development = no margin for delay in the project!

” Project

Project Requirements • New buildings must be future proof and able to support RTL and new prospects alike for a minimum of 10 years post build; adapting to new workflow challenges as required

• If possible: must be an all-IP production environment • SMPTE ST 2022-6/-7 (stream redundancy) • AES67 for audio signals in the same network • No separate master clock pulse distribution  PTP (IEEE1588) • Spine-Leaf network architecture preferred • System upgradeable to VSF TR03 • Independent consultancy and testing by IRT (Institut für Rundfunktechnik GmbH) • Phased project timeline with key milestones • On-air target date is Early 2017

Objectives and Concept • Flexibility  upgradeability and future expansion • Format agnostic  1080p, UHD, HDR

• Cost saving  less cabling, easy installation • Enabling of virtualised services • Highest reliability (no less than what has come before)

• No limitation to a single vendor (best in class products) • Key technological criteria: • Futureproof through a maximum use of existing standards and ready to accept emerging standards • Easier service and support through homogeneous structure • Taking advantage of the benefits of standard IT-based technology at an affordable price

• • • •

Validation of the network concept Validation of available broadcast IP technology Evaluation of the results - identify advantages, disadvantages and risk Suggestions for further proof of concept tests

• • • •

“PoI” with multiple vendors at IRT Multi-vendor network switch architecture Determine interoperability level, to allow for a multi-vendor solution Control management layer by Lawo VSM supported

NOVEMBER 2014

SEPTEMBER 2015

The Concept

Proof of Interoperability

JULY 2015 Proof of Concept

• • • •

“PoC” with multiple vendors at BCE Preparation of test cases and measurement Analysis and assessment of “PoC” test results Document results and present recommendations

Proof of Interoperability Proof of Interoperability test criteria: • SMPTE ST 2022-6 Interoperability • SMPTE ST 2022-7 “Hitless” Failover • Switching Performance • “Dirty” switching, recovery time • “Clean” switching – latency (errors) • Audio Interoperability • Dante & AES67 • MADI Input & Output • PTP (IEEE1588) SMPTE 2059-2 support

• SDN (Network Control & Northbound Interface)

VENDOR A

VENDOR D

VENDOR B

VENDOR E

VENDOR C

VENDOR F

IRT’s “PoI” Learnings • SMPTE ST 2022-6 is interoperable

• SMPTE ST 2022-7 is essential, but not implemented on all vendor products • AES67 and PTP have come a long way, Dante/Ravenna need to interoperate • Overall latency might be an issue for live production • Standard Definition (SD) format must still be considered • Spine-Leaf architecture reduces size of single components, but brings complexity • Clean switching is not mandatory for all signals and could save network bandwidth • On-going innovation necessary – upgradeability to TR03

• Installation to start with SMPTE ST 2022-6/-7 and AES67 as a minimum • In general, various standards offered and technology not fully mature • Conclusion – more extensive “PoC” required with select vendors

• • • •

“PoI” with multiple vendors at IRT Multi-vendor network switch architecture Determine interoperability level, to allow for a multi-vendor solution Control management layer by Lawo VSM supported

NOVEMBER 2014

SEPTEMBER 2015

The Concept

Proof of Interoperability

• • • •

JULY 2015

DECEMBER 2015

Proof of Concept (pt. 1)

Proof of Concept (pt. 2)

“PoC” with multiple vendors at BCE Preparation of test cases and measurement Analysis and assessment of “PoC” test results Document results and present recommendations

• • •

Extensive “PoC” with select vendors at BCE Final decision to be taken on technology and standards Pilot and acceptance test criteria to be defined

Having been successfully awarded the contract, the customers design then had to be scaled out and delivered

Network connectivity planned across several floors/buildings

Large amount of SDI & MADI to IP conversion required

Native IP devices required

Centralised core switches with LR Optics to edge devices

Audio de-mux and mux required for every edge device

Mixture of 10G & 40G IP native devices

Technical Challenges • Network topology and connectivity

• System timing • Implementing VSF TR04

• Switching performance with a COTS network switch • Migrating to VSF TR03 (SMPTE ST 2110)

• Control and Monitoring System

Network and Connectivity • Edge devices based on 40G technology (IP Gateways, Production Switcher, etc.). 10G connectivity accommodated via “Breakout mode” 1 x 40G to 4 x 10G” (GV Cameras, Harmonic Playout Server, Studer Audio Mixer, etc.) • Customer insisting on different switch vendors (hardware and software redundancy) • Data plane performance variance must be within buffer range of edge device receiver in order to provide ST 2022-7 redundancy

•

SMPTE ST 2022-7 Seamless protection of ST 2022 IP Datagrams

•

Originally for SMPTE ST 2022-6 streams. Good for TR-03 RTP Video, AES67 Audio & Metadata streams

•

Requires copy of Multicast Source

•

Two identical network interfaces

•

Two IP Routers or two IP Router cards

•

Packets received / joined at Host

•

Packet-by-packet merging / arbitration

•

Works for individual packet or full stream loss

Edge Device Host Transmitter

Edge Device Host Receiver Port 1

Port 1

Port 2

Packet Merge / Arbitrate

Buffer / Delay 1 Frame / 20ms

• Customer insisting on different switch vendors (hardware and software redundancy) • Data plane performance variance must be within buffer range of edge device receiver in order to provide ST 2022-7 redundancy • Monolithic “core” network switches preferred • Removes complexity as well as the associated blocking/hashing issues related to Spine-Leaf Architecture • Long range optics deployed due to centralised core switch approach • QSFP+ transceivers, four CWDM lanes, 1310nm single mode using LC

System timing • Redundant PTP Grandmaster with Tri-Level / Black Burst support and changeover unit deployed – Tektronix SPG8000A with ECO8000 • Meinberg LANTIME M3000 used to distribute PTP to the edge devices (slaves) • SMPTE ST 2059-2 profile used across all edge devices • Transparent clock mode implemented using “Hybrid” mode on edge devices • Boundary clock mode not considered due to network switch vendor roadmap timescales

PTP Grandmaster

Tektronix SPG8000A

Camera

BB / TL Slave

PTP Slave

* Slave

Meinberg LANTIME M1000

Production Switcher Multi-format SDI references 1 x PTP (RJ45), 1 x PTP(SFP) Free-run or Genlock GPS option

1. Multicast to Edge Devices 2. Unicast to PTP Grandmaster * Reduces multicast traffic, protects edge devices

Modular (No SDI) Up to 4 PTP ports/modules Free-run or Genlock GPS option *M1000 can be slaved to SPG8000A to circumvent high client count issues

PTP Slave Target ΔT between all devices: ≤ 1.0 microsecond (Lock time < 5s) IP Gateways

Implementing VSF TR04 • Standardisation and interoperability benefits of SMPTE ST 2022-6 • Separate essence stream for audio generated by each edge device – 16 channels or 64 channels @ 125us • Audio bandwidth is double per signal (2022-6 + AES67) – has little or no impact on 40G connected edge devices. IQMIX gateway, only 8 x 3G = 24G usage per QSFP+ • AES67 native devices provide a limited number of channels (typically 8), packet time not to exceed 125us (latency) and must provide stream redundancy

• MADI to AES67 (IQAMD) conversion provided - 64 channels @ 125us • Audio routing and shuffling across multiple AES67 streams possible

SDI generated AES67 audio stream configured as 16 ch. @ 125us

ST 2022-6

AES67 (16 ch.)

SDI

40G IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI

IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI 40G

ST 2022-6 SDI

Rx IQAMD (IP Gateway – MADI) 8 x MADI to IP and 8 x IP to MADI

MADI

AES67 (64 ch.)

MADI generated AES67 audio stream configured as 64 ch. @ 125us

Rx 10G

AES67 (16 ch.)

SDI generated AES67 audio stream configured as 16 ch. @ 125us

ST 2022-6

AES67 (16 ch.)

SDI

40G IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI

IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI 40G

ST 2022-6 SDI

AES67 (16 ch.)

IQAMD (IP Gateway – MADI) 8 x MADI to IP and 8 x IP to MADI

MADI

AES67 (64 ch.)

MADI generated AES67 audio stream configured as 64 ch. @ 125us

Edge Device receiver will only take first 16 channels of AES67 stream (if more channels are present, they are ignored)

Rx 10G

Only a single AES67 stream can be connected to the Edge Device receiver at any given time

SDI generated AES67 audio stream configured as 16 ch. @ 125us

ST 2022-6

AES67 (16 ch.)

SDI

40G IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI

IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI 40G

ST 2022-6 SDI

AES67 (16 ch.)

IQAMD (IP Gateway – MADI) 8 x MADI to IP and 8 x IP to MADI

MADI

AES67 (64 ch.)

Rx 10G

Tx 3 x AES67 Receivers (144 ch.)

MADI generated AES67 audio stream configured as 64 ch. @ 125us

10G

Audio XS (Audio Routing & Processing) 64 x AES67 receivers (4096 ch.) 64 x AES67 transmitters (4096 ch.)

1 x AES67 Transmitter (16 ch.) Audio “shuffled” in Audio XS

COTS Switching Performance • Juniper and Arista switches running L3 with PIM-SSM enabled

• No SDN intervention required – network or broadcast • Destination based switching on the Edge Devices using IGMPv3

• IGMP “join” requests actioned within an average of 25ms • All “SAM” edge devices behave in exactly the same way (i.e. IP Gateways, Production Switcher, Servers, Multiviewers, etc.) • Two operational modes of “clean” switching available with or without frame-sync (low-latency mode)

Break-before-Make Clean, Very fast & Visibly undetectable (One frame repeat) Good for 95%+ of applications! Stream A

Stream B

Stream D

Make!

Stream A

Stream C

Break!

Tx Stream C

Stream E

Stream D

Stream C

Stream E

Stream B

Stream C

Stream D Spare BW

Make!

Break!

Stream A

Stream B

Stream C

Stream D

Stream E

Make-before-Break ‘Clean’ (Switches on frame boundary) Bandwidth burden for switch duration Stream A

New

Stream D New

Stream B Stream C

Stream D New

Preferred mode set for each Destination

Migrating to VSF TR03 • Important to understand, more flows to manage (control system) – need to consider if the network switch (ASIC) can handle the increased number of multicast traffic • Supporting both VSF TR04 (ST 2022-6 & AES67) and TR03 in the same system a must for seamless migration – requirement for 3rd party edge devices •

Edge device “transmitters” can be configured as ST 2022-6, RFC4175, AES67, etc.

•

Edge device “receivers” must operate in all modes – SAM edge devices using extended header in multicast stream (self described streams)

•

SAM edge devices deployed supporting TR03 (RFC4175 & AES67) today, software upgrade path to SMPTE ST 2110 planned

SDI generated AES67 audio stream configured as 16 ch. @ 125us

ST 2022-6

AES67 (16 ch.)

SDI

40G IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI

IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI 40G

ST 2022-6 SDI

Rx IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI

RFC 4175

Rx 40G

SDI AES67 (16 ch.)

SDI generated AES67 audio stream configured as 16 ch. @ 125us

AES67 (16 ch.)

SDI generated AES67 audio stream configured as 16 ch. @ 125us

ST 2022-6

AES67 (16 ch.)

SDI

40G IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI

IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI 40G

ST 2022-6 SDI

Rx IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI

RFC 4175

Rx 40G

SDI AES67 (16 ch.)

SDI generated AES67 audio stream configured as 16 ch. @ 125us

AES67 (16 ch.)

SDI generated AES67 audio stream configured as 16 ch. @ 125us

ST 2022-6

AES67 (16 ch.)

SDI

40G IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI

IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI 40G

ST RFC2022-6 4175 SDI

Rx IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI

RFC 4175

Rx 40G

SDI AES67 (16 ch.)

SDI generated AES67 audio stream configured as 16 ch. @ 125us

AES67 (16 ch.)

SDI generated AES67 audio stream configured as 16 ch. @ 125us

ST 2022-6

AES67 (16 ch.)

SDI

40G IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI

IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI 40G

RFC2022-6 4175 ST SDI

Rx IQMIX (IP Gateway – SDI) 8 x SDI to IP and 8 x IP to SDI

RFC 4175

Rx 40G

SDI AES67 (16 ch.)

SDI generated AES67 audio stream configured as 16 ch. @ 125us

AES67 (16 ch.)

Redundant Routers ‘Outband’ Edge Control

Both Controllers communicate with Edge Devices & each other! i.e. Controller Redundancy

Upstream Client Control Network

Controller 1

Device Control 1

GbE

Device Control 2

10GbE 10GbE

Controller 2

Juniper (Main)

Arista (Backup)

Device Control 1

GbE

Device Control 2

Downstream Device Control Network

Redundant Trunk

Edge Devices

Main Trunk Aggregation Gateway Card

sQ – Video Server

IQ EDGE - Processing

IQMIX (SDI) & IQAMD (MADI) Kahuna IP - Switcher Backup To Edge Device External Control Ports

MV820 - Multiviewer

Main

Control & Monitoring System • Control system designed to abstract the technology from the operators • Human interfaces must be familiar (hardware and software panels), operator should not care / or be aware if the underlying technology is SDI or IP • “Out-of-band” edge device control implemented in order to secure the media network • “Clean” and “Fast” switching with SDI like performance • Auto-discovery and registration of SAM edge devices within the control system • Bulk addressing of edge devices, removing the need for manual entry of source IP and multicast addresses (and then having to remember them to route signals) • Multi-level routing functionality to accommodated “Audio Breakaways”

Control & Monitoring System • Northbound “SWP-08” interface provided to Lawo Virtual Studio Manager (VSM)

• 3rd Party edge device drivers created for Studer audio desk (Dante), Grass Valley LDX camera (MCP) and Harmonic Electra X2 servers (NMX) • Critical monitoring of all core system components provided: • Juniper and Network switches via SNMP (i.e. chassis, fans, supervisors, line-cards, port bandwidth, etc.) • SAM edge device monitoring (i.e. signal / stream type, network traffic, link / redundancy status, SFP/QSFP+ status, etc.) • Log servers deployed to monitor the IP sub-system and provide a northbound interface to the Skyline DataMiner monitoring system

Lawo Virtual Studio Manager (VSM) : SWP-08 Protocol Skyline DataMiner : RollCall Protocol

LiveTouch & sQ Servers

IP Routing Controller & RollLog Server

Redundant Network Infrastructure â&#x20AC;&#x201C; 10, 40 & 100GbE Connectivity

AES67 ST 2022-6 ST 2022-7 TR04 TR03

Grass Valley Cameras

Kahuna IP Production Switcher

IQMIX & IQAMD SDI / IP Gateways MADI / IP Gateways

Audio XS Audio Processing

MV820 IP Multi-viewers

Studer Audio Mixing Console

Harmonic Playout Servers

Closing comments • SAM provided a full portfolio of IP enabled products to BCE for the RTL City project, this included – IQMIX (SDI), IQAMD (MADI), IQEdge (Processing) Kahuna IP (Production Switcher), MV-820 (Multiviewers) as well an IP routing control and monitoring solution • In addition, the Juniper QFX10008 and Arista 7508R series COTS switches were also supplied and configured by SAM as part of the project • Third party edge device control drivers for the Grass Valley (cameras), Harmonic (servers) and Studer (audio desk) were implemented by SAM • The RTL City project is great example of a TR04 deployment at scale (2,400+ IP video ports). Customer is currently on-air with phase 1 of the project

phil.myers@s-a-m.com

(Booth SL1805)