STF Feature: ONE HOP BEYOND

Creating Regional Connections From Data Centers Through Fiber Selection and High-Powered Amplifiers

BY WAYNE PELOUCH AND LEIGH FRAME

INTRODUCTION

In recent times the way subsea cables interconnect the worldwide data network has changed. Historically submarine systems were terminated in cable landing stations sited as close to population centers and to the landing locations as possible. This reflected the origins of the data flows which were – if we go back far enough – circuits connecting users, and then more recently, user or enterprise-originated data paths, possibly to a data center or another user or enterprise. If you had been able to drill down and interrogate the majority of data flows, they would have been a transaction recognizable to a human, perhaps a phone call, perhaps a demand for information or streaming data in real time. But then, sometime in our recent lifetime, the world changed…

It is only 20 years since Google built its first data center (DC), roughly coinciding with the founding of Equinix out of Digital Equipment Corporation. It is roughly 10 years since Facebook’s first DC, and only 15 years since the first iPhone. That is a complete data revolution in 20 years. The consequence is that what fills the pipes now is mostly machine-generated data flows based on algorithms. Recent data suggests that between 50 and 65% of all internet traffic is ‘bots’. Good bots would be aggregator crawlers, search engine crawlers, social network bots, shop bots and monitoring bots, while the best known bad-bot would be a Distributed Denial-of-Service (DDoS) botnet. Unfortunately, research suggests that those bad bots account for perhaps 50% more traffic than the good guys. These patterns of data traffic have led to industry and market revolutions – compare the customer base and funding sources for subsea cable systems today to twenty years ago, and to an imperative for technology to keep up!

Fig 1: Crossover cost point for repeatered vs unrepeatered links as distance increases, for 100Tb/s and 20Tb/s capacities.

When subsea cables connected people, the signals arrived at the cable landing station and from here the signal was regenerated and transmitted terrestrially inland towards PoPs and then DCs based in cities and hubs. Now that machines spend much of their time and energy (literally) exchanging with each other systems are primarily designed to connect Data Centers to Data Centers, directly facilitating an end-to-end network that as far as possible resembles one pane of glass. Those Data Centers are also in locations where proximity to people is almost secondary, compared to energy cost, green considerations, land cost, legal and physical security and other factors, which may dominate.

As the number of new trans-oceanic systems accumulates, they increasingly connect seemingly random places optimized around machines, geo-politics and perhaps the environment; regional connectivity needs to be added and improved to support onward transmission of critical data, both in-country and to Data Centers in neighboring countries. Decommissioning of existing multi-landing trans-oceanic systems, such as TAT-14 or CANTAT-3 results in a reduction of local links extending from regional hubs. Similarly, a number of existing unrepeatered links are approaching their end of life and can be replaced by much higher fiber count unrepeatered or SDM (Space Division Multiplexing) systems, utilizing modern higher quality fiber.

High optical performance and transmission capacity are achieved subsea because the newly deployed subsea fibers are low attenuation and often high effective area, and the amplifier spacing is at regular and relatively short intervals. However, this is typically not the case in terrestrial fiber links and in unrepeatered subsea links (~ 200 to 500 km) that do not have electrically powered subsea amplifiers. Links that fall within these categories are often more challenging and result in a bottleneck of transmission capacity. There is also a trend to extend the higher performance subsea links through these lower performance sections to avoid regeneration costs at a subsea cable landing station. Methods to improve the performance of these bottleneck sections are discussed below.

CHALLENGING SCENARIOS

Long unrepeatered (UR) spans are the most difficult and performance-limiting case, as you move beyond around 300km. Although the technology exists to receive data more than 500km away on an unrepeatered link, and although in theory an unrepeatered cable can contain many multiples of fibers compared to a repeatered cable, the number of wavelengths that can be transmitted and the data rate of each wavelength means that beyond 400 to 450 kilometers, it is usually more cost-effective (measured as cost/bit) to switch to repeatered technology, notwithstanding the inherent limitations on number of fiber pairs and the cost of the additional subsea and dry plant.

Fig 2: Schematic of UR system with ROPAs.

The science behind making difficult UR spans work.

For UR spans that approach this pivot point, it is well known that utilizing forward- and backward-pumped Raman significantly improves the Optical Signal-to-Noise Ratio (OSNR).[1][2] The Raman modules can produce both significant gain within the line fiber and pump a Remote Optically-Pumped Amplifier (ROPA). In standard core area fibers (G.654.C) the maximum gain that can be used is about 25 dB before multi-path interference (fed by Rayleigh scattering) within the fiber starts becoming an issue.[2] Xtera has a Raman module with this gain that can additionally act as a ROPA pump using the residual Raman power after about 100 km or more of distributed Raman amplification.

However, newer UR deployments are making use of lower attenuation and larger core area fibers (G.654.D). The lower attenuation reduces the overall span loss and generally results in lower Rayleigh scattering and MPI penalties. For example, 0.02 dB/km lower fiber attenuation over 400 km reduces the span loss and increases the OSNR by 8 dB, a significant improvement. The larger core area allows a higher channel power in the fiber before nonlinear effects become dominant. Unfortunately, the larger core area also results in a negative effect, by reducing the Raman gain. The Raman gain scales inversely with the core effective area (Aeff) so that a 112 µm2 fiber has only ~ 70% of the gain in dB compared to 80 µm2 fiber. In other words, 25 dB gain in 80 µm2 fiber becomes < 18 dB gain in 112 µm2 fiber resulting in a loss of ~ 14 dB Raman gain with forward and backward Raman pumping. In many cases, the UR performance in standard Aeff fiber is better than higher Aeff fiber due to this gain limitation. To compensate for this lower Raman gain, specific high-power Raman modules have been developed for larger Aeff fiber to maximize the optical performance in these newer fibers.

Terrestrial fiber spans are typically longer than subsea spans and quite often use existing G.652.A fiber with standard 0.20 dB/km attenuation. A shorter terrestrial fiber section can have a much larger impact on performance than a much longer subsea section. For example, terrestrial fiber spans of 100 km @ 0.2 dB/km (20 dB) have more than 10x lower OSNR versus subsea 60 km spans @ 0.16 dB/km (9.6 dB). Thus, 10 terrestrial spans result in a lower ONSR than 100 subsea spans in this case, not even considering the lower nonlinear penalties in larger Aeff subsea fiber. This is another case where deploying a lower gain (10 to 13 dB) Raman module on the longer terrestrial spans can significantly improve performance by about 4.5 dB per span.[2]

Terrestrial deployment for TIM Brasil using multiple Raman + ROPA spans over the Amazon jungle[3]

In some cases, terrestrial fiber links contain one or more very long spans in the middle, possibly over a difficult access geography point (like the Amazon jungle or subsea section). Similar to the discussion above regarding unrepeatered links, a span with +10 dB loss (~ 50 km longer) relative to other spans has the equivalent OSNR as 10 shorter spans with the same amplifier equipment. This bottleneck can be solved in the same way as the single UR span, noting that the optical power in the line fiber needs to be optimized differently for a multiple-span link compared to a single-span. We have even deployed ROPAs in multiple spans of a long terrestrial network to optimize capacity.[3]

WHAT COMES NEXT?

A recent trend in optical communications is to develop a disaggregated equipment model such that different parts of the system can accommodate different equipment vendors and interoperate through a unified management system. We are developing disaggregated Raman modules using the latest pump technology and incorporating upgrades based on decades of experience. For example, one such upgrade is an integrated fiber cut and repair detection mechanism that can work over extremely high loss spans of 80 dB or more and through unpumped ROPAs (which are normally opaque without pumping). Fiber cuts on long spans have always been a difficult operational issue and there has been no simple way of detecting when a repair has been completed since the Raman pumps are turned off for safety reasons. Having a continuously operating monitoring system (that complies with laser safety regulations) allows autonomous restart after a repair has been completed.

Another upgrade is a gain control mechanism of a forward Raman module which is normally blind to the gain that occurs in the line fiber ahead of it. This is particularly helpful as the forward Raman module can adjust the pumps based on varying channel capacity. Equipment variants of medium gain C-band, high gain C+L-band (with ROPA pumping) and high-power C+L-band (for large Aeff fiber with ROPA pumping) will all share a common mechanical platform and interface. This will allow any network to operate the best-in-class Raman solution for these cases where it is needed to overcome performance limitations.

CONCLUSIONS

The last decade has seen a huge emphasis on SDM technology from Xtera and the major players in the repeatered subsea space, partly for the capacity gains, and partly for the inherent cost reduction that accompanies this shift in technol- ogy. SDM technology has been one of the key facilitators that allows the world map of Data Centers to look the way it does. The next decade is likely to see a shift in focus, from a network perspective, to provide the capillary connectivity from these hyper scale data hubs that will distribute the data regionally, inter-data-center, and to some less well served destinations. The technologies discussed above will take their turn as the facilitators of this next generation of network build out.

Wayne Pelouch, Vice President of Photonics at Xtera

WAYNE PELOUCH is Vice President of Photonics at Xtera. Wayne Pelouch received his B.A. degree in Physics (Honors) and B.S. degree from Northwestern University in 1987, and his Ph.D. degree in Applied Physics from Cornell University in 1992. His thesis covered ultrafast lasers and nonlinear optical processes with applications in semiconductor physics. After graduation he joined the University of New Mexico / Air Force Research Labs in Albuquerque, NM and then worked for the Lions Eye Institute in Perth, Australia as a Laser Physicist. Subsequently, he joined Coherent Technologies (Lafayette, CO) where he was Principal Investigator on a number of SBIR programs related to waveguide laser and amplifier technology. Dr. Pelouch joined Xtera Communications in 2001 and is currently the Vice President of Photonics at Xtera Inc. (Allen, TX) where he leads all Photonics activities including research, product development, and system network design. He has authored numerous publications and patents related to fiber optics and amplifier technology.

Leigh Frame, Chief Operating and Sales Officer of Xtera

LEIGH FRAME is Chief Operating & Sales Officer at Xtera. Leigh joined Xtera in early 2018 and serves as the Chief Operating Officer and key architect of the company’s strategy. Leigh brings with him a wealth of senior level and hands-on experience, backed up by a network of strong industry relationships at all levels in the customer, supplier and finance communities. Previously Leigh had an extensive career with Alcatel Submarine Networks, which includes positions as COO, VP Projects and Customer Support, and Director Marketing and Business Development. In addition, he served as an active Board member of the Apollo Submarine Cable System for ten years. His broad experience in the subsea industry over three decades covers corporate strategy, a track record of sales success, marketing, M&A work, operations management, and high risk/high value turnkey project delivery. Leigh holds a Degree in Economics from Leeds University.

REFERENCES H. Fevrier, B. Clesca, P. Perrier, D. Chang and W. Pelouch, “Unrepeatered Transmission,” in Undersea Fiber Communication Systems, José Chesnoy, Ed. Oxford: Academic Press, 2016, pp. 261-300. Wayne Pelouch, “Raman Amplification: an Enabling Technology for Long-Haul, Coherent Transmission Systems”, J. Lightwave Tech., vol. 34, no. 4, 2016. TIM Brasil Case Study: A 2000 km Backbone Network Deployment on a Power Grid with Ultra-Long Spans in the Amazon Region

To read more SubTel Forum Magazine Feature Articles, visit https://subtelforum.com/products/subtel-forum-magazine/