How Tarrant Regional Water District Responded to the Texas Winter Storm

How Tarrant Regional Water District Responded to the Texas Winter Storm

An air/vacuum release valve on TRWD’s system that froze and broke during the storm.

Tarrant Regional Water District (TRWD) supplies raw water to more than 2 million people in and around the city of Fort Worth and manages the federal floodway that runs through the city. Like all critical utilities in Texas, TRWD put extraordinary efforts into continuing to provide its essential services during the winter storm of February 13–17, 2021. In this interview, TRWD Director of Operations Darrell Beason and TRWD Water Resource Engineering Director Rachel Ickert tell Municipal Water Leader about their preparations for the storm, the response to it in real time, and the lessons TRWD will be taking forward to future emergency events.

Municipal Water Leader: Please tell us about your backgrounds and how you came to be in your current positions.

Rachel Ickert: I joined the district in 2014 as a full-time employee. I worked for Freese & Nichols before that and had worked on projects for TRWD for 11 years, meaning that I’ve been working with the district since 2003. I’m a licensed professional civil engineer and have worked in water supply, design, and planning and in flood control. My group at TRWD oversees our flood modeling and monitoring efforts; water supply management and planning, both long term and shorter term; energy management; and geospatial services, which includes GIS and survey services.

The floor of one of TRWD’s transmission pump stations.

Darrell Beason: I started at TRWD in 1994, at which point it was known as the Tarrant County Water Control and Improvement District No. 1. I started in the carpentry shop, which was similar to what we call now facilities maintenance. Back then we actually did things like building cabinets. I started at the bottom and worked my way through to the positions of construction coordinator, Fort Worth operations manager, western division operations assistant director, and western division operations director, eventually reaching my current position, director of operations. About 8 years ago, we combined the two operational divisions.

Taking advantage of TRWD’s tuition reimbursement program, I went back to college and graduated in 2011 with a degree in management. I like to say that I was on the 23‐year degree plan.

My current group, the operations department, covers pipeline operations and maintenance; floodway operations and maintenance, including all the maintenance on the on the Trinity River floodway through Fort Worth; reservoirs operation and maintenance; law enforcement; our newly formed emergency management department; and fleet maintenance and management.

Municipal Water Leader: Please tell us about TRWD’s services and infrastructure.

Darrell Beason: TRWD has been around for almost 100 years. Our two primary responsibilities are supplying raw water to municipal customers who treat and distribute it to more than 2 million people and managing the 28‐mile federal floodway that runs through Fort Worth. We have a recreation mission as well, which covers the recreation around the floodway, lakes, and parks. We serve 11 counties centered around Tarrant County. About 80 percent of the water that we supply to Fort Worth is pumped from reservoirs in East Texas.

Municipal Water Leader: Please tell us about your experience of the February storm.

An operations crew isolating a broken valve in order to replace it. The line was in operation, soaking them with water.

Darrell Beason: My group started our preparations when our emergency management group sent out its first situation warnings. We started doing a lot of preparation on our generators and checking supplies. At the time, we thought that we might suffer short-term power outages based on ice loading on the power lines. We had no way of knowing that we would end up suffering massive, long-term blackouts caused by equipment failures. I think the last comparable storm in this region occurred when I was 13 years old. I remember skating on Lake Worth in my tennis shoes and playing hockey with friends.

Rachel Ickert: In the days leading up to the event, the weather forecasts were starting to come in, and we were getting indications from our energy providers that we would see elevated energy prices in the market. We are able to prepurchase part of our expected energy load, but another part of our load rides on the real-time market. The real-time market price is generally $20–$40 per megawatt-hour. We started to see prices in the hundreds and then thousands and we eventually realized that the price would likely hit the $9,000 price cap. We started having conversations about reducing our pumping load before the storm hit. The prices began to increase on February 12 and 13. We reduced the amount of water we were pumping from our East Texas reservoirs on February 13 in response and tried to stay within our prepurchased load when possible.

We were in a pretty good operational position for a few days, but on the night of February 15, Oncor Electric Delivery Company called our energy manager to say that it might have to start taking down transmission grids. We responded that we were a priority user because we provide a critical resource to over 2 million people, but Oncor replied that that might not matter because the grids were at a critical point. We asked if we would get a warning before it took down the grids, and the answer was no. Having our transmission-level pump stations go down with no warning could cause significant damage to our water delivery system. A sudden shutdown is hard on the pumps and the pipeline.

After that call, we all spoke by phone. It was around 11:00 p.m., and a lot of us didn’t have power in our homes at the time. I was sitting by the fireplace. We needed to keep moving our water, but since there was a chance that our stations would go down with no warning, we decided that we might need to take them down proactively. The potential damage would have been worst if we lost power to our Richland Chambers and Cedar Creek stations in East Texas. A power failure to our Benbrook pump station, which pumps out of Benbrook Lake, could have caused some damage, but it wouldn’t have been nearly as serious. We made the decision to shut down pumping at Richland Chambers and Cedar Creek Reservoirs and to use the Benbrook Lake pump station and the water we had in the earthen balancing reservoirs that sit at high points in our system to supply the system and meet customer water demands. At the time, that all seemed fine and good, but over the next few days, February 15–18, our customer demand started skyrocketing. Our peak primary customer demand on February 18, 536 million gallons, was higher than the demand on a dry summer day. It was difficult to meet that level of demand without pumping, and we estimated that we had less than 1 day of storage left in the balancing reservoirs at that water demand rate.

As time went on and the grid became more stable, we got an opportunity to start bringing some of our pump stations back online. Our customers Joe Smolinski and Jeff Price from the City of Mansfield called and connected us with David Cook, a state representative who is the former mayor of Mansfield, and Oncor’s vice president of regulatory affairs, Liz Jones. After being apprised of the situation, Ms. Jones assured us that we could bring on the pump stations we needed and that we would be considered priority, so that if transmission-level power had to go out, our pump stations wouldn’t go out. We were in uncharted waters. We weren’t 100 percent sure that that meant it was safe to bring everything back up, but we went ahead and started slowly increasing our pumping again.

These decisions involved Darrell’s operations team, water resources engineering, the SCADA team, and our executives. We made the decisions to take the system down and bring it back up as a team. It was a question of managing system risk while meeting customer demands and keeping an eye on energy prices. Through this whole event, we had staff in constant communication with our customers as parts of their systems were failing. Not once did our customers run out of water from us—I am proud to say that we met our customer demands through it all.

Municipal Water Leader: How can losing power damage your pumps and pipelines?

Darrell Beason: Our normal shutdown process starts with slowing down our pumps. We start closing the valves and reducing the flow. We pinch the pump control valve down to the point at which the water is barely moving through the pump before stopping the pump. It’s a slow, controlled, methodical decrease. In an emergency stop, as would occur with a power outage, the flow in the line stops instantaneously when the pumps stop. This can induce a transient pressure wave, and the pumps can spin backward for a short period of time. The wave takes 3–7 minutes to go from one end of the pipe to the other, and it can cycle back and forth for hours. You risk rupturing a line, which is one of the worst things that can happen. In this case, the roads were covered with ice, and our rights of way were covered with snow, so we wouldn’t even have been able to mobilize equipment to begin a pipeline repair. A repair could have taken weeks, instead of our usual failure response time of 48 hours. Combined with the skyrocketing demand, it would have put us in a bad position. That scenario was discussed during that late-night call on February 15.

Municipal Water Leader: In addition to concerns related to power supply, was there any damage to TRWD’s system caused by power outages or freezing water?

Darrell Beason: February 15 is when we were hit with the possibility of an electric grid failure, but before that, we experienced extremely cold temperatures and rolling blackouts. A lot of our instrumentation lines are protected by a heat trace on the piping itself. As we lost power, the heat trace failed, and the lines froze. We experienced relatively minor damage because of that. A lot of our office buildings went 2–3 days without any electricity or heat, and we had lines freeze, rupture, and flood facilities throughout our system. That damage was spread out pretty evenly over the full length of our system. On the pipeline system, 12 small air pots froze and burst. These caused relatively small leaks and were repaired within a week.

Municipal Water Leader: How did operations staff respond to these unexpected circumstances?

Darrell Beason: Our electricians have two-wheel-drive company trucks, and many used their personal trucks instead to get to our office buildings and maintenance facility sites. A number of employees used their own vehicles, their own tools, and their own equipment to keep systems going. Ice loading prevented some of our security gates from operating. Some guys climbed over razor-wire fences to get to pump stations so that they could adjust the controls or take other actions. One electrician was borrowing heaters and propane from his dad and setting them up on site to keep our pressure transmitters from freezing—those are essentially the eyes of our SCADA operators. Electricians built little makeshift houses out of wood to shelter some of these devices and keep the heat from the heaters from blowing away. The innovation and dedication of our guys was appreciated. They knew that the failure of those systems was not an option. They are the unsung heroes of this event.

Cows take advantage of the water from a broken valve on TRWD’s system; all the other ponds were frozen over.

Municipal Water Leader: Did TRWD’s emergency planning processes help the agency respond to this event?

Rachel Ickert: Our regular and emergency planning put us in a really good position to be able to communicate and work through this. Our staff know our system so well. Even without power, and even when the SCADA manager couldn’t see everything, he knew what to do and how to do it. Members of my team knew how to get through our contingency operations and how long the water in storage would last. They had all that information readily available, even when the power was out where they were working. There is a lot of preparation that you don’t even realize people are always doing until it’s needed. A lot of hard work went into preparing for things like this, even though there is no way we could have predicted exactly what would happen.

Darrell Beason: During emergency management preparation efforts, people often think, “My time would be better spent solving a problem or fixing something,” but our preparation on the front end was so valuable as things started to fall apart. We had an idea of what each other’s departments were going to be doing. We were in as frequent communication as we could be. Cellular towers started to fail toward the end of the event, but up until that point, most of us had phone connections. The internet was in and out, so we were doing most things by telephone.

One silver lining of the COVID‐19 pandemic is that all our planning and management employees were fully capable of working from home. Knowing what the other groups were probably doing gave us a lot of peace of mind. We were able to react, not just to sit paralyzed, waiting for confirmation. We’d already worked through the scenario, and knew that if A happened, then B would happen. All the labor-hours we spent on emergency management training definitely paid off.

Municipal Water Leader: Did you cooperate with other local agencies in your response?

Darrell Beason: During this event, we had some capacity in our flood control group, and we offered assistance to some of our customers. The City of Fort Worth had more than 600 water main breaks. We embedded some of our crews with the city’s crews to help it respond. At one point, we were asked by the emergency management coordinator if we could provide potable water to some locations in the city that were completely out of water. Some of our equipment operators brought in large trucks with potable-water tanks. More than 10,000 gallons was distributed in 1‐ to 5‐gallon containers. One of our operators told me, nearly with tears in his eyes, that these people hadn’t had water in a week. These guys had the ability to help others out, and they jumped on that opportunity.

Municipal Water Leader: What changes will you make in your emergency response plans based on your experience of this event?

Rachel Ickert: We knew this event was coming, and we started preparations from an operations and energydemand-reduction standpoint. However, we did not foresee its severity, and we did not have a dedicated meeting about it. Next time something like this is coming in, we need to have a premeeting. We need to run through who will be where and run through some potential scenarios.

We have also noted that we need to remember in the future to make sure that all our balancing reservoirs are as full as possible before an event like this, just as they were this time around.

This event brought to light the fact that certain critical facilities throughout the state were not registered as critical with the power providers. TRWD’s pump stations were registered as critical facilities prior to this event, but we are making sure all the facilities we consider critical are registered, and we want to make sure that that actually means something. We had that criticality designation, but at a certain point, everyone needed to pitch in and help save the grid. From the larger, statewide energy grid standpoint, I sure hope there will be some after-action reviews and planning. That is somewhat out of our hands, except where we may be able to comment on potential legislation.

We are also having after-action meetings with our customers. We hope they will share information with us and tell us the things they plan to do in the future. For example, if we have an ability to get them water during an emergency, do they still have the ability to treat it?

Darrell Beason: We need to have deliberate discussions about the potential scenarios in front of us for an event like this. I’m grateful that we were talking every day, but our conversations were more along the lines of, “This is what we’re doing today; this is what we’re doing tomorrow,” than, “Let’s think about 4 or 5 days out.”

Most of the items on the after-action report were small tweaks. We want to change the design of future gates so that their tracks do not fill with ice. We will change the battery capacity on some of our vehicle gate actuators and add manual personnel gates. Our pressure transmitters will be placed in below-ground vaults to decrease the likelihood of freeze damage. We can’t react in a knee-jerk fashion to an event like this. We have to keep in mind the frequency with which these events happen and use our funding in the best possible way. The easy and cheap things that bring the biggest bang for the buck are what we’re focused on today.

We’ve definitely looked at our backup generators. The likelihood of them failing was greater than we previously anticipated. Some of them are old and probably should have been replaced sooner. We will try to replace those with something a little newer and more reliable. Those generators are not big enough to run our pumps; they are simply for communications and to keep our control centers up and running.

There is talk about getting larger generators that would be able to spin up a pump at one of our pump stations during a true black-sky event of a longer duration. We have talked about seeking Federal Emergency Management Agency funding for some of that. We’ve also talked about seeking grants for solar fields. There are a lot of things in the pipeline. We are going to work through them, do business cases on each one, and route them through our capital improvement plan process. Some have been considered in the past. The big difference is that the what if scenarios are now remember when scenarios.

Darrell Beason is the director of operations at Tarrant Regional Water District.

Rachel Ickert is the water resource engineering director at Tarrant Regional Water District. For more about TRWD, visit www.trwd.com.