Solution
Although the LLNL computing center holds some of the world’s largest, fastest and most advanced classified systems, the facility’s existing power and cooling capacity was unable to handle the next generation of high-performance computers.
This project called for doubling the capacity of the building’s electrical and cooling infrastructure. Our design team was hired to plan, design and provide engineering support during construction of the utility systems and support facilities. The project was handled in two phases. The initial site preparation phase consisted of design and construction support for removing light poles, rerouting utilities, removing buildings and clearing the site in preparation for the second phase. The second phase included detailed design and construction support of a new utility substation and electrical distribution system, an 18,000-ton process water cooling plant, and a process cooling water distribution system.
Engineering solutions were developed for civil, structural, architectural, mechanical and electrical systems. The design solution included the addition of an electrical substation with accompanying overhead power distribution. A process cooling water (PCW) plant composed of cooling towers, heat exchangers (HX), pumps, underground and overhead piping and controls with supporting underground utilities were also included in the design.
The new electrical substation consists of two transformers — each 40/50/60 MW — and reduces the utility-supplied voltage of 115-kV down to 13.8-kV. The addition of this substation increases the facility power supply from 45 MW to 85 MW. A new aboveground, parallel 115-kV transmission line is routed to the new substation yard from new utility poles interconnected 3,000 feet away.
With the substation constructed in a tight space, its development required working closely with the Western Area Power Administration (WAPA) to coordinate outages and reroute power. A highlight of the partnership included using a single set of poles to develop parallel power feeds through the substation to the computing center. This move increases redundancy and resiliency by providing backup power should one of the power lines fail. During work on this aspect of the project, several utility poles needed replacing and pole availability threatened the project’s deadlines. Leveraging our relationship with a nearby utility, LLNL was able to secure the poles needed and continue the project on schedule.
The process cooling water meets American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) W3 grade water standards and will be utilized to provide direct cooling of the high-performance computers using 86°F process water. Six modules of cooling towers, pumps and heat exchangers are being used to provide the heat rejection requirements. One module is entirely redundant. A new six-cell, 28,000-gallons-per-minute (GPM) cooling tower was constructed on the site. The cooling tower is installed over a concrete basin with a collection trough and sump pump with vertical turbine pumps for circulating the tower water to the process cooling water heat exchangers.
Two water loops were created — one open loop for the cooling towers and one closed loop for the process water, separated by plate heat exchangers. The process cooling water loop is constructed of stainless steel 30 inches in diameter, plus combination below- and aboveground piping. The below- ground piping is direct-buried and the aboveground piping is located on structural steel supports. The process cooling water loop uses cartridge-style fabric filters to keep the water clean for the high-performance computers.
A life cycle cost analysis and an hourly energy analysis were used to evaluate the most viable option of 12 different heat rejection configurations, including packaged cooling towers, field-erected cooling towers and fluid coolers. We recommended using a factory-fabricated and field-assembled counterflow cooling tower as the basis of our design because of the superior life cycle cost analysis. The cooling towers and variable frequency drives were procured early under a Government Furnished Equipment (GFE) bid package so the materials could be ready for assembly early in the construction schedule.
