Liquid Cooling for AI Data Centers: What Buyers Should Know
The thermal math for modern AI has become unforgiving. An NVIDIA H100 SXM5 GPU dissipates up to 700 watts. A single rack of eight H100s already approaches 6–7 kW just in GPU heat — before accounting for CPUs, memory, storage, or networking. Multiply that across a training cluster of hundreds of nodes and you are looking at power densities that conventional computer-room air conditioning was simply never designed to handle. Liquid cooling for AI data centers is no longer an exotic option reserved for national laboratories; it is rapidly becoming the baseline expectation for any serious AI infrastructure build.
Understanding the technology landscape — and the procurement decisions that come with it — is critical for federal agencies, SLED institutions, healthcare systems, and enterprise buyers evaluating their next AI infrastructure investment. This guide walks through the core cooling approaches, what the economics actually look like, how HPE's current portfolio addresses the challenge, and what facility teams need to plan for before the first server ships.
Why Air Cooling Has Hit a Ceiling for AI Workloads
Traditional computer-room air conditioning (CRAC) and computer-room air handling (CRAH) units move chilled air across hot-aisle/cold-aisle rows at densities that peak around 15–25 kW per rack in well-optimized designs. Even aggressive approaches like rear-door heat exchangers can push that ceiling to roughly 30–40 kW per rack. Modern AI accelerators blow past that limit without breaking a sweat.
The root issue is thermodynamic: air is a poor heat carrier. Water moves approximately 3,500 times more heat per unit volume than air at equivalent flow rates. Once a single rack in an AI training cluster exceeds 30–40 kW — a threshold that eight H100s can cross — air cooling either fails outright or demands floor space, airflow volume, and chiller capacity that become prohibitively expensive. Cooling alone can account for 30–40% of a data center's total electricity consumption, according to data from the U.S. National Renewable Energy Laboratory. For high-density AI workloads, that fraction climbs higher unless liquid is introduced into the thermal path.
The Three Dominant Liquid Cooling Approaches
Buyers entering the liquid cooling market for the first time will encounter three architectures, each with distinct tradeoffs:
1. Direct-to-Chip (Direct Liquid Cooling / DLC) Cold plates machined to fit CPUs and GPUs are plumbed directly into a recirculating liquid loop. Heat is extracted at the source and transferred to a facility water loop via a Cooling Distribution Unit (CDU). The rest of the server — memory DIMMs, SSDs, power supplies, NICs — may still be air-cooled. DLC is the most widely deployed approach in enterprise AI clusters today, commanding roughly 47% of the liquid cooling market. It is the approach HPE uses across its ProLiant and Cray XD server lines.
2. Rear-Door Heat Exchangers (RDHx) A liquid-cooled door replaces the standard rear exhaust panel on a rack. Hot exhaust air passes through the exchanger before leaving the rack, reducing (but not eliminating) the thermal burden on room air systems. RDHx is a useful retrofit option for moderate-density racks (30–60 kW) and requires less plumbing work than full DLC, but it does not match DLC's efficiency at the extreme densities demanded by large-scale AI clusters.
3. Immersion Cooling Servers are submerged in a thermally conductive dielectric fluid — either single-phase (liquid stays liquid) or two-phase (fluid boils, vapor condenses and falls back). Immersion achieves the highest possible cooling density, with some commercial systems handling 250 kW or more per tank. The trade-off is significant: custom server designs or delidded components, specialized fluids, longer maintenance windows, and a smaller ecosystem of compatible hardware. Two-phase immersion is best suited for ultra-dense AI pods with standardized workloads rather than general-purpose enterprise deployments.
| Approach | Typical Rack Density | Facility Water Required | Best Fit |
|---|---|---|---|
| Air cooling (baseline) | Up to ~25 kW | No | Light compute, edge |
| Rear-door heat exchanger | 30–60 kW | Yes (chilled loop) | Moderate AI inference, retrofit |
| Direct liquid cooling (DLC) | 50–150+ kW | Yes (chilled or warm water) | AI training clusters, HPC |
| Single-phase immersion | 100–250 kW | Yes (facility loop) | Ultra-dense AI pods |
| Two-phase immersion | Up to 1,500 W/cm² density | Yes (specialized) | Research, specialized hyperscale |
HPE's Liquid Cooling Portfolio: What Exists Today
HPE has invested in liquid cooling technology for over two decades through its Cray supercomputing heritage, resulting in a portfolio that spans from enterprise rack servers to leadership-class supercomputers.
HPE Cray Supercomputing EX Series The HPE Cray Supercomputing EX is the flagship platform for extreme-scale AI and HPC workloads. It uses a sealed, direct liquid-cooled cabinet architecture built around the EX4000 (a standalone cabinet) and the EX2500 (a liquid-cooled rack with an integrated CDU). Neither cabinet exhausts heated air into the data center — all heat is transferred to the facility water loop via the CDU, making the system effectively invisible to the room's air management. A single in-row CDU supports up to four EX4000 cabinets and is rated for up to 1.6 MW of cooling. The portfolio is notable for being built on what HPE describes as the industry's first 100% fanless DLC architecture — an approach now deployed at national research laboratories and in sovereign AI initiatives internationally.
The newest generation, announced in late 2024 and continuing to ship in 2025, includes the HPE Cray Supercomputing EX154n Accelerator Blade — capable of housing up to 224 NVIDIA Blackwell GPUs in a single cabinet — and the EX4252 Gen 2 Compute Blade, which can deliver up to 98,304 CPU cores per cabinet using AMD EPYC 5th Gen processors. These systems are designed for AI model training, large-scale inference, and scientific computing at the scale of government agencies and large enterprises.
HPE Cray XD670 The XD670 is a 5U chassis that houses a single dual-CPU node with eight NVIDIA H100 or H200 SXM5 Tensor Core GPUs. It is available with a direct liquid cooling option that places cold plates directly on the GPUs and CPUs, handling roughly 70% of the server's total heat through the liquid loop while the remaining 30% (low-heat components) continues to use air. This hybrid approach means the XD670 can operate in data centers with existing air infrastructure as long as a facility water loop and CDU are present.
HPE ProLiant Compute XD685 Aimed at large enterprises and service providers training their own AI models, the XD685 is a 5U server available with eight NVIDIA H200 SXM Tensor Core GPUs or NVIDIA Blackwell GPUs. It carries forward HPE's multi-decade liquid cooling expertise in a more operationally accessible form factor than the full Cray EX cabinet system.
HPE ProLiant DL380 Gen11 and DL560 Gen11 with DLC Options For organizations that already operate HPE ProLiant environments and need to step up to AI workloads incrementally, the DL380 Gen11 and DL560 Gen11 both support direct liquid cooling configurations. The DL380 Gen11 can support up to three NVIDIA H100 80GB accelerators — and when configured with all three, a DLC kit is required. The DL560 Gen11 similarly supports liquid cooling options for high-density GPU configurations. These servers allow buyers to introduce DLC at the server level without immediately committing to a full HPC-class infrastructure overhaul.
If you are evaluating which ProLiant or Cray platform fits your workload, the AI infrastructure buying guide at Uniqcli walks through the decision tree in detail.
Facility Requirements Buyers Often Underestimate
The cooling technology inside the server is only half the equation. Liquid cooling places non-trivial demands on the physical facility that must be scoped before procurement begins.
- Facility water loop: DLC systems require a building-side chilled or warm water supply. HPE's Cray XD and Cray EX systems use a closed secondary loop — the CDU circulates its own heat transfer fluid to the server cold plates, and transfers captured heat to the facility loop via a liquid-to-liquid heat exchanger. This means facility water never directly contacts server components, reducing contamination risk.
- Dew point management: CDUs actively monitor ambient conditions and maintain secondary loop temperatures above the room's dew point to prevent condensation on cold plates. Buyers must provide accurate environmental data to HPE and their facilities team during planning.
- Pipe and floor penetration planning: Bringing liquid supply and return lines to each rack row requires coordination with building management, structural engineering, and potentially electrical teams (for leak detection systems). Lead times for this infrastructure can dwarf server delivery times.
- CDU placement and power: The CDU itself requires power for its circulating pumps and control systems. In-row CDUs for Cray EX installations are rated for 1.6 MW of cooling capacity but also consume power — plan accordingly in the facility power budget.
- Water quality and treatment: Facility water chemistry must meet HPE's specifications. Deionized water or treated glycol solutions are typical; untreated tap water can cause corrosion and mineral scaling that degrades heat transfer efficiency over time.
Federal agencies and healthcare institutions should engage their facilities management teams early — sometimes 12–18 months before server deployment — to assess pipe routing, structural floor loading from CDUs, and compatibility with existing building management systems.
The Economics: TCO, PUE, and Payback Period
The capital cost of liquid cooling infrastructure is meaningfully higher than air cooling. Rough industry benchmarks place air-cooled data center build-out at $1.5–2M per MW of IT capacity, versus $3–4M per MW for fully liquid-cooled infrastructure. That premium is real, and procurement teams rightly scrutinize it.
The total cost of ownership calculation changes the picture substantially, however:
- PUE improvement: Air-cooled data centers supporting dense AI workloads commonly run PUE of 1.4–1.6. Well-implemented DLC systems at the same IT load can achieve PUE of 1.03–1.15. For a 10 MW AI training facility, reducing PUE from 1.5 to 1.10 saves 4 MW of continuous power draw — roughly $2.8 million annually at $0.08/kWh, and significantly more in energy markets where power costs are higher.
- Floor space efficiency: Liquid cooling enables rack densities three to five times higher than air cooling for the same IT capacity, reducing the total floor space and associated build-out cost.
- Hardware reliability: Lower, more consistent operating temperatures reduce thermal stress on components, which correlates with longer mean time between failure (MTBF) for GPUs, CPUs, and memory.
- Payback period: At rack densities above 50 kW, liquid cooling installations typically reach payback within 18–36 months through energy savings alone, before accounting for deferred CRAC/CRAH unit costs and reduced hardware replacement rates.
For federal and SLED buyers subject to energy efficiency mandates, the PUE and Water Usage Effectiveness (WUE) metrics also directly affect compliance reporting. Closed-loop DLC systems that use no evaporative water — recirculating the same heat transfer fluid — can achieve WUE values well below 0.5 L/kWh, compared to 1.0–2.0 L/kWh for evaporative cooling towers used in conventional chiller plants.
For a broader view of how liquid cooling fits into AI infrastructure procurement strategy, see the Uniqcli AI infrastructure buying guide.
Comparing Liquid Cooling Readiness: What Buyers Should Assess
Not every organization needs a Cray EX supercomputer to benefit from liquid cooling. The right entry point depends on current rack density, projected GPU count, facility constraints, and budget cycle timing.
| Buyer Profile | Recommended Entry Point | Key Consideration |
|---|---|---|
| Single-rack AI inference (up to 30 kW) | Air cooling or RDHx | DLC ROI is weak at these densities |
| Small AI training cluster (30–80 kW/rack) | HPE ProLiant DL560 Gen11 with DLC | Requires CDU and facility water loop |
| Enterprise AI factory (8+ H200 nodes) | HPE ProLiant Compute XD685 | Full DLC required; plan facility work early |
| HPC/large model training (multi-rack) | HPE Cray XD670 with DLC | Fanless option available with full CDU |
| Supercomputing / sovereign AI | HPE Cray Supercomputing EX (EX2500/EX4000) | 100% DLC, sealed cabinet, in-row CDU |
Procurement Considerations for Federal, SLED, and Healthcare Buyers
Public-sector and regulated-industry buyers face constraints that private enterprise does not, and liquid cooling procurement has several sector-specific wrinkles.
Contract vehicles: HPE products are available through major federal and SLED contract vehicles, which can streamline procurement without requiring a full competitive bid for commodity configurations. Uniqcli can help identify the appropriate vehicle for your agency or institution — visit the Uniqcli shop or request a quote to start the conversation.
Data sovereignty and facility security: For classified or sensitive workloads, the sealed CDU architecture of the HPE Cray EX is actually an advantage — no air is exhausted into the room, reducing acoustic and thermal side-channel risk.
Sustainability mandates: Federal Executive Orders and state-level green IT policies increasingly require agencies to report PUE, WUE, and carbon footprint for data center operations. Liquid-cooled infrastructure typically makes compliance easier, not harder.
Healthcare-specific concerns: Healthcare organizations evaluating AI infrastructure for imaging AI, genomics, or clinical decision support should note that liquid cooling does not introduce additional HIPAA risk — the cooling loop is physically separated from IT network infrastructure. The primary risk management consideration is leak detection: reputable DLC deployments include moisture sensors at every drip tray and quick-disconnect fittings that seal automatically if a line is disconnected under pressure.
Long procurement cycles: Hospital systems and government agencies often operate on 18–36 month procurement cycles. Given that DLC infrastructure requires facility preparation work that may itself take 6–18 months, buyers should initiate planning conversations well before the formal solicitation process begins. The Uniqcli guides library includes resources on data center planning that can support early-stage business case development.
What to Ask Your Vendor Before Signing
Whether you are evaluating HPE or another manufacturer, the following questions surface issues that generic marketing materials typically do not address:
- What are the exact facility water specifications (supply temperature, pressure, flow rate, water chemistry) required for this cooling system?
- Who is responsible for CDU installation and commissioning — the server vendor, a third-party integrator, or your facilities team?
- What is the warranty posture if a leak occurs — does the server warranty cover component damage from coolant?
- Is the DLC system modular — can individual cold plates or CDU components be replaced without taking the entire cluster offline?
- What monitoring and management integration is available — can coolant temperature, flow rate, and pump health be surfaced in the same management plane as server health (HPE iLO, for example)?
- What is the roadmap for the next GPU generation — will the same CDU and cold plate infrastructure support the next-generation accelerator, or will a forklift upgrade be required?
HPE's approach to the last question is worth noting: the EX series CDU infrastructure is designed to be decoupled from the compute blades, meaning that as accelerators evolve, organizations can upgrade compute density without necessarily replacing their entire cooling distribution infrastructure.
How Uniqcli Helps
Uniqcli is an authorized HPE and HPE Aruba Networking partner with experience supporting federal agencies, SLED institutions, healthcare systems, and enterprise buyers through complex AI infrastructure procurement. We can help you scope the right cooling approach for your workload, identify the appropriate HPE platform, and coordinate with your facilities team on the pre-deployment planning that liquid cooling requires.
Whether you are beginning to evaluate options or are ready to specify a configuration, request a quote or contact our team to start a conversation. We work on your timeline and your contract vehicles — and we do not push hardware that does not fit your actual requirements.