Artificial intelligence is changing the way data centers are designed.

Traditional data centers already require substantial electrical and mechanical infrastructure, but AI introduces a new challenge: enormous amounts of computing power concentrated into a relatively small space.

Modern AI systems can connect thousands—or even tens of thousands—of Graphics Processing Units, or GPUs, into large computing clusters. These GPUs consume tremendous amounts of electricity, and almost all of that electrical energy eventually becomes heat that must be continuously removed.

That creates two fundamental engineering challenges:

• How do you deliver enough electrical power to high-density GPU racks?
• How do you remove the enormous amount of heat they generate?

Those two questions are driving major changes in data-center electrical distribution, cooling systems, controls, and even how data centers are constructed.

Why AI Data Centers Are Different

Traditional data centers rely heavily on CPUs—or Central Processing Units—to process information and run applications.

AI systems increasingly rely on GPUs—or Graphics Processing Units.

Unlike CPUs, GPUs are designed to perform huge numbers of calculations simultaneously. This parallel-processing capability makes them particularly effective for artificial intelligence, machine learning, large language models, and other computationally intensive workloads.

Instead of operating independently, GPUs can be connected through extremely high-speed networks to create enormous computing clusters.

Modern rack-scale systems demonstrate just how concentrated this computing power has become. Some current systems combine dozens of GPUs into a single liquid-cooled rack, allowing the rack to operate almost like one enormous computing system.

But increased computing density creates increased power density and heat density.

The Power and Heat Density Problem

A traditional server rack might consume approximately 5 to 15 kilowatts, although rack loads vary considerably depending on the application.

High-density AI racks operate at a very different scale.

Modern GPU racks can require 40, 80, 100 kilowatts or more, and newer rack-scale AI systems can exceed 100 kW.

Consider a 100-kW GPU rack.

Almost all the electrical power consumed by the IT equipment ultimately becomes heat.

That means:

100 kW ≈ 341,000 BTU/hr

Converting that into familiar HVAC terminology:

341,000 BTU/hr ÷ 12,000 ≈ 28.4 tons of heat

That’s approximately 28 tons of heat from a single rack.

Put ten 100-kW racks together and you have approximately:

• 1 megawatt of IT load
• 3.41 million BTU/hr of heat
• 284 tons of heat

And all of it can be concentrated into a relatively small portion of the data hall.

The challenge isn’t simply the amount of heat.

It’s the concentration of that heat.

Getting Power to the GPUs

Before the heat can be removed, the electrical power first has to reach the computing equipment.

The basic electrical path through an AI data center is similar to other mission-critical data centers:

Utility → Transformers → Switchgear → UPS → Power Distribution → GPU Racks

Backup generators, UPS systems, redundant electrical paths, and other systems help maintain operation when utility power or individual components fail.

What changes with AI is the scale and concentration of the electrical load.

A relatively short row of high-density GPU racks can represent megawatts of electrical demand. That affects transformers, switchgear, UPS capacity, busway, conductors, rack distribution, and ultimately the utility infrastructure serving the facility.

And every kilowatt delivered to the computing equipment creates approximately another kilowatt of heat that ultimately has to be removed.

Why Air Cooling Becomes More Difficult

Traditional data centers have relied primarily on air cooling.

Cool air enters the front of a server rack, passes through the equipment, absorbs heat, and leaves the rear of the rack as hot air.

Hot-aisle and cold-aisle arrangements help prevent these two air streams from mixing and improve cooling efficiency.

This remains an effective solution for many data centers.

The problem occurs as rack density increases.

Removing more heat with air requires moving increasingly large quantities of air through the equipment. That means greater airflow, larger cooling systems, more fan energy, and sufficient space to distribute all that air.

At very high rack densities, removing enough heat directly from the highest-powered processors with air becomes increasingly difficult.

The solution is to move the cooling medium closer to the heat source.

That’s where liquid cooling becomes particularly important.

Direct-to-Chip Liquid Cooling

One of the most important cooling technologies for high-density AI equipment is direct-to-chip liquid cooling.

Instead of relying entirely on air to remove heat, a cold plate is installed directly against high-heat components such as GPUs and CPUs.

Coolant flows through small passages inside the cold plate and absorbs heat directly from the processor.

The basic heat path becomes:

GPU/CPU → Cold Plate → Coolant → Rack Manifold

Supply and return manifolds distribute coolant to multiple servers within the rack.

Liquid is particularly effective because it can transport large quantities of heat without requiring the enormous airflow that would otherwise be necessary.

But liquid cooling doesn’t eliminate the heat.

It simply gives us a much more effective way to capture it and move it somewhere else.

What Is a Cooling Distribution Unit?

The next major component is the Cooling Distribution Unit, or CDU.

Think of the CDU as the bridge between the liquid cooling the computer equipment and the mechanical cooling system serving the building.

Warm coolant returning from the GPU racks enters the CDU and transfers its heat through a heat exchanger.

On the opposite side of that heat exchanger is the facility water system.

The two fluid circuits remain separated while heat passes between them.

A CDU can also contain components such as:

• Pumps
• Heat exchangers
• Filters
• Temperature sensors
• Pressure sensors
• Flow monitoring
• Controls
• Leak-detection systems

The cooling path can now be expanded:

GPU → Cold Plate → Rack Manifold → CDU → Facility Water System

But the heat still has to leave the building.

Getting the Heat Out of the Data Center

Once the heat reaches the facility water system, several types of mechanical equipment can ultimately reject it outdoors.

Depending on the facility, climate, water temperatures, and system design, this can include:

• Chillers
• Cooling towers
• Dry coolers
• Evaporative or adiabatic equipment
• Economizer systems
• Combinations of these technologies

One important advantage of some liquid-cooled systems is the potential to operate at warmer water temperatures than traditional chilled-water cooling.

Under suitable outdoor conditions, warmer water temperatures may allow heat to be rejected through dry coolers or other economizer strategies with reduced reliance on mechanical refrigeration.

The complete heat journey might therefore look something like this:

GPU → Cold Plate → Manifold → CDU → Facility Water → Heat Rejection Equipment → Outdoors

The exact equipment changes from one facility to another, but the objective remains the same:

Capture the heat, transport it efficiently, and reject it outside.

AI Data Centers Can Use Both Air and Liquid Cooling

Liquid cooling doesn’t necessarily eliminate air cooling.

Cold plates can capture heat directly from the highest-powered components, including GPUs and CPUs.

Other equipment within the servers and racks may still reject heat into the surrounding air, including:

• Power supplies
• Memory
• Storage
• Networking equipment
• Other electronic components

This creates a hybrid cooling system.

Liquid handles the highest heat-density components while air cooling handles the remaining rack and room heat.

The exact split varies by equipment and system architecture.

The important concept is that liquid cooling captures a significant portion of the heat before that heat ever enters the data hall.

Other Liquid-Cooling Technologies

Direct-to-chip cooling isn’t the only liquid-cooling technology available.

Rear-Door Heat Exchangers

A liquid-cooled heat exchanger is installed directly behind the server rack.

Hot air leaving the servers passes through the heat exchanger, allowing much of the heat to be captured before it enters the data hall.

Immersion Cooling

Immersion cooling takes a completely different approach.

Computing equipment is submerged in a specially engineered dielectric fluid that doesn’t conduct electricity.

Heat transfers directly from the electronic components into the fluid and is then transported to the cooling system.

Each cooling method has advantages and limitations depending on equipment density, application, facility design, maintainability, and cost.

Efficiency Matters at AI Scale

When a data center consumes tens or hundreds of megawatts, relatively small efficiency improvements can represent substantial amounts of energy.

One of the most common data-center efficiency metrics is Power Usage Effectiveness, or PUE.

PUE compares the total energy consumed by the facility with the energy actually consumed by the IT equipment:

PUE = Total Facility Energy ÷ IT Equipment Energy

A theoretical PUE of 1.0 would mean that all facility energy is being used directly by the IT equipment.

Real facilities also require energy for cooling, pumps, fans, electrical losses, lighting, controls, and other supporting infrastructure.

Energy isn’t the only consideration.

Cooling-system selection can also affect water consumption, particularly when cooling towers or other evaporative cooling technologies are used.

AI data-center design therefore involves balancing several objectives:

• Computing performance
• Electrical efficiency
• Cooling efficiency
• Water consumption
• Reliability
• Capital cost
• Operating cost

Reliability and Controls

A high-density AI data center can concentrate an extraordinary amount of computing capacity into a relatively small area.

That makes reliability critical.

A power interruption can stop thousands of GPUs, while loss of cooling can cause equipment temperatures to increase very quickly.

Critical infrastructure may therefore incorporate redundant:

• Electrical sources
• UPS systems
• Pumps
• CDUs
• Cooling equipment
• Controls
• Heat-rejection equipment

Monitoring is equally important.

Sensors and control systems continuously monitor parameters such as:

Power | Temperature | Coolant Flow | Pressure | Equipment Status | Leak Detection

Data Center Infrastructure Management systems, building automation systems, and equipment-level controls allow operators to identify abnormal conditions and respond before they affect computing operations.

Modular AI Data Centers

There is another major challenge facing AI data centers:

Speed of deployment.

Demand for AI computing capacity can grow much faster than conventional data centers can be designed and constructed.

One solution is modular data-center construction.

Instead of assembling every system independently at the construction site, major portions of the infrastructure can be manufactured, assembled, integrated, and tested in a factory.

Depending on the design, modules can contain:

• IT equipment
• Electrical distribution
• UPS equipment
• Cooling systems
• Pumps
• CDUs
• Controls
• Piping
• Other supporting infrastructure

The completed modules are transported to the project site and interconnected.

Additional modules can potentially be added as computing demand increases.

This approach can reduce field installation requirements, improve factory quality control, and accelerate deployment.

We’ll explore this subject separately in our upcoming article and video on Modular Data Centers and the concept sometimes referred to as a “Data Center in a Box.”

The Big Picture

AI may run on software, but the infrastructure supporting it is very physical.

Thousands of GPUs require enormous amounts of electrical power.

That power becomes heat.

And that heat has to be continuously captured, transported, and rejected.

The entire engineering problem can therefore be reduced to one simple concept:

POWER IN → GPU COMPUTING → HEAT OUT

As GPU densities continue to increase, data centers are evolving with them.

Higher-capacity electrical systems, high-density power distribution, direct-to-chip liquid cooling, CDUs, facility water systems, advanced heat rejection, sophisticated controls, and modular construction are all becoming increasingly important parts of the AI data-center infrastructure.

The technology inside the racks will continue to evolve.

But the fundamental engineering challenge remains the same:

Deliver the power. Keep the equipment operating. And get the heat out.

For more detailed explanations of data-center electrical systems, UPS systems, redundancy, chilled-water systems, CRAC and CRAH units, liquid cooling, immersion cooling, and other critical infrastructure, visit the MEP Academy Data Center video series.