Server immersion cooling

Example of server immersion cooling of one server

Server immersion cooling is a computer cooling practice by which computer components or servers are submerged in a thermally but not electrically conductive liquid. The coolant liquid is commonly a mineral-based oil with dielectric properties.

Server immersion cooling has the potential of becoming a popular server cooling solution for green data centers,^[1] as it allows them to drastically reduce their Power Usage Effectiveness (PUE). Multiple relevant brands like Intel and Facebook have already validated the advantages of submerging servers.^[2][3]

Servers and other IT hardware cooled by immersion cooling don’t require fans, thus these are removed together with any thermal paste to avoid it from dissolving in the oil. The oils used in immersion cooling usually operate at temperatures between 35 to 50 °C (95 to 122 °F), this allows data centers to use much more efficient evaporative or adiabatic cooling towers^[4] instead of chiller-based air cooling.

Current commercial applications for immersion cooling range from data center oriented solutions for commodity server cooling,^[5][6] server clusters, HPCC applications^[7] and Bitcoin Mining.^[8]

History

Although still rarely used for cooling of IT hardware, the usage of mineral or vegetable-based oil for cooling large power distribution components such as transformers and industrial machinery is a common practice since the beginning of the 1900s^[9]

Some extreme density supercomputers such as the Cray-2 (released in 1985) and Cray T90 (released in 1995) used large liquid-to-chilled liquid heat exchangers and single or two-phase immersion cooling oils for heat removal^[10]

Server Immersion Cooling techniques

Open or Semi-Open Bath Immersion Cooling

Open or semi-open bath immersion cooling is a data center cooling technique that implies fully submerging servers in dielectric coolant fluid. Its called open or semi-open bath because servers are placed side-by-side in large tanks that assimilate bath tubs. These baths operate at atmospheric pressure and allow the coolant fluid to be pumped through the hardware components or servers submerged in it.^[11]

Open Bath Immersion Cooling

Open Bath Immersion Cooling uses single-phase oils. These oils always remain in liquid state while operating. They never boil or freeze. The oil is usually pumped to a heat exchanger where its cooled thanks to heat exchange with a cooler water-circuit. This technique uses “open baths“, as there’s little (or no) risk of the coolant evaporating:

Semi-Open Bath Immersion Cooling

In semi-open bath Immersion Cooling, two-phase oil is the working fluid. It literally boils thanks to its low boiling point^[12][13] and thus exists in both a liquid and gas phase. The system takes advantage of a concept known as “latent heat” which is the heat (thermal energy) required to change the phase of a fluid (in this case two-phase dielectric mineral oil). The oil is only cooled by boiling and thus remains at the boiling point (“saturation temperature”). Energy transferred from the servers into the two-phase oil will cause a portion of it to boil off into a gas (this is the second phase of the oil). The gas rises above the liquid oil level where it contacts a condenser which is cooler than the saturation temperature. This causes the vapoured oil to condense back into a liquid form and fall (rain) back into the bath.^[14]

The term "semi-open bath" means, that when the system operates, its sealed to avoid the evaporation or the coolant:

Sealed Server Immersion Cooling

A different approach to open or semi-open bath immersion cooling are sealed server immersion cooling solutions, where servers are specially built in an oil-tight casing. Dielectric oil is pumped inside each server to collect heat from the components. The heat is then taken to a backplane in the rack which is cooled by a water circuit.^[15]

The main advantage of this approach is that servers don't require to be placed in horizontal baths. The main disadvantage is that not any hardware can be used as the vendor defines the hardware specs of the sealed servers.^[16][17]

References