Cool your chips: What's ahead in energy management
April 12, 2007
Appeared in Computerworld
There are already ways to add liquid to your existing rack server, and cooling the chips themselves is a logical next step, some believe. Various technologies are at different stages of commercialization.
"There is going to be a return to liquid-cooled electronics like the old mainframe days," said Terry Rodgers, a consultant at Syska Hennessy Group, Inc. "A lot of the newer products out there today are very effective in getting heat out of a rack and allowing data centers to remain in operation, but they are not saving energy. The move to direct-component cooling is going to be needed if we are going to create truly more efficient systems."
Liquid is 3,500% more efficient than air, said Patchen Noelke, director of marketing at Spraycool, a division of ISR Inc. With many companies now embracing cabinet-level cooling, "it's only a short step to putting liquid inside the server, where the benefits are enormous for power savings," Noelke said.
Spraycool has two commercial products. The M-Series is a direct chip-cooling technology in which a module is attached to the surface of a processor or other system component. Inside the module, liquid is sprayed across a cold plate on top of the processor, removing as much as half the heat. Spraycool also sells the G-Series, used primarily in defense contracts, which sprays nonconductive fluid directly across an entire motherboard.
The M-Series modules are being evaluated by a range of businesses and labs, and the next major step in commercialization will be to get server makers to begin offering platforms that incorporate the technology. One partner in this effort is Smart Modular Technologies Inc. (SMT), a provider of memory subsystems that in February announced it would begin offering a family of DIMMs that combine the Spraycool modules with its own flexible circuit-board technology.
"This is a complete shift in how to design systems," said Arthur Sainio, senior marketing manager at SMT. "But it's really a no-brainer. It will be hard for [resellers] to dispute better performance, lower power and lower operating costs."
The Pacific Northwest National Laboratory (PNNL) is entering its third phase of cooling tests designed to evaluate and measure the effectiveness of the Spraycool technology. In the first two phases, the lab used Hewlett-Packard Co. servers with Itanium 2 processors to test the modules. The lab is now testing the modules on IBM servers using quad-core Xeon processors.
Stephen Elbert, associate division director for computational sciences and mathematics at PNNL, said the lab will need new cooling technologies by the end of the decade as it deploys hotter next-generation servers. The current tests are expected to yield metrics that can be used to justify further deployment of chip-level cooling techniques.
John Breakey, president and CEO of Unis Lumin Inc., a systems integrator and services provider, sees an opportunity for the Spraycool technology inside his firm's own data center and in the data center of its clients. Unis Lumin has been running a rack of x86-based servers using the Spraycool modules for nearly a year. He said he believes the technology delivers a 50% reduction in heat.
In addition, Spraycool-enabled processors are able to work at maximum performance levels, while most processors generally run at only 80% or less of rated power because of associated heat issues, Breakey said. The heat reduction and performance improvement will allow users to maximize data center density without having to buy new air-cooling equipment.
"We are dealing with projections of our power costs increasing by as much as 100% per kilowatt hour over the next year or two," Breakey said. "If my electricity costs are going up $150,000 a year, I might be motivated to spend $25,000 to $100,000 to fix that problem."
Other companies are also working on chip-level cooling technology. In 2005, Liebert Corp. acquired Cooligy Inc., which is developing an approach that sprays chemically treated water on top of hot components. More than 100 microchannels direct the coolant onto specific hot areas inside the chips. The technique has already been used in commercial workstations, but Liebert expects cabinet-level systems such as its XD series of cooling equipment, in combination with more traditional air conditioning, will remain the primary answer for spot cooling within data centers for the next several years.
"What's out on the market today is very effective for today's heat loads," said Steve Madara, vice president and general manager of Liebert's precision cooling business. "I do think these direct-chip cooling methods will find their way into enterprise servers in the next decade."
A team within HP was been working on a chip-cooling technique for several years that would use ink-jet spray heads developed within HP's printer division to spread drops of coolant on top of hot chips. Commercialization of this product, however, is apparently distant.
"The actual adoption or acceptance of chip-cooling technology will be driven by multiple factors, including the rate of change in compute density when more universal cooling solutions no longer work effectively," said Ron Mann, director of engineering for infrastructure solutions at HP.
Nevertheless, the move to chip-level cooling is actually moving faster than many may realize. Working committees within the American Society of Heating, Refrigeration and Air Conditioning Engineers, which has representation from server manufacturers including HP, IBM and Sun Microsystems Inc., late last year published guidelines for using liquid cooling within data centers.
Benefits of direct chip-level cooling "are so overwhelming that it will have to become a mass-market product," Spraycool's Noelke said. "That will need to begin with adoption by at least one leading server manufacturer."
That adoption could come this year, Noelke said, based on his ongoing discussions with multiple resellers including Dell Inc. and HP. There seems to be some acceptance of the chip-cooling notion within the server-provider camp, because Noelke said he has never seen a server reseller void the warranty on a system -- or decline to support one -- that has been retrofitted with Spraycool modules.