The Impact of Virtualization on Data Center Infrastructure
VMWare led the industry in server virtualization technology, or the “process of creating a software-based representation of something rather than a physical one. This applies to applications, servers, storage and networks.” (https://www.vmware.com/solutions/virtualization.html). As this technology hit the mainstream data center market it allowed for a more efficient IT platform by running a fewer number of servers at a higher power utilization rate.
Those who designed and build data centers took great interest in what this new technology and how it would impact the power and cooling infrastructure. At the same time the industry was re-examining how data centers were engineered and constructed to obtain greater energy efficiencies while maintaining reliability and uptime.
The Legacy Data Center
Up until about ten years ago most data centers operated as with servers placed on top of a raised floor system which was utilized for power and fiber pathways as well as the air delivery system used to maintain a very cold temperature (55 -60 degrees). Cabinet power densities were relatively low (around 2-3 kW/cabinet), and the IT cabinets saw utilization rates in the order of magnitude of 15%. Almost every watt of power in a server is rejected to the atmosphere, and needs to be cooled. Computer Room Air Conditioners (CRAC’s) were run at full speed.
Raised floor data center, Image courtesy of Cisco Systems
This resulted in very inefficient system and very high operational expenses. Data center operators didn’t care because the avoidance of thermal failure of a server and associated downtime took precedent over energy efficiency.
The Virtualized Data Center
Virtualization initially resulted in a fewer number of cabinets with higher power densities, close to 10 kW/cabinet. There was concern that in legacy data centers this would result in “hot spots”, where the existing cooling infrastructure could not handle this increased load.
In parallel, the American Society of Heating, Refrigeration, and Air Conditioning Engineers (ASHRAE) Technical Committee 9.9 (https://tc0909.ashraetcs.org) had been establishing the best practices for the cooling of datacom spaces, which based on years of research recommended higher inlet temperatures for IT servers, from 55 degrees to up to 80 degrees. To take advantage of these allowed higher temperatures the design and construction industry utilized a solution known as “aisle containment”, which is the installation of a physical barrier to eliminate the air flow from the back (hot side) of the cabinet from recirculating and entering into the inlet (cold side) of the server.
Example of “Hot Aisle Containment”, Image courtesy of Subzero Engineering,
What Happened to the Raised Floor?
As the implementation of virtualization became more commonplace and servers became more efficient and powerful, data center operators no longer feared operating IT cabinets at higher power densities (kW/cab). The evolution of data center design also questioned the need for raised access flooring as a cooling air delivery pathway as the higher cabinet power densities began to exceed the physical limitations of volumetric flow and air velocities of the perforated supply floor tiles.
Facebook recognized in 2011 that data centers are a necessity in our society, however the environmental impact was potentially a negative given the huge carbon footprint. As noted in a 2015 Business Insider article by Julie Bort, Facebook decided to go public with their designs in the Open Compute Project in an effort to push the data center industry into more efficient design and operations (http://www.businessinsider.com/facebook-open-compute-project-history-2015-6).
One of the first major data centers I had heard of without a raised floor was Facebook’s Prineville, OR facility (other data centers may have done this first, but not been as public about it). This approach of installing the server cabinets directly onto the slab while integrating aisle containment and delivering the cooling air from above or laterally into the data hall was unique. Since then it has become mainstream and I have used it many times in my own data center designs. This approach can provide airflow cooling to cabinet power densities of up to 30 kW/cabinet.
Data Center installed on Slab on Grade, Image courtesy of Facebook
Virtualization Has Changed the Industry
Virtualization is driving the industry to the model of a software defined data center, which allows for more efficient IT utilization. Inherently this would leave one to believe that potentially data centers could become physically smaller when in fact the opposite has happened. The major cloud providers are using virtualization and designing facilities capable of handling over 50 Megawatts of critical IT computing power, and High Performance Computing centers can see individual cabinet power densities in the range of 60 to 100 kW/cabinet. The engineering and construction industry has responded with new technologies and methods in cooling and power delivery based on industry research from organizations such as ASHRAE. With each new advance in IT based technology, the engineering and construction industry must evaluate it and respond with solutions that maintain both data center reliability and energy efficiency.
Users who have LIKED this post: