From Energy Visibility to Energy Savings in the Data Center

The Green Grid™ reports that the average Power Usage Effectiveness (PUE) of survey respondents who measure it, is 2.03 – essentially unchanged from the prior year (other industry surveys indicate an even higher average). This means that, on average, as much or more energy is being spent on infrastructure as on IT equipment – not a desirable goal. The report goes on to say that even the largest (Tier 4) data centers show an average PUE of 2.1. Clearly, energy savings opportunities exist at all tiers of data centers.

However, the best-in-class data centers have been able to achieve a PUE of 1.2 – a 40% improvement over the average. How do they do it? A strategy of “Measure and Manage” can have a huge impact. Here are four steps to gaining visibility into data center energy usage and realizing substantial savings using Arch Rock's Energy Optimizer wireless energy monitoring system.

Step 1. Start Monitoring Power and Thermals

1. The first step to savings is gaining visibility through instrumentation:
2. Install wireless thermal sensors at rack inlets and exhausts. ASHRAE recommends measuring racks at multiple elevations at the inlet and exhaust side of the racks.
3. Install wireless thermal sensors at CRAH returns and supply.
4. Install wireless differential pressure sensors at key points throughout the raised floor and underlying cold air plenum.
5. Install wireless power sub-meters on the IT loads (UPS, PDUs) and mechanical loads (Chiller, CRAHs).
6. Baseline the starting PUE and monitor it in real time as you make changes using the Arch Rock Energy Optimizer dashboard.

Note that the rack inlet temperatures at the right are out of ASHRAE guidelines, which has generated an alert. Also note that the CRAH return temperatures (much cooler than expected) at lower right indicate possible significant bypass airflow. The first of these two findings (hot spots) represents a challenge: you will not be able to raise the supply air temperature set points unless other measures are implemented first, as indicated below.

The second finding (bypass airflow) represents an opportunity for savings. If bypass airflow is prevented, you may save on over-sized cold air delivery efforts as a primary effect, and on costly re-humidification in the CRAHs as a secondary effect (from condensation during cooling of the already cool return air below its dew point).

Step 2. Take Action While Tracking PUE and Rack Inlet Temperatures in Real Time

Design guidelines published by the California electric utility PG&E for high performance data centers indicate that proper airflow management alone can improve the efficiency of CRAC units by up to 50%. In a data center with a PUE of 2.3, this can result in 23% savings from your baseline (assuming 80% of the infrastructure load is the cooling system and 20% is UPS loss plus lighting). This would translate to a PUE of 1.78.

Once the instrumentation is in place and a baseline established, you can take action. As shown above, thermal maps can help you fix hot spots and find the bypass air going back into CRAHs. Air pressure differentials around the raised floor can help to locate dead spots, obstructions, or too many perforated tiles installed. Replacing selective perforated tiles with solid tiles may actually improve airflow of the cold air and CRAC efficiency. It may even be possible to safely turn off redundant CRAC units to save energy, while watching for temperature alerts from the sensors. Continuous monitoring with real-time data will allow you to iterate this process and realize maximum improvement in a very dynamic environment.

With tighter airflows and alerts on hot spots, you can safely raise the temperature set points on the cooling plant. The PG&E paper finds that every degree Fahrenheit rise in the chiller water can improve chiller plant efficiency by 1-2%. Increasing the set point of the chilled water from 44° to 57° F enables an additional 5-8% savings from baseline in the same data center, and results in a PUE of 1.59.

Step 3. Upgrade to Improve PUE and Save Energy or Gain Capacity

Cold- or hot-aisle containment is an increasingly popular method of improving airflow and gaining even greater savings. One study suggests that up to 40% improvement may be possible in the efficiency of CRACs, resulting in another 5-8% savings from baseline and a PUE of 1.4.

This Energy Optimizer trend graph shows the results of one cold-aisle containment project. Before containment, rack inlet temperatures had an 8° F spread and CRAH return air averaged 75° F. Afterwards, rack inlet temperatures are lower and have a smaller spread, while CRAH return temperatures have risen. The data center’s rack space is now cooler and more efficient.

Additional measures that deliver even more savings, such as retrofits of the HVAC motors and pumps to VFD, airside and waterside economizers for free cooling, more efficient UPS, etc., can reduce the data center PUE even further, matching industry-leading levels of around 1.2.

Savings Summary

The table below summarizes how a data center with a PUE of 2.3 can save up to 39% in infrastructure energy costs and lower its PUE to 1.4 with the help of active real-time monitoring from Arch Rock Energy Optimizer.

Step 4. Maximize Your Gains with Continuous Monitoring

As every facility and IT manager knows, data centers are highly dynamic environments, with significant moves/adds/changes occurring daily. Reconfigured racks are often left with missing blanking panels. Real-time, ongoing monitoring is the only way to stay on top of the evolving status of the data center to preserve the gains made in PUE, capacity and savings. Threshold alerts sent from Energy Optimizer can tell you:

• When a rack’s inlet temperature has exceeded ASHRAE guidelines, pinpointing possible air mixing/missing blanking plates in or near the rack
• When the differential pressure in a contained aisle has dropped, indicating doors left open; or when the differential pressure across the supply air plenum and raised floor has dropped, indicating possible open tiles or obstructed air paths
• When the CRAH return temperatures have dropped, indicating possible air bypass

Next Steps

The best practices mentioned in this paper -- managing air flow to prevent bypass, raising HVAC set points while monitoring rack inlet temperatures, preventing hot-cold air mixing by aisle containment, etc. -- will help to improve PUE and save energy or gain capacity in data centers small and large. But without creating a baseline at the start and monitoring the environment continuously, these gains may be smaller to begin with and will rapidly degrade over time.

Arch Rock Energy Optimizer can be deployed with zero downtime, no disruptive wiring, and no risk to the ongoing operation of the data center's power or cooling infrastructure. For more information, visit http://www.archrock.com, or contact Arch Rock at +49 6403 6099300, Hans-Georg Bieschke

References

Energy Measurement Survey Results: White Paper #26. The Green Grid. 2009.
High Performance Data Centers: A Design Guidelines Sourcebook. PG&E. 2006.
Cold Aisle Containment for Improved Data Center Cooling Efficiency: Rittal White Paper 506. Rittal Corporation. 2009.

Revision 1.1. March 10, 2010