Making Ice to Lower PUE and TCO

At night, demand on the grid is lower, energy costs tend to be lower, and temperatures are also lower. These three factors make night an attractive time to produce thermal storage. This allows facility managers to time-shift HVAC-related energy costs to reduce peak demands on the grid and lower energy costs.

Although some facility managers have developed their own methods for time-shifting HVAC energy requirements, Ice Energy may be the first vendor to market a product specifically designed to do this. The Ice Bear* distributed energy storage system provides up to 5 tons of cooling load during peak hours.

It's good to see innovative products like this coming to market.

--kb

SSD Potential Power Savings Writ Large

E-week has an interesting article about an iSuppli report suggesting that if all data centers moved 10% of their hard disks to SSDs (solid-state disks), it would save more power worldwide than the total 2006 power consumption of the African country of Gambia. This equates to more than 166,000 Megawatt hours.

However, since the analysis assumed an SSD averaged 7W, SSDs that use less than 2W could save more than 3x that amount. Adding in HVAC and power infrastructure savings, the savings could be even higher.

--kb

Building Codes and Roof Anchors

One of the biggest impediments to installing rooftop solar panels or wind energy devices such as helical windmills is the cost and risk associated with properly anchoring them to a building's roof. Extra effort is required to ensure retrofit roof penetrations do not leak; in many cases, installation of the roof anchors takes the majority of the installation time. The noise from cutting through the roof can significantly disrupt those on the floor or two below.

If "rooftop renewables" are designed into a building during initial construction, the cost is substantially lower. However, it may not be feasible to install these rooftop renewables when the building is first built (due to limited capital or other reasons).

A middle ground is to provide rooftop anchors during initial construction, regardless of whether or not rooftop renewables are installed with initial construction. That way, solar or wind devices could be installed at a later date much more easily and with no need to breach the roof seal.

All new data centers should be designed for the later installation of rooftop renewables, even if they aren't part of the initial build-out.

Taking this a step further, I advocate the following: building codes should be revised to **REQUIRE** all new commercial buildings with a roof area greater than 1000 square feet to install roof anchors every x (20?) feet, with a TBD load rating for each anchor. (These anchors should also be required when major roof renovations are initiated as well.) Requiring these anchors will facilitate the broader adoption of rooftop renewables in data centers and other commercial buildings.

I hope others will adopt this cause; together we can effect real changes,
--kb

Evaluating Google's Battery-backed Server Approach

As noted previously, Google has disclosed that they put batteries on every server (see this picture of a Google rack), essentially powering their servers like the way laptops have traditionally been powered. The batteries are needed on laptops because they need to be mobile, which is not generally a consideration for servers.

Are batteries in servers a good idea?

There are some definite advantages in Google's approach:

1. No need to pay for UPS systems (saves CapEx dollars)
2. Eliminates two conversion stages found in a traditional AC double-conversion UPS
3. Reduces dedicated floor space/real estate commonly devoted to UPS/battery rooms
4. Localizes fault domains for a failed server to just one server
5. Scales linearly with the number of servers deployed

All of these add up to a solution that works just as well for one server as it does for one thousand servers. Coupled with Google's efforts to increase energy efficiency through founding and support for the Climate Savers Computing Initiative (CSCI) and its target of 92% power supply efficiency, this solution appears to be very efficient.

However, there are some down sides to Google's approach:

1. A lot of batteries to wire up and monitor
2. Increased air impedance from blocking airflow
3. Lower battery reliability with increased ambient temperatures
4. Higher environmental impact due to increased battery materials
5. Individual server supplies are exposed to a higher level of power transients and harmonics
6. Potential phase imbalances and stranded power in data centers

Issue #1 is self-obvious. Issue #2 can be seen from this picture from Green Data Center Blog; the physical mass of the batteries blocks a good portion of the air space in front of the server, which increases the resistance and in turn requires more fan power to move the same amount of air.

Issues #3 and #4 are somewhat related. Google, Microsoft, and other leading internet companies have advocated moving the ambient temperatures of data centers to higher temperatures, with some advising 35°C, 40°C, or even occasionally 50°C ambient temperatures. There are clear savings to be had here, but it may run counter to the battery approach used by Google. Assuming the Google batteries are conventional lead-acid batteries, a common rule is that the useful life of batteries drops by ~50% for every 10°C above 25°C ambient temperatures. Thus, a 4-year battery would only be good for ~2 years in a 35°C environment. In comparison, conventional UPS batteries are often rated for 10, 15, or 20 years. When consolidated in a UPS battery cabinet, the batteries can be protected from the higher ambient temperatures through localized cooling (batteries dissipate almost no heat) for increased life.

Lots of little batteries like Google uses results in more materials usage compared to the use of larger batteries. Couple that with reduced battery life at higher temperatures, and the result is not as good as it first seems. According to http://www.batterycouncil.org/LeadAcidBatteries/BatteryRecycling/tabid/71/Default.aspx, more than 97% of lead from lead-acid batteries is recycled, but this also states that 60-80% the lead and plastic of new batteries is recycled material. Looking at this last stat a different way, 20-40% of lead-acid battery materials are not recycled. Thus, even if Google performs 100% battery recycling, using lots of new batteries still results in the use of a lot of new materials.

I'll address issues #5 and #6 in a future post.

--kb

Suggestion for Energy Star Measurement of Blade Power Consumption

The US EPA is developing an Energy Star for Servers specification. Based on information in the latest draft of the specification, it looks like the EPA may be backing away from including servers in the first release ("Tier 1") of the server Energy Star specification. Given the increasing prevalance of blade servers in data centers, this would be unfortunate.

Ideally, there would be a standardized benchmark like SPECpower_ssj2008 that would be able to measure power consumption on a per-blade basis, but the current benchmark doesn't have provisions to handle chassis.

As an alternative, here are suggestions for how the EPA could measure power consumption for Energy Star (until a chassis-friendly industry specification is developed by an industry group like SPEC):

• Apply Energy Star to blades, not to chassis. Chassis are ineligible to meet Energy Star, but the blades that go in them can be Energy Star certified.
• Configure a chassis with the minimal amount of chassis management modules and external modules required for operation, but include all supported power supplies for a given chassis and all the fan/cooling modules typically used (don't remove redundant fans or power supplies).
• Run a sample workload on all servers to keep them minimally active. Install the same server configuration in all server slots.

Measure total power consumption to all power feeds in the chassis under two conditions and with the following calculations:

1. Condition 1: Determine power consumption P1 with all N server blade slots installed.
2. Condition 2: Remove servers so that N/2 (round up) servers are evenly distributed in the chassis; call that number N'. Determine power consumption P2 at this level.
3. P3 = P1 / N. This is the weighted average power per server blade in a full chassis.
4. P4 = P2 / N'. This is the weighted average power per server blade in a half-full chassis.
5. P5 = (P3 + P4) / 2. This is the weighted average power per server blade.

Notes:

• This accounts for chassis overhead, including fans, power supplies, management modules, and network connectivity. There is a slight penalty to blades here since rack-mount servers don't include any allocation for network switch power, but represents the minimum configuration needed to use those blades. Additionally, many vendors have low-energy networking elements (i.e., passthrough blades) that minimize this impact.
• If the chassis contains power supplies to convert input voltages to a different voltage supplied on the backplane, the power supplies used in the chassis must meet the power supply qualification requirements outlined elsewhere in the Energy Star for Servers specification.
• If a chassis contains redundant power supplies, the server blades are eligible for an allowance of 20W per redundant power supply, divided by the number of servers. For example, if a chassis has 2+2 power supplies (2 redundant power supplies and 2 minimum power supplies for a fully loaded chassis) and 10 blades, then each server would get a 4W/server allowance (2 * 20W / 10 servers).

With all the notes above, this may look to be complicated, but it's actually a fairly simple configuration that provides a close analog to how standalone rack-mount servers are tested. This could be used in the initial version ("Tier 1") of the Energy Star for Servers specification if the EPA wanted to use it.

--kb

Eliminating the UPS Efficiency Penalty with -48Vdc: Part II

In Eliminating the UPS Efficiency Penalty with -48Vdc, there is a discussion of how a non-redundant AC and DC configuration can have nearly equivalent efficiency in facilities without a UPS. However, when redundancy is figured in, the advantages of DC power become more pronounced.

Let's start by looking at the power supply unit (PSU) component by itself. Based on the information in the quantitative analysis by The Green Grid, high-efficiency AC and DC power supplies look like this when compared to each other:

The graph shifts to the right when redundant power supplies are considered. Since there are numerous different voltage converters in a server (modern servers often have in excess of 25 voltage rails used internally), it's really impractical to try to duplicate every voltage converter in a server--at least if you want it for a reasonable price. However, servers with redundant power supplies provide three principal benefits:

1. Connectivity to separate primary power sources (i.e., different utility feeds)
2. Protection against failure in upstream power equipment (i.e., failure in a PDU)
3. Cabling problem or service failure (i.e., accidentally unplugging the wrong server)

In an AC system, separate power supplies are required to have redundant feeds, since each power feed might be slightly out of phase with the other feed by the time the power signal gets to the server (relative phasing can shift in different parts of the data center based on relative cable lengths). If a server has two power supplies equally sharing the load as is commonly done, then each power supply <50%>

In contrast, a DC system has no phasing issues to deal with. Therefore, DC-based equipment has two main options: full duplicate power supplies (like AC) or using a technique called diode OR'ing (or FET OR'ing) to safely combine power from two separate DC sources as inputs to a single power supply. [Since there are numerous downstream power converters that are not redundant, there's no need for the power supply itself to be redundant--it just needs to be fed from multiple inputs.] Many DC power supplies do this today, as this approach is commonly used in the highly-reliable telecommunications system with -48Vdc systems. The result is a wider gap between the net AC power supply efficiency and the DC power supply efficiency:

Taking this a step further, look at the typical operating point for servers vs. their power supply ratings. For example, look at the various published reports for SPECpower_ssj2008: you'll notice there are numerous cases where the power supply shipped with the system is 2-4 times the maximum power draw in the sytem. If the power supply in a system is 2x the necessary power, then the system would normally operate in the left half of the graph immediately above. If the average power is considerably less than the maximum power draw, then the system could spend the bulk of its time operating at the 25% load level or less in the graph above.

At these lower loads, the efficiency benefits of -48Vdc systems become more apparent, even when there's no UPS in the picture. If an installation uses UPSes, the efficiency gap widens further in favor of -48Vdc.