In my prior blog post, I compared the capabilities offered by server vendors in their power calculator tools. In this post, I'll give my recommendations for how to use these tools, talk about their accuracy, and suggest some improvements.
Power Calculator Do's and Don'ts
Rule No. 1: Don’t expect calculator to match exactly what you'd measure in the datacenter. That doesn't mean they're inaccurate; they just can’t predict your precise workload, nor can they account for everything that impacts power use. So, they all provide conservative (high) estimates.
With that in mind, here's my list of Do's and Don'ts for using server power calculators:
| How to Use Server Power Calculators | |
| Do |
|
| Don't |
|
| When Comparing Results... |
|
* Except for Fujitsu SystemArchitect.
Can You Trust Power Calculators for Circuit Sizing?
You can use calculators to predict what servers can safely be grouped onto an existing power circuit. However, most of these calculators’ designers say that you shouldn’t use them to define what circuits to lay out in your datacenter. (In fact, electrical codes in some jurisdictions require that circuits be designed based on faceplate numbers.)
The notes with one of IBM's calculators are explicit that their calculator “should not be used in planning the electrical supply infrastructure." That echoes a note from Oracle's calculators: "[They] should not be used to calculate power or HVAC requirements."
A Cisco white paper puts it best, saying the Cisco tool can be used to determine what PDUs are adequate but stresses that you should use site planning guides rather than calculators for the circuit sizing itself.
Where Does the Data Come from?
Vendors start with actual lab measurements on various configurations of each server model. Usually, power values are also measured for each subcomponent, such as a DIMM, allowing the calculators to extrapolate results from their limited set of measurements and cover all configurations. Generally, this extrapolation is not just a straight sum of component values; calculators take into account, for example, power supply efficiency at various power levels.
The initial lab measurements are typically done while running HPC benchmarks like LINPACK or server stress tests like Prime95. These applications put much more emphasis on the processor and memory subsystems than normal programs, and stress multiple subsystems of the server simultaneously. Thus, disclaimers on the power calculators from IBM, HP, and Cisco say that their estimates are likely much higher than you’d see from normal applications. Likely, but not certain: Cisco’s calculator contains a disclaimer that it’s possible with LINPACK to see higher consumption that the calculator shows.
How Accurate Are They?
That's probably impossible to say. "Actual results may vary," as the Dell ESSA disclaimer laconically puts it.
The primary reason for the variances is that different workloads can show dramatically different power consumption, as described in this blog post from Tony Harvey (coincidentally, someone involved with creating power calculators at both HP and Cisco). Another reason your results will vary: environmental conditions, such as ambient temperature and humidity, will impact actual consumption.
A less obvious reason for inaccurate calculator predictions is that two identically configured server models may yield different measurements, even under identical conditions. That’s because there are differences in the subcomponents used that might not be apparent or even discoverable. For example, vendors commonly use hard drives or memory DIMMs from multiple component suppliers when assembling a single server model.
I haven’t seen any comprehensive studies on power calculator accuracy. Most users with whom I speak agree that the calculator estimates are usually higher than actual consumption. The only study on the topic I’ve seen was a recent Cisco-commissioned report that monitored actual power consumption and then compared those values to what was reported by the Cisco and HP power calculators. The report called into question just how conservative HP's Power Calculator results were, since it measured higher results than the calculator showed. However, the report also highlighted that even Cisco’s calculator results were 15% to 25% different than the an independent lab measurements.
Comparing Calculators to Power Benchmarks
Sources of data that you can use to ensure that vendors are at least reporting consistent power measurements are Energy Star, SPECpower, and TPC-Energy results. These public reports include actual system-level measurements of power consumption for specific configurations.
| Energy Star Reports vs. Power Calculators | |||
| Value reported in Energy Star report | Value given by power calculator for same configuration | Difference | |
| Dell R710 - Idle | 129 watts | 150 watts | + 16% |
| Dell R710 – 100% | 248 watts | 288 watts | + 16% |
| HP DL380 G6 - Idle | 139 watts | 142 watts | + 2% |
| HP DL380 G6 – 100% | 261 watts | 316 watts | + 21% |
In all the cases I checked, Energy Star idle numbers were pretty close to what was in the Energy Star reports, while max power numbers averaged 20% higher in the calculators than in the Energy Star reports. That agrees with vendor documentation that says power calculators tend to be conservatively high.
Since SPECPower requires that server power be measured at different utilization levels, it should provide not just an accuracy check max power consumption, but also for all of the utilization levels in between.
The public SPECpower reports show lower power than the calculators. Again, that can be explained by workload differences between SPECPower and the more aggressive workloads used to populate the calculators. However, my comparisons show that most power calculators show a linear relationship of power versus utilization, while SPECPower shows a distinctly nonlinear relationship:
A non-linear relationship would be expected in actual measurements. While today’s power supplies and regulators are a lot better at being efficient over a wide range of output levels, they still show relatively poor efficiency when running at sub-30% utilization.
But as you can see below, most of the power calculators appear to model a pure linear relationship between power and utilization:
Only Fujitsu’s SystemArchitect consistantly shows non-linearity. Dell’s ESSA is linear except for one point: idle power (and I found examples where even that point was linear). Occasionally the idle point in Cisco’s new Power Calculator was slightly off from linear (including the example plotted above), but the difference was negligible.
Are Incorrect Utilization Curves a Problem?
So, it appears that power calculators aren’t accurately modeling power versus utilization. However, I don't think that's a problem for most people. Using a single “utilization %” number to specify utilization is a flawed concept, one that outweighs concerns over power calculator accuracy. The basic problem is: What does “50% utilized” mean? Does it mean that exactly half the transistors in a server are switching? Or does it mean the server is achieving 50% of its potential max score on some benchmark? Or that the operating system reports 50% utilization on the CPU? Or something else?
So, as long as you accept that the utilization-based numbers in power calculators yield ballpark and not precise estimates, linear curves for utilization versus power are OK.
Like utilization curves, power calculators appear to be linear for power consumption for partially-filled blade enclosures. Again, that doesn’t match what you’d expect based on power supply efficiency, but it could reflect the intelligent power management happening behind the scenes in blade enclosures.
What Vendors Can Do to Improve Power Calculators
The greatest favor vendors could do for power calculator end users would be to create a standard for how power is measured by the calculators and then describe it. (I’ve heard rumors that a standard is actually under consideration.) However, there are some less comprehensive changes vendors might consider:
- Show the impact of power-saving modes.
- Document how the data in the calculator was measured.
- Include modifiers for workload types.
- Cover more direct-attach storage gear within the calculators.
- Add release notes, so changes in each version can be understood.
Comments