Power usage effectiveness in data centre management

October 3rd, 2017, Published in Articles: Energize

Power usage effectiveness (PUE) is growing in popularity as an efficiency metric by the data centre market. While PUE is a tool to deploy an approach, or develop an implementation within a data centre, the measurement of PUE and its accuracy could have an increasingly negative impact.

Where the energy manager of a data centre relies on the reported PUE value to manage power supply, to adjust IT load, to improve the cooling system, and fully understand the running capacity of the data centre, or to decide when expansion is necessary, it is vital that the manager understands that the reported value, which is the measured value, might be different to the real operational PUE value of the data centre, and could jeopardise all of these important decisions, even when this variation is of a small percentage.

Some examples of possible consequences include:

  • Rectifying efficiency issues currently existing within the facility becomes intolerable
  • Correctly addressing power allocation into IT loads
  • Planning expansion projects timely or effectively becomes impossible
  • Not knowing capacity limits to provide clients with a high quality of data processing ability

 PUE, energy cost and accuracy: “the problem”

The severity of this variation can best be understood when it is translated into actual cost.

According to the Green Grid Association’s (Green Grid) recommendation for PUE measurement, three levels have been introduced for the implementation of measurement techniques within a data centre.

  • Level 1: The simplest way, with the least number of measuring points, which is also the cheapest in terms of implementation costs.
  • Level 2: The medium category
  • Level 3: The most advanced and expensive way

Assume a data centre with 1 MW capacity, operating with PUE of 1,6, has a total energy cost of R7-million/year.

The total energy cost has a direct relation to the value of the PUE reported. Any variation in PUE results in a variation with the total annual cost. The challenge here, is that if the PUE reported is not correct or accurate enough, how can an energy manager forecast expansion, modify systems or rectify energy allocation to improve efficiency within the current operation of their data centre?

It is impossible to make any correct decisions when one is in that situation, if the reported PUE is not close in value to the actual PUE in the centre. The three level measurement technique frameworks by Green Grid can help us.

Fig 1 Simulated ammeter for ease of reading.

Level 1

Level 1 is the least sophisticated, simplest PUE measurement category. Our example data centre is 1 MW and the actual PUE that it is operating is 1,6. Considering ±30% accuracy in PUE value for this level, the actual PUE could have been anything between 1,12 (-30% of 1,6 reported) to 2,08 (+30% of 1,6 reported).

These two new boundaries provide two different total electricity costs:

R4,9-million as the lower limit and R9,1-million as the upper limit. The difference between the two is R4,2-million. This is the result of the difference in initial PUE translated into cost value. The variation value is 60% of the initial annual electricity cost (the R7-million mentioned earlier).

The same calculation can be applied in similar manner to levels 2 and 3. In level 2, consider ±10% accuracy in measurement. Applying this percentage to PUE of 1,6 result is 1,76 (for +10%) and 1,44 (for -10%).

 PUE total annual energy cost

This variance translates this PUE into a cost value of R7,7-million (upper limit) and R6,3-million (lower limit). The difference between these two-upper limit and lower limit is R1,4-million. Again, if the data centre was running at a PUE of 1,6 – which was its original reported value, it would have had R7-million as its total annual electrical cost. The difference between the upper and lower limits, which are the actual PUE, compared with this R7-million considered is equal to 20% of annual electricity cost of the data centre.

This indeed, is a great amount of capital to be planned, budgeted or considered for in operation. The difference here is less if compared to the other two levels but individually, it is still a lot of money.

Therefore, measurement of the PUE and accuracy of this measurement plays an important role in considering the current or future value, or future expansion planning, so as to make the right decision. We know that whichever level one chooses to measure, one will experience this issue. In some levels it will be less, such as level 3, while in some cases it will be far higher as in the case of level 1.

Objective of the study

Our objective here is to assist in the avoidance of inaccuracies, whichever level the measurement technique uses. To provide an accurate, correct value, a value reported close enough to the real value to minimise the risk of mismanagement of budget, the misconception of values and to help energy managers make an informed executive decision with reliable numbers which are accurate. In other words, information which is reliable enough to help managers operate and plan their data centres properly.

 Having highlighted relevant issues with PUE measurement, the most important areas are discussed below.

Fig. 2: Touch screen control panel with multiple results displayed.

 

Accuracy

The measurement unit should be highly accurate, such as Class 1. This will enable the correct reporting, especially when it is duplicated to multiple breakers. As far as measurement is involved, metering and display is an important capability. Obviously in this application, if metering can be integrated into components of the distribution unit, it will help to reduce the complexity and improve the simplicity within an electrical network of a data centre. In terms of functionality this needs to meet the IEC61557-12 standard in terms of the methodology in which measurement is collected.

Measurement of earth fault current becomes highly advisable and the ammeter should be able to operate on a self-supplied mode or with an auxiliary voltage (Fig. 1). To go beyond a simple ammeter measurement of current, it is recommended that the measuring unit acts as a full multimeter to measure:

  • Voltage: phase-to-phase, phase-to-neutral
  • Power: active, reactive, apparent
  • Energy: active, reactive, apparent
  • Frequency
  • Power factor by phase and total
  • Peak factor

Frequency of data collection

In PUE measurement, frequency of data collection is highly important. Thus, measurement should be followed with information regarding frequency of measurement or maximum and average values per phase. Information should be stored regarding:

  • Date, time, fault current per phase and type of protection tripped over the last 30 trips
  • Date, time and type of operation for the last 200 events (for example: opening/closing of circuit-breaker, pre-alarms, editing settings)
  • Number of mechanical and electrical operation
  • Total operating time
  • Contact wear
  • Date and time of the last maintenance carried out, in addition to the estimate of the next maintenance required
  • Circuit-breaker identifying data: type, serial number, firmware version, name of the device as assigned by the user.

Communication protocol

Supervision systems which transfer data for PUE measurements often vary in terms of protocol from one subsystem to another. The reality is, having multiple sections communicating individually on separate protocol becomes a dilemma, and is an extremely hard task to have measured results reported as one integral value for the operator. If products within the distribution network are capable of communicating with different protocols, this predicament can easily be resolved. The following protocols to be recognised are:

  • IEC 61850
  • Modbus TCP
  • Modbus RS-485
  • Profibus
  • Profinet
  • DeviceNet
  • EtherNet/IP

Redundancy and reliability

Data centres are sensitive in character and operational relevance, and the PUE measurement is highly sensitive too. Loss of data for any reason, through maintenance or service on the supervisory system or actual distribution components which will disrupt the continuous measurement or reporting. Avoiding loss of data is crucially important and any operation related to this is highly sensitive.

A solution to this problem is the introduction of redundancy communication functionality, where on a single device, with a single unique IP address, two networks can be connected and communicate at the same time. One can be deployed for measurement and reporting purposes and the other for monitoring and maintenance. If either one of these networks is disrupted, the second network will still be active and can continue communicating with the device to receive information and report values of measurement.

A touch-screen display is a great advantage in that it simplifies information; this information can then be communicated to a central command hub in addition to the physical switchboard (see Fig. 2). This can be all gathered in a common point or alternatively can be installed elsewhere further from the switchboard.

Watch dog function

As far as a sensitive operation such as a data centre is involved, having a watch dog functionality can be beneficial in that it can enable proper functionality of relay actions within the circuit breaker. Having a mechanism which can periodically control the continuity of its own internal connections (trip coil, rating plug and current sensors-Rogowski coil) is of high importance. Having a monitoring circuit which can be set to indicate alarms or simulate commands such as a circuit breaker failure or circuit breaker opening, can help to detect a malfunction and enable the chain of items involved to be checked.

In addition, in some cases when the HVAC experiences difficulties and the temperature inside the data centre increases, this higher temperature starts to affect the measurement quality and can damage the equipment involved without any indicative signal. Self-protection against abnormal temperature inside the protection trip unit can overcome these scenarios by merely indicating an alarm or opening the circuit-breaker, either of which can be set by operators.

 Power quality analyser

A network analyser can help in evaluating the quality of energy within the electrical distribution of a data centre. The harmonic content or changes in voltage (in the long or short term) cause significant damage to the switchgear. These malfunctions reduce the life span and increase losses which contribute in the reduction of energy efficiency.

With the use of a power analyser, the root of a problem with an increase in power loss in transformers, motors or a reduction in the lifespan of cables and capacitors can be identified. If this analyser is embedded within components of a distribution system, there will be no need to install any external instrument.

The analyser monitors the system and its energy quality continuously and interfaces the result either through a display or through a communication output.

The following values should be monitored:

  • Hourly average voltage value: Based on an international standard, it must remain within 10% of rated value. An operator can set different limits depending on the requirements of the system.
  • Positive sequence voltage: This is obtained from three line voltages compared with a set limit. If values exceed preset limits, the power quality analyser generates a signalling event indicating this difference. These events are stored in a counter which can keep operation data for the past seven days. Calculation intervals can be set between five minutes and two hours measuring positive and negative sequence voltages as well as positive and negative sequence currents.
  • Interruptions or short dips in voltage: If the voltage remains below the threshold for more than 40 ms, the power quality analyser generates an event which is recorded in a dedicated log. The voltage is monitored on all lines.
  • Voltage transients, spikes: If the voltage exceeds the set threshold for 40 ms, set for a pre-determined time, the network analyser generates an event log.
  • Slow voltage sags and swells: When the voltage exceeds an acceptable range of limit values for a duration greater than the preset time, the network analyser generates an event log. Three values can be configured for voltage sags and two for voltage swells, each of which associated to a time limit. This enables the verification of network quality by understanding whether the voltage remained within a curve of values in an acceptable range for different loads (such as servers and computers) or not. The voltage is monitored on all lines.
  • Voltage unbalances: If the voltage is unequal on all phases or phase displacement is not exactly 120°, an imbalance occurs which is manifested with a negative sequence voltage value. If this limit exceeds the threshold value set, an event is stored which is counted in the network analyser’s log.
  • Harmonic analysis: The harmonic content of voltages and currents measured to the 50th harmonic as well as the value of total harmonic distortion (THD) is available in real time on a display or through communication modules. The power quality analyser can generate an alarm if the value of the THD or at least one of the harmonics exceeds pre-determined values. The voltage and current is monitored on all phases.

Conclusion

PUE measurement, its translation to energy cost and the accuracy impact that this variation will have, has been demonstrated. Using cost as an indication, any variation efficiency results in a huge amount of cost. Furthermore, solutions have been introduced which can help distinguish these factors. High measurement accuracy, frequency of data collection, communication protocol, avoidance in loss of measured data and power quality analysis capability are some of these features.

These functionalities can assist energy managers of data centres to make executive decisions with credible data. These decisions may be to enlarge the facility, or to modify the approach to maintain service quality, but whichever it is, it requires promptness in action and confidence in the data driving the decision to consider any proposal.

If the energy data is the basis on which to act, then the data needs to be credible and accurate enough to advise the decision makers. That is exactly what these features will provide as a tool for executive decisions and a guarantee for business success.

Contact Lebo Kgaye, ABB, Tel 010 202-5246, lebohang.kgaye@za.abb.com

 

Subscribe to our leading email newsletters

FREE-OF-CHARGE

CLICK for other EE Publishers information products