Server rack design and capacity planning

Our VMware local SE team has got a great Christmas present from regional Intel BU. Four rack servers with very nice technical specifications and the latest Intel Optane technology. 

Here is the server technical spec: 

Node Configuration	Description	Quantity
CPU	Intel Platinum 8280L (28 cores, max memory 4.5TB)	2
DDR4 Memory	768GB DDR4 DRAM RDIMM	12 x 64GB
Intel Persistent Memory	3TB Intel Persistent Memory	12 x 256GB
Caching Tier	Intel Optane SSD DC P4800X Series (750GB, 2.5in PCIe* x4, 3D XPoint™)	2
Capacity Tier	Intel SSD DC P4510 Series (4.0TB, 2.5in PCIe* 3.1 x4, 3D2, TLC)	4
Networking        + transceivers, cables	Intel® Ethernet Network Adapter XXV710-DA2 (25G, 2 ports)	1

These servers are vSAN Ready and the local VMware team is planning to use them for demonstration purposes of VMware SDDC (vSphere, vSAN, NSX, vRealize), therefore VMware Cloud Foundation (VCF) is a very logical choice.

Anyway, even Software-Defined Data Center requires power and cooling, so I've been asked to help with server rack design with proper power capacity planning. To be honest, the server rack plan and design is not rocket science. It is just simple math & elementary physics, however, you have to know the power consumption of each computer component. I did some research and here is the math exercise with a power consumption of each component:

• CPU - 2x CPU Intel Platinum 8280L (110 W Idle, 150 W Computational,  360 W Peak load)
• Estimation: 2x150 W = 300 W
• RAM - 12x 64 GB DDR4 DRAM RDIMM (768 GB)
• Estimation: 12x 24 Watt = 288 W
• Persistent RAM - 12x 256GB (3TB) Intel Persistent Memory
• Estimation: 12x 15 Watt = 180 W
• vSAN Caching Tier - 2x Intel Optane SSD DC P4800X 750GB
• Estimation: 2x18W =>  36W
• vSAN Capacity Tier - 4x Intel SSD DC P4510 Series 4TB
• Estimation: 4x 16W => 64 W
• NIC - 1x Intel® Ethernet Network Adapter XXV710-DA2 (25G, 2 ports)
• Estimation: 15 W

If we sum the power consumption above, we will get 883 Watt per single server.  

To validate the estimation above, I used the DellEMC Enterprise Infrastructure Planning Tool available at http://dell-ui-eipt.azurewebsites.net/#/, where you can place infrastructure devices and get the Power and Heating calculations. You can see the IDLE and COMPUTATIONAL consumptions below.

Idle Power Consumption

Computational Power Consumption

POWER CONSUMPTION

Based on the above calculations, the server power consumption range between 300 and 900 Watts, so it is good to plan a 1 kW power budget per server which in our case would be 4 kW / 17.4 Amp per a single power brach, which would mean 1x32 Amp PDUs just for 4 servers. 

For a full 45U Rack with 21 servers, it would be 21 kW / 91.3 Amp, which would mean 3x32 Amp per a single branch in the rack.

HEATING AND COOLING

Heating and cooling are other considerations. Based on Dell Infrastructure Planning Tool, the temperature in the environment will rise by 9°C (idle load) or even 15 °C (computational load). This also requires appropriate cooling and electricity planning.

Conclusion

1 kW per server is a pretty decent consumption. When you design your cool SDDC, do not forget for basics - Power and Cooling.