HP ML110 G6, CPU and ESXi

Before installing VMware ESXi (or any other OS) it is good practice to explore the BIOS of the server hardware. Poorly configured BIOS settings can have major performance impact on the installed OS and default settings are not automatically the best settings.

While writing this post, I stumbled onto some BIOS settings, I did not fully understand. That makes it difficult to choose the correct settings. In this post, I will show you how to deal with some “CPU Power Management” settings on a HP ML110 G6 server.

Before you start, make sure which CPU’s are supported in the server, some BIOS settings are not relevant for your CPU. Example: the ML110 G6 supports[1] Intel Xeon X3430, X3450, X3460, In the BIOS under “Advanced Processor Options” there is the “Processor Hyper-Threading” option. The X3430 does not support Hyperthreading, which makes this option irrelevant. However the other models do support Hyperthreading.

The first question is, which processor is installed? A HP ML110 G6 will show the CPU type during the boot process. In case you have VMware ESXi already installed, you can also determine the CPU type from the vSphere Client, under Configuration.

Now it is time getting to know your processor. Today, both Intel and AMD have a wide range of CPUs with many different options. In this example, the ML110 G6 is equipped with an Intel Xeon X3430 CPU. With a little help from Google, you will find this information from Intel:

Wikipedia provides some compact background information:

Now it is time for some advanced setting. In this post, I have already explained some of the “Advanced Processor Options”. Now let’s have a look at the “CPU Power Management” settings.

BIOS => Advanced => Advanced Processor Options => CPU power Management

EIST (GV3)

EIST is the abbreviation of: Enhanced Intel SpeedStep.  According to Wikipedia, “SpeedStep is a trademark for a series of dynamic frequency scaling technologies (codenamed Geyserville and including SpeedStep, SpeedStep II, and SpeedStep III) built into some Intel microprocessors that allow the clock speed of the processor to be dynamically changed (to different P-states) by software. This allows the processor to meet the instantaneous performance needs of the operation being performed, while minimizing power draw and heat dissipation.”
By the way, I was not able to find out what GV3 means, anyone out there?
The Intel X3430 does support EIST,

Turbo Mode

Turbo Mode reflects to the Intel Turbo Boost Technology, which is available on the Intel X3430.

According to Wikipedia, “Turbo Boost is a technology that enables the processor to run above its base operating frequency via dynamic control of the CPU’s “clock rate”. It is activated when the operating system requests the highest performance state of the processor. Processor performance states are defined by the Advanced Configuration and Power Interface (ACPI) specification, an open standard supported by all major operating systems; no additional software or drivers are required to support the technology. The design concept behind Turbo Boost is commonly referred to as “dynamic overclocking”

The clock rate of an Intel X3440 CPU is 2,4 GHz. Under circumstances Turbo Boost allows a core to run above this clock speed.

CPU C1E

A good summary about ACPI, Processor and Performance States is (again) this Wiki Page. In this article, the C1 and C1E state are also mentioned. “C1 (often known as Halt) is a state where the processor is not executing instructions, but can return to an executing state essentially instantaneously. All ACPI-conformant processors must support this power state. Some processors, such as the Pentium 4, also support an Enhanced C1 state (C1E or Enhanced Halt State) for lower power consumption.”
With enabling C1 Enhanced Mode, all logical processors in the physical processor have entered the C1 state, the processor will reduce the core clock frequency to system bus ratio and VID

In this blog post Intel’s Taylor Kidd explains that C states is all about Idle states and there are a lot of different C-states, like Core C-states, Processor C-states and Logical C-states. C0 is the operational state, where the CPU is doing useful work and on the other end, C3 and C6 are “deep sleep” states.

Idle states are supported on a Intel X3440.

CPU C3 Report, as the comprehensive item specific help states, this setting reflects to the core C3 state which matches the ACPI C2 state (C2. often known as Stop-Clock,  is a state where the processor maintains all software-visible state, but may take longer to wake up).

And finally, CPU C6 Report, as the comprehensive item specific help states, this setting reflects to the core C6 state which matches the ACPI C3 state (often known as Sleep, is a state where the processor does not need to keep its cache coherent, but maintains other state. Some processors have variations on the C3 state (Deep Sleep, Deeper Sleep, etc.) that differ in how long it takes to wake the processor).
So the C3 and C6 Report options allows you to enable extra sleep states.

And finally, the question remains, what are the optimal BIOS settings?
Before we conclude let us have a look at this VMware document “Performance Best Practices for VMware vSphere 4.1”

The “Hardware BIOS Settings” section has a few interesting points related to the Intel X3440 CPU:

• Make sure you are running the latest BIOS for your hardware.
  OK, always a good idea imho.
• ESX (and I assume ESXi also…) supports Enhanced Intel Speedstep (EIST) CPU Power management technologies that can save power when a host is not fully utilized. However because these and other power-saving technologies can reduce multi-threaded performance and increase I/O latency in some situations, you should consider disabling them when these particular performance considerations outweigh power considerations.
• Enable “Turbo Mode” if your processor supports it.
• Disable C1E halt state in the BIOS if multi-threaded performance and I/O latency are important considerations. Consider enabling C1E if “Turbo Mode” is present and single-threaded performance of  certain workloads is more relevant. (See note above regarding performance considerations versus power considerations.)
  The explanation is that to enable “Turbo Mode ” for one core, the CPU has to shut off other cores which is only possible with C1E state enabled. This gives you the “single-threaded” performance. The downside of enabling C1E state is that a few CPU cores may shut off. When these cores are needed, there will be some I/O latency.
• For the highest performance, potentially at the expense of higher power consumption, set any BIOS power-saving options to high-performance mode. Depending on the manufacturer, this might involve disabling an option (for example, “Demand-Based Power Management”) or enabling an option (for example, “Static High Performance Mode”).

This seems to relate to this BIOS setting:

If we want optimal performance, I would go for these settings:

and…

Notice that the “Turbo Mode” option is only available if EIST is enabled.

Final thoughts, in my opinion, one should care for his environment and energy savings.
This post is meant as an exercise how to handle advanced CPU options. In real life, you probably have a cluster of vSphere servers. In that case I would recommend that you consider using the Distributed Power Management(DPM) feature and configure the individual servers for optimal performance. The win of a server shut down by DPM is much bigger than tweaking CPU sleep states and possible performance loss.

As always enjoy reading and I welcome your feedback.

---

[1] See “HP PrLiant ML110 G6 Server Maintenance and Service Guide”