ThinkSystem Hidden UEFI ParametersPlanning / Implementation

Published
15 May 2018
Form Number
LP0891
PDF size
6 pages, 73 KB

Abstract

The Lenovo server Unified Extensible Firmware Interface (UEFI) provides the interface between an operating system and the server firmware. Many UEFI tuning parameters can be changed by system administrators to fine-tune the operation of a server. Other UEFI parameters are more limited in their applicability to customer use cases and are not exposed in UEFI menus but can be accessed using Lenovo command line utilities.

This paper describes how to view and change the values of these hidden UEFI parameters on Lenovo ThinkSystem servers with Intel Xeon Scalable Family processors. The intended audience is IT administrators and technical professionals who are responsible for server performance.

Tip: System x servers with Intel Xeon E5 v3 or E5 v4 processors also have hidden UEFI parameters as described in the paper System x Hidden UEFI Parameters.

Introduction

The Lenovo server Unified Extensible Firmware Interface (UEFI) provides the interface between an operating system and the server firmware. Many UEFI parameters are exposed to server administrators in UEFI menus that can be accessed by pressing F1 when a server is rebooted, through the XClarity Controller (XCC) service processor, or through command line utilities such as Lenovo’s Advanced Settings Utility (ASU) or OneCLI.

These parameters are exposed because they are commonly changed from their default values to fine tune server performance for a wide variety of customer use cases. Other UEFI settings are more limited in their applicability to customer use cases and are not exposed in UEFI menus but can be accessed using the command line utilities.

In this paper, we describe the following commands that are only accessible via command-line utilities:

  • Snoop Response Hold Off
  • Dead Line LLC Allocation
  • Stale/Directory AtoS

These commands are available for ThinkSystem servers that use the Intel Xeon Scalable Family processors.

Snoop Response Hold Off

This hidden parameter is accessible on Lenovo ThinkSystem servers that use the Intel Xeon Scalable Family processors, as well as System x servers with Intel Xeon E5 v3 or E5 v4 processors.

For some workloads in which throughput and latency are critical, it is better to constantly poll the status of an I/O device rather than use an interrupt. Network adapter device drivers commonly use a thread to continuously poll in a fast loop so that incoming requests can be handled as fast as possible.

This can create contention between a processor core running the polling thread and the processor’s Integrated I/O feature (IIO) for an I/O-owned line in cache. This contention can cause an I/O operation to lose ownership of the cache line it has just acquired. It must then spend more time reacquiring the cache line to write it back.

When there are a large number of network ports each servicing small packets, the system may not be able to achieve the full throughput required due to excessive I/O and core contentions of cache lines. For this situation, the I/O operation should delay its response to core snoops and hold onto its cache lines until it successfully completes its write.

The Snoop Response Hold Off parameter allows the I/O operation to delay its snoop response by a selected amount to achieve this delay. It is possible to adjust this parameter using Lenovo’s Advanced Settings Utility (ASU) or the OneCLI tool as follows.

ASU command:

To show the current setting or to change the setting with ASU, use these commands:

asu64 show Processors.SnoopResponseHoldOff --override
asu64 set Processors.SnoopResponseHoldOff value --override

Where value is a number from 0 to 15 . The default (initial value) is 6. The parameter --override is required when viewing or changing the setting.

OneCLI command:

To show the current setting or to change the setting with OneCLI, use these commands:

onecli config show Processors.SnoopResponseHoldOff --override
onecli config set Processors.SnoopResponseHoldOff value --override

Where value is a number from 0 to 15. The default (initial value) is 6. The parameter --override is required when viewing or changing the setting.

For both ASU and OneCLI, the parameter value can be between 0 and 15, which corresponds to a number cycles as shown in following table. The cycles in the table are IIO clock cycles which are 2 ns per cycle. For Intel Omni-Path network adapter use, we recommend setting this parameter to 9 as a starting point. Network performance tests should be performed to determine the most optimal value for each customer’s workload.

Table 1. Possible values of the value parameter in the Processors.SnoopResponseHoldOff command
Value Number of cycles (1 cycle = 2 nanoseconds)
0 Disabled
1 8
2 16
3 32
4 64
5 128
6 256
7 512
8 1K
9 2K
10 4K
11 8K
12 16K
13 32K
14 64K
15 128K

Dead Line LLC Allocation

This hidden parameter is accessible on Lenovo ThinkSystem servers that use the Intel Xeon Scalable processors.

With the Intel Xeon Scalable processors non-inclusive cache scheme, mid-level cache (MLC) evictions are filled into the last level cache (LLC) if the data is shared across processor cores. When cache lines are evicted from the MLC, the processor core can flag them as “dead” meaning they are not likely to be read again.

With this command, the LLC can be configured to drop dead lines and not fill them in the LLC. By setting LLCdeadlinealloc to Disabled, dead lines will be dropped. This provides better utilization in the LLC and prevents the LLC from evicting useful data. By setting LLCdeadlinealloc to Enabled, then the processor will determine will determine whether to keep or drop deadlines. The default (initial value) is Enabled.

ASU command:

To show the current setting or to change the setting with ASU, use these commands:

asu64 show Processors.LLCdeadlinealloc --override
asu64 set Processors.LLCdeadlinealloc value --override

Where value is Auto, Disable or Enable. The parameter --override is required when viewing or changing the setting.

OneCLI command:

To show the current setting or to change the setting with OneCLI, use these commands:

onecli config show Processors.LLCdeadlinealloc --override
onecli config set Processors.LLCdeadlinealloc value --override

Where value is Auto, Disable or Enable. The parameter --override is required when viewing or changing the setting.

Stale/Directory AtoS

This hidden parameter is accessible on Lenovo ThinkSystem servers that use the Intel Xeon Scalable processors.

The in-memory directory has three states:

  • Invalid: The invalid (I) state means the data is clean and does not exist in any other processor’s cache.
  • snoopAll: The snoopAll (A) state means the data may exist in another processor in an exclusive or modified state.
  • Shared: The shared (S) state means the data is clean and may be shared across one or more processor’s caches.

When executing a read to memory, if the directory line is in the A state, the processor must snoop all other processors because another processor may have the line in modified state. If this is the case, the snoop will return the modified data. However, it may be the case that a line is read in A state and all the snoops come back a miss. This can happen if another processor read the line earlier and then silently dropped it from its cache without modifying it.

By setting StaleAtoS to Enabled, a line in A state returns only snoop misses and the line will transition to S state. Subsequent reads to the line will encounter it in S state and not have to snoop, saving latency and snoop bandwidth. When StaleAtoS is disabled, it is possible that a line is read in the A state and all the snoops come back a miss. This can happen if another socket read the line earlier and then silently dropped it from its cache without modifying it. This consumes snoop bandwidth unnecessarily. The default (initial value) is Disabled.

ASU command:

To show the current setting or to change the setting with ASU, use these commands:

asu64 show Processors.StaleAtoS --override
asu64 set Processors.StaleAtoS value --override

Where value is Auto, Disable or Enable. The parameter --override is required when viewing or changing the setting.

OneCLI command:

To show the current setting or to change the setting with OneCLI, use these commands:

onecli config show Processors.StaleAtoS --override
onecli config set Processors.StaleAtoS value --override

Where value is Auto, Disable or Enable. The parameter --override is required when viewing or changing the setting.

Related product families

Product families related to this document are the following:

Trademarks

Lenovo and the Lenovo logo are trademarks or registered trademarks of Lenovo in the United States, other countries, or both. A current list of Lenovo trademarks is available on the Web at https://www.lenovo.com/us/en/legal/copytrade/.

The following terms are trademarks of Lenovo in the United States, other countries, or both:
Advanced Settings Utility
Lenovo®
System x®
ThinkSystem

The following terms are trademarks of other companies:

Intel® and Xeon® are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

Other company, product, or service names may be trademarks or service marks of others.