Lenovo ThinkSystem SD665-N V3 Neptune DWC Server

The ThinkSystem SD665-N V3 Neptune DWC node is the next-generation high-performance server based on the fifth generation Lenovo Neptune™ direct water cooling platform.

With one or two fourth-generation AMD EPYC processors and four powerful NVIDIA H100 Tensor Core GPUs, the ThinkSystem SD665-N V3 server features the latest technology from AMD and NVIDIA, combined with Lenovo's market-leading water-cooling solution, which results in extreme performance in an extreme dense packaging, supporting your application from Exascale to Everyscale™.

The direct water cooled solution is designed to operate by using warm water, up to 45°C (113°F) depending on the configuration. Chillers are not needed for most customers, meaning even greater savings and a lower total cost of ownership. The nodes are housed in the upgraded ThinkSystem DW612S enclosure, a 6U rack mount unit that fits in a standard 19-inch rack.

Figure 1. The Lenovo ThinkSystem SD665-N V3 server tray with two processors and four NVIDIA GPUs

The ThinkSystem SD665-N V3 server tray and DW612S enclosure with direct water cooling provide the ultimate in data center cooling efficiencies and performance. The embedded networking chips allow for direct GPU-to-GPU communication without having to go through the CPU or PCIe switches. This enables ideal scaling, from a single rack with a single chassis and a single tray, all the way to a full sustained-Exaflop system realized in fewer than 200 racks and fewer than 6,000 nodes.

The Lenovo ThinkSystem SD665-N V3 server tray is designed for High Performance Computing (HPC), large-scale cloud, heavy simulations, and modeling. It implements Lenovo Neptune™ Direct Water Cooling (DWC) technology to optimally support workloads from technical computing, grid deployments, analytics, and is ideally suited for fields such as research, life sciences, energy, simulation, and engineering.

The unique design of ThinkSystem SD665-N V3 provides the optimal balance of serviceability, performance, and efficiency. By using a standard rack with the ThinkSystem DW612S enclosure equipped with patented stainless steel drip-less quick connectors, the SD665-N V3 provides easy serviceability and extreme density that is well suited for clusters ranging from small enterprises to the world's largest supercomputers.

The Lenovo Neptune™ direct liquid cooling doesn't use risky plastic retrofitting but instead custom-designed copper water loops, so you have peace of mind implementing a platform with liquid cooling at the core of the design.

Compared to other technology, the SD665-N V3 direct water cooling:

• Reduces data center energy costs by up to 40%
• Increases system performance by up to 10%
• Delivers up to 100% heat removal efficiency (depending on the environment)
• Creates a quieter data center with its fan-less design
• Enables data center growth without adding computer room air conditioning

Lenovo’s direct water-cooled solutions are factory-integrated and are re-tested at the rack-level to ensure that a rack can be directly deployed at the customer site. This careful and consistent quality testing has been developed as a result of over a decade of experience designing and deploying DWC solutions to the very highest standards.

Scalability and performance

The ThinkSystem SD665-N V3 server tray and DW612S enclosure offer the following features to boost performance, improve scalability, and reduce costs:

• Each SD665-N V3 node supports two fourth-generation AMD EPYC processors, four NVIDIA H100 SXM GPUs, 24x TruDDR5 DIMMs, two OSFP 800G cages for high-speed I/O, and up to two drive bays, all in a 1U form factor.
• Up to 6x SD665-N V3 nodes are installed in the DW612S enclosure, occupying only 6U of rack space. It is a highly dense, scalable, and price-optimized offering.
• Supports two fourth-generation AMD EPYC 9004 processors
  • Up to 128 cores and 256 threads
  • Core speed of up to 4.1 GHz
  • Nominal TDP rating of up to 360 W, configurable TDP up to 400 W
• Supports four NVIDIA H100 GPUs
  • 700W SXM5 GPUs with configurable EDP (Electrical Design Point)
  • 80GB HBM3 or 94GB HBM2e GPU memory per GPU
  • Interconnected using dual NVLink 4.0 connections
  • Up to 400 Gb/s NDR connectivity to each through four NVIDIA ConnectX-7 embedded network controllers
• Support for DDR5 memory DIMMs to maximize the performance of the memory subsystem:
  • Up to 24 DDR5 memory DIMMs, 12 DIMMs per processor
  • 12 memory channels per processor (1 DIMM per channel)
  • DIMM speeds up to 4800 MHz
  • Using 128GB 3DS RDIMMs, the node supports up to 3TB of system memory
• Supports high-speed GPU Direct networking with dual InfiniBand NDRx2 800Gb/s connections 
  • Choice of two OSFP-DD or alternatively OSFP ports 
  • Each port supports OSFP 800G (2x400 Gb/s) or OSFP 400G (400 Gb/s) connectivity 
  • Direct connections to the GPUs - each OSFP port connects to two GPUs 
• Supports up to two NVMe SSDs, as follows:
  • Two E3.S 1T NVMe SSDs, or
  • Two 7mm NVMe SSDs, or
  • One 15mm NVMe SSD
• Drives are NVMe to maximize I/O performance in terms of throughput, bandwidth, and latency.
• Supports a PCIe 4.0 x4 high-speed M.2 NVMe drive installed in an adapter for convenient operating system boot and internal storage functions.
• The node includes one Gigabit and two 25 Gb Ethernet onboard ports for cost effective networking.

Energy efficiency

The direct water cooled solution offers the following energy efficiency features to save energy, reduce operational costs, increase energy availability, and contribute to a green environment:

• Water cooling eliminates power that is drawn by cooling fans in the enclosure and dramatically reduces the required air movement in the server room, which also saves power. In combination with an Energy Aware Runtime environment, savings as much as 40% are possible in the data center due to the reduced need for air conditioning.
• Water chillers may not be required with a direct water cooled solution. Chillers are a major expense for most geographies and can be reduced or even eliminated because the water temperature can now be 45°C instead of 18°C in an air-cooled environment.
• Up to 100% heat recovery is possible with the direct water cooled design, depending on water temperature chosen. Heat energy absorbed may be reused for heating buildings in the winter, or generating cold through Adsorption Chillers, for further operating expense savings.
• The processors and other microelectronics are run at lower temperatures because they are water cooled, which uses less power, and allows for higher performance through Turbo Mode.
• The processors and accelerators are run at uniform temperatures because they are cooled in parallel loops, which avoid thermal jitter and provides higher and more reliable performance at same power.
• Low-voltage 1.1V DDR5 memory offers energy savings compared to 1.2V DDR4 DIMMs, an approximately 20% decrease in power consumption
• 80 Plus Titanium power supplies ensure energy efficiency.
• There are power monitoring and management capabilities through the System Management Module in the DW612S enclosure.
• Lenovo power/energy meter based on TI INA226 measures DC power for the CPU and the GPU board at higher than 97% accuracy and 100 Hz sampling frequency to the XCC and can be leveraged both in-band and out-of-band using IPMI raw commands.
• Optional Lenovo XClarity Energy Manager provide advanced data center power notification, analysis, and policy-based management to help achieve lower heat output and reduced cooling needs.
• Optional Energy Aware Runtime provides sophisticated power monitoring and energy optimization on a job-level during the application runtime without impacting performance negatively.

Manageability and security

The following powerful systems management features simplify local and remote management of the SD665-N V3 server:

• The server includes an XClarity Controller 2 (XCC2) to monitor server availability. Optional upgrade to XCC Platinum to provide remote control (keyboard video mouse) functions, support for the mounting of remote media files, FIPS 140-3 security, enhanced NIST 800-193 support, boot capture, power capping, and other management and security features.
• Support for industry standard management protocols, IPMI 2.0, SNMP 3.0, Redfish REST API, serial console via IPMI
• Integrated Trusted Platform Module (TPM) 2.0 support enables advanced cryptographic functionality, such as digital signatures and remote attestation.
• Supports AMD Secure Root-of-Trust, Secure Run and Secure Move features to minimize potential attacks and protect data as the OS is booted, as applications are run and as applications are migrated from server to server.
• Supports Secure Boot to ensure only a digitally signed operating system can be used.
• Industry-standard Advanced Encryption Standard (AES) NI support for faster, stronger encryption.
• With the System Management Module (SMM) installed in the enclosure, only one Ethernet connection is needed to provide remote systems management functions for all SD665-N V3 servers and the enclosure.
• The SMM management module has two Ethernet ports which allows a single Ethernet connection to be daisy chained across 7 enclosures and 84 servers, thereby significantly reducing the number of Ethernet switch ports needed to manage an entire rack of SD665-N V3 servers and DW612S enclosures.
• The DW612S enclosure includes drip sensors that monitor the inlet and outlet manifold quick connect couplers; leaks are reported via the SMM.
• The server supports Lenovo XClarity suite software with Lenovo XClarity Administrator, Lenovo XClarity Provisioning Manager, and XClarity Energy Manager. They are described further in the Software section of this product guide.
• Lenovo HPC & AI Software Stack provides our HPC customers you with a fully tested and supported open-source software stack to enable your administrators and users with for the most effective and environmentally sustainable consumption of Lenovo supercomputing capabilities.
• Our Confluent management system and Lenovo Intelligent Computing Orchestration (LiCO) web portal provides an interface designed to abstract the users from the complexity of HPC cluster orchestration and AI workloads management, making open-source HPC software consumable for every customer.
• LiCO web portal provides workflows for both AI and HPC, and supports multiple AI frameworks, allowing you to leverage a single cluster for diverse workload requirements.

Availability and serviceability

The SD665-N V3 node and DW612S enclosure provide the following features to simplify serviceability and increase system uptime:

• Designed to run 24 hours a day, 7 days a week
• Depending on the configuration and node population, the DW612S enclosure supports N+1 power policies for its power supplies, which means greater system uptime.
• All supported power supplies are hot-swappable, including the water-cooled power supplies.
• Toolless cover removal on the trays provides easy access to upgrades and serviceable parts, such as adapters and memory.
• The server uses ECC memory and supports memory RAS features including Single Device Data Correction (SDDC, also known as Chipkill), Patrol/Demand Scrubbing, Bounded Fault, DRAM Address Command Parity with Replay, DRAM Uncorrected ECC Error Retry, On-die ECC, ECC Error Check and Scrub (ECS), and Post Package Repair.
• Proactive Platform Alerts (including PFA and SMART alerts): Processors, voltage regulators, memory, internal storage (HDDs and SSDs, NVMe SSDs, M.2 storage), fans, power supplies, and server ambient and subcomponent temperatures. Alerts can be surfaced through the XClarity Controller to managers such as Lenovo XClarity Administrator and other standards-based management applications. These proactive alerts let you take appropriate actions in advance of possible failure, thereby increasing server uptime and application availability.
• The XCC offers optional remote management capability and can enable remote keyboard, video, and mouse (KVM) control and remote media for the node.
• Built-in diagnostics in UEFI, using Lenovo XClarity Provisioning Manager, speed up troubleshooting tasks to reduce service time.
• Lenovo XClarity Provisioning Manager supports diagnostics and can save service data to a USB key drive or remote CIFS share folder for troubleshooting and reduce service time.
• Auto restart in the event of a momentary loss of AC power (based on power policy setting in the XClarity Controller service processor)
• Virtual reseat is a supported feature of the System Management Module (SMM2) which simulates physically removing the node from A/C power and reconnecting the node to AC power from a remote location.
• There is a three-year customer replaceable unit and onsite limited warranty, with next business day 9x5 coverage. Optional warranty upgrades and extensions are available.
• With water cooling, system fans are not required. This results in significantly reduced noise levels on the data center floor, a significant benefit to personnel having to work on site.

The front of the SD665-N V3 node is shown in the following figure.

Figure 2. Front view of the ThinkSystem SD665-N V3 node

The following figure shows key components internal to the server tray.

Figure 3. Inside view of the SD665-N V3 node in the water-cooled tray

The rear of the DW612S enclosure contains the power supplies, cooling water manifolds, and the System Management Module. The following figure shows rear of the enclosure with 6x air-cooled power supplies.

Figure 5. Rear view of the DW612S enclosure with 6 air-cooled power supplies

The also supports water-cooled power supplies for an increased level of heat removal using water. The following figure shows the enclosure with 3 water-cooled power supplies installed.

Figure 6. Rear view of the DW612S enclosure with 3 water-cooled power supplies

The following figure shows the architectural block diagram of the SD665-N V3 with two processors installed. Each GPU is connected to a processor with a PCIe 5.0 x16 link.

Figure 7. SD665-N V3 system architectural block diagram (2 processors)

The following figure shows the block diagram of the SD665-N V3 with one processor installed. Each GPU is connected to the processor with a PCIe 5.0 x8 link.

Figure 8. SD665-N V3 system architectural block diagram (1 processor)

The following table lists the standard specifications of the SD665-N V3 server tray.

The ThinkSystem DW612S enclosure provides shared high-efficiency power supplies. The SD665-N V3 servers connect to the midplane of the DW612S enclosure. This midplane connection is for power and control only; the midplane does not provide any I/O connectivity.

The following table lists the standard specifications of the enclosure.

There are no standard SD665-N V3 models; all servers must be configured by using the configure-to-order (CTO) process with the Lenovo Data Center Solution Configurator (DCSC) or the Lenovo Cluster Solutions configurator (x-config). The ThinkSystem SD665-N V3 machine type is 7DAZ.

The following table lists the base CTO model and base feature code

There are no standard models of the DW612S enclosure. All enclosures must be configured by using the CTO process. The machine type is 7D1L.

The following table lists the base CTO model and base feature code

The manifold provides the water supply and return to the DW612S Enclosure. It can be connected through the Eaton Ball Valves (Stainless steel V2A, Type FD83-2046-16-16) to a water loop in the data center that is connected to a centralized coolant distribution unit (CDU) or be ordered with an in-rack CDU.

Figure 9. DW612S enclosure and manifold assembly

The manifold is ordered using the CTO process in the configurators using machine type 5469. The following table lists the base CTO model.

The following table lists the base feature code for CTO configurations when connecting to a data center level water distribution. Select the correct feature code based on the number of enclosures installed in the rack. The feature code for the water-cooled power supplies (PSU) will be auto-derived when you select the PSUs in the configuration and is only supported with 6 Enclosures.

The following table lists the base feature code for CTO configurations when connecting to the in-rack CDU.

To support the onsite setup for the direct water-cooled solution, a Commissioning Kit is available providing flow meter, bleed hose, pressure gauge and vent valve to Lenovo partners and customers.

For additional information, see the Cooling section.

The SD665-N V3 supports processors in the fourth-generation AMD EPYC family of processors. The node supports one or two processors.

Field upgrades from one processor to two processors are currently not supported.

Topics in this section:

• Processor options
• Processor features
• UEFI operating modes

The following table lists the feature codes that allow you to specify the mode you wish to preset in the factory for CTO orders.

The preset modes for the SD665-N V3 are as follows:

• Maximum Efficiency Mode (feature BFYA): Maximizes performance/watt efficiency while maintaining reasonable performance
• Maximum Performance Mode (feature BFYB): Achieves maximum performance but with higher power consumption and lower energy efficiency.

For details about these preset modes, and all other performance and power efficiency UEFI settings offered in the SD665-N V3, see the paper "Tuning UEFI Settings for Performance and Energy Efficiency on AMD Processor-Based ThinkSystem Servers", available from https://lenovopress.lenovo.com/lp1267.

The SD665-N V3 uses Lenovo TruDDR5 memory operating at up to 4800 MHz. The server supports up to 24 DIMMs with 2 processors. The processors have 12 memory channels and support 1 DIMM per channel. The server supports up to 3TB of memory using 24x 128GB 3DS RDIMMs and two processors.

The following table lists the memory options that are available for the server.

Lenovo TruDDR5 memory uses the highest quality components that are sourced from Tier 1 DRAM suppliers and only memory that meets the strict requirements of Lenovo is selected. It is compatibility tested and tuned to maximize performance and reliability. From a service and support standpoint, Lenovo TruDDR5 memory automatically assumes the system warranty, and Lenovo provides service and support worldwide.

9x4 RDIMMs (also known as Optimized or EC4 RDIMMs) are a new lower-cost DDR5 memory option supported in ThinkSystem V3 servers. 9x4 DIMMs offer the same performance as standard RDIMMs (known as 10x4 or EC8 modules), however they support lower fault-tolerance characteristics. Standard RDIMMs and 3DS RDIMMs support two 40-bit subchannels (that is, a total of 80 bits), whereas 9x4 RDIMMs support two 36-bit subchannels (a total of 72 bits). The extra bits in the subchannels allow standard RDIMMs and 3DS RDIMMs to support Single Device Data Correction (SDDC), however 9x4 RDIMMs do not support SDDC. Note, however, that all DDR5 DIMMs, including 9x4 RDIMMs, support Bounded Fault correction, which enables the server to correct most common types of DRAM failures.

For more information on DDR5 memory, see the Lenovo Press paper, Introduction to DDR5 Memory, available from https://lenovopress.com/lp1618.

The following rules apply when selecting the memory configuration:

• The SD665-N V3 only supports quantities of 8 or 12 DIMMs per processor; other quantities not supported
• 128GB DIMMs require a water inlet temperature of 32°C or less
• The server supports four types of DIMMs: 9x4 RDIMMs, 10x4 RDIMMs, x8 RDIMMs and 3DS RDIMMs; UDIMMs and LRDIMMs are not supported
• All memory DIMMs must be identical part numbers

Note: Memory mirroring and memory rank sparing are not supported.

A key feature of the SD665-N V3 is the integration of a 4x SXM4 GPU complex on the left half of the server as shown in the Components and connectors section. The server supports four NVIDIA HGX H100 GPU modules that are connected together using high-speed fourth-generation NVLink interconnects.

The GPUs supported are listed in the following table.

The NVIDIA H100 Tensor Core GPU delivers unprecedented performance, scalability and security to every data center and includes NVIDIA AI Enterprise software suite for streamlined AI development and deployment.

The NVIDIA H100 supports granular power management by using the Total Graphics Power (TGP) setting. This setting determines what the maximum power each GPU can use, and in turn will dictate how many nodes can be installed in the enclosure and how hot the inlet water can be to properly cool all nodes.

Lenovo supports pre-set TGP of 500W, 600W and 700W. With full 400W processor configuration and the GPUs at 700W up to 35°C system inlet water temperature can be supported based on 4 lpm flow rate per tray. With 600W set on the GPUs 40°C are supported and with 500W the full 45°C. The supported trays per chassis are shown in the Power supplies section.

The desired TGP setting is executed in the factory by specifying the matching feature code in the configurator. The following table lists the feature codes that can be selected.

The SD665-N V3 node supports one or two SSDs drives internally in the node. These are internal drives that are not front accessible and are not hot-swap. See the Components and connectors section for the location of the drives.

The SD665-N V3 supports either:

• 2x E3.S 1T drives
• 2x 2.5-inch 7mm drives
• 1x 2.5-inch 15mm drive

Configuration notes:

• The node only supports NVMe drives; SATA and SAS drives are not supported
• The drives are connected to onboard controllers; No RAID support
• NVMe drives are connected to CPU 1 in all configurations
• When 2x 7mm or 2x E3.S drives are installed in a node, they are numbered drive 2 (bottom) and 3 (top). When 1x 15mm drive is installed, it is numbered 2.

In addition, the SD665-N V3 node a single high-performance M.2 NVMe drive, installed in an adapter mounted on top of the front processor. For details, see the M.2 drive section.

The feature codes to select the appropriate storage cage are listed in the following table:

The necessary storage cables are auto-derived by the configurator.

To upgrade systems installed in the field with storage options, there are separate kits available that contain both the cage and the necessary cables. The option part numbers of the upgrade kits are listed in the following table.

The M.2 enablement kit has the following features:

• Supports one NVMe M.2 drive
• Supports 80mm and 110mm drive form factors (2280 and 22110)
• PCIe 4.0 x4 NVMe interface to the drive
• Connects to CPU 1 via onboard NVMe connector
• Supports monitoring and reporting of events and temperature through I2C
• Firmware update via Lenovo firmware update tools
• Water-cooled via the attached cold plate

The drives of the SD665-N V3 are connected to CPU 1 using PCIe 4.0 x4 connections.

The node hardware does not provide any RAID functionality. RAID support is provided by the operating system.

The following tables list the drive options for internal storage of the server.

The server supports the external USB optical drive listed in the following table.

The drive is based on the Lenovo Slim DVD Burner DB65 drive and supports the following formats: DVD-RAM, DVD-RW, DVD+RW, DVD+R, DVD-R, DVD-ROM, DVD-R DL, CD-RW, CD-R, CD-ROM.

The SD665-N V3 offers I/O expansion in the form of high-speed GPU Direct connections to the four NVIDIA GPUs in the system. These InfiniBand NDR connections with OSFP cages are in addition to two onboard 25 GbE ports with SRP28 cages.

The location of these ports is shown in the following figure.

Figure 12. SD665-N V3 networking

The SD665-N V3 has five network ports, one 1Gb, two 25Gb, and two 800Gb ports. There is no support for PCIe network adapters.

Topics in this section:

• Onboard 25Gb and 1Gb ports
• OSFP800 ports

Onboard 25Gb and 1Gb ports

The SD665-N V3 has three onboard network ports:

• 2x 25GbE ports, connected to an onboard Mellanox ConnectX-6 Lx controller, implemented with SFP28 cages for optical or copper connections. Supports 1Gb, 10Gb and 25Gb connections.
• 1x 1GbE port, connected to an onboard Intel I210 controller, implemented with an RJ45 port for copper cabling

Locations of these ports is shown in the Components and connectors section. The 1GbE port and 25GbE Port 1 both support NC-SI for remote management. For factory orders, to specify which ports should have NC-SI enabled, use the feature codes listed in the Remote Management section. If neither is chosen, both ports will have NC-SI disabled by default.

For the specifications of the 25GbE ports including the supported transceivers and cables, see the Mellanox ConnectX-6 Lx product guide:
https://lenovopress.lenovo.com/lp1364-thinksystem-mellanox-connectx-6-lx-25gbe-sfp28-ethernet-adapters

The following table lists the transceiver supported by ThinkSystem SD665-N, SD650-N V3 4x NDR Infiniband Interface (BRK8).

For the specifications of the OSFP ports including the supported transceivers and cables, see the NVIDIA ConnectX-7 product guide:
https://lenovopress.lenovo.com/lp1692-thinksystem-nvidia-connectx-7-ndr-infiniband-osfp400-adapters

The following table lists the supported cables for ThinkSystem SD665-N, SD650-N V3 4x NDR Infiniband Interface, BRK8.

The following table lists the supported cables for ThinkSystem SD665-N, SD650-N V3 4x NDR200 Infiniband Interface, BRK9.

The SD665-N V3 does not support storage host bus adapters.

The SD665-N V3 does not support Flash storage adapters.

One of the most notable features of the ThinkSystem SD665-N V3 offering is direct water cooling. Direct water cooling (DWC) is achieved by circulating the cooling water directly through cold plates that contact the CPU thermal case, DIMMs, and other high-heat-producing components in the node.

One of the main advantages of direct water cooling is the water can be relatively warm and still be effective because water conducts heat much more effectively than air. Depending on the server and power supply configuration as well as environmentals like water and air temperature, effectively 100% of the heat can be removed by water cooling; in configurations that stay slightly below that, the rest can be easily managed by a standard computer room air conditioner. Measured data at a customer data center shows 98% heat capture at 45°C water inlet temperature and 99% heat capture at 40°C water inlet temperature and 26.6°C ambient temperature with insulated racks using the SD650-N V2.

Allowable inlet temperatures for the water can be as high as 45°C (113°F) with the SD665-N V3. In most climates, water-side economizers can supply water at temperatures below 45°C for most of the year. This ability allows the data center chilled water system to be bypassed thus saving energy because the chiller is the most significant energy consumer in the data center. Typical economizer systems, such as dry-coolers, use only a fraction of the energy that is required by chillers, which produce 6-10 °C (43-50 °F) water. The facility energy savings are the largest component of the total energy savings that are realized when the SD665-N V3 is deployed.

The advantages of the use of water cooling over air cooling result from water’s higher specific heat capacity, density, and thermal conductivity. These features allow water to transmit heat over greater distances with much less volumetric flow and reduced temperature difference as compared to air.

For cooling IT equipment, this heat transfer capability is its primary advantage. Water has a tremendously increased ability to transport heat away from its source to a secondary cooling surface, which allows for large, more optimally designed radiators or heat exchangers rather than small, inefficient fins that are mounted on or near a heat source, such as a CPU.

The ThinkSystem SD665-N V3 offering uses the benefits of water by distributing it directly to the highest heat generating node subsystem components. By doing so, the offering realizes 7% - 10% direct energy savings when compared to an air-cooled equivalent. That energy savings results from the removal of the system fans and the lower operating temp of the direct water-cooled system components.

The direct energy savings at the enclosure level, combined with the potential for significant facility energy savings, makes the SD665-N V3 an excellent choice for customers that are burdened by high energy costs or with a sustainability mandate.

Water is delivered to each of the nodes from a coolant distribution unit (CDU) via the water manifold. As shown in the following figure, each manifold section attaches to an enclosure and connects directly to the water inlet and outlet connectors for each compute node to deliver water safely and reliably to and from each server tray.

The DWC Manifold is modular and is available in multiple configurations that are based on the number of enclosure drops that are required in a rack. The Manifold scales to support up to six Enclosures in a single rack, as shown in the following figure. Ordering information for the water manifold is in the Manifold assembly section.

Figure 14. DW612S enclosure and manifold assembly

The water flows through the SD665-N V3 tray to cool all major heat-producing components. The inlet water is split into two parallel paths, one for each node in the tray. Each path is then split further to cool the processors, memory, drives (including the M.2 drive) and adapters.

When the DW612S is configured with water-cooled power supplies, an additional water manifold is used to supply water to each of the three power supplies, as shown in the following figure. Ordering information for the manifold is in the Manifold assembly section.

Figure 15. DW612S enclosure with water-cooled power supplies and manifold

During the manufacturing and test cycle, Lenovo’s water-cooled nodes are pressure tested with Helium according to ASTM E499 / E499M – 11 (Standard Practice for Leaks Using the Mass Spectrometer Leak Detector in the Detector Probe Mode) and later again with Nitrogen to detect micro-leaks which may be undetectable by pressure testing with water and/or a water/glycol mixture as Helium and Nitrogen have smaller molecule sizes.

This approach also allows Lenovo to ship the systems pressurized without needing to send hazardous antifreeze-components to our customers.

Onsite the materials used within the water loop from the CDU to the nodes should be limited to copper alloys with brazed joints, Stainless steels with TIG and MIG welded joints and EPDM rubber. In some instances, PVC might be an acceptable choice within the facility.

The water the system is filled with must be reasonably clean, bacteria-free water (< 100 CFU/ml) such as de-mineralized water, reverse osmosis water, de-ionized water, or distilled water. It must be filtered with in-line 50 micron filter. Biocide and Corrosion inhibitors ensure a clean operation without microbiological growth or corrosion.

Lenovo Data Center Power and Cooling Services can support you in the design, implementation and maintenance of the facility water-cooling infrastructure. 

Limitations based on GPU power requirements

The following table shows the power limits based on the configured Peak EDP (EDPp) setting for a high-end dual-socket configuration (2x 360W CPU e.g. 9554 64C 3.1GHz, 24x 64GB Memory, 2x NVMe, 1x M.2).

The following table shows the power limits based on the configured EDPp setting for a high-end single-socket configuration (1x 360W CPU e.g. 9554 64C 3.1GHz, 12x 64 GB Memory, no local storage).

Power cables

The power supplies in the DW612S enclosure have C19 connectors and support the following rack power cables.

The SD665-N V3 contains an integrated service processor, XClarity Controller 2 (XCC), which provides advanced control, monitoring, and alerting functions. The XCC2 is based on the AST2600 baseboard management controller (BMC) using a dual-core ARM Cortex A7 32-bit RISC service processor running at 1.2 GHz.

Topics in this section:

• Local console
• External Diagnostics Handset
• System status with XClarity Mobile
• Remote management
• XCC2 Platinum
• Remote management using the SMM
• Lenovo HPC & AI Software Stack
• Lenovo XClarity Provisioning Manager
• Lenovo XClarity Essentials
• Lenovo XClarity Administrator
• Lenovo XClarity Integrators
• Lenovo XClarity Energy Manager
• Lenovo Capacity Planner

Local console

The SD665-N V3 node supports a local console with the use of a console breakout cable. The cable connects to the port on the front of the node as shown in the following figure.

Figure 18. Console breakout cable

The cable has the following connectors:

• VGA port
• Serial port
• USB 3.1 Gen 1 (5 Gb/s) port

As well as local console functions, the USB port on the breakout cable also supports the use of the XClarity Mobile app as described in the next section.

Ordering information for the cable is listed in the following table.

Figure 19. SD665-N V3 External Diagnostics Handset

Ordering information for the External Diagnostics Handset with is listed in the following table.

The steps to connect the mobile device are as follows:

1. Enable USB Management on the server, by holding down the ID button for 3 seconds (or pressing the dedicated USB management button if one is present)
2. Connect the mobile device via a USB cable to the server's USB port with the management symbol
3. In iOS or Android settings, enable Personal Hotspot or USB Tethering
4. Launch the Lenovo XClarity Mobile app

Remote management

Remote server management is provided through industry-standard interfaces:

• Intelligent Platform Management Interface (IPMI) Version 2.0
• Simple Network Management Protocol (SNMP) Version 3 (no SET commands; no SNMP v1)
• Common Information Model (CIM-XML)
• Representational State Transfer (REST) support
• Redfish support (DMTF compliant)
• Web browser - HTML 5-based browser interface (Java and ActiveX not required) using a responsive design (content optimized for device being used - laptop, tablet, phone) with NLS support

The 1Gb port and 25Gb Port 1 support NC-SI. You can enable NC-SI in the factory using the feature codes listed in the following table. If neither feature code is selected, both ports will have NC-SI disabled.

XCC2 Platinum adds the following Enterprise and Advanced functions:

• Remotely viewing video with graphics resolutions up to 1600x1200 at 75 Hz with up to 23 bits per pixel, regardless of the system state
• Remotely accessing the server using the keyboard and mouse from a remote client
• International keyboard mapping support
• Syslog alerting
• Redirecting serial console via SSH
• Component replacement log (Maintenance History log)
• Access restriction (IP address blocking)
• Lenovo SED security key management
• Displaying graphics for real-time and historical power usage data and temperature
• Boot video capture and crash video capture
• Virtual console collaboration - Ability for up to 6 remote users to be log into the remote session simultaneously
• Remote console Java client
• Mapping the ISO and image files located on the local client as virtual drives for use by the server
• Mounting the remote ISO and image files via HTTPS, SFTP, CIFS, and NFS
• Power capping
• System utilization data and graphic view
• Single sign on with Lenovo XClarity Administrator
• Update firmware from a repository
• License for XClarity Energy Manager

With XCC2 Platinum, for CTO orders, you can request that System Guard be enabled in the factory and the first configuration snapshot be recorded. To add this to an order, select feature code listed in the following table. The selection is made in the Security tab of the DCSC configurator.

For more information about System Guard, see https://pubs.lenovo.com/xcc2/NN1ia_c_systemguard

The SMM has the following ports and LEDs:

• 2x Gigabit Ethernet RJ45 ports for remote management access
• USB port and activation button for service
• SMM reset button
• System error LED (yellow)
• Identification (ID) LED (blue)
• Status LED (green)
• System power LED (green)

The USB service button and USB service port are used to gather service data in the event of an error. Pressing the service button copies First Failure Data Collection (FFDC) data to a USB key installed in the USB service port. The reset button is used to perform an SMM reset (short press) or to restore the SMM back to factory defaults (press for 4+ seconds).

The use of two RJ45 Ethernet ports enables the ability to daisy-chain the Ethernet management connections thereby reducing the number of ports you need in your management switches and reducing the overall cable density needed for systems management. With this feature you can connect the first SMM to your management network and the SMM in a second enclosure connects to the first SMM. The SMM in the third enclosure can then connect to the SMM in the second enclosure.

Up to 7 enclosures can be connected in a daisy-chain configuration and all servers in those enclosures can be managed remotely via one single Ethernet connection.

Notes:

• If you are using IEEE 802.1D spanning tree protocol (STP) then at most 6 enclosures can be connected together
• Do not form a loop with the network cabling. The dual-port SMM at the end of the chain should not be connected back to the switch that is connected to the top of the SMM chain.

Lenovo HPC & AI Software Stack

The Lenovo HPC & AI Software Stack combines open-source with proprietary best-of-breed Supercomputing software to provide the most consumable open-source HPC software stack embraced by all Lenovo HPC customers.

It provides a fully tested and supported, complete but customizable HPC software stack to enable the administrators and users in optimally and environmentally sustainable utilizing their Lenovo Supercomputers.

The Lenovo HPC & AI Software Stack is built on the most widely adopted and maintained HPC community software for orchestration and management. It integrates third party components especially around programming environments and performance optimization to complement and enhance the capabilities, creating the organic umbrella in software and service to add value for our customers.

• SLURM Orchestration

  Slurm is integrated as an open source, flexible, and modern choice to manage complex workloads for faster processing and optimal utilization of the large-scale and specialized high-performance and AI resource capabilities needed per workload provided by Lenovo systems. Lenovo provides support in partnership with SchedMD.

• LiCO Webportal

  Lenovo Intelligent Computing Orchestration (LiCO) is a Lenovo-developed consolidated Graphical User Interface (GUI) for monitoring, managing and using cluster resources. The webportal provides workflows for both AI and HPC, and supports multiple AI frameworks, including TensorFlow, Caffe, Neon, and MXNet, allowing you to leverage a single cluster for diverse workload requirements.

• Energy Aware Runtime

  EAR is a powerful European open-source energy management suite supporting anything from monitoring over power capping to live-optimization during the application runtime. Lenovo is collaborating with Barcelona Supercomputing Centre (BSC) and EAS4DC on the continuous development and support and offers three versions with differentiating capabilities.

For more information and ordering information, see the Lenovo HPC & AI Software Stack product guide:
https://lenovopress.com/lp1651

Because Lenovo XClarity Administrator does not require any agent software to be installed on the managed endpoints, there are no CPU cycles spent on agent execution, and no memory is used, which means that up to 1GB of RAM and 1 - 2% CPU usage is saved, compared to a typical managed system where an agent is required.

Lenovo XClarity Administrator is an optional software component for the SD665-N V3. The software can be downloaded and used at no charge to discover and monitor the SD665-N V3 and to manage firmware upgrades.

If software support is required for Lenovo XClarity Administrator, or premium features such as configuration management and operating system deployment are required, Lenovo XClarity Pro software subscription should be ordered. Lenovo XClarity Pro is licensed on a per managed system basis, that is, each managed Lenovo system requires a license.

The following table lists the Lenovo XClarity software license options.

For more information, refer to the Lenovo XClarity Administrator Product Guide:
http://lenovopress.com/tips1200

For more information about all the available Lenovo XClarity Integrators, see the Lenovo XClarity Administrator Product Guide: https://lenovopress.com/tips1200-lenovo-xclarity-administrator

LXEM is a licensed product. A single-node LXEM license is included with the XClarity Controller Platinum upgrade as described in the XCC2 Platinum section. If your server does not have the XCC Platinum upgrade, Energy Manager licenses can be ordered as shown in the following table.

Note: The SD665-N V3 does not support the following Energy Manager functions:

• Power capping
• Policy-based management

Topics in this section:

• Security features
• Platform Firmware Resiliency - Lenovo ThinkShield
• Security standards

Security features

The server offers the following electronic security features:

• System Guard (part of XCC Platinum) - Proactive monitoring of hardware inventory for unexpected component changes
• Administrator and power-on password
• Trusted Platform Module (TPM) supporting TPM 2.0 (no support for TPM 1.2)
• Support for an optional Nationz TPM 2.0, available only in China (CTO only)

The server is NIST SP 800-147B compliant.

The following table lists the security options for the SD665-N V3.

Platform Firmware Resiliency - Lenovo ThinkShield

Lenovo's ThinkShield Security is a transparent and comprehensive approach to security that extends to all dimensions of our data center products: from development, to supply chain, and through the entire product lifecycle.

The ThinkSystem SD665-N V3 includes Platform Firmware Resiliency (PFR) hardware Root of Trust (RoT) which enables the system to be NIST SP800-193 compliant. This offering further enhances key platform subsystem protections against unauthorized firmware updates and corruption, to restore firmware to an integral state, and to closely monitor firmware for possible compromise from cyber-attacks.

PFR operates upon the following server components:

• UEFI image – the low-level server firmware that connects the operating system to the server hardware
• XCC image – the management “engine” software that controls and reports on the server status separate from the server operating system
• FPGA image – the code that runs the server’s lowest level hardware controller on the motherboard

The Lenovo Platform Root of Trust Hardware performs the following three main functions:

• Detection – Measures the firmware and updates for authenticity
• Recovery – Recovers a corrupted image to a known-safe image
• Protection – Monitors the system to ensure the known-good firmware is not maliciously written

These enhanced protection capabilities are implemented using a dedicated, discrete security processor whose implementation has been rigorously validated by leading third-party security firms. Security evaluation results and design details are available for customer review – providing unprecedented transparency and assurance.

The SD665-N V3 includes support for Secure Boot, a UEFI firmware security feature developed by the UEFI Consortium that ensures only immutable and signed software are loaded during the boot time. The use of Secure Boot helps prevent malicious code from being loaded and helps prevent attacks, such as the installation of rootkits. Lenovo offers the capability to enable secure boot in the factory, to ensure end-to-end protection. Alternatively, Secure Boot can be left disabled in the factory, allowing the customer to enable it themselves at a later point, if desired.

The following table lists the relevant feature code(s).

Security standards

The SD665-N V3 supports the following security standards and capabilities:

• Industry Standard Security Capabilities
  • AMD CPU Enablement
    • AES-NI (Advanced Encryption Standard New Instructions)
    • GMET (Guest Mode Execute Trap)
    • Hardware-based side channel attack resilience enhancements
    • NX (No eXecute)
    • PSB (Platform Secure Boot)
    • Shadow Stack
    • SEV (Secure Encrypted Virtualization)
    • SEV-ES (Encrypted State register encryption)
    • SEV-SNP (Secure Nested Paging)
    • SVM (Secure Virtual Machine)
    • SME (Secure Memory Encryption)
    • UMIP (User Mode Instruction Prevention)
  • Microsoft Windows Security Enablement
    • Credential Guard
    • Device Guard
    • Host Guardian Service
  • TCG (Trusted Computing Group) TPM (Trusted Platform Module) 2.0
  • UEFI (Unified Extensible Firmware Interface) Forum Secure Boot
• Hardware Root of Trust and Security
  • Independent security subsystem providing platform-wide NIST SP800-193 compliant Platform Firmware Resilience (PFR)
  • Host domain RoT supplemented by AMD Platform Secure Boot (PSB)
  • Management domain RoT supplemented by the Secure Boot features of XCC
• Platform Security
  • Boot and run-time firmware integrity monitoring with rollback to known-good firmware (e.g., “self-healing”)
  • Non-volatile storage bus security monitoring and filtering
  • Resilient firmware implementation, such as to detect and defeat unauthorized flash writes or SMM (System Management Mode) memory incursions
  • Patented IPMI KCS channel privileged access authorization (USPTO Patent# 11,256,810)
  • Host and management domain authorization, including integration with CyberArk for enterprise password management
  • KMIP (Key Management Interoperability Protocol) compliant, including support for IBM SKLM and Thales KeySecure
  • Reduced “out of box” attack surface
  • Configurable network services
  • FIPS 140-3 (in progress) validated cryptography for XCC
  • CNSA Suite 1.0 Quantum-resistant cryptography for XCC
  • Lenovo System Guard

  For more information on platform security, see the paper “How to Harden the Security of your ThinkSystem Server and Management Applications” available from https://lenovopress.com/lp1260-how-to-harden-the-security-of-your-thinksystem-server.

• Standards Compliance and/or Support
  • NIST SP800-131A rev 2 “Transitioning the Use of Cryptographic Algorithms and Key Lengths”
  • NIST SP800-147B “BIOS Protection Guidelines for Servers”
  • NIST SP800-193 “Platform Firmware Resiliency Guidelines”
  • ISO/IEC 11889 “Trusted Platform Module Library”
  • Common Criteria TCG Protection Profile for “PC Client Specific TPM 2.0”
  • European Union Commission Regulation 2019/424 (“ErP Lot 9”) “Ecodesign Requirements for Servers and Data Storage Products” Secure Data Deletion
  • Optional FIPS 140-2 validated Self-Encrypting Disks (SEDs) with external KMIP-based key management
• Product and Supply Chain Security
  • Suppliers validated through Lenovo’s Trusted Supplier Program
  • Developed in accordance with Lenovo’s Secure Development Lifecycle (LSDL)
  • Continuous firmware security validation through automated testing, including static code analysis, dynamic network and web vulnerability testing, software composition analysis, and subsystem-specific testing, such as UEFI security configuration validation
  • Ongoing security reviews by US-based security experts, with attestation letters available from our third-party security partners
  • Digitally signed firmware, stored and built on US-based infrastructure and signed on US-based Hardware Security Modules (HSMs)
  • TAA (Trade Agreements Act) compliant manufacturing, by default in Mexico for North American markets with additional US and EU manufacturing options
  • US 2019 NDAA (National Defense Authorization Act) Section 889 compliant

The server supports the following operating systems:

• Red Hat Enterprise Linux 8.6
• Red Hat Enterprise Linux 9.0
• SUSE Linux Enterprise Server 15 SP4
• SUSE Linux Enterprise Server 15 SP5
• SUSE Linux Enterprise Server 15 Xen SP4
• SUSE Linux Enterprise Server 15 Xen SP5
• Ubuntu 20.04 LTS 64-bit
• Ubuntu 22.04 LTS 64-bit

The server is also certified or tested with the following operating systems:

• Ubuntu
• Rocky Linux
• AlmaLinux

See Operating System Interoperability Guide (OSIG) for the complete list of supported, certified, and tested operating systems, including version and point releases: https://lenovopress.lenovo.com/osig#servers=sd665-n-v3-7daz&support=all

Also review the latest LeSI Best Recipe to see the operating systems that are supported via Lenovo Scalable Infrastructure (LeSI):
https://support.lenovo.com/us/en/solutions/HT505184#5
 

Six SD665-N V3 server trays are installed in the DW612S enclosure. Each SD665-N V3 tray has the following dimensions:

• Width: 438 mm (17.2 inches)
• Height: 41 mm (1.6 inches)
• Depth: 714 mm (28.1 inches) (769 mm, including the water connections at the rear of the server)

The DW612S enclosure has the following overall physical dimensions, excluding components that extend outside the standard chassis, such as EIA flanges, front security bezel (if any), and power supply handles:

• Width: 447 mm (17.6 inches)
• Height: 264 mm (10.4 inches)
• Depth: 933 mm (36.7 inches)

The following table lists the detailed dimensions. See the figure below for the definition of each dimension.

Figure 21. Enclosure dimensions

The SD665-N V3 tray has the following maximum weight:

• 23.6 kg (52.0 lb)

The DW612S enclosure has the following weight:

• Empty enclosure (with midplane and cables): 24.3 kg (53.5 lb)
• Fully configured enclosure:
  • With 9x air-cooled power supplies and 6x SD665-N V3 server trays (6 nodes): 182.9 kg (403 lb) (without water manifold)
  • with 3x water-cooled power supplies and 6x SD665-N V3 server trays (6 nodes): 188.7 kg (416 lb) (without water manifold)

The enclosure has the following electrical specifications for AC input power supplies:

• Input voltage:
  • 200 to 240 (nominal) Vac, 50 Hz or 60 Hz
  • 180 to 300 Vdc (China only)
• Max current for 2600W power supplies:
  • 200-208V AC: 13.2A
  • 220-240V AC: 13A
  • 240V DC: 11.9A (China only)
• Max current for 7200W power supplies (each of 3 inputs):
  • 200-208V AC: 12.7A
  • 220-240V AC: 12A
  • 240V DC: 11A (China only)

The SD665-N V3 server trays and DW612S enclosure are supported in the following environment:

Water requirements

7200W (220-240 Vac and 240 Vdc) DWC power supply:

• Water temperature:
  • ASHRAE class W+: up to 50°C (122°F)
• Maximum pressure: 4.4 bars
• Minimum water flow rate:1.5 liters per minute per power supply
  • For inlet water temperatures up to 45°C (113°F), 1.5 liters per minute per power supply
  • For inlet water temperatures between 45°C - 50°C (113°F - 122°F), 2.0 liters per minute per power supply

SD665-N V3 tray and DW612S enclosure are supported in the following environment:

• Water requirements
  • Water temperature: up to 45°C (113°F)
    • CPUs up to 400W TDP
    • DIMMs up to 96GB capacity
    • GPUs up to 600W TDP
    • NVIDIA network card up to 800Gb/sec
  • Water temperature: up to 40°C (104°F)
    • CPUs up to 400W TDP
    • DIMMs up to 96GB capacity
    • GPUs up to 700W TDP
    • NVIDIA network card up to 800Gb/sec
• Water requirement exceptions:
  • Water temperature: ASHRAE W32, up to 32°C (89.6°F)
    • 128 GB DIMMs
  • Water temperature: up to 40°C (104°F) with 5 LPM with 4 trays
    • AMD EPYC 9554 64C and 9554P 64C processors.
  • Water temperature: up to 35°C (95°F) with 5 LPM with 4 trays
    • AMD EPYC 9374F 32C, 9274F 24C, 9174F 16C, and 9474F 48C processors.
• Maximum pressure: 4.4 bars
• Water flow rates (1 tray consists of 1 compute node and 1 GPU node):
  • Water flow rate for 45°C (113°F): 20 liters per minute (lpm) per enclosure, assuming 5.0 liters per tray with 4 trays per enclosure.
  • Water flow rate for 40°C (104°F): 16 liters per minute (lpm) per enclosure, assuming 4.0 liters per tray with 4 trays per enclosure.
  • Water flow rate for 35°C (95°F): 17.5 liters per minute (lpm) per enclosure, assuming 3.5 liters per tray with 5 trays per enclosure.
  • Water flow rate for 35°C (95°F): 21 liters per minute (lpm) per enclosure, assuming 3.5 liters per tray with 6 trays per enclosure.

Note: The water required to initially fill the system side cooling loop must be reasonably clean, bacteria-free water (<100 CFU/ml) such as de-mineralized water, reverse osmosis water, de-ionized water, or distilled water. The water must be filtered with an in-line 50 micron filter (approximately 288 mesh). The water must be treated with anti-biological and anti-corrosion measures.

Air temperature requirements

• Operating: ASHRAE A2: 10°C to 35°C (50°F to 95°F); when the altitude exceeds 900 m (2953 ft), the maximum ambient temperature value decreases by 1°C (1.8°F) with every 300 m (984 ft) of altitude increase.
• Powered off: 5°C to 45°C (41°F to 113°F)
• Shipping/storage: -40°C to 60°C (-40°F to 140°F)

Relative humidity (non-condensing)

• Operating: ASHRAE Class A2: 8% - 80%, maximum dew point: 21°C (70°F)
• Shipment/storage: 8% - 90%

The following specifications indicate the limits of particulates that the system can tolerate:

• Reactive gases:
  • The copper reactivity level shall be less than 200 Angstroms per month (Å/month)
  • The silver reactivity level shall be less than 200 Å/month
• Airborne particulates:
  • The room air should be continuously filtered with MERV 8 filters.
  • Air entering a data center should be filtered with MERV 11 or preferably MERV 13 filters.
  • The deliquescent relative humidity of the particulate contamination should be more than 60% RH
  • Environment must be free of zinc whiskers

For additional information, see the Specifications section of the documentation for the server, available from the Lenovo Documents site, https://pubs.lenovo.com/

The SD665-N V3 conforms to the following standards:

• ANSI/UL 62368-1
• IEC 62368-1 (CB Certificate and CB Test Report)
• CSA C22.2 No. 62368-1
• Mexico NOM-019
• Brazil INMETRO
• South Africa NRCS LOA
• Ukraine UkrCEPRO
• Morocco CMIM Certification (CM)
• Russia, Belorussia and Kazakhstan, TP EAC 037/2016 (for RoHS)
• CE, UKCA Mark (EN55032 Class A, EN62368-1, EN55035, EN61000-3-11, EN61000-3-12, (EU) 2019/424, and EN IEC 63000 (RoHS))
• FCC - Verified to comply with Part 15 of the FCC Rules, Class A
• Canada ICES-003, issue 7, Class A
• CISPR 32, Class A, CISPR 35
• Korea KN32, Class A, KN35
• Japan VCCI, Class A
• Taiwan BSMI CNS15936, Class A; Section 5 of CNS15663
• Australia/New Zealand AS/NZS CISPR 32, Class A; AS/NZS 62368.1
• SGS, VOC Emission
• Energy Star 4.0
• EPEAT (NSF/ ANSI 426) Bronze
• Japanese Energy-Saving Act
• EU2019/424 Energy Related Product (ErP Lot9)
• China CELP certificate, HJ 2507-2011

The DW612S conforms to the following standards:

• ANSI/UL 62368-1
• IEC 62368-1 (CB Certificate and CB Test Report)
• CSA C22.2 No. 62368-1
• Mexico NOM-019
• Brazil INMETRO
• South Africa NRCS LOA
• Ukraine UkrCEPRO
• Morocco CMIM Certification (CM)
• Russia, Belorussia and Kazakhstan, TP EAC 037/2016 (for RoHS)
• CE, UKCA Mark (EN55032 Class A, EN62368-1, EN55035, EN61000-3-11, EN61000-3-12, (EU) 2019/424, and EN IEC 63000 (RoHS))
• FCC - Verified to comply with Part 15 of the FCC Rules, Class A
• Canada ICES-003, issue 7, Class A
• CISPR 32, Class A, CISPR 35
• Korea KN32, Class A, KN35
• Japan VCCI, Class A
• Taiwan BSMI CNS15936, Class A; Section 5 of CNS15663
• Australia/New Zealand AS/NZS CISPR 32, Class A; AS/NZS 62368.1
• SGS, VOC Emission
• Energy Star 4.0
• EPEAT (NSF/ ANSI 426) Bronze
• Japanese Energy-Saving Act
• EU2019/424 Energy Related Product (ErP Lot9)
• China CELP certificate, HJ 2507-2011

The server and enclosure have the following warranty:

• Lenovo ThinkSystem SD665-N V3 (7DAZ) - 3-year warranty
• Lenovo ThinkSystem DW612S Enclosure (7D1L) - 3-year warranty
• Lenovo Neptune DWC Node Manifold (5469) - 3-year warranty
• Lenovo Neptune DWC RM100 In-Rack CDU (7DBL) -  1-year warranty through the vendor (Cooltera)
• Genie Lift GL-8 Material Lift (7D5Y) - 3-year warranty

The standard warranty terms are customer-replaceable unit (CRU) and onsite (for field-replaceable units FRUs only) with standard call center support during normal business hours and 9x5 Next Business Day Parts Delivered.

Lenovo’s additional support services provide a sophisticated, unified support structure for your data center, with an experience consistently ranked number one in customer satisfaction worldwide. Available offerings include:

• Premier Support

  Premier Support provides a Lenovo-owned customer experience and delivers direct access to technicians skilled in hardware, software, and advanced troubleshooting, in addition to the following:

  • Direct technician-to-technician access through a dedicated phone line
  • 24x7x365 remote support
  • Single point of contact service
  • End to end case management
  • Third-party collaborative software support
  • Online case tools and live chat support
  • On-demand remote system analysis
• Warranty Upgrade (Preconfigured Support)

  Services are available to meet the on-site response time targets that match the criticality of your systems.

  • 3, 4, or 5 years of service coverage
  • 1-year or 2-year post-warranty extensions
  • Foundation Service: 9x5 service coverage with next business day onsite response. YourDrive YourData is an optional extra (see below).
  • Essential Service: 24x7 service coverage with 4-hour onsite response or 24-hour committed repair (available only in select markets). Bundled with YourDrive YourData.
  • Advanced Service: 24x7 service coverage with 2-hour onsite response or 6-hour committed repair (available only in select markets). Bundled with YourDrive YourData.
• Managed Services

  Lenovo Managed Services provides continuous 24x7 remote monitoring (plus 24x7 call center availability) and proactive management of your data center using state-of-the-art tools, systems, and practices by a team of highly skilled and experienced Lenovo services professionals.

  Quarterly reviews check error logs, verify firmware & OS device driver levels, and software as needed. We’ll also maintain records of latest patches, critical updates, and firmware levels, to ensure you systems are providing business value through optimized performance.

• Technical Account Management (TAM)

  A Lenovo Technical Account Manager helps you optimize the operation of your data center based on a deep understanding of your business. You gain direct access to your Lenovo TAM, who serves as your single point of contact to expedite service requests, provide status updates, and furnish reports to track incidents over time. In addition, your TAM will help proactively make service recommendations and manage your service relationship with Lenovo to make certain your needs are met.

• Enterprise Server Software Support

  Enterprise Software Support is an additional support service providing customers with software support on Microsoft, Red Hat, SUSE, and VMware applications and systems. Around the clock availability for critical problems plus unlimited calls and incidents helps customers address challenges fast, without incremental costs. Support staff can answer troubleshooting and diagnostic questions, address product comparability and interoperability issues, isolate causes of problems, report defects to software vendors, and more.

• YourDrive YourData

  Lenovo’s YourDrive YourData is a multi-drive retention offering that ensures your data is always under your control, regardless of the number of drives that are installed in your Lenovo server. In the unlikely event of a drive failure, you retain possession of your drive while Lenovo replaces the failed drive part. Your data stays safely on your premises, in your hands. The YourDrive YourData service can be purchased in convenient bundles and is optional with Foundation Service. It is bundled with Essential Service and Advanced Service.

• Health Check

  Having a trusted partner who can perform regular and detailed health checks is central to maintaining efficiency and ensuring that your systems and business are always running at their best. Health Check supports Lenovo-branded server, storage, and networking devices, as well as select Lenovo-supported products from other vendors that are sold by Lenovo or a Lenovo-Authorized Reseller.

Examples of region-specific warranty terms are second or longer business day parts delivery or parts-only base warranty.

If warranty terms and conditions include onsite labor for repair or replacement of parts, Lenovo will dispatch a service technician to the customer site to perform the replacement. Onsite labor under base warranty is limited to labor for replacement of parts that have been determined to be field-replaceable units (FRUs). Parts that are determined to be customer-replaceable units (CRUs) do not include onsite labor under base warranty.

If warranty terms include parts-only base warranty, Lenovo is responsible for delivering only replacement parts that are under base warranty (including FRUs) that will be sent to a requested location for self-service. Parts-only service does not include a service technician being dispatched onsite. Parts must be changed at customer’s own cost and labor and defective parts must be returned following the instructions supplied with the spare parts.

Lenovo Services is a dedicated partner to your success. Our goal is to reduce your capital outlays, mitigate your IT risks, and accelerate your time to productivity.

Here’s a more in-depth look at what we can do for you:

• Asset Recovery Services

  Asset Recovery Services (ARS) helps customers recover the maximum value from their end-of-life equipment in a cost-effective and secure way. On top of simplifying the transition from old to new equipment, ARS mitigates environmental and data security risks associated with data center equipment disposal. Lenovo ARS is a cash-back solution for equipment based on its remaining market value, yielding maximum value from aging assets and lowering total cost of ownership for your customers. For more information, see the ARS page, https://lenovopress.com/lp1266-reduce-e-waste-and-grow-your-bottom-line-with-lenovo-ars.

• Assessment Services

  An Assessment helps solve your IT challenges through an onsite, multi-day session with a Lenovo technology expert. We perform a tools-based assessment which provides a comprehensive and thorough review of a company's environment and technology systems. In addition to the technology based functional requirements, the consultant also discusses and records the non-functional business requirements, challenges, and constraints. Assessments help organizations like yours, no matter how large or small, get a better return on your IT investment and overcome challenges in the ever-changing technology landscape.

• Design Services

  Professional Services consultants perform infrastructure design and implementation planning to support your strategy. The high-level architectures provided by the assessment service are turned into low level designs and wiring diagrams, which are reviewed and approved prior to implementation. The implementation plan will demonstrate an outcome-based proposal to provide business capabilities through infrastructure with a risk-mitigated project plan.

• Basic Hardware Installation

  Lenovo experts can seamlessly manage the physical installation of your server, storage, or networking hardware. Working at a time convenient for you (business hours or off shift), the technician will unpack and inspect the systems on your site, install options, mount in a rack cabinet, connect to power and network, check and update firmware to the latest levels, verify operation, and dispose of the packaging, allowing your team to focus on other priorities.

• Deployment Services

  When investing in new IT infrastructures, you need to ensure your business will see quick time to value with little to no disruption. Lenovo deployments are designed by development and engineering teams who know our Products & Solutions better than anyone else, and our technicians own the process from delivery to completion. Lenovo will conduct remote preparation and planning, configure & integrate systems, validate systems, verify and update appliance firmware, train on administrative tasks, and provide post-deployment documentation. Customer’s IT teams leverage our skills to enable IT staff to transform with higher level roles and tasks.

• Integration, Migration, and Expansion Services

  Move existing physical & virtual workloads easily, or determine technical requirements to support increased workloads while maximizing performance. Includes tuning, validation, and documenting ongoing run processes. Leverage migration assessment planning documents to perform necessary migrations.

• Data Center Power and Cooling Services

  The Data Center Infrastructure team will provide solution design and implementation services to support the power and cooling needs of the multi-node chassis and multi-rack solutions. This includes designing for various levels of power redundancy and integration into the customer power infrastructure. The Infrastructure team will work with site engineers to design an effective cooling strategy based on facility constraints or customer goals and optimize a cooling solution to ensure high efficiency and availability. The Infrastructure team will provide the detailed solution design and complete integration of the cooling solution into the customer data center. In addition, the Infrastructure team will provide rack and chassis level commissioning and stand-up of the water-cooled solution which includes setting and tuning of the flow rates based on water temperature and heat recovery targets. Lastly, the Infrastructure team will provide cooling solution optimization and performance validation to ensure the highest overall operational efficiency of the solution.

The DW612S enclosure is supported in the following racks:

• Lenovo EveryScale 42U Onyx Heavy Duty Rack Cabinet, model 1410-O42
• Lenovo EveryScale 42U Pearl Heavy Duty Rack Cabinet, model 1410-P42
• Lenovo EveryScale 48U Onyx Heavy Duty Rack Cabinet, model 1410-O48
• Lenovo EveryScale 48U Pearl Heavy Duty Rack Cabinet, model 1410-P48

Considering the weight of the trays in the enclosure, an onsite material lift is required to allow service by a single person. If you do not already have a material lift available, Lenovo offers the Genie Lift GL-8 material lift as configurable option to the rack cabinets. Ordering information is listed in the following table.

Lenovo Financial Services reinforces Lenovo’s commitment to deliver pioneering products and services that are recognized for their quality, excellence, and trustworthiness. Lenovo Financial Services offers financing solutions and services that complement your technology solution anywhere in the world.

We are dedicated to delivering a positive finance experience for customers like you who want to maximize your purchase power by obtaining the technology you need today, protect against technology obsolescence, and preserve your capital for other uses.

We work with businesses, non-profit organizations, governments and educational institutions to finance their entire technology solution. We focus on making it easy to do business with us. Our highly experienced team of finance professionals operates in a work culture that emphasizes the importance of providing outstanding customer service. Our systems, processes and flexible policies support our goal of providing customers with a positive experience.

We finance your entire solution. Unlike others, we allow you to bundle everything you need from hardware and software to service contracts, installation costs, training fees, and sales tax. If you decide weeks or months later to add to your solution, we can consolidate everything into a single invoice.

Our Premier Client services provide large accounts with special handling services to ensure these complex transactions are serviced properly. As a premier client, you have a dedicated finance specialist who manages your account through its life, from first invoice through asset return or purchase. This specialist develops an in-depth understanding of your invoice and payment requirements. For you, this dedication provides a high-quality, easy, and positive financing experience.

The following sales training courses are offered for employees and partners (login required). Courses are listed in date order.

1. Partner Technical Webinar - ISG Portfolio Update
   2024-04-15 | 60 minutes | Employees and Partners
   Details
   Partner Technical Webinar - ISG Portfolio Update
   
   

   In this 60-minute replay, Mark Bica, NA ISG Server Product Manager reviewed the Lenovo ISG portfolio. He covered new editions such as the SR680a \ SR685a, dense servers, and options that are strategic for any workload.

   Published: 2024-04-15
   Length: 60 minutes
   Employee link: Grow@Lenovo
   Partner link: Lenovo Partner Learning
   Course code: 041224
2. Partner Technical Webinar – StorMagic
   2024-03-19 | 60 minutes | Employees and Partners
   Details
   Partner Technical Webinar – StorMagic
   
   

   March 08, 2024 – In this 60-minute replay, Stuart Campbell and Wes Ganeko of StorMagic joined us and provided an overview of StorMagic on Lenovo. They also demonstrated the interface while sharing some interesting use cases.

   Published: 2024-03-19
   Length: 60 minutes
   Employee link: Grow@Lenovo
   Partner link: Lenovo Partner Learning
   Course code: 030824
3. Family Introduction ThinkSystem Supercomputing Servers
   2024-02-21 | 10 minutes | Employees and Partners
   Details
   Family Introduction ThinkSystem Supercomputing Servers
   
   

   After completing this introductory course about the supercomputing server family, the learner will be able to define the characteristics of high-density servers, describe the supercomputing servers, recognize when a product from the supercomputing server family might be used, and identify keywords or buzzwords that indicate opportunities to introduce customers to the supercomputing server family products.

   Published: 2024-02-21
   Length: 10 minutes
   Employee link: Grow@Lenovo
   Partner link: Lenovo Partner Learning
   Course code: SXXW2526
4. Intel Transparent Supply Chain on Lenovo Servers
   2024-01-29 | 12 minutes | Employees and Partners
   Details
   Intel Transparent Supply Chain on Lenovo Servers
   
   

   This course introduces the Intel Transparent Supply Chain (TSC) program, explains how the program works, and discusses the benefits of the Intel TSC program to customers. Adding the Intel TSC feature to an order is explained.
   
   Course objectives:
   • Describe the Intel® Transparent Supply Chain program
   • Explain how the Intel® Transparent Supply Chain program works
   • Discuss the benefits of the Intel® Transparent Supply Chain program to Lenovo customers
   • Explain how to add Intel® Transparent Supply Chain program feature to an order

   Published: 2024-01-29
   Length: 12 minutes
   Employee link: Grow@Lenovo
   Partner link: Lenovo Partner Learning
   Course code: SXXW1230
5. Family Portfolio: Storage Controller Options
   2024-01-23 | 25 minutes | Employees and Partners
   Details
   Family Portfolio: Storage Controller Options
   
   

   This course covers the storage controller options available for use in Lenovo servers. The classes of storage controller are discussed, along with a discussion of where they are used, and which to choose.
   
   After completing this course, you will be able to:
   • Describe the classes of storage controllers
   • Discuss where each controller class is used
   • Describe the available options in each controller class

   Published: 2024-01-23
   Length: 25 minutes
   Employee link: Grow@Lenovo
   Partner link: Lenovo Partner Learning
   Course code: SXXW1111
6. Lenovo-Intel Sustainable Solutions QH
   2024-01-22 | 10 minutes | Employees and Partners
   Details
   Lenovo-Intel Sustainable Solutions QH
   
   

   This Quick Hit explains how Lenovo and Intel are committed to sustainability, and introduces the Lenovo-Intel joint sustainability campaign. You will learn how to use this campaign to show customers what that level of commitment entails, how to use the campaign's unsolicited proposal approach, and how to use the campaign as a conversation starter which may lead to increased sales.

   Published: 2024-01-22
   Length: 10 minutes
   Employee link: Grow@Lenovo
   Partner link: Lenovo Partner Learning
   Course code: SXXW2524a
7. FY24Q3 Intel Servers Update
   2023-12-11 | 15 minutes | Employees and Partners
   Details
   FY24Q3 Intel Servers Update
   
   

   This update is designed to help you discuss the features and customer benefits of Lenovo servers that use the 5th Gen Intel® Xeon® processors. Lenovo has also introduced a new server, the ThinkSystem SD650-N V3, which expands the supercomputer server family. Reasons to call your customer and talk about refreshing their infrastructure are also included as a guideline.

   Published: 2023-12-11
   Length: 15 minutes
   Employee link: Grow@Lenovo
   Partner link: Lenovo Partner Learning
   Course code: SXXW2522a
8. Lenovo Data Center Product Portfolio
   2023-07-21 | 15 minutes | Employees and Partners
   Details
   Lenovo Data Center Product Portfolio
   
   

   This course introduces the Lenovo data center portfolio, and covers servers, storage, storage networking, and software-defined infrastructure products. After completing this course about Lenovo data center products, you will be able to identify product types within each data center family, describe Lenovo innovations that this product family or category uses, and recognize when a specific product should be selected.

   Published: 2023-07-21
   Length: 15 minutes
   Employee link: Grow@Lenovo
   Partner link: Lenovo Partner Learning
   Course code: SXXW1110r6
9. Partner Technical Webinar - Data Center Limits and ISG TAA Compliance
   2023-05-16 | 60 minutes | Employees and Partners
   Details
   Partner Technical Webinar - Data Center Limits and ISG TAA Compliance
   
   

   In this 60-minute replay, we had two topics. First Vinod Kamath, Lenovo Distinguished Engineer for Data Center Cooling presented on the Systems Configuration and Data Center Ambient Limits. Second, Shama Patari, Lenovo Trade Council, and Glenn Johnson, Lenovo Principal Engineer for Supply Chain presented on ISG TAA Compliance.

   Published: 2023-05-16
   Length: 60 minutes
   Employee link: Grow@Lenovo
   Partner link: Lenovo Partner Learning
   Course code: 051223
10. Introduction to the AMD EPYC Gen4 Processors
    2022-12-30 | 10 minutes | Employees and Partners
    Details
    Introduction to the AMD EPYC Gen4 Processors
    
    

    After completing this course, you should be able to describe the Gen4 AMD EPYC™ processors, discuss the new and improved features of the Gen4 processors, and describe the customer benefits of the AMD EPYC Gen4 processors. You should also be able to list the Lenovo servers that use AMD EPYC Gen4 processors.

    Published: 2022-12-30
    Length: 10 minutes
    Employee link: Grow@Lenovo
    Partner link: Lenovo Partner Learning
    Course code: SXXW2501
11. Partner Technical Webinar - ThinkSystem AMD V3 servers
    2022-11-22 | 60 minutes | Partners Only
    Details
    Partner Technical Webinar - ThinkSystem AMD V3 servers
    
    

    In this 60-minute replay, Mark Bica and Joe Murphy presented the details of the new announced ThinkSystem Servers V3 based on the new 4th Generation AMD EPYC processor. They presented the ThinkSytem SR645 V3, SR665 V3, SR675 V3, SD665 V3 and SD665-N V3.

    Published: 2022-11-22
    Length: 60 minutes
    Partner link: Lenovo Partner Learning
    Course code: 111822
12. ThinkSystem AMD Dense Servers Preview Oct 22
    2022-10-26 | 4 minutes | Employees and Partners
    Details
    ThinkSystem AMD Dense Servers Preview Oct 22
    
    

    This Quick Hit previews the new Lenovo ThinkSystem high-density servers with AMD EPYC Gen4 processors. The SD665 V3 and SD665-N V3 are water-cooled servers that support TruDDR5 memory and PCIe Gen5.

    Published: 2022-10-26
    Length: 4 minutes
    Employee link: Grow@Lenovo
    Partner link: Lenovo Partner Learning
    Course code: SXXW1501a
13. Family Portfolio: AMD Dense Servers
    2022-10-26 | 15 minutes | Employees and Partners
    Details
    Family Portfolio: AMD Dense Servers
    
    

    This course introduces the first Lenovo dense servers with AMD processors – the SD665 V3 and SD665-N V3. After completing this course, you should be able to identify products and characteristics within the family, describe innovative features used in the family, and recognize when a specific server should be selected.

    Published: 2022-10-26
    Length: 15 minutes
    Employee link: Grow@Lenovo
    Partner link: Lenovo Partner Learning
    Course code: SXXW1501
14. Lenovo Infrastructure Solutions Launch
    2022-09-16 | 8 minutes | Employees and Partners
    Details
    Lenovo Infrastructure Solutions Launch
    
    

    This Quick Hit introduces a wealth of new products, solutions, and services announced as part of the Lenovo ThinkSystem 30th Anniversary celebration.

    Published: 2022-09-16
    Length: 8 minutes
    Employee link: Grow@Lenovo
    Partner link: Lenovo Partner Learning
    Course code: FY23Q2a
15. Neptune Direct Water Cooling Preview
    2022-09-16 | 3 minutes | Employees and Partners
    Details
    Neptune Direct Water Cooling Preview
    
    

    This Quick Hit introduces the 5th Generation Lenovo Neptune cooling solutions. An industry leader in liquid cooling, Lenovo Neptune delivers alternative cooling solutions that range from liquid-assisted air cooling to direct water-cooled CPUs and GPUs.

    Published: 2022-09-16
    Length: 3 minutes
    Employee link: Grow@Lenovo
    Partner link: Lenovo Partner Learning
    Course code: SXXW2509a
16. Lenovo Sustainable Computing
    2022-09-16 | 4 minutes | Employees and Partners
    Details
    Lenovo Sustainable Computing
    
    

    This Quick Hit describes the Lenovo sustainable computing program, and the many ways in which Lenovo strives to respect and protect the environment.

    Published: 2022-09-16
    Length: 4 minutes
    Employee link: Grow@Lenovo
    Partner link: Lenovo Partner Learning
    Course code: SXXW2504a
17. Introduction to DDR5 Memory
    2022-08-23 | 10 minutes | Employees and Partners
    Details
    Introduction to DDR5 Memory
    
    

    This course introduces DDR5 memory, describes new features of this memory generation, and discusses the advantages to customers of this new memory generation.

    Published: 2022-08-23
    Length: 10 minutes
    Employee link: Grow@Lenovo
    Partner link: Lenovo Partner Learning
    Course code: SXXW2502

1-10 of 17 courses. Show All

For more information, see these resources:

• ThinkSystem SD665-N V3 product page
  https://www.lenovo.com/us/en/p/servers-storage/servers/high-density/thinksystem-sd665-n-v3/len21ts0011
• ThinkSystem SD665 V3 datasheet
  https://lenovopress.com/DS0153
• Interactive 3D Tour of the ThinkSystem SD665-N V3:
  https://lenovopress.lenovo.com/lp1633-thinksystem-sd665-n-v3-interactive-3d-tour
• ThinkSystem SD665-N V3 drivers and support
  http://datacentersupport.lenovo.com/products/servers/thinksystem/sd665nv3/7daz/downloads
• Lenovo Hardware Installation & Removal Videos on the ThinkSystem DW612S Enclosure:
  https://www.youtube.com/playlist?list=PLYV5R7hVcs-A9Ws8PYIbxz9JbIrq1zoYd
• Lenovo ThinkSystem SD665-N V3 product publications:
  https://pubs.lenovo.com/sd665-n-v3/
  • User Guide
  • Rack Installation Guide
  • Messages and Codes Reference
  • UEFI Manual
• Lenovo ThinkSystem DW612S product publications:
  https://pubs.lenovo.com/dw612s_neptune_enclosure/
  • User Guide
  • Rack Installation Guide
  • SMM2 Messages and Codes Reference
• ServerProven hardware compatibility:
  http://www.lenovo.com/us/en/serverproven
• Data Center Solution Configurator (DCSC)
  https://dcsc.lenovo.com
• Lenovo Cluster solutions configurator (x-config)
  https://lesc.lenovo.com/products/hardware/configurator/worldwide/bhui/asit/index.html