Difference between revisions of "Leonhard"

From ScientificComputing
Jump to: navigation, search
 
(14 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
{{LeoOpen_obsolete}}
 +
 +
This page describes the hardware of the Leonhard cluster. For information on how to use the cluster, please check the [[Tutorials|tutorials]] page.
 +
 
==Introduction==
 
==Introduction==
''Leonhard'' is a new cluster designed for “big data” applications. This cluster is financed jointly by a small group of shareholders and IT Services, and by a special grant from the Executive Board of ETH.
+
''Leonhard'' is a cluster designed for “big data” applications. This cluster is financed jointly by a small group of shareholders and IT Services, and by a special grant from the Executive Board of ETH.
  
As the name implies, Leonhard is closely related to [[Euler]]. It is operated by the same group and offers the same software environment. One part of Leonhard is a normal system intended for shareholders dealing with ''open'' (public) research data; the other part is a secure system intended for ''confidential'' data, like ''personalized medicine''. In general, the name ''Leonhard'' refers to the whole cluster, and its two parts are called respectively ''Leonhard Open'' and ''Leonhard Med''. For security reasons, each part has its own dedicated storage, network and login nodes.
+
As the name implies, Leonhard is closely related to [[Euler]]. It is operated by the same group and offers a similar software environment. One part of Leonhard is a normal system intended for shareholders dealing with ''open'' (public) research data; the other part is a secure system intended for ''confidential'' data, like ''personalized medicine''. In general, the name ''Leonhard'' refers to the whole cluster, and its two parts are called respectively ''Leonhard Open'' and ''Leonhard Med''. For security reasons, each part has its own dedicated storage, network and login nodes.
  
 
Unlike [[Euler]], which is open to all members of ETH without restriction, Leonhard is reserved exclusively to the groups who have invested in it (the so-called ''shareholders''). Temporary ''guest accounts'' will be created on demand for prospective shareholders who want to try it out before buying in.
 
Unlike [[Euler]], which is open to all members of ETH without restriction, Leonhard is reserved exclusively to the groups who have invested in it (the so-called ''shareholders''). Temporary ''guest accounts'' will be created on demand for prospective shareholders who want to try it out before buying in.
 
==Accessing the cluster==
 
Beta testers of the Leonhard, can access the cluster via [[Accessing_the_clusters#SSH|SSH]]. You can find a documentation about the cluster on our [[Leonhard_beta_testing|Leonhard beta testing]] wiki page.
 
 
==Life time==
 
The cluster's components were delivered in December 2016. Leonhard Med was installed in December already is currently in "alpha" testing.
 
 
The installation of Leonhard Open is on-going. We plan to start "alpha" testing in March and to open it to "beta" users in April.
 
  
 
==Specifications==
 
==Specifications==
 
===Storage===
 
===Storage===
Each part of Leonhard is attached to a dedicated high-performance parallel file system — based on GPFS — with a capacity of 2.0 PB for Leonhard Open and 1.5 PB for Leonhard Med (1 PB = 10^15 bytes).
+
Each part of Leonhard is attached to a dedicated high-performance parallel file system with a capacity of 2.0 PB each (1 PB = 10^15 bytes).
 
 
Both file systems have built-in support for ''data encryption''.
 
 
 
Since Leonhard is intended for big data processing, storage is offered in blocks of '''20 TB''' (up to 100 TB) and '''50 TB''' (above 100 TB) and can only be purchased ''in combination'' with compute nodes (minimum 1 node).
 
  
 
===Compute nodes===
 
===Compute nodes===
Leonhard contains three types of compute nodes:
+
Leonhard contains several types of compute nodes:
* ''Standard nodes'' equipped with two '''18-core''' Intel Xeon E5-2697v4 processors and '''128 GB''' of memory
+
* 36 compute nodes equipped with two '''18-core''' Intel Xeon E5-2697v4 processors and '''128''' or '''512 GB''' of memory
* ''Large-memory nodes'' equipped with two '''18-core''' Intel Xeon E5-2697v4 processors and '''512 GB''' of memory
+
* 48 compute nodes equipped with two '''18-core''' Intel Xeon Gold 6140 processors and '''96''', '''384''' or '''768 GB''' of memory
* ''GPU nodes'' equipped with two '''10-core''' Xeon E5-2650v4 processors, '''256 GB''' of memory and '''eight''' Nvidia GTX 1080 GPUs
+
* 5 huge-memory nodes equipped with two '''24-core''' AMD Epyc 7451 processors and '''2048 GB''' of memory
 +
* 22 GPU nodes equipped with two '''10-core''' Xeon E5-2630v4 processors, '''256 GB''' of memory and '''8''' Nvidia '''GTX 1080''' GPUs
 +
* 70 GPU nodes equipped with two '''10-core''' Xeon E5-2630v4 processors, '''256 GB''' of memory and '''8''' Nvidia '''GTX 1080 Ti''' GPUs
 +
* 4 high-end GPU nodes (Nvidia DGX-1) equipped with 4 '''20-core''' Xeon E5-2698v4 processors, '''512 GB''' of memory and '''8''' Nvidia '''V100''' GPUs
  
Since Leonhard is intended for big data processing, compute nodes can only be purchased ''in combination'' with storage (minimum 20 TB).
+
In total Leonhard contains 5,128 CPU cores and 2,334,720 CUDA cores. The GPUs alone have a peak performance (single precision) of 7.0 PF (1 PF = 10^15 floating-point operations per second).
  
 
===Network===
 
===Network===
All compute nodes are connected together and to the cluster's parallel file systems via a 100 Gb/s InfiniBand EDR network.
+
All compute nodes are connected together and to the cluster's parallel file systems via a 100 Gb/s Ethernet network.
 
 
==Prices==
 
 
 
Prices cannot be published on-line. Prospective shareholders are invited to contact [mailto:cluster-support@id.ethz.ch Cluster Support] for more information.
 
 
 
<!--
 
==Wiki==
 
 
 
==Compute nodes==
 
 
 
==Networking==
 
  
==Trivia==
+
==Service description==
  
==External Links==
+
The official service description and the current price list are available on the [https://ethz.ch/services/en/it-services/katalog/server-cluster/hpc.html IT service catalogue].
-->
 

Latest revision as of 08:43, 16 September 2021

This page contains information about the Leonhard Open cluster, which is now obsolete as the cluster has been integrated into the Euler cluster on 14/15 September 2021

This page describes the hardware of the Leonhard cluster. For information on how to use the cluster, please check the tutorials page.

Introduction

Leonhard is a cluster designed for “big data” applications. This cluster is financed jointly by a small group of shareholders and IT Services, and by a special grant from the Executive Board of ETH.

As the name implies, Leonhard is closely related to Euler. It is operated by the same group and offers a similar software environment. One part of Leonhard is a normal system intended for shareholders dealing with open (public) research data; the other part is a secure system intended for confidential data, like personalized medicine. In general, the name Leonhard refers to the whole cluster, and its two parts are called respectively Leonhard Open and Leonhard Med. For security reasons, each part has its own dedicated storage, network and login nodes.

Unlike Euler, which is open to all members of ETH without restriction, Leonhard is reserved exclusively to the groups who have invested in it (the so-called shareholders). Temporary guest accounts will be created on demand for prospective shareholders who want to try it out before buying in.

Specifications

Storage

Each part of Leonhard is attached to a dedicated high-performance parallel file system with a capacity of 2.0 PB each (1 PB = 10^15 bytes).

Compute nodes

Leonhard contains several types of compute nodes:

  • 36 compute nodes equipped with two 18-core Intel Xeon E5-2697v4 processors and 128 or 512 GB of memory
  • 48 compute nodes equipped with two 18-core Intel Xeon Gold 6140 processors and 96, 384 or 768 GB of memory
  • 5 huge-memory nodes equipped with two 24-core AMD Epyc 7451 processors and 2048 GB of memory
  • 22 GPU nodes equipped with two 10-core Xeon E5-2630v4 processors, 256 GB of memory and 8 Nvidia GTX 1080 GPUs
  • 70 GPU nodes equipped with two 10-core Xeon E5-2630v4 processors, 256 GB of memory and 8 Nvidia GTX 1080 Ti GPUs
  • 4 high-end GPU nodes (Nvidia DGX-1) equipped with 4 20-core Xeon E5-2698v4 processors, 512 GB of memory and 8 Nvidia V100 GPUs

In total Leonhard contains 5,128 CPU cores and 2,334,720 CUDA cores. The GPUs alone have a peak performance (single precision) of 7.0 PF (1 PF = 10^15 floating-point operations per second).

Network

All compute nodes are connected together and to the cluster's parallel file systems via a 100 Gb/s Ethernet network.

Service description

The official service description and the current price list are available on the IT service catalogue.