.. _Hardware: ==================== Hardware information ==================== .. Contents:: Niflheim overview ================= Niflheim currently (as of March 2026) consists of the following hardware: * A total of **698 compute nodes**. * The nodes contain a total of 31160 CPU_cores_ and **106 Graphical Processing Units** (GPU_). * Aggregate theoretical peak CPU performance of more than **2380 TeraFLOPS** (TFLOPS_) or **2.3 PetaFLOPS** (PFLOPS_). * Total aggregate RAM memory is 313 TB_. .. _CPU_cores: https://en.wikipedia.org/wiki/Multi-core_processor .. _GPU: https://en.wikipedia.org/wiki/Graphics_processing_unit .. _GFLOPS: http://en.wikipedia.org/wiki/FLOPS .. _TFLOPS: http://en.wikipedia.org/wiki/FLOPS .. _PFLOPS: http://en.wikipedia.org/wiki/FLOPS CPU and GPU architectures =========================== Niflheim comprises several generations of hardware as described below. Lenovo AMD EPYC 9474F servers ------------------------------ * **80 96-core nodes** Lenovo SD665_V3_ with two AMD EPYC Zen4_ 9474F_ 48-core CPUs (a total of 6528 cores) running at 3.60 GHz Base frequency (up to 4.10 GHz in Boost Clock) and including AVX-512_ vector instructions. The Direct Water Cooled servers are mounted in Lenovo *Neptune* DW612S_ chasisses. Peak speed: 6500 GFLOPS_/node, 520 TFLOPS_ in total for all nodes. The RAM memory is 768 GB_ (8.0 GB_/core) 4800 MHz DDR5 dual-rank memory, 52.2 TB_ in total for all nodes. Each server has a 1.9 TB_ local NVMe hard disk. The network interconnect is **100 Gbit/s** water-cooled `ThinkSystem NVIDIA ConnectX-7 NDR200 InfiniBand QSFP112 Adapters `_. Installed in **February 2024, March and December 2025**. .. _SD665_V3: https://lenovopress.lenovo.com/lp1612-lenovo-thinksystem-sd665-v3-server .. _DW612S: https://pubs.lenovo.com/dw612s_neptune_enclosure/ .. _9474F: https://www.amd.com/en/products/processors/server/epyc/4th-generation-9004-and-8004-series/amd-epyc-9474f.html .. _Zen4: https://www.amd.com/en/campaigns/epyc-9004-architecture Lenovo GPU servers with 4xH200 SXM4 --------------------------------------- * **2 GPU nodes** Lenovo SD650-N_V3_ 1U servers that feature the Intel Xeon family of processors and 4 times NVIDIA HGX H200_ *H200-SXM4-141GB* GPUs and an NVLink_ interconnect. The processors are Intel Xeon Gold 6442Y 96 CPU cores (2*24 cores with Hyperthreading_) running at 2.60 GHz Base frequency. The RAM memory is 512 GB_ 5600 MHz DDR5 dual-rank memory per node. CPU peak speed: 3994 GFLOPS_/node. The Direct Water Cooled servers are mounted in Lenovo *Neptune* DW612S_ chasisses. Installed in **July 2025**. .. _SD650-N_V3: https://lenovopress.lenovo.com/lp1613-thinksystem-sd650-n-v3-server .. _H200: https://www.nvidia.com/en-us/data-center/h200/ Lenovo Intel servers with 4 TB RAM memory -------------------------------------------- * **4 32-core nodes** Lenovo SR850_V3_ 2U servers. The four processors are Intel(R) Xeon(R) Gold 6434H_ running at 3.70 GHz Base frequency (up to 4.10 GHz in Turbo_mode_) and including AVX-512_ vector instructions.. The RAM memory is 4096 GB_ (128 GB_/core) 4800 MHz DDR5 dual-rank memory, 16 TB_ in total for all nodes. Peak speed: 3789 GFLOPS_/node, 15 TFLOPS_ in total for all nodes. Each server has 15 TB_ local NVMe_ scratch disk (striped over 4 disks of 3.9 TB_ for performance). Installed in **February 2024**. .. _SR850_V3: https://lenovopress.lenovo.com/lp1605-thinksystem-sr850-v3-server .. _NVMe: https://en.wikipedia.org/wiki/NVM_Express .. _6434H: https://www.intel.com/content/www/us/en/products/sku/232387/intel-xeon-gold-6434h-processor-22-5m-cache-3-70-ghz/specifications.html Lenovo GPU servers with 4xA100 SXM4 --------------------------------------- * **4 GPU nodes** Lenovo SD650-N_V2_ 1U servers that feature the Intel Xeon family of processors and 4 times NVIDIA A100_ *A100-SXM4-40GB* GPUs and an NVLink_ interconnect. The processors are Intel Xeon Platinum 8358_ 2 times 32-core CPUs running at 2.60 GHz Base frequency (up to 3.40 GHz in Turbo_mode_) and including AVX-512_ vector instructions. The RAM memory is 512 GB_ 3200 MHz DDR4 dual-rank memory per node, 2 TB_ in total for all nodes. CPU peak speed: 5325 GFLOPS_/node, 21 TFLOPS_ in total for all nodes. The Direct Water Cooled servers are mounted in Lenovo *Neptune* DW612S_ chasisses. Installed in **February 2024**. .. _SD650-N_V2: https://lenovopress.lenovo.com/lp1613-thinksystem-sd650-n-v2-server .. _A100: https://www.nvidia.com/en-us/data-center/a100/ .. _NVLink: https://en.wikipedia.org/wiki/NVLink .. _8358: https://www.intel.com/content/www/us/en/products/sku/212282/intel-xeon-platinum-8358-processor-48m-cache-2-60-ghz/specifications.html Dell AMD server with Instinct MI210 GPU ---------------------------------------- * **1 GPU node** Dell R7525_ with two AMD Instinct_ MI210_ GPUs with 64 GB_ HBM2e memory. This node has two AMD EPYC 7313 16-Core Processor and 256 GB_ RAM memory. Installed in **December 2023**. .. _R7525: https://www.dell.com/en-us/shop/cty/pdp/spd/poweredge-r7525/pe_r7525_tm_vi_vp_sb .. _Instinct: https://www.amd.com/en/products/accelerators/instinct.html .. _MI210: https://www.amd.com/en/products/accelerators/instinct/mi200/mi210.html Dell Ice Lake servers --------------------- * **96 56-core nodes** Dell R650_ with two Intel IceLake_ Xeon_Gold_6348_ 28-core CPUs (a total of 5376 cores) running at 2.60 GHz Base frequency (up to 3.50 GHz in Turbo_mode_) and including AVX-512_ vector instructions. Peak speed: 4659 GFLOPS_/node, 447 TFLOPS_ in total for all nodes. The RAM memory is 512 GB_ (9.14 GB_/core) 3200 MHz DDR4 dual-rank memory, 49.1 TB_ in total for all nodes. Each server has a 480 GB_ local SSD hard disk. The network interconnect is **100 Gbit/s** Cornelis_Networks_ (previously Intel) OmniPath_. Installed in **December 2021**. SuperMicro servers ------------------ * **8 40-core nodes** SYS-4029GP-TRT2_ from Nextron/SuperMicro with Intel Xeon_Gold_5218R_ 20-core CPUs @2.10 GHz (a total of 320 cores). Hyperthreading_ is enabled so that each node has 80 threads or virtual CPUs. Peak speed: 2688 GFLOPS_/node, 21 TFLOPS_ in total for all nodes. The RAM memory is 198 GB_ (4 nodes) and 768 GB_ (4 nodes), 3.81 TB_ in total for all nodes. These servers are equipped with Nvidia GPUs. Each server has a 960 GB_ local NVMe hard disk and a 1 Gbit Ethernet. Installed in **December 2020 and July 2021**. .. _SYS-4029GP-TRT2: https://www.supermicro.com/en/products/system/4U/4029/SYS-4029GP-TRT2.cfm .. _Hyperthreading: https://en.wikipedia.org/wiki/Hyper-threading Dell Cascade Lake Refresh servers --------------------------------- * **128 40-core nodes** Dell R640_ with two Intel Cascade_Lake_ Xeon_Gold_6242R_ 20-core CPUs (a total of 5120 cores) running at 3.10 GHz Base frequency (up to 4.10 GHz in Turbo_mode_) and including AVX-512_ vector instructions. Peak speed: 3968 GFLOPS_/node, 508 TFLOPS_ in total for all nodes. The RAM memory is 384 GB_ (9.6 GB_/core) 2933 MHz DDR4 dual-rank memory, 49.1 TB_ in total for all nodes. Each server has a 240 GB_ local SSD hard disk. The network interconnect is **100 Gbit/s** Cornelis_Networks_ (previously Intel) OmniPath_. Installed in **October 2020**. Dell Skylake servers -------------------- * **208 40-core nodes** Dell C6420_ and R640_ with two Intel Skylake_ Xeon_Gold_6148_ 20-core CPUs (a total of 8320 cores) running at 2.40 GHz Base frequency (up to 3.70 GHz in Turbo_mode_) and including AVX-512_ vector instructions. Peak speed: 3072 GFLOPS_/node, 639 TFLOPS_ in total for all nodes. The RAM memory type is 2666 MHz DDR4 dual-rank memory: * 196 C6420_ nodes have 384 GB_ of memory (9.6 GB_/core), 75.3 TB_ in total for all nodes. * 12 R640_ nodes have 768 GB_ of memory (19.2 GB_/core), 9.2 TB_ in total for all nodes. Each server has a 240 GB_ local SSD hard disk. The network interconnect is **100 Gbit/s** Cornelis_Networks_ (previously Intel) OmniPath_. Installed in **April 2019**. Huawei Broadwell servers ------------------------ * **192 24-core nodes** `Huawei XH620 v3 `_ with two Intel Broadwell_ Xeon_E5-2650_v4_ 12-core CPUs (a total of 4608 cores) running at 2.20 GHz (up to 2.90 GHz in Turbo_mode_). Peak speed: 845 GFLOPS_/node, 162 TFLOPS_ in total for all nodes. The RAM memory type is 2400 MHz DDR4 dual-rank memory: * 180 nodes have 256 GB_ of memory (10.7 GB_/core), 46.1 TB_ in total for all nodes. * 12 nodes have 512 GB_ of memory (21.3 GB_/core), 6.1 TB_ in total for all nodes. Each server has a 240 GB_ local SSD hard disk. The network interconnect is **100 Gbit/s** Cornelis_Networks_ (previously Intel) OmniPath_. Installed in **December 2016, March 2017, November 2017**. .. _OmniPath: https://www.cornelisnetworks.com/products/ .. _Cornelis_Networks: https://www.cornelisnetworks.com/ .. _Infiniband: http://en.wikipedia.org/wiki/InfiniBand .. _IceLake: https://en.wikipedia.org/wiki/Ice_Lake_(microprocessor) .. _Cascade_Lake: https://en.wikipedia.org/wiki/Cascade_Lake_(microarchitecture) .. _Skylake: https://en.wikipedia.org/wiki/Skylake_(microarchitecture) .. _Broadwell: https://en.wikipedia.org/wiki/Broadwell_%28microarchitecture%29 .. _AVX-512: https://en.wikipedia.org/wiki/AVX-512 .. _Xeon_Gold_6348: https://www.intel.com/content/www/us/en/products/sku/212456/intel-xeon-gold-6348-processor-42m-cache-2-60-ghz/specifications.html .. _Xeon_Gold_5218R: https://ark.intel.com/content/www/us/en/ark/products/199342/intel-xeon-gold-5218r-processor-27-5m-cache-2-10-ghz.html .. _Xeon_Gold_6242R: https://ark.intel.com/content/www/us/en/ark/products/199352/intel-xeon-gold-6242r-processor-35-75m-cache-3-10-ghz.html .. _Xeon_Gold_6148: https://ark.intel.com/content/www/us/en/ark/products/120489/intel-xeon-gold-6148-processor-27-5m-cache-2-40-ghz.html .. _Xeon_E5-2650_v4: https://ark.intel.com/content/www/us/en/ark/products/91767/intel-xeon-processor-e5-2650-v4-30m-cache-2-20-ghz.html .. _Xeon_E5-2650_v2: https://ark.intel.com/content/www/us/en/ark/products/75269/intel-xeon-processor-e5-2650-v2-20m-cache-2-60-ghz.html .. _Xeon_E5-2670: https://ark.intel.com/content/www/us/en/ark/products/64595/intel-xeon-processor-e5-2670-20m-cache-2-60-ghz-8-00-gt-s-intel-qpi.html .. _Xeon_X5550: https://ark.intel.com/content/www/us/en/ark/products/37106/intel-xeon-processor-x5550-8m-cache-2-66-ghz-6-40-gt-s-intel-qpi.html .. _Xeon_X5570: https://ark.intel.com/content/www/us/en/ark/products/37111/intel-xeon-processor-x5570-8m-cache-2-93-ghz-6-40-gt-s-intel-qpi.html .. _C6420: https://www.dell.com/en-us/work/shop/povw/poweredge-c6420 .. _R640: https://www.dell.com/en-us/work/shop/povw/poweredge-r640 .. _R650: https://www.dell.com/en-us/work/shop/povw/poweredge-r650 .. _Turbo_mode: https://en.wikipedia.org/wiki/Intel_Turbo_Boost File servers ============ Several Linux file servers are available for the departmental user groups. Each group is assigned a file-system on one of the existing file servers. Depending on disk requirements, group file-systems can be from 1 TB_ and up. The file servers are standard Linux servers with large disk arrays, sharing the file-systems using NFS. We do not use any parallel file servers (for example, Lustre_ etc.). The file server total available disk spaces are: * Server niflfs1: 218 TB_ * Server niflfs2: 218 TB_ * Server niflfs3: 218 TB_ * Server niflfs8: 163 TB_ * Server niflfs9: 163 TB_ * Server niflfs10: 163 TB_ * Server niflfs11: 196 TB_ * Server niflfs12: 218 TB_ * Server niflfs13: 218 TB_ A maximum disk capacity of 1.77 PB_ disk space is available for user applications. .. _GB: https://en.wikipedia.org/wiki/Gigabyte .. _TB: https://simple.wikipedia.org/wiki/Terabyte .. _PB: https://simple.wikipedia.org/wiki/Petabyte .. _Lustre: https://en.wikipedia.org/wiki/Lustre_%28file_system%29