Testbeds

JLSE provides testbeds for the evaluation of high-performance computing systems.

Note: Access to certain testbeds are restricted as it may require an NDA with the vendor for your home institution or it is provisionally allowed for certain projects. If you need access to any of these resources please send an email to help@jlse.anl.gov with a link to your project information on the website, as well as the users and home institutions that need access.

Status Current
Service Date
Hostname(s) grace00, grace01

NVIDIA GH200

2x Supermicro servers (ARS-111GL-NHR)
1x NVIDIA 72-core NVIDIA Grace CPU on GH200
480GB ECC LPDDR5X, 96GB ECC HBM3
1x NVIDIA H100 Tensor Core GPU on GH200
ConnectX 7 network interface
Access Restricted (gpu_gh200 queue)

Status Current
Service Date
Hostname(s) hopper00, hopper01

NVIDIA H100

Supermicro servers
2x Intel(R) Xeon(R) Platinum 8468 at 2100 MHz (48 cores/96 threads)
512GB DDR5 RAM,
4x NVIDIA H100 SXM5 80GB GPUs
ConnectX 7 network interface (not yet available)
Access Restricted (gpu_h100, hopper00, hopper01 queue)

Status Current
Service Date
Hostname(s) nextsilicon00, nextsilicon01

NextSilicon Maverick-1

2x Intel M50CYP

  • 2x Intel(R) Xeon(R) Gold 6336Y CPU @ 2.40GHz
  • 512GB DDR4-3200
  • 1x NextSilicon Maverick-1

Access Restricted (next silicon queue)

Status Current
Service Date
Hostname(s) amdgpu[04-05]

AMD GPU MI250

Supermicro AS-4124GQ-TNMI
– 2x AMD EPYC 7713 64c (Milan) 64C/128T 2.0GHz 256M 225W
– x4 AMD Instinct MI250 OAM 64GBx2 530W HBM2e PCIe Gen4
– 512GB DDR4-3200
– 1x EDR IB

Status Current
Service Date
Hostname(s) bluefield00 – bluefield03

NVIDIA Bluefield-2

4x ProLiant DL325, each of which has:
1X AMD EPYC 7433P 24-Core Processor (2850MHz)
256GB DDR4
1x Bluefield-2 NICs
Access Restricted

Status Current
Service Date
Hostname(s) Florentia

Florentia

6 Nodes – Intel® Server Board codenamed Denali Pass
4x Intel® Data Center GPU, codenamed Ponte Vecchio (PVC)
2x 4th Gen Intel® Xeon® Scalable processor
1TB: 16x 64GB DDR5 @ 4800
Intel® SSD DC P4511 Series (1.0TB, M.2 110mm PCIe 3.1 x4, 3D2, TLC)
Mellanox ConnectX-6: EDR InfiniBand (100 Gbps)

Status Current
Service Date
Hostname(s) altra00, altra01

NVIDIA ARM Dev Kit

2x Gigabyte G242-P32, 2U server
– 1x Ampere Altra Q80-30 (Arm processor)
– 512 GB DDR4 memory
– 2x NVIDIA A100 GPU (40GB HBM)
Access Restricted (arm-users)

Status Current
Service Date
Hostname(s) a64fx

Fujisu A64FX

HPE Apollo 80
8x Fujitsu A64FX ARM CPU
32GB HBM2 per CPU
Access Restricted (arm-users)

Status Current
Service Date

Intel Pre-production Development Platform

1x Node – Intel Pre-production Development Platform
2x Next Gen Intel Xeon Scalable processor
256GB: 16x 16GB DDR5 @ 4800
Mellanox ConnectX-6: EDR InfiniBand (100 Gbps)
Access restricted

Status Current
Service Date
Hostname(s) gpu04

NVIDIA A40

Gigabyte G242-Z11
AMD 7532 32c 2.4Ghz
DDR4-3200 256GB (8x32G DIMMs) RAM
1x Nvidia A40 48GB GDDR6 PCIe 4.0
Mellanox ConnectX-6 EDR
Intel P4510 2TB NVMe
Access Restricted (gpu_a40 queue)

Status Current
Service Date
Hostname(s) amdgpu02 and amdgpu03

AMD GPU MI100

– 2x AMD EPYC 7543 32c (Milan)
– 4x AMD MI100 32GB GPUs
– Infinity Fabric
– 512GB DDR4-3200
Access Restricted

Status Current
Service Date
Hostname(s) gpu[06-07]

Nvidia A100

Gigabyte G242-Z11
AMD 7532 32c 2.4Ghz
DDR4-3200 256GB (8x32G DIMMs) RAM
1x Nvidia A100 40GB PCIe 4.0
Mellanox ConnectX-6 EDR
Intel P4510 2TB NVMe
Access Restricted (gpu_a100 queue)

Status Current
Service Date
Hostname(s) amdgpu[00-01]

AMD GPU MI50

Gigabyte G482-Z51
– 2x 7742 64c Rome
– 4x AMD MI50 32GB GPUs
– Infinity Fabric
– 256GB DDR-3200 RAM
Access Restricted

Status Current
Service Date
Hostname(s) Arcticus

Arcticus

17 Nodes – Intel® Server Board code named Coyote Pass
2x Intel development GPU card (Codename XeHP_SDV)
2x Intel(R) Xeon Gold 6336Y CPU (48 physical cores total) 2.4Ghz
256GB: 16x 16GB DDR4 @ 3200
Intel® SSD D3-S4610 Series (960GB, 2.5in SATA 6Gb/s, 3D2)
Mellanox ConnectX-6: EDR InfiniBand (100 Gbps)

Status Current
Service Date
Hostname(s) cpl00-01

CPL “Cooper Lake”

Qty 2 – Supermicro SYS-240P-TNRT
– 4 Intel(R) Xeon(R) Platinum 8354H CPU @ 3.10GHz 18 cores/36 threads
– 384GiB DDR4 3200 MHz Memory
– 2x 1TB INTEL SSDPE2KX010T8
– 4x Mellanox MT27800
– 2x Intel X710 10GBASE-T
– 2x Intel X710 10 Gigabit SFP+
Access Restricted (cpl_8354H queue)

Status Current
Service Date
Hostname(s) cascade00-19

Cascade

Qty 20 – Supermicro SYS-1029GQ-TNRT
– 2x Intel Xeon Gold 6226R
– 12 x 16 GB DDR4-2666
– 1x Mellanox ConnectX-5 EDR
– 2x 1.0TB Intel SSD DC P4510 NVMe
Access Restricted (cascade queue)

Status Current
Service Date
Hostname(s) witherspoon00

Witherspoon

Qty 1 – IBM Power9 servers with 2 x 20-core Power9 processors @ 2.4GHz (3.00GHz turbo mode)
– 4x NVIDIA Volta V100 16GB SMX2 GPUs
– 512GB DDR4 2666Mhz memory
– 2x 960GB internal SSDs
– 1x 1.6TB NVMe Flash adapter
– Mellanox ConnectX-5 EDR
Access Restricted (gpu_power9_v100_smx2 queue)

Status Current
Service Date
Hostname(s) iris

Iris

Qty 20 – SuperMicro X11SSH-GF-1585 Server Motherboard
– Intel Xeon E3-1585 v5 CPU w/ Intel Iris Pro Graphics P580
– 4x 16GB DDR4-2666 SODIMMs (operating at DDR4-2133)
– 1GbE Onboard
Access Restricted

Status Current
Service Date
Hostname(s) gpu00-02

GPU_V100_SMX2

Qty 3 – SuperMicro SuperServer 1029GQ-TVRT
– 4x NVIDIA Tesla V100 SXM2 w/32GB HBM2
– 2x Intel Xeon Gold 6152 CPU 22c 2.10GHz
– 192GB RAM DDR4-2666
– Mellanox ConnectX-5 EDR
Access Restricted (DGX Queue)

Status Current
Service Date
Hostname(s) i2s00

I2S – SDN Testbed

Qty: 1 – Intel HNS2600KPR, 2x E5-2667v4 16c 3.20GHz, 128GB RAM, Mellanox ConnectX-4 HCA
Qty: 1 – IBM x3650 M4, Mellanox ConnectX-4 HCA
Qty: 2 – Mellanox SN2100 16x 100GbE Switch
Access Restricted

Status Current
Service Date
Hostname(s) epyc00-01

EPYC

Qty 2 – Gigabyte R181-Z91, 2x AMD EPYC 7601 32c 2.20Ghz, 128GB RAM, Mellanox ConnectX-5 EDR

Status Current
Service Date
Hostname(s) comanche

HPE Comanche

HPE Apollo 70 Prototype “Comanche”
– 2x Marvell ThunderX2 CPU CN9975 v2.1 @ 2.20GHz (CN9975-2200LG4077-Y21-G)
– 256GB DDR4-2400 (16x 16GB)
– 1x EDR – Mellanox ConnectX-5
Access Restricted (arm-users)

Status Current
Service Date
Hostname(s) skylake00-12

Skylake

Qty 13 – Intel S2600WF, (12 – 2x Intel Xeon Platinum 8180M CPU @ 2.50GHz, 1 2x Intel Xeon Platinum 8176 CPU @ 2.10GHz), 768GB RAM, EDR IB, Intel P4500 1TB NVMe SSD, Intel P4800X 375GB NVMe Optane SSD

Status Current
Service Date
Hostname(s) atos00

ATOS

Atos Quantum Learning Machine (QLM-35), Bull Bullion based system with 4x Intel E7-8890 v4, 3TB RAM
Access Restricted

Status Current
Service Date
Hostname(s) dgx2

DGX

Nvidia DGX-1, 8x Nvidia V100 GPUs with NVlink, https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/dgx-1/NV…
Access Restricted (DGX Queue)

Status Current
Service Date
Hostname(s) gomez00-04

Gomez

Qty 5 – SuperMicro 4048B-TR4FT, 4 Socket – Intel Haswell-EX E7-8867v3 16c 2.5Ghz. 1TB RAM, Mellanox EDR HCA, 200GB SATA SSD

Status Current
Service Date
Hostname(s) it00-12

It

Qty 13 – Intel HNS2600KPR, 2x E5-2699v4 22c 2.2Ghz, 128GB RAM, Omni-path HCA, 200GB SATA SSD

Status Current
Service Date
Hostname(s) firestone

Firestone

Qty 1 – IBM S822LC, OpenPower, Power8 10c 2.92Ghz, Mellanox EDR, Nvidia K80 GPU, Qty: 2 – 400GB SATA SSD, 128GB RAM

Status Current
Service Date
Hostname(s) knl00,01,09,10

KNL

Qty 4 – Intel Adam’s Pass S7200AP, (2 – KNL 7210 64c 1.3Ghz, 2 KNL 7250), 192GB RAM, (2 – Samsung 128GB NVMe SSD), Omni-path HCA, 200GB SATA SSD
Access Restricted (Open Intel Resources)

Status Current
Service Date
Hostname(s) thing00-07

Thing

Qty 8 – SuperMicro X10DRi, Intel Haswell-EP E5-2699v3, 128GB RAM, Mellanox EDR, 200GB SATA SSD, 4RU

Status Current

DevEP

32 Nodes – Coyote Pass Chassis
2x Intel development GPU card (Code name XeHP_SDV)
2x Intel(R) Xeon(R) Gold 6336Y CPU
Mellanox Fabic (100gbe and 200 IB)