GPU

NVIDIA H800

Edit@3 months ago

Intergrated Memory(VRAM)
Capacity

80 GB

(HBM3 5120-bit)

Bandwidth

3361 GB/s

480 Token/s

Vector Compute
FP64
1.04 T
FP32
66.91 T
FP16
133.80 T
BF16
133.80 T
INT32
33.45 T
INT8
X

NVIDIA H800 General-Purpose Float-Point performance (Vector Performance / Scalar Performance)

FP64: 1.04 TFLOPS

FP32: 66.91 TFLOPS

FP16: 133.80 TFLOPS

BF16: 133.80 TFLOPS

INT32: 33.45 TOPS

Matirx Compute
FP64
1.04 T
2.09 T
FP32
X
FP16
989.43 T
1978.86 T
FP8
1978.86 T
3957.71 T
TF32
494.71 T
989.43 T
BF16
989.43 T
1978.86 T
INT16
X
INT8
X
INT4
X

NVIDIA H800 AI performance (Tensor Performance / Matrix Performance)

FP64: 1.04 TFLOPS, with sparsity: 2.09 TFLOPS

FP16: 989.43 TFLOPS, with sparsity: 1978.86 TFLOPS

FP8: 1978.86 TFLOPS, with sparsity: 3957.71 TFLOPS

TF32: 494.71 TFLOPS, with sparsity: 989.43 TFLOPS

BF16: 989.43 TFLOPS, with sparsity: 1978.86 TFLOPS

Hardware Specs
NVIDIA H800 is a 5nm chip, has 80000 million transistors, launched by NVIDIA at 2023. It has 80 GB built-in(On-Board/On-Chip) memory with bandwidth up to 3361 GB/s. It has 16896 general-purpose ALUs(CUDA cores/Shader cores) and 528 matrix cores(Tensor cores) .
Process Node
5 nm
Launch Year
2023
Vector(CUDA) Cores
16896
Matrix(Tensor) Cores
528
Core Frequency
1095 ~ 1755 MHz
Cache
50MB