💾 CPU + GPU, NVIDIA Superchip SuperPOD

2023 Aug Week 3 Datumo Newsletter

Editor: Jodie Jung

NVIDIA's Supercomputer and AI Ecosystem

Introducing the 'GH200' chip a combination of CPU and GPU

Source: NVIDIA Keynote at SIGGRAPH 2023. YouTube.

Last week, SIGGRAPH 2023, the largest conference in the computer graphics field, was held in Los Angeles. SIGGRAPH started in 1974 and this year marks its 50th anniversary.

There were many presentations, but NVIDIA's CEO Jensen Huang's keynote was definitely one of the hottest topics. As you may know, NVIDIA's current stock price has tripled since the beginning of the year. During his keynote, Jensen Huang introduced new supercomputers and chips, as well as a generative AI app development platform called AI WORKBENCH.

Let's peek into Huang's SIGGRAPH 2023 keynote to see what made it so interesting :)

Combining CPU and GPU: Grace Hopper Superchip

SuperPOD 'NVIDIA DGX GH200'

Source: NVIDIA Keynote at COMPUTEX 2023. YouTube.

Huang introduced NVIDIA's supercomputer, NVIDIA DGX GH200, which is capable of handling massive AI models with trillions of parameters. The DGX GH200 connects 256 'NVIDIA GH200 Grace Hopper Superchips' as a single GPU, providing up to 144 terabytes (TB) of GPU memory space.

The computing speed is 1 EFLOPS (exaflops). An exa is a unit representing 100 quintillion, so 1 EF means 100 quintillion floating-point operations. (If you're curious about FLOPS, please refer to the previous newsletter.)

The names are quite long - the computer's name is 'DGX GH200,' and the chip's name is 'GH200 Grace Hopper Superchip.' DGX refers to NVIDIA's supercomputing product line, while Grace denotes the CPU, and Hopper represents the GPU. In other words, the DGX GH200 supercomputer is built on the Grace Hopper Superchip, which combines the Grace CPU and the Hopper GPU (H 100).

GH200 Grace Hopper Superchip architecture.

Source: NVIDIA Grace Hopper Superchip Architecture Whitepaper.

The above illustration represents the Grace Hopper Superchip architecture. NVIDIA's technology of connecting the CPU and GPU as such is called NVLink. Generally, CPUs have a large memory capacity but small bandwidth (speed), and GPUs have a small memory capacity but large bandwidth. By combining the two, an optimized chip for AI is developed.

Huang emphasized in his keynote speech that the Grace Hopper Superchip is the world's largest single GPU. The Grace Hopper Superchip is currently in production, with a sample version expected to come out around the end of the year and production to be completed by the end of the second quarter next year. The price is yet to be announced.

"The world's fastest memory now connected to Grace Hopper, we're calling it GH200. The chips are in production, we'll sample it at the end of the year or so, and be in production by the end of the second quarter. This processor is designed for scale-out of the world's data centers," Jensen Huang, NVIDIA Keynote at SIGGRAPH 2023.

NVIDIA CEO Jensen Huang presenting in front of a 1:1 size image of DGX GH200.

Source: NVIDIA Keynote at SIGGRAPH 2023. YouTube.

GPU Hours, MFU, and Quantization

Source: NVIDIA AI Workbench | Fine Tuning Generative AI. YouTube.

NVIDIA also announced that it will soon launch the 'NVIDIA AI Workbench.' AI Workbench is a single platform that can manage data, models, resources, and computing requirements.

NVIDIA emphasizes that it is 'accelerating the adoption of customized generative AI across global enterprises through the AI Workbench.' This means that everything from selecting foundation models to building the project environment and fine-tuning can all be done within AI Workbench.

Moreover, AI Workbench integrates with various platforms and developer tools such as GitHub, Hugging Face, JupyterLab, and VS Code.

AI developers select models, create projects, and fine-tune them.

Source: NVIDIA Tech Blog.

In the publicly released demo video, you can see the fine-tuning of the Llama 2 70B model and the interface carried out through AI Workbench.

In this keynote speech, which lasted over 80 minutes, Jensen Huang introduced not only the 'GH200' and 'AI Workbench' but also the metaverse development platform NVIDIA Omniverse. He showcased the growth and depth of NVIDIA's AI landscape, from infrastructure to development platforms to user tools.