My CartNVIDIA, a leading innovator in the graphics processing unit (GPU) market, has once again demonstrated its prowess with a series of groundbreaking announcements and technological advancements. From introducing a revolutionary slot design in its next-generation AI GPUs to collaborating with MediaTek on AI PC chips, NVIDIA is setting the stage for a new era in computing and artificial intelligence. This blog will delve into NVIDIA's latest chip roadmap, its impact on various industries, and the future implications of these innovations.
NVIDIA's Revolutionary Slot Design for AI GPUs
On October 11, 2024, NVIDIA announced a significant innovation in the GPU domain: the introduction of a new slot design in its next-generation AI GPUs. According to the latest report by Trendforce, NVIDIA plans to adopt this revolutionary design in its GB300 series, following the GB200 shipments in the fourth quarter of 2024. This new design replaces the traditional on-board memory (OAM) solution with an independent socket type, a transition more commonly seen in the CPU server market.
The slot design not only changes how users interact with GPUs but also simplifies the after-sales maintenance and upgrade process, providing a more seamless user experience. With this technology, users can effortlessly replace and upgrade their GPUs without needing to replace the entire server board or undergo complex repairs. This design presents a significant opportunity for companies like Foxconn and interconnect component suppliers such as LOTES, creating higher flexibility across the supply chain.
The GB300 series, expected to become NVIDIA's flagship product in the second half of 2025, will feature the FP4 architecture, which is highly suitable for inference scenarios, offering superior computational efficiency and lower latency. This advancement is a crucial driver for the proliferation and development of AI applications, particularly in data processing and deep learning, where it exhibits high market competitiveness.
The modular design extends beyond the hardware level, profoundly impacting the user experience. In practical use, users can flexibly adjust device performance based on their needs without worrying about current hardware limitations. This flexibility enables users to handle gaming, video rendering, or complex data analysis with ease, enhancing the overall experience. For developers and professional users who frequently upgrade their systems, this innovation offers an attractive option.
In the current market, NVIDIA's modular design holds a competitive edge. Compared to traditional fixed GPUs, the flexibility and upgrade potential of the slot design will attract more enterprise clients and users with urgent high-performance needs. Although competitors like AMD are also innovating, NVIDIA's modular design is likely to stay ahead, redefining market standards. This change will benefit the entire industry, reducing maintenance costs and enhancing device sustainability.
NVIDIA's GPU Shortage and the AI Boom
NVIDIA's GPUs have become popular as AI chips, particularly since the release of ChatGPT by OpenAI, which sparked a surge in generative AI. However, GPU production faces two bottlenecks: TSMC's CoWoS packaging and high-bandwidth memory (HBM). These bottlenecks have led to a global shortage of GPUs, with the H100 model experiencing particularly high demand and prices soaring to $40,000, sparking what has been termed the "NVIDIA GPU frenzy."
To address this, TSMC doubled its silicon interposer capacity from 15,000 units per month in summer 2023 to over 30,000 units per month by summer 2024. Additionally, Samsung Electronics and Micron Technology, which have received NVIDIA certification, began supplying advanced HBM, previously dominated by SK Hynix. These efforts shortened the delivery time of the highly demanded H100 from 52 weeks to 20 weeks.
According to DIGITIMES Research's "Global Annual Server Shipments, 2023-2024," AI servers are categorized into two types: general AI servers with two or more AI accelerators but no HBM, and high-end AI servers with at least four AI accelerators equipped with HBM. ChatGPT-level generative AI requires high-end AI servers rather than general ones.
The report predicts that while general AI server shipments will increase from 344,000 units in 2022 to 725,000 units in 2024, high-end AI server shipments will grow from 34,000 units in 2022 to 564,000 units in 2024. However, these shipments barely meet the demand from cloud service providers (CSPs) like Google, Amazon, and Microsoft.
ChatGPT's development and operation require 30,000 NVIDIA DGX H100 high-end AI servers, each equipped with eight H100 chips priced at 460,000 per system. Thus, building ChatGPT-level AI requires an investment of $13.8 billion. In 2022, only one ChatGPT-level AI system could be built due to the limited availability of high-end AI servers. By 2024, with an expected shipment of 564,000 high-end AI servers, up to 18-19 ChatGPT-level AI systems could be constructed.
NVIDIA and MediaTek's Collaboration on AI PC Chips
NVIDIA has also ventured into the AI PC chip market, collaborating with MediaTek to release a chip in the second half of next year. Currently in the tape-out stage, this chip will utilize TSMC's 3nm process, adopt an Arm architecture, and potentially integrate NVIDIA's iGPU to enhance performance.
NVIDIA's interest in the custom chip business, evident from its attempted acquisition of Arm, aligns with the surge in demand for its GPUs in the AI market. The collaboration has attracted interest from major OEM manufacturers like Lenovo, Dell, HP, and ASUS.
The AI PC chip industry includes players like Intel and AMD in the X86 camp and Qualcomm in the Arm camp. With Qualcomm's exclusive agreement with Microsoft expiring, both NVIDIA and MediaTek have strong reasons to enter the Arm-based PC chip market. However, Qualcomm's efforts in this segment have yet to yield remarkable results, while Intel's latest Core Ultra 200V platform has shown significant improvements in power consumption and graphics performance, without compatibility issues faced by Arm-based PCs.
NVIDIA's Chip Roadmap and Future Prospects
NVIDIA CEO Huang Renxun announced the company's chip roadmap during his keynote speech titled "Opening the Prelude to the New Industrial Revolution" at Computex 2024. Huang highlighted NVIDIA's technological achievements from GTC 2024 and unveiled the latest chip plan, sharing insights and breakthroughs in chip equipment, digital twin technology, and robotics.
NVIDIA's new architecture, Blackwell, announced just three months prior, has already begun mass production. Huang stated that NVIDIA plans to upgrade its AI accelerator annually, with the Blackwell Ultra chip expected in 2025. In 2026, Blackwell's successor, the Rubin chip, will be launched using HBM4 memory. The following year, Rubin Ultra, a new generation Arm-based Vera CPU, and NVLink 6 Switch (3600GB/s) will be introduced.
Conclusion
NVIDIA's latest innovations in AI GPUs and its collaboration with MediaTek on AI PC chips mark significant strides in the computing and AI landscape. The revolutionary slot design in its next-generation GPUs promises to simplify maintenance and upgrades, enhancing user experience and flexibility. Meanwhile, NVIDIA's entry into the AI PC chip market, coupled with its ambitious chip roadmap, positions it as a key player in shaping the future of technology.
As the demand for AI continues to grow, NVIDIA's advancements will play a crucial role in meeting this demand, driving innovation and efficiency across various industries. With competitors like AMD and Qualcomm also pushing boundaries, the tech landscape is poised for exciting changes, and NVIDIA's contributions will undoubtedly be at the forefront. The future of computing and AI is bright, and NVIDIA is leading the charge.
