tech

In the dual context of the global fervor for large models and the slowdown of Moore's Law (which can be seen as a super scissors difference between demand and technology curves), TSMC recently announced its breakthrough progress in wafer-level computing (integration) technology and mass production plans, that is, System on Wafer (SoW). It is currently expected that SoW will greatly change the development process of computing systems, providing greater computational support and expansion capabilities for large models and future supercomputing. The mass production announcement of SoW also means that the industry is already in full swing preparing for a new computing paradigm that transcends the traditional GPGPU architecture.

So far, only Cerebras and Tesla have developed wafer-level chips using SoW technology. This is also the current tough challenge faced by wafer-level design and 3D IC: compared with the excellent performance and energy efficiency of wafer chips, the complexity of design and production is obviously higher, especially the need to solve the yield issues facing thermal and stress balance. This also means that the traditional EDA "chip design -> packaging design -> packaging" process is invalid in the field of 3D IC design.

Advertisement

After the breakthrough progress in the development of wafer-level system integration processes, TSMC is confident that the usage rate of wafer-level design will not only increase significantly but also the major trends such as artificial intelligence and HPC will require more complex solutions: vertically stacked wafer system design. Given TSMC's ability to know the technical direction of the world's most advanced Design House, we believe that vertically stacked wafer-level systems will develop rapidly.

01

System on Wafer and 3D IC

In the system on wafer, the wafer and its associated components (such as power modules and cooling modules) form a complete system, not just a single chip.

The system on wafer can bring many benefits to designers, including:

Improving computing power (increasing the number of transistors on a single chip)

Reducing data center space occupationImproving Performance per Watt (Energy Efficiency)

Essentially, the system on a wafer metaphorically expands the chip area to the entire wafer (increasing the number of transistors and computing power), and reduces the distance between chips (reducing interconnect power consumption, and reducing the space occupied by interconnect wiring).

TSMC's wafer-level system is abbreviated as SoW, with the goal of providing disruptive performance improvements for artificial intelligence applications such as data centers and large models. (Note that SoW is also a registered trademark of TSMC.)

TSMC's current SoW solutions include InFO-SoW and CoW-SoW (note that the official term is not CoWoS-SoW). It is said that InFO-SoW is already in production, with customers probably including Tesla.

In the wafer-level system, the logic wafer plane is the foundation of the entire system. The wafer design needs to fully consider its structural hierarchy, microarchitecture, NoC, storage hierarchy, and compilation framework. The wafer plane is interconnected through SoW integration technology. In addition to the wafer plane, the wafer-level system also includes stacked storage chips, heat dissipators, and power supply units.

When it comes to SoW, it is inevitable to mention 3D IC. 3D IC, or three-dimensional chip, is a concept that contrasts with the traditional planar IC process. For example, the CMOS 2.0 proposed by the Belgian Microelectronics Research Center (IMEC) divides cache and memory into independent units. Then, they are stacked in a 3D arrangement on top of the logic chip functions. Unlike AMD's 3D V-Cache (which stacks L3 memory on top or below the processor), IMEC stacks L1, L2, and L3 caches vertically.

But no matter how the stacking is done, the essence is still to bring logic and storage closer together, in order to achieve near-memory computing in the integration of storage and computing.CoW-SoW

CoW stands for Chip on Wafer, which is part of TSMC's SoIC (System on Integrated Chip) process series.

Currently, many solutions use InFO-SoW or other wafer-level integration methods. Processor designers need to integrate a large amount of SRAM on the logic Die to enhance performance. However, this may not be enough for the next generation of AI computing. InFO-SoW also requires the entire wafer to be processed with a single manufacturing technology, making it difficult to integrate different process Dies, especially those with Memory (including SRAM and DRAM) process Dies. This deficiency has given rise to the emergence of CoW-SoW.

According to TSMC's roadmap, CoW-SoW is expected to enter mass production in 2027. This technology allows the stacking of Memory or logic Dies on top of SoW using the CoW (Chip on Wafer) method. Based on the information currently circulating in the industry, the earliest mass production goal is to integrate HBM4 with the wafer-level processor through stacking. Compared to the existing 2.5D packaging method, this approach can obviously integrate much larger capacity HBM on the logic chip. (The reason AMD chose HBM was because the capacity of DDR could not be increased around the CPU)

This technology has similarities with Info-SoW, but it is clear that using the CoW method can achieve better interconnect bandwidth, and the cost will also be significantly higher than Info-SoW.HBM4 and HBM4E

HBM4 and HBM4E are clearly associated with CoW-SoW technology.

It is reported that SK Hynix plans to launch the first HBM4 products with 12-layer DRAM stacking in the second half of 2025, and to introduce their HBM4E memory as early as 2026 (the bandwidth will be 1.4 times that of HBM4). In Hynix's HBM4 and HBM4E schemes, HBM is directly stacked on or under the logic Die, and the interconnect distance is reduced to the chip thickness, eliminating the need for planar connections at the chip width distance.

Micron is expected to bring HBM4 with 12-Hi and 16-Hi stacks of 36GB to 48GB to the market between 2026 and 2027. In 2028, they will introduce HBM4E, with capacities increased to 48GB to 64GB per stack.

Info-SoW

Info stands for Integrated Fan-Out. Compared with CoWoS, Info has the advantage of low cost.

The Tesla Dojo wafer-level processor is the first solution mass-produced based on TSMC InFO-SoW technology, offering low latency, high bandwidth, high performance, and high bandwidth density compared to system-in-package (SiP), with lower power supply resistance. Each training module of Dojo is arranged in a 5x5 D1 chip array, interconnected in a two-dimensional Mesh structure. The on-chip cross-core SRAM reaches an astonishing 11GB, and of course, the power consumption also reaches an astonishing indicator of 15kW.Of course, the Info-SoW structure is inherently difficult to directly integrate with ultra-high-capacity HBM. I recall criticism that Groq's 2019 plan did not include integrated HBM, but the real reason lies in the fact that: to achieve the integration of peer-demand HBM, it can only be stacked above or below the HBM logic Die, and at that time, there was no mature process solution.

05

Compact Universal Photonic Engine

By the way, let's talk about the optical interconnection technology closely related to CoW-SoW. The SoIC-X chip stacking technology stacks electronic chips on photonic chips (EIC-on-PIC), interconnecting between chips with lower impedance. By introducing silicon photonic connections through SoIC-X technology, better interconnection energy efficiency can be achieved.

COUPE can provide up to 12.8 Tbps of optical connections. The first generation of COUPE is expected to be experimentally launched in pluggable OSFP devices in 2025, operating at a speed of 1.6 Tbps, which is twice the speed of the current copper cable Ethernet (800 Gbps). The second generation is expected to be launched in 2026 as a "co-packaged optical device" (CPO) integrated into the CoWoS package, directly introducing silicon photonic interconnection into the package at a speed of 6.4 Tbps. The third generation of COUPE implementation will be directly integrated into the interposer layer, directly providing chip-level optical interconnection, with a speed of 12.8 Tbps.

06

Impact on the Semiconductor IndustryTSMC's SoW technology is expected to drive the rapid development of wafer-level computing technology, which will have a profound impact on the semiconductor industry. Traditional computing cluster interconnections (through fiber optics and switches) will quickly integrate into wafer computing models, making desktop supercomputers and desktop clusters possible.

In addition to artificial intelligence, SoW technology also has a very broad application space. It is expected that data centers and high-throughput network services will adopt SoW technology in large quantities to reduce IDC costs. Combined with new silicon photonic interconnection integration technology, Moore's Law will be further extended, and new computing architectures will gradually enter mainstream applications.