In particular, for high-performance computing chips, the combination of sophisticated packaging and wafer production techniques can meet the demands of computing power, latency, and increased bandwidth. The maximum memory bandwidth is needed by the AI accelerators used to train AI models in data centers.
Modern data centre architectures employ a number of strategies to get around memory constraints, as opposed to prior systems, which were memory-constrained.
High bandwidth memory (HBM) is one of the methods frequently used to increase bandwidth and memory capacity. Even though AI has historically been a specialized technology, it has been a significant driver of HBM on GPUs.
Using silicon vias (TSVs) and micro-bumps, HBM technology connects DRAM chips stacked vertically on top of one another. When the DRAMs are stacked, HBM may provide terabytes per second, which is a considerable amount of processing power needed for AI training and high-performance applications. Although providing extremely high bandwidth for the off-chip memory required by data centre AI accelerators, HBM has drawbacks such as high cost and heat restrictions.
When the DRAM stack and the SoC are combined into a single package substrate, HBM is becoming more widely used thanks to technological advancements, improved performance, and decreased power consumption with each iteration. This is what is causing its adoption in AI applications.
The primary motivator for packing sophisticated logic with HBMs is CoWoS.
CoWoS (chip-on-wafer-on-substrate), a 2.5D IC integration technology that merges logic processing and HBM chips by mounting them on a silicon interposer and then placing them directly on a package substrate, is one of the integration techniques that is frequently utilized. TSV/RDL interposers are used in semiconductor IC applications requiring high I/O, high performance, and high density.
To create a chip-on-wafer (CoW) with fine pitch and high-density interconnect routing between the devices, the logic and HBMs are first bonded side by side on the silicon interposer. Each HBM has a logic foundation with TSVs running directly through it, along with DRAMs with microbubbles. The characteristic that makes it possible for 2.5D and 3D advanced packaging is called Through-Silicon Via (TSV). Smaller container sizes and denser interconnects are made possible by TSVs, which are electrical connection channels that are short vertical columns that run through the silicon wafer or die. TSVs also allow many chips to be stacked together and are used in products like HBM. The TSV interposer’s bigger bump assembly on a package substrate is now complete.
Benefiting from AI computing’s demand
CoWoS is also a problem for businesses.
Growing silicon interposer dimensions to enable CPUs and HBM stacks have been the main emphasis of CoWoS technology development throughout the years. The most popular assembly method for silicon-to-silicon flip chip bonding today is CoW with C2 bump via TCB method. With time, the CoW flip chip using a bumpless approach known as the hybrid bonding method, which is currently under R&D, will gain popularity.
The primary constraint for AI shipments recently has been CoWoS. The emphasis has been on increasing CoWoS capacity to meet the growing demand, which is being pushed by NVIDIA, AMD, Broadcom, and Amazon.
overcoming obstacles to CoWoS expansion
To overcome these obstacles, the current market participants must expand their capacities and work in partnership with OSATs, interposer producers, and suppliers of packaging equipment.
How ought pure-play foundries to proceed?
Because of TSMC’s capacity limitations with regard to CoWoS, NVIDIA and other clients may experience certain bottlenecks. The equipment suppliers’ and the manufacturers’ interposers’ bottleneck is one of the causes in this case. In order to stabilize volume in the near- to mid-term and meet the rising demand for advanced packaging in AI and HPC applications, TSMC should seek to cooperate and possibly outsource to its partners. Therefore, in the long run, TSMC should cooperate with machinery producers to optimize CoWoS processing methods like TCB (Thermal Compression Bonding) and hybrid bonding.
What should suppliers of equipment do?
In order to scale the CoWoS packaging, equipment players should work closely with foundries and jointly create optimised methods. For instance, Applied Materials is working hard to create novel hybrid bonding and TSV technologies that will promote heterogeneous chip integration and aid chipmakers in integrating chiplets into sophisticated 2.5D and 3D packaging. Similar partnerships could be anticipated from other players including KLA Tencor, Lam Research, ASMPT, and BESI, to promote CoWoS packaging or other rival techniques like hybrid bonding to enable advancements in fine pitch, I/O density, and power consumption. A comprehensive set of tools and technologies for creating and prototyping different package designs will be made available to semiconductor and systems businesses through research collaborations, enabling further improvements in power, performance, area, cost, and time-to-market.
What ought IDMs to do?
In order to build a strong enough client base for premium solutions, Samsung and other IDMs are aggressively striving to capture a portion of the advanced packaging market and improve their design and production capabilities. For instance, the HBM-PIM technology from Samsung Electronics combines processing chips and cutting-edge memory chips like HBM with its unique 2.5D or 3D packaging technology. In order to potentially replace its current HBM2 product, Samsung has created HBM-PIM, in which the bottom four of eight memory chips are swapped out for chips with DRAM and computation cores.
The neural network is effectively sped up and less power is used to transfer data by performing. Some of the computation in the DRAM rather than in the main memory. Samsung, the only firm with a chip business portfolio spanning memory chips to contract-based processing chip fabrication and packaging. Will be able to harness its sophisticated packaging solutions and technologies as this technology advances and compete with TSMC.
IDMs with established node capabilities can also take advantage of advanced packaging. To improve the performance of their current product portfolios and gain a sizable market share.
Benefiting from AI computing’s demand
As a result of developments in artificial intelligence, Chin has decided to adjust the price objective. Since they think that memory, a vital component of the semiconductor sector, will see significant growth (AI). The analyst highlighted memory’s benefits from the development of AI, calling it “the unsung beneficiary of growth in AI/accelerated computing.”
DRAM (dynamic random access memory) is anticipated to surpass other wafer fab equipment manufacturing markets in 2024, according to Chin. They predict that memory will increase wafer fab equipment by between $4 billion and $5 billion. Chin also emphasized Lam Research’s excellent tactical setup in spite of the industry-wide AI tailwind.
The majority of top-rated analysts currently have a favorable outlook on Lam Research. With 75% of them giving the company a Strong Buy or Buy rating. No analysts have recommended or strongly recommended selling the stock, and 25% of people consider it a hold.
Lam Research has shown to be a competitive force in the market in terms of performance. The stock price has increased by 8.5% since June 25, 2023, the date of its most recent quarterly report. The stock has significantly increased by 38.1% year over year. Throughout this time, Lam Research has outpaced the S&P 500, which has grown by 5.6%.
Benefiting from AI computing’s demand
The Wall Street community holds Brian Chin, the Stifel Nicolaus analyst who provided the rating, in high regard. Chin is ranked in the top 12% of the 4,308 Wall Street analysts by WallStreetZen. They have a 65% win rate and an average return of 19.8%. Chin is an expert in real estate and technology.
A leading manufacturer, marketer, refurbisher, and service provider of semiconductor processing equipment, is based in Fremont, California. The creation of integrated circuits uses the company’s goods. Systems for film deposition, electrochemical deposition, and thermal processing. Plasma-enhanced chemical vapour deposition, atomic layer deposition, etching, and metrology are all available from Lam Research. The corporation operates in the United States, China, Europe, Japan, Korea, Southeast Asia, and Taiwan. It provides services to the semiconductor industry on a global scale.
