CPO (co-packaged optoelectronics) technology has been around for some time but is still in its developmental stages. Andreas Matiss, senior manager of optical components and integration at Corning Optical Communications, explained how glass plays a key role in placing silicon-based electro-optical converters as close as possible to silicon processors.
Data center networks are rapidly evolving, and this momentum has accelerated with the rise of AI and the large-scale deployment of AI clusters. Recent progress in this area has been significant, particularly with the deployment of NVIDIA's DGX SuperPOD architecture and Google's TPU clusters. This shift is driven by the demand for high-performance computing to support AI training and inference tasks. NVIDIA alone is expected to ship millions of AI-optimized GPU units annually within the next five years, reaching a significant scale by 2028.
The number of transceiver units required to build these networks will reach tens of millions annually, and these devices will need to operate at maximum speeds of 1.6Tbps and 3.2Tbps. Industry analysts predict that each accelerator (GPU) will be equipped with more than 10 transceivers in the future, which means that the demand for fiber optic connections will increase approximately 10 times compared to current deployment levels.
In a typical data center, a standard pluggable Ethernet transceiver consumes approximately 20 watts of power. Next-generation transceivers are expected to consume nearly double that power. Based on current shipments, it's estimated that approximately 200 megawatts (MW) of power will be deployed to power transceivers in 2024. Based on the trajectory of transceiver development and an expected tenfold increase in demand for optical connectivity, transceiver power deployment is projected to rise to 2 gigawatts (GW) per year, equivalent to the power generated by a large nuclear power plant. This does not include the power required to power the host-side electronics and electrical retimers used to transmit data from the integrated circuits to the transceivers on the device front end.
For example, for an AI data center equipped with one million GPUs, introducing CPO technology could save the data center approximately 150 megawatts of power generation capacity. In addition to reducing the investment required to build the corresponding power generation facilities, this technology also significantly reduces operating costs—depending on regional energy price differences, annual electricity savings could easily exceed €100 million. In China, with the advancement of the "East-West Computing" initiative, the demand for high-bandwidth, low-power optical interconnects is surging in supercomputing centers (such as the Wuxi Sunway TaihuLight) and intelligent computing centers (such as the AI computing clusters in Beijing and Shenzhen). CPO technology is expected to be key to reducing energy consumption and increasing efficiency for domestically produced GPUs. Faced with this unsustainable energy consumption trend, innovation is crucial.
Introduction of CPO technology
CPO is the technology most likely to overcome this energy consumption bottleneck in the short term. This technology shifts the electro-optical conversion module from the transceiver on the front panel to the device interior, ideally integrating it directly onto the CPU or GPU package substrate. This minimizes power loss in the copper channel, resulting in a more energy-efficient link. Compared to pluggable transceivers, power consumption can be reduced by over 50%, and in some cases, up to 75%. This energy-saving advantage is achieved not only by reducing the use of high-loss copper channels but also by simplifying or even eliminating the digital signal processor (DSP) required to compensate for electrical signal transmission losses.
In summary, CPO technology offers high-speed, low-power, and low-latency optical connectivity. These characteristics are key to advanced AI networks.
Another energy-saving alternative worth considering is the Linear Pluggable Optical Module (LPO). By eliminating the DSP chip, it reduces power consumption and latency while maintaining the form factor and ecosystem of a front-panel pluggable transceiver. While CPO offers better signal integrity and lower latency, LPO is more cost-effective, particularly for short-reach applications. LPO's cost-effectiveness and low power consumption, combined with its rapid time-to-market, may delay the widespread adoption of CPO technology.
However, as link speeds increase to 200G and beyond, LPO consumes more power than CPO and becomes significantly more difficult to manage in order to ensure high signal quality. As technology continues to advance, CPO is expected to become the preferred solution in the future.
Glass Empowers CPO Technology
Glass is expected to play a key role in the next generation of CPO technology. To bring electro-optical converters (primarily silicon photonics chips) as close as possible to the actual silicon processors (CPUs and GPUs), a new packaging technology is required that not only supports larger substrate sizes but also enables optical connectivity to the silicon photonics chips.
Semiconductor packaging has traditionally relied primarily on organic substrates. These materials have a higher coefficient of thermal expansion than silicon, limiting the maximum size of semiconductor packages. As the industry continues to push for larger package substrates on existing organic technology platforms, reliability issues (such as solder joint integrity issues and increased risk of delamination) and manufacturing challenges (such as high-quality fine-pitch interconnect structures and high-density wiring) have become increasingly prominent, leading to increasing packaging and testing costs. However, through optimized design, glass can achieve a thermal expansion coefficient that more closely matches that of silicon chips, surpassing traditional organic substrates. This specially processed glass substrate exhibits exceptional thermal stability, reducing mechanical stress and damage during temperature fluctuations. Its superior mechanical strength and flatness provide a solid foundation for chip packaging reliability. Furthermore, glass substrates support higher interconnect density and finer pitches, improving electrical performance and reducing parasitic effects. These properties make glass a highly reliable and precise choice for advanced semiconductor packaging. Consequently, the semiconductor packaging industry is actively developing advanced glass substrate technology as the next-generation substrate technology.
Glass Waveguide Substrates
In addition to its excellent thermal and mechanical properties, glass can also be manipulated to function as an optical waveguide. Waveguides in glass are typically created through a process called ion exchange: ions in the glass are replaced with different ions from a salt solution, thereby changing the glass's refractive index. By confining light to regions with a higher refractive index, these modified regions can guide the light. This technique enables precise tuning of waveguide properties, making it suitable for a variety of optical applications. Consequently, in optical waveguides with fiber-like structures, light can propagate along integrated glass waveguides and be efficiently coupled into optical fibers or silicon photonic chips. This makes glass an attractive material choice for advanced CPO applications.
Integrating electrical and optical interconnects on the same substrate also helps address the interconnect density challenges companies face when building large AI clusters. Currently, the number of optical channels is limited by the geometry of optical fibers—the diameter of a typical optical fiber cladding is 127 microns, about the thickness of a human hair. Glass waveguides, however, enable denser arrangements, significantly increasing input/output (I/O) density compared to direct fiber-to-chip connections.
The integration of electrical and optical interconnects not only addresses density issues but also improves the overall performance and scalability of AI clusters. The compact nature of glass waveguides allows for more optical channels to be accommodated within the same physical space, thereby increasing the system's data transmission capacity and efficiency. This advancement is crucial for driving the development of next-generation AI infrastructure—in scenarios where AI systems must process massive amounts of data, high-density interconnect technology is key to efficient management.
By integrating glass waveguides, a complete optical system can be built on the same substrate, enabling photonic integrated circuits to communicate directly through optical waveguides. This process eliminates the need for optical fiber interconnects and significantly improves the bandwidth and coverage of inter-chip communication. In high-density systems with numerous interconnected components, the use of glass waveguides can achieve lower signal loss, higher bandwidth density, and greater durability compared to discrete optical fibers. These advantages make glass waveguides an ideal choice for high-performance optical interconnect systems.
Applying CPO technology to next-generation data centers and AI supercomputer networks can increase chip-escape bandwidth, opening up new possibilities for high-speed, high-radix switches of 102T and above. Network architects now have a unique opportunity to reimagine and redesign network architectures. Thanks to increased bandwidth and simplified network architectures, they will achieve superior network performance, driving operational efficiency improvements and process optimization.
Conclusion
CPO technology has the potential to revolutionize AI interconnect architecture on multiple levels. It can significantly reduce energy consumption and improve sustainability, making AI systems more environmentally friendly and cost-effective. Furthermore, CPO improves the efficiency and scalability of AI systems, enabling them to easily handle larger and more complex tasks. By addressing density issues, CPO can increase data transmission rates, ensuring faster and more reliable communication between AI components. This will also help reduce bottlenecks in future AI systems, ensuring smoother and more efficient system operation.
Future AI interconnects are expected to introduce direct optical links, eliminating the need for computing switches. This innovation will broaden the bandwidth for AI tasks and improve the speed and efficiency of processing large datasets. Glass, with its superior data transmission capabilities and scalability, is an ideal material for enabling these technological advancements. Glass-based optical links will become a critical enabler for next-generation AI systems, forming an indispensable infrastructure for high-performance computing and advanced AI applications.
NEW LIGHT OPTICS TECHNOLOGY LIMITED will strive to seize every opportunity and contribute.
CPO (co-packaged optoelectronics) technology has been around for some time but is still in its developmental stages. Andreas Matiss, senior manager of optical components and integration at Corning Optical Communications, explained how glass plays a key role in placing silicon-based electro-optical converters as close as possible to silicon processors.
Data center networks are rapidly evolving, and this momentum has accelerated with the rise of AI and the large-scale deployment of AI clusters. Recent progress in this area has been significant, particularly with the deployment of NVIDIA's DGX SuperPOD architecture and Google's TPU clusters. This shift is driven by the demand for high-performance computing to support AI training and inference tasks. NVIDIA alone is expected to ship millions of AI-optimized GPU units annually within the next five years, reaching a significant scale by 2028.
The number of transceiver units required to build these networks will reach tens of millions annually, and these devices will need to operate at maximum speeds of 1.6Tbps and 3.2Tbps. Industry analysts predict that each accelerator (GPU) will be equipped with more than 10 transceivers in the future, which means that the demand for fiber optic connections will increase approximately 10 times compared to current deployment levels.
In a typical data center, a standard pluggable Ethernet transceiver consumes approximately 20 watts of power. Next-generation transceivers are expected to consume nearly double that power. Based on current shipments, it's estimated that approximately 200 megawatts (MW) of power will be deployed to power transceivers in 2024. Based on the trajectory of transceiver development and an expected tenfold increase in demand for optical connectivity, transceiver power deployment is projected to rise to 2 gigawatts (GW) per year, equivalent to the power generated by a large nuclear power plant. This does not include the power required to power the host-side electronics and electrical retimers used to transmit data from the integrated circuits to the transceivers on the device front end.
For example, for an AI data center equipped with one million GPUs, introducing CPO technology could save the data center approximately 150 megawatts of power generation capacity. In addition to reducing the investment required to build the corresponding power generation facilities, this technology also significantly reduces operating costs—depending on regional energy price differences, annual electricity savings could easily exceed €100 million. In China, with the advancement of the "East-West Computing" initiative, the demand for high-bandwidth, low-power optical interconnects is surging in supercomputing centers (such as the Wuxi Sunway TaihuLight) and intelligent computing centers (such as the AI computing clusters in Beijing and Shenzhen). CPO technology is expected to be key to reducing energy consumption and increasing efficiency for domestically produced GPUs. Faced with this unsustainable energy consumption trend, innovation is crucial.
Introduction of CPO technology
CPO is the technology most likely to overcome this energy consumption bottleneck in the short term. This technology shifts the electro-optical conversion module from the transceiver on the front panel to the device interior, ideally integrating it directly onto the CPU or GPU package substrate. This minimizes power loss in the copper channel, resulting in a more energy-efficient link. Compared to pluggable transceivers, power consumption can be reduced by over 50%, and in some cases, up to 75%. This energy-saving advantage is achieved not only by reducing the use of high-loss copper channels but also by simplifying or even eliminating the digital signal processor (DSP) required to compensate for electrical signal transmission losses.
In summary, CPO technology offers high-speed, low-power, and low-latency optical connectivity. These characteristics are key to advanced AI networks.
Another energy-saving alternative worth considering is the Linear Pluggable Optical Module (LPO). By eliminating the DSP chip, it reduces power consumption and latency while maintaining the form factor and ecosystem of a front-panel pluggable transceiver. While CPO offers better signal integrity and lower latency, LPO is more cost-effective, particularly for short-reach applications. LPO's cost-effectiveness and low power consumption, combined with its rapid time-to-market, may delay the widespread adoption of CPO technology.
However, as link speeds increase to 200G and beyond, LPO consumes more power than CPO and becomes significantly more difficult to manage in order to ensure high signal quality. As technology continues to advance, CPO is expected to become the preferred solution in the future.
Glass Empowers CPO Technology
Glass is expected to play a key role in the next generation of CPO technology. To bring electro-optical converters (primarily silicon photonics chips) as close as possible to the actual silicon processors (CPUs and GPUs), a new packaging technology is required that not only supports larger substrate sizes but also enables optical connectivity to the silicon photonics chips.
Semiconductor packaging has traditionally relied primarily on organic substrates. These materials have a higher coefficient of thermal expansion than silicon, limiting the maximum size of semiconductor packages. As the industry continues to push for larger package substrates on existing organic technology platforms, reliability issues (such as solder joint integrity issues and increased risk of delamination) and manufacturing challenges (such as high-quality fine-pitch interconnect structures and high-density wiring) have become increasingly prominent, leading to increasing packaging and testing costs. However, through optimized design, glass can achieve a thermal expansion coefficient that more closely matches that of silicon chips, surpassing traditional organic substrates. This specially processed glass substrate exhibits exceptional thermal stability, reducing mechanical stress and damage during temperature fluctuations. Its superior mechanical strength and flatness provide a solid foundation for chip packaging reliability. Furthermore, glass substrates support higher interconnect density and finer pitches, improving electrical performance and reducing parasitic effects. These properties make glass a highly reliable and precise choice for advanced semiconductor packaging. Consequently, the semiconductor packaging industry is actively developing advanced glass substrate technology as the next-generation substrate technology.
Glass Waveguide Substrates
In addition to its excellent thermal and mechanical properties, glass can also be manipulated to function as an optical waveguide. Waveguides in glass are typically created through a process called ion exchange: ions in the glass are replaced with different ions from a salt solution, thereby changing the glass's refractive index. By confining light to regions with a higher refractive index, these modified regions can guide the light. This technique enables precise tuning of waveguide properties, making it suitable for a variety of optical applications. Consequently, in optical waveguides with fiber-like structures, light can propagate along integrated glass waveguides and be efficiently coupled into optical fibers or silicon photonic chips. This makes glass an attractive material choice for advanced CPO applications.
Integrating electrical and optical interconnects on the same substrate also helps address the interconnect density challenges companies face when building large AI clusters. Currently, the number of optical channels is limited by the geometry of optical fibers—the diameter of a typical optical fiber cladding is 127 microns, about the thickness of a human hair. Glass waveguides, however, enable denser arrangements, significantly increasing input/output (I/O) density compared to direct fiber-to-chip connections.
The integration of electrical and optical interconnects not only addresses density issues but also improves the overall performance and scalability of AI clusters. The compact nature of glass waveguides allows for more optical channels to be accommodated within the same physical space, thereby increasing the system's data transmission capacity and efficiency. This advancement is crucial for driving the development of next-generation AI infrastructure—in scenarios where AI systems must process massive amounts of data, high-density interconnect technology is key to efficient management.
By integrating glass waveguides, a complete optical system can be built on the same substrate, enabling photonic integrated circuits to communicate directly through optical waveguides. This process eliminates the need for optical fiber interconnects and significantly improves the bandwidth and coverage of inter-chip communication. In high-density systems with numerous interconnected components, the use of glass waveguides can achieve lower signal loss, higher bandwidth density, and greater durability compared to discrete optical fibers. These advantages make glass waveguides an ideal choice for high-performance optical interconnect systems.
Applying CPO technology to next-generation data centers and AI supercomputer networks can increase chip-escape bandwidth, opening up new possibilities for high-speed, high-radix switches of 102T and above. Network architects now have a unique opportunity to reimagine and redesign network architectures. Thanks to increased bandwidth and simplified network architectures, they will achieve superior network performance, driving operational efficiency improvements and process optimization.
Conclusion
CPO technology has the potential to revolutionize AI interconnect architecture on multiple levels. It can significantly reduce energy consumption and improve sustainability, making AI systems more environmentally friendly and cost-effective. Furthermore, CPO improves the efficiency and scalability of AI systems, enabling them to easily handle larger and more complex tasks. By addressing density issues, CPO can increase data transmission rates, ensuring faster and more reliable communication between AI components. This will also help reduce bottlenecks in future AI systems, ensuring smoother and more efficient system operation.
Future AI interconnects are expected to introduce direct optical links, eliminating the need for computing switches. This innovation will broaden the bandwidth for AI tasks and improve the speed and efficiency of processing large datasets. Glass, with its superior data transmission capabilities and scalability, is an ideal material for enabling these technological advancements. Glass-based optical links will become a critical enabler for next-generation AI systems, forming an indispensable infrastructure for high-performance computing and advanced AI applications.
NEW LIGHT OPTICS TECHNOLOGY LIMITED will strive to seize every opportunity and contribute.
주소
방 C, 9층, 리 빌딩, 72-76 Wing Lok Street, Sheung Wan, 홍콩
전화
00-86-13534063703
