Marvell Technology announced that it has incorporated co-packaged photoelectric (CPO) technology into its custom XPU architecture, further enhancing the company's leadership in custom semiconductor technology. Marvell has previously announced its high-bandwidth in-memory computing architecture, and now the company is taking the performance of AI servers one step further with the introduction of CPO technology. This technology enables customers to seamlessly integrate XPU with CPO to scale AI servers, from the current copper interconnect connecting dozens of XPUs in a single rack to the efficient connection of multiple racks and hundreds of XPU through CPO, thereby significantly improving the performance of AI servers.
Marvell's innovative architecture enables cloud hyperscale data center operators to develop custom XPUs that break through bandwidth density constraints and provide longer connection distances between XPU within a single AI server, while optimizing latency and power consumption. The technology is now available to Marvell customers for next-generation custom XPU designs.
Optoelectronic co-packaging: break the shackles of traditional electrical connections
Marvell's custom AI accelerator architecture integrates XPU computing silicon, HBM and other chip components on the same substrate as Marvell's 3D SiPho Engine, eliminating the need for electrical signals to be transmitted over copper wires or printed circuit boards (PCBs) through high-speed SerDes, chip-to-chip interfaces and advanced packaging technologies. Through the integration of optical technology, the connection between the XPU enables higher data transfer speeds while transmitting distances up to 100 times longer than traditional copper cables. This technology makes the extended connection within the AI server more efficient, and the XPU interconnection across racks can ensure efficient data transmission with optimized latency and power consumption.
CPO technology minimizes the length of the electrical path by integrating optical components directly into the package. This tightly coupled design greatly reduces signal loss, enhances the integrity of high-speed signals, and significantly reduces latency. Compared to traditional copper connections, CPOs utilize high-bandwidth silicon optical engines that are able to provide higher data transmission rates and exhibit greater immunity to interference in the face of electromagnetic interference. In addition, CPO integration significantly improves power efficiency, reducing the need for high-power electrical drivers, signal amplifiers, and retimers. By enabling longer transmission distances and higher density XPU interconnects, CPO technology supports the development of next-generation accelerated infrastructure that optimizes computing performance and power consumption for AI servers.
Figure: Marvell introduces a custom XPU architecture and achieves a breakthrough in optoelectronic co-packaging technology
The industry's first 3D silicon photonics engine helps CPO technology
At OFC 2024, Marvell debuted its industry-leading 3D silicon photonics engine, which supports 200Gbps electrical and optical interfaces and is a key building block for integrating CPO technology into the XPU. The 6.4T 3D silicon photonics engine integrates 32 200G electrical and optical interfaces, hundreds of modules, photodetectors, modem drivers, transimpedance amplifiers, microcontrollers, and other passive components to deliver twice the bandwidth and 30% lower power per bit than 100G interface devices. Several customers are evaluating this technology and plan to integrate it into their next-generation solutions.
Marvell's portfolio of optoelectronic technologies
Marvell has more than eight years of experience in silicon optics technology to deliver high-performance, low-power COLORZ optical transceivers for data centers. The technology has been proven in high volumes and is being used by several leading hyperscale cloud computing companies to meet their growing inter-data center bandwidth needs. Marvell's silicon optical devices have accumulated more than 10 billion hours of operation in real-world applications.
Breakthrough optoelectronic technologies are accelerating the evolution of infrastructure
Marvell is committed to revolutionizing interconnect technologies that improve the performance, scalability and economics of accelerated infrastructure. Marvell's interconnect portfolio includes a range of technology solutions including high-performance SerDes, chip-to-chip technology IP, high-efficiency PCIe retimers, breakthrough CXL devices, active cables for short-distance connections and optical cable digital signal processors (DSPs) that enable connectivity from within a rack to interconnects thousands of kilometers across data centers.
Future outlook: Broad prospects for CPO technology
"Custom AI accelerator architectures with integrated CPO technology enable operators of cloud computing hyperscale data centers to develop higher-density, higher-performance AI servers," said Marvell executives. "As AI servers scale to require higher signal speeds and longer connection distances, integrating CPO technology is the next step in improving performance, enabling greater bandwidth and longer distances," said Nick Kucharewski, senior vice president of Marvell's Network Switching business unit. ”
Industry experts expect the market to grow rapidly from less than 50,000 ports today to more than 18 million ports by 2029 as CPO technology becomes more widely available, with the majority of these ports going to be used for XPU connections within servers. "Marvell's deep experience in optical technology and custom XPU is poised to be a key enabler of this advancement, unlocking the potential of CPO for hyperscale cloud computing companies and integrating it into their infrastructure to meet the increasing performance demands of AI applications."
