Recently, a new chip connection system that is expected to break through the limitations of the "memory wall" and accelerate the development of AI has attracted attention. Led by the University of Michigan (U-M) and funded by a $2 million grant from the National Science Foundation's Semiconductor Futures Program, the technology brings together researchers from the University of Washington, the University of Pennsylvania, Lawrence Berkeley National Laboratory, and industry giants such as Google, Hewlett-Packard Enterprise, Microsoft, and Nvidia to support and guide the project.
At present, AI is developing rapidly, but the speed of data transfer between memory and processor has become a key bottleneck that restricts the scale of AI models and the improvement of computing speed. Data shows that in the last 20 years, data processing has increased by 60,000 times, while data transfer speed has only increased by 30 times. Since 1998, AI models have grown 400 times every two years, and slow data transfer rates have severely constrained AI performance improvements.
Di Liang, a professor of electrical and computer engineering at the University of Michigan and principal researcher of the project, pointed out that the technology they proposed uses optical connections instead of traditional electrical connections, and is expected to transmit tens of terabits of data per second, more than 100 times faster than the existing state-of-the-art electrical connections, allowing high-performance computing to keep up with the demand for massive data processing by AI models.
Currently, data is mainly transmitted between memory and processor chips via metal connections soldered to interposers. However, there are many drawbacks to this hard connection, such as high-speed electrical signals are prone to heat loss and electromagnetic interference, which limits the data transmission bandwidth and signal integrity. As AI models continue to expand, with more than 900,000 cores on a single supercomputer chip, and counting, traditional connectivity methods are becoming increasingly difficult to keep up.
Figure: New Chip Connectivity System: Breaking through the "memory wall" and opening a new era of AI
Li Mo, a professor of electrical and computer engineering at the University of Washington and co-principal investigator, said that optical connectivity will become the key to solving communication problems due to its advantages of long propagation distance and low energy loss. In the new interposer design, light pulses are propagated between chips through an optical waveguide, and a receiver on the chip converts the optical signal into an electrical signal for computer processing.
More innovatively, the special phase change material in the interposer allows the optical waveguide network to be flexibly reconstructed. When irradiated by a laser or voltage, the refractive index of the material changes, and the direction of light propagation in the waveguide changes, just like the opening and closing of a road can be flexibly regulated. Feng Liang, a professor of materials science and electrical and systems engineering at the University of Pennsylvania, believes that based on this technology, companies can rewrite the connections between different batches of chips and servers without changing the layout of other components when selling chips.
In addition, the research team plans to develop flow control software that monitors interposer communication needs in real time and establishes an ideal connection on the fly through voltage switching to accommodate the operation and training needs of different AI models. According to Retuparna Das, associate professor of computer science and engineering, this flexible connection allows the network to be reconfigured according to actual needs.
Not only does the project have the potential to break through the "memory wall" in terms of technology, but it will also provide University of Michigan students with the opportunity to work deeply with the industry to help them gain hands-on experience and meet the challenges posed by rapidly evolving technologies. Professor Leung said that industry collaboration can expose students to cutting-edge issues that cannot be covered in textbooks, and effectively enhance their professional skills.
Industry insiders believe that once this new chip connection system is successfully applied, it will inject strong impetus into the development of AI, significantly improve computing efficiency, and is expected to promote AI technology to new heights and bring more possibilities for the intelligent era.
