Semiconductor Technology Superconducting Digital Technology revolutionizes AI and Machine Learning

Related Vendors

imec Interuniversity Micro Electronics Centre

Wuxi InfiMotion Technology Co., Ltd.

Superconducting digital components could advance artificial intelligence (AI) and machine learning (ML) as they promise 100 times higher energy efficiency and 1000 times higher computing density. What is the state of the art and the scaling roadmap?

Experts from the Semiconductor Industry Association have predicted that by the year 2040 nearly 50% of the globally generated electrical energy will be used for data processing [1]. This forecast was made before the explosion of AI-based language models, so the actual proportion is likely to be even higher. The predicted computing power will be far beyond the capabilities of even the most advanced CMOS computing and storage systems.

Take the progress of AI as an example. The total AI computing time, expressed in petaFLOP/s-days, has increased by a factor of 1800 over the past two years [2]. Smaller structure widths and architectural improvements only accounted for a factor of six growth. The lion's share was due to the expansion of data centers and the associated higher energy consumption. From a cost and environmental point of view, this is unsustainable.

Gallery

Gallery with 6 images

At least unless new computing systems are developed that offer significantly higher energy efficiency and computing density. This is exactly what superconducting digital technology brings, as the electrical resistance is almost zero at low temperatures. Initial calculations predict a hundred times higher energy efficiency and a thousand times higher computing density compared to modern CMOS processors [3].

Driving the development of deep learning and quantum computers

The superconducting digital technology is not intended to replace conventional technologies, but to complement CMOS in selected application areas and drive innovation in new areas, such as AI and ML techniques.

Through superconductivity, energy-efficient, compact servers are possible that bridge the gap between edge and cloud. The potential of such server technology is enormous. It opens doors for online training of AI models based on real data that are part of an actively changing environment, such as in autopilots.

Real-time interaction with simultaneous cloud communication will benefit many application areas, e.g. smart grids, smart cities, mobile health systems, connected manufacturing and smart agriculture.

In addition, elements of superconductivity could benefit quantum technology. Scalable measurement systems need to be developed to read out and control the growing number of qubits, perform error corrections, and process data. Here, superconductivity can provide a solution that cannot be achieved with other technologies.

Co-development of systems and technologies

The leap in energy efficiency and computing density will result from the introduction of superconducting materials and new components, thereby changing the technological basis of classical digital technologies. However, the advances achieved through material and device innovations can easily be undone by bottlenecks at the module or system level.

To ensure that these effectively reflect on the entire system, innovations in the whole system structure—from the core components of the devices to the algorithms—are developed together from the beginning. This requires a continuous, iterative loop between top-down system-driven decisions and the maturation of technology and bottom-up integration. Such a systems-level approach takes into account the fact that solutions are determined by application requirements, implementation constraints, and workloads.

One might wonder, for example, whether the energy and cost efficiency of low-temperature technology would not be easily negated by the necessary cooling effort. To this end, the electricity costs of both a conventional AI system and a superconducting AI system were modeled and compared for different orders of magnitude (i.e., the number of petaFLOPS). At larger scales, i.e., between ten and hundreds of petaFLOPS, the cooling effort decreases significantly and the superconducting system becomes more energy-efficient than its classical cousin.

In other words: the larger the scale of operation, the higher the profits. Cooling can be done by commercial cryocoolers, which have the size of three conventional server cabinets.

A data center as small as a shoebox

Design tools were used to optimize the functional partitioning of a superconducting system for AI processing. A look at one of the boards shows many similarities with a classic 3D system-on-chip that is created by heterogeneous integration of CMOS technologies. The board is equipped in a similar way with computing chips, a CPU with embedded SRAM-like cache memories, DRAM memory stacks, and switches that are connected via silicon interposer or bridge techniques.

But there are also some striking differences. With classic CMOS technology, it is very difficult to stack computer chips on top of each other, as a lot of power loss is generated per chip. In the case of superconductivity, the low power loss can be well compensated by the cooling system. Logic chips can be stacked directly using 3D integration, which leads to shorter and faster connections between chips and to area advantages.

Several boards can be stacked one on top of the other with a small distance between them. An estimate of the performance and space requirements of a stack of 100 boards shows that the system can deliver 20 AI exaFLOPS (BF16 dense or 16-bit floating point format dense—which corresponds to 80 exaFLOPS FP8 with sparsity, as is often used in GPU servers), exceeding the most modern supercomputer in today's data centers. Moreover, the system only consumes 1 kW at cryogenic cold and 500 kW at room temperature, an energy efficiency of more than 100 TOPS (Tera operations per second) per watt, all on the base area of a shoebox.

Superconductivity: basics, building blocks and state of the art

To better understand why superconducting digital technology is so energy efficient and how it can be used to build digital electronics, let's look at the physics behind it.

In classic logic and memory systems, the digital information is derived from the voltage levels of the analog signals in the form of ones and zeros. With superconducting elements, the zero resistance rules out a voltage drop. The data encoding here is based on the Meissner-Ochsenfeld effect (magnetic fields are displaced from a superconducting material when it is brought below its critical temperature).

If a ring made of a superconducting material is brought into a magnetic field, the magnetic flux is only an integer multiple. This so-called magnetic flux quantum can be used as a basis for the encoding of information. Unlike the voltage ranges used in classical computing, this quantum bit is fundamentally exact. This effect forms the basis for a superconducting memory.

Once generated, a quantum bit exists forever. To perform digital operations, a switch is needed that can change the state of the ring. This is where the Josephson effect comes into play (a tunnel current between two superconductors).

This effect is exploited in the Josephson contact, where a thin layer of non-superconducting material is embedded between two superconducting layers. Until a critical current is reached, Cooper pairs tunnel through the barrier and a supercurrent flows.

If the critical current is exceeded, a tiny voltage pulse is produced over the junction, releasing 2x10^-20 J in a picosecond. This effect is used for data encoding, for example by defining a digital one for a positive pulse when the superconducting phase switches from low to high, and a digital zero for a negative pulse when it returns from high to low. During operation, only the "small" switching processes contribute to the power loss of the system.

Zero resistance, Meissner-Ochsenfeld effect, and Josephson effect

These three physical phenomena—zero resistance, Meissner-Ochsenfeld effect, and Josephson effect—form the basis for the construction of superconducting digital electronics: superconducting logic chips, Josephson SRAMs, resistance-free wires, switches, supplemented by tunable metal-insulator-metal capacitors (MIMCAPs) for power supply and cryo-DRAM [4, 5].

And although the materials and functional principles fundamentally differ from CMOS technology, the logical operations and data coding have many similarities. In the design phase, conventional EDA tools can be used, superconducting CPUs can be synthesized with conventional code, and multi-chip modules (MCM) can be built.

Superconducting electronics is not a new computational paradigm. It is a classic technique that has been around for a long time. Over the years, a variety of powerful, energy-efficient 8- and 16-bit CPUs and MCMs have been developed.

Scaling superconducting technology: key criteria and first stages

The fabrication processes and materials used to manufacture superconducting CPUs do not offer the ability to scale to the computing density required for the revolution of the AI and ML roadmaps. Imec aims to scale from today's 0.25 µm lithography node to a 28 nm node.

When scaling superconducting interconnects to 50 nm, the product of clock rate and device density is comparable to what a 7 nm CMOS logic technology node offers. In terms of interconnect performance, expressed in Gbit/line, the 28 nm superconducting technology is expected to outperform the 7 nm CMOS technology by two to three orders of magnitude while being about 50 times more energy efficient.

Two important factors contribute to scaling. First, the processing is shifted to a 300-mm chip cleanroom to take advantage of the processes and equipment that have enabled the continuous scaling of CMOS technology. This gives the researchers access to unique processes such as 193 nm immersion lithography for structure production and advanced integration methods such as the semi-Damascene process for the construction of the connection layers.

Secondly, niobium, the superconducting material currently used, is replaced by NbTiN, a superconducting compound with significantly better scaling potential. This alloy is used for the construction of the connections and for the production of new types of Josephson contacts and MIM capacitors. Unlike niobium, NbTiN can withstand the temperatures used in standard CMOS processes, and it interacts much less with the surrounding layers.

Superconducting logic components

The research focuses on developing modules for MIM capacitors, Josephson contacts, and connections and validating them at cryogenic temperatures. Recently, the scientists demonstrated short loops with metal lines and vias made of NbTiN, which were manufactured in the 300-mm cleanroom through direct etching of metals and a semi-Damascene approach. The manufactured lines are only 50 nm wide, chemically stable, have a critical current density of 100 mA/µm² and a critical temperature of 14 K.

The ability to achieve the critical dimension of 50 nm through optimal process control forms a solid basis for the production of superconducting connections on a two-metal level. The feasibility of such a two-metal scheme was also demonstrated. The module is designed to allow multi-level expansion due to the planar single layers from which the metal and via layers are built, allowing multiple metal layers to be stacked on top of each other. The results on the conductor tracks and vias were presented at the IEEE International Interconnect Technology Conference (IITC) [6] in 2023.

The two-metal level results form the basis for the possibility of embedding superconducting digital logic components, such as tunable HZO capacitors with NbTiN electrodes and Josephson contacts with αSi barrier and NbTiN electrodes. Progress has been made in the development of superconducting HZO capacitors with NbTiN electrodes, using a manufacturing concept similar to that of standard room temperature HZO capacitors with TiN electrodes.

With the αSi Josephson contacts, the integration of the new NbTiN electrodes resulted in an improvement compared to previously reported αSi junctions with Nb electrodes. The use of clean rooms allows for high-quality interfaces and sufficient control of uniformity within the component, which are crucial for proper functionality. Cross-sections of the connections, HZO capacitors, and Josephson junctions can be seen in image 3a and 3b.

In addition to shrinking the Josephson contacts and the size of the connections over three successive generations, the roadmap also extends to 3D integration and cooling technologies. For the first generation, the roadmap envisages stacking about 100 boards to achieve a performance of 20 exaFLOPS BF16 dense (80 exaFLOPS FP8 with sparsity). Gradually, more and more logic chips are stacked and the number of boards is reduced. This will further increase performance while reducing complexity and costs.

Conclusion: Superconducting digital systems are likely to turn AI and ML roadmaps upside down. They promise a big leap in energy efficiency and computing density, deeply rooted in the physical fundamentals of superconducting technology. A joint optimization of system and technology is necessary to ensure that the gains apply to the entire system structure.

On the technology side, a roadmap based on shrinking Josephson contacts and connections, increasing clock frequency, and stacking circuit boards and individual logic chips, will enable the shrinking of data servers to the size of a shoe box. Initial milestones have been reached in the miniaturization of NbTiN-based conductors, HZO capacitors, and αSi Josephson contacts. For all three process modules, the use of materials and integration schemes compatible with CMOS manufacturing is key to success. (kr)

*Quentin and Anna Herr are scientific directors at imec in Leuven, Belgium.

This research work was also reported in IEEE Spectrum

References

[1] Semiconductor Industry Association. “Rebooting the IT Revolution: A Call to Action.” Retrieved March 14 (2015): 2019.

[2] https://openai.com/research/ai-and-compute

[3] ‘Superconducting digital technology: enabling sustainable hardware for deep learning and quantum computing’, Anna Herr, presented at the 2022 imec technology forum (ITF) USA

[4] ‘Scaling NbTiN-based ac-powered Josephson digital to 400M devices/cm2’, A. Herr et al., arXiv preprint arXiv:2303.16792 (2023).

[5] ‘Superconducting pulse conserving logic and Josephson-SRAM’, Q. Herr et al., Appl. Phys. Lett. 122, 182604 (2023)

[6] ‘Towards enabling two metal level semi-damascene interconnects for superconducting digital logic: fabrication, characterization and electrical measurements of superconducting NbxTi_(1-x)N’, A. Ponkhrel et al., 2023 IITC