Calculating with Light An SDK for Photonic AI Accelerators Unveiled

Related Vendors

FAULHABER Drive Systems

Taiwan Machine Tool & Accessory Builders' Association (TMBA)

SIMTOS (Seoul International Machine Tool Show)

Q.ANT is releasing an open-source SDK for its photonic accelerator systems. Developers can use it to write applications in C and Python, emulate workloads on standard CPUs, and adapt existing AI models for the company’s Native Processing Units. The emulation enables functional development before the code runs on the actual hardware.

Photonics computing is set to become more accessible to software developers. To this end, Q.ANT is providing an open-source software development kit (SDK) for its photonic accelerator systems. The tools include interfaces for C and Python, examples, documentation, and a backend for execution on standard CPUs. This will enable developers to create and test photonic applications on their own computers and prepare them for execution on actual hardware.

“The computational operations are abstracted to such an extent that they can be used as standard mathematical functions,” explains Utz Bacher, Vice President of Software at Q.ANT. Developers can therefore use Python as usual in their preferred programming environment and work with familiar data types and libraries. No knowledge of photonics is required to get started. This allows developers to focus entirely on the development and application of, for example, AI models.

According to Q.ANT, the SDK provides a software interface to the company's photonic accelerator systems. Planned applications include, among others, neural networks, image processing, and the direct execution of key mathematical operations on the photonic hardware.

Data Transfer as an Energy Bottleneck

Q.ANT’s approach primarily addresses the energy requirements of today’s AI systems. In data center applications, a large portion of power consumption is not attributable to the actual computational operations, but rather to data movement. Bacher estimates this share to be as high as 95 percent of the available energy. Among other things, data is transferred between different server nodes as well as between main memory, CPU cores, caches, registers, GPUs, and GPU memory.

“The actual computational operations account for only about five percent of power consumption,” says Bacher. According to him, these figures refer to AI tasks in data centers. Any assessment must take into account the specific system architecture under consideration, the utilization rate, and the inclusion of memory and interface components.

Q.ANT aims to reduce energy consumption not only by performing individual computational operations optically, but also by using more compact and expressive AI models. In photonics, nonlinear functions such as cosine functions, Fourier transforms, and more complex dynamic systems could be directly mapped in the optical system. This is possible with the help of oscillating elements. Such operations are computationally intensive and therefore energy-intensive in classical CMOS systems, explains Bacher. If mathematical functions were executed directly on photonic chips, they could be integrated more efficiently into AI models. According to Q.ANT, this allows for the development of models with significantly fewer parameters that achieve equivalent results. Fewer parameters also mean less data movement.

"So we want to use smarter and smaller models instead of scaling the systems to become ever larger and more energy-intensive," says Bacher.

Calculating with Phase Shifts

Q.ANT's photonic accelerators are based on photonic integrated circuits made of lithium niobate. This material makes it possible to modulate the phase of laser light using electrical voltages. To do this, voltages are applied to modulators within the photonic chip. The electric field changes the optical refractive index of the lithium niobate and thus the phase of the light. This relationship is based on the Pockels effect. According to Bacher, the phase shift occurs very quickly and with low energy consumption because it requires only an electric field. The information is encoded via the phase of the laser light.

However, the interface to the digital world remains electrical. D/A converters require energy to convert digital data into analog voltages. After optical processing, the signals are detected by photodetectors and then converted back into digital data using A/D converters. “The actual computation, however, consumes very little energy,” says Bacher. The energy requirements of the entire system therefore depend not only on the photonic computing core, but also on the converters and the remaining data paths.

By combining multiple optical elements, complex calculations could be performed in a single pass. Q.ANT sees this as a significant advantage over an architecture in which such functions are approximated step by step through numerous electronic computational operations.

Mach-Zehnder Interferometers Map Operations

As an example of this implementation, Bacher cites multiplication using two Mach-Zehnder interferometers (MZI). In an MZI, the laser light is first split into two optical arms. By applying voltages, the light signals in the two arms can be modulated. When the sub-beams are subsequently recombined, their interference generates an output signal that contains the encoded information. According to Q.ANT, coupling two MZIs allows for native optical multiplication. The result can then be converted back into a digital signal using a photodetector.

The optical transmission characteristics can also be used for nonlinear operations. If a different operating point is selected, the first MZI can, for example, map a cosine function. The second MZI then performs the multiplication by a weighting factor. The limiting factor here lies at the interface between the digital and analog worlds. The D/A and A/D converters significantly determine resolution, noise, and speed. The optical chip itself offers high resolution and speed and is therefore not the limiting element, says Bacher.

However, Mach-Zehnder interferometers are just one example of optical computing elements. Q.ANT is already working on other elements that can be used to implement additional nonlinear functions.

CPU Emulation for Functional Testing

In addition to providing access to real hardware, the SDK also offers an emulation of the Q.ANT accelerators on the CPU. Developers can use it to test algorithms, workflows, and integrations. According to Bacher, this approach is also suitable for more complex algorithms. As an example, he cites a diffusion model based on a convolutional neural network that changes the style of input images (style transfer). The emulation focuses on a functional representation of the mathematical operations. Physical interference effects from the real hardware—such as noise from the D/A and A/D converters—are deliberately not modeled at this stage.

For linear operations such as matrix multiplication, the differences between functional emulation and actual hardware are barely noticeable. For nonlinear operations, however—and especially with complex models—minor inaccuracies may occur. “In our experience, AI models perform very well on our hardware and are sufficiently precise,” Bacher emphasizes. The emulation is thus primarily intended to facilitate the development of workloads, the verification of algorithms, and the adaptation of existing software. The transition to photonic hardware is generally straightforward.

Pre-Trained Models Can Be Ported

Bacher explains that retraining is not strictly necessary when using existing AI models. The examples included in the SDK demonstrate how pre-trained models can be adapted to the Q.ANT hardware. These include handwriting recognition, image classification, and the segmentation of brain scan image data into benign and potentially malignant tissue areas. In addition, the SDK includes a learning algorithm designed to demonstrate the use of nonlinearities. Compared to traditional neural networks, this is intended to achieve better training quality with fewer parameters.

The SDK is modeled after NumPy and PyTorch. With this approach, Q.ANT aims to lower the barrier to entry for a larger developer community. The examples demonstrate how existing workflows—such as those based on PyTorch—can be adapted to the SDK with relatively little effort. According to Q.ANT, direct PyTorch integration is already available internally but is not yet part of the published SDK. A compiler toolchain for PyTorch and other frameworks is currently under development. For now, developers must therefore determine which model operations can be mapped to the available photonic functions and which parts of the workload will be executed on classical processors.

Scaling Across Multiple Optical Elements

For future systems, Q.ANT plans to connect multiple optical elements in series. This approach is intended to enable high-performance computations in a single pass. The optical components could be integrated onto a single photonic integrated circuit (PIC). It may also be possible in the future to distribute these functions across multiple optical chips. The SDK provides the software foundation on which developers can build their own frameworks and more advanced tools. Q.ANT also plans to open up access to its photonic accelerators in the coming months. Plans include cloud access via collaboration with IONOS as well as on-premises installations of the Native Processing Server.

This shifts the focus of further development from purely providing software to the question of how photonic hardware can be integrated into existing computing environments. For developers, the available operations, the quality of model porting, the characteristics of the data converters, and the comparison between functional emulation and real hardware will be particularly crucial. (heh)

Calculating with Light

Q.ANT Demonstrates Generative AI and xLSTM on Photonic Hardware