Xpeng relies on Large Language Models for the Tianji operating system

Artificial Intelligence Xpeng relies on Large Language Models for the Tianji operating system

2024-05-30 From Henrik Bork * | Translated by AI 4 min Reading Time

Related Vendors

Will artificial intelligence cause the next complete upheaval in the automotive industry? Will it be as "disruptive" in the vernacular as electrification? The Chinese startup Xpeng firmly believes so.

Automated driving without Lidar but with XNet, XPlanner, and XBrain – that's what the company announced on its AI Day.(Image: Xpeng) — Automated driving without Lidar but with XNet, XPlanner, and XBrain – that's what the company announced on its AI Day.
(Image: Xpeng)

Henrik Bork, long-time China correspondent for the German Süddeutsche Zeitung and the Frankfurter Rundschau, is Managing Director at Asia Waypoint, a consulting agency based in Beijing specialized in China.

The electric car manufacturer Xpeng has just unveiled a new car operating system packed with artificial intelligence (AI). The founder, He Xiaopeng, called it "the first AI-powered in-car OS in the automotive industry" on the "AI Day" of the company named after him. The self-developed operating system (OS) named XOS 5.1.0 or "Tianji" uses Large Language Models (LLM) for a range of functions, which, according to the manufacturer, also include "improved" navigation, object detection, and decision-making ability.

Improved voice assistant

Just like with several other manufacturers, the AI moves into the cockpit of Xpeng's electric cars with the new operating system, improving, for example, the ability of the voice assistant to react to natural statements from drivers. Now, you no longer have to say "Hello, Xiao P" first, but can get straight to the point. If you say, for example, "It's a bit stuffy in here, Xiao P", the voice assistant suggests opening a window.

Optimize automated driving functions

If that were all, then we probably wouldn't have to bother with Xpeng's AI uproar. After all, skeptics about the relevance of AI in cars would say, you could just open the window straight away. But the automaker and its founder insist that their new OS does more than just possibly use generative AI to boost sales for the voice assistant in the cockpit.

Rather, with "Tianji", AI is now also being used to optimize automated driving functions and vehicle control, according to the manufacturer. The OS operates with the help of "XNet", a neural network by Xpeng for "Deep Vision". It not only makes driving more comfortable, but also safer, the manufacturer claims.

Automated driving without Lidar

Almost in passing, Xpeng has therefore also confirmed at its AI Day that it wants to do without Lidar from now on – similar to Tesla. Xnet allows autonomous driving or ADAS capabilities "comparable to Lidar" even without Lidar, according to Xpeng. XNet offers a "2K network for pure machine seeing" that maps the real world with "more than two million grids," according to Xpeng. The range of its perception is greater than that of previous systems, more precisely the width of 1.8 football fields, and XNet can "identify more than 50 different types of objects."

Also new is the "XPlanner" planning model, which collaborates with XNet. Similar to the human brain, it can continuously learn, here only with the help of enormous amounts of data, and can therefore make decisions in driverless driving that are increasingly "human-like", according to the car manufacturer.

The third new component is "XBrain", an AI-LLM architecture, according to Xpeng. The use of AI significantly improves the processing of "complex and even unknown scenarios".

All three new components together, i.e. XNet, XBrain and XBrain, could constantly learn with the help of artificial intelligence. Over the next 18 months, Xpeng expects a 30-fold improvement in autonomous driving functions. "By 2025 Xpeng will achieve a level L4 driving experience in China," said company founder He Xiaopeng at the event. The new "end-to-end" capabilities of the navigation system XNGP would also be tested abroad, according to He. "The transformation of smart cars by LLM is disruptive," said the startup founder at his AI Day in Beijing. "It's not a proportional change, not a few dozen percent, but a change with a multiplication factor of several dozen," said the head of Xpeng.

From electrification to smartification

Whether this will happen remains to be seen. For now, it can at least be noted that Xpeng has irrevocably linked its positioning as a car manufacturer with the new buzzword AI. The auto industry is moving from "electrification to smartification" and Xpeng itself is striving for a "new positioning in the market as the global pioneer and advocate of AI smart driving", according to the automaker's press announcement.

It can also be noted that this strong bet on AI as the "Next Big Thing" in the auto industry is expensive and risky. This year alone, Xpeng has announced plans to spend 3.5 billion yuan, or about 450 million euros, on research & development and hire 4,000 additional engineers to continue being the pioneer of "smart driving" in the age of AI.

Subscribe to the newsletter now

Don't Miss out on Our Best Content

Business E-mail

Please enter a valid mailadress.

By clicking on „Subscribe to Newsletter“ I agree to the processing and use of my data according to the consent form (please expand for details) and accept the Terms of Use. For more information, please see our Privacy Policy. The consent declaration relates, among other things, to the sending of editorial newsletters by email and to data matching for marketing purposes with selected advertising partners (e.g., LinkedIn, Google, Meta)

Date: 08.12.2025

Naturally, we always handle your personal data responsibly. Any personal data we receive from you is processed in accordance with applicable data protection legislation. For detailed information please see our privacy policy.

Consent to the use of data for promotional purposes

I hereby consent to Vogel Communications Group GmbH & Co. KG, Max-Planck-Str. 7-9, 97082 Würzburg including any affiliated companies according to §§ 15 et seq. AktG (hereafter: Vogel Communications Group) using my e-mail address to send editorial newsletters. A list of all affiliated companies can be found here

Newsletter content may include all products and services of any companies mentioned above, including for example specialist journals and books, events and fairs as well as event-related products and services, print and digital media offers and services such as additional (editorial) newsletters, raffles, lead campaigns, market research both online and offline, specialist webportals and e-learning offers. In case my personal telephone number has also been collected, it may be used for offers of aforementioned products, for services of the companies mentioned above, and market research purposes.

Additionally, my consent also includes the processing of my email address and telephone number for data matching for marketing purposes with select advertising partners such as LinkedIn, Google, and Meta. For this, Vogel Communications Group may transmit said data in hashed form to the advertising partners who then use said data to determine whether I am also a member of the mentioned advertising partner portals. Vogel Communications Group uses this feature for the purposes of re-targeting (up-selling, cross-selling, and customer loyalty), generating so-called look-alike audiences for acquisition of new customers, and as basis for exclusion for on-going advertising campaigns. Further information can be found in section “data matching for marketing purposes”.

In case I access protected data on Internet portals of Vogel Communications Group including any affiliated companies according to §§ 15 et seq. AktG, I need to provide further data in order to register for the access to such content. In return for this free access to editorial content, my data may be used in accordance with this consent for the purposes stated here. This does not apply to data matching for marketing purposes.

Right of revocation

I understand that I can revoke my consent at will. My revocation does not change the lawfulness of data processing that was conducted based on my consent leading up to my revocation. One option to declare my revocation is to use the contact form found at https://contact.vogel.de. In case I no longer wish to receive certain newsletters, I have subscribed to, I can also click on the unsubscribe link included at the end of a newsletter. Further information regarding my right of revocation and the implementation of it as well as the consequences of my revocation can be found in the data protection declaration, section editorial newsletter.

AI as a lifeline?

That's a bold statement for a company that has sold just 21,821 electric cars in the first quarter of 2024, even fewer than the other somewhat promising electric car startup in China, Nio (30,053 cars in Q1). At least the number of vehicles delivered in the first quarter at Xpeng grew significantly compared to the previous year, while it fell at Nio. But BYD sold several hundred thousand electric cars in the same period.

So Xpeng is now throwing all its energy into AI, hoping to outdo the competition. It's impossible to predict the outcome of this audacious bet at this point in time. Either He Xiaoping will be celebrated in the future as a pioneer of AI-supported driving in China, similar (se)