When Algorithms Clean up Diesel Exhaust Fumes Artificial Intelligence Significantly Reduces Nitrogen Oxides

Related Vendors

FAULHABER Drive Systems

dataTec AG

SIMTOS (Seoul International Machine Tool Show)

Nitrogen oxides are a by-product of the combustion of fossil fuels. They are among the most strictly regulated pollutants and are among the most harmful to human health. The compounds collectively known as "NOx" cause the brown smog that still hangs over some city skylines despite stricter air pollution regulations.

The California Air Resources Board is setting limits for pollutant emissions from off-road diesel engines. The proposed Tier 5 standards, which are to be introduced gradually over the next decade, require a 90 percent reduction in NOx emissions in some performance categories. Manufacturers in the USA are already responding to stricter requirements with more complex exhaust gas aftertreatment: SCR catalytic converters (selective catalytic reduction) are being used in diesel engines for industrial applications, particularly in the power range above 56 kW—a consequence of the NOx limits of the EPA Tier 4 final regulations in force since 2014.

With ever stricter emission limits, additional NOx reductions are required, resulting in ever more complex and technically sophisticated systems and further increasing development costs. As a result, the need for more efficient solutions is increasing. Engineers at Yanmar America, a manufacturer of agricultural equipment, energy systems and industrial engines, are using deep reinforcement learning to optimize their exhaust aftertreatment technology.

When the engineers at Yanmar America began working on a solution a few years ago, their engines were already equipped with SCR catalytic converters. These minimized emissions through a chemical reaction between ammonia and NOx. Ammonia is added to the exhaust gas flow via a Diesel Exhaust Fluid (DEF) injector. DEF is a urea-water solution that decomposes into ammonia in the hot exhaust gases. In the catalytic converter, the ammonia reacts with the NOx compounds and converts them into harmless nitrogen—a natural component of the air.

Despite this cleaning process, small amounts of NOx can still escape into the air and ammonia, itself a harmful pollutant, can also escape into the air. This undesirable phenomenon is known as ammonia slip. "You should try to store as much ammonia as possible in the SCR to increase the NOx conversion capability," explains Martin Muinos, Research and Development Engineer at Yanmar America. "However, this is not easy because ammonia storage is temperature-dependent." For example, a rapid increase in exhaust gas temperature can lead to ammonia slip.

The size of the SCR is another limiting factor. "The performance of SCR systems is highly dependent on size. This means that they do not have unlimited space for ammonia injection," says Shota Nomura, test engineer at Yanmar America.

From Test Bench to Simulation

SCR development typically involves a lengthy calibration process to ensure adequate NOx reduction—often requiring over 240 man-hours. "There are over 20 calibration maps that define SCR controls, and we need extensive data to calibrate these maps," Muinos emphasizes. "To collect this data, an engine test bench has to be operated for a few weeks at a time, then the maps calibrated and then the performance checked." And Nomura adds: "There is no autonomous way to calibrate the system. Since SCR affects several maps at the same time, every adjustment has an impact on performance." Achieving the optimum balance based on the test results therefore required extensive manual adjustments in the past.

To reduce the time and cost of physical testing, the Yanmar team switched to a model-in-the-loop approach. Engineers used Simulink and separate catalyst simulation software to model their SCR. "Without Simulink, we would have to propose changes to our controls, hand them over to our engine control unit supplier, have the control logic and software developed and then test them on a real test bench," says Muinos. "But Simulink is like a sandbox for the development of control systems."

Using the sandbox, the team quickly realized that their current SCR control methods would not be sufficient to meet the new Tier 5 requirements.

Nomura wondered whether artificial intelligence (AI), in particular deep reinforcement learning, could help. Reinforcement learning is an AI method that learns by absorbing information from the environment in order to achieve a desired result. It evolves through trial and error. Because the team was already using Matlab and Simulink but had little knowledge of AI, they used MathWorks Consulting Services to find out if deep reinforcement learning could help determine the best DEF dosage rate for their SCR.

Using the Reinforcement Learning Toolbox and Deep Learning Toolbox, MathWorks consultants helped Nomura select the right reinforcement learning algorithms for Yanmar's project from the various reinforcement learning algorithms provided. "After working with the team and analyzing the pros and cons of the approach proposed by Yanmar, we realized that the Deep Q-Network algorithm would help them develop a good reinforcement learning agent," summarized Mohammad Muquit, Senior Technical Consultant at MathWorks.

Learning from Experience

Equipped with the DQN algorithm, the Yanmar team began training its Deep Reinforcement Learning (DRL) agent to optimize the ammonia dosing profile for SCR performance. The model output was used to understand the extent to which the calibration values suggested by the agent matched those from existing systems. The visualizations of the reinforcement learning toolbox also made it easier for them to understand the model results.

Muinos explains: "Once we have a good understanding of why the reinforcement learning agent delivers better emission results, we can understand which control mechanisms we need to implement." "This gives us a better starting point for calibration." The results speak for themselves: after the simulation, which took just 30 minutes, the system delivered an optimal dosing profile that improved NOx emission reduction by 60 percent, according to Nomura.

The agent also halved the calibration time, reducing the total number of hours required for the project by 30 percent. "It only took six months—we hadn't expected such a short development time," says Nomura. "It far exceeded my expectations."

The time saved and the reduction in physical checks also led to a cost saving of 41 percent compared to the normal manual calibration process.

The AI-powered SCR calibration tool currently provides Yanmar engineers with a better starting point for calibrating their engines. The next step is to develop prototypes for rapid control. The Yanmar team recently procured the necessary hardware to put their modeling results to the test. The goal is to further automate this process. The DRL agent is currently searching for optimal values, but these still have to be manually transferred to the control units. For future projects, the engineers hope to use the AI in such a way that it automatically generates initial calibration values for the control units and reduces manual adjustments.

Matlab and Simulink played a crucial role in this project, and the team plans to use them and other related toolboxes for future developments. (se)

Christoph Stockhammer is a Senior Application Engineer at MathWorks.

AI in the industry

ETMM Dossier – Artificial Intelligence