AI Research: Fluid Mixing Optimization Using Reinforcement Learning (RL)

AI researchers at the Tokyo University of Science (TUS Japan) have adopted a Reinforcement Learning (RL) model to optimize the laminar flow fluid mixing processes. This application would benefit the industries that widely use laminar flow fluid mixing processes. Researchers at the TUS Japan are now working on a RL technique that could also be deployed to optimize turbulent flow fluid mixing. In our latest AI ML project coverage, we feature TUS Japan’s improvement in RL algorithm for fluid mixing.

Fluid mixing is one of the most common activity in chemistry. It involves the mixing of two or more heterogeneous immiscible liquids or gaseous substances to enhance chemical and thermal properties of the final homogeneous product. In most chemical engineering processes, fluid mixing is done at either macro-scale or micro-scale. For various industries, chemical mixing of fluid is a very complex process as it is mostly carried out on a ‘trial and error’ basis leading to poor outcomes. In critical operations such as DNA analysis, genomics and microfluidic micro-mixing processes, homogeneous results are not achieved due to these trial and error methods. Turbulent mixing could be an alternative to laminar flow fluid mixing, but this process is seldom used due to the damage it often causes to the material involved. Laminar flow micro-scale mixing is still preferred and that’s why AI researchers at the TUS Japan chose to optimize this approach using machine learning algorithms using Reinforcement Learning technique.

Types of Fluid Mixing

In the industrial applications, six  types of mixing processes for fluids are used.

1. Blending (applied for miscible liquids)
2. Soluble dissolution (liquid + solid mixing)
3. Chemical reaction (mixing by adding heat or pressure)
4. Insoluble suspension
5. Dispersion
6. Emulsification

Reinforcement Learning Algorithm is Effective in Laminar Flow Optimization

When the AI team at the TUS Japan questioned – “Can an optimized mixing be achieved for laminar flows instead?”, they found it is possible to do so by using machine learning.  AI researchers Mikito Konishi, Professor Masanobu Inubushi and Professor Susumu Goto (Osaka University) adopted an approach called “reinforcement learning” (RL), in which intelligent agents take actions in an environment to maximize the cumulative reward (as opposed to an instantaneous reward).

Associate Professor Masanobu Inubushi, the corresponding author of the study, explains the reasoning behind their choice.

“Since RL maximizes the cumulative reward, which is global-in-time, it can be expected to be suitable for tackling the problem of efficient fluid mixing, which is also a global-in-time optimization problem. Personally, I have a conviction that it is important to find the right algorithm for the right problem rather than blindly apply a machine learning algorithm. Luckily, in this study, we managed to connect the two fields (fluid mixing and reinforcement learning) after considering their physical and mathematical characteristics.” 

It wasn’t an easy task to move ahead with RL for laminar flow mixing optimization. The team found that laminar flow fluid mixing posed a big problem to their study due to the involvement of high-dimensional state spaces. Fluid mixing involves a system where a large number of variables are required to define their dimensions using multiple parameters. That’s where RL algorithm came into the picture that just cut down the requirement to rely on multiple dimensional states to define the fluid motion, and instead used only one parameter to solve the optimization problem.

The team created a machine learning RL algorithm in terms of a “Markov decision process” (MDP). MDP is a mathematical framework for decision making in situations where the outcomes are part random and part controlled by the decision maker. Using this approach, the team showed that RL was effective in optimizing fluid mixing.

Dr. Inubushi explained – “We tested our RL-based algorithm for the two-dimensional fluid mixing problem and found that the algorithm identified an effective flow control, which culminated in an exponentially fast mixing without any prior knowledge. The mechanism underlying this efficient mixing was explained by looking at the flow around the fixed points from a dynamical system theory perspective.”

But, why RL for mixing optimization?

Apart from solving the problem with working in a high-dimensional state space, RL algorithm had another significant advantage. This MDP-based RL method was an effective transfer learning (applying the knowledge gained to a different but related problem) of the trained “mixer.”

In the context of fluid mixing, this implied that a mixer trained at a certain Péclet number (the ratio of the rate of advection to the rate of diffusion in the mixing process) could be used to solve a mixing problem at another Péclet number. This greatly reduced the time and cost of training the RL algorithm.

While these results are encouraging, Dr. Inubishi points out that this is still the first step. “There are still many issues to be solved, such as the method’s application to more realistic fluid mixing problems and improvement of RL algorithms and their implementation methods,” he remarks.

While it is certainly true that two-dimensional mixing is not representative of the actual mixing problems in the real world,  this study provides a useful starting point. Moreover, while it focuses on mixing in laminar flows, the method is extendable to turbulent mixing as well. It is, therefore, versatile and has potential for major applications across various industries employing fluid mixing.

Conclusion

Clearly, the AI researchers were able to optimize fluid mixing process using an RL algorithm for two-dimensional state space. We can expect industrial adoption of RL algorithms for training of ML-based mixers. Mixers can be trained with sensory data of the mixture states as an input to the RL neural network.

[To share your insights with us, please write to sghosh@martechseries.com]

Source:

Title of original paper	:	Fluid Mixing Optimization with Reinforcement Learning
Journal	:	Scientific Reports
DOI	:	10.1038/s41598-022-18037-7