Controlling thermoacoustic instability of a laminar premixed flame with deep reinforcement learning and neural autoregressive models

Thermoacoustic instabilities pose challenges for several combustion applications, such as rockets, ramjets, aeroengines and boilers. The mitigation of these instabilities requires decoupling unsteady heat release and acoustics of the system. While existing strategies rely in theoretical approaches, this paper introduces a fully data-driven approach for modelling and control of systems with sustained pressure oscillations. A nonlinear autoregressive model (NARX) with neural networks was trained on experimental data obtained from a laminar premixed flame exhibiting a thermoacoustic instability at 166 Hz. The NARX model showed good prediction capabilities using closed-loop measurements. Furthermore, given the limitations that traditional control techniques face for nonlinear systems, this work explores the application of offline reinforcement learning for tuning the parameters of a phase-shift controller. The reinforcement learning model is trained using the NARX model as the environment. The study demonstrates the potential of reinforcement learning for control of thermoacoustic instabilities and shows that the parameters suggested by the model fall in the range where the thermoacoustic instability can be reduced.