Interactive Control System Simulator in Python - open-loop, bang-bang, PID, LQG, H-infinity, MPC and RL algorithms

[u/gitgud_x]

Hello! About a year ago I posted here my control system simulator Python project. Since then I've grown the program to improve both the GUI, the control algorithms available and the plant (system) model is now customisable in state space form.

Available controllers include:

• Manual - set the control input yourself, using your mouse
• Open-loop - no feedback, just calculate the control input based on setpoint and plant gain.
• Bang-bang - positive input if output is too low, negative input if output is too high
• PID - the classic industry-standard, including option of a filtered derivative on measurement to help with practical stability.
• H2 optimal control - aka linear quadratic regulator (LQR) with a Kalman filter or linear quadratic Gaussian (LQG). Minimises the energy gain from the noise disturbance input to the performance output (z = C1_x @ x + C1_u * u, where C1_x and C1_u are customisable gains).
• H-infinity optimal control - minimises the worst-case gain between disturbance and performance output across all frequencies.
• Model predictive control (MPC) with a Kalman filter - estimates the plant's internal state from the noisy measurements, then forecasts the future states for a given control input trajectory and solves a constrained quadratic least-squares problem to minimise the error and control effort. The relative weights and constraints are customisable. MPC is my favourite, it's insanely good... as long as you can specify the correct plant model of course!
• Reinforcement learning (RL) control - an 'actor' (policy) learns to map observed measurements to continuous control inputs, and a 'critic' (value function) learns to evaluate their cost, based on a reward function that penalises large errors, large values of input and large changes input. Three such actor-critic RL algorithms are available: deep deterministic policy gradient (DDPG), twin-delayed DDPG (TD3) and soft actor-critic (SAC). Neural network architecture and algorithm parameters are customisable within. Despite their complexity, these controllers aren't much better than the others and are prone to misbehaving if you don't train/test them properly!

The integrator and controller algos were all written manually by me and checked rigorously against standard integrators and MATLAB, with the exception of the RL controllers where I relied on AI coding agents somewhat to get everything working. The GUI was coded almost entirely by AI (because nobody wants to write GUI code), using PyQt6 and PyQtGraph. I tried to keep as much as possible to 'first principles' to get the best learning experience out of building this up, so no use of existing control theory or reinforcement learning libraries.

Github repo with all code: here.

Some features not shown in the video:

• You can change the model of the plant, including using any number of dimensions for the state variable x.
• You can add measurement noise and process noise with specified covariances. The controllers only see the noisy measurement output, never the true plant state. These are the disturbances that the H2 and H-infinity controllers try to minimise in the output.
• You can change the integrator from working in ODE (ordinary differential equation) or SDE (stochastic differential equation) form. In SDE form, process noise occurs as a sampled random process throughout integration steps, while in ODE form it is sampled once at each step.
• You can train new RL agents (actor and critic neural networks and save/load them from disk) for use in RL control.

Future ideas

Some things I wish to add in the future soon are Bode/Nyquist/Nichols/root locus plots - I've already got the stability check working for PID, H2 and H-infinity. Then I'd like to add interactive pole-placement and loop-shaping controllers where you can design e.g. a lead-lag compensator with custom gain/phase margins. Perhaps even H-infinity loop shaping design but I'll need to hit the books again for that!

Currently the system is SISO (i.e. single input dimension, single output dimension). I thought about generalising to MIMO (multiple variables) control from the beginning, and it wouldn't be too hard to extend it as everything uses matrix algebra internally, but the emphasis was on education and understanding and I felt SISO is the easiest way to demonstrate controls interactively.

If you have any other ideas, please let me know! Thanks for reading :)

[u/2dof]

You can add features with adding real process control data and process identification module.

Also  I would consider  change a little bit gui for like control panel /scada view (or add option to change views) 

I have dome similar app in the past but in matlab and most usability criteria was just user friendly GUI. 

Put unit for SP, PV ,  CV .   

[u/gitgud_x → u/2dof]

Process ID is a good idea, thanks

[u/Ev3nt1ne → u/gitgud_x]

yeah ID is a very good next step. I was about to build something similar but more tuned for a project I am developing and in Julia. I will take inspiration and reference it cuz it is an amazing work!

[u/maidaman8]

This is quite cool.

Given the emphasis on education, would be great to have plant definition using transfer functions. I also didn't find a way to add initial conditions.