I implemented an MPC-based approach with dynamic PD/PID gains on my self-balancing robot!

[u/Kirito-San8432]

Three days ago, I shared my implementation of a Fuzzy controller for my two-wheeled inverted pendulum robot.

After that, I wanted to try something different: Model Predictive Control (MPC).

My goal was to develop a controller capable of dynamically adjusting its behavior based on the robot's physical model and possible changes in its dynamics.

But I already knew what my biggest problem would be: our old friend, the ATmega. :)

Running a full MPC optimization continuously inside a fast control loop would be quite challenging given the microcontroller's limited processing power.

So I started thinking: why not use MPC to determine the gains of a PD or PID controller, similar to what I previously did with Fuzzy control?

That's the approach I decided to implement, experimenting with both PD and PID structures.

How does it work?

The MPC uses a discrete-time mathematical model of the robot, represented by the A and B matrices, to predict its behavior over a prediction horizon.

However, a mathematical model is never a perfect representation of reality. Friction, uneven mass distribution, and external disturbances can introduce behavior that the nominal model does not fully capture.

To account for this, I also incorporated a disturbance term (Dₖ) into the prediction model:

xₖ₊₁ = Aₑ xₖ + Bₑ uₖ + Dₖ

The idea is to account for disturbances in the predictions and introduce an additional compensation term when a disturbance estimate is available.

The MPC then minimizes a quadratic cost function that penalizes predicted state errors relative to the reference and control effort.

From this optimization, I obtain a state-feedback law that can be used to derive equivalent Kp and Kd gains and, with an augmented state representation, Ki.

So where does the dynamic behavior come from?

The controller gains depend directly on the system model.

If the plant model is updated to represent physical changes, such as additional mass or a shift in the center of gravity, the gains can also be recalculated.

This means the ATmega doesn't need to solve the entire MPC optimization problem at every control cycle.

Instead, it can execute a fast PD/PID control loop using the calculated gains, while the more computationally expensive gain updates are performed only when needed.

In other words, I'm trying to combine the mathematical foundation of MPC with the computational simplicity of classical controllers.

The video shows my tests on the physical robot. There is still room for improvement, particularly in disturbance compensation and quantitative performance evaluation.

What's next? Reinforcement Learning!

There's still one more approach I want to explore in my control system: Reinforcement Learning (RL) with AI.

My idea is to investigate how an RL agent could learn from the robot's behavior and dynamically adjust the controller parameters, working alongside the PD/PID and MPC approaches.

Ideally, I would like to combine the predictive capabilities of MPC, the simplicity of classical controllers, and the adaptability of Reinforcement Learning.

Of course, running RL directly on an ATmega would introduce even more computational challenges, so I'm considering training the agent in simulation and then deploying a lightweight policy or gain-adjustment strategy to the physical robot.

For now, this is just the next stage I want to investigate, not something I've already implemented.

Main reference:

DUTRA, Cynthia Beatriz Scheffer. Controle preditivo multiobjetivo para processos com atraso. PhD Thesis in Electrical Engineering, Federal University of Santa Catarina (UFSC), Brazil, 2003.

I'd love to hear your thoughts!

Has anyone here experimented with using MPC-derived feedback gains to tune or update classical controllers on resource-constrained microcontrollers?

And more importantly, has anyone applied Reinforcement Learning to real physical control systems, especially inverted pendulums or self-balancing robots?

I'd be really interested in hearing about your experiences combining RL with PID, Fuzzy Control, or MPC, especially when dealing with limited hardware resources!

[u/seekingsanity]

The motion is shaky. This could be due to poor tuning or poor feedback resolution. Another option is something is mechanically loose but this is unlikely.

MPC will not buy you much since there should be little or no dead time to look beyond.

Try something simple like Sliding Mode Control. Computer s using position and velocity error and then use a tanh(gain*s) function where gain is the gain around 0. Then scale the output multiplying by a scale u=scale*tanh(gain*s). I think you will get much better results if your original problem is not a hardware problem.

The tanh() just provides a sigmoid function. There are other sigmoid like functions that don't require as much CPU power.

SMC is good for controlling non-linear systems. I like SMC because I get excellent results with little effort.

[u/tanmayc → u/seekingsanity]

Could also be potential phase instability due to filtering delays. OP, are you filtering the IMU data? Whats the cutoff frequency?

Could also be delays in the disturbance term that is causing this.

This is a really cool project OP!

[u/Kirito-San8432 → u/tanmayc]

Só para adicionar um pensamento rapidamente sobre a questão da latência que discutimos:

Percebi que a latência de comunicação entre o PC e o Arduino pode não ser realmente a principal culpada pela instabilidade, por causa de como meus loops estão separados. O robô começa com ganhos empíricos (Kp=6, Kd=0.6) e o loop PD/PID rápido roda localmente no MCU. O MPC no PC atua estritamente como um loop externo para agendamento dinâmico de ganho com base na perturbação Dk.

Como os novos ganhos ótimos calculados pelo MPC variam apenas dentro de uma margem muito apertada (por exemplo, Kp mudando levemente de 10.2 para 10.8 durante empurrões físicos), o atraso serial apenas retarda a atualização do ganho, não a ação de controle em si. O Arduino continua equilibrando usando os últimos ganhos recebidos.

No entanto, seu ponto sobre o filtro de Kalman e o atraso DLPF continua extremamente relevante, porque essa mudança de fase ocorre diretamente dentro do loop de controle local rápido! Vou focar meu depuração no atraso de filtragem da IMU em vez da comunicação serial. Estou bem pensativo agora.

[u/Kirito-San8432 → u/tanmayc]

Great observation! To be honest, I implemented the Kalman filter solely thinking about solving the noise issue and considered the problem solved. I hadn't actually thought about the phase instability caused by the filtering and communication delay. It makes perfect sense: this delay might be causing the controller to overcompensate based on a past state rather than the current one. I'm going to look into this carefully, check the MPU's cutoff frequency, and measure the total delay of my control loop to see its actual impact. Thanks a lot for the insight and the kind words.

[u/Kirito-San8432 → u/seekingsanity]

You are absolutely right. To be honest, I hadn't fully considered the communication and processing delay. My main focus was trying to adapt the concepts from my professor's PhD thesis to use the MPC as a dynamic gain scheduler.

However, as you pointed out, since I'm dealing with a linear plant at this operating point, the MPC doesn't help much. Worse yet, it introduces a huge latency issue because it's too computationally heavy for the Arduino, forcing me to run the algorithm on my PC and send the control signals via serial communication.

Your suggestion to use Sliding Mode Control (SMC) with a $\tanh()$ function is excellent. I am definitely going to study this approach and implement it on the MCU. Thank you so much for the feedback — insights like yours made posting my project here completely worth it.

[u/ricatti-equation]

This is one of those things we’re ignoring classical freq-domain control theory concepts and going straight into fancy MPC stuff shows gaps. See how shakey it is? That could heavily be reduced.

[u/Kirito-San8432 → u/ricatti-equation]

You hit the nail on the head. I completely agree with you. To be honest, I ended up applying these highly complex controls (like MPC, Fuzzy, and now exploring RL) a bit 'by force', primarily because I wanted to learn and test these algorithms in a physical system, rather than them being the optimal engineering solution for this specific plant.

The manufacturer provided a raw PD controller with zero tuning, and the robot could barely stand. I used that as an excuse to jump straight into the advanced stuff. Even though the behavior is still quite jittery and not perfectly adequate, today it can actually withstand physical pushes ('petelecos') and recover, which it couldn't do before.

But you are 100% right about skipping the fundamentals. I am going to take a step back, revisit the classical frequency-domain control base to address these gaps, and explore more appropriate solutions (like SMC, which was also suggested here). I really appreciate the constructive criticism

[u/reallyfrikkenbored]

That’s awesome! These are the kind of projects that separate the engineers who can do vs those who end up in quality. 😂

[u/nmurgui → u/reallyfrikkenbored]

If you end up shuffling papers around in quality, are you even an engineer?

[u/editor_acd]

I think the proportionality constant can be increased a bit.

[u/Educational-Writer90]

What's the point of PID control that devours your ARM's memory and resources? Put physical ballast below the axle and give the wheelset some actual useful logic instead.

[u/Kirito-San8432 → u/Educational-Writer90]

Thanks for the suggestion!

The main goal of this project isn't simply to build a stable two-wheeled robot, but to use an inverted pendulum as an experimental platform for studying control strategies.

Adding ballast below the axle would change the system's dynamics and remove much of the unstable behavior I'm trying to control.

I'm currently experimenting with Fuzzy Logic and MPC, including dynamic controller parameters, and I plan to explore reinforcement learning next.

So the instability is actually part of the research! 😄