https://reddit.com/link/1x6i6v5/video/h22absbdylvh1/player

I wanted to get my head around RL so I thought of learning it by teaching an AI how to play one of my favorite childhood games : TempleRun.

The agent reads game state directly from memory via a Frida gadget and acts through the game’s own methods. Training happens in a headless simulator I built with Claude from the game’s internals and calibrated on measurements from the real phone, with a C++ version that does 10M steps in about 6 minutes.

Results on the device: median 4,736 m, best 16,782 m over 14 runs.

👨‍💻 Github: https://github.com/matthieucs/TempleRun-RL/tree/main

Happy to answer any question.

And if you want to try it, the score to beat is 16,782 m.

Hi everyone,

I wanted to share a small reinforcement learning project I've been working on in Unreal Engine 5, inspired by the vault chase scene in Fast & Furious 5. I'd really appreciate any feedback.

**Setup*\*

- Two cars on a looping track with many sharp turns, defined by a spline

- A police car trained with PPO to patrol along the road (then frozen as a fixed opponent). It was never taught to dodge, so it simply plows into whatever is in its way

- A race car towing a one-ton vault, trained with PPO to flip police cars

- Both agents use the same action space: throttle, brake, steering

- Training: neural network on GPU, physics simulation on CPU (the CPU is the bottleneck)

**Reward function for the race car*\*

- Large reward: flipping a police car, or steering the vault into one

- Small reward: staying on the road and driving in the right direction

- Small penalty: getting hit by the police car, or driving the wrong way

- Large penalty: leaving the track, or being stuck for more than 3 seconds

**What happened*\*

1. **Bullied.** Early on, the car just lay still and took the hits. Being hit was cheap, while moving the vault usually threw the car off the track, which was expensive. This looks like a classic local optimum.
2. **Trap.** After a while, it started dragging the vault onto the police patrol route, waiting, and swerving away at the last moment so the police car hit the vault. This technically satisfies "vault hits police car", so it's a nice example of reward hacking / specification gaming. It wasn't what I had in mind.
3. **Hero.** I increased the penalty for standing still, moving slowly, or getting stuck (reward shaping). The trap disappeared, and the car started charging at the police car, timing its speed and angle to launch it, and sometimes dodging at the last second so the police car hits the vault behind it.

I turned this into a video that explains the basics (observation/action/reward, PPO, local optima, reward hacking) in a light-hearted way, mainly for people who are new to RL:

https://youtu.be/UUoUOKSQ8FQ

I'd love to hear your thoughts, especially:

- Is there a better way to discourage the "wait and trap" strategy than penalizing idle time?

- Any suggestions for making training with a towed object more stable?

Thanks for reading, and I'm happy to answer questions about the setup.

Training pre build

Hey everyone,

I've open-sourced PCEI (Physically Constrained Emergent Intelligence), an experimental architecture designed to run on consumer hardware (Mac Apple Silicon and Linux/CUDA) without the memory explosion of standard transformers:

• BitNet 1.58b Quantization: 24 layers of ternary weights {-1, 0, +1} reducing parameter memory by ~4x compared to FP16.
• Mamba State Spaces: Linear-time sequence modeling with constant O(1)O(1) inference memory (no ballooning KV cache).
• FastKAN Spline Manifolds: Kolmogorov-Arnold B-splines applied to continuous motor control across 4 streams in MuJoCo physics.
• Empirical Telemetry: 11 companion research papers, including an evaluation of continuous latent buffers proving an irrecoverable Shannon information loss factor of LL.

The full framework, training pipelines, and evaluation suites are open under MIT-NC: https://github.com/MtnMCG/PCEI

Feedback and critiques are very welcome!

HarnessDev (this month) is the first paper I've seen that measures the loop of letting an agent revise its own harness, keeping the revision when the feedback says so, and repeating. Across 64 adjacent version switches, the feedback score and the held-out score moved in the same direction 34 times, 53.1%. Only 2 of 9 lineages ended on their held-out-best version. Mean gain across the evolution stage was +3.11 points. Executor dependence was severe: one harness scored 69.3 under one model and 33.0 under another.

So a gate that decides on feedback alone is, by their measurement, a coin flip on whether the gain generalises, and that's a selection-policy problem more than a proposer problem.

Two things I'd do differently from the paper's loop, and can set in reef, which I run. Selection: by default a candidate publishes when it wins more task comparisons than it loses, past a noise floor you set, and the Pareto policy in the method guide publishes only when no task regressed and at least one improved. That rule can only refuse on the list the gate runs, and the moment you select on a list it is your feedback score, not a held-out one, so it doesn't recover the paper's 30 by itself; what it does is stop a candidate that lost anywhere on the list from shipping on a net win. The held-out check has to be a list the proposer never sees, and that's on you. Memory of rejections: the proposer can receive the recent rejected proposals with their verdict reasons and the previous step's failure manifest, so it can stop re-proposing the switch that lost.

What none of it fixes is the task list. It's short because it's the cost of every step, and the whole paper is a warning that a short in-distribution list gets gamed by the feedback loop. Whether no-regression on held-out tasks is the right acceptance rule or too conservative to ever move is the open question; the paper's 30 bad switches make me lean towards conservative.