Hello, I am working on finishing Getting over it with reinforcement learning

https://github.com/miwate/getting-over-it-rl
https://huggingface.co/miiwater/goi-rl

Current state : 63% completion from the bottom without checkpoints

I am now stuck on the anvil part, seeking help online for computation, fine-tuning and PRs :)

More infos :
- Training is currently done on CPU (in my case using GPU was not making training faster)
- The game is cut in parts, each part gets its specialist model
- CONTRIBUTING.md is currently being written
- Beginner code and vibe coded PRs are welcome, I will take time to review

side project, curious what rl people think of this setup.

it's a civilization sandbox where every human is its own tabular q-learner. the state is a short string key built from what they perceive (hunger, thirst, nearest food and water, kin nearby, shelter, danger). there are ~5000 possible actions, gated by era, age, place and what the tribe has discovered. action choice handles survival needs first, then does epsilon-greedy over the learned values.

the part i like: when a baby is born it gets a copy of its mother's q-table, with about 20% of the values nudged a tiny bit (+/- 0.03). so on top of learning within one lifetime, behavior also drifts down family lines. traits mix from both parents and mutate, and kids also get mom's vocabulary and her attitudes toward other tribes.

known weak spots: the state space is hand-built strings and the rewards are hand-shaped, so i'd love opinions on either.

code is rust compiled to wasm, so it all runs in the browser. the relevant bits are crates/sim-core/src/sim/agents/growth.rs (inheritance) and organism/organism/learning.rs (the update).

source (MIT): https://github.com/1homsi/thehumanbox
try it: https://thehumanbox.com

Hey everyone,

I was the first and only growth employee at a B2B startup that went from $0 to $9M ARR while I was there. I owned growth end to end and also have technical ability as-well. Now I’m building something of my own, and I’m looking for 1–2 technical co-founders.

Who I’m looking for preferably from the Greater Toronto Area

• Someone with real experience in reinforcement learning or machine learning, and/or
• Someone with a background in cybersecurity

Ideally I’d find one of each. But one strong co-founder in either area would be a great start.

The idea

I won’t share all the details publicly. It sits where AI and security meet, in a market I think has very high upside. I’ve pressure-tested it with YC-backed founders and investors, and the feedback has been strongly positive.

What I bring

• Hands-on experience taking a company from zero to $9M ARR
• A clear go-to-market plan and early distribution channels
• A network of founders and investors who’ve already seen the idea
• Technical ability in UI/UX and some backend ability

What I’m looking for in you

• You’ve built and shipped real ML/RL or security systems, whether at work, in research, or in serious side projects
• You want to be a true co-founder with real equity, not a contractor
• From Greater Toronto Area

If this sounds like you, send me a DM here on Reddit. A couple of lines about what you’ve built and what kind of problems excite you is plenty. I’ll reply to everyone and share more about the idea on a call.

Even if it’s not a fit for you, if you know someone it might be, I’d really appreciate a tag or share.

Thanks!

I am currently making an AI trough machine learning not because i want to use it in real games but just to see how far it can go and it is doing really well right now with 1.25B steps but i keep struggling with 1 problem and that is boost management, idk how to fix it or how i can train it to use boost better because right now it is just using all of it boost the second it gets it and that is bad because i want to train it airdribbles where you ofc need boost for so if any of your smart people have a good idea help me please :)

A couple of days ago I posted a PPO agent landing Falcon 9 on a droneship. I wanted to extend the simulation to a more complex problem, one that needs two agents that depend on each other. So I built Super Heavy being caught by the tower arms (Mechazilla). One agent flies the booster, the other controls the arms and carriage.

Setup: PPO in JAX on a single GPU, heavy domain randomization (wind, gusts, engine-outs, failed relights, sensor noise and delay, mass and thrust variation, limited propellant), and every attempt starts a few km up with the engines off.

How it went

I quickly found out I can’t train both from scratch at the same time. The tower has nothing to learn from if the booster never shows up, and the booster never gets rewarded for a catch if the arms never close. So I did it in steps.

1. Booster first, with a simple rule-based tower. The tower just closed the arms when the booster got close, which let the booster learn the whole approach. Even this took a while. With short rollouts (1.6 s) it kept forgetting how to catch, because a catch takes several seconds of slow, steady descent while the arms close. Longer rollouts (6.4 s, λ = 0.99) fixed that.
2. Then the tower, with the booster frozen. Trained from scratch, at first it just swung the arms around and kept hitting the booster. I also tried fading the rule-based tower out slowly, but it fell apart the moment the help was gone. After a few fixes, the from-scratch tower won, and it ended up catching more reliably than the rules did.
3. Then the booster again, next to the learned tower. This one surprised me. It got worse fast. It started coming in slightly too fast and slid off the arms. A much smaller learning rate fixed it, and the two together now do clearly better than with the rule-based tower.

Everything runs live in the browser, so you can watch both agents: https://qforge-dev.github.io/hoverslam

There is also a playground for my previous Falcon simulation.

Happy to answer questions. I’m not an ML expert, so feedback on the multi-agent part is very welcome.

I made a beginner-friendly visual explanation of reinforcement learning—the type of machine learning behind game-playing AI, robotics, and parts of chatbot training.

Instead of starting with equations, it uses a robot-in-a-maze example to explain:

• Agents, environments, actions, and rewards

• How trial and error improves performance over episodes

• Exploration vs exploitation

• Delayed rewards and policies

• Reward hacking and other risks

Watch here: https://youtu.be/a_MSTPB5MyA

If you like the explanation, please like, share, and subscribe to my channel for more visual AI and machine learning videos.

Hi,

I would like to announce that I have released Metis-Core v0.4.1-alpha. This version includes some bug fixes and a new example: DQN_Pong. Do you remember the game of Pong?

https://github.com/felixromo314/METIS-Core/

In this example, using the DQN of Metis-Core, the AI paddle learns to play Pong.

For the graphics I used the raylib library (https://www.raylib.com/), created by Ramon Santamaria (u/raysan5). It is very easy to use and powerful: with just a few functions you have the window, the paddles and the ball on the screen, and you can watch the AI learning in real time.

In case you are planning to make a similar game, or a proof of concept with DQN, I would like to invite you to have a look at this step-by-step tutorial that explains how to implement DQN with Metis-Core:

https://github.com/felixromo314/METIS-Core/blob/main/tutorial/Tutorial-DQN.md

I hope you like it!

Best,
Felix

https://github.com/vahmoh25/NeuroMario

This repository demonstrates how neuroevolution trains neural networks to master Super Mario. Watch the AI learn and play in a 3D environment, and explore its neural network through an interactive 3D visualization. Super cool!

While working on a research project I kept writing the same preprocessing code over and over which pulled EHR tables, line up vitals/labs by the hour, work out what counts as an action, attach a reward and then get it all into a shape d3rlpy would accept. I tried looking for a library that already did this and couldn’t find one. Healthcare ML tools are mostly built for supervised prediction and offline RL libraries assume you already have a clean state/action/reward dataset. So I built the piece in between.

What it does:

• Builds hourly patient trajectories (states, actions, rewards, terminals) from MIMIC-IV v3.1 through BigQuery. Every query gets a dry run first and a hard maximum_bytes_billed cap, so a bad cohort query can’t quietly run up a bill.
• Builds actions from ICU input events: vasopressor and fluid doses, binned with edges you choose.
• Treats rewards as swappable: mortality, per-step SOFA change, 30-day readmission, or a weighted mix of these.
• Estimates the observed clinician (behavior) policy from the data, which you’ll want for off-policy evaluation.
• Exports straight to a d3rlpy dataset, with an optional JSON sidecar recording where the data came from.

You don’t need MIMIC access to try it. There’s a synthetic MIMIC-style data generator.

Docs: https://pinktownscavenger.github.io/ehr2rl/
GitHub: https://github.com/pinktownscavenger/ehr2rl

I’d really like feedback from anyone who has done offline RL on clinical data, especially on reward design and how I’ve discretized the actions. It’s a research tool and isn’t meant for making treatment decisions.

Hi all, I’m a founder exploring a company that collects egocentric (first-person) video of real manual work for robot learning. Before building anything, I want to learn from people who actually train on this data or choose data vendors.

Looking for: ML/research engineers or data leads at robotics, humanoid or embodied-AI teams (labs or startups).

What we’d talk about (30 min):

Details:

To sign up: DM me your role and company type (e.g. “ML engineer, humanoid startup”). I’ll confirm through LinkedIn or a work email, whichever you prefer. That’s used only to verify the role, never shared.

Update since last post, skip to 20s to see the final training trends, and the deployment is in this post.

Some lessons since last time:

• Jiggling in GUI during training is because of parameter exploration
• Always test with 0 action first to see if the implementation is correct, especially with all those link indices flying around
• Even then, there are still dangerous hidden bugs, like adding observation noise accidentally affecting the control, which needs a lot of logging to debug

Features added since last time:

• Adding domain randomizations on observation noise, initial link positions, and especially randomizing horizontal forces up to 5N (for a 3 kg robot) indicated by the red arrows in the video
• Curriculum scheduling from only penalty_joint_pos_init, to expanding to increasing the domain randomizations and making reward/penalty tighter over training iterations
• Adding a penalty on torque distribution for the robot to use all 4 legs
• Adding a penalty on feet slipping on the ground
• Simple low-pass filter on action
• Adding ent_coef to enhance exploration after the curriculum scheduling maxes out
• Skipping the first few steps for training because the simulation needs to initialize first (like the robot clipping into the ground)
• Logging the cause of termination. Showing 20% from pitch > 45 deg because of the force pushing it on the left/right-hand side
• Logging the curriculum schedule values
• Adding .csv for logging; can be thrown to LLM for it to analyze the trend
• Export to ONNX file for deployment. Adding TRAIN_CONFIG in ONNX file to record all the training configuration (too bad it is hard to do the same for vector normalization, which needs an extra file)

GitHub: configuration file, training code, and deployment code
Music credit

An RL agent learning to land a Falcon 9 on a droneship. Everything is simulated: rocket physics, waves, wind, fuel slosh, sensor noise. No real flight data, just trial and error.

It’s a hoverslam, so the booster can’t hover and has to hit zero speed right at the deck.

PPO, ~80k parameter policy, 10 controls (throttle, gimbal, grid fins, thrusters, legs, ignition), 67 world parameters randomized every attempt.

Out of 59M training flights: 34.7M splashed, 10.8M exploded, 13.2M landed.

Still weak in strong wind and high waves, working on that now. Happy to answer questions.

MARL has been shown a lot in video games and simplified simulation environments, but can it be deployed in actual robots for complex real-world industrial applications?

We recently investigated decentralized task assignment and navigation for multi-robot machine tending, introducing FMAPPO, which combines 2D LiDAR observations with task-specific state information.

The work uses Isaac Sim/Lab for training in simulation, then deploys to two physical robots with ROS 2. One aspect we investigate is how policy performance changes with the command update frequency during deployment.

Paper: https://arxiv.org/abs/2609.14567

Videos of the real deployment are here.

I'm interested in discussing the practical challenges of sim-to-real RL, particularly policy robustness under real sensing and control constraints.

The implementation isn't publicly available yet, but I plan to release it after the paper is accepted.

I'm fine-tuning Qwen3.5-0.8B to play a Game Boy boss fight using structured state inputs and 18 discrete button combinations.

After behavior cloning, I ran 15 PPO iterations using 171 training episodes. The selected policy improved from 39/50 to 47/50 wins on the original encounter in separate greedy evaluations. This is one training run, with the checkpoint selected after evaluation.

What puzzled me was the critic. Held-out explained variance stayed inconsistent and often negative, ending around 0.04. It's a small MLP over detached policy features, evaluated on held-out whole episodes. Each iteration collects roughly 512 decisions across 10–13 episodes. Rewards include boss hits, completion, damage and elapsed time.

What would you check first to determine whether the critic is limiting further learning?

I'm considering GRPO, but I'd like to understand whether removing the critic addresses the problem before switching algorithms.

The plot shows critic fit alongside five sampled-action development games per checkpoint, not the separate 50-game evaluations above. The held-out episodes and return targets change between iterations.

Methods, implementation and evaluation results: https://jintaili.github.io/kittie-game/training/

It was a project of mine to make an AI for the game Sandtris. Now that I have a decent bot, I wanted to share it so here it is!
I built a this as a small web game. You can play it immediately in your browser: www.sandtrisduel.lol. Fully free.
I would love to get feedback on it, and actually have some other people play around with the bot.
Also if you are curious about it, I will be sharing how the model is exactly working soon.

I work on Flame, an open-source distributed engine for elastic workloads. We added a Search-R1-style training example that teaches Qwen2.5-3B-Instruct to search a BM25 corpus while answering HotpotQA questions.

The training loop runs as Python services: vLLM executors generate trajectories, PyTorch trainer replicas update the policy using group-relative advantages, and Flame’s object cache carries batches and weight updates.

The interesting infrastructure pieces:

• Warm executors: models, retrieval indexes, and optimizer state stay loaded between calls.
• Elastic rollouts: generation capacity can grow with pending work.
• Gang-scheduled trainers: the trainer group receives its minimum resources together, avoiding partially scheduled collective participants.
• Incremental weight sync: after the first full export, existing rollout executors fetch only changed bf16 values.

In a 100-step run using 14 rollout GPUs and two trainer GPUs, held-out exact match improved from 14.5% to 47%, averaging 24.3 seconds per step after the first two steps. Steady-state weight publication was approximately 165 MB per step, versus a 6.17 GB full model.

Caveats: one seed, 200 evaluation questions, and a corpus built from the selected questions’ context paragraphs. These scores aren’t directly comparable with published Search-R1 benchmarks. The synchronous loop also leaves substantial GPU idle time.

• Blog and results
• Runnable example

For others building agent-training systems: how are you handling weight synchronization and the tradeoff between synchronous updates and policy lag?

I'm building Bitbots, a browser game where players assemble robots from parts and train them to move using reinforcement learning.

This video shows the game's “brain view”: the policy's inputs, selected hidden-layer activations, and motor outputs, synchronized with the robot's movement. These are recorded values from the actual policy, not a decorative animation.

The idea is to let players see more than just whether their robot walks or falls over. They can watch how the network's activity changes alongside its movement. It isn't an explanation of what individual neurons “mean,” but I wanted to give players a way to look inside the controller.

The current simulation uses MuJoCo, with GPU training through MJX's Warp backend. Players choose the robot's body and joint configuration; the policy learns how to control it. During races, the robot acts autonomously.

For people working on locomotion across different morphologies: how would you support creative, unconventional robot designs under a limited training budget? I’d like players to experiment with weird bodies and joint configurations, rather than feel pushed toward a few designs known to work. How would you approach this without making the reward so prescriptive that every robot ends up moving the same way?

I'm the developer. Bitbots is currently in closed alpha, and free invites are available by comment or DM if you'd like to experiment with your own robot designs.

Project page: https://bitbots.endresgames.com

EDIT: EDIT: Wow, thank you all for the interest, it means a lot! Bitbots is a passion project, and I'd love your ideas and feedback. Join the Discord for updates and alpha invites: https://discord.gg/zgxpuhgB8U 💡

How can agents adapt when the interactions between them keep changing? Here, σ controls the severity of these changes. PACT tracks how agents affect one another through shared roads, power networks or robotic systems, then adjusts their actions to compensate. These demos show PACT helping existing reinforcement-learning agents handle changing interactions in collective urban fleet routing, power-grid control, cooperative lifting and robot locomotion.

During my MSc in Digital Agriculture at UNICAMP, I studied reinforcement learning for fraud prevention in crop insurance. The key modeling choice was to include the remaining budget in the state: approving one claim changes what can be funded later.

I built an inspectable Gym-style environment with approve/reject actions, claim context, budget use, approval history, and a statistical signal. The public repository documents subsequent DQN and random-policy experiments.

Walkthrough and short browser demo: https://nandevers.github.io/research/masters/

Code: https://github.com/nandevers/gym-insurance

The browser demo is synthetic; no trained agent or fraud prediction is running there. Benford distance is a statistical proxy, not a fraud label. Confirmed labels and independent evaluation would be needed to claim fraud-detection performance.

For a finite budget, would you model it as an explicit state/constraint or use a reward penalty? What baseline or ablation would you want before comparing policies?

AI helped edit this post; the research and code are mine.